authorgravatar for git@paperclover.netclover caruso <git@paperclover.net> 2026-09-26 14:32:33-07:00
committergravatar for git@paperclover.netclover caruso <git@paperclover.net> 2026-09-27 04:23:49-07:00
logb305654ab5ef9281fe63f30857a5ae23fa611558
treee462cf86d464276cd529148fdf369876f76b5ec3
parented9e69d6d927206decb398664df5f135e7244603
signature Signed by SSH key SHA256:52mNGHRsVFBDED9IAX5pe+LRWUefqTbxEReunq21QvU

feat(data): conflict pages, SMB like OneNote, and no whole-image edit API left

Conflicts publish as OneNote 2010 conflict pages (one-sided merge, conflict page under the page, OneNote's delete) instead of a review; cache schema 16 drops the dead columns. SMB publishes in compound requests with the spec's write order, the remote stamp is required, and our locks match OneNote's (three OneNote and three Rust clients converge). Every whole-page and whole-image edit API is gone from onestore and notebook: tests, examples, fuzz targets and evidence generators use ops, and protected sections have an ops path. Edits carry the author's name and initials. New corpus rows: conflict-page, page-date, extended-ascii. Assisted-by: claude-opus-5.5 Assisted-by: claude-opus-5

472 files changed, 12831 insertions(+), 8187 deletions(-)

corpus/conflict-page/README.md created+77
......@@ -0,0 +1,77 @@
1# Conflict pages
2
3What OneNote 2010 does when a sync finds that two clients changed the same object, and
4that it reads and deletes the conflict pages Rust writes. `tools/native_conflict.py`
5made every capture; `tools/test_conflict_page.py` checks them without a VM.
6
7`native` is two OneNote clients on one Samba notebook (`native_conflict.py sync`, fixture
8`native-ink/cold-ui-ink`): client A, offline, rewrote the first outline's paragraph and the
9second outline's; client B rewrote the same first paragraph and the left table cell and
10published (`b-published`); A reconnected and synced (`conflict`). A kept B's version of the
11paragraph both changed and merged its own disjoint edit into the page; its own version of the
12page became a conflict page:
13
14- its own object space, listed only in the page manifest's `ChildGraphSpaceElementNodes`
15 (`0x2c001d63`), never in the page series, with a copy of its metadata in the manifest's
16 `0x24003442` whose identity is the space's XOR the metadata salt (MS-ONE 2.2.81);
17- metadata root `jcidConflictPageMetaData` (`0x20038`): `CachedTitleString`, a fresh
18 `NotebookManagementEntityGuid`, `PageLevel`, both schema revisions (`0x28`),
19 `TopologyCreationTimeStamp` of the merge, `ConflictingUserName` and
20 `ConflictingUserInitials` of the client whose version it is;
21- a page node with `IsReadOnly`, `Deletable` and `IsConflictPage`, holding A's whole page;
22 only the rich text of the paragraph both changed has `IsConflictObjectForSelection` and
23 `IsConflictObjectForRender`;
24- the page's metadata and the section's copy of it gain `HasConflictPages`.
25
26Both clients show the same UI (`a/`, `b/`): the page's tab carries a conflict mark and a
27yellow bar reads "This page has changes that could not be merged during synchronization.
28Click here to show versions of the page with unmerged changes." Clicking it lists the
29conflict pages under the page, each as its date and user ("9/26/2026 snow"), and opens the
30first; that page's bar reads "Conflicting changes are highlighted in red. This page cannot be
31edited, but you can copy changes to the primary page. Click here for more options." over a
32light red highlight (RGB 255, 214, 214), and its menu offers Delete Conflict Page, Copy Page
33To..., Select Previous/Next Conflicting Change and Collapse Conflict Pages.
34
35`native-three` is `evidence/m8/conflict-abrupt-cold-01`: three clients' conflict pages under
36one page. OneNote appends each to `ChildGraphSpaceElementNodes` and lists them last first
37(`conflicts.png`); the metadata copies are in their own order.
38
39`native-delete` is OneNote's Delete Conflict Page on `native/conflict` in a fresh clone
40(`native_conflict.py cold --delete`): the manifest loses `ChildGraphSpaceElementNodes` (its
41last entry) but keeps the metadata copy, the page's metadata and the section's copy lose
42`HasConflictPages`, and the conflict page's space is left as it was.
43
44`native-pages` and `native-restore` are page-list merges (`native_conflict.py pages`,
45`restore`): client A, offline, moved pages through COM and edited `Target`'s body while
46client B moved pages and deleted `Target`; A reconnected (`merged`, both clients' `listed.json`
47and screenshots). OneNote merges page series, not page positions. A page a client moves
48gets a series of its own, and a COM move also re-series every page it jumps over (a drag
49re-series only the dragged pages, `page-lifecycle/movement`). The merge keeps each page in
50the series of the client that re-seriesed it, the server's where both did, leaving the
51other's series empty (`native-pages` keeps two, which MS-ONE's one-page minimum does not
52allow); series only A made go after the server's. `Target` comes back as a new page space
53holding A's edit, in a new series where A had it; B's version stays in the recycle bin.
54
55- `native-pages`: from One, Two, Three, Four, Target, A moved Four before One and Two last
56 (re-seriesing One, Two, Three), B moved Four last and Three before One (re-seriesing One,
57 Two, Four): One, Two, Four, Three, Target.
58- `native-restore`: from One, Two, Target, Three, Four, A moved Four before One
59 (re-seriesing One, Two, Target, Three) and B deleted Target: Four, One, Two, Target, Three.
60
61`page_moves_and_a_removed_page_merge_as_onenote_merges_them` in
62`crates/notebook/tests/sync_pages.rs` replays both with Snowbound's queue, A's moves as
63moves of the pages A re-seriesed, and reaches OneNote's order.
64
65`candidate` is `a_conflict_page_is_stored_as_onenote_stores_one` in
66`crates/onestore/tests/conflict.rs` (`ONESTORE_CONFLICT_EXPORT`): the remote renamed the
67first word of the fixture's first paragraph, this machine did too, and `SectionOp::Conflict`
68kept this machine's page for "Clover Snow", whose author objects carry her initials beside
69her name as OneNote's do. `cold` is OneNote's fresh read and the same UI
70walk, ending in Delete Conflict Page on the Rust-written page: its bar and list entry show,
71the conflict page opens (window title) with the paragraph highlighted, and the deletion
72stores what `native-delete` did.
73
74`candidate-delete` is `deleting_a_conflict_page_stores_what_onenote_stores`
75(`ONESTORE_CONFLICT_DELETE_EXPORT`): `SectionOp::Delete` of `native/conflict`'s conflict page,
76which stores the objects OneNote's deletion did; `cold-delete` is its fresh read, without a
77bar.
corpus/conflict-page/candidate-delete/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/candidate-delete/Open Notebook.onetoc2 differ
corpus/conflict-page/candidate-delete/synthetic.one created
Binary files /dev/null and b/corpus/conflict-page/candidate-delete/synthetic.one differ
corpus/conflict-page/candidate/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/candidate/Open Notebook.onetoc2 differ
corpus/conflict-page/candidate/synthetic.one created
Binary files /dev/null and b/corpus/conflict-page/candidate/synthetic.one differ
corpus/conflict-page/cold-delete/commands.jsonl created+3
......@@ -0,0 +1,3 @@
1{"command": "powershell -NoProfile -Command \"Expand-Archive -LiteralPath C:\\one-tests\\transfer.zip -DestinationPath C:\\one-tests\\runs\\capture\\notebook\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-4D3235A0 -ExpectedPages 1 -KeepOpen", "exit": 0, "stdout": "Read 1 sections and 1 pages.\r\n", "stderr": "", "error": null}
3{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\read -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
corpus/conflict-page/cold-delete/machine.json created+1
......@@ -0,0 +1 @@
1{"name": "m6-4d3235a0", "hostname": "ONE-M6-4D3235A0"}
corpus/conflict-page/cold-delete/read/af1af4b8987109ce1e335e538910fb3860b7c57eb3b6110a06aa2114e06d69a7.attachment created+1
......@@ -0,0 +1 @@
1Fictitious attachment for native corpus.
\ No newline at end of file
corpus/conflict-page/cold-delete/read/environment.json created+7
......@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-4D3235A0",
5 "cold": true,
6 "onenote": "14.0.4763.1000"
7}
corpus/conflict-page/cold-delete/read/hierarchy.xml created+2
......@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{1421A7BE-8880-40AC-A90F-B8EBEDAA41C7}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-27T01:42:28.000Z" color="#9595AA"><one:Section name="synthetic" ID="{F27BEF77-EE9A-0A3A-12FB-EA2DDBE6F411}{1}{B0}" path="C:\one-tests\runs\capture\notebook\synthetic.one" lastModifiedTime="2026-09-27T01:42:28.000Z" color="#8AA8E4"><one:Page ID="{DE321CF1-6751-492B-959C-42BC27DEE80E}{1}{B0}" name="Client B conflicting edit." dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1"/></one:Section></one:Notebook>
corpus/conflict-page/cold-delete/read/page-000.navigation.json created+13
......@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Client B conflicting edit. - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/cold-delete/read/page-000.png created
Binary files /dev/null and b/corpus/conflict-page/cold-delete/read/page-000.png differ
corpus/conflict-page/cold-delete/read/page-000.xml created+21
......@@ -0,0 +1,21 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{DE321CF1-6751-492B-959C-42BC27DEE80E}{1}{B0}" name="Client B conflicting edit." dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="ONE-M6-D63654A2" authorInitials="4A2" lastModifiedBy="ONE-M6-D63654A2" lastModifiedByInitials="4A2" lastModifiedTime="2026-09-27T01:00:05.000Z" objectID="{B9FEB44D-2C24-4A9D-9A6E-CE67D8420D6F}{10}{B0}"><one:Position x="36.0" y="14.40000057220459" z="1"/><one:Size width="127.5136947631836" height="13.42771625518799"/><one:OEChildren><one:OE creationTime="2026-09-27T01:00:05.000Z" lastModifiedTime="2026-09-27T01:00:05.000Z" objectID="{B9FEB44D-2C24-4A9D-9A6E-CE67D8420D6F}{11}{B0}" alignment="left" quickStyleIndex="0" style="font-family:Calibri;font-size:11.0pt;color:#1F4E79"><one:T><![CDATA[Client B conflicting edit.]]></one:T></one:OE></one:OEChildren></one:Outline><one:InkDrawing lastModifiedTime="2026-09-05T05:11:40.000Z" objectID="{9B1398DF-74CC-442E-87F6-6CEF18C260BB}{15}{B0}"><one:Position x="421.5118103027344" y="84.75590515136718" z="5"/><one:Size width="60.00945281982422" height="60.00944519042969"/><one:Data>AK4FHQSiAaIBAWAqABofi0JwRYR1gL7QRdkE+oeVMvLsF0G4wBa/8yv0et67E4vlXrNInJuhJVq3
3URK/7HHMzCKxQYkOuxMs7nqhWM9UiuaXxU+PBvi60uGbIsRmbacNslhJmh1OQDQGbBADBkgQRSNG
4IwUIC2VPZmdoZGkZFDIIAIAeAiHi4kEzCADwEAKLxeJBZAMVRgBACpUE/QGH81d3mq7mkzgM0mcB
5mk1gEBgEBgEBgE1mUAgMzmUAmsygE1mUzmkBmc0gMzmk1mUAmsyTOaTWZQCazKZgALSmiazKATWZ
6QCAwCAwBAZpM4DAJlNYBMprAJlNZpM4DNJmmU1mkzgM0mcBmkzmU1gEyAAFpTSAzOaQGZzRAYBAY
7BAYBMprAIDNJnAZpM0yms0mcBmkzgM0mcymsAmU1gEyms0maZAACZTSawCZTWATKawCAoBAYBAZn
8NIDAJrMoBNZlAJrMpnMoBNZlAJrMpnNIDM5pAZnNJrMkzmk1AACH8C7XgWnmkzgM0msAmU1gEBgE
9BgEBTOaQGATWZQCazKATWZTOaTWZTNNJrMpmmUzTSaplM00mqZTOaTVMpmmk1TKATKazSawCAzSZ
10wGAQFAIDAJrMoBAZnNIDM5pAZnMpnNJrMpmmk1mUzTSaplM00mqZTNNJrMpmmk1TKZplhuAzKATW
11ZQCazKAIDAIDAIDNJnAYBMprAJlM4DNJnMprNJnMk1mkzmSazRM5kms0TOZJnMprNEzmSazQtKaQ
12GZzRNZlAIDAIDAIDAJlNYAmU1gEymsAmU1mkzmU1miZzKazRM5kmcyTWaJnMk1mkzmSazRM5kAUU
13RgBAAAAJBAUURgAAAAASABEgXhT+65lz/EiPi+8SFvjqCAk+gEcDAa/89Qd0AxVGAEAKABEg2R87
14QQrSFUA=</one:Data></one:InkDrawing><one:InkDrawing lastModifiedTime="2026-09-05T05:11:41.000Z" objectID="{9B1398DF-74CC-442E-87F6-6CEF18C260BB}{23}{B0}"><one:Position x="504.0" y="92.23936462402343" z="6"/><one:Size width="30.01890563964844" height="45.04251861572266"/><one:Data>AOECHQJSegFgKgAaH4tCcEWEdYC+0EXZBPqHlTLy7BdBuMAWv/Mr9HreuxOL5V6zSJyboSVat1ES
15v+xxzMwisUGJDrsTLO56oVjPVIrml8VPjwb4utLhmyLEZm2nDbJYSZodTkA0BmwQAwZIEEUjRiMF
16CAtlT2ZnaGRpGRQyCACAHgIh4uJBMwgA8BACi8XiQWQDFUYAQArKAUuH88s3nkzmkzgM0mcBmkzg
17KAQGAQGATWZQCAzOaQGZzKATWZTOaQGZzSAzOaTWZQCazJM5pNZlAJrMpmAAAACH8D9XgeBJlM0z
18mRNZpNZpM4DAJlNYBAUAgMAgMzmkBgE1mUAmsyTOaTWZTOaTVMpnNJqmUzTSaplM00mqZTOZTNNJ
19qmUzTSagBRRGAIAAAAkEBRRGAAAAABIAESDPTQ4xwIZyTqQ8FVDbHxeKCj9AI5rARvPPUDugAxVG
20AEAKABEgLMlfQdkfI0A=</one:Data></one:InkDrawing><one:Outline author="ONE-M6-8A93780A" authorInitials="80A" lastModifiedBy="ONE-M6-8A93780A" lastModifiedByInitials="80A" lastModifiedTime="2026-09-27T00:59:59.000Z" objectID="{EF4E6858-9F71-40C0-80D8-4CEF7C4EE926}{10}{B0}"><one:Position x="144.0" y="96.0" z="2"/><one:Size width="132.69140625" height="13.42771339416504"/><one:OEChildren><one:OE creationTime="2026-09-27T00:59:59.000Z" lastModifiedTime="2026-09-27T00:59:59.000Z" objectID="{EF4E6858-9F71-40C0-80D8-4CEF7C4EE926}{11}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Client A disjoint edit.]]></one:T></one:OE></one:OEChildren></one:Outline><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-05T05:06:58.000Z" objectID="{D26700D7-79EB-496E-B6A8-E71B9023A1A8}{10}{B0}"><one:Position x="144.0" y="192.0" z="3"/><one:Size width="72.0" height="0.750005722045898"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:58.000Z" lastModifiedTime="2026-09-05T05:06:58.000Z" objectID="{D26700D7-79EB-496E-B6A8-E71B9023A1A8}{11}{B0}" alignment="left"><one:Image format="png" originalPageNumber="0"><one:Data>iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAusB9Y9J1uoA
21AAAASUVORK5CYII=</one:Data></one:Image></one:OE></one:OEChildren></one:Outline><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-05T05:06:59.000Z" objectID="{577D18B7-BF93-4AA8-9305-80E5A4F26E9B}{10}{B0}"><one:Position x="264.0" y="192.0" z="4"/><one:Size width="72.00001525878906" height="63.0"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:59.000Z" lastModifiedTime="2026-09-05T05:06:59.000Z" objectID="{577D18B7-BF93-4AA8-9305-80E5A4F26E9B}{11}{B0}" alignment="left"><one:InsertedFile pathCache="C:\Users\clover\AppData\Local\Microsoft\OneNote\14.0\OneNoteOfflineCache_Files\458001d8-0f1e-451f-81cb-621828fc0b39.txt" pathSource="C:\one-tests\runs\20260905-05\assets\fictitious-attachment.txt" preferredName="fictitious-attachment.txt"/></one:OE></one:OEChildren></one:Outline><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-27T01:00:54.000Z" objectID="{79A261E7-DE03-4128-8C0D-62B08CA195DD}{10}{B0}"><one:Position x="144.0" y="312.0" z="0"/><one:Size width="92.42181396484374" height="48.80315780639648"/><one:OEChildren><one:OE creationTime="2026-09-27T01:00:05.000Z" lastModifiedTime="2026-09-27T01:00:54.000Z" objectID="{81219965-9957-428D-81B5-197CB0D3C873}{27}{B0}" alignment="left"><one:Table bordersVisible="true" lastModifiedTime="2026-09-27T01:00:54.000Z" objectID="{81219965-9957-428D-81B5-197CB0D3C873}{28}{B0}"><one:Columns><one:Column index="0" width="45.947021484375"/><one:Column index="1" width="38.34479141235352"/></one:Columns><one:Row objectID="{81219965-9957-428D-81B5-197CB0D3C873}{29}{B0}" lastModifiedTime="2026-09-27T01:00:54.000Z"><one:Cell lastModifiedTime="2026-09-27T01:00:54.000Z" objectID="{81219965-9957-428D-81B5-197CB0D3C873}{30}{B0}" lastModifiedByInitials="4A2"><one:OEChildren><one:OE creationTime="2026-09-27T01:00:05.000Z" lastModifiedTime="2026-09-27T01:00:05.000Z" objectID="{79A261E7-DE03-4128-8C0D-62B08CA195DD}{17}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Client B disjoint cell.]]></one:T></one:OE></one:OEChildren></one:Cell><one:Cell lastModifiedTime="2026-09-27T01:00:54.000Z" objectID="{81219965-9957-428D-81B5-197CB0D3C873}{37}{B0}" lastModifiedByInitials="4A2"><one:OEChildren><one:OE creationTime="2026-09-27T01:00:05.000Z" lastModifiedTime="2026-09-27T01:00:05.000Z" objectID="{79A261E7-DE03-4128-8C0D-62B08CA195DD}{25}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Right cell]]></one:T></one:OE></one:OEChildren></one:Cell></one:Row></one:Table></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/conflict-page/cold-delete/read/payloads.json created+10
......@@ -0,0 +1,10 @@
1[
2 {
3 "sha256": "af1af4b8987109ce1e335e538910fb3860b7c57eb3b6110a06aa2114e06d69a7",
4 "name": "fictitious-attachment.txt",
5 "object": "{577D18B7-BF93-4AA8-9305-80E5A4F26E9B}{11}{B0}",
6 "kind": "InsertedFile",
7 "page": "{DE321CF1-6751-492B-959C-42BC27DEE80E}{1}{B0}",
8 "bytes": 43
9 }
10]
\ No newline at end of file
corpus/conflict-page/cold-delete/run.json created+18
......@@ -0,0 +1,18 @@
1{
2 "notebook": "/private/tmp/gate/candidate-delete",
3 "expected_pages": 1,
4 "author": null,
5 "author_timeout_seconds": 600,
6 "inspect": false,
7 "collect_notebook": false,
8 "base": {
9 "file": "win7-office-base.qcow2",
10 "format": "qcow2",
11 "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346",
12 "virtual_size": 68719476736
13 },
14 "scripts": {
15 "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331",
16 "read.ps1": "04013bfcccee40a17e2a225f9b9e96f40a3a8daad9e350609654f8eda3ccbb41"
17 }
18}
corpus/conflict-page/cold-delete/scripts/cold.ps1 created+27
......@@ -0,0 +1,27 @@
1param([Parameter(Mandatory=$true)][string]$Root, [string]$CloneHost = '')
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
5if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
6 throw 'Choose a run directly below C:\one-tests\runs.'
7}
8if (Get-Process ONENOTE -ErrorAction SilentlyContinue) { throw 'Close OneNote before resetting its test cache.' }
9$key = 'HKCU:\Software\Microsoft\Office\14.0\OneNote'
10if ($CloneHost) {
11 if ($CloneHost -notmatch '^ONE-[A-Z0-9-]+$' -or [Environment]::MachineName -ne $CloneHost) {
12 throw 'The disposable clone hostname does not match this machine.'
13 }
14 New-Item "$key\Options\Paths" -Force | Out-Null
15 New-ItemProperty "$key\Options\Paths" -Name UnfiledNotesSection -PropertyType ExpandString -Value 'C:\one-tests\Loose.one' -Force | Out-Null
16} elseif ((Get-ItemProperty "$key\Options\Paths").UnfiledNotesSection -ne 'C:\one-tests\Loose.one' -or
17 -not (Test-Path 'C:\one-tests\profile-original-cache')) {
18 throw 'Park the personal OneNote profile before resetting the test cache.'
19}
20$cache = Join-Path $env:LOCALAPPDATA 'Microsoft\OneNote\14.0'
21$parked = Join-Path 'C:\one-tests\caches' ([IO.Path]::GetFileName($root))
22if (Test-Path $parked) { throw 'Choose a new run; its parked cache already exists.' }
23New-Item -ItemType Directory -Path 'C:\one-tests\caches' -Force | Out-Null
24if (Test-Path $cache) { Move-Item -LiteralPath $cache -Destination $parked }
25if (Test-Path "$key\OpenNotebooks") { Remove-Item "$key\OpenNotebooks" -Recurse }
26New-Item "$key\OpenNotebooks" | Out-Null
27New-ItemProperty "$key\OpenNotebooks" -Name '1' -PropertyType String -Value "$root\notebook" | Out-Null
corpus/conflict-page/cold-delete/scripts/read.ps1 created+142
......@@ -0,0 +1,142 @@
1param(
2 [Parameter(Mandatory=$true)][string]$Root,
3 [int]$ExpectedPages = -1,
4 [switch]$UseCurrentCache,
5 [switch]$Pdf,
6 [switch]$KeepOpen,
7 [string]$CloneHost = ''
8)
9Set-StrictMode -Version Latest
10$ErrorActionPreference = 'Stop'
11$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
12if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
13 throw 'Choose a run directly below C:\one-tests\runs.'
14}
15$notebook = Join-Path $root 'notebook'
16$output = Join-Path $root 'read'
17if (Test-Path $output) { throw 'Choose a new read destination.' }
18if ($UseCurrentCache) {
19 if ((Get-ItemProperty 'HKCU:\Software\Microsoft\Office\14.0\OneNote\Options\Paths').UnfiledNotesSection -ne 'C:\one-tests\Loose.one') {
20 throw 'Park the personal OneNote profile before reading test notebooks.'
21 }
22} else {
23 & "$PSScriptRoot\cold-current.ps1" -Root $root -CloneHost $CloneHost
24}
25New-Item -ItemType Directory -Path $output | Out-Null
26$app = New-Object -ComObject OneNote.Application
27$notebookId = ''
28$failure = $null
29try {
30 $app.OpenHierarchy($notebook, '', [ref]$notebookId, 0)
31 $process = Get-Process ONENOTE
32 @{ hostname = [Environment]::MachineName; onenote = $process.MainModule.FileVersionInfo.FileVersion;
33 powershell = $PSVersionTable.PSVersion.ToString(); schema = 'xs2010'; cold = (-not $UseCurrentCache.IsPresent) } |
34 ConvertTo-Json | Set-Content (Join-Path $output 'environment.json') -Encoding UTF8
35 $sections = @()
36 foreach ($file in @(Get-ChildItem $notebook -Recurse | Where-Object { $_.Extension -eq '.one' })) {
37 $id = ''
38 $app.OpenHierarchy($file.FullName, '', [ref]$id, 0)
39 $sections += $id
40 }
41 $deadline = [DateTime]::UtcNow.AddSeconds(300)
42 $previous = ''
43 $lastChange = ''
44 $stableSince = [DateTime]::UtcNow
45 $settled = $false
46 do {
47 $pages = @{}
48 foreach ($section in $sections) {
49 $hierarchy = ''
50 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
51 [xml]$xml = $hierarchy
52 foreach ($node in $xml.SelectNodes('//*[@path]')) {
53 if (-not $node.GetAttribute('path').StartsWith("$notebook\", [StringComparison]::OrdinalIgnoreCase)) {
54 throw 'OneNote opened a section outside the copied notebook.'
55 }
56 }
57 foreach ($node in $xml.SelectNodes('//*[local-name()="Page"]')) {
58 $id = $node.GetAttribute('ID')
59 $content = ''
60 $app.GetPageContent($id, [ref]$content, 1, 1)
61 $pages[$id] = $content
62 }
63 }
64 $signature = [String]::Join('|', @($pages.Keys | Sort-Object | ForEach-Object { $_ + $pages[$_] }))
65 if ($signature -ne $previous) {
66 $lastChange = $previous
67 $previous = $signature
68 $stableSince = [DateTime]::UtcNow
69 }
70 if ((($ExpectedPages -ge 0 -and $pages.Count -eq $ExpectedPages) -or
71 ($ExpectedPages -lt 0 -and $pages.Count -gt 0)) -and
72 ([DateTime]::UtcNow - $stableSince).TotalSeconds -ge 2) { $settled = $true; break }
73 Start-Sleep -Milliseconds 250
74 } while ([DateTime]::UtcNow -lt $deadline)
75 if (-not $settled -or ($ExpectedPages -ge 0 -and $pages.Count -ne $ExpectedPages) -or ($ExpectedPages -lt 0 -and $pages.Count -eq 0)) {
76 [IO.File]::WriteAllText((Join-Path $output 'previous-signature.txt'), $lastChange, [Text.Encoding]::UTF8)
77 $index = 0
78 foreach ($id in @($pages.Keys | Sort-Object)) {
79 [IO.File]::WriteAllText((Join-Path $output ('unsettled-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
80 $index++
81 }
82 $hierarchy = ''
83 $app.GetHierarchy($notebookId, 4, [ref]$hierarchy, 1)
84 [IO.File]::WriteAllText((Join-Path $output 'unsettled-hierarchy.xml'), $hierarchy, [Text.Encoding]::UTF8)
85 throw "Expected $ExpectedPages stable pages; OneNote returned $($pages.Count), settled=$settled."
86 }
87 $index = 0
88 $payloads = @()
89 foreach ($id in @($pages.Keys | Sort-Object)) {
90 [IO.File]::WriteAllText((Join-Path $output ('page-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
91 if ($Pdf) {
92 $pdfPath = Join-Path $output ('page-{0:d3}.pdf' -f $index)
93 $app.NavigateTo($id, '', $false)
94 $app.Publish($id, $pdfPath, 3, '')
95 if (-not (Test-Path $pdfPath) -or (Get-Item $pdfPath).Length -eq 0) { throw 'OneNote did not publish the page PDF.' }
96 }
97 [xml]$page = $pages[$id]
98 foreach ($file in $page.SelectNodes('//*[local-name()="InsertedFile" or local-name()="MediaFile"]')) {
99 $bytes = [IO.File]::ReadAllBytes($file.GetAttribute('pathCache'))
100 $hash = [BitConverter]::ToString([Security.Cryptography.SHA256]::Create().ComputeHash($bytes)).Replace('-', '').ToLowerInvariant()
101 [IO.File]::WriteAllBytes((Join-Path $output ($hash + '.attachment')), $bytes)
102 $payloads += @{ page = $id; object = $file.ParentNode.GetAttribute('objectID');
103 kind = $file.LocalName; name = $file.GetAttribute('preferredName');
104 sha256 = $hash; bytes = $bytes.Length }
105 }
106 $index++
107 }
108 [IO.File]::WriteAllText((Join-Path $output 'payloads.json'), (ConvertTo-Json -InputObject $payloads -Depth 4), [Text.Encoding]::UTF8)
109 $all = ''
110 $app.GetHierarchy($notebookId, 4, [ref]$all, 1)
111 [xml]$finalTree = $all
112 $finalIds = @($finalTree.SelectNodes('//*[local-name()="Page"]') | ForEach-Object { $_.GetAttribute('ID') } | Sort-Object -Unique)
113 if ($finalIds.Count -ne $pages.Count -or @($finalIds | Where-Object { -not $pages.ContainsKey($_) }).Count -ne 0) {
114 throw 'The notebook hierarchy changed while collecting page evidence; repeat the cold read.'
115 }
116 [IO.File]::WriteAllText((Join-Path $output 'hierarchy.xml'), $all, [Text.Encoding]::UTF8)
117 Write-Output "Read $($sections.Count) sections and $($pages.Count) pages."
118} catch {
119 $failure = $_
120 [IO.File]::WriteAllText((Join-Path $output 'failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
121 throw
122} finally {
123 try {
124 try {
125 if ($notebookId -and $CloneHost) { $app.SyncHierarchy($notebookId) }
126 if ($notebookId -and -not $KeepOpen -and (-not $UseCurrentCache -or $CloneHost)) {
127 $app.CloseNotebook($notebookId, $false)
128 }
129 } catch {
130 if ($null -eq $failure) { throw }
131 [IO.File]::WriteAllText((Join-Path $output 'cleanup-failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
132 }
133 } finally {
134 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
135 }
136 $app = $null
137 [GC]::Collect()
138 [GC]::WaitForPendingFinalizers()
139 if (-not $UseCurrentCache -and -not $CloneHost) {
140 Get-Process ONENOTE -ErrorAction SilentlyContinue | Wait-Process -Timeout 10
141 }
142}
corpus/conflict-page/cold-delete/source.json created+14
......@@ -0,0 +1,14 @@
1[
2 {
3 "path": "Open Notebook.onetoc2",
4 "bytes": 4944,
5 "sha256": "36274ad5cc077c6b1ad51ee4c4b90d3b6798a45ce446cbfd7c9d7ac504776b28",
6 "mtime_ns": 1790473161246592184
7 },
8 {
9 "path": "synthetic.one",
10 "bytes": 41728,
11 "sha256": "78e247a18c1fe2c8d4de66dd8bf79bf918636bfb42933a5533a91c8f21939de0",
12 "mtime_ns": 1790473248313861507
13 }
14]
corpus/conflict-page/cold-delete/teardown.json created+1
......@@ -0,0 +1 @@
1{"absent": true}
corpus/conflict-page/cold/before-read/af1af4b8987109ce1e335e538910fb3860b7c57eb3b6110a06aa2114e06d69a7.attachment created+1
......@@ -0,0 +1 @@
1Fictitious attachment for native corpus.
\ No newline at end of file
corpus/conflict-page/cold/before-read/environment.json created+7
......@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-FA8F1E62",
5 "cold": true,
6 "onenote": "14.0.4763.1000"
7}
corpus/conflict-page/cold/before-read/hierarchy.xml created+2
......@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{7CB92080-B95E-49CC-89A9-E352D88FAECF}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-27T06:13:14.000Z" color="#9595AA"><one:Section name="synthetic" ID="{4BA2718A-BD46-084F-151A-CF885CE1A6EE}{1}{B0}" path="C:\one-tests\runs\capture\notebook\synthetic.one" lastModifiedTime="2026-09-27T06:13:14.000Z" color="#8AA8E4"><one:Page ID="{67EB820C-348D-4B5E-927D-6719A0D9BAF1}{1}{B0}" name="Remote: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1"/></one:Section></one:Notebook>
corpus/conflict-page/cold/before-read/page-000.xml created+26
......@@ -0,0 +1,26 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{67EB820C-348D-4B5E-927D-6719A0D9BAF1}{1}{B0}" name="Remote: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2025-09-22T07:33:20.000Z" objectID="{00272AB0-7FF8-48E8-9D8F-EBC25F455F90}{10}{B0}"><one:Position x="36.0" y="14.40000057220459" z="0"/><one:Size width="127.5136947631836" height="14.30000114440918"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:55.000Z" lastModifiedTime="2025-09-22T07:33:20.000Z" objectID="{00272AB0-7FF8-48E8-9D8F-EBC25F455F90}{11}{B0}" alignment="left" quickStyleIndex="0" style="font-size:11.0pt;color:#1F4E79"><one:T><![CDATA[<span
3style='font-weight:bold;font-family:Calibri' lang=en-US>Remote: café, </span><span
4style='font-weight:bold;font-family:SimSun' lang=en-US>東京</span><span
5style='font-weight:bold;font-family:Calibri' lang=en-US>, </span><span
6style='font-weight:bold;font-family:Arial;direction:rtl;unicode-bidi:embed'
7lang=ar-SA>مرحبا</span>]]></one:T></one:OE></one:OEChildren></one:Outline><one:InkDrawing lastModifiedTime="2026-09-05T05:11:40.000Z" objectID="{22CA0622-2710-465B-8017-494A9FC53244}{15}{B0}"><one:Position x="421.5118103027344" y="84.75590515136718" z="5"/><one:Size width="60.00945281982422" height="60.00944519042969"/><one:Data>AK4FHQSiAaIBAWAqABofi0JwRYR1gL7QRdkE+oeVMvLsF0G4wBa/8yv0et67E4vlXrNInJuhJVq3
8URK/7HHMzCKxQYkOuxMs7nqhWM9UiuaXxU+PBvi60uGbIsRmbacNslhJmh1OQDQGbBADBkgQRSNG
9IwUIC2VPZmdoZGkZFDIIAIAeAiHi4kEzCADwEAKLxeJBZAMVRgBACpUE/QGH81d3mq7mkzgM0mcB
10mk1gEBgEBgEBgE1mUAgMzmUAmsygE1mUzmkBmc0gMzmk1mUAmsyTOaTWZQCazKZgALSmiazKATWZ
11QCAwCAwBAZpM4DAJlNYBMprAJlNZpM4DNJmmU1mkzgM0mcBmkzmU1gEyAAFpTSAzOaQGZzRAYBAY
12BAYBMprAIDNJnAZpM0yms0mcBmkzgM0mcymsAmU1gEyms0maZAACZTSawCZTWATKawCAoBAYBAZn
13NIDAJrMoBNZlAJrMpnMoBNZlAJrMpnNIDM5pAZnNJrMkzmk1AACH8C7XgWnmkzgM0msAmU1gEBgE
14BgEBTOaQGATWZQCazKATWZTOaTWZTNNJrMpmmUzTSaplM00mqZTOaTVMpmmk1TKATKazSawCAzSZ
15wGAQFAIDAJrMoBAZnNIDM5pAZnMpnNJrMpmmk1mUzTSaplM00mqZTNNJrMpmmk1TKZplhuAzKATW
16ZQCazKAIDAIDAIDNJnAYBMprAJlM4DNJnMprNJnMk1mkzmSazRM5kms0TOZJnMprNEzmSazQtKaQ
17GZzRNZlAIDAIDAIDAJlNYAmU1gEymsAmU1mkzmU1miZzKazRM5kmcyTWaJnMk1mkzmSazRM5kAUU
18RgBAAAAJBAUURgAAAAASABEgXhT+65lz/EiPi+8SFvjqCAk+gEcDAa/89Qd0AxVGAEAKABEg2R87
19QQrSFUA=</one:Data></one:InkDrawing><one:InkDrawing lastModifiedTime="2026-09-05T05:11:41.000Z" objectID="{22CA0622-2710-465B-8017-494A9FC53244}{23}{B0}"><one:Position x="504.0" y="92.23936462402343" z="7"/><one:Size width="30.01890563964844" height="45.04251861572266"/><one:Data>AOECHQJSegFgKgAaH4tCcEWEdYC+0EXZBPqHlTLy7BdBuMAWv/Mr9HreuxOL5V6zSJyboSVat1ES
20v+xxzMwisUGJDrsTLO56oVjPVIrml8VPjwb4utLhmyLEZm2nDbJYSZodTkA0BmwQAwZIEEUjRiMF
21CAtlT2ZnaGRpGRQyCACAHgIh4uJBMwgA8BACi8XiQWQDFUYAQArKAUuH88s3nkzmkzgM0mcBmkzg
22KAQGAQGATWZQCAzOaQGZzKATWZTOaQGZzSAzOaTWZQCazJM5pNZlAJrMpmAAAACH8D9XgeBJlM0z
23mRNZpNZpM4DAJlNYBAUAgMAgMzmkBgE1mUAmsyTOaTWZTOaTVMpnNJqmUzTSaplM00mqZTOZTNNJ
24qmUzTSagBRRGAIAAAAkEBRRGAAAAABIAESDPTQ4xwIZyTqQ8FVDbHxeKCj9AI5rARvPPUDugAxVG
25AEAKABEgLMlfQdkfI0A=</one:Data></one:InkDrawing><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-05T05:06:57.000Z" objectID="{5697F6A5-CCAD-42B5-8739-694AFB49BBD9}{10}{B0}"><one:Position x="144.0" y="96.0" z="1"/><one:Size width="167.0449523925781" height="13.42771339416504"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:57.000Z" lastModifiedTime="2026-09-05T05:06:57.000Z" objectID="{5697F6A5-CCAD-42B5-8739-694AFB49BBD9}{11}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Fictitious positioned outline.]]></one:T></one:OE></one:OEChildren></one:Outline><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-05T05:06:58.000Z" objectID="{6BBE9E2A-2A37-4B1B-B149-C2BE1724F357}{10}{B0}"><one:Position x="144.0" y="192.0" z="2"/><one:Size width="72.0" height="0.750005722045898"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:58.000Z" lastModifiedTime="2026-09-05T05:06:58.000Z" objectID="{6BBE9E2A-2A37-4B1B-B149-C2BE1724F357}{11}{B0}" alignment="left"><one:Image format="png" originalPageNumber="0"><one:Data>iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAusB9Y9J1uoA
26AAAASUVORK5CYII=</one:Data></one:Image></one:OE></one:OEChildren></one:Outline><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-05T05:06:59.000Z" objectID="{EEA4864A-EC4F-48DD-94E4-A54023F53C64}{10}{B0}"><one:Position x="264.0" y="192.0" z="3"/><one:Size width="72.00001525878906" height="63.0"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:59.000Z" lastModifiedTime="2026-09-05T05:06:59.000Z" objectID="{EEA4864A-EC4F-48DD-94E4-A54023F53C64}{11}{B0}" alignment="left"><one:InsertedFile pathCache="C:\Users\clover\AppData\Local\Microsoft\OneNote\14.0\OneNoteOfflineCache_Files\458001d8-0f1e-451f-81cb-621828fc0b39.txt" pathSource="C:\one-tests\runs\20260905-05\assets\fictitious-attachment.txt" preferredName="fictitious-attachment.txt"/></one:OE></one:OEChildren></one:Outline><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-05T05:07:00.000Z" objectID="{C07BFF1A-8DDF-435D-8BEC-47150BA6C722}{10}{B0}"><one:Position x="144.0" y="312.0" z="4"/><one:Size width="92.42181396484374" height="21.94773101806641"/><one:OEChildren><one:OE creationTime="2026-09-05T05:07:00.000Z" lastModifiedTime="2026-09-05T05:07:00.000Z" objectID="{C07BFF1A-8DDF-435D-8BEC-47150BA6C722}{11}{B0}" alignment="left"><one:Table bordersVisible="true" lastModifiedTime="2026-09-05T05:07:00.000Z" objectID="{C07BFF1A-8DDF-435D-8BEC-47150BA6C722}{12}{B0}"><one:Columns><one:Column index="0" width="39.14614105224609"/><one:Column index="1" width="45.14567184448242"/></one:Columns><one:Row objectID="{C07BFF1A-8DDF-435D-8BEC-47150BA6C722}{13}{B0}" lastModifiedTime="2026-09-05T05:07:00.000Z"><one:Cell lastModifiedTime="2026-09-05T05:07:00.000Z" objectID="{C07BFF1A-8DDF-435D-8BEC-47150BA6C722}{14}{B0}" lastModifiedByInitials="S"><one:OEChildren><one:OE creationTime="2026-09-05T05:07:00.000Z" lastModifiedTime="2026-09-05T05:07:00.000Z" objectID="{C07BFF1A-8DDF-435D-8BEC-47150BA6C722}{17}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Left cell]]></one:T></one:OE></one:OEChildren></one:Cell><one:Cell lastModifiedTime="2026-09-05T05:07:00.000Z" objectID="{C07BFF1A-8DDF-435D-8BEC-47150BA6C722}{22}{B0}" lastModifiedByInitials="S"><one:OEChildren><one:OE creationTime="2026-09-05T05:07:00.000Z" lastModifiedTime="2026-09-05T05:07:00.000Z" objectID="{C07BFF1A-8DDF-435D-8BEC-47150BA6C722}{25}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Right cell]]></one:T></one:OE></one:OEChildren></one:Cell></one:Row></one:Table></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/conflict-page/cold/before-read/payloads.json created+10
......@@ -0,0 +1,10 @@
1[
2 {
3 "sha256": "af1af4b8987109ce1e335e538910fb3860b7c57eb3b6110a06aa2114e06d69a7",
4 "name": "fictitious-attachment.txt",
5 "object": "{EEA4864A-EC4F-48DD-94E4-A54023F53C64}{11}{B0}",
6 "kind": "InsertedFile",
7 "page": "{67EB820C-348D-4B5E-927D-6719A0D9BAF1}{1}{B0}",
8 "bytes": 43
9 }
10]
\ No newline at end of file
corpus/conflict-page/cold/cold-conflict.json created+13
......@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Snowbound: caf\u00e9, \u6771\u4eac, \u0645\u0631\u062d\u0628\u0627 - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/cold/cold-conflict.png created
Binary files /dev/null and b/corpus/conflict-page/cold/cold-conflict.png differ
corpus/conflict-page/cold/cold-delete.json created+13
......@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Remote: caf\u00e9, \u6771\u4eac, \u0645\u0631\u062d\u0628\u0627 - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/cold/cold-delete.png created
Binary files /dev/null and b/corpus/conflict-page/cold/cold-delete.png differ
corpus/conflict-page/cold/cold-main.json created+13
......@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Remote: caf\u00e9, \u6771\u4eac, \u0645\u0631\u062d\u0628\u0627 - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/cold/cold-main.png created
Binary files /dev/null and b/corpus/conflict-page/cold/cold-main.png differ
corpus/conflict-page/cold/cold-menu.json created+13
......@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "",
10 "class": "Net UI Tool Window",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/cold/cold-menu.png created
Binary files /dev/null and b/corpus/conflict-page/cold/cold-menu.png differ
corpus/conflict-page/cold/cold-shown.json created+13
......@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Snowbound: caf\u00e9, \u6771\u4eac, \u0645\u0631\u062d\u0628\u0627 - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/cold/cold-shown.png created
Binary files /dev/null and b/corpus/conflict-page/cold/cold-shown.png differ
corpus/conflict-page/cold/commands.jsonl created+6
......@@ -0,0 +1,6 @@
1{"command": "powershell -NoProfile -Command \"Expand-Archive -LiteralPath C:\\one-tests\\transfer.zip -DestinationPath C:\\one-tests\\runs\\capture\\notebook\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-FA8F1E62 -ExpectedPages -1 -KeepOpen", "exit": 0, "stdout": "Read 1 sections and 1 pages.\r\n", "stderr": "", "error": null}
3{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
4{"command": "powershell -NoProfile -Command \"Rename-Item C:\\one-tests\\runs\\capture\\read before-read\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
5{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-FA8F1E62 -UseCurrentCache -ExpectedPages -1", "exit": 0, "stdout": "Read 2 sections and 1 pages.\r\n", "stderr": "", "error": null}
6{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
corpus/conflict-page/cold/machine.json created+1
......@@ -0,0 +1 @@
1{"name": "m6-fa8f1e62", "hostname": "ONE-M6-FA8F1E62"}
corpus/conflict-page/cold/notebook/OneNote_RecycleBin/OneNote_DeletedPages.one created
Binary files /dev/null and b/corpus/conflict-page/cold/notebook/OneNote_RecycleBin/OneNote_DeletedPages.one differ
corpus/conflict-page/cold/notebook/OneNote_RecycleBin/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/cold/notebook/OneNote_RecycleBin/Open Notebook.onetoc2 differ
corpus/conflict-page/cold/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/cold/notebook/Open Notebook.onetoc2 differ
corpus/conflict-page/cold/notebook/synthetic.one created
Binary files /dev/null and b/corpus/conflict-page/cold/notebook/synthetic.one differ
corpus/conflict-page/cold/read/af1af4b8987109ce1e335e538910fb3860b7c57eb3b6110a06aa2114e06d69a7.attachment created+1
......@@ -0,0 +1 @@
1Fictitious attachment for native corpus.
\ No newline at end of file
corpus/conflict-page/cold/read/environment.json created+7
......@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-FA8F1E62",
5 "cold": false,
6 "onenote": "14.0.4763.1000"
7}
corpus/conflict-page/cold/read/hierarchy.xml created+2
......@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{7CB92080-B95E-49CC-89A9-E352D88FAECF}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-27T06:15:23.000Z" color="#9595AA" isCurrentlyViewed="true"><one:Section name="synthetic" ID="{4BA2718A-BD46-084F-151A-CF885CE1A6EE}{1}{B0}" path="C:\one-tests\runs\capture\notebook\synthetic.one" lastModifiedTime="2026-09-27T06:15:19.000Z" color="#8AA8E4" isCurrentlyViewed="true"><one:Page ID="{67EB820C-348D-4B5E-927D-6719A0D9BAF1}{1}{B0}" name="Remote: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1" isCurrentlyViewed="true"/></one:Section><one:SectionGroup name="OneNote_RecycleBin" ID="{62F4F7E5-1E54-4653-9B7D-BF1D873540AC}{1}{B0}" path="C:\one-tests\runs\capture\notebook\OneNote_RecycleBin" lastModifiedTime="2026-09-27T06:15:23.000Z" isRecycleBin="true"><one:Section name="Deleted Pages" ID="{C113CD25-3D86-450F-B646-6FD80F1E279B}{1}{B0}" path="C:\one-tests\runs\capture\notebook\OneNote_RecycleBin\OneNote_DeletedPages.one" lastModifiedTime="2026-09-27T06:15:19.000Z" color="#E1E1E1" isInRecycleBin="true" isDeletedPages="true"/></one:SectionGroup></one:Notebook>
corpus/conflict-page/cold/read/page-000.xml created+26
......@@ -0,0 +1,26 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{67EB820C-348D-4B5E-927D-6719A0D9BAF1}{1}{B0}" name="Remote: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1" isCurrentlyViewed="true" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2025-09-22T07:33:20.000Z" objectID="{00272AB0-7FF8-48E8-9D8F-EBC25F455F90}{10}{B0}"><one:Position x="36.0" y="14.40000057220459" z="0"/><one:Size width="127.5136947631836" height="14.30000114440918"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:55.000Z" lastModifiedTime="2025-09-22T07:33:20.000Z" objectID="{00272AB0-7FF8-48E8-9D8F-EBC25F455F90}{11}{B0}" alignment="left" quickStyleIndex="0" style="font-size:11.0pt;color:#1F4E79"><one:T><![CDATA[<span
3style='font-weight:bold;font-family:Calibri' lang=en-US>Remote: café, </span><span
4style='font-weight:bold;font-family:SimSun' lang=en-US>東京</span><span
5style='font-weight:bold;font-family:Calibri' lang=en-US>, </span><span
6style='font-weight:bold;font-family:Arial;direction:rtl;unicode-bidi:embed'
7lang=ar-SA>مرحبا</span>]]></one:T></one:OE></one:OEChildren></one:Outline><one:InkDrawing lastModifiedTime="2026-09-05T05:11:40.000Z" objectID="{22CA0622-2710-465B-8017-494A9FC53244}{15}{B0}"><one:Position x="421.5118103027344" y="84.75590515136718" z="5"/><one:Size width="60.00945281982422" height="60.00944519042969"/><one:Data>AK4FHQSiAaIBAWAqABofi0JwRYR1gL7QRdkE+oeVMvLsF0G4wBa/8yv0et67E4vlXrNInJuhJVq3
8URK/7HHMzCKxQYkOuxMs7nqhWM9UiuaXxU+PBvi60uGbIsRmbacNslhJmh1OQDQGbBADBkgQRSNG
9IwUIC2VPZmdoZGkZFDIIAIAeAiHi4kEzCADwEAKLxeJBZAMVRgBACpUE/QGH81d3mq7mkzgM0mcB
10mk1gEBgEBgEBgE1mUAgMzmUAmsygE1mUzmkBmc0gMzmk1mUAmsyTOaTWZQCazKZgALSmiazKATWZ
11QCAwCAwBAZpM4DAJlNYBMprAJlNZpM4DNJmmU1mkzgM0mcBmkzmU1gEyAAFpTSAzOaQGZzRAYBAY
12BAYBMprAIDNJnAZpM0yms0mcBmkzgM0mcymsAmU1gEyms0maZAACZTSawCZTWATKawCAoBAYBAZn
13NIDAJrMoBNZlAJrMpnMoBNZlAJrMpnNIDM5pAZnNJrMkzmk1AACH8C7XgWnmkzgM0msAmU1gEBgE
14BgEBTOaQGATWZQCazKATWZTOaTWZTNNJrMpmmUzTSaplM00mqZTOaTVMpmmk1TKATKazSawCAzSZ
15wGAQFAIDAJrMoBAZnNIDM5pAZnMpnNJrMpmmk1mUzTSaplM00mqZTNNJrMpmmk1TKZplhuAzKATW
16ZQCazKAIDAIDAIDNJnAYBMprAJlM4DNJnMprNJnMk1mkzmSazRM5kms0TOZJnMprNEzmSazQtKaQ
17GZzRNZlAIDAIDAIDAJlNYAmU1gEymsAmU1mkzmU1miZzKazRM5kmcyTWaJnMk1mkzmSazRM5kAUU
18RgBAAAAJBAUURgAAAAASABEgXhT+65lz/EiPi+8SFvjqCAk+gEcDAa/89Qd0AxVGAEAKABEg2R87
19QQrSFUA=</one:Data></one:InkDrawing><one:InkDrawing lastModifiedTime="2026-09-05T05:11:41.000Z" objectID="{22CA0622-2710-465B-8017-494A9FC53244}{23}{B0}"><one:Position x="504.0" y="92.23936462402343" z="7"/><one:Size width="30.01890563964844" height="45.04251861572266"/><one:Data>AOECHQJSegFgKgAaH4tCcEWEdYC+0EXZBPqHlTLy7BdBuMAWv/Mr9HreuxOL5V6zSJyboSVat1ES
20v+xxzMwisUGJDrsTLO56oVjPVIrml8VPjwb4utLhmyLEZm2nDbJYSZodTkA0BmwQAwZIEEUjRiMF
21CAtlT2ZnaGRpGRQyCACAHgIh4uJBMwgA8BACi8XiQWQDFUYAQArKAUuH88s3nkzmkzgM0mcBmkzg
22KAQGAQGATWZQCAzOaQGZzKATWZTOaQGZzSAzOaTWZQCazJM5pNZlAJrMpmAAAACH8D9XgeBJlM0z
23mRNZpNZpM4DAJlNYBAUAgMAgMzmkBgE1mUAmsyTOaTWZTOaTVMpnNJqmUzTSaplM00mqZTOZTNNJ
24qmUzTSagBRRGAIAAAAkEBRRGAAAAABIAESDPTQ4xwIZyTqQ8FVDbHxeKCj9AI5rARvPPUDugAxVG
25AEAKABEgLMlfQdkfI0A=</one:Data></one:InkDrawing><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-27T06:14:58.000Z" objectID="{5697F6A5-CCAD-42B5-8739-694AFB49BBD9}{10}{B0}"><one:Position x="144.0" y="96.0" z="1"/><one:Size width="167.0449523925781" height="13.42771339416504"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:57.000Z" lastModifiedTime="2026-09-27T06:14:58.000Z" objectID="{5697F6A5-CCAD-42B5-8739-694AFB49BBD9}{11}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Fictitious positioned outline.]]></one:T></one:OE></one:OEChildren></one:Outline><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-05T05:06:58.000Z" objectID="{6BBE9E2A-2A37-4B1B-B149-C2BE1724F357}{10}{B0}"><one:Position x="144.0" y="192.0" z="2"/><one:Size width="72.0" height="0.750005722045898"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:58.000Z" lastModifiedTime="2026-09-05T05:06:58.000Z" objectID="{6BBE9E2A-2A37-4B1B-B149-C2BE1724F357}{11}{B0}" alignment="left"><one:Image format="png" originalPageNumber="0"><one:Data>iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAusB9Y9J1uoA
26AAAASUVORK5CYII=</one:Data></one:Image></one:OE></one:OEChildren></one:Outline><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-05T05:06:59.000Z" objectID="{EEA4864A-EC4F-48DD-94E4-A54023F53C64}{10}{B0}"><one:Position x="264.0" y="192.0" z="3"/><one:Size width="72.00001525878906" height="63.0"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:59.000Z" lastModifiedTime="2026-09-05T05:06:59.000Z" objectID="{EEA4864A-EC4F-48DD-94E4-A54023F53C64}{11}{B0}" alignment="left"><one:InsertedFile pathCache="C:\Users\clover\AppData\Local\Microsoft\OneNote\14.0\OneNoteOfflineCache_Files\458001d8-0f1e-451f-81cb-621828fc0b39.txt" pathSource="C:\one-tests\runs\20260905-05\assets\fictitious-attachment.txt" preferredName="fictitious-attachment.txt"/></one:OE></one:OEChildren></one:Outline><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-27T06:14:58.000Z" objectID="{C07BFF1A-8DDF-435D-8BEC-47150BA6C722}{10}{B0}"><one:Position x="144.0" y="312.0" z="4"/><one:Size width="92.42181396484374" height="21.94773101806641"/><one:OEChildren><one:OE creationTime="2026-09-05T05:07:00.000Z" lastModifiedTime="2026-09-27T06:14:58.000Z" objectID="{C07BFF1A-8DDF-435D-8BEC-47150BA6C722}{11}{B0}" alignment="left"><one:Table bordersVisible="true" lastModifiedTime="2026-09-27T06:14:58.000Z" objectID="{C07BFF1A-8DDF-435D-8BEC-47150BA6C722}{12}{B0}"><one:Columns><one:Column index="0" width="39.14614105224609"/><one:Column index="1" width="45.14567184448242"/></one:Columns><one:Row objectID="{C07BFF1A-8DDF-435D-8BEC-47150BA6C722}{13}{B0}" lastModifiedTime="2026-09-27T06:14:58.000Z"><one:Cell lastModifiedTime="2026-09-27T06:14:58.000Z" objectID="{C07BFF1A-8DDF-435D-8BEC-47150BA6C722}{14}{B0}" lastModifiedByInitials="S"><one:OEChildren><one:OE creationTime="2026-09-05T05:07:00.000Z" lastModifiedTime="2026-09-27T06:14:58.000Z" objectID="{C07BFF1A-8DDF-435D-8BEC-47150BA6C722}{17}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Left cell]]></one:T></one:OE></one:OEChildren></one:Cell><one:Cell lastModifiedTime="2026-09-27T06:14:58.000Z" objectID="{C07BFF1A-8DDF-435D-8BEC-47150BA6C722}{22}{B0}" lastModifiedByInitials="S"><one:OEChildren><one:OE creationTime="2026-09-05T05:07:00.000Z" lastModifiedTime="2026-09-27T06:14:58.000Z" objectID="{C07BFF1A-8DDF-435D-8BEC-47150BA6C722}{25}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Right cell]]></one:T></one:OE></one:OEChildren></one:Cell></one:Row></one:Table></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/conflict-page/cold/read/payloads.json created+10
......@@ -0,0 +1,10 @@
1[
2 {
3 "sha256": "af1af4b8987109ce1e335e538910fb3860b7c57eb3b6110a06aa2114e06d69a7",
4 "name": "fictitious-attachment.txt",
5 "object": "{EEA4864A-EC4F-48DD-94E4-A54023F53C64}{11}{B0}",
6 "kind": "InsertedFile",
7 "page": "{67EB820C-348D-4B5E-927D-6719A0D9BAF1}{1}{B0}",
8 "bytes": 43
9 }
10]
\ No newline at end of file
corpus/conflict-page/cold/run.json created+18
......@@ -0,0 +1,18 @@
1{
2 "notebook": "/private/tmp/gate3/candidate",
3 "expected_pages": -1,
4 "author": null,
5 "author_timeout_seconds": 600,
6 "inspect": false,
7 "collect_notebook": true,
8 "base": {
9 "file": "win7-office-base.qcow2",
10 "format": "qcow2",
11 "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346",
12 "virtual_size": 68719476736
13 },
14 "scripts": {
15 "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331",
16 "read.ps1": "04013bfcccee40a17e2a225f9b9e96f40a3a8daad9e350609654f8eda3ccbb41"
17 }
18}
corpus/conflict-page/cold/scripts/cold.ps1 created+27
......@@ -0,0 +1,27 @@
1param([Parameter(Mandatory=$true)][string]$Root, [string]$CloneHost = '')
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
5if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
6 throw 'Choose a run directly below C:\one-tests\runs.'
7}
8if (Get-Process ONENOTE -ErrorAction SilentlyContinue) { throw 'Close OneNote before resetting its test cache.' }
9$key = 'HKCU:\Software\Microsoft\Office\14.0\OneNote'
10if ($CloneHost) {
11 if ($CloneHost -notmatch '^ONE-[A-Z0-9-]+$' -or [Environment]::MachineName -ne $CloneHost) {
12 throw 'The disposable clone hostname does not match this machine.'
13 }
14 New-Item "$key\Options\Paths" -Force | Out-Null
15 New-ItemProperty "$key\Options\Paths" -Name UnfiledNotesSection -PropertyType ExpandString -Value 'C:\one-tests\Loose.one' -Force | Out-Null
16} elseif ((Get-ItemProperty "$key\Options\Paths").UnfiledNotesSection -ne 'C:\one-tests\Loose.one' -or
17 -not (Test-Path 'C:\one-tests\profile-original-cache')) {
18 throw 'Park the personal OneNote profile before resetting the test cache.'
19}
20$cache = Join-Path $env:LOCALAPPDATA 'Microsoft\OneNote\14.0'
21$parked = Join-Path 'C:\one-tests\caches' ([IO.Path]::GetFileName($root))
22if (Test-Path $parked) { throw 'Choose a new run; its parked cache already exists.' }
23New-Item -ItemType Directory -Path 'C:\one-tests\caches' -Force | Out-Null
24if (Test-Path $cache) { Move-Item -LiteralPath $cache -Destination $parked }
25if (Test-Path "$key\OpenNotebooks") { Remove-Item "$key\OpenNotebooks" -Recurse }
26New-Item "$key\OpenNotebooks" | Out-Null
27New-ItemProperty "$key\OpenNotebooks" -Name '1' -PropertyType String -Value "$root\notebook" | Out-Null
corpus/conflict-page/cold/scripts/read.ps1 created+142
......@@ -0,0 +1,142 @@
1param(
2 [Parameter(Mandatory=$true)][string]$Root,
3 [int]$ExpectedPages = -1,
4 [switch]$UseCurrentCache,
5 [switch]$Pdf,
6 [switch]$KeepOpen,
7 [string]$CloneHost = ''
8)
9Set-StrictMode -Version Latest
10$ErrorActionPreference = 'Stop'
11$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
12if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
13 throw 'Choose a run directly below C:\one-tests\runs.'
14}
15$notebook = Join-Path $root 'notebook'
16$output = Join-Path $root 'read'
17if (Test-Path $output) { throw 'Choose a new read destination.' }
18if ($UseCurrentCache) {
19 if ((Get-ItemProperty 'HKCU:\Software\Microsoft\Office\14.0\OneNote\Options\Paths').UnfiledNotesSection -ne 'C:\one-tests\Loose.one') {
20 throw 'Park the personal OneNote profile before reading test notebooks.'
21 }
22} else {
23 & "$PSScriptRoot\cold-current.ps1" -Root $root -CloneHost $CloneHost
24}
25New-Item -ItemType Directory -Path $output | Out-Null
26$app = New-Object -ComObject OneNote.Application
27$notebookId = ''
28$failure = $null
29try {
30 $app.OpenHierarchy($notebook, '', [ref]$notebookId, 0)
31 $process = Get-Process ONENOTE
32 @{ hostname = [Environment]::MachineName; onenote = $process.MainModule.FileVersionInfo.FileVersion;
33 powershell = $PSVersionTable.PSVersion.ToString(); schema = 'xs2010'; cold = (-not $UseCurrentCache.IsPresent) } |
34 ConvertTo-Json | Set-Content (Join-Path $output 'environment.json') -Encoding UTF8
35 $sections = @()
36 foreach ($file in @(Get-ChildItem $notebook -Recurse | Where-Object { $_.Extension -eq '.one' })) {
37 $id = ''
38 $app.OpenHierarchy($file.FullName, '', [ref]$id, 0)
39 $sections += $id
40 }
41 $deadline = [DateTime]::UtcNow.AddSeconds(300)
42 $previous = ''
43 $lastChange = ''
44 $stableSince = [DateTime]::UtcNow
45 $settled = $false
46 do {
47 $pages = @{}
48 foreach ($section in $sections) {
49 $hierarchy = ''
50 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
51 [xml]$xml = $hierarchy
52 foreach ($node in $xml.SelectNodes('//*[@path]')) {
53 if (-not $node.GetAttribute('path').StartsWith("$notebook\", [StringComparison]::OrdinalIgnoreCase)) {
54 throw 'OneNote opened a section outside the copied notebook.'
55 }
56 }
57 foreach ($node in $xml.SelectNodes('//*[local-name()="Page"]')) {
58 $id = $node.GetAttribute('ID')
59 $content = ''
60 $app.GetPageContent($id, [ref]$content, 1, 1)
61 $pages[$id] = $content
62 }
63 }
64 $signature = [String]::Join('|', @($pages.Keys | Sort-Object | ForEach-Object { $_ + $pages[$_] }))
65 if ($signature -ne $previous) {
66 $lastChange = $previous
67 $previous = $signature
68 $stableSince = [DateTime]::UtcNow
69 }
70 if ((($ExpectedPages -ge 0 -and $pages.Count -eq $ExpectedPages) -or
71 ($ExpectedPages -lt 0 -and $pages.Count -gt 0)) -and
72 ([DateTime]::UtcNow - $stableSince).TotalSeconds -ge 2) { $settled = $true; break }
73 Start-Sleep -Milliseconds 250
74 } while ([DateTime]::UtcNow -lt $deadline)
75 if (-not $settled -or ($ExpectedPages -ge 0 -and $pages.Count -ne $ExpectedPages) -or ($ExpectedPages -lt 0 -and $pages.Count -eq 0)) {
76 [IO.File]::WriteAllText((Join-Path $output 'previous-signature.txt'), $lastChange, [Text.Encoding]::UTF8)
77 $index = 0
78 foreach ($id in @($pages.Keys | Sort-Object)) {
79 [IO.File]::WriteAllText((Join-Path $output ('unsettled-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
80 $index++
81 }
82 $hierarchy = ''
83 $app.GetHierarchy($notebookId, 4, [ref]$hierarchy, 1)
84 [IO.File]::WriteAllText((Join-Path $output 'unsettled-hierarchy.xml'), $hierarchy, [Text.Encoding]::UTF8)
85 throw "Expected $ExpectedPages stable pages; OneNote returned $($pages.Count), settled=$settled."
86 }
87 $index = 0
88 $payloads = @()
89 foreach ($id in @($pages.Keys | Sort-Object)) {
90 [IO.File]::WriteAllText((Join-Path $output ('page-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
91 if ($Pdf) {
92 $pdfPath = Join-Path $output ('page-{0:d3}.pdf' -f $index)
93 $app.NavigateTo($id, '', $false)
94 $app.Publish($id, $pdfPath, 3, '')
95 if (-not (Test-Path $pdfPath) -or (Get-Item $pdfPath).Length -eq 0) { throw 'OneNote did not publish the page PDF.' }
96 }
97 [xml]$page = $pages[$id]
98 foreach ($file in $page.SelectNodes('//*[local-name()="InsertedFile" or local-name()="MediaFile"]')) {
99 $bytes = [IO.File]::ReadAllBytes($file.GetAttribute('pathCache'))
100 $hash = [BitConverter]::ToString([Security.Cryptography.SHA256]::Create().ComputeHash($bytes)).Replace('-', '').ToLowerInvariant()
101 [IO.File]::WriteAllBytes((Join-Path $output ($hash + '.attachment')), $bytes)
102 $payloads += @{ page = $id; object = $file.ParentNode.GetAttribute('objectID');
103 kind = $file.LocalName; name = $file.GetAttribute('preferredName');
104 sha256 = $hash; bytes = $bytes.Length }
105 }
106 $index++
107 }
108 [IO.File]::WriteAllText((Join-Path $output 'payloads.json'), (ConvertTo-Json -InputObject $payloads -Depth 4), [Text.Encoding]::UTF8)
109 $all = ''
110 $app.GetHierarchy($notebookId, 4, [ref]$all, 1)
111 [xml]$finalTree = $all
112 $finalIds = @($finalTree.SelectNodes('//*[local-name()="Page"]') | ForEach-Object { $_.GetAttribute('ID') } | Sort-Object -Unique)
113 if ($finalIds.Count -ne $pages.Count -or @($finalIds | Where-Object { -not $pages.ContainsKey($_) }).Count -ne 0) {
114 throw 'The notebook hierarchy changed while collecting page evidence; repeat the cold read.'
115 }
116 [IO.File]::WriteAllText((Join-Path $output 'hierarchy.xml'), $all, [Text.Encoding]::UTF8)
117 Write-Output "Read $($sections.Count) sections and $($pages.Count) pages."
118} catch {
119 $failure = $_
120 [IO.File]::WriteAllText((Join-Path $output 'failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
121 throw
122} finally {
123 try {
124 try {
125 if ($notebookId -and $CloneHost) { $app.SyncHierarchy($notebookId) }
126 if ($notebookId -and -not $KeepOpen -and (-not $UseCurrentCache -or $CloneHost)) {
127 $app.CloseNotebook($notebookId, $false)
128 }
129 } catch {
130 if ($null -eq $failure) { throw }
131 [IO.File]::WriteAllText((Join-Path $output 'cleanup-failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
132 }
133 } finally {
134 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
135 }
136 $app = $null
137 [GC]::Collect()
138 [GC]::WaitForPendingFinalizers()
139 if (-not $UseCurrentCache -and -not $CloneHost) {
140 Get-Process ONENOTE -ErrorAction SilentlyContinue | Wait-Process -Timeout 10
141 }
142}
corpus/conflict-page/cold/source.json created+14
......@@ -0,0 +1,14 @@
1[
2 {
3 "path": "Open Notebook.onetoc2",
4 "bytes": 4944,
5 "sha256": "36274ad5cc077c6b1ad51ee4c4b90d3b6798a45ce446cbfd7c9d7ac504776b28",
6 "mtime_ns": 1788585164198018949
7 },
8 {
9 "path": "synthetic.one",
10 "bytes": 43112,
11 "sha256": "ab763188e885724957e2440a78eb29c457f47d538f78c3cc40e9c1a396be5c3b",
12 "mtime_ns": 1790489509247358867
13 }
14]
corpus/conflict-page/cold/teardown.json created+1
......@@ -0,0 +1 @@
1{"absent": true}
corpus/conflict-page/native-delete/before-read/af1af4b8987109ce1e335e538910fb3860b7c57eb3b6110a06aa2114e06d69a7.attachment created+1
......@@ -0,0 +1 @@
1Fictitious attachment for native corpus.
\ No newline at end of file
corpus/conflict-page/native-delete/before-read/environment.json created+7
......@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-7691E04E",
5 "cold": true,
6 "onenote": "14.0.4763.1000"
7}
corpus/conflict-page/native-delete/before-read/hierarchy.xml created+2
......@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{D55F46CA-9FD0-412D-8ED0-322FCEAD2763}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-27T01:30:09.000Z" color="#9595AA"><one:Section name="synthetic" ID="{BC6E92D1-2D86-0BD5-3D78-3FFC33523E50}{1}{B0}" path="C:\one-tests\runs\capture\notebook\synthetic.one" lastModifiedTime="2026-09-27T01:30:09.000Z" color="#8AA8E4"><one:Page ID="{90276157-A44D-48C4-BA1F-976DCF6A224F}{1}{B0}" name="Client B conflicting edit." dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1"/></one:Section></one:Notebook>
corpus/conflict-page/native-delete/before-read/page-000.xml created+21
......@@ -0,0 +1,21 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{90276157-A44D-48C4-BA1F-976DCF6A224F}{1}{B0}" name="Client B conflicting edit." dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="ONE-M6-D63654A2" authorInitials="4A2" lastModifiedBy="ONE-M6-D63654A2" lastModifiedByInitials="4A2" lastModifiedTime="2026-09-27T01:00:05.000Z" objectID="{F7EBC9EB-EF38-4B72-B5ED-1BB630F6C72E}{10}{B0}"><one:Position x="36.0" y="14.40000057220459" z="1"/><one:Size width="127.5136947631836" height="13.42771625518799"/><one:OEChildren><one:OE creationTime="2026-09-27T01:00:05.000Z" lastModifiedTime="2026-09-27T01:00:05.000Z" objectID="{F7EBC9EB-EF38-4B72-B5ED-1BB630F6C72E}{11}{B0}" alignment="left" quickStyleIndex="0" style="font-family:Calibri;font-size:11.0pt;color:#1F4E79"><one:T><![CDATA[Client B conflicting edit.]]></one:T></one:OE></one:OEChildren></one:Outline><one:InkDrawing lastModifiedTime="2026-09-05T05:11:40.000Z" objectID="{D506E579-B7D0-45C1-A875-B93EF076AAFA}{15}{B0}"><one:Position x="421.5118103027344" y="84.75590515136718" z="5"/><one:Size width="60.00945281982422" height="60.00944519042969"/><one:Data>AK4FHQSiAaIBAWAqABofi0JwRYR1gL7QRdkE+oeVMvLsF0G4wBa/8yv0et67E4vlXrNInJuhJVq3
3URK/7HHMzCKxQYkOuxMs7nqhWM9UiuaXxU+PBvi60uGbIsRmbacNslhJmh1OQDQGbBADBkgQRSNG
4IwUIC2VPZmdoZGkZFDIIAIAeAiHi4kEzCADwEAKLxeJBZAMVRgBACpUE/QGH81d3mq7mkzgM0mcB
5mk1gEBgEBgEBgE1mUAgMzmUAmsygE1mUzmkBmc0gMzmk1mUAmsyTOaTWZQCazKZgALSmiazKATWZ
6QCAwCAwBAZpM4DAJlNYBMprAJlNZpM4DNJmmU1mkzgM0mcBmkzmU1gEyAAFpTSAzOaQGZzRAYBAY
7BAYBMprAIDNJnAZpM0yms0mcBmkzgM0mcymsAmU1gEyms0maZAACZTSawCZTWATKawCAoBAYBAZn
8NIDAJrMoBNZlAJrMpnMoBNZlAJrMpnNIDM5pAZnNJrMkzmk1AACH8C7XgWnmkzgM0msAmU1gEBgE
9BgEBTOaQGATWZQCazKATWZTOaTWZTNNJrMpmmUzTSaplM00mqZTOaTVMpmmk1TKATKazSawCAzSZ
10wGAQFAIDAJrMoBAZnNIDM5pAZnMpnNJrMpmmk1mUzTSaplM00mqZTNNJrMpmmk1TKZplhuAzKATW
11ZQCazKAIDAIDAIDNJnAYBMprAJlM4DNJnMprNJnMk1mkzmSazRM5kms0TOZJnMprNEzmSazQtKaQ
12GZzRNZlAIDAIDAIDAJlNYAmU1gEymsAmU1mkzmU1miZzKazRM5kmcyTWaJnMk1mkzmSazRM5kAUU
13RgBAAAAJBAUURgAAAAASABEgXhT+65lz/EiPi+8SFvjqCAk+gEcDAa/89Qd0AxVGAEAKABEg2R87
14QQrSFUA=</one:Data></one:InkDrawing><one:InkDrawing lastModifiedTime="2026-09-05T05:11:41.000Z" objectID="{D506E579-B7D0-45C1-A875-B93EF076AAFA}{23}{B0}"><one:Position x="504.0" y="92.23936462402343" z="6"/><one:Size width="30.01890563964844" height="45.04251861572266"/><one:Data>AOECHQJSegFgKgAaH4tCcEWEdYC+0EXZBPqHlTLy7BdBuMAWv/Mr9HreuxOL5V6zSJyboSVat1ES
15v+xxzMwisUGJDrsTLO56oVjPVIrml8VPjwb4utLhmyLEZm2nDbJYSZodTkA0BmwQAwZIEEUjRiMF
16CAtlT2ZnaGRpGRQyCACAHgIh4uJBMwgA8BACi8XiQWQDFUYAQArKAUuH88s3nkzmkzgM0mcBmkzg
17KAQGAQGATWZQCAzOaQGZzKATWZTOaQGZzSAzOaTWZQCazJM5pNZlAJrMpmAAAACH8D9XgeBJlM0z
18mRNZpNZpM4DAJlNYBAUAgMAgMzmkBgE1mUAmsyTOaTWZTOaTVMpnNJqmUzTSaplM00mqZTOZTNNJ
19qmUzTSagBRRGAIAAAAkEBRRGAAAAABIAESDPTQ4xwIZyTqQ8FVDbHxeKCj9AI5rARvPPUDugAxVG
20AEAKABEgLMlfQdkfI0A=</one:Data></one:InkDrawing><one:Outline author="ONE-M6-8A93780A" authorInitials="80A" lastModifiedBy="ONE-M6-8A93780A" lastModifiedByInitials="80A" lastModifiedTime="2026-09-27T00:59:59.000Z" objectID="{A15B15FE-5C6D-412F-AF5B-993E94FA2367}{10}{B0}"><one:Position x="144.0" y="96.0" z="2"/><one:Size width="132.69140625" height="13.42771339416504"/><one:OEChildren><one:OE creationTime="2026-09-27T00:59:59.000Z" lastModifiedTime="2026-09-27T00:59:59.000Z" objectID="{A15B15FE-5C6D-412F-AF5B-993E94FA2367}{11}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Client A disjoint edit.]]></one:T></one:OE></one:OEChildren></one:Outline><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-05T05:06:58.000Z" objectID="{9C727D71-BAF7-4881-992B-32CA78976BE9}{10}{B0}"><one:Position x="144.0" y="192.0" z="3"/><one:Size width="72.0" height="0.750005722045898"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:58.000Z" lastModifiedTime="2026-09-05T05:06:58.000Z" objectID="{9C727D71-BAF7-4881-992B-32CA78976BE9}{11}{B0}" alignment="left"><one:Image format="png" originalPageNumber="0"><one:Data>iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAusB9Y9J1uoA
21AAAASUVORK5CYII=</one:Data></one:Image></one:OE></one:OEChildren></one:Outline><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-05T05:06:59.000Z" objectID="{19686511-7C8F-4B47-BC86-55344C46A4DA}{10}{B0}"><one:Position x="264.0" y="192.0" z="4"/><one:Size width="72.00001525878906" height="63.0"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:59.000Z" lastModifiedTime="2026-09-05T05:06:59.000Z" objectID="{19686511-7C8F-4B47-BC86-55344C46A4DA}{11}{B0}" alignment="left"><one:InsertedFile pathCache="C:\Users\clover\AppData\Local\Microsoft\OneNote\14.0\OneNoteOfflineCache_Files\458001d8-0f1e-451f-81cb-621828fc0b39.txt" pathSource="C:\one-tests\runs\20260905-05\assets\fictitious-attachment.txt" preferredName="fictitious-attachment.txt"/></one:OE></one:OEChildren></one:Outline><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-27T01:00:54.000Z" objectID="{37B71C41-1D1F-40C7-A38E-B76164155F9C}{10}{B0}"><one:Position x="144.0" y="312.0" z="0"/><one:Size width="92.42181396484374" height="48.80315780639648"/><one:OEChildren><one:OE creationTime="2026-09-27T01:00:05.000Z" lastModifiedTime="2026-09-27T01:00:54.000Z" objectID="{CF34E4C3-5A4B-4362-AE36-CCAD58670232}{27}{B0}" alignment="left"><one:Table bordersVisible="true" lastModifiedTime="2026-09-27T01:00:54.000Z" objectID="{CF34E4C3-5A4B-4362-AE36-CCAD58670232}{28}{B0}"><one:Columns><one:Column index="0" width="45.947021484375"/><one:Column index="1" width="38.34479141235352"/></one:Columns><one:Row objectID="{CF34E4C3-5A4B-4362-AE36-CCAD58670232}{29}{B0}" lastModifiedTime="2026-09-27T01:00:54.000Z"><one:Cell lastModifiedTime="2026-09-27T01:00:54.000Z" objectID="{CF34E4C3-5A4B-4362-AE36-CCAD58670232}{30}{B0}" lastModifiedByInitials="4A2"><one:OEChildren><one:OE creationTime="2026-09-27T01:00:05.000Z" lastModifiedTime="2026-09-27T01:00:05.000Z" objectID="{37B71C41-1D1F-40C7-A38E-B76164155F9C}{17}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Client B disjoint cell.]]></one:T></one:OE></one:OEChildren></one:Cell><one:Cell lastModifiedTime="2026-09-27T01:00:54.000Z" objectID="{CF34E4C3-5A4B-4362-AE36-CCAD58670232}{37}{B0}" lastModifiedByInitials="4A2"><one:OEChildren><one:OE creationTime="2026-09-27T01:00:05.000Z" lastModifiedTime="2026-09-27T01:00:05.000Z" objectID="{37B71C41-1D1F-40C7-A38E-B76164155F9C}{25}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Right cell]]></one:T></one:OE></one:OEChildren></one:Cell></one:Row></one:Table></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/conflict-page/native-delete/before-read/payloads.json created+10
......@@ -0,0 +1,10 @@
1[
2 {
3 "sha256": "af1af4b8987109ce1e335e538910fb3860b7c57eb3b6110a06aa2114e06d69a7",
4 "name": "fictitious-attachment.txt",
5 "object": "{19686511-7C8F-4B47-BC86-55344C46A4DA}{11}{B0}",
6 "kind": "InsertedFile",
7 "page": "{90276157-A44D-48C4-BA1F-976DCF6A224F}{1}{B0}",
8 "bytes": 43
9 }
10]
\ No newline at end of file
corpus/conflict-page/native-delete/cold-conflict.json created+13
......@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Client A conflicting edit. - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/native-delete/cold-conflict.png created
Binary files /dev/null and b/corpus/conflict-page/native-delete/cold-conflict.png differ
corpus/conflict-page/native-delete/cold-delete.json created+13
......@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Client B conflicting edit. - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/native-delete/cold-delete.png created
Binary files /dev/null and b/corpus/conflict-page/native-delete/cold-delete.png differ
corpus/conflict-page/native-delete/cold-main.json created+13
......@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Client B conflicting edit. - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/native-delete/cold-main.png created
Binary files /dev/null and b/corpus/conflict-page/native-delete/cold-main.png differ
corpus/conflict-page/native-delete/cold-menu.json created+13
......@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "",
10 "class": "Net UI Tool Window",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/native-delete/cold-menu.png created
Binary files /dev/null and b/corpus/conflict-page/native-delete/cold-menu.png differ
corpus/conflict-page/native-delete/cold-shown.json created+13
......@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Client A conflicting edit. - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/native-delete/cold-shown.png created
Binary files /dev/null and b/corpus/conflict-page/native-delete/cold-shown.png differ
corpus/conflict-page/native-delete/commands.jsonl created+6
......@@ -0,0 +1,6 @@
1{"command": "powershell -NoProfile -Command \"Expand-Archive -LiteralPath C:\\one-tests\\transfer.zip -DestinationPath C:\\one-tests\\runs\\capture\\notebook\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-7691E04E -ExpectedPages -1 -KeepOpen", "exit": 0, "stdout": "Read 1 sections and 1 pages.\r\n", "stderr": "", "error": null}
3{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
4{"command": "powershell -NoProfile -Command \"Rename-Item C:\\one-tests\\runs\\capture\\read before-read\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
5{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-7691E04E -UseCurrentCache -ExpectedPages -1", "exit": 0, "stdout": "Read 2 sections and 1 pages.\r\n", "stderr": "", "error": null}
6{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
corpus/conflict-page/native-delete/machine.json created+1
......@@ -0,0 +1 @@
1{"name": "m6-7691e04e", "hostname": "ONE-M6-7691E04E"}
corpus/conflict-page/native-delete/notebook/OneNote_RecycleBin/OneNote_DeletedPages.one created
Binary files /dev/null and b/corpus/conflict-page/native-delete/notebook/OneNote_RecycleBin/OneNote_DeletedPages.one differ
corpus/conflict-page/native-delete/notebook/OneNote_RecycleBin/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/native-delete/notebook/OneNote_RecycleBin/Open Notebook.onetoc2 differ
corpus/conflict-page/native-delete/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/native-delete/notebook/Open Notebook.onetoc2 differ
corpus/conflict-page/native-delete/notebook/synthetic.one created
Binary files /dev/null and b/corpus/conflict-page/native-delete/notebook/synthetic.one differ
corpus/conflict-page/native-delete/read/af1af4b8987109ce1e335e538910fb3860b7c57eb3b6110a06aa2114e06d69a7.attachment created+1
......@@ -0,0 +1 @@
1Fictitious attachment for native corpus.
\ No newline at end of file
corpus/conflict-page/native-delete/read/environment.json created+7
......@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-7691E04E",
5 "cold": false,
6 "onenote": "14.0.4763.1000"
7}
corpus/conflict-page/native-delete/read/hierarchy.xml created+2
......@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{D55F46CA-9FD0-412D-8ED0-322FCEAD2763}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-27T01:32:11.000Z" color="#9595AA" isCurrentlyViewed="true"><one:Section name="synthetic" ID="{BC6E92D1-2D86-0BD5-3D78-3FFC33523E50}{1}{B0}" path="C:\one-tests\runs\capture\notebook\synthetic.one" lastModifiedTime="2026-09-27T01:32:07.000Z" color="#8AA8E4" isCurrentlyViewed="true"><one:Page ID="{90276157-A44D-48C4-BA1F-976DCF6A224F}{1}{B0}" name="Client B conflicting edit." dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1" isCurrentlyViewed="true"/></one:Section><one:SectionGroup name="OneNote_RecycleBin" ID="{1FBDA5C2-882E-488F-98BC-34FC12A65A9E}{1}{B0}" path="C:\one-tests\runs\capture\notebook\OneNote_RecycleBin" lastModifiedTime="2026-09-27T01:32:11.000Z" isRecycleBin="true"><one:Section name="Deleted Pages" ID="{07CEB579-3338-4C03-8027-1BBBF7B2DF88}{1}{B0}" path="C:\one-tests\runs\capture\notebook\OneNote_RecycleBin\OneNote_DeletedPages.one" lastModifiedTime="2026-09-27T01:32:07.000Z" color="#E1E1E1" isInRecycleBin="true" isDeletedPages="true"/></one:SectionGroup></one:Notebook>
corpus/conflict-page/native-delete/read/page-000.xml created+21
......@@ -0,0 +1,21 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{90276157-A44D-48C4-BA1F-976DCF6A224F}{1}{B0}" name="Client B conflicting edit." dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1" isCurrentlyViewed="true" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="ONE-M6-D63654A2" authorInitials="4A2" lastModifiedBy="ONE-M6-D63654A2" lastModifiedByInitials="4A2" lastModifiedTime="2026-09-27T01:31:45.000Z" objectID="{F7EBC9EB-EF38-4B72-B5ED-1BB630F6C72E}{10}{B0}"><one:Position x="36.0" y="14.40000057220459" z="1"/><one:Size width="127.5136947631836" height="13.42771625518799"/><one:OEChildren><one:OE creationTime="2026-09-27T01:00:05.000Z" lastModifiedTime="2026-09-27T01:31:45.000Z" objectID="{F7EBC9EB-EF38-4B72-B5ED-1BB630F6C72E}{11}{B0}" alignment="left" quickStyleIndex="0" style="font-family:Calibri;font-size:11.0pt;color:#1F4E79"><one:T><![CDATA[Client B conflicting edit.]]></one:T></one:OE></one:OEChildren></one:Outline><one:InkDrawing lastModifiedTime="2026-09-05T05:11:40.000Z" objectID="{D506E579-B7D0-45C1-A875-B93EF076AAFA}{15}{B0}"><one:Position x="421.5118103027344" y="84.75590515136718" z="5"/><one:Size width="60.00945281982422" height="60.00944519042969"/><one:Data>AK4FHQSiAaIBAWAqABofi0JwRYR1gL7QRdkE+oeVMvLsF0G4wBa/8yv0et67E4vlXrNInJuhJVq3
3URK/7HHMzCKxQYkOuxMs7nqhWM9UiuaXxU+PBvi60uGbIsRmbacNslhJmh1OQDQGbBADBkgQRSNG
4IwUIC2VPZmdoZGkZFDIIAIAeAiHi4kEzCADwEAKLxeJBZAMVRgBACpUE/QGH81d3mq7mkzgM0mcB
5mk1gEBgEBgEBgE1mUAgMzmUAmsygE1mUzmkBmc0gMzmk1mUAmsyTOaTWZQCazKZgALSmiazKATWZ
6QCAwCAwBAZpM4DAJlNYBMprAJlNZpM4DNJmmU1mkzgM0mcBmkzmU1gEyAAFpTSAzOaQGZzRAYBAY
7BAYBMprAIDNJnAZpM0yms0mcBmkzgM0mcymsAmU1gEyms0maZAACZTSawCZTWATKawCAoBAYBAZn
8NIDAJrMoBNZlAJrMpnMoBNZlAJrMpnNIDM5pAZnNJrMkzmk1AACH8C7XgWnmkzgM0msAmU1gEBgE
9BgEBTOaQGATWZQCazKATWZTOaTWZTNNJrMpmmUzTSaplM00mqZTOaTVMpmmk1TKATKazSawCAzSZ
10wGAQFAIDAJrMoBAZnNIDM5pAZnMpnNJrMpmmk1mUzTSaplM00mqZTNNJrMpmmk1TKZplhuAzKATW
11ZQCazKAIDAIDAIDNJnAYBMprAJlM4DNJnMprNJnMk1mkzmSazRM5kms0TOZJnMprNEzmSazQtKaQ
12GZzRNZlAIDAIDAIDAJlNYAmU1gEymsAmU1mkzmU1miZzKazRM5kmcyTWaJnMk1mkzmSazRM5kAUU
13RgBAAAAJBAUURgAAAAASABEgXhT+65lz/EiPi+8SFvjqCAk+gEcDAa/89Qd0AxVGAEAKABEg2R87
14QQrSFUA=</one:Data></one:InkDrawing><one:InkDrawing lastModifiedTime="2026-09-05T05:11:41.000Z" objectID="{D506E579-B7D0-45C1-A875-B93EF076AAFA}{23}{B0}"><one:Position x="504.0" y="92.23936462402343" z="6"/><one:Size width="30.01890563964844" height="45.04251861572266"/><one:Data>AOECHQJSegFgKgAaH4tCcEWEdYC+0EXZBPqHlTLy7BdBuMAWv/Mr9HreuxOL5V6zSJyboSVat1ES
15v+xxzMwisUGJDrsTLO56oVjPVIrml8VPjwb4utLhmyLEZm2nDbJYSZodTkA0BmwQAwZIEEUjRiMF
16CAtlT2ZnaGRpGRQyCACAHgIh4uJBMwgA8BACi8XiQWQDFUYAQArKAUuH88s3nkzmkzgM0mcBmkzg
17KAQGAQGATWZQCAzOaQGZzKATWZTOaQGZzSAzOaTWZQCazJM5pNZlAJrMpmAAAACH8D9XgeBJlM0z
18mRNZpNZpM4DAJlNYBAUAgMAgMzmkBgE1mUAmsyTOaTWZTOaTVMpnNJqmUzTSaplM00mqZTOZTNNJ
19qmUzTSagBRRGAIAAAAkEBRRGAAAAABIAESDPTQ4xwIZyTqQ8FVDbHxeKCj9AI5rARvPPUDugAxVG
20AEAKABEgLMlfQdkfI0A=</one:Data></one:InkDrawing><one:Outline author="ONE-M6-8A93780A" authorInitials="80A" lastModifiedBy="ONE-M6-8A93780A" lastModifiedByInitials="80A" lastModifiedTime="2026-09-27T01:31:45.000Z" objectID="{A15B15FE-5C6D-412F-AF5B-993E94FA2367}{10}{B0}"><one:Position x="144.0" y="96.0" z="2"/><one:Size width="132.69140625" height="13.42771339416504"/><one:OEChildren><one:OE creationTime="2026-09-27T00:59:59.000Z" lastModifiedTime="2026-09-27T01:31:45.000Z" objectID="{A15B15FE-5C6D-412F-AF5B-993E94FA2367}{11}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Client A disjoint edit.]]></one:T></one:OE></one:OEChildren></one:Outline><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-05T05:06:58.000Z" objectID="{9C727D71-BAF7-4881-992B-32CA78976BE9}{10}{B0}"><one:Position x="144.0" y="192.0" z="3"/><one:Size width="72.0" height="0.750005722045898"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:58.000Z" lastModifiedTime="2026-09-05T05:06:58.000Z" objectID="{9C727D71-BAF7-4881-992B-32CA78976BE9}{11}{B0}" alignment="left"><one:Image format="png" originalPageNumber="0"><one:Data>iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAusB9Y9J1uoA
21AAAASUVORK5CYII=</one:Data></one:Image></one:OE></one:OEChildren></one:Outline><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-05T05:06:59.000Z" objectID="{19686511-7C8F-4B47-BC86-55344C46A4DA}{10}{B0}"><one:Position x="264.0" y="192.0" z="4"/><one:Size width="72.00001525878906" height="63.0"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:59.000Z" lastModifiedTime="2026-09-05T05:06:59.000Z" objectID="{19686511-7C8F-4B47-BC86-55344C46A4DA}{11}{B0}" alignment="left"><one:InsertedFile pathCache="C:\Users\clover\AppData\Local\Microsoft\OneNote\14.0\OneNoteOfflineCache_Files\458001d8-0f1e-451f-81cb-621828fc0b39.txt" pathSource="C:\one-tests\runs\20260905-05\assets\fictitious-attachment.txt" preferredName="fictitious-attachment.txt"/></one:OE></one:OEChildren></one:Outline><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-27T01:31:45.000Z" objectID="{37B71C41-1D1F-40C7-A38E-B76164155F9C}{10}{B0}"><one:Position x="144.0" y="312.0" z="0"/><one:Size width="92.42181396484374" height="48.80315780639648"/><one:OEChildren><one:OE creationTime="2026-09-27T01:00:05.000Z" lastModifiedTime="2026-09-27T01:31:45.000Z" objectID="{CF34E4C3-5A4B-4362-AE36-CCAD58670232}{27}{B0}" alignment="left"><one:Table bordersVisible="true" lastModifiedTime="2026-09-27T01:31:45.000Z" objectID="{CF34E4C3-5A4B-4362-AE36-CCAD58670232}{28}{B0}"><one:Columns><one:Column index="0" width="45.947021484375"/><one:Column index="1" width="38.34479141235352"/></one:Columns><one:Row objectID="{CF34E4C3-5A4B-4362-AE36-CCAD58670232}{29}{B0}" lastModifiedTime="2026-09-27T01:31:45.000Z"><one:Cell lastModifiedTime="2026-09-27T01:31:45.000Z" objectID="{CF34E4C3-5A4B-4362-AE36-CCAD58670232}{30}{B0}" lastModifiedByInitials="4A2"><one:OEChildren><one:OE creationTime="2026-09-27T01:00:05.000Z" lastModifiedTime="2026-09-27T01:31:45.000Z" objectID="{37B71C41-1D1F-40C7-A38E-B76164155F9C}{17}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Client B disjoint cell.]]></one:T></one:OE></one:OEChildren></one:Cell><one:Cell lastModifiedTime="2026-09-27T01:31:45.000Z" objectID="{CF34E4C3-5A4B-4362-AE36-CCAD58670232}{37}{B0}" lastModifiedByInitials="4A2"><one:OEChildren><one:OE creationTime="2026-09-27T01:00:05.000Z" lastModifiedTime="2026-09-27T01:31:45.000Z" objectID="{37B71C41-1D1F-40C7-A38E-B76164155F9C}{25}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Right cell]]></one:T></one:OE></one:OEChildren></one:Cell></one:Row></one:Table></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/conflict-page/native-delete/read/payloads.json created+10
......@@ -0,0 +1,10 @@
1[
2 {
3 "sha256": "af1af4b8987109ce1e335e538910fb3860b7c57eb3b6110a06aa2114e06d69a7",
4 "name": "fictitious-attachment.txt",
5 "object": "{19686511-7C8F-4B47-BC86-55344C46A4DA}{11}{B0}",
6 "kind": "InsertedFile",
7 "page": "{90276157-A44D-48C4-BA1F-976DCF6A224F}{1}{B0}",
8 "bytes": 43
9 }
10]
\ No newline at end of file
corpus/conflict-page/native-delete/run.json created+18
......@@ -0,0 +1,18 @@
1{
2 "notebook": "/private/tmp/gate/native-conflict",
3 "expected_pages": -1,
4 "author": null,
5 "author_timeout_seconds": 600,
6 "inspect": false,
7 "collect_notebook": true,
8 "base": {
9 "file": "win7-office-base.qcow2",
10 "format": "qcow2",
11 "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346",
12 "virtual_size": 68719476736
13 },
14 "scripts": {
15 "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331",
16 "read.ps1": "04013bfcccee40a17e2a225f9b9e96f40a3a8daad9e350609654f8eda3ccbb41"
17 }
18}
corpus/conflict-page/native-delete/scripts/cold.ps1 created+27
......@@ -0,0 +1,27 @@
1param([Parameter(Mandatory=$true)][string]$Root, [string]$CloneHost = '')
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
5if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
6 throw 'Choose a run directly below C:\one-tests\runs.'
7}
8if (Get-Process ONENOTE -ErrorAction SilentlyContinue) { throw 'Close OneNote before resetting its test cache.' }
9$key = 'HKCU:\Software\Microsoft\Office\14.0\OneNote'
10if ($CloneHost) {
11 if ($CloneHost -notmatch '^ONE-[A-Z0-9-]+$' -or [Environment]::MachineName -ne $CloneHost) {
12 throw 'The disposable clone hostname does not match this machine.'
13 }
14 New-Item "$key\Options\Paths" -Force | Out-Null
15 New-ItemProperty "$key\Options\Paths" -Name UnfiledNotesSection -PropertyType ExpandString -Value 'C:\one-tests\Loose.one' -Force | Out-Null
16} elseif ((Get-ItemProperty "$key\Options\Paths").UnfiledNotesSection -ne 'C:\one-tests\Loose.one' -or
17 -not (Test-Path 'C:\one-tests\profile-original-cache')) {
18 throw 'Park the personal OneNote profile before resetting the test cache.'
19}
20$cache = Join-Path $env:LOCALAPPDATA 'Microsoft\OneNote\14.0'
21$parked = Join-Path 'C:\one-tests\caches' ([IO.Path]::GetFileName($root))
22if (Test-Path $parked) { throw 'Choose a new run; its parked cache already exists.' }
23New-Item -ItemType Directory -Path 'C:\one-tests\caches' -Force | Out-Null
24if (Test-Path $cache) { Move-Item -LiteralPath $cache -Destination $parked }
25if (Test-Path "$key\OpenNotebooks") { Remove-Item "$key\OpenNotebooks" -Recurse }
26New-Item "$key\OpenNotebooks" | Out-Null
27New-ItemProperty "$key\OpenNotebooks" -Name '1' -PropertyType String -Value "$root\notebook" | Out-Null
corpus/conflict-page/native-delete/scripts/read.ps1 created+142
......@@ -0,0 +1,142 @@
1param(
2 [Parameter(Mandatory=$true)][string]$Root,
3 [int]$ExpectedPages = -1,
4 [switch]$UseCurrentCache,
5 [switch]$Pdf,
6 [switch]$KeepOpen,
7 [string]$CloneHost = ''
8)
9Set-StrictMode -Version Latest
10$ErrorActionPreference = 'Stop'
11$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
12if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
13 throw 'Choose a run directly below C:\one-tests\runs.'
14}
15$notebook = Join-Path $root 'notebook'
16$output = Join-Path $root 'read'
17if (Test-Path $output) { throw 'Choose a new read destination.' }
18if ($UseCurrentCache) {
19 if ((Get-ItemProperty 'HKCU:\Software\Microsoft\Office\14.0\OneNote\Options\Paths').UnfiledNotesSection -ne 'C:\one-tests\Loose.one') {
20 throw 'Park the personal OneNote profile before reading test notebooks.'
21 }
22} else {
23 & "$PSScriptRoot\cold-current.ps1" -Root $root -CloneHost $CloneHost
24}
25New-Item -ItemType Directory -Path $output | Out-Null
26$app = New-Object -ComObject OneNote.Application
27$notebookId = ''
28$failure = $null
29try {
30 $app.OpenHierarchy($notebook, '', [ref]$notebookId, 0)
31 $process = Get-Process ONENOTE
32 @{ hostname = [Environment]::MachineName; onenote = $process.MainModule.FileVersionInfo.FileVersion;
33 powershell = $PSVersionTable.PSVersion.ToString(); schema = 'xs2010'; cold = (-not $UseCurrentCache.IsPresent) } |
34 ConvertTo-Json | Set-Content (Join-Path $output 'environment.json') -Encoding UTF8
35 $sections = @()
36 foreach ($file in @(Get-ChildItem $notebook -Recurse | Where-Object { $_.Extension -eq '.one' })) {
37 $id = ''
38 $app.OpenHierarchy($file.FullName, '', [ref]$id, 0)
39 $sections += $id
40 }
41 $deadline = [DateTime]::UtcNow.AddSeconds(300)
42 $previous = ''
43 $lastChange = ''
44 $stableSince = [DateTime]::UtcNow
45 $settled = $false
46 do {
47 $pages = @{}
48 foreach ($section in $sections) {
49 $hierarchy = ''
50 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
51 [xml]$xml = $hierarchy
52 foreach ($node in $xml.SelectNodes('//*[@path]')) {
53 if (-not $node.GetAttribute('path').StartsWith("$notebook\", [StringComparison]::OrdinalIgnoreCase)) {
54 throw 'OneNote opened a section outside the copied notebook.'
55 }
56 }
57 foreach ($node in $xml.SelectNodes('//*[local-name()="Page"]')) {
58 $id = $node.GetAttribute('ID')
59 $content = ''
60 $app.GetPageContent($id, [ref]$content, 1, 1)
61 $pages[$id] = $content
62 }
63 }
64 $signature = [String]::Join('|', @($pages.Keys | Sort-Object | ForEach-Object { $_ + $pages[$_] }))
65 if ($signature -ne $previous) {
66 $lastChange = $previous
67 $previous = $signature
68 $stableSince = [DateTime]::UtcNow
69 }
70 if ((($ExpectedPages -ge 0 -and $pages.Count -eq $ExpectedPages) -or
71 ($ExpectedPages -lt 0 -and $pages.Count -gt 0)) -and
72 ([DateTime]::UtcNow - $stableSince).TotalSeconds -ge 2) { $settled = $true; break }
73 Start-Sleep -Milliseconds 250
74 } while ([DateTime]::UtcNow -lt $deadline)
75 if (-not $settled -or ($ExpectedPages -ge 0 -and $pages.Count -ne $ExpectedPages) -or ($ExpectedPages -lt 0 -and $pages.Count -eq 0)) {
76 [IO.File]::WriteAllText((Join-Path $output 'previous-signature.txt'), $lastChange, [Text.Encoding]::UTF8)
77 $index = 0
78 foreach ($id in @($pages.Keys | Sort-Object)) {
79 [IO.File]::WriteAllText((Join-Path $output ('unsettled-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
80 $index++
81 }
82 $hierarchy = ''
83 $app.GetHierarchy($notebookId, 4, [ref]$hierarchy, 1)
84 [IO.File]::WriteAllText((Join-Path $output 'unsettled-hierarchy.xml'), $hierarchy, [Text.Encoding]::UTF8)
85 throw "Expected $ExpectedPages stable pages; OneNote returned $($pages.Count), settled=$settled."
86 }
87 $index = 0
88 $payloads = @()
89 foreach ($id in @($pages.Keys | Sort-Object)) {
90 [IO.File]::WriteAllText((Join-Path $output ('page-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
91 if ($Pdf) {
92 $pdfPath = Join-Path $output ('page-{0:d3}.pdf' -f $index)
93 $app.NavigateTo($id, '', $false)
94 $app.Publish($id, $pdfPath, 3, '')
95 if (-not (Test-Path $pdfPath) -or (Get-Item $pdfPath).Length -eq 0) { throw 'OneNote did not publish the page PDF.' }
96 }
97 [xml]$page = $pages[$id]
98 foreach ($file in $page.SelectNodes('//*[local-name()="InsertedFile" or local-name()="MediaFile"]')) {
99 $bytes = [IO.File]::ReadAllBytes($file.GetAttribute('pathCache'))
100 $hash = [BitConverter]::ToString([Security.Cryptography.SHA256]::Create().ComputeHash($bytes)).Replace('-', '').ToLowerInvariant()
101 [IO.File]::WriteAllBytes((Join-Path $output ($hash + '.attachment')), $bytes)
102 $payloads += @{ page = $id; object = $file.ParentNode.GetAttribute('objectID');
103 kind = $file.LocalName; name = $file.GetAttribute('preferredName');
104 sha256 = $hash; bytes = $bytes.Length }
105 }
106 $index++
107 }
108 [IO.File]::WriteAllText((Join-Path $output 'payloads.json'), (ConvertTo-Json -InputObject $payloads -Depth 4), [Text.Encoding]::UTF8)
109 $all = ''
110 $app.GetHierarchy($notebookId, 4, [ref]$all, 1)
111 [xml]$finalTree = $all
112 $finalIds = @($finalTree.SelectNodes('//*[local-name()="Page"]') | ForEach-Object { $_.GetAttribute('ID') } | Sort-Object -Unique)
113 if ($finalIds.Count -ne $pages.Count -or @($finalIds | Where-Object { -not $pages.ContainsKey($_) }).Count -ne 0) {
114 throw 'The notebook hierarchy changed while collecting page evidence; repeat the cold read.'
115 }
116 [IO.File]::WriteAllText((Join-Path $output 'hierarchy.xml'), $all, [Text.Encoding]::UTF8)
117 Write-Output "Read $($sections.Count) sections and $($pages.Count) pages."
118} catch {
119 $failure = $_
120 [IO.File]::WriteAllText((Join-Path $output 'failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
121 throw
122} finally {
123 try {
124 try {
125 if ($notebookId -and $CloneHost) { $app.SyncHierarchy($notebookId) }
126 if ($notebookId -and -not $KeepOpen -and (-not $UseCurrentCache -or $CloneHost)) {
127 $app.CloseNotebook($notebookId, $false)
128 }
129 } catch {
130 if ($null -eq $failure) { throw }
131 [IO.File]::WriteAllText((Join-Path $output 'cleanup-failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
132 }
133 } finally {
134 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
135 }
136 $app = $null
137 [GC]::Collect()
138 [GC]::WaitForPendingFinalizers()
139 if (-not $UseCurrentCache -and -not $CloneHost) {
140 Get-Process ONENOTE -ErrorAction SilentlyContinue | Wait-Process -Timeout 10
141 }
142}
corpus/conflict-page/native-delete/source.json created+14
......@@ -0,0 +1,14 @@
1[
2 {
3 "path": "Open Notebook.onetoc2",
4 "bytes": 4944,
5 "sha256": "36274ad5cc077c6b1ad51ee4c4b90d3b6798a45ce446cbfd7c9d7ac504776b28",
6 "mtime_ns": 1790472520423354873
7 },
8 {
9 "path": "synthetic.one",
10 "bytes": 40600,
11 "sha256": "e8d43dd57e02fe76e88c8ef45a7e18487bbf28928d0baf077288a4bb1415692c",
12 "mtime_ns": 1790472520423873581
13 }
14]
corpus/conflict-page/native-delete/teardown.json created+1
......@@ -0,0 +1 @@
1{"absent": true}
corpus/conflict-page/native-pages/a/a-merged.json created+13
......@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Fictitious: caf\u00e9, \u6771\u4eac, \u0645\u0631\u062d\u0628\u0627 - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/native-pages/a/a-merged.png created
Binary files /dev/null and b/corpus/conflict-page/native-pages/a/a-merged.png differ
corpus/conflict-page/native-pages/a/command-0001.json created+1
......@@ -0,0 +1 @@
1{"action": "pages", "titles": ["One", "Two", "Three", "Four", "Target"]}
\ No newline at end of file
corpus/conflict-page/native-pages/a/command-0002.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-pages/a/command-0003.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-pages/a/command-0004.json created+1
......@@ -0,0 +1 @@
1{"action": "move", "page": "Four", "before": "One"}
\ No newline at end of file
corpus/conflict-page/native-pages/a/command-0005.json created+1
......@@ -0,0 +1 @@
1{"action": "move", "page": "Two", "before": null}
\ No newline at end of file
corpus/conflict-page/native-pages/a/command-0006.json created+1
......@@ -0,0 +1 @@
1{"action": "edit", "expected": "Body Target.", "text": "Body Target edited offline."}
\ No newline at end of file
corpus/conflict-page/native-pages/a/command-0007.json created+1
......@@ -0,0 +1 @@
1{"action": "snapshot"}
\ No newline at end of file
corpus/conflict-page/native-pages/a/command-0008.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-pages/a/command-0009.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-pages/a/command-0010.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-pages/a/command-0011.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-pages/a/command-0012.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-pages/a/command-0013.json created+1
......@@ -0,0 +1 @@
1{"action": "snapshot"}
\ No newline at end of file
corpus/conflict-page/native-pages/a/command-0014.json created+1
......@@ -0,0 +1 @@
1{"action": "close"}
\ No newline at end of file
corpus/conflict-page/native-pages/a/commands.jsonl created+16
......@@ -0,0 +1,16 @@
1{"command": "mkdir C:\\one-tests\\runs\\capture", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -ExecutionPolicy Bypass -File C:\\one-tests\\network.ps1 -LabMac 52:54:02:a8:e9:55", "exit": 0, "stdout": "{\"renewalResult\":0,\"releaseResult\":0,\"mac\":\"52:54:02:a8:e9:55\",\"address\":\"192.168.77.42\"}\r\n", "stderr": "", "error": null}
3{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0001.tmp C:\\one-tests\\runs\\capture\\inbox\\0001.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
4{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0002.tmp C:\\one-tests\\runs\\capture\\inbox\\0002.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
5{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0003.tmp C:\\one-tests\\runs\\capture\\inbox\\0003.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
6{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0004.tmp C:\\one-tests\\runs\\capture\\inbox\\0004.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
7{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0005.tmp C:\\one-tests\\runs\\capture\\inbox\\0005.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
8{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0006.tmp C:\\one-tests\\runs\\capture\\inbox\\0006.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
9{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0007.tmp C:\\one-tests\\runs\\capture\\inbox\\0007.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
10{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0008.tmp C:\\one-tests\\runs\\capture\\inbox\\0008.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
11{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0009.tmp C:\\one-tests\\runs\\capture\\inbox\\0009.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
12{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0010.tmp C:\\one-tests\\runs\\capture\\inbox\\0010.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
13{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0011.tmp C:\\one-tests\\runs\\capture\\inbox\\0011.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
14{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0012.tmp C:\\one-tests\\runs\\capture\\inbox\\0012.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
15{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0013.tmp C:\\one-tests\\runs\\capture\\inbox\\0013.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
16{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0014.tmp C:\\one-tests\\runs\\capture\\inbox\\0014.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
corpus/conflict-page/native-pages/a/hierarchy-0007.xml created+2
......@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Section xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="synthetic" ID="{63142E76-6879-0236-0E9B-223917E9FE4E}{1}{B0}" path="\\192.168.77.1\agent\m6-conflict\synthetic.one" lastModifiedTime="2026-09-27T09:22:37.000Z" color="#8AA8E4"><one:Page ID="{4F5DDDF0-E1B2-4127-89FC-8AA8EBD1E251}{1}{B0}" name="Fictitious: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1"/><one:Page ID="{FE831102-E2A2-47B2-A9AB-FA1A7ED19095}{1}{B0}" name="Four" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:20:19.000Z" pageLevel="1"/><one:Page ID="{5DBAC284-CA61-451B-B041-C2719A268F2A}{1}{B0}" name="One" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:20:19.000Z" pageLevel="1"/><one:Page ID="{540C9694-2DCE-4E68-8CCE-0B8D44BA9AAB}{1}{B0}" name="Three" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:20:19.000Z" pageLevel="1"/><one:Page ID="{4A9CC72B-5F8D-4666-BC1D-7E8CEF0D5351}{1}{B0}" name="Target" dateTime="2026-09-27T09:20:18.000Z" lastModifiedTime="2026-09-27T09:20:19.000Z" pageLevel="1"/><one:Page ID="{DB35495A-2D25-485D-B817-34DF84D8C5BF}{1}{B0}" name="Two" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:20:19.000Z" pageLevel="1"/></one:Section>
corpus/conflict-page/native-pages/a/hierarchy-0013.xml created+2
......@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Section xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="synthetic" ID="{63142E76-6879-0236-0E9B-223917E9FE4E}{1}{B0}" path="\\192.168.77.1\agent\m6-conflict\synthetic.one" lastModifiedTime="2026-09-27T09:22:59.000Z" color="#8AA8E4"><one:Page ID="{4F5DDDF0-E1B2-4127-89FC-8AA8EBD1E251}{1}{B0}" name="Fictitious: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1"/><one:Page ID="{5DBAC284-CA61-451B-B041-C2719A268F2A}{1}{B0}" name="One" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:20:19.000Z" pageLevel="1"/><one:Page ID="{DB35495A-2D25-485D-B817-34DF84D8C5BF}{1}{B0}" name="Two" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:20:19.000Z" pageLevel="1"/><one:Page ID="{FE831102-E2A2-47B2-A9AB-FA1A7ED19095}{1}{B0}" name="Four" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:20:19.000Z" pageLevel="1"/><one:Page ID="{540C9694-2DCE-4E68-8CCE-0B8D44BA9AAB}{1}{B0}" name="Three" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:20:19.000Z" pageLevel="1"/><one:Page ID="{B32E58B6-6903-44FC-A2CF-974EDF7A1EF2}{1}{B0}" name="Target" dateTime="2026-09-27T09:20:18.000Z" lastModifiedTime="2026-09-27T09:22:49.000Z" pageLevel="1"/></one:Section>
corpus/conflict-page/native-pages/a/listed.json created+1
......@@ -0,0 +1 @@
1["Fictitious: café, 東京, مرحبا", "One", "Two", "Four", "Three", "Target"]
\ No newline at end of file
corpus/conflict-page/native-pages/a/machine.json created+1
......@@ -0,0 +1 @@
1{"name": "m6-327953db", "hostname": "ONE-M6-327953DB"}
corpus/conflict-page/native-pages/a/teardown.json created+1
......@@ -0,0 +1 @@
1{"absent": true}
corpus/conflict-page/native-pages/b-published/notebook/OneNote_RecycleBin/OneNote_DeletedPages.one created
Binary files /dev/null and b/corpus/conflict-page/native-pages/b-published/notebook/OneNote_RecycleBin/OneNote_DeletedPages.one differ
corpus/conflict-page/native-pages/b-published/notebook/OneNote_RecycleBin/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/native-pages/b-published/notebook/OneNote_RecycleBin/Open Notebook.onetoc2 differ
corpus/conflict-page/native-pages/b-published/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/native-pages/b-published/notebook/Open Notebook.onetoc2 differ
corpus/conflict-page/native-pages/b-published/notebook/synthetic.one created
Binary files /dev/null and b/corpus/conflict-page/native-pages/b-published/notebook/synthetic.one differ
corpus/conflict-page/native-pages/b/b-merged.json created+13
......@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Fictitious: caf\u00e9, \u6771\u4eac, \u0645\u0631\u062d\u0628\u0627 - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/native-pages/b/b-merged.png created
Binary files /dev/null and b/corpus/conflict-page/native-pages/b/b-merged.png differ
corpus/conflict-page/native-pages/b/command-0001.json created+1
......@@ -0,0 +1 @@
1{"action": "snapshot"}
\ No newline at end of file
corpus/conflict-page/native-pages/b/command-0002.json created+1
......@@ -0,0 +1 @@
1{"action": "move", "page": "Four", "before": null}
\ No newline at end of file
corpus/conflict-page/native-pages/b/command-0003.json created+1
......@@ -0,0 +1 @@
1{"action": "move", "page": "Three", "before": "One"}
\ No newline at end of file
corpus/conflict-page/native-pages/b/command-0004.json created+1
......@@ -0,0 +1 @@
1{"action": "delete-page", "page": "Target"}
\ No newline at end of file
corpus/conflict-page/native-pages/b/command-0005.json created+1
......@@ -0,0 +1 @@
1{"action": "snapshot"}
\ No newline at end of file
corpus/conflict-page/native-pages/b/command-0006.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-pages/b/command-0007.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-pages/b/command-0008.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-pages/b/command-0009.json created+1
......@@ -0,0 +1 @@
1{"action": "snapshot"}
\ No newline at end of file
corpus/conflict-page/native-pages/b/command-0010.json created+1
......@@ -0,0 +1 @@
1{"action": "close"}
\ No newline at end of file
corpus/conflict-page/native-pages/b/commands.jsonl created+12
......@@ -0,0 +1,12 @@
1{"command": "mkdir C:\\one-tests\\runs\\capture", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -ExecutionPolicy Bypass -File C:\\one-tests\\network.ps1 -LabMac 52:54:02:19:c7:6d", "exit": 0, "stdout": "{\"renewalResult\":0,\"releaseResult\":0,\"mac\":\"52:54:02:19:c7:6d\",\"address\":\"192.168.77.210\"}\r\n", "stderr": "", "error": null}
3{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0001.tmp C:\\one-tests\\runs\\capture\\inbox\\0001.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
4{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0002.tmp C:\\one-tests\\runs\\capture\\inbox\\0002.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
5{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0003.tmp C:\\one-tests\\runs\\capture\\inbox\\0003.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
6{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0004.tmp C:\\one-tests\\runs\\capture\\inbox\\0004.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
7{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0005.tmp C:\\one-tests\\runs\\capture\\inbox\\0005.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
8{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0006.tmp C:\\one-tests\\runs\\capture\\inbox\\0006.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
9{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0007.tmp C:\\one-tests\\runs\\capture\\inbox\\0007.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
10{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0008.tmp C:\\one-tests\\runs\\capture\\inbox\\0008.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
11{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0009.tmp C:\\one-tests\\runs\\capture\\inbox\\0009.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
12{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0010.tmp C:\\one-tests\\runs\\capture\\inbox\\0010.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
corpus/conflict-page/native-pages/b/hierarchy-0001.xml created+2
......@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Section xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="synthetic" ID="{D7EAB610-D407-0FFC-036F-02FDAAF1BCEF}{1}{B0}" path="\\192.168.77.1\agent\m6-conflict\synthetic.one" lastModifiedTime="2026-09-27T09:21:21.000Z" color="#8AA8E4" isUnread="true"><one:Page ID="{FBA34596-5DCC-4CED-8408-AA6C56C9A0F0}{1}{B0}" name="Fictitious: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1"/><one:Page ID="{E9445AE2-761F-48D1-BDB5-E2B5273ECD8B}{1}{B0}" name="One" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:21:21.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{6FCBD13C-915B-4597-B5E3-141B39C0871E}{1}{B0}" name="Two" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:21:21.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{E0F20EF2-91B0-43A2-813A-2B49F9A2D80A}{1}{B0}" name="Three" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:21:21.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{4A7D8964-5EDC-4A78-A45F-DADEC3C9D234}{1}{B0}" name="Four" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:21:21.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{FE625F4D-E3F3-4BAC-B1E9-5E48521511F0}{1}{B0}" name="Target" dateTime="2026-09-27T09:20:18.000Z" lastModifiedTime="2026-09-27T09:21:21.000Z" pageLevel="1" isUnread="true"/></one:Section>
corpus/conflict-page/native-pages/b/hierarchy-0005.xml created+2
......@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Section xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="synthetic" ID="{D7EAB610-D407-0FFC-036F-02FDAAF1BCEF}{1}{B0}" path="\\192.168.77.1\agent\m6-conflict\synthetic.one" lastModifiedTime="2026-09-27T09:22:44.000Z" color="#8AA8E4" isUnread="true"><one:Page ID="{FBA34596-5DCC-4CED-8408-AA6C56C9A0F0}{1}{B0}" name="Fictitious: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1"/><one:Page ID="{E0F20EF2-91B0-43A2-813A-2B49F9A2D80A}{1}{B0}" name="Three" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:21:21.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{E9445AE2-761F-48D1-BDB5-E2B5273ECD8B}{1}{B0}" name="One" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:21:21.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{6FCBD13C-915B-4597-B5E3-141B39C0871E}{1}{B0}" name="Two" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:21:21.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{4A7D8964-5EDC-4A78-A45F-DADEC3C9D234}{1}{B0}" name="Four" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:21:21.000Z" pageLevel="1" isUnread="true"/></one:Section>
corpus/conflict-page/native-pages/b/hierarchy-0009.xml created+2
......@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Section xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="synthetic" ID="{D7EAB610-D407-0FFC-036F-02FDAAF1BCEF}{1}{B0}" path="\\192.168.77.1\agent\m6-conflict\synthetic.one" lastModifiedTime="2026-09-27T09:23:22.000Z" color="#8AA8E4" isUnread="true"><one:Page ID="{FBA34596-5DCC-4CED-8408-AA6C56C9A0F0}{1}{B0}" name="Fictitious: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1"/><one:Page ID="{E9445AE2-761F-48D1-BDB5-E2B5273ECD8B}{1}{B0}" name="One" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:21:21.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{6FCBD13C-915B-4597-B5E3-141B39C0871E}{1}{B0}" name="Two" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:21:21.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{4A7D8964-5EDC-4A78-A45F-DADEC3C9D234}{1}{B0}" name="Four" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:21:21.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{E0F20EF2-91B0-43A2-813A-2B49F9A2D80A}{1}{B0}" name="Three" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:21:21.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{07D0C0D0-D57D-4936-AF3B-B78A62625C53}{1}{B0}" name="Target" dateTime="2026-09-27T09:20:18.000Z" lastModifiedTime="2026-09-27T09:23:22.000Z" pageLevel="1" isUnread="true"/></one:Section>
corpus/conflict-page/native-pages/b/listed.json created+1
......@@ -0,0 +1 @@
1["Fictitious: café, 東京, مرحبا", "One", "Two", "Four", "Three", "Target"]
\ No newline at end of file
corpus/conflict-page/native-pages/b/machine.json created+1
......@@ -0,0 +1 @@
1{"name": "m6-39da335d", "hostname": "ONE-M6-39DA335D"}
corpus/conflict-page/native-pages/b/teardown.json created+1
......@@ -0,0 +1 @@
1{"absent": true}
corpus/conflict-page/native-pages/clients.json created+4
......@@ -0,0 +1,4 @@
1{
2 "a": "m6-327953db",
3 "b": "m6-39da335d"
4}
\ No newline at end of file
corpus/conflict-page/native-pages/initial/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/native-pages/initial/notebook/Open Notebook.onetoc2 differ
corpus/conflict-page/native-pages/initial/notebook/synthetic.one created
Binary files /dev/null and b/corpus/conflict-page/native-pages/initial/notebook/synthetic.one differ
corpus/conflict-page/native-pages/linux-teardown.json created+1
......@@ -0,0 +1 @@
1{"absent": true}
corpus/conflict-page/native-pages/linux.json created+11
......@@ -0,0 +1,11 @@
1{
2 "cpus": 2,
3 "disk_gb": 16,
4 "hostname": "one-conflict2",
5 "lab_address": "192.168.77.1",
6 "lab_mac": "52:54:04:00:03:8c",
7 "memory_mb": 2048,
8 "samba_port": 14450,
9 "ssh_port": 22000,
10 "wan_mac": "52:54:03:c3:23:a2"
11}
\ No newline at end of file
corpus/conflict-page/native-pages/merged.json created+49
......@@ -0,0 +1,49 @@
1{
2 "order": [
3 "",
4 "One",
5 "Two",
6 "Four",
7 "Three",
8 "Target"
9 ],
10 "pages": [
11 [
12 "",
13 "Fictitious positioned outline.",
14 "Fictitious: café, 東京, مرحبا",
15 "Left cell",
16 "Right cell"
17 ],
18 [
19 "2:20 AM",
20 "Body One.",
21 "One",
22 "Sunday, September 27, 2026"
23 ],
24 [
25 "2:20 AM",
26 "Body Two.",
27 "Sunday, September 27, 2026",
28 "Two"
29 ],
30 [
31 "2:20 AM",
32 "Body Four.",
33 "Four",
34 "Sunday, September 27, 2026"
35 ],
36 [
37 "2:20 AM",
38 "Body Three.",
39 "Sunday, September 27, 2026",
40 "Three"
41 ],
42 [
43 "2:20 AM",
44 "Body Target edited offline.",
45 "Sunday, September 27, 2026",
46 "Target"
47 ]
48 ]
49}
\ No newline at end of file
corpus/conflict-page/native-pages/merged/notebook/OneNote_RecycleBin/OneNote_DeletedPages.one created
Binary files /dev/null and b/corpus/conflict-page/native-pages/merged/notebook/OneNote_RecycleBin/OneNote_DeletedPages.one differ
corpus/conflict-page/native-pages/merged/notebook/OneNote_RecycleBin/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/native-pages/merged/notebook/OneNote_RecycleBin/Open Notebook.onetoc2 differ
corpus/conflict-page/native-pages/merged/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/native-pages/merged/notebook/Open Notebook.onetoc2 differ
corpus/conflict-page/native-pages/merged/notebook/synthetic.one created
Binary files /dev/null and b/corpus/conflict-page/native-pages/merged/notebook/synthetic.one differ
corpus/conflict-page/native-pages/native_conflict.py created+362
......@@ -0,0 +1,362 @@
1#!/usr/bin/env python3
2"""OneNote 2010 conflict pages: how they arise, what they store and how OneNote shows them.
3
4`sync LINUX OUTPUT` runs two disposable OneNote clients on one Samba notebook: client A edits
5two paragraphs offline while client B edits one of them and a table cell online; A then
6reconnects, which makes OneNote write a conflict page. The server file is kept before and
7after, A and B are screenshotted with the information bar, the page list shown, the conflict
8page open and its menu open, and A then deletes the conflict page through that menu.
9
10`pages LINUX OUTPUT` runs the same two clients on page-list changes: client A, offline, moves
11a page that client B also moves, moves another page, and edits a page B deletes; A then
12reconnects. The server's page order and pages, with the recycle bin, are kept at each step.
13
14`restore LINUX OUTPUT` runs them on a page in the middle of the list: client A, offline, edits
15it and moves a page B leaves alone while B deletes it; A then reconnects.
16
17`cold NOTEBOOK OUTPUT [--delete]` cold-opens a notebook holding a conflict page in a fresh
18clone and screenshots the same views; `--delete` deletes the conflict page through the menu
19and captures the notebook OneNote leaves.
20"""
21import argparse
22from contextlib import ExitStack
23from concurrent.futures import ThreadPoolExecutor
24import base64
25import json
26from pathlib import Path
27import shutil
28import subprocess
29import sys
30import time
31import xml.etree.ElementTree as ET
32
33sys.path.insert(0, str(Path(__file__).resolve().parent))
34sys.path.insert(0, str(Path(__file__).resolve().parent / 'w7'))
35import native_runner as runner
36from native_runner import ROOT, clone, command, windows, vm
37import linux_vm
38
39FIXTURE = ROOT / 'corpus/native-ink/cold-ui-ink/notebook'
40SHARE = 'm6-conflict'
41ORIGINAL = 'Fictitious: café, 東京, مرحبا'
42OTHER_OUTLINE = 'Fictitious positioned outline.'
43CELL = 'Left cell'
44
45# Coordinates on OneNote 2010 maximized at 800x600, from the stage-5, m8 and sync-01 captures:
46# the information bar's text, and the first conflict page listed under its page.
47BAR = (332, 100)
48FIRST_CONFLICT = (725, 142)
49# Delete Conflict Page, first in the menu the bar opens on a conflict page.
50DELETE = (420, 113)
51
52
53def shot(name, output, label, script):
54 """Runs an AutoHotkey step on the clone, keeping its screenshot as `label.png`."""
55 (output / f'{label}.ahk').write_text(script)
56 result = windows.do_exec(script, target=name, timeout_ms=120000, shot_delay_ms=1500)
57 screenshot = result.pop('png_b64', None)
58 if screenshot:
59 (output / f'{label}.png').write_bytes(base64.b64decode(screenshot))
60 (output / f'{label}.json').write_text(json.dumps(result, indent=2))
61 if result.get('error') or result.get('exit') != 0:
62 raise RuntimeError(f'{label}: {result}')
63 return result
64
65
66def click(x, y):
67 return ('WinActivate("ahk_exe ONENOTE.EXE")\nCoordMode("Mouse", "Screen")\n'
68 f'Click({x}, {y})\nSleep(1200)\n')
69
70
71# Reports any dialog OneNote raised, so a confirmation shows in the step's record.
72DIALOG = ('if WinExist("ahk_class #32770 ahk_exe ONENOTE.EXE") {\n'
73 ' FileAppend("dialog: " WinGetTitle() " | " WinGetText() "`n", "*", "UTF-8")\n'
74 '}\n')
75
76
77def conflict_views(name, output, page, prefix, delete=False):
78 """Screenshots a page with a conflict page: the bar, the list shown, the conflict page and
79 its menu; `delete` then chooses Delete Conflict Page."""
80 shot(name, output, f'{prefix}-main', runner.navigation_script(page) + 'Sleep(2000)\n')
81 shot(name, output, f'{prefix}-shown', click(*BAR))
82 shot(name, output, f'{prefix}-conflict', click(*FIRST_CONFLICT))
83 shot(name, output, f'{prefix}-menu', click(*BAR))
84 if delete:
85 shot(name, output, f'{prefix}-delete', click(*DELETE) + DIALOG)
86 else:
87 shot(name, output, f'{prefix}-closed', 'WinActivate("ahk_exe ONENOTE.EXE")\nSend("{Esc}")\nSleep(800)\n')
88
89
90def cold(notebook, output, delete):
91 def interaction(name, output):
92 page = ET.parse(sorted((output / 'read').glob('page-*.xml'))[0]).getroot().attrib['ID']
93 conflict_views(name, output, page, 'cold', delete)
94 if delete:
95 # OneNote writes the deletion in the background; give it time before the re-read.
96 time.sleep(20)
97 runner.capture(notebook, output, collect_notebook=True, interaction=interaction)
98
99
100class Lab:
101 """Two OneNote clients on one Samba notebook: `action` runs a controller command,
102 `state` copies the share and exports its section, `settle` syncs until a check holds."""
103
104
105def page_list(model):
106 """The section's pages in order, each as its sorted texts."""
107 from document_model import ordered_pages, walk
108 return [sorted(node['kind']['text'] for _, node in walk(page, oid) if node['kind']['type'] == 'RichText')
109 for _, _, page, oid in ordered_pages(model)]
110
111
112def edits(lab):
113 from native_collaboration import reachable_page_text
114 a, b, output, action, settle = lab.a, lab.b, lab.output, lab.action, lab.settle
115 state = lambda label: reachable_page_text(lab.state(label))
116 texts = lambda state: {text for values in state.values() for text in values}
117 state('initial')
118 vm.qmp(a['name'], 'set_link', {'name': 'lab', 'up': False})
119 try:
120 action(a, 'edit', expected=ORIGINAL, text='Client A conflicting edit.')
121 action(a, 'edit', expected=OTHER_OUTLINE, text='Client A disjoint edit.')
122 action(a, 'snapshot')
123 action(b, 'edit', expected=ORIGINAL, text='Client B conflicting edit.')
124 action(b, 'edit', expected=CELL, text='Client B disjoint cell.')
125 settle(b, lambda label: {'Client B conflicting edit.', 'Client B disjoint cell.'} <= texts(state(label)), 'b-published')
126 finally:
127 vm.qmp(a['name'], 'set_link', {'name': 'lab', 'up': True})
128 settle(a, lambda label: len(found := state(label)) > 1 and 'Client A disjoint edit.' in texts(found), 'conflict')
129 (output / 'conflict.json').write_text(json.dumps(state('conflict'), indent=2, ensure_ascii=False))
130 page = action(a, 'snapshot')[0].attrib['ID']
131 conflict_views(a['name'], output / 'a', page, 'a')
132 action(b, 'sync')
133 time.sleep(5)
134 page_b = action(b, 'snapshot')[0].attrib['ID']
135 conflict_views(b['name'], output / 'b', page_b, 'b')
136 conflict_views(a['name'], output / 'a', page, 'a-again', delete=True)
137 settle(a, lambda label: len(state(label)) == 1, 'deleted')
138 action(b, 'sync')
139 time.sleep(5)
140 shot(b['name'], output / 'b', 'b-after-delete', runner.navigation_script(page_b) + 'Sleep(2000)\n')
141
142
143PAGES = ['One', 'Two', 'Three', 'Four', 'Target']
144
145
146def titled(model, titles):
147 """The section's pages in order, each named by the title among `titles` it holds."""
148 return [next((text for text in texts if text in titles), '') for texts in page_list(model)]
149
150
151def offline(lab, titles, a_actions, b_actions):
152 """Creates the titled pages; A runs its actions offline while B runs and publishes its
153 own; A then reconnects. Keeps the merged order and pages, and what each client lists."""
154 a, b, output, action = lab.a, lab.b, lab.output, lab.action
155 action(a, 'pages', titles=titles)
156 lab.settle(a, lambda label: 'Body Target.' in str(page_list(lab.state(label))), 'initial')
157 deadline = time.monotonic() + 300
158 while not any(node.get('name') == 'Target' for node in action(b, 'snapshot')):
159 action(b, 'sync')
160 if time.monotonic() > deadline: raise TimeoutError('B never listed the new pages')
161 vm.qmp(a['name'], 'set_link', {'name': 'lab', 'up': False})
162 try:
163 for kind, parameters in a_actions:
164 action(a, kind, **parameters)
165 action(a, 'snapshot')
166 for kind, parameters in b_actions:
167 action(b, kind, **parameters)
168 action(b, 'snapshot')
169 lab.settle(b, lambda label: 'Body Target.' not in str(page_list(lab.state(label))), 'b-published')
170 finally:
171 vm.qmp(a['name'], 'set_link', {'name': 'lab', 'up': True})
172 # A's changes reach the server when its sync writes; wait until the file stops changing.
173 previous, steady = None, 0
174 deadline = time.monotonic() + 300
175 while steady < 3 and time.monotonic() < deadline:
176 action(a, 'sync')
177 time.sleep(10)
178 current = page_list(lab.state('merged'))
179 steady = steady + 1 if current == previous else 0
180 previous = current
181 (output / 'merged.json').write_text(json.dumps({'order': titled(lab.state('merged'), titles), 'pages': previous}, indent=2, ensure_ascii=False))
182 for client, label in ((a, 'a'), (b, 'b')):
183 action(client, 'sync')
184 time.sleep(5)
185 listed = action(client, 'snapshot')
186 (output / label / 'listed.json').write_text(json.dumps([node.get('name') for node in listed], ensure_ascii=False))
187 if listed:
188 shot(client['name'], output / label, f'{label}-merged', runner.navigation_script(listed[0].get('ID')) + 'Sleep(2000)\n')
189
190
191def pages(lab):
192 """Competing page moves, and an offline edit of a page the other client deletes."""
193 offline(lab, PAGES, [
194 ('move', {'page': 'Four', 'before': 'One'}),
195 ('move', {'page': 'Two', 'before': None}),
196 ('edit', {'expected': 'Body Target.', 'text': 'Body Target edited offline.'}),
197 ], [
198 ('move', {'page': 'Four', 'before': None}),
199 ('move', {'page': 'Three', 'before': 'One'}),
200 ('delete-page', {'page': 'Target'}),
201 ])
202
203
204def restore(lab):
205 """An offline edit of a page in the middle of the list the other client deletes, with an
206 offline move of a page the other client leaves alone."""
207 offline(lab, ['One', 'Two', 'Target', 'Three', 'Four'], [
208 ('edit', {'expected': 'Body Target.', 'text': 'Body Target edited offline.'}),
209 ('move', {'page': 'Four', 'before': 'One'}),
210 ], [
211 ('delete-page', {'page': 'Target'}),
212 ])
213
214
215def sync(server, output, scenario):
216 from document_model import EXPORTER
217
218 if linux_vm.instance_path(server).exists():
219 raise ValueError('Choose a new Linux VM name; existing machines are not owned by this run.')
220 output = output.resolve()
221 output.mkdir(parents=True, exist_ok=False)
222 scripts = output / 'scripts'
223 scripts.mkdir()
224 for name in ('cold.ps1', 'collaborate.ps1', 'network.ps1', 'text.ps1', 'stress.ps1'):
225 shutil.copyfile(ROOT / 'tools/native' / name, scripts / name)
226 shutil.copyfile(Path(__file__), output / 'native_conflict.py')
227
228 def ssh(text, timeout=90):
229 result = linux_vm.run_ssh(server, text, timeout=timeout)
230 with (output / 'server.jsonl').open('a') as log:
231 log.write(json.dumps({'command': text, 'exit': result.returncode, 'stdout': result.stdout, 'stderr': result.stderr}) + '\n')
232 result.check_returncode()
233 return result.stdout
234
235 def server_state(label):
236 """Copies the server's notebook, recycle bin included, until its section parses,
237 keeping it as `label/notebook`; returns the section's exported model."""
238 deadline = time.monotonic() + 120
239 while True:
240 target = output / label
241 shutil.rmtree(target, ignore_errors=True)
242 (target / 'notebook').mkdir(parents=True)
243 archive = subprocess.run(linux_vm.ssh_argv(server, f"tar cf - -C '/srv/agent/{SHARE}' ."), capture_output=True, check=True, timeout=60).stdout
244 subprocess.run(['tar', 'xf', '-', '-C', target / 'notebook'], input=archive, check=True)
245 result = subprocess.run([EXPORTER, target / 'notebook/synthetic.one', target / 'model'], capture_output=True, text=True)
246 if result.returncode == 0:
247 return json.loads((target / 'model/document.json').read_text())
248 if time.monotonic() > deadline:
249 raise RuntimeError(f'{label}: {result.stderr}')
250 time.sleep(1)
251
252 linux_vm.create_instance(server)
253 try:
254 linux_vm.launch(server)
255 linux_vm.wait_instance(server, 600)
256 (output / 'linux.json').write_text(json.dumps(linux_vm.load_instance(server), indent=2))
257 ssh(f'mkdir /srv/agent/{SHARE}')
258 for name in ('synthetic.one', 'Open Notebook.onetoc2'):
259 with (FIXTURE / name).open('rb') as stream:
260 subprocess.run(linux_vm.ssh_argv(server, f"cat > '/srv/agent/{SHARE}/{name}'"), stdin=stream, check=True)
261 ssh(f'chmod u+w /srv/agent/{SHARE}/*')
262 with ExitStack() as stack:
263 labels = ['a', 'b']
264 for label in labels:
265 (output / label).mkdir()
266
267 def start(label):
268 manager = clone(output / label)
269 name = manager.__enter__()
270 return label, name, manager
271 with ThreadPoolExecutor(max_workers=2) as pool:
272 started = list(pool.map(start, labels))
273 for _, _, manager in started:
274 stack.push(manager)
275 clients = {}
276 for label, name, _ in started:
277 folder = output / label
278 for local in scripts.iterdir():
279 remote = 'cold-current.ps1' if local.name == 'cold.ps1' else local.name
280 result = windows.do_put(local, 'C:\\one-tests\\' + remote, name)
281 if result.get('error'): raise RuntimeError(result['error'])
282 command(name, 'mkdir C:\\one-tests\\runs\\capture', folder)
283 command(name, 'powershell -NoProfile -ExecutionPolicy Bypass -File C:\\one-tests\\network.ps1 -LabMac ' + vm.lab_mac(name), folder)
284 result = windows.do_spawn('cmd /c powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\collaborate.ps1 -Root C:\\one-tests\\runs\\capture -SharedPath \\\\192.168.77.1\\agent\\' + SHARE + ' -CloneHost ONE-' + name.upper() + ' > C:\\one-tests\\runs\\capture\\controller.log 2>&1', name)
285 if result.get('error'): raise RuntimeError(result['error'])
286 deadline = time.monotonic() + 180
287 while windows.do_cmd('if exist C:\\one-tests\\runs\\capture\\ready (echo ready)', target=name).get('stdout', '').find('ready') < 0:
288 if time.monotonic() > deadline: raise RuntimeError('The collaboration controller did not open the notebook')
289 time.sleep(1)
290 clients[label] = {'name': name, 'folder': folder, 'sequence': 0}
291 (output / 'clients.json').write_text(json.dumps({label: client['name'] for label, client in clients.items()}, indent=2))
292
293 def action(client, kind, **parameters):
294 client['sequence'] += 1
295 sequence = client['sequence']
296 local = client['folder'] / f'command-{sequence:04}.json'
297 local.write_text(json.dumps({'action': kind, **parameters}, ensure_ascii=False))
298 inbox = f'C:\\one-tests\\runs\\capture\\inbox\\{sequence:04}'
299 result = windows.do_put(local, inbox + '.tmp', client['name'])
300 if result.get('error'): raise RuntimeError(result['error'])
301 command(client['name'], f'move {inbox}.tmp {inbox}.json', client['folder'])
302 remote = f'C:\\one-tests\\runs\\capture\\outbox\\{sequence}'
303 deadline = time.monotonic() + 600
304 while True:
305 result = windows.do_cmd(f'if exist {remote}\\error (type {remote}\\error & exit /b 1) else (if exist {remote}\\done (echo complete))', target=client['name'])
306 if result.get('exit') != 0: raise RuntimeError(str(result))
307 if 'complete' in result.get('stdout', ''): break
308 if time.monotonic() > deadline: raise TimeoutError(f'{kind} did not complete')
309 time.sleep(.5)
310 if kind != 'snapshot':
311 return None
312 hierarchy = client['folder'] / f'hierarchy-{sequence:04}.xml'
313 windows.do_get(remote + '\\hierarchy.xml', hierarchy, client['name'])
314 pages = [node for node in ET.fromstring(hierarchy.read_text(encoding='utf-8-sig').strip()).iter() if node.tag.endswith('}Page')]
315 for i in range(len(pages)):
316 windows.do_get(remote + f'\\page-{i}.xml', client['folder'] / f'snapshot-{sequence:04}-{i}.xml', client['name'])
317 return pages
318
319 def settle(client, check, label):
320 deadline = time.monotonic() + 300
321 while True:
322 action(client, 'sync')
323 if check(label): return
324 if time.monotonic() > deadline: raise TimeoutError(label)
325 time.sleep(2)
326
327 lab = Lab()
328 lab.a, lab.b, lab.output = clients['a'], clients['b'], output
329 lab.action, lab.state, lab.settle = action, server_state, settle
330 scenario(lab)
331
332 for client in clients.values():
333 try:
334 action(client, 'close')
335 except Exception as error:
336 (client['folder'] / 'close-failure.txt').write_text(str(error))
337 server_state('final')
338 finally:
339 if linux_vm.running(server):
340 linux_vm.shutdown(server, 120)
341 linux_vm.delete_instance(server)
342 (output / 'linux-teardown.json').write_text(json.dumps({'absent': not linux_vm.instance_path(server).exists()}) + '\n')
343
344
345if __name__ == '__main__':
346 parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
347 commands = parser.add_subparsers(dest='mode', required=True)
348 scenarios = {'sync': edits, 'pages': pages, 'restore': restore}
349 for mode in scenarios:
350 lab = commands.add_parser(mode)
351 lab.add_argument('linux')
352 lab.add_argument('output', type=Path)
353 reopen = commands.add_parser('cold')
354 reopen.add_argument('notebook', type=Path)
355 reopen.add_argument('output', type=Path)
356 reopen.add_argument('--delete', action='store_true')
357 args = parser.parse_args()
358 if args.mode in scenarios:
359 sync(args.linux, args.output, scenarios[args.mode])
360 else:
361 cold(args.notebook, args.output, args.delete)
362 print('Captured', args.output)
corpus/conflict-page/native-pages/scripts/cold.ps1 created+27
......@@ -0,0 +1,27 @@
1param([Parameter(Mandatory=$true)][string]$Root, [string]$CloneHost = '')
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
5if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
6 throw 'Choose a run directly below C:\one-tests\runs.'
7}
8if (Get-Process ONENOTE -ErrorAction SilentlyContinue) { throw 'Close OneNote before resetting its test cache.' }
9$key = 'HKCU:\Software\Microsoft\Office\14.0\OneNote'
10if ($CloneHost) {
11 if ($CloneHost -notmatch '^ONE-[A-Z0-9-]+$' -or [Environment]::MachineName -ne $CloneHost) {
12 throw 'The disposable clone hostname does not match this machine.'
13 }
14 New-Item "$key\Options\Paths" -Force | Out-Null
15 New-ItemProperty "$key\Options\Paths" -Name UnfiledNotesSection -PropertyType ExpandString -Value 'C:\one-tests\Loose.one' -Force | Out-Null
16} elseif ((Get-ItemProperty "$key\Options\Paths").UnfiledNotesSection -ne 'C:\one-tests\Loose.one' -or
17 -not (Test-Path 'C:\one-tests\profile-original-cache')) {
18 throw 'Park the personal OneNote profile before resetting the test cache.'
19}
20$cache = Join-Path $env:LOCALAPPDATA 'Microsoft\OneNote\14.0'
21$parked = Join-Path 'C:\one-tests\caches' ([IO.Path]::GetFileName($root))
22if (Test-Path $parked) { throw 'Choose a new run; its parked cache already exists.' }
23New-Item -ItemType Directory -Path 'C:\one-tests\caches' -Force | Out-Null
24if (Test-Path $cache) { Move-Item -LiteralPath $cache -Destination $parked }
25if (Test-Path "$key\OpenNotebooks") { Remove-Item "$key\OpenNotebooks" -Recurse }
26New-Item "$key\OpenNotebooks" | Out-Null
27New-ItemProperty "$key\OpenNotebooks" -Name '1' -PropertyType String -Value "$root\notebook" | Out-Null
corpus/conflict-page/native-pages/scripts/collaborate.ps1 created+145
......@@ -0,0 +1,145 @@
1param([Parameter(Mandatory=$true)][string]$Root,
2 [Parameter(Mandatory=$true)][string]$SharedPath,
3 [Parameter(Mandatory=$true)][string]$CloneHost,
4 [switch]$ResumeCache,
5 [int]$StartSequence = 1)
6function Keep-Container($page, $target) {
7 $container = $target
8 while ($container.ParentNode -ne $page.DocumentElement) { $container = $container.ParentNode }
9 foreach ($sibling in @($page.DocumentElement.ChildNodes)) {
10 if ($sibling -ne $container -and $sibling.LocalName -in @('Outline', 'Title', 'Image', 'InkDrawing')) {
11 [void]$page.DocumentElement.RemoveChild($sibling)
12 }
13 }
14}
15Set-StrictMode -Version Latest
16$ErrorActionPreference = 'Stop'
17. "$PSScriptRoot\text.ps1"
18if ($SharedPath -notmatch '^\\\\192\.168\.77\.1\\agent\\m6-[a-z0-9-]+$') { throw 'Choose a disposable Linux lab notebook.' }
19if ($env:COMPUTERNAME -ne $CloneHost) { throw 'The collaboration controller requires its owned clone.' }
20if (-not $ResumeCache) { & "$PSScriptRoot\cold-current.ps1" -Root $Root -CloneHost $CloneHost }
21$shared = $SharedPath
22if (-not (Test-Path "$shared\synthetic.one")) { throw 'Create the shared fixture first.' }
23New-Item -ItemType Directory "$Root\inbox", "$Root\outbox" -Force | Out-Null
24$user = 'HKCU:\Software\Microsoft\Office\Common\UserInfo'
25New-Item $user -Force | Out-Null
26New-ItemProperty $user -Name UserName -Value $CloneHost -PropertyType String -Force | Out-Null
27New-ItemProperty $user -Name UserInitials -Value $CloneHost.Substring($CloneHost.Length - 3) -PropertyType String -Force | Out-Null
28$app = New-Object -ComObject OneNote.Application
29$notebook = ''
30try {
31 $app.OpenHierarchy($shared, '', [ref]$notebook, 0)
32 $section = ''
33 $app.OpenHierarchy('synthetic.one', $notebook, [ref]$section, 0)
34 [IO.File]::WriteAllText("$Root\ready", 'ready')
35 $deadline = [DateTime]::UtcNow.AddMinutes(30)
36 $sequence = $StartSequence
37 while ([DateTime]::UtcNow -lt $deadline) {
38 $name = '{0:d4}.json' -f $sequence
39 $input = "$Root\inbox\$name"
40 if (-not (Test-Path $input)) { Start-Sleep -Milliseconds 100; continue }
41 $command = Get-Content -Raw -Encoding UTF8 $input | ConvertFrom-Json
42 $output = "$Root\outbox\$sequence"
43 New-Item -ItemType Directory $output | Out-Null
44 try {
45 if ($command.action -in @('prepare-stress', 'stress')) {
46 . "$PSScriptRoot\stress.ps1"
47 Invoke-Stress $app $section $command $output $shared
48 }
49 elseif ($command.action -eq 'sync') { $app.SyncHierarchy($notebook) }
50 elseif ($command.action -in @('pages', 'move', 'delete-page')) {
51 $namespace = 'http://schemas.microsoft.com/office/onenote/2010/onenote'
52 $hierarchy = ''
53 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
54 [xml]$tree = $hierarchy
55 $listed = @($tree.SelectNodes('//*[local-name()="Page"]'))
56 $named = { param($title) @($listed | Where-Object { $_.GetAttribute('name') -eq $title }) }
57 if ($command.action -eq 'pages') {
58 # New titled pages with one body paragraph each, appended in order.
59 foreach ($title in $command.titles) {
60 $page = ''
61 $app.CreateNewPage($section, [ref]$page, 2)
62 $body = [Security.SecurityElement]::Escape('Body ' + $title + '.')
63 $name = [Security.SecurityElement]::Escape($title)
64 $xml = '<one:Page xmlns:one="' + $namespace + '" ID="' + $page + '"><one:Title><one:OE><one:T>' + $name + '</one:T></one:OE></one:Title><one:Outline><one:OEChildren><one:OE><one:T>' + $body + '</one:T></one:OE></one:OEChildren></one:Outline></one:Page>'
65 $app.UpdatePageContent($xml, [DateTime]::MinValue, 1, $false)
66 }
67 } elseif ($command.action -eq 'move') {
68 # Moves the page titled `page` before the one titled `before`, or last.
69 $moved = @(& $named $command.page)
70 if ($moved.Count -ne 1) { throw 'Expected one page to move.' }
71 $order = [Collections.ArrayList]@($listed | Where-Object { $_ -ne $moved[0] })
72 $at = $order.Count
73 if ($command.before) {
74 $anchor = @(& $named $command.before)
75 if ($anchor.Count -ne 1) { throw 'Expected one anchor page.' }
76 $at = $order.IndexOf($anchor[0])
77 }
78 $order.Insert($at, $moved[0])
79 $xml = '<one:Section xmlns:one="' + $namespace + '" ID="' + $section + '">'
80 foreach ($node in $order) {
81 $xml += '<one:Page ID="' + $node.GetAttribute('ID') + '" pageLevel="' + $node.GetAttribute('pageLevel') + '"/>'
82 }
83 $app.UpdateHierarchy($xml + '</one:Section>', 1)
84 } else {
85 $doomed = @(& $named $command.page)
86 if ($doomed.Count -ne 1) { throw 'Expected one page to delete.' }
87 $app.DeleteHierarchy($doomed[0].GetAttribute('ID'), [DateTime]::MinValue, $false)
88 }
89 }
90 elseif ($command.action -eq 'kill-process') {
91 Get-Process ONENOTE | Stop-Process -Force
92 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
93 $app = New-Object -ComObject OneNote.Application
94 $app.OpenHierarchy($shared, '', [ref]$notebook, 0)
95 $app.OpenHierarchy('synthetic.one', $notebook, [ref]$section, 0)
96 }
97 elseif ($command.action -eq 'edit' -or $command.action -eq 'snapshot') {
98 $hierarchy = ''
99 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
100 [IO.File]::WriteAllText("$output\hierarchy.xml", $hierarchy, [Text.Encoding]::UTF8)
101 [xml]$tree = $hierarchy
102 $matches = 0
103 $editedPage = $null
104 $i = 0
105 foreach ($node in $tree.SelectNodes('//*[local-name()="Page"]')) {
106 $content = ''
107 $app.GetPageContent($node.GetAttribute('ID'), [ref]$content, 1, 1)
108 [IO.File]::WriteAllText("$output\page-$i.xml", $content, [Text.Encoding]::UTF8)
109 [xml]$page = $content
110 if ($command.action -eq 'edit') {
111 $targets = @($page.SelectNodes('//*[local-name()="T"]') | Where-Object {
112 (Get-ParagraphText $_.InnerText) -eq $command.expected
113 })
114 if ($targets.Count -gt 1) { throw 'Expected one matching text object.' }
115 if ($targets.Count -eq 1) {
116 $matches++
117 if ($matches -gt 1) { throw 'The text occurs on multiple pages.' }
118 $targets[0].InnerText = [Security.SecurityElement]::Escape($command.text)
119 Keep-Container $page $targets[0]
120 $editedPage = $page
121 }
122 }
123 $i++
124 }
125 if ($command.action -eq 'edit' -and $matches -ne 1) { throw 'Expected one matching text object.' }
126 if ($editedPage) { $app.UpdatePageContent($editedPage.OuterXml, [DateTime]::MinValue, 1, $false) }
127 } elseif ($command.action -ne 'close') { throw 'Unknown collaboration command.' }
128 [IO.File]::WriteAllText("$output\done", 'ok')
129 } catch {
130 [IO.File]::WriteAllText("$output\error", $_.Exception.ToString())
131 }
132 if ($command.action -eq 'close') { break }
133 $sequence++
134 }
135} finally {
136 try { if ($notebook) { $app.CloseNotebook($notebook, $false) } }
137 finally {
138 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
139 $app = $null
140 [GC]::Collect()
141 [GC]::WaitForPendingFinalizers()
142 Get-Process ONENOTE -ErrorAction SilentlyContinue | Wait-Process -Timeout 10
143 [IO.File]::WriteAllText("$Root\closed", 'closed')
144 }
145}
corpus/conflict-page/native-pages/scripts/network.ps1 created+20
......@@ -0,0 +1,20 @@
1param([Parameter(Mandatory=$true)][string]$LabMac)
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$adapter = @(Get-WmiObject Win32_NetworkAdapterConfiguration | Where-Object { $_.MACAddress -eq $LabMac })
5if ($adapter.Count -ne 1) { throw 'The clone lab adapter is ambiguous.' }
6$enabled = $adapter[0].EnableDHCP()
7if ($enabled.ReturnValue -notin @(0, 1)) {
8 throw ('Unable to enable DHCP on the clone lab adapter; result ' + $enabled.ReturnValue)
9}
10$release = $adapter[0].ReleaseDHCPLease()
11$result = $adapter[0].RenewDHCPLease()
12$deadline = [DateTime]::UtcNow.AddSeconds(60)
13do {
14 $adapter = @(Get-WmiObject Win32_NetworkAdapterConfiguration | Where-Object { $_.MACAddress -eq $LabMac })
15 $addresses = @($adapter[0].IPAddress | Where-Object { $_ -match '^192\.168\.77\.\d+$' -and $_ -ne '192.168.77.1' })
16 if ($addresses.Count -eq 1) { break }
17 Start-Sleep -Milliseconds 250
18} while ([DateTime]::UtcNow -lt $deadline)
19if ($addresses.Count -ne 1) { throw ('The clone did not receive a lab IPv4 address; renewal result ' + $result.ReturnValue + '; adapter ' + ($adapter[0] | Select-Object MACAddress,DHCPEnabled,IPEnabled,IPAddress | ConvertTo-Json -Compress)) }
20@{mac=$LabMac;address=$addresses[0];releaseResult=$release.ReturnValue;renewalResult=$result.ReturnValue} | ConvertTo-Json -Compress
corpus/conflict-page/native-pages/scripts/stress.ps1 created+83
......@@ -0,0 +1,83 @@
1function Invoke-Stress($app, $section, $command, $output, $shared) {
2 $hierarchy = ''
3 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
4 [IO.File]::WriteAllText("$output\hierarchy.xml", $hierarchy, [Text.Encoding]::UTF8)
5 [xml]$tree = $hierarchy
6 $pages = @($tree.SelectNodes('//*[local-name()="Page"]'))
7 if ($pages.Count -ne 1) { throw 'Expected one stress-test page.' }
8 $pageId = $pages[0].GetAttribute('ID')
9 if ($command.action -eq 'prepare-stress') {
10 for ($i = 0; $i -lt $command.clients; $i++) {
11 $xml = '<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="' + $pageId + '"><one:Outline><one:Position x="48" y="' + (150 + 75 * $i) + '" z="' + ($i + 1) + '"/><one:OEChildren><one:OE><one:T><![CDATA[Native ' + $i + ':]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>'
12 $app.UpdatePageContent($xml, [DateTime]::MinValue, 1, $false)
13 }
14 return
15 }
16 $random = New-Object Random([int]$command.seed)
17 $events = New-Object IO.StreamWriter("$output\events.jsonl", $false, [Text.Encoding]::UTF8)
18 $events.AutoFlush = $true
19 try {
20 [IO.File]::WriteAllText("$output\ready", 'ready')
21 $deadline = [DateTime]::UtcNow.AddMinutes(10)
22 while (-not (Test-Path "$shared\stress-start")) {
23 if ([DateTime]::UtcNow -gt $deadline) { throw 'Stress start timed out.' }
24 Start-Sleep -Milliseconds 20
25 }
26 for ($i = 0; $i -lt $command.operations; $i++) {
27 if (($command.PSObject.Properties.Name -contains 'maintenance') -and $command.maintenance -and $i -eq [Math]::Floor($command.operations / 2)) {
28 [IO.File]::WriteAllText("$shared\maintenance-paused-n$($command.actor)", 'paused')
29 while (-not (Test-Path "$shared\maintenance-resume")) {
30 if ([DateTime]::UtcNow -gt $deadline) { throw 'Maintenance pause timed out.' }
31 Start-Sleep -Milliseconds 100
32 }
33 }
34 Start-Sleep -Milliseconds $random.Next(10, 100)
35 $started = [DateTime]::UtcNow.Ticks
36 $content = ''
37 $app.GetPageContent($pageId, [ref]$content, 1, 1)
38 [xml]$page = $content
39 $matches = @($page.SelectNodes('//*[local-name()="T"]') | Where-Object {
40 (Get-ParagraphText $_.InnerText).StartsWith($command.prefix)
41 })
42 if ($matches.Count -ne 1) { throw 'Expected one stress-test paragraph.' }
43 $before = Get-ParagraphText $matches[0].InnerText
44 $token = ' [n' + $command.actor + ':' + $i + ']'
45 $start = $end = $before.Length
46 $replacement = $token
47 $bold = $italic = $false
48 if ($command.edit) {
49 $boundaries = New-Object 'Collections.Generic.List[int]'
50 for ($at = $command.prefix.Length; $at -le $before.Length; $at++) {
51 $boundaries.Add($at)
52 if ($at -lt $before.Length -and [char]::IsHighSurrogate($before[$at])) { $at++ }
53 }
54 $first = $boundaries[$random.Next($boundaries.Count)]
55 $second = $boundaries[$random.Next($boundaries.Count)]
56 $start = [Math]::Min($first, $second)
57 $end = [Math]::Max($first, $second)
58 $replacement = ' caf' + [char]0xe9 + ' ' + [char]::ConvertFromUtf32(0x1f980) + $token
59 $bold = $random.Next(2) -eq 1
60 $italic = $random.Next(2) -eq 1
61 }
62 $after = $before.Substring(0, $start) + $replacement + $before.Substring($end)
63 $html = [Security.SecurityElement]::Escape($after)
64 if ($command.edit) {
65 $weight = if ($bold) { 'bold' } else { 'normal' }
66 $style = if ($italic) { 'italic' } else { 'normal' }
67 $html = '<span style="font-weight:' + $weight + ';font-style:' + $style + '">' + $html + '</span>'
68 }
69 $matches[0].InnerText = $html
70 Keep-Container $page $matches[0]
71 $updateStarted = [DateTime]::UtcNow.Ticks
72 $app.UpdatePageContent($page.OuterXml, [DateTime]::MinValue, 1, $false)
73 $updated = [DateTime]::UtcNow.Ticks
74 $events.WriteLine((@{ operation=$i; before=$before; token=$token; started_ticks=$started;
75 range=@($start, $end); replacement=$replacement; bold=$bold; italic=$italic;
76 update_started_ticks=$updateStarted; updated_ticks=$updated } | ConvertTo-Json -Compress))
77 if ($i -eq 0) { [IO.File]::WriteAllText("$output\editing", 'editing') }
78 if ($command.sync_every -gt 0 -and (($i + 1) % $command.sync_every) -eq 0) {
79 $app.SyncHierarchy($section)
80 }
81 }
82 } finally { $events.Dispose() }
83}
corpus/conflict-page/native-pages/scripts/text.ps1 created+13
......@@ -0,0 +1,13 @@
1function Get-ParagraphText([string]$html) {
2 $plain = [regex]::Replace($html, '<[^>]*>', '')
3 $plain = [regex]::Replace($plain, '&#(?:x([0-9a-fA-F]+)|([0-9]+));', {
4 param($match)
5 $value = if ($match.Groups[1].Success) {
6 [Convert]::ToInt32($match.Groups[1].Value, 16)
7 } else { [int]$match.Groups[2].Value }
8 # The Win7 framework leaves supplementary numeric entities undecoded.
9 if ($value -gt 0xffff -and $value -le 0x10ffff) { [char]::ConvertFromUtf32($value) }
10 else { $match.Value }
11 })
12 [Net.WebUtility]::HtmlDecode($plain)
13}
corpus/conflict-page/native-pages/server.jsonl created+2
......@@ -0,0 +1,2 @@
1{"command": "mkdir /srv/agent/m6-conflict", "exit": 0, "stdout": "", "stderr": ""}
2{"command": "chmod u+w /srv/agent/m6-conflict/*", "exit": 0, "stdout": "", "stderr": ""}
corpus/conflict-page/native-restore/a/a-merged.json created+13
......@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Fictitious: caf\u00e9, \u6771\u4eac, \u0645\u0631\u062d\u0628\u0627 - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/native-restore/a/a-merged.png created
Binary files /dev/null and b/corpus/conflict-page/native-restore/a/a-merged.png differ
corpus/conflict-page/native-restore/a/command-0001.json created+1
......@@ -0,0 +1 @@
1{"action": "pages", "titles": ["One", "Two", "Target", "Three", "Four"]}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0002.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0003.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0004.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0005.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0006.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0007.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0008.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0009.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0010.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0011.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0012.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0013.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0014.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0015.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0016.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0017.json created+1
......@@ -0,0 +1 @@
1{"action": "edit", "expected": "Body Target.", "text": "Body Target edited offline."}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0018.json created+1
......@@ -0,0 +1 @@
1{"action": "move", "page": "Four", "before": "One"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0019.json created+1
......@@ -0,0 +1 @@
1{"action": "snapshot"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0020.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0021.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0022.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0023.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0024.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0025.json created+1
......@@ -0,0 +1 @@
1{"action": "snapshot"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0026.json created+1
......@@ -0,0 +1 @@
1{"action": "close"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/commands.jsonl created+28
......@@ -0,0 +1,28 @@
1{"command": "mkdir C:\\one-tests\\runs\\capture", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -ExecutionPolicy Bypass -File C:\\one-tests\\network.ps1 -LabMac 52:54:02:b6:9a:97", "exit": 0, "stdout": "{\"renewalResult\":0,\"releaseResult\":0,\"mac\":\"52:54:02:b6:9a:97\",\"address\":\"192.168.77.119\"}\r\n", "stderr": "", "error": null}
3{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0001.tmp C:\\one-tests\\runs\\capture\\inbox\\0001.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
4{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0002.tmp C:\\one-tests\\runs\\capture\\inbox\\0002.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
5{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0003.tmp C:\\one-tests\\runs\\capture\\inbox\\0003.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
6{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0004.tmp C:\\one-tests\\runs\\capture\\inbox\\0004.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
7{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0005.tmp C:\\one-tests\\runs\\capture\\inbox\\0005.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
8{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0006.tmp C:\\one-tests\\runs\\capture\\inbox\\0006.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
9{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0007.tmp C:\\one-tests\\runs\\capture\\inbox\\0007.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
10{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0008.tmp C:\\one-tests\\runs\\capture\\inbox\\0008.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
11{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0009.tmp C:\\one-tests\\runs\\capture\\inbox\\0009.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
12{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0010.tmp C:\\one-tests\\runs\\capture\\inbox\\0010.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
13{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0011.tmp C:\\one-tests\\runs\\capture\\inbox\\0011.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
14{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0012.tmp C:\\one-tests\\runs\\capture\\inbox\\0012.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
15{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0013.tmp C:\\one-tests\\runs\\capture\\inbox\\0013.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
16{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0014.tmp C:\\one-tests\\runs\\capture\\inbox\\0014.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
17{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0015.tmp C:\\one-tests\\runs\\capture\\inbox\\0015.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
18{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0016.tmp C:\\one-tests\\runs\\capture\\inbox\\0016.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
19{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0017.tmp C:\\one-tests\\runs\\capture\\inbox\\0017.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
20{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0018.tmp C:\\one-tests\\runs\\capture\\inbox\\0018.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
21{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0019.tmp C:\\one-tests\\runs\\capture\\inbox\\0019.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
22{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0020.tmp C:\\one-tests\\runs\\capture\\inbox\\0020.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
23{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0021.tmp C:\\one-tests\\runs\\capture\\inbox\\0021.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
24{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0022.tmp C:\\one-tests\\runs\\capture\\inbox\\0022.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
25{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0023.tmp C:\\one-tests\\runs\\capture\\inbox\\0023.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
26{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0024.tmp C:\\one-tests\\runs\\capture\\inbox\\0024.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
27{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0025.tmp C:\\one-tests\\runs\\capture\\inbox\\0025.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
28{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0026.tmp C:\\one-tests\\runs\\capture\\inbox\\0026.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
corpus/conflict-page/native-restore/a/hierarchy-0019.xml created+2
......@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Section xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="synthetic" ID="{FA5C2511-AB32-007B-27C3-C7F92431FBB5}{1}{B0}" path="\\192.168.77.1\agent\m6-conflict\synthetic.one" lastModifiedTime="2026-09-27T09:56:46.000Z" color="#8AA8E4"><one:Page ID="{D615D697-22F9-436A-A0A4-6F68D809E7AA}{1}{B0}" name="Fictitious: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1"/><one:Page ID="{E269A741-C23A-4D0D-9598-77B0FEECEE98}{1}{B0}" name="Four" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:55:48.000Z" pageLevel="1"/><one:Page ID="{64456FBD-67B9-4F61-88FD-16458A15913B}{1}{B0}" name="One" dateTime="2026-09-27T09:55:47.000Z" lastModifiedTime="2026-09-27T09:55:48.000Z" pageLevel="1"/><one:Page ID="{B3F46589-DF47-43C8-9EA9-0AEDC9806EF7}{1}{B0}" name="Two" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:55:48.000Z" pageLevel="1"/><one:Page ID="{8443300C-6402-4B3D-969D-D6F992F2BF2C}{1}{B0}" name="Target" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:56:46.000Z" pageLevel="1"/><one:Page ID="{F81D581F-A816-4C4B-960F-E41C28A79ED3}{1}{B0}" name="Three" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:55:48.000Z" pageLevel="1"/></one:Section>
corpus/conflict-page/native-restore/a/hierarchy-0025.xml created+2
......@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Section xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="synthetic" ID="{FA5C2511-AB32-007B-27C3-C7F92431FBB5}{1}{B0}" path="\\192.168.77.1\agent\m6-conflict\synthetic.one" lastModifiedTime="2026-09-27T09:57:04.000Z" color="#8AA8E4"><one:Page ID="{D615D697-22F9-436A-A0A4-6F68D809E7AA}{1}{B0}" name="Fictitious: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1"/><one:Page ID="{E269A741-C23A-4D0D-9598-77B0FEECEE98}{1}{B0}" name="Four" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:55:48.000Z" pageLevel="1"/><one:Page ID="{64456FBD-67B9-4F61-88FD-16458A15913B}{1}{B0}" name="One" dateTime="2026-09-27T09:55:47.000Z" lastModifiedTime="2026-09-27T09:55:48.000Z" pageLevel="1"/><one:Page ID="{B3F46589-DF47-43C8-9EA9-0AEDC9806EF7}{1}{B0}" name="Two" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:55:48.000Z" pageLevel="1"/><one:Page ID="{77E5F974-8FC1-41FC-BAC4-EE995A884F9A}{1}{B0}" name="Target" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:56:55.000Z" pageLevel="1"/><one:Page ID="{F81D581F-A816-4C4B-960F-E41C28A79ED3}{1}{B0}" name="Three" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:55:48.000Z" pageLevel="1"/></one:Section>
corpus/conflict-page/native-restore/a/listed.json created+1
......@@ -0,0 +1 @@
1["Fictitious: café, 東京, مرحبا", "Four", "One", "Two", "Target", "Three"]
\ No newline at end of file
corpus/conflict-page/native-restore/a/machine.json created+1
......@@ -0,0 +1 @@
1{"name": "m6-d281b38e", "hostname": "ONE-M6-D281B38E"}
corpus/conflict-page/native-restore/a/teardown.json created+1
......@@ -0,0 +1 @@
1{"absent": true}
corpus/conflict-page/native-restore/b-published/notebook/OneNote_RecycleBin/OneNote_DeletedPages.one created
Binary files /dev/null and b/corpus/conflict-page/native-restore/b-published/notebook/OneNote_RecycleBin/OneNote_DeletedPages.one differ
corpus/conflict-page/native-restore/b-published/notebook/OneNote_RecycleBin/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/native-restore/b-published/notebook/OneNote_RecycleBin/Open Notebook.onetoc2 differ
corpus/conflict-page/native-restore/b-published/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/native-restore/b-published/notebook/Open Notebook.onetoc2 differ
corpus/conflict-page/native-restore/b-published/notebook/synthetic.one created
Binary files /dev/null and b/corpus/conflict-page/native-restore/b-published/notebook/synthetic.one differ
corpus/conflict-page/native-restore/b/b-merged.json created+13
......@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Fictitious: caf\u00e9, \u6771\u4eac, \u0645\u0631\u062d\u0628\u0627 - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/native-restore/b/b-merged.png created
Binary files /dev/null and b/corpus/conflict-page/native-restore/b/b-merged.png differ
corpus/conflict-page/native-restore/b/command-0001.json created+1
......@@ -0,0 +1 @@
1{"action": "snapshot"}
\ No newline at end of file
corpus/conflict-page/native-restore/b/command-0002.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/b/command-0003.json created+1
......@@ -0,0 +1 @@
1{"action": "snapshot"}
\ No newline at end of file
corpus/conflict-page/native-restore/b/command-0004.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/b/command-0005.json created+1
......@@ -0,0 +1 @@
1{"action": "snapshot"}
\ No newline at end of file
corpus/conflict-page/native-restore/b/command-0006.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/b/command-0007.json created+1
......@@ -0,0 +1 @@
1{"action": "snapshot"}
\ No newline at end of file
corpus/conflict-page/native-restore/b/command-0008.json created+1
......@@ -0,0 +1 @@
1{"action": "delete-page", "page": "Target"}
\ No newline at end of file
corpus/conflict-page/native-restore/b/command-0009.json created+1
......@@ -0,0 +1 @@
1{"action": "snapshot"}
\ No newline at end of file
corpus/conflict-page/native-restore/b/command-0010.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/b/command-0011.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/b/command-0012.json created+1
......@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/b/command-0013.json created+1
......@@ -0,0 +1 @@
1{"action": "snapshot"}
\ No newline at end of file
corpus/conflict-page/native-restore/b/command-0014.json created+1
......@@ -0,0 +1 @@
1{"action": "close"}
\ No newline at end of file
corpus/conflict-page/native-restore/b/commands.jsonl created+16
......@@ -0,0 +1,16 @@
1{"command": "mkdir C:\\one-tests\\runs\\capture", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -ExecutionPolicy Bypass -File C:\\one-tests\\network.ps1 -LabMac 52:54:02:7b:1a:4d", "exit": 0, "stdout": "{\"renewalResult\":0,\"releaseResult\":0,\"mac\":\"52:54:02:7b:1a:4d\",\"address\":\"192.168.77.40\"}\r\n", "stderr": "", "error": null}
3{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0001.tmp C:\\one-tests\\runs\\capture\\inbox\\0001.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
4{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0002.tmp C:\\one-tests\\runs\\capture\\inbox\\0002.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
5{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0003.tmp C:\\one-tests\\runs\\capture\\inbox\\0003.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
6{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0004.tmp C:\\one-tests\\runs\\capture\\inbox\\0004.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
7{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0005.tmp C:\\one-tests\\runs\\capture\\inbox\\0005.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
8{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0006.tmp C:\\one-tests\\runs\\capture\\inbox\\0006.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
9{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0007.tmp C:\\one-tests\\runs\\capture\\inbox\\0007.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
10{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0008.tmp C:\\one-tests\\runs\\capture\\inbox\\0008.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
11{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0009.tmp C:\\one-tests\\runs\\capture\\inbox\\0009.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
12{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0010.tmp C:\\one-tests\\runs\\capture\\inbox\\0010.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
13{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0011.tmp C:\\one-tests\\runs\\capture\\inbox\\0011.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
14{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0012.tmp C:\\one-tests\\runs\\capture\\inbox\\0012.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
15{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0013.tmp C:\\one-tests\\runs\\capture\\inbox\\0013.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
16{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0014.tmp C:\\one-tests\\runs\\capture\\inbox\\0014.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
corpus/conflict-page/native-restore/b/hierarchy-0001.xml created+2
......@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Section xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="synthetic" ID="{E9F59793-A58A-0A1E-0F4D-7805A7589855}{1}{B0}" path="\\192.168.77.1\agent\m6-conflict\synthetic.one" lastModifiedTime="2026-09-27T09:56:24.000Z" color="#8AA8E4" areAllPagesAvailable="false"><one:Page ID="{C5BC6415-2C41-490F-882A-D0945B60844A}{1}{B0}" name="Fictitious: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1"/></one:Section>
corpus/conflict-page/native-restore/b/hierarchy-0003.xml created+2
......@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Section xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="synthetic" ID="{E9F59793-A58A-0A1E-0F4D-7805A7589855}{1}{B0}" path="\\192.168.77.1\agent\m6-conflict\synthetic.one" lastModifiedTime="2026-09-27T09:56:36.000Z" color="#8AA8E4" areAllPagesAvailable="false"><one:Page ID="{C5BC6415-2C41-490F-882A-D0945B60844A}{1}{B0}" name="Fictitious: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1"/></one:Section>
corpus/conflict-page/native-restore/b/hierarchy-0005.xml created+2
......@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Section xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="synthetic" ID="{E9F59793-A58A-0A1E-0F4D-7805A7589855}{1}{B0}" path="\\192.168.77.1\agent\m6-conflict\synthetic.one" lastModifiedTime="2026-09-27T09:56:37.000Z" color="#8AA8E4" areAllPagesAvailable="false"><one:Page ID="{C5BC6415-2C41-490F-882A-D0945B60844A}{1}{B0}" name="Fictitious: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1"/><one:Page ID="{77ECDD3F-6901-4504-A073-A9B9097CF2DB}{1}{B0}" name="One" dateTime="2026-09-27T09:55:47.000Z" lastModifiedTime="2026-09-27T09:56:37.000Z" pageLevel="1"/></one:Section>
corpus/conflict-page/native-restore/b/hierarchy-0007.xml created+2
......@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Section xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="synthetic" ID="{E9F59793-A58A-0A1E-0F4D-7805A7589855}{1}{B0}" path="\\192.168.77.1\agent\m6-conflict\synthetic.one" lastModifiedTime="2026-09-27T09:56:39.000Z" color="#8AA8E4"><one:Page ID="{C5BC6415-2C41-490F-882A-D0945B60844A}{1}{B0}" name="Fictitious: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1"/><one:Page ID="{77ECDD3F-6901-4504-A073-A9B9097CF2DB}{1}{B0}" name="One" dateTime="2026-09-27T09:55:47.000Z" lastModifiedTime="2026-09-27T09:56:37.000Z" pageLevel="1"/><one:Page ID="{A05DD70B-D1FF-49AD-B627-B5114AE90D17}{1}{B0}" name="Two" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:56:38.000Z" pageLevel="1"/><one:Page ID="{97EA828E-6ABA-4158-BE13-6905119BDCCC}{1}{B0}" name="Target" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:56:39.000Z" pageLevel="1"/><one:Page ID="{EBB4EA9D-A6AE-462E-BE81-5BE0ABCEFD33}{1}{B0}" name="Three" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:56:39.000Z" pageLevel="1"/><one:Page ID="{F1C015C3-CC82-4768-BD16-C84C7D858D78}{1}{B0}" name="Four" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:56:39.000Z" pageLevel="1"/></one:Section>
corpus/conflict-page/native-restore/b/hierarchy-0009.xml created+2
......@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Section xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="synthetic" ID="{E9F59793-A58A-0A1E-0F4D-7805A7589855}{1}{B0}" path="\\192.168.77.1\agent\m6-conflict\synthetic.one" lastModifiedTime="2026-09-27T09:56:49.000Z" color="#8AA8E4" isUnread="true"><one:Page ID="{C5BC6415-2C41-490F-882A-D0945B60844A}{1}{B0}" name="Fictitious: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1"/><one:Page ID="{77ECDD3F-6901-4504-A073-A9B9097CF2DB}{1}{B0}" name="One" dateTime="2026-09-27T09:55:47.000Z" lastModifiedTime="2026-09-27T09:56:37.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{A05DD70B-D1FF-49AD-B627-B5114AE90D17}{1}{B0}" name="Two" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:56:38.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{EBB4EA9D-A6AE-462E-BE81-5BE0ABCEFD33}{1}{B0}" name="Three" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:56:39.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{F1C015C3-CC82-4768-BD16-C84C7D858D78}{1}{B0}" name="Four" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:56:39.000Z" pageLevel="1" isUnread="true"/></one:Section>
corpus/conflict-page/native-restore/b/hierarchy-0013.xml created+2
......@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Section xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="synthetic" ID="{E9F59793-A58A-0A1E-0F4D-7805A7589855}{1}{B0}" path="\\192.168.77.1\agent\m6-conflict\synthetic.one" lastModifiedTime="2026-09-27T09:57:25.000Z" color="#8AA8E4" isUnread="true"><one:Page ID="{C5BC6415-2C41-490F-882A-D0945B60844A}{1}{B0}" name="Fictitious: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1"/><one:Page ID="{F1C015C3-CC82-4768-BD16-C84C7D858D78}{1}{B0}" name="Four" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:56:39.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{77ECDD3F-6901-4504-A073-A9B9097CF2DB}{1}{B0}" name="One" dateTime="2026-09-27T09:55:47.000Z" lastModifiedTime="2026-09-27T09:56:37.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{A05DD70B-D1FF-49AD-B627-B5114AE90D17}{1}{B0}" name="Two" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:56:38.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{644C4BF6-8179-4B99-924A-5165D9E12C7A}{1}{B0}" name="Target" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:57:25.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{EBB4EA9D-A6AE-462E-BE81-5BE0ABCEFD33}{1}{B0}" name="Three" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:56:39.000Z" pageLevel="1" isUnread="true"/></one:Section>
corpus/conflict-page/native-restore/b/listed.json created+1
......@@ -0,0 +1 @@
1["Fictitious: café, 東京, مرحبا", "Four", "One", "Two", "Target", "Three"]
\ No newline at end of file
corpus/conflict-page/native-restore/b/machine.json created+1
......@@ -0,0 +1 @@
1{"name": "m6-a205cc61", "hostname": "ONE-M6-A205CC61"}
corpus/conflict-page/native-restore/b/teardown.json created+1
......@@ -0,0 +1 @@
1{"absent": true}
corpus/conflict-page/native-restore/clients.json created+4
......@@ -0,0 +1,4 @@
1{
2 "a": "m6-d281b38e",
3 "b": "m6-a205cc61"
4}
\ No newline at end of file
corpus/conflict-page/native-restore/initial/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/native-restore/initial/notebook/Open Notebook.onetoc2 differ
corpus/conflict-page/native-restore/initial/notebook/synthetic.one created
Binary files /dev/null and b/corpus/conflict-page/native-restore/initial/notebook/synthetic.one differ
corpus/conflict-page/native-restore/linux-teardown.json created+1
......@@ -0,0 +1 @@
1{"absent": true}
corpus/conflict-page/native-restore/linux.json created+11
......@@ -0,0 +1,11 @@
1{
2 "cpus": 2,
3 "disk_gb": 16,
4 "hostname": "one-conflict3",
5 "lab_address": "192.168.77.1",
6 "lab_mac": "52:54:04:08:6d:03",
7 "memory_mb": 2048,
8 "samba_port": 14450,
9 "ssh_port": 22000,
10 "wan_mac": "52:54:03:bb:9c:87"
11}
\ No newline at end of file
corpus/conflict-page/native-restore/merged.json created+49
......@@ -0,0 +1,49 @@
1{
2 "order": [
3 "",
4 "Four",
5 "One",
6 "Two",
7 "Target",
8 "Three"
9 ],
10 "pages": [
11 [
12 "",
13 "Fictitious positioned outline.",
14 "Fictitious: café, 東京, مرحبا",
15 "Left cell",
16 "Right cell"
17 ],
18 [
19 "2:55 AM",
20 "Body Four.",
21 "Four",
22 "Sunday, September 27, 2026"
23 ],
24 [
25 "2:55 AM",
26 "Body One.",
27 "One",
28 "Sunday, September 27, 2026"
29 ],
30 [
31 "2:55 AM",
32 "Body Two.",
33 "Sunday, September 27, 2026",
34 "Two"
35 ],
36 [
37 "2:55 AM",
38 "Body Target edited offline.",
39 "Sunday, September 27, 2026",
40 "Target"
41 ],
42 [
43 "2:55 AM",
44 "Body Three.",
45 "Sunday, September 27, 2026",
46 "Three"
47 ]
48 ]
49}
\ No newline at end of file
corpus/conflict-page/native-restore/merged/notebook/OneNote_RecycleBin/OneNote_DeletedPages.one created
Binary files /dev/null and b/corpus/conflict-page/native-restore/merged/notebook/OneNote_RecycleBin/OneNote_DeletedPages.one differ
corpus/conflict-page/native-restore/merged/notebook/OneNote_RecycleBin/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/native-restore/merged/notebook/OneNote_RecycleBin/Open Notebook.onetoc2 differ
corpus/conflict-page/native-restore/merged/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/native-restore/merged/notebook/Open Notebook.onetoc2 differ
corpus/conflict-page/native-restore/merged/notebook/synthetic.one created
Binary files /dev/null and b/corpus/conflict-page/native-restore/merged/notebook/synthetic.one differ
corpus/conflict-page/native-restore/native_conflict.py created+362
......@@ -0,0 +1,362 @@
1#!/usr/bin/env python3
2"""OneNote 2010 conflict pages: how they arise, what they store and how OneNote shows them.
3
4`sync LINUX OUTPUT` runs two disposable OneNote clients on one Samba notebook: client A edits
5two paragraphs offline while client B edits one of them and a table cell online; A then
6reconnects, which makes OneNote write a conflict page. The server file is kept before and
7after, A and B are screenshotted with the information bar, the page list shown, the conflict
8page open and its menu open, and A then deletes the conflict page through that menu.
9
10`pages LINUX OUTPUT` runs the same two clients on page-list changes: client A, offline, moves
11a page that client B also moves, moves another page, and edits a page B deletes; A then
12reconnects. The server's page order and pages, with the recycle bin, are kept at each step.
13
14`restore LINUX OUTPUT` runs them on a page in the middle of the list: client A, offline, edits
15it and moves a page B leaves alone while B deletes it; A then reconnects.
16
17`cold NOTEBOOK OUTPUT [--delete]` cold-opens a notebook holding a conflict page in a fresh
18clone and screenshots the same views; `--delete` deletes the conflict page through the menu
19and captures the notebook OneNote leaves.
20"""
21import argparse
22from contextlib import ExitStack
23from concurrent.futures import ThreadPoolExecutor
24import base64
25import json
26from pathlib import Path
27import shutil
28import subprocess
29import sys
30import time
31import xml.etree.ElementTree as ET
32
33sys.path.insert(0, str(Path(__file__).resolve().parent))
34sys.path.insert(0, str(Path(__file__).resolve().parent / 'w7'))
35import native_runner as runner
36from native_runner import ROOT, clone, command, windows, vm
37import linux_vm
38
39FIXTURE = ROOT / 'corpus/native-ink/cold-ui-ink/notebook'
40SHARE = 'm6-conflict'
41ORIGINAL = 'Fictitious: café, 東京, مرحبا'
42OTHER_OUTLINE = 'Fictitious positioned outline.'
43CELL = 'Left cell'
44
45# Coordinates on OneNote 2010 maximized at 800x600, from the stage-5, m8 and sync-01 captures:
46# the information bar's text, and the first conflict page listed under its page.
47BAR = (332, 100)
48FIRST_CONFLICT = (725, 142)
49# Delete Conflict Page, first in the menu the bar opens on a conflict page.
50DELETE = (420, 113)
51
52
53def shot(name, output, label, script):
54 """Runs an AutoHotkey step on the clone, keeping its screenshot as `label.png`."""
55 (output / f'{label}.ahk').write_text(script)
56 result = windows.do_exec(script, target=name, timeout_ms=120000, shot_delay_ms=1500)
57 screenshot = result.pop('png_b64', None)
58 if screenshot:
59 (output / f'{label}.png').write_bytes(base64.b64decode(screenshot))
60 (output / f'{label}.json').write_text(json.dumps(result, indent=2))
61 if result.get('error') or result.get('exit') != 0:
62 raise RuntimeError(f'{label}: {result}')
63 return result
64
65
66def click(x, y):
67 return ('WinActivate("ahk_exe ONENOTE.EXE")\nCoordMode("Mouse", "Screen")\n'
68 f'Click({x}, {y})\nSleep(1200)\n')
69
70
71# Reports any dialog OneNote raised, so a confirmation shows in the step's record.
72DIALOG = ('if WinExist("ahk_class #32770 ahk_exe ONENOTE.EXE") {\n'
73 ' FileAppend("dialog: " WinGetTitle() " | " WinGetText() "`n", "*", "UTF-8")\n'
74 '}\n')
75
76
77def conflict_views(name, output, page, prefix, delete=False):
78 """Screenshots a page with a conflict page: the bar, the list shown, the conflict page and
79 its menu; `delete` then chooses Delete Conflict Page."""
80 shot(name, output, f'{prefix}-main', runner.navigation_script(page) + 'Sleep(2000)\n')
81 shot(name, output, f'{prefix}-shown', click(*BAR))
82 shot(name, output, f'{prefix}-conflict', click(*FIRST_CONFLICT))
83 shot(name, output, f'{prefix}-menu', click(*BAR))
84 if delete:
85 shot(name, output, f'{prefix}-delete', click(*DELETE) + DIALOG)
86 else:
87 shot(name, output, f'{prefix}-closed', 'WinActivate("ahk_exe ONENOTE.EXE")\nSend("{Esc}")\nSleep(800)\n')
88
89
90def cold(notebook, output, delete):
91 def interaction(name, output):
92 page = ET.parse(sorted((output / 'read').glob('page-*.xml'))[0]).getroot().attrib['ID']
93 conflict_views(name, output, page, 'cold', delete)
94 if delete:
95 # OneNote writes the deletion in the background; give it time before the re-read.
96 time.sleep(20)
97 runner.capture(notebook, output, collect_notebook=True, interaction=interaction)
98
99
100class Lab:
101 """Two OneNote clients on one Samba notebook: `action` runs a controller command,
102 `state` copies the share and exports its section, `settle` syncs until a check holds."""
103
104
105def page_list(model):
106 """The section's pages in order, each as its sorted texts."""
107 from document_model import ordered_pages, walk
108 return [sorted(node['kind']['text'] for _, node in walk(page, oid) if node['kind']['type'] == 'RichText')
109 for _, _, page, oid in ordered_pages(model)]
110
111
112def edits(lab):
113 from native_collaboration import reachable_page_text
114 a, b, output, action, settle = lab.a, lab.b, lab.output, lab.action, lab.settle
115 state = lambda label: reachable_page_text(lab.state(label))
116 texts = lambda state: {text for values in state.values() for text in values}
117 state('initial')
118 vm.qmp(a['name'], 'set_link', {'name': 'lab', 'up': False})
119 try:
120 action(a, 'edit', expected=ORIGINAL, text='Client A conflicting edit.')
121 action(a, 'edit', expected=OTHER_OUTLINE, text='Client A disjoint edit.')
122 action(a, 'snapshot')
123 action(b, 'edit', expected=ORIGINAL, text='Client B conflicting edit.')
124 action(b, 'edit', expected=CELL, text='Client B disjoint cell.')
125 settle(b, lambda label: {'Client B conflicting edit.', 'Client B disjoint cell.'} <= texts(state(label)), 'b-published')
126 finally:
127 vm.qmp(a['name'], 'set_link', {'name': 'lab', 'up': True})
128 settle(a, lambda label: len(found := state(label)) > 1 and 'Client A disjoint edit.' in texts(found), 'conflict')
129 (output / 'conflict.json').write_text(json.dumps(state('conflict'), indent=2, ensure_ascii=False))
130 page = action(a, 'snapshot')[0].attrib['ID']
131 conflict_views(a['name'], output / 'a', page, 'a')
132 action(b, 'sync')
133 time.sleep(5)
134 page_b = action(b, 'snapshot')[0].attrib['ID']
135 conflict_views(b['name'], output / 'b', page_b, 'b')
136 conflict_views(a['name'], output / 'a', page, 'a-again', delete=True)
137 settle(a, lambda label: len(state(label)) == 1, 'deleted')
138 action(b, 'sync')
139 time.sleep(5)
140 shot(b['name'], output / 'b', 'b-after-delete', runner.navigation_script(page_b) + 'Sleep(2000)\n')
141
142
143PAGES = ['One', 'Two', 'Three', 'Four', 'Target']
144
145
146def titled(model, titles):
147 """The section's pages in order, each named by the title among `titles` it holds."""
148 return [next((text for text in texts if text in titles), '') for texts in page_list(model)]
149
150
151def offline(lab, titles, a_actions, b_actions):
152 """Creates the titled pages; A runs its actions offline while B runs and publishes its
153 own; A then reconnects. Keeps the merged order and pages, and what each client lists."""
154 a, b, output, action = lab.a, lab.b, lab.output, lab.action
155 action(a, 'pages', titles=titles)
156 lab.settle(a, lambda label: 'Body Target.' in str(page_list(lab.state(label))), 'initial')
157 deadline = time.monotonic() + 300
158 while not any(node.get('name') == 'Target' for node in action(b, 'snapshot')):
159 action(b, 'sync')
160 if time.monotonic() > deadline: raise TimeoutError('B never listed the new pages')
161 vm.qmp(a['name'], 'set_link', {'name': 'lab', 'up': False})
162 try:
163 for kind, parameters in a_actions:
164 action(a, kind, **parameters)
165 action(a, 'snapshot')
166 for kind, parameters in b_actions:
167 action(b, kind, **parameters)
168 action(b, 'snapshot')
169 lab.settle(b, lambda label: 'Body Target.' not in str(page_list(lab.state(label))), 'b-published')
170 finally:
171 vm.qmp(a['name'], 'set_link', {'name': 'lab', 'up': True})
172 # A's changes reach the server when its sync writes; wait until the file stops changing.
173 previous, steady = None, 0
174 deadline = time.monotonic() + 300
175 while steady < 3 and time.monotonic() < deadline:
176 action(a, 'sync')
177 time.sleep(10)
178 current = page_list(lab.state('merged'))
179 steady = steady + 1 if current == previous else 0
180 previous = current
181 (output / 'merged.json').write_text(json.dumps({'order': titled(lab.state('merged'), titles), 'pages': previous}, indent=2, ensure_ascii=False))
182 for client, label in ((a, 'a'), (b, 'b')):
183 action(client, 'sync')
184 time.sleep(5)
185 listed = action(client, 'snapshot')
186 (output / label / 'listed.json').write_text(json.dumps([node.get('name') for node in listed], ensure_ascii=False))
187 if listed:
188 shot(client['name'], output / label, f'{label}-merged', runner.navigation_script(listed[0].get('ID')) + 'Sleep(2000)\n')
189
190
191def pages(lab):
192 """Competing page moves, and an offline edit of a page the other client deletes."""
193 offline(lab, PAGES, [
194 ('move', {'page': 'Four', 'before': 'One'}),
195 ('move', {'page': 'Two', 'before': None}),
196 ('edit', {'expected': 'Body Target.', 'text': 'Body Target edited offline.'}),
197 ], [
198 ('move', {'page': 'Four', 'before': None}),
199 ('move', {'page': 'Three', 'before': 'One'}),
200 ('delete-page', {'page': 'Target'}),
201 ])
202
203
204def restore(lab):
205 """An offline edit of a page in the middle of the list the other client deletes, with an
206 offline move of a page the other client leaves alone."""
207 offline(lab, ['One', 'Two', 'Target', 'Three', 'Four'], [
208 ('edit', {'expected': 'Body Target.', 'text': 'Body Target edited offline.'}),
209 ('move', {'page': 'Four', 'before': 'One'}),
210 ], [
211 ('delete-page', {'page': 'Target'}),
212 ])
213
214
215def sync(server, output, scenario):
216 from document_model import EXPORTER
217
218 if linux_vm.instance_path(server).exists():
219 raise ValueError('Choose a new Linux VM name; existing machines are not owned by this run.')
220 output = output.resolve()
221 output.mkdir(parents=True, exist_ok=False)
222 scripts = output / 'scripts'
223 scripts.mkdir()
224 for name in ('cold.ps1', 'collaborate.ps1', 'network.ps1', 'text.ps1', 'stress.ps1'):
225 shutil.copyfile(ROOT / 'tools/native' / name, scripts / name)
226 shutil.copyfile(Path(__file__), output / 'native_conflict.py')
227
228 def ssh(text, timeout=90):
229 result = linux_vm.run_ssh(server, text, timeout=timeout)
230 with (output / 'server.jsonl').open('a') as log:
231 log.write(json.dumps({'command': text, 'exit': result.returncode, 'stdout': result.stdout, 'stderr': result.stderr}) + '\n')
232 result.check_returncode()
233 return result.stdout
234
235 def server_state(label):
236 """Copies the server's notebook, recycle bin included, until its section parses,
237 keeping it as `label/notebook`; returns the section's exported model."""
238 deadline = time.monotonic() + 120
239 while True:
240 target = output / label
241 shutil.rmtree(target, ignore_errors=True)
242 (target / 'notebook').mkdir(parents=True)
243 archive = subprocess.run(linux_vm.ssh_argv(server, f"tar cf - -C '/srv/agent/{SHARE}' ."), capture_output=True, check=True, timeout=60).stdout
244 subprocess.run(['tar', 'xf', '-', '-C', target / 'notebook'], input=archive, check=True)
245 result = subprocess.run([EXPORTER, target / 'notebook/synthetic.one', target / 'model'], capture_output=True, text=True)
246 if result.returncode == 0:
247 return json.loads((target / 'model/document.json').read_text())
248 if time.monotonic() > deadline:
249 raise RuntimeError(f'{label}: {result.stderr}')
250 time.sleep(1)
251
252 linux_vm.create_instance(server)
253 try:
254 linux_vm.launch(server)
255 linux_vm.wait_instance(server, 600)
256 (output / 'linux.json').write_text(json.dumps(linux_vm.load_instance(server), indent=2))
257 ssh(f'mkdir /srv/agent/{SHARE}')
258 for name in ('synthetic.one', 'Open Notebook.onetoc2'):
259 with (FIXTURE / name).open('rb') as stream:
260 subprocess.run(linux_vm.ssh_argv(server, f"cat > '/srv/agent/{SHARE}/{name}'"), stdin=stream, check=True)
261 ssh(f'chmod u+w /srv/agent/{SHARE}/*')
262 with ExitStack() as stack:
263 labels = ['a', 'b']
264 for label in labels:
265 (output / label).mkdir()
266
267 def start(label):
268 manager = clone(output / label)
269 name = manager.__enter__()
270 return label, name, manager
271 with ThreadPoolExecutor(max_workers=2) as pool:
272 started = list(pool.map(start, labels))
273 for _, _, manager in started:
274 stack.push(manager)
275 clients = {}
276 for label, name, _ in started:
277 folder = output / label
278 for local in scripts.iterdir():
279 remote = 'cold-current.ps1' if local.name == 'cold.ps1' else local.name
280 result = windows.do_put(local, 'C:\\one-tests\\' + remote, name)
281 if result.get('error'): raise RuntimeError(result['error'])
282 command(name, 'mkdir C:\\one-tests\\runs\\capture', folder)
283 command(name, 'powershell -NoProfile -ExecutionPolicy Bypass -File C:\\one-tests\\network.ps1 -LabMac ' + vm.lab_mac(name), folder)
284 result = windows.do_spawn('cmd /c powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\collaborate.ps1 -Root C:\\one-tests\\runs\\capture -SharedPath \\\\192.168.77.1\\agent\\' + SHARE + ' -CloneHost ONE-' + name.upper() + ' > C:\\one-tests\\runs\\capture\\controller.log 2>&1', name)
285 if result.get('error'): raise RuntimeError(result['error'])
286 deadline = time.monotonic() + 180
287 while windows.do_cmd('if exist C:\\one-tests\\runs\\capture\\ready (echo ready)', target=name).get('stdout', '').find('ready') < 0:
288 if time.monotonic() > deadline: raise RuntimeError('The collaboration controller did not open the notebook')
289 time.sleep(1)
290 clients[label] = {'name': name, 'folder': folder, 'sequence': 0}
291 (output / 'clients.json').write_text(json.dumps({label: client['name'] for label, client in clients.items()}, indent=2))
292
293 def action(client, kind, **parameters):
294 client['sequence'] += 1
295 sequence = client['sequence']
296 local = client['folder'] / f'command-{sequence:04}.json'
297 local.write_text(json.dumps({'action': kind, **parameters}, ensure_ascii=False))
298 inbox = f'C:\\one-tests\\runs\\capture\\inbox\\{sequence:04}'
299 result = windows.do_put(local, inbox + '.tmp', client['name'])
300 if result.get('error'): raise RuntimeError(result['error'])
301 command(client['name'], f'move {inbox}.tmp {inbox}.json', client['folder'])
302 remote = f'C:\\one-tests\\runs\\capture\\outbox\\{sequence}'
303 deadline = time.monotonic() + 600
304 while True:
305 result = windows.do_cmd(f'if exist {remote}\\error (type {remote}\\error & exit /b 1) else (if exist {remote}\\done (echo complete))', target=client['name'])
306 if result.get('exit') != 0: raise RuntimeError(str(result))
307 if 'complete' in result.get('stdout', ''): break
308 if time.monotonic() > deadline: raise TimeoutError(f'{kind} did not complete')
309 time.sleep(.5)
310 if kind != 'snapshot':
311 return None
312 hierarchy = client['folder'] / f'hierarchy-{sequence:04}.xml'
313 windows.do_get(remote + '\\hierarchy.xml', hierarchy, client['name'])
314 pages = [node for node in ET.fromstring(hierarchy.read_text(encoding='utf-8-sig').strip()).iter() if node.tag.endswith('}Page')]
315 for i in range(len(pages)):
316 windows.do_get(remote + f'\\page-{i}.xml', client['folder'] / f'snapshot-{sequence:04}-{i}.xml', client['name'])
317 return pages
318
319 def settle(client, check, label):
320 deadline = time.monotonic() + 300
321 while True:
322 action(client, 'sync')
323 if check(label): return
324 if time.monotonic() > deadline: raise TimeoutError(label)
325 time.sleep(2)
326
327 lab = Lab()
328 lab.a, lab.b, lab.output = clients['a'], clients['b'], output
329 lab.action, lab.state, lab.settle = action, server_state, settle
330 scenario(lab)
331
332 for client in clients.values():
333 try:
334 action(client, 'close')
335 except Exception as error:
336 (client['folder'] / 'close-failure.txt').write_text(str(error))
337 server_state('final')
338 finally:
339 if linux_vm.running(server):
340 linux_vm.shutdown(server, 120)
341 linux_vm.delete_instance(server)
342 (output / 'linux-teardown.json').write_text(json.dumps({'absent': not linux_vm.instance_path(server).exists()}) + '\n')
343
344
345if __name__ == '__main__':
346 parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
347 commands = parser.add_subparsers(dest='mode', required=True)
348 scenarios = {'sync': edits, 'pages': pages, 'restore': restore}
349 for mode in scenarios:
350 lab = commands.add_parser(mode)
351 lab.add_argument('linux')
352 lab.add_argument('output', type=Path)
353 reopen = commands.add_parser('cold')
354 reopen.add_argument('notebook', type=Path)
355 reopen.add_argument('output', type=Path)
356 reopen.add_argument('--delete', action='store_true')
357 args = parser.parse_args()
358 if args.mode in scenarios:
359 sync(args.linux, args.output, scenarios[args.mode])
360 else:
361 cold(args.notebook, args.output, args.delete)
362 print('Captured', args.output)
corpus/conflict-page/native-restore/scripts/cold.ps1 created+27
......@@ -0,0 +1,27 @@
1param([Parameter(Mandatory=$true)][string]$Root, [string]$CloneHost = '')
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
5if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
6 throw 'Choose a run directly below C:\one-tests\runs.'
7}
8if (Get-Process ONENOTE -ErrorAction SilentlyContinue) { throw 'Close OneNote before resetting its test cache.' }
9$key = 'HKCU:\Software\Microsoft\Office\14.0\OneNote'
10if ($CloneHost) {
11 if ($CloneHost -notmatch '^ONE-[A-Z0-9-]+$' -or [Environment]::MachineName -ne $CloneHost) {
12 throw 'The disposable clone hostname does not match this machine.'
13 }
14 New-Item "$key\Options\Paths" -Force | Out-Null
15 New-ItemProperty "$key\Options\Paths" -Name UnfiledNotesSection -PropertyType ExpandString -Value 'C:\one-tests\Loose.one' -Force | Out-Null
16} elseif ((Get-ItemProperty "$key\Options\Paths").UnfiledNotesSection -ne 'C:\one-tests\Loose.one' -or
17 -not (Test-Path 'C:\one-tests\profile-original-cache')) {
18 throw 'Park the personal OneNote profile before resetting the test cache.'
19}
20$cache = Join-Path $env:LOCALAPPDATA 'Microsoft\OneNote\14.0'
21$parked = Join-Path 'C:\one-tests\caches' ([IO.Path]::GetFileName($root))
22if (Test-Path $parked) { throw 'Choose a new run; its parked cache already exists.' }
23New-Item -ItemType Directory -Path 'C:\one-tests\caches' -Force | Out-Null
24if (Test-Path $cache) { Move-Item -LiteralPath $cache -Destination $parked }
25if (Test-Path "$key\OpenNotebooks") { Remove-Item "$key\OpenNotebooks" -Recurse }
26New-Item "$key\OpenNotebooks" | Out-Null
27New-ItemProperty "$key\OpenNotebooks" -Name '1' -PropertyType String -Value "$root\notebook" | Out-Null
corpus/conflict-page/native-restore/scripts/collaborate.ps1 created+145
......@@ -0,0 +1,145 @@
1param([Parameter(Mandatory=$true)][string]$Root,
2 [Parameter(Mandatory=$true)][string]$SharedPath,
3 [Parameter(Mandatory=$true)][string]$CloneHost,
4 [switch]$ResumeCache,
5 [int]$StartSequence = 1)
6function Keep-Container($page, $target) {
7 $container = $target
8 while ($container.ParentNode -ne $page.DocumentElement) { $container = $container.ParentNode }
9 foreach ($sibling in @($page.DocumentElement.ChildNodes)) {
10 if ($sibling -ne $container -and $sibling.LocalName -in @('Outline', 'Title', 'Image', 'InkDrawing')) {
11 [void]$page.DocumentElement.RemoveChild($sibling)
12 }
13 }
14}
15Set-StrictMode -Version Latest
16$ErrorActionPreference = 'Stop'
17. "$PSScriptRoot\text.ps1"
18if ($SharedPath -notmatch '^\\\\192\.168\.77\.1\\agent\\m6-[a-z0-9-]+$') { throw 'Choose a disposable Linux lab notebook.' }
19if ($env:COMPUTERNAME -ne $CloneHost) { throw 'The collaboration controller requires its owned clone.' }
20if (-not $ResumeCache) { & "$PSScriptRoot\cold-current.ps1" -Root $Root -CloneHost $CloneHost }
21$shared = $SharedPath
22if (-not (Test-Path "$shared\synthetic.one")) { throw 'Create the shared fixture first.' }
23New-Item -ItemType Directory "$Root\inbox", "$Root\outbox" -Force | Out-Null
24$user = 'HKCU:\Software\Microsoft\Office\Common\UserInfo'
25New-Item $user -Force | Out-Null
26New-ItemProperty $user -Name UserName -Value $CloneHost -PropertyType String -Force | Out-Null
27New-ItemProperty $user -Name UserInitials -Value $CloneHost.Substring($CloneHost.Length - 3) -PropertyType String -Force | Out-Null
28$app = New-Object -ComObject OneNote.Application
29$notebook = ''
30try {
31 $app.OpenHierarchy($shared, '', [ref]$notebook, 0)
32 $section = ''
33 $app.OpenHierarchy('synthetic.one', $notebook, [ref]$section, 0)
34 [IO.File]::WriteAllText("$Root\ready", 'ready')
35 $deadline = [DateTime]::UtcNow.AddMinutes(30)
36 $sequence = $StartSequence
37 while ([DateTime]::UtcNow -lt $deadline) {
38 $name = '{0:d4}.json' -f $sequence
39 $input = "$Root\inbox\$name"
40 if (-not (Test-Path $input)) { Start-Sleep -Milliseconds 100; continue }
41 $command = Get-Content -Raw -Encoding UTF8 $input | ConvertFrom-Json
42 $output = "$Root\outbox\$sequence"
43 New-Item -ItemType Directory $output | Out-Null
44 try {
45 if ($command.action -in @('prepare-stress', 'stress')) {
46 . "$PSScriptRoot\stress.ps1"
47 Invoke-Stress $app $section $command $output $shared
48 }
49 elseif ($command.action -eq 'sync') { $app.SyncHierarchy($notebook) }
50 elseif ($command.action -in @('pages', 'move', 'delete-page')) {
51 $namespace = 'http://schemas.microsoft.com/office/onenote/2010/onenote'
52 $hierarchy = ''
53 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
54 [xml]$tree = $hierarchy
55 $listed = @($tree.SelectNodes('//*[local-name()="Page"]'))
56 $named = { param($title) @($listed | Where-Object { $_.GetAttribute('name') -eq $title }) }
57 if ($command.action -eq 'pages') {
58 # New titled pages with one body paragraph each, appended in order.
59 foreach ($title in $command.titles) {
60 $page = ''
61 $app.CreateNewPage($section, [ref]$page, 2)
62 $body = [Security.SecurityElement]::Escape('Body ' + $title + '.')
63 $name = [Security.SecurityElement]::Escape($title)
64 $xml = '<one:Page xmlns:one="' + $namespace + '" ID="' + $page + '"><one:Title><one:OE><one:T>' + $name + '</one:T></one:OE></one:Title><one:Outline><one:OEChildren><one:OE><one:T>' + $body + '</one:T></one:OE></one:OEChildren></one:Outline></one:Page>'
65 $app.UpdatePageContent($xml, [DateTime]::MinValue, 1, $false)
66 }
67 } elseif ($command.action -eq 'move') {
68 # Moves the page titled `page` before the one titled `before`, or last.
69 $moved = @(& $named $command.page)
70 if ($moved.Count -ne 1) { throw 'Expected one page to move.' }
71 $order = [Collections.ArrayList]@($listed | Where-Object { $_ -ne $moved[0] })
72 $at = $order.Count
73 if ($command.before) {
74 $anchor = @(& $named $command.before)
75 if ($anchor.Count -ne 1) { throw 'Expected one anchor page.' }
76 $at = $order.IndexOf($anchor[0])
77 }
78 $order.Insert($at, $moved[0])
79 $xml = '<one:Section xmlns:one="' + $namespace + '" ID="' + $section + '">'
80 foreach ($node in $order) {
81 $xml += '<one:Page ID="' + $node.GetAttribute('ID') + '" pageLevel="' + $node.GetAttribute('pageLevel') + '"/>'
82 }
83 $app.UpdateHierarchy($xml + '</one:Section>', 1)
84 } else {
85 $doomed = @(& $named $command.page)
86 if ($doomed.Count -ne 1) { throw 'Expected one page to delete.' }
87 $app.DeleteHierarchy($doomed[0].GetAttribute('ID'), [DateTime]::MinValue, $false)
88 }
89 }
90 elseif ($command.action -eq 'kill-process') {
91 Get-Process ONENOTE | Stop-Process -Force
92 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
93 $app = New-Object -ComObject OneNote.Application
94 $app.OpenHierarchy($shared, '', [ref]$notebook, 0)
95 $app.OpenHierarchy('synthetic.one', $notebook, [ref]$section, 0)
96 }
97 elseif ($command.action -eq 'edit' -or $command.action -eq 'snapshot') {
98 $hierarchy = ''
99 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
100 [IO.File]::WriteAllText("$output\hierarchy.xml", $hierarchy, [Text.Encoding]::UTF8)
101 [xml]$tree = $hierarchy
102 $matches = 0
103 $editedPage = $null
104 $i = 0
105 foreach ($node in $tree.SelectNodes('//*[local-name()="Page"]')) {
106 $content = ''
107 $app.GetPageContent($node.GetAttribute('ID'), [ref]$content, 1, 1)
108 [IO.File]::WriteAllText("$output\page-$i.xml", $content, [Text.Encoding]::UTF8)
109 [xml]$page = $content
110 if ($command.action -eq 'edit') {
111 $targets = @($page.SelectNodes('//*[local-name()="T"]') | Where-Object {
112 (Get-ParagraphText $_.InnerText) -eq $command.expected
113 })
114 if ($targets.Count -gt 1) { throw 'Expected one matching text object.' }
115 if ($targets.Count -eq 1) {
116 $matches++
117 if ($matches -gt 1) { throw 'The text occurs on multiple pages.' }
118 $targets[0].InnerText = [Security.SecurityElement]::Escape($command.text)
119 Keep-Container $page $targets[0]
120 $editedPage = $page
121 }
122 }
123 $i++
124 }
125 if ($command.action -eq 'edit' -and $matches -ne 1) { throw 'Expected one matching text object.' }
126 if ($editedPage) { $app.UpdatePageContent($editedPage.OuterXml, [DateTime]::MinValue, 1, $false) }
127 } elseif ($command.action -ne 'close') { throw 'Unknown collaboration command.' }
128 [IO.File]::WriteAllText("$output\done", 'ok')
129 } catch {
130 [IO.File]::WriteAllText("$output\error", $_.Exception.ToString())
131 }
132 if ($command.action -eq 'close') { break }
133 $sequence++
134 }
135} finally {
136 try { if ($notebook) { $app.CloseNotebook($notebook, $false) } }
137 finally {
138 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
139 $app = $null
140 [GC]::Collect()
141 [GC]::WaitForPendingFinalizers()
142 Get-Process ONENOTE -ErrorAction SilentlyContinue | Wait-Process -Timeout 10
143 [IO.File]::WriteAllText("$Root\closed", 'closed')
144 }
145}
corpus/conflict-page/native-restore/scripts/network.ps1 created+20
......@@ -0,0 +1,20 @@
1param([Parameter(Mandatory=$true)][string]$LabMac)
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$adapter = @(Get-WmiObject Win32_NetworkAdapterConfiguration | Where-Object { $_.MACAddress -eq $LabMac })
5if ($adapter.Count -ne 1) { throw 'The clone lab adapter is ambiguous.' }
6$enabled = $adapter[0].EnableDHCP()
7if ($enabled.ReturnValue -notin @(0, 1)) {
8 throw ('Unable to enable DHCP on the clone lab adapter; result ' + $enabled.ReturnValue)
9}
10$release = $adapter[0].ReleaseDHCPLease()
11$result = $adapter[0].RenewDHCPLease()
12$deadline = [DateTime]::UtcNow.AddSeconds(60)
13do {
14 $adapter = @(Get-WmiObject Win32_NetworkAdapterConfiguration | Where-Object { $_.MACAddress -eq $LabMac })
15 $addresses = @($adapter[0].IPAddress | Where-Object { $_ -match '^192\.168\.77\.\d+$' -and $_ -ne '192.168.77.1' })
16 if ($addresses.Count -eq 1) { break }
17 Start-Sleep -Milliseconds 250
18} while ([DateTime]::UtcNow -lt $deadline)
19if ($addresses.Count -ne 1) { throw ('The clone did not receive a lab IPv4 address; renewal result ' + $result.ReturnValue + '; adapter ' + ($adapter[0] | Select-Object MACAddress,DHCPEnabled,IPEnabled,IPAddress | ConvertTo-Json -Compress)) }
20@{mac=$LabMac;address=$addresses[0];releaseResult=$release.ReturnValue;renewalResult=$result.ReturnValue} | ConvertTo-Json -Compress
corpus/conflict-page/native-restore/scripts/stress.ps1 created+83
......@@ -0,0 +1,83 @@
1function Invoke-Stress($app, $section, $command, $output, $shared) {
2 $hierarchy = ''
3 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
4 [IO.File]::WriteAllText("$output\hierarchy.xml", $hierarchy, [Text.Encoding]::UTF8)
5 [xml]$tree = $hierarchy
6 $pages = @($tree.SelectNodes('//*[local-name()="Page"]'))
7 if ($pages.Count -ne 1) { throw 'Expected one stress-test page.' }
8 $pageId = $pages[0].GetAttribute('ID')
9 if ($command.action -eq 'prepare-stress') {
10 for ($i = 0; $i -lt $command.clients; $i++) {
11 $xml = '<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="' + $pageId + '"><one:Outline><one:Position x="48" y="' + (150 + 75 * $i) + '" z="' + ($i + 1) + '"/><one:OEChildren><one:OE><one:T><![CDATA[Native ' + $i + ':]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>'
12 $app.UpdatePageContent($xml, [DateTime]::MinValue, 1, $false)
13 }
14 return
15 }
16 $random = New-Object Random([int]$command.seed)
17 $events = New-Object IO.StreamWriter("$output\events.jsonl", $false, [Text.Encoding]::UTF8)
18 $events.AutoFlush = $true
19 try {
20 [IO.File]::WriteAllText("$output\ready", 'ready')
21 $deadline = [DateTime]::UtcNow.AddMinutes(10)
22 while (-not (Test-Path "$shared\stress-start")) {
23 if ([DateTime]::UtcNow -gt $deadline) { throw 'Stress start timed out.' }
24 Start-Sleep -Milliseconds 20
25 }
26 for ($i = 0; $i -lt $command.operations; $i++) {
27 if (($command.PSObject.Properties.Name -contains 'maintenance') -and $command.maintenance -and $i -eq [Math]::Floor($command.operations / 2)) {
28 [IO.File]::WriteAllText("$shared\maintenance-paused-n$($command.actor)", 'paused')
29 while (-not (Test-Path "$shared\maintenance-resume")) {
30 if ([DateTime]::UtcNow -gt $deadline) { throw 'Maintenance pause timed out.' }
31 Start-Sleep -Milliseconds 100
32 }
33 }
34 Start-Sleep -Milliseconds $random.Next(10, 100)
35 $started = [DateTime]::UtcNow.Ticks
36 $content = ''
37 $app.GetPageContent($pageId, [ref]$content, 1, 1)
38 [xml]$page = $content
39 $matches = @($page.SelectNodes('//*[local-name()="T"]') | Where-Object {
40 (Get-ParagraphText $_.InnerText).StartsWith($command.prefix)
41 })
42 if ($matches.Count -ne 1) { throw 'Expected one stress-test paragraph.' }
43 $before = Get-ParagraphText $matches[0].InnerText
44 $token = ' [n' + $command.actor + ':' + $i + ']'
45 $start = $end = $before.Length
46 $replacement = $token
47 $bold = $italic = $false
48 if ($command.edit) {
49 $boundaries = New-Object 'Collections.Generic.List[int]'
50 for ($at = $command.prefix.Length; $at -le $before.Length; $at++) {
51 $boundaries.Add($at)
52 if ($at -lt $before.Length -and [char]::IsHighSurrogate($before[$at])) { $at++ }
53 }
54 $first = $boundaries[$random.Next($boundaries.Count)]
55 $second = $boundaries[$random.Next($boundaries.Count)]
56 $start = [Math]::Min($first, $second)
57 $end = [Math]::Max($first, $second)
58 $replacement = ' caf' + [char]0xe9 + ' ' + [char]::ConvertFromUtf32(0x1f980) + $token
59 $bold = $random.Next(2) -eq 1
60 $italic = $random.Next(2) -eq 1
61 }
62 $after = $before.Substring(0, $start) + $replacement + $before.Substring($end)
63 $html = [Security.SecurityElement]::Escape($after)
64 if ($command.edit) {
65 $weight = if ($bold) { 'bold' } else { 'normal' }
66 $style = if ($italic) { 'italic' } else { 'normal' }
67 $html = '<span style="font-weight:' + $weight + ';font-style:' + $style + '">' + $html + '</span>'
68 }
69 $matches[0].InnerText = $html
70 Keep-Container $page $matches[0]
71 $updateStarted = [DateTime]::UtcNow.Ticks
72 $app.UpdatePageContent($page.OuterXml, [DateTime]::MinValue, 1, $false)
73 $updated = [DateTime]::UtcNow.Ticks
74 $events.WriteLine((@{ operation=$i; before=$before; token=$token; started_ticks=$started;
75 range=@($start, $end); replacement=$replacement; bold=$bold; italic=$italic;
76 update_started_ticks=$updateStarted; updated_ticks=$updated } | ConvertTo-Json -Compress))
77 if ($i -eq 0) { [IO.File]::WriteAllText("$output\editing", 'editing') }
78 if ($command.sync_every -gt 0 -and (($i + 1) % $command.sync_every) -eq 0) {
79 $app.SyncHierarchy($section)
80 }
81 }
82 } finally { $events.Dispose() }
83}
corpus/conflict-page/native-restore/scripts/text.ps1 created+13
......@@ -0,0 +1,13 @@
1function Get-ParagraphText([string]$html) {
2 $plain = [regex]::Replace($html, '<[^>]*>', '')
3 $plain = [regex]::Replace($plain, '&#(?:x([0-9a-fA-F]+)|([0-9]+));', {
4 param($match)
5 $value = if ($match.Groups[1].Success) {
6 [Convert]::ToInt32($match.Groups[1].Value, 16)
7 } else { [int]$match.Groups[2].Value }
8 # The Win7 framework leaves supplementary numeric entities undecoded.
9 if ($value -gt 0xffff -and $value -le 0x10ffff) { [char]::ConvertFromUtf32($value) }
10 else { $match.Value }
11 })
12 [Net.WebUtility]::HtmlDecode($plain)
13}
corpus/conflict-page/native-restore/server.jsonl created+2
......@@ -0,0 +1,2 @@
1{"command": "mkdir /srv/agent/m6-conflict", "exit": 0, "stdout": "", "stderr": ""}
2{"command": "chmod u+w /srv/agent/m6-conflict/*", "exit": 0, "stdout": "", "stderr": ""}
corpus/conflict-page/native-three/commands.jsonl created+6
......@@ -0,0 +1,6 @@
1{"command": "powershell -NoProfile -Command \"Expand-Archive -LiteralPath C:\\one-tests\\transfer.zip -DestinationPath C:\\one-tests\\runs\\capture\\notebook\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-C6F6C14F -ExpectedPages -1 -Pdf -KeepOpen", "exit": 0, "stdout": "Read 1 sections and 1 pages.\r\n", "stderr": "", "error": null}
3{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\read -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
4{"command": "powershell -NoProfile -Command \"Rename-Item C:\\one-tests\\runs\\capture\\read before-read\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
5{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-C6F6C14F -UseCurrentCache -ExpectedPages -1 -Pdf", "exit": 0, "stdout": "Read 1 sections and 1 pages.\r\n", "stderr": "", "error": null}
6{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
corpus/conflict-page/native-three/conflicts.png created
Binary files /dev/null and b/corpus/conflict-page/native-three/conflicts.png differ
corpus/conflict-page/native-three/expand.png created
Binary files /dev/null and b/corpus/conflict-page/native-three/expand.png differ
corpus/conflict-page/native-three/machine.json created+1
......@@ -0,0 +1 @@
1{"name": "m6-c6f6c14f", "hostname": "ONE-M6-C6F6C14F"}
corpus/conflict-page/native-three/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/native-three/notebook/Open Notebook.onetoc2 differ
corpus/conflict-page/native-three/notebook/synthetic.one created
Binary files /dev/null and b/corpus/conflict-page/native-three/notebook/synthetic.one differ
corpus/conflict-page/native-three/run.json created+17
......@@ -0,0 +1,17 @@
1{
2 "notebook": "/Users/clo/dev/one/evidence/m8/conflict-abrupt-01/conflict-closed/notebook",
3 "expected_pages": -1,
4 "author": null,
5 "author_timeout_seconds": 600,
6 "inspect": true,
7 "base": {
8 "file": "win7-office-base.qcow2",
9 "format": "qcow2",
10 "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346",
11 "virtual_size": 68719476736
12 },
13 "scripts": {
14 "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331",
15 "read.ps1": "086789e467779add0fc071ad8c524c2b02d3d0d609aea4a41f3128cadba371e8"
16 }
17}
corpus/conflict-page/native-three/source.json created+14
......@@ -0,0 +1,14 @@
1[
2 {
3 "path": "Open Notebook.onetoc2",
4 "bytes": 3780,
5 "sha256": "f21c6d4eefc25ce78aee0420fdee0b409da52fdf3119b66302ac6ca4690c55ae",
6 "mtime_ns": 1788733155312083279
7 },
8 {
9 "path": "synthetic.one",
10 "bytes": 46416,
11 "sha256": "7ac7a42932848380b90aa478800ab141fce4fb591aae2bf1e167f1e516e5edc0",
12 "mtime_ns": 1788733155311971612
13 }
14]
corpus/conflict-page/native-three/teardown.json created+1
......@@ -0,0 +1 @@
1{"absent": true}
corpus/conflict-page/native/a/a-conflict.json created+13
......@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Client A conflicting edit. - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/native/a/a-conflict.png created
Binary files /dev/null and b/corpus/conflict-page/native/a/a-conflict.png differ
corpus/conflict-page/native/a/a-main.json created+13
......@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Client B conflicting edit. - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/native/a/a-main.png created
Binary files /dev/null and b/corpus/conflict-page/native/a/a-main.png differ
corpus/conflict-page/native/a/a-menu.json created+13
......@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "",
10 "class": "Net UI Tool Window",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/native/a/a-menu.png created
Binary files /dev/null and b/corpus/conflict-page/native/a/a-menu.png differ
corpus/conflict-page/native/a/a-shown.json created+13
......@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Client A conflicting edit. - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/native/a/a-shown.png created
Binary files /dev/null and b/corpus/conflict-page/native/a/a-shown.png differ
corpus/conflict-page/native/a/commands.jsonl created+124
......@@ -0,0 +1,124 @@
1{"command": "mkdir C:\\one-tests\\runs\\capture", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -ExecutionPolicy Bypass -File C:\\one-tests\\network.ps1 -LabMac 52:54:02:a3:5c:e3", "exit": 0, "stdout": "{\"renewalResult\":0,\"releaseResult\":0,\"mac\":\"52:54:02:a3:5c:e3\",\"address\":\"192.168.77.180\"}\r\n", "stderr": "", "error": null}
3{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0001.tmp C:\\one-tests\\runs\\capture\\inbox\\0001.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
4{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0002.tmp C:\\one-tests\\runs\\capture\\inbox\\0002.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
5{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0003.tmp C:\\one-tests\\runs\\capture\\inbox\\0003.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
6{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0004.tmp C:\\one-tests\\runs\\capture\\inbox\\0004.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
7{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0005.tmp C:\\one-tests\\runs\\capture\\inbox\\0005.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
8{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0006.tmp C:\\one-tests\\runs\\capture\\inbox\\0006.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
9{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0007.tmp C:\\one-tests\\runs\\capture\\inbox\\0007.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
10{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0008.tmp C:\\one-tests\\runs\\capture\\inbox\\0008.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
11{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0009.tmp C:\\one-tests\\runs\\capture\\inbox\\0009.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
12{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0010.tmp C:\\one-tests\\runs\\capture\\inbox\\0010.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
13{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0011.tmp C:\\one-tests\\runs\\capture\\inbox\\0011.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
14{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0012.tmp C:\\one-tests\\runs\\capture\\inbox\\0012.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
15{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0013.tmp C:\\one-tests\\runs\\capture\\inbox\\0013.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
16{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0014.tmp C:\\one-tests\\runs\\capture\\inbox\\0014.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
17{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0015.tmp C:\\one-tests\\runs\\capture\\inbox\\0015.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
18{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0016.tmp C:\\one-tests\\runs\\capture\\inbox\\0016.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
19{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0017.tmp C:\\one-tests\\runs\\capture\\inbox\\0017.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
20{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0018.tmp C:\\one-tests\\runs\\capture\\inbox\\0018.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
21{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0019.tmp C:\\one-tests\\runs\\capture\\inbox\\0019.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
22{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0020.tmp C:\\one-tests\\runs\\capture\\inbox\\0020.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
23{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0021.tmp C:\\one-tests\\runs\\capture\\inbox\\0021.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
24{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0022.tmp C:\\one-tests\\runs\\capture\\inbox\\0022.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
25{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0023.tmp C:\\one-tests\\runs\\capture\\inbox\\0023.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
26{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0024.tmp C:\\one-tests\\runs\\capture\\inbox\\0024.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
27{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0025.tmp C:\\one-tests\\runs\\capture\\inbox\\0025.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
28{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0026.tmp C:\\one-tests\\runs\\capture\\inbox\\0026.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
29{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0027.tmp C:\\one-tests\\runs\\capture\\inbox\\0027.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
30{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0028.tmp C:\\one-tests\\runs\\capture\\inbox\\0028.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
31{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0029.tmp C:\\one-tests\\runs\\capture\\inbox\\0029.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
32{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0030.tmp C:\\one-tests\\runs\\capture\\inbox\\0030.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
33{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0031.tmp C:\\one-tests\\runs\\capture\\inbox\\0031.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
34{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0032.tmp C:\\one-tests\\runs\\capture\\inbox\\0032.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
35{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0033.tmp C:\\one-tests\\runs\\capture\\inbox\\0033.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
36{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0034.tmp C:\\one-tests\\runs\\capture\\inbox\\0034.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
37{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0035.tmp C:\\one-tests\\runs\\capture\\inbox\\0035.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
38{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0036.tmp C:\\one-tests\\runs\\capture\\inbox\\0036.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
39{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0037.tmp C:\\one-tests\\runs\\capture\\inbox\\0037.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
40{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0038.tmp C:\\one-tests\\runs\\capture\\inbox\\0038.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
41{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0039.tmp C:\\one-tests\\runs\\capture\\inbox\\0039.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
42{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0040.tmp C:\\one-tests\\runs\\capture\\inbox\\0040.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
43{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0041.tmp C:\\one-tests\\runs\\capture\\inbox\\0041.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
44{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0042.tmp C:\\one-tests\\runs\\capture\\inbox\\0042.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
45{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0043.tmp C:\\one-tests\\runs\\capture\\inbox\\0043.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
46{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0044.tmp C:\\one-tests\\runs\\capture\\inbox\\0044.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
47{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0045.tmp C:\\one-tests\\runs\\capture\\inbox\\0045.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
48{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0046.tmp C:\\one-tests\\runs\\capture\\inbox\\0046.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
49{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0047.tmp C:\\one-tests\\runs\\capture\\inbox\\0047.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
50{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0048.tmp C:\\one-tests\\runs\\capture\\inbox\\0048.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
51{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0049.tmp C:\\one-tests\\runs\\capture\\inbox\\0049.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
52{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0050.tmp C:\\one-tests\\runs\\capture\\inbox\\0050.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
53{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0051.tmp C:\\one-tests\\runs\\capture\\inbox\\0051.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
54{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0052.tmp C:\\one-tests\\runs\\capture\\inbox\\0052.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
55{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0053.tmp C:\\one-tests\\runs\\capture\\inbox\\0053.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
56{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0054.tmp C:\\one-tests\\runs\\capture\\inbox\\0054.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
57{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0055.tmp C:\\one-tests\\runs\\capture\\inbox\\0055.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
58{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0056.tmp C:\\one-tests\\runs\\capture\\inbox\\0056.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
59{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0057.tmp C:\\one-tests\\runs\\capture\\inbox\\0057.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
60{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0058.tmp C:\\one-tests\\runs\\capture\\inbox\\0058.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
61{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0059.tmp C:\\one-tests\\runs\\capture\\inbox\\0059.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
62{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0060.tmp C:\\one-tests\\runs\\capture\\inbox\\0060.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
63{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0061.tmp C:\\one-tests\\runs\\capture\\inbox\\0061.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
64{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0062.tmp C:\\one-tests\\runs\\capture\\inbox\\0062.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
65{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0063.tmp C:\\one-tests\\runs\\capture\\inbox\\0063.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
66{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0064.tmp C:\\one-tests\\runs\\capture\\inbox\\0064.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
67{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0065.tmp C:\\one-tests\\runs\\capture\\inbox\\0065.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
68{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0066.tmp C:\\one-tests\\runs\\capture\\inbox\\0066.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
69{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0067.tmp C:\\one-tests\\runs\\capture\\inbox\\0067.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
70{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0068.tmp C:\\one-tests\\runs\\capture\\inbox\\0068.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
71{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0069.tmp C:\\one-tests\\runs\\capture\\inbox\\0069.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
72{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0070.tmp C:\\one-tests\\runs\\capture\\inbox\\0070.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
73{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0071.tmp C:\\one-tests\\runs\\capture\\inbox\\0071.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
74{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0072.tmp C:\\one-tests\\runs\\capture\\inbox\\0072.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
75{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0073.tmp C:\\one-tests\\runs\\capture\\inbox\\0073.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
76{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0074.tmp C:\\one-tests\\runs\\capture\\inbox\\0074.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
77{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0075.tmp C:\\one-tests\\runs\\capture\\inbox\\0075.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
78{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0076.tmp C:\\one-tests\\runs\\capture\\inbox\\0076.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
79{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0077.tmp C:\\one-tests\\runs\\capture\\inbox\\0077.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
80{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0078.tmp C:\\one-tests\\runs\\capture\\inbox\\0078.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
81{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0079.tmp C:\\one-tests\\runs\\capture\\inbox\\0079.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
82{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0080.tmp C:\\one-tests\\runs\\capture\\inbox\\0080.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
83{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0081.tmp C:\\one-tests\\runs\\capture\\inbox\\0081.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
84{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0082.tmp C:\\one-tests\\runs\\capture\\inbox\\0082.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
85{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0083.tmp C:\\one-tests\\runs\\capture\\inbox\\0083.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
86{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0084.tmp C:\\one-tests\\runs\\capture\\inbox\\0084.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
87{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0085.tmp C:\\one-tests\\runs\\capture\\inbox\\0085.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
88{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0086.tmp C:\\one-tests\\runs\\capture\\inbox\\0086.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
89{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0087.tmp C:\\one-tests\\runs\\capture\\inbox\\0087.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
90{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0088.tmp C:\\one-tests\\runs\\capture\\inbox\\0088.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
91{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0089.tmp C:\\one-tests\\runs\\capture\\inbox\\0089.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
92{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0090.tmp C:\\one-tests\\runs\\capture\\inbox\\0090.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
93{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0091.tmp C:\\one-tests\\runs\\capture\\inbox\\0091.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
94{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0092.tmp C:\\one-tests\\runs\\capture\\inbox\\0092.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
95{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0093.tmp C:\\one-tests\\runs\\capture\\inbox\\0093.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
96{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0094.tmp C:\\one-tests\\runs\\capture\\inbox\\0094.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
97{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0095.tmp C:\\one-tests\\runs\\capture\\inbox\\0095.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
98{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0096.tmp C:\\one-tests\\runs\\capture\\inbox\\0096.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
99{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0097.tmp C:\\one-tests\\runs\\capture\\inbox\\0097.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
100{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0098.tmp C:\\one-tests\\runs\\capture\\inbox\\0098.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
101{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0099.tmp C:\\one-tests\\runs\\capture\\inbox\\0099.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
102{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0100.tmp C:\\one-tests\\runs\\capture\\inbox\\0100.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
103{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0101.tmp C:\\one-tests\\runs\\capture\\inbox\\0101.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
104{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0102.tmp C:\\one-tests\\runs\\capture\\inbox\\0102.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
105{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0103.tmp C:\\one-tests\\runs\\capture\\inbox\\0103.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
106{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0104.tmp C:\\one-tests\\runs\\capture\\inbox\\0104.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
107{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0105.tmp C:\\one-tests\\runs\\capture\\inbox\\0105.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
108{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0106.tmp C:\\one-tests\\runs\\capture\\inbox\\0106.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
109{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0107.tmp C:\\one-tests\\runs\\capture\\inbox\\0107.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
110{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0108.tmp C:\\one-tests\\runs\\capture\\inbox\\0108.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
111{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0109.tmp C:\\one-tests\\runs\\capture\\inbox\\0109.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
112{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0110.tmp C:\\one-tests\\runs\\capture\\inbox\\0110.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
113{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0111.tmp C:\\one-tests\\runs\\capture\\inbox\\0111.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
114{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0112.tmp C:\\one-tests\\runs\\capture\\inbox\\0112.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
115{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0113.tmp C:\\one-tests\\runs\\capture\\inbox\\0113.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
116{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0114.tmp C:\\one-tests\\runs\\capture\\inbox\\0114.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
117{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0115.tmp C:\\one-tests\\runs\\capture\\inbox\\0115.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
118{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0116.tmp C:\\one-tests\\runs\\capture\\inbox\\0116.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
119{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0117.tmp C:\\one-tests\\runs\\capture\\inbox\\0117.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
120{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0118.tmp C:\\one-tests\\runs\\capture\\inbox\\0118.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
121{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0119.tmp C:\\one-tests\\runs\\capture\\inbox\\0119.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
122{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0120.tmp C:\\one-tests\\runs\\capture\\inbox\\0120.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
123{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0121.tmp C:\\one-tests\\runs\\capture\\inbox\\0121.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
124{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "ZipArchiveHelper : The process cannot access the file \r\n'C:\\one-tests\\runs\\capture\\controller.log' because it is being used by another \r\nprocess.\r\nAt C:\\Windows\\system32\\WindowsPowerShell\\v1.0\\Modules\\Microsoft.PowerShell.Arch\r\nive\\Microsoft.PowerShell.Archive.psm1:631 char:30\r\n+ ... sArchived = ZipArchiveHelper $sourceFilePaths $destinationPath $compr ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (C:\\one-tests\\runs\\capture\\con \r\n troller.log:String) [Write-Error], IOException\r\n + FullyQualifiedErrorId : CompressArchiveUnauthorizedAccessError,ZipArchiv \r\n eHelper\r\n \r\n", "error": null}
corpus/conflict-page/native/a/machine.json created+1
......@@ -0,0 +1 @@
1{"name": "m6-8a93780a", "hostname": "ONE-M6-8A93780A"}
corpus/conflict-page/native/a/teardown.json created+1
......@@ -0,0 +1 @@
1{"absent": true}
corpus/conflict-page/native/b-published/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/native/b-published/notebook/Open Notebook.onetoc2 differ
corpus/conflict-page/native/b-published/notebook/synthetic.one created
Binary files /dev/null and b/corpus/conflict-page/native/b-published/notebook/synthetic.one differ
corpus/conflict-page/native/b/b-main.json created+13
......@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Client B conflicting edit. - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/native/b/b-main.png created
Binary files /dev/null and b/corpus/conflict-page/native/b/b-main.png differ
corpus/conflict-page/native/b/b-shown.json created+13
......@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Client A conflicting edit. - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/native/b/b-shown.png created
Binary files /dev/null and b/corpus/conflict-page/native/b/b-shown.png differ
corpus/conflict-page/native/b/commands.jsonl created+21
......@@ -0,0 +1,21 @@
1{"command": "mkdir C:\\one-tests\\runs\\capture", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -ExecutionPolicy Bypass -File C:\\one-tests\\network.ps1 -LabMac 52:54:02:e7:d0:a7", "exit": 0, "stdout": "{\"renewalResult\":0,\"releaseResult\":0,\"mac\":\"52:54:02:e7:d0:a7\",\"address\":\"192.168.77.42\"}\r\n", "stderr": "", "error": null}
3{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0001.tmp C:\\one-tests\\runs\\capture\\inbox\\0001.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
4{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0002.tmp C:\\one-tests\\runs\\capture\\inbox\\0002.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
5{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0003.tmp C:\\one-tests\\runs\\capture\\inbox\\0003.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
6{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0004.tmp C:\\one-tests\\runs\\capture\\inbox\\0004.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
7{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0005.tmp C:\\one-tests\\runs\\capture\\inbox\\0005.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
8{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0006.tmp C:\\one-tests\\runs\\capture\\inbox\\0006.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
9{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0007.tmp C:\\one-tests\\runs\\capture\\inbox\\0007.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
10{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0008.tmp C:\\one-tests\\runs\\capture\\inbox\\0008.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
11{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0009.tmp C:\\one-tests\\runs\\capture\\inbox\\0009.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
12{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0010.tmp C:\\one-tests\\runs\\capture\\inbox\\0010.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
13{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0011.tmp C:\\one-tests\\runs\\capture\\inbox\\0011.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
14{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0012.tmp C:\\one-tests\\runs\\capture\\inbox\\0012.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
15{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0013.tmp C:\\one-tests\\runs\\capture\\inbox\\0013.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
16{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0014.tmp C:\\one-tests\\runs\\capture\\inbox\\0014.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
17{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0015.tmp C:\\one-tests\\runs\\capture\\inbox\\0015.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
18{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0016.tmp C:\\one-tests\\runs\\capture\\inbox\\0016.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
19{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0017.tmp C:\\one-tests\\runs\\capture\\inbox\\0017.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
20{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0018.tmp C:\\one-tests\\runs\\capture\\inbox\\0018.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
21{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "ZipArchiveHelper : The process cannot access the file \r\n'C:\\one-tests\\runs\\capture\\controller.log' because it is being used by another \r\nprocess.\r\nAt C:\\Windows\\system32\\WindowsPowerShell\\v1.0\\Modules\\Microsoft.PowerShell.Arch\r\nive\\Microsoft.PowerShell.Archive.psm1:631 char:30\r\n+ ... sArchived = ZipArchiveHelper $sourceFilePaths $destinationPath $compr ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (C:\\one-tests\\runs\\capture\\con \r\n troller.log:String) [Write-Error], IOException\r\n + FullyQualifiedErrorId : CompressArchiveUnauthorizedAccessError,ZipArchiv \r\n eHelper\r\n \r\n", "error": null}
corpus/conflict-page/native/b/machine.json created+1
......@@ -0,0 +1 @@
1{"name": "m6-d63654a2", "hostname": "ONE-M6-D63654A2"}
corpus/conflict-page/native/b/teardown.json created+1
......@@ -0,0 +1 @@
1{"absent": true}
corpus/conflict-page/native/clients.json created+4
......@@ -0,0 +1,4 @@
1{
2 "a": "m6-8a93780a",
3 "b": "m6-d63654a2"
4}
\ No newline at end of file
corpus/conflict-page/native/conflict.json created+15
......@@ -0,0 +1,15 @@
1{
2 "{64183EF4-4E19-0CBE-3D3B-88FF17D58B1C},1": [
3 "Client A disjoint edit.",
4 "Client B conflicting edit.",
5 "Client B disjoint cell.",
6 "Right cell"
7 ],
8 "{9C2E0DE1-2E23-094F-3F9F-A919195FC40D},1": [
9 "",
10 "Client A conflicting edit.",
11 "Client A disjoint edit.",
12 "Left cell",
13 "Right cell"
14 ]
15}
\ No newline at end of file
corpus/conflict-page/native/conflict/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/native/conflict/notebook/Open Notebook.onetoc2 differ
corpus/conflict-page/native/conflict/notebook/synthetic.one created
Binary files /dev/null and b/corpus/conflict-page/native/conflict/notebook/synthetic.one differ
corpus/conflict-page/native/initial/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/native/initial/notebook/Open Notebook.onetoc2 differ
corpus/conflict-page/native/initial/notebook/synthetic.one created
Binary files /dev/null and b/corpus/conflict-page/native/initial/notebook/synthetic.one differ
corpus/conflict-page/native/linux-teardown.json created+1
......@@ -0,0 +1 @@
1{"absent": true}
corpus/conflict-page/native/linux.json created+11
......@@ -0,0 +1,11 @@
1{
2 "cpus": 2,
3 "disk_gb": 16,
4 "hostname": "one-conflict",
5 "lab_address": "192.168.77.1",
6 "lab_mac": "52:54:04:69:39:00",
7 "memory_mb": 2048,
8 "samba_port": 14450,
9 "ssh_port": 22000,
10 "wan_mac": "52:54:03:76:a2:ae"
11}
\ No newline at end of file
corpus/conflict-page/native/native_conflict.py created+261
......@@ -0,0 +1,261 @@
1#!/usr/bin/env python3
2"""OneNote 2010 conflict pages: how they arise, what they store and how OneNote shows them.
3
4`sync LINUX OUTPUT` runs two disposable OneNote clients on one Samba notebook: client A edits
5two paragraphs offline while client B edits one of them and a table cell online; A then
6reconnects, which makes OneNote write a conflict page. The server file is kept before and
7after, A and B are screenshotted with the information bar, the page list shown, the conflict
8page open and its menu open, and A then deletes the conflict page through that menu.
9
10`cold NOTEBOOK OUTPUT [--delete]` cold-opens a notebook holding a conflict page in a fresh
11clone and screenshots the same views; `--delete` deletes the conflict page through the menu
12and captures the notebook OneNote leaves.
13"""
14import argparse
15from contextlib import ExitStack
16from concurrent.futures import ThreadPoolExecutor
17import base64
18import json
19from pathlib import Path
20import shutil
21import subprocess
22import sys
23import time
24import xml.etree.ElementTree as ET
25
26sys.path.insert(0, str(Path(__file__).resolve().parent))
27sys.path.insert(0, str(Path(__file__).resolve().parent / 'w7'))
28import native_runner as runner
29from native_runner import ROOT, clone, command, windows, vm
30import linux_vm
31
32FIXTURE = ROOT / 'corpus/native-ink/cold-ui-ink/notebook'
33SHARE = 'm6-conflict'
34ORIGINAL = 'Fictitious: café, 東京, مرحبا'
35OTHER_OUTLINE = 'Fictitious positioned outline.'
36CELL = 'Left cell'
37
38# Coordinates on OneNote 2010 maximized at 800x600, from the stage-5 and m8 captures: the
39# information bar's text, and the first conflict page listed under its page.
40BAR = (332, 100)
41FIRST_CONFLICT = (725, 142)
42
43
44def shot(name, output, label, script):
45 """Runs an AutoHotkey step on the clone, keeping its screenshot as `label.png`."""
46 (output / f'{label}.ahk').write_text(script)
47 result = windows.do_exec(script, target=name, timeout_ms=120000, shot_delay_ms=1500)
48 screenshot = result.pop('png_b64', None)
49 if screenshot:
50 (output / f'{label}.png').write_bytes(base64.b64decode(screenshot))
51 (output / f'{label}.json').write_text(json.dumps(result, indent=2))
52 if result.get('error') or result.get('exit') != 0:
53 raise RuntimeError(f'{label}: {result}')
54 return result
55
56
57def click(x, y):
58 return ('WinActivate("ahk_exe ONENOTE.EXE")\nCoordMode("Mouse", "Screen")\n'
59 f'Click({x}, {y})\nSleep(1200)\n')
60
61
62# Reports any dialog OneNote raised, so a confirmation shows in the step's record.
63DIALOG = ('if WinExist("ahk_class #32770 ahk_exe ONENOTE.EXE") {\n'
64 ' FileAppend("dialog: " WinGetTitle() " | " WinGetText() "`n", "*", "UTF-8")\n'
65 '}\n')
66
67
68def conflict_views(name, output, page, prefix, delete=False):
69 """Screenshots a page with a conflict page: the bar, the list shown, the conflict page and
70 its menu; `delete` then chooses Delete Conflict Page."""
71 shot(name, output, f'{prefix}-main', runner.navigation_script(page) + 'Sleep(2000)\n')
72 shot(name, output, f'{prefix}-shown', click(*BAR))
73 shot(name, output, f'{prefix}-conflict', click(*FIRST_CONFLICT))
74 shot(name, output, f'{prefix}-menu', click(*BAR))
75 if delete:
76 shot(name, output, f'{prefix}-delete', 'WinActivate("ahk_exe ONENOTE.EXE")\nSend("d")\nSleep(1500)\n' + DIALOG)
77 else:
78 shot(name, output, f'{prefix}-closed', 'WinActivate("ahk_exe ONENOTE.EXE")\nSend("{Esc}")\nSleep(800)\n')
79
80
81def cold(notebook, output, delete):
82 def interaction(name, output):
83 page = ET.parse(sorted((output / 'read').glob('page-*.xml'))[0]).getroot().attrib['ID']
84 conflict_views(name, output, page, 'cold', delete)
85 if delete:
86 # OneNote writes the deletion in the background; give it time before the re-read.
87 time.sleep(20)
88 runner.capture(notebook, output, collect_notebook=True, interaction=interaction)
89
90
91def sync(server, output):
92 from native_collaboration import reachable_page_text
93 from document_model import EXPORTER
94
95 if linux_vm.instance_path(server).exists():
96 raise ValueError('Choose a new Linux VM name; existing machines are not owned by this run.')
97 output = output.resolve()
98 output.mkdir(parents=True, exist_ok=False)
99 scripts = output / 'scripts'
100 scripts.mkdir()
101 for name in ('cold.ps1', 'collaborate.ps1', 'network.ps1', 'text.ps1', 'stress.ps1'):
102 shutil.copyfile(ROOT / 'tools/native' / name, scripts / name)
103 shutil.copyfile(Path(__file__), output / 'native_conflict.py')
104
105 def ssh(text, timeout=90):
106 result = linux_vm.run_ssh(server, text, timeout=timeout)
107 with (output / 'server.jsonl').open('a') as log:
108 log.write(json.dumps({'command': text, 'exit': result.returncode, 'stdout': result.stdout, 'stderr': result.stderr}) + '\n')
109 result.check_returncode()
110 return result.stdout
111
112 def server_state(label):
113 """Copies the server's notebook until it parses, keeping it as `label/notebook`."""
114 deadline = time.monotonic() + 120
115 while True:
116 target = output / label
117 shutil.rmtree(target, ignore_errors=True)
118 (target / 'notebook').mkdir(parents=True)
119 for name in ('synthetic.one', 'Open Notebook.onetoc2'):
120 with (target / 'notebook' / name).open('wb') as stream:
121 subprocess.run(linux_vm.ssh_argv(server, f"cat '/srv/agent/{SHARE}/{name}'"), stdout=stream, check=True, timeout=60)
122 result = subprocess.run([EXPORTER, target / 'notebook/synthetic.one', target / 'model'], capture_output=True, text=True)
123 if result.returncode == 0:
124 model = json.loads((target / 'model/document.json').read_text())
125 return reachable_page_text(model)
126 if time.monotonic() > deadline:
127 raise RuntimeError(f'{label}: {result.stderr}')
128 time.sleep(1)
129
130 linux_vm.create_instance(server)
131 try:
132 linux_vm.launch(server)
133 linux_vm.wait_instance(server, 600)
134 (output / 'linux.json').write_text(json.dumps(linux_vm.load_instance(server), indent=2))
135 ssh(f'mkdir /srv/agent/{SHARE}')
136 for name in ('synthetic.one', 'Open Notebook.onetoc2'):
137 with (FIXTURE / name).open('rb') as stream:
138 subprocess.run(linux_vm.ssh_argv(server, f"cat > '/srv/agent/{SHARE}/{name}'"), stdin=stream, check=True)
139 ssh(f'chmod u+w /srv/agent/{SHARE}/*')
140 with ExitStack() as stack:
141 labels = ['a', 'b']
142 for label in labels:
143 (output / label).mkdir()
144
145 def start(label):
146 manager = clone(output / label)
147 name = manager.__enter__()
148 return label, name, manager
149 with ThreadPoolExecutor(max_workers=2) as pool:
150 started = list(pool.map(start, labels))
151 for _, _, manager in started:
152 stack.push(manager)
153 clients = {}
154 for label, name, _ in started:
155 folder = output / label
156 for local in scripts.iterdir():
157 remote = 'cold-current.ps1' if local.name == 'cold.ps1' else local.name
158 result = windows.do_put(local, 'C:\\one-tests\\' + remote, name)
159 if result.get('error'): raise RuntimeError(result['error'])
160 command(name, 'mkdir C:\\one-tests\\runs\\capture', folder)
161 command(name, 'powershell -NoProfile -ExecutionPolicy Bypass -File C:\\one-tests\\network.ps1 -LabMac ' + vm.lab_mac(name), folder)
162 result = windows.do_spawn('cmd /c powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\collaborate.ps1 -Root C:\\one-tests\\runs\\capture -SharedPath \\\\192.168.77.1\\agent\\' + SHARE + ' -CloneHost ONE-' + name.upper() + ' > C:\\one-tests\\runs\\capture\\controller.log 2>&1', name)
163 if result.get('error'): raise RuntimeError(result['error'])
164 deadline = time.monotonic() + 180
165 while windows.do_cmd('if exist C:\\one-tests\\runs\\capture\\ready (echo ready)', target=name).get('stdout', '').find('ready') < 0:
166 if time.monotonic() > deadline: raise RuntimeError('The collaboration controller did not open the notebook')
167 time.sleep(1)
168 clients[label] = {'name': name, 'folder': folder, 'sequence': 0}
169 (output / 'clients.json').write_text(json.dumps({label: client['name'] for label, client in clients.items()}, indent=2))
170
171 def action(client, kind, **parameters):
172 client['sequence'] += 1
173 sequence = client['sequence']
174 local = client['folder'] / f'command-{sequence:04}.json'
175 local.write_text(json.dumps({'action': kind, **parameters}, ensure_ascii=False))
176 inbox = f'C:\\one-tests\\runs\\capture\\inbox\\{sequence:04}'
177 result = windows.do_put(local, inbox + '.tmp', client['name'])
178 if result.get('error'): raise RuntimeError(result['error'])
179 command(client['name'], f'move {inbox}.tmp {inbox}.json', client['folder'])
180 remote = f'C:\\one-tests\\runs\\capture\\outbox\\{sequence}'
181 deadline = time.monotonic() + 600
182 while True:
183 result = windows.do_cmd(f'if exist {remote}\\error (type {remote}\\error & exit /b 1) else (if exist {remote}\\done (echo complete))', target=client['name'])
184 if result.get('exit') != 0: raise RuntimeError(str(result))
185 if 'complete' in result.get('stdout', ''): break
186 if time.monotonic() > deadline: raise TimeoutError(f'{kind} did not complete')
187 time.sleep(.5)
188 if kind != 'snapshot':
189 return None
190 hierarchy = client['folder'] / f'hierarchy-{sequence:04}.xml'
191 windows.do_get(remote + '\\hierarchy.xml', hierarchy, client['name'])
192 pages = [node for node in ET.fromstring(hierarchy.read_text(encoding='utf-8-sig').strip()).iter() if node.tag.endswith('}Page')]
193 for i in range(len(pages)):
194 windows.do_get(remote + f'\\page-{i}.xml', client['folder'] / f'snapshot-{sequence:04}-{i}.xml', client['name'])
195 return pages
196
197 def settle(client, check, label):
198 deadline = time.monotonic() + 300
199 while True:
200 action(client, 'sync')
201 state = server_state(label)
202 if check(state): return state
203 if time.monotonic() > deadline: raise TimeoutError(f'{label}: {state}')
204 time.sleep(2)
205
206 a, b = clients['a'], clients['b']
207 texts = lambda state: {text for values in state.values() for text in values}
208 server_state('initial')
209 vm.qmp(a['name'], 'set_link', {'name': 'lab', 'up': False})
210 try:
211 action(a, 'edit', expected=ORIGINAL, text='Client A conflicting edit.')
212 action(a, 'edit', expected=OTHER_OUTLINE, text='Client A disjoint edit.')
213 action(a, 'snapshot')
214 action(b, 'edit', expected=ORIGINAL, text='Client B conflicting edit.')
215 action(b, 'edit', expected=CELL, text='Client B disjoint cell.')
216 settle(b, lambda state: {'Client B conflicting edit.', 'Client B disjoint cell.'} <= texts(state), 'b-published')
217 finally:
218 vm.qmp(a['name'], 'set_link', {'name': 'lab', 'up': True})
219 state = settle(a, lambda state: len(state) > 1 and 'Client A disjoint edit.' in texts(state), 'conflict')
220 (output / 'conflict.json').write_text(json.dumps(state, indent=2, ensure_ascii=False))
221 page = action(a, 'snapshot')[0].attrib['ID']
222 conflict_views(a['name'], output / 'a', page, 'a')
223 action(b, 'sync')
224 time.sleep(5)
225 page_b = action(b, 'snapshot')[0].attrib['ID']
226 conflict_views(b['name'], output / 'b', page_b, 'b')
227 conflict_views(a['name'], output / 'a', page, 'a-again', delete=True)
228 state = settle(a, lambda state: len(state) == 1, 'deleted')
229 (output / 'deleted.json').write_text(json.dumps(state, indent=2, ensure_ascii=False))
230 action(b, 'sync')
231 time.sleep(5)
232 shot(b['name'], output / 'b', 'b-after-delete', runner.navigation_script(page_b) + 'Sleep(2000)\n')
233 for client in clients.values():
234 try:
235 action(client, 'close')
236 except Exception as error:
237 (client['folder'] / 'close-failure.txt').write_text(str(error))
238 server_state('final')
239 finally:
240 if linux_vm.running(server):
241 linux_vm.shutdown(server, 120)
242 linux_vm.delete_instance(server)
243 (output / 'linux-teardown.json').write_text(json.dumps({'absent': not linux_vm.instance_path(server).exists()}) + '\n')
244
245
246if __name__ == '__main__':
247 parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
248 commands = parser.add_subparsers(dest='mode', required=True)
249 lab = commands.add_parser('sync')
250 lab.add_argument('linux')
251 lab.add_argument('output', type=Path)
252 reopen = commands.add_parser('cold')
253 reopen.add_argument('notebook', type=Path)
254 reopen.add_argument('output', type=Path)
255 reopen.add_argument('--delete', action='store_true')
256 args = parser.parse_args()
257 if args.mode == 'sync':
258 sync(args.linux, args.output)
259 else:
260 cold(args.notebook, args.output, args.delete)
261 print('Captured', args.output)
corpus/conflict-page/native/scripts/cold.ps1 created+27
......@@ -0,0 +1,27 @@
1param([Parameter(Mandatory=$true)][string]$Root, [string]$CloneHost = '')
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
5if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
6 throw 'Choose a run directly below C:\one-tests\runs.'
7}
8if (Get-Process ONENOTE -ErrorAction SilentlyContinue) { throw 'Close OneNote before resetting its test cache.' }
9$key = 'HKCU:\Software\Microsoft\Office\14.0\OneNote'
10if ($CloneHost) {
11 if ($CloneHost -notmatch '^ONE-[A-Z0-9-]+$' -or [Environment]::MachineName -ne $CloneHost) {
12 throw 'The disposable clone hostname does not match this machine.'
13 }
14 New-Item "$key\Options\Paths" -Force | Out-Null
15 New-ItemProperty "$key\Options\Paths" -Name UnfiledNotesSection -PropertyType ExpandString -Value 'C:\one-tests\Loose.one' -Force | Out-Null
16} elseif ((Get-ItemProperty "$key\Options\Paths").UnfiledNotesSection -ne 'C:\one-tests\Loose.one' -or
17 -not (Test-Path 'C:\one-tests\profile-original-cache')) {
18 throw 'Park the personal OneNote profile before resetting the test cache.'
19}
20$cache = Join-Path $env:LOCALAPPDATA 'Microsoft\OneNote\14.0'
21$parked = Join-Path 'C:\one-tests\caches' ([IO.Path]::GetFileName($root))
22if (Test-Path $parked) { throw 'Choose a new run; its parked cache already exists.' }
23New-Item -ItemType Directory -Path 'C:\one-tests\caches' -Force | Out-Null
24if (Test-Path $cache) { Move-Item -LiteralPath $cache -Destination $parked }
25if (Test-Path "$key\OpenNotebooks") { Remove-Item "$key\OpenNotebooks" -Recurse }
26New-Item "$key\OpenNotebooks" | Out-Null
27New-ItemProperty "$key\OpenNotebooks" -Name '1' -PropertyType String -Value "$root\notebook" | Out-Null
corpus/conflict-page/native/scripts/collaborate.ps1 created+105
......@@ -0,0 +1,105 @@
1param([Parameter(Mandatory=$true)][string]$Root,
2 [Parameter(Mandatory=$true)][string]$SharedPath,
3 [Parameter(Mandatory=$true)][string]$CloneHost,
4 [switch]$ResumeCache,
5 [int]$StartSequence = 1)
6function Keep-Container($page, $target) {
7 $container = $target
8 while ($container.ParentNode -ne $page.DocumentElement) { $container = $container.ParentNode }
9 foreach ($sibling in @($page.DocumentElement.ChildNodes)) {
10 if ($sibling -ne $container -and $sibling.LocalName -in @('Outline', 'Title', 'Image', 'InkDrawing')) {
11 [void]$page.DocumentElement.RemoveChild($sibling)
12 }
13 }
14}
15Set-StrictMode -Version Latest
16$ErrorActionPreference = 'Stop'
17. "$PSScriptRoot\text.ps1"
18if ($SharedPath -notmatch '^\\\\192\.168\.77\.1\\agent\\m6-[a-z0-9-]+$') { throw 'Choose a disposable Linux lab notebook.' }
19if ($env:COMPUTERNAME -ne $CloneHost) { throw 'The collaboration controller requires its owned clone.' }
20if (-not $ResumeCache) { & "$PSScriptRoot\cold-current.ps1" -Root $Root -CloneHost $CloneHost }
21$shared = $SharedPath
22if (-not (Test-Path "$shared\synthetic.one")) { throw 'Create the shared fixture first.' }
23New-Item -ItemType Directory "$Root\inbox", "$Root\outbox" -Force | Out-Null
24$user = 'HKCU:\Software\Microsoft\Office\Common\UserInfo'
25New-Item $user -Force | Out-Null
26New-ItemProperty $user -Name UserName -Value $CloneHost -PropertyType String -Force | Out-Null
27New-ItemProperty $user -Name UserInitials -Value $CloneHost.Substring($CloneHost.Length - 3) -PropertyType String -Force | Out-Null
28$app = New-Object -ComObject OneNote.Application
29$notebook = ''
30try {
31 $app.OpenHierarchy($shared, '', [ref]$notebook, 0)
32 $section = ''
33 $app.OpenHierarchy('synthetic.one', $notebook, [ref]$section, 0)
34 [IO.File]::WriteAllText("$Root\ready", 'ready')
35 $deadline = [DateTime]::UtcNow.AddMinutes(30)
36 $sequence = $StartSequence
37 while ([DateTime]::UtcNow -lt $deadline) {
38 $name = '{0:d4}.json' -f $sequence
39 $input = "$Root\inbox\$name"
40 if (-not (Test-Path $input)) { Start-Sleep -Milliseconds 100; continue }
41 $command = Get-Content -Raw -Encoding UTF8 $input | ConvertFrom-Json
42 $output = "$Root\outbox\$sequence"
43 New-Item -ItemType Directory $output | Out-Null
44 try {
45 if ($command.action -in @('prepare-stress', 'stress')) {
46 . "$PSScriptRoot\stress.ps1"
47 Invoke-Stress $app $section $command $output $shared
48 }
49 elseif ($command.action -eq 'sync') { $app.SyncHierarchy($notebook) }
50 elseif ($command.action -eq 'kill-process') {
51 Get-Process ONENOTE | Stop-Process -Force
52 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
53 $app = New-Object -ComObject OneNote.Application
54 $app.OpenHierarchy($shared, '', [ref]$notebook, 0)
55 $app.OpenHierarchy('synthetic.one', $notebook, [ref]$section, 0)
56 }
57 elseif ($command.action -eq 'edit' -or $command.action -eq 'snapshot') {
58 $hierarchy = ''
59 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
60 [IO.File]::WriteAllText("$output\hierarchy.xml", $hierarchy, [Text.Encoding]::UTF8)
61 [xml]$tree = $hierarchy
62 $matches = 0
63 $editedPage = $null
64 $i = 0
65 foreach ($node in $tree.SelectNodes('//*[local-name()="Page"]')) {
66 $content = ''
67 $app.GetPageContent($node.GetAttribute('ID'), [ref]$content, 1, 1)
68 [IO.File]::WriteAllText("$output\page-$i.xml", $content, [Text.Encoding]::UTF8)
69 [xml]$page = $content
70 if ($command.action -eq 'edit') {
71 $targets = @($page.SelectNodes('//*[local-name()="T"]') | Where-Object {
72 (Get-ParagraphText $_.InnerText) -eq $command.expected
73 })
74 if ($targets.Count -gt 1) { throw 'Expected one matching text object.' }
75 if ($targets.Count -eq 1) {
76 $matches++
77 if ($matches -gt 1) { throw 'The text occurs on multiple pages.' }
78 $targets[0].InnerText = [Security.SecurityElement]::Escape($command.text)
79 Keep-Container $page $targets[0]
80 $editedPage = $page
81 }
82 }
83 $i++
84 }
85 if ($command.action -eq 'edit' -and $matches -ne 1) { throw 'Expected one matching text object.' }
86 if ($editedPage) { $app.UpdatePageContent($editedPage.OuterXml, [DateTime]::MinValue, 1, $false) }
87 } elseif ($command.action -ne 'close') { throw 'Unknown collaboration command.' }
88 [IO.File]::WriteAllText("$output\done", 'ok')
89 } catch {
90 [IO.File]::WriteAllText("$output\error", $_.Exception.ToString())
91 }
92 if ($command.action -eq 'close') { break }
93 $sequence++
94 }
95} finally {
96 try { if ($notebook) { $app.CloseNotebook($notebook, $false) } }
97 finally {
98 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
99 $app = $null
100 [GC]::Collect()
101 [GC]::WaitForPendingFinalizers()
102 Get-Process ONENOTE -ErrorAction SilentlyContinue | Wait-Process -Timeout 10
103 [IO.File]::WriteAllText("$Root\closed", 'closed')
104 }
105}
corpus/conflict-page/native/scripts/network.ps1 created+20
......@@ -0,0 +1,20 @@
1param([Parameter(Mandatory=$true)][string]$LabMac)
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$adapter = @(Get-WmiObject Win32_NetworkAdapterConfiguration | Where-Object { $_.MACAddress -eq $LabMac })
5if ($adapter.Count -ne 1) { throw 'The clone lab adapter is ambiguous.' }
6$enabled = $adapter[0].EnableDHCP()
7if ($enabled.ReturnValue -notin @(0, 1)) {
8 throw ('Unable to enable DHCP on the clone lab adapter; result ' + $enabled.ReturnValue)
9}
10$release = $adapter[0].ReleaseDHCPLease()
11$result = $adapter[0].RenewDHCPLease()
12$deadline = [DateTime]::UtcNow.AddSeconds(60)
13do {
14 $adapter = @(Get-WmiObject Win32_NetworkAdapterConfiguration | Where-Object { $_.MACAddress -eq $LabMac })
15 $addresses = @($adapter[0].IPAddress | Where-Object { $_ -match '^192\.168\.77\.\d+$' -and $_ -ne '192.168.77.1' })
16 if ($addresses.Count -eq 1) { break }
17 Start-Sleep -Milliseconds 250
18} while ([DateTime]::UtcNow -lt $deadline)
19if ($addresses.Count -ne 1) { throw ('The clone did not receive a lab IPv4 address; renewal result ' + $result.ReturnValue + '; adapter ' + ($adapter[0] | Select-Object MACAddress,DHCPEnabled,IPEnabled,IPAddress | ConvertTo-Json -Compress)) }
20@{mac=$LabMac;address=$addresses[0];releaseResult=$release.ReturnValue;renewalResult=$result.ReturnValue} | ConvertTo-Json -Compress
corpus/conflict-page/native/scripts/stress.ps1 created+83
......@@ -0,0 +1,83 @@
1function Invoke-Stress($app, $section, $command, $output, $shared) {
2 $hierarchy = ''
3 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
4 [IO.File]::WriteAllText("$output\hierarchy.xml", $hierarchy, [Text.Encoding]::UTF8)
5 [xml]$tree = $hierarchy
6 $pages = @($tree.SelectNodes('//*[local-name()="Page"]'))
7 if ($pages.Count -ne 1) { throw 'Expected one stress-test page.' }
8 $pageId = $pages[0].GetAttribute('ID')
9 if ($command.action -eq 'prepare-stress') {
10 for ($i = 0; $i -lt $command.clients; $i++) {
11 $xml = '<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="' + $pageId + '"><one:Outline><one:Position x="48" y="' + (150 + 75 * $i) + '" z="' + ($i + 1) + '"/><one:OEChildren><one:OE><one:T><![CDATA[Native ' + $i + ':]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>'
12 $app.UpdatePageContent($xml, [DateTime]::MinValue, 1, $false)
13 }
14 return
15 }
16 $random = New-Object Random([int]$command.seed)
17 $events = New-Object IO.StreamWriter("$output\events.jsonl", $false, [Text.Encoding]::UTF8)
18 $events.AutoFlush = $true
19 try {
20 [IO.File]::WriteAllText("$output\ready", 'ready')
21 $deadline = [DateTime]::UtcNow.AddMinutes(10)
22 while (-not (Test-Path "$shared\stress-start")) {
23 if ([DateTime]::UtcNow -gt $deadline) { throw 'Stress start timed out.' }
24 Start-Sleep -Milliseconds 20
25 }
26 for ($i = 0; $i -lt $command.operations; $i++) {
27 if (($command.PSObject.Properties.Name -contains 'maintenance') -and $command.maintenance -and $i -eq [Math]::Floor($command.operations / 2)) {
28 [IO.File]::WriteAllText("$shared\maintenance-paused-n$($command.actor)", 'paused')
29 while (-not (Test-Path "$shared\maintenance-resume")) {
30 if ([DateTime]::UtcNow -gt $deadline) { throw 'Maintenance pause timed out.' }
31 Start-Sleep -Milliseconds 100
32 }
33 }
34 Start-Sleep -Milliseconds $random.Next(10, 100)
35 $started = [DateTime]::UtcNow.Ticks
36 $content = ''
37 $app.GetPageContent($pageId, [ref]$content, 1, 1)
38 [xml]$page = $content
39 $matches = @($page.SelectNodes('//*[local-name()="T"]') | Where-Object {
40 (Get-ParagraphText $_.InnerText).StartsWith($command.prefix)
41 })
42 if ($matches.Count -ne 1) { throw 'Expected one stress-test paragraph.' }
43 $before = Get-ParagraphText $matches[0].InnerText
44 $token = ' [n' + $command.actor + ':' + $i + ']'
45 $start = $end = $before.Length
46 $replacement = $token
47 $bold = $italic = $false
48 if ($command.edit) {
49 $boundaries = New-Object 'Collections.Generic.List[int]'
50 for ($at = $command.prefix.Length; $at -le $before.Length; $at++) {
51 $boundaries.Add($at)
52 if ($at -lt $before.Length -and [char]::IsHighSurrogate($before[$at])) { $at++ }
53 }
54 $first = $boundaries[$random.Next($boundaries.Count)]
55 $second = $boundaries[$random.Next($boundaries.Count)]
56 $start = [Math]::Min($first, $second)
57 $end = [Math]::Max($first, $second)
58 $replacement = ' caf' + [char]0xe9 + ' ' + [char]::ConvertFromUtf32(0x1f980) + $token
59 $bold = $random.Next(2) -eq 1
60 $italic = $random.Next(2) -eq 1
61 }
62 $after = $before.Substring(0, $start) + $replacement + $before.Substring($end)
63 $html = [Security.SecurityElement]::Escape($after)
64 if ($command.edit) {
65 $weight = if ($bold) { 'bold' } else { 'normal' }
66 $style = if ($italic) { 'italic' } else { 'normal' }
67 $html = '<span style="font-weight:' + $weight + ';font-style:' + $style + '">' + $html + '</span>'
68 }
69 $matches[0].InnerText = $html
70 Keep-Container $page $matches[0]
71 $updateStarted = [DateTime]::UtcNow.Ticks
72 $app.UpdatePageContent($page.OuterXml, [DateTime]::MinValue, 1, $false)
73 $updated = [DateTime]::UtcNow.Ticks
74 $events.WriteLine((@{ operation=$i; before=$before; token=$token; started_ticks=$started;
75 range=@($start, $end); replacement=$replacement; bold=$bold; italic=$italic;
76 update_started_ticks=$updateStarted; updated_ticks=$updated } | ConvertTo-Json -Compress))
77 if ($i -eq 0) { [IO.File]::WriteAllText("$output\editing", 'editing') }
78 if ($command.sync_every -gt 0 -and (($i + 1) % $command.sync_every) -eq 0) {
79 $app.SyncHierarchy($section)
80 }
81 }
82 } finally { $events.Dispose() }
83}
corpus/conflict-page/native/scripts/text.ps1 created+13
......@@ -0,0 +1,13 @@
1function Get-ParagraphText([string]$html) {
2 $plain = [regex]::Replace($html, '<[^>]*>', '')
3 $plain = [regex]::Replace($plain, '&#(?:x([0-9a-fA-F]+)|([0-9]+));', {
4 param($match)
5 $value = if ($match.Groups[1].Success) {
6 [Convert]::ToInt32($match.Groups[1].Value, 16)
7 } else { [int]$match.Groups[2].Value }
8 # The Win7 framework leaves supplementary numeric entities undecoded.
9 if ($value -gt 0xffff -and $value -le 0x10ffff) { [char]::ConvertFromUtf32($value) }
10 else { $match.Value }
11 })
12 [Net.WebUtility]::HtmlDecode($plain)
13}
corpus/conflict-page/native/server.jsonl created+2
......@@ -0,0 +1,2 @@
1{"command": "mkdir /srv/agent/m6-conflict", "exit": 0, "stdout": "", "stderr": ""}
2{"command": "chmod u+w /srv/agent/m6-conflict/*", "exit": 0, "stdout": "", "stderr": ""}
corpus/extended-ascii/README.md created+9
......@@ -0,0 +1,9 @@
1# Extended ASCII text
2
3`TextExtendedAscii` (`0x1c003498`, MS-ONE 2.2.89) names no code page. Each `candidate`
4is `m6/ansi/notebook` with the paragraph's `RichEditTextLangID` (`0x10001cfe`) set to
50x0409 (`en`), 0x0419 (`ru`, ANSI code page 1251) or 0x0411 (`ja`, code page 932); its
6text holds the bytes 0x80–0xFF. `cold` is each one's fresh OneNote 2010 read: all three
7export U+0080–U+00FF, so OneNote reads the bytes as Latin-1 whatever the language, and
8`read/page-000.png` shows 0x80–0x9F as nothing and 0xC0 as À rather than Cyrillic А.
9OneNote left the section files unchanged.
corpus/extended-ascii/en/candidate/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/extended-ascii/en/candidate/Open Notebook.onetoc2 differ
corpus/extended-ascii/en/candidate/synthetic.one created
Binary files /dev/null and b/corpus/extended-ascii/en/candidate/synthetic.one differ
corpus/extended-ascii/en/cold/commands.jsonl created+3
......@@ -0,0 +1,3 @@
1{"command": "powershell -NoProfile -Command \"Expand-Archive -LiteralPath C:\\one-tests\\transfer.zip -DestinationPath C:\\one-tests\\runs\\capture\\notebook\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-A375E105 -ExpectedPages 1 -KeepOpen", "exit": 0, "stdout": "Read 1 sections and 1 pages.\r\n", "stderr": "", "error": null}
3{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
corpus/extended-ascii/en/cold/machine.json created+1
......@@ -0,0 +1 @@
1{"name": "m6-a375e105", "hostname": "ONE-M6-A375E105"}
corpus/extended-ascii/en/cold/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/extended-ascii/en/cold/notebook/Open Notebook.onetoc2 differ
corpus/extended-ascii/en/cold/notebook/synthetic.one created
Binary files /dev/null and b/corpus/extended-ascii/en/cold/notebook/synthetic.one differ
corpus/extended-ascii/en/cold/read/environment.json created+7
......@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-A375E105",
5 "cold": true,
6 "onenote": "14.0.4763.1000"
7}
corpus/extended-ascii/en/cold/read/hierarchy.xml created+2
......@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{EE076CAD-41AC-4A4A-8929-025A57318F7D}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-27T05:33:41.000Z" color="#9595AA"><one:Section name="synthetic" ID="{AF80AC8F-943F-0D15-0824-72674F116D2B}{1}{B0}" path="C:\one-tests\runs\capture\notebook\synthetic.one" lastModifiedTime="2026-09-27T05:33:41.000Z" color="#8AA8E4"><one:Page ID="{83C95F09-1DF4-4E04-8F43-DAF6B3297134}{1}{B0}" name="Fictitious plain text." dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-27T05:33:41.000Z" pageLevel="1"/></one:Section></one:Notebook>
corpus/extended-ascii/en/cold/read/page-000.navigation.json created+13
......@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Extended bytes: \u0080 \u0081 \u0082 \u0083 \u0084 \u0085 \u0086 \u0087 \u0088 \u0089 \u008a \u008b \u008c \u008d \u008e \u008f \u0090 - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/extended-ascii/en/cold/read/page-000.png created
Binary files /dev/null and b/corpus/extended-ascii/en/cold/read/page-000.png differ
corpus/extended-ascii/en/cold/read/page-000.xml created+2
......@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{83C95F09-1DF4-4E04-8F43-DAF6B3297134}{1}{B0}" name="Fictitious plain text." dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-27T05:33:41.000Z" pageLevel="1" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-05T05:06:55.000Z" objectID="{E405F7B5-5681-4DB2-80B1-562D4CB59455}{10}{B0}"><one:Position x="36.0" y="14.40000057220459" z="0"/><one:Size width="468.0" height="40.28314971923828"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:55.000Z" lastModifiedTime="2026-09-05T05:06:55.000Z" objectID="{E405F7B5-5681-4DB2-80B1-562D4CB59455}{11}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Extended bytes: €  ‚ ƒ „ … † ‡ ˆ ‰ Š ‹ Œ  Ž   ‘ ’ “ ” • – — ˜ ™ š › œ  ž Ÿ &nbsp; ¡ ¢ £ ¤ ¥ ¦ § ¨ © ª « ¬ ­ ® ¯ ° ± ² ³ ´ µ ¶ · ¸ ¹ º » ¼ ½ ¾ ¿ À Á Â Ã Ä Å Æ Ç È É Ê Ë Ì Í Î Ï Ð Ñ Ò Ó Ô Õ Ö × Ø Ù Ú Û Ü Ý Þ ß à á â ã ä å æ ç è é ê ë ì í î ï ð ñ ò ó ô õ ö ÷ ø ù ú û ü ý þ ÿ]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/extended-ascii/en/cold/read/payloads.json created+3
......@@ -0,0 +1,3 @@
1[
2
3]
\ No newline at end of file
corpus/extended-ascii/en/cold/run.json created+18
......@@ -0,0 +1,18 @@
1{
2 "notebook": "/private/tmp/ansi-lang/en/notebook",
3 "expected_pages": 1,
4 "author": null,
5 "author_timeout_seconds": 600,
6 "inspect": false,
7 "collect_notebook": true,
8 "base": {
9 "file": "win7-office-base.qcow2",
10 "format": "qcow2",
11 "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346",
12 "virtual_size": 68719476736
13 },
14 "scripts": {
15 "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331",
16 "read.ps1": "04013bfcccee40a17e2a225f9b9e96f40a3a8daad9e350609654f8eda3ccbb41"
17 }
18}
corpus/extended-ascii/en/cold/scripts/cold.ps1 created+27
......@@ -0,0 +1,27 @@
1param([Parameter(Mandatory=$true)][string]$Root, [string]$CloneHost = '')
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
5if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
6 throw 'Choose a run directly below C:\one-tests\runs.'
7}
8if (Get-Process ONENOTE -ErrorAction SilentlyContinue) { throw 'Close OneNote before resetting its test cache.' }
9$key = 'HKCU:\Software\Microsoft\Office\14.0\OneNote'
10if ($CloneHost) {
11 if ($CloneHost -notmatch '^ONE-[A-Z0-9-]+$' -or [Environment]::MachineName -ne $CloneHost) {
12 throw 'The disposable clone hostname does not match this machine.'
13 }
14 New-Item "$key\Options\Paths" -Force | Out-Null
15 New-ItemProperty "$key\Options\Paths" -Name UnfiledNotesSection -PropertyType ExpandString -Value 'C:\one-tests\Loose.one' -Force | Out-Null
16} elseif ((Get-ItemProperty "$key\Options\Paths").UnfiledNotesSection -ne 'C:\one-tests\Loose.one' -or
17 -not (Test-Path 'C:\one-tests\profile-original-cache')) {
18 throw 'Park the personal OneNote profile before resetting the test cache.'
19}
20$cache = Join-Path $env:LOCALAPPDATA 'Microsoft\OneNote\14.0'
21$parked = Join-Path 'C:\one-tests\caches' ([IO.Path]::GetFileName($root))
22if (Test-Path $parked) { throw 'Choose a new run; its parked cache already exists.' }
23New-Item -ItemType Directory -Path 'C:\one-tests\caches' -Force | Out-Null
24if (Test-Path $cache) { Move-Item -LiteralPath $cache -Destination $parked }
25if (Test-Path "$key\OpenNotebooks") { Remove-Item "$key\OpenNotebooks" -Recurse }
26New-Item "$key\OpenNotebooks" | Out-Null
27New-ItemProperty "$key\OpenNotebooks" -Name '1' -PropertyType String -Value "$root\notebook" | Out-Null
corpus/extended-ascii/en/cold/scripts/read.ps1 created+142
......@@ -0,0 +1,142 @@
1param(
2 [Parameter(Mandatory=$true)][string]$Root,
3 [int]$ExpectedPages = -1,
4 [switch]$UseCurrentCache,
5 [switch]$Pdf,
6 [switch]$KeepOpen,
7 [string]$CloneHost = ''
8)
9Set-StrictMode -Version Latest
10$ErrorActionPreference = 'Stop'
11$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
12if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
13 throw 'Choose a run directly below C:\one-tests\runs.'
14}
15$notebook = Join-Path $root 'notebook'
16$output = Join-Path $root 'read'
17if (Test-Path $output) { throw 'Choose a new read destination.' }
18if ($UseCurrentCache) {
19 if ((Get-ItemProperty 'HKCU:\Software\Microsoft\Office\14.0\OneNote\Options\Paths').UnfiledNotesSection -ne 'C:\one-tests\Loose.one') {
20 throw 'Park the personal OneNote profile before reading test notebooks.'
21 }
22} else {
23 & "$PSScriptRoot\cold-current.ps1" -Root $root -CloneHost $CloneHost
24}
25New-Item -ItemType Directory -Path $output | Out-Null
26$app = New-Object -ComObject OneNote.Application
27$notebookId = ''
28$failure = $null
29try {
30 $app.OpenHierarchy($notebook, '', [ref]$notebookId, 0)
31 $process = Get-Process ONENOTE
32 @{ hostname = [Environment]::MachineName; onenote = $process.MainModule.FileVersionInfo.FileVersion;
33 powershell = $PSVersionTable.PSVersion.ToString(); schema = 'xs2010'; cold = (-not $UseCurrentCache.IsPresent) } |
34 ConvertTo-Json | Set-Content (Join-Path $output 'environment.json') -Encoding UTF8
35 $sections = @()
36 foreach ($file in @(Get-ChildItem $notebook -Recurse | Where-Object { $_.Extension -eq '.one' })) {
37 $id = ''
38 $app.OpenHierarchy($file.FullName, '', [ref]$id, 0)
39 $sections += $id
40 }
41 $deadline = [DateTime]::UtcNow.AddSeconds(300)
42 $previous = ''
43 $lastChange = ''
44 $stableSince = [DateTime]::UtcNow
45 $settled = $false
46 do {
47 $pages = @{}
48 foreach ($section in $sections) {
49 $hierarchy = ''
50 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
51 [xml]$xml = $hierarchy
52 foreach ($node in $xml.SelectNodes('//*[@path]')) {
53 if (-not $node.GetAttribute('path').StartsWith("$notebook\", [StringComparison]::OrdinalIgnoreCase)) {
54 throw 'OneNote opened a section outside the copied notebook.'
55 }
56 }
57 foreach ($node in $xml.SelectNodes('//*[local-name()="Page"]')) {
58 $id = $node.GetAttribute('ID')
59 $content = ''
60 $app.GetPageContent($id, [ref]$content, 1, 1)
61 $pages[$id] = $content
62 }
63 }
64 $signature = [String]::Join('|', @($pages.Keys | Sort-Object | ForEach-Object { $_ + $pages[$_] }))
65 if ($signature -ne $previous) {
66 $lastChange = $previous
67 $previous = $signature
68 $stableSince = [DateTime]::UtcNow
69 }
70 if ((($ExpectedPages -ge 0 -and $pages.Count -eq $ExpectedPages) -or
71 ($ExpectedPages -lt 0 -and $pages.Count -gt 0)) -and
72 ([DateTime]::UtcNow - $stableSince).TotalSeconds -ge 2) { $settled = $true; break }
73 Start-Sleep -Milliseconds 250
74 } while ([DateTime]::UtcNow -lt $deadline)
75 if (-not $settled -or ($ExpectedPages -ge 0 -and $pages.Count -ne $ExpectedPages) -or ($ExpectedPages -lt 0 -and $pages.Count -eq 0)) {
76 [IO.File]::WriteAllText((Join-Path $output 'previous-signature.txt'), $lastChange, [Text.Encoding]::UTF8)
77 $index = 0
78 foreach ($id in @($pages.Keys | Sort-Object)) {
79 [IO.File]::WriteAllText((Join-Path $output ('unsettled-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
80 $index++
81 }
82 $hierarchy = ''
83 $app.GetHierarchy($notebookId, 4, [ref]$hierarchy, 1)
84 [IO.File]::WriteAllText((Join-Path $output 'unsettled-hierarchy.xml'), $hierarchy, [Text.Encoding]::UTF8)
85 throw "Expected $ExpectedPages stable pages; OneNote returned $($pages.Count), settled=$settled."
86 }
87 $index = 0
88 $payloads = @()
89 foreach ($id in @($pages.Keys | Sort-Object)) {
90 [IO.File]::WriteAllText((Join-Path $output ('page-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
91 if ($Pdf) {
92 $pdfPath = Join-Path $output ('page-{0:d3}.pdf' -f $index)
93 $app.NavigateTo($id, '', $false)
94 $app.Publish($id, $pdfPath, 3, '')
95 if (-not (Test-Path $pdfPath) -or (Get-Item $pdfPath).Length -eq 0) { throw 'OneNote did not publish the page PDF.' }
96 }
97 [xml]$page = $pages[$id]
98 foreach ($file in $page.SelectNodes('//*[local-name()="InsertedFile" or local-name()="MediaFile"]')) {
99 $bytes = [IO.File]::ReadAllBytes($file.GetAttribute('pathCache'))
100 $hash = [BitConverter]::ToString([Security.Cryptography.SHA256]::Create().ComputeHash($bytes)).Replace('-', '').ToLowerInvariant()
101 [IO.File]::WriteAllBytes((Join-Path $output ($hash + '.attachment')), $bytes)
102 $payloads += @{ page = $id; object = $file.ParentNode.GetAttribute('objectID');
103 kind = $file.LocalName; name = $file.GetAttribute('preferredName');
104 sha256 = $hash; bytes = $bytes.Length }
105 }
106 $index++
107 }
108 [IO.File]::WriteAllText((Join-Path $output 'payloads.json'), (ConvertTo-Json -InputObject $payloads -Depth 4), [Text.Encoding]::UTF8)
109 $all = ''
110 $app.GetHierarchy($notebookId, 4, [ref]$all, 1)
111 [xml]$finalTree = $all
112 $finalIds = @($finalTree.SelectNodes('//*[local-name()="Page"]') | ForEach-Object { $_.GetAttribute('ID') } | Sort-Object -Unique)
113 if ($finalIds.Count -ne $pages.Count -or @($finalIds | Where-Object { -not $pages.ContainsKey($_) }).Count -ne 0) {
114 throw 'The notebook hierarchy changed while collecting page evidence; repeat the cold read.'
115 }
116 [IO.File]::WriteAllText((Join-Path $output 'hierarchy.xml'), $all, [Text.Encoding]::UTF8)
117 Write-Output "Read $($sections.Count) sections and $($pages.Count) pages."
118} catch {
119 $failure = $_
120 [IO.File]::WriteAllText((Join-Path $output 'failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
121 throw
122} finally {
123 try {
124 try {
125 if ($notebookId -and $CloneHost) { $app.SyncHierarchy($notebookId) }
126 if ($notebookId -and -not $KeepOpen -and (-not $UseCurrentCache -or $CloneHost)) {
127 $app.CloseNotebook($notebookId, $false)
128 }
129 } catch {
130 if ($null -eq $failure) { throw }
131 [IO.File]::WriteAllText((Join-Path $output 'cleanup-failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
132 }
133 } finally {
134 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
135 }
136 $app = $null
137 [GC]::Collect()
138 [GC]::WaitForPendingFinalizers()
139 if (-not $UseCurrentCache -and -not $CloneHost) {
140 Get-Process ONENOTE -ErrorAction SilentlyContinue | Wait-Process -Timeout 10
141 }
142}
corpus/extended-ascii/en/cold/source.json created+14
......@@ -0,0 +1,14 @@
1[
2 {
3 "path": "Open Notebook.onetoc2",
4 "bytes": 4816,
5 "sha256": "b2f857961b76258a1582b402c048b340e89962257681426b3bfc58df8e0adbfa",
6 "mtime_ns": 1790487072902156985
7 },
8 {
9 "path": "synthetic.one",
10 "bytes": 12456,
11 "sha256": "aa6f1188773f1cc41cc5c619f213ce04b30592320c6b2e32b2b4c91639423ab4",
12 "mtime_ns": 1790487072902335569
13 }
14]
corpus/extended-ascii/en/cold/teardown.json created+1
......@@ -0,0 +1 @@
1{"absent": true}
corpus/extended-ascii/ja/candidate/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/extended-ascii/ja/candidate/Open Notebook.onetoc2 differ
corpus/extended-ascii/ja/candidate/synthetic.one created
Binary files /dev/null and b/corpus/extended-ascii/ja/candidate/synthetic.one differ
corpus/extended-ascii/ja/cold/commands.jsonl created+3
......@@ -0,0 +1,3 @@
1{"command": "powershell -NoProfile -Command \"Expand-Archive -LiteralPath C:\\one-tests\\transfer.zip -DestinationPath C:\\one-tests\\runs\\capture\\notebook\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-8DC7DCC7 -ExpectedPages 1 -KeepOpen", "exit": 0, "stdout": "Read 1 sections and 1 pages.\r\n", "stderr": "", "error": null}
3{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
corpus/extended-ascii/ja/cold/machine.json created+1
......@@ -0,0 +1 @@
1{"name": "m6-8dc7dcc7", "hostname": "ONE-M6-8DC7DCC7"}
corpus/extended-ascii/ja/cold/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/extended-ascii/ja/cold/notebook/Open Notebook.onetoc2 differ
corpus/extended-ascii/ja/cold/notebook/synthetic.one created
Binary files /dev/null and b/corpus/extended-ascii/ja/cold/notebook/synthetic.one differ
corpus/extended-ascii/ja/cold/read/environment.json created+7
......@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-8DC7DCC7",
5 "cold": true,
6 "onenote": "14.0.4763.1000"
7}
corpus/extended-ascii/ja/cold/read/hierarchy.xml created+2
......@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{5B6DFD6A-ECAF-406E-B615-008B44758AA3}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-27T05:37:38.000Z" color="#9595AA"><one:Section name="synthetic" ID="{3B11134F-882C-06AE-07C4-197CA488888D}{1}{B0}" path="C:\one-tests\runs\capture\notebook\synthetic.one" lastModifiedTime="2026-09-27T05:37:38.000Z" color="#8AA8E4"><one:Page ID="{1758E0C9-01E7-45BF-80A3-B1ED58B09492}{1}{B0}" name="Fictitious plain text." dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-27T05:37:38.000Z" pageLevel="1"/></one:Section></one:Notebook>
corpus/extended-ascii/ja/cold/read/page-000.navigation.json created+13
......@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Extended bytes: \u0080 \u0081 \u0082 \u0083 \u0084 \u0085 \u0086 \u0087 \u0088 \u0089 \u008a \u008b \u008c \u008d \u008e \u008f \u0090 - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/extended-ascii/ja/cold/read/page-000.png created
Binary files /dev/null and b/corpus/extended-ascii/ja/cold/read/page-000.png differ
corpus/extended-ascii/ja/cold/read/page-000.xml created+2
......@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{1758E0C9-01E7-45BF-80A3-B1ED58B09492}{1}{B0}" name="Fictitious plain text." dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-27T05:37:38.000Z" pageLevel="1" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-05T05:06:55.000Z" objectID="{70944875-4A92-4609-8F51-3D36A72C71F3}{10}{B0}"><one:Position x="36.0" y="14.40000057220459" z="0"/><one:Size width="468.0" height="40.28314971923828"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:55.000Z" lastModifiedTime="2026-09-05T05:06:55.000Z" objectID="{70944875-4A92-4609-8F51-3D36A72C71F3}{11}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Extended bytes: €  ‚ ƒ „ … † ‡ ˆ ‰ Š ‹ Œ  Ž   ‘ ’ “ ” • – — ˜ ™ š › œ  ž Ÿ &nbsp; ¡ ¢ £ ¤ ¥ ¦ § ¨ © ª « ¬ ­ ® ¯ ° ± ² ³ ´ µ ¶ · ¸ ¹ º » ¼ ½ ¾ ¿ À Á Â Ã Ä Å Æ Ç È É Ê Ë Ì Í Î Ï Ð Ñ Ò Ó Ô Õ Ö × Ø Ù Ú Û Ü Ý Þ ß à á â ã ä å æ ç è é ê ë ì í î ï ð ñ ò ó ô õ ö ÷ ø ù ú û ü ý þ ÿ]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/extended-ascii/ja/cold/read/payloads.json created+3
......@@ -0,0 +1,3 @@
1[
2
3]
\ No newline at end of file
corpus/extended-ascii/ja/cold/run.json created+18
......@@ -0,0 +1,18 @@
1{
2 "notebook": "/private/tmp/ansi-lang/ja/notebook",
3 "expected_pages": 1,
4 "author": null,
5 "author_timeout_seconds": 600,
6 "inspect": false,
7 "collect_notebook": true,
8 "base": {
9 "file": "win7-office-base.qcow2",
10 "format": "qcow2",
11 "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346",
12 "virtual_size": 68719476736
13 },
14 "scripts": {
15 "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331",
16 "read.ps1": "04013bfcccee40a17e2a225f9b9e96f40a3a8daad9e350609654f8eda3ccbb41"
17 }
18}
corpus/extended-ascii/ja/cold/scripts/cold.ps1 created+27
......@@ -0,0 +1,27 @@
1param([Parameter(Mandatory=$true)][string]$Root, [string]$CloneHost = '')
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
5if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
6 throw 'Choose a run directly below C:\one-tests\runs.'
7}
8if (Get-Process ONENOTE -ErrorAction SilentlyContinue) { throw 'Close OneNote before resetting its test cache.' }
9$key = 'HKCU:\Software\Microsoft\Office\14.0\OneNote'
10if ($CloneHost) {
11 if ($CloneHost -notmatch '^ONE-[A-Z0-9-]+$' -or [Environment]::MachineName -ne $CloneHost) {
12 throw 'The disposable clone hostname does not match this machine.'
13 }
14 New-Item "$key\Options\Paths" -Force | Out-Null
15 New-ItemProperty "$key\Options\Paths" -Name UnfiledNotesSection -PropertyType ExpandString -Value 'C:\one-tests\Loose.one' -Force | Out-Null
16} elseif ((Get-ItemProperty "$key\Options\Paths").UnfiledNotesSection -ne 'C:\one-tests\Loose.one' -or
17 -not (Test-Path 'C:\one-tests\profile-original-cache')) {
18 throw 'Park the personal OneNote profile before resetting the test cache.'
19}
20$cache = Join-Path $env:LOCALAPPDATA 'Microsoft\OneNote\14.0'
21$parked = Join-Path 'C:\one-tests\caches' ([IO.Path]::GetFileName($root))
22if (Test-Path $parked) { throw 'Choose a new run; its parked cache already exists.' }
23New-Item -ItemType Directory -Path 'C:\one-tests\caches' -Force | Out-Null
24if (Test-Path $cache) { Move-Item -LiteralPath $cache -Destination $parked }
25if (Test-Path "$key\OpenNotebooks") { Remove-Item "$key\OpenNotebooks" -Recurse }
26New-Item "$key\OpenNotebooks" | Out-Null
27New-ItemProperty "$key\OpenNotebooks" -Name '1' -PropertyType String -Value "$root\notebook" | Out-Null
corpus/extended-ascii/ja/cold/scripts/read.ps1 created+142
......@@ -0,0 +1,142 @@
1param(
2 [Parameter(Mandatory=$true)][string]$Root,
3 [int]$ExpectedPages = -1,
4 [switch]$UseCurrentCache,
5 [switch]$Pdf,
6 [switch]$KeepOpen,
7 [string]$CloneHost = ''
8)
9Set-StrictMode -Version Latest
10$ErrorActionPreference = 'Stop'
11$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
12if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
13 throw 'Choose a run directly below C:\one-tests\runs.'
14}
15$notebook = Join-Path $root 'notebook'
16$output = Join-Path $root 'read'
17if (Test-Path $output) { throw 'Choose a new read destination.' }
18if ($UseCurrentCache) {
19 if ((Get-ItemProperty 'HKCU:\Software\Microsoft\Office\14.0\OneNote\Options\Paths').UnfiledNotesSection -ne 'C:\one-tests\Loose.one') {
20 throw 'Park the personal OneNote profile before reading test notebooks.'
21 }
22} else {
23 & "$PSScriptRoot\cold-current.ps1" -Root $root -CloneHost $CloneHost
24}
25New-Item -ItemType Directory -Path $output | Out-Null
26$app = New-Object -ComObject OneNote.Application
27$notebookId = ''
28$failure = $null
29try {
30 $app.OpenHierarchy($notebook, '', [ref]$notebookId, 0)
31 $process = Get-Process ONENOTE
32 @{ hostname = [Environment]::MachineName; onenote = $process.MainModule.FileVersionInfo.FileVersion;
33 powershell = $PSVersionTable.PSVersion.ToString(); schema = 'xs2010'; cold = (-not $UseCurrentCache.IsPresent) } |
34 ConvertTo-Json | Set-Content (Join-Path $output 'environment.json') -Encoding UTF8
35 $sections = @()
36 foreach ($file in @(Get-ChildItem $notebook -Recurse | Where-Object { $_.Extension -eq '.one' })) {
37 $id = ''
38 $app.OpenHierarchy($file.FullName, '', [ref]$id, 0)
39 $sections += $id
40 }
41 $deadline = [DateTime]::UtcNow.AddSeconds(300)
42 $previous = ''
43 $lastChange = ''
44 $stableSince = [DateTime]::UtcNow
45 $settled = $false
46 do {
47 $pages = @{}
48 foreach ($section in $sections) {
49 $hierarchy = ''
50 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
51 [xml]$xml = $hierarchy
52 foreach ($node in $xml.SelectNodes('//*[@path]')) {
53 if (-not $node.GetAttribute('path').StartsWith("$notebook\", [StringComparison]::OrdinalIgnoreCase)) {
54 throw 'OneNote opened a section outside the copied notebook.'
55 }
56 }
57 foreach ($node in $xml.SelectNodes('//*[local-name()="Page"]')) {
58 $id = $node.GetAttribute('ID')
59 $content = ''
60 $app.GetPageContent($id, [ref]$content, 1, 1)
61 $pages[$id] = $content
62 }
63 }
64 $signature = [String]::Join('|', @($pages.Keys | Sort-Object | ForEach-Object { $_ + $pages[$_] }))
65 if ($signature -ne $previous) {
66 $lastChange = $previous
67 $previous = $signature
68 $stableSince = [DateTime]::UtcNow
69 }
70 if ((($ExpectedPages -ge 0 -and $pages.Count -eq $ExpectedPages) -or
71 ($ExpectedPages -lt 0 -and $pages.Count -gt 0)) -and
72 ([DateTime]::UtcNow - $stableSince).TotalSeconds -ge 2) { $settled = $true; break }
73 Start-Sleep -Milliseconds 250
74 } while ([DateTime]::UtcNow -lt $deadline)
75 if (-not $settled -or ($ExpectedPages -ge 0 -and $pages.Count -ne $ExpectedPages) -or ($ExpectedPages -lt 0 -and $pages.Count -eq 0)) {
76 [IO.File]::WriteAllText((Join-Path $output 'previous-signature.txt'), $lastChange, [Text.Encoding]::UTF8)
77 $index = 0
78 foreach ($id in @($pages.Keys | Sort-Object)) {
79 [IO.File]::WriteAllText((Join-Path $output ('unsettled-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
80 $index++
81 }
82 $hierarchy = ''
83 $app.GetHierarchy($notebookId, 4, [ref]$hierarchy, 1)
84 [IO.File]::WriteAllText((Join-Path $output 'unsettled-hierarchy.xml'), $hierarchy, [Text.Encoding]::UTF8)
85 throw "Expected $ExpectedPages stable pages; OneNote returned $($pages.Count), settled=$settled."
86 }
87 $index = 0
88 $payloads = @()
89 foreach ($id in @($pages.Keys | Sort-Object)) {
90 [IO.File]::WriteAllText((Join-Path $output ('page-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
91 if ($Pdf) {
92 $pdfPath = Join-Path $output ('page-{0:d3}.pdf' -f $index)
93 $app.NavigateTo($id, '', $false)
94 $app.Publish($id, $pdfPath, 3, '')
95 if (-not (Test-Path $pdfPath) -or (Get-Item $pdfPath).Length -eq 0) { throw 'OneNote did not publish the page PDF.' }
96 }
97 [xml]$page = $pages[$id]
98 foreach ($file in $page.SelectNodes('//*[local-name()="InsertedFile" or local-name()="MediaFile"]')) {
99 $bytes = [IO.File]::ReadAllBytes($file.GetAttribute('pathCache'))
100 $hash = [BitConverter]::ToString([Security.Cryptography.SHA256]::Create().ComputeHash($bytes)).Replace('-', '').ToLowerInvariant()
101 [IO.File]::WriteAllBytes((Join-Path $output ($hash + '.attachment')), $bytes)
102 $payloads += @{ page = $id; object = $file.ParentNode.GetAttribute('objectID');
103 kind = $file.LocalName; name = $file.GetAttribute('preferredName');
104 sha256 = $hash; bytes = $bytes.Length }
105 }
106 $index++
107 }
108 [IO.File]::WriteAllText((Join-Path $output 'payloads.json'), (ConvertTo-Json -InputObject $payloads -Depth 4), [Text.Encoding]::UTF8)
109 $all = ''
110 $app.GetHierarchy($notebookId, 4, [ref]$all, 1)
111 [xml]$finalTree = $all
112 $finalIds = @($finalTree.SelectNodes('//*[local-name()="Page"]') | ForEach-Object { $_.GetAttribute('ID') } | Sort-Object -Unique)
113 if ($finalIds.Count -ne $pages.Count -or @($finalIds | Where-Object { -not $pages.ContainsKey($_) }).Count -ne 0) {
114 throw 'The notebook hierarchy changed while collecting page evidence; repeat the cold read.'
115 }
116 [IO.File]::WriteAllText((Join-Path $output 'hierarchy.xml'), $all, [Text.Encoding]::UTF8)
117 Write-Output "Read $($sections.Count) sections and $($pages.Count) pages."
118} catch {
119 $failure = $_
120 [IO.File]::WriteAllText((Join-Path $output 'failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
121 throw
122} finally {
123 try {
124 try {
125 if ($notebookId -and $CloneHost) { $app.SyncHierarchy($notebookId) }
126 if ($notebookId -and -not $KeepOpen -and (-not $UseCurrentCache -or $CloneHost)) {
127 $app.CloseNotebook($notebookId, $false)
128 }
129 } catch {
130 if ($null -eq $failure) { throw }
131 [IO.File]::WriteAllText((Join-Path $output 'cleanup-failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
132 }
133 } finally {
134 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
135 }
136 $app = $null
137 [GC]::Collect()
138 [GC]::WaitForPendingFinalizers()
139 if (-not $UseCurrentCache -and -not $CloneHost) {
140 Get-Process ONENOTE -ErrorAction SilentlyContinue | Wait-Process -Timeout 10
141 }
142}
corpus/extended-ascii/ja/cold/source.json created+14
......@@ -0,0 +1,14 @@
1[
2 {
3 "path": "Open Notebook.onetoc2",
4 "bytes": 4816,
5 "sha256": "b2f857961b76258a1582b402c048b340e89962257681426b3bfc58df8e0adbfa",
6 "mtime_ns": 1790487072903313695
7 },
8 {
9 "path": "synthetic.one",
10 "bytes": 12456,
11 "sha256": "b792b045a793ef97ded2a9d6ab456070fb5d9d3f48763067ed5e24bde482358a",
12 "mtime_ns": 1790487072903438737
13 }
14]
corpus/extended-ascii/ja/cold/teardown.json created+1
......@@ -0,0 +1 @@
1{"absent": true}
corpus/extended-ascii/ru/candidate/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/extended-ascii/ru/candidate/Open Notebook.onetoc2 differ
corpus/extended-ascii/ru/candidate/synthetic.one created
Binary files /dev/null and b/corpus/extended-ascii/ru/candidate/synthetic.one differ
corpus/extended-ascii/ru/cold/commands.jsonl created+3
......@@ -0,0 +1,3 @@
1{"command": "powershell -NoProfile -Command \"Expand-Archive -LiteralPath C:\\one-tests\\transfer.zip -DestinationPath C:\\one-tests\\runs\\capture\\notebook\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-2B8202A3 -ExpectedPages 1 -KeepOpen", "exit": 0, "stdout": "Read 1 sections and 1 pages.\r\n", "stderr": "", "error": null}
3{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
corpus/extended-ascii/ru/cold/machine.json created+1
......@@ -0,0 +1 @@
1{"name": "m6-2b8202a3", "hostname": "ONE-M6-2B8202A3"}
corpus/extended-ascii/ru/cold/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/extended-ascii/ru/cold/notebook/Open Notebook.onetoc2 differ
corpus/extended-ascii/ru/cold/notebook/synthetic.one created
Binary files /dev/null and b/corpus/extended-ascii/ru/cold/notebook/synthetic.one differ
corpus/extended-ascii/ru/cold/read/environment.json created+7
......@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-2B8202A3",
5 "cold": true,
6 "onenote": "14.0.4763.1000"
7}
corpus/extended-ascii/ru/cold/read/hierarchy.xml created+2
......@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{6A0D391B-07F4-4CA2-B500-0760660A4E33}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-27T05:33:41.000Z" color="#9595AA"><one:Section name="synthetic" ID="{7392C207-21C6-0CE2-3853-9AD75F71833E}{1}{B0}" path="C:\one-tests\runs\capture\notebook\synthetic.one" lastModifiedTime="2026-09-27T05:33:41.000Z" color="#8AA8E4"><one:Page ID="{5FDB3181-A80D-4FF3-BF34-3246A3499F21}{1}{B0}" name="Fictitious plain text." dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-27T05:33:41.000Z" pageLevel="1"/></one:Section></one:Notebook>
corpus/extended-ascii/ru/cold/read/page-000.navigation.json created+13
......@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Extended bytes: \u0080 \u0081 \u0082 \u0083 \u0084 \u0085 \u0086 \u0087 \u0088 \u0089 \u008a \u008b \u008c \u008d \u008e \u008f \u0090 - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/extended-ascii/ru/cold/read/page-000.png created
Binary files /dev/null and b/corpus/extended-ascii/ru/cold/read/page-000.png differ
corpus/extended-ascii/ru/cold/read/page-000.xml created+2
......@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{5FDB3181-A80D-4FF3-BF34-3246A3499F21}{1}{B0}" name="Fictitious plain text." dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-27T05:33:41.000Z" pageLevel="1" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-05T05:06:55.000Z" objectID="{3817993D-E378-4C45-B0C6-BE9D5CD57A40}{10}{B0}"><one:Position x="36.0" y="14.40000057220459" z="0"/><one:Size width="468.0" height="40.28314971923828"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:55.000Z" lastModifiedTime="2026-09-05T05:06:55.000Z" objectID="{3817993D-E378-4C45-B0C6-BE9D5CD57A40}{11}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Extended bytes: €  ‚ ƒ „ … † ‡ ˆ ‰ Š ‹ Œ  Ž   ‘ ’ “ ” • – — ˜ ™ š › œ  ž Ÿ &nbsp; ¡ ¢ £ ¤ ¥ ¦ § ¨ © ª « ¬ ­ ® ¯ ° ± ² ³ ´ µ ¶ · ¸ ¹ º » ¼ ½ ¾ ¿ À Á Â Ã Ä Å Æ Ç È É Ê Ë Ì Í Î Ï Ð Ñ Ò Ó Ô Õ Ö × Ø Ù Ú Û Ü Ý Þ ß à á â ã ä å æ ç è é ê ë ì í î ï ð ñ ò ó ô õ ö ÷ ø ù ú û ü ý þ ÿ]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/extended-ascii/ru/cold/read/payloads.json created+3
......@@ -0,0 +1,3 @@
1[
2
3]
\ No newline at end of file
corpus/extended-ascii/ru/cold/run.json created+18
......@@ -0,0 +1,18 @@
1{
2 "notebook": "/private/tmp/ansi-lang/ru/notebook",
3 "expected_pages": 1,
4 "author": null,
5 "author_timeout_seconds": 600,
6 "inspect": false,
7 "collect_notebook": true,
8 "base": {
9 "file": "win7-office-base.qcow2",
10 "format": "qcow2",
11 "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346",
12 "virtual_size": 68719476736
13 },
14 "scripts": {
15 "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331",
16 "read.ps1": "04013bfcccee40a17e2a225f9b9e96f40a3a8daad9e350609654f8eda3ccbb41"
17 }
18}
corpus/extended-ascii/ru/cold/scripts/cold.ps1 created+27
......@@ -0,0 +1,27 @@
1param([Parameter(Mandatory=$true)][string]$Root, [string]$CloneHost = '')
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
5if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
6 throw 'Choose a run directly below C:\one-tests\runs.'
7}
8if (Get-Process ONENOTE -ErrorAction SilentlyContinue) { throw 'Close OneNote before resetting its test cache.' }
9$key = 'HKCU:\Software\Microsoft\Office\14.0\OneNote'
10if ($CloneHost) {
11 if ($CloneHost -notmatch '^ONE-[A-Z0-9-]+$' -or [Environment]::MachineName -ne $CloneHost) {
12 throw 'The disposable clone hostname does not match this machine.'
13 }
14 New-Item "$key\Options\Paths" -Force | Out-Null
15 New-ItemProperty "$key\Options\Paths" -Name UnfiledNotesSection -PropertyType ExpandString -Value 'C:\one-tests\Loose.one' -Force | Out-Null
16} elseif ((Get-ItemProperty "$key\Options\Paths").UnfiledNotesSection -ne 'C:\one-tests\Loose.one' -or
17 -not (Test-Path 'C:\one-tests\profile-original-cache')) {
18 throw 'Park the personal OneNote profile before resetting the test cache.'
19}
20$cache = Join-Path $env:LOCALAPPDATA 'Microsoft\OneNote\14.0'
21$parked = Join-Path 'C:\one-tests\caches' ([IO.Path]::GetFileName($root))
22if (Test-Path $parked) { throw 'Choose a new run; its parked cache already exists.' }
23New-Item -ItemType Directory -Path 'C:\one-tests\caches' -Force | Out-Null
24if (Test-Path $cache) { Move-Item -LiteralPath $cache -Destination $parked }
25if (Test-Path "$key\OpenNotebooks") { Remove-Item "$key\OpenNotebooks" -Recurse }
26New-Item "$key\OpenNotebooks" | Out-Null
27New-ItemProperty "$key\OpenNotebooks" -Name '1' -PropertyType String -Value "$root\notebook" | Out-Null
corpus/extended-ascii/ru/cold/scripts/read.ps1 created+142
......@@ -0,0 +1,142 @@
1param(
2 [Parameter(Mandatory=$true)][string]$Root,
3 [int]$ExpectedPages = -1,
4 [switch]$UseCurrentCache,
5 [switch]$Pdf,
6 [switch]$KeepOpen,
7 [string]$CloneHost = ''
8)
9Set-StrictMode -Version Latest
10$ErrorActionPreference = 'Stop'
11$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
12if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
13 throw 'Choose a run directly below C:\one-tests\runs.'
14}
15$notebook = Join-Path $root 'notebook'
16$output = Join-Path $root 'read'
17if (Test-Path $output) { throw 'Choose a new read destination.' }
18if ($UseCurrentCache) {
19 if ((Get-ItemProperty 'HKCU:\Software\Microsoft\Office\14.0\OneNote\Options\Paths').UnfiledNotesSection -ne 'C:\one-tests\Loose.one') {
20 throw 'Park the personal OneNote profile before reading test notebooks.'
21 }
22} else {
23 & "$PSScriptRoot\cold-current.ps1" -Root $root -CloneHost $CloneHost
24}
25New-Item -ItemType Directory -Path $output | Out-Null
26$app = New-Object -ComObject OneNote.Application
27$notebookId = ''
28$failure = $null
29try {
30 $app.OpenHierarchy($notebook, '', [ref]$notebookId, 0)
31 $process = Get-Process ONENOTE
32 @{ hostname = [Environment]::MachineName; onenote = $process.MainModule.FileVersionInfo.FileVersion;
33 powershell = $PSVersionTable.PSVersion.ToString(); schema = 'xs2010'; cold = (-not $UseCurrentCache.IsPresent) } |
34 ConvertTo-Json | Set-Content (Join-Path $output 'environment.json') -Encoding UTF8
35 $sections = @()
36 foreach ($file in @(Get-ChildItem $notebook -Recurse | Where-Object { $_.Extension -eq '.one' })) {
37 $id = ''
38 $app.OpenHierarchy($file.FullName, '', [ref]$id, 0)
39 $sections += $id
40 }
41 $deadline = [DateTime]::UtcNow.AddSeconds(300)
42 $previous = ''
43 $lastChange = ''
44 $stableSince = [DateTime]::UtcNow
45 $settled = $false
46 do {
47 $pages = @{}
48 foreach ($section in $sections) {
49 $hierarchy = ''
50 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
51 [xml]$xml = $hierarchy
52 foreach ($node in $xml.SelectNodes('//*[@path]')) {
53 if (-not $node.GetAttribute('path').StartsWith("$notebook\", [StringComparison]::OrdinalIgnoreCase)) {
54 throw 'OneNote opened a section outside the copied notebook.'
55 }
56 }
57 foreach ($node in $xml.SelectNodes('//*[local-name()="Page"]')) {
58 $id = $node.GetAttribute('ID')
59 $content = ''
60 $app.GetPageContent($id, [ref]$content, 1, 1)
61 $pages[$id] = $content
62 }
63 }
64 $signature = [String]::Join('|', @($pages.Keys | Sort-Object | ForEach-Object { $_ + $pages[$_] }))
65 if ($signature -ne $previous) {
66 $lastChange = $previous
67 $previous = $signature
68 $stableSince = [DateTime]::UtcNow
69 }
70 if ((($ExpectedPages -ge 0 -and $pages.Count -eq $ExpectedPages) -or
71 ($ExpectedPages -lt 0 -and $pages.Count -gt 0)) -and
72 ([DateTime]::UtcNow - $stableSince).TotalSeconds -ge 2) { $settled = $true; break }
73 Start-Sleep -Milliseconds 250
74 } while ([DateTime]::UtcNow -lt $deadline)
75 if (-not $settled -or ($ExpectedPages -ge 0 -and $pages.Count -ne $ExpectedPages) -or ($ExpectedPages -lt 0 -and $pages.Count -eq 0)) {
76 [IO.File]::WriteAllText((Join-Path $output 'previous-signature.txt'), $lastChange, [Text.Encoding]::UTF8)
77 $index = 0
78 foreach ($id in @($pages.Keys | Sort-Object)) {
79 [IO.File]::WriteAllText((Join-Path $output ('unsettled-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
80 $index++
81 }
82 $hierarchy = ''
83 $app.GetHierarchy($notebookId, 4, [ref]$hierarchy, 1)
84 [IO.File]::WriteAllText((Join-Path $output 'unsettled-hierarchy.xml'), $hierarchy, [Text.Encoding]::UTF8)
85 throw "Expected $ExpectedPages stable pages; OneNote returned $($pages.Count), settled=$settled."
86 }
87 $index = 0
88 $payloads = @()
89 foreach ($id in @($pages.Keys | Sort-Object)) {
90 [IO.File]::WriteAllText((Join-Path $output ('page-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
91 if ($Pdf) {
92 $pdfPath = Join-Path $output ('page-{0:d3}.pdf' -f $index)
93 $app.NavigateTo($id, '', $false)
94 $app.Publish($id, $pdfPath, 3, '')
95 if (-not (Test-Path $pdfPath) -or (Get-Item $pdfPath).Length -eq 0) { throw 'OneNote did not publish the page PDF.' }
96 }
97 [xml]$page = $pages[$id]
98 foreach ($file in $page.SelectNodes('//*[local-name()="InsertedFile" or local-name()="MediaFile"]')) {
99 $bytes = [IO.File]::ReadAllBytes($file.GetAttribute('pathCache'))
100 $hash = [BitConverter]::ToString([Security.Cryptography.SHA256]::Create().ComputeHash($bytes)).Replace('-', '').ToLowerInvariant()
101 [IO.File]::WriteAllBytes((Join-Path $output ($hash + '.attachment')), $bytes)
102 $payloads += @{ page = $id; object = $file.ParentNode.GetAttribute('objectID');
103 kind = $file.LocalName; name = $file.GetAttribute('preferredName');
104 sha256 = $hash; bytes = $bytes.Length }
105 }
106 $index++
107 }
108 [IO.File]::WriteAllText((Join-Path $output 'payloads.json'), (ConvertTo-Json -InputObject $payloads -Depth 4), [Text.Encoding]::UTF8)
109 $all = ''
110 $app.GetHierarchy($notebookId, 4, [ref]$all, 1)
111 [xml]$finalTree = $all
112 $finalIds = @($finalTree.SelectNodes('//*[local-name()="Page"]') | ForEach-Object { $_.GetAttribute('ID') } | Sort-Object -Unique)
113 if ($finalIds.Count -ne $pages.Count -or @($finalIds | Where-Object { -not $pages.ContainsKey($_) }).Count -ne 0) {
114 throw 'The notebook hierarchy changed while collecting page evidence; repeat the cold read.'
115 }
116 [IO.File]::WriteAllText((Join-Path $output 'hierarchy.xml'), $all, [Text.Encoding]::UTF8)
117 Write-Output "Read $($sections.Count) sections and $($pages.Count) pages."
118} catch {
119 $failure = $_
120 [IO.File]::WriteAllText((Join-Path $output 'failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
121 throw
122} finally {
123 try {
124 try {
125 if ($notebookId -and $CloneHost) { $app.SyncHierarchy($notebookId) }
126 if ($notebookId -and -not $KeepOpen -and (-not $UseCurrentCache -or $CloneHost)) {
127 $app.CloseNotebook($notebookId, $false)
128 }
129 } catch {
130 if ($null -eq $failure) { throw }
131 [IO.File]::WriteAllText((Join-Path $output 'cleanup-failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
132 }
133 } finally {
134 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
135 }
136 $app = $null
137 [GC]::Collect()
138 [GC]::WaitForPendingFinalizers()
139 if (-not $UseCurrentCache -and -not $CloneHost) {
140 Get-Process ONENOTE -ErrorAction SilentlyContinue | Wait-Process -Timeout 10
141 }
142}
corpus/extended-ascii/ru/cold/source.json created+14
......@@ -0,0 +1,14 @@
1[
2 {
3 "path": "Open Notebook.onetoc2",
4 "bytes": 4816,
5 "sha256": "b2f857961b76258a1582b402c048b340e89962257681426b3bfc58df8e0adbfa",
6 "mtime_ns": 1790487072902780320
7 },
8 {
9 "path": "synthetic.one",
10 "bytes": 12456,
11 "sha256": "3d9562625271091ff5c85c24325ab2bf12a670e81aab2caffb6b00c724d7e19f",
12 "mtime_ns": 1790487072903016820
13 }
14]
corpus/extended-ascii/ru/cold/teardown.json created+1
......@@ -0,0 +1 @@
1{"absent": true}
corpus/m6/native-template-controls-01/ORACLE.md+4-2
......@@ -1,7 +1,9 @@
11# Native default page template and unaligned image container
22
3The captured UI operations selected Hearts from the native Templates pane, selected Hearts - Decorative as the section default, and created a new page. The default background appeared automatically; `default-template.png` and `ui-operations.json` record that result. `read/` contains the three native page exports and PDFs. The clone was deleted.
3The captured UI operations selected Hearts from the native Templates pane, selected Hearts - Decorative as the section default, and created a new page. The default background appeared automatically; `default-template.png` and `ui-operations.json` record that result. The clone was deleted.
44
55The section property 0x2c001d62 refers to a separate page object space whose role-2 metadata has JCID 0x2003e. Property 0x1c001ca5 contains the UTF-16 template name Hearts. These native template fields are not in the published MS-ONE object list; the typed model identifies them as observed native metadata. The original personal notebook uses the same relationship with the name Ivy.
66
7The embedded image object starts at byte 11396, four bytes past an eight-byte file boundary. Its payload is 10056 bytes; the footer begins at 21492 after four padding bytes. Native XML exports the same image bytes (SHA-256 2035407a0540e1c4f7934db08ba4add750fcb9a62863ddd9553e7871c81a99e3). MS-ONESTORE 2.6.13 pads the object length, independent of its file position. This fixture exposed the reader's incorrect absolute-offset alignment. The regression relocates the object through all eight offsets and separately corrupts its footer.
7The embedded image object starts at byte 11396, four bytes past an eight-byte file boundary. Its payload is 10056 bytes; the footer begins at 21492 after four padding bytes. MS-ONESTORE 2.6.13 pads the object length, independent of its file position. This fixture exposed the reader's incorrect absolute-offset alignment. The regression relocates the object through all eight offsets and separately corrupts its footer.
8
9The stored picture is Snowbound's rendering of Hearts (SHA-256 6a0593db927c1a34a94cf11093e5bd8700843323ed4fe8c905df0c595cb09c26), written over Microsoft's at the same length with a PNG comment as padding, so the file is otherwise native. `read/` is a cold OneNote read of that notebook, whose XML exports the same image bytes; `default-template.png` still shows Microsoft's art as OneNote drew it during the capture.
corpus/m6/native-template-controls-01/notebook/synthetic.one
Binary files a/corpus/m6/native-template-controls-01/notebook/synthetic.one and b/corpus/m6/native-template-controls-01/notebook/synthetic.one differ
corpus/m6/native-template-controls-01/read/environment.json+2-2
......@@ -1,7 +1,7 @@
11{
22 "powershell": "5.1.14409.1005",
33 "schema": "xs2010",
4 "hostname": "ONE-M6-A33C1CD2",
5 "cold": false,
4 "hostname": "ONE-M6-F782B029",
5 "cold": true,
66 "onenote": "14.0.4763.1000"
77}
corpus/m6/native-template-controls-01/read/hierarchy.xml+1-1
......@@ -1,2 +1,2 @@
11<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{BF109DC2-6C00-44C4-B8B2-9892971AF8B9}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-06T04:25:01.000Z" color="#B49EDE" isCurrentlyViewed="true"><one:Section name="synthetic" ID="{9C3D193E-0294-060F-08C2-1BB16FEA3244}{1}{B0}" path="C:\one-tests\runs\capture\notebook\synthetic.one" lastModifiedTime="2026-09-06T04:25:01.000Z" color="#8AA8E4" isCurrentlyViewed="true"><one:Page ID="{F3B3040C-0089-05FF-2C9F-3BFB91647BCD}{1}{B0}" name="A notebook created in Rust." dateTime="2026-09-05T07:11:28.000Z" lastModifiedTime="2026-09-06T04:18:00.000Z" pageLevel="1" isCurrentlyViewed="true"/><one:Page ID="{CEEEB7B7-FAC8-4A77-917B-DBF7BAF6813C}{1}{B0}" name="Hearts template control" dateTime="2026-09-06T04:21:25.000Z" lastModifiedTime="2026-09-06T04:23:18.000Z" pageLevel="1" stationeryName="Hearts"><one:Meta name="PageLocID" content="00040"/><one:Meta name="LocVer" content="1"/><one:Meta name="PageVerComment" content=""/><one:Meta name="PageOveride" content="0"/><one:Meta name="PageName" content="Untitled page"/></one:Page><one:Page ID="{92E09529-B4AC-4953-828F-1D1072EB4782}{1}{B0}" name="Default template page" dateTime="2026-09-06T04:24:38.000Z" lastModifiedTime="2026-09-06T04:24:51.000Z" pageLevel="1" stationeryName="Hearts"/></one:Section></one:Notebook>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{B9AA08C0-7C3A-40B1-910B-631014E2286E}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-27T08:50:30.000Z" color="#B49EDE" isCurrentlyViewed="true"><one:Section name="synthetic" ID="{E2261F82-C226-039C-33BA-1D20ADCA3D7B}{1}{B0}" path="C:\one-tests\runs\capture\notebook\synthetic.one" lastModifiedTime="2026-09-27T08:50:30.000Z" color="#8AA8E4" isCurrentlyViewed="true"><one:Page ID="{8DA802B0-C03B-006C-17E7-3D6A534474F2}{1}{B0}" name="A notebook created in Rust." dateTime="2026-09-05T07:11:28.000Z" lastModifiedTime="2026-09-27T08:50:30.000Z" pageLevel="1"/><one:Page ID="{B0F5B10B-3A7A-4FE4-AA03-DD6678D68E03}{1}{B0}" name="Hearts template control" dateTime="2026-09-06T04:21:25.000Z" lastModifiedTime="2026-09-06T04:23:18.000Z" pageLevel="1" stationeryName="Hearts"><one:Meta name="PageLocID" content="00040"/><one:Meta name="LocVer" content="1"/><one:Meta name="PageVerComment" content=""/><one:Meta name="PageOveride" content="0"/><one:Meta name="PageName" content="Untitled page"/></one:Page><one:Page ID="{ECFB9395-741E-4CC0-B9F7-1B81B0CB48BD}{1}{B0}" name="Default template page" dateTime="2026-09-06T04:24:38.000Z" lastModifiedTime="2026-09-06T04:24:51.000Z" pageLevel="1" stationeryName="Hearts" isCurrentlyViewed="true"/></one:Section></one:Notebook>
corpus/m6/native-template-controls-01/read/page-000.pdf
Binary files a/corpus/m6/native-template-controls-01/read/page-000.pdf and b/corpus/m6/native-template-controls-01/read/page-000.pdf differ
corpus/m6/native-template-controls-01/read/page-000.xml+1-178
......@@ -1,179 +1,2 @@
11<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{92E09529-B4AC-4953-828F-1D1072EB4782}{1}{B0}" name="Default template page" dateTime="2026-09-06T04:24:38.000Z" lastModifiedTime="2026-09-06T04:24:52.000Z" pageLevel="1" stationeryName="Hearts" isCurrentlyViewed="true"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Title lang="en-US"><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-06T04:24:51.000Z" lastModifiedTime="2026-09-06T04:24:51.000Z" objectID="{40E51230-3D0A-4B15-8CFD-067DA64C81AA}{18}{B0}" alignment="left" quickStyleIndex="0" style="font-family:Calibri;font-size:17.0pt"><one:Meta name="oeLocID" content="0001"/><one:Meta name="oeLocComment" content=""/><one:T><![CDATA[Default template page]]></one:T></one:OE></one:Title><one:Image format="png" backgroundImage="true" originalPageNumber="0" lastModifiedTime="2009-07-28T18:49:14.000Z" objectID="{40E51230-3D0A-4B15-8CFD-067DA64C81AA}{14}{B0}"><one:Position x="-90.0" y="-3.599996089935303" z="0"/><one:Size width="89.21739196777343" height="287.9999694824219" isSetByUser="true"/><one:Data>iVBORw0KGgoAAAANSUhEUgAAAKsAAAIoCAMAAAAhjet0AAADAFBMVEX////d0dncvM/S6+W+4qyw
33t7akbqZ06ir2qrewNLB5OHcxtTr9/fp1eH9+/z9+fvcmL7tyN274avlzdu44d/cvc/jydjP6uTE
45ub25O779/n58/bbscncw9Lbuc2f1q3Y7++03quy3t6i1qncztez3r7gxNXm9Oq14ODu3ufbpsTJ
55+Pcz9jak7v6/f327vPcwdHcydXam7/F5uLjrcvy5u35/PrcvtDcy9bL6OPO6uq/5OT1+/vczdf7
68ve24N/cv9D46/L06vDcxdPw4uqf1aiq0rS84uDbtsyb06jfn8Os27jw+fm64uLhpsew3KrM6dTa
7lrzaocHD5eGm2Knamb20397outTZ79/x1uXbq8fbs8vs9+/n0d7z+vTi8/Pqwdjr2eTcytXcwtK/
85MnN6eTT7Ozd8fHansCy3aq64uC54Kvcx9S336um2bPao8Ok16n03erbrsjbqcWp2anG5s7H5uK7
90sDJ6Oid1Kjf8eTp9fLn9fW+4+Gt26q54cPvz+HmtNDM0cyd0qvZvs/U0dLJobXQ0c+y293Y0dW/
100cPOxtOu0rey0rr0+vnTpbbA0djYlrnd8OzT7NnRnLfLyNS12dy419u71drH0cni8u7SwtHRxNKh
110q7Gzda20r3m9PDY7ujD0cb8/v22yK7h1t3KvrLg8vDWwNDDz9el0rG9rbLNnbb7/frexNTIw7HP
12r7TMubPV7Of7+fq909m2yLnq4uepzbHPpMPWm7i/3Ky6srHFza/d2eSxuq6dzqnn9OC+17LGyLDb
1379Du4Oju5+vKsMi8yrzIytXL6Mru+PW1tq/Gq7Ot267A47r9/P273cTewtPk0dzf1Nvh4uDfq8Xy
147vHp2OLy6e7M6eiq1Kr6/fzFytXQ6sXz+vDr9vTVqsbZ3dm02KvS7OnY1+HD0q7B16339PbBqbPa
157+rCxMDXm77T7Oeu3LjYydLL6ejQpLvKuc3G5snXp7za1dfx+ffgzdne0dq94rCy2dvE2t+lxqvA
16ysDRqrXNtLPU7OkRzzSdAAAAAWJLR0QAiAUdSAAAAAxjbVBQSkNtcDA3MTIAAAADSABzvAAAI95J
17REFUeF7NnXdgFMe9x/eW050k1AtCgBoCgYSECiAJ05Gw6YhmejXVgOk2YGxckzixndhOc3p7qS8v
18eekvyeu9l7zee++9vzd9Z+vM7MzsZf4B3e3dfva33/m1mbtzHJujZrC3urHCJWPGkcnZNs+m8d7r
19T/RQSu/fnt5ZGm9p56Wzq5k5g8AnauycMuW7DjaGLeo9UrE+5dtaeNnsRFLIfMLCWVO9ZW+STclz
201ane2fSLamZIoLrutwPs7NgpFbiEXtNGUn4/aVTXLbWrVUB1ZygbwugLaiKcf7x6lxk9t+qbCX2V
21D7xH9e1NHj8p5QG8g0po2BpZF8ByGZOGUnuvakWzum7J0phZyqjupJoxzB2tbla30dzZld6pJs6s
22Q0PT4ZgYGgsfoXQGcwdHOYG1Eyfn5bzRf7ItgDto7vwq7xRKWcYu8pyEuH96M49bGsEGZ9bQOGdQ
23/r/zhjjY0uSxy3x3dyyOFFJP9w4tzeQ6wrNOxNiUPDzMji1N/sJlLc0nk1FzuQkGqzIjTB3LybV5
24lQg1l2OaNXV+7n1mD072wjE5GJMhDzJLyaDm5lFvYJZ19mS13x3NOBFRMnv14LDYqtz8Msg6GNU6
25cd2K6qATP0HtKphW9EL6iWFNsdb0JqT5M/zJJ82yx/qlzJrLkQhmhrWmV5COzuBtS1mT/KrvKlbh
26G2GEdb1E4lztNaaIpIckrQoOw5mMAdYauQSvh5mWyFXarFQE+qySvRMASJMPzDomb9YcDl7arPKo
27rNeDWS8qsM6DL9COsSqoFBazRiSB8fTwBdq5C/OVLBYl/gd10ZQlgOPsEU0NeOFSjhW1/NChknGA
28mBqy6jbglPo8iBHAon/lwitVBUy2NOuC9ZLW5A9b5iBvLJFgcfKFKbdmveVLmgPcQxPD43j+zBu/
292MbVpbjrruAFSPqit8oRWzqPTYQc/byLFLcHxq21yqwVelMrRgIPxeT6J7lKTyHAkrJL02VFLkwM
30JTjOYSaFh5TtqlnGRrRQBfVTP62epiuzaq5zhVnXCqPRSZw4q7Nq9glDHmutRPa8CsEqsg65uo3t
31IKsMai7XvzYNq+4aV5BV0r9DWEW7rnV1e/CBCCt9flBFSx+L56B+W9s/t5olxEpm/6pmNdZ52nJ1
32/KwqmdOwysG53Lj+Yrc/FkiqFZtW2Mby+d/prqZ3dRxfjG1W8+5KR7dpJgMglfD1fBUjvBLrWl2P
33BWB5R6A2W5RQc/oScBy+M2CRdVxfAn7BWmSdbkACDi5H8LDIOqRZvuAsnROBRVbdvAWzzvbsqubd
34VebWSd1qm1Q/Xhvbns9q00xdaaHmLVXZiwW/plcVeq/2XKywJFC579yxq4zMLEjsGVatkyIP/kum
35zMrFrjb50ysd+evmWJlhLQn2W+ZQHYe5ArU8T9a2v2qSlfU1FfsTcrDf+nmTrA7rwJnyBC+8/hgY
36r78Ar+YrRlEdttfKTOh6/qUiGS8+k8t90Swrqw8UqsN4AXzkWYoK/r3/DwyjOkwFBvIXZlRCfNM0
37LFVBs7ZiP8gZFf93y1HDtNQX6MaDj4RQi8WOw4ZhafmtsHAZIdnnea162PsNw5K+hmJvPcB7f4RZ
384UOGYek2YZ3p9UwMqnFYuv1aqf3it+v7Y1mLhjVLujDpVfB6PGqxw7A3IBlX6ugVp1bsugzPLwKb
39Mt96PsGs4CnTQQGX4FK7rsIe67Fk1uICw5bFsHLrBkFafmZtKAfjyiYfvWkVkOZGmvnFSaD7Qh6N
40hh23eVrDXtZx8OaHFLH2zQzrSUyKRjkH+4hhEdDCVh32RUq1g0PN5y9tsBdrU8NSpHIfKhBCN4M1
41rljQ5ELdQ0XLvpEQbQiggj/3MFjD0QtqahYqbdVgqVwvhVk9WNM+FsLirXBKsCTJ3hOBms9fIZY1
42P7tQ6wjqYEh+dS5HAizxVkHN0glmOCsgjmU22ikin3ThLLs70qz5/AUSFoy7WEILSwXx1nESvV7A
43d/l8DGt+B37ehmBx0xuaVjL3xmn2pjjUfB57rvnGwwF7w0mgWvFWDWhazBo9s9AFXEAH2JlcmBc6
44hGaZHZmPIRR/yPIbGXtZ3q5HX7v3CDL2zTujRuwNd45JTDHM2hCvAWJYj2n/FixhNDpOmfEQg0C2
45EyLvhVhvJ6ASxdLIdQdZlB+nzNh2WY9QCChzSZAruArkCnDCPXovSApte8eIEJz1je5Y4noCCgV8
46Mhg2cQNjXdARgQoeOmUG1hmsTqRFrF2JGsjD4AXtuj+aFDy6xYwOQEbT2zM2HKdbxBodYNkF3ACH
47ANRTsagGYcG6+JHmiehWImJNNmv+ScSahGq2UVezrHEoyriQNSJ19dF3QdZkVBAtTMkAaX/W5Iy2
48UAsBssYlLhQYhK75ryEB3L7RBV3xpR00WeRkYUyzZKKGcWVY8+AWQ6hubg42lPsLc/C0+ZwhgAtZ
49g5VWSL7IepsCuVhDyLY2krHZ1d5Mg8sEUqxXwnH4fNC0hoKC31uDz1I9hNPxxyRZI6/nkq/lAZRi
50dH4x5FlH3DbocyVZYxKxBq6JAHViQwW4MhsDpoVlrEgDl+JzxiCsmawrHLJBO2EY5doi1q6EfKHB
51LwNTmUGIFnxW6aQMa1J2m7/kn2C2DAsaNc2rJOyaHILP+9KEKUMpV/htBt21oOa+IcgHBE/D1IYN
52i4VZr/tVcYwVsPola82uYHPHl7VZaQ8B29Yi6+BfisotCYHwvsAiq/PPwBYSOImHkOaMbbs6s8AZ
53BHWB8FJQRUaGTbs684X1lpCVNJKs29W5KQ5cQlhuJcSqXe+I+gNC0nyeiwdWWUfFBZeQFlZkWejV
54Af0pIYzggMxYgWAFzQzhpXisHVY14CzQn1yeXk0nhbWt9ceX4lF/tdZxOoRVt8iwnh8wuih+9fjm
55gm883QqafyIYwfNsidHgCmNtfQAUUf/9f8WvbUhdhBe3zJm1vs5vUvbX9+mlsCwfMNbOWLc9hhQ8
56/DNS9os7iOZZxjbyHIgnBc98XAOWmdXUil0yaqHwzdSwDbQ6NOWvRKiF70nNSvtapnzAukQBoCc/
57lxIWdpLhMNXTrI1yVQH8lIalYjWF6hwXm7VQ+KE0hqWo80213VbLoBY+n4KVopqaVo6zT4r1FXVW
58kgd0mHJWjlMrhVoo/IkiLO1sbzG4decpP+veywfnbAQ7DNrnHNw1wD+lGA9oW9toC2spBzRysJ3/
59RoD2gxzt/6rY9QJxq/ONdgY5CYwc4kHR/9vPsSt5qzwrXYZ5xFxihQqKqxRm4FqIFD6wiz4vzcpI
60zc0pUvrUE5a9vrvPYTNYObte2IMTgPkpl13WHahfuhRmp6AsObDaX58Rucaiuu07ydVIlIgNO/By
61RsfNdDq9ui+QQ29/ClRRbOC0dSDOqsDAcwiraOHgUjkBPZXOpKuPRyb7+zzjYpA5kVolD+7Fx+yJ
62F8Gl8+WkqNpyM+V8qo0tS+rqqWERxq4kVPcgZmWLyF1Pwr3QZOzp7iYZ6iP3phakjvtJZUlhOzYt
63Tgeh848f7TxrF/Kc833j3tTUnZTmJCZbF1fr4VPXXYXHtYrN6rpIBG+HdgWB895+g0FTDhWc/QBl
64DccAv5EvQ9Y3AL1e2DCV+i4ntIdqk62KTAtgoV13JioAPIkE+4byfMPfpHNFoi6WVKZ3ALFeFrEi
65r/WZroZ/FJ003fNyCXRhHWSNDq7cBRyCrN/I/2Q6FOGros26d1cfGudoLKqDfkAkV9eFrB/7HeFJ
66Ux4QVuvOy7zD33gd48IQK5IAZv3Bz6ZEEb0sVEWPhG/0QWJc4dRCrK/8gOicaZ9nmR72pQPXI23X
67h54ckbLrr6RFEb6OZnoYdW+cIg/BvF/MuhEcxYKy8NyqB/hYd8YnUe0gJIlZoc9qVUWQPt7HmjTP
68AawcK59HSmNIHcizJrv69p1i1uvAX0idNtVBfOMvOYlyD50Tzi0wB+3JFadPeAhRxKFgpFBYl8pk
69ci/yWPuEZhMeMFDYLHfWdEd53f/E8kSICQ8ALut4Ogq5V3l9Sn3Wa3YlwAlWn/WyXQk4Dkte9Fl3
702vQCUCZMBMLsVKRZINdA/0NOhvJHsWRb2w9ctxkI8AU9LetfRXY9V0AVr81Bw8GAiEXwfLvtmQWN
71QJvAmoK9jmpzy4PWBrv0DLu3zl6K5VmA1IdJPUDxZRyy7bAwL21nHBQTxR+xKxOzOg5ZYpEo/mJZ
722wcsZoO+SUCml4Zh+zIyq+OsxpE2oeASOayBpyx7AO/tSe2dOnYdtJq4BsxA0gJx7h9p4PYBe+Vr
73xP3Ckt2bzhX0Pe2srFqZmQrIRoZUKmj/sf8sg2PFmYx4SSs+RRr7lnciUjTuy4YWww4ISu+QSJ57
74m0cK/9eZiRRwxhW/Mhip5efe5UcFSsjEtLizIe6vcNBh1LKyhVuzMG0rkoFKwhUQALbxwmNZwGLN
75ysP+RFAA5O9sYNEeLFnYT74phrUsExnUok7MuYT1bE6t745DzUizuAqX8gbvi0UFml2chWadq1C0
76AxKpARcDwtQrMmF1VqPkQBhu35Jg1syCguMcgKYdEYSweLXii6jKxrJOLSwYBxJN+95ks5aVbcuI
77FexugELYmZDKRIYBH382qQGySCv0tSNxtM9xvvWBR6eBsfyJB/2mzsgX4Nt3IIH2HYyr6TQkRWPJ
78XB/tfZmpANkWKmEkqsD9EqV6gJGC/7Q87IPNxskyi6yG+7Z2Xg75hEhUQLubh81QsaQvcwD2Pfde
799+FS5+qzKtYBB5udK/DUtvopmCbsve6lCZ/CRGtaeAUQ0XKwdzNVLD0Zxj1H3QIpBxaFUX2WzXZ2
80cYZBO/nw7tfnsO3CCkDsnmYXlsSunheD0QzLdW6EAhDsA0wGmSSycQYBS+MgFf8AgqmMUgB8rGUN
81hT1bQsOC4FsHdmrhvMULAkHo5ZS1ZIIlPmx74RoKsDFqRdyVBHZmSe0KW6B/hUginQCxcAvNDUrM
826rR+DaI+GKdWX0goNavzx5D1iUTWacSwJXUE0FCdkHV5MitxsllVB7H37xZ0rsmo01rw7Co560wA
838bCAdRpOD/2sK6uOdXZWZcq/AkDsFrHihIvjOnZrIXW7M89mltpKyHXatNM+1pWdDBQDZ9RWduC5
84RGadhj0B7sYuDpLCp7LJxMGJmsSsSLAI9e42evN9/67IwKGtBGcUeFcaZ2FSuPhMJCl4MINObZXM
851MI5N8gHVsKZGDes52GQNSkZIPJYBGeQszUwpwLUthMxyBqfDzIhQ9azx5JRrc8wyCqeWtOQXRPu
86P3nKbsNegRXzzH24chEYj1ayeoG/BKvFLmCVcFnIrnA0LfFuwulKfxsJPm91kUmJtSkwC1u8wpEa
87d4XFgLtYzq6PwrsfkTYsCpn2jMWMPLnWoncc1FxrIt2FV+ZSy1rMvBLKbc4/PFD2QEwDoaUp4B62
882VOBHGtTbP9g2rSgQ7CXx8yMb2NwduWmf8gbB2Ww0NrquBxrYrhgRTlRg7VYe0bKrsmhbXnAG9hS
89bGdiz0Ui/MJD+KYyzBws+a27UnFLxOxfW7C1KFplhLXFrwJbs8sIq38dxJoItok6GaL7j5/3Ld7d
90siTYmTL5q5jXN71sCbZTpi4Qs07zKdaSXY/J1FsSrE/weYEl1q3iFpEE6TSWjiNkS6zOwoS8RIqS
91HJSBXZ2ZMkWMBDSfHNqya2dyB16CEh/CFTTW1u2qyuLW4aQx0YF0vQZowd6SjSFHwLFaTLfNTC6O
921V7N1SnswUuJwWO1uNdgq5mMwPMD9iTgOAsFa0ZSZuWCrK2UEHrC+3bL0SQexdaYy2y2M5xjJgTL
938gG7e7kWm0gLWQJrtVfoOLfQbjetwfJXaxU3idtnJZY3kq+EFVy20myWYazUTgloMmCzp0lwV2h6
94LaqADFCdTr0wSxsvWaA6W9foTCyKestWc8ifDm+TWDiKuxyKajO08rQaoetRXMFus5dcBaqM9KGL
95pFdnsrn/EJvmWsslFjx5LZzG2dXMzIwKYfH+nJakTWUResVLXNvsrheGCs0VS1B9pxRsl6AcIGtS
96x5kJPeyjKvELk97K9O5jE98HuwRN0ons6UpIuu2szbQ6tsnQCViXS1bfLUtgH3tFZwb7RiJ5zwDW
97J2Ry7uW7wdRfceZudj4qxDsTzK0HoQRe/dDjcLzn1b8IzftFSyqbts08c7YEEuV5P/rXcBfW8i88
98/lH+0Zdffvn3EPnjn/jEJ85WVVWV0JiM66e+AIy4pOxDPlJbDbSk962q6uz8u9/4o/9bsmgRTlM/
99/Oqr73n88X95+bfwq17+0IfRw5UWexDi615c1XlrW1lT5aLY7tqHX/VUWWmvZSZC3doJdlPNVYhE
100TSVivXsf3KCkAAo1UArWlWfgpr+5lYp91RKwVsGtwmUPolxEaSzJem5VwY3C6jaFF7Xc2vJJ5NTC
101pGXcZ/FULJvNZ7cx+GK8jS5qc5IU8sPZ5cx38d7ENan7Ertt96KoFohRIz/fJmVVkBBkNLm2km20
102igWT/yoezCR1riJ7U/XWAJ+wtVOFdwTHyGJo1AccJQWAvFYGkYuizk09rcgFzbWeoMINt2hIl3Zx
103tn7AttdaSfdRazX7EP6jtnaqUMGyjfSK/Z0I47ZYW6HGsOgTWChdUZhFcYc2WV1FgZ+8MKRWcAF2
104QxfOVuDQaPcyMy+3Gbo8s5pZAZ5rsZnifUJBL2RRyz5sMXR5n/uQafCIZ5/FQoaFAbl912LWRfaK
105A+awTLG22Pu4pucFzOwNBQ24TlHjIe3z3Ie/zMytafY6Gsy5gv+IxShzRKW1MMuz6iaE+EqWWKtm
106eVbtTQCIdVGZrbyQ/1ididwFLHFZ21vD+QGpDzlKKNbabkDfhwDNeII1tibXWV6wZjYxNpVZKrq2
107+lj1ixigkSZrkcv/8Wrt4hCwVlpzBP4PWGsX3YjVVpT1cm2kBs1mBnQTlWXWilmf15L7pGOy46q0
1085rQcLoNFlk3cvdQiUZNVWvyKv4Bhy5bEm61FxlGADSxpsz7h6wKKLYuHbVlTalavmUF9bYxlwT4q
109mZoM2NViLRv6toXIjAtuTpLZigeOs7joHv6+hfCawSL0kdvSszq0/8oF3Cd8i4anyZ5PmW1CYNuF
110RbuCnTW+tAD/wRZjT+9mH2OW6STaZnWCjgvjrmkCg35gDapahhVcmFW7OouTvs0Eca+4K8kKDrPL
111KoRdsRimuhJ6hV9AZZlVAHvfYgdmZBJ+AH41hm3WRFjY/IOSlmCFW6+tszrIclFjG4pDMCuXqMjg
1125y8yYHXI6nGAtxPXT/BRibIcbmjNgjXqe5jYF3GhKxBvK4FHCfMlMwesPMO3N7Zxv5uBWBMSRpxK
113oo8LmUGReZeqzpnQ28685dtICb7JBQzhMjj6fj+Zs9g8BlfnwgUQuFs0gxVkwZXiGScSAYzI3y6s
114Aq8Fv8zHWs0tLxviyZLTF5Q7/kMWv2uWCE46NMmGRUnZ156Wt4CdI2namJS/4M82/X5hnx0E6Xel
115EfjBhHiAPoXxJndvqWFZ+zPex55GZn23u3GgsNTyr1wmm9hrf8b6LVyVfcB1D+0s1GX1s3GR1Cz4
116xnUTycdx3wK+dbv9XKGwub50/sArIGMkSz6R/Rz63Zg54DePC/us/zBnjBa4yjyy9Um+6vdd9Cdu
117Nl4Gvz9fVyJcLgOLgKUfxfsU93M813aB33Sv25fpb8hhS/NdhNBXzbGvF3uv/9e5EO7m+qzdgq+j
118ONcfbBnqm0I/JNZ+EPwGfeHpjI3r73jwbaTlrDXztqgfPZsD3EJhcwY/1epNtUAPfO5uEsJauO+6
119gB4rYmzclTUt2iLPj4crweC/VSwsAQqOnNjm7JTAth4GmdnfvBcImvca8GEZBl9uf0w07ieTfm+y
120/TL82anMwll0Q5FxR84sDn8ONG1WQhCo4H2in/Fsh3OscDybTAG2NmPHl0So4PnrEHb7Oum8WedA
121/zK+HztRrcwhgEBWqMvG2cZ/STLIXGXGIQhbOK5jMOnXxsG+E2eD4oFhsynKYmADWUsCM4bdnskM
122i4R9h9ig7IgsYSMmmAoqqMigYTOybFXwu/3VUF33YIaw/q/3f6cwCIQEguJtIaMWzUr0WU84/uO3
123ZT0ATwwz8Ky8AfBxdztvgZby96b7/XnQ8YDjKWlfaeLAzSl/fh5F20Ihu5TWcepT/u6468LsG4Tb
1247OrG2rqUZnXdjdiwS03cXKn3OJ5OrWiWYV9QyKr1tbqQ2qyg7YWnVyGbDNHZN6AQWkOH9mHW7VJ3
125UPeg1eJf7U26FqLYbBzXvoLqz8770WGTA/qCDFKu1YVzOhJw3WuYNYvMu15nZqGrJKwF6062tq4g
12691vj8cYnIrBfJRzQlYDrEk9gX7HbtSXgzqEisJzDrCtoegEgDca6Wdd9Jr/+eCF12sIE3E7tajl4
127wd8O1R6M1WrD4GqhcE0bFReJ1gMt+PV2vaAFL5TptVCwGbvqCuK8Reh+qc8ClrVYIAAvMCKUwGUR
128LK4Q0bDotfBvCAtGn+AQmmhBVospN/jla/jjzMmjT3AMLrnwsMdaC95dXBIAOV5PuBpSymJWe32N
129VvDu9PfE42ng1ImXAW4U2bcrkKsEK1oeGImZYJwPsKqBz/6TFCvW40CUDtDiHD+steV/+BUF1kJh
13058GAbQ8GSe1Vs7/7h9AiYr2Cg45j2w3sOoh+gB5U2nP6aIbNm9VWojX67x+TYoXus5XAIq6REdIS
131CNx/8Kcll3X05/I/LcUKS79W5+kwWPgRS6Xs4Z/Nd30Dni3JdaLbDd1Aq1MLwoZwWDHr6NSefP7J
132P4cnF6avcPkVpCQSsDYacEenOm7k8/nyNyD1CSIsSvhg+lQrkoH++tHoAt/YP3VqS7G7C6DmryBW
133UcWNpjtO9WDoiB9LNXLXo3em7s3vKIZG9w1EyliT6wKcR5O0tHVzPGpqrS6YIpS3u8vLu7oIHEZk
134o/w70ZmTO0TY3dOOSm19XTTtvnQ9l9H997Atu8u7GgJ4flZy2qQihiRRXi0cRbv5eCpSCrppz/lE
135TgjdhfWaZFjSVvOHo6v7OCnULa1P1yU+egrrs3tHkj3Zc4Q1vpKdQwJUyBnVtrbWg3G1Ne18WjAf
1363/s9F6RI8/m3EhEMkCAf9F4si049b2L6KJS0XHjv2ZVQ1sJAVAKz0cuizG4rPUpsekXWphD4u705
1373RdMpdEmNzrS3ukos45O4bt/O9o5xUni4xzOQB/nDub0cWW02SJqwSMY9Yb87Uf43/T7yp0jl/v6
138+i6P+DjhIQaTfWLUTeclp5R3WFLI9J6rM9apHN1C7v8lZdT8/0jBGvMCh0nE36B4/9GFfU6G1VgC
139rYWa/zosDkXDlFop6qY0VgWG/bwI1NwKNkUtptAqEsGfCg272ZBvZajl6tOKvIILB5EmrkuXmIQ8
140B/UAxQ2pUb2cIFoMplCdUzTlV4tW/gv7+tsTJKtTlvhMu4CidmuYNZ//TVwdRAyDHzMigbVYlEtW
1414y7o/Pd/Jho1ZVkSFeT2s6JPy6z5hmLxx5eGaLc/ZWj+I3RmVj0J5PPdxQ5n3XGurbJ934FU9VNs
1422nCHmTW9w8I3pLxYPAxP0wrqkqutrWYxET+pVQHxk3oayHcVi1PGMqnIN/KaFDoeC11msbjFLqvX
143TtFm3VAsjlqF9Vh1NZDfUyzeyYhVd27lnywWb2bEquuz4OSanxFrUdMPwGhQrLEJy7Uq1WvCwNWB
1449/rljFi1RdBdLP63TcPyLWBdrwVYv1xt0bA8q65hbxSLX3UH7cGy1AVCayoWZATF5h57KrjJG3aT
145Ssct7DYg69+6R6wZ9jDPqlVxoUyr+Eyba0+ypIlJkPckONkGkUQg6+s5i7BeAotw42EbNoj8xHnw
146+sdyAPaILc36DRubxjbsKUqy5oabeyx5A79i4yzbADI+ESvUwIs5MOYBHcyyMsV8rgDKIKKpdQmg
147FkVeArLeD1kt0pLOK/MIG0JtrR2b4JOixhzHmsv1D689st68bY+GVlv9XfgLIHjCIaMBYlds3Ivf
148VW0cd4HPycI/Nt1gtj1/BT7QAaagKqsdXK+h4VHfvlJeXn6jG939YsdhCVZoft6uRLoXf9GsdaNg
149OWN3HHaAqEVVTjQrIB5vqzgx25x2E2EBqgPIbwgqB3gLwnZF1u2/ODZj0liESICFqJBV1J+FNyKG
150FeAOj7nVpoy7IGLvBZLBFtT6ETstUBsmsSLaRkN+ga7FBpzCKdyggI8mdz1BzZ3Mimh7lpkR7lTY
151tI/Q9gQEuZIoWNDLELHmctObTdEeZf14bNyOKdb1QX8nRi4YhhP0ij1Yrr/NdQ2lNqOveRH33n7u
152diHWJK91AR0BckLRGB9z3UZDs2x0wRQYry3w6wqljklrdTtkWXP9E2DnxgljHiysf5zmJhgWheLi
153R0RWRc+PN7tuhSGXEDFVyYp97Moi7BCB8YwUa65/CJi20U6O6zgkVsTGAyyB4vNyrMAhANiKSTP+
154K/guRzFLbKBFXqBYlEUFvhbowFZlRrse0QEBBS0Jl8VdyyoI22PIIfhNm7wUStLxl+TtmsshWNeG
155DlhxvilifhG1Ft+swprrXwthbfQ+vNZXSAYNOCMvFt+oxEpgZ5h3tcRrRblZMrGKz6qhgoiLLDvD
156uPMa5TKbbj4zAG0OMlB3QGlgzVYYn2GcYYubvCWmC9SqRUW5oouyBMsblpW6l5hRVSIBZ/txtK3T
157uGVfo/ea/nvbMyl86NNKt58ePGwHNtihCbCreSx2YSCjtWBZtuMoaGD89wup7EqcgVth2BskluZK
158QYu/rHnIGbim/WywqcgbWDEQcLRYsu4Mw2lXoCLjWONbA2JpPIRhTYfb2PklnbpGoPdjFZhOZEa9
159bSe+GSZRFSbY9yJmNb6UFymDT6d0ApQfJQYgJpjOY/hgS2z7bPqJhWlx+AJFmOH55YR6X9qouRyZ
160XqYl6zijv+DTqgHU3DxiWNd4zvUdz9zv0b6oqVWsAmpY0172z8B7P//BT0Pc97+kK1UyvahiTavg
161K2L3rn4EcQWGE4MvqoNIvIJEWsPh60clzqx+SD+dXSYjQs2/qYPIvILOLpNO9kdkTpziGCYCg4b9
1621xQcMi9hLtacYZetkjlxmmNItmUwh2lMgyH1GtiVxcPQDp9ZE1LnTXMQbMqSYab4OnEyDYbUazjW
163E0byrQqp06Y66KRn1x4TrOuHUmFIvYilBIDZRLpVPV3qtKkO4lkNiKDGHU+FIfUiWnVBKRgQwTK3
164X+q0qQ7i5pbr6nuCI2tTUci9iCUE0LD6a3XuQ3KnTXWUF7cAa6+uJ1jvqk2ti/MUoFd5LstERVut
165OLWmr1WQN25u0qFdd/W4aonLdLdN2rBemoVxNTUwy3WlT40OBF5IGtY3s/RZl6myQu8uCetl2kQF
166mnatdhUjLIpEUrD+iWVAAz2qrDgSPSSeYHD7Q2Do2RXIVdGuJBKtFc1IProS5Ao91vXARGpzi0XN
167RLc8zysIPNtqdgx7XcVQgLZf4DE2HHeVYMt01NBMtBqVWaEf6mG0UbIdDroqyq2ZD1Qos8KbWzFJ
168T9/cNuyLueMX40DBKzQ73HBKq+kVUTpHuHvcPNQ2HY6JIdpri1SA7prMIHhXNT+Aw6ZTMyOaJ+lR
169zY8ogKilyHoSszqzgHrUhm6DALgBd0xJA3A3HkpCZivCai/OnoAnVmIF2xwxqyqs9i5Z4LJcpZyQ
170xHgUgJQsq+lbwekQa6xLjzA4lgBJRGfJTzAD67LoZCrdLJKPkMBew7uupJlmYnsZen+FyUUzUpaE
171TMrMMDO7jrEt5Ks9PLP4YmQWklHi6NWMV8Qw+BzShSzNW3xF3mAyrbHPfWFWWRGQnSGh5eB4WoMf
1727XDIvZP0BCzch2b1rMmwS6g4MqnfFOKyc8IqZ1gvKY1qoNSs7z1C1dDY2LvMRAfTV0fQOSGjWC5/
1731kxE05Uy7M6J25p8qm/03sqSsyncLCj1+vkUWrPGk4ULHOe5m2RY+MkMb+hmd+lgYU5IRnO8DOYF
1746hLtlCkVLMy12WiL7k+Mh4rSksjVgTUMZ9qJULQdn/DdfXSsZpmfyqjwRcFAPjYxTLSwavziRFRD
175wnVLIwGSwCanHsFnS+MFgF25ySVLrN31TyuC2bKE7Djje+7k0Wm7R5q5ZGZVF4GBxT95QwaOrJE2
176KD7QePqkQl6tBFtCBYCLqpGp7uj1GCidVewYOpb1J8UG1l5M0wKFLxZXouQqGs0UpDrAsiootQDk
177G1MVpUoDArdBootm7aOvyoqoEWi2pyTVYNxlJDWmGr9Nbj9jr+mNdrQ9J0pTBiRro2bZkQDujOpl
178NkH/HxtpiBH77U/NAAAAAElFTkSuQmCC
179</one:Data></one:Image></one:Page>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{8DA802B0-C03B-006C-17E7-3D6A534474F2}{1}{B0}" name="A notebook created in Rust." dateTime="2026-09-05T07:11:28.000Z" lastModifiedTime="2026-09-27T08:50:30.000Z" pageLevel="1" style="font-family:Calibri;font-size:11.0pt" lang="en-US"><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="Fixture Author" lastModifiedBy="Fixture Author" lastModifiedTime="2026-09-05T07:11:28.000Z" objectID="{E2261F82-C226-039C-33BA-1D20ADCA3D7B}{23}{B0}"><one:Position x="36.0" y="36.0" z="0"/><one:Size width="191.1661224365234" height="13.4277172088623"/><one:Indents><one:Indent level="0" indent="0.0"/></one:Indents><one:OEChildren><one:OE creationTime="2026-09-05T07:11:28.000Z" lastModifiedTime="2026-09-05T07:11:28.000Z" objectID="{E2261F82-C226-039C-33BA-1D20ADCA3D7B}{24}{B0}" alignment="left"><one:T><![CDATA[A notebook created in Rust.]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/m6/native-template-controls-01/read/page-001.pdf
Binary files a/corpus/m6/native-template-controls-01/read/page-001.pdf and b/corpus/m6/native-template-controls-01/read/page-001.pdf differ
corpus/m6/native-template-controls-01/read/page-001.xml+177-177
......@@ -1,179 +1,179 @@
11<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{CEEEB7B7-FAC8-4A77-917B-DBF7BAF6813C}{1}{B0}" name="Hearts template control" dateTime="2026-09-06T04:21:25.000Z" lastModifiedTime="2026-09-06T04:23:18.000Z" pageLevel="1" stationeryName="Hearts"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:Meta name="PageLocID" content="00040"/><one:Meta name="LocVer" content="1"/><one:Meta name="PageVerComment" content=""/><one:Meta name="PageOveride" content="0"/><one:Meta name="PageName" content="Untitled page"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Title lang="en-US"><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-06T04:23:18.000Z" lastModifiedTime="2026-09-06T04:23:18.000Z" objectID="{37F0B5A1-103E-4EE9-9AA9-94166A9D8F91}{18}{B0}" alignment="left" quickStyleIndex="0" style="font-family:Calibri;font-size:17.0pt"><one:Meta name="oeLocID" content="0001"/><one:Meta name="oeLocComment" content=""/><one:T><![CDATA[Hearts template control]]></one:T></one:OE></one:Title><one:Image format="png" backgroundImage="true" originalPageNumber="0" lastModifiedTime="2009-07-28T18:49:14.000Z" objectID="{37F0B5A1-103E-4EE9-9AA9-94166A9D8F91}{14}{B0}"><one:Position x="-90.0" y="-3.599996089935303" z="0"/><one:Size width="89.21739196777343" height="287.9999694824219" isSetByUser="true"/><one:Data>iVBORw0KGgoAAAANSUhEUgAAAKsAAAIoCAMAAAAhjet0AAADAFBMVEX////d0dncvM/S6+W+4qyw
33t7akbqZ06ir2qrewNLB5OHcxtTr9/fp1eH9+/z9+fvcmL7tyN274avlzdu44d/cvc/jydjP6uTE
45ub25O779/n58/bbscncw9Lbuc2f1q3Y7++03quy3t6i1qncztez3r7gxNXm9Oq14ODu3ufbpsTJ
55+Pcz9jak7v6/f327vPcwdHcydXam7/F5uLjrcvy5u35/PrcvtDcy9bL6OPO6uq/5OT1+/vczdf7
68ve24N/cv9D46/L06vDcxdPw4uqf1aiq0rS84uDbtsyb06jfn8Os27jw+fm64uLhpsew3KrM6dTa
7lrzaocHD5eGm2Knamb20397outTZ79/x1uXbq8fbs8vs9+/n0d7z+vTi8/Pqwdjr2eTcytXcwtK/
85MnN6eTT7Ozd8fHansCy3aq64uC54Kvcx9S336um2bPao8Ok16n03erbrsjbqcWp2anG5s7H5uK7
90sDJ6Oid1Kjf8eTp9fLn9fW+4+Gt26q54cPvz+HmtNDM0cyd0qvZvs/U0dLJobXQ0c+y293Y0dW/
100cPOxtOu0rey0rr0+vnTpbbA0djYlrnd8OzT7NnRnLfLyNS12dy419u71drH0cni8u7SwtHRxNKh
110q7Gzda20r3m9PDY7ujD0cb8/v22yK7h1t3KvrLg8vDWwNDDz9el0rG9rbLNnbb7/frexNTIw7HP
12r7TMubPV7Of7+fq909m2yLnq4uepzbHPpMPWm7i/3Ky6srHFza/d2eSxuq6dzqnn9OC+17LGyLDb
1379Du4Oju5+vKsMi8yrzIytXL6Mru+PW1tq/Gq7Ot267A47r9/P273cTewtPk0dzf1Nvh4uDfq8Xy
147vHp2OLy6e7M6eiq1Kr6/fzFytXQ6sXz+vDr9vTVqsbZ3dm02KvS7OnY1+HD0q7B16339PbBqbPa
157+rCxMDXm77T7Oeu3LjYydLL6ejQpLvKuc3G5snXp7za1dfx+ffgzdne0dq94rCy2dvE2t+lxqvA
16ysDRqrXNtLPU7OkRzzSdAAAAAWJLR0QAiAUdSAAAAAxjbVBQSkNtcDA3MTIAAAADSABzvAAAI95J
17REFUeF7NnXdgFMe9x/eW050k1AtCgBoCgYSECiAJ05Gw6YhmejXVgOk2YGxckzixndhOc3p7qS8v
18eekvyeu9l7zee++9vzd9Z+vM7MzsZf4B3e3dfva33/m1mbtzHJujZrC3urHCJWPGkcnZNs+m8d7r
19T/RQSu/fnt5ZGm9p56Wzq5k5g8AnauycMuW7DjaGLeo9UrE+5dtaeNnsRFLIfMLCWVO9ZW+STclz
201ane2fSLamZIoLrutwPs7NgpFbiEXtNGUn4/aVTXLbWrVUB1ZygbwugLaiKcf7x6lxk9t+qbCX2V
21D7xH9e1NHj8p5QG8g0po2BpZF8ByGZOGUnuvakWzum7J0phZyqjupJoxzB2tbla30dzZld6pJs6s
22Q0PT4ZgYGgsfoXQGcwdHOYG1Eyfn5bzRf7ItgDto7vwq7xRKWcYu8pyEuH96M49bGsEGZ9bQOGdQ
23/r/zhjjY0uSxy3x3dyyOFFJP9w4tzeQ6wrNOxNiUPDzMji1N/sJlLc0nk1FzuQkGqzIjTB3LybV5
24lQg1l2OaNXV+7n1mD072wjE5GJMhDzJLyaDm5lFvYJZ19mS13x3NOBFRMnv14LDYqtz8Msg6GNU6
25cd2K6qATP0HtKphW9EL6iWFNsdb0JqT5M/zJJ82yx/qlzJrLkQhmhrWmV5COzuBtS1mT/KrvKlbh
26G2GEdb1E4lztNaaIpIckrQoOw5mMAdYauQSvh5mWyFXarFQE+qySvRMASJMPzDomb9YcDl7arPKo
27rNeDWS8qsM6DL9COsSqoFBazRiSB8fTwBdq5C/OVLBYl/gd10ZQlgOPsEU0NeOFSjhW1/NChknGA
28mBqy6jbglPo8iBHAon/lwitVBUy2NOuC9ZLW5A9b5iBvLJFgcfKFKbdmveVLmgPcQxPD43j+zBu/
292MbVpbjrruAFSPqit8oRWzqPTYQc/byLFLcHxq21yqwVelMrRgIPxeT6J7lKTyHAkrJL02VFLkwM
30JTjOYSaFh5TtqlnGRrRQBfVTP62epiuzaq5zhVnXCqPRSZw4q7Nq9glDHmutRPa8CsEqsg65uo3t
31IKsMai7XvzYNq+4aV5BV0r9DWEW7rnV1e/CBCCt9flBFSx+L56B+W9s/t5olxEpm/6pmNdZ52nJ1
32/KwqmdOwysG53Lj+Yrc/FkiqFZtW2Mby+d/prqZ3dRxfjG1W8+5KR7dpJgMglfD1fBUjvBLrWl2P
33BWB5R6A2W5RQc/oScBy+M2CRdVxfAn7BWmSdbkACDi5H8LDIOqRZvuAsnROBRVbdvAWzzvbsqubd
34VebWSd1qm1Q/Xhvbns9q00xdaaHmLVXZiwW/plcVeq/2XKywJFC579yxq4zMLEjsGVatkyIP/kum
35zMrFrjb50ysd+evmWJlhLQn2W+ZQHYe5ArU8T9a2v2qSlfU1FfsTcrDf+nmTrA7rwJnyBC+8/hgY
36r78Ar+YrRlEdttfKTOh6/qUiGS8+k8t90Swrqw8UqsN4AXzkWYoK/r3/DwyjOkwFBvIXZlRCfNM0
37LFVBs7ZiP8gZFf93y1HDtNQX6MaDj4RQi8WOw4ZhafmtsHAZIdnnea162PsNw5K+hmJvPcB7f4RZ
384UOGYek2YZ3p9UwMqnFYuv1aqf3it+v7Y1mLhjVLujDpVfB6PGqxw7A3IBlX6ugVp1bsugzPLwKb
39Mt96PsGs4CnTQQGX4FK7rsIe67Fk1uICw5bFsHLrBkFafmZtKAfjyiYfvWkVkOZGmvnFSaD7Qh6N
40hh23eVrDXtZx8OaHFLH2zQzrSUyKRjkH+4hhEdDCVh32RUq1g0PN5y9tsBdrU8NSpHIfKhBCN4M1
41rljQ5ELdQ0XLvpEQbQiggj/3MFjD0QtqahYqbdVgqVwvhVk9WNM+FsLirXBKsCTJ3hOBms9fIZY1
42P7tQ6wjqYEh+dS5HAizxVkHN0glmOCsgjmU22ikin3ThLLs70qz5/AUSFoy7WEILSwXx1nESvV7A
43d/l8DGt+B37ehmBx0xuaVjL3xmn2pjjUfB57rvnGwwF7w0mgWvFWDWhazBo9s9AFXEAH2JlcmBc6
44hGaZHZmPIRR/yPIbGXtZ3q5HX7v3CDL2zTujRuwNd45JTDHM2hCvAWJYj2n/FixhNDpOmfEQg0C2
45EyLvhVhvJ6ASxdLIdQdZlB+nzNh2WY9QCChzSZAruArkCnDCPXovSApte8eIEJz1je5Y4noCCgV8
46Mhg2cQNjXdARgQoeOmUG1hmsTqRFrF2JGsjD4AXtuj+aFDy6xYwOQEbT2zM2HKdbxBodYNkF3ACH
47ANRTsagGYcG6+JHmiehWImJNNmv+ScSahGq2UVezrHEoyriQNSJ19dF3QdZkVBAtTMkAaX/W5Iy2
48UAsBssYlLhQYhK75ryEB3L7RBV3xpR00WeRkYUyzZKKGcWVY8+AWQ6hubg42lPsLc/C0+ZwhgAtZ
49g5VWSL7IepsCuVhDyLY2krHZ1d5Mg8sEUqxXwnH4fNC0hoKC31uDz1I9hNPxxyRZI6/nkq/lAZRi
50dH4x5FlH3DbocyVZYxKxBq6JAHViQwW4MhsDpoVlrEgDl+JzxiCsmawrHLJBO2EY5doi1q6EfKHB
51LwNTmUGIFnxW6aQMa1J2m7/kn2C2DAsaNc2rJOyaHILP+9KEKUMpV/htBt21oOa+IcgHBE/D1IYN
52i4VZr/tVcYwVsPola82uYHPHl7VZaQ8B29Yi6+BfisotCYHwvsAiq/PPwBYSOImHkOaMbbs6s8AZ
53BHWB8FJQRUaGTbs684X1lpCVNJKs29W5KQ5cQlhuJcSqXe+I+gNC0nyeiwdWWUfFBZeQFlZkWejV
54Af0pIYzggMxYgWAFzQzhpXisHVY14CzQn1yeXk0nhbWt9ceX4lF/tdZxOoRVt8iwnh8wuih+9fjm
55gm883QqafyIYwfNsidHgCmNtfQAUUf/9f8WvbUhdhBe3zJm1vs5vUvbX9+mlsCwfMNbOWLc9hhQ8
56/DNS9os7iOZZxjbyHIgnBc98XAOWmdXUil0yaqHwzdSwDbQ6NOWvRKiF70nNSvtapnzAukQBoCc/
57lxIWdpLhMNXTrI1yVQH8lIalYjWF6hwXm7VQ+KE0hqWo80213VbLoBY+n4KVopqaVo6zT4r1FXVW
58kgd0mHJWjlMrhVoo/IkiLO1sbzG4decpP+veywfnbAQ7DNrnHNw1wD+lGA9oW9toC2spBzRysJ3/
59RoD2gxzt/6rY9QJxq/ONdgY5CYwc4kHR/9vPsSt5qzwrXYZ5xFxihQqKqxRm4FqIFD6wiz4vzcpI
60zc0pUvrUE5a9vrvPYTNYObte2IMTgPkpl13WHahfuhRmp6AsObDaX58Rucaiuu07ydVIlIgNO/By
61RsfNdDq9ui+QQ29/ClRRbOC0dSDOqsDAcwiraOHgUjkBPZXOpKuPRyb7+zzjYpA5kVolD+7Fx+yJ
62F8Gl8+WkqNpyM+V8qo0tS+rqqWERxq4kVPcgZmWLyF1Pwr3QZOzp7iYZ6iP3phakjvtJZUlhOzYt
63Tgeh848f7TxrF/Kc833j3tTUnZTmJCZbF1fr4VPXXYXHtYrN6rpIBG+HdgWB895+g0FTDhWc/QBl
64DccAv5EvQ9Y3AL1e2DCV+i4ntIdqk62KTAtgoV13JioAPIkE+4byfMPfpHNFoi6WVKZ3ALFeFrEi
65r/WZroZ/FJ003fNyCXRhHWSNDq7cBRyCrN/I/2Q6FOGros26d1cfGudoLKqDfkAkV9eFrB/7HeFJ
66Ux4QVuvOy7zD33gd48IQK5IAZv3Bz6ZEEb0sVEWPhG/0QWJc4dRCrK/8gOicaZ9nmR72pQPXI23X
67h54ckbLrr6RFEb6OZnoYdW+cIg/BvF/MuhEcxYKy8NyqB/hYd8YnUe0gJIlZoc9qVUWQPt7HmjTP
68AawcK59HSmNIHcizJrv69p1i1uvAX0idNtVBfOMvOYlyD50Tzi0wB+3JFadPeAhRxKFgpFBYl8pk
69ci/yWPuEZhMeMFDYLHfWdEd53f/E8kSICQ8ALut4Ogq5V3l9Sn3Wa3YlwAlWn/WyXQk4Dkte9Fl3
702vQCUCZMBMLsVKRZINdA/0NOhvJHsWRb2w9ctxkI8AU9LetfRXY9V0AVr81Bw8GAiEXwfLvtmQWN
71QJvAmoK9jmpzy4PWBrv0DLu3zl6K5VmA1IdJPUDxZRyy7bAwL21nHBQTxR+xKxOzOg5ZYpEo/mJZ
722wcsZoO+SUCml4Zh+zIyq+OsxpE2oeASOayBpyx7AO/tSe2dOnYdtJq4BsxA0gJx7h9p4PYBe+Vr
73xP3Ckt2bzhX0Pe2srFqZmQrIRoZUKmj/sf8sg2PFmYx4SSs+RRr7lnciUjTuy4YWww4ISu+QSJ57
74m0cK/9eZiRRwxhW/Mhip5efe5UcFSsjEtLizIe6vcNBh1LKyhVuzMG0rkoFKwhUQALbxwmNZwGLN
75ysP+RFAA5O9sYNEeLFnYT74phrUsExnUok7MuYT1bE6t745DzUizuAqX8gbvi0UFml2chWadq1C0
76AxKpARcDwtQrMmF1VqPkQBhu35Jg1syCguMcgKYdEYSweLXii6jKxrJOLSwYBxJN+95ks5aVbcuI
77FexugELYmZDKRIYBH382qQGySCv0tSNxtM9xvvWBR6eBsfyJB/2mzsgX4Nt3IIH2HYyr6TQkRWPJ
78XB/tfZmpANkWKmEkqsD9EqV6gJGC/7Q87IPNxskyi6yG+7Z2Xg75hEhUQLubh81QsaQvcwD2Pfde
799+FS5+qzKtYBB5udK/DUtvopmCbsve6lCZ/CRGtaeAUQ0XKwdzNVLD0Zxj1H3QIpBxaFUX2WzXZ2
80cYZBO/nw7tfnsO3CCkDsnmYXlsSunheD0QzLdW6EAhDsA0wGmSSycQYBS+MgFf8AgqmMUgB8rGUN
81hT1bQsOC4FsHdmrhvMULAkHo5ZS1ZIIlPmx74RoKsDFqRdyVBHZmSe0KW6B/hUginQCxcAvNDUrM
826rR+DaI+GKdWX0goNavzx5D1iUTWacSwJXUE0FCdkHV5MitxsllVB7H37xZ0rsmo01rw7Co560wA
838bCAdRpOD/2sK6uOdXZWZcq/AkDsFrHihIvjOnZrIXW7M89mltpKyHXatNM+1pWdDBQDZ9RWduC5
84RGadhj0B7sYuDpLCp7LJxMGJmsSsSLAI9e42evN9/67IwKGtBGcUeFcaZ2FSuPhMJCl4MINObZXM
851MI5N8gHVsKZGDes52GQNSkZIPJYBGeQszUwpwLUthMxyBqfDzIhQ9azx5JRrc8wyCqeWtOQXRPu
86P3nKbsNegRXzzH24chEYj1ayeoG/BKvFLmCVcFnIrnA0LfFuwulKfxsJPm91kUmJtSkwC1u8wpEa
87d4XFgLtYzq6PwrsfkTYsCpn2jMWMPLnWoncc1FxrIt2FV+ZSy1rMvBLKbc4/PFD2QEwDoaUp4B62
882VOBHGtTbP9g2rSgQ7CXx8yMb2NwduWmf8gbB2Ww0NrquBxrYrhgRTlRg7VYe0bKrsmhbXnAG9hS
89bGdiz0Ui/MJD+KYyzBws+a27UnFLxOxfW7C1KFplhLXFrwJbs8sIq38dxJoItok6GaL7j5/3Ld7d
90siTYmTL5q5jXN71sCbZTpi4Qs07zKdaSXY/J1FsSrE/weYEl1q3iFpEE6TSWjiNkS6zOwoS8RIqS
91HJSBXZ2ZMkWMBDSfHNqya2dyB16CEh/CFTTW1u2qyuLW4aQx0YF0vQZowd6SjSFHwLFaTLfNTC6O
921V7N1SnswUuJwWO1uNdgq5mMwPMD9iTgOAsFa0ZSZuWCrK2UEHrC+3bL0SQexdaYy2y2M5xjJgTL
938gG7e7kWm0gLWQJrtVfoOLfQbjetwfJXaxU3idtnJZY3kq+EFVy20myWYazUTgloMmCzp0lwV2h6
94LaqADFCdTr0wSxsvWaA6W9foTCyKestWc8ifDm+TWDiKuxyKajO08rQaoetRXMFus5dcBaqM9KGL
95pFdnsrn/EJvmWsslFjx5LZzG2dXMzIwKYfH+nJakTWUResVLXNvsrheGCs0VS1B9pxRsl6AcIGtS
96x5kJPeyjKvELk97K9O5jE98HuwRN0ons6UpIuu2szbQ6tsnQCViXS1bfLUtgH3tFZwb7RiJ5zwDW
97J2Ry7uW7wdRfceZudj4qxDsTzK0HoQRe/dDjcLzn1b8IzftFSyqbts08c7YEEuV5P/rXcBfW8i88
98/lH+0Zdffvn3EPnjn/jEJ85WVVWV0JiM66e+AIy4pOxDPlJbDbSk962q6uz8u9/4o/9bsmgRTlM/
99/Oqr73n88X95+bfwq17+0IfRw5UWexDi615c1XlrW1lT5aLY7tqHX/VUWWmvZSZC3doJdlPNVYhE
100TSVivXsf3KCkAAo1UArWlWfgpr+5lYp91RKwVsGtwmUPolxEaSzJem5VwY3C6jaFF7Xc2vJJ5NTC
101pGXcZ/FULJvNZ7cx+GK8jS5qc5IU8sPZ5cx38d7ENan7Ertt96KoFohRIz/fJmVVkBBkNLm2km20
102igWT/yoezCR1riJ7U/XWAJ+wtVOFdwTHyGJo1AccJQWAvFYGkYuizk09rcgFzbWeoMINt2hIl3Zx
103tn7AttdaSfdRazX7EP6jtnaqUMGyjfSK/Z0I47ZYW6HGsOgTWChdUZhFcYc2WV1FgZ+8MKRWcAF2
104QxfOVuDQaPcyMy+3Gbo8s5pZAZ5rsZnifUJBL2RRyz5sMXR5n/uQafCIZ5/FQoaFAbl912LWRfaK
105A+awTLG22Pu4pucFzOwNBQ24TlHjIe3z3Ie/zMytafY6Gsy5gv+IxShzRKW1MMuz6iaE+EqWWKtm
106eVbtTQCIdVGZrbyQ/1ididwFLHFZ21vD+QGpDzlKKNbabkDfhwDNeII1tibXWV6wZjYxNpVZKrq2
107+lj1ixigkSZrkcv/8Wrt4hCwVlpzBP4PWGsX3YjVVpT1cm2kBs1mBnQTlWXWilmf15L7pGOy46q0
1085rQcLoNFlk3cvdQiUZNVWvyKv4Bhy5bEm61FxlGADSxpsz7h6wKKLYuHbVlTalavmUF9bYxlwT4q
109mZoM2NViLRv6toXIjAtuTpLZigeOs7joHv6+hfCawSL0kdvSszq0/8oF3Cd8i4anyZ5PmW1CYNuF
110RbuCnTW+tAD/wRZjT+9mH2OW6STaZnWCjgvjrmkCg35gDapahhVcmFW7OouTvs0Eca+4K8kKDrPL
111KoRdsRimuhJ6hV9AZZlVAHvfYgdmZBJ+AH41hm3WRFjY/IOSlmCFW6+tszrIclFjG4pDMCuXqMjg
1125y8yYHXI6nGAtxPXT/BRibIcbmjNgjXqe5jYF3GhKxBvK4FHCfMlMwesPMO3N7Zxv5uBWBMSRpxK
113oo8LmUGReZeqzpnQ28685dtICb7JBQzhMjj6fj+Zs9g8BlfnwgUQuFs0gxVkwZXiGScSAYzI3y6s
114Aq8Fv8zHWs0tLxviyZLTF5Q7/kMWv2uWCE46NMmGRUnZ156Wt4CdI2namJS/4M82/X5hnx0E6Xel
115EfjBhHiAPoXxJndvqWFZ+zPex55GZn23u3GgsNTyr1wmm9hrf8b6LVyVfcB1D+0s1GX1s3GR1Cz4
116xnUTycdx3wK+dbv9XKGwub50/sArIGMkSz6R/Rz63Zg54DePC/us/zBnjBa4yjyy9Um+6vdd9Cdu
117Nl4Gvz9fVyJcLgOLgKUfxfsU93M813aB33Sv25fpb8hhS/NdhNBXzbGvF3uv/9e5EO7m+qzdgq+j
118ONcfbBnqm0I/JNZ+EPwGfeHpjI3r73jwbaTlrDXztqgfPZsD3EJhcwY/1epNtUAPfO5uEsJauO+6
119gB4rYmzclTUt2iLPj4crweC/VSwsAQqOnNjm7JTAth4GmdnfvBcImvca8GEZBl9uf0w07ieTfm+y
120/TL82anMwll0Q5FxR84sDn8ONG1WQhCo4H2in/Fsh3OscDybTAG2NmPHl0So4PnrEHb7Oum8WedA
121/zK+HztRrcwhgEBWqMvG2cZ/STLIXGXGIQhbOK5jMOnXxsG+E2eD4oFhsynKYmADWUsCM4bdnskM
122i4R9h9ig7IgsYSMmmAoqqMigYTOybFXwu/3VUF33YIaw/q/3f6cwCIQEguJtIaMWzUr0WU84/uO3
123ZT0ATwwz8Ky8AfBxdztvgZby96b7/XnQ8YDjKWlfaeLAzSl/fh5F20Ihu5TWcepT/u6468LsG4Tb
1247OrG2rqUZnXdjdiwS03cXKn3OJ5OrWiWYV9QyKr1tbqQ2qyg7YWnVyGbDNHZN6AQWkOH9mHW7VJ3
125UPeg1eJf7U26FqLYbBzXvoLqz8770WGTA/qCDFKu1YVzOhJw3WuYNYvMu15nZqGrJKwF6062tq4g
12691vj8cYnIrBfJRzQlYDrEk9gX7HbtSXgzqEisJzDrCtoegEgDca6Wdd9Jr/+eCF12sIE3E7tajl4
127wd8O1R6M1WrD4GqhcE0bFReJ1gMt+PV2vaAFL5TptVCwGbvqCuK8Reh+qc8ClrVYIAAvMCKUwGUR
128LK4Q0bDotfBvCAtGn+AQmmhBVospN/jla/jjzMmjT3AMLrnwsMdaC95dXBIAOV5PuBpSymJWe32N
129VvDu9PfE42ng1ImXAW4U2bcrkKsEK1oeGImZYJwPsKqBz/6TFCvW40CUDtDiHD+steV/+BUF1kJh
13058GAbQ8GSe1Vs7/7h9AiYr2Cg45j2w3sOoh+gB5U2nP6aIbNm9VWojX67x+TYoXus5XAIq6REdIS
131CNx/8Kcll3X05/I/LcUKS79W5+kwWPgRS6Xs4Z/Nd30Dni3JdaLbDd1Aq1MLwoZwWDHr6NSefP7J
132P4cnF6avcPkVpCQSsDYacEenOm7k8/nyNyD1CSIsSvhg+lQrkoH++tHoAt/YP3VqS7G7C6DmryBW
133UcWNpjtO9WDoiB9LNXLXo3em7s3vKIZG9w1EyliT6wKcR5O0tHVzPGpqrS6YIpS3u8vLu7oIHEZk
134o/w70ZmTO0TY3dOOSm19XTTtvnQ9l9H997Atu8u7GgJ4flZy2qQihiRRXi0cRbv5eCpSCrppz/lE
135TgjdhfWaZFjSVvOHo6v7OCnULa1P1yU+egrrs3tHkj3Zc4Q1vpKdQwJUyBnVtrbWg3G1Ne18WjAf
1363/s9F6RI8/m3EhEMkCAf9F4si049b2L6KJS0XHjv2ZVQ1sJAVAKz0cuizG4rPUpsekXWphD4u705
1373RdMpdEmNzrS3ukos45O4bt/O9o5xUni4xzOQB/nDub0cWW02SJqwSMY9Yb87Uf43/T7yp0jl/v6
138+i6P+DjhIQaTfWLUTeclp5R3WFLI9J6rM9apHN1C7v8lZdT8/0jBGvMCh0nE36B4/9GFfU6G1VgC
139rYWa/zosDkXDlFop6qY0VgWG/bwI1NwKNkUtptAqEsGfCg272ZBvZajl6tOKvIILB5EmrkuXmIQ8
140B/UAxQ2pUb2cIFoMplCdUzTlV4tW/gv7+tsTJKtTlvhMu4CidmuYNZ//TVwdRAyDHzMigbVYlEtW
1414y7o/Pd/Jho1ZVkSFeT2s6JPy6z5hmLxx5eGaLc/ZWj+I3RmVj0J5PPdxQ5n3XGurbJ934FU9VNs
1422nCHmTW9w8I3pLxYPAxP0wrqkqutrWYxET+pVQHxk3oayHcVi1PGMqnIN/KaFDoeC11msbjFLqvX
143TtFm3VAsjlqF9Vh1NZDfUyzeyYhVd27lnywWb2bEquuz4OSanxFrUdMPwGhQrLEJy7Uq1WvCwNWB
1449/rljFi1RdBdLP63TcPyLWBdrwVYv1xt0bA8q65hbxSLX3UH7cGy1AVCayoWZATF5h57KrjJG3aT
145Ssct7DYg69+6R6wZ9jDPqlVxoUyr+Eyba0+ypIlJkPckONkGkUQg6+s5i7BeAotw42EbNoj8xHnw
146+sdyAPaILc36DRubxjbsKUqy5oabeyx5A79i4yzbADI+ESvUwIs5MOYBHcyyMsV8rgDKIKKpdQmg
147FkVeArLeD1kt0pLOK/MIG0JtrR2b4JOixhzHmsv1D689st68bY+GVlv9XfgLIHjCIaMBYlds3Ivf
148VW0cd4HPycI/Nt1gtj1/BT7QAaagKqsdXK+h4VHfvlJeXn6jG939YsdhCVZoft6uRLoXf9GsdaNg
149OWN3HHaAqEVVTjQrIB5vqzgx25x2E2EBqgPIbwgqB3gLwnZF1u2/ODZj0liESICFqJBV1J+FNyKG
150FeAOj7nVpoy7IGLvBZLBFtT6ETstUBsmsSLaRkN+ga7FBpzCKdyggI8mdz1BzZ3Mimh7lpkR7lTY
151tI/Q9gQEuZIoWNDLELHmctObTdEeZf14bNyOKdb1QX8nRi4YhhP0ij1Yrr/NdQ2lNqOveRH33n7u
152diHWJK91AR0BckLRGB9z3UZDs2x0wRQYry3w6wqljklrdTtkWXP9E2DnxgljHiysf5zmJhgWheLi
153R0RWRc+PN7tuhSGXEDFVyYp97Moi7BCB8YwUa65/CJi20U6O6zgkVsTGAyyB4vNyrMAhANiKSTP+
154K/guRzFLbKBFXqBYlEUFvhbowFZlRrse0QEBBS0Jl8VdyyoI22PIIfhNm7wUStLxl+TtmsshWNeG
155DlhxvilifhG1Ft+swprrXwthbfQ+vNZXSAYNOCMvFt+oxEpgZ5h3tcRrRblZMrGKz6qhgoiLLDvD
156uPMa5TKbbj4zAG0OMlB3QGlgzVYYn2GcYYubvCWmC9SqRUW5oouyBMsblpW6l5hRVSIBZ/txtK3T
157uGVfo/ea/nvbMyl86NNKt58ePGwHNtihCbCreSx2YSCjtWBZtuMoaGD89wup7EqcgVth2BskluZK
158QYu/rHnIGbim/WywqcgbWDEQcLRYsu4Mw2lXoCLjWONbA2JpPIRhTYfb2PklnbpGoPdjFZhOZEa9
159bSe+GSZRFSbY9yJmNb6UFymDT6d0ApQfJQYgJpjOY/hgS2z7bPqJhWlx+AJFmOH55YR6X9qouRyZ
160XqYl6zijv+DTqgHU3DxiWNd4zvUdz9zv0b6oqVWsAmpY0172z8B7P//BT0Pc97+kK1UyvahiTavg
161K2L3rn4EcQWGE4MvqoNIvIJEWsPh60clzqx+SD+dXSYjQs2/qYPIvILOLpNO9kdkTpziGCYCg4b9
1621xQcMi9hLtacYZetkjlxmmNItmUwh2lMgyH1GtiVxcPQDp9ZE1LnTXMQbMqSYab4OnEyDYbUazjW
163E0byrQqp06Y66KRn1x4TrOuHUmFIvYilBIDZRLpVPV3qtKkO4lkNiKDGHU+FIfUiWnVBKRgQwTK3
164X+q0qQ7i5pbr6nuCI2tTUci9iCUE0LD6a3XuQ3KnTXWUF7cAa6+uJ1jvqk2ti/MUoFd5LstERVut
165OLWmr1WQN25u0qFdd/W4aonLdLdN2rBemoVxNTUwy3WlT40OBF5IGtY3s/RZl6myQu8uCetl2kQF
166mnatdhUjLIpEUrD+iWVAAz2qrDgSPSSeYHD7Q2Do2RXIVdGuJBKtFc1IProS5Ao91vXARGpzi0XN
167RLc8zysIPNtqdgx7XcVQgLZf4DE2HHeVYMt01NBMtBqVWaEf6mG0UbIdDroqyq2ZD1Qos8KbWzFJ
168T9/cNuyLueMX40DBKzQ73HBKq+kVUTpHuHvcPNQ2HY6JIdpri1SA7prMIHhXNT+Aw6ZTMyOaJ+lR
169zY8ogKilyHoSszqzgHrUhm6DALgBd0xJA3A3HkpCZivCai/OnoAnVmIF2xwxqyqs9i5Z4LJcpZyQ
170xHgUgJQsq+lbwekQa6xLjzA4lgBJRGfJTzAD67LoZCrdLJKPkMBew7uupJlmYnsZen+FyUUzUpaE
171TMrMMDO7jrEt5Ks9PLP4YmQWklHi6NWMV8Qw+BzShSzNW3xF3mAyrbHPfWFWWRGQnSGh5eB4WoMf
1727XDIvZP0BCzch2b1rMmwS6g4MqnfFOKyc8IqZ1gvKY1qoNSs7z1C1dDY2LvMRAfTV0fQOSGjWC5/
1731kxE05Uy7M6J25p8qm/03sqSsyncLCj1+vkUWrPGk4ULHOe5m2RY+MkMb+hmd+lgYU5IRnO8DOYF
1746hLtlCkVLMy12WiL7k+Mh4rSksjVgTUMZ9qJULQdn/DdfXSsZpmfyqjwRcFAPjYxTLSwavziRFRD
175wnVLIwGSwCanHsFnS+MFgF25ySVLrN31TyuC2bKE7Djje+7k0Wm7R5q5ZGZVF4GBxT95QwaOrJE2
176KD7QePqkQl6tBFtCBYCLqpGp7uj1GCidVewYOpb1J8UG1l5M0wKFLxZXouQqGs0UpDrAsiootQDk
177G1MVpUoDArdBootm7aOvyoqoEWi2pyTVYNxlJDWmGr9Nbj9jr+mNdrQ9J0pTBiRro2bZkQDujOpl
178NkH/HxtpiBH77U/NAAAAAElFTkSuQmCC
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{B0F5B10B-3A7A-4FE4-AA03-DD6678D68E03}{1}{B0}" name="Hearts template control" dateTime="2026-09-06T04:21:25.000Z" lastModifiedTime="2026-09-06T04:23:18.000Z" pageLevel="1" stationeryName="Hearts"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:Meta name="PageLocID" content="00040"/><one:Meta name="LocVer" content="1"/><one:Meta name="PageVerComment" content=""/><one:Meta name="PageOveride" content="0"/><one:Meta name="PageName" content="Untitled page"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Title lang="en-US"><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-06T04:23:18.000Z" lastModifiedTime="2026-09-06T04:23:18.000Z" objectID="{49EBB31D-D08C-4B7A-A1D1-9287A8BD80AE}{18}{B0}" alignment="left" quickStyleIndex="0" style="font-family:Calibri;font-size:17.0pt"><one:Meta name="oeLocID" content="0001"/><one:Meta name="oeLocComment" content=""/><one:T><![CDATA[Hearts template control]]></one:T></one:OE></one:Title><one:Image format="png" backgroundImage="true" originalPageNumber="0" lastModifiedTime="2009-07-28T18:49:14.000Z" objectID="{49EBB31D-D08C-4B7A-A1D1-9287A8BD80AE}{14}{B0}"><one:Position x="-90.0" y="-3.599996089935303" z="0"/><one:Size width="89.21739196777343" height="287.9999694824219" isSetByUser="true"/><one:Data>iVBORw0KGgoAAAANSUhEUgAAAKsAAAIoCAMAAAAhjet0AAADAFBMVEUAAAABAQECAgIDAwMXHhna
3kbrakbrakbrakbrakbrakbrakbrakbrakbrakrvakrvblr3bmr/TnLjbnMDcn8LdosPPo7bdpsXd
4q8jdr8ncssqEtpHdt83duc7du8/evNDdvNDcvM/cvM/cvM/cvM/cvM/cvM/cvM/cvM/cvM/cvM/c
5vM/cvc/cvc/avc/TvtDTv9DTv9DUv9DYwNHbwNHcwNHcwdLewdLcwtLdw9PdxNPdxtS6yK7dyNXd
6ydXeydbgydfizNrnzd3lztzd0Nnd0Nnd0Njd0Nnd0Nnd0Nnd0Nno0d/d0dnd0dnd0dnd0dnd0dnd
70dnd0dnd0dnd0dnd0dnd0dmY0qeh0q2Z06iZ06iZ06iZ06iZ06iZ06iZ06iZ06iZ06ia06ie06mZ
806ig06uk1LTq1OHh1Nyc1Kib1Kie1ana1duq1sHt1+Tl1+Dr2+Ss287u3Oex3Kux3Ku03auz3auw
93t6w3t6w3t7v3uiw3t6w3t6w3t6x3t6x3t6w3t6w3t6w3t6w3t7V3uC938i737+937m336uy396z
1039+939W9387U3+C24N/y4Ory4Oq64Ku54Ku/4La/4LS94Ni44d+84ay94ay64uC+4qy+4qy74uC8
114uC94q694q++4qy+4qy+4qy+4qy+4qy+4qzy4uu+4ty+4qy+4q2+4qy+4qy+4+C+4+K+49+94+G/
125OHQ5OPN6eXN6eLN6ebP6uXR6+TS6+LS6+XS6+XS6+XS6+XS6+XS6+XS6+XS6+XS6+XT7OHT7N/T
137N/T7ODT7OLT7OLT7OTT7OLX7ujc8Ovl8/Hm9PHn9PPp9fTs9vXy9vfw9/fv9/f09/j39/j5+vr7
14/fz9/f39/f39/f78/v79/v79/v79/v79/v79/v79/v78/v79/v79/v79/v7/////////////////
15////////////////////////////////////////////////////////////////////////////
16//////////////+3cRl2AAAAGXRFWHRTbm93Ym91bmQgdGVtcGxhdGUAaGVhcnRzVrgCNgAAGoNJ
17REFUeNrt3btPI1uaAHBSR+AxWhsEInLoXd0AjMwFDNeZI0czEzkCjahtQFhNYMlkyyKkJeAO9BJc
18jeSgpI26hCEiWQHSCkPftmzKnmz+h/oLth4uux6nqs7j+6rbmv6kkWbm8vjdw3e+8yzXlMYYg2I2
19OuoaRkxhULPZYu87sJazlCF/c+uRBcmt6LEcji0Pvq21pxtWNzYrwyhtrq3kgvNg8E2tv+Q2ShV3
20lDZWgrD53je0djYqpCgV4sGyWDuVoCitxoGdgqDqsUnuafnBN7GGUgMTAbKDTQFR9dggloTyN7Ae
21RForJWIeHMVulSsVXmwnZuvf/1LhxhYGmNaeXP9Fj0P5fUCdrSHYQzSreujs0WVrDnJYocXm8LLA
22a1UPfCWyPtD+XqGOTbws8FjreVJBlzv01soaWi1wWXtBE+ktBmtlBWv4clrlvD07Lazp4ZjurbFY
23SSl7CGyVh8m1Oc69Qo7DWlkn/GVUUKtFXXHPT0trpnaDyVpBatgpFzW3SZ7usbUrqRZAZKxt7Rg/
24cLkU0LEZrZVVlFIwtKpGt1otBXWWAqO1RKixYFajWK0E/+rVinjDdoCsR4EJMMSyWgkZewBjNTIg
25x+xhLAV5GKsxB9gEpZJqbAfCatSAVVgqqXcdQlgPwDNAD/9EtghgVdmLPUUQplsDceshRrOSkkAW
26t+pFoFCBjxz40DVlTgRKCFb/cLAlbDV6FgKVkLB5YWsBJwVIQ5dg55rqsU9P6WIDfDSYMtK1ghLg
27hWBKr1jLOFbwQjC1hZSupPWsqBVl0Aqwbv1zWTdwrIUJsq79sMZkzf5zWdfj6lsFcesk1azCBFlX
28JsianRTrFtIShjTPEjzvNOaESEULfF/TnGujdK4SvNVYw6CsDTfgrcb2APTGmxnr8GsYc5MQIwnW
294NeG5l4Gwh4RaWtbdM1tbr1hVIJl6KmLsZ9VxOldWYQ9IutkC7x3baBYB1mMOcE6/BGXsa9dxsjY
30Ao7VOt7E71odAKtmtgHw6iCLsgdvNyxojd1AOOW0zg0L4N2LMGqVYazWdWHIddcKfNcaWgdWrwXM
31gizSuaGdsYApu4HQtUZ3HbbMH7cMhSWkK9DZsWbdGgfEriB0rfF9l0PrJwKdIWfhTw0dVvuBjFWI
32liVd1OvBWbVeHi4NCjTp2nt7ejTi6a3ParXvkkFgc1ETwv7z1W51HLtXX9msmn1FMyc65yIst10j
33wduVZBIlSarp/7G4z0zW8TNEgvOY9dCR4O3UdNbOmsP4dLxjeHfbDFYHVqyHkZ46GWWpKd25bLqj
34ZmhP+/RW60qZcB6QUsBO1yeJJDW1+j/40Ke3joqBMYXjbtqNwOraNxpV+tgkxvGengd9eqsTm+Nd
35L64GVdevRt/faQbFmY79wGB1Yr0XTAVSwJxnfzX+/rVmcBzr//yRwaqpzovQPIlAGrSMC3ptnboX
36Rm02d3Rsj8Hqfvwxt8asLZDTta079s5Cqc1L/WuuWKzawPUAJHNFyBHnrpEJYIb+RRKTdfQI5Ehb
37EkyBstbfpaGaheuNzap13M8ZsGQCOQXMYhVJbX4M611Bz+6onkcNctS9jJQC6pWRrJ/orFesVm8e
38GBWMqt6SBoLC1ypVBljWU3Yr4SkOmlQgDQSHlBnAbyU0rfEkL/tAkH00mvUjjbXGbdV6W/5fHJ65
39xBTYpW1W0/rIZx0/I0PLJaXAf9Bmq1lfq8+8Vm1wRPqrZpfXS9QpYDTrXpMuXfdqx39rNv/n16cO
40h5X0+Flw65JWBD/RN+uO48t+7XBY9ZFhKxvE9XS15YBmrR43meO///Pxjd0aotW5m6XQFDCbda/J
41EztV6bHPbA3T6oXMTl7S83s/V0Mn2JFTmWd2q17ADoK12WUzeXNBKVDjs36SnGMu0+cPqAf5EG6u
42sBaUAtVLPqu5UhhhGT8vYyAXsmxhpoDU5A3JgWX9HBJ9dCgzWT+IWWvmn+WZ02o8mZynt1YF0tWI
43PfPftc9rZWlcK12Pm2JJYGUBp9XIXKpP0DHnAnRzrLAkMFe33FajiB0WKK38VHNGO2xYEasxQhyE
44p27hg6j10rJK4lYjdcO4R1WxMqDHcDu5DWEN5bbBrFdAVpNLyt0inHUXzmp2NV9lkB9Fy+vIWgW1
45GqOE7J6NDa6ErWe29Q3YatbdcfKWze0WMesxotWsZMNskAGsNdv6jGM1s6FsPMdfFbZKtvURzWp2
46Nm1oFRhiL6vxWDUA6ygFsPJ13LLCVik265uoddys+kzrO7eOmxV8LIC2Opr1NCYr77Lgcq/qKAPx
47WGviGWCceOBan4WsNedtCA3b+ihidVIh1jCIVhdVeG2IanVTzTMEYatc3p4zYmm7/g5o3XFRrbNE
48Iau6v5BwxuyS58N/n6zfxLyj+Umq+ptVxPq+lPDH7Lbqr1ms663anpsq9cSsKklqRNKh5bKeeRp1
49dIrEa60nE4GRLAtYz3a80tG9Ek7rdiI05t75rJ9qvjYdZQCvdSkREUnZaa1SQnf2qoQYXTHjspYT
500VFmsn76SGpRxz4xr1W2PdMpPYISd4nOevnxuCYFOd23CTis6qz5V07P29sA85lUANa21qz4aIf1
51PyVJ2qtGhPPiA4fV6FezC+7NoIX0tB9bHFm540kTs84mUgv+bbZFgna/LyaV3Efe7NZ6IkPev1z0
52Z4IsRN31XOBltxbnA3eGM77S9UcB6pX3HiSzdbAYsuU+782Df+GWfvBf02K2HoaeD/iwP3P++UmX
53M1itnYhzl4UZt/UPYFKEdy14W5Y9C06DLukxWtXo07c5oYY9fQ6+WcxolSmOCtO8Dbt79cxyXzsq
54jiisizMc1tOr58iHUBitWzRnsPPuhv1TeG06vXp8o3tWht4qy7TWrHsAm7869YXxHMzz2xvb8zxU
55VnnbysHkHN3FAXfDJjWgoLDKjp6dznI0rByX1b1eTdFZ3ROD7Zis8rACJVMps8jTWRddA0I6Hqts
56rFCmh0uABX3+n6HDzroaVo3DqiYNqWNmtZDhSQI5DqueAMn5LEcsYiRsqFVfWSUXs1zhWt1m8K36
572nqak+qeFCTxrXpdneekehIW3SpTl/7IoUvGti7xZ4AeLmsd2aqKNGs2OwNfCIKt+4mEQLO6pwRF
58ZOsC7eBPUQgyyNYk7Xj67a3vAgXLZ03jWmXaORVV0cK1bieSE2RN/bD+sE6QdWZirPUJqlmy4FgQ
59p1VLiI2xsVrTYp0rVmtRbE4Yq7UuNteO1arNCFWteK1Fod6VjtX6nhBZHMZrNW5fpGGsaXTru3Fz
60BcSaQbcaGTu9wGlNxWxVZ/g334Ktave13W6/drt9SKtRY3mxSe/dFzO6baUxDuXldzirkQWcWMIG
61rPpy0/DGRVuFsqpGF5meh7B27yycotw+6HHbUn47N/+flz6M1UzZxDR7NVjwXNZSTem55RxFy2zo
62iy8wVk02N9FSrOXAs6fZvtBN160HX9xeG9o7GOsQm0gv8lv/8JvRpgSpEcoJLZbm3NDCTqdZ2tY5
63FKQNjfIQFLfGP36BsWrv9i9OzfFY53XKSeshOFpGGnRhrI5zzulUhi4XxodxPxnU24ewMGquAmTV
64BwXHNnVyNhNdxFJO6sX9Q3gYxasNZfVdeU6m0unM/HzksEVFNRv2BsyqZ20mQY5kyhnuCyTz0Qlg
65xL2RsSqcNeDie2hQUh8eLmiSgO2+y3t5loWaaoQWK9bexXxHTy5Sc2eMwnnz8O2sBrfsvYlHjn8z
66MuCe2nqBYTWzYX87EwFOU2eAZW1gWYctXN/ezmQCksLIgPOH78bqUJuxv21FXf/v/2X8+hal1Zht
67nes/pvePGKy+6F8wNOuD/je4uGub/7X9ez9u6wtDtj7cuue2L+1uN07rBX0R0NPVkyv3+ncrr3FZ
68X41m/fzAHZ/NdWM8VoUlBYJG3YaixmDtmivUewHrvbnIvejiW82edf0gEq2GF4tkFU6BYcq6sDhW
69tcEyEAQWMgvbx7W2hdN13LDj9TiO1dpkEaQOM7bR+IJqvRDvWsORt+FYiaFYrXRVhK32tuIrovUL
70kNXeq71BtL5AlAFHwg43ZVCsCpD11rbeff/Wh9HuN561AVJeHVYrCTCsVhloPABaX7CsXXjr3QRZ
71FSzrC5S1NT5b+u6tyg8rivUC36oAWW8bsVlvH+BSANvaAkyB8+/c2nKe3n/nVucB/h2yVXCu7exZ
721rnH92u9dlq7yPVVzNpyXd/QkK2fRaj3Jw1vumJahdbc7psxr9jWC6BhQI8+trUBRh3uEn2X1tsT
73t7WLZ31tiA0GXqp9+Im4huG1tjzU0fUSTCtfgVW8V+NGZ8oY1r6A9d5/ja+LaR3uZfAU2Na5jzq+
74DYViHf5C5s3iW3+jNm76uFaFq2jdKyd+qvPeFqr1Vljqug+HuP/KUrRanxvEcN0zRLVSFoJb5Zws
75bSh9dGu7QT8rvFUuGg0qKo61S1m0bpXPJ43gOO9q8VlDCsF9S7kJc/pbFet8qxFSCHSlcn3eiAwf
76Fcl6QygE9y0Ded2gjDv/rRccq11gFR2oCz9fX580mOL8lfBTcawvDbFQiHc2v0frecCNTRxrV4R6
77F+/9LAGr0g38qUh3SLjbtBvyQ5GsN1x5ehd+DRrzbg7jH/816hkezDtPLFCaR6OQrG1wqLC1q4fx
78QJ4Zd4ojqJ13X6gf5eOxdn83YQ3RUF6+MD1yyGTtd3VjAyDOlZfXLnMbUVt15g2AUjGUfY0r6J4z
79er27EDe+fOFFUlvVtsIP1IXtbjeWzyfU+q+8UKWvQUf4c8cCf/oLLU6r9Zgod8RoFZTGaW2LdvzY
80rF2Ait+Nx9puNCbE2r9rTIq1DzLix2LtUo36J9fXv+mjUssM906QvbPS1rCt3Yj+f36tC0P3q23r
81i4Zs7YdQz28Umo1qJSZrcK5eK7Sb/3FZ74KgDPduY7ISV58nCtulsHisXaKU9SqzfVFF0RCt/RsA
82aUxWfwZc81wJjMPa9TfqA0/cxmD1lqtz3nuW+FZvs15wPx6Ab1WgqPhWT7NeCzx0gW69g2rV0RVb
83PKtrznIidH35Gtn6xdWsYrdssa2uFIC5uIpmdQ6vos9fIltVuAwYWe+QrM6KJfyo6DXunPAFsFlj
84tIo/gYtsVcCKQKzWeyhrF9168zA5VmWCrC0wq4ZkHdeBh8mxngNYz5GG2KH1C1x1tefaaFYV3vqC
85ZR3NswCs91jbr975K4C1hVWyvOuCCzBrH81qX10He1j4RsOz2lXrFsh6h2jtn8NOs9qIVpAH7xxD
86QRfTOixbJ0DlVUO1dmEmLy20dHXuFb+CVFira70iW2E+7sZKJRXbamHPxVYxJ0gTF69V+3IuWgpu
870SqW74zTvDkiUmMVvBTwn3OrL4oI9gKtCpDvZajdtqex7tlSoBufVTPukP2f/evvlfPPbClwo8Vq
881bR/9L4Yl1r/95qpjJ2gFVct+j6hwjTuojZrpPWCqYidYzZrpJVpnqjgjQO01hZTs3YnwqogrbUR
89rLeuDKhv78tQL5JnqAOU+Wp+8HPX/sbhiyHSc0vbZVmOxUo/P/jsqgGq98UBunlfxrW+0m4aeJK1
90Tv4s89m5Ir84ytqnXIgrnjlLMfST4pfKMoKVMgm8VC0d+Sn8c9sytFWlaVgfVaV7w8Fc+R3Sai5s
91Iu4T+AtrnfrdEbNFGc5qlq2wKcG9+SH57llAkeVVF7OUrUthNS9EBl8qMT/j4Kbr/h73O0/m5tJp
927xtO3LEgw1gtbMBbflrm9pXvFVru927YL41ZnDfQZPPsPoh1eIP7phUg9T8mKrsZvpdIzaVnk+xa
93ymeizMW45xLs8OMYbgjz1W0XIuDtk/MZL3i2DmHV+i/W/t+5dUtbUYZPkyvEmbU7XUPehbU4l3Z5
94F1QAK/GRo5uXgB+dJKZrkDfjeL9gsg5hNbkv9sdwGE/hBbZBRLqGc5dUEOswiC/DcMY+Tbp6X4aX
95nrZnC++A1shwjwS0bxxctF+Ok5RjtLomLtMsb5mbDsOiWN0vi2J5fd9iOgSLYXV3LcZX0M6biZBU
96Y7KWudJ11LRmSUjHZHW//oz59ZjW+waX4rG6ulaS3Zo1h716LFbWkYCMnVVjsMocIwHpVY7FGKx1
97oa7leD/iO751W7BrWdVg2t+9EKwLol1rlLLv6NYM96jlipQvYxGsIqOWcwDT/yjxWrnfm242bB3Z
98KkOUgWHGLsVqzQrEdGIW2brNO3klDQhyjNaUiHXOk7Dw1iUw66L9cnc0awamZBkxk1iYGGsqkZkY
99axrbmgQaCuKwQg0FP6xo1lnsOgBoTWHXV0DrdGJ/UqwL6POBtPgqdhiZxIyGa4UbC1Lo81cw6wL+
100umABap4165lqf8fz1wXPjBDDug+0hkklZlRsK9DaUE/7soZtVRMQRWuesF3MYu29vb09Pw7j6lSP
101q8dRvI2+bFZ4S1OnTnvGAUpr/+1Jl32ohsfj6OuX2Bey3mnunE71nySHW78+X51W6eJ09E1l9o2X
102dNKZ1+ZhTFFjsH59pGV6rO/s+9p6dqbsfynrSI5wXBBkbV9JVbaQNHLChp1yO2upUYzTelhLoLJG
103af3KDDVi/P1F9p1Czz0C2nPD5w9VrvgaUGGnFxitM9sBJ91e6/NulTPGRctz1YFinM2/l4fTiJnM
104PuWZ/Bu3tFp9Gv8Yz+2s6LFryxxEZDn8QqfT2j+tCsSjFtS7phejrIdUY5HD2pZEqNWrwPlL9AFH
105j9H6WBWLU+ePzSRYimxRY7KK/f19VjnBgu0wWfsMhWpPcgSxwGreA7lw7IHGYo2iStJOrfbx48cm
106KWokq+8CbGpRMANs62mIsnZ81gyNj77BwKxA3muD0wGla2HAZL0KgtYimC7rm/sny74LgynCCLaY
107Lmss1jYRunPcpIzhN3gf3Nv3YROzHu1iZnpGZbH2JFKLXjabjNZHLRqbSKZHqbA4NzsdMKMKtPqT
108VTpusoQUYNX2yVddk6nU6OpuRmOx+jOg1mzCWDV5NhEeaZXJ6p2uSJeMVLtonRJ+vJoJpzJd2596
109FmzUcGtgHlgJwPZs6tSuMDXcqqlLAdKZssYWU+7h8yMH1S6wu0G/471IaNuZIvMDvy7r3llTwFoN
110/i3q/oKbm9nX2MNlPW7yRaTVrAnlYiaTSWcyC9ucTxpNCeYqvVU8HFaJl9rci916xm2V4rZyZ0D8
1111r1PwtZ+XFaBZrUHg7e4rJeTY5Wak2OtAVi1mKxnE2QVodoTgpis0g8rirUGYJUmyHo6EdZarFYJ
112wnooT4z137PZwmFPQ7fuAVg3zJ2fwqGKZt0Vn7pY1j+ON1RlpM+kagN0LnP++rNzA7AsDxCsw603
113SdT653/17qzL8NbhlualoPVn/z4wdE+bsreKdwSsxvf/RD4LgMwFY1/7WbBhz/Tv/lPwKUsH0mph
114+TO2Rk6BcS7UVTir9iaJlALd+ueIQ0yQxrXP4k4F1gZSeLMOG1c8c0fnsb0r3m3CZmDP8jauCmQ1
115MuHxry4DZW87pmlWK36Roax6fP11LK1RlrEaZbMKp4Lvvkvn6ddf//brX2sS9Xas5JgKUES+PoCy
116WvFMv7YNSoFlcG2A9ZG+4O6QUyBXKq0FcvNc3UzYehnQrIWKHqW1XBCXo21DrDWRZs2WKlaUCjmo
117TAjJ1xplsxLHrJXKODYDuPk6iPWNduYVNGZtVpxR2lgJqGAA1j5lvh4HjFm5ijcCUvcXVdhqrhai
118d+Uv96rk4rpWIcRmgfSlhwNR6xNNwn6SqhE9i6Zx87KgtS9RJIFeA6of8kEFixykxi0PhKw0lcCk
119ytE9K7px6Zo2+A70h4izZDMBPvTLVD3LExs+7S8DEauRBSHYS4N62h+QmnW9Ehmb3iKW7whYta9S
120yIWC2p51NVNm6FkR2rqA1cSSc9ZsVMm4kFVm61nuxC2w5UHo8wXmKkzyac92zC1M40hTZe5Zbq27
121bfM9fuvw+vZezZEJZzVzn+bUOniTeXpWcCbkZQGr1n/cte9AGiFZVwVO7SPCIu2YFaJ11QRZwKr5
122n4s4fRodaKvcPcsZrnp7IGQ1etnz46kZV0+uQ1dSCqxWmKO0SoUVe36LlAIbFY5wJsIBilUV7Vnk
123+nWAYZUFimtY0x4gWMsQPYtUbI/grXApMCwI4aVLxAqaAlbT5sJGMBHrIWgKWFh7+6MwgLUWgFPA
124hS2CWnvgKWBGIbB/CVhl+BRwYXuA1i2EFHBiIT/zDycFHNgjMGtHaJZN08HyYM/Jk9K1AhTDOnsA
125ZS2DTAeDsNYPVIGseaDpIDnWCQ3LbVWRKpYdK/6G5bYS0nUZkDpM2QMQ6yFaxXJlwQDCWkSrWHbk
126vJNDbityuuqx4R28eK09rAHW27164lYZO1312PT0Ll7rEddOJmMYQ21B3HqAnq52KVCFrVt4kwFH
127jXVXAl5rHnckGIYxOSyLWgeYExd378qLWjvCW5l0YfzgDrx1A8O66kxYTutRTNZ151IGzlrCsBqV
128YAveWkEJ52gAZl3BsRpDl6D1IC7rqrh1Ky7rmmO+PQnWDrR1Fce6gWFd+2GdNOtggqzgNauAY10X
129tx7FWLPyk2ItiM9d5FiWMBVzj+BI0NqJcZ4lOtdWY5y/qoJWLaZ1wQbA2pCwTYi13joQtpZjKbAl
130kL0MGX+b0JsCkHuaCJ1rGWSfkLBJtIHRrM6NbW5rHAm74trSBD2HKeE2K+j5FnASGKdGWxqElVBh
131gacEq1nPJ22Cnh2zJkHo129mPc0qYB2In24UIjIgPwCyAhwZrBRCqd5bWgLWjnDDrgR+g0n1XtUD
132vvPE1rD6AmWF+B3mnf7iANBK6F1sKxmjp+f8p7jWnU0fVew+YUHw/Ni6I7BaIjQqgQp+pzRXYrbq
1332vG/YWnduupSBL6jRzyRY9mDcyZPYW1jfa1g39A7ID1qIGYlXdOjP5Y9kEnPKGWzQbfhBd+3Qbqq
134ST0tkDV1iyTdovzsetbBKy+QssZg3/FptwIf4hF9jwnxwQ067F+GeXToKCf5sA8qEH7nCikLlku0
135zWqFKh8dbG1tHcpCz8NQZEGRs2UPmH+V+LtseqTOnIvuYO/fwEp+3CyydHF8gAbEO4LI2JVSeL1i
136j/8H9g/ABkdaki4AAAlPdEVYdENvbW1lbnQAICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
137ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
138ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
139ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
140ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
141ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
142ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
143ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
144ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
145ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
146ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
147ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
148ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
149ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
150ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
151ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
152ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
153ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
154ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
155ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
156ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
157ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
158ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
159ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
160ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
161ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
162ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
163ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
164ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
165ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
166ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
167ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
168ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
169ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
170ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
171ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
172ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
173ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
174ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
175ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
176ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
177ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
178ICAgICAgICBmyrzgAAAAAElFTkSuQmCC
179179</one:Data></one:Image></one:Page>
corpus/m6/native-template-controls-01/read/page-002.pdf
Binary files a/corpus/m6/native-template-controls-01/read/page-002.pdf and b/corpus/m6/native-template-controls-01/read/page-002.pdf differ
corpus/m6/native-template-controls-01/read/page-002.xml+178-1
......@@ -1,2 +1,179 @@
11<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{F3B3040C-0089-05FF-2C9F-3BFB91647BCD}{1}{B0}" name="A notebook created in Rust." dateTime="2026-09-05T07:11:28.000Z" lastModifiedTime="2026-09-06T04:21:25.000Z" pageLevel="1" style="font-family:Calibri;font-size:11.0pt" lang="en-US"><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="Fixture Author" lastModifiedBy="Fixture Author" lastModifiedTime="2026-09-05T07:11:28.000Z" objectID="{9C3D193E-0294-060F-08C2-1BB16FEA3244}{23}{B0}"><one:Position x="36.0" y="36.0" z="0"/><one:Size width="191.1661224365234" height="13.4277172088623"/><one:Indents><one:Indent level="0" indent="0.0"/></one:Indents><one:OEChildren><one:OE creationTime="2026-09-05T07:11:28.000Z" lastModifiedTime="2026-09-05T07:11:28.000Z" objectID="{9C3D193E-0294-060F-08C2-1BB16FEA3244}{24}{B0}" alignment="left"><one:T><![CDATA[A notebook created in Rust.]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{ECFB9395-741E-4CC0-B9F7-1B81B0CB48BD}{1}{B0}" name="Default template page" dateTime="2026-09-06T04:24:38.000Z" lastModifiedTime="2026-09-06T04:24:51.000Z" pageLevel="1" stationeryName="Hearts"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Title lang="en-US"><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-06T04:24:51.000Z" lastModifiedTime="2026-09-06T04:24:51.000Z" objectID="{3EFE148C-FDB8-4E86-B785-00EC646C8E95}{18}{B0}" alignment="left" quickStyleIndex="0" style="font-family:Calibri;font-size:17.0pt"><one:Meta name="oeLocID" content="0001"/><one:Meta name="oeLocComment" content=""/><one:T><![CDATA[Default template page]]></one:T></one:OE></one:Title><one:Image format="png" backgroundImage="true" originalPageNumber="0" lastModifiedTime="2009-07-28T18:49:14.000Z" objectID="{3EFE148C-FDB8-4E86-B785-00EC646C8E95}{14}{B0}"><one:Position x="-90.0" y="-3.599996089935303" z="0"/><one:Size width="89.21739196777343" height="287.9999694824219" isSetByUser="true"/><one:Data>iVBORw0KGgoAAAANSUhEUgAAAKsAAAIoCAMAAAAhjet0AAADAFBMVEUAAAABAQECAgIDAwMXHhna
3kbrakbrakbrakbrakbrakbrakbrakbrakbrakrvakrvblr3bmr/TnLjbnMDcn8LdosPPo7bdpsXd
4q8jdr8ncssqEtpHdt83duc7du8/evNDdvNDcvM/cvM/cvM/cvM/cvM/cvM/cvM/cvM/cvM/cvM/c
5vM/cvc/cvc/avc/TvtDTv9DTv9DUv9DYwNHbwNHcwNHcwdLewdLcwtLdw9PdxNPdxtS6yK7dyNXd
6ydXeydbgydfizNrnzd3lztzd0Nnd0Nnd0Njd0Nnd0Nnd0Nnd0Nno0d/d0dnd0dnd0dnd0dnd0dnd
70dnd0dnd0dnd0dnd0dnd0dmY0qeh0q2Z06iZ06iZ06iZ06iZ06iZ06iZ06iZ06iZ06ia06ie06mZ
806ig06uk1LTq1OHh1Nyc1Kib1Kie1ana1duq1sHt1+Tl1+Dr2+Ss287u3Oex3Kux3Ku03auz3auw
93t6w3t6w3t7v3uiw3t6w3t6w3t6x3t6x3t6w3t6w3t6w3t6w3t7V3uC938i737+937m336uy396z
1039+939W9387U3+C24N/y4Ory4Oq64Ku54Ku/4La/4LS94Ni44d+84ay94ay64uC+4qy+4qy74uC8
114uC94q694q++4qy+4qy+4qy+4qy+4qy+4qzy4uu+4ty+4qy+4q2+4qy+4qy+4+C+4+K+49+94+G/
125OHQ5OPN6eXN6eLN6ebP6uXR6+TS6+LS6+XS6+XS6+XS6+XS6+XS6+XS6+XS6+XS6+XT7OHT7N/T
137N/T7ODT7OLT7OLT7OTT7OLX7ujc8Ovl8/Hm9PHn9PPp9fTs9vXy9vfw9/fv9/f09/j39/j5+vr7
14/fz9/f39/f39/f78/v79/v79/v79/v79/v79/v79/v78/v79/v79/v79/v7/////////////////
15////////////////////////////////////////////////////////////////////////////
16//////////////+3cRl2AAAAGXRFWHRTbm93Ym91bmQgdGVtcGxhdGUAaGVhcnRzVrgCNgAAGoNJ
17REFUeNrt3btPI1uaAHBSR+AxWhsEInLoXd0AjMwFDNeZI0czEzkCjahtQFhNYMlkyyKkJeAO9BJc
18jeSgpI26hCEiWQHSCkPftmzKnmz+h/oLth4uux6nqs7j+6rbmv6kkWbm8vjdw3e+8yzXlMYYg2I2
19OuoaRkxhULPZYu87sJazlCF/c+uRBcmt6LEcji0Pvq21pxtWNzYrwyhtrq3kgvNg8E2tv+Q2ShV3
20lDZWgrD53je0djYqpCgV4sGyWDuVoCitxoGdgqDqsUnuafnBN7GGUgMTAbKDTQFR9dggloTyN7Ae
21RForJWIeHMVulSsVXmwnZuvf/1LhxhYGmNaeXP9Fj0P5fUCdrSHYQzSreujs0WVrDnJYocXm8LLA
22a1UPfCWyPtD+XqGOTbws8FjreVJBlzv01soaWi1wWXtBE+ktBmtlBWv4clrlvD07Lazp4ZjurbFY
23SSl7CGyVh8m1Oc69Qo7DWlkn/GVUUKtFXXHPT0trpnaDyVpBatgpFzW3SZ7usbUrqRZAZKxt7Rg/
24cLkU0LEZrZVVlFIwtKpGt1otBXWWAqO1RKixYFajWK0E/+rVinjDdoCsR4EJMMSyWgkZewBjNTIg
25x+xhLAV5GKsxB9gEpZJqbAfCatSAVVgqqXcdQlgPwDNAD/9EtghgVdmLPUUQplsDceshRrOSkkAW
26t+pFoFCBjxz40DVlTgRKCFb/cLAlbDV6FgKVkLB5YWsBJwVIQ5dg55rqsU9P6WIDfDSYMtK1ghLg
27hWBKr1jLOFbwQjC1hZSupPWsqBVl0Aqwbv1zWTdwrIUJsq79sMZkzf5zWdfj6lsFcesk1azCBFlX
28JsianRTrFtIShjTPEjzvNOaESEULfF/TnGujdK4SvNVYw6CsDTfgrcb2APTGmxnr8GsYc5MQIwnW
294NeG5l4Gwh4RaWtbdM1tbr1hVIJl6KmLsZ9VxOldWYQ9IutkC7x3baBYB1mMOcE6/BGXsa9dxsjY
30Ao7VOt7E71odAKtmtgHw6iCLsgdvNyxojd1AOOW0zg0L4N2LMGqVYazWdWHIddcKfNcaWgdWrwXM
31gizSuaGdsYApu4HQtUZ3HbbMH7cMhSWkK9DZsWbdGgfEriB0rfF9l0PrJwKdIWfhTw0dVvuBjFWI
32liVd1OvBWbVeHi4NCjTp2nt7ejTi6a3ParXvkkFgc1ETwv7z1W51HLtXX9msmn1FMyc65yIst10j
33wduVZBIlSarp/7G4z0zW8TNEgvOY9dCR4O3UdNbOmsP4dLxjeHfbDFYHVqyHkZ46GWWpKd25bLqj
34ZmhP+/RW60qZcB6QUsBO1yeJJDW1+j/40Ke3joqBMYXjbtqNwOraNxpV+tgkxvGengd9eqsTm+Nd
35L64GVdevRt/faQbFmY79wGB1Yr0XTAVSwJxnfzX+/rVmcBzr//yRwaqpzovQPIlAGrSMC3ptnboX
36Rm02d3Rsj8Hqfvwxt8asLZDTta079s5Cqc1L/WuuWKzawPUAJHNFyBHnrpEJYIb+RRKTdfQI5Ehb
37EkyBstbfpaGaheuNzap13M8ZsGQCOQXMYhVJbX4M611Bz+6onkcNctS9jJQC6pWRrJ/orFesVm8e
38GBWMqt6SBoLC1ypVBljWU3Yr4SkOmlQgDQSHlBnAbyU0rfEkL/tAkH00mvUjjbXGbdV6W/5fHJ65
39xBTYpW1W0/rIZx0/I0PLJaXAf9Bmq1lfq8+8Vm1wRPqrZpfXS9QpYDTrXpMuXfdqx39rNv/n16cO
40h5X0+Flw65JWBD/RN+uO48t+7XBY9ZFhKxvE9XS15YBmrR43meO///Pxjd0aotW5m6XQFDCbda/J
41EztV6bHPbA3T6oXMTl7S83s/V0Mn2JFTmWd2q17ADoK12WUzeXNBKVDjs36SnGMu0+cPqAf5EG6u
42sBaUAtVLPqu5UhhhGT8vYyAXsmxhpoDU5A3JgWX9HBJ9dCgzWT+IWWvmn+WZ02o8mZynt1YF0tWI
43PfPftc9rZWlcK12Pm2JJYGUBp9XIXKpP0DHnAnRzrLAkMFe33FajiB0WKK38VHNGO2xYEasxQhyE
44p27hg6j10rJK4lYjdcO4R1WxMqDHcDu5DWEN5bbBrFdAVpNLyt0inHUXzmp2NV9lkB9Fy+vIWgW1
45GqOE7J6NDa6ErWe29Q3YatbdcfKWze0WMesxotWsZMNskAGsNdv6jGM1s6FsPMdfFbZKtvURzWp2
46Nm1oFRhiL6vxWDUA6ygFsPJ13LLCVik265uoddys+kzrO7eOmxV8LIC2Opr1NCYr77Lgcq/qKAPx
47WGviGWCceOBan4WsNedtCA3b+ihidVIh1jCIVhdVeG2IanVTzTMEYatc3p4zYmm7/g5o3XFRrbNE
48Iau6v5BwxuyS58N/n6zfxLyj+Umq+ptVxPq+lPDH7Lbqr1ms663anpsq9cSsKklqRNKh5bKeeRp1
49dIrEa60nE4GRLAtYz3a80tG9Ek7rdiI05t75rJ9qvjYdZQCvdSkREUnZaa1SQnf2qoQYXTHjspYT
500VFmsn76SGpRxz4xr1W2PdMpPYISd4nOevnxuCYFOd23CTis6qz5V07P29sA85lUANa21qz4aIf1
51PyVJ2qtGhPPiA4fV6FezC+7NoIX0tB9bHFm540kTs84mUgv+bbZFgna/LyaV3Efe7NZ6IkPev1z0
52Z4IsRN31XOBltxbnA3eGM77S9UcB6pX3HiSzdbAYsuU+782Df+GWfvBf02K2HoaeD/iwP3P++UmX
53M1itnYhzl4UZt/UPYFKEdy14W5Y9C06DLukxWtXo07c5oYY9fQ6+WcxolSmOCtO8Dbt79cxyXzsq
54jiisizMc1tOr58iHUBitWzRnsPPuhv1TeG06vXp8o3tWht4qy7TWrHsAm7869YXxHMzz2xvb8zxU
55VnnbysHkHN3FAXfDJjWgoLDKjp6dznI0rByX1b1eTdFZ3ROD7Zis8rACJVMps8jTWRddA0I6Hqts
56rFCmh0uABX3+n6HDzroaVo3DqiYNqWNmtZDhSQI5DqueAMn5LEcsYiRsqFVfWSUXs1zhWt1m8K36
572nqak+qeFCTxrXpdneekehIW3SpTl/7IoUvGti7xZ4AeLmsd2aqKNGs2OwNfCIKt+4mEQLO6pwRF
58ZOsC7eBPUQgyyNYk7Xj67a3vAgXLZ03jWmXaORVV0cK1bieSE2RN/bD+sE6QdWZirPUJqlmy4FgQ
59p1VLiI2xsVrTYp0rVmtRbE4Yq7UuNteO1arNCFWteK1Fod6VjtX6nhBZHMZrNW5fpGGsaXTru3Fz
60BcSaQbcaGTu9wGlNxWxVZ/g334Ktave13W6/drt9SKtRY3mxSe/dFzO6baUxDuXldzirkQWcWMIG
61rPpy0/DGRVuFsqpGF5meh7B27yycotw+6HHbUn47N/+flz6M1UzZxDR7NVjwXNZSTem55RxFy2zo
62iy8wVk02N9FSrOXAs6fZvtBN160HX9xeG9o7GOsQm0gv8lv/8JvRpgSpEcoJLZbm3NDCTqdZ2tY5
63FKQNjfIQFLfGP36BsWrv9i9OzfFY53XKSeshOFpGGnRhrI5zzulUhi4XxodxPxnU24ewMGquAmTV
64BwXHNnVyNhNdxFJO6sX9Q3gYxasNZfVdeU6m0unM/HzksEVFNRv2BsyqZ20mQY5kyhnuCyTz0Qlg
65xL2RsSqcNeDie2hQUh8eLmiSgO2+y3t5loWaaoQWK9bexXxHTy5Sc2eMwnnz8O2sBrfsvYlHjn8z
66MuCe2nqBYTWzYX87EwFOU2eAZW1gWYctXN/ezmQCksLIgPOH78bqUJuxv21FXf/v/2X8+hal1Zht
67nes/pvePGKy+6F8wNOuD/je4uGub/7X9ez9u6wtDtj7cuue2L+1uN07rBX0R0NPVkyv3+ncrr3FZ
68X41m/fzAHZ/NdWM8VoUlBYJG3YaixmDtmivUewHrvbnIvejiW82edf0gEq2GF4tkFU6BYcq6sDhW
69tcEyEAQWMgvbx7W2hdN13LDj9TiO1dpkEaQOM7bR+IJqvRDvWsORt+FYiaFYrXRVhK32tuIrovUL
70kNXeq71BtL5AlAFHwg43ZVCsCpD11rbeff/Wh9HuN561AVJeHVYrCTCsVhloPABaX7CsXXjr3QRZ
71FSzrC5S1NT5b+u6tyg8rivUC36oAWW8bsVlvH+BSANvaAkyB8+/c2nKe3n/nVucB/h2yVXCu7exZ
721rnH92u9dlq7yPVVzNpyXd/QkK2fRaj3Jw1vumJahdbc7psxr9jWC6BhQI8+trUBRh3uEn2X1tsT
73t7WLZ31tiA0GXqp9+Im4huG1tjzU0fUSTCtfgVW8V+NGZ8oY1r6A9d5/ja+LaR3uZfAU2Na5jzq+
74DYViHf5C5s3iW3+jNm76uFaFq2jdKyd+qvPeFqr1Vljqug+HuP/KUrRanxvEcN0zRLVSFoJb5Zws
75bSh9dGu7QT8rvFUuGg0qKo61S1m0bpXPJ43gOO9q8VlDCsF9S7kJc/pbFet8qxFSCHSlcn3eiAwf
76Fcl6QygE9y0Ded2gjDv/rRccq11gFR2oCz9fX580mOL8lfBTcawvDbFQiHc2v0frecCNTRxrV4R6
77F+/9LAGr0g38qUh3SLjbtBvyQ5GsN1x5ehd+DRrzbg7jH/816hkezDtPLFCaR6OQrG1wqLC1q4fx
78QJ4Zd4ojqJ13X6gf5eOxdn83YQ3RUF6+MD1yyGTtd3VjAyDOlZfXLnMbUVt15g2AUjGUfY0r6J4z
79er27EDe+fOFFUlvVtsIP1IXtbjeWzyfU+q+8UKWvQUf4c8cCf/oLLU6r9Zgod8RoFZTGaW2LdvzY
80rF2Ait+Nx9puNCbE2r9rTIq1DzLix2LtUo36J9fXv+mjUssM906QvbPS1rCt3Yj+f36tC0P3q23r
81i4Zs7YdQz28Umo1qJSZrcK5eK7Sb/3FZ74KgDPduY7ISV58nCtulsHisXaKU9SqzfVFF0RCt/RsA
82aUxWfwZc81wJjMPa9TfqA0/cxmD1lqtz3nuW+FZvs15wPx6Ab1WgqPhWT7NeCzx0gW69g2rV0RVb
83PKtrznIidH35Gtn6xdWsYrdssa2uFIC5uIpmdQ6vos9fIltVuAwYWe+QrM6KJfyo6DXunPAFsFlj
84tIo/gYtsVcCKQKzWeyhrF9168zA5VmWCrC0wq4ZkHdeBh8mxngNYz5GG2KH1C1x1tefaaFYV3vqC
85ZR3NswCs91jbr975K4C1hVWyvOuCCzBrH81qX10He1j4RsOz2lXrFsh6h2jtn8NOs9qIVpAH7xxD
86QRfTOixbJ0DlVUO1dmEmLy20dHXuFb+CVFira70iW2E+7sZKJRXbamHPxVYxJ0gTF69V+3IuWgpu
870SqW74zTvDkiUmMVvBTwn3OrL4oI9gKtCpDvZajdtqex7tlSoBufVTPukP2f/evvlfPPbClwo8Vq
881bR/9L4Yl1r/95qpjJ2gFVct+j6hwjTuojZrpPWCqYidYzZrpJVpnqjgjQO01hZTs3YnwqogrbUR
89rLeuDKhv78tQL5JnqAOU+Wp+8HPX/sbhiyHSc0vbZVmOxUo/P/jsqgGq98UBunlfxrW+0m4aeJK1
90Tv4s89m5Ir84ytqnXIgrnjlLMfST4pfKMoKVMgm8VC0d+Sn8c9sytFWlaVgfVaV7w8Fc+R3Sai5s
91Iu4T+AtrnfrdEbNFGc5qlq2wKcG9+SH57llAkeVVF7OUrUthNS9EBl8qMT/j4Kbr/h73O0/m5tJp
927xtO3LEgw1gtbMBbflrm9pXvFVru927YL41ZnDfQZPPsPoh1eIP7phUg9T8mKrsZvpdIzaVnk+xa
93ymeizMW45xLs8OMYbgjz1W0XIuDtk/MZL3i2DmHV+i/W/t+5dUtbUYZPkyvEmbU7XUPehbU4l3Z5
94F1QAK/GRo5uXgB+dJKZrkDfjeL9gsg5hNbkv9sdwGE/hBbZBRLqGc5dUEOswiC/DcMY+Tbp6X4aX
95nrZnC++A1shwjwS0bxxctF+Ok5RjtLomLtMsb5mbDsOiWN0vi2J5fd9iOgSLYXV3LcZX0M6biZBU
96Y7KWudJ11LRmSUjHZHW//oz59ZjW+waX4rG6ulaS3Zo1h716LFbWkYCMnVVjsMocIwHpVY7FGKx1
97oa7leD/iO751W7BrWdVg2t+9EKwLol1rlLLv6NYM96jlipQvYxGsIqOWcwDT/yjxWrnfm242bB3Z
98KkOUgWHGLsVqzQrEdGIW2brNO3klDQhyjNaUiHXOk7Dw1iUw66L9cnc0awamZBkxk1iYGGsqkZkY
99axrbmgQaCuKwQg0FP6xo1lnsOgBoTWHXV0DrdGJ/UqwL6POBtPgqdhiZxIyGa4UbC1Lo81cw6wL+
100umABap4165lqf8fz1wXPjBDDug+0hkklZlRsK9DaUE/7soZtVRMQRWuesF3MYu29vb09Pw7j6lSP
101q8dRvI2+bFZ4S1OnTnvGAUpr/+1Jl32ohsfj6OuX2Bey3mnunE71nySHW78+X51W6eJ09E1l9o2X
102dNKZ1+ZhTFFjsH59pGV6rO/s+9p6dqbsfynrSI5wXBBkbV9JVbaQNHLChp1yO2upUYzTelhLoLJG
103af3KDDVi/P1F9p1Czz0C2nPD5w9VrvgaUGGnFxitM9sBJ91e6/NulTPGRctz1YFinM2/l4fTiJnM
104PuWZ/Bu3tFp9Gv8Yz+2s6LFryxxEZDn8QqfT2j+tCsSjFtS7phejrIdUY5HD2pZEqNWrwPlL9AFH
105j9H6WBWLU+ePzSRYimxRY7KK/f19VjnBgu0wWfsMhWpPcgSxwGreA7lw7IHGYo2iStJOrfbx48cm
106KWokq+8CbGpRMANs62mIsnZ81gyNj77BwKxA3muD0wGla2HAZL0KgtYimC7rm/sny74LgynCCLaY
107Lmss1jYRunPcpIzhN3gf3Nv3YROzHu1iZnpGZbH2JFKLXjabjNZHLRqbSKZHqbA4NzsdMKMKtPqT
108VTpusoQUYNX2yVddk6nU6OpuRmOx+jOg1mzCWDV5NhEeaZXJ6p2uSJeMVLtonRJ+vJoJpzJd2596
109FmzUcGtgHlgJwPZs6tSuMDXcqqlLAdKZssYWU+7h8yMH1S6wu0G/471IaNuZIvMDvy7r3llTwFoN
110/i3q/oKbm9nX2MNlPW7yRaTVrAnlYiaTSWcyC9ucTxpNCeYqvVU8HFaJl9rci916xm2V4rZyZ0D8
1111r1PwtZ+XFaBZrUHg7e4rJeTY5Wak2OtAVi1mKxnE2QVodoTgpis0g8rirUGYJUmyHo6EdZarFYJ
112wnooT4z137PZwmFPQ7fuAVg3zJ2fwqGKZt0Vn7pY1j+ON1RlpM+kagN0LnP++rNzA7AsDxCsw603
113SdT653/17qzL8NbhlualoPVn/z4wdE+bsreKdwSsxvf/RD4LgMwFY1/7WbBhz/Tv/lPwKUsH0mph
114+TO2Rk6BcS7UVTir9iaJlALd+ueIQ0yQxrXP4k4F1gZSeLMOG1c8c0fnsb0r3m3CZmDP8jauCmQ1
115MuHxry4DZW87pmlWK36Roax6fP11LK1RlrEaZbMKp4Lvvkvn6ddf//brX2sS9Xas5JgKUES+PoCy
116WvFMv7YNSoFlcG2A9ZG+4O6QUyBXKq0FcvNc3UzYehnQrIWKHqW1XBCXo21DrDWRZs2WKlaUCjmo
117TAjJ1xplsxLHrJXKODYDuPk6iPWNduYVNGZtVpxR2lgJqGAA1j5lvh4HjFm5ijcCUvcXVdhqrhai
118d+Uv96rk4rpWIcRmgfSlhwNR6xNNwn6SqhE9i6Zx87KgtS9RJIFeA6of8kEFixykxi0PhKw0lcCk
119ytE9K7px6Zo2+A70h4izZDMBPvTLVD3LExs+7S8DEauRBSHYS4N62h+QmnW9Ehmb3iKW7whYta9S
120yIWC2p51NVNm6FkR2rqA1cSSc9ZsVMm4kFVm61nuxC2w5UHo8wXmKkzyac92zC1M40hTZe5Zbq27
121bfM9fuvw+vZezZEJZzVzn+bUOniTeXpWcCbkZQGr1n/cte9AGiFZVwVO7SPCIu2YFaJ11QRZwKr5
122n4s4fRodaKvcPcsZrnp7IGQ1etnz46kZV0+uQ1dSCqxWmKO0SoUVe36LlAIbFY5wJsIBilUV7Vnk
123+nWAYZUFimtY0x4gWMsQPYtUbI/grXApMCwI4aVLxAqaAlbT5sJGMBHrIWgKWFh7+6MwgLUWgFPA
124hS2CWnvgKWBGIbB/CVhl+BRwYXuA1i2EFHBiIT/zDycFHNgjMGtHaJZN08HyYM/Jk9K1AhTDOnsA
125ZS2DTAeDsNYPVIGseaDpIDnWCQ3LbVWRKpYdK/6G5bYS0nUZkDpM2QMQ6yFaxXJlwQDCWkSrWHbk
126vJNDbityuuqx4R28eK09rAHW27164lYZO1312PT0Ll7rEddOJmMYQ21B3HqAnq52KVCFrVt4kwFH
127jXVXAl5rHnckGIYxOSyLWgeYExd378qLWjvCW5l0YfzgDrx1A8O66kxYTutRTNZ151IGzlrCsBqV
128YAveWkEJ52gAZl3BsRpDl6D1IC7rqrh1Ky7rmmO+PQnWDrR1Fce6gWFd+2GdNOtggqzgNauAY10X
129tx7FWLPyk2ItiM9d5FiWMBVzj+BI0NqJcZ4lOtdWY5y/qoJWLaZ1wQbA2pCwTYi13joQtpZjKbAl
130kL0MGX+b0JsCkHuaCJ1rGWSfkLBJtIHRrM6NbW5rHAm74trSBD2HKeE2K+j5FnASGKdGWxqElVBh
131gacEq1nPJ22Cnh2zJkHo129mPc0qYB2In24UIjIgPwCyAhwZrBRCqd5bWgLWjnDDrgR+g0n1XtUD
132vvPE1rD6AmWF+B3mnf7iANBK6F1sKxmjp+f8p7jWnU0fVew+YUHw/Ni6I7BaIjQqgQp+pzRXYrbq
1332vG/YWnduupSBL6jRzyRY9mDcyZPYW1jfa1g39A7ID1qIGYlXdOjP5Y9kEnPKGWzQbfhBd+3Qbqq
134ST0tkDV1iyTdovzsetbBKy+QssZg3/FptwIf4hF9jwnxwQ067F+GeXToKCf5sA8qEH7nCikLlku0
135zWqFKh8dbG1tHcpCz8NQZEGRs2UPmH+V+LtseqTOnIvuYO/fwEp+3CyydHF8gAbEO4LI2JVSeL1i
136j/8H9g/ABkdaki4AAAlPdEVYdENvbW1lbnQAICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
137ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
138ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
139ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
140ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
141ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
142ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
143ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
144ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
145ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
146ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
147ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
148ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
149ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
150ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
151ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
152ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
153ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
154ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
155ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
156ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
157ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
158ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
159ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
160ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
161ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
162ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
163ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
164ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
165ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
166ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
167ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
168ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
169ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
170ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
171ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
172ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
173ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
174ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
175ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
176ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
177ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
178ICAgICAgICBmyrzgAAAAAElFTkSuQmCC
179</one:Data></one:Image></one:Page>
corpus/outline-edit/README.md+1-1
......@@ -46,7 +46,7 @@ The controller selects the target again after activating the native window and
4646transfers XML as UTF-8 data. Captured authoring scripts retain their exact source;
4747identical immutable artifacts link to one canonical corpus copy.
4848
49`layout/candidate` applies `PreparedEdit::outline` to the five geometry and saved
49`layout/candidate` applies Rust outline edits to the five geometry and saved
5050collapse cases in `before`. `layout/cold` captures that Rust output through a fresh
5151OneNote cache. Every active identity, child/content/structure reference, child level,
5252and collapse default survives; text, fields, tags and explicit formatting compare
corpus/page-model/README.md+1-1
......@@ -1,6 +1,6 @@
11# Page-model edits through a cold native reopen
22
3`candidate` is `corpus/outline-edit/before` after eight compound `PreparedEdit::page`
3`candidate` is `corpus/outline-edit/before` after eight compound page-model
44publications, one per edited page, each combining several kinds of change in a single
55transaction: paragraph reorder and Unicode replacement with a bold range; a subtree
66move with a new nested child and an italic/colored/resized range; a subtree deletion
corpus/paragraph-edit/README.md+1-1
......@@ -26,7 +26,7 @@ notebook before completing inspection. Cold-open the split notebook for the join
2626pass. Captured COM IDs belong to their originating sessions; discover fresh IDs
2727when regenerating the corpus.
2828
29`rust-split` retains twelve `ParagraphSplit` outputs, their cold native captures,
29`rust-split` retains twelve Rust paragraph-split outputs, their cold native captures,
3030and a second native session typing into four empty boundary paragraphs. The
3131hyperlink case has no output: the operation rejected fields when this row was
3232captured, and the writer now splits before a link as OneNote does. The Rust
corpus/protected-edit/README.md+1-1
......@@ -1,7 +1,7 @@
11# Protected page edit
22
33`candidate/notebook` is `native-encrypted/encrypted-01` after
4`Notebook::save_unlocked` appended text to the first paragraph and added a
4a Rust edit of the unlocked page appended text to the first paragraph and added a
55paragraph, exported by `an_unlocked_page_is_saved_under_the_section_key`
66(`ONESTORE_PROTECTED_EXPORT`). `candidate/read` is OneNote 2010's COM read from a
77fresh clone with an empty cache after unlocking the section in its UI with the
corpus/table-edit/README.md+1-1
......@@ -27,7 +27,7 @@ modification time, the indent array copied from a sibling cell (OneNote's
2727default without one), and the flags every native cell carries, rebuilds each table's row and cell order to the
2828model's, and writes the column count, widths, locks and border flag. New
2929tables start as an empty paragraph whose content the structure pass replaces.
30An emptied cell keeps a replacement paragraph, as the tree writer provides.
30An edit that ends with an empty cell is refused; its ops refill the cell themselves.
3131`tools/test_table_edit.py` checks all three captures without a VM. Regenerate
3232with `ONESTORE_TABLE_EXPORT`, `ONESTORE_TABLE_EDIT_EXPORT` and
3333`ONESTORE_NESTED_TABLE_EXPORT` set to new absolute directories while running
crates/notebook/Cargo.toml+4
......@@ -36,3 +36,7 @@ required-features = ["smb"]
3636[[example]]
3737name = "smb_concurrent_client"
3838required-features = ["smb"]
39
40[[example]]
41name = "smb_publish_probe"
42required-features = ["smb"]
crates/notebook/README.md+37-29
......@@ -11,7 +11,7 @@ each by hash. Local success does not acknowledge publication to a shared noteboo
1111editor ── Edit ──► section thread ─ Section::apply, INSERT edit ─┐
1212 (the parsed section: base + sealed + open) │ one fsync per burst
1313sync thread: remote.stamp() == base ? publish(sealed batch) ──────┤ base chunks += transaction
14 otherwise read once, replay the queue on it ─────────┘ conflict, or base := remote
14 otherwise read once, replay the queue on it ─────────┘ base := remote, conflict pages queued
1515```
1616
1717## Edits
......@@ -56,15 +56,11 @@ same queue after a relaunch. A section is `Send + Sync`: `apply(author, edit)`
5656returns at once and reports a refusal as `Event::Rejected`; `events()` drains
5757remote changes (`Changed(spaces)`), publication outcomes, unreachable files and
5858failures; `notify` runs on a background thread whenever an event waits.
59`conflict()`, `remote_page(space)` and `resolve(id, Resolution)` present and end a
60conflict; `release(id, archive, Resolution)` ends an uncertain attempt.
59`conflicts()` lists each page's conflict pages, which `page` reads and `delete_pages`
60removes; `release(id, archive, Resolution)` ends an uncertain attempt. The author an
61edit names is the host's: the app passes the account's full name.
6162`import_page` creates a page holding a copy of another; `delete_pages` removes pages.
6263
63Until the application emits ops, `save`, `queue_save`, `saved`, `saving`,
64`conflicts`, `queue`, `review` and `Event::Refreshed` keep the whole-page API: a
65save lowers the page it is given against the stored page (`onestore::op::lower_page`)
66and applies the ops.
67
6864`Section::resume(file, replica, notify)` starts from an owned `Replica` without
6965consulting the remote file; with the `smb` feature, `Section::resume_smb(path,
7066replica, limit, connect, notify)` binds a share-relative path, `connect` running on
......@@ -81,10 +77,15 @@ queue replays on it (`merge.rs`): an op whose objects the remote left alone appl
8177as it is; a text op shifts past the remote's changes to the same text when its
8278range stays clear of them; a move, deletion or property the remote already made is
8379done; a page the remote already holds as the local edits leave it drops their ops.
84Anything else is a `Conflict` on the batch: the queue stays on its base and the
85remote image is kept for review. `resolve(id, Mine)` rewrites the remote page to the
86local one (`lower_page`, once), `Theirs` drops the local ops on that page; either
87applies from the conflicted batch on.
80Anything else conflicts, as in OneNote 2010: the op is dropped with every later op naming
81what it named, the remote's version stays the page, and the local version becomes a
82conflict page under it (`SectionOp::Conflict`, queued as one more edit and published with
83the rest), its conflicting objects marked, named for the author of the first edit that did
84not replay. Page lists merge as OneNote 2010 merges page series: a page the remote moved
85keeps the remote's place, a page moved here goes before the next page in the local order
86the remote left in place, or last, and a page the remote removed comes back as a new copy
87of the local one, placed the same way (`corpus/conflict-page/native-pages`,
88`native-restore`). Nothing blocks the queue.
8889
8990Publication attempts are recorded before network I/O. An attempt confirms only
9091when the remote holds its revisions, or every page it changed as it changed them,
......@@ -96,7 +97,9 @@ A durable `Published` receipt survives reopening.
9697A schema-14 cache is converted when opened: it is exported to
9798`<cache>.v14-recovery`, each queued page is lowered against the page the conversion
9899has so far, and every converted page must equal the page in the old working image,
99or the conversion rolls back and the open fails naming the archive.
100or the conversion rolls back and the open fails naming the archive; a page it held in
101conflict becomes a conflict page. A schema-15 cache's batches lose the review conflict
102they recorded when opened.
100103
101104With the optional `smb` feature, `SmbRemote::new(client, path, limit)` binds an
102105`notebook::smb::Client` to one share-relative file and snapshot limit. Remote identity uses the logical root
......@@ -161,7 +164,7 @@ conflict policy lives in the client.
161164## Recovery archives
162165
163166`export_recovery(new_path)` captures the base and remote images, the edit queue,
164uncertain attempts, conflicts, receipts, downloaded media and the edit-ID sequence in one
167uncertain attempts, receipts, downloaded media and the edit-ID sequence in one
165168SQLite snapshot. It refuses existing destinations and leaves the live queue
166169unchanged. Export to a local directory from a background thread: copying holds
167170the cache mutex while capturing the database. Failure after the final rename can
......@@ -261,16 +264,16 @@ targets are `None`.
261264
262265With the optional `protected` feature, `Notebook::unlock(path, password)`
263266reads a `Locked` section as `Unlocked { pages, .. }`, and
264`Notebook::save_unlocked(path, password, &mut unlocked, space, page, author)`
265saves an edited page under the section's key, straight to the file: nothing of
266a protected section is cached or queued, a section written since `unlocked`
267was read fails the save, and a wrong password is
267`Notebook::apply_unlocked(path, password, &mut unlocked, author, edit)`
268applies page ops and stores them under the section's key, straight to the file:
269nothing of a protected section is cached or queued, a section written since
270`unlocked` was read refuses the edit, and a wrong password is
268271`Error::Protected(PasswordMismatch)`.
269272
270273`Section::import_page(page, author)` copies a page, usually read from another
271section, to the end of this one as a page creation and a save queued like the
274section, to the end of this one as one `SectionOp::Import` queued like the
272275user's own edits, under fresh identities (`Page::copy`); payloads travel with
273the model, so the copy publishes offline later like any save. Content outside
276the model, so the copy publishes offline later like any edit. Content outside
274277the model refuses to copy. A move is an import here followed by
275278`Section::delete_pages` there.
276279
......@@ -309,15 +312,20 @@ let document = onestore::document::Document::parse(&revisions)?;
309312```
310313
311314Paths are relative to the share. The read limit bounds the complete physical
312snapshot. Call from a background thread outside a Tokio runtime. Use identities
313from the document and the same snapshot with `Client::commit_text` or
314`Client::commit_property_bytes`; their errors retain `onestore::CommitState`.
315`PreparedEdit::page` separates preparation from I/O: inspect the immutable image
316and persist the intended revision identity before `Client::commit_transaction` publishes
317its `transaction()`. `Client::stamp` reads a file's header and length without coordination,
318for polling. `Client::confirm_snapshot` checks and flushes an observed image's stamp, then refreshes
319its header version metadata without adding a revision. The caller must first
320establish which intents that image contains and reread before another commit.
315snapshot. Call from a background thread outside a Tokio runtime.
316`Client::commit_transaction` publishes a `Transaction` from `Section::seal`; its errors
317retain `onestore::CommitState`. `Client::stamp` reads a file's header and length in one
318round trip without coordination, for polling. `Client::confirm` checks and flushes a
319stamp, then refreshes its header version metadata without adding a revision. The caller
320must first establish which intents that image contains and reread before another commit.
321
322A commit is eight round trips while its appended bytes and patches fit fifteen 64 KiB
323writes: open; the coordination locks, identity queries and the stamp check's reads as one
324compound request; the path's identity; the appended bytes and patches, the header, the
325counter (twice at a carry) and the version, each as its writes and a flush in one
326compound; close. The server
327performs a compound in order and answers its flush once the writes are durable, so the
328writes carry no write-through flag.
321329
322330Readers use shared native guards while writers publish under native write-open
323331and byte-lock exclusion. Maintenance is excluded during each operation; pathname
crates/notebook/examples/cache_probe.rs+13-4
......@@ -32,6 +32,17 @@ fn content(bytes: &[u8]) -> (ExGuid, ExGuid, String) {
3232 .unwrap()
3333}
3434
35/// The images the queue leaves and started from, as a recovery archive records them.
36fn images(cache: &Replica, path: &Path) -> Result<(Vec<u8>, Vec<u8>), Box<dyn std::error::Error>> {
37 let archive = path.with_extension(format!("probe-{}", std::process::id()));
38 cache.export_recovery(&archive)?;
39 let recovery = notebook::Recovery::open(&archive)?;
40 let images = (recovery.snapshot()?, recovery.remote_snapshot()?);
41 drop(recovery);
42 std::fs::remove_file(&archive)?;
43 Ok(images)
44}
45
3546fn payload(operation: u64, size: usize) -> String {
3647 format!("{operation}:🦀{}", "x".repeat(size))
3748}
......@@ -73,8 +84,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
7384 }
7485 let cache = Replica::open(path)?;
7586 if mode == "read" {
76 let snapshot = cache.snapshot()?;
77 let base = cache.remote_snapshot()?;
87 let (snapshot, base) = images(&cache, path)?;
7888 let (sid, target, mut expected) = content(&base);
7989 let store = Store::parse(&snapshot)?;
8090 assert!(store.checksum_mismatches.is_empty());
......@@ -164,8 +174,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
164174 let instruction = lines.next().unwrap()?;
165175 let (operation, acknowledgement) = instruction.split_once(' ').unwrap();
166176 let operation: u64 = operation.parse()?;
167 let source = cache.snapshot()?;
168 let (sid, oid, text) = content(&source);
177 let (sid, oid, text) = content(&images(&cache, path)?.0);
169178 let replacement = payload(operation, size);
170179 println!("editing {operation}");
171180 io::stdout().flush()?;
crates/notebook/examples/keystroke_probe.rs+5-4
......@@ -66,15 +66,15 @@ impl Remote for Memory {
6666 self.read += self.bytes.len() as u64;
6767 Ok(self.bytes.clone())
6868 }
69 fn stamp(&mut self) -> io::Result<Option<Stamp>> {
69 fn stamp(&mut self) -> io::Result<Stamp> {
7070 self.read += 1024;
71 Ok(Stamp::of(&self.bytes).ok())
71 Stamp::of(&self.bytes).map_err(io::Error::other)
7272 }
7373 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
7474 transaction.commit(self)
7575 }
76 fn confirm(&mut self, snapshot: &[u8]) -> Result<(), CommitError> {
77 onestore::confirm_snapshot(self, snapshot)
76 fn confirm(&mut self, base: &Stamp) -> Result<(), CommitError> {
77 onestore::confirm(self, base)
7878 }
7979}
8080
......@@ -135,6 +135,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
135135 identity: None,
136136 created: None,
137137 margin_origin: [0.0; 2],
138 color: None,
138139 objects: vec![PageObject::Outline(Outline {
139140 id: new_id()?,
140141 title: false,
crates/notebook/examples/migrate_probe.rs+13-9
......@@ -28,14 +28,14 @@ impl Remote for Memory {
2828 fn read(&mut self) -> io::Result<Vec<u8>> {
2929 Ok(self.0.clone())
3030 }
31 fn stamp(&mut self) -> io::Result<Option<Stamp>> {
32 Ok(Stamp::of(&self.0).ok())
31 fn stamp(&mut self) -> io::Result<Stamp> {
32 Stamp::of(&self.0).map_err(io::Error::other)
3333 }
3434 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
3535 transaction.commit(self)
3636 }
37 fn confirm(&mut self, snapshot: &[u8]) -> Result<(), CommitError> {
38 onestore::confirm_snapshot(self, snapshot)
37 fn confirm(&mut self, base: &Stamp) -> Result<(), CommitError> {
38 onestore::confirm(self, base)
3939 }
4040}
4141
......@@ -72,11 +72,11 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
7272 let pending = replica.pending()?;
7373 let summary = replica.recovery_summary()?;
7474 println!(
75 "{path}: converted in {:.1} ms; {} edits; {:?}; conflict {:?}",
75 "{path}: converted in {:.1} ms; {} edits; {:?}; conflict pages {:?}",
7676 elapsed.as_secs_f64() * 1e3,
7777 pending.len(),
7878 summary,
79 replica.conflict()?
79 replica.conflicts()?
8080 );
8181 for edit in &pending {
8282 println!(
......@@ -89,11 +89,15 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
8989 for (space, title, level) in replica.pages()? {
9090 println!(" page {space} {level} {title:?}");
9191 }
92 let blocked = replica.conflict()?.is_some()
93 || replica.recovery_summary()?.uncertain_edits > 0;
92 let blocked = replica.recovery_summary()?.uncertain_edits > 0;
9493 if !pending.is_empty() && !blocked {
9594 let local = pages(&replica)?;
96 let mut remote = Memory(replica.remote_snapshot()?);
95 // The remote the queue was made on, as a recovery archive records it.
96 let archive = copy.with_extension("probe-recovery");
97 replica.export_recovery(&archive)?;
98 let mut remote =
99 Memory(notebook::Recovery::open(&archive)?.remote_snapshot()?);
100 std::fs::remove_file(&archive)?;
97101 let synced = replica.sync_once(&mut remote)?;
98102 let published = {
99103 let arena = onestore::Arena::default();
crates/notebook/examples/open_probe.rs+5-4
......@@ -44,6 +44,7 @@ fn generated(count: usize) -> Result<Vec<u8>, Box<dyn std::error::Error>> {
4444 identity: None,
4545 created: None,
4646 margin_origin: [0.0; 2],
47 color: None,
4748 objects: vec![PageObject::Outline(Outline {
4849 id: new_id()?,
4950 title: false,
......@@ -106,14 +107,14 @@ impl Remote for Memory {
106107 fn read(&mut self) -> io::Result<Vec<u8>> {
107108 Ok(self.0.clone())
108109 }
109 fn stamp(&mut self) -> io::Result<Option<Stamp>> {
110 Ok(Stamp::of(&self.0).ok())
110 fn stamp(&mut self) -> io::Result<Stamp> {
111 Stamp::of(&self.0).map_err(io::Error::other)
111112 }
112113 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
113114 transaction.commit(self)
114115 }
115 fn confirm(&mut self, snapshot: &[u8]) -> Result<(), CommitError> {
116 onestore::confirm_snapshot(self, snapshot)
116 fn confirm(&mut self, base: &Stamp) -> Result<(), CommitError> {
117 onestore::confirm(self, base)
117118 }
118119}
119120
crates/notebook/examples/queue_scale.rs+45-47
......@@ -1,8 +1,8 @@
11use notebook::{EditStatus, Remote, Replica};
22use onestore::{
3 CommitError, PageCreation, PreparedEdit, RevisionIndex, Store, Transaction,
4 document::Document,
5 page::{Page, PageObject, Paragraph, text::Edit},
3 Arena, CommitError, ExGuid, PageCreation, Section, Stamp, Transaction,
4 op::{Edit, Op, PageOp, SectionOp},
5 page::PageObject,
66};
77use std::{io, path::PathBuf, time::Instant};
88
......@@ -12,11 +12,14 @@ impl Remote for FileRemote {
1212 fn read(&mut self) -> io::Result<Vec<u8>> {
1313 onestore::read_file(&self.0)
1414 }
15 fn stamp(&mut self) -> io::Result<Stamp> {
16 Stamp::of(&self.read()?).map_err(io::Error::other)
17 }
1518 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
1619 transaction.commit_file(&self.0)
1720 }
18 fn confirm(&mut self, snapshot: &[u8]) -> Result<(), CommitError> {
19 onestore::confirm_file_snapshot(&self.0, snapshot)
21 fn confirm(&mut self, base: &Stamp) -> Result<(), CommitError> {
22 onestore::confirm_file(&self.0, base)
2023 }
2124}
2225
......@@ -29,13 +32,14 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
2932 std::fs::create_dir(&directory)?;
3033 let source = onestore::create_section("queue.one", "First", "Fixture")?;
3134 let creation = PageCreation::new(None, Some("Second"), "Fixture")?;
32 let source = PreparedEdit::create_page(&source, &creation)?
33 .as_bytes()
34 .to_vec();
35 let pages = {
36 let store = Store::parse(&source)?;
37 let index = RevisionIndex::parse(&store)?;
38 Document::parse(&index)?.pages()?
35 let (source, pages) = {
36 let arena = Arena::default();
37 let mut section = Section::open(&arena, source)?;
38 let ops = vec![Op::Section(SectionOp::Create(creation))];
39 section.apply("Fixture", &Edit { at: 133_000_000_000_000_000, ops })?;
40 section.seal()?;
41 let pages: Vec<ExGuid> = section.pages()?.into_iter().map(|(space, ..)| space).collect();
42 (section.image(), pages)
3943 };
4044 assert_eq!(pages.len(), 2);
4145 let path = directory.join("cache.sqlite");
......@@ -48,40 +52,32 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
4852 for n in 0..count {
4953 let edit_start = Instant::now();
5054 let slot = n % 2;
51 let space = pages[slot].0;
52 let mut page = cache.page(space)?;
55 let space = pages[slot];
56 let page = cache.page(space)?;
5357 let text = page
5458 .objects
55 .iter_mut()
59 .iter()
5660 .find_map(|object| match object {
57 PageObject::Outline(outline) => {
58 outline.paragraphs.iter_mut().find_map(|p| p.text_mut())
59 }
6061 PageObject::Title(title) => title
6162 .outlines
62 .iter_mut()
63 .flat_map(|outline| &mut outline.paragraphs)
64 .find_map(|p| p.text_mut().filter(|text| text.date_field.is_none())),
63 .iter()
64 .flat_map(|outline| &outline.paragraphs)
65 .find_map(|p| p.text().filter(|text| text.date_field.is_none())),
6566 _ => None,
6667 })
6768 .unwrap();
6869 expected[slot] = format!("Edit {n} 🦀 e\u{301}");
6970 let end = u32::try_from(text.text.text().encode_utf16().count())?;
70 let format = text.text.format_at(0)?.clone();
71 text.text.apply(Edit {
71 let op = PageOp::Text {
72 text: text.id,
7273 range: 0..end,
73 replacement: Paragraph::new(expected[slot].clone(), format),
74 })?;
75 let before = cache.page(space)?;
76 let ops = onestore::op::lower_page(&before, &page)?;
74 with: expected[slot].clone(),
75 };
7776 let id = cache.apply(
7877 "Fixture",
79 onestore::op::Edit {
78 Edit {
8079 at: 133_000_000_000_000_000,
81 ops: ops
82 .into_iter()
83 .map(|op| onestore::op::Op::Page { space, op })
84 .collect(),
80 ops: vec![Op::Page { space, op }],
8581 },
8682 )?;
8783 assert!(ids.last().is_none_or(|previous| *previous < id));
......@@ -108,8 +104,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
108104 .collect::<Vec<_>>(),
109105 ids
110106 );
111 let source = cache.snapshot()?;
112 verify(&source, &pages, &expected)?;
107 verify(&cache, &pages, &expected)?;
113108 println!(
114109 "{}",
115110 serde_json::json!({"phase":"reopen","seconds":start.elapsed().as_secs_f64()})
......@@ -132,9 +127,13 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
132127 serde_json::json!({"phase":"publish","count":ids.len(),"ms":step.elapsed().as_secs_f64()*1000.0})
133128 );
134129 assert!(cache.pending()?.is_empty());
135 let published = cache.snapshot()?;
136 assert_eq!(onestore::read_file(&remote.0)?, published);
137 verify(&published, &pages, &expected)?;
130 let published = onestore::read_file(&remote.0)?;
131 verify(&cache, &pages, &expected)?;
132 let arena = Arena::default();
133 let section = Section::open(&arena, published.clone())?;
134 for space in &pages {
135 assert_eq!(section.page(*space)?, cache.page(*space)?);
136 }
138137 drop(cache);
139138 let cache = Replica::open(&path)?;
140139 for id in &ids {
......@@ -152,7 +151,11 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
152151 .collect::<Vec<_>>(),
153152 ids
154153 );
155 verify(&archive.snapshot()?, &pages, &expected)?;
154 let arena = Arena::default();
155 let section = Section::open(&arena, archive.snapshot()?)?;
156 for (slot, space) in pages.iter().enumerate() {
157 assert_eq!(section.page(*space)?.title, expected[slot]);
158 }
156159 println!(
157160 "{}",
158161 serde_json::json!({"phase":"complete","count":count,
......@@ -163,17 +166,12 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
163166}
164167
165168fn verify(
166 bytes: &[u8],
167 pages: &[(onestore::ExGuid, onestore::ExGuid)],
169 cache: &Replica,
170 pages: &[ExGuid],
168171 expected: &[String; 2],
169172) -> Result<(), Box<dyn std::error::Error>> {
170 let store = Store::parse(bytes)?;
171 assert!(store.checksum_mismatches.is_empty());
172 let index = RevisionIndex::parse(&store)?;
173 index.validate_current()?;
174 let document = Document::parse(&index)?;
175 for (slot, (space, _)) in pages.iter().enumerate() {
176 assert_eq!(Page::from_space(&document, *space)?.title, expected[slot]);
173 for (slot, space) in pages.iter().enumerate() {
174 assert_eq!(cache.page(*space)?.title, expected[slot]);
177175 }
178176 Ok(())
179177}
crates/notebook/examples/recovery_probe.rs+9-7
......@@ -1,10 +1,10 @@
11mod support {
22 pub mod view;
33}
4use support::view::view;
4use support::view::{cached, view};
55
66use notebook::{EditStatus, Remote, Replica};
7use onestore::{CommitError, CommitIo, Transaction};
7use onestore::{CommitError, CommitIo, Stamp, Transaction};
88use serde_json::json;
99use std::{
1010 env,
......@@ -73,16 +73,19 @@ impl Remote for Disk {
7373 phase("publish-after");
7474 result
7575 }
76 fn confirm(&mut self, snapshot: &[u8]) -> Result<(), CommitError> {
76 fn stamp(&mut self) -> io::Result<Stamp> {
77 Stamp::of(&self.read()?).map_err(io::Error::other)
78 }
79 fn confirm(&mut self, base: &Stamp) -> Result<(), CommitError> {
7780 phase("confirm-before");
78 let result = onestore::confirm_snapshot(self, snapshot);
81 let result = onestore::confirm(self, base);
7982 phase("confirm-after");
8083 result
8184 }
8285}
8386
8487fn report(cache: &Replica, root: &Path) -> Result<(), Box<dyn std::error::Error>> {
85 let local = view(&cache.snapshot()?)?;
88 let (_, _, local) = cached(cache)?;
8689 let remote = view(&fs::read(root.join("remote.one"))?)?;
8790 let (status, revision) = match cache.status(1)? {
8891 Some(EditStatus::Pending) => ("pending", None),
......@@ -90,14 +93,13 @@ fn report(cache: &Replica, root: &Path) -> Result<(), Box<dyn std::error::Error>
9093 ("uncertain", Some(revision.to_string()))
9194 }
9295 Some(EditStatus::Published { revision }) => ("published", Some(revision.to_string())),
93 Some(EditStatus::Conflict(_)) => ("conflict", None),
9496 Some(EditStatus::Archived { archive }) => ("archived", Some(archive)),
9597 None => ("missing", None),
9698 };
9799 println!(
98100 "{}",
99101 json!({"event":"state", "status":status, "revision":revision,
100 "local_text":local.text, "remote_text":remote.text, "remote_revision":remote.revision.to_string(),
102 "local_text":local, "remote_text":remote.text, "remote_revision":remote.revision.to_string(),
101103 "pending":cache.pending()?.iter().map(|pending| match &pending.edit.ops[..] {
102104 [onestore::op::Op::Page { op: onestore::op::PageOp::Text { range, with, .. }, .. }] =>
103105 json!({"id":pending.id,"replacement":with,"range":[range.start,range.end]}),
crates/notebook/examples/save_probe.rs+32-30
......@@ -1,12 +1,13 @@
1//! Times importing a generated page into a copy of a section, then saving a one-character
2//! edit of it: `save_probe SECTION PARAGRAPHS`.
1//! Times importing a generated page into a copy of a section, then typing one character
2//! into it until durable and queued: `save_probe SECTION PARAGRAPHS`.
33
44use notebook::session::Section;
55use onestore::{
66 document::{Format, Layout},
7 op::{Edit, Op, PageOp},
78 page::{
89 Outline, Page, PageObject, PageParagraph, Paragraph, ParagraphContent, TextObject,
9 text::{Edit, new_id},
10 text::new_id,
1011 },
1112};
1213use std::time::{Duration, Instant};
......@@ -49,6 +50,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
4950 identity: None,
5051 created: None,
5152 margin_origin: [0.0; 2],
53 color: None,
5254 objects: vec![PageObject::Outline(Outline {
5355 id: new_id()?,
5456 title: false,
......@@ -67,38 +69,38 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
6769 let start = Instant::now();
6870 let space = section.import_page(&imported, "Probe")?;
6971 println!("import {count}: {:?}", start.elapsed());
70 let before = section.page(space)?;
71 let edit = |page: &Page, word: &str| -> Result<Page, Box<dyn std::error::Error>> {
72 let mut after = page.clone();
73 let text = after
74 .objects
75 .iter_mut()
76 .find_map(|object| match object {
77 PageObject::Outline(outline) if outline.paragraphs.len() == count => {
78 outline.paragraphs[count / 2].text_mut()
79 }
80 _ => None,
81 })
82 .ok_or("the imported outline is missing")?;
83 let format = text.text.format_at(0)?.clone();
84 text.text.apply(Edit {
85 range: 0..0,
86 replacement: Paragraph::new(word.into(), format),
87 })?;
88 Ok(after)
72 let text = section
73 .page(space)?
74 .objects
75 .iter()
76 .find_map(|object| match object {
77 PageObject::Outline(outline) if outline.paragraphs.len() == count => {
78 outline.paragraphs[count / 2].text().map(|text| text.id)
79 }
80 _ => None,
81 })
82 .ok_or("the imported outline is missing")?;
83 let typed = |word: &str| Edit {
84 at: 133_000_000_000_000_000,
85 ops: vec![Op::Page {
86 space,
87 op: PageOp::Text {
88 text,
89 range: 0..0,
90 with: word.into(),
91 },
92 }],
8993 };
90 let after = edit(&before, "x")?;
9194 let start = Instant::now();
92 section.save(space, &before, &after, "Probe")?;
93 println!("save: {:?}", start.elapsed());
94 let next = edit(&after, "y")?;
95 section.replica().apply("Probe", typed("x"))?;
96 println!("typed, durable: {:?}", start.elapsed());
9597 let start = Instant::now();
96 section.queue_save(space, after, next, "Probe")?;
97 println!("queue_save: {:?}", start.elapsed());
98 section.apply("Probe", typed("y"))?;
99 println!("typed, queued: {:?}", start.elapsed());
98100 let deadline = start + Duration::from_secs(600);
99101 while Instant::now() < deadline {
100 if let Some((_, save)) = section.saved().pop() {
101 println!("queued save done: {:?} {save:?}", start.elapsed());
102 if section.replica().pending()?.len() == 2 {
103 println!("queued edit durable: {:?}", start.elapsed());
102104 break;
103105 }
104106 std::thread::sleep(Duration::from_millis(1));
crates/notebook/examples/session_client.rs+36-37
......@@ -1,15 +1,16 @@
1//! Drives a section the way the application does: open it through a session, save
2//! page-model edits, wait for their publication and report the page texts.
1//! Drives a section the way the application does: open it through a session, apply
2//! edits, wait for their publication and report the page texts.
33//! `session_client SECTION CACHE_DIR LABEL` edits the first body paragraph of every
44//! page whose title starts with "Move" and adds one outline per page; each launch
55//! reports the pending queue it found and the receipts it obtained.
66
77use notebook::{
88 EditStatus,
9 session::{Event, Save, Section},
9 session::{Event, Section},
1010};
1111use onestore::{
1212 ExGuid,
13 op::PageOp,
1314 page::{Outline, Page, PageObject, PageParagraph, ParagraphContent, TextObject, text::new_id},
1415};
1516use std::{
......@@ -46,28 +47,9 @@ fn texts(page: &Page) -> Vec<String> {
4647 out
4748}
4849
49fn edit(page: &mut Page, label: &str) {
50/// Types `label` before the first body paragraph and adds an outline below the body.
51fn edit(page: &Page, label: &str) -> Vec<PageOp> {
5052 let text = body_text(page).expect("a body paragraph");
51 for object in &mut page.objects {
52 let PageObject::Outline(outline) = object else {
53 continue;
54 };
55 if let Some(paragraph) = outline
56 .paragraphs
57 .iter_mut()
58 .find(|p| p.text().is_some_and(|t| t.id == text))
59 {
60 let target = paragraph.text_mut().unwrap();
61 let format = target.text.format_at(0).unwrap().clone();
62 target
63 .text
64 .apply(onestore::page::text::Edit {
65 range: 0..0,
66 replacement: onestore::page::Paragraph::new(format!("{label} "), format),
67 })
68 .unwrap();
69 }
70 }
7153 let template = page
7254 .objects
7355 .iter()
......@@ -110,12 +92,21 @@ fn edit(page: &mut Page, label: &str) {
11092 paragraphs: vec![paragraph],
11193 unsupported: Vec::new(),
11294 };
113 let at = page
114 .objects
115 .iter()
116 .position(|o| matches!(o, PageObject::Title(_)))
117 .unwrap_or(page.objects.len());
118 page.objects.insert(at, PageObject::Outline(outline));
95 let title = page.objects.iter().find_map(|object| match object {
96 PageObject::Title(title) => Some(title.id),
97 _ => None,
98 });
99 vec![
100 PageOp::Text {
101 text,
102 range: 0..0,
103 with: format!("{label} "),
104 },
105 PageOp::Add {
106 object: PageObject::Outline(outline),
107 before: title,
108 },
109 ]
119110}
120111
121112fn main() -> Result<(), Box<dyn std::error::Error>> {
......@@ -138,13 +129,21 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
138129 .iter()
139130 .filter(|(_, title, _)| title.starts_with("Move"))
140131 {
141 let before = section.page(*space)?;
142 let mut after = before.clone();
143 edit(&mut after, label);
144 match section.save(*space, &before, &after, "session client")? {
145 Save::Queued(id) => queued.push((id, *space, title.clone(), texts(&after))),
146 other => return Err(format!("{title}: {other:?}").into()),
132 let mut page = section.page(*space)?;
133 let ops = edit(&page, label);
134 for op in &ops {
135 onestore::op::predict(&mut page, op)?;
147136 }
137 let unix = std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH)?;
138 let edit = onestore::op::Edit {
139 at: (unix.as_secs() + 11_644_473_600) * 10_000_000,
140 ops: ops
141 .into_iter()
142 .map(|op| onestore::op::Op::Page { space: *space, op })
143 .collect(),
144 };
145 let id = section.replica().apply("session client", edit)?;
146 queued.push((id, *space, title.clone(), texts(&page)));
148147 }
149148 let deadline = Instant::now() + Duration::from_secs(120);
150149 let mut receipts = Vec::new();
......@@ -176,7 +175,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
176175 }
177176 Event::Failed(error) => return Err(error.into()),
178177 Event::Rejected { error, .. } => return Err(error.into()),
179 Event::Refreshed | Event::Changed(_) => {}
178 Event::Changed(_) => {}
180179 }
181180 }
182181 // A batch reports its newest edit; each edit's receipt is its own.
crates/notebook/examples/smb_concurrent_client.rs+6-1
......@@ -1,5 +1,7 @@
11#[path = "../../onestore/examples/support/concurrent.rs"]
22mod concurrent;
3#[path = "support/edit.rs"]
4mod edit;
35
46use notebook::smb::{Client, Credentials};
57use std::{env, time::Duration};
......@@ -16,7 +18,10 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
1618 &args,
1719 |path| client.read(path, 256 * 1024 * 1024),
1820 |path, source, space, object, range, replacement| {
19 client.commit_text(path, source, space, object, range, replacement)
21 client.commit_transaction(
22 path,
23 &edit::replaced(source, space, object, range, replacement)?,
24 )
2025 },
2126 )
2227}
crates/notebook/examples/smb_offline_client.rs+47-43
......@@ -3,7 +3,7 @@ mod concurrent;
33
44use notebook::smb::{Client, Credentials};
55use notebook::{EditStatus, Error, Remote, Replica, SmbRemote};
6use onestore::{CommitError, CommitState, ExGuid, RevisionIndex, Store, Transaction};
6use onestore::{CommitError, CommitState, ExGuid, RevisionIndex, Stamp, Store, Transaction};
77use serde_json::json;
88use std::{
99 env,
......@@ -18,7 +18,7 @@ mod support {
1818 pub mod view;
1919}
2020use concurrent::{DocumentView, document_view};
21use support::view::view;
21use support::view::{cached, view};
2222
2323impl DocumentView {
2424 fn changes(&self, before: &Self) -> Self {
......@@ -79,6 +79,13 @@ impl<R: Remote> Remote for Traced<R> {
7979 Ok(bytes)
8080 }
8181 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
82 // Publication reads nothing while the stamp holds; observe an unseen base once.
83 if Stamp::of(&self.read).ok().as_ref() != Some(transaction.base()) {
84 self.read().map_err(|error| CommitError {
85 state: CommitState::NotCommitted,
86 error,
87 })?;
88 }
8289 let mut image = self.read.clone();
8390 transaction.apply(&mut image).map_err(|error| CommitError {
8491 state: CommitState::NotCommitted,
......@@ -193,10 +200,22 @@ impl<R: Remote> Remote for Traced<R> {
193200 thread::sleep(Duration::from_millis(10));
194201 }
195202 }
203 if result.is_ok() {
204 self.read = image;
205 self.before = Some((after.text, documents));
206 }
196207 result
197208 }
198 fn confirm(&mut self, snapshot: &[u8]) -> Result<(), CommitError> {
209 fn stamp(&mut self) -> io::Result<Stamp> {
210 self.remote.stamp()
211 }
212 fn confirm(&mut self, base: &Stamp) -> Result<(), CommitError> {
199213 let captured = (|| -> Result<_, Box<dyn std::error::Error>> {
214 // A confirmation follows the read of its image, except for edits that changed nothing.
215 if Stamp::of(&self.read).ok().as_ref() != Some(base) {
216 self.read = self.remote.read()?;
217 }
218 let snapshot = &self.read[..];
200219 let store = Store::parse(snapshot)?;
201220 let index = RevisionIndex::parse(&store)?;
202221 let revisions = index
......@@ -240,7 +259,7 @@ impl<R: Remote> Remote for Traced<R> {
240259 error: io::Error::other(error.to_string()),
241260 })?;
242261 let started = now();
243 let result = self.remote.confirm(snapshot);
262 let result = self.remote.confirm(base);
244263 println!(
245264 "{}",
246265 json!({"event":"remote_confirm", "started_us":started, "finished_us":now(), "revisions":revisions, "current_revisions":current, "capture":capture.as_ref().and_then(|path| path.file_name()).map(|name| name.to_string_lossy()), "text":self.before.as_ref().map(|(text,_)|text),
......@@ -343,6 +362,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
343362 }
344363 })?;
345364 let mut ids = Vec::new();
365 let mut tokens = Vec::new();
346366 let result = (|| -> Result<(), Box<dyn std::error::Error>> {
347367 let mut generated = 0;
348368 let mut received = 0;
......@@ -380,62 +400,46 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
380400 8
381401 };
382402 if generated < operations && pending.len() < capacity {
383 let source = cache.snapshot()?;
384 let target = view(&source)?;
385 let at = u32::try_from(target.text.encode_utf16().count())?;
403 let (space, object, text) = cached(&cache)?;
404 let at = u32::try_from(text.encode_utf16().count())?;
386405 let token = format!(" [{}:{}]", args[2], generated);
387406 let started = now();
388407 match cache.apply(
389408 "Offline document writer",
390 append(target.space, target.object, at, &token),
409 append(space, object, at, &token),
391410 ) {
392411 Ok(id) => {
393412 println!(
394413 "{}",
395 json!({"event":"local_commit", "id":id, "operation":generated, "space":target.space.to_string(), "object":target.object.to_string(), "before":target.text, "token":token, "started_us":started, "finished_us":now()})
414 json!({"event":"local_commit", "id":id, "operation":generated, "space":space.to_string(), "object":object.to_string(), "before":text, "token":token, "started_us":started, "finished_us":now()})
396415 );
397416 ids.push(id);
417 tokens.push(token);
398418 generated += 1;
399419 }
400420 other => return Err(format!("Unexpected local result: {other:?}").into()),
401421 }
402422 }
403 if let Some(conflict) = cache.conflict()? {
404 // Concurrent appends meet at the end of the text: take the remote text and
405 // append the queued tokens it lacks after it again.
406 let remote = view(&cache.remote_snapshot()?)?;
407 let tokens: Vec<String> = cache
408 .pending()?
423 if received == ids.len() && cache.pending()?.is_empty() {
424 // Concurrent appends meet at the end of the text: a merge keeps the remote's
425 // text and this writer's on a conflict page. Append the published tokens
426 // the text lacks after it again.
427 let (space, object, text) = cached(&cache)?;
428 let missing: Vec<String> = tokens
409429 .iter()
410 .flat_map(|queued| queued.edit.ops.iter())
411 .filter_map(|op| match op {
412 onestore::op::Op::Page {
413 op: onestore::op::PageOp::Text { with, .. },
414 ..
415 } => Some(with.clone()),
416 _ => None,
417 })
418 .filter(|token| !remote.text.contains(token.as_str()))
430 .filter(|token| !text.contains(token.as_str()))
431 .cloned()
419432 .collect();
420 match cache.resolve(conflict.id, notebook::Resolution::Theirs) {
421 Ok(()) => {
422 let mut at = u32::try_from(remote.text.encode_utf16().count())?;
423 for token in &tokens {
424 cache.apply(
425 "Offline document writer",
426 append(remote.space, remote.object, at, token),
427 )?;
428 at += u32::try_from(token.encode_utf16().count())?;
429 }
430 println!(
431 "{}",
432 json!({"event":"reviewed_append", "id":conflict.id, "remote":remote.text, "tokens":tokens, "at_us":now()})
433 );
434 }
435 Err(Error::Io(error))
436 if [io::ErrorKind::WouldBlock, io::ErrorKind::InvalidInput]
437 .contains(&error.kind()) => {}
438 Err(error) => return Err(error.into()),
433 let mut at = u32::try_from(text.encode_utf16().count())?;
434 for token in &missing {
435 ids.push(cache.apply("Offline document writer", append(space, object, at, token))?);
436 at += u32::try_from(token.encode_utf16().count())?;
437 }
438 if !missing.is_empty() {
439 println!(
440 "{}",
441 json!({"event":"reviewed_append", "remote":text, "tokens":missing, "at_us":now()})
442 );
439443 }
440444 }
441445 if generated == operations && received == ids.len() {
crates/notebook/examples/smb_publish_probe.rs created+110
......@@ -0,0 +1,110 @@
1//! Publishes typed keystrokes to a Samba lab one at a time and reports each publication's
2//! wall time: `smb_publish_probe ADDRESS KEYSTROKES [CONTROL]`. With `CONTROL`, the
3//! `tools/smb-proxy.py` control file, each publication is marked in the proxy's trace.
4
5use notebook::{
6 EditStatus, Replica, SmbRemote,
7 smb::{Client, Credentials},
8};
9use onestore::op::{Edit, Op, PageOp};
10use std::{
11 env, fs,
12 time::{Duration, Instant, SystemTime, UNIX_EPOCH},
13};
14
15fn main() -> Result<(), Box<dyn std::error::Error>> {
16 let args: Vec<_> = env::args().skip(1).collect();
17 let [address, count, control @ ..] = &args[..] else {
18 return Err("smb_publish_probe ADDRESS KEYSTROKES [CONTROL]".into());
19 };
20 let count: u32 = count.parse()?;
21 let mark = |phase: String| -> std::io::Result<()> {
22 if let [control] = control {
23 fs::write(
24 format!("{control}.tmp"),
25 format!("{{\"phase\":\"{phase}\"}}"),
26 )?;
27 fs::rename(format!("{control}.tmp"), control)?;
28 // The proxy polls its control file every 50 ms.
29 std::thread::sleep(Duration::from_millis(120));
30 }
31 Ok(())
32 };
33 let connect = || {
34 Client::connect(
35 address,
36 "agent",
37 Credentials::default(),
38 Duration::from_secs(5),
39 )
40 };
41 let path = format!(
42 "probe-{}.one",
43 SystemTime::now().duration_since(UNIX_EPOCH)?.as_nanos()
44 );
45 let source = onestore::create_section(&path, "Body", "Probe")?;
46 connect()?.create(&path, &source)?;
47 let directory = tempfile::tempdir()?;
48 let replica = Replica::create(directory.path().join("cache.sqlite"), &source)?;
49 let space = replica.pages()?[0].0;
50 let text = replica
51 .page(space)?
52 .objects
53 .iter()
54 .find_map(|object| match object {
55 onestore::page::PageObject::Outline(outline) => outline
56 .paragraphs
57 .iter()
58 .find_map(|p| p.text().map(|t| t.id)),
59 _ => None,
60 })
61 .ok_or("no text")?;
62 let mut remote = SmbRemote::new(connect()?, path.clone(), 1 << 24);
63 mark("idle".into())?;
64 let mut idle = Vec::new();
65 for _ in 0..count {
66 let started = Instant::now();
67 assert_eq!(replica.sync_once(&mut remote)?.edit, None);
68 idle.push(started.elapsed());
69 }
70 let mut published = Vec::new();
71 for n in 0..count {
72 replica.apply(
73 "Probe",
74 Edit {
75 at: 133_000_000_000_000_000,
76 ops: vec![Op::Page {
77 space,
78 op: PageOp::Text {
79 text,
80 range: 4 + n..4 + n,
81 with: "x".into(),
82 },
83 }],
84 },
85 )?;
86 mark(format!("publish-{n}"))?;
87 let started = Instant::now();
88 let synced = replica.sync_once(&mut remote)?;
89 published.push(started.elapsed());
90 assert!(matches!(
91 synced.edit,
92 Some((_, EditStatus::Published { .. }))
93 ));
94 }
95 mark("done".into())?;
96 let quantiles = |mut times: Vec<Duration>| {
97 times.sort();
98 let at = |q: f64| times[((times.len() - 1) as f64 * q) as usize].as_secs_f64() * 1000.0;
99 format!(
100 "p50 {:.2} ms, p90 {:.2} ms, max {:.2} ms",
101 at(0.5),
102 at(0.9),
103 at(1.0)
104 )
105 };
106 println!("idle poll: {}", quantiles(idle));
107 println!("publish: {}", quantiles(published));
108 connect()?.delete(&path)?;
109 Ok(())
110}
crates/notebook/examples/smb_reconnect_client.rs+17-17
......@@ -1,9 +1,11 @@
11#[path = "../../onestore/examples/support/concurrent.rs"]
22mod concurrent;
3#[path = "support/edit.rs"]
4mod edit;
35
46use notebook::smb::{Client, Credentials};
57use onestore::{
6 CommitError, CommitState, ExGuid, RevisionIndex, Store,
8 CommitError, CommitState, ExGuid, RevisionIndex, Stamp, Store,
79 document::{Document, Kind},
810};
911use serde_json::json;
......@@ -155,14 +157,14 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
155157 &args,
156158 read,
157159 |path, source, space, object, range, replacement| {
158 let outcome = client.borrow().as_ref().unwrap().commit_text(
159 path,
160 source,
161 space,
162 object,
163 range,
164 replacement,
165 );
160 let outcome =
161 edit::replaced(source, space, object, range, replacement).and_then(|transaction| {
162 client
163 .borrow()
164 .as_ref()
165 .unwrap()
166 .commit_transaction(path, &transaction)
167 });
166168 let Err(error) = outcome else {
167169 return Ok(());
168170 };
......@@ -209,14 +211,12 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
209211 error: io::Error::other(failure.to_string()),
210212 })?;
211213 if published {
212 let confirmation = client.borrow().as_ref().unwrap().commit_text(
213 path,
214 &current,
215 space,
216 object,
217 0..0,
218 "",
219 );
214 let confirmation = Stamp::of(&current)
215 .map_err(|failure| CommitError {
216 state: CommitState::NotCommitted,
217 error: io::Error::other(failure.to_string()),
218 })
219 .and_then(|base| client.borrow().as_ref().unwrap().confirm(path, &base));
220220 if let Err(failure) = confirmation
221221 && failure.state != CommitState::Committed
222222 {
crates/notebook/examples/support/edit.rs created+49
......@@ -0,0 +1,49 @@
1use onestore::{
2 Arena, CommitError, CommitState, ExGuid, Section, Transaction,
3 op::{Edit, Op, PageOp},
4};
5use std::{
6 io,
7 ops::Range,
8 time::{SystemTime, UNIX_EPOCH},
9};
10
11/// The transaction replacing `range` of `text` in `image`, typed now.
12pub fn replaced(
13 image: &[u8],
14 space: ExGuid,
15 text: ExGuid,
16 range: Range<u32>,
17 with: &str,
18) -> Result<Transaction, CommitError> {
19 let failed = |error: String| CommitError {
20 state: CommitState::NotCommitted,
21 error: io::Error::new(io::ErrorKind::InvalidData, error),
22 };
23 let arena = Arena::default();
24 let mut section = Section::open(&arena, image.to_vec()).map_err(|e| failed(e.to_string()))?;
25 let at = SystemTime::now()
26 .duration_since(UNIX_EPOCH)
27 .map_err(|e| failed(e.to_string()))?
28 .as_nanos() as u64
29 / 100
30 + 116_444_736_000_000_000;
31 let edit = Edit {
32 at,
33 ops: vec![Op::Page {
34 space,
35 op: PageOp::Text {
36 text,
37 range,
38 with: with.to_owned(),
39 },
40 }],
41 };
42 section
43 .apply("Concurrent client", &edit)
44 .map_err(|e| failed(format!("{e:?}")))?;
45 section
46 .seal()
47 .map_err(|e| failed(e.to_string()))?
48 .ok_or_else(|| failed("The edit changed nothing".into()))
49}
crates/notebook/examples/support/view.rs+41
......@@ -1,3 +1,5 @@
1#![allow(dead_code)]
2
13use onestore::{
24 ExGuid, RevisionIndex, Store,
35 document::{Document, Kind},
......@@ -55,3 +57,42 @@ pub fn view(bytes: &[u8]) -> Result<View, Box<dyn std::error::Error>> {
5557 }
5658 Ok(found.pop().unwrap())
5759}
60
61/// The concurrent-edit paragraph of `page`: its text object and text.
62fn in_page(page: &onestore::page::Page) -> Option<(ExGuid, String)> {
63 let mut pending: Vec<&onestore::page::PageParagraph> = page
64 .objects
65 .iter()
66 .flat_map(|object| match object {
67 onestore::page::PageObject::Outline(outline) => outline.paragraphs.iter().collect(),
68 _ => Vec::new(),
69 })
70 .collect();
71 pending.reverse();
72 while let Some(paragraph) = pending.pop() {
73 if let Some(text) = paragraph.text()
74 && text.text.text().starts_with("Concurrent edits:")
75 {
76 return Some((text.id, text.text.text().to_owned()));
77 }
78 if let onestore::page::ParagraphContent::Table(table) = &paragraph.content {
79 pending.extend(table.rows.iter().flat_map(|row| &row.cells).flat_map(|cell| &cell.paragraphs).rev());
80 }
81 }
82 None
83}
84
85/// The concurrent-edit paragraph as the queued edits leave it: its page, text object and
86/// text.
87pub fn cached(cache: &notebook::Replica) -> Result<(ExGuid, ExGuid, String), Box<dyn std::error::Error>> {
88 let mut found = Vec::new();
89 for (space, ..) in cache.pages()? {
90 if let Some((object, text)) = in_page(&cache.page(space)?) {
91 found.push((space, object, text));
92 }
93 }
94 match <[_; 1]>::try_from(found) {
95 Ok([found]) => Ok(found),
96 Err(_) => Err("Expected one concurrent-edit paragraph".into()),
97 }
98}
crates/notebook/src/bin/onestore-diagnostic.rs+104-11
......@@ -1,4 +1,11 @@
1use onestore::{ExGuid, Insertion, PreparedEdit, TextAttribute};
1use onestore::{
2 Arena, ExGuid, Section, TextAttribute,
3 document::{Format, Layout},
4 op::{Op, PageOp},
5 page::{
6 Outline, PageObject, PageParagraph, Paragraph, ParagraphContent, TextObject, text::new_id,
7 },
8};
29use serde::Deserialize;
310use serde_json::{Value, json};
411use std::{
......@@ -40,6 +47,33 @@ enum Action {
4047 },
4148}
4249
50/// A new paragraph holding `text` in OneNote's default text style.
51fn paragraph(text: String) -> Result<PageParagraph, Box<dyn std::error::Error>> {
52 let format = Format {
53 font: Some("Calibri".into()),
54 font_size: Some(11.0),
55 language: Some(0x409),
56 ..Default::default()
57 };
58 Ok(PageParagraph {
59 id: new_id()?,
60 parent: None,
61 level: 1,
62 style: None,
63 format: Format::default(),
64 content: ParagraphContent::Text(TextObject {
65 id: new_id()?,
66 date_field: None,
67 text: Paragraph::new(text, format),
68 tags: Vec::new(),
69 }),
70 lists: Vec::new(),
71 tags: Vec::new(),
72 media: Default::default(),
73 collapsed: false,
74 })
75}
76
4377fn run(args: &[std::ffi::OsString]) -> Result<Value, Box<dyn std::error::Error>> {
4478 if let [mode, root] = args
4579 && mode == "catalog"
......@@ -79,34 +113,93 @@ fn run(args: &[std::ffi::OsString]) -> Result<Value, Box<dyn std::error::Error>>
79113 let edit: Edit = serde_json::from_reader(io::stdin().lock())?;
80114 let sid = edit.space;
81115 let oid = edit.object;
82 let prepared = match edit.action {
116 let (op, author) = match edit.action {
83117 Action::Text {
84118 start,
85119 end,
86120 replacement,
87 } => PreparedEdit::text(&bytes, sid, oid, start..end, &replacement)?,
121 } => (
122 PageOp::Text {
123 text: oid,
124 range: start..end,
125 with: replacement,
126 },
127 String::new(),
128 ),
88129 Action::Format {
89130 start,
90131 end,
91132 attributes,
92 } => PreparedEdit::format(&bytes, sid, oid, start..end, &attributes)?,
133 } => (
134 PageOp::Format {
135 text: oid,
136 range: start..end,
137 set: attributes,
138 clear: Vec::new(),
139 },
140 String::new(),
141 ),
93142 Action::Paragraph {
94143 before,
95144 text,
96145 author,
97 } => PreparedEdit::insert(
98 &bytes,
99 sid,
100 &Insertion::paragraph(oid, before, &text, &author)?,
101 )?,
146 } => (
147 PageOp::Insert {
148 container: oid,
149 before,
150 paragraphs: vec![paragraph(text)?],
151 },
152 author,
153 ),
102154 Action::Outline { x, y, text, author } => {
103 PreparedEdit::insert(&bytes, sid, &Insertion::outline(oid, x, y, &text, &author)?)?
155 let store = onestore::Store::parse(&bytes)?;
156 let index = onestore::RevisionIndex::parse(&store)?;
157 if !onestore::document::Document::parse(&index)?
158 .pages()?
159 .contains(&(sid, oid))
160 {
161 return Err("Choose the page to hold the outline".into());
162 }
163 (
164 PageOp::Add {
165 object: PageObject::Outline(Outline {
166 id: new_id()?,
167 title: false,
168 min_width: None,
169 layout: Layout {
170 x: Some(x),
171 y: Some(y),
172 ..Default::default()
173 },
174 indents: Vec::new(),
175 paragraphs: vec![paragraph(text)?],
176 unsupported: Vec::new(),
177 }),
178 before: None,
179 },
180 author,
181 )
104182 }
105183 };
184 let arena = Arena::default();
185 let mut section = Section::open(&arena, bytes)?;
186 let unix = std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH)?;
187 let at = (unix.as_secs() + 11_644_473_600) * 10_000_000 + u64::from(unix.subsec_nanos() / 100);
188 section.apply(
189 &author,
190 &onestore::op::Edit {
191 at,
192 ops: vec![Op::Page { space: sid, op }],
193 },
194 )?;
195 let transaction = section.seal()?;
106196 if check {
107197 return Ok(json!({"ok": true}));
108198 }
109 Ok(match prepared.commit_file(&args[1]) {
199 let Some(transaction) = transaction else {
200 return Ok(json!({"ok": true, "state": "Unchanged"}));
201 };
202 Ok(match transaction.commit_file(&args[1]) {
110203 Ok(()) => json!({"ok": true, "state": "Committed"}),
111204 Err(error) => json!({"ok": false, "state": format!("{:?}", error.state),
112205 "kind": format!("{:?}", error.error.kind()), "error": error.error.to_string()}),
crates/notebook/src/lib.rs+11-16
......@@ -29,7 +29,7 @@ mod schema;
2929pub mod session;
3030pub use recovery::{Recovery, RecoverySummary};
3131mod sync;
32pub use sync::{ConflictKind, EditStatus, Remote, Synced};
32pub use sync::{EditStatus, Remote, Synced};
3333mod worker;
3434mod working;
3535#[cfg(feature = "smb")]
......@@ -72,19 +72,8 @@ pub struct PendingEdit {
7272 pub edit: Edit,
7373}
7474
75/// Unpublished work the remote section no longer accepts: the batch holding edit `id`
76/// changed page `space` where the remote changed it too.
77#[derive(Debug, Clone, PartialEq, Eq)]
78pub struct Conflict {
79 pub id: u64,
80 pub space: ExGuid,
81 pub kind: ConflictKind,
82}
83
84/// How a conflict or an uncertain attempt ends. `Mine` keeps the local pages: a conflicted
85/// page is rewritten from the remote page to the local one; a released attempt publishes
86/// again. `Theirs` drops the local edits to the conflicted page, or the whole unpublished
87/// branch of a released attempt.
75/// How an uncertain attempt ends after review: `Mine` publishes it again, `Theirs` drops
76/// the whole unpublished branch.
8877#[derive(Debug, Clone, Copy, PartialEq, Eq)]
8978pub enum Resolution {
9079 Mine,
......@@ -161,6 +150,7 @@ impl Replica {
161150 // the file as it was.
162151 match version {
163152 migrate::VERSION => migrate::migrate(&mut connection, path)?,
153 schema::PREVIOUS => schema::upgrade(&mut connection)?,
164154 schema::VERSION => {}
165155 _ => {
166156 return Err(io::Error::new(
......@@ -248,9 +238,14 @@ impl Replica {
248238 self.ask(|reply| working::Request::Pages { reply })
249239 }
250240
241 /// The conflict pages of each page that has them (`onestore::Section::conflicts`).
242 pub fn conflicts(&self) -> Result<Vec<(ExGuid, Vec<onestore::ConflictPage>)>> {
243 self.ask(|reply| working::Request::Conflicts { reply })
244 }
245
251246 /// The section image the queued edits leave, the unsealed ones sealed as one more
252 /// revision whose identities differ per call: O(section), for tests and diagnostics.
253 pub fn snapshot(&self) -> Result<Vec<u8>> {
247 /// revision whose identities differ per call: O(section).
248 pub(crate) fn snapshot(&self) -> Result<Vec<u8>> {
254249 self.ask(|reply| working::Request::Flush { reply })?;
255250 working::image(&*self.lock()?)
256251 }
crates/notebook/src/merge.rs+354-296
......@@ -2,11 +2,16 @@
22//! section when the objects it reads are as the base image had them; text ops on a text
33//! the remote also changed shift past the remote's changes when they stay clear of them;
44//! a move, deletion or outline setting the remote already made is dropped as done.
5//! Anything else is a conflict: the queue stays on its base until it is resolved.
5//! An op on a page the remote changed where it did is dropped, with every later op naming
6//! the objects it named, and the page conflicts: as OneNote 2010 does, the remote's version
7//! stays the page and the local one becomes a conflict page under it (`conflict_page`).
8//! Page-list edits merge as OneNote 2010 merges page series: a page the remote moved keeps
9//! the remote's place, and a page the queue placed goes before the next page in the queue's
10//! order the remote left in place (`corpus/conflict-page/native-pages`, `native-restore`).
611//! Objects the queue itself created are not in the base, so nothing remote can have
712//! changed them.
813
9use crate::{ConflictKind, PendingEdit, Result};
14use crate::{PendingEdit, Result};
1015use onestore::{
1116 ExGuid, OutlineEdit, PageCreation, PageEdit, PagePosition, Section,
1217 document::{Format, Layout, Tag},
......@@ -21,143 +26,113 @@ use std::{
2126 ops::Range,
2227};
2328
24/// A conflict being resolved: from edit `first` on, ops on `space` are dropped. Keeping
25/// mine replaces the first of them by the ops rewriting the remote page to `target` (or
26/// putting `target` back as a page where the remote removed it), or, for the section's
27/// page list, keeps what of each section op still applies.
28pub(crate) struct Resolving<'p> {
29 pub first: u64,
30 pub space: ExGuid,
31 pub mine: bool,
32 pub target: Option<&'p Page>,
33}
29/// The remote changed what an op reads: the op does not replay.
30struct Conflicting;
3431
35pub(crate) enum Outcome {
36 /// Every edit applied; these edits' ops changed. An edit whose ops were all dropped
37 /// stays, empty, to be published and acknowledged with its batch.
38 Applied(Vec<(u64, Edit)>),
39 Conflict {
40 id: u64,
41 space: ExGuid,
42 kind: ConflictKind,
43 },
32/// The queue replayed on the remote section.
33pub(crate) struct Merged {
34 /// Edits whose ops changed. An edit whose ops were all dropped stays, empty, to be
35 /// published and acknowledged with its batch.
36 pub rewritten: Vec<(u64, Edit)>,
37 /// Pages whose local version the remote could not take, with who made the first edit
38 /// that did not replay and the objects the dropped ops named.
39 pub conflicts: BTreeMap<ExGuid, (String, BTreeSet<ExGuid>)>,
4440}
4541
4642/// Replays `edits` on `new`, the remote section; `old` is the base they were made on.
43/// Ops on the pages in `converged`, which the remote holds as the edits leave them, are done.
4744pub(crate) fn rebase(
4845 old: &mut Section<'_>,
4946 new: &mut Section<'_>,
5047 edits: &[PendingEdit],
5148 converged: &BTreeSet<ExGuid>,
52 resolving: Option<Resolving<'_>>,
53) -> Result<Outcome> {
54 let root = new.root();
49) -> Result<Merged> {
5550 let before: BTreeMap<ExGuid, ExGuid> = old.revisions().collect();
5651 let after: BTreeMap<ExGuid, ExGuid> = new.revisions().collect();
5752 let orders = (order(old)?, order(new)?);
5853 let mut state = Replay {
5954 before,
6055 after,
56 local: orders.0.clone(),
57 unmoved: unmoved(old, new)?,
6158 orders,
6259 created: BTreeSet::new(),
6360 diffs: BTreeMap::new(),
6461 };
65 let mut rewritten = Vec::new();
66 let mut resolved = false;
62 let mut merged = Merged {
63 rewritten: Vec::new(),
64 conflicts: BTreeMap::new(),
65 };
6766 for queued in edits {
68 let branch = resolving
69 .as_ref()
70 .filter(|resolving| queued.id >= resolving.first);
7167 let mut ops = Vec::new();
7268 let mut changed = false;
7369 for op in &queued.edit.ops {
74 let conflict = |kind, space| Outcome::Conflict {
75 id: queued.id,
76 space,
77 kind,
70 let apply = |state: &mut Replay, new: &mut Section<'_>, op: &Op| {
71 state.apply(new, &queued.author, queued.edit.at, op)
7872 };
79 if let Some(resolving) = branch {
80 let dropped = match op {
81 Op::Page { space, .. } => *space == resolving.space,
82 Op::Section(_) => resolving.space == root,
83 };
84 if dropped {
85 changed = true;
86 let rewrite = match op {
87 Op::Page { space, .. } if !resolved && resolving.mine => {
88 resolving.target.map(|target| (*space, target))
73 match op {
74 Op::Page { space, op: page_op } => {
75 if converged.contains(space) {
76 changed = true;
77 continue;
78 }
79 // An op naming what a dropped op named would build on it.
80 let named = named(page_op);
81 if let Some((_, dropped)) = merged.conflicts.get_mut(space)
82 && named.iter().any(|id| dropped.contains(id))
83 {
84 dropped.extend(named);
85 changed = true;
86 continue;
87 }
88 let replayed = match state.check(old, new, *space, page_op)? {
89 Ok(None) => {
90 changed = true;
91 continue;
92 }
93 Ok(Some(mapped)) => {
94 let mapped = Op::Page {
95 space: *space,
96 op: mapped,
97 };
98 apply(&mut state, new, &mapped)?.then_some(mapped)
8999 }
90 _ => None,
100 Err(Conflicting) => None,
91101 };
92 resolved |= matches!(op, Op::Page { .. });
93 if let Some((space, target)) = rewrite {
94 if !state.listed(space) {
95 // Kept while the remote removed it: the page comes back.
96 match state.restore(
97 new,
98 space,
99 target,
100 &queued.author,
101 queued.edit.at,
102 )? {
103 Ok(restored) => ops.extend(restored),
104 Err(kind) => return Ok(conflict(kind, space)),
105 }
106 continue;
102 match replayed {
103 Some(mapped) => {
104 changed |= mapped != *op;
105 ops.push(mapped);
107106 }
108 let current = new.page(space)?;
109 for op in onestore::op::lower_page(&current, target)
110 .map_err(|error| OpError::Unsupported(error.message))?
111 {
112 let op = Op::Page { space, op };
113 if let Some(kind) =
114 state.apply(new, &queued.author, queued.edit.at, &op)?
115 {
116 return Ok(conflict(kind, space));
117 }
118 ops.push(op);
107 None => {
108 changed = true;
109 merged
110 .conflicts
111 .entry(*space)
112 .or_insert_with(|| (queued.author.clone(), BTreeSet::new()))
113 .1
114 .extend(named);
119115 }
120116 }
121 if let (Op::Section(op), true) = (op, resolving.mine) {
122 // Keeping mine for the section list: the first form that applies.
123 for op in state.salvage(new, op) {
124 let op = Op::Section(op);
125 if state
126 .apply(new, &queued.author, queued.edit.at, &op)?
127 .is_none()
128 {
129 ops.push(op);
130 break;
131 }
117 }
118 Op::Section(section_op) => {
119 advance(&mut state.local, section_op);
120 // The first form of the edit the remote's page list takes, or none.
121 let mut kept = None;
122 for form in state.forms(new, section_op)? {
123 let form = Op::Section(form);
124 if apply(&mut state, new, &form)? {
125 kept = Some(form);
126 break;
132127 }
133128 }
134 continue;
135 }
136 }
137 let space = match op {
138 Op::Page { space, .. } => *space,
139 Op::Section(_) => root,
140 };
141 if matches!(op, Op::Page { .. }) && converged.contains(&space) {
142 changed = true;
143 continue;
144 }
145 let mapped = match state.check(old, new, op)? {
146 Ok(Some(mapped)) => mapped,
147 Ok(None) => {
148 changed = true;
149 continue;
129 changed |= kept.as_ref() != Some(op);
130 ops.extend(kept);
150131 }
151 Err(kind) => return Ok(conflict(kind, space)),
152 };
153 changed |= mapped != *op;
154 if let Some(kind) = state.apply(new, &queued.author, queued.edit.at, &mapped)? {
155 return Ok(conflict(kind, space));
156132 }
157 ops.push(mapped);
158133 }
159134 if changed {
160 rewritten.push((
135 merged.rewritten.push((
161136 queued.id,
162137 Edit {
163138 at: queued.edit.at,
......@@ -166,7 +141,176 @@ pub(crate) fn rebase(
166141 ));
167142 }
168143 }
169 Ok(Outcome::Applied(rewritten))
144 Ok(merged)
145}
146
147/// The stored objects a page op names; a conflict page marks those it holds.
148fn named(op: &PageOp) -> Vec<ExGuid> {
149 match op {
150 PageOp::Text { text, .. }
151 | PageOp::Format { text, .. }
152 | PageOp::Link { text, .. }
153 | PageOp::Equation { text, .. }
154 | PageOp::Split { text, .. } => vec![*text],
155 PageOp::Date { fields, .. } => fields.iter().map(|(text, _)| *text).collect(),
156 PageOp::Color(_) => Vec::new(),
157 PageOp::Join { left, right } => vec![*left, *right],
158 PageOp::Insert { container, .. } => vec![*container],
159 PageOp::Move { object, .. } | PageOp::Delete { object } | PageOp::Outline { object, .. } => {
160 vec![*object]
161 }
162 PageOp::Level { paragraph, .. }
163 | PageOp::Paragraph { paragraph, .. }
164 | PageOp::Style { paragraph, .. }
165 | PageOp::List { paragraph, .. } => vec![*paragraph],
166 PageOp::Tags { target, .. } => vec![*target],
167 PageOp::Add { object, .. } => vec![object.id()],
168 PageOp::Picture { picture: object, .. }
169 | PageOp::Attachment {
170 attachment: object, ..
171 }
172 | PageOp::Strokes { ink: object, .. } => vec![*object],
173 PageOp::Table { table, edit } => match edit {
174 TableEdit::Cell { cell, .. } => vec![*cell],
175 TableEdit::DeleteRow(row) => vec![*row],
176 _ => vec![*table],
177 },
178 }
179}
180
181/// The edit keeping `page`, the version of page `space` the remote could not take, and
182/// applies it to `new`: a conflict page under the remote's page marking `objects` where
183/// `page` holds them, or, where the remote removed the page, the page put back as a copy
184/// where `local`, the section as the queue leaves it, has it. Content outside the page
185/// model stays on the remote's page only; `at` is now.
186#[allow(clippy::too_many_arguments)]
187pub(crate) fn conflict_page(
188 new: &mut Section<'_>,
189 old: &mut Section<'_>,
190 local: &mut Section<'_>,
191 space: ExGuid,
192 page: &Page,
193 author: &str,
194 objects: &BTreeSet<ExGuid>,
195 at: u64,
196) -> Result<Edit> {
197 let mut page = page.clone();
198 page.objects.retain(|object| !matches!(object, PageObject::Unsupported(_)));
199 let listed = order(new)?;
200 let ops = if listed.iter().any(|(listed, _)| *listed == space) {
201 let index = Index::of(&page);
202 let mut marked: Vec<ExGuid> = objects
203 .iter()
204 .copied()
205 .filter(|id| index.nodes.contains_key(id))
206 .collect();
207 let copy = page.copy_with(&mut marked)?;
208 let titled = page
209 .objects
210 .iter()
211 .any(|object| matches!(object, PageObject::Title(_)));
212 vec![Op::Section(SectionOp::Conflict {
213 of: space,
214 creation: PageCreation::new(None, titled.then_some(page.title.as_str()), author)?,
215 page: copy,
216 objects: marked,
217 })]
218 } else {
219 // A copy under fresh identities, as OneNote 2010 puts a removed page back.
220 let (base, unmoved) = (order(old)?, unmoved(old, new)?);
221 let queued = order(local)?;
222 let at = queued.iter().position(|(queued, _)| *queued == space);
223 let before = at
224 .map_or(&[][..], |at| &queued[at + 1..])
225 .iter()
226 .map(|(next, _)| *next)
227 .find(|next| {
228 listed.iter().any(|(listed, _)| listed == next)
229 && (unmoved.contains(next) || base.iter().all(|(page, _)| page != next))
230 });
231 let creation = PageCreation::new(before, Some(&page.title), author)?;
232 let mut ops = vec![Op::Section(SectionOp::Import {
233 creation: creation.clone(),
234 page: page.copy()?,
235 })];
236 if let Some(level) = at.map(|at| queued[at].1).filter(|level| *level > 1) {
237 ops.push(Op::Section(SectionOp::Pages(vec![PageEdit::set_level(
238 creation.space(),
239 level,
240 )?])));
241 }
242 ops
243 };
244 let edit = Edit { at, ops };
245 new.apply(author, &edit)?;
246 Ok(edit)
247}
248
249/// Pages the base and the remote both list that the remote left in place: in the same
250/// series at the same level, in base order among themselves. OneNote 2010 gives a page it
251/// moves a series of its own; a page moved within its series is out of base order.
252fn unmoved(old: &mut Section<'_>, new: &mut Section<'_>) -> Result<BTreeSet<ExGuid>> {
253 let series = (old.series()?, new.series()?);
254 let base: BTreeMap<ExGuid, (usize, u32)> = order(old)?
255 .into_iter()
256 .enumerate()
257 .map(|(at, (space, level))| (space, (at, level)))
258 .collect();
259 let kept: Vec<(ExGuid, usize)> = order(new)?
260 .into_iter()
261 .filter_map(|(space, level)| {
262 let (at, before) = base.get(&space)?;
263 (*before == level && series.0.get(&space) == series.1.get(&space)).then_some((space, *at))
264 })
265 .collect();
266 // A longest subsequence of `kept` in base order (patience sorting).
267 let mut tails: Vec<usize> = Vec::new();
268 let mut previous = vec![None; kept.len()];
269 for (i, (_, at)) in kept.iter().enumerate() {
270 let k = tails.partition_point(|tail| kept[*tail].1 < *at);
271 previous[i] = k.checked_sub(1).map(|k| tails[k]);
272 if k == tails.len() {
273 tails.push(i);
274 } else {
275 tails[k] = i;
276 }
277 }
278 let mut unmoved = BTreeSet::new();
279 let mut next = tails.last().copied();
280 while let Some(i) = next {
281 unmoved.insert(kept[i].0);
282 next = previous[i];
283 }
284 Ok(unmoved)
285}
286
287/// Applies a queued page-list edit to `local`, the page order as the queue leaves it.
288fn advance(local: &mut Order, op: &SectionOp) {
289 let insert = |local: &mut Order, page: (ExGuid, u32), before: Option<ExGuid>| {
290 let at = before
291 .and_then(|before| local.iter().position(|(space, _)| *space == before))
292 .unwrap_or(local.len());
293 local.insert(at, page);
294 };
295 match op {
296 SectionOp::Create(creation) | SectionOp::Import { creation, .. } => {
297 insert(local, (creation.space(), 1), creation.before());
298 }
299 SectionOp::Pages(edits) => {
300 for edit in edits {
301 let Some(at) = local.iter().position(|(space, _)| *space == edit.space()) else {
302 continue;
303 };
304 local[at].1 = edit.level();
305 if let PagePosition::Before(before) = edit.position() {
306 let page = local.remove(at);
307 insert(local, page, before);
308 }
309 }
310 }
311 SectionOp::Delete(spaces) => local.retain(|(space, _)| !spaces.contains(space)),
312 SectionOp::Conflict { .. } | SectionOp::Color(_) => {}
313 }
170314}
171315
172316fn order(section: &mut Section<'_>) -> Result<Order> {
......@@ -186,6 +330,10 @@ struct Replay {
186330 after: BTreeMap<ExGuid, ExGuid>,
187331 /// Page order of the base and the remote.
188332 orders: (Order, Order),
333 /// Page order as the queue leaves it, up to the op replaying.
334 local: Order,
335 /// Pages the remote left in place (`unmoved`).
336 unmoved: BTreeSet<ExGuid>,
189337 /// Page spaces the replayed edits created.
190338 created: BTreeSet<ExGuid>,
191339 /// Per page the remote changed, what it changed; `None` when it left the page alone.
......@@ -193,14 +341,8 @@ struct Replay {
193341}
194342
195343impl Replay {
196 /// Applies one op to the remote section; `Some` names why it did not apply.
197 fn apply(
198 &mut self,
199 new: &mut Section<'_>,
200 author: &str,
201 at: u64,
202 op: &Op,
203 ) -> Result<Option<ConflictKind>> {
344 /// Applies one op to the remote section; false when the section refused it.
345 fn apply(&mut self, new: &mut Section<'_>, author: &str, at: u64, op: &Op) -> Result<bool> {
204346 match new.apply(
205347 author,
206348 &Edit {
......@@ -210,64 +352,49 @@ impl Replay {
210352 ) {
211353 Ok(()) => {
212354 if let Op::Section(
213 SectionOp::Create(creation) | SectionOp::Import { creation, .. },
355 SectionOp::Create(creation)
356 | SectionOp::Import { creation, .. }
357 | SectionOp::Conflict { creation, .. },
214358 ) = op
215359 {
216360 self.created.insert(creation.space());
217361 }
218 Ok(None)
362 Ok(true)
219363 }
220364 Err(OpError::Failed(error)) => Err(error.into()),
221 Err(error) => Ok(Some(match op {
222 // A page list the remote reordered refuses the op, however it says so.
223 Op::Section(_) => ConflictKind::StructureChanged,
224 Op::Page { .. } if matches!(error, OpError::Unsupported(_)) => {
225 ConflictKind::UnsupportedEdit
226 }
227 Op::Page { space, .. } if !self.listed(*space) => ConflictKind::TargetUnavailable,
228 Op::Page { .. } => ConflictKind::ContentChanged,
229 })),
365 Err(_) => Ok(false),
230366 }
231367 }
232368
233 /// The op as it replays on the remote section, or why it cannot.
369 /// The page op as it replays on the remote section; `None` when the remote made it.
234370 fn check(
235371 &mut self,
236372 old: &mut Section<'_>,
237373 new: &mut Section<'_>,
238 op: &Op,
239 ) -> Result<std::result::Result<Option<Op>, ConflictKind>> {
240 match op {
241 Op::Page { space, op } => {
242 if !self.listed(*space) {
243 return Ok(Err(ConflictKind::TargetUnavailable));
244 }
245 let diff = match self.diffs.get_mut(space) {
246 Some(diff) => diff,
247 None => {
248 let diff = match (self.before.get(space), self.after.get(space)) {
249 (Some(before), Some(after)) if before != after => Some(Diff {
250 old: Index::of(&old.page(*space)?),
251 new: Index::of(&new.page(*space)?),
252 regions: BTreeMap::new(),
253 }),
254 _ => None,
255 };
256 self.diffs.entry(*space).or_insert(diff)
257 }
374 space: ExGuid,
375 op: &PageOp,
376 ) -> Result<std::result::Result<Option<PageOp>, Conflicting>> {
377 if !self.listed(space) {
378 return Ok(Err(Conflicting));
379 }
380 let diff = match self.diffs.get_mut(&space) {
381 Some(diff) => diff,
382 None => {
383 let diff = match (self.before.get(&space), self.after.get(&space)) {
384 (Some(before), Some(after)) if before != after => Some(Diff {
385 old: Index::of(&old.page(space)?),
386 new: Index::of(&new.page(space)?),
387 regions: BTreeMap::new(),
388 }),
389 _ => None,
258390 };
259 Ok(match diff {
260 None => Ok(Some(Op::Page {
261 space: *space,
262 op: op.clone(),
263 })),
264 Some(diff) => diff
265 .check(op)
266 .map(|op| op.map(|op| Op::Page { space: *space, op })),
267 })
391 self.diffs.entry(space).or_insert(diff)
268392 }
269 Op::Section(section) => Ok(self.section(section).map(|()| Some(op.clone()))),
270 }
393 };
394 Ok(match diff {
395 None => Ok(Some(op.clone())),
396 Some(diff) => diff.check(op),
397 })
271398 }
272399
273400 /// Whether the remote section lists the page, or the replayed edits created it; a page
......@@ -276,151 +403,82 @@ impl Replay {
276403 self.created.contains(&space) || self.orders.1.iter().any(|(page, _)| *page == space)
277404 }
278405
279 /// Puts `page`, which the remote removed, back into the section and returns the ops
280 /// that did: a copy under fresh identities at its level, before the first page after it
281 /// that the remote still has and that can take a page before it, or last.
282 fn restore(
283 &mut self,
284 new: &mut Section<'_>,
285 space: ExGuid,
286 page: &Page,
287 author: &str,
288 at: u64,
289 ) -> Result<std::result::Result<Vec<Op>, ConflictKind>> {
290 let old = self.orders.0.clone();
291 let position = old.iter().position(|(listed, _)| *listed == space);
292 let anchors = position
293 .map_or(&[][..], |position| &old[position + 1..])
294 .iter()
295 .map(|(next, _)| Some(*next))
296 .filter(|next| next.is_some_and(|next| self.listed(next)))
297 .chain([None])
298 .collect::<Vec<_>>();
299 let copy = page.copy()?;
300 let mut refusal = ConflictKind::StructureChanged;
301 for before in anchors {
302 let creation = PageCreation::new(before, Some(&page.title), author)?;
303 let import = Op::Section(SectionOp::Import {
304 creation: creation.clone(),
305 page: copy.clone(),
306 });
307 match self.apply(new, author, at, &import)? {
308 Some(kind) => refusal = kind,
309 None => {
310 let mut ops = vec![import];
311 if let Some(level) = position.map(|at| old[at].1).filter(|level| *level > 1) {
312 let indent = Op::Section(SectionOp::Pages(vec![PageEdit::set_level(
313 creation.space(),
314 level,
315 )?]));
316 if let Some(kind) = self.apply(new, author, at, &indent)? {
317 return Ok(Err(kind));
318 }
319 ops.push(indent);
320 }
321 return Ok(Ok(ops));
322 }
323 }
324 }
325 Ok(Err(refusal))
406 /// Whether the remote moved or re-leveled a page the base listed.
407 fn placed(&self, page: ExGuid) -> bool {
408 !self.unmoved.contains(&page) && self.orders.0.iter().any(|(listed, _)| *listed == page)
326409 }
327410
328 fn section(&self, op: &SectionOp) -> std::result::Result<(), ConflictKind> {
329 let (old, new) = &self.orders;
330 let find =
331 |list: &[(ExGuid, u32)], space: ExGuid| list.iter().position(|(s, _)| *s == space);
332 match op {
333 SectionOp::Create(_) | SectionOp::Import { .. } => Ok(()),
334 SectionOp::Delete(pages) => {
335 for page in pages {
336 if find(new, *page).is_none() && !self.created.contains(page) {
337 return Err(ConflictKind::TargetUnavailable);
338 }
339 if self.before.get(page) != self.after.get(page) {
340 return Err(ConflictKind::ContentChanged);
341 }
411 /// Where a page the queue put before `before` goes on the remote: before the first page
412 /// from `before` on, in the queue's order, that the remote lists where the base had it,
413 /// or last, as OneNote 2010 places a page series only one side changed.
414 fn anchor(&self, listed: &Order, page: ExGuid, before: Option<ExGuid>) -> Option<ExGuid> {
415 let from = self.local.iter().position(|(space, _)| Some(*space) == before)?;
416 self.local[from..].iter().map(|(space, _)| *space).find(|space| {
417 *space != page && !self.placed(*space) && listed.iter().any(|(listed, _)| listed == space)
418 })
419 }
420
421 /// Forms of a section op to try on the remote section, in order: a page placed by
422 /// `anchor`, the page edits of pages the remote still has and did not move, removals of
423 /// pages the remote still has (a page it changed is not removed), a conflict page's
424 /// content as a page of its own where the remote removed its page.
425 fn forms(&self, new: &mut Section<'_>, op: &SectionOp) -> Result<Vec<SectionOp>> {
426 let conflicts: BTreeSet<ExGuid> = new
427 .conflicts()?
428 .into_iter()
429 .flat_map(|(_, pages)| pages.into_iter().map(|page| page.space))
430 .collect();
431 let listed = order(new)?;
432 let placed = |creation: &PageCreation| {
433 creation.reposition(self.anchor(&listed, creation.space(), creation.before()))
434 };
435 Ok(match op {
436 SectionOp::Create(creation) => vec![SectionOp::Create(placed(creation)?)],
437 SectionOp::Import { creation, page } => vec![SectionOp::Import {
438 creation: placed(creation)?,
439 page: page.clone(),
440 }],
441 SectionOp::Color(_) => vec![op.clone()],
442 SectionOp::Conflict {
443 of, creation, page, ..
444 } => vec![if self.listed(*of) {
445 op.clone()
446 } else {
447 SectionOp::Import {
448 creation: creation.clone(),
449 page: page.clone(),
342450 }
343 Ok(())
344 }
451 }],
452 // A page the remote moved or re-leveled keeps the remote's placement.
345453 SectionOp::Pages(edits) => {
346 // Where the batch puts each page, from the base order.
347 let mut wanted: Vec<ExGuid> = old.iter().map(|(space, _)| *space).collect();
454 let mut kept = Vec::new();
348455 for edit in edits {
349 if let PagePosition::Before(before) = edit.position()
350 && let Some(at) = wanted.iter().position(|space| *space == edit.space())
456 if self.placed(edit.space())
457 || listed.iter().all(|(listed, _)| *listed != edit.space())
351458 {
352 let space = wanted.remove(at);
353 let at = before
354 .and_then(|before| wanted.iter().position(|space| *space == before))
355 .unwrap_or(wanted.len());
356 wanted.insert(at, space);
357 }
358 }
359 let wanted: Vec<(ExGuid, u32)> =
360 wanted.into_iter().map(|space| (space, 0)).collect();
361 for edit in edits {
362 let space = edit.space();
363 let Some(at) = find(old, space) else {
364459 continue;
365 };
366 let Some(now) = find(new, space) else {
367 return Err(ConflictKind::TargetUnavailable);
368 };
369 if old[at].1 != new[now].1 && new[now].1 != edit.level() {
370 return Err(ConflictKind::StructureChanged);
371 }
372 if edit.position() != PagePosition::Keep {
373 // Each page the remote also has must keep its side of the moved page,
374 // or already be on the side the batch wants.
375 let side = |list: &[(ExGuid, u32)], peer: ExGuid| {
376 Some(find(list, peer)? < find(list, space)?)
377 };
378 let conflicting = old.iter().any(|(peer, _)| {
379 *peer != space
380 && find(new, *peer).is_some()
381 && side(new, *peer) != side(old, *peer)
382 && side(new, *peer) != side(&wanted, *peer)
383 });
384 if conflicting {
385 return Err(ConflictKind::StructureChanged);
386 }
387460 }
461 kept.push(match edit.position() {
462 PagePosition::Keep => edit.clone(),
463 PagePosition::Before(before) => edit.reposition(
464 PagePosition::Before(self.anchor(&listed, edit.space(), before)),
465 edit.level(),
466 )?,
467 });
388468 }
389 Ok(())
390 }
391 }
392 }
393
394 /// Forms of a section op to try on the remote section list, in order: as it is, a
395 /// creation appended instead, moves and removals of the pages still there.
396 fn salvage(&self, new: &mut Section<'_>, op: &SectionOp) -> Vec<SectionOp> {
397 let pages: BTreeSet<ExGuid> = new
398 .pages()
399 .map(|pages| pages.into_iter().map(|(space, ..)| space).collect())
400 .unwrap_or_default();
401 let present = |space: &ExGuid| pages.contains(space) || self.created.contains(space);
402 match op {
403 SectionOp::Create(creation) => std::iter::once(creation.clone())
404 .chain(creation.reposition(None).ok())
405 .map(SectionOp::Create)
406 .collect(),
407 SectionOp::Import { .. } => vec![op.clone()],
408 SectionOp::Pages(edits) => {
409 let edits: Vec<PageEdit> = edits
410 .iter()
411 .filter(|edit| present(&edit.space()))
412 .cloned()
413 .collect();
414 if edits.is_empty() {
469 if kept.is_empty() {
415470 Vec::new()
416471 } else {
417 vec![SectionOp::Pages(edits)]
472 vec![SectionOp::Pages(kept)]
418473 }
419474 }
420475 SectionOp::Delete(deleted) => {
421476 let deleted: Vec<ExGuid> = deleted
422477 .iter()
423 .filter(|space| present(space))
478 .filter(|space| {
479 self.listed(**space) && self.before.get(space) == self.after.get(space)
480 || conflicts.contains(space)
481 })
424482 .copied()
425483 .collect();
426484 if deleted.is_empty() {
......@@ -429,7 +487,7 @@ impl Replay {
429487 vec![SectionOp::Delete(deleted)]
430488 }
431489 }
432 }
490 })
433491 }
434492}
435493
......@@ -491,7 +549,7 @@ impl Diff {
491549
492550 /// Where the remote changed a text's characters and formats; its tags and date field
493551 /// are not positions an op names.
494 fn region(&mut self, text: ExGuid) -> std::result::Result<Option<&mut Changes>, ConflictKind> {
552 fn region(&mut self, text: ExGuid) -> std::result::Result<Option<&mut Changes>, Conflicting> {
495553 if !self.regions.contains_key(&text) {
496554 if self.ours(text) {
497555 return Ok(None);
......@@ -500,7 +558,7 @@ impl Diff {
500558 self.old.nodes.get(&text).map(|node| &node.own),
501559 self.new.nodes.get(&text).map(|node| &node.own),
502560 ) else {
503 return Err(ConflictKind::ContentChanged);
561 return Err(Conflicting);
504562 };
505563 if a.text == b.text {
506564 return Ok(None);
......@@ -512,21 +570,20 @@ impl Diff {
512570 }
513571
514572 /// The op as it replays on the remote page; `None` when the remote already made it.
515 fn check(&mut self, op: &PageOp) -> std::result::Result<Option<PageOp>, ConflictKind> {
516 use ConflictKind::ContentChanged as Changed;
517 let require = |ok: bool| if ok { Ok(()) } else { Err(Changed) };
573 fn check(&mut self, op: &PageOp) -> std::result::Result<Option<PageOp>, Conflicting> {
574 let require = |ok: bool| if ok { Ok(()) } else { Err(Conflicting) };
518575 let mut op = op.clone();
519576 match &mut op {
520577 PageOp::Text { text, range, with } => {
521578 if let Some(region) = self.region(*text)? {
522 let mapped = region.map(range).ok_or(Changed)?;
579 let mapped = region.map(range).ok_or(Conflicting)?;
523580 region.replaced(range, with.encode_utf16().count() as u32);
524581 *range = mapped;
525582 }
526583 }
527584 PageOp::Format { text, range, .. } => {
528585 if let Some(region) = self.region(*text)? {
529 *range = region.map(range).ok_or(Changed)?;
586 *range = region.map(range).ok_or(Conflicting)?;
530587 }
531588 }
532589 PageOp::Link { text, .. } | PageOp::Equation { text, .. } => {
......@@ -547,7 +604,7 @@ impl Diff {
547604 if let Some(region) = self.region(*text)? {
548605 // The cut moves with the character after it: text the remote typed at
549606 // the cut stays left of it.
550 let cut = region.map(&(*at..*at + 1)).ok_or(Changed)?;
607 let cut = region.map(&(*at..*at + 1)).ok_or(Conflicting)?;
551608 let right_changes = region.split(*at);
552609 if !right_changes.0.is_empty() {
553610 self.regions.insert(*right, right_changes);
......@@ -565,7 +622,8 @@ impl Diff {
565622 self.region(text)?;
566623 }
567624 }
568 PageOp::Insert { .. } | PageOp::Add { .. } => {}
625 // A page's colour is the latest one set, whichever side set it.
626 PageOp::Insert { .. } | PageOp::Add { .. } | PageOp::Color(_) => {}
569627 PageOp::Move {
570628 object,
571629 parent,
......@@ -633,7 +691,7 @@ impl Diff {
633691 }
634692 _ => false,
635693 };
636 return if done { Ok(None) } else { Err(Changed) };
694 return if done { Ok(None) } else { Err(Conflicting) };
637695 }
638696 require(before.is_some())?
639697 }
crates/notebook/src/migrate.rs+58-17
......@@ -1,7 +1,8 @@
11//! Converts a schema-14 cache, whose queue holds whole-page edits over a working image,
22//! to this schema's op queue. The cache is exported first; each queued page is lowered to
33//! ops against the page the conversion has so far, and every converted page must equal
4//! the page in the old working image, or nothing changes.
4//! the page in the old working image, or nothing changes. A page edit schema 14 held in
5//! conflict becomes the conflict page a merge makes now, the remote's page staying.
56
67use crate::{Result, base, queue, schema};
78use onestore::{
......@@ -87,7 +88,6 @@ struct Converted {
8788 edit: Edit,
8889 /// Schema 14's publication evidence, when this edit was attempted.
8990 attempted: Option<String>,
90 conflict: Option<(i64, ExGuid)>,
9191}
9292
9393pub(crate) fn migrate(connection: &mut Connection, path: &Path) -> Result<()> {
......@@ -107,6 +107,53 @@ pub(crate) fn migrate(connection: &mut Connection, path: &Path) -> Result<()> {
107107 Ok(())
108108}
109109
110/// The conflict page keeping `local`, the version of page `space` its merge with `remote`
111/// did not take, marking the texts `remote` does not hold as they are.
112fn conflict(space: ExGuid, remote: &Page, local: &Page, author: &str) -> Result<Op> {
113 let texts = |page: &Page| -> BTreeMap<ExGuid, String> {
114 let mut texts = BTreeMap::new();
115 let mut pending: Vec<&onestore::page::PageParagraph> = Vec::new();
116 for object in &page.objects {
117 match object {
118 onestore::page::PageObject::Outline(outline) => pending.extend(&outline.paragraphs),
119 onestore::page::PageObject::Title(title) => {
120 pending.extend(title.outlines.iter().flat_map(|outline| &outline.paragraphs))
121 }
122 _ => {}
123 }
124 }
125 while let Some(paragraph) = pending.pop() {
126 match &paragraph.content {
127 onestore::page::ParagraphContent::Text(text) => {
128 texts.insert(text.id, format!("{:?}", text.text));
129 }
130 onestore::page::ParagraphContent::Table(table) => pending.extend(
131 table.rows.iter().flat_map(|row| &row.cells).flat_map(|cell| &cell.paragraphs),
132 ),
133 _ => {}
134 }
135 }
136 texts
137 };
138 let kept = texts(remote);
139 let mut objects: Vec<ExGuid> = texts(local)
140 .into_iter()
141 .filter(|(id, text)| kept.get(id) != Some(text))
142 .map(|(id, _)| id)
143 .collect();
144 let page = local.copy_with(&mut objects)?;
145 let titled = local
146 .objects
147 .iter()
148 .any(|object| matches!(object, onestore::page::PageObject::Title(_)));
149 Ok(Op::Section(SectionOp::Conflict {
150 of: space,
151 creation: PageCreation::new(None, titled.then_some(local.title.as_str()), author)?,
152 page,
153 objects,
154 }))
155}
156
110157fn convert(transaction: &rusqlite::Transaction<'_>) -> Result<()> {
111158 let (base, patch): (Vec<u8>, Vec<u8>) =
112159 transaction.query_row("SELECT base, working FROM replica WHERE id=1", [], |row| {
......@@ -118,9 +165,9 @@ fn convert(transaction: &rusqlite::Transaction<'_>) -> Result<()> {
118165 Ok((row.get(0)?, row.get(1)?))
119166 })
120167 .optional()?;
121 let conflicts: BTreeMap<i64, i64> = transaction
122 .prepare("SELECT edit_id, kind FROM conflicts")?
123 .query_map([], |row| Ok((row.get(0)?, row.get(1)?)))?
168 let conflicts: std::collections::BTreeSet<i64> = transaction
169 .prepare("SELECT edit_id FROM conflicts")?
170 .query_map([], |row| row.get(0))?
124171 .collect::<rusqlite::Result<_>>()?;
125172 let queued: Vec<(i64, String, String)> = transaction
126173 .prepare("SELECT id, space, operation FROM edits ORDER BY id")?
......@@ -146,6 +193,11 @@ fn convert(transaction: &rusqlite::Transaction<'_>) -> Result<()> {
146193 let operation: Operation = serde_json::from_str(&operation)
147194 .map_err(|error| invalid(format!("Queued edit {id} is unreadable: {error}")))?;
148195 let (author, ops) = match operation {
196 Operation::Page(intent) if conflicts.contains(&id) => {
197 let current = section.page(space)?;
198 let op = conflict(space, &current, &intent.after, &intent.author)?;
199 (intent.author, vec![op])
200 }
149201 Operation::Page(intent) => {
150202 let current = section.page(space)?;
151203 let ops = onestore::op::lower_page(&current, &intent.after)?;
......@@ -185,17 +237,12 @@ fn convert(transaction: &rusqlite::Transaction<'_>) -> Result<()> {
185237 }
186238 sealed = Some(section.seal()?);
187239 }
188 // A conflicting page-list edit is the section's conflict, whichever page it names.
189 let conflicted = match edit.ops.first() {
190 Some(Op::Section(_)) => section.root(),
191 _ => space,
192 };
240 // A schema-14 conflict is queued work: the next rebase keeps both versions.
193241 converted.push(Converted {
194242 id,
195243 author,
196244 edit,
197245 attempted,
198 conflict: conflicts.get(&id).map(|kind| (*kind, conflicted)),
199246 });
200247 }
201248 let listed = |image: &[u8]| -> Result<Vec<ExGuid>> {
......@@ -258,12 +305,6 @@ fn convert(transaction: &rusqlite::Transaction<'_>) -> Result<()> {
258305 } else {
259306 batch = Some(current);
260307 }
261 if let Some((kind, space)) = edit.conflict {
262 transaction.execute(
263 "UPDATE batches SET conflict=?1, space=?2 WHERE id=?3",
264 params![kind, space.to_string(), current],
265 )?;
266 }
267308 queue::insert(
268309 transaction,
269310 Some(crate::unsigned(edit.id)?),
crates/notebook/src/queue.rs+5-1
......@@ -74,7 +74,7 @@ fn slots(edit: &mut Edit, mut visit: impl FnMut(&mut Payload)) {
7474 } => cells(list, &mut visit),
7575 _ => {}
7676 },
77 Op::Section(SectionOp::Import { page, .. }) => {
77 Op::Section(SectionOp::Import { page, .. } | SectionOp::Conflict { page, .. }) => {
7878 for added in &mut page.objects {
7979 object(added, &mut visit);
8080 }
......@@ -235,8 +235,12 @@ pub(crate) fn spaces(edit: &Edit, root: onestore::ExGuid) -> Vec<onestore::ExGui
235235 SectionOp::Create(creation) | SectionOp::Import { creation, .. } => {
236236 spaces.push(creation.space());
237237 }
238 SectionOp::Conflict { of, creation, .. } => {
239 spaces.extend([*of, creation.space()]);
240 }
238241 SectionOp::Pages(edits) => spaces.extend(edits.iter().map(|edit| edit.space())),
239242 SectionOp::Delete(pages) => spaces.extend(pages),
243 SectionOp::Color(_) => {}
240244 }
241245 }
242246 }
crates/notebook/src/recovery.rs+4-6
......@@ -8,7 +8,6 @@ const RECOVERY_ID: u32 = 0x4f4e4552;
88#[derive(Debug, Clone, Copy, PartialEq, Eq, serde::Serialize)]
99pub struct RecoverySummary {
1010 pub queued_edits: u64,
11 pub conflicts: u64,
1211 pub uncertain_edits: u64,
1312 pub published_receipts: u64,
1413 /// Bytes of the image the queued edits apply to.
......@@ -38,7 +37,8 @@ impl Recovery {
3837 io::Error::new(io::ErrorKind::InvalidData, "Not a recovery archive").into(),
3938 );
4039 }
41 if version != schema::VERSION {
40 // A schema-15 archive differs only in columns nothing reads.
41 if ![schema::PREVIOUS, schema::VERSION].contains(&version) {
4242 return Err(io::Error::new(
4343 io::ErrorKind::InvalidData,
4444 format!(
......@@ -166,18 +166,16 @@ fn summary(connection: &Connection) -> Result<RecoverySummary> {
166166 let length = |image| -> Result<u64> {
167167 Ok(base::stamp(connection, image)?.map_or(0, |stamp| stamp.length))
168168 };
169 let (queued_edits, conflicts, uncertain_edits, published_receipts): (i64, i64, i64, i64) =
169 let (queued_edits, uncertain_edits, published_receipts): (i64, i64, i64) =
170170 connection.query_row(
171171 "SELECT (SELECT count(*) FROM edits),
172 (SELECT count(*) FROM edits JOIN batches ON batches.id=edits.batch WHERE conflict IS NOT NULL),
173172 (SELECT count(*) FROM edits JOIN batches ON batches.id=edits.batch WHERE attempted=1),
174173 (SELECT count(*) FROM receipts)",
175174 [],
176 |row| Ok((row.get(0)?, row.get(1)?, row.get(2)?, row.get(3)?)),
175 |row| Ok((row.get(0)?, row.get(1)?, row.get(2)?)),
177176 )?;
178177 Ok(RecoverySummary {
179178 queued_edits: unsigned(queued_edits)?,
180 conflicts: unsigned(conflicts)?,
181179 uncertain_edits: unsigned(uncertain_edits)?,
182180 published_receipts: unsigned(published_receipts)?,
183181 base_bytes: length(base::Image::Base)?,
crates/notebook/src/schema.rs+59-6
......@@ -1,7 +1,9 @@
11use super::*;
2use rusqlite::Transaction;
2use rusqlite::{OptionalExtension, Transaction};
33
4pub(crate) const VERSION: u32 = 15;
4pub(crate) const VERSION: u32 = 16;
5/// The schema whose batches still recorded review conflicts.
6pub(crate) const PREVIOUS: u32 = 15;
57
68/// Tables a schema-14 cache shares with this one.
79pub(crate) const KEPT: &str = "
......@@ -28,11 +30,8 @@ pub(crate) const QUEUE: &str = "
2830 sealed TEXT,
2931 revisions TEXT,
3032 attempted INTEGER NOT NULL DEFAULT 0 CHECK(attempted IN (0,1)),
31 conflict INTEGER CHECK(conflict BETWEEN 0 AND 3),
32 space TEXT,
3333 CHECK((sealed IS NULL) = (revisions IS NULL)),
34 CHECK(attempted=0 OR sealed IS NOT NULL),
35 CHECK((conflict IS NULL) = (space IS NULL))
34 CHECK(attempted=0 OR sealed IS NOT NULL)
3635 ) STRICT;
3736 CREATE TABLE edits (
3837 id INTEGER PRIMARY KEY AUTOINCREMENT CHECK(id>0),
......@@ -51,3 +50,57 @@ pub(crate) fn create(transaction: &Transaction<'_>) -> Result<()> {
5150 transaction.execute_batch(KEPT)?;
5251 Ok(())
5352}
53
54/// Converts a schema-15 cache: its batches lose the review conflict and its page, which
55/// a merge no longer records. A conflict it held is queued work the next rebase keeps as
56/// conflict pages. Foreign keys stay off while `edits` briefly references the old table.
57pub(crate) fn upgrade(connection: &mut Connection) -> Result<()> {
58 connection.execute_batch("PRAGMA foreign_keys=OFF")?;
59 let upgraded = (|| -> Result<()> {
60 let transaction = connection.transaction_with_behavior(TransactionBehavior::Exclusive)?;
61 let sequence: Option<i64> = transaction
62 .query_row(
63 "SELECT seq FROM sqlite_sequence WHERE name='batches'",
64 [],
65 |row| row.get(0),
66 )
67 .optional()?;
68 transaction.execute_batch(
69 "CREATE TABLE upgraded (
70 id INTEGER PRIMARY KEY AUTOINCREMENT CHECK(id>0),
71 sealed TEXT,
72 revisions TEXT,
73 attempted INTEGER NOT NULL DEFAULT 0 CHECK(attempted IN (0,1)),
74 CHECK((sealed IS NULL) = (revisions IS NULL)),
75 CHECK(attempted=0 OR sealed IS NOT NULL)
76 ) STRICT;
77 INSERT INTO upgraded(id, sealed, revisions, attempted)
78 SELECT id, sealed, revisions, attempted FROM batches;
79 DROP TABLE batches;
80 ALTER TABLE upgraded RENAME TO batches;",
81 )?;
82 if let Some(sequence) = sequence {
83 transaction.execute(
84 "UPDATE sqlite_sequence SET seq=max(seq, ?1) WHERE name='batches'",
85 [sequence],
86 )?;
87 }
88 let dangling: bool = transaction.query_row(
89 "SELECT EXISTS(SELECT 1 FROM pragma_foreign_key_check)",
90 [],
91 |row| row.get(0),
92 )?;
93 if dangling {
94 return Err(io::Error::new(
95 io::ErrorKind::InvalidData,
96 "The schema-15 cache holds edits without their batch",
97 )
98 .into());
99 }
100 transaction.pragma_update(None, "user_version", VERSION)?;
101 transaction.commit()?;
102 Ok(())
103 })();
104 connection.execute_batch("PRAGMA foreign_keys=ON")?;
105 upgraded
106}
crates/notebook/src/session.rs+263-280
......@@ -2,11 +2,11 @@
22//! publishes their edits to the section file in the background.
33
44use crate::{
5 Conflict, ConflictKind, EditStatus, Error, PendingEdit, Remote, Replica, Resolution, Result,
5 EditStatus, Error, PendingEdit, Remote, Replica, Resolution, Result,
66 SyncWorker, discover,
77};
88use onestore::{
9 CommitError, ExGuid, PageCreation, PreparedEdit, RevisionIndex, Stamp, Store, Transaction,
9 CommitError, ExGuid, PageCreation, RevisionIndex, Stamp, Store, Transaction,
1010 document::Document,
1111 op::{Edit, Op, SectionOp},
1212 page::Page,
......@@ -18,7 +18,6 @@ use std::{
1818 path::{Path, PathBuf},
1919 sync::{
2020 Arc, Mutex,
21 atomic::{AtomicUsize, Ordering},
2221 mpsc::{self, Receiver, Sender},
2322 },
2423 time::Duration,
......@@ -81,25 +80,19 @@ pub trait Storage: Send + Sync {
8180 fn create_directory(&self, path: &str) -> Result<()>;
8281 /// Renames or moves a file or directory; an existing target is an error.
8382 fn rename(&self, from: &str, to: &str) -> Result<()>;
83 /// Deletes a file or an empty directory.
84 fn delete(&self, path: &str) -> Result<()>;
8485 /// Names a section or TOC file for its notebook, as `onestore::place`.
8586 fn place(&self, path: &str, ancestor: [u8; 16], name: &str) -> Result<()>;
86 fn commit(&self, path: &str, edit: &PreparedEdit<'_>) -> Result<()>;
87 fn set_property(
88 &self,
89 path: &str,
90 source: &[u8],
91 space: ExGuid,
92 object: ExGuid,
93 property: u32,
94 value: &[u8],
95 ) -> Result<()>;
87 /// Publishes a transaction made on the file's current image.
88 fn commit(&self, path: &str, transaction: &Transaction) -> Result<()>;
9689}
9790
9891/// A password-protected section as `Notebook::unlock` read it.
9992#[cfg(feature = "protected")]
10093pub struct Unlocked {
10194 pub pages: Vec<(ExGuid, Page)>,
102 /// The stored section the pages came from, which a save must still find in place.
95 /// The stored section the pages came from, which an edit must still find in place.
10396 snapshot: Vec<u8>,
10497}
10598
......@@ -145,31 +138,21 @@ impl Storage for Directory {
145138 Ok(std::fs::rename(self.path(from), self.path(to))?)
146139 }
147140
141 fn delete(&self, path: &str) -> Result<()> {
142 let path = self.path(path);
143 Ok(if path.is_dir() {
144 std::fs::remove_dir(path)?
145 } else {
146 std::fs::remove_file(path)?
147 })
148 }
149
148150 fn place(&self, path: &str, ancestor: [u8; 16], name: &str) -> Result<()> {
149151 Ok(onestore::place_file(self.path(path), ancestor, name)?)
150152 }
151153
152 fn commit(&self, path: &str, edit: &PreparedEdit<'_>) -> Result<()> {
153 Ok(edit.commit_file(self.path(path))?)
154 }
155
156 fn set_property(
157 &self,
158 path: &str,
159 source: &[u8],
160 space: ExGuid,
161 object: ExGuid,
162 property: u32,
163 value: &[u8],
164 ) -> Result<()> {
165 Ok(onestore::commit_file_property(
166 self.path(path),
167 source,
168 space,
169 object,
170 property,
171 value,
172 )?)
154 fn commit(&self, path: &str, transaction: &Transaction) -> Result<()> {
155 Ok(transaction.commit_file(self.path(path))?)
173156 }
174157}
175158
......@@ -225,33 +208,18 @@ impl Storage for Share {
225208 Ok(self.client.rename(&self.path(from), &self.path(to))?)
226209 }
227210
211 fn delete(&self, path: &str) -> Result<()> {
212 Ok(self.client.delete(&self.path(path))?)
213 }
214
228215 fn place(&self, path: &str, ancestor: [u8; 16], name: &str) -> Result<()> {
229216 Ok(self.client.place(&self.path(path), ancestor, name)?)
230217 }
231218
232 fn commit(&self, path: &str, edit: &PreparedEdit<'_>) -> Result<()> {
219 fn commit(&self, path: &str, transaction: &Transaction) -> Result<()> {
233220 Ok(self
234221 .client
235 .commit_transaction(&self.path(path), &edit.transaction())?)
236 }
237
238 fn set_property(
239 &self,
240 path: &str,
241 source: &[u8],
242 space: ExGuid,
243 object: ExGuid,
244 property: u32,
245 value: &[u8],
246 ) -> Result<()> {
247 Ok(self.client.commit_property_bytes(
248 &self.path(path),
249 source,
250 space,
251 object,
252 property,
253 value,
254 )?)
222 .commit_transaction(&self.path(path), transaction)?)
255223 }
256224}
257225
......@@ -279,6 +247,22 @@ impl Notebook {
279247 Self::with(Box::new(Directory(root.clone())), Some(root), cache)
280248 }
281249
250 /// Creates a notebook in the new folder `root`, as OneNote 2010 creates one: a table of
251 /// contents in the notebook colour `color` (COLORREF) and "New Section 1" holding the
252 /// page `page` creates.
253 pub fn create(
254 root: impl AsRef<Path>,
255 cache: impl AsRef<Path>,
256 color: u32,
257 page: &PageCreation,
258 ) -> Result<Self> {
259 std::fs::create_dir(root.as_ref())?;
260 let mut notebook = Self::open(root, cache)?;
261 notebook.edit_toc("", &[onestore::TocEdit::Color(color)])?;
262 notebook.create_section("", "New Section 1", page)?;
263 Ok(notebook)
264 }
265
282266 /// Opens the notebook at `root` on the share `client` is connected to. Sections open
283267 /// through `Section::resume_smb` with the catalog's paths.
284268 #[cfg(feature = "smb")]
......@@ -393,18 +377,38 @@ impl Notebook {
393377
394378 fn edit_toc(&self, folder: &str, edits: &[onestore::TocEdit]) -> Result<()> {
395379 let (toc, _) = self.toc(folder)?;
396 let source = self.storage.read(&toc)?;
397 self.storage
398 .commit(&toc, &PreparedEdit::table_of_contents(&source, edits)?)
380 self.commit_toc(&toc, edits)
399381 }
400382
401 /// Creates `name.one` in `folder` with one empty page and lists it last in the folder's
402 /// TOC, as OneNote creates a section. Returns the new catalog path.
403 pub fn create_section(&mut self, folder: &str, name: &str, author: &str) -> Result<String> {
383 fn commit_toc(&self, toc: &str, edits: &[onestore::TocEdit]) -> Result<()> {
384 let source = self.storage.read(toc)?;
385 match onestore::edit_table_of_contents(&source, edits)? {
386 Some(transaction) => self.storage.commit(toc, &transaction),
387 None => Ok(()),
388 }
389 }
390
391 /// Creates `name.one` in `folder` holding the page `page` creates, in the next of
392 /// OneNote's section colours, and lists it last in the folder's TOC, as OneNote creates
393 /// a section. Returns the new catalog path.
394 pub fn create_section(&mut self, folder: &str, name: &str, page: &PageCreation) -> Result<String> {
404395 let filename = format!("{name}.one");
405 self.folder(folder)?;
396 let color = next_color(self.folder(folder)?);
406397 let (_, ancestor) = self.toc(folder)?;
407 let bytes = onestore::create_section(&filename, "", author)?;
398 let mut bytes = onestore::create_empty_section(&filename, Some(color))?;
399 let transaction = {
400 let arena = onestore::Arena::default();
401 let mut section = onestore::Section::open(&arena, bytes.clone())?;
402 let edit = Edit {
403 at: crate::now(),
404 ops: vec![Op::Section(SectionOp::Create(page.clone()))],
405 };
406 section.apply(page.author(), &edit)?;
407 section.seal()?
408 };
409 if let Some(transaction) = transaction {
410 transaction.apply(&mut bytes)?;
411 }
408412 let path = catalog_path(folder, &filename);
409413 self.storage.create(&path, &bytes)?;
410414 self.storage.place(&path, ancestor, &filename)?;
......@@ -483,23 +487,16 @@ impl Notebook {
483487 /// Sets a section's colour (COLORREF) in its own metadata, where OneNote keeps it.
484488 pub fn set_section_color(&mut self, path: &str, color: Option<u32>) -> Result<()> {
485489 let path = self.section_path(path)?.path.clone();
486 let source = self.storage.read(&path)?;
487 let store = onestore::Store::parse(&source)?;
488 let index = onestore::RevisionIndex::parse(&store)?;
489 let document = Document::parse(&index)?;
490 let revision = document.active(document.root)?;
491 let metadata = *revision
492 .roots
493 .get(&2)
494 .ok_or_else(|| io::Error::from(io::ErrorKind::InvalidData))?;
495 self.storage.set_property(
496 &path,
497 &source,
498 document.root,
499 metadata,
500 0x14001cbe,
501 &color.unwrap_or(0xffff_ffff).to_le_bytes(),
502 )?;
490 let arena = onestore::Arena::default();
491 let mut section = onestore::Section::open(&arena, self.storage.read(&path)?)?;
492 let edit = Edit {
493 at: crate::now(),
494 ops: vec![Op::Section(SectionOp::Color(color))],
495 };
496 section.apply("", &edit)?;
497 if let Some(transaction) = section.seal()? {
498 self.storage.commit(&path, &transaction)?;
499 }
503500 self.refresh()?;
504501 Ok(())
505502 }
......@@ -518,15 +515,51 @@ impl Notebook {
518515 self.refresh().map(drop)
519516 }
520517
521 /// Deletes a section the way OneNote does: the file moves into the notebook's
522 /// `OneNote_RecycleBin` folder, a section group with its own TOC that lists it (and that
523 /// the root TOC lists), and its own folder's TOC entry goes.
518 /// Deletes a section or section group the way OneNote 2010 does. A section's file moves
519 /// into the notebook's `OneNote_RecycleBin` folder, a section group with its own TOC that
520 /// lists it (and that the root TOC lists). A group's sections, in its groups too, move
521 /// there the same way, then its folders go. Either way its folder's TOC entry goes.
524522 pub fn delete(&mut self, path: &str) -> Result<()> {
525523 let (folder, entry) = split(path);
526524 let (filename, identity) = self.entry(folder, entry)?;
527 if !filename.to_ascii_lowercase().ends_with(".one") {
525 if path == RECYCLE_BIN {
528526 return Err(io::Error::from(io::ErrorKind::InvalidInput).into());
529527 }
528 if filename.to_ascii_lowercase().ends_with(".one") {
529 self.bin_section(path, &filename, identity)?;
530 } else {
531 // Sections first, then each folder once it is empty, deepest first.
532 let mut sections = Vec::new();
533 let mut folders = Vec::new();
534 let mut pending = vec![self.folder(path)?];
535 while let Some(group) = pending.pop() {
536 sections.extend(
537 group
538 .sections
539 .iter()
540 .map(|section| (section.path.clone(), section.file_id)),
541 );
542 folders.push((group.path.clone(), group.toc.as_ref().map(|toc| toc.filename.clone())));
543 pending.extend(&group.groups);
544 }
545 for (section, identity) in sections {
546 let (_, name) = split(&section);
547 self.bin_section(&section, name, identity)?;
548 }
549 for (group, toc) in folders.into_iter().rev() {
550 if let Some(toc) = toc {
551 self.storage.delete(&catalog_path(&group, &toc))?;
552 }
553 self.storage.delete(&group)?;
554 }
555 }
556 self.edit_toc(folder, &[onestore::TocEdit::Remove { identity }])?;
557 self.refresh().map(drop)
558 }
559
560 /// The recycle bin's TOC path and file identity, creating the bin as OneNote does: a
561 /// section group the root TOC lists.
562 fn bin(&self) -> Result<(String, [u8; 16])> {
530563 let bin_toc = catalog_path(RECYCLE_BIN, TOC);
531564 if !self.storage.exists(&bin_toc) {
532565 if !self.storage.exists(RECYCLE_BIN) {
......@@ -545,37 +578,116 @@ impl Notebook {
545578 }],
546579 )?;
547580 }
548 let mut target = filename.clone();
581 let source = self.storage.read(&bin_toc)?;
582 Ok((bin_toc, onestore::Store::parse(&source)?.header.file_id))
583 }
584
585 /// Moves the section file at `path` into the recycle bin, under a name no binned
586 /// section has, and lists it there; its own folder's TOC is the caller's.
587 fn bin_section(&self, path: &str, filename: &str, identity: [u8; 16]) -> Result<()> {
588 let (bin_toc, bin_identity) = self.bin()?;
589 let mut target = filename.to_owned();
549590 let mut attempt = 1;
550591 while self.storage.exists(&catalog_path(RECYCLE_BIN, &target)) {
551592 attempt += 1;
552 let (stem, extension) = filename.rsplit_once('.').unwrap_or((&filename, ""));
593 let (stem, extension) = filename.rsplit_once('.').unwrap_or((filename, ""));
553594 target = format!("{stem} ({attempt}).{extension}");
554595 }
555596 let binned = catalog_path(RECYCLE_BIN, &target);
556 self.storage
557 .rename(&catalog_path(folder, &filename), &binned)?;
558 let source = self.storage.read(&bin_toc)?;
559 self.storage.place(
560 &binned,
561 onestore::Store::parse(&source)?.header.file_id,
562 &target,
563 )?;
564 self.storage.commit(
597 self.storage.rename(path, &binned)?;
598 self.storage.place(&binned, bin_identity, &target)?;
599 self.commit_toc(
565600 &bin_toc,
566 &PreparedEdit::table_of_contents(
567 &source,
601 &[onestore::TocEdit::Add {
602 filename: target,
603 identity,
604 group: false,
605 }],
606 )
607 }
608
609 /// Keeps copies of `pages` in the notebook's recycle bin, as OneNote 2010 does with a
610 /// page it deletes: in `OneNote_RecycleBin/OneNote_DeletedPages.one`, each at the top
611 /// level with its identity, title, date and creation time. The caller then deletes the
612 /// pages from their section.
613 pub fn recycle_pages(&mut self, pages: &[Page], author: &str) -> Result<()> {
614 const DELETED: &str = "OneNote_DeletedPages.one";
615 let (bin_toc, bin_identity) = self.bin()?;
616 let path = catalog_path(RECYCLE_BIN, DELETED);
617 if !self.storage.exists(&path) {
618 // OneNote's "Deleted Pages" section is grey.
619 let bytes = onestore::create_empty_section(DELETED, Some(0x00e1e1e1))?;
620 self.storage.create(&path, &bytes)?;
621 self.storage.place(&path, bin_identity, DELETED)?;
622 self.commit_toc(
623 &bin_toc,
568624 &[onestore::TocEdit::Add {
569 filename: target,
570 identity,
625 filename: DELETED.into(),
626 identity: onestore::Store::parse(&bytes)?.header.file_id,
571627 group: false,
572628 }],
573 )?,
574 )?;
575 self.edit_toc(folder, &[onestore::TocEdit::Remove { identity }])?;
629 )?;
630 }
631 let arena = onestore::Arena::default();
632 let mut section = onestore::Section::open(&arena, self.storage.read(&path)?)?;
633 let mut ops = Vec::new();
634 for page in pages {
635 let mut creation = PageCreation::new(None, Some(&page.title), author)?;
636 if let (Some(identity), Some(created)) = (page.identity, page.created) {
637 creation = creation.keeping(identity, created)?;
638 }
639 if let Some([date, time]) = page.date_text() {
640 creation = creation.dated(&date, &time)?;
641 }
642 ops.push(Op::Section(SectionOp::Import {
643 creation,
644 page: page.copy()?,
645 }));
646 }
647 section.apply(author, &Edit { at: crate::now(), ops })?;
648 if let Some(transaction) = section.seal()? {
649 self.storage.commit(&path, &transaction)?;
650 }
576651 self.refresh().map(drop)
577652 }
578653
654 /// Moves a section or section group into the folder at catalog path `folder`, last, as
655 /// OneNote moves one dragged onto a group: the file or folder moves and each TOC's
656 /// entry follows it. Returns its new catalog path.
657 pub fn move_entry(&mut self, path: &str, folder: &str) -> Result<String> {
658 let (from, entry) = split(path);
659 let (filename, identity) = self.entry(from, entry)?;
660 self.folder(folder)?;
661 let group = !filename.to_ascii_lowercase().ends_with(".one");
662 let target = catalog_path(folder, &filename);
663 if from == folder
664 || path == RECYCLE_BIN
665 || group && (folder == path || folder.starts_with(&format!("{path}/")))
666 || self.storage.exists(&target)
667 {
668 return Err(io::Error::from(io::ErrorKind::InvalidInput).into());
669 }
670 let (_, ancestor) = self.toc(folder)?;
671 self.storage.rename(path, &target)?;
672 let placed = if group {
673 catalog_path(&target, TOC)
674 } else {
675 target.clone()
676 };
677 self.storage.place(&placed, ancestor, &filename)?;
678 self.edit_toc(
679 folder,
680 &[onestore::TocEdit::Add {
681 filename,
682 identity,
683 group,
684 }],
685 )?;
686 self.edit_toc(from, &[onestore::TocEdit::Remove { identity }])?;
687 self.refresh()?;
688 Ok(target)
689 }
690
579691 /// The stored filename and TOC identity of a folder's section or group.
580692 fn entry(&self, folder: &str, name: &str) -> Result<(String, [u8; 16])> {
581693 let parent = self.folder(folder)?;
......@@ -660,30 +772,31 @@ impl Notebook {
660772 Ok(Unlocked { pages, snapshot })
661773 }
662774
663 /// Saves an edited page of `unlocked` under the section's key, straight to the file:
664 /// nothing of a protected section is cached or queued. A section written since
665 /// `unlocked` was read fails the save; unlock again for its pages.
775 /// Applies `edit` to pages of `unlocked` and stores it under the section's key, straight
776 /// to the file: nothing of a protected section is cached or queued. A section written
777 /// since `unlocked` was read refuses the edit; unlock again for its pages.
666778 #[cfg(feature = "protected")]
667 pub fn save_unlocked(
779 pub fn apply_unlocked(
668780 &self,
669781 path: &str,
670782 password: &str,
671783 unlocked: &mut Unlocked,
672 space: ExGuid,
673 page: &Page,
674784 author: &str,
785 edit: &Edit,
675786 ) -> Result<()> {
676787 let path = self.section_path(path)?.path.clone();
677 let edit = onestore::PreparedEdit::page_protected(
678 &unlocked.snapshot,
679 password,
680 space,
681 page,
682 author,
683 )?;
684 self.storage.commit(&path, &edit)?;
685 let written = edit.as_bytes().to_vec();
686 unlocked.snapshot = written;
788 let transaction = {
789 let store = Store::parse(&unlocked.snapshot)?;
790 let index = RevisionIndex::parse(&store)?;
791 onestore::protected::UnlockedSection::open(
792 &index,
793 password,
794 onestore::protected::Limits::default(),
795 )?
796 .apply(author, edit)?
797 };
798 self.storage.commit(&path, &transaction)?;
799 transaction.apply(&mut unlocked.snapshot)?;
687800 Ok(())
688801 }
689802
......@@ -706,6 +819,27 @@ impl Notebook {
706819 }
707820}
708821
822/// The section colours OneNote 2010 gives new sections, COLORREF, in the order it gives
823/// them (`evidence/notebook-management/new-notebook`).
824const SECTION_COLORS: [u32; 3] = [0x00e4a88a, 0x0078b0f6, 0x00bba4d5];
825
826/// The colour OneNote gives a new section in `folder`: the first of its colours none of
827/// the folder's sections has, or the next in turn when all are taken.
828fn next_color(folder: &discover::Folder) -> u32 {
829 let taken: Vec<u32> = folder
830 .sections
831 .iter()
832 .filter_map(|section| match section.state {
833 discover::SectionState::Readable { color, .. } => color,
834 _ => None,
835 })
836 .collect();
837 SECTION_COLORS
838 .into_iter()
839 .find(|color| !taken.contains(color))
840 .unwrap_or(SECTION_COLORS[folder.sections.len() % SECTION_COLORS.len()])
841}
842
709843fn component(name: &str) -> bool {
710844 !name.is_empty() && !name.contains(['/', '\\', '\0']) && name != "." && name != ".."
711845}
......@@ -733,8 +867,6 @@ pub enum Event {
733867 /// The section refused an edit `apply` handed it; the pages it names are as they were
734868 /// before it, so an editor showing them should reload them.
735869 Rejected { spaces: Vec<ExGuid>, error: String },
736 /// Old session API, until the application reloads on `Changed`: pages changed remotely.
737 Refreshed,
738870 /// A publication attempt finished with this durable state.
739871 Attempt { id: u64, status: EditStatus },
740872 /// The section file could not be reached; the replica keeps its state.
......@@ -743,25 +875,6 @@ pub enum Event {
743875 Failed(String),
744876}
745877
746/// The outcome of saving an edited page (old session API).
747#[derive(Debug, PartialEq, Eq)]
748pub enum Save {
749 /// The model equals the stored page.
750 Unchanged,
751 /// The edit is durable locally under this id.
752 Queued(u64),
753 /// The stored page no longer matches `before`; reload it before saving again.
754 Stale,
755}
756
757/// A queued edit and its durable state.
758#[derive(Debug, Clone, PartialEq, Eq)]
759pub struct QueuedEdit {
760 pub id: u64,
761 pub space: ExGuid,
762 pub status: EditStatus,
763}
764
765878type Notify = Arc<dyn Fn() + Send + Sync>;
766879
767880/// A section file with its replica and background publication. `Send + Sync`: edits
......@@ -773,10 +886,6 @@ pub struct Section {
773886 events: Mutex<Receiver<Event>>,
774887 sender: Sender<Event>,
775888 notify: Notify,
776 saved: Mutex<Receiver<(ExGuid, std::result::Result<Save, String>)>>,
777 saves: Sender<(ExGuid, std::result::Result<Save, String>)>,
778 /// Saves queued and not yet reported.
779 unsaved: Arc<AtomicUsize>,
780889}
781890
782891impl Section {
......@@ -850,7 +959,6 @@ impl Section {
850959 ) -> Result<Self> {
851960 let replica = Arc::new(replica);
852961 let (sender, events) = mpsc::channel();
853 let (saves, saved) = mpsc::channel();
854962 let notify = Mutex::new(notify);
855963 let notify: Notify = Arc::new(move || {
856964 if let Ok(notify) = notify.lock() {
......@@ -865,7 +973,6 @@ impl Section {
865973 Ok(synced) => {
866974 if !synced.changed.is_empty() {
867975 events.push(Event::Changed(synced.changed.clone()));
868 events.push(Event::Refreshed);
869976 }
870977 if let Some((id, status)) = &synced.edit {
871978 events.push(Event::Attempt {
......@@ -899,9 +1006,6 @@ impl Section {
8991006 events: Mutex::new(events),
9001007 sender,
9011008 notify,
902 saved: Mutex::new(saved),
903 saves,
904 unsaved: Arc::new(AtomicUsize::new(0)),
9051009 })
9061010 }
9071011
......@@ -971,8 +1075,8 @@ impl Section {
9711075 Ok(space)
9721076 }
9731077
974 /// Removes pages permanently, queued like the user's own edits (a move across
975 /// sections is `import_page` there, then this here).
1078 /// Removes pages or conflict pages permanently, queued like the user's own edits (a
1079 /// move across sections is `import_page` there, then this here).
9761080 pub fn delete_pages(&self, pages: &[ExGuid]) -> Result<u64> {
9771081 self.replica.apply(
9781082 "",
......@@ -991,23 +1095,10 @@ impl Section {
9911095 self.replica.pending()
9921096 }
9931097
994 /// The unpublished batch the remote no longer accepts, if any. Show `remote_page`
995 /// against `page` for its space, then `resolve`.
996 pub fn conflict(&self) -> Result<Option<Conflict>> {
997 self.replica.conflict()
998 }
999
1000 /// The page as last observed in the section file, for reviewing a conflict.
1001 pub fn remote_page(&self, space: ExGuid) -> Result<Page> {
1002 let snapshot = self.replica.remote_snapshot()?;
1003 let arena = onestore::Arena::default();
1004 Ok(onestore::Section::open(&arena, snapshot)?.page(space)?)
1005 }
1006
1007 /// Ends a conflict: `Mine` publishes the local page over the remote change, `Theirs`
1008 /// drops the local edits to the conflicted page (and its later ones).
1009 pub fn resolve(&self, id: u64, resolution: Resolution) -> Result<()> {
1010 self.replica.resolve(id, resolution)
1098 /// The conflict pages of each page that has them, as the local edits leave them
1099 /// (`onestore::Section::conflicts`); `page` reads one, `delete_pages` removes it.
1100 pub fn conflicts(&self) -> Result<Vec<(ExGuid, Vec<onestore::ConflictPage>)>> {
1101 self.replica.conflicts()
10111102 }
10121103
10131104 /// Retires an uncertain attempt after review, exporting the queue to `archive` first:
......@@ -1056,114 +1147,6 @@ impl Section {
10561147 None => Ok(()),
10571148 }
10581149 }
1059
1060 // Old whole-page session API, kept until the application emits ops.
1061
1062 /// Saves an edited page. `before` is the model the edit started from; a stored page
1063 /// that differs from it means the section changed underneath the editor.
1064 pub fn save(&self, space: ExGuid, before: &Page, after: &Page, author: &str) -> Result<Save> {
1065 let outcome = self.replica.ask(|reply| crate::working::Request::Save {
1066 space,
1067 before: before.clone(),
1068 after: after.clone(),
1069 author: author.to_owned(),
1070 reply,
1071 })?;
1072 Ok(outcome.unwrap_or(Save::Unchanged))
1073 }
1074
1075 /// `save` on the section thread, returning at once; `saved` reports the outcome.
1076 /// Saves queue in order, and one whose `before` is the previous save's `after`
1077 /// continues it: pass each save's `after` as the next one's `before`.
1078 pub fn queue_save(&self, space: ExGuid, before: Page, after: Page, author: &str) -> Result<()> {
1079 let (saves, notify, unsaved) = (
1080 self.saves.clone(),
1081 Arc::clone(&self.notify),
1082 Arc::clone(&self.unsaved),
1083 );
1084 self.unsaved.fetch_add(1, Ordering::Relaxed);
1085 self.replica.send(crate::working::Request::Save {
1086 space,
1087 before,
1088 after,
1089 author: author.to_owned(),
1090 reply: Box::new(move |outcome| {
1091 unsaved.fetch_sub(1, Ordering::Release);
1092 let outcome = match outcome {
1093 Ok(None) => return,
1094 Ok(Some(save)) => Ok(save),
1095 Err(error) => Err(error.to_string()),
1096 };
1097 if saves.send((space, outcome)).is_ok() {
1098 notify();
1099 }
1100 }),
1101 })
1102 }
1103
1104 /// Whether a queued save has yet to report, so the stored page may trail the editor's.
1105 pub fn saving(&self) -> bool {
1106 self.unsaved.load(Ordering::Acquire) > 0
1107 }
1108
1109 /// Outcomes of queued saves since the last poll, oldest first; `notify` runs for each.
1110 /// Consecutive saves continuing one another finish as one, reported once.
1111 pub fn saved(&self) -> Vec<(ExGuid, std::result::Result<Save, String>)> {
1112 self.saved
1113 .lock()
1114 .map(|saved| saved.try_iter().collect())
1115 .unwrap_or_default()
1116 }
1117
1118 /// Every queued edit with its state: pending, awaiting confirmation of a retained
1119 /// attempt, or a conflict awaiting review. `space` is the first page it edits.
1120 pub fn queue(&self) -> Result<Vec<QueuedEdit>> {
1121 self.replica
1122 .pending()?
1123 .into_iter()
1124 .map(|edit| {
1125 Ok(QueuedEdit {
1126 id: edit.id,
1127 space: edit
1128 .edit
1129 .ops
1130 .iter()
1131 .find_map(|op| match op {
1132 Op::Page { space, .. } => Some(*space),
1133 Op::Section(_) => None,
1134 })
1135 .unwrap_or(self.replica.root),
1136 status: self.replica.status(edit.id)?.unwrap_or(EditStatus::Pending),
1137 })
1138 })
1139 .collect()
1140 }
1141
1142 /// The conflict as a queue entry, for the review dialog.
1143 pub fn conflicts(&self) -> Result<Vec<(QueuedEdit, ConflictKind)>> {
1144 Ok(self
1145 .replica
1146 .conflict()?
1147 .map(|conflict| {
1148 (
1149 QueuedEdit {
1150 id: conflict.id,
1151 space: conflict.space,
1152 status: EditStatus::Conflict(conflict.kind),
1153 },
1154 conflict.kind,
1155 )
1156 })
1157 .into_iter()
1158 .collect())
1159 }
1160
1161 /// Resolves the conflict holding `id` with a page reviewed against `remote_page`: the
1162 /// remote page is rewritten to `after`.
1163 pub fn review(&self, id: u64, after: &Page) -> Result<()> {
1164 self.replica
1165 .resolve_with(id, Resolution::Mine, Some(after.clone()))
1166 }
11671150}
11681151
11691152/// The section file itself as the publication target, under OneNote-compatible exclusion.
......@@ -1176,21 +1159,21 @@ impl Remote for FileRemote {
11761159
11771160 /// Read without the whole-file lock, which would block OneNote's readers: a change
11781161 /// detector, never a snapshot to edit.
1179 fn stamp(&mut self) -> io::Result<Option<Stamp>> {
1162 fn stamp(&mut self) -> io::Result<Stamp> {
11801163 use std::io::Read;
11811164 let mut file = std::fs::File::open(&self.0)?;
11821165 let mut header = [0; 1024];
11831166 file.read_exact(&mut header)?;
11841167 let length = file.metadata()?.len();
1185 Ok(Some(Stamp { header, length }))
1168 Ok(Stamp { header, length })
11861169 }
11871170
11881171 fn publish(&mut self, transaction: &Transaction) -> std::result::Result<(), CommitError> {
11891172 transaction.commit_file(&self.0)
11901173 }
11911174
1192 fn confirm(&mut self, snapshot: &[u8]) -> std::result::Result<(), CommitError> {
1193 onestore::confirm_file_snapshot(&self.0, snapshot)
1175 fn confirm(&mut self, base: &Stamp) -> std::result::Result<(), CommitError> {
1176 onestore::confirm_file(&self.0, base)
11941177 }
11951178}
11961179
crates/notebook/src/smb/directory.rs+1-5
......@@ -35,11 +35,7 @@ impl Client {
3535 create_contexts: Vec::new(),
3636 },
3737 )?;
38 let file = File {
39 client: self,
40 id: Some(response.file_id),
41 length: response.end_of_file,
42 };
38 let file = File::new(self, &response);
4339 let mut entries = BTreeMap::new();
4440 loop {
4541 let response: io::Result<QueryDirectoryResponse> =
crates/notebook/src/smb/mod.rs+335-150
......@@ -1,9 +1,9 @@
11//! Blocking SMB access to OneNote sections, table-of-contents files and payloads.
22
3use onestore::{CommitError, CommitIo, CommitState, ExGuid};
3use onestore::{CommitError, CommitIo, CommitState};
44use smb2::{
55 Session, Tree,
6 client::connection::{Connection, NegotiatedParams},
6 client::connection::{CompoundOp, Connection, Frame, NegotiatedParams},
77 msg::{
88 close::{CloseRequest, CloseResponse},
99 create::{
......@@ -22,7 +22,7 @@ use smb2::{
2222 flags::{Capabilities, FileAccessMask},
2323 },
2424};
25use std::{io, ops::Range, sync::Mutex, time::Duration};
25use std::{io, sync::Mutex, time::Duration};
2626use tokio::runtime::{Handle, Runtime};
2727
2828mod directory;
......@@ -149,6 +149,54 @@ impl Client {
149149 .map_err(|_| io::Error::from(io::ErrorKind::TimedOut))
150150 .and_then(|result| result.map_err(io::Error::other))
151151 .inspect_err(|_| self.retire())?;
152 self.unpack(&self.body(command, frame)?)
153 }
154
155 /// Sends `requests` as one related compound request, which the server performs in order,
156 /// and returns each response body or status error. A request after a CREATE names the
157 /// file it opened as `FileId::SENTINEL`; each request fits one credit.
158 fn compound(&self, requests: &[(Command, &dyn Pack)]) -> io::Result<Vec<io::Result<Vec<u8>>>> {
159 if Handle::try_current().is_ok() {
160 return Err(io::ErrorKind::InvalidInput.into());
161 }
162 let connection = self
163 .connection
164 .lock()
165 .map_err(|_| io::ErrorKind::Other)?
166 .clone()
167 .ok_or(io::ErrorKind::NotConnected)?;
168 let operations: Vec<_> = requests
169 .iter()
170 .map(|(command, body)| CompoundOp::new(*command, *body, Some(self.tree.tree_id)))
171 .collect();
172 let frames = {
173 let runtime = self.runtime.lock().map_err(|_| io::ErrorKind::Other)?;
174 runtime
175 .as_ref()
176 .ok_or(io::ErrorKind::NotConnected)?
177 .block_on(async {
178 tokio::time::timeout(self.timeout, connection.execute_compound(&operations))
179 .await
180 })
181 };
182 let frames = frames
183 .map_err(|_| io::Error::from(io::ErrorKind::TimedOut))
184 .and_then(|result| result.map_err(io::Error::other))
185 .inspect_err(|_| self.retire())?;
186 Ok(frames
187 .into_iter()
188 .zip(requests)
189 .map(|(frame, (command, _))| {
190 let frame = frame
191 .map_err(io::Error::other)
192 .inspect_err(|_| self.retire())?;
193 self.body(*command, frame)
194 })
195 .collect())
196 }
197
198 /// The body of a response to `command`, or its status as an error.
199 fn body(&self, command: Command, frame: Frame) -> io::Result<Vec<u8>> {
152200 if frame.header.command != command {
153201 self.retire();
154202 return Err(io::ErrorKind::InvalidData.into());
......@@ -170,12 +218,17 @@ impl Client {
170218 },
171219 ));
172220 }
173 T::unpack(&mut ReadCursor::new(&frame.body)).map_err(|error| {
221 Ok(frame.body)
222 }
223
224 fn unpack<T: Unpack>(&self, body: &[u8]) -> io::Result<T> {
225 T::unpack(&mut ReadCursor::new(body)).map_err(|error| {
174226 self.retire();
175227 io::Error::new(io::ErrorKind::InvalidData, error)
176228 })
177229 }
178230
231 /// Opens as OneNote 2010 does: readers share everything, a writer denies other writers.
179232 fn open(&self, path: &str, write: bool) -> io::Result<File<'_>> {
180233 self.open_shared(path, write, if write { 5 } else { 7 })
181234 }
......@@ -186,7 +239,7 @@ impl Client {
186239 if write { 0xc0000000 } else { 0x80000000 },
187240 sharing,
188241 CreateDisposition::FileOpen,
189 0x42,
242 0x40,
190243 )
191244 }
192245
......@@ -198,28 +251,11 @@ impl Client {
198251 disposition: CreateDisposition,
199252 options: u32,
200253 ) -> io::Result<File<'_>> {
201 if path.is_empty() || path.contains('\0') || path.encode_utf16().count() > 32767 {
202 return Err(io::ErrorKind::InvalidInput.into());
203 }
204254 let response: CreateResponse = self.request(
205255 Command::Create,
206 CreateRequest {
207 requested_oplock_level: OplockLevel::None,
208 impersonation_level: ImpersonationLevel::Impersonation,
209 desired_access: FileAccessMask::new(access),
210 file_attributes: 0,
211 share_access: ShareAccess(sharing),
212 create_disposition: disposition,
213 create_options: options,
214 name: smb2::encode_path(&path.replace('\\', "/")),
215 create_contexts: Vec::new(),
216 },
256 create_request(path, access, sharing, disposition, options)?,
217257 )?;
218 Ok(File {
219 client: self,
220 id: Some(response.file_id),
221 length: response.end_of_file,
222 })
258 Ok(File::new(self, &response))
223259 }
224260
225261 /// Creates a file holding `bytes`; an existing file is an error.
......@@ -286,7 +322,7 @@ impl Client {
286322 /// Names a section or TOC file for its notebook (`onestore::place`) under native
287323 /// writer coordination.
288324 pub fn place(&self, path: &str, ancestor: [u8; 16], name: &str) -> Result<(), CommitError> {
289 self.commit(path, |file| {
325 self.commit(path, &[(0, 1024)], |file| {
290326 onestore::place(file, ancestor, name).map_err(|error| CommitError {
291327 state: CommitState::NotCommitted,
292328 error,
......@@ -338,7 +374,9 @@ impl Client {
338374 path: &str,
339375 snapshot: impl FnOnce(&mut File<'_>) -> io::Result<Option<Vec<u8>>>,
340376 ) -> io::Result<Vec<u8>> {
341 let mut file = self.open(path, false)?.coordinate(path, false)?;
377 let mut file = self
378 .open(path, false)?
379 .coordinate(path, false, &[(0, 1024)])?;
342380 let result = snapshot(&mut file)
343381 .and_then(|snapshot| snapshot.ok_or_else(|| io::ErrorKind::WouldBlock.into()));
344382 let closed = file.close();
......@@ -347,79 +385,67 @@ impl Client {
347385 Ok(snapshot)
348386 }
349387
350 pub fn commit_text(
351 &self,
352 path: &str,
353 source: &[u8],
354 space: ExGuid,
355 object: ExGuid,
356 range: Range<u32>,
357 replacement: &str,
358 ) -> Result<(), CommitError> {
359 self.commit(path, |file| {
360 onestore::commit_text(file, source, space, object, range, replacement)
361 })
362 }
363
364 pub fn commit_property_bytes(
365 &self,
366 path: &str,
367 source: &[u8],
368 space: ExGuid,
369 object: ExGuid,
370 property: u32,
371 value: &[u8],
372 ) -> Result<(), CommitError> {
373 self.commit(path, |file| {
374 onestore::commit_property_bytes(file, source, space, object, property, value)
375 })
376 }
377
378388 /// Publishes a transaction using the same native writer coordination as text commits.
379389 pub fn commit_transaction(
380390 &self,
381391 path: &str,
382392 transaction: &onestore::Transaction,
383393 ) -> Result<(), CommitError> {
384 self.commit(path, |file| transaction.commit(file))
394 self.commit(path, &checked(transaction.base()), |file| {
395 transaction.commit(file)
396 })
385397 }
386398
387 /// The header and length of a revision store, read without writer coordination or path
388 /// identity checks: a change detector for polling, never a snapshot to edit.
399 /// The header and length of a revision store, in one round trip, without writer
400 /// coordination or path identity checks: a change detector for polling, never a snapshot
401 /// to edit.
389402 pub fn stamp(&self, path: &str) -> io::Result<onestore::Stamp> {
390 let mut file = self.open(path, false)?;
391 let mut header = [0; 1024];
392 let mut read = 0;
393 let result = loop {
394 match file.read_at(read as u64, &mut header[read..]) {
395 Ok(0) => break Err(io::ErrorKind::UnexpectedEof.into()),
396 Ok(count) => read += count,
397 Err(error) => break Err(error),
398 }
399 if read == header.len() {
400 break Ok(());
401 }
403 let create = create_request(path, 0x80000000, 7, CreateDisposition::FileOpen, 0x40)?;
404 let read = read_request(FileId::SENTINEL, 0, 1024);
405 let close = CloseRequest {
406 file_id: FileId::SENTINEL,
407 flags: 0,
402408 };
403 let length = file.length;
404 let closed = file.close();
405 result?;
409 let [created, read, closed]: [_; 3] = self
410 .compound(&[
411 (Command::Create, &create),
412 (Command::Read, &read),
413 (Command::Close, &close),
414 ])?
415 .try_into()
416 .map_err(|_| io::ErrorKind::InvalidData)?;
417 let created: CreateResponse = self.unpack(&created?)?;
418 if closed.is_err() {
419 self.retire();
420 }
421 let read: ReadResponse = self.unpack(&read?)?;
406422 closed?;
407 Ok(onestore::Stamp { header, length })
423 let header = read
424 .data
425 .try_into()
426 .map_err(|_| io::ErrorKind::UnexpectedEof)?;
427 Ok(onestore::Stamp {
428 header,
429 length: created.end_of_file,
430 })
408431 }
409432
410 /// Confirms an observed snapshot's durability under native writer coordination.
411 pub fn confirm_snapshot(&self, path: &str, source: &[u8]) -> Result<(), CommitError> {
412 self.commit(path, |file| onestore::confirm_snapshot(file, source))
433 /// Confirms that the file still has `base`'s stamp and is durable, then refreshes its
434 /// version, under native writer coordination.
435 pub fn confirm(&self, path: &str, base: &onestore::Stamp) -> Result<(), CommitError> {
436 self.commit(path, &checked(base), |file| onestore::confirm(file, base))
413437 }
414438
439 /// Opens `path` for writing under OneNote's coordination, making `reads` with the locks.
415440 fn commit(
416441 &self,
417442 path: &str,
443 reads: &[(u64, usize)],
418444 operation: impl FnOnce(&mut File<'_>) -> Result<(), CommitError>,
419445 ) -> Result<(), CommitError> {
420446 let mut file = self
421447 .open(path, true)
422 .and_then(|file| file.coordinate(path, true))
448 .and_then(|file| file.coordinate(path, true, reads))
423449 .map_err(|error| CommitError {
424450 state: CommitState::NotCommitted,
425451 error,
......@@ -443,24 +469,103 @@ impl Drop for Client {
443469struct File<'a> {
444470 client: &'a Client,
445471 id: Option<FileId>,
446 /// The end of file when opened.
447 length: u64,
472 /// Writes `flush` sends ahead of its flush, in order.
473 writes: Vec<(u64, Vec<u8>)>,
474 /// Reads made under the coordination locks, each returned once to the same `read_at`.
475 prefetched: Vec<(u64, usize, Vec<u8>)>,
448476}
449impl File<'_> {
450 fn coordinate(self, path: &str, write: bool) -> io::Result<Self> {
451 self.lock(0xfffffffb, 0x11)?;
477impl<'a> File<'a> {
478 fn new(client: &'a Client, created: &CreateResponse) -> Self {
479 Self {
480 client,
481 id: Some(created.file_id),
482 writes: Vec::new(),
483 prefetched: Vec::new(),
484 }
485 }
486
487 /// Takes OneNote 2010's coordination locks in one request (the shared reader byte, and
488 /// to write the exclusive writer byte), makes `reads` under them, then checks that the
489 /// handle is still the file at `path`: maintenance replaces a file while holding its
490 /// locks.
491 fn coordinate(mut self, path: &str, write: bool, reads: &[(u64, usize)]) -> io::Result<Self> {
492 let id = self.id.unwrap();
493 let mut locks = vec![LockElement {
494 offset: 0xfffffffb,
495 length: 1,
496 flags: 0x11,
497 }];
452498 if write {
453 self.lock(0xfffffffd, 0x12)?;
499 locks.push(LockElement {
500 offset: 0xfffffffd,
501 length: 1,
502 flags: 0x12,
503 });
504 }
505 let lock = LockRequest {
506 file_id: id,
507 lock_sequence: 0,
508 locks,
509 };
510 let [index, volume] = identity_requests(id);
511 let reading: Vec<_> = reads
512 .iter()
513 .map(|(offset, length)| read_request(id, *offset, *length))
514 .collect();
515 let mut requests: Vec<(Command, &dyn Pack)> = vec![
516 (Command::Lock, &lock),
517 (Command::QueryInfo, &index),
518 (Command::QueryInfo, &volume),
519 ];
520 requests.extend(
521 reading
522 .iter()
523 .map(|read| (Command::Read, read as &dyn Pack)),
524 );
525 let mut responses = self.client.compound(&requests)?.into_iter();
526 let mut next = || responses.next().ok_or(io::ErrorKind::InvalidData);
527 let _: LockResponse = self.client.unpack(&next()??)?;
528 let own = self.client.identity(next()?, next()?)?;
529 for (offset, length) in reads {
530 let data = match next()? {
531 Err(error) if error.kind() == io::ErrorKind::UnexpectedEof => Vec::new(),
532 body => self.client.unpack::<ReadResponse>(&body?)?.data,
533 };
534 if data.len() > *length {
535 self.client.retire();
536 return Err(io::ErrorKind::InvalidData.into());
537 }
538 self.prefetched.push((*offset, *length, data));
539 }
540 let create = create_request(path, 0x80, 7, CreateDisposition::FileOpen, 0)?;
541 let [index, volume] = identity_requests(FileId::SENTINEL);
542 let close = CloseRequest {
543 file_id: FileId::SENTINEL,
544 flags: 0,
545 };
546 let [created, index, volume, closed]: [_; 4] = self
547 .client
548 .compound(&[
549 (Command::Create, &create),
550 (Command::QueryInfo, &index),
551 (Command::QueryInfo, &volume),
552 (Command::Close, &close),
553 ])?
554 .try_into()
555 .map_err(|_| io::ErrorKind::InvalidData)?;
556 created?;
557 if closed.is_err() {
558 self.client.retire();
454559 }
455 let current = self.client.open(path, false)?;
456 let same = self.identity()? == current.identity()?;
457 current.close()?;
458 if !same {
560 let current = self.client.identity(index, volume)?;
561 closed?;
562 if own != current {
459563 return Err(io::ErrorKind::ResourceBusy.into());
460564 }
461565 Ok(self)
462566 }
463567
568 #[cfg(test)]
464569 fn lock(&self, offset: u64, flags: u32) -> io::Result<()> {
465570 let _: LockResponse = self.client.request(
466571 Command::Lock,
......@@ -477,47 +582,6 @@ impl File<'_> {
477582 Ok(())
478583 }
479584
480 fn identity(&self) -> io::Result<(u64, u32)> {
481 let index: QueryInfoResponse = self.client.request(
482 Command::QueryInfo,
483 QueryInfoRequest {
484 info_type: InfoType::File,
485 file_info_class: 6,
486 output_buffer_length: 8,
487 additional_information: 0,
488 flags: 0,
489 file_id: self.id.unwrap(),
490 input_buffer: Vec::new(),
491 },
492 )?;
493 let index = u64::from_le_bytes(
494 index
495 .output_buffer
496 .try_into()
497 .map_err(|_| io::ErrorKind::InvalidData)?,
498 );
499 if index == 0 {
500 return Err(io::ErrorKind::Unsupported.into());
501 }
502 let volume: QueryInfoResponse = self.client.request(
503 Command::QueryInfo,
504 QueryInfoRequest {
505 info_type: InfoType::Filesystem,
506 file_info_class: 1,
507 output_buffer_length: 1024,
508 additional_information: 0,
509 flags: 0,
510 file_id: self.id.unwrap(),
511 input_buffer: Vec::new(),
512 },
513 )?;
514 let serial = volume
515 .output_buffer
516 .get(8..12)
517 .ok_or(io::ErrorKind::InvalidData)?;
518 Ok((index, u32::from_le_bytes(serial.try_into().unwrap())))
519 }
520
521585 fn close(mut self) -> io::Result<()> {
522586 self.release()
523587 }
......@@ -540,6 +604,87 @@ impl Drop for File<'_> {
540604 let _ = self.release();
541605 }
542606}
607
608impl Client {
609 /// A file's volume serial number and index, from the responses to `identity_requests`.
610 fn identity(
611 &self,
612 index: io::Result<Vec<u8>>,
613 volume: io::Result<Vec<u8>>,
614 ) -> io::Result<(u64, u32)> {
615 let index: QueryInfoResponse = self.unpack(&index?)?;
616 let index = u64::from_le_bytes(
617 index
618 .output_buffer
619 .try_into()
620 .map_err(|_| io::ErrorKind::InvalidData)?,
621 );
622 if index == 0 {
623 return Err(io::ErrorKind::Unsupported.into());
624 }
625 let volume: QueryInfoResponse = self.unpack(&volume?)?;
626 let serial = volume
627 .output_buffer
628 .get(8..12)
629 .ok_or(io::ErrorKind::InvalidData)?;
630 Ok((index, u32::from_le_bytes(serial.try_into().unwrap())))
631 }
632}
633
634/// FileInternalInformation and FileFsVolumeInformation queries.
635fn identity_requests(file_id: FileId) -> [QueryInfoRequest; 2] {
636 let query = |info_type, file_info_class, output_buffer_length| QueryInfoRequest {
637 info_type,
638 file_info_class,
639 output_buffer_length,
640 additional_information: 0,
641 flags: 0,
642 file_id,
643 input_buffer: Vec::new(),
644 };
645 [
646 query(InfoType::File, 6, 8),
647 query(InfoType::Filesystem, 1, 1024),
648 ]
649}
650
651fn create_request(
652 path: &str,
653 access: u32,
654 sharing: u32,
655 disposition: CreateDisposition,
656 options: u32,
657) -> io::Result<CreateRequest> {
658 if path.is_empty() || path.contains('\0') || path.encode_utf16().count() > 32767 {
659 return Err(io::ErrorKind::InvalidInput.into());
660 }
661 Ok(CreateRequest {
662 requested_oplock_level: OplockLevel::None,
663 impersonation_level: ImpersonationLevel::Impersonation,
664 desired_access: FileAccessMask::new(access),
665 file_attributes: 0,
666 share_access: ShareAccess(sharing),
667 create_disposition: disposition,
668 create_options: options,
669 name: smb2::encode_path(&path.replace('\\', "/")),
670 create_contexts: Vec::new(),
671 })
672}
673
674fn read_request(file_id: FileId, offset: u64, length: usize) -> ReadRequest {
675 ReadRequest {
676 file_id,
677 offset,
678 length: length as u32,
679 minimum_count: 1,
680 flags: 0,
681 padding: 0,
682 channel: 0,
683 remaining_bytes: 0,
684 read_channel_info: Vec::new(),
685 }
686}
687
543688fn read_size(params: &NegotiatedParams, credits: u16, requested: usize) -> usize {
544689 let budget = if params.dialect != Dialect::Smb2_0_2
545690 && params.capabilities.contains(Capabilities::LARGE_MTU)
......@@ -559,6 +704,15 @@ impl CommitIo for File<'_> {
559704 if output.is_empty() {
560705 return Ok(0);
561706 }
707 if let Some(at) = self
708 .prefetched
709 .iter()
710 .position(|(start, length, _)| *start == offset && *length == output.len())
711 {
712 let (_, _, data) = self.prefetched.swap_remove(at);
713 output[..data.len()].copy_from_slice(&data);
714 return Ok(data.len());
715 }
562716 let mut size = 0;
563717 let response: io::Result<ReadResponse> =
564718 self.client.request_with(Command::Read, |connection| {
......@@ -595,40 +749,71 @@ impl CommitIo for File<'_> {
595749 output[..response.data.len()].copy_from_slice(&response.data);
596750 Ok(response.data.len())
597751 }
752 /// Queues at most one request's worth of `bytes`; `flush` sends it.
598753 fn write_at(&mut self, offset: u64, bytes: &[u8]) -> io::Result<usize> {
599 if bytes.is_empty() {
600 return Ok(0);
601 }
602754 let size = bytes.len().min(65536);
603 let response: WriteResponse = self.client.request(
604 Command::Write,
605 WriteRequest {
606 file_id: self.id.unwrap(),
755 if size > 0 {
756 self.writes.push((offset, bytes[..size].to_vec()));
757 }
758 Ok(size)
759 }
760
761 /// Sends the queued writes and then the flush in related compound requests of at most
762 /// sixteen, awaiting each: the server performs a compound in order, and answers the flush
763 /// once every write before it is durable.
764 fn flush(&mut self) -> io::Result<()> {
765 let file_id = self.id.unwrap();
766 let writes: Vec<_> = self
767 .writes
768 .drain(..)
769 .map(|(offset, data)| WriteRequest {
770 file_id,
607771 offset,
608 data: bytes[..size].to_vec(),
772 data,
609773 data_offset: 112,
610 flags: 1,
774 flags: 0,
611775 channel: 0,
612776 remaining_bytes: 0,
613777 write_channel_info_offset: 0,
614778 write_channel_info_length: 0,
615 },
616 )?;
617 if response.count as usize > size {
618 return Err(io::ErrorKind::InvalidData.into());
779 })
780 .collect();
781 let flush = FlushRequest { file_id };
782 let mut requests: Vec<(Command, &dyn Pack)> = writes
783 .iter()
784 .map(|write| (Command::Write, write as &dyn Pack))
785 .collect();
786 requests.push((Command::Flush, &flush));
787 let mut sizes = writes.iter().map(|write| write.data.len());
788 for compound in requests.chunks(16) {
789 for (response, (command, _)) in
790 self.client.compound(compound)?.into_iter().zip(compound)
791 {
792 let body = response?;
793 if *command == Command::Flush {
794 let _: FlushResponse = self.client.unpack(&body)?;
795 continue;
796 }
797 let written: WriteResponse = self.client.unpack(&body)?;
798 let size = sizes.next().ok_or(io::ErrorKind::InvalidData)?;
799 if written.count as usize != size {
800 return Err(if (written.count as usize) < size {
801 io::ErrorKind::WriteZero
802 } else {
803 io::ErrorKind::InvalidData
804 }
805 .into());
806 }
807 }
619808 }
620 Ok(response.count as usize)
621 }
622 fn flush(&mut self) -> io::Result<()> {
623 let _: FlushResponse = self.client.request(
624 Command::Flush,
625 FlushRequest {
626 file_id: self.id.unwrap(),
627 },
628 )?;
629809 Ok(())
630810 }
631811}
632812
813/// The reads `Stamp::check` makes: the header and a probe of the last byte.
814fn checked(stamp: &onestore::Stamp) -> [(u64, usize); 2] {
815 [(0, 1024), (stamp.length.saturating_sub(1), 2)]
816}
817
633818#[cfg(test)]
634819mod tests;
crates/notebook/src/smb/remote.rs+4-4
......@@ -26,15 +26,15 @@ impl Remote for SmbRemote {
2626 self.client.read(&self.path, self.limit)
2727 }
2828
29 fn stamp(&mut self) -> io::Result<Option<Stamp>> {
30 self.client.stamp(&self.path).map(Some)
29 fn stamp(&mut self) -> io::Result<Stamp> {
30 self.client.stamp(&self.path)
3131 }
3232
3333 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
3434 self.client.commit_transaction(&self.path, transaction)
3535 }
3636
37 fn confirm(&mut self, snapshot: &[u8]) -> Result<(), CommitError> {
38 self.client.confirm_snapshot(&self.path, snapshot)
37 fn confirm(&mut self, base: &Stamp) -> Result<(), CommitError> {
38 self.client.confirm(&self.path, base)
3939 }
4040}
crates/notebook/src/smb/tests.rs+67-38
......@@ -1,7 +1,8 @@
11use super::*;
22use onestore::{
3 RevisionIndex, Store,
3 ExGuid, RevisionIndex, Store, Transaction,
44 document::{Document, Kind},
5 op::{Edit, Op, PageOp},
56};
67use std::{
78 fs,
......@@ -117,11 +118,7 @@ fn create(client: &Client, path: &str, bytes: &[u8]) {
117118 },
118119 )
119120 .unwrap();
120 let mut file = File {
121 client,
122 id: Some(response.file_id),
123 length: response.end_of_file,
124 };
121 let mut file = File::new(client, &response);
125122 let mut offset = 0;
126123 while offset < bytes.len() {
127124 offset += file.write_at(offset as u64, &bytes[offset..]).unwrap();
......@@ -129,6 +126,43 @@ fn create(client: &Client, path: &str, bytes: &[u8]) {
129126 file.flush().unwrap();
130127 file.close().unwrap();
131128}
129/// The transaction replacing `range` of the first text in `image` with `with`, as typed now.
130fn replaced(image: &[u8], range: std::ops::Range<u32>, with: &str) -> Transaction {
131 let (space, text, _) = text(image);
132 let arena = onestore::Arena::default();
133 let mut section = onestore::Section::open(&arena, image.to_vec()).unwrap();
134 let at = SystemTime::now()
135 .duration_since(UNIX_EPOCH)
136 .unwrap()
137 .as_nanos() as u64
138 / 100
139 + 116_444_736_000_000_000;
140 section
141 .apply(
142 "Fixture",
143 &Edit {
144 at,
145 ops: vec![Op::Page {
146 space,
147 op: PageOp::Text {
148 text,
149 range,
150 with: with.to_owned(),
151 },
152 }],
153 },
154 )
155 .unwrap();
156 section.seal().unwrap().unwrap()
157}
158
159/// `image` after `transaction`.
160fn applied(image: &[u8], transaction: &Transaction) -> Vec<u8> {
161 let mut image = image.to_vec();
162 transaction.apply(&mut image).unwrap();
163 image
164}
165
132166fn text(bytes: &[u8]) -> (ExGuid, ExGuid, String) {
133167 let store = Store::parse(bytes).unwrap();
134168 assert!(store.checksum_mismatches.is_empty());
......@@ -171,21 +205,15 @@ fn live_coordination() {
171205 client
172206 .open(&path, false)
173207 .unwrap()
174 .coordinate(&path, false)
208 .coordinate(&path, false, &[])
175209 .unwrap()
176210 })
177211 .collect();
178 let (sid, oid, before) = text(&source);
212 let (_, _, before) = text(&source);
179213 let replacement = "After café 🦀";
214 let everything = 0..before.encode_utf16().count() as u32;
180215 writer
181 .commit_text(
182 &path,
183 &source,
184 sid,
185 oid,
186 0..before.encode_utf16().count() as u32,
187 replacement,
188 )
216 .commit_transaction(&path, &replaced(&source, everything.clone(), replacement))
189217 .unwrap();
190218 let after = writer.read(&path, 1 << 20).unwrap();
191219 assert_eq!(text(&after).2, replacement);
......@@ -196,7 +224,7 @@ fn live_coordination() {
196224 assert_eq!(snapshot, after);
197225 }
198226 let error = writer
199 .commit_text(&path, &source, sid, oid, 0..1, "X")
227 .commit_transaction(&path, &replaced(&source, 0..1, "X"))
200228 .unwrap_err();
201229 assert_eq!(error.state, CommitState::NotCommitted);
202230 assert_eq!(error.error.kind(), io::ErrorKind::ResourceBusy);
......@@ -262,7 +290,7 @@ fn live_coordination() {
262290 .unwrap();
263291 drop(connection);
264292 guard.close().unwrap();
265 let error = stale.coordinate(&path, true).err().unwrap();
293 let error = stale.coordinate(&path, true, &[]).err().unwrap();
266294 assert_eq!(error.kind(), io::ErrorKind::ResourceBusy);
267295 assert_eq!(writer.read(&path, 1 << 20).unwrap(), source);
268296
......@@ -270,7 +298,7 @@ fn live_coordination() {
270298 let file = retiring
271299 .open(&path, true)
272300 .unwrap()
273 .coordinate(&path, true)
301 .coordinate(&path, true, &[])
274302 .unwrap();
275303 retiring.retire();
276304 assert_eq!(
......@@ -282,7 +310,7 @@ fn live_coordination() {
282310 loop {
283311 match writer
284312 .open(&path, true)
285 .and_then(|file| file.coordinate(&path, true))
313 .and_then(|file| file.coordinate(&path, true, &[]))
286314 {
287315 Ok(file) => {
288316 file.close().unwrap();
......@@ -300,7 +328,7 @@ fn live_coordination() {
300328 let mut unfinished = writer
301329 .open(&path, true)
302330 .unwrap()
303 .coordinate(&path, true)
331 .coordinate(&path, true, &[])
304332 .unwrap();
305333 assert_eq!(
306334 unfinished
......@@ -314,14 +342,7 @@ fn live_coordination() {
314342 assert_eq!(snapshot.len(), source.len() + 11);
315343 assert_eq!(text(&snapshot).2, before);
316344 writer
317 .commit_text(
318 &path,
319 &snapshot,
320 sid,
321 oid,
322 0..before.encode_utf16().count() as u32,
323 "Recovered café 🦀",
324 )
345 .commit_transaction(&path, &replaced(&snapshot, everything, "Recovered café 🦀"))
325346 .unwrap();
326347 let recovered = writer.read(&path, 1 << 20).unwrap();
327348 assert_eq!(text(&recovered).2, "Recovered café 🦀");
......@@ -376,19 +397,27 @@ fn live_stamp_and_transaction() {
376397 onestore::Stamp::of(&source).unwrap()
377398 );
378399 guard.close().unwrap();
379 let (sid, oid, _) = text(&source);
380 let edit = onestore::PreparedEdit::text(&source, sid, oid, 0..0, "Changed ").unwrap();
381 let transaction = edit.transaction();
400 let transaction = replaced(&source, 0..0, "Changed ");
401 let edited = applied(&source, &transaction);
382402 poller.commit_transaction(&path, &transaction).unwrap();
383403 assert_eq!(
384404 poller.stamp(&path).unwrap(),
385 onestore::Stamp::of(edit.as_bytes()).unwrap()
405 onestore::Stamp::of(&edited).unwrap()
386406 );
387 assert_eq!(poller.read(&path, 1 << 20).unwrap(), edit.as_bytes());
407 assert_eq!(poller.read(&path, 1 << 20).unwrap(), edited);
388408 let stale = poller.commit_transaction(&path, &transaction).unwrap_err();
389409 assert_eq!(stale.state, CommitState::NotCommitted);
390410 assert_eq!(stale.error.kind(), io::ErrorKind::ResourceBusy);
391 assert_eq!(poller.read(&path, 1 << 20).unwrap(), edit.as_bytes());
411 assert_eq!(poller.read(&path, 1 << 20).unwrap(), edited);
412 let base = onestore::Stamp::of(&edited).unwrap();
413 poller.confirm(&path, &base).unwrap();
414 let confirmed = poller.read(&path, 1 << 20).unwrap();
415 assert_eq!(confirmed[..212], edited[..212]);
416 assert_ne!(confirmed[212..252], edited[212..252]);
417 assert_eq!(confirmed[252..], edited[252..]);
418 let stale = poller.confirm(&path, &base).unwrap_err();
419 assert_eq!(stale.state, CommitState::NotCommitted);
420 assert_eq!(stale.error.kind(), io::ErrorKind::ResourceBusy);
392421 poller.delete(&path).unwrap();
393422}
394423
......@@ -488,7 +517,7 @@ fn live_reader_hold() {
488517 let mut file = client
489518 .open(&path, false)
490519 .unwrap()
491 .coordinate(&path, false)
520 .coordinate(&path, false, &[])
492521 .unwrap();
493522 fs::write(output.join("ready"), b"held").unwrap();
494523 let deadline = Instant::now() + Duration::from_secs(300);
......@@ -580,13 +609,13 @@ fn live_structure() {
580609 let mut notebook =
581610 Notebook::open_smb(std::sync::Arc::clone(&client), &root, cache.path()).unwrap();
582611 assert_eq!(
583 notebook.create_section("", "Second", "Author").unwrap(),
612 notebook.create_section("", "Second", &onestore::PageCreation::new(None, Some(""), "Author").unwrap()).unwrap(),
584613 "Second.one"
585614 );
586615 assert_eq!(notebook.create_group("", "Archive").unwrap(), "Archive");
587616 assert_eq!(
588617 notebook
589 .create_section("Archive", "Inner", "Author")
618 .create_section("Archive", "Inner", &onestore::PageCreation::new(None, Some(""), "Author").unwrap())
590619 .unwrap(),
591620 "Archive/Inner.one"
592621 );
crates/notebook/src/smb/tests/faults.rs+12-33
......@@ -297,7 +297,7 @@ fn live_message_loss() {
297297 &source,
298298 )
299299 .unwrap();
300 let (sid, oid, before) = text(&source);
300 let (_, _, before) = text(&source);
301301 let after = if fixture == "chunked" {
302302 "After café 🦀 ".repeat(6000)
303303 } else {
......@@ -318,15 +318,11 @@ fn live_message_loss() {
318318 );
319319 std::thread::sleep(Duration::from_millis(5));
320320 }
321 let expected = onestore::replace_text(
321 let everything = 0..before.encode_utf16().count() as u32;
322 let new = snapshot(&applied(
322323 &source,
323 sid,
324 oid,
325 0..before.encode_utf16().count() as u32,
326 &after,
327 )
328 .unwrap();
329 let new = snapshot(&expected);
324 &replaced(&source, everything.clone(), &after),
325 ));
330326 let baseline_path = format!("fault-{prefix}-{fixture}-baseline.one");
331327 create(&observer, &baseline_path, &source);
332328 configure(&output, json!({"phase":format!("{fixture}-setup")}));
......@@ -340,13 +336,9 @@ fn live_message_loss() {
340336 let start = configure(&output, json!({"phase":format!("{fixture}-baseline")}));
341337 let began = stamp();
342338 baseline
343 .commit_text(
339 .commit_transaction(
344340 &baseline_path,
345 &source,
346 sid,
347 oid,
348 0..before.encode_utf16().count() as u32,
349 &after,
341 &replaced(&source, everything.clone(), &after),
350342 )
351343 .unwrap();
352344 let events = records(&output);
......@@ -397,15 +389,9 @@ fn live_message_loss() {
397389 json!({"phase":name, "direction":direction, "cut":command, "status":status, "occurrence":occurrence}),
398390 );
399391 let began = stamp();
392 let transaction = replaced(&source, everything.clone(), &after);
400393 let error = interrupted
401 .commit_text(
402 &path,
403 &source,
404 sid,
405 oid,
406 0..before.encode_utf16().count() as u32,
407 &after,
408 )
394 .commit_transaction(&path, &transaction)
409395 .unwrap_err();
410396 assert_eq!(
411397 interrupted.read(&path, 1 << 20).unwrap_err().kind(),
......@@ -429,7 +415,7 @@ fn live_message_loss() {
429415 loop {
430416 match fresh
431417 .open(&path, true)
432 .and_then(|file| file.coordinate(&path, true))
418 .and_then(|file| file.coordinate(&path, true, &[]))
433419 {
434420 Ok(file) => {
435421 file.close().unwrap();
......@@ -458,14 +444,7 @@ fn live_message_loss() {
458444 }
459445 if !visible {
460446 fresh
461 .commit_text(
462 &path,
463 &saved,
464 sid,
465 oid,
466 0..before.encode_utf16().count() as u32,
467 &after,
468 )
447 .commit_transaction(&path, &replaced(&saved, everything.clone(), &after))
469448 .unwrap();
470449 }
471450 let saved = fresh.read(&path, 1 << 20).unwrap();
......@@ -475,7 +454,7 @@ fn live_message_loss() {
475454 );
476455 let end = after.encode_utf16().count() as u32;
477456 fresh
478 .commit_text(&path, &saved, sid, oid, end..end, suffix)
457 .commit_transaction(&path, &replaced(&saved, end..end, suffix))
479458 .unwrap();
480459 let recovered = fresh.read(&path, 1 << 20).unwrap();
481460 assert_eq!(text(&recovered).2, format!("{after}{suffix}"));
crates/notebook/src/sync.rs+40-193
......@@ -1,5 +1,5 @@
11use super::*;
2use crate::working::{Rebased, Request, Resolve, Sealed};
2use crate::working::{Request, Sealed};
33use onestore::{CommitError, CommitState, RevisionIndex, Stamp, Store, Transaction};
44use rusqlite::OptionalExtension;
55use std::{
......@@ -12,44 +12,12 @@ use std::{
1212/// cached readers.
1313pub trait Remote {
1414 fn read(&mut self) -> io::Result<Vec<u8>>;
15 /// The file's stamp without reading its body, when the remote can tell. While it is the
16 /// last observed image's, synchronization neither reads nor revalidates the file.
17 fn stamp(&mut self) -> io::Result<Option<Stamp>> {
18 Ok(None)
19 }
15 /// The file's stamp without reading its body or coordinating with writers. While it is
16 /// the last observed image's, synchronization neither reads nor revalidates the file.
17 fn stamp(&mut self) -> io::Result<Stamp>;
2018 fn publish(&mut self, transaction: &Transaction) -> std::result::Result<(), CommitError>;
21 fn confirm(&mut self, snapshot: &[u8]) -> std::result::Result<(), CommitError>;
22}
23
24#[derive(Debug, Clone, Copy, PartialEq, Eq)]
25#[repr(i64)]
26pub enum ConflictKind {
27 /// The edited page or a selected page no longer exists remotely.
28 TargetUnavailable = 0,
29 /// The remote section no longer accepts the edit.
30 UnsupportedEdit = 1,
31 /// Page order or indentation changed remotely in a way the batch cannot absorb.
32 StructureChanged = 2,
33 /// The remote page changed where the local edits changed it too; review is required.
34 ContentChanged = 3,
35}
36
37impl ConflictKind {
38 fn from_stored(kind: i64) -> Result<Self> {
39 Ok(match kind {
40 0 => Self::TargetUnavailable,
41 1 => Self::UnsupportedEdit,
42 2 => Self::StructureChanged,
43 3 => Self::ContentChanged,
44 _ => {
45 return Err(io::Error::new(
46 io::ErrorKind::InvalidData,
47 "Unknown cached conflict kind",
48 )
49 .into());
50 }
51 })
52 }
19 /// Confirms that the file still has `base`'s stamp and is durable (`onestore::confirm`).
20 fn confirm(&mut self, base: &Stamp) -> std::result::Result<(), CommitError>;
5321}
5422
5523#[derive(Debug, Clone, PartialEq, Eq)]
......@@ -59,7 +27,6 @@ pub enum EditStatus {
5927 AwaitingConfirmation {
6028 revision: ExGuid,
6129 },
62 Conflict(ConflictKind),
6330 /// Historical confirmation; later remote edits or restores may remove the effect.
6431 Published {
6532 revision: ExGuid,
......@@ -84,15 +51,13 @@ struct State {
8451 base: Stamp,
8552 /// The observed remote image's stamp while it is not the base.
8653 remote: Option<Stamp>,
87 /// The first batch waiting on a remote answer: sealed, attempted, or in conflict.
54 /// The first batch waiting on a remote answer to an attempt.
8855 blocked: Option<Blocked>,
8956 queued: bool,
9057}
9158
9259struct Blocked {
9360 batch: i64,
94 attempted: bool,
95 conflict: Option<ConflictKind>,
9661 revisions: Option<BTreeMap<ExGuid, ExGuid>>,
9762}
9863
......@@ -101,24 +66,14 @@ fn state(connection: &Connection) -> Result<State> {
10166 .ok_or_else(|| io::Error::new(io::ErrorKind::InvalidData, "The cache has no base image"))?;
10267 let blocked = connection
10368 .query_row(
104 "SELECT id, attempted, conflict, revisions FROM batches
105 WHERE attempted=1 OR conflict IS NOT NULL ORDER BY id LIMIT 1",
69 "SELECT id, revisions FROM batches WHERE attempted=1 ORDER BY id LIMIT 1",
10670 [],
107 |row| {
108 Ok((
109 row.get::<_, i64>(0)?,
110 row.get::<_, bool>(1)?,
111 row.get::<_, Option<i64>>(2)?,
112 row.get::<_, Option<String>>(3)?,
113 ))
114 },
71 |row| Ok((row.get::<_, i64>(0)?, row.get::<_, Option<String>>(1)?)),
11572 )
11673 .optional()?
117 .map(|(batch, attempted, conflict, revisions)| {
74 .map(|(batch, revisions)| {
11875 Ok::<_, Error>(Blocked {
11976 batch,
120 attempted,
121 conflict: conflict.map(ConflictKind::from_stored).transpose()?,
12277 revisions: revisions
12378 .map(|revisions| decode_revisions(&revisions))
12479 .transpose()?,
......@@ -186,7 +141,7 @@ impl Replica {
186141
187142 /// The last observed remote image. Observation alone does not acknowledge any pending
188143 /// edit's remote durability.
189 pub fn remote_snapshot(&self) -> Result<Vec<u8>> {
144 pub(crate) fn remote_snapshot(&self) -> Result<Vec<u8>> {
190145 let connection = self.lock()?;
191146 let image = match base::read(&connection, base::Image::Remote)? {
192147 Some(image) => Some(image),
......@@ -197,26 +152,6 @@ impl Replica {
197152 })?)
198153 }
199154
200 /// The unpublished batch the remote no longer accepts, if any.
201 pub fn conflict(&self) -> Result<Option<Conflict>> {
202 let connection = self.lock()?;
203 let row: Option<(i64, i64, String)> = connection
204 .query_row(
205 "SELECT id, conflict, space FROM batches WHERE conflict IS NOT NULL ORDER BY id LIMIT 1",
206 [],
207 |row| Ok((row.get(0)?, row.get(1)?, row.get(2)?)),
208 )
209 .optional()?;
210 row.map(|(batch, kind, space)| {
211 Ok(Conflict {
212 id: newest(&connection, batch)?,
213 space: space.parse()?,
214 kind: ConflictKind::from_stored(kind)?,
215 })
216 })
217 .transpose()
218 }
219
220155 /// Publishes the oldest unpublished batch, rebasing the queue first when the remote
221156 /// changed. Reads the remote image only when its stamp moved; network I/O holds
222157 /// synchronization ownership without holding the cache mutex. Uncertain attempts are
......@@ -225,8 +160,8 @@ impl Replica {
225160 let _owner = self.sync_owner()?;
226161 let state = state(&*self.lock()?)?;
227162 let observed = remote.stamp().map_err(Error::RemoteIo)?;
228 if let (Some(observed), Some(blocked)) = (&observed, &state.blocked)
229 && Some(observed) == state.remote.as_ref().or(Some(&state.base))
163 if let Some(blocked) = &state.blocked
164 && observed == *state.remote.as_ref().unwrap_or(&state.base)
230165 {
231166 // Nothing new to decide: the remote is as it was when the batch blocked.
232167 let id = newest(&*self.lock()?, blocked.batch)?;
......@@ -239,7 +174,7 @@ impl Replica {
239174 });
240175 }
241176 let image = match observed {
242 Some(observed) if observed == state.base => None,
177 observed if observed == state.base => None,
243178 _ => {
244179 let image = remote.read().map_err(Error::RemoteIo)?;
245180 // A read image is compared whole: a stamp stands for it only when read alone.
......@@ -249,13 +184,7 @@ impl Replica {
249184 };
250185 let mut changed = Vec::new();
251186 if let Some(image) = image {
252 if let Some(Blocked {
253 batch,
254 attempted: true,
255 revisions,
256 ..
257 }) = &state.blocked
258 {
187 if let Some(Blocked { batch, revisions }) = &state.blocked {
259188 let id = newest(&*self.lock()?, *batch)?;
260189 let revisions = revisions.clone().unwrap_or_default();
261190 let store = Store::parse(&image)?;
......@@ -308,7 +237,7 @@ impl Replica {
308237 .collect(),
309238 )
310239 };
311 if let Err(error) = remote.confirm(&image) {
240 if let Err(error) = remote.confirm(&Stamp::of(&image)?) {
312241 if error.state == CommitState::Committed {
313242 self.acknowledge(*batch, sealed.as_ref(), receipts.as_ref())?;
314243 }
......@@ -316,49 +245,26 @@ impl Replica {
316245 }
317246 self.acknowledge(*batch, sealed.as_ref(), receipts.as_ref())?;
318247 let revision = self.receipt(id)?;
319 changed = self.rebase(Some(image), None)?.unwrap_or_default();
248 changed = self.rebase(Some(image))?;
320249 return Ok(Synced {
321250 edit: Some((id, EditStatus::Published { revision })),
322251 changed,
323252 });
324253 }
325 match self.rebase(Some(image), None)? {
326 Ok(spaces) => changed = spaces,
327 Err(conflict) => {
328 return Ok(Synced {
329 edit: Some((conflict.id, EditStatus::Conflict(conflict.kind))),
330 changed,
331 });
332 }
333 }
254 changed = self.rebase(Some(image))?;
334255 } else if let Some(blocked) = &state.blocked {
335256 if state.remote.is_some() {
336257 // The remote is back at the base: what blocked the queue is gone.
337 let connection = self.lock()?;
338 base::clear(&connection, base::Image::Remote)?;
339 if blocked.conflict.is_some() {
340 connection.execute("UPDATE batches SET conflict=NULL, space=NULL", [])?;
341 }
342 }
343 if blocked.attempted {
344 let id = newest(&*self.lock()?, blocked.batch)?;
345 return Ok(Synced {
346 edit: Some((
347 id,
348 status(&*self.lock()?, id)?.unwrap_or(EditStatus::Pending),
349 )),
350 changed,
351 });
352 }
353 if let Some(kind) = blocked.conflict
354 && state.remote.is_none()
355 {
356 let id = newest(&*self.lock()?, blocked.batch)?;
357 return Ok(Synced {
358 edit: Some((id, EditStatus::Conflict(kind))),
359 changed,
360 });
258 base::clear(&*self.lock()?, base::Image::Remote)?;
361259 }
260 let id = newest(&*self.lock()?, blocked.batch)?;
261 return Ok(Synced {
262 edit: Some((
263 id,
264 status(&*self.lock()?, id)?.unwrap_or(EditStatus::Pending),
265 )),
266 changed,
267 });
362268 } else if !state.queued {
363269 return Ok(Synced::default());
364270 }
......@@ -383,7 +289,9 @@ impl Replica {
383289 let id = newest(&*self.lock()?, batch)?;
384290 let Some(transaction) = transaction else {
385291 // Edits that changed nothing are published once the remote's image is durable.
386 let base = base::read(&*self.lock()?, base::Image::Base)?.unwrap_or_default();
292 let base = base::stamp(&*self.lock()?, base::Image::Base)?.ok_or_else(|| {
293 io::Error::new(io::ErrorKind::InvalidData, "The cache holds no base image")
294 })?;
387295 if let Err(error) = remote.confirm(&base) {
388296 if error.state == CommitState::Committed {
389297 self.acknowledge(batch, None, None)?;
......@@ -439,22 +347,10 @@ impl Replica {
439347 ))
440348 }
441349
442 /// Asks the section thread to replay the queue on `image`; `Err` is the conflict found.
443 fn rebase(
444 &self,
445 image: Option<Vec<u8>>,
446 resolve: Option<Resolve>,
447 ) -> Result<std::result::Result<Vec<ExGuid>, Conflict>> {
448 Ok(
449 match self.ask(|reply| Request::Rebase {
450 image,
451 resolve,
452 reply,
453 })? {
454 Rebased::Applied { changed } => Ok(changed),
455 Rebased::Conflict { id, space, kind } => Err(Conflict { id, space, kind }),
456 },
457 )
350 /// Asks the section thread to replay the queue on `image`, returning the pages the
351 /// remote changed.
352 fn rebase(&self, image: Option<Vec<u8>>) -> Result<Vec<ExGuid>> {
353 self.ask(|reply| Request::Rebase { image, reply })
458354 }
459355
460356 fn receipt(&self, id: u64) -> Result<ExGuid> {
......@@ -501,49 +397,6 @@ impl Replica {
501397 Ok(())
502398 }
503399
504 /// Resolves the conflict holding edit `id`: the batch and everything queued after it
505 /// drop their ops on the conflicted page; `Mine` rewrites the remote page to the local
506 /// one first. A conflict on another page may follow.
507 pub fn resolve(&self, id: u64, resolution: Resolution) -> Result<()> {
508 self.resolve_with(id, resolution, None)
509 }
510
511 pub(crate) fn resolve_with(&self, id: u64, keep: Resolution, page: Option<Page>) -> Result<()> {
512 let owner = self.sync_owner()?;
513 let (first, space) = {
514 let connection = self.lock()?;
515 let row: Option<(i64, String)> = connection
516 .query_row(
517 "SELECT (SELECT min(id) FROM edits WHERE batch=batches.id), space FROM batches
518 WHERE conflict IS NOT NULL AND id=(SELECT batch FROM edits WHERE id=?1)",
519 [signed(id)?],
520 |row| Ok((row.get(0)?, row.get(1)?)),
521 )
522 .optional()?;
523 let Some((first, space)) = row else {
524 return Err(io::Error::new(
525 io::ErrorKind::InvalidInput,
526 "The edit is not in conflict",
527 )
528 .into());
529 };
530 (unsigned(first)?, space.parse()?)
531 };
532 // A conflict on another page is recorded like any other.
533 let _ = self.rebase(
534 None,
535 Some(Resolve {
536 first,
537 space,
538 keep,
539 page,
540 }),
541 )?;
542 drop(owner);
543 self.wake_sync();
544 Ok(())
545 }
546
547400 /// Retires the uncertain attempt of the batch holding edit `id` after review. The
548401 /// queue is first exported to `archive` (a new file), which is the record: no receipt
549402 /// is written. `Mine` publishes the batch again against the current remote; `Theirs`
......@@ -620,10 +473,7 @@ impl Replica {
620473 None if !state.queued => state.base,
621474 None => return Ok(false),
622475 };
623 let Some(stamp) = remote.stamp().map_err(Error::RemoteIo)? else {
624 return Ok(false);
625 };
626 Ok(stamp == expected)
476 Ok(remote.stamp().map_err(Error::RemoteIo)? == expected)
627477 }
628478}
629479
......@@ -651,26 +501,23 @@ pub(crate) fn status(connection: &Connection, id: u64) -> Result<Option<EditStat
651501 {
652502 return Ok(Some(EditStatus::Archived { archive }));
653503 }
654 let record: Option<(bool, Option<i64>, Option<String>, String)> = connection
504 let record: Option<(bool, Option<String>, String)> = connection
655505 .query_row(
656 "SELECT batches.attempted, batches.conflict, batches.revisions, edits.edit
506 "SELECT batches.attempted, batches.revisions, edits.edit
657507 FROM edits JOIN batches ON batches.id=edits.batch WHERE edits.id=?1",
658508 [id],
659 |row| Ok((row.get(0)?, row.get(1)?, row.get(2)?, row.get(3)?)),
509 |row| Ok((row.get(0)?, row.get(1)?, row.get(2)?)),
660510 )
661511 .optional()?;
662512 Ok(match record {
663513 None => None,
664 Some((true, _, revisions, edit)) => {
514 Some((true, revisions, edit)) => {
665515 let revisions = decode_revisions(revisions.as_deref().unwrap_or("{}"))?;
666516 let edit: onestore::op::Edit = serde_json::from_str(&edit)
667517 .map_err(|error| io::Error::new(io::ErrorKind::InvalidData, error))?;
668518 let revision = revision_of(&edit, &revisions)?;
669519 Some(EditStatus::AwaitingConfirmation { revision })
670520 }
671 Some((false, Some(kind), ..)) => {
672 Some(EditStatus::Conflict(ConflictKind::from_stored(kind)?))
673 }
674 Some((false, None, ..)) => Some(EditStatus::Pending),
521 Some((false, ..)) => Some(EditStatus::Pending),
675522 })
676523}
crates/notebook/src/working.rs+120-327
......@@ -4,12 +4,11 @@
44//! and, when the remote changed, to rebase the queue.
55
66use crate::{
7 ConflictKind, Resolution, Result, base, lock, merge, queue, session::Save, signed,
8 worker::Signal,
7 Result, base, lock, merge, queue, worker::Signal,
98};
109use onestore::{
1110 Arena, ExGuid, Section, Transaction,
12 op::{Edit, Op, OpError},
11 op::{Edit, OpError},
1312 page::Page,
1413};
1514use rusqlite::{Connection, OptionalExtension, TransactionBehavior, params};
......@@ -28,15 +27,6 @@ pub(crate) enum Request {
2827 edit: Edit,
2928 reply: Reply<u64>,
3029 },
31 /// A whole-page save of the old session API: `None` answers a save a later one
32 /// continuing it absorbed.
33 Save {
34 space: ExGuid,
35 before: Page,
36 after: Page,
37 author: String,
38 reply: Reply<Option<Save>>,
39 },
4030 Page {
4131 space: ExGuid,
4232 reply: Reply<Page>,
......@@ -44,6 +34,9 @@ pub(crate) enum Request {
4434 Pages {
4535 reply: Reply<Vec<(ExGuid, String, u32)>>,
4636 },
37 Conflicts {
38 reply: Reply<Vec<(ExGuid, Vec<onestore::ConflictPage>)>>,
39 },
4740 /// Answers once the edits before it are written.
4841 Flush {
4942 reply: Reply<()>,
......@@ -53,11 +46,11 @@ pub(crate) enum Request {
5346 Seal {
5447 reply: Reply<Option<Sealed>>,
5548 },
56 /// Replays the queue on `image` (the stored remote image, or the base, when `None`).
49 /// Replays the queue on `image` (the stored remote image, or the base, when `None`);
50 /// answers with the pages the remote changed.
5751 Rebase {
5852 image: Option<Vec<u8>>,
59 resolve: Option<Resolve>,
60 reply: Reply<Rebased>,
53 reply: Reply<Vec<ExGuid>>,
6154 },
6255 /// Rereads the queue after the sync thread replaced it.
6356 Reopen {
......@@ -78,34 +71,14 @@ pub(crate) struct Sealed {
7871 pub transaction: Option<Transaction>,
7972}
8073
81/// A conflict's resolution: the batch it holds and every later edit drop their ops on
82/// `space`; `Mine` first rewrites the rebased page to `page`, the local one unless given.
83pub(crate) struct Resolve {
84 pub first: u64,
85 pub space: ExGuid,
86 pub keep: Resolution,
87 pub page: Option<Page>,
88}
89
90pub(crate) enum Rebased {
91 /// The queue now applies to the image, which is the base; these pages changed.
92 Applied { changed: Vec<ExGuid> },
93 /// The batch holding edit `id` no longer applies; nothing changed but the record.
94 Conflict {
95 id: u64,
96 space: ExGuid,
97 kind: ConflictKind,
98 },
99}
100
10174impl Request {
10275 fn fail(self, message: &str) {
10376 let error = || io::Error::other(message.to_owned()).into();
10477 match self {
10578 Self::Apply { reply, .. } => reply(Err(error())),
106 Self::Save { reply, .. } => reply(Err(error())),
10779 Self::Page { reply, .. } => reply(Err(error())),
10880 Self::Pages { reply } => reply(Err(error())),
81 Self::Conflicts { reply } => reply(Err(error())),
10982 Self::Flush { reply } => reply(Err(error())),
11083 Self::Seal { reply } => reply(Err(error())),
11184 Self::Rebase { reply, .. } => reply(Err(error())),
......@@ -244,8 +217,7 @@ fn fail(error: crate::Error, backlog: VecDeque<Request>, requests: mpsc::Receive
244217enum Job {
245218 Rebase {
246219 image: Option<Vec<u8>>,
247 resolve: Option<Resolve>,
248 reply: Reply<Rebased>,
220 reply: Reply<Vec<ExGuid>>,
249221 },
250222 Reopen {
251223 reply: Reply<()>,
......@@ -253,18 +225,14 @@ enum Job {
253225}
254226
255227/// Runs `job`, rereads the section and takes the requests over from the thread that
256/// started it; a rebase that found a conflict changed nothing that thread serves.
228/// started it; a failed rebase changed nothing that thread serves.
257229fn build(shared: Arc<Thread>, job: Job, signal: mpsc::Sender<Request>) {
258230 let answer: Box<dyn FnOnce() + Send> = match job {
259231 Job::Reopen { reply } => Box::new(move || reply(Ok(()))),
260 Job::Rebase {
261 image,
262 resolve,
263 reply,
264 } => match rebase(&shared.connection, image, resolve) {
265 Ok(applied @ Rebased::Applied { .. }) => Box::new(move || reply(Ok(applied))),
266 other => {
267 reply(other);
232 Job::Rebase { image, reply } => match rebase(&shared.connection, image) {
233 Ok(changed) => Box::new(move || reply(Ok(changed))),
234 Err(error) => {
235 reply(Err(error));
268236 let _ = signal.send(Request::Resume);
269237 return;
270238 }
......@@ -298,21 +266,7 @@ fn build(shared: Arc<Thread>, job: Job, signal: mpsc::Sender<Request>) {
298266struct Accepted {
299267 author: String,
300268 edit: Edit,
301 done: Done,
302}
303
304enum Done {
305 Apply(Reply<u64>),
306 Save(Reply<Option<Save>>),
307}
308
309impl Done {
310 fn reply(self, written: Result<u64>) {
311 match self {
312 Self::Apply(reply) => reply(written),
313 Self::Save(reply) => reply(written.map(|id| Some(Save::Queued(id)))),
314 }
315 }
269 reply: Reply<u64>,
316270}
317271
318272struct Working<'a> {
......@@ -321,8 +275,6 @@ struct Working<'a> {
321275 open: Option<i64>,
322276 /// Spaces the open batch's edits change.
323277 touched: BTreeSet<ExGuid>,
324 /// The last whole-page save: its space, the page it was given, and the page as stored.
325 saved: Option<(ExGuid, Page, Page)>,
326278}
327279
328280impl<'a> Working<'a> {
......@@ -332,7 +284,6 @@ impl<'a> Working<'a> {
332284 section,
333285 open,
334286 touched,
335 saved: None,
336287 })
337288 }
338289
......@@ -367,6 +318,9 @@ impl<'a> Working<'a> {
367318 reply(self.section.page(space).map_err(Into::into))
368319 }
369320 Request::Pages { reply } => reply(self.section.pages().map_err(Into::into)),
321 Request::Conflicts { reply } => {
322 reply(self.section.conflicts().map_err(Into::into))
323 }
370324 Request::Handover(to) => {
371325 let mut waiting = held.take().unwrap_or_default();
372326 waiting.extend(burst.drain(..).chain(later.drain(..)));
......@@ -386,86 +340,9 @@ impl<'a> Working<'a> {
386340 edit,
387341 reply,
388342 } => self
389 .accept(
390 author,
391 edit,
392 Done::Apply(reply),
393 &mut accepted,
394 connection,
395 worker,
396 )
343 .accept(author, edit, reply, &mut accepted, connection, worker)
397344 .err()
398345 .unwrap_or(false),
399 Request::Save {
400 space,
401 before,
402 mut after,
403 author,
404 reply,
405 } => {
406 // Saves that continue one another are written once.
407 let mut replies = vec![reply];
408 while let Some(Request::Save {
409 space: next,
410 before: continued,
411 author: by,
412 ..
413 }) = burst.front()
414 && (*next, continued, by) == (space, &after, &author)
415 {
416 let Some(Request::Save {
417 after: later,
418 reply,
419 ..
420 }) = burst.pop_front()
421 else {
422 unreachable!()
423 };
424 after = later;
425 replies.push(reply);
426 }
427 let last = replies.pop().unwrap();
428 for folded in replies {
429 folded(Ok(None));
430 }
431 match self.save(space, &before, &after) {
432 Ok(Some(edit)) => match self.accept(
433 author,
434 edit,
435 Done::Save(last),
436 &mut accepted,
437 connection,
438 worker,
439 ) {
440 Ok(()) => {
441 self.saved = self
442 .section
443 .page(space)
444 .ok()
445 .map(|stored| (space, after, stored));
446 false
447 }
448 Err(broken) => broken,
449 },
450 Ok(None) => {
451 self.saved = self
452 .section
453 .page(space)
454 .ok()
455 .map(|stored| (space, after, stored));
456 last(Ok(Some(Save::Unchanged)));
457 false
458 }
459 Err(Stale::Stale) => {
460 last(Ok(Some(Save::Stale)));
461 false
462 }
463 Err(Stale::Error(error)) => {
464 last(Err(error));
465 false
466 }
467 }
468 }
469346 Request::Page { space, reply } => {
470347 reply(self.section.page(space).map_err(Into::into));
471348 false
......@@ -474,6 +351,10 @@ impl<'a> Working<'a> {
474351 reply(self.section.pages().map_err(Into::into));
475352 false
476353 }
354 Request::Conflicts { reply } => {
355 reply(self.section.conflicts().map_err(Into::into));
356 false
357 }
477358 request @ (Request::Flush { .. } | Request::Seal { .. })
478359 if !burst.is_empty() =>
479360 {
......@@ -502,50 +383,15 @@ impl<'a> Working<'a> {
502383 failed
503384 }
504385 }
505 Request::Rebase {
506 image,
507 mut resolve,
508 reply,
509 } => {
510 let written = self.flush(connection, worker, &mut accepted);
511 // The page a `Mine` resolution keeps is read here, before the
512 // rebuild; the root space lists pages and holds none.
513 let root = self.section.root();
514 let kept = match &mut resolve {
515 Some(Resolve {
516 keep: Resolution::Mine,
517 page: page @ None,
518 space,
519 ..
520 }) if *space != root => {
521 self.section.page(*space).map(|local| *page = Some(local))
522 }
523 _ => Ok(()),
524 };
525 match (written, kept) {
526 (true, Ok(())) => {
527 held = self
528 .rebuild(
529 shared,
530 Job::Rebase {
531 image,
532 resolve,
533 reply,
534 },
535 )
536 .then(VecDeque::new);
537 false
538 }
539 (_, Err(error)) => {
540 reply(Err(error.into()));
541 !written
542 }
543 (false, _) => {
544 reply(Err(
545 io::Error::other("The queue could not be written").into()
546 ));
547 true
548 }
386 Request::Rebase { image, reply } => {
387 if self.flush(connection, worker, &mut accepted) {
388 held = self
389 .rebuild(shared, Job::Rebase { image, reply })
390 .then(VecDeque::new);
391 false
392 } else {
393 reply(Err(io::Error::other("The queue could not be written").into()));
394 true
549395 }
550396 }
551397 Request::Reopen { reply } => {
......@@ -598,7 +444,7 @@ impl<'a> Working<'a> {
598444 &mut self,
599445 author: String,
600446 edit: Edit,
601 done: Done,
447 reply: Reply<u64>,
602448 accepted: &mut Vec<Accepted>,
603449 connection: &Mutex<Connection>,
604450 worker: &Worker,
......@@ -607,7 +453,11 @@ impl<'a> Working<'a> {
607453 Ok(()) => {
608454 self.touched
609455 .extend(queue::spaces(&edit, self.section.root()));
610 accepted.push(Accepted { author, edit, done });
456 accepted.push(Accepted {
457 author,
458 edit,
459 reply,
460 });
611461 Ok(())
612462 }
613463 Err(error) => {
......@@ -615,40 +465,12 @@ impl<'a> Working<'a> {
615465 if broken {
616466 self.flush(connection, worker, accepted);
617467 }
618 done.reply(Err(error.into()));
468 reply(Err(error.into()));
619469 Err(broken)
620470 }
621471 }
622472 }
623473
624 /// The edit that takes the stored page from `before` to `after`. A save continuing
625 /// the previous one finds the page as that one stored it, which may read back
626 /// normalized.
627 fn save(
628 &mut self,
629 space: ExGuid,
630 before: &Page,
631 after: &Page,
632 ) -> std::result::Result<Option<Edit>, Stale> {
633 let current = self
634 .section
635 .page(space)
636 .map_err(|error| Stale::Error(error.into()))?;
637 let expected = match &self.saved {
638 Some((saved, given, stored)) if *saved == space && given == before => stored,
639 _ => before,
640 };
641 if current != *expected {
642 return Err(Stale::Stale);
643 }
644 let ops = onestore::op::lower_page(&current, after)
645 .map_err(|error| Stale::Error(OpError::Unsupported(error.message).into()))?;
646 Ok((!ops.is_empty()).then(|| Edit {
647 at: crate::now(),
648 ops: ops.into_iter().map(|op| Op::Page { space, op }).collect(),
649 }))
650 }
651
652474 /// Writes the accepted edits in one transaction, then answers their senders; false
653475 /// when the write failed and the section holds edits the queue lacks.
654476 fn flush(
......@@ -689,15 +511,14 @@ impl<'a> Working<'a> {
689511 match written {
690512 Ok(ids) => {
691513 for (edit, id) in accepted.drain(..).zip(ids) {
692 edit.done.reply(Ok(id));
514 (edit.reply)(Ok(id));
693515 }
694516 crate::wake(worker);
695517 }
696518 Err(error) => {
697519 let message = error.to_string();
698520 for edit in accepted.drain(..) {
699 edit.done
700 .reply(Err(io::Error::other(message.clone()).into()));
521 (edit.reply)(Err(io::Error::other(message.clone()).into()));
701522 }
702523 }
703524 }
......@@ -712,7 +533,7 @@ impl<'a> Working<'a> {
712533 {
713534 let connection = lock(connection)?;
714535 let waiting: bool = connection.query_row(
715 "SELECT EXISTS(SELECT 1 FROM batches WHERE sealed IS NOT NULL OR conflict IS NOT NULL)",
536 "SELECT EXISTS(SELECT 1 FROM batches WHERE sealed IS NOT NULL)",
716537 [],
717538 |row| row.get(0),
718539 )?;
......@@ -746,12 +567,10 @@ impl<'a> Working<'a> {
746567 }
747568}
748569
749/// Replays every queued edit on `image`, keeping it as the base when all apply.
750fn rebase(
751 connection: &Mutex<Connection>,
752 image: Option<Vec<u8>>,
753 resolve: Option<Resolve>,
754) -> Result<Rebased> {
570/// Replays every queued edit on `image`, which becomes the base; each page whose local
571/// version the remote could not take gains a conflict page holding it, queued as a new edit.
572/// Returns the pages the remote changed.
573fn rebase(connection: &Mutex<Connection>, image: Option<Vec<u8>>) -> Result<Vec<ExGuid>> {
755574 let (base_image, image, edits) = {
756575 let connection = lock(connection)?;
757576 if connection.query_row(
......@@ -777,117 +596,91 @@ fn rebase(
777596 (base_image, image, queue::load(&connection, None)?)
778597 };
779598 let old_arena = Arena::default();
780 let mut old = Section::open(&old_arena, base_image.clone())?;
599 let mut old = Section::open(&old_arena, base_image)?;
781600 let before: BTreeMap<ExGuid, ExGuid> = old.revisions().collect();
782 let mut after: BTreeMap<ExGuid, ExGuid>;
601 let remote_arena = Arena::default();
602 let remote = Section::open(&remote_arena, image.clone())?;
603 if old.root() != remote.root() {
604 return Err(io::Error::new(
605 io::ErrorKind::InvalidInput,
606 "Remote snapshot belongs to another document",
607 )
608 .into());
609 }
610 let after: BTreeMap<ExGuid, ExGuid> = remote.revisions().collect();
611 // The pages as the queue leaves them, read once a page conflicts: O(section).
612 let local_arena = Arena::default();
613 let mut local = None;
783614 // Pages the remote already holds as the local edits leave them: their ops are done.
784615 let mut converged = BTreeSet::new();
785 let outcome = loop {
616 let (rewritten, added) = loop {
786617 let arena = Arena::default();
787618 let mut new = Section::open(&arena, image.clone())?;
788 if old.root() != new.root() {
789 return Err(io::Error::new(
790 io::ErrorKind::InvalidInput,
791 "Remote snapshot belongs to another document",
792 )
793 .into());
619 let merged = merge::rebase(&mut old, &mut new, &edits, &converged)?;
620 if !merged.conflicts.is_empty() && local.is_none() {
621 local = Some(replay(&local_arena, &*lock(connection)?)?.0);
794622 }
795 after = new.revisions().collect();
796 let outcome = merge::rebase(
797 &mut old,
798 &mut new,
799 &edits,
800 &converged,
801 resolve.as_ref().map(|resolve| merge::Resolving {
802 first: resolve.first,
803 space: resolve.space,
804 mine: resolve.keep == Resolution::Mine,
805 target: resolve.page.as_ref(),
806 }),
807 )?;
808 if let merge::Outcome::Conflict { space, .. } = &outcome
809 && !converged.contains(space)
810 && let Some(local) = local_page(connection, *space)?
811 && Section::open(&Arena::default(), image.clone())?
812 .page(*space)
813 .is_ok_and(|remote| remote == local)
814 {
815 converged.insert(*space);
623 let pages: BTreeMap<ExGuid, Page> = merged
624 .conflicts
625 .keys()
626 .filter_map(|space| Some((*space, local.as_ref()?.page(*space).ok()?)))
627 .collect();
628 let settled: Vec<ExGuid> = pages
629 .iter()
630 .filter(|(space, page)| remote.page(**space).is_ok_and(|remote| remote == **page))
631 .map(|(space, _)| *space)
632 .collect();
633 if !settled.is_empty() {
634 converged.extend(settled);
816635 continue;
817636 }
818 break outcome;
819 };
820 let mut connection = lock(connection)?;
821 let transaction = connection.transaction_with_behavior(TransactionBehavior::Immediate)?;
822 let rebased = match outcome {
823 merge::Outcome::Conflict { id, space, kind } => {
824 transaction.execute("UPDATE batches SET conflict=NULL, space=NULL", [])?;
825 transaction.execute(
826 "UPDATE batches SET conflict=?1, space=?2 WHERE id=(SELECT batch FROM edits WHERE id=?3)",
827 params![kind as i64, space.to_string(), signed(id)?],
637 let mut added = Vec::new();
638 for (space, (author, objects)) in &merged.conflicts {
639 // A page the queue went on to delete keeps no version of its own.
640 let (Some(page), Some(local)) = (pages.get(space), local.as_mut()) else {
641 continue;
642 };
643 let edit = merge::conflict_page(
644 &mut new,
645 &mut old,
646 local,
647 *space,
648 page,
649 author,
650 objects,
651 crate::now(),
828652 )?;
829 if image == base_image {
830 base::clear(&transaction, base::Image::Remote)?;
831 } else {
832 base::write(&transaction, base::Image::Remote, &image)?;
833 }
834 // A conflict is reported under its batch's newest edit.
835 let id = crate::unsigned(transaction.query_row(
836 "SELECT max(id) FROM edits WHERE batch=(SELECT batch FROM edits WHERE id=?1)",
837 [signed(id)?],
838 |row| row.get(0),
839 )?)?;
840 Rebased::Conflict { id, space, kind }
841 }
842 merge::Outcome::Applied(rewritten) => {
843 base::write(&transaction, base::Image::Base, &image)?;
844 base::clear(&transaction, base::Image::Remote)?;
845 transaction.execute("INSERT INTO batches DEFAULT VALUES", [])?;
846 let batch = transaction.last_insert_rowid();
847 transaction.execute("UPDATE edits SET batch=?1", [batch])?;
848 transaction.execute("DELETE FROM batches WHERE id<>?1", [batch])?;
849 for (id, edit) in rewritten {
850 queue::rewrite(&transaction, id, &edit)?;
851 }
852 if transaction
853 .query_row("SELECT count(*) FROM edits", [], |row| row.get::<_, i64>(0))?
854 == 0
855 {
856 transaction.execute("DELETE FROM batches", [])?;
857 }
858 queue::collect(&transaction)?;
859 let mut changed: BTreeSet<ExGuid> = before
860 .iter()
861 .filter(|(space, rid)| after.get(space) != Some(rid))
862 .map(|(space, _)| *space)
863 .chain(
864 after
865 .keys()
866 .filter(|space| !before.contains_key(space))
867 .copied(),
868 )
869 .collect();
870 changed.extend(resolve.as_ref().map(|resolve| resolve.space));
871 changed.extend(converged);
872 Rebased::Applied {
873 changed: changed.into_iter().collect(),
874 }
653 added.push((author.clone(), edit));
875654 }
655 break (merged.rewritten, added);
876656 };
657 let mut connection = lock(connection)?;
658 let transaction = connection.transaction_with_behavior(TransactionBehavior::Immediate)?;
659 base::write(&transaction, base::Image::Base, &image)?;
660 base::clear(&transaction, base::Image::Remote)?;
661 transaction.execute("INSERT INTO batches DEFAULT VALUES", [])?;
662 let batch = transaction.last_insert_rowid();
663 transaction.execute("UPDATE edits SET batch=?1", [batch])?;
664 transaction.execute("DELETE FROM batches WHERE id<>?1", [batch])?;
665 for (id, edit) in rewritten {
666 queue::rewrite(&transaction, id, &edit)?;
667 }
668 for (author, edit) in &added {
669 queue::insert(&transaction, None, batch, author, edit)?;
670 }
671 if transaction.query_row("SELECT count(*) FROM edits", [], |row| row.get::<_, i64>(0))? == 0 {
672 transaction.execute("DELETE FROM batches", [])?;
673 }
674 queue::collect(&transaction)?;
877675 transaction.commit()?;
878 Ok(rebased)
879}
880
881/// A page as the queued edits leave it, read from the cache: O(section).
882fn local_page(connection: &Mutex<Connection>, space: ExGuid) -> Result<Option<Page>> {
883 let arena = Arena::default();
884 let (section, ..) = replay(&arena, &*lock(connection)?)?;
885 Ok(section.page(space).ok())
886}
887
888enum Stale {
889 Stale,
890 Error(crate::Error),
676 let mut changed: BTreeSet<ExGuid> = before
677 .iter()
678 .filter(|(space, rid)| after.get(space) != Some(rid))
679 .map(|(space, _)| *space)
680 .chain(after.keys().filter(|space| !before.contains_key(space)).copied())
681 .collect();
682 changed.extend(converged);
683 Ok(changed.into_iter().collect())
891684}
892685
893686/// Batches in order with their sealed transactions: `None` while open, `Some(None)` when
crates/notebook/tests/assets.rs+16-9
......@@ -15,6 +15,7 @@ use std::{
1515mod model_ops;
1616#[path = "support/server.rs"]
1717mod server;
18use server::{remote_snapshot, snapshot};
1819
1920const FIXTURE: &str = "../../corpus/native-external-assets/notebook";
2021
......@@ -86,8 +87,8 @@ impl notebook::discover::Source for Payload {
8687fn downloaded_media_survives_reopen_and_recovery_with_the_queue_intact() {
8788 let directory = tempfile::tempdir().unwrap();
8889 let cache = queued_cache(directory.path());
89 let working = cache.snapshot().unwrap();
90 let remote = cache.remote_snapshot().unwrap();
90 let working = snapshot(&cache);
91 let remote = remote_snapshot(&cache);
9192 let pending = cache.pending().unwrap();
9293 assert_eq!(
9394 pending.iter().map(|edit| edit.id).collect::<Vec<_>>(),
......@@ -118,10 +119,10 @@ fn downloaded_media_survives_reopen_and_recovery_with_the_queue_intact() {
118119 (2, 1024)
119120 );
120121 assert_eq!(
121 server::pages(&cache.snapshot().unwrap()),
122 server::pages(&snapshot(&cache)),
122123 server::pages(&working)
123124 );
124 assert_eq!(cache.remote_snapshot().unwrap(), remote);
125 assert_eq!(remote_snapshot(&cache), remote);
125126 assert_eq!(cache.pending().unwrap(), pending);
126127 assert_eq!(cache.status(1).unwrap(), uncertain);
127128 cache
......@@ -227,7 +228,7 @@ fn download_network_wait_does_not_block_local_edits() {
227228 .unwrap()
228229 });
229230 waiting.recv_timeout(Duration::from_secs(5)).unwrap();
230 let text = first_text(&cache.snapshot().unwrap());
231 let text = first_text(&snapshot(&cache));
231232 let id = model_ops::save(&cache, text, |page| {
232233 model_ops::replace_text(page, text, 0..0, "during download ")
233234 })
......@@ -272,12 +273,18 @@ fn a_native_refresh_removing_the_reference_rejects_an_inflight_download() {
272273 struct Native;
273274 impl notebook::Remote for Native {
274275 fn read(&mut self) -> io::Result<Vec<u8>> {
275 fs::read("../../corpus/native-external-assets/native/synthetic.one")
276 let mut image = fs::read("../../corpus/native-external-assets/native/synthetic.one")?;
277 // The fixture keeps the old header; a native commit writes a new file version.
278 image[212] ^= 1;
279 Ok(image)
276280 }
277281 fn publish(&mut self, _: &onestore::Transaction) -> Result<(), onestore::CommitError> {
278282 panic!("Unexpected publication")
279283 }
280 fn confirm(&mut self, _: &[u8]) -> Result<(), onestore::CommitError> {
284 fn stamp(&mut self) -> io::Result<onestore::Stamp> {
285 onestore::Stamp::of(&self.read()?).map_err(io::Error::other)
286 }
287 fn confirm(&mut self, _: &onestore::Stamp) -> Result<(), onestore::CommitError> {
281288 panic!("Unexpected confirmation")
282289 }
283290 }
......@@ -301,8 +308,8 @@ fn a_native_refresh_removing_the_reference_rejects_an_inflight_download() {
301308 );
302309 assert!(cache.cached_asset(&name, 1024).unwrap().is_none());
303310 assert_eq!(
304 cache.snapshot().unwrap(),
305 fs::read("../../corpus/native-external-assets/native/synthetic.one").unwrap()
311 snapshot(&cache),
312 notebook::Remote::read(&mut Native).unwrap()
306313 );
307314}
308315
crates/notebook/tests/cache.rs+22-22
......@@ -1,3 +1,5 @@
1#[path = "../../onestore/tests/support/ops.rs"]
2mod ops;
13use notebook::{Error, Replica};
24use onestore::{
35 ExGuid, RevisionIndex, Store,
......@@ -17,6 +19,7 @@ use std::{
1719mod model_ops;
1820#[path = "support/server.rs"]
1921mod server;
22use server::snapshot;
2023
2124fn target(source: &[u8]) -> (ExGuid, ExGuid, String) {
2225 let store = Store::parse(source).unwrap();
......@@ -60,7 +63,7 @@ fn cache_reopen_preserves_the_base_and_queued_edits() {
6063 let path = dir.path().join("section.sqlite");
6164 let source = onestore::create_section("section.one", "café 🦀", "Fixture").unwrap();
6265 let replica = Replica::create(&path, &source).unwrap();
63 assert_eq!(replica.snapshot().unwrap(), source);
66 assert_eq!(snapshot(&replica), source);
6467 assert!(replica.pending().unwrap().is_empty());
6568 let (sid, oid, _) = target(&source);
6669 let unchanged = |page: &mut onestore::page::Page| {
......@@ -73,7 +76,7 @@ fn cache_reopen_preserves_the_base_and_queued_edits() {
7376 })
7477 .unwrap()
7578 .unwrap();
76 assert_eq!(target(&replica.snapshot().unwrap()).2, "café 🐈 日本語");
79 assert_eq!(target(&snapshot(&replica)).2, "café 🐈 日本語");
7780 let edits = replica.pending().unwrap();
7881 assert_eq!(edits.len(), 1);
7982 assert_eq!(
......@@ -99,14 +102,14 @@ fn cache_reopen_preserves_the_base_and_queued_edits() {
99102 "closing checkpoints and removes the log"
100103 );
101104 let replica = Replica::open(&path).unwrap();
102 assert_eq!(target(&replica.snapshot().unwrap()).2, "café 🐈 日本語");
105 assert_eq!(target(&snapshot(&replica)).2, "café 🐈 日本語");
103106 assert_eq!(replica.pending().unwrap(), edits);
104107 let second = replica
105108 .apply(model_ops::AUTHOR, typed(sid, oid, "Recovered "))
106109 .unwrap();
107110 assert!(second > first);
108111 assert_eq!(
109 target(&replica.snapshot().unwrap()).2,
112 target(&snapshot(&replica)).2,
110113 "Recovered café 🐈 日本語"
111114 );
112115 assert_eq!(replica.pending().unwrap().len(), 2);
......@@ -160,7 +163,7 @@ fn refused_edits_and_failed_writes_leave_the_queue_as_it_was() {
160163 replica.apply("Author", typed(ExGuid::default(), oid, "Elsewhere ")),
161164 Err(Error::Rejected(_))
162165 ));
163 assert_eq!(replica.snapshot().unwrap(), source);
166 assert_eq!(snapshot(&replica), source);
164167 assert!(replica.pending().unwrap().is_empty());
165168 drop(replica);
166169 let connection = rusqlite::Connection::open(&path).unwrap();
......@@ -168,10 +171,10 @@ fn refused_edits_and_failed_writes_leave_the_queue_as_it_was() {
168171 drop(connection);
169172 let replica = Replica::open(&path).unwrap();
170173 assert!(replica.apply("Author", typed(sid, oid, "lost? ")).is_err());
171 assert_eq!(replica.snapshot().unwrap(), source);
174 assert_eq!(snapshot(&replica), source);
172175 drop(replica);
173176 let replica = Replica::open(&path).unwrap();
174 assert_eq!(replica.snapshot().unwrap(), source);
177 assert_eq!(snapshot(&replica), source);
175178 assert!(replica.pending().unwrap().is_empty());
176179}
177180
......@@ -230,7 +233,7 @@ fn concurrent_recovery_exports_capture_one_complete_acknowledged_queue() {
230233 }
231234 });
232235 assert_eq!(replica.pending().unwrap().len(), 60);
233 let content = target(&replica.snapshot().unwrap()).2;
236 let content = target(&snapshot(&replica)).2;
234237 for writer in 0..3 {
235238 for edit in 0..20 {
236239 assert_eq!(content.matches(&format!("[{writer}:{edit}] ")).count(), 1);
......@@ -255,7 +258,7 @@ fn ownership_and_foreign_file_rejection_preserve_existing_data() {
255258 assert!(
256259 matches!(Replica::create(&path, &source), Err(Error::Io(error)) if error.kind() == ErrorKind::AlreadyExists)
257260 );
258 assert_eq!(replica.snapshot().unwrap(), source);
261 assert_eq!(snapshot(&replica), source);
259262 }
260263 drop(replica);
261264 let current: u32 = rusqlite::Connection::open(&path)
......@@ -264,10 +267,10 @@ fn ownership_and_foreign_file_rejection_preserve_existing_data() {
264267 .unwrap();
265268 for sql in [
266269 "PRAGMA application_id=0".to_owned(),
267 // Schema 14 converts; older caches do not.
270 // Schemas 14 and 15 convert; older caches do not.
268271 format!(
269272 "PRAGMA application_id=1330529615; PRAGMA user_version={}",
270 current - 2
273 current - 3
271274 ),
272275 format!("PRAGMA user_version={}", current + 1),
273276 ] {
......@@ -348,7 +351,7 @@ fn twelve_local_editors_queue_every_edit_once_in_order() {
348351 }
349352 let ids: BTreeSet<_> = outcomes.into_iter().flatten().collect();
350353 assert_eq!(ids.len(), 240);
351 let content = target(&replica.snapshot().unwrap()).2;
354 let content = target(&snapshot(&replica)).2;
352355 assert!(content.ends_with("Shared café 🦀"));
353356 for writer in 0..12 {
354357 for edit in 0..20 {
......@@ -362,7 +365,7 @@ fn twelve_local_editors_queue_every_edit_once_in_order() {
362365 );
363366 drop(replica);
364367 let reopened = Replica::open(&path).unwrap();
365 assert_eq!(target(&reopened.snapshot().unwrap()).2, content);
368 assert_eq!(target(&snapshot(&reopened)).2, content);
366369 assert_eq!(reopened.pending().unwrap(), pending);
367370}
368371
......@@ -410,7 +413,7 @@ fn seeded_unicode_edits_and_restarts_match_an_independent_text_model() {
410413 }
411414 text = expected;
412415 assert_eq!(
413 target(&replica.snapshot().unwrap()).2,
416 target(&snapshot(&replica)).2,
414417 text,
415418 "seed 911, step {step}"
416419 );
......@@ -418,7 +421,7 @@ fn seeded_unicode_edits_and_restarts_match_an_independent_text_model() {
418421 let pending = replica.pending().unwrap();
419422 drop(replica);
420423 replica = Replica::open(&path).unwrap();
421 assert_eq!(target(&replica.snapshot().unwrap()).2, text);
424 assert_eq!(target(&snapshot(&replica)).2, text);
422425 assert_eq!(replica.pending().unwrap(), pending);
423426 }
424427 }
......@@ -508,13 +511,10 @@ fn twelve_local_clients_preserve_inserted_identities_and_dependent_edits() {
508511 .flat_map(|(ids, _)| ids.iter().copied())
509512 .collect();
510513 assert_eq!(ids.len(), 84);
511 let snapshot = cache.snapshot().unwrap();
514 let image = snapshot(&cache);
512515 drop(cache);
513516 let cache = Replica::open(&path).unwrap();
514 assert_eq!(
515 server::pages(&cache.snapshot().unwrap()),
516 server::pages(&snapshot)
517 );
517 assert_eq!(server::pages(&snapshot(&cache)), server::pages(&image));
518518 assert_eq!(
519519 cache
520520 .pending()
......@@ -524,7 +524,7 @@ fn twelve_local_clients_preserve_inserted_identities_and_dependent_edits() {
524524 .collect::<BTreeSet<_>>(),
525525 ids
526526 );
527 let store = Store::parse(&snapshot).unwrap();
527 let store = Store::parse(&image).unwrap();
528528 let index = RevisionIndex::parse(&store).unwrap();
529529 let doc = Document::parse(&index).unwrap();
530530 let s = &doc.spaces[&sid];
......@@ -559,7 +559,7 @@ fn unrecognized_persisted_ops_are_rejected_without_dropping_fields() {
559559 let path = directory.path().join("unknown.sqlite");
560560 let first = onestore::create_section("unknown.one", "Original", "Author").unwrap();
561561 let second = onestore::PageCreation::new(None, Some("Second"), "Author").unwrap();
562 let source = onestore::PreparedEdit::create_page(&first, &second)
562 let source = ops::section_op(&first, SectionOp::Create(second.clone()))
563563 .unwrap()
564564 .as_bytes()
565565 .to_vec();
crates/notebook/tests/changes.rs+2-2
......@@ -24,9 +24,9 @@ fn external_structure_changes_are_reported_by_identity() {
2424 let mut other = Notebook::open(&root, temporary.path().join("other")).unwrap();
2525 assert_eq!(watcher.refresh().unwrap(), []);
2626
27 other.create_section("", "Second", "Author").unwrap();
27 other.create_section("", "Second", &onestore::PageCreation::new(None, Some(""), "Author").unwrap()).unwrap();
2828 other.create_group("", "Group").unwrap();
29 other.create_section("Group", "Inner", "Author").unwrap();
29 other.create_section("Group", "Inner", &onestore::PageCreation::new(None, Some(""), "Author").unwrap()).unwrap();
3030 assert_eq!(
3131 watcher.refresh().unwrap(),
3232 [
crates/notebook/tests/copy.rs+4-13
......@@ -1,7 +1,4 @@
1use notebook::{
2 EditStatus,
3 session::{Event, Section},
4};
1use notebook::session::{Event, Section};
52use onestore::{
63 RevisionIndex, Store,
74 document::Document,
......@@ -197,13 +194,7 @@ fn pages_copy_into_another_section_with_their_content_and_fresh_identities() {
197194 ]
198195 );
199196 let deadline = Instant::now() + Duration::from_secs(120);
200 while section
201 .queue()
202 .unwrap()
203 .iter()
204 .any(|edit| !matches!(edit.status, EditStatus::Published { .. }))
205 || !section.queue().unwrap().is_empty()
206 {
197 while !section.pending().unwrap().is_empty() {
207198 for event in section.events() {
208199 if let Event::Failed(message) = event {
209200 panic!("{message}");
......@@ -212,7 +203,7 @@ fn pages_copy_into_another_section_with_their_content_and_fresh_identities() {
212203 assert!(
213204 Instant::now() < deadline,
214205 "publication did not finish: {:?}",
215 section.queue().unwrap()
206 section.pending().unwrap()
216207 );
217208 std::thread::sleep(Duration::from_millis(50));
218209 }
......@@ -233,7 +224,7 @@ fn pages_copy_into_another_section_with_their_content_and_fresh_identities() {
233224 }
234225 section.delete_pages(&[expected.last().unwrap().0]).unwrap();
235226 let deadline = Instant::now() + Duration::from_secs(60);
236 while !section.queue().unwrap().is_empty() {
227 while !section.pending().unwrap().is_empty() {
237228 assert!(Instant::now() < deadline);
238229 std::thread::sleep(Duration::from_millis(50));
239230 }
crates/notebook/tests/migrate.rs+112-37
......@@ -1,8 +1,12 @@
11//! Converting schema-14 caches: whole-page edits become ops, verified page by page against
2//! the old working image, with a recovery archive first and nothing changed on a mismatch.
2//! the old working image, with a recovery archive first and nothing changed on a mismatch;
3//! and schema-15 caches, whose batches lose the review conflict.
34
4use notebook::{ConflictKind, EditStatus, Replica, Resolution};
5use onestore::{ExGuid, PageCreation, PageEdit, PreparedEdit, page::Page};
5#[path = "../../onestore/tests/support/ops.rs"]
6mod ops;
7
8use notebook::{EditStatus, Replica};
9use onestore::{ExGuid, PageCreation, PageEdit, op::SectionOp, page::Page};
610use rusqlite::{Connection, params};
711use serde_json::json;
812use std::path::Path;
......@@ -48,17 +52,17 @@ fn v14(path: &Path, base: &[u8], queue: &[Queued], working: Option<&[u8]>) -> Ve
4852 let (space, operation, next) = match entry {
4953 Queued::Page(space, after) => {
5054 let before = model_ops::page_of(&image, *space);
51 let next = PreparedEdit::page(&image, *space, after, "Author").unwrap();
55 let next = ops::saved(&image, *space, after).unwrap();
5256 (
5357 *space,
5458 json!({"Page": {"before": before, "after": after, "author": "Author"}}),
55 next.as_bytes().to_vec(),
59 next.as_slice().to_vec(),
5660 )
5761 }
5862 Queued::Create(creation) => (
5963 creation.space(),
6064 json!({"CreatePage": creation}),
61 PreparedEdit::create_page(&image, creation)
65 ops::section_op(&image, SectionOp::Create(creation.clone()))
6266 .unwrap()
6367 .as_bytes()
6468 .to_vec(),
......@@ -71,7 +75,7 @@ fn v14(path: &Path, base: &[u8], queue: &[Queued], working: Option<&[u8]>) -> Ve
7175 (
7276 root(&image),
7377 json!({"Pages": {"edits": batch, "observed": observed}}),
74 PreparedEdit::pages(&image, batch)
78 ops::section_op(&image, SectionOp::Pages(batch.to_vec()))
7579 .unwrap()
7680 .as_bytes()
7781 .to_vec(),
......@@ -80,7 +84,7 @@ fn v14(path: &Path, base: &[u8], queue: &[Queued], working: Option<&[u8]>) -> Ve
8084 Queued::Delete(spaces) => (
8185 root(&image),
8286 json!({"DeletePages": spaces}),
83 PreparedEdit::delete_pages_permanently(&image, spaces)
87 ops::section_op(&image, SectionOp::Delete(spaces.to_vec()))
8488 .unwrap()
8589 .as_bytes()
8690 .to_vec(),
......@@ -187,9 +191,9 @@ fn a_schema_14_queue_converts_to_ops_that_reach_its_working_pages_and_publish()
187191 Queued::Page(
188192 a,
189193 appended(
190 PreparedEdit::page(SOURCE, a, &first, "Author")
194 ops::saved(SOURCE, a, &first)
191195 .unwrap()
192 .as_bytes(),
196 .as_slice(),
193197 a,
194198 a_text,
195199 " two",
......@@ -216,12 +220,12 @@ fn a_schema_14_queue_converts_to_ops_that_reach_its_working_pages_and_publish()
216220 }
217221 })
218222 );
219 let local = pages(&cache.snapshot().unwrap());
223 let local = pages(&snapshot(&cache));
220224 assert_eq!(local, pages(&working));
221225 drop(cache);
222226 // A converted cache opens as it is.
223227 let cache = Replica::open(&path).unwrap();
224 assert_eq!(pages(&cache.snapshot().unwrap()), local);
228 assert_eq!(pages(&snapshot(&cache)), local);
225229 let next = cache
226230 .apply(
227231 "Author",
......@@ -245,7 +249,7 @@ fn a_schema_14_queue_converts_to_ops_that_reach_its_working_pages_and_publish()
245249 Some((id, EditStatus::Published { .. })) if id == next
246250 ));
247251 let published = pages(&server.durable);
248 assert_eq!(published, pages(&cache.snapshot().unwrap()));
252 assert_eq!(published, pages(&snapshot(&cache)));
249253 assert!(
250254 published
251255 .iter()
......@@ -255,7 +259,7 @@ fn a_schema_14_queue_converts_to_ops_that_reach_its_working_pages_and_publish()
255259}
256260
257261#[test]
258fn an_uncertain_attempt_and_a_conflict_keep_their_states() {
262fn an_uncertain_attempt_keeps_its_state_and_a_conflict_becomes_a_conflict_page() {
259263 let body = body_pages(SOURCE);
260264 let (a, a_text) = body[0];
261265 let directory = tempfile::tempdir().unwrap();
......@@ -263,9 +267,9 @@ fn an_uncertain_attempt_and_a_conflict_keep_their_states() {
263267 // The oldest edit was attempted: its schema-14 evidence names the revision it published.
264268 let path = directory.path().join("attempted.sqlite");
265269 let after = appended(SOURCE, a, a_text, " attempted");
266 let published = PreparedEdit::page(SOURCE, a, &after, "Author").unwrap();
270 let published = ops::saved(SOURCE, a, &after).unwrap();
267271 let revision =
268 onestore::RevisionIndex::parse(&onestore::Store::parse(published.as_bytes()).unwrap())
272 onestore::RevisionIndex::parse(&onestore::Store::parse(published.as_slice()).unwrap())
269273 .unwrap()
270274 .active(a)
271275 .unwrap();
......@@ -286,7 +290,7 @@ fn an_uncertain_attempt_and_a_conflict_keep_their_states() {
286290 Some((2, awaiting))
287291 );
288292 assert_eq!(unchanged.publications, 0);
289 let mut landed = Server::new(published.as_bytes());
293 let mut landed = Server::new(published.as_slice());
290294 assert_eq!(
291295 cache.sync_once(&mut landed).unwrap().edit,
292296 Some((2, EditStatus::Published { revision }))
......@@ -294,17 +298,17 @@ fn an_uncertain_attempt_and_a_conflict_keep_their_states() {
294298 assert_eq!(landed.publications, 0);
295299 drop(cache);
296300
297 // The oldest edit was in conflict: the conversion lowered it onto the remote page the
298 // conflict was recorded against, and keeping mine publishes it.
301 // The oldest edit was in conflict: the conversion keeps the remote page the conflict was
302 // recorded against and queues the local version as its conflict page.
299303 let path = directory.path().join("conflicted.sqlite");
300 let remote = PreparedEdit::page(SOURCE, a, &appended(SOURCE, a, a_text, " remote"), "Native")
304 let remote = ops::saved(SOURCE, a, &appended(SOURCE, a, a_text, " remote"))
301305 .unwrap()
302 .as_bytes()
306 .as_slice()
303307 .to_vec();
304308 let local = appended(SOURCE, a, a_text, " local");
305 let working = PreparedEdit::page(SOURCE, a, &local, "Author")
309 let working = ops::saved(SOURCE, a, &local)
306310 .unwrap()
307 .as_bytes()
311 .as_slice()
308312 .to_vec();
309313 {
310314 // Schema 14 stored the remote as the base and kept the local working image.
......@@ -324,27 +328,26 @@ fn an_uncertain_attempt_and_a_conflict_keep_their_states() {
324328 .unwrap();
325329 }
326330 let cache = Replica::open(&path).unwrap();
331 assert_eq!(cache.status(2).unwrap(), Some(EditStatus::Pending));
327332 assert_eq!(
328 cache.status(2).unwrap(),
329 Some(EditStatus::Conflict(ConflictKind::ContentChanged))
330 );
331 assert_eq!(
332 model_ops::page_of(&cache.snapshot().unwrap(), a),
333 model_ops::page_of(&working, a)
333 model_ops::page_of(&snapshot(&cache), a),
334 model_ops::page_of(&remote, a)
334335 );
335336 let mut server = Server::new(&remote);
336 assert_eq!(
337 cache.sync_once(&mut server).unwrap().edit,
338 Some((2, EditStatus::Conflict(ConflictKind::ContentChanged)))
339 );
340 cache.resolve(2, Resolution::Mine).unwrap();
341337 assert!(matches!(
342338 cache.sync_once(&mut server).unwrap().edit,
343339 Some((2, EditStatus::Published { .. }))
344340 ));
345341 assert_eq!(
346342 model_ops::page_of(&server.durable, a),
347 model_ops::page_of(&working, a)
343 model_ops::page_of(&remote, a)
344 );
345 assert_eq!(
346 conflicts(&server.durable),
347 [(
348 a,
349 vec![("Author".to_owned(), page_texts(&model_ops::page_of(&working, a)))]
350 )]
348351 );
349352}
350353
......@@ -355,9 +358,9 @@ fn a_page_the_conversion_cannot_reproduce_leaves_the_cache_untouched() {
355358 let directory = tempfile::tempdir().unwrap();
356359 let path = directory.path().join("cache.sqlite");
357360 // The working image holds a page the queued edit does not reach.
358 let stray = PreparedEdit::page(SOURCE, a, &appended(SOURCE, a, a_text, " stray"), "Author")
361 let stray = ops::saved(SOURCE, a, &appended(SOURCE, a, a_text, " stray"))
359362 .unwrap()
360 .as_bytes()
363 .as_slice()
361364 .to_vec();
362365 v14(
363366 &path,
......@@ -382,3 +385,75 @@ fn a_page_the_conversion_cannot_reproduce_leaves_the_cache_untouched() {
382385 assert!(Replica::open(&path).is_err());
383386 assert_eq!(std::fs::read(&path).unwrap(), before);
384387}
388
389/// A schema-15 cache upgrades in place: its queue, receipts and recovery archives stay, and
390/// a conflict it recorded is queued work the next sync keeps as a conflict page.
391#[test]
392fn a_schema_15_cache_upgrades_in_place_and_its_conflict_becomes_a_conflict_page() {
393 let (space, text) = body_pages(SOURCE)[0];
394 let directory = tempfile::tempdir().unwrap();
395 let path = directory.path().join("cache.sqlite");
396 let cache = Replica::create(&path, SOURCE).unwrap();
397 let id = model_ops::save(&cache, text, |page| {
398 model_ops::replace_text(page, text, 0..0, "Local ")
399 })
400 .unwrap()
401 .unwrap();
402 let pending = cache.pending().unwrap();
403 let archive = directory.path().join("before.sqlite");
404 drop(cache);
405 {
406 // Schema 15 recorded the review conflict and its page on the batch.
407 let connection = Connection::open(&path).unwrap();
408 connection
409 .execute_batch(&format!(
410 "PRAGMA foreign_keys=OFF;
411 BEGIN;
412 CREATE TABLE old (
413 id INTEGER PRIMARY KEY AUTOINCREMENT CHECK(id>0),
414 sealed TEXT,
415 revisions TEXT,
416 attempted INTEGER NOT NULL DEFAULT 0 CHECK(attempted IN (0,1)),
417 conflict INTEGER CHECK(conflict BETWEEN 0 AND 3),
418 space TEXT,
419 CHECK((sealed IS NULL) = (revisions IS NULL)),
420 CHECK(attempted=0 OR sealed IS NOT NULL),
421 CHECK((conflict IS NULL) = (space IS NULL))
422 ) STRICT;
423 INSERT INTO old SELECT id, sealed, revisions, attempted, 3, '{space}' FROM batches;
424 DROP TABLE batches;
425 ALTER TABLE old RENAME TO batches;
426 PRAGMA user_version=15;
427 COMMIT;"
428 ))
429 .unwrap();
430 }
431 let cache = Replica::open(&path).unwrap();
432 assert_eq!(cache.pending().unwrap(), pending);
433 assert_eq!(cache.status(id).unwrap(), Some(EditStatus::Pending));
434 cache.export_recovery(&archive).unwrap();
435 drop(cache);
436 let connection = Connection::open(&path).unwrap();
437 let version: u32 = connection
438 .pragma_query_value(None, "user_version", |row| row.get(0))
439 .unwrap();
440 let columns: Vec<String> = connection
441 .prepare("SELECT name FROM pragma_table_info('batches')")
442 .unwrap()
443 .query_map([], |row| row.get(0))
444 .unwrap()
445 .collect::<Result<_, _>>()
446 .unwrap();
447 assert_eq!(version, 16);
448 assert_eq!(columns, ["id", "sealed", "revisions", "attempted"]);
449 drop(connection);
450 assert_eq!(notebook::Recovery::open(&archive).unwrap().pending().unwrap(), pending);
451 let cache = Replica::open(&path).unwrap();
452 let remote = typed(SOURCE, space, text, 0..0, "Remote ");
453 let mut server = Server::new(&remote);
454 assert!(matches!(
455 cache.sync_once(&mut server).unwrap().edit,
456 Some((_, EditStatus::Published { .. }))
457 ));
458 assert!(conflicted(&server.durable, space));
459}
crates/notebook/tests/protected.rs+15-6
......@@ -28,8 +28,8 @@ fn a_locked_section_unlocks_for_reading() {
2828 ));
2929}
3030
31/// An unlocked page is edited and saved under the section's key, then reads back with the
32/// same password. `ONESTORE_PROTECTED_EXPORT` names a directory receiving the notebook for
31/// An unlocked page is edited as ops stored under the section's key, then reads back with
32/// the same password. `ONESTORE_PROTECTED_EXPORT` names a directory receiving the notebook for
3333/// a cold reopen in OneNote.
3434#[test]
3535fn an_unlocked_page_is_saved_under_the_section_key() {
......@@ -59,6 +59,7 @@ fn an_unlocked_page_is_saved_under_the_section_key() {
5959 .clone();
6060 let mut unlocked = notebook.unlock(&section, password).unwrap();
6161 let (space, mut page) = unlocked.pages[0].clone();
62 let before = page.clone();
6263 let paragraphs = page
6364 .objects
6465 .iter_mut()
......@@ -95,20 +96,28 @@ fn an_unlocked_page_is_saved_under_the_section_key() {
9596 format,
9697 ))
9798 .unwrap();
99 let edit = onestore::op::Edit {
100 at: 134_000_000_000_000_000,
101 ops: onestore::op::lower_page(&before, &page)
102 .unwrap()
103 .into_iter()
104 .map(|op| onestore::op::Op::Page { space, op })
105 .collect(),
106 };
98107 assert!(matches!(
99 notebook.save_unlocked(&section, "wrong", &mut unlocked, space, &page, "Rust"),
108 notebook.apply_unlocked(&section, "wrong", &mut unlocked, "Rust", &edit),
100109 Err(notebook::Error::Protected(
101110 onestore::protected::Error::PasswordMismatch
102111 ))
103112 ));
104113 let mut stale = notebook.unlock(&section, password).unwrap();
105114 notebook
106 .save_unlocked(&section, password, &mut unlocked, space, &page, "Rust")
115 .apply_unlocked(&section, password, &mut unlocked, "Rust", &edit)
107116 .unwrap();
108 // A view read before that save no longer matches the file.
117 // A view read before that edit no longer matches the file.
109118 assert!(
110119 notebook
111 .save_unlocked(&section, password, &mut stale, space, &page, "Rust")
120 .apply_unlocked(&section, password, &mut stale, "Rust", &edit)
112121 .is_err()
113122 );
114123 let (_, stored) = notebook.unlock(&section, password).unwrap().pages.remove(0);
crates/notebook/tests/session.rs+98-125
......@@ -1,8 +1,14 @@
1#[path = "../../onestore/tests/support/ops.rs"]
2mod ops;
13use notebook::{
2 ConflictKind, EditStatus, Recovery, Resolution,
3 session::{Event, Notebook, QueuedEdit, Save, Section},
4 EditStatus, Recovery, Resolution,
5 session::{Event, Notebook, Section},
6};
7use onestore::{
8 ExGuid,
9 op::{Edit, Op, PageOp},
10 page::Page,
411};
5use onestore::{ExGuid, PreparedEdit, page::Page};
612use std::{
713 path::Path,
814 sync::{
......@@ -35,6 +41,21 @@ fn edited(page: &Page, text: &str) -> Page {
3541 after
3642}
3743
44/// Replaces `range` of the page's first body text as the editor does: the edit's id once
45/// it is durable.
46fn typed(section: &Section, space: ExGuid, page: &Page, range: std::ops::Range<u32>, text: &str) -> u64 {
47 let op = PageOp::Text {
48 text: first_text(page),
49 range,
50 with: text.into(),
51 };
52 let edit = Edit {
53 at: model_ops::now(),
54 ops: vec![Op::Page { space, op }],
55 };
56 section.replica().apply("Editor", edit).unwrap()
57}
58
3859fn open(file: &Path, cache: &Path) -> (Section, Arc<AtomicUsize>) {
3960 let notified = Arc::new(AtomicUsize::new(0));
4061 let counter = Arc::clone(&notified);
......@@ -105,14 +126,8 @@ fn a_section_opens_through_its_replica_and_a_save_reaches_the_file_and_survives_
105126 assert_eq!(pages.len(), 1);
106127 let space = pages[0].0;
107128 let before = section.page(space).unwrap();
108 assert_eq!(
109 section.save(space, &before, &before, "Editor").unwrap(),
110 Save::Unchanged
111 );
112129 let after = edited(&before, "Saved ");
113 let Save::Queued(id) = section.save(space, &before, &after, "Editor").unwrap() else {
114 panic!()
115 };
130 let id = typed(&section, space, &before, 0..0, "Saved ");
116131 assert_same(section.page(space).unwrap(), &after);
117132 published(&section, id);
118133 // The worker reports the step after writing its receipt.
......@@ -167,9 +182,7 @@ fn saves_wait_for_an_unreachable_file_and_publish_after_relaunch() {
167182 let permissions = std::fs::metadata(&file).unwrap().permissions();
168183 std::fs::set_permissions(&file, std::fs::Permissions::from_mode(0o444)).unwrap();
169184 let after = edited(&before, "Offline ");
170 let Save::Queued(id) = section.save(space, &before, &after, "Editor").unwrap() else {
171 panic!()
172 };
185 let id = typed(&section, space, &before, 0..0, "Offline ");
173186 wait(&section, |event| matches!(event, Event::Unreachable(_)));
174187 section.close().unwrap();
175188 assert_eq!(
......@@ -198,7 +211,7 @@ fn saves_wait_for_an_unreachable_file_and_publish_after_relaunch() {
198211}
199212
200213#[test]
201fn an_external_change_refreshes_the_page_and_stales_a_save_from_the_old_model() {
214fn an_external_change_reloads_the_page_and_later_edits_apply_to_it() {
202215 let directory = tempfile::tempdir().unwrap();
203216 let file = directory.path().join("notes.one");
204217 let cache = directory.path().join("cache");
......@@ -214,7 +227,7 @@ fn an_external_change_refreshes_the_page_and_stales_a_save_from_the_old_model()
214227 let deadline = Instant::now() + Duration::from_secs(20);
215228 loop {
216229 let bytes = onestore::read_file(&file).unwrap();
217 match PreparedEdit::page(&bytes, space, &native, "Native")
230 match ops::save(&bytes, space, &native)
218231 .unwrap()
219232 .commit_file(&file)
220233 {
......@@ -227,29 +240,18 @@ fn an_external_change_refreshes_the_page_and_stales_a_save_from_the_old_model()
227240 }
228241 }
229242 section.wake();
230 let deadline = Instant::now() + Duration::from_secs(20);
231 while section.page(space).unwrap().title == before.title {
232 assert!(Instant::now() < deadline, "the refresh did not arrive");
233 wait(&section, |event| matches!(event, Event::Refreshed));
234 }
243 wait(&section, |event| matches!(event, Event::Changed(spaces) if spaces.contains(&space)));
235244 assert_same(section.page(space).unwrap(), &native);
236 let native = section.page(space).unwrap();
237 let stale = edited(&before, "Local ");
238 assert_eq!(
239 section.save(space, &before, &stale, "Editor").unwrap(),
240 Save::Stale
241 );
245 // An edit made on the page shown before the change applies to the page as it is now.
246 let id = typed(&section, space, &before, 0..0, "Local ");
242247 let after = edited(&native, "Local ");
243 let Save::Queued(id) = section.save(space, &native, &after, "Editor").unwrap() else {
244 panic!()
245 };
246248 published(&section, id);
247249 assert_same(stored_page(&file, space), &after);
248250 section.close().unwrap();
249251}
250252
251253#[test]
252fn queued_saves_continue_each_other_detect_stale_pages_and_finish_before_close() {
254fn edits_apply_in_order_without_waiting_and_finish_before_close() {
253255 let directory = tempfile::tempdir().unwrap();
254256 let file = directory.path().join("notes.one");
255257 let cache = directory.path().join("cache");
......@@ -260,36 +262,44 @@ fn queued_saves_continue_each_other_detect_stale_pages_and_finish_before_close()
260262 .unwrap();
261263 let (section, _) = open(&file, &cache);
262264 let space = section.pages().unwrap()[0].0;
263 let mut before = section.page(space).unwrap();
265 let original = section.page(space).unwrap();
266 let text = first_text(&original);
267 let apply = |word: &str| {
268 let op = PageOp::Text {
269 text,
270 range: 0..0,
271 with: word.into(),
272 };
273 let edit = Edit {
274 at: model_ops::now(),
275 ops: vec![Op::Page { space, op }],
276 };
277 section.apply("Editor", edit).unwrap();
278 };
279 let mut expected = original.clone();
264280 for word in ["one ", "two ", "three ", "four ", "five "] {
265 let after = edited(&before, word);
266 section
267 .queue_save(space, before, after.clone(), "Editor")
268 .unwrap();
269 before = after;
281 apply(word);
282 expected = edited(&expected, word);
270283 }
271284 let deadline = Instant::now() + Duration::from_secs(20);
272285 loop {
273286 let mut stored = section.page(space).unwrap();
274 stored.title = before.title.clone();
275 if stored == before {
287 stored.title = expected.title.clone();
288 if stored == expected {
276289 break;
277290 }
278 assert!(Instant::now() < deadline, "the saves did not arrive");
279 for (_, save) in section.saved() {
280 assert!(matches!(save, Ok(Save::Queued(_))), "{save:?}");
281 }
291 assert!(Instant::now() < deadline, "the edits did not arrive");
282292 std::thread::sleep(Duration::from_millis(20));
283293 }
284294 while !section.pending().unwrap().is_empty() {
285 assert!(Instant::now() < deadline, "the saves were not published");
295 assert!(Instant::now() < deadline, "the edits were not published");
286296 std::thread::sleep(Duration::from_millis(20));
287297 }
288298 let native = edited(&section.page(space).unwrap(), "Native ");
289299 let deadline = Instant::now() + Duration::from_secs(20);
290300 loop {
291301 let bytes = onestore::read_file(&file).unwrap();
292 match PreparedEdit::page(&bytes, space, &native, "Native")
302 match ops::save(&bytes, space, &native)
293303 .unwrap()
294304 .commit_file(&file)
295305 {
......@@ -302,29 +312,11 @@ fn queued_saves_continue_each_other_detect_stale_pages_and_finish_before_close()
302312 }
303313 }
304314 section.wake();
305 while !section.page(space).unwrap().objects.eq(&native.objects) {
306 assert!(Instant::now() < deadline, "the refresh did not arrive");
307 wait(&section, |event| matches!(event, Event::Refreshed));
308 }
309 let stale = edited(&before, "Local ");
310 section.queue_save(space, before, stale, "Editor").unwrap();
311 // Outcomes of the earlier saves may still be arriving.
312 while !section
313 .saved()
314 .into_iter()
315 .any(|saved| saved == (space, Ok(Save::Stale)))
316 {
317 assert!(Instant::now() < deadline, "the stale save was not reported");
318 std::thread::sleep(Duration::from_millis(20));
319 }
320 let native = section.page(space).unwrap();
321 let after = edited(&native, "Local ");
322 section
323 .queue_save(space, native, after.clone(), "Editor")
324 .unwrap();
315 wait(&section, |event| matches!(event, Event::Changed(spaces) if spaces.contains(&space)));
316 apply("Local ");
325317 section.close().unwrap();
326318 let (section, _) = open(&file, &cache);
327 assert_same(section.page(space).unwrap(), &after);
319 assert_same(section.page(space).unwrap(), &edited(&native, "Local "));
328320 section.close().unwrap();
329321}
330322
......@@ -426,9 +418,7 @@ fn an_absent_remote_does_not_prevent_local_relaunch_or_further_saves() {
426418 let space = section.pages().unwrap()[0].0;
427419 let before = section.page(space).unwrap();
428420 let after = edited(&before, "First ");
429 let Save::Queued(id) = section.save(space, &before, &after, "Editor").unwrap() else {
430 panic!()
431 };
421 let id = typed(&section, space, &before, 0..0, "First ");
432422 wait(
433423 &section,
434424 |event| matches!(event, Event::Unreachable(error) if error.kind() == std::io::ErrorKind::NotFound),
......@@ -442,9 +432,7 @@ fn an_absent_remote_does_not_prevent_local_relaunch_or_further_saves() {
442432 assert_eq!(section.status(id).unwrap(), Some(EditStatus::Pending));
443433 let before = section.page(space).unwrap();
444434 let after = edited(&before, "Second ");
445 let Save::Queued(next) = section.save(space, &before, &after, "Editor").unwrap() else {
446 panic!()
447 };
435 let next = typed(&section, space, &before, 0..0, "Second ");
448436 assert!(!file.exists());
449437 let restored = directory.path().join("restored.one");
450438 std::fs::write(&restored, &source).unwrap();
......@@ -470,9 +458,7 @@ fn resuming_against_another_document_preserves_both_remote_and_pending_edit() {
470458 let space = section.pages().unwrap()[0].0;
471459 let before = section.page(space).unwrap();
472460 let after = edited(&before, "Local ");
473 let Save::Queued(id) = section.save(space, &before, &after, "Editor").unwrap() else {
474 panic!()
475 };
461 let id = typed(&section, space, &before, 0..0, "Local ");
476462 wait(&section, |event| matches!(event, Event::Unreachable(_)));
477463 section.close().unwrap();
478464
......@@ -499,10 +485,7 @@ fn offline_save_process() {
499485 let section = Section::resume(root.join("absent.one"), replica, || {}).unwrap();
500486 let space = section.pages().unwrap()[0].0;
501487 let before = section.page(space).unwrap();
502 let after = edited(&before, "Durable ");
503 let Save::Queued(id) = section.save(space, &before, &after, "Editor").unwrap() else {
504 panic!()
505 };
488 let id = typed(&section, space, &before, 0..0, "Durable ");
506489 std::fs::write(root.join("acknowledged"), id.to_string()).unwrap();
507490 // Terminate without running Session or SQLite destructors after acknowledgement.
508491 std::process::exit(0);
......@@ -562,9 +545,7 @@ fn smb_connection_failure_keeps_the_session_locally_editable_and_retries() {
562545 let space = section.pages().unwrap()[0].0;
563546 let before = section.page(space).unwrap();
564547 let after = edited(&before, "Offline SMB ");
565 let Save::Queued(id) = section.save(space, &before, &after, "Editor").unwrap() else {
566 panic!()
567 };
548 let id = typed(&section, space, &before, 0..0, "Offline SMB ");
568549 section.wake();
569550 wait(
570551 &section,
......@@ -611,9 +592,7 @@ fn live_smb_session_save_publishes_and_reopens() {
611592 let space = section.pages().unwrap()[0].0;
612593 let before = section.page(space).unwrap();
613594 let after = edited(&before, "Session SMB ");
614 let Save::Queued(id) = section.save(space, &before, &after, "Editor").unwrap() else {
615 panic!()
616 };
595 let id = typed(&section, space, &before, 0..0, "Session SMB ");
617596 published(&section, id);
618597 let remote = client.read("session.one", 1024 * 1024).unwrap();
619598 assert_same(model_ops::page_of(&remote, space), &after);
......@@ -661,9 +640,7 @@ fn dropping_during_connection_keeps_cache_owned_until_the_worker_finishes() {
661640 let space = section.pages().unwrap()[0].0;
662641 let before = section.page(space).unwrap();
663642 let after = edited(&before, "Saved during connection ");
664 let Save::Queued(id) = section.save(space, &before, &after, "Editor").unwrap() else {
665 panic!()
666 };
643 let id = typed(&section, space, &before, 0..0, "Saved during connection ");
667644 let (dropped, finished) = mpsc::channel();
668645 let owner = std::thread::spawn(move || {
669646 drop(section);
......@@ -692,14 +669,14 @@ fn dropping_during_connection_keeps_cache_owned_until_the_worker_finishes() {
692669 };
693670 assert_eq!(replica.status(id).unwrap(), Some(EditStatus::Pending));
694671 assert_same(
695 model_ops::page_of(&replica.snapshot().unwrap(), space),
672 model_ops::page_of(&server::snapshot(&replica), space),
696673 &after,
697674 );
698675 assert!(connecting.try_recv().is_err());
699676}
700677
701678#[test]
702fn a_conflicting_save_is_reviewed_against_the_remote_page_and_archived_for_recovery() {
679fn a_conflicting_save_keeps_the_native_page_and_a_conflict_page_the_session_deletes() {
703680 let directory = tempfile::tempdir().unwrap();
704681 let file = directory.path().join("notes.one");
705682 let cache = directory.path().join("cache");
......@@ -717,9 +694,7 @@ fn a_conflicting_save_is_reviewed_against_the_remote_page_and_archived_for_recov
717694 std::fs::set_permissions(&file, std::fs::Permissions::from_mode(0o444)).unwrap();
718695 let mut local = before.clone();
719696 model_ops::replace_text(&mut local, text, 0..8, "Local");
720 let Save::Queued(id) = section.save(space, &before, &local, "Editor").unwrap() else {
721 panic!()
722 };
697 let id = typed(&section, space, &before, 0..8, "Local");
723698 wait(&section, |event| matches!(event, Event::Unreachable(_)));
724699 std::fs::set_permissions(&file, permissions).unwrap();
725700 let mut native = before.clone();
......@@ -727,7 +702,7 @@ fn a_conflicting_save_is_reviewed_against_the_remote_page_and_archived_for_recov
727702 let deadline = Instant::now() + Duration::from_secs(20);
728703 loop {
729704 let bytes = onestore::read_file(&file).unwrap();
730 match PreparedEdit::page(&bytes, space, &native, "Native")
705 match ops::save(&bytes, space, &native)
731706 .unwrap()
732707 .commit_file(&file)
733708 {
......@@ -740,40 +715,38 @@ fn a_conflicting_save_is_reviewed_against_the_remote_page_and_archived_for_recov
740715 }
741716 }
742717 section.wake();
743 wait(
744 &section,
745 |event| matches!(event, Event::Attempt { id: n, status: EditStatus::Conflict(ConflictKind::ContentChanged) } if *n == id),
746 );
747 let conflicts = section.conflicts().unwrap();
718 wait(&section, |event| {
719 matches!(event, Event::Attempt { id: n, status: EditStatus::Published { .. } } if *n > id)
720 });
721 assert!(matches!(
722 section.status(id).unwrap(),
723 Some(EditStatus::Published { .. })
724 ));
725 // As OneNote does, the native version stays the page and the local one is kept beside it.
726 assert_same(section.page(space).unwrap(), &native);
727 assert_same(stored_page(&file, space), &native);
728 let listed = section.conflicts().unwrap();
729 assert_eq!(listed.len(), 1);
730 assert_eq!(listed[0].0, space);
731 let conflict = &listed[0].1[0];
732 assert_eq!(conflict.user, "Editor");
733 let kept = section.page(conflict.space).unwrap();
748734 assert_eq!(
749 conflicts,
750 [(
751 QueuedEdit {
752 id,
753 space,
754 status: EditStatus::Conflict(ConflictKind::ContentChanged)
755 },
756 ConflictKind::ContentChanged
757 )]
735 model_ops::paragraph_with(&kept, first_text(&kept))
736 .and_then(|paragraph| paragraph.text())
737 .map(|text| text.text.text().to_owned()),
738 Some("Local".to_owned())
758739 );
759 assert_same(section.page(space).unwrap(), &local);
760 let remote = section.remote_page(space).unwrap();
761 assert_same(remote.clone(), &native);
762 let archive = directory.path().join("review.sqlite");
763 section.export_recovery(&archive).unwrap();
764 let recovery = Recovery::open(&archive).unwrap();
765 assert_eq!(recovery.pending().unwrap().len(), 1);
766740 assert_eq!(
767 recovery.status(id).unwrap(),
768 Some(EditStatus::Conflict(ConflictKind::ContentChanged))
741 server::conflicts(&onestore::read_file(&file).unwrap())[0].1,
742 [("Editor".to_owned(), vec!["Local".to_owned()])]
769743 );
770 let mut reviewed = remote.clone();
771 model_ops::replace_text(&mut reviewed, text, 0..6, "Native and local");
772 section.review(id, &reviewed).unwrap();
773 assert_eq!(section.status(id).unwrap(), Some(EditStatus::Pending));
774 published(&section, id);
775 assert_same(stored_page(&file, space), &reviewed);
776 assert!(section.queue().unwrap().is_empty());
744 // Merged by hand, the conflict page is deleted.
745 let deleted = section.delete_pages(&[conflict.space]).unwrap();
746 published(&section, deleted);
747 assert!(section.conflicts().unwrap().is_empty());
748 assert!(server::conflicts(&onestore::read_file(&file).unwrap()).is_empty());
749 assert!(section.pending().unwrap().is_empty());
777750 section.close().unwrap();
778751}
779752
......@@ -819,7 +792,7 @@ fn an_uncertain_attempt_is_released_after_restart_by_review() {
819792 );
820793 assert_eq!(
821794 section
822 .queue()
795 .pending()
823796 .unwrap()
824797 .iter()
825798 .map(|edit| edit.id)
......@@ -850,7 +823,7 @@ fn an_uncertain_attempt_is_released_after_restart_by_review() {
850823 recovery.status(id).unwrap(),
851824 Some(EditStatus::AwaitingConfirmation { .. })
852825 ));
853 assert!(section.queue().unwrap().is_empty());
826 assert!(section.pending().unwrap().is_empty());
854827 assert!(
855828 section
856829 .release(
crates/notebook/tests/smb_queue.rs+10-6
......@@ -17,6 +17,10 @@ use std::{
1717 time::{Duration, Instant, SystemTime, UNIX_EPOCH},
1818};
1919
20#[path = "support/server.rs"]
21mod server;
22use server::snapshot;
23
2024fn client() -> Client {
2125 Client::connect(
2226 &std::env::var("ONESTORE_SMB_LAB").unwrap(),
......@@ -49,15 +53,15 @@ impl Remote for Counted {
4953 self.reads += 1;
5054 self.remote.read()
5155 }
52 fn stamp(&mut self) -> io::Result<Option<Stamp>> {
56 fn stamp(&mut self) -> io::Result<Stamp> {
5357 self.stamps += 1;
5458 self.remote.stamp()
5559 }
5660 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
5761 self.remote.publish(transaction)
5862 }
59 fn confirm(&mut self, snapshot: &[u8]) -> Result<(), CommitError> {
60 self.remote.confirm(snapshot)
63 fn confirm(&mut self, base: &Stamp) -> Result<(), CommitError> {
64 self.remote.confirm(base)
6165 }
6266}
6367
......@@ -140,7 +144,7 @@ fn live_keystrokes_publish_reading_only_the_header_and_a_native_change_merges()
140144 assert_eq!(remote.stamps, 20);
141145 assert_eq!(
142146 lab.read(&path, 1 << 24).unwrap(),
143 replica.snapshot().unwrap()
147 snapshot(&replica)
144148 );
145149 for _ in 0..5 {
146150 assert_eq!(replica.sync_once(&mut remote).unwrap().edit, None);
......@@ -173,7 +177,7 @@ fn live_keystrokes_publish_reading_only_the_header_and_a_native_change_merges()
173177 assert!(merged.ends_with(" local"), "{merged}");
174178 assert_eq!(
175179 lab.read(&path, 1 << 24).unwrap(),
176 replica.snapshot().unwrap()
180 snapshot(&replica)
177181 );
178182 lab.delete(&path).unwrap();
179183}
......@@ -244,6 +248,6 @@ fn live_session_applies_edits_through_the_embedded_client_and_reopens() {
244248 section.close().unwrap();
245249 let replica = Replica::open(&cache).unwrap();
246250 assert_eq!(text_of(&replica, space, text), "0123456789Session");
247 assert_eq!(replica.snapshot().unwrap(), published);
251 assert_eq!(snapshot(&replica), published);
248252 lab.delete(&path).unwrap();
249253}
crates/notebook/tests/structure.rs+2-2
......@@ -53,13 +53,13 @@ fn sections_and_groups_are_created_renamed_coloured_ordered_and_deleted() {
5353 onestore::place_file(root.join("First.one"), toc_id, "First.one").unwrap();
5454 let mut notebook = Notebook::open(&root, temporary.path().join("cache")).unwrap();
5555 assert_eq!(
56 notebook.create_section("", "Second", "Author").unwrap(),
56 notebook.create_section("", "Second", &onestore::PageCreation::new(None, Some(""), "Author").unwrap()).unwrap(),
5757 "Second.one"
5858 );
5959 assert_eq!(notebook.create_group("", "Archive").unwrap(), "Archive");
6060 assert_eq!(
6161 notebook
62 .create_section("Archive", "Inner", "Author")
62 .create_section("Archive", "Inner", &onestore::PageCreation::new(None, Some(""), "Author").unwrap())
6363 .unwrap(),
6464 "Archive/Inner.one"
6565 );
crates/notebook/tests/support/model_ops.rs+12-13
......@@ -287,17 +287,16 @@ pub fn save(
287287 text: ExGuid,
288288 edit: impl FnOnce(&mut Page),
289289) -> Result<Option<u64>, notebook::Error> {
290 let (space, _) = locate(&cache.snapshot()?, text);
291 save_page(cache, space, edit)
292}
293
294/// Resolves a conflict by taking the remote page, then queues `edit` of it.
295pub fn review(
296 cache: &Replica,
297 id: u64,
298 text: ExGuid,
299 edit: impl FnOnce(&mut Page),
300) -> Result<Option<u64>, notebook::Error> {
301 cache.resolve(id, notebook::Resolution::Theirs)?;
302 save(cache, text, edit)
290 let mut space = None;
291 for (listed, ..) in cache.pages()? {
292 if paragraph_with(&cache.page(listed)?, text).is_some() {
293 space = Some(listed);
294 break;
295 }
296 }
297 save_page(
298 cache,
299 space.expect("text object belongs to an active page"),
300 edit,
301 )
303302}
crates/notebook/tests/support/model_schedule.rs+68-39
......@@ -1,16 +1,20 @@
11//! A deterministic multi-actor schedule over page-model saves: several replicas edit one
2//! page offline, publish through a fault-injecting remote, reopen, and review conflicts.
2//! page offline, publish through a fault-injecting remote, reopen, and delete the conflict
3//! pages the merges leave.
34
45use crate::model_ops::{self, AUTHOR};
5use crate::server::{Fault, Server};
6use notebook::{EditStatus, Remote, Replica, Resolution};
6use crate::server::{Fault, Server, remote_snapshot, snapshot};
7use notebook::{EditStatus, Remote, Replica};
78use onestore::{
8 CommitError, ExGuid, PreparedEdit, RevisionIndex, Store, Transaction,
9 CommitError, ExGuid, RevisionIndex, Stamp, Store, Transaction,
910 document::Document,
1011 page::{Outline, Page, PageObject},
1112};
1213use std::{io, sync::LazyLock};
1314
15#[path = "../../../onestore/tests/support/ops.rs"]
16mod ops;
17
1418static SOURCE: LazyLock<(Vec<u8>, ExGuid)> = LazyLock::new(|| {
1519 let source =
1620 onestore::create_section("model-schedule.one", "Original 🦀 é 0", "Author").unwrap();
......@@ -23,9 +27,9 @@ static SOURCE: LazyLock<(Vec<u8>, ExGuid)> = LazyLock::new(|| {
2327 for at in 1..8 {
2428 anchor = model_ops::insert_after(&mut page, anchor, &format!("Original 🦀 é {at}")).1;
2529 }
26 let source = PreparedEdit::page(&source, space, &page, "Author")
30 let source = ops::saved(&source, space, &page)
2731 .unwrap()
28 .as_bytes()
32 .as_slice()
2933 .to_vec();
3034 (source, space)
3135});
......@@ -139,6 +143,10 @@ impl Remote for Session<'_> {
139143 self.server.read()
140144 }
141145
146 fn stamp(&mut self) -> io::Result<Stamp> {
147 self.server.stamp()
148 }
149
142150 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
143151 assert!(!self.retired, "a retired attempt was replayed");
144152 self.publications += 1;
......@@ -146,8 +154,8 @@ impl Remote for Session<'_> {
146154 self.server.publish(transaction)
147155 }
148156
149 fn confirm(&mut self, snapshot: &[u8]) -> Result<(), CommitError> {
150 self.server.confirm(snapshot)
157 fn confirm(&mut self, base: &Stamp) -> Result<(), CommitError> {
158 self.server.confirm(base)
151159 }
152160}
153161
......@@ -165,11 +173,17 @@ fn statuses(cache: &Replica) -> Vec<(u64, Option<EditStatus>)> {
165173 .collect()
166174}
167175
168/// Fails unless the durable remote holds `revision` in the edited page's space.
176/// Fails unless the durable remote holds `revision`: in the edited page's space, or, for
177/// the edit adding a conflict page, in the space it changed first.
169178fn durable(server: &Server, revision: ExGuid) {
170179 let store = Store::parse(&server.durable).unwrap();
171180 let index = RevisionIndex::parse(&store).unwrap();
172 assert!(index.spaces[&SOURCE.1].revisions.contains_key(&revision));
181 assert!(
182 index
183 .spaces
184 .values()
185 .any(|space| space.revisions.contains_key(&revision))
186 );
173187}
174188
175189pub fn run(input: &[u8]) {
......@@ -181,10 +195,10 @@ pub fn run(input: &[u8]) {
181195 let actor = usize::from(step[0]) % replicas.len();
182196 let path = directory.path().join(format!("{actor}.sqlite"));
183197 let cache = replicas[actor].get_or_insert_with(|| Replica::create(&path, source).unwrap());
184 let snapshot = cache.snapshot().unwrap();
198 let image = snapshot(cache);
185199 let before = local(cache);
186200 let pending = cache.pending().unwrap();
187 let rows_now = rows(&snapshot);
201 let rows_now = rows(&image);
188202 match step[1] % 8 {
189203 0..=2 => {
190204 let row = rows_now[usize::from(step[2]) % rows_now.len()].clone();
......@@ -266,12 +280,17 @@ pub fn run(input: &[u8]) {
266280 };
267281 server.fault = Fault::None;
268282 let next = cache.pending().unwrap();
269 // Edits leave the queue oldest first, each with a durable receipt.
270 let left = pending.len() - next.len();
271 assert!(
272 next.iter()
273 .all(|edit| pending[left..].iter().any(|kept| kept.id == edit.id))
274 );
283 // Edits leave the queue oldest first, each with a durable receipt; a rebase
284 // queues the conflict pages it makes after them.
285 let queued = |edit: &notebook::PendingEdit| pending.iter().any(|kept| kept.id == edit.id);
286 let left = pending.len() - next.iter().filter(|edit| queued(edit)).count();
287 assert!(next.iter().all(|edit| {
288 pending[left..].iter().any(|kept| kept.id == edit.id)
289 || matches!(
290 &edit.edit.ops[..],
291 [onestore::op::Op::Section(onestore::op::SectionOp::Conflict { .. })]
292 )
293 }));
275294 for edit in &pending[..left] {
276295 let Some(EditStatus::Published { revision }) = cache.status(edit.id).unwrap()
277296 else {
......@@ -283,7 +302,7 @@ pub fn run(input: &[u8]) {
283302 assert_eq!(
284303 local(cache),
285304 shape(&model_ops::page_of(
286 &cache.remote_snapshot().unwrap(),
305 &remote_snapshot(cache),
287306 *space
288307 ))
289308 );
......@@ -301,31 +320,41 @@ pub fn run(input: &[u8]) {
301320 &char::from(b'a' + step[3] % 26).to_string(),
302321 );
303322 let visible = server.visible.clone();
304 PreparedEdit::page(&visible, *space, &page, "Remote")
323 ops::save(&visible, *space, &page)
305324 .unwrap()
306325 .commit(&mut server)
307326 .unwrap();
308327 }
309328 }
310329 6 => {
311 if let Some(conflict) = cache.conflict().unwrap() {
312 let keep = if step[2] & 1 == 0 {
313 Resolution::Mine
314 } else {
315 Resolution::Theirs
316 };
317 let remote = model_ops::page_of(&cache.remote_snapshot().unwrap(), *space);
318 cache.resolve(conflict.id, keep).unwrap();
319 match cache.conflict().unwrap() {
320 Some(next) => assert_ne!(next.space, conflict.space),
321 None => assert_eq!(
322 local(cache),
323 match keep {
324 Resolution::Mine => before.clone(),
325 Resolution::Theirs => shape(&remote),
326 }
327 ),
328 }
330 // The writer merged a conflict page's version by hand and deletes it.
331 let listed = cache.conflicts().unwrap();
332 if let Some(conflict) = listed
333 .iter()
334 .find(|(page, _)| page == space)
335 .map(|(_, pages)| pages[usize::from(step[2]) % pages.len()].space)
336 {
337 let delete = onestore::op::Op::Section(onestore::op::SectionOp::Delete(vec![
338 conflict,
339 ]));
340 cache
341 .apply(
342 AUTHOR,
343 onestore::op::Edit {
344 at: model_ops::now(),
345 ops: vec![delete],
346 },
347 )
348 .unwrap();
349 assert!(
350 cache
351 .conflicts()
352 .unwrap()
353 .iter()
354 .flat_map(|(_, pages)| pages)
355 .all(|page| page.space != conflict)
356 );
357 assert_eq!(local(cache), before);
329358 let (result, publications) = {
330359 let mut session = Session {
331360 retired: false,
......@@ -356,6 +385,6 @@ pub fn run(input: &[u8]) {
356385 }
357386 }
358387 rows(&server.durable);
359 rows(&replicas[actor].as_ref().unwrap().snapshot().unwrap());
388 rows(&snapshot(replicas[actor].as_ref().unwrap()));
360389 }
361390}
crates/notebook/tests/support/page_schedule.rs+34-40
......@@ -1,8 +1,12 @@
1use crate::{disk, model_ops};
2use notebook::{EditStatus, Remote, Replica, Resolution};
1use crate::{
2 disk, model_ops,
3 server::{remote_snapshot, snapshot},
4};
5use notebook::{Remote, Replica};
36use onestore::{
4 CommitError, ExGuid, PageEdit, PreparedEdit, RevisionIndex, Store, Transaction,
7 CommitError, ExGuid, PageEdit, RevisionIndex, Stamp, Store, Transaction,
58 document::{Document, Format, Kind, Layout},
9 op::SectionOp,
610 page::{
711 Outline, Page, PageObject, PageParagraph, Paragraph, ParagraphContent, TextObject,
812 text::new_id,
......@@ -10,6 +14,9 @@ use onestore::{
1014};
1115use std::{io, sync::LazyLock};
1216
17#[path = "../../../onestore/tests/support/ops.rs"]
18mod ops;
19
1320const BODY: &str = "Body 🦋 é";
1421
1522#[path = "../../../onestore/tests/support/current.rs"]
......@@ -19,7 +26,7 @@ static SOURCE: LazyLock<Vec<u8>> = LazyLock::new(|| {
1926 let mut source = onestore::create_section("page-schedule.one", "Original", "Author").unwrap();
2027 for _ in 0..3 {
2128 let page = onestore::PageCreation::new(None, Some("Same title"), "Author").unwrap();
22 source = PreparedEdit::create_page(&source, &page)
29 source = ops::section_op(&source, SectionOp::Create(page.clone()))
2330 .unwrap()
2431 .as_bytes()
2532 .to_vec();
......@@ -107,6 +114,10 @@ impl Remote for Session<'_> {
107114 Ok(self.disk.visible.clone())
108115 }
109116
117 fn stamp(&mut self) -> io::Result<Stamp> {
118 Stamp::of(&self.disk.visible).map_err(io::Error::other)
119 }
120
110121 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
111122 let mut image = self.disk.visible.clone();
112123 transaction.apply(&mut image).unwrap();
......@@ -121,8 +132,8 @@ impl Remote for Session<'_> {
121132 result
122133 }
123134
124 fn confirm(&mut self, snapshot: &[u8]) -> Result<(), CommitError> {
125 onestore::confirm_snapshot(self.disk, snapshot)
135 fn confirm(&mut self, base: &Stamp) -> Result<(), CommitError> {
136 onestore::confirm(self.disk, base)
126137 }
127138}
128139
......@@ -152,8 +163,8 @@ pub fn run(input: &[u8]) {
152163 let actor = usize::from(step[0]) % replicas.len();
153164 let path = directory.path().join(format!("{actor}.sqlite"));
154165 let cache = replicas[actor].get_or_insert_with(|| Replica::create(&path, &SOURCE).unwrap());
155 let snapshot = cache.snapshot().unwrap();
156 let listed = pages(&snapshot);
166 let image = snapshot(cache);
167 let listed = pages(&image);
157168 let pending = cache.pending().unwrap();
158169 match step[1] % 8 {
159170 0 | 1 => {
......@@ -183,7 +194,7 @@ pub fn run(input: &[u8]) {
183194 .collect();
184195 replicas[actor] = None;
185196 let cache = Replica::open(&path).unwrap();
186 assert_eq!(pages(&cache.snapshot().unwrap()), listed);
197 assert_eq!(pages(&snapshot(&cache)), listed);
187198 assert_eq!(cache.pending().unwrap(), pending);
188199 assert_eq!(
189200 pending
......@@ -200,24 +211,9 @@ pub fn run(input: &[u8]) {
200211 disk.fail_at =
201212 (step[2] != 0).then_some(usize::from(u16::from_le_bytes([step[2], step[3]])));
202213 disk.random = u64::from(step[4]) + 1;
203 let prior = pending
204 .first()
205 .and_then(|edit| cache.status(edit.id).unwrap());
206214 let result = cache.sync_once(&mut Session { disk: &mut disk });
207 if matches!(prior, Some(EditStatus::AwaitingConfirmation { .. })) {
208 assert!(
209 result.is_err()
210 || !matches!(
211 result.as_ref().unwrap().edit,
212 Some((_, EditStatus::Conflict(_)))
213 )
214 );
215 }
216215 if result.is_ok() && cache.pending().unwrap().is_empty() {
217 assert_eq!(
218 pages(&cache.snapshot().unwrap()),
219 pages(&cache.remote_snapshot().unwrap())
220 );
216 assert_eq!(pages(&snapshot(cache)), pages(&remote_snapshot(cache)));
221217 }
222218 }
223219 5 => {
......@@ -244,36 +240,34 @@ pub fn run(input: &[u8]) {
244240 }
245241 })
246242 .collect();
247 let expected = PreparedEdit::pages(&snapshot, &edits);
243 let expected = ops::section_op(&image, SectionOp::Pages(edits.to_vec()));
248244 match section_op(cache, onestore::op::SectionOp::Pages(edits)) {
249245 Ok(_) => assert_eq!(
250 pages(&cache.snapshot().unwrap()),
246 pages(&snapshot(cache)),
251247 pages(expected.unwrap().as_bytes())
252248 ),
253249 Err(_) => {
254250 assert!(expected.is_err());
255 assert_eq!(pages(&cache.snapshot().unwrap()), listed);
251 assert_eq!(pages(&snapshot(cache)), listed);
256252 assert_eq!(cache.pending().unwrap(), pending);
257253 }
258254 }
259255 }
260256 7 => {
261 if let Some(conflict) = cache.conflict().unwrap() {
262 // Taking theirs for the page list would drop the pages this actor made.
263 let root = listed.iter().all(|(sid, ..)| *sid != conflict.space);
264 let keep = if root || step[2] & 1 == 0 {
265 Resolution::Mine
266 } else {
267 Resolution::Theirs
268 };
269 if cache.resolve(conflict.id, keep).is_err() {
270 assert_eq!(cache.pending().unwrap(), pending);
271 }
257 // The actor merged a conflict page's version by hand and deletes it.
258 if let Some(conflict) = cache
259 .conflicts()
260 .unwrap()
261 .first()
262 .map(|(_, pages)| pages[usize::from(step[2]) % pages.len()].space)
263 {
264 section_op(cache, onestore::op::SectionOp::Delete(vec![conflict])).unwrap();
265 assert_eq!(pages(&snapshot(cache)), listed);
272266 }
273267 }
274268 _ => unreachable!(),
275269 }
276 let local = pages(&replicas[actor].as_ref().unwrap().snapshot().unwrap());
270 let local = pages(&snapshot(replicas[actor].as_ref().unwrap()));
277271 for page in &owned[actor] {
278272 assert!(local.iter().any(|(sid, oid, _)| (*sid, *oid) == *page));
279273 }
crates/notebook/tests/support/server.rs+101-4
......@@ -75,8 +75,8 @@ impl Remote for Server {
7575 fn read(&mut self) -> io::Result<Vec<u8>> {
7676 Ok(self.visible.clone())
7777 }
78 fn stamp(&mut self) -> io::Result<Option<Stamp>> {
79 Ok(Stamp::of(&self.visible).ok())
78 fn stamp(&mut self) -> io::Result<Stamp> {
79 Stamp::of(&self.visible).map_err(io::Error::other)
8080 }
8181 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
8282 self.publications += 1;
......@@ -99,13 +99,13 @@ impl Remote for Server {
9999 _ => Ok(()),
100100 }
101101 }
102 fn confirm(&mut self, snapshot: &[u8]) -> Result<(), CommitError> {
102 fn confirm(&mut self, base: &Stamp) -> Result<(), CommitError> {
103103 self.confirmations += 1;
104104 if matches!(self.fault, Fault::Confirm) {
105105 self.fault = Fault::None;
106106 return Err(failure(CommitState::Unknown));
107107 }
108 onestore::confirm_snapshot(self, snapshot)?;
108 onestore::confirm(self, base)?;
109109 if matches!(self.fault, Fault::ConfirmCommitted) {
110110 self.fault = Fault::None;
111111 return Err(failure(CommitState::Committed));
......@@ -165,3 +165,100 @@ pub fn section_op(cache: &notebook::Replica, op: onestore::op::SectionOp) -> u64
165165 )
166166 .unwrap()
167167}
168
169/// The image the queue leaves, as a recovery archive of the cache records it.
170pub fn snapshot(cache: &notebook::Replica) -> Vec<u8> {
171 archived(cache, |recovery| recovery.snapshot())
172}
173
174/// The remote image the cache last observed, as a recovery archive records it.
175pub fn remote_snapshot(cache: &notebook::Replica) -> Vec<u8> {
176 archived(cache, |recovery| recovery.remote_snapshot())
177}
178
179fn archived(
180 cache: &notebook::Replica,
181 image: impl FnOnce(&notebook::Recovery) -> Result<Vec<u8>, notebook::Error>,
182) -> Vec<u8> {
183 let directory = tempfile::tempdir().unwrap();
184 let path = directory.path().join("recovery.sqlite");
185 cache.export_recovery(&path).unwrap();
186 image(&notebook::Recovery::open(&path).unwrap()).unwrap()
187}
188
189/// `image` after another writer replaces `range` of the text object `text` with `with`.
190pub fn typed(
191 image: &[u8],
192 space: ExGuid,
193 text: ExGuid,
194 range: std::ops::Range<u32>,
195 with: &str,
196) -> Vec<u8> {
197 let op = onestore::op::PageOp::Text {
198 text,
199 range,
200 with: with.into(),
201 };
202 edited(image, vec![onestore::op::Op::Page { space, op }]).unwrap()
203}
204
205/// `image` after another writer's `ops`, sealed as one revision per space they change.
206pub fn edited(image: &[u8], ops: Vec<onestore::op::Op>) -> Result<Vec<u8>, onestore::op::OpError> {
207 let arena = onestore::Arena::default();
208 let mut section =
209 onestore::Section::open(&arena, image.to_vec()).map_err(onestore::op::OpError::Failed)?;
210 let edit = onestore::op::Edit {
211 at: 134_000_000_000_000_000,
212 ops,
213 };
214 section.apply("Other writer", &edit)?;
215 section.seal().map_err(onestore::op::OpError::Failed)?;
216 Ok(section.image())
217}
218
219/// A conflict page as the tests compare it: whose version it is and its texts.
220pub type Version = (String, Vec<String>);
221
222/// Each page's conflict pages in list order.
223pub fn conflicts(image: &[u8]) -> Vec<(ExGuid, Vec<Version>)> {
224 let arena = onestore::Arena::default();
225 let mut section = onestore::Section::open(&arena, image.to_vec()).unwrap();
226 section
227 .conflicts()
228 .unwrap()
229 .into_iter()
230 .map(|(page, conflicts)| {
231 let conflicts = conflicts
232 .into_iter()
233 .map(|conflict| {
234 let page = section.page(conflict.space).unwrap();
235 (conflict.user, page_texts(&page))
236 })
237 .collect();
238 (page, conflicts)
239 })
240 .collect()
241}
242
243/// The texts of a page's outlines and title, in page order.
244pub fn page_texts(page: &onestore::page::Page) -> Vec<String> {
245 use onestore::page::PageObject;
246 page.objects
247 .iter()
248 .flat_map(|object| match object {
249 PageObject::Outline(outline) => outline.paragraphs.clone(),
250 PageObject::Title(title) => title
251 .outlines
252 .iter()
253 .flat_map(|outline| outline.paragraphs.clone())
254 .collect(),
255 _ => Vec::new(),
256 })
257 .filter_map(|paragraph| Some(paragraph.text()?.text.text().to_owned()))
258 .collect()
259}
260
261/// Whether `image` lists a conflict page under page `space`.
262pub fn conflicted(image: &[u8], space: ExGuid) -> bool {
263 conflicts(image).iter().any(|(page, _)| *page == space)
264}
crates/notebook/tests/sync.rs+148-270
......@@ -1,6 +1,6 @@
1//! Replica protocol: durable receipts, retired attempts, conflict resolution and contention.
1//! Replica protocol: durable receipts, retired attempts, conflict pages and contention.
22
3use notebook::{ConflictKind, EditStatus, Error, Remote, Replica, Resolution};
3use notebook::{EditStatus, Error, Remote, Replica};
44use onestore::{CommitError, CommitState, ExGuid, Transaction};
55use std::{io, ops::Range};
66
......@@ -29,14 +29,14 @@ fn an_unchanged_stamp_publishes_and_settles_without_reading_the_remote() {
2929 self.reads += 1;
3030 self.server.read()
3131 }
32 fn stamp(&mut self) -> io::Result<Option<onestore::Stamp>> {
32 fn stamp(&mut self) -> io::Result<onestore::Stamp> {
3333 self.server.stamp()
3434 }
3535 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
3636 self.server.publish(transaction)
3737 }
38 fn confirm(&mut self, snapshot: &[u8]) -> Result<(), CommitError> {
39 self.server.confirm(snapshot)
38 fn confirm(&mut self, base: &onestore::Stamp) -> Result<(), CommitError> {
39 self.server.confirm(base)
4040 }
4141 }
4242 let directory = tempfile::tempdir().unwrap();
......@@ -56,11 +56,11 @@ fn an_unchanged_stamp_publishes_and_settles_without_reading_the_remote() {
5656 assert_eq!(remote.reads, 0);
5757 // Another writer's commit moves the header, so the next step reads the file.
5858 let native =
59 onestore::replace_text(&remote.server.visible, sid, object, 0..0, "Native ").unwrap();
59 typed(&remote.server.visible, sid, object, 0..0, "Native ");
6060 remote.server.visible.clone_from(&native);
6161 assert_eq!(cache.sync_once(&mut remote).unwrap().edit, None);
6262 assert_eq!(remote.reads, 1);
63 assert_eq!(cache.snapshot().unwrap(), native);
63 assert_eq!(snapshot(&cache), native);
6464}
6565
6666#[test]
......@@ -89,14 +89,13 @@ fn recovery_archive_preserves_the_queue_uncertainty_and_receipts_without_becomin
8989 let created = onestore::PageCreation::new(None, Some("Recovery 🦀"), "Fixture").unwrap();
9090 section_op(&cache, onestore::op::SectionOp::Create(created));
9191 save(&cache, object, 0..0, "Last ").unwrap();
92 let working = cache.snapshot().unwrap();
93 let remote = cache.remote_snapshot().unwrap();
92 let working = snapshot(&cache);
93 let remote = remote_snapshot(&cache);
9494 let pending = cache.pending().unwrap();
9595 let uncertain = cache.status(attempted).unwrap();
9696 let source_file = std::fs::read(&path).unwrap();
9797 let summary = RecoverySummary {
9898 queued_edits: 4,
99 conflicts: 0,
10099 uncertain_edits: 1,
101100 published_receipts: 1,
102101 base_bytes: remote.len() as u64,
......@@ -173,28 +172,21 @@ fn recovery_archive_preserves_the_queue_uncertainty_and_receipts_without_becomin
173172}
174173
175174#[test]
176fn recovery_archive_retains_conflict_images_and_rejects_foreign_or_future_archives() {
175fn recovery_archive_retains_the_queue_and_rejects_foreign_or_future_archives() {
177176 use notebook::Recovery;
178177
179178 let directory = tempfile::tempdir().unwrap();
180179 let source = onestore::create_section("recovery.one", "Original", "Fixture").unwrap();
181 let (space, object, _) = text(&source);
180 let (_, object, _) = text(&source);
182181 let cache = Replica::create(directory.path().join("live.sqlite"), &source).unwrap();
183182 let id = save(&cache, object, 0..8, "Local").unwrap();
184 let changed = onestore::replace_text(&source, space, object, 0..8, "Remote").unwrap();
185 let mut server = Server::new(&changed);
186 let outcome = cache.sync_once(&mut server).unwrap().edit.unwrap();
187 assert_eq!(
188 outcome,
189 (id, EditStatus::Conflict(ConflictKind::ContentChanged))
190 );
191 let path = directory.path().join("conflict.sqlite");
183 let path = directory.path().join("queued.sqlite");
192184 cache.export_recovery(&path).unwrap();
193185 let archive = Recovery::open(&path).unwrap();
194186 assert_eq!(text(&archive.snapshot().unwrap()).2, "Local");
195 assert_eq!(archive.remote_snapshot().unwrap(), changed);
196 assert_eq!(archive.status(id).unwrap(), Some(outcome.1.clone()));
197 assert_eq!(archive.summary().unwrap().conflicts, 1);
187 assert_eq!(archive.remote_snapshot().unwrap(), source);
188 assert_eq!(archive.status(id).unwrap(), Some(EditStatus::Pending));
189 assert_eq!(archive.summary().unwrap().queued_edits, 1);
198190 assert_eq!(archive.summary().unwrap().uncertain_edits, 0);
199191 drop(archive);
200192 for sql in [
......@@ -208,8 +200,8 @@ fn recovery_archive_retains_conflict_images_and_rejects_foreign_or_future_archiv
208200 assert!(Recovery::open(&path).is_err());
209201 assert_eq!(std::fs::read(&path).unwrap(), before);
210202 }
211 assert_eq!(cache.status(id).unwrap(), Some(outcome.1.clone()));
212 assert_eq!(text(&cache.snapshot().unwrap()).2, "Local");
203 assert_eq!(cache.status(id).unwrap(), Some(EditStatus::Pending));
204 assert_eq!(text(&snapshot(&cache)).2, "Local");
213205}
214206
215207#[test]
......@@ -220,13 +212,13 @@ fn disjoint_remote_changes_merge_and_persist_the_remote_receipt() {
220212 let (sid, oid, _) = text(&source);
221213 let cache = Replica::create(&path, &source).unwrap();
222214 let id = save(&cache, oid, 2..4, "🐈").unwrap();
223 let remote = onestore::replace_text(&source, sid, oid, 0..6, "Xab🦀cYd").unwrap();
215 let remote = typed(&source, sid, oid, 0..6, "Xab🦀cYd");
224216 let mut server = Server::new(&remote);
225217 let outcome = cache.sync_once(&mut server).unwrap().edit.unwrap();
226218 assert_eq!(outcome.0, id);
227219 assert!(matches!(outcome.1, EditStatus::Published { .. }));
228220 assert_eq!(text(&server.durable).2, "Xab🐈cYd");
229 assert_eq!(cache.snapshot().unwrap(), server.durable);
221 assert_eq!(snapshot(&cache), server.durable);
230222 assert!(cache.pending().unwrap().is_empty());
231223 drop(cache);
232224 let cache = Replica::open(&path).unwrap();
......@@ -238,31 +230,38 @@ fn disjoint_remote_changes_merge_and_persist_the_remote_receipt() {
238230 assert!(next > id);
239231}
240232
233/// As OneNote 2010 does, the remote's version stays the page and the local one becomes a
234/// conflict page under it, published with the rest of the queue.
241235#[test]
242fn overlapping_changes_preserve_both_images_and_survive_restart() {
236fn overlapping_changes_keep_both_versions_and_survive_restart() {
243237 let dir = tempfile::tempdir().unwrap();
244238 let path = dir.path().join("cache.sqlite");
245239 let source = onestore::create_section("sync.one", "abc", "Fixture").unwrap();
246240 let (sid, oid, _) = text(&source);
247241 let cache = Replica::create(&path, &source).unwrap();
248242 let id = save(&cache, oid, 1..2, "L").unwrap();
249 let local = cache.snapshot().unwrap();
250 let remote = onestore::replace_text(&source, sid, oid, 1..2, "R").unwrap();
243 let remote = typed(&source, sid, oid, 1..2, "R");
251244 let mut server = Server::new(&remote);
252 assert_eq!(
245 assert!(matches!(
253246 cache.sync_once(&mut server).unwrap().edit,
254 Some((id, EditStatus::Conflict(ConflictKind::ContentChanged)))
247 Some((newest, EditStatus::Published { .. })) if newest > id
248 ));
249 assert_eq!(server.publications, 1);
250 assert_eq!(content(&server.durable, oid), "aRc");
251 assert_eq!(
252 conflicts(&server.durable),
253 [(sid, vec![(model_ops::AUTHOR.to_owned(), vec!["aLc".to_owned()])])]
255254 );
256 assert_eq!(server.publications, 0);
255 assert_eq!(snapshot(&cache), server.durable);
256 assert!(cache.pending().unwrap().is_empty());
257257 drop(cache);
258258 let cache = Replica::open(&path).unwrap();
259 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));
260 assert_eq!(cache.remote_snapshot().unwrap(), remote);
261 assert_eq!(cache.pending().unwrap().len(), 1);
262 assert_eq!(
259 assert!(matches!(
263260 cache.status(id).unwrap(),
264 Some(EditStatus::Conflict(ConflictKind::ContentChanged))
265 );
261 Some(EditStatus::Published { .. })
262 ));
263 assert_eq!(cache.conflicts().unwrap()[0].0, sid);
264 assert_eq!(cache.sync_once(&mut server).unwrap().edit, None);
266265}
267266
268267#[test]
......@@ -452,16 +451,19 @@ fn twelve_local_editors_progress_during_remote_reads_publication_and_confirmatio
452451 }
453452 impl Remote for Paused {
454453 fn read(&mut self) -> io::Result<Vec<u8>> {
455 self.wait("read");
456454 self.server.read()
457455 }
456 fn stamp(&mut self) -> io::Result<onestore::Stamp> {
457 self.wait("read");
458 self.server.stamp()
459 }
458460 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
459461 self.wait("publish");
460462 self.server.publish(transaction)
461463 }
462 fn confirm(&mut self, source: &[u8]) -> Result<(), CommitError> {
464 fn confirm(&mut self, base: &onestore::Stamp) -> Result<(), CommitError> {
463465 self.wait("confirm");
464 self.server.confirm(source)
466 self.server.confirm(base)
465467 }
466468 }
467469 for phase in ["read", "publish", "confirm"] {
......@@ -505,9 +507,6 @@ fn twelve_local_editors_progress_during_remote_reads_publication_and_confirmatio
505507 assert!(
506508 matches!(cache.sync_once(&mut Server::new(&source)),Err(Error::Io(error)) if error.kind()==io::ErrorKind::WouldBlock)
507509 );
508 assert!(
509 matches!(cache.resolve(first, Resolution::Theirs), Err(Error::Io(error)) if error.kind() == io::ErrorKind::WouldBlock)
510 );
511510 let started = std::time::Instant::now();
512511 let handles: Vec<_> = (0..12)
513512 .map(|writer| {
......@@ -550,7 +549,7 @@ fn twelve_local_editors_progress_during_remote_reads_publication_and_confirmatio
550549 cache.pending().unwrap().len(),
551550 if phase == "read" { 0 } else { 12 }
552551 );
553 let expected = text(&cache.snapshot().unwrap()).2;
552 let expected = text(&snapshot(&cache)).2;
554553 for writer in 0..12 {
555554 assert_eq!(expected.matches(&format!("[{writer}] ")).count(), 1);
556555 }
......@@ -562,7 +561,7 @@ fn twelve_local_editors_progress_during_remote_reads_publication_and_confirmatio
562561 }
563562 assert_eq!(server.publications, if phase == "read" { 1 } else { 2 });
564563 assert_eq!(text(&server.durable).2, expected);
565 assert_eq!(cache.snapshot().unwrap(), server.durable);
564 assert_eq!(snapshot(&cache), server.durable);
566565 }
567566}
568567
......@@ -578,12 +577,12 @@ fn unrelated_remote_files_never_replace_a_local_cache() {
578577 if pending {
579578 save(&cache, oid, 0..0, "Local ").unwrap();
580579 }
581 let before = pages(&cache.snapshot().unwrap());
580 let before = pages(&snapshot(&cache));
582581 assert!(
583582 matches!(cache.sync_once(&mut server),Err(Error::Io(error)) if error.kind()==io::ErrorKind::InvalidInput)
584583 );
585 assert_eq!(pages(&cache.snapshot().unwrap()), before);
586 assert_eq!(cache.remote_snapshot().unwrap(), source);
584 assert_eq!(pages(&snapshot(&cache)), before);
585 assert_eq!(remote_snapshot(&cache), source);
587586 assert_eq!(server.publications, 0);
588587 }
589588}
......@@ -600,165 +599,76 @@ fn content(bytes: &[u8], text: ExGuid) -> String {
600599 .to_owned()
601600}
602601
602/// A conflict never holds the queue: later edits of the page apply to the remote's version.
603603#[test]
604fn a_conflict_holds_later_edits_until_resolved_either_way_and_survives_reopen() {
605 for resolution in [Resolution::Mine, Resolution::Theirs] {
606 let dir = tempfile::tempdir().unwrap();
607 let path = dir.path().join("cache.sqlite");
608 let source = onestore::create_section("resolve.one", "abc", "Fixture").unwrap();
609 let (sid, oid, _) = text(&source);
610 let cache = Replica::create(&path, &source).unwrap();
611 let first = save(&cache, oid, 1..2, "L").unwrap();
612 let remote = onestore::replace_text(&source, sid, oid, 0..3, "aRcZ").unwrap();
613 let mut server = Server::new(&remote);
614 let conflict = EditStatus::Conflict(ConflictKind::ContentChanged);
615 assert_eq!(
616 cache.sync_once(&mut server).unwrap().edit,
617 Some((first, conflict.clone()))
618 );
619 let mut dependent = None;
620 for n in 0..12 {
621 dependent = save(&cache, oid, 0..0, &format!("[{n}] "));
622 }
623 let dependent = dependent.unwrap();
624 assert_ne!(dependent, first);
625 assert_eq!(cache.status(dependent).unwrap(), Some(conflict.clone()));
626 let local = content(&cache.snapshot().unwrap(), oid);
627 assert!(local.ends_with("[0] aLc"), "{local}");
628 let pending = cache.pending().unwrap();
629 assert_eq!(pending.len(), 13);
630 assert_eq!(
631 cache.sync_once(&mut server).unwrap().edit,
632 Some((dependent, conflict.clone())),
633 "an unchanged remote leaves the conflict as it was"
634 );
635 assert_eq!(server.publications, 0);
636 drop(cache);
637 let cache = Replica::open(&path).unwrap();
638 assert_eq!(cache.pending().unwrap(), pending);
639 assert_eq!(content(&cache.snapshot().unwrap(), oid), local);
640 assert_eq!(
641 cache.conflict().unwrap(),
642 Some(notebook::Conflict {
643 id: dependent,
644 space: sid,
645 kind: ConflictKind::ContentChanged
646 })
647 );
648 cache.resolve(first, resolution).unwrap();
649 assert_eq!(cache.conflict().unwrap(), None);
650 assert_eq!(cache.remote_snapshot().unwrap(), remote);
651 let expected = match resolution {
652 Resolution::Mine => local.clone(),
653 Resolution::Theirs => "aRcZ".to_owned(),
654 };
655 assert_eq!(content(&cache.snapshot().unwrap(), oid), expected);
656 drop(cache);
657 let cache = Replica::open(&path).unwrap();
658 match resolution {
659 Resolution::Mine => {
660 assert!(matches!(
661 cache.sync_once(&mut server).unwrap().edit,
662 Some((_, EditStatus::Published { .. }))
663 ));
664 assert_eq!(server.publications, 1);
665 for edit in &pending {
666 assert!(matches!(
667 cache.status(edit.id).unwrap(),
668 None | Some(EditStatus::Published { .. })
669 ));
670 }
671 assert!(matches!(
672 cache.status(first).unwrap(),
673 Some(EditStatus::Published { .. })
674 ));
675 }
676 Resolution::Theirs => {
677 // The emptied edits are acknowledged once the remote image is flushed.
678 assert!(matches!(
679 cache.sync_once(&mut server).unwrap().edit,
680 Some((id, EditStatus::Published { .. })) if id == dependent
681 ));
682 assert_eq!((server.publications, server.confirmations), (0, 1));
683 assert!(matches!(
684 cache.status(first).unwrap(),
685 Some(EditStatus::Published { .. })
686 ));
687 }
688 }
689 assert_eq!(content(&server.durable, oid), expected);
690 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&server.durable));
691 assert!(cache.pending().unwrap().is_empty());
692 }
693}
694
695#[test]
696fn resolution_refuses_edits_that_are_not_in_conflict() {
604fn edits_after_a_conflict_apply_to_the_remote_version_and_survive_reopen() {
697605 let dir = tempfile::tempdir().unwrap();
606 let path = dir.path().join("cache.sqlite");
698607 let source = onestore::create_section("resolve.one", "abc", "Fixture").unwrap();
699608 let (sid, oid, _) = text(&source);
700 let cache = Replica::create(dir.path().join("cache.sqlite"), &source).unwrap();
701 let id = save(&cache, oid, 1..2, "L").unwrap();
702 let refused = |cache: &Replica, id| matches!(cache.resolve(id, Resolution::Mine), Err(Error::Io(error)) if error.kind() == io::ErrorKind::InvalidInput);
703 assert!(refused(&cache, id));
704 let remote = onestore::replace_text(&source, sid, oid, 0..3, "🦀Rc").unwrap();
609 let cache = Replica::create(&path, &source).unwrap();
610 save(&cache, oid, 1..2, "L").unwrap();
611 let remote = typed(&source, sid, oid, 0..3, "aRcZ");
705612 let mut server = Server::new(&remote);
706613 assert!(matches!(
707614 cache.sync_once(&mut server).unwrap().edit,
708 Some((_, EditStatus::Conflict(_)))
615 Some((_, EditStatus::Published { .. }))
709616 ));
617 assert_eq!(content(&snapshot(&cache), oid), "aRcZ");
618 let mut expected = "aRcZ".to_owned();
619 for n in 0..12 {
620 save(&cache, oid, 0..0, &format!("[{n}] ")).unwrap();
621 expected.insert_str(0, &format!("[{n}] "));
622 }
710623 let pending = cache.pending().unwrap();
711 assert!(refused(&cache, id + 1));
624 drop(cache);
625 let cache = Replica::open(&path).unwrap();
712626 assert_eq!(cache.pending().unwrap(), pending);
713 cache.resolve(id, Resolution::Mine).unwrap();
714 server.fault = Fault::UnknownBefore;
715 assert!(
716 matches!(cache.sync_once(&mut server), Err(Error::Remote(error)) if error.state == CommitState::Unknown)
717 );
718 let attempted = cache.status(id).unwrap();
719627 assert!(matches!(
720 attempted,
721 Some(EditStatus::AwaitingConfirmation { .. })
628 cache.sync_once(&mut server).unwrap().edit,
629 Some((id, EditStatus::Published { .. })) if id == pending.last().unwrap().id
722630 ));
723 assert!(refused(&cache, id));
724 assert_eq!(cache.status(id).unwrap(), attempted);
725 assert_eq!(server.publications, 1);
631 assert_eq!(server.publications, 2);
632 assert_eq!(content(&server.durable, oid), expected);
633 assert_eq!(
634 conflicts(&server.durable),
635 [(sid, vec![(model_ops::AUTHOR.to_owned(), vec!["aLc".to_owned()])])]
636 );
637 assert!(cache.pending().unwrap().is_empty());
726638}
727639
640/// A rebase that cannot be stored leaves the queue, and every page, as it was.
728641#[test]
729fn a_failed_resolution_leaves_the_conflict_as_it_was() {
642fn a_failed_rebase_leaves_the_queue_as_it_was() {
730643 let dir = tempfile::tempdir().unwrap();
731644 let path = dir.path().join("cache.sqlite");
732645 let source = onestore::create_section("resolve.one", "abc", "Fixture").unwrap();
733646 let (sid, oid, _) = text(&source);
734647 let cache = Replica::create(&path, &source).unwrap();
735648 let id = save(&cache, oid, 1..2, "L").unwrap();
736 let remote = onestore::replace_text(&source, sid, oid, 0..3, "XaRc").unwrap();
649 let remote = typed(&source, sid, oid, 0..3, "XaRc");
737650 let mut server = Server::new(&remote);
738 cache.sync_once(&mut server).unwrap();
739 let local = pages(&cache.snapshot().unwrap());
651 let local = pages(&snapshot(&cache));
740652 let pending = cache.pending().unwrap();
741 let status = cache.status(id).unwrap();
742653 drop(cache);
743654 let connection = rusqlite::Connection::open(&path).unwrap();
744 connection.execute_batch("CREATE TRIGGER fail_clear BEFORE DELETE ON batches BEGIN SELECT RAISE(ABORT, 'test resolution failure'); END;").unwrap();
655 connection.execute_batch("CREATE TRIGGER fail_clear BEFORE DELETE ON batches BEGIN SELECT RAISE(ABORT, 'test rebase failure'); END;").unwrap();
745656 drop(connection);
746657 let cache = Replica::open(&path).unwrap();
747658 assert!(matches!(
748 cache.resolve(id, Resolution::Mine),
659 cache.sync_once(&mut server),
749660 Err(Error::Database(_))
750661 ));
751662 drop(cache);
752663 let cache = Replica::open(&path).unwrap();
753664 assert_eq!(cache.pending().unwrap(), pending);
754 assert_eq!(cache.status(id).unwrap(), status);
755 assert_eq!(pages(&cache.snapshot().unwrap()), local);
756 assert_eq!(cache.remote_snapshot().unwrap(), remote);
665 assert_eq!(cache.status(id).unwrap(), Some(EditStatus::Pending));
666 assert_eq!(pages(&snapshot(&cache)), local);
757667 assert_eq!(server.publications, 0);
758668}
759669
760670#[test]
761fn seeded_conflicts_resolve_to_the_remote_text_and_later_edits_publish_once() {
671fn seeded_conflicts_keep_the_remote_text_and_later_edits_publish_once() {
762672 let dir = tempfile::tempdir().unwrap();
763673 let original: Vec<_> = "abcdefghij🦀klmnop".chars().collect();
764674 let mut seed = 911_u64;
......@@ -779,70 +689,71 @@ fn seeded_conflicts_resolve_to_the_remote_text_and_later_edits_publish_once() {
779689 .unwrap();
780690 let (sid, oid, _) = text(&source);
781691 let cache = Replica::create(dir.path().join(format!("{case}.sqlite")), &source).unwrap();
782 let id = save(&cache, oid, start..end, "λ🦊μ").unwrap();
692 save(&cache, oid, start..end, "λ🦊μ").unwrap();
783693 save(&cache, oid, start + 1..start + 3, "🐕").unwrap();
694 let local = content(&snapshot(&cache), oid);
784695 let mut remote_text = original.clone();
785696 remote_text.splice(at..at + 1, "Ω🐈π".chars());
786697 let remote_text = prefix.clone() + &remote_text.iter().collect::<String>() + &suffix;
787 let remote = onestore::replace_text(
698 let remote = typed(
788699 &source,
789700 sid,
790701 oid,
791702 0..original.iter().map(|ch| ch.len_utf16() as u32).sum(),
792703 &remote_text,
793 )
794 .unwrap();
704 );
795705 let mut server = Server::new(&remote);
796706 assert!(
797707 matches!(
798708 cache.sync_once(&mut server).unwrap().edit,
799 Some((_, EditStatus::Conflict(ConflictKind::ContentChanged)))
709 Some((_, EditStatus::Published { .. }))
800710 ),
801711 "case {case}, seed {seed}"
802712 );
713 assert_eq!(content(&server.durable, oid), remote_text, "case {case}, seed {seed}");
714 assert_eq!(
715 conflicts(&server.durable),
716 [(sid, vec![(model_ops::AUTHOR.to_owned(), vec![local])])],
717 "case {case}, seed {seed}"
718 );
803719 let at_remote = start + prefix.len() as u32;
804 let reviewed = model_ops::review(&cache, id, oid, |page| {
805 model_ops::replace_text(page, oid, at_remote..at_remote + 4, "λ🐕μ")
806 })
807 .unwrap()
808 .unwrap();
720 let later = save(&cache, oid, at_remote..at_remote + 4, "λ🐕μ").unwrap();
809721 assert!(
810 matches!(cache.sync_once(&mut server).unwrap().edit, Some((actual, EditStatus::Published { .. })) if actual == reviewed),
722 matches!(cache.sync_once(&mut server).unwrap().edit, Some((actual, EditStatus::Published { .. })) if actual == later),
811723 "case {case}, seed {seed}"
812724 );
813725 let mut expected = original.clone();
814726 expected.splice(at..at + 1, "λ🐕μ".chars());
815727 let expected = prefix + &expected.iter().collect::<String>() + &suffix;
816 assert_eq!(
817 text(&server.durable).2,
818 expected,
819 "case {case}, seed {seed}"
820 );
821 assert_eq!(server.publications, 1);
728 assert_eq!(content(&server.durable, oid), expected, "case {case}, seed {seed}");
729 assert_eq!(server.publications, 2);
822730 assert!(cache.pending().unwrap().is_empty());
823731 }
824732}
825733
826734#[test]
827fn remote_changes_after_resolution_are_never_overwritten() {
828 struct ChangedAfterRead {
735fn remote_changes_after_a_conflict_are_never_overwritten() {
736 struct ChangedAfterStamp {
829737 server: Server,
830738 change: Option<Vec<u8>>,
831739 }
832 impl Remote for ChangedAfterRead {
740 impl Remote for ChangedAfterStamp {
833741 fn read(&mut self) -> io::Result<Vec<u8>> {
834 let snapshot = self.server.read()?;
742 self.server.read()
743 }
744 fn stamp(&mut self) -> io::Result<onestore::Stamp> {
745 let stamp = self.server.stamp()?;
835746 if let Some(changed) = self.change.take() {
836747 self.server.visible = changed.clone();
837748 self.server.durable = changed;
838749 }
839 Ok(snapshot)
750 Ok(stamp)
840751 }
841752 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
842753 self.server.publish(transaction)
843754 }
844 fn confirm(&mut self, snapshot: &[u8]) -> Result<(), CommitError> {
845 self.server.confirm(snapshot)
755 fn confirm(&mut self, base: &onestore::Stamp) -> Result<(), CommitError> {
756 self.server.confirm(base)
846757 }
847758 }
848759 for after_read in [false, true] {
......@@ -850,50 +761,41 @@ fn remote_changes_after_resolution_are_never_overwritten() {
850761 let source = onestore::create_section("resolve.one", "abc", "Fixture").unwrap();
851762 let (sid, oid, _) = text(&source);
852763 let cache = Replica::create(dir.path().join("cache.sqlite"), &source).unwrap();
853 let id = save(&cache, oid, 1..2, "L").unwrap();
854 let remote = onestore::replace_text(&source, sid, oid, 1..2, "R").unwrap();
855 let mut remote = ChangedAfterRead {
764 save(&cache, oid, 1..2, "L").unwrap();
765 let remote = typed(&source, sid, oid, 1..2, "R");
766 let mut remote = ChangedAfterStamp {
856767 server: Server::new(&remote),
857768 change: None,
858769 };
859770 assert!(matches!(
860771 cache.sync_once(&mut remote).unwrap().edit,
861 Some((_, EditStatus::Conflict(_)))
772 Some((_, EditStatus::Published { .. }))
862773 ));
863 let id = model_ops::review(&cache, id, oid, |page| {
864 model_ops::replace_text(page, oid, 1..2, "L")
865 })
866 .unwrap()
867 .unwrap();
868 let local = content(&cache.snapshot().unwrap(), oid);
869 assert_eq!(local, "aLc");
870 let changed = onestore::replace_text(&remote.server.visible, sid, oid, 1..2, "Q").unwrap();
774 // The local version again, typed over the remote's.
775 let id = save(&cache, oid, 1..2, "L").unwrap();
776 assert_eq!(content(&snapshot(&cache), oid), "aLc");
777 let changed = typed(&remote.server.visible, sid, oid, 1..2, "Q");
871778 if after_read {
872779 remote.change = Some(changed.clone());
873 } else {
874 remote.server.visible = changed.clone();
875 remote.server.durable = changed.clone();
876 }
877 if after_read {
878780 assert!(
879781 matches!(cache.sync_once(&mut remote), Err(Error::Remote(error)) if error.state == CommitState::NotCommitted)
880782 );
881783 assert_eq!(cache.status(id).unwrap(), Some(EditStatus::Pending));
784 } else {
785 remote.server.visible = changed.clone();
786 remote.server.durable = changed.clone();
882787 }
883 assert_eq!(
884 cache.sync_once(&mut remote).unwrap().edit,
885 Some((id, EditStatus::Conflict(ConflictKind::ContentChanged)))
886 );
887 assert_eq!(remote.server.durable, changed);
888 assert_eq!(remote.server.visible, changed);
889 assert_eq!(remote.server.publications, usize::from(after_read));
890 assert_eq!(content(&cache.snapshot().unwrap(), oid), local);
891 assert!(cache.pending().unwrap()[0].edit.ops.is_empty());
892788 assert!(matches!(
893 &cache.pending().unwrap()[1].edit.ops[..],
894 [onestore::op::Op::Page { op: onestore::op::PageOp::Text { range, with, .. }, .. }]
895 if *range == (1..2) && with == "L"
789 cache.sync_once(&mut remote).unwrap().edit,
790 Some((_, EditStatus::Published { .. }))
896791 ));
792 assert_eq!(content(&remote.server.durable, oid), "aQc");
793 let versions = [(model_ops::AUTHOR.to_owned(), vec!["aLc".to_owned()])];
794 assert_eq!(
795 conflicts(&remote.server.durable),
796 [(sid, [versions.clone(), versions].concat())]
797 );
798 assert!(cache.pending().unwrap().is_empty());
897799 }
898800}
899801
......@@ -908,7 +810,7 @@ fn remote_restore_retains_historical_receipts_without_replaying_them() {
908810 let published = save(&cache, object, 0..0, "Published ").unwrap();
909811 let (_, receipt) = cache.sync_once(&mut server).unwrap().edit.unwrap();
910812 assert!(matches!(receipt, EditStatus::Published { .. }));
911 let published_image = cache.snapshot().unwrap();
813 let published_image = snapshot(&cache);
912814 cache
913815 .export_recovery(directory.path().join("published.sqlite"))
914816 .unwrap();
......@@ -918,8 +820,8 @@ fn remote_restore_retains_historical_receipts_without_replaying_them() {
918820 server.durable.clone_from(&source);
919821 let cache = Replica::open(&path).unwrap();
920822 assert_eq!(cache.sync_once(&mut server).unwrap().edit, None);
921 assert_eq!(cache.snapshot().unwrap(), source);
922 assert_eq!(cache.remote_snapshot().unwrap(), source);
823 assert_eq!(snapshot(&cache), source);
824 assert_eq!(remote_snapshot(&cache), source);
923825 assert_eq!(cache.status(published).unwrap(), Some(receipt.clone()));
924826 assert_eq!(server.publications, 1);
925827 assert!(cache.pending().unwrap().is_empty());
......@@ -929,7 +831,7 @@ fn remote_restore_retains_historical_receipts_without_replaying_them() {
929831 assert_eq!(text(&archive.snapshot().unwrap()).2, "Published Original");
930832 drop(cache);
931833 let cache = Replica::open(path).unwrap();
932 assert_eq!(text(&cache.snapshot().unwrap()).2, "Original");
834 assert_eq!(text(&snapshot(&cache)).2, "Original");
933835 assert_eq!(cache.status(published).unwrap(), Some(receipt.clone()));
934836}
935837
......@@ -945,14 +847,14 @@ fn remote_restore_rebases_unsent_work_but_never_replays_an_uncertain_attempt() {
945847 let mut server = Server::new(&source);
946848 let published = save(&cache, object, 0..0, "Published ").unwrap();
947849 let (_, receipt) = cache.sync_once(&mut server).unwrap().edit.unwrap();
948 let end = u32::try_from(text(&cache.snapshot().unwrap()).2.encode_utf16().count()).unwrap();
850 let end = u32::try_from(text(&snapshot(&cache)).2.encode_utf16().count()).unwrap();
949851 let queued = save(&cache, object, end..end, " Local").unwrap();
950852 if uncertain {
951853 server.fault = fault;
952854 assert!(cache.sync_once(&mut server).is_err());
953855 }
954856 let prior_status = cache.status(queued).unwrap();
955 let local = cache.snapshot().unwrap();
857 let local = snapshot(&cache);
956858 let pending = cache.pending().unwrap();
957859 cache
958860 .export_recovery(directory.path().join("before-restore.sqlite"))
......@@ -969,7 +871,7 @@ fn remote_restore_rebases_unsent_work_but_never_replays_an_uncertain_attempt() {
969871 assert_eq!(Some(status.clone()), prior_status);
970872 assert_eq!(server.publications, attempts);
971873 assert_eq!(server.visible, source);
972 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));
874 assert_eq!(pages(&snapshot(&cache)), pages(&local));
973875 assert_eq!(cache.pending().unwrap(), pending);
974876 } else {
975877 assert!(matches!(status, EditStatus::Published { .. }));
......@@ -989,8 +891,10 @@ fn remote_restore_rebases_unsent_work_but_never_replays_an_uncertain_attempt() {
989891 }
990892}
991893
894/// A remote restored to an older image conflicts with local work on what it undid: the
895/// restored page stays and the local version becomes its conflict page.
992896#[test]
993fn restore_conflicts_keep_dependent_work_until_mine_publishes_it() {
897fn a_remote_restore_keeps_local_work_on_a_conflict_page() {
994898 let directory = tempfile::tempdir().unwrap();
995899 let path = directory.path().join("cache.sqlite");
996900 let source = onestore::create_section("restore.one", "Original", "Fixture").unwrap();
......@@ -1002,52 +906,26 @@ fn restore_conflicts_keep_dependent_work_until_mine_publishes_it() {
1002906 let head = save(&cache, object, 0..9, "Revised").unwrap();
1003907 server.visible.clone_from(&source);
1004908 server.durable.clone_from(&source);
1005 let conflict = EditStatus::Conflict(ConflictKind::ContentChanged);
1006 assert_eq!(
909 assert!(matches!(
1007910 cache.sync_once(&mut server).unwrap().edit,
1008 Some((head, conflict.clone()))
1009 );
1010 let end = u32::try_from(text(&cache.snapshot().unwrap()).2.encode_utf16().count()).unwrap();
1011 let dependent = save(&cache, object, end..end, " Later").unwrap();
1012 assert_ne!(head, dependent);
1013 let local = text(&cache.snapshot().unwrap()).2;
1014 assert_eq!(local, "Revised Original Later");
1015 let pending = cache.pending().unwrap();
1016 assert_eq!(pending.len(), 2);
1017 cache
1018 .export_recovery(directory.path().join("conflicted.sqlite"))
1019 .unwrap();
1020 drop(cache);
1021 let cache = Replica::open(&path).unwrap();
1022 assert_eq!(cache.status(head).unwrap(), Some(conflict.clone()));
1023 assert_eq!(cache.status(dependent).unwrap(), Some(conflict.clone()));
1024 assert_eq!(cache.pending().unwrap(), pending);
1025 let changed = onestore::replace_text(&source, space, object, 8..8, " Remote").unwrap();
1026 server.visible.clone_from(&changed);
1027 server.durable.clone_from(&changed);
911 Some((_, EditStatus::Published { .. }))
912 ));
913 assert_eq!(content(&server.durable, object), "Original");
1028914 assert_eq!(
1029 cache.sync_once(&mut server).unwrap().edit,
1030 Some((dependent, conflict.clone()))
915 conflicts(&server.durable),
916 [(
917 space,
918 vec![(model_ops::AUTHOR.to_owned(), vec!["Revised Original".to_owned()])]
919 )]
1031920 );
1032 assert_eq!(cache.remote_snapshot().unwrap(), changed);
1033 assert_eq!(server.publications, 1);
1034 cache.resolve(head, Resolution::Mine).unwrap();
1035921 drop(cache);
1036922 let cache = Replica::open(&path).unwrap();
1037 assert!(matches!(
1038 cache.sync_once(&mut server).unwrap().edit,
1039 Some((_, EditStatus::Published { .. }))
1040 ));
1041 assert_eq!(text(&server.durable).2, local);
1042923 assert!(matches!(
1043924 cache.status(head).unwrap(),
1044925 Some(EditStatus::Published { .. })
1045926 ));
1046 assert_eq!(cache.status(published).unwrap(), Some(receipt.clone()));
1047 let archive = notebook::Recovery::open(directory.path().join("conflicted.sqlite")).unwrap();
1048 assert_eq!(text(&archive.snapshot().unwrap()).2, local);
1049 assert_eq!(archive.pending().unwrap(), pending);
1050 assert_eq!(archive.status(head).unwrap(), Some(conflict.clone()));
927 assert_eq!(cache.status(published).unwrap(), Some(receipt));
928 assert!(cache.pending().unwrap().is_empty());
1051929}
1052930
1053931#[test]
crates/notebook/tests/sync/tree.rs+41-49
......@@ -34,9 +34,9 @@ fn fixture() -> (Vec<u8>, ExGuid, ExGuid, [ExGuid; 4], [ExGuid; 4]) {
3434 for at in 1..4 {
3535 anchor = model_ops::insert_after(&mut page, anchor, &format!("Sibling {at}")).1;
3636 }
37 let source = PreparedEdit::page(&source, space, &page, "Author")
37 let source = ops::saved(&source, space, &page)
3838 .unwrap()
39 .as_bytes()
39 .as_slice()
4040 .to_vec();
4141 let stored = page_of(&source, space);
4242 let siblings = &outline_of(&stored, outline).paragraphs;
......@@ -87,12 +87,12 @@ fn move_preserves_remote_content_and_dependent_edits_through_reopen() {
8787 })
8888 .unwrap()
8989 .unwrap();
90 let local = cache.snapshot().unwrap();
90 let local = snapshot(&cache);
9191 let queue = cache.pending().unwrap();
9292 drop(cache);
9393 let cache = Replica::open(&path).unwrap();
9494 assert_eq!(cache.pending().unwrap(), queue);
95 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));
95 assert_eq!(pages(&snapshot(&cache)), pages(&local));
9696 let mut server = Server::new(&remote_with(&source, space, |page| {
9797 insert_child(page, paragraphs[0], "New remote child");
9898 restyle(page, texts[1], 0..7, |format| format.bold = Some(true));
......@@ -185,14 +185,16 @@ fn native_empty_child_list_normalization_merges_with_a_local_deletion() {
185185 }
186186}
187187
188/// Deleting what the remote changed keeps the remote's content, and the local version on a
189/// conflict page; the edits that do not depend on the deletion merge.
188190#[test]
189fn deletion_requires_review_of_remote_content_and_preserves_later_work() {
191fn deletion_of_remote_content_keeps_a_conflict_page_and_merges_later_work() {
190192 let (source, space, outline, paragraphs, texts) = fixture();
191193 let mut page = page_of(&source, space);
192194 insert_child(&mut page, paragraphs[0], "Descendant");
193 let source = PreparedEdit::page(&source, space, &page, "Author")
195 let source = ops::saved(&source, space, &page)
194196 .unwrap()
195 .as_bytes()
197 .as_slice()
196198 .to_vec();
197199 let child = outline_of(&page_of(&source, space), outline).paragraphs[1]
198200 .text()
......@@ -215,24 +217,20 @@ fn deletion_requires_review_of_remote_content_and_preserves_later_work() {
215217 replace_text(page, texts[1], 0..0, "Local ");
216218 };
217219 let id = save(&cache, texts[1], change).unwrap().unwrap();
218 let queue = cache.pending().unwrap();
219 let local = cache.snapshot().unwrap();
220220 let mut server = Server::new(&remote_with(&source, space, remote));
221 assert!(conflicted(&cache, &mut server, space));
221222 assert_eq!(
222 cache.sync_once(&mut server).unwrap().edit,
223 Some((id, EditStatus::Conflict(ConflictKind::ContentChanged)))
223 text_of(&page_of(&server.durable, space), texts[1]),
224 "Local Sibling 1"
224225 );
225 assert_eq!(server.publications, 0);
226 assert_eq!(cache.pending().unwrap(), queue);
227 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));
228226 drop(cache);
229227 let cache = Replica::open(&path).unwrap();
230 assert_eq!(
231 cache.status(id).unwrap(),
232 Some(EditStatus::Conflict(ConflictKind::ContentChanged))
233 );
234 review(&cache, id, texts[1], change).unwrap();
235228 published(&cache, &mut server, id);
229 // The writer deletes the paragraph again, remote changes and all.
230 let again = save(&cache, texts[1], |page| delete_paragraph(page, texts[0]))
231 .unwrap()
232 .unwrap();
233 published(&cache, &mut server, again);
236234 assert_eq!(
237235 order(&server.durable, space, outline),
238236 paragraphs[1..].to_vec()
......@@ -247,7 +245,7 @@ fn deletion_requires_review_of_remote_content_and_preserves_later_work() {
247245#[test]
248246fn sibling_and_ancestry_changes_have_explicit_merge_or_conflict() {
249247 let (source, space, outline, paragraphs, texts) = fixture();
250 // The published order of the fixture's paragraphs, or a conflict.
248 // The published order of the fixture's paragraphs, or a conflict page.
251249 let cases: [(Change<Page>, Option<&[usize]>); 5] = [
252250 (
253251 Box::new(move |page| {
......@@ -279,14 +277,16 @@ fn sibling_and_ancestry_changes_have_explicit_merge_or_conflict() {
279277 .unwrap()
280278 .unwrap();
281279 let mut server = Server::new(&remote_with(&source, space, remote));
282 let result = cache.sync_once(&mut server).unwrap().edit.unwrap();
283 assert_eq!(result.0, id);
280 let conflict = conflicted(&cache, &mut server, space);
281 assert!(matches!(
282 cache.status(id).unwrap(),
283 Some(EditStatus::Published { .. })
284 ));
284285 let Some(expected) = expected else {
285 assert_eq!(result.1, EditStatus::Conflict(ConflictKind::ContentChanged));
286 assert_eq!(server.publications, 0);
286 assert!(conflict);
287287 continue;
288288 };
289 assert!(matches!(result.1, EditStatus::Published { .. }));
289 assert!(!conflict);
290290 let published: Vec<ExGuid> = order(&server.durable, space, outline)
291291 .into_iter()
292292 .filter(|id| paragraphs.contains(id))
......@@ -352,7 +352,7 @@ fn cell_mut(page: &mut Page, id: ExGuid) -> &mut TableCell {
352352}
353353
354354/// Moves a cell's sole paragraph to the end of another cell, or deletes it, leaving the
355/// cell an empty paragraph as the writers do.
355/// cell a new empty paragraph, as an edit that empties a cell must.
356356fn empty_cell(page: &mut Page, cell: ExGuid, destination: Option<ExGuid>) {
357357 let emptied = cell_mut(page, cell);
358358 let mut replacement = model_ops::fresh_paragraph(&emptied.paragraphs[0], "");
......@@ -419,7 +419,7 @@ fn emptied_cell_replacement_is_durable_and_cannot_be_silently_omitted_on_replay(
419419 empty_cell(page, cell, move_out.then_some(neighbour));
420420 })
421421 .unwrap();
422 let local = cache.snapshot().unwrap();
422 let local = snapshot(&cache);
423423 let queue = cache.pending().unwrap();
424424 {
425425 let page = page_of(&local, space);
......@@ -431,7 +431,7 @@ fn emptied_cell_replacement_is_durable_and_cannot_be_silently_omitted_on_replay(
431431 drop(cache);
432432 let cache = Replica::open(&path).unwrap();
433433 assert_eq!(cache.pending().unwrap(), queue);
434 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));
434 assert_eq!(pages(&snapshot(&cache)), pages(&local));
435435 let mut server = Server::new(&if competing {
436436 remote_with(source, space, |page| {
437437 let target = cell_mut(page, cell);
......@@ -574,28 +574,20 @@ fn native_tree_and_layout_changes_reconcile_without_discarding_unreviewed_conten
574574 let path = directory.path().join("native.sqlite");
575575 let cache = Replica::create(&path, BEFORE).unwrap();
576576 let id = save(&cache, dependent, save_both).unwrap().unwrap();
577 let local = cache.snapshot().unwrap();
578 let queue = cache.pending().unwrap();
579 let merged = match cache.sync_once(&mut server).unwrap().edit.unwrap() {
580 (actual, EditStatus::Published { .. }) => {
581 assert_eq!(actual, id, "{name}");
582 true
583 }
584 result => {
585 assert_eq!(
586 result,
587 (id, EditStatus::Conflict(ConflictKind::ContentChanged)),
588 "{name}"
589 );
590 assert_eq!(cache.pending().unwrap(), queue);
591 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));
592 drop(cache);
593 let cache = Replica::open(&path).unwrap();
594 review(&cache, id, dependent, save_both).unwrap();
595 published(&cache, &mut server, id);
596 false
577 let merged = !conflicted(&cache, &mut server, space);
578 assert!(matches!(
579 cache.status(id).unwrap(),
580 Some(EditStatus::Published { .. })
581 ));
582 if !merged {
583 // The dependent text merged; the writer makes the change again.
584 drop(cache);
585 let cache = Replica::open(&path).unwrap();
586 // Unless the remote made the same change.
587 if let Some(again) = save(&cache, dependent, &change).unwrap() {
588 published(&cache, &mut server, again);
597589 }
598 };
590 }
599591 let durable = page_of(&server.durable, space);
600592 assert!(
601593 text_of(&durable, dependent).starts_with("Offline "),
crates/notebook/tests/sync_formatting.rs+42-49
......@@ -1,21 +1,23 @@
11//! Formatting and insertion as page-model differences: what reconciles with an independent
2//! remote change, what conflicts, and what a reviewed conflict publishes.
2//! remote change, what keeps a conflict page, and what the edit made again publishes.
33
4use notebook::{ConflictKind, EditStatus, Replica};
4#[path = "../../onestore/tests/support/ops.rs"]
5mod ops;
6
7use notebook::{EditStatus, Replica};
58use onestore::{
6 ExGuid, PreparedEdit, RevisionIndex, Store,
9 ExGuid, RevisionIndex, Store,
710 document::{Document, Format},
811 page::{Page, PageObject},
912};
1013
1114#[path = "support/server.rs"]
1215mod server;
13use server::{Fault, Server, pages};
16use server::{Fault, Server, conflicted, pages, snapshot};
1417#[path = "support/model_ops.rs"]
1518mod model_ops;
1619use model_ops::{
17 AUTHOR, insert_after, insert_outline, page_of, paragraph_with, replace_text, restyle, review,
18 save,
20 AUTHOR, insert_after, insert_outline, page_of, paragraph_with, replace_text, restyle, save,
1921};
2022
2123const OUTLINES: &[u8] =
......@@ -80,9 +82,9 @@ fn remote_with(space: ExGuid, change: impl FnOnce(&mut Page)) -> Server {
8082 let mut page = page_of(OUTLINES, space);
8183 change(&mut page);
8284 Server::new(
83 PreparedEdit::page(OUTLINES, space, &page, "Native author")
85 ops::saved(OUTLINES, space, &page)
8486 .unwrap()
85 .as_bytes(),
87 .as_slice(),
8688 )
8789}
8890
......@@ -112,7 +114,7 @@ fn offline_formatting_merges_with_a_remote_edit_to_another_paragraph() {
112114 .all(|format| format.bold == Some(true))
113115 );
114116 assert_eq!(text_of(&published, ids[3]), "Remote Trailing sibling");
115 assert_eq!(cache.snapshot().unwrap(), server.durable);
117 assert_eq!(snapshot(&cache), server.durable);
116118}
117119
118120#[test]
......@@ -127,7 +129,7 @@ fn offline_formatting_conflicts_only_with_remote_changes_it_overlaps() {
127129 })
128130 .unwrap()
129131 .unwrap();
130 let local = cache.snapshot().unwrap();
132 let local = snapshot(&cache);
131133 let mut server = remote_with(space, |page| {
132134 if typed {
133135 replace_text(page, ids[0], 0..0, "Remote ");
......@@ -150,18 +152,18 @@ fn offline_formatting_conflicts_only_with_remote_changes_it_overlaps() {
150152 );
151153 }
152154 } else {
153 assert_eq!(
154 outcome,
155 Some((id, EditStatus::Conflict(ConflictKind::ContentChanged)))
156 );
157 assert_eq!(server.publications, 0);
158 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));
155 assert!(matches!(outcome, Some((_, EditStatus::Published { .. }))));
156 assert!(conflicted(&server.durable, space));
157 let remote = page_of(&server.durable, space);
158 assert!(formats(&remote, ids[0]).iter().all(|format| format.bold != Some(true)));
159 assert_eq!(pages(&snapshot(&cache)), pages(&server.durable));
160 assert_ne!(pages(&local), pages(&server.durable));
159161 }
160162 }
161163}
162164
163165#[test]
164fn competing_formatting_of_one_paragraph_conflicts_until_reviewed() {
166fn competing_formatting_of_one_paragraph_keeps_a_conflict_page_until_made_again() {
165167 let (space, page) = page_titled(OUTLINES, PAGE);
166168 let ids = plain(&page);
167169 let directory = tempfile::tempdir().unwrap();
......@@ -172,24 +174,22 @@ fn competing_formatting_of_one_paragraph_conflicts_until_reviewed() {
172174 })
173175 .unwrap()
174176 .unwrap();
175 let pending = cache.pending().unwrap();
176177 let mut server = remote_with(space, |page| {
177178 restyle(page, ids[0], 0..3, |format| format.italic = Some(true));
178179 });
179 for _ in 0..2 {
180 assert_eq!(
181 cache.sync_once(&mut server).unwrap().edit,
182 Some((id, EditStatus::Conflict(ConflictKind::ContentChanged)))
183 );
184 }
185 assert_eq!(cache.pending().unwrap(), pending);
180 assert!(matches!(
181 cache.sync_once(&mut server).unwrap().edit,
182 Some((_, EditStatus::Published { .. }))
183 ));
184 assert!(conflicted(&server.durable, space));
185 assert_eq!(cache.sync_once(&mut server).unwrap().edit, None);
186186 drop(cache);
187187 let cache = Replica::open(&path).unwrap();
188 assert_eq!(
188 assert!(matches!(
189189 cache.status(id).unwrap(),
190 Some(EditStatus::Conflict(ConflictKind::ContentChanged))
191 );
192 review(&cache, id, ids[0], |page| {
190 Some(EditStatus::Published { .. })
191 ));
192 save(&cache, ids[0], |page| {
193193 restyle(page, ids[0], 0..6, |format| format.font_size = Some(14.0));
194194 })
195195 .unwrap();
......@@ -287,7 +287,7 @@ fn seeded_formatting_reconciles_exactly_when_the_remote_left_the_paragraph_alone
287287 })
288288 .unwrap()
289289 .unwrap();
290 let local = cache.snapshot().unwrap();
290 let local = snapshot(&cache);
291291 let mut server = remote_with(space, |page| {
292292 for (step, &at) in remote.iter().enumerate() {
293293 if step % 2 == 0 {
......@@ -313,17 +313,10 @@ fn seeded_formatting_reconciles_exactly_when_the_remote_left_the_paragraph_alone
313313 }
314314 if underlined[target] && start == 0 {
315315 conflicts += 1;
316 assert_eq!(
317 outcome.1,
318 EditStatus::Conflict(ConflictKind::ContentChanged),
319 "seed {seed}"
320 );
321 assert_eq!(server.publications, 0, "seed {seed}");
322 assert_eq!(
323 pages(&cache.snapshot().unwrap()),
324 pages(&local),
325 "seed {seed}"
326 );
316 assert!(matches!(outcome.1, EditStatus::Published { .. }), "seed {seed}");
317 assert!(conflicted(&server.durable, space), "seed {seed}");
318 assert_eq!(page_of(&server.durable, space), remote_page, "seed {seed}");
319 assert_ne!(pages(&snapshot(&cache)), pages(&local), "seed {seed}");
327320 continue;
328321 }
329322 published += 1;
......@@ -345,7 +338,7 @@ fn seeded_formatting_reconciles_exactly_when_the_remote_left_the_paragraph_alone
345338 "seed {seed}"
346339 );
347340 }
348 assert_eq!(cache.snapshot().unwrap(), server.durable, "seed {seed}");
341 assert_eq!(snapshot(&cache), server.durable, "seed {seed}");
349342 }
350343 assert!(
351344 published >= 16,
......@@ -366,11 +359,11 @@ fn offline_insertions_survive_reopen_and_carry_their_formatting() {
366359 })
367360 .unwrap()
368361 .unwrap();
369 let local = cache.snapshot().unwrap();
362 let local = snapshot(&cache);
370363 let pending = cache.pending().unwrap();
371364 drop(cache);
372365 cache = Replica::open(&path).unwrap();
373 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));
366 assert_eq!(pages(&snapshot(&cache)), pages(&local));
374367 assert_eq!(cache.pending().unwrap(), pending);
375368 let mut server = remote_with(space, |page| replace_text(page, ids[3], 0..0, "Remote "));
376369 assert!(
......@@ -399,7 +392,7 @@ fn offline_insertions_survive_reopen_and_carry_their_formatting() {
399392 for (offset, format) in styles.iter().enumerate() {
400393 assert_eq!(format.bold == Some(true), (8..17).contains(&offset));
401394 }
402 assert_eq!(cache.snapshot().unwrap(), server.durable);
395 assert_eq!(snapshot(&cache), server.durable);
403396}
404397
405398#[test]
......@@ -415,7 +408,7 @@ fn uncertain_formatting_attempts_keep_the_original_attempt_and_never_replay() {
415408 })
416409 .unwrap()
417410 .unwrap();
418 let local = cache.snapshot().unwrap();
411 let local = snapshot(&cache);
419412 let mut server = Server::new(OUTLINES);
420413 server.fault = fault;
421414 assert!(cache.sync_once(&mut server).is_err());
......@@ -433,7 +426,7 @@ fn uncertain_formatting_attempts_keep_the_original_attempt_and_never_replay() {
433426 }
434427 assert_eq!(server.publications, 1);
435428 assert_eq!(server.durable, OUTLINES);
436 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));
429 assert_eq!(pages(&snapshot(&cache)), pages(&local));
437430 continue;
438431 }
439432 assert!(matches!(
......@@ -474,7 +467,7 @@ fn a_format_the_remote_already_carries_is_confirmed_without_republishing() {
474467 );
475468 assert_eq!((server.publications, server.confirmations), (0, 2));
476469 assert!(cache.pending().unwrap().is_empty());
477 let published = page_of(&cache.snapshot().unwrap(), space);
470 let published = page_of(&snapshot(&cache), space);
478471 assert_eq!(published, page_of(&server.durable, space));
479472 assert!(
480473 formats(&published, ids[0])
......@@ -552,7 +545,7 @@ fn paragraph_formatting_survives_reopen_rebase_and_a_lost_publication_reply() {
552545 let id = save(&cache, ids[0], |page| restyle(page, ids[0], 0..6, apply))
553546 .unwrap()
554547 .unwrap();
555 let expected = formats(&page_of(&cache.snapshot().unwrap(), space), ids[0]);
548 let expected = formats(&page_of(&snapshot(&cache), space), ids[0]);
556549 drop(cache);
557550 let cache = Replica::open(directory.path().join("cache.sqlite")).unwrap();
558551 let mut server = remote_with(space, |page| replace_text(page, ids[3], 0..0, "Remote "));
crates/notebook/tests/sync_model.rs+70-66
......@@ -1,8 +1,11 @@
11//! Page-model saves: one intent per page until it is attempted, reconciliation against the
22//! remote page, reviewed conflicts, and independence between pages.
33
4use notebook::{ConflictKind, EditStatus, Replica};
5use onestore::{ExGuid, PreparedEdit, RevisionIndex, Store, document::Document};
4#[path = "../../onestore/tests/support/ops.rs"]
5mod ops;
6
7use notebook::{EditStatus, Replica};
8use onestore::{ExGuid, RevisionIndex, Store, document::Document};
69
710#[path = "support/server.rs"]
811mod server;
......@@ -80,7 +83,7 @@ fn a_saved_page_model_publishes_and_reads_back() {
8083 .unwrap()
8184 .unwrap();
8285 assert_eq!(cache.status(id).unwrap(), Some(EditStatus::Pending));
83 assert_eq!(page_of(&cache.snapshot().unwrap(), space), after);
86 assert_eq!(page_of(&snapshot(&cache), space), after);
8487 let pending = cache.pending().unwrap();
8588 assert_eq!(pending.len(), 1);
8689 assert_eq!(pending[0].author, "Model author");
......@@ -113,7 +116,7 @@ fn saves_before_an_attempt_publish_together() {
113116 let id = model_ops::save_as(&cache, space, &after, "Model author")
114117 .unwrap()
115118 .unwrap();
116 let mut later = page_of(&cache.snapshot().unwrap(), space);
119 let mut later = page_of(&snapshot(&cache), space);
117120 append(&mut later, " two");
118121 let second = model_ops::save_as(&cache, space, &later, "Model author")
119122 .unwrap()
......@@ -133,7 +136,7 @@ fn saves_before_an_attempt_publish_together() {
133136 first_text(&page_of(&server.durable, space)),
134137 first_text(&later)
135138 );
136 let mut third = page_of(&cache.snapshot().unwrap(), space);
139 let mut third = page_of(&snapshot(&cache), space);
137140 append(&mut third, " three");
138141 let next = model_ops::save_as(&cache, space, &third, "Model author")
139142 .unwrap()
......@@ -157,7 +160,7 @@ fn an_attempted_save_is_never_rewritten() {
157160 cache.status(id).unwrap(),
158161 Some(EditStatus::AwaitingConfirmation { .. })
159162 ));
160 let mut more = page_of(&cache.snapshot().unwrap(), space);
163 let mut more = page_of(&snapshot(&cache), space);
161164 append(&mut more, " more");
162165 let next = model_ops::save_as(&cache, space, &more, "Model author")
163166 .unwrap()
......@@ -165,7 +168,7 @@ fn an_attempted_save_is_never_rewritten() {
165168 assert!(next > id);
166169 let pending = cache.pending().unwrap();
167170 assert_eq!(pending.iter().map(|p| p.id).collect::<Vec<_>>(), [id, next]);
168 assert_eq!(page_of(&cache.snapshot().unwrap(), space), more);
171 assert_eq!(page_of(&snapshot(&cache), space), more);
169172 assert!(matches!(
170173 cache.status(id).unwrap(),
171174 Some(EditStatus::AwaitingConfirmation { .. })
......@@ -173,8 +176,10 @@ fn an_attempted_save_is_never_rewritten() {
173176 assert_eq!(cache.status(next).unwrap(), Some(EditStatus::Pending));
174177}
175178
179/// A remote change to the saved text keeps the remote's page and the local one as its
180/// conflict page; the writer's change made again publishes over the remote's.
176181#[test]
177fn a_remote_change_to_the_saved_page_conflicts_until_reviewed() {
182fn a_remote_change_to_the_saved_page_keeps_a_conflict_page() {
178183 let (space, mut after) = page_titled(OUTLINES, "Move leaf down");
179184 let directory = tempfile::tempdir().unwrap();
180185 let cache = Replica::create(directory.path().join("cache.sqlite"), OUTLINES).unwrap();
......@@ -184,49 +189,37 @@ fn a_remote_change_to_the_saved_page_conflicts_until_reviewed() {
184189 .unwrap();
185190 let mut remote_page = page_of(OUTLINES, space);
186191 append(&mut remote_page, " remote");
187 let remote = PreparedEdit::page(OUTLINES, space, &remote_page, "Native author").unwrap();
188 let mut server = Server::new(remote.as_bytes());
189 assert_eq!(
192 let remote = ops::saved(OUTLINES, space, &remote_page).unwrap();
193 let mut server = Server::new(remote.as_slice());
194 assert!(matches!(
190195 cache.sync_once(&mut server).unwrap().edit,
191 Some((id, EditStatus::Conflict(ConflictKind::ContentChanged)))
196 Some((_, EditStatus::Published { .. }))
197 ));
198 assert!(matches!(
199 cache.status(id).unwrap(),
200 Some(EditStatus::Published { .. })
201 ));
202 assert_eq!(server.publications, 1);
203 let published = page_of(&server.durable, space);
204 assert_eq!(first_text(&published), first_text(&remote_page));
205 assert_eq!(
206 conflicts(&server.durable)[0].1[0].0,
207 "Model author",
208 "the conflict page names whose version it is"
192209 );
193 assert_eq!(server.publications, 0);
194 assert!(cache.pending().unwrap().len() == 1);
195 let observed = cache.remote_snapshot().unwrap();
196 assert_eq!(observed, remote.as_bytes());
197 let mut reviewed = page_of(&observed, space);
198 append(&mut reviewed, " local");
199 assert!(cache.resolve(id + 1, notebook::Resolution::Theirs).is_err());
200 let text = model_ops::paragraph_with(&reviewed, {
201 let outline = reviewed
202 .objects
203 .iter()
204 .find_map(|object| match object {
205 PageObject::Outline(outline) => Some(outline),
206 _ => None,
207 })
208 .unwrap();
209 outline.paragraphs[0].text().unwrap().id
210 })
211 .unwrap()
212 .text()
213 .unwrap()
214 .id;
215 let reviewed_id = model_ops::review(&cache, id, text, |page| append(page, " local"))
210 let text = published
211 .objects
212 .iter()
213 .find_map(|object| match object {
214 PageObject::Outline(outline) => outline.paragraphs[0].text().map(|text| text.id),
215 _ => None,
216 })
217 .unwrap();
218 let again = model_ops::save(&cache, text, |page| append(page, " local"))
216219 .unwrap()
217220 .unwrap();
218 // The conflicted edit stays, emptied, and publishes with the reviewed one.
219 assert_eq!(cache.status(id).unwrap(), Some(EditStatus::Pending));
220 assert_eq!(
221 cache.status(reviewed_id).unwrap(),
222 Some(EditStatus::Pending)
223 );
224221 assert!(
225 matches!(cache.sync_once(&mut server).unwrap().edit, Some((n, EditStatus::Published { .. })) if n == reviewed_id)
226 );
227 assert_eq!(
228 first_text(&page_of(&server.durable, space)),
229 first_text(&reviewed)
222 matches!(cache.sync_once(&mut server).unwrap().edit, Some((n, EditStatus::Published { .. })) if n == again)
230223 );
231224 assert!(first_text(&page_of(&server.durable, space)).ends_with(" remote local"));
232225}
......@@ -242,14 +235,14 @@ fn remote_changes_to_other_pages_do_not_conflict() {
242235 .unwrap()
243236 .unwrap();
244237 append(&mut other_page, " elsewhere");
245 let remote = PreparedEdit::page(OUTLINES, other, &other_page, "Native author").unwrap();
246 let mut server = Server::new(remote.as_bytes());
238 let remote = ops::saved(OUTLINES, other, &other_page).unwrap();
239 let mut server = Server::new(remote.as_slice());
247240 assert!(
248241 matches!(cache.sync_once(&mut server).unwrap().edit, Some((n, EditStatus::Published { .. })) if n == id)
249242 );
250243 assert_eq!(page_of(&server.durable, space), after);
251244 assert_eq!(page_of(&server.durable, other), other_page);
252 assert_eq!(page_of(&cache.snapshot().unwrap(), other), other_page);
245 assert_eq!(page_of(&snapshot(&cache), other), other_page);
253246}
254247
255248#[test]
......@@ -267,7 +260,7 @@ fn pending_saves_survive_reopening_the_cache() {
267260 let reopened = Replica::open(&path).unwrap();
268261 assert_eq!(reopened.pending().unwrap(), pending);
269262 assert_eq!(reopened.status(id).unwrap(), Some(EditStatus::Pending));
270 assert_eq!(page_of(&reopened.snapshot().unwrap(), space), after);
263 assert_eq!(page_of(&snapshot(&reopened), space), after);
271264}
272265
273266/// Edits paragraph `index` of the first body outline: replaces its text range with `replacement`.
......@@ -309,11 +302,7 @@ fn texts(page: &Page) -> Vec<String> {
309302fn remote_with(space: ExGuid, change: impl FnOnce(&mut Page)) -> Server {
310303 let mut page = page_of(OUTLINES, space);
311304 change(&mut page);
312 Server::new(
313 PreparedEdit::page(OUTLINES, space, &page, "Native author")
314 .unwrap()
315 .as_bytes(),
316 )
305 Server::new(ops::saved(OUTLINES, space, &page).unwrap().as_slice())
317306}
318307
319308#[test]
......@@ -366,11 +355,16 @@ fn concurrent_edits_to_one_paragraph_merge_unless_their_ranges_overlap() {
366355 .unwrap()
367356 .unwrap();
368357 let mut server = remote_with(space, |page| edit_paragraph(page, 0, 0..6, "Remote"));
369 assert_eq!(
358 assert!(matches!(
370359 cache.sync_once(&mut server).unwrap().edit,
371 Some((id, EditStatus::Conflict(ConflictKind::ContentChanged)))
372 );
373 assert_eq!(server.publications, 0);
360 Some((_, EditStatus::Published { .. }))
361 ));
362 assert!(matches!(
363 cache.status(id).unwrap(),
364 Some(EditStatus::Published { .. })
365 ));
366 assert!(conflicted(&server.durable, space));
367 assert_eq!(server.publications, 1);
374368}
375369
376370#[test]
......@@ -428,7 +422,7 @@ fn a_local_insertion_merges_with_a_remote_deletion_elsewhere() {
428422}
429423
430424#[test]
431fn a_remote_deletion_of_the_edited_paragraph_conflicts() {
425fn a_remote_deletion_of_the_edited_paragraph_keeps_a_conflict_page() {
432426 let (space, mut after) = page_titled(OUTLINES, "Move leaf down");
433427 let directory = tempfile::tempdir().unwrap();
434428 let cache = Replica::create(directory.path().join("cache.sqlite"), OUTLINES).unwrap();
......@@ -447,10 +441,15 @@ fn a_remote_deletion_of_the_edited_paragraph_conflicts() {
447441 .unwrap();
448442 outline.paragraphs.pop();
449443 });
450 assert_eq!(
444 assert!(matches!(
451445 cache.sync_once(&mut server).unwrap().edit,
452 Some((id, EditStatus::Conflict(ConflictKind::ContentChanged)))
453 );
446 Some((_, EditStatus::Published { .. }))
447 ));
448 assert!(matches!(
449 cache.status(id).unwrap(),
450 Some(EditStatus::Published { .. })
451 ));
452 assert!(conflicted(&server.durable, space));
454453}
455454
456455#[test]
......@@ -515,10 +514,15 @@ fn outline_moves_merge_with_remote_text_edits_but_not_with_remote_moves() {
515514 };
516515 outline.layout.x = Some(50.0);
517516 });
518 assert_eq!(
517 assert!(matches!(
519518 cache.sync_once(&mut server).unwrap().edit,
520 Some((id, EditStatus::Conflict(ConflictKind::ContentChanged)))
521 );
519 Some((_, EditStatus::Published { .. }))
520 ));
521 assert!(matches!(
522 cache.status(id).unwrap(),
523 Some(EditStatus::Published { .. })
524 ));
525 assert!(conflicted(&server.durable, space));
522526}
523527
524528#[test]
crates/notebook/tests/sync_outline.rs+82-73
......@@ -1,9 +1,12 @@
11//! Outline layout through page-model saves: reconciliation against the remote page,
22//! review of competing layout, twelve offline writers, and the native outline fixtures.
33
4use notebook::{ConflictKind, EditStatus, Replica};
4#[path = "../../onestore/tests/support/ops.rs"]
5mod ops;
6
7use notebook::{EditStatus, Replica};
58use onestore::{
6 ExGuid, PreparedEdit, RevisionIndex, Store,
9 ExGuid, RevisionIndex, Store,
710 document::Document,
811 page::{Outline, Page, PageObject, PageParagraph},
912};
......@@ -11,12 +14,12 @@ use serde_json::json;
1114
1215#[path = "support/server.rs"]
1316mod server;
14use server::{Fault, Server, pages};
17use server::{Fault, Server, conflicts, pages, snapshot};
1518#[path = "support/model_ops.rs"]
1619mod model_ops;
1720use model_ops::{
1821 delete_paragraph, insert_outline, outlines_mut, page_of, paragraph_with, replace_text, restyle,
19 review, save,
22 save,
2023};
2124#[path = "support/model_schedule.rs"]
2225mod model_schedule;
......@@ -99,12 +102,30 @@ pub(crate) fn text_of(page: &Page, text: ExGuid) -> String {
99102pub(crate) fn remote_with(source: &[u8], space: ExGuid, change: impl FnOnce(&mut Page)) -> Vec<u8> {
100103 let mut page = page_of(source, space);
101104 change(&mut page);
102 PreparedEdit::page(source, space, &page, "Native author")
105 ops::saved(source, space, &page)
103106 .unwrap()
104 .as_bytes()
107 .as_slice()
105108 .to_vec()
106109}
107110
111/// Runs one synchronization step, which publishes; true when the remote's version of
112/// `space` stayed and the local one became a conflict page under it.
113pub(crate) fn conflicted(cache: &Replica, server: &mut Server, space: ExGuid) -> bool {
114 let count = |image: &[u8]| {
115 conflicts(image)
116 .into_iter()
117 .filter(|(page, _)| *page == space)
118 .map(|(_, pages)| pages.len())
119 .sum::<usize>()
120 };
121 let before = count(&server.durable);
122 assert!(matches!(
123 cache.sync_once(server).unwrap().edit,
124 Some((_, EditStatus::Published { .. }))
125 ));
126 count(&server.durable) > before
127}
128
108129/// Publishes the batch holding `id`, unless an earlier step already did.
109130pub(crate) fn published(cache: &Replica, server: &mut Server, id: u64) {
110131 if !matches!(
......@@ -149,13 +170,13 @@ fn layout_and_dependent_text_survive_reopen_and_a_remote_format_of_the_same_page
149170 let path = directory.path().join("cache.sqlite");
150171 let cache = Replica::create(&path, &source).unwrap();
151172 let id = save(&cache, text_id, &change).unwrap().unwrap();
152 let expected = page_of(&cache.snapshot().unwrap(), space);
173 let expected = page_of(&snapshot(&cache), space);
153174 let pending = cache.pending().unwrap();
154 let local = cache.snapshot().unwrap();
175 let local = snapshot(&cache);
155176 drop(cache);
156177 let cache = Replica::open(&path).unwrap();
157178 assert_eq!(cache.pending().unwrap(), pending);
158 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));
179 assert_eq!(pages(&snapshot(&cache)), pages(&local));
159180 let mut server = Server::new(&remote_with(&source, space, |page| {
160181 restyle(page, text_id, 0..8, |format| format.bold = Some(true));
161182 }));
......@@ -208,9 +229,9 @@ fn an_uncertain_move_confirms_from_the_remote_revision_and_adopts_the_remote_tit
208229 let (source, space, outline, _) = fixture();
209230 let mut page = page_of(&source, space);
210231 insert_outline(&mut page, 144.0, 36.0, "Second");
211 let source = PreparedEdit::page(&source, space, &page, "Author")
232 let source = ops::saved(&source, space, &page)
212233 .unwrap()
213 .as_bytes()
234 .as_slice()
214235 .to_vec();
215236 let second = text_with(&page_of(&source, space), "Second");
216237 let directory = tempfile::tempdir().unwrap();
......@@ -242,8 +263,9 @@ fn an_uncertain_move_confirms_from_the_remote_revision_and_adopts_the_remote_tit
242263 assert_eq!((server.publications, server.confirmations), (1, 1));
243264}
244265
266/// Competing layout keeps the remote's on the page and the local one on its conflict page.
245267#[test]
246fn competing_layout_conflicts_until_the_local_page_is_kept() {
268fn competing_layout_keeps_the_local_one_on_a_conflict_page() {
247269 let (source, space, outline, text_id) = fixture();
248270 let cases: [(Change<Outline>, Change<Outline>); 2] = [
249271 (Box::new(moved(180.0, 216.0)), Box::new(moved(288.0, 216.0))),
......@@ -256,46 +278,40 @@ fn competing_layout_conflicts_until_the_local_page_is_kept() {
256278 let directory = tempfile::tempdir().unwrap();
257279 let path = directory.path().join("cache.sqlite");
258280 let cache = Replica::create(&path, &source).unwrap();
259 let id = save(&cache, text_id, |page| {
281 save(&cache, text_id, |page| {
260282 local_change(outline_mut(page, outline))
261283 })
262284 .unwrap()
263285 .unwrap();
264 let local = cache.snapshot().unwrap();
265 let mut server = Server::new(&remote_with(&source, space, |page| {
286 let local = page_of(&snapshot(&cache), space);
287 let remote = remote_with(&source, space, |page| {
266288 remote_change(outline_mut(page, outline));
267 }));
289 });
290 let mut server = Server::new(&remote);
291 assert!(conflicted(&cache, &mut server, space));
292 assert_eq!(server.publications, 1);
293 let durable = page_of(&server.durable, space);
268294 assert_eq!(
269 cache.sync_once(&mut server).unwrap().edit,
270 Some((id, EditStatus::Conflict(ConflictKind::ContentChanged)))
295 outline_of(&durable, outline).layout,
296 outline_of(&page_of(&remote, space), outline).layout
271297 );
272 assert_eq!(server.publications, 0);
273 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));
298 let arena = onestore::Arena::default();
299 let mut section = onestore::Section::open(&arena, server.durable.clone()).unwrap();
300 let conflict = section.conflicts().unwrap()[0].1[0].space;
301 let kept = section.page(conflict).unwrap();
302 assert_eq!(body_outlines(&kept)[0].layout, outline_of(&local, outline).layout);
274303 let dependent = save(&cache, text_id, |page| {
275304 replace_text(page, text_id, 0..0, "Local ")
276305 })
277306 .unwrap()
278307 .unwrap();
279 let local = cache.snapshot().unwrap();
280 drop(cache);
281 let cache = Replica::open(&path).unwrap();
282 assert_eq!(
283 cache.status(id).unwrap(),
284 Some(EditStatus::Conflict(ConflictKind::ContentChanged))
285 );
286 cache.resolve(id, notebook::Resolution::Mine).unwrap();
287 assert_eq!(cache.status(id).unwrap(), Some(EditStatus::Pending));
288308 drop(cache);
289309 let cache = Replica::open(&path).unwrap();
290 for id in [id, dependent] {
291 published(&cache, &mut server, id);
292 }
293 let durable = page_of(&server.durable, space);
310 published(&cache, &mut server, dependent);
294311 assert_eq!(
295 outline_of(&durable, outline).layout,
296 outline_of(&page_of(&local, space), outline).layout
312 text_of(&page_of(&server.durable, space), text_id),
313 "Local Original 🦀 é"
297314 );
298 assert_eq!(text_of(&durable, text_id), "Local Original 🦀 é");
299315 }
300316}
301317
......@@ -346,15 +362,14 @@ fn twelve_offline_writers_preserve_all_layout_intents_through_review_and_restart
346362 paragraph_mut(page, text_id).collapsed = (actor + round) % 2 != 0;
347363 };
348364 let id = save(&cache, text_id, change).unwrap().unwrap();
349 let expected = page_of(&cache.snapshot().unwrap(), space);
365 let expected = page_of(&snapshot(&cache), space);
350366 drop(cache);
351367 let cache = Replica::open(&path).unwrap();
352 if cache.sync_once(&mut server).unwrap().edit
353 == Some((id, EditStatus::Conflict(ConflictKind::ContentChanged)))
354 {
355 review(&cache, id, text_id, change).unwrap();
368 if conflicted(&cache, &mut server, space) {
369 // The writer makes the change again over the remote's version.
370 let again = save(&cache, text_id, change).unwrap().unwrap();
356371 reviewed += 1;
357 published(&cache, &mut server, id);
372 published(&cache, &mut server, again);
358373 }
359374 assert!(matches!(
360375 cache.status(id).unwrap(),
......@@ -388,7 +403,7 @@ fn an_uncertain_layout_attempt_confirms_by_revision_or_by_an_equal_remote_page()
388403 })
389404 .unwrap()
390405 .unwrap();
391 let local = cache.snapshot().unwrap();
406 let local = snapshot(&cache);
392407 let mut server = Server::new(&source);
393408 server.fault = fault;
394409 assert!(cache.sync_once(&mut server).is_err());
......@@ -416,7 +431,7 @@ fn an_uncertain_layout_attempt_confirms_by_revision_or_by_an_equal_remote_page()
416431 cache.sync_once(&mut server).unwrap().edit,
417432 Some((id, status.clone()))
418433 );
419 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));
434 assert_eq!(pages(&snapshot(&cache)), pages(&local));
420435 server.visible = remote_with(&source, space, |page| {
421436 resized(144.0, true)(outline_mut(page, outline));
422437 });
......@@ -483,7 +498,7 @@ pub(crate) fn native_pages(source: &[u8]) -> Vec<(String, ExGuid, ExGuid, ExGuid
483498}
484499
485500#[test]
486fn native_moves_deletions_and_layout_changes_merge_or_require_review() {
501fn native_moves_deletions_and_layout_changes_merge_or_keep_a_conflict_page() {
487502 let mut server = Server::new(NATIVE);
488503 let mut records = Vec::new();
489504 for (name, space, outline, target, dependent) in native_pages(BEFORE) {
......@@ -496,28 +511,20 @@ fn native_moves_deletions_and_layout_changes_merge_or_require_review() {
496511 let path = directory.path().join("cache.sqlite");
497512 let cache = Replica::create(&path, BEFORE).unwrap();
498513 let id = save(&cache, dependent, save_both).unwrap().unwrap();
499 let local = cache.snapshot().unwrap();
500 let queue = cache.pending().unwrap();
501 let merged = match cache.sync_once(&mut server).unwrap().edit.unwrap() {
502 (actual, EditStatus::Published { .. }) => {
503 assert_eq!(actual, id, "{name}");
504 true
505 }
506 result => {
507 assert_eq!(
508 result,
509 (id, EditStatus::Conflict(ConflictKind::ContentChanged)),
510 "{name}"
511 );
512 assert_eq!(cache.pending().unwrap(), queue);
513 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));
514 drop(cache);
515 let cache = Replica::open(&path).unwrap();
516 review(&cache, id, dependent, save_both).unwrap();
517 published(&cache, &mut server, id);
518 false
514 let merged = !conflicted(&cache, &mut server, space);
515 assert!(matches!(
516 cache.status(id).unwrap(),
517 Some(EditStatus::Published { .. })
518 ));
519 if !merged {
520 // The dependent text merged; the writer makes the change again.
521 drop(cache);
522 let cache = Replica::open(&path).unwrap();
523 // Unless the remote made the same change.
524 if let Some(again) = save(&cache, dependent, &change).unwrap() {
525 published(&cache, &mut server, again);
519526 }
520 };
527 }
521528 let durable = page_of(&server.durable, space);
522529 assert!(
523530 text_of(&durable, dependent).starts_with("Local "),
......@@ -564,7 +571,7 @@ fn native_moves_deletions_and_layout_changes_merge_or_require_review() {
564571}
565572
566573#[test]
567fn a_new_native_wrap_reservation_requires_review_before_width_replacement() {
574fn a_new_native_wrap_reservation_keeps_the_local_width_on_a_conflict_page() {
568575 let (_, space, outline, _, text_id) = native_pages(BEFORE)
569576 .into_iter()
570577 .find(|(name, ..)| name == "Move outline")
......@@ -584,15 +591,17 @@ fn a_new_native_wrap_reservation_requires_review_before_width_replacement() {
584591 .unwrap()
585592 .unwrap();
586593 let mut server = Server::new(NATIVE);
587 assert_eq!(
588 cache.sync_once(&mut server).unwrap().edit,
589 Some((id, EditStatus::Conflict(ConflictKind::ContentChanged)))
590 );
591 review(&cache, id, text_id, |page| {
594 assert!(conflicted(&cache, &mut server, space));
595 assert!(matches!(
596 cache.status(id).unwrap(),
597 Some(EditStatus::Published { .. })
598 ));
599 let again = save(&cache, text_id, |page| {
592600 resized(144.0, true)(outline_mut(page, outline))
593601 })
602 .unwrap()
594603 .unwrap();
595 published(&cache, &mut server, id);
604 published(&cache, &mut server, again);
596605 let layout = outline_of(&page_of(&server.durable, space), outline)
597606 .layout
598607 .clone();
crates/notebook/tests/sync_page.rs+85-92
......@@ -1,7 +1,10 @@
1//! Page creation reconciliation: dependent body saves, uncertain publication, anchor review,
1//! Page creation reconciliation: dependent body saves, uncertain publication, lost anchors,
22//! and deterministic multi-actor schedules.
33
4use notebook::{ConflictKind, EditStatus, Recovery, Replica, Resolution};
4#[path = "../../onestore/tests/support/ops.rs"]
5mod ops;
6
7use notebook::{EditStatus, Recovery, Replica};
58use onestore::op::SectionOp;
69use onestore::{
710 ExGuid, PageCreation, RevisionIndex, Store,
......@@ -67,11 +70,11 @@ fn offline_page_creation_rebases_with_dependent_edits_and_duplicate_titles() {
6770 assert!(
6871 matches!(&original[0].edit.ops[..], [onestore::op::Op::Section(SectionOp::Create(retained))] if retained == &page)
6972 );
70 let local = cache.snapshot().unwrap();
73 let local = snapshot(&cache);
7174 drop(cache);
7275 let cache = Replica::open(&path).unwrap();
7376 assert_eq!(cache.pending().unwrap(), original);
74 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));
77 assert_eq!(pages(&snapshot(&cache)), pages(&local));
7578 assert!(matches!(
7679 cache.sync_once(&mut server).unwrap().edit,
7780 Some((published, EditStatus::Published { .. })) if published == dependent
......@@ -120,7 +123,7 @@ fn uncertain_page_publication_retains_both_revisions_and_never_replays() {
120123 let cache = Replica::create(&path, &source).unwrap();
121124 let page = PageCreation::new(None, Some("Created"), "Author").unwrap();
122125 let id = section_op(&cache, SectionOp::Create(page.clone()));
123 let local = cache.snapshot().unwrap();
126 let local = snapshot(&cache);
124127 let mut server = Server::new(&source);
125128 server.fault = fault;
126129 let _ = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
......@@ -155,12 +158,12 @@ fn uncertain_page_publication_retains_both_revisions_and_never_replays() {
155158 assert_eq!(server.confirmations, usize::from(visible));
156159 if visible {
157160 assert!(matches!(result.1, EditStatus::Published { .. }));
158 assert!(cache.snapshot().unwrap() == observed);
161 assert!(snapshot(&cache) == observed);
159162 assert!(observed[..212] == server.durable[..212]);
160163 assert!(observed[252..] == server.durable[252..]);
161164 } else {
162165 assert_eq!(result, (id, attempted.clone()));
163 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));
166 assert_eq!(pages(&snapshot(&cache)), pages(&local));
164167 }
165168 }
166169}
......@@ -235,8 +238,10 @@ fn surviving_page_revision_alone_does_not_confirm_section_publication() {
235238 assert_eq!(server.publications, 1);
236239}
237240
241/// A page created before a page the remote moved goes before the next page the remote left
242/// in place, its content under the identities it was made with.
238243#[test]
239fn changed_page_anchor_conflicts_until_kept_without_regenerating_dependent_identities() {
244fn a_moved_page_anchor_places_the_page_without_regenerating_dependent_identities() {
240245 let source = include_bytes!("../../../corpus/page-lifecycle/03-renamed/notebook/Lifecycle.one");
241246 let remote = include_bytes!("../../../corpus/page-lifecycle/04-nested/notebook/Lifecycle.one");
242247 let store = Store::parse(source).unwrap();
......@@ -252,30 +257,29 @@ fn changed_page_anchor_conflicts_until_kept_without_regenerating_dependent_ident
252257 let save = model_ops::save_as(&cache, page.space(), &created, "Author")
253258 .unwrap()
254259 .unwrap();
255 let local = cache.snapshot().unwrap();
256 let pending = cache.pending().unwrap();
260 let local = snapshot(&cache);
257261 let mut server = Server::new(remote);
258 assert_eq!(
262 assert!(matches!(
259263 cache.sync_once(&mut server).unwrap().edit,
260 Some((save, EditStatus::Conflict(ConflictKind::StructureChanged)))
261 );
262 cache.resolve(id, Resolution::Mine).unwrap();
263 assert_eq!(cache.pending().unwrap()[1], pending[1]);
264 Some((published, EditStatus::Published { .. })) if published == save
265 ));
264266 assert!(matches!(
265 &cache.pending().unwrap()[0].edit.ops[..],
266 [onestore::op::Op::Section(SectionOp::Create(reviewed))] if *reviewed == page.reposition(None).unwrap()
267 cache.status(id).unwrap(),
268 Some(EditStatus::Published { .. })
267269 ));
270 assert_eq!(server.publications, 1);
268271 assert_eq!(
269 model_ops::page_of(&cache.snapshot().unwrap(), page.space()),
272 model_ops::page_of(&server.durable, page.space()),
270273 model_ops::page_of(&local, page.space())
271274 );
272 drop(cache);
273 let cache = Replica::open(&path).unwrap();
274 assert!(matches!(
275 cache.sync_once(&mut server).unwrap().edit,
276 Some((_, EditStatus::Published { .. }))
277 ));
278 assert_eq!(server.publications, 1);
275 let arena = onestore::Arena::default();
276 let listed = onestore::Section::open(&arena, server.durable.clone())
277 .unwrap()
278 .pages()
279 .unwrap();
280 // The remote made `child` and `grandchild` subpages; `Renamed` stayed.
281 let at = listed.iter().position(|listed| listed.0 == pages[6].0).unwrap();
282 assert_eq!(listed[at - 1].0, page.space());
279283 let store = Store::parse(&server.durable).unwrap();
280284 let index = RevisionIndex::parse(&store).unwrap();
281285 let document = Document::parse(&index).unwrap();
......@@ -285,8 +289,9 @@ fn changed_page_anchor_conflicts_until_kept_without_regenerating_dependent_ident
285289 Kind::RichText { text, .. } if text == "Retained body"));
286290}
287291
292/// Edits of a page the remote removed put the local version back where it stood.
288293#[test]
289fn keeping_mine_puts_back_a_page_the_remote_removed_where_it_stood() {
294fn a_page_the_remote_removed_comes_back_where_it_stood() {
290295 let source = include_bytes!("../../../corpus/page-lifecycle/04-nested/notebook/Lifecycle.one");
291296 let order = |image: &[u8]| -> Vec<(ExGuid, String, u32)> {
292297 let arena = onestore::Arena::default();
......@@ -319,73 +324,61 @@ fn keeping_mine_puts_back_a_page_the_remote_removed_where_it_stood() {
319324 _ => None,
320325 })
321326 .unwrap();
322 for keep in [Resolution::Mine, Resolution::Theirs] {
323 let directory = tempfile::tempdir().unwrap();
324 let cache = Replica::create(directory.path().join("pages.sqlite"), source).unwrap();
325 let id = model_ops::save(&cache, text, |page| {
326 model_ops::replace_text(page, text, 0..0, "Kept ")
327 })
327 let directory = tempfile::tempdir().unwrap();
328 let cache = Replica::create(directory.path().join("pages.sqlite"), source).unwrap();
329 model_ops::save(&cache, text, |page| {
330 model_ops::replace_text(page, text, 0..0, "Kept ")
331 })
332 .unwrap()
333 .unwrap();
334 let local = cache.page(space).unwrap();
335 let removed = ops::section_op(source, SectionOp::Delete([space].to_vec()))
328336 .unwrap()
329 .unwrap();
330 let local = cache.page(space).unwrap();
331 let removed = onestore::PreparedEdit::delete_pages_permanently(source, &[space])
332 .unwrap()
333 .as_bytes()
334 .to_vec();
335 let mut server = Server::new(&removed);
336 assert_eq!(
337 cache.sync_once(&mut server).unwrap().edit,
338 Some((id, EditStatus::Conflict(ConflictKind::TargetUnavailable)))
339 );
340 assert_eq!(
341 server.publications, 0,
342 "a removed page is not edited in place"
343 );
344 cache.resolve(id, keep).unwrap();
345 assert!(matches!(
346 cache.sync_once(&mut server).unwrap().edit,
347 Some((_, EditStatus::Published { .. }))
348 ));
349 let published = order(&server.durable);
350 assert!(published.iter().all(|(page, ..)| *page != space));
351 match keep {
352 Resolution::Mine => {
353 assert_eq!(published.len(), pages.len());
354 let (restored, title, restored_level) = published[at].clone();
355 assert_eq!(
356 (title.as_str(), restored_level),
357 (pages[at].1.as_str(), level)
358 );
359 let page = model_ops::page_of(&server.durable, restored);
360 let texts = |page: &onestore::page::Page| -> Vec<String> {
361 page.objects
362 .iter()
363 .filter_map(|object| match object {
364 onestore::page::PageObject::Outline(outline) => Some(outline),
365 _ => None,
366 })
367 .flat_map(|outline| &outline.paragraphs)
368 .filter_map(|p| p.text().map(|t| t.text.text().to_owned()))
369 .collect()
370 };
371 assert_eq!(texts(&page), texts(&local));
372 assert!(texts(&page)[0].starts_with("Kept "));
373 let others: Vec<_> = published
374 .iter()
375 .map(|(page, ..)| *page)
376 .filter(|page| *page != restored)
377 .collect();
378 let before: Vec<_> = pages
379 .iter()
380 .map(|(page, ..)| *page)
381 .filter(|page| *page != space)
382 .collect();
383 assert_eq!(others, before);
384 }
385 Resolution::Theirs => assert_eq!(published, order(&removed)),
386 }
387 assert_eq!(pages_of(&cache), published);
388 }
337 .as_bytes()
338 .to_vec();
339 let mut server = Server::new(&removed);
340 assert!(matches!(
341 cache.sync_once(&mut server).unwrap().edit,
342 Some((_, EditStatus::Published { .. }))
343 ));
344 assert_eq!(
345 server.publications, 1,
346 "a removed page is not edited in place"
347 );
348 let published = order(&server.durable);
349 assert!(published.iter().all(|(page, ..)| *page != space));
350 assert_eq!(published.len(), pages.len());
351 let (restored, title, restored_level) = published[at].clone();
352 assert_eq!(
353 (title.as_str(), restored_level),
354 (pages[at].1.as_str(), level)
355 );
356 let page = model_ops::page_of(&server.durable, restored);
357 let texts = |page: &onestore::page::Page| -> Vec<String> {
358 page.objects
359 .iter()
360 .filter_map(|object| match object {
361 onestore::page::PageObject::Outline(outline) => Some(outline),
362 _ => None,
363 })
364 .flat_map(|outline| &outline.paragraphs)
365 .filter_map(|p| p.text().map(|t| t.text.text().to_owned()))
366 .collect()
367 };
368 assert_eq!(texts(&page), texts(&local));
369 assert!(texts(&page)[0].starts_with("Kept "));
370 let others: Vec<_> = published
371 .iter()
372 .map(|(page, ..)| *page)
373 .filter(|page| *page != restored)
374 .collect();
375 let before: Vec<_> = pages
376 .iter()
377 .map(|(page, ..)| *page)
378 .filter(|page| *page != space)
379 .collect();
380 assert_eq!(others, before);
381 assert_eq!(pages_of(&cache), published);
389382}
390383
391384fn pages_of(cache: &Replica) -> Vec<(ExGuid, String, u32)> {
crates/notebook/tests/sync_pages.rs+165-116
......@@ -1,10 +1,14 @@
11//! Page batch reconciliation: atomic batches with dependent body saves, competing moves and
2//! indentation, uncertain batches, and native page movement fixtures.
2//! indentation (a page the remote moved keeps the remote's placement), uncertain batches, and
3//! native page movement fixtures.
34
4use notebook::{ConflictKind, EditStatus, Recovery, Replica, Resolution};
5#[path = "../../onestore/tests/support/ops.rs"]
6mod ops;
7
8use notebook::{EditStatus, Recovery, Replica};
59use onestore::op::{Op, SectionOp};
610use onestore::{
7 ExGuid, PageEdit, PagePosition, PreparedEdit, RevisionIndex, Store,
11 ExGuid, PageEdit, PagePosition, RevisionIndex, Store,
812 document::{Document, Format, Kind, Layout},
913 page::{
1014 Outline, Page, PageObject, PageParagraph, Paragraph, ParagraphContent, TextObject,
......@@ -135,12 +139,12 @@ fn atomic_page_batches_survive_reopen_with_dependent_text() {
135139 assert!(
136140 matches!(&queue[0].edit.ops[..], [Op::Section(SectionOp::Pages(batch))] if *batch == edits)
137141 );
138 let local = cache.snapshot().unwrap();
142 let local = snapshot(&cache);
139143 drop(cache);
140144 let cache = Replica::open(&path).unwrap();
141145 assert_eq!(cache.pending().unwrap(), queue);
142146 assert_eq!(
143 server::pages(&cache.snapshot().unwrap()),
147 server::pages(&snapshot(&cache)),
144148 server::pages(&local)
145149 );
146150 let mut server = Server::new(SOURCE);
......@@ -164,7 +168,7 @@ fn atomic_page_batches_survive_reopen_with_dependent_text() {
164168}
165169
166170#[test]
167fn indentation_only_edits_preserve_remote_page_movement() {
171fn indenting_a_page_the_remote_moved_keeps_the_remote_placement() {
168172 let directory = tempfile::tempdir().unwrap();
169173 let cache = Replica::create(directory.path().join("pages.sqlite"), SOURCE).unwrap();
170174 let before = order(SOURCE);
......@@ -174,86 +178,56 @@ fn indentation_only_edits_preserve_remote_page_movement() {
174178 SectionOp::Pages([PageEdit::set_level(sid, 3).unwrap()].to_vec()),
175179 );
176180 let mut server = Server::new(SOURCE);
177 PreparedEdit::pages(
178 SOURCE,
179 &[PageEdit::move_to(sid, Some(before[7].0), 2).unwrap()],
180 )
181 ops::section_op(SOURCE, SectionOp::Pages([PageEdit::move_to(sid, Some(before[7].0), 2).unwrap()].to_vec()))
181182 .unwrap()
182183 .commit(&mut server)
183184 .unwrap();
184 let mut expected = order(&server.durable);
185 expected.iter_mut().find(|p| p.0 == sid).unwrap().1 = 3;
185 let expected = order(&server.durable);
186186 assert!(matches!(cache.sync_once(&mut server).unwrap().edit,
187187 Some((published, EditStatus::Published { .. })) if published == id));
188188 assert_eq!(order(&server.durable), expected);
189189}
190190
191/// A page both sides moved stays where the remote put it, as OneNote 2010 keeps it
192/// (corpus/conflict-page/native-pages).
191193#[test]
192fn competing_page_moves_conflict_until_mine_is_kept_with_its_identities() {
194fn competing_page_moves_keep_the_remote_placement() {
193195 let directory = tempfile::tempdir().unwrap();
194196 let cache = Replica::create(directory.path().join("pages.sqlite"), SOURCE).unwrap();
195197 let pages = order(SOURCE);
196198 let sid = pages[6].0;
197 let original = PageEdit::move_to(sid, None, 1).unwrap();
198 let id = section_op(&cache, SectionOp::Pages(vec![original.clone()]));
199 let id = section_op(&cache, SectionOp::Pages(vec![PageEdit::move_to(sid, None, 1).unwrap()]));
199200 let mut server = Server::new(SOURCE);
200 PreparedEdit::pages(
201 SOURCE,
202 &[PageEdit::move_to(sid, Some(pages[0].0), 1).unwrap()],
203 )
201 ops::section_op(SOURCE, SectionOp::Pages([PageEdit::move_to(sid, Some(pages[0].0), 1).unwrap()].to_vec()))
204202 .unwrap()
205203 .commit(&mut server)
206204 .unwrap();
207 let remote = server.visible.clone();
208 let local = cache.snapshot().unwrap();
209 assert_eq!(
210 cache.sync_once(&mut server).unwrap().edit,
211 Some((id, EditStatus::Conflict(ConflictKind::StructureChanged)))
212 );
213 assert_eq!(server.publications, 0);
214 assert_eq!(
215 server::pages(&cache.snapshot().unwrap()),
216 server::pages(&local)
217 );
218 // Keeping mine moves the page last again, from where the remote put it.
219 cache.resolve(id, Resolution::Mine).unwrap();
220 assert!(matches!(&cache.pending().unwrap()[0].edit.ops[..],
221 [Op::Section(SectionOp::Pages(batch))] if *batch == [original.clone()]));
222 let expected = PreparedEdit::pages(&remote, &[original]).unwrap();
205 let expected = order(&server.visible);
223206 assert!(matches!(cache.sync_once(&mut server).unwrap().edit,
224207 Some((published, EditStatus::Published { .. })) if published == id));
225 assert_eq!(order(&server.durable), order(expected.as_bytes()));
208 assert_eq!(order(&server.durable), expected);
226209}
227210
228211#[test]
229fn one_competing_level_prevents_publication_of_the_whole_batch() {
212fn a_competing_level_keeps_the_remote_level_and_the_rest_of_the_batch() {
230213 let directory = tempfile::tempdir().unwrap();
231214 let cache = Replica::create(directory.path().join("pages.sqlite"), SOURCE).unwrap();
232215 let pages = order(SOURCE);
233 let id = section_op(
234 &cache,
235 SectionOp::Pages(vec![
236 PageEdit::set_level(pages[4].0, 3).unwrap(),
237 PageEdit::set_level(pages[5].0, 2).unwrap(),
238 ]),
239 );
240 let local = cache.snapshot().unwrap();
216 let edits = vec![
217 PageEdit::set_level(pages[4].0, 3).unwrap(),
218 PageEdit::set_level(pages[5].0, 2).unwrap(),
219 ];
220 let id = section_op(&cache, SectionOp::Pages(edits.clone()));
241221 let mut server = Server::new(SOURCE);
242 PreparedEdit::pages(SOURCE, &[PageEdit::set_level(pages[5].0, 1).unwrap()])
222 ops::section_op(SOURCE, SectionOp::Pages([PageEdit::set_level(pages[5].0, 1).unwrap()].to_vec()))
243223 .unwrap()
244224 .commit(&mut server)
245225 .unwrap();
246 let remote = server.durable.clone();
247 assert_eq!(
248 cache.sync_once(&mut server).unwrap().edit,
249 Some((id, EditStatus::Conflict(ConflictKind::StructureChanged)))
250 );
251 assert_eq!(server.publications, 0);
252 assert_eq!(server.durable, remote);
253 assert_eq!(
254 server::pages(&cache.snapshot().unwrap()),
255 server::pages(&local)
256 );
226 let expected =
227 ops::section_op(&server.durable.clone(), SectionOp::Pages(edits[..1].to_vec())).unwrap();
228 assert!(matches!(cache.sync_once(&mut server).unwrap().edit,
229 Some((published, EditStatus::Published { .. })) if published == id));
230 assert_eq!(order(&server.durable), order(expected.as_bytes()));
257231}
258232
259233#[test]
......@@ -273,7 +247,7 @@ fn uncertain_page_batches_survive_recovery_and_do_not_replay() {
273247 PageEdit::set_level(pages[5].0, 2).unwrap(),
274248 ];
275249 let id = section_op(&cache, SectionOp::Pages(edits.to_vec()));
276 let local = cache.snapshot().unwrap();
250 let local = snapshot(&cache);
277251 let queue = cache.pending().unwrap();
278252 let mut server = Server::new(SOURCE);
279253 server.fault = fault;
......@@ -293,7 +267,6 @@ fn uncertain_page_batches_survive_recovery_and_do_not_replay() {
293267 server::pages(&recovery.snapshot().unwrap()),
294268 server::pages(&local)
295269 );
296 assert!(cache.resolve(id, Resolution::Mine).is_err());
297270 let result = cache.sync_once(&mut server).unwrap().edit.unwrap();
298271 assert_eq!(server.publications, 1);
299272 if matches!(fault, Fault::UnknownAfter | Fault::PanicAfter) {
......@@ -319,9 +292,23 @@ fn an_unchanged_section_revision_cannot_confirm_a_changed_page() {
319292 let copy = *view.nodes.iter().find(|(_, node)|
320293 matches!(&node.kind, Kind::Metadata { title: Some(title), .. } if title == "child"))
321294 .unwrap().0;
322 let source =
323 onestore::replace_property_bytes(SOURCE, root, copy, 0x14001dff, &3_u32.to_le_bytes())
324 .unwrap();
295 // The section's copy of that page's metadata says level 3, stored where it lies.
296 let revision = index.resolve_active(root).unwrap();
297 let onestore::ObjectData::Properties(data) = revision.objects[&copy].data else {
298 panic!()
299 };
300 let properties = onestore::PropertySets::parse(data).unwrap();
301 let onestore::Value::Bytes(level) = properties.sets[0]
302 .iter()
303 .find(|property| property.id == 0x14001dff)
304 .unwrap()
305 .value
306 else {
307 panic!()
308 };
309 let at = level.as_ptr().addr() - SOURCE.as_ptr().addr();
310 let mut source = SOURCE.to_vec();
311 source[at..at + 4].copy_from_slice(&3_u32.to_le_bytes());
325312 let directory = tempfile::tempdir().unwrap();
326313 let path = directory.path().join("pages.sqlite");
327314 let cache = Replica::create(&path, &source).unwrap();
......@@ -368,7 +355,7 @@ fn an_unchanged_section_revision_cannot_confirm_a_changed_page() {
368355}
369356
370357#[test]
371fn an_explicit_anchor_is_retained_even_when_originally_in_place() {
358fn an_explicit_anchor_yields_to_a_remote_move() {
372359 let directory = tempfile::tempdir().unwrap();
373360 let cache = Replica::create(directory.path().join("pages.sqlite"), SOURCE).unwrap();
374361 let pages = order(SOURCE);
......@@ -380,18 +367,18 @@ fn an_explicit_anchor_is_retained_even_when_originally_in_place() {
380367 ]),
381368 );
382369 let mut server = Server::new(SOURCE);
383 PreparedEdit::pages(
384 SOURCE,
385 &[PageEdit::move_to(pages[6].0, Some(pages[0].0), 1).unwrap()],
386 )
370 ops::section_op(SOURCE, SectionOp::Pages([PageEdit::move_to(pages[6].0, Some(pages[0].0), 1).unwrap()].to_vec()))
387371 .unwrap()
388372 .commit(&mut server)
389373 .unwrap();
390 assert_eq!(
391 cache.sync_once(&mut server).unwrap().edit,
392 Some((id, EditStatus::Conflict(ConflictKind::StructureChanged)))
393 );
394 assert_eq!(server.publications, 0);
374 let expected = ops::section_op(
375 &server.durable.clone(),
376 SectionOp::Pages(vec![PageEdit::set_level(pages[4].0, 3).unwrap()]),
377 )
378 .unwrap();
379 assert!(matches!(cache.sync_once(&mut server).unwrap().edit,
380 Some((published, EditStatus::Published { .. })) if published == id));
381 assert_eq!(order(&server.durable), order(expected.as_bytes()));
395382}
396383
397384#[test]
......@@ -404,7 +391,7 @@ fn convergent_page_indentation_requires_confirmation_without_republishing() {
404391 SectionOp::Pages([PageEdit::set_level(sid, 1).unwrap()].to_vec()),
405392 );
406393 let mut server = Server::new(SOURCE);
407 PreparedEdit::pages(SOURCE, &[PageEdit::set_level(sid, 1).unwrap()])
394 ops::section_op(SOURCE, SectionOp::Pages([PageEdit::set_level(sid, 1).unwrap()].to_vec()))
408395 .unwrap()
409396 .commit(&mut server)
410397 .unwrap();
......@@ -418,7 +405,7 @@ fn convergent_page_indentation_requires_confirmation_without_republishing() {
418405 assert_eq!(server.confirmations, 2);
419406}
420407#[test]
421fn native_page_changes_reconcile_with_atomic_offline_batches_and_review() {
408fn native_page_changes_reconcile_with_atomic_offline_batches() {
422409 let root = std::path::Path::new(env!("CARGO_MANIFEST_DIR"))
423410 .join("../../corpus/page-lifecycle/movement");
424411 let provenance: serde_json::Value =
......@@ -455,48 +442,36 @@ fn native_page_changes_reconcile_with_atomic_offline_batches_and_review() {
455442 .unwrap();
456443 let native = std::fs::read(root.join(phase).join("notebook/Lifecycle.one")).unwrap();
457444 let mut server = Server::new(&native);
458 let local = cache.snapshot().unwrap();
459 let expected_conflict = if mode == "indent" {
460 matches!(
461 phase.as_str(),
462 "04-subpage-move" | "07-promoted-root" | "08-collapsed-group-move"
463 )
464 } else {
465 !matches!(
466 phase.as_str(),
467 "02-promoted-parent" | "03-selected-group-move" | "06-promoted-level-two"
468 )
469 };
470 let first = if expected_conflict {
471 cache.sync_once(&mut server).unwrap().edit.unwrap()
472 } else {
473 (text_id, EditStatus::Pending)
445 // The page list takes the edits of the pages OneNote left in their series, at
446 // their level and after the same page.
447 let both: Vec<ExGuid> = order(&native)
448 .into_iter()
449 .map(|(sid, _)| sid)
450 .filter(|sid| original.iter().any(|(s, _)| s == sid))
451 .collect();
452 let placement = |image: &[u8]| -> BTreeMap<ExGuid, (ExGuid, Option<ExGuid>, u32)> {
453 let arena = onestore::Arena::default();
454 let series = onestore::Section::open(&arena, image.to_vec()).unwrap().series().unwrap();
455 let shared: Vec<_> = order(image).into_iter().filter(|(sid, _)| both.contains(sid)).collect();
456 (0..shared.len())
457 .map(|at| {
458 let (sid, level) = shared[at];
459 (sid, (series[&sid], at.checked_sub(1).map(|b| shared[b].0), level))
460 })
461 .collect()
474462 };
475 let mut reviewed = edits.clone();
476 if expected_conflict {
477 assert_eq!(
478 first.1,
479 EditStatus::Conflict(ConflictKind::StructureChanged),
480 "{mode}:{phase}"
481 );
482 assert_eq!(server.publications, 0);
483 assert_eq!(
484 server::pages(&cache.snapshot().unwrap()),
485 server::pages(&local)
486 );
487 // Keeping mine drops the moves that no longer apply.
488 cache.resolve(id, Resolution::Mine).unwrap();
489 reviewed = match &cache.pending().unwrap()[0].edit.ops[..] {
490 [Op::Section(SectionOp::Pages(kept))] => kept.clone(),
491 [] => Vec::new(),
492 other => panic!("{other:?}"),
493 };
494 }
463 let (before, after) = (placement(SOURCE), placement(&native));
464 let reviewed: Vec<PageEdit> = edits
465 .iter()
466 .filter(|edit| after.get(&edit.space()) == before.get(&edit.space()))
467 .cloned()
468 .collect();
469 let trimmed = reviewed != edits;
495470 assert!(
496471 matches!(cache.sync_once(&mut server).unwrap().edit, Some((published, EditStatus::Published { .. })) if published == text_id),
497472 "{mode}:{phase}"
498473 );
499 counts[usize::from(expected_conflict)] += 1;
474 counts[usize::from(trimmed)] += 1;
500475 drop(cache);
501476 let cache = Replica::open(&path).unwrap();
502477 assert!(cache.pending().unwrap().is_empty());
......@@ -525,7 +500,7 @@ fn native_page_changes_reconcile_with_atomic_offline_batches_and_review() {
525500 path.join("manifest.json"),
526501 serde_json::to_vec_pretty(
527502 &serde_json::json!({"mode": mode, "phase": phase, "original": edits,
528 "reviewed": reviewed, "conflict": expected_conflict,
503 "reviewed": reviewed, "conflict": trimmed,
529504 "publications": server.publications, "confirmations": server.confirmations,
530505 "page_receipt": format!("{:?}", cache.status(id).unwrap()),
531506 "text_receipt": format!("{:?}", cache.status(text_id).unwrap())}),
......@@ -536,7 +511,7 @@ fn native_page_changes_reconcile_with_atomic_offline_batches_and_review() {
536511 }
537512 }
538513 }
539 assert_eq!(counts, [9, 9]);
514 assert_eq!(counts.iter().sum::<usize>(), 18);
540515}
541516
542517#[test]
......@@ -545,7 +520,7 @@ fn twelve_disjoint_page_batches_merge_with_dependent_bodies_without_review() {
545520 let mut created = Vec::new();
546521 for _ in 0..36 {
547522 let page = onestore::PageCreation::new(None, Some("Same title"), "Author").unwrap();
548 source = PreparedEdit::create_page(&source, &page)
523 source = ops::section_op(&source, SectionOp::Create(page.clone()))
549524 .unwrap()
550525 .as_bytes()
551526 .to_vec();
......@@ -564,7 +539,7 @@ fn twelve_disjoint_page_batches_merge_with_dependent_bodies_without_review() {
564539 ];
565540 section_op(&cache, SectionOp::Pages(edits.to_vec()));
566541 let body = format!("Actor {actor} 🦀 e\u{301}");
567 let local = cache.snapshot().unwrap();
542 let local = snapshot(&cache);
568543 let mut model = model_ops::page_of(&local, group[1].space());
569544 let text = body_outline(&mut model, &body);
570545 model_ops::save_as(&cache, group[1].space(), &model, "Author")
......@@ -594,7 +569,7 @@ fn twelve_disjoint_page_batches_merge_with_dependent_bodies_without_review() {
594569 for (path, queue) in &queues {
595570 let cache = Replica::open(path).unwrap();
596571 assert_eq!(cache.sync_once(&mut server).unwrap().edit, None);
597 assert_eq!(order(&cache.snapshot().unwrap()), expected);
572 assert_eq!(order(&snapshot(&cache)), expected);
598573 for edit in queue {
599574 assert!(matches!(
600575 cache.status(edit.id).unwrap(),
......@@ -627,3 +602,77 @@ fn twelve_disjoint_page_batches_merge_with_dependent_bodies_without_review() {
627602 .unwrap();
628603 }
629604}
605
606/// Two OneNote 2010 clients' page merges (`corpus/conflict-page/native-*`), replayed: client
607/// A, offline, `moves` pages (each last, or before another) and edits `Target`'s body while
608/// client B publishes `b-published`. The pages the merge lists after the first, and
609/// `Target`'s space and texts.
610fn native_page_merge(capture: &str, moves: &[(&str, Option<&str>)]) -> (Vec<String>, ExGuid, Vec<String>) {
611 let root = std::path::Path::new(env!("CARGO_MANIFEST_DIR"))
612 .join("../../corpus/conflict-page")
613 .join(capture);
614 let read = |phase: &str| std::fs::read(root.join(phase).join("notebook/synthetic.one")).unwrap();
615 let pages = |image: &[u8]| {
616 let arena = onestore::Arena::default();
617 onestore::Section::open(&arena, image.to_vec()).unwrap().pages().unwrap()
618 };
619 let initial = read("initial");
620 let listed = pages(&initial);
621 let space = |title: &str| listed.iter().find(|page| page.1 == title).unwrap().0;
622 let directory = tempfile::tempdir().unwrap();
623 let cache = Replica::create(directory.path().join("pages.sqlite"), &initial).unwrap();
624 for (page, before) in moves {
625 section_op(
626 &cache,
627 SectionOp::Pages(vec![PageEdit::move_to(space(page), before.map(space), 1).unwrap()]),
628 );
629 }
630 let body = texts(&initial)
631 .into_iter()
632 .find(|(_, text)| text == "Body Target.")
633 .unwrap()
634 .0
635 .1;
636 model_ops::save(&cache, body, |page| {
637 model_ops::replace_text(page, body, 0..12, "Body Target edited offline.")
638 })
639 .unwrap()
640 .unwrap();
641 let mut server = Server::new(&read("b-published"));
642 for _ in 0..moves.len() + 2 {
643 cache.sync_once(&mut server).unwrap();
644 }
645 assert!(cache.pending().unwrap().is_empty(), "{capture}");
646 let merged = pages(&server.durable);
647 let target = merged.iter().find(|page| page.1 == "Target").unwrap().0;
648 let texts = texts(&server.durable)
649 .into_iter()
650 .filter(|((page, _), _)| *page == target)
651 .map(|(_, text)| text)
652 .collect();
653 (merged[1..].iter().map(|page| page.1.clone()).collect(), target, texts)
654}
655
656/// Each side keeps the pages it gave a series of their own, the remote's where both did;
657/// a page the remote deleted comes back as a new page where the queue had it. OneNote's
658/// COM moves re-series the pages a page jumps over, so its moves replay as moves of those.
659#[test]
660fn page_moves_and_a_removed_page_merge_as_onenote_merges_them() {
661 for (capture, moves) in [
662 ("native-pages", &[("One", Some("Target")), ("Three", Some("Target")), ("Two", None)][..]),
663 ("native-restore", &[("One", None), ("Two", None), ("Target", None), ("Three", None)][..]),
664 ("native-restore", &[("Four", Some("One"))][..]),
665 ] {
666 let root = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../../corpus/conflict-page");
667 let native: serde_json::Value =
668 serde_json::from_slice(&std::fs::read(root.join(capture).join("merged.json")).unwrap()).unwrap();
669 let (order, target, texts) = native_page_merge(capture, moves);
670 let native: Vec<String> = serde_json::from_value(native["order"].clone()).unwrap();
671 assert_eq!(order, native[1..], "{capture}");
672 assert!(texts.contains(&"Body Target edited offline.".to_owned()), "{capture}");
673 let arena = onestore::Arena::default();
674 let initial = std::fs::read(root.join(capture).join("initial/notebook/synthetic.one")).unwrap();
675 let listed = onestore::Section::open(&arena, initial).unwrap().pages().unwrap();
676 assert!(listed.iter().all(|page| page.0 != target), "{capture}: a new page");
677 }
678}
crates/notebook/tests/sync_paragraph.rs+89-69
......@@ -1,9 +1,12 @@
11//! Paragraph splits and joins expressed as page-model differences: reconciliation against
22//! native OneNote captures, dependent edits, uncertain attempts and competing clients.
33
4use notebook::{ConflictKind, EditStatus, Replica};
4#[path = "../../onestore/tests/support/ops.rs"]
5mod ops;
6
7use notebook::{EditStatus, Replica};
58use onestore::{
6 ExGuid, PreparedEdit, RevisionIndex, Store,
9 ExGuid, RevisionIndex, Store,
710 document::Document,
811 page::{Page, PageObject, PageParagraph, ParagraphContent, TextObject, text::new_id},
912};
......@@ -11,7 +14,7 @@ use std::path::Path;
1114
1215#[path = "support/server.rs"]
1316mod server;
14use server::{Fault, Server, pages};
17use server::{Fault, Server, conflicts, pages, snapshot, typed};
1518#[path = "support/model_ops.rs"]
1619mod model_ops;
1720use model_ops::{AUTHOR, outlines_mut, page_of, replace_text};
......@@ -175,18 +178,27 @@ fn save_page(
175178 text: ExGuid,
176179 edit: impl FnOnce(&mut Page),
177180) -> Result<Option<u64>, notebook::Error> {
178 let source = cache.snapshot()?;
181 let source = snapshot(cache);
179182 let (space, mut page) = holder(&source, text);
180183 edit(&mut page);
181184 model_ops::save_as(cache, space, &page, AUTHOR)
182185}
183186
184/// Resolves a conflict by taking the remote page, then saves `edit` of it.
185fn review_page(cache: &Replica, id: u64, text: ExGuid, edit: impl FnOnce(&mut Page)) {
186 cache.resolve(id, notebook::Resolution::Theirs).unwrap();
187/// Makes `edit` again on the page as the remote's version left it, as a writer does after
188/// a conflict page kept the first try.
189fn redo_page(cache: &Replica, text: ExGuid, edit: impl FnOnce(&mut Page)) {
187190 save_page(cache, text, edit).unwrap().unwrap();
188191}
189192
193/// The conflict pages `image` lists under page `space`.
194fn conflict_pages(image: &[u8], space: ExGuid) -> usize {
195 conflicts(image)
196 .into_iter()
197 .filter(|(page, _)| *page == space)
198 .map(|(_, pages)| pages.len())
199 .sum()
200}
201
190202/// The styled text of an object anywhere on a page.
191203fn content(page: &Page, text: ExGuid) -> onestore::page::Paragraph {
192204 locate_in(&mut page.clone(), text, |list, index| {
......@@ -250,12 +262,15 @@ fn reconcile_controls() -> (Vec<u8>, Vec<serde_json::Value>) {
250262 drop(cache);
251263 cache = Replica::open(&path).unwrap();
252264 assert_eq!(cache.pending().unwrap(), pending);
253 let (actual, status) = cache.sync_once(&mut server).unwrap().edit.unwrap();
254 assert_eq!(actual, id);
255 let reviewed = matches!(status, EditStatus::Conflict(_));
265 let kept = conflict_pages(&server.visible, space);
266 let (_, status) = cache.sync_once(&mut server).unwrap().edit.unwrap();
267 assert!(matches!(
268 cache.status(id).unwrap(),
269 Some(EditStatus::Published { .. })
270 ));
271 let reviewed = conflict_pages(&server.visible, space) > kept;
256272 let receipt = if reviewed {
257 assert_eq!(cache.pending().unwrap(), pending);
258 review_page(&cache, id, left, apply);
273 redo_page(&cache, left, apply);
259274 cache.sync_once(&mut server).unwrap().edit.unwrap().1
260275 } else {
261276 automatic += 1;
......@@ -301,7 +316,7 @@ fn reconcile_controls() -> (Vec<u8>, Vec<serde_json::Value>) {
301316 panic!("{name} dependent: {status:?}")
302317 };
303318 assert!(cache.pending().unwrap().is_empty());
304 assert_eq!(cache.snapshot().unwrap(), server.durable);
319 assert_eq!(snapshot(&cache), server.durable);
305320 recorded.push(
306321 serde_json::json!({"case": name, "space": space, "intent": intent,
307322 "outcome": if reviewed { "reviewed" } else { "automatic" },
......@@ -309,12 +324,13 @@ fn reconcile_controls() -> (Vec<u8>, Vec<serde_json::Value>) {
309324 );
310325 }
311326 assert_eq!((recorded.len(), automatic), (16, 8));
312 assert_eq!(server.publications, 32);
327 // Each edit and its dependent, and each conflict page before its edit made again.
328 assert_eq!(server.publications, 32 + 16 - automatic);
313329 (server.durable, recorded)
314330}
315331
316332#[test]
317fn native_keyboard_edits_merge_with_offline_splits_and_joins_or_survive_review() {
333fn native_keyboard_edits_merge_with_offline_splits_and_joins_or_keep_a_conflict_page() {
318334 let (durable, cases) = reconcile_controls();
319335 let expected = [
320336 ("Split remote prefix", "automatic", vec!["Xab", "🦀Cd"]),
......@@ -349,8 +365,9 @@ fn native_keyboard_edits_merge_with_offline_splits_and_joins_or_survive_review()
349365}
350366
351367#[test]
352fn a_native_text_object_replacement_conflicts_with_every_offline_split_and_join() {
368fn a_native_text_object_replacement_keeps_a_conflict_page_for_every_offline_split_and_join() {
353369 let mut server = Server::new(REPLACED);
370 let mut count = 0;
354371 for (space, name, split) in controls() {
355372 let original = page_of(BEFORE, space);
356373 let left = body(&original)[0].text().unwrap().id;
......@@ -366,24 +383,23 @@ fn a_native_text_object_replacement_conflicts_with_every_offline_split_and_join(
366383 let id = model_ops::save_as(&cache, space, &after, AUTHOR)
367384 .unwrap()
368385 .unwrap();
369 let local = cache.snapshot().unwrap();
370 let pending = cache.pending().unwrap();
371 assert_eq!(
372 cache.sync_once(&mut server).unwrap().edit,
373 Some((id, EditStatus::Conflict(ConflictKind::ContentChanged))),
386 let remote = page_of(&server.visible, space);
387 assert!(
388 matches!(
389 cache.sync_once(&mut server).unwrap().edit,
390 Some((_, EditStatus::Published { .. }))
391 ),
374392 "{name}"
375393 );
376 drop(cache);
377 let cache = Replica::open(&path).unwrap();
378 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));
379 assert_eq!(cache.pending().unwrap(), pending);
380 assert_eq!(
381 cache.status(id).unwrap(),
382 Some(EditStatus::Conflict(ConflictKind::ContentChanged))
394 assert!(
395 matches!(cache.status(id).unwrap(), Some(EditStatus::Published { .. })),
396 "{name}"
383397 );
398 assert_eq!(page_of(&server.visible, space), remote, "{name}");
399 assert_eq!(conflict_pages(&server.visible, space), 1, "{name}");
400 count += 1;
384401 }
385 assert_eq!(server.publications, 0);
386 assert_eq!(server.visible, REPLACED);
402 assert_eq!(server.publications, count);
387403}
388404
389405/// Replays a recorded native split or join intent as a page-model difference.
......@@ -459,7 +475,7 @@ fn offline_paragraphs(output: Option<&Path>) {
459475 {
460476 let intent = &case["intent"];
461477 let left = origin(intent, split);
462 let (space, page) = holder(&cache.snapshot().unwrap(), left);
478 let (space, page) = holder(&snapshot(&cache), left);
463479 let offset = if split {
464480 intent["offset"].as_u64().unwrap()
465481 } else {
......@@ -468,7 +484,7 @@ fn offline_paragraphs(output: Option<&Path>) {
468484 let prefixed = offset > 0 && !content(&page, left).text().is_empty();
469485 if prefixed {
470486 server.visible =
471 onestore::replace_text(&server.visible, space, left, 0..0, "\u{2602}").unwrap();
487 typed(&server.visible, space, left, 0..0, "\u{2602}");
472488 server.durable.clone_from(&server.visible);
473489 }
474490 let Ok(queued) = save_page(&cache, left, |page| replay(page, intent, split)) else {
......@@ -484,6 +500,8 @@ fn offline_paragraphs(output: Option<&Path>) {
484500 let case = entry["case"].as_str().unwrap().to_owned();
485501 let intent = entry["intent"].clone();
486502 let mut uncertain = id % 2 == 0;
503 let mut kept = conflict_pages(&server.visible, space);
504 let mut id = id;
487505 let revision = loop {
488506 if uncertain {
489507 server.fault = Fault::UnknownAfter;
......@@ -501,7 +519,7 @@ fn offline_paragraphs(output: Option<&Path>) {
501519 Ok(notebook::Synced {
502520 edit: Some((_, EditStatus::Published { revision })),
503521 ..
504 }) => {
522 }) if conflict_pages(&server.visible, space) == kept => {
505523 assert!(matches!(
506524 cache.status(id).unwrap(),
507525 Some(EditStatus::Published { .. })
......@@ -509,20 +527,21 @@ fn offline_paragraphs(output: Option<&Path>) {
509527 break revision;
510528 }
511529 Ok(notebook::Synced {
512 edit: Some((actual, EditStatus::Conflict(_))),
530 edit: Some((_, EditStatus::Published { .. })),
513531 ..
514532 }) => {
515 assert_eq!(actual, id);
533 kept += 1;
516534 server.fault = Fault::None;
517535 reviewed.push(case.clone());
518 // The reviewed split sits after the remote's one-unit prefix.
536 // The split made again sits after the remote's one-unit prefix.
519537 let mut placed = intent.clone();
520538 if split && entry["remote_prefix"] == true {
521539 placed["offset"] = (intent["offset"].as_u64().unwrap() + 1).into();
522540 }
523 review_page(&cache, id, origin(&intent, split), |page| {
541 redo_page(&cache, origin(&intent, split), |page| {
524542 replay(page, &placed, split)
525543 });
544 id = cache.pending().unwrap().last().unwrap().id;
526545 }
527546 other => panic!("{name} {case}: {other:?}"),
528547 }
......@@ -543,7 +562,7 @@ fn offline_paragraphs(output: Option<&Path>) {
543562 );
544563 assert_eq!(server.publications, entries.len());
545564 assert_eq!(cache.sync_once(&mut server).unwrap().edit, None);
546 assert_eq!(cache.snapshot().unwrap(), server.durable);
565 assert_eq!(snapshot(&cache), server.durable);
547566 if output.is_some() {
548567 cache
549568 .export_recovery(folder.join("recovery.sqlite"))
......@@ -573,9 +592,9 @@ fn remote_with(space: ExGuid, change: impl FnOnce(&mut Page)) -> Server {
573592 let mut page = page_of(BEFORE, space);
574593 change(&mut page);
575594 Server::new(
576 PreparedEdit::page(BEFORE, space, &page, "Native author")
595 ops::saved(BEFORE, space, &page)
577596 .unwrap()
578 .as_bytes(),
597 .as_slice(),
579598 )
580599}
581600
......@@ -611,7 +630,7 @@ fn a_split_publishes_two_paragraphs_and_a_join_puts_them_back() {
611630 ["ab\u{1f980}cd", "Right", "Preserved sibling"]
612631 );
613632 assert_eq!(server.publications, 2);
614 assert_eq!(cache.snapshot().unwrap(), server.durable);
633 assert_eq!(snapshot(&cache), server.durable);
615634}
616635
617636#[test]
......@@ -637,7 +656,7 @@ fn a_split_merges_with_a_remote_edit_to_another_paragraph() {
637656}
638657
639658#[test]
640fn a_join_conflicts_when_the_remote_changed_the_paragraph_it_removes() {
659fn a_join_keeps_a_conflict_page_when_the_remote_changed_the_paragraph_it_removes() {
641660 let (space, page) = control("Join remote prefix");
642661 let (left, right) = (
643662 body(&page)[0].text().unwrap().id,
......@@ -649,12 +668,16 @@ fn a_join_conflicts_when_the_remote_changed_the_paragraph_it_removes() {
649668 .unwrap()
650669 .unwrap();
651670 let mut server = remote_with(space, |page| replace_text(page, right, 0..0, "Remote "));
652 assert_eq!(
671 assert!(matches!(
653672 cache.sync_once(&mut server).unwrap().edit,
654 Some((id, EditStatus::Conflict(ConflictKind::ContentChanged)))
655 );
656 assert_eq!(server.publications, 0);
657 review_page(&cache, id, left, |page| join_paragraphs(page, left, right));
673 Some((_, EditStatus::Published { .. }))
674 ));
675 assert!(matches!(
676 cache.status(id).unwrap(),
677 Some(EditStatus::Published { .. })
678 ));
679 assert_eq!(conflict_pages(&server.durable, space), 1);
680 redo_page(&cache, left, |page| join_paragraphs(page, left, right));
658681 assert!(matches!(
659682 cache.sync_once(&mut server).unwrap().edit,
660683 Some((_, EditStatus::Published { .. }))
......@@ -680,7 +703,7 @@ fn edits_dependent_on_a_published_split_survive_reopen() {
680703 .unwrap()
681704 .unwrap();
682705 cache.sync_once(&mut server).unwrap().edit.unwrap();
683 let tail = body(&page_of(&cache.snapshot().unwrap(), space))[1]
706 let tail = body(&page_of(&snapshot(&cache), space))[1]
684707 .text()
685708 .unwrap()
686709 .id;
......@@ -689,17 +712,17 @@ fn edits_dependent_on_a_published_split_survive_reopen() {
689712 })
690713 .unwrap()
691714 .unwrap();
692 let local = cache.snapshot().unwrap();
715 let local = snapshot(&cache);
693716 let pending = cache.pending().unwrap();
694717 drop(cache);
695718 cache = Replica::open(&path).unwrap();
696 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));
719 assert_eq!(pages(&snapshot(&cache)), pages(&local));
697720 assert_eq!(cache.pending().unwrap(), pending);
698721 let mut remote = page_of(&server.durable, space);
699722 replace_text(&mut remote, other, 0..0, "Remote ");
700 server.visible = PreparedEdit::page(&server.durable, space, &remote, "Native author")
723 server.visible = ops::saved(&server.durable, space, &remote)
701724 .unwrap()
702 .as_bytes()
725 .as_slice()
703726 .to_vec();
704727 server.durable.clone_from(&server.visible);
705728 assert!(
......@@ -742,7 +765,7 @@ fn uncertain_splits_and_joins_keep_the_original_attempt_across_reopen() {
742765 })
743766 .unwrap()
744767 .unwrap();
745 let local = cache.snapshot().unwrap();
768 let local = snapshot(&cache);
746769 let pending = cache.pending().unwrap();
747770 let mut server = Server::new(BEFORE);
748771 server.fault = fault;
......@@ -764,7 +787,7 @@ fn uncertain_splits_and_joins_keep_the_original_attempt_across_reopen() {
764787 Fault::UnknownBefore => {
765788 assert!(matches!(state, EditStatus::AwaitingConfirmation { .. }));
766789 assert_eq!(cache.pending().unwrap(), pending);
767 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));
790 assert_eq!(pages(&snapshot(&cache)), pages(&local));
768791 for _ in 0..3 {
769792 assert_eq!(
770793 cache.sync_once(&mut server).unwrap().edit,
......@@ -777,7 +800,7 @@ fn uncertain_splits_and_joins_keep_the_original_attempt_across_reopen() {
777800 }
778801 Fault::UnknownAfter => {
779802 assert!(matches!(state, EditStatus::AwaitingConfirmation { .. }));
780 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));
803 assert_eq!(pages(&snapshot(&cache)), pages(&local));
781804 assert!(matches!(
782805 cache.sync_once(&mut server).unwrap().edit.unwrap().1,
783806 EditStatus::Published { .. }
......@@ -797,13 +820,13 @@ fn uncertain_splits_and_joins_keep_the_original_attempt_across_reopen() {
797820 };
798821 assert_eq!(texts(&page_of(&server.durable, space)), expected);
799822 assert_eq!(cache.sync_once(&mut server).unwrap().edit, None);
800 assert_eq!(cache.snapshot().unwrap(), server.durable);
823 assert_eq!(snapshot(&cache), server.durable);
801824 }
802825 }
803826}
804827
805828#[test]
806fn a_competing_join_leaves_the_other_client_a_reviewable_split() {
829fn a_competing_join_leaves_the_other_client_its_split_on_a_conflict_page() {
807830 let (space, page) = control("Join remote prefix");
808831 let (left, right) = (
809832 body(&page)[0].text().unwrap().id,
......@@ -819,27 +842,24 @@ fn a_competing_join_leaves_the_other_client_a_reviewable_split() {
819842 let id = save_page(&second, right, |page| split_paragraph(page, right, 2))
820843 .unwrap()
821844 .unwrap();
822 let local = second.snapshot().unwrap();
823 let pending = second.pending().unwrap();
824845 let mut server = Server::new(BEFORE);
825846 assert!(matches!(
826847 first.sync_once(&mut server).unwrap().edit.unwrap().1,
827848 EditStatus::Published { .. }
828849 ));
829 assert_eq!(
850 assert!(matches!(
830851 second.sync_once(&mut server).unwrap().edit,
831 Some((id, EditStatus::Conflict(ConflictKind::ContentChanged)))
832 );
833 assert_eq!(server.publications, 1);
852 Some((_, EditStatus::Published { .. }))
853 ));
854 assert_eq!(server.publications, 2);
855 assert_eq!(conflict_pages(&server.durable, space), 1);
834856 drop(second);
835857 let second = Replica::open(&path).unwrap();
836 assert_eq!(pages(&second.snapshot().unwrap()), pages(&local));
837 assert_eq!(second.pending().unwrap(), pending);
838 assert_eq!(
858 assert!(matches!(
839859 second.status(id).unwrap(),
840 Some(EditStatus::Conflict(ConflictKind::ContentChanged))
841 );
842 review_page(&second, id, left, |page| {
860 Some(EditStatus::Published { .. })
861 ));
862 redo_page(&second, left, |page| {
843863 split_paragraph(page, left, 2);
844864 });
845865 assert!(matches!(
crates/notebook/tests/sync_worker.rs+116-115
......@@ -1,6 +1,7 @@
11//! The background worker: waking, backoff, ownership and error propagation.
22
3use notebook::{ConflictKind, EditStatus, Error, Remote, Replica};
3use notebook::{EditStatus, Error, Remote, Replica};
4use onestore::Stamp;
45use onestore::{
56 CommitError, CommitState, ExGuid, RevisionIndex, Store, Transaction, document::Document,
67};
......@@ -25,14 +26,14 @@ impl Remote for Shared {
2526 fn read(&mut self) -> io::Result<Vec<u8>> {
2627 self.0.lock().unwrap().read()
2728 }
28 fn stamp(&mut self) -> io::Result<Option<onestore::Stamp>> {
29 fn stamp(&mut self) -> io::Result<Stamp> {
2930 self.0.lock().unwrap().stamp()
3031 }
3132 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
3233 self.0.lock().unwrap().publish(transaction)
3334 }
34 fn confirm(&mut self, snapshot: &[u8]) -> Result<(), CommitError> {
35 self.0.lock().unwrap().confirm(snapshot)
35 fn confirm(&mut self, base: &Stamp) -> Result<(), CommitError> {
36 self.0.lock().unwrap().confirm(base)
3637 }
3738}
3839
......@@ -75,11 +76,17 @@ fn reconnects_after_connect_read_and_uncertain_publish_without_replaying() {
7576 }
7677 self.shared.read()
7778 }
79 fn stamp(&mut self) -> io::Result<Stamp> {
80 if self.fail_read {
81 return Err(io::ErrorKind::ConnectionReset.into());
82 }
83 self.shared.stamp()
84 }
7885 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
7986 self.shared.publish(transaction)
8087 }
81 fn confirm(&mut self, snapshot: &[u8]) -> Result<(), CommitError> {
82 self.shared.confirm(snapshot)
88 fn confirm(&mut self, base: &Stamp) -> Result<(), CommitError> {
89 self.shared.confirm(base)
8390 }
8491 }
8592 let dir = tempfile::tempdir().unwrap();
......@@ -142,7 +149,7 @@ fn reconnects_after_connect_read_and_uncertain_publish_without_replaying() {
142149 drop(cache);
143150 let cache = Replica::open(&path).unwrap();
144151 assert_eq!(cache.status(id).unwrap(), Some(published));
145 let cached = cache.snapshot().unwrap();
152 let cached = snapshot(&cache);
146153 // Confirmation changes reader-notification fields without changing the committed graph.
147154 assert_eq!(cached[..212], server.durable[..212]);
148155 assert_eq!(cached[252..], server.durable[252..]);
......@@ -159,7 +166,7 @@ fn reconnects_after_connect_read_and_uncertain_publish_without_replaying() {
159166fn local_saves_wake_an_idle_worker_and_publish_every_writer_marker() {
160167 let dir = tempfile::tempdir().unwrap();
161168 let source = onestore::create_section("worker.one", "abc", "Fixture").unwrap();
162 let (_, oid, _) = text(&source);
169 let (sid, oid, _) = text(&source);
163170 let cache = Arc::new(Replica::create(dir.path().join("cache.sqlite"), &source).unwrap());
164171 let server = Arc::new(Mutex::new(Server::new(&source)));
165172 let shared = Shared(Arc::clone(&server));
......@@ -190,13 +197,19 @@ fn local_saves_wake_an_idle_worker_and_publish_every_writer_marker() {
190197 (0..12)
191198 .map(|writer| {
192199 let cache = &cache;
200 // Typed as an op: a page read before another writer's edit would lower stale.
193201 scope.spawn(move || {
194 let marker = format!("[{writer}] ");
195 model_ops::save(cache, oid, |page| {
196 model_ops::replace_text(page, oid, 0..0, &marker)
197 })
198 .unwrap()
199 .unwrap()
202 let op = onestore::op::PageOp::Text {
203 text: oid,
204 range: 0..0,
205 with: format!("[{writer}] "),
206 };
207 let ops = vec![onestore::op::Op::Page { space: sid, op }];
208 let edit = onestore::op::Edit {
209 at: model_ops::now(),
210 ops,
211 };
212 cache.apply(model_ops::AUTHOR, edit).unwrap()
200213 })
201214 })
202215 .collect::<Vec<_>>()
......@@ -221,7 +234,7 @@ fn local_saves_wake_an_idle_worker_and_publish_every_writer_marker() {
221234 Some(EditStatus::Published { .. })
222235 ));
223236 }
224 let content = text(&cache.snapshot().unwrap()).2;
237 let content = text(&snapshot(&cache)).2;
225238 for writer in 0..12 {
226239 assert_eq!(content.matches(&format!("[{writer}] ")).count(), 1);
227240 }
......@@ -230,7 +243,7 @@ fn local_saves_wake_an_idle_worker_and_publish_every_writer_marker() {
230243 assert_eq!(text(&server.durable).2, content);
231244 // Edits queued while the worker was busy publish together.
232245 assert_eq!(server.publications, 1);
233 assert_eq!(cache.snapshot().unwrap(), server.durable);
246 assert_eq!(snapshot(&cache), server.durable);
234247}
235248
236249#[test]
......@@ -244,13 +257,16 @@ fn dropping_during_publication_is_nonblocking_and_retains_ownership_until_recove
244257 fn read(&mut self) -> io::Result<Vec<u8>> {
245258 self.shared.read()
246259 }
260 fn stamp(&mut self) -> io::Result<Stamp> {
261 self.shared.stamp()
262 }
247263 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
248264 self.entered.send(()).unwrap();
249265 self.resume.recv_timeout(Duration::from_secs(5)).unwrap();
250266 self.shared.publish(transaction)
251267 }
252 fn confirm(&mut self, snapshot: &[u8]) -> Result<(), CommitError> {
253 self.shared.confirm(snapshot)
268 fn confirm(&mut self, base: &Stamp) -> Result<(), CommitError> {
269 self.shared.confirm(base)
254270 }
255271 }
256272 let dir = tempfile::tempdir().unwrap();
......@@ -358,81 +374,65 @@ fn cache_failures_stop_retries_and_return_the_error_without_remote_publication()
358374 assert!(rx.try_iter().next().is_none());
359375 assert_eq!(server.lock().unwrap().publications, 0);
360376 assert_eq!(cache.status(id).unwrap(), Some(EditStatus::Pending));
361 assert_eq!(text(&cache.snapshot().unwrap()).2, "L abc");
377 assert_eq!(text(&snapshot(&cache)).2, "L abc");
362378}
363379
364380#[test]
365fn polling_reports_a_blocked_queue_once_per_remote_change_without_replay() {
366 for uncertain in [false, true] {
367 let dir = tempfile::tempdir().unwrap();
368 let source = onestore::create_section("worker.one", "abc", "Fixture").unwrap();
369 let (sid, oid, _) = text(&source);
370 let cache = Arc::new(Replica::create(dir.path().join("cache.sqlite"), &source).unwrap());
371 let id = save(&cache, oid, 1..2, "L").unwrap();
372 let local = text(&cache.snapshot().unwrap()).2;
373 let mut server = if uncertain {
374 Server::new(&source)
375 } else {
376 Server::new(&onestore::replace_text(&source, sid, oid, 1..2, "R").unwrap())
377 };
378 if uncertain {
379 server.fault = Fault::UnknownBefore;
380 }
381 let server = Arc::new(Mutex::new(server));
382 let shared = Shared(Arc::clone(&server));
383 let (tx, rx) = mpsc::channel();
384 let worker = cache
385 .start_sync(
386 Duration::from_millis(10),
387 move || Ok(shared.clone()),
388 move |result| {
389 tx.send(
390 result
391 .as_ref()
392 .map(|synced| synced.edit.clone())
393 .map_err(|_| ()),
394 )
395 .unwrap();
396 },
397 )
398 .unwrap();
399 if uncertain {
400 assert!(rx.recv_timeout(Duration::from_secs(5)).unwrap().is_err());
401 }
402 let expected = |status: &EditStatus| {
403 if uncertain {
404 matches!(status, EditStatus::AwaitingConfirmation { .. })
405 } else {
406 *status == EditStatus::Conflict(ConflictKind::ContentChanged)
407 }
408 };
409 for round in 0..3 {
410 let (actual, status) = rx
411 .recv_timeout(Duration::from_secs(5))
412 .unwrap()
413 .unwrap()
381fn polling_reports_an_uncertain_attempt_once_per_remote_change_without_replay() {
382 let dir = tempfile::tempdir().unwrap();
383 let source = onestore::create_section("worker.one", "abc", "Fixture").unwrap();
384 let (sid, oid, _) = text(&source);
385 let cache = Arc::new(Replica::create(dir.path().join("cache.sqlite"), &source).unwrap());
386 let id = save(&cache, oid, 1..2, "L").unwrap();
387 let local = text(&snapshot(&cache)).2;
388 let mut server = Server::new(&source);
389 server.fault = Fault::UnknownBefore;
390 let server = Arc::new(Mutex::new(server));
391 let shared = Shared(Arc::clone(&server));
392 let (tx, rx) = mpsc::channel();
393 let worker = cache
394 .start_sync(
395 Duration::from_millis(10),
396 move || Ok(shared.clone()),
397 move |result| {
398 tx.send(
399 result
400 .as_ref()
401 .map(|synced| synced.edit.clone())
402 .map_err(|_| ()),
403 )
414404 .unwrap();
415 assert_eq!(actual, id);
416 assert!(expected(&status), "{status:?}");
417 // An unchanged remote is not read or reported again.
418 assert!(rx.recv_timeout(Duration::from_millis(100)).is_err());
419 // Another writer's change elsewhere in the text is read and decided again.
420 let mut server = server.lock().unwrap();
421 let changed =
422 onestore::replace_text(&server.visible, sid, oid, 0..0, &round.to_string())
423 .unwrap();
424 server.visible = changed.clone();
425 server.durable = changed;
426 drop(server);
427 worker.wake();
428 }
429 worker.stop().unwrap();
430 assert_eq!(text(&cache.snapshot().unwrap()).2, local);
431 assert_eq!(cache.pending().unwrap().len(), 1);
432 let server = server.lock().unwrap();
433 assert_eq!(server.publications, usize::from(uncertain));
434 assert_eq!(server.confirmations, 0);
405 },
406 )
407 .unwrap();
408 assert!(rx.recv_timeout(Duration::from_secs(5)).unwrap().is_err());
409 for round in 0..3 {
410 let (actual, status) = rx
411 .recv_timeout(Duration::from_secs(5))
412 .unwrap()
413 .unwrap()
414 .unwrap();
415 assert_eq!(actual, id);
416 assert!(
417 matches!(status, EditStatus::AwaitingConfirmation { .. }),
418 "{status:?}"
419 );
420 // An unchanged remote is not read or reported again.
421 assert!(rx.recv_timeout(Duration::from_millis(100)).is_err());
422 // Another writer's change elsewhere in the text is read and decided again.
423 let mut server = server.lock().unwrap();
424 let changed = typed(&server.visible, sid, oid, 0..0, &round.to_string());
425 server.visible = changed.clone();
426 server.durable = changed;
427 drop(server);
428 worker.wake();
435429 }
430 worker.stop().unwrap();
431 assert_eq!(text(&snapshot(&cache)).2, local);
432 assert_eq!(cache.pending().unwrap().len(), 1);
433 let server = server.lock().unwrap();
434 assert_eq!(server.publications, 1);
435 assert_eq!(server.confirmations, 0);
436436}
437437
438438#[test]
......@@ -533,51 +533,46 @@ fn invalid_intervals_and_callback_panics_leave_worker_ownership_recoverable() {
533533 .unwrap();
534534 assert_eq!(rx.recv_timeout(Duration::from_secs(5)).unwrap(), None);
535535 worker.stop().unwrap();
536 assert_eq!(cache.snapshot().unwrap(), source);
536 assert_eq!(snapshot(&cache), source);
537537}
538538
539/// A conflict never blocks the worker: one step publishes the remote's version with the
540/// local one as its conflict page.
539541#[test]
540fn a_resolved_conflict_wakes_the_worker_and_publishes_the_original_edit_once() {
542fn a_conflict_publishes_its_conflict_page_in_one_worker_step() {
541543 let dir = tempfile::tempdir().unwrap();
542544 let source = onestore::create_section("resolve.one", "abc", "Fixture").unwrap();
543545 let (sid, oid, _) = text(&source);
544546 let cache = Arc::new(Replica::create(dir.path().join("cache.sqlite"), &source).unwrap());
545547 let id = save(&cache, oid, 1..2, "L").unwrap();
546 let remote = onestore::replace_text(&source, sid, oid, 1..2, "R").unwrap();
548 let remote = typed(&source, sid, oid, 1..2, "R");
547549 let server = Arc::new(Mutex::new(Server::new(&remote)));
548550 let shared = Shared(Arc::clone(&server));
549551 let (tx, rx) = mpsc::channel();
550 let (resume_tx, resume_rx) = mpsc::channel();
551 let mut first = true;
552552 let worker = cache
553553 .start_sync(
554554 Duration::from_secs(3600),
555555 move || Ok(shared.clone()),
556556 move |result| {
557557 tx.send(result.as_ref().unwrap().edit.clone()).unwrap();
558 if first {
559 first = false;
560 resume_rx.recv_timeout(Duration::from_secs(5)).unwrap();
561 }
562558 },
563559 )
564560 .unwrap();
565 assert_eq!(
566 rx.recv_timeout(Duration::from_secs(5)).unwrap(),
567 Some((id, EditStatus::Conflict(ConflictKind::ContentChanged)))
568 );
569 cache.resolve(id, notebook::Resolution::Mine).unwrap();
570 assert_eq!(cache.status(id).unwrap(), Some(EditStatus::Pending));
571 resume_tx.send(()).unwrap();
572561 assert!(
573 matches!(rx.recv_timeout(Duration::from_secs(5)).unwrap(), Some((actual, EditStatus::Published { .. })) if actual == id)
562 matches!(rx.recv_timeout(Duration::from_secs(5)).unwrap(), Some((actual, EditStatus::Published { .. })) if actual > id)
574563 );
575564 worker.stop().unwrap();
576565 assert!(cache.pending().unwrap().is_empty());
577566 let server = server.lock().unwrap();
578567 assert_eq!(server.publications, 1);
579 assert_eq!(text(&server.durable).2, "aLc");
580 assert_eq!(cache.snapshot().unwrap(), server.durable);
568 assert_eq!(
569 conflicts(&server.durable),
570 [(
571 sid,
572 vec![(model_ops::AUTHOR.to_owned(), vec!["aLc".to_owned()])]
573 )]
574 );
575 assert_eq!(snapshot(&cache), server.durable);
581576}
582577
583578#[test]
......@@ -589,11 +584,14 @@ fn ordinary_read_and_unpublished_write_contention_reuse_the_connection() {
589584 }
590585 impl Remote for Busy {
591586 fn read(&mut self) -> io::Result<Vec<u8>> {
587 self.server.read()
588 }
589 fn stamp(&mut self) -> io::Result<Stamp> {
592590 self.reads += 1;
593591 match self.reads {
594592 1 => Err(io::ErrorKind::WouldBlock.into()),
595593 2 => Err(io::ErrorKind::ResourceBusy.into()),
596 _ => self.server.read(),
594 _ => self.server.stamp(),
597595 }
598596 }
599597 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
......@@ -611,8 +609,8 @@ fn ordinary_read_and_unpublished_write_contention_reuse_the_connection() {
611609 _ => self.server.publish(transaction),
612610 }
613611 }
614 fn confirm(&mut self, source: &[u8]) -> Result<(), CommitError> {
615 self.server.confirm(source)
612 fn confirm(&mut self, base: &Stamp) -> Result<(), CommitError> {
613 self.server.confirm(base)
616614 }
617615 }
618616 let dir = tempfile::tempdir().unwrap();
......@@ -648,7 +646,7 @@ fn ordinary_read_and_unpublished_write_contention_reuse_the_connection() {
648646 matches!(rx.recv_timeout(Duration::from_secs(5)).unwrap().unwrap(), Some((actual, EditStatus::Published { .. })) if actual == id)
649647 );
650648 worker.stop().unwrap();
651 assert_eq!(text(&cache.snapshot().unwrap()).2, "L abc");
649 assert_eq!(text(&snapshot(&cache)).2, "L abc");
652650 assert!(cache.pending().unwrap().is_empty());
653651}
654652
......@@ -659,6 +657,9 @@ fn publication_backoff_drains_local_wakes_without_waiting_for_the_idle_poll() {
659657 fn read(&mut self) -> io::Result<Vec<u8>> {
660658 self.0.read()
661659 }
660 fn stamp(&mut self) -> io::Result<Stamp> {
661 self.0.stamp()
662 }
662663 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
663664 let server = &mut self.0;
664665 if server.publications == 0 {
......@@ -670,8 +671,8 @@ fn publication_backoff_drains_local_wakes_without_waiting_for_the_idle_poll() {
670671 }
671672 server.publish(transaction)
672673 }
673 fn confirm(&mut self, source: &[u8]) -> Result<(), CommitError> {
674 self.0.confirm(source)
674 fn confirm(&mut self, base: &Stamp) -> Result<(), CommitError> {
675 self.0.confirm(base)
675676 }
676677 }
677678 let dir = tempfile::tempdir().unwrap();
......@@ -714,6 +715,6 @@ fn publication_backoff_drains_local_wakes_without_waiting_for_the_idle_poll() {
714715 }
715716 worker.stop().unwrap();
716717 assert!(cache.pending().unwrap().is_empty());
717 assert_eq!(content(&cache.snapshot().unwrap(), oid), "L abc");
718 assert_eq!(pages(&cache.snapshot().unwrap()), 2);
718 assert_eq!(content(&snapshot(&cache), oid), "L abc");
719 assert_eq!(pages(&snapshot(&cache)), 2);
719720}
crates/onestore/README.md+48-52
......@@ -49,32 +49,27 @@ harness also accepts `--client-profile release`.
4949| `page::Page`, `page::Paragraph`, `page::Ink`, `page::Math` | Build an editable page model (title, outlines, paragraphs with coalesced text spans, tables, images, attachments, ink drawings and handwriting decoded to stroke polylines in page points) with stored identities; equations parse from their linear text and run data into a tree that renders the MathML OneNote exports; content outside the model is retained as `Unsupported` |
5050| `protected::UnlockedSection` (optional feature) | Own decoded buffers for explicit known-password inspection; clear those buffers on drop; derived document strings/exports remain caller-owned |
5151| `create_section` | Create one page containing one plain-text paragraph and an author, including Unicode |
52| `PageCreation`, `PreparedEdit::create_page` | Add an empty top-level page and its section entry atomically, retaining page identities across retries |
53| `PageEdit`, `PreparedEdit::pages` | Publish explicitly selected page moves and indentation changes together, preserving page content and historical revisions |
54| `PreparedEdit::delete_pages_permanently` | Remove explicit pages and their section references atomically while retaining stored revisions |
52| `PageCreation`, `SectionOp::Create` | Add an empty top-level page and its section entry atomically, under identities the intent retains across retries |
53| `PageEdit`, `SectionOp::Pages` | Publish explicitly selected page moves and indentation changes together, preserving page content and historical revisions |
54| `SectionOp::Delete` | Remove explicit pages and their section references atomically while retaining stored revisions |
5555| `create_table_of_contents` | Create ordered section entries from filenames and file identities |
56| `TocEdit`, `PreparedEdit::table_of_contents` | Add, rename, colour, order and remove a table of contents' section and group entries as one revision |
56| `TocEdit`, `edit_table_of_contents` | Add, rename, order and remove a table of contents' section and group entries, and colour the notebook, as one revision's `Transaction`; a section's colour is `SectionOp::Color` in its own metadata |
5757| `place`, `place_file` | Name a file for its notebook as OneNote does on adoption (parent TOC identity and name CRC in the header), so OneNote keeps its identity, on any `CommitIo` or under the filesystem adapter |
58| `replace_property_bytes` | Append one scalar-property revision; preserve prior revisions and unrelated property values and references |
59| `replace_text`, `commit_text`, `commit_file_text` | Replace a UTF-16 range across ordinary text runs; publish text, run boundaries and modification time together |
60| `Insertion`, `PreparedEdit::insert` | Insert paragraphs into editable containers or positioned outlines into a page, retaining intent identities across rebases |
61| `ParagraphSplit`, `PreparedEdit::split` | Split ordinary text at a UTF-16 scalar boundary, retaining the original left identities and moving children to the right |
62| `ParagraphJoin`, `PreparedEdit::join` | Join adjacent ordinary text while preserving inherited character styles and native text-identity rules |
63| `TextAttribute`, `PreparedEdit::format` | Change character formatting over a UTF-16 range while sharing immutable styles; preserve unselected runs |
64| `OutlineEdit`, `PreparedEdit::outline` | Change ordinary outline position/width or a paragraph's saved expansion default, preserving identities and content |
65| `PreparedEdit::page` | Publish an edited `page::Page` as one revision per changed space: text, character and paragraph formatting, hyperlinks (external and, via `page::link`, to pages, paragraphs and sections), bullets and numbering, note tags, table rows, columns, cell shading and indents, nested tables, inserted pictures (in paragraphs or on the page) and attachments, picture position, size and description, a stored attachment's shown name, source path and icon size, paragraph styles on new paragraphs, ink drawings and handwriting (strokes added and erased), equations (built from `page::Math` trees), paragraph insertion/split/join/move/deletion, outline insertion/deletion/position/width and saved collapse state, lowered onto the typed writers with the model's identities |
66| `TreeEdit`, `PreparedEdit::tree` | Move or delete a subtree on one page, normalize surviving containers, and replace an emptied table cell's paragraph atomically |
67| `Transaction`, `Stamp`, `PreparedEdit::transaction` | The bytes a commit writes (appended data, in-place list-tail and log patches, header) and the header and length it requires unchanged; `commit` under caller-held exclusion, `commit_file` under the conservative filesystem adapter, `apply` to the base image in memory; serializable for queues |
58| `TextAttribute`, `PageOp::Format` | Change character formatting over a UTF-16 range while sharing immutable styles; preserve unselected runs |
59| `OutlineEdit`, `PageOp::Outline` | Change ordinary outline position/width or a paragraph's saved expansion default, preserving identities and content |
60| `Transaction`, `Stamp` | The bytes a commit writes (appended data, in-place list-tail and log patches, header) and the header and length it requires unchanged; `commit` under caller-held exclusion, `commit_file` under the conservative filesystem adapter, `apply` to the base image in memory; serializable for queues |
6861| `Arena`, `Section` | Keep a section parsed across edits: `open` validates an image once; `seal` appends one revision per changed space as a `Transaction` on `stamp`, checking only what it appends; `replay` applies a queued transaction; `image` and `page` read the result |
69| `op::{Edit, Op, PageOp, TableEdit, SectionOp}`, `Section::apply` | Object-level edits (text, formatting, links, equations, paragraph insertion/split/join/move/deletion/levels, outlines, lists, tags, styles, paragraph formatting, tables, pictures, attachments, ink, page creation/import/moves/removal) applied whole or not to a kept-open section with emitter-chosen identities, UTF-16 ranges and the edit's time; refusals name the target, identity or structure at fault |
70| `op::lower`, `op::lower_page`, `Section::apply_page` | The ops turning a range of paragraphs, or a whole page model, into another, computed from the models alone |
62| `op::{Edit, Op, PageOp, TableEdit, SectionOp}`, `Section::apply` | Object-level edits (text, formatting, links, equations, paragraph insertion/split/join/move/deletion/levels, outlines, lists, tags, styles, paragraph formatting, tables, pictures, attachments, ink, page creation/import/moves/removal, conflict pages) applied whole or not to a kept-open section with emitter-chosen identities, UTF-16 ranges and the edit's time; refusals name the target, identity or structure at fault |
63| `SectionOp::Conflict`, `Section::conflicts`, `ConflictPage` | Keep the version of a page a merge could not take as OneNote 2010 does: a read-only conflict page (`jcidConflictPageMetaData`, `IsConflictPage`, conflict objects marked) under the page's manifest, which says it has conflict pages; list each page's, newest first, and delete one (`SectionOp::Delete`) as OneNote's Delete Conflict Page does |
64| `op::lower`, `op::lower_page` | The ops turning a range of paragraphs, or a whole page model, into another, computed from the models alone, for editors and imports |
65| `op::predict` | The page an op leaves, as the section stores it and reading it back shows it |
7166| `read_file` | Read a snapshot under whole-file exclusion |
7267| `read_snapshot` | Read a validated snapshot through fresh positioned I/O while the caller excludes maintenance |
73| `commit_file_property` | Lock, check the source snapshot's stamp, append and flush, then publish the revision |
74| `CommitIo`, `commit_property_bytes` | Supply another storage backend with equivalent exclusion and ordered durability |
68| `CommitIo`, `confirm`, `confirm_file` | Supply another storage backend with equivalent exclusion and ordered durability; check that a stamp still holds |
7569
76Scalar edits accept encoded values and require the caller to maintain MS-ONE
77semantics. Text edits maintain run boundaries, inherit the insertion run's formatting,
70Edits enter a kept-open `Section` as ops and leave as one appended revision per
71changed space; only section and page creation, imports and opening files handle
72whole images. Text edits maintain run boundaries, inherit the insertion run's formatting,
7873and promote legacy text to Unicode when needed. Explicit and body-derived
7974navigation titles update in the same transaction; unsupported fields,
8075protected objects and split surrogate pairs are
......@@ -85,19 +80,19 @@ sections retain their encrypted structure and payloads. With the optional
8580`protected` feature, `protected::UnlockedSection` opens native OneNote 2010
8681AES-128/CBC, SHA-1 password wrappers into a borrowed document view. Incorrect
8782passwords, unsupported protection profiles and work-limit failures remain distinct.
88The source stays encrypted. `PreparedEdit::page_protected` publishes a page-model
89edit stored under the section's key (fresh IV per object, payloads under the file
90IV); the other writers reject protected sections.
83The source stays encrypted. `UnlockedSection::apply` applies page ops and seals them
84under the section's key (fresh IV per object, payloads under the file IV); `Section`
85rejects protected sections.
9186`PageCreation::new` appends, or inserts before the first page space of an existing
9287series. `Some("")` creates an empty title field; `None` omits the title node.
9388The page has no body outlines, generated date/time text or applied template.
9489Retain the intent to preserve its page, title and space identities; existing
9590identities require reconciliation before retry. Body insertion and title edits
96use those identities through the existing APIs. [Native page-creation fixtures](../../corpus/page-lifecycle/creation/README.md)
91use those identities through page ops. [Native page-creation fixtures](../../corpus/page-lifecycle/creation/README.md)
9792cover duplicate Unicode titles, native edits and Rust follow-up edits.
9893`PageEdit::set_level` changes one page's indentation in place. `PageEdit::move_to`
9994moves before an existing page space, or appends for `None`, and sets its level.
100`PreparedEdit::pages` applies moves in slice order and publishes the final order,
95`SectionOp::Pages` applies moves in slice order and publishes the final order,
10196series membership and metadata levels in one transaction. Each page occurs once;
10297levels are 1–3 and the final first page must have level 1. A following deeper-level
10398page remains in place unless explicitly selected. To move a group, supply all its
......@@ -105,40 +100,39 @@ pages in order. Retain the intents across retries so newly formed series keep
105100their identities; `reposition(PagePosition, level)` revises their placement while
106101preserving those identities. [Native page-edit fixtures](../../corpus/page-lifecycle/page-edits/README.md)
107102cover individual tabs, selected and collapsed groups, nesting and promotion.
108`PreparedEdit::delete_pages_permanently` removes exactly the supplied page spaces,
103`SectionOp::Delete` removes exactly the supplied page spaces,
109104including subpages only when selected explicitly. The first remaining page becomes
110105top-level; other page levels and surviving content are retained. The operation
111106creates no recycle-bin copies and preserves prior revisions, so it is not secure
112107erasure. Duplicate, missing or non-page identities reject the entire batch;
113an empty selection leaves the file unchanged. Commit uses the exact source
114snapshot and rejects a stale prepared removal before writing.
115Insertions update child references, reference counts, modification times and automatic
116titles atomically. Paragraphs can be nested or inserted into table cells; outline
117coordinates use points. `Insertion::with_formatting` includes nonoverlapping UTF-16
118spans in that same publication; gaps keep the default insertion style. Empty text
119accepts one `0..0` span for subsequent typing. Retain the `Insertion` value for rebasing: creating another
120value creates different object identities. Duplicate insertion identities require
121reconciliation. Formatting accepts explicit attributes, preserves inherited values,
108an empty selection leaves the file unchanged.
109`PageOp::Insert` and `PageOp::Add` update child references, reference counts,
110modification times and automatic titles atomically. Paragraphs can be nested or
111inserted into table cells; outline coordinates use points. Inserted text keeps its
112spans' formats; `PageOp::Format` over an empty text's `0..0` sets its insertion style.
113New objects carry the emitter's identities, so a queued edit replays onto another
114image unchanged; an identity already on the page refuses the edit. Formatting accepts explicit attributes, preserves inherited values,
122115and gives retired immutable styles zero current references while retaining history.
123116Outline layout edits use points. Width is at least 36 points; an explicit user width
124117and an automatic maximum-width hint remain distinct. Native layout generates the
125118rendered height. Saved paragraph collapse defaults can be overridden by the native
126119client's cached view. [Native layout captures](../../corpus/outline-edit/README.md)
127120verify these edits through a fresh OneNote cache, including fields and nested content.
128`TreeEdit::move_to` takes an existing parent and an optional direct sibling to insert
121`PageOp::Move` takes an existing parent and an optional direct sibling to insert
129122before; `None` appends. Paragraphs retain their descendants and explicit list styles.
130123Outlines remain page children, so reordering changes their stacking order while
131retaining coordinates. `TreeEdit::delete` removes the selected subtree from the active
124retaining coordinates. `PageOp::Delete` removes the selected subtree from the active
132125graph and preserves historical objects. Empty outlines/groups are removed; surviving
133group indentation is normalized without shifting other paragraphs. Empty table cells
134receive new empty paragraph/text objects with identities retained in the intent.
126group indentation is normalized without shifting other paragraphs. An edit that
127leaves a table cell without a paragraph is refused; insert its replacement in the
128same edit.
135129Title/protected content, ambiguous ancestry, cycles, and incompatible destinations
136130reject before publication. Modification times, move attribution and automatic titles
137131publish with the tree change. These explicit destinations differ from keyboard list
138132indentation, which can also substitute list markers.
139133Paragraph splits preserve character formatting, retain tags on the left, and clone
140mutable list objects without restarting numbering. The new right paragraph/text
141identities belong to the retained `ParagraphSplit` intent. Its publication includes
134mutable list objects without restarting numbering. `PageOp::Split` names
135the new right paragraph/text identities. Its publication includes
142136the complete child graph and title metadata; repeating an existing identity requires
143137reconciliation. Title containers, generated fields, recording-linked text and
144138associated run metadata are rejected before I/O. Native split controls and subsequent
......@@ -206,9 +200,8 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
206200```
207201
208202`Store::parse` exposes checksum mismatches for diagnostic readers; the writer
209rejects them. The resolved graph borrows the snapshot. Select the object-space ID,
210object ID, and property from that graph, then pass those IDs, the same snapshot,
211and the replacement's encoded bytes to `commit_file_property`. The
203rejects them. The resolved graph borrows the snapshot. Open it as a `Section`,
204apply ops naming objects from that graph, seal, and commit the `Transaction`. The
212205`edit_property` example demonstrates selection by JCID/property/expected bytes,
213206with optional explicit IDs when conflict copies contain identical text.
214207
......@@ -228,7 +221,7 @@ the header (MS-ONESTORE 2.3.1), so the body is never compared. Live readers must
228221equivalent exclusion.
229222Native conflict creation can still expose cross-space references before their
230223targets are saved; such snapshots must be rejected and reread while synchronization
231proceeds. `read_file` and `commit_file_property` serialize within the process because
224proceeds. `read_file` and `Transaction::commit_file` serialize within the process because
232225macOS SMB locks can be reentrant. The lock is nonblocking across processes;
233226contention requires a fresh read and a later retry.
234227
......@@ -245,13 +238,16 @@ native acceptance captures. A no-op still flushes; a failed flush has an unknown
245238durability outcome.
246239
247240The filesystem adapter uses whole-file locking. On macOS it acquires the lock
248as part of opening the file (`O_EXLOCK | O_NONBLOCK`): separate open and `flock`
249calls allowed overlapping exclusive holders and stranded server locks under
250multi-process SMB contention. `tools/smb_lock_race.py` reproduces that failure
251without notebook parsing or writing. On the tested macOS SMB mount,
252POSIX byte-range locks returned `ENOTSUP`; whole-file locks excluded native
253OneNote's lock ranges. This adapter serializes readers and writers during each
254operation; it does not reproduce native reader/writer concurrency. The
241as part of opening the file (`O_EXLOCK` to commit, `O_SHLOCK` to read, with
242`O_NONBLOCK`): separate open and `flock` calls allowed overlapping exclusive holders
243and stranded server locks under multi-process SMB contention.
244`tools/smb_lock_race.py` reproduces that failure without notebook parsing or writing,
245and with `--shared-reads` checks that shared readers never overlap a writer. On the
246tested macOS SMB mount, POSIX byte-range locks returned `ENOTSUP`, and `flock` of
247either kind became an exclusive lock over the whole file, which fails OneNote's reads.
248smbfs sends `O_SHLOCK` and `O_EXLOCK` as share modes: a read excludes writers,
249OneNote's included, but not OneNote's readers; a commit excludes everyone. It does not
250reproduce native reader/writer concurrency. The
255251[locking audit](../../evidence/LOCKING.md) records native coordination bytes, write-open share
256252modes, and a working macOS SMB-specific byte-range lock probe. `sync_all` falls
257253back to `fsync` on macOS only when `F_FULLFSYNC` is unsupported. Successful SMB FLUSH replies were observed on the
crates/onestore/examples/concurrent_client.rs+11-1
......@@ -2,6 +2,8 @@
22mod concurrent;
33#[path = "../src/flush.rs"]
44mod flush;
5#[path = "support/typing.rs"]
6mod typing;
57
68use std::{env, fs, io::Write};
79
......@@ -22,7 +24,15 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
2224 &args,
2325 |path| onestore::read_file(path),
2426 |path, source, space, object, range, replacement| {
25 onestore::commit_file_text(path, source, space, object, range, replacement)
27 let edit = typing::text(space, object, range, replacement);
28 match typing::sealed(source, "Concurrent client", &edit) {
29 Ok(Some(transaction)) => transaction.commit_file(path),
30 Ok(None) => Ok(()),
31 Err(error) => Err(onestore::CommitError {
32 state: onestore::CommitState::NotCommitted,
33 error: std::io::Error::new(std::io::ErrorKind::InvalidData, error),
34 }),
35 }
2636 },
2737 )
2838}
crates/onestore/examples/edit_property.rs+63-6
......@@ -1,9 +1,15 @@
1//! Changes one stored property value: a text object's text (`6000e 1c001c22`, UTF-16LE,
2//! a trailing NUL dropped) as a text op, the notebook's colour on the table of contents root
3//! (`20001 14001cbe`) as a TOC edit, each appending a revision; any other value of the same
4//! length only in a copy, patched where it is stored, as a fixture that is no edit.
5
16#[path = "../src/flush.rs"]
27mod flush;
8#[path = "support/typing.rs"]
9mod typing;
310
411use onestore::{
5 ExGuid, ObjectData, PropertySets, RevisionIndex, Store, Value, commit_file_property,
6 replace_property_bytes,
12 ExGuid, ObjectData, PropertySets, RevisionIndex, Store, TocEdit, Transaction, Value,
713};
814use std::{env, fs, io::Write};
915
......@@ -59,16 +65,67 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
5965 }
6066 }
6167 let (space, object) = selected.ok_or("No matching object")?;
68 let transaction: Option<Transaction> = match (jcid, id) {
69 (0x6000e, 0x1c001c22) => {
70 let units = |bytes: &[u8]| -> Vec<u16> {
71 let mut units: Vec<u16> = bytes
72 .chunks_exact(2)
73 .map(|pair| u16::from_le_bytes([pair[0], pair[1]]))
74 .collect();
75 if units.last() == Some(&0) {
76 units.pop();
77 }
78 units
79 };
80 let length = u32::try_from(units(&expected).len())?;
81 let text = String::from_utf16(&units(&value))?;
82 let edit = typing::text(space, object, 0..length, &text);
83 typing::sealed(&bytes, "Property editor", &edit)?
84 }
85 (0x20001, 0x14001cbe) => {
86 if index.resolve_active(space)?.roots.get(&1) != Some(&object) {
87 return Err("Only the notebook's colour, on the TOC root, changes".into());
88 }
89 let color = u32::from_le_bytes(value.as_slice().try_into()?);
90 onestore::edit_table_of_contents(&bytes, &[TocEdit::Color(color)])?
91 }
92 _ if args[1] != "--in-place" && value.len() == expected.len() => {
93 let revision = index.resolve_active(space)?;
94 let ObjectData::Properties(data) = revision.objects[&object].data else {
95 return Err("The object stores no properties".into());
96 };
97 let properties = PropertySets::parse(data)?;
98 let Some(Value::Bytes(stored)) = properties.sets[0]
99 .iter()
100 .find(|property| property.id == id)
101 .map(|property| &property.value)
102 else {
103 return Err("The property stores no bytes".into());
104 };
105 let at = stored.as_ptr().addr() - bytes.as_ptr().addr();
106 let mut patched = bytes.clone();
107 patched[at..at + value.len()].copy_from_slice(&value);
108 write(&args[1], &patched)?;
109 return Ok(());
110 }
111 _ => return Err("Only text and TOC colours change as edits; other values are patched into a copy of equal length".into()),
112 };
113 let transaction = transaction.ok_or("The value is already stored")?;
62114 if args[1] == "--in-place" {
63 commit_file_property(&args[0], &bytes, space, object, id, &value)?;
115 transaction.commit_file(&args[0])?;
64116 return Ok(());
65117 }
66 let written = replace_property_bytes(&bytes, space, object, id, &value)?;
118 let mut written = bytes.clone();
119 transaction.apply(&mut written)?;
120 write(&args[1], &written)
121}
122
123fn write(path: &str, written: &[u8]) -> Result<(), Box<dyn std::error::Error>> {
67124 let mut output = fs::OpenOptions::new()
68125 .write(true)
69126 .create_new(true)
70 .open(&args[1])?;
71 output.write_all(&written)?;
127 .open(path)?;
128 output.write_all(written)?;
72129 flush::flush(&output)?;
73130 Ok(())
74131}
crates/onestore/examples/header_faults.rs+19-15
......@@ -1,24 +1,28 @@
1#[path = "support/typing.rs"]
2mod typing;
3
14use onestore::*;
25fn main() -> Result<(), Box<dyn std::error::Error>> {
36 let source =
47 std::fs::read("corpus/native/20260905-05/snapshots/02-text/notebook/synthetic.one")?;
58 let store = Store::parse(&source)?;
69 let index = RevisionIndex::parse(&store)?;
7 let mut target = None;
8 for (osid, space) in &index.spaces {
9 let revision = index.resolve(*osid, space.labels[&(ExGuid::default(), 1)])?;
10 for (oid, object) in revision.objects {
11 if let ObjectData::Properties(data) = object.data {
12 for p in &PropertySets::parse(data)?.sets[0] {
13 if p.value == Value::Bytes(b"Fictitious plain text.") {
14 target = Some((*osid, oid, p.id));
15 }
16 }
17 }
18 }
19 }
20 let (osid, oid, property) = target.unwrap();
21 let output = replace_property_bytes(&source, osid, oid, property, b"A durable edit.")?;
10 let document = document::Document::parse(&index)?;
11 let (osid, oid) = document
12 .spaces
13 .iter()
14 .find_map(|(sid, space)| {
15 space.active()?.nodes.iter().find_map(|(oid, node)| {
16 matches!(&node.kind, document::Kind::RichText { text, .. } if text == "Fictitious plain text.")
17 .then_some((*sid, *oid))
18 })
19 })
20 .ok_or("Missing native fixture text")?;
21 let end = "Fictitious plain text.".encode_utf16().count() as u32;
22 let edit = typing::text(osid, oid, 0..end, "A durable edit.");
23 let transaction = typing::sealed(&source, "Header faults", &edit)?.ok_or("No change")?;
24 let mut output = source.clone();
25 transaction.apply(&mut output)?;
2226 let mut invalid = Vec::new();
2327 for prefix in 0..=1024 {
2428 let mut torn = output.clone();
crates/onestore/examples/insert.rs+85-24
......@@ -1,37 +1,96 @@
11#[path = "../src/flush.rs"]
22mod flush;
3#[path = "support/typing.rs"]
4mod typing;
35
4use onestore::{Insertion, PreparedEdit};
6use onestore::{
7 document::{Format, Layout},
8 op::{Edit, Op, PageOp},
9 page::{Outline, PageObject, PageParagraph, Paragraph, ParagraphContent, TextObject, text::new_id},
10};
511use std::{fs, io::Write};
612
13/// A paragraph holding `text` in OneNote's default text style.
14fn paragraph(text: &str) -> Result<PageParagraph, Box<dyn std::error::Error>> {
15 let format = Format {
16 font: Some("Calibri".into()),
17 font_size: Some(11.0),
18 language: Some(0x409),
19 ..Format::default()
20 };
21 Ok(PageParagraph {
22 id: new_id()?,
23 parent: None,
24 level: 1,
25 style: None,
26 format: Format::default(),
27 content: ParagraphContent::Text(TextObject {
28 id: new_id()?,
29 date_field: None,
30 text: Paragraph::new(text.into(), format),
31 tags: Vec::new(),
32 }),
33 lists: Vec::new(),
34 tags: Vec::new(),
35 media: Default::default(),
36 collapsed: false,
37 })
38}
39
740fn main() -> Result<(), Box<dyn std::error::Error>> {
841 let args: Vec<_> = std::env::args().skip(1).collect();
9 let usage = "Usage: insert paragraph INPUT OUTPUT|--in-place SPACE PARENT BEFORE|- TEXT AUTHOR\n insert outline INPUT OUTPUT|--in-place SPACE PAGE X Y TEXT AUTHOR";
10 let insertion = match args.first().map(String::as_str) {
11 Some("paragraph") if args.len() == 8 => Insertion::paragraph(
12 args[4].parse()?,
13 if args[5] == "-" {
14 None
15 } else {
16 Some(args[5].parse()?)
17 },
18 &args[6],
19 &args[7],
20 )?,
21 Some("outline") if args.len() == 9 => Insertion::outline(
22 args[4].parse()?,
23 args[5].parse()?,
24 args[6].parse()?,
25 &args[7],
26 &args[8],
27 )?,
42 let usage = "Usage: insert paragraph INPUT OUTPUT|--in-place SPACE CONTAINER BEFORE|- TEXT AUTHOR\n insert outline INPUT OUTPUT|--in-place SPACE X Y TEXT AUTHOR";
43 let (op, object, text, author) = match args.first().map(String::as_str) {
44 Some("paragraph") if args.len() == 8 => {
45 let paragraph = paragraph(&args[6])?;
46 let (object, text) = (paragraph.id, paragraph.text().unwrap().id);
47 let op = PageOp::Insert {
48 container: args[4].parse()?,
49 before: if args[5] == "-" {
50 None
51 } else {
52 Some(args[5].parse()?)
53 },
54 paragraphs: vec![paragraph],
55 };
56 (op, object, text, &args[7])
57 }
58 Some("outline") if args.len() == 8 => {
59 let paragraph = paragraph(&args[6])?;
60 let text = paragraph.text().unwrap().id;
61 let object = new_id()?;
62 let op = PageOp::Add {
63 object: PageObject::Outline(Outline {
64 id: object,
65 title: false,
66 min_width: None,
67 layout: Layout {
68 x: Some(args[4].parse()?),
69 y: Some(args[5].parse()?),
70 ..Layout::default()
71 },
72 indents: Vec::new(),
73 paragraphs: vec![paragraph],
74 unsupported: Vec::new(),
75 }),
76 before: None,
77 };
78 (op, object, text, &args[7])
79 }
2880 _ => return Err(usage.into()),
2981 };
3082 let source = onestore::read_file(&args[1])?;
31 let prepared = PreparedEdit::insert(&source, args[3].parse()?, &insertion)?;
32 let mut record = serde_json::json!({"intent": insertion, "object": insertion.object(), "text_object": insertion.text_object()});
83 let edit = Edit {
84 at: typing::now(),
85 ops: vec![Op::Page {
86 space: args[3].parse()?,
87 op,
88 }],
89 };
90 let transaction = typing::sealed(&source, author, &edit)?.ok_or("The insertion stores nothing")?;
91 let mut record = serde_json::json!({"edit": edit, "object": object, "text_object": text});
3392 if args[2] == "--in-place" {
34 if let Err(error) = prepared.commit_file(&args[1]) {
93 if let Err(error) = transaction.commit_file(&args[1]) {
3594 record["state"] = format!("{:?}", error.state).into();
3695 record["error"] = error.error.to_string().into();
3796 println!("{record}");
......@@ -39,11 +98,13 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
3998 }
4099 record["state"] = "Committed".into();
41100 } else {
101 let mut written = source;
102 transaction.apply(&mut written)?;
42103 let mut file = fs::OpenOptions::new()
43104 .write(true)
44105 .create_new(true)
45106 .open(&args[2])?;
46 file.write_all(prepared.as_bytes())?;
107 file.write_all(&written)?;
47108 flush::flush(&file)?;
48109 record["state"] = "Created".into();
49110 }
crates/onestore/examples/maintenance_fixture.rs+7-1
......@@ -1,3 +1,6 @@
1#[path = "support/typing.rs"]
2mod typing;
3
14use onestore::{
25 ExGuid, RevisionIndex, Store,
36 document::{Document, Kind},
......@@ -46,7 +49,10 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
4649 } else {
4750 format!("Revision {revision}: {}", "x".repeat(65536))
4851 };
49 bytes = onestore::replace_text(&bytes, sid, oid, 0..end, &text)?;
52 let edit = typing::text(sid, oid, 0..end, &text);
53 if let Some(transaction) = typing::sealed(&bytes, "Fixture", &edit)? {
54 transaction.apply(&mut bytes)?;
55 }
5056 }
5157 let store = Store::parse(&bytes)?;
5258 let toc = onestore::create_table_of_contents(
crates/onestore/examples/power_loss.rs+49-36
......@@ -8,6 +8,8 @@ use trace::{Event, Trace};
88
99#[path = "../tests/support/checkpoint.rs"]
1010mod checkpoint;
11#[path = "support/typing.rs"]
12mod typing;
1113
1214use onestore::{
1315 ExGuid, RevisionIndex, Store,
......@@ -15,6 +17,16 @@ use onestore::{
1517};
1618use std::{collections::BTreeMap, fs, path::PathBuf};
1719
20/// `source`, a table of contents, with the notebook recoloured to `color`.
21fn recolored(source: &[u8], color: u32) -> Result<Vec<u8>, onestore::Error> {
22 let mut image = source.to_vec();
23 let edit = onestore::TocEdit::Color(color);
24 if let Some(transaction) = onestore::edit_table_of_contents(source, &[edit])? {
25 transaction.apply(&mut image)?;
26 }
27 Ok(image)
28}
29
1830fn main() -> Result<(), Box<dyn std::error::Error>> {
1931 let mut args = std::env::args_os().skip(1);
2032 let destination = PathBuf::from(args.next().ok_or("Provide a new evidence directory")?);
......@@ -69,12 +81,11 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
6981 let index = RevisionIndex::parse(&store)?;
7082 let document = Document::parse(&index)?;
7183 let toc = store.header.file_type == onestore::FileType::TableOfContents;
84 // A table of contents changes the notebook's colour.
7285 let (sid, oid) = if toc {
7386 let root = &document.spaces[&document.root];
74 (
75 document.root,
76 root.revisions[&root.contexts[&ExGuid::default()]].roots[&1],
77 )
87 let root = &root.revisions[&root.contexts[&ExGuid::default()]];
88 (document.root, root.roots[&1])
7889 } else {
7990 document
8091 .spaces
......@@ -95,37 +106,43 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
95106 })
96107 .ok_or("Missing native fixture text")?
97108 };
98 if name.ends_with("checkpoint") {
99 source =
100 checkpoint::pending(&source, sid, oid, if toc { 0x14001cbe } else { 0x14001d7a });
101 }
102 if name == "toc-rollover" {
103 loop {
109 match name {
110 "checkpoint" => source = checkpoint::pending(&source, sid, oid),
111 "toc-checkpoint" => {
112 for value in 0.. {
113 let store = Store::parse(&source)?;
114 let index = RevisionIndex::parse(&store)?;
115 let rid = index.spaces[&sid].labels[&(ExGuid::default(), 1)];
116 let depth = std::iter::successors(Some(rid), |id| {
117 index.spaces[&sid].revisions[id].dependency
118 })
119 .count();
120 if depth == 512 {
121 break;
122 }
123 source = recolored(&source, value)?;
124 }
125 }
126 "toc-rollover" => loop {
104127 let count = Store::parse(&source)?.header.transaction_count;
105128 if count == 255 {
106129 break;
107130 }
108131 assert!(count < 255);
109 source = onestore::replace_property_bytes(
110 &source,
111 sid,
112 oid,
113 0x14001cbe,
114 &count.to_le_bytes(),
115 )?;
116 }
132 source = recolored(&source, count)?;
133 },
134 _ => {}
117135 }
118136 let apply = |io: &mut Trace,
119137 bytes: &[u8],
120138 recovering: bool|
121139 -> Result<(), Box<dyn std::error::Error>> {
122140 if toc {
123 let value = if recovering {
124 [0x44, 0x55, 0x66, 0]
125 } else {
126 [0x33, 0x66, 0x99, 0]
127 };
128 onestore::commit_property_bytes(io, bytes, sid, oid, 0x14001cbe, &value)?;
141 let color = if recovering { 0x665544 } else { 0x996633 };
142 let edit = onestore::TocEdit::Color(color);
143 let transaction =
144 onestore::edit_table_of_contents(bytes, &[edit])?.ok_or("No change")?;
145 transaction.commit(io)?;
129146 } else {
130147 let store = Store::parse(bytes)?;
131148 let index = RevisionIndex::parse(&store)?;
......@@ -136,18 +153,14 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
136153 unreachable!()
137154 };
138155 let end = u32::try_from(text.encode_utf16().count())?;
139 onestore::commit_text(
140 io,
141 bytes,
142 sid,
143 oid,
144 end..end,
145 if recovering {
146 " [recovered]"
147 } else {
148 " [durable café 🦀]"
149 },
150 )?;
156 let with = if recovering {
157 " [recovered]"
158 } else {
159 " [durable café 🦀]"
160 };
161 let edit = typing::text(sid, oid, end..end, with);
162 let transaction = typing::sealed(bytes, "Power loss", &edit)?.ok_or("No change")?;
163 transaction.commit(io)?;
151164 }
152165 Ok(())
153166 };
crates/onestore/examples/random_edit.rs+12-3
......@@ -1,5 +1,7 @@
11#[path = "../src/flush.rs"]
22mod flush;
3#[path = "support/typing.rs"]
4mod typing;
35
46use onestore::{
57 ExGuid, RevisionIndex, Store,
......@@ -111,13 +113,20 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
111113 end
112114 };
113115 let range = offset + boundaries[start]..offset + boundaries[end];
114 let written = match onestore::replace_text(&source, sid, oid, range.clone(), replacement) {
115 Ok(written) => written,
116 let edit = typing::text(sid, oid, range.clone(), replacement);
117 let transaction = match typing::sealed(&source, "Random editor", &edit) {
118 Ok(Some(transaction)) => transaction,
119 Ok(None) => {
120 rejected.push("The edit stores nothing".into());
121 continue;
122 }
116123 Err(error) => {
117124 rejected.push(error.to_string());
118125 continue;
119126 }
120127 };
128 let mut written = source.clone();
129 transaction.apply(&mut written)?;
121130 let space = &document.spaces[&sid];
122131 let revision = &space.revisions[&space.contexts[&ExGuid::default()]];
123132 let Kind::RichText { runs, .. } = &revision.nodes[&oid].kind else {
......@@ -133,7 +142,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
133142 "run_before": actual[selected].text, "run_start": runs[selected].start, "source_md5": format!("{:x}", md5::compute(&source)),
134143 "started_ms": started, "rejected_candidates": rejected});
135144 let failed = if args[1] == "--in-place" {
136 match onestore::commit_file_text(&args[0], &source, sid, oid, range, replacement) {
145 match transaction.commit_file(&args[0]) {
137146 Ok(()) => {
138147 record["state"] = "Committed".into();
139148 false
crates/onestore/examples/scratch_dump.rs created+29
......@@ -0,0 +1,29 @@
1use onestore::{ObjectData, PropertySets, RevisionIndex, Store, Value, document::Document};
2fn main() -> Result<(), Box<dyn std::error::Error>> {
3 let args: Vec<String> = std::env::args().skip(1).collect();
4 let bytes = std::fs::read(&args[0])?;
5 let store = Store::parse(&bytes)?;
6 let index = RevisionIndex::parse(&store)?;
7 let pages = Document::parse(&index).and_then(|document| document.pages()).unwrap_or_default();
8 let space = if args[1] == "root" { index.root } else { pages[args[1].parse::<usize>()?].0 };
9 println!("space {space}; pages {:?}", pages.iter().map(|p| p.0).collect::<Vec<_>>());
10 let raw = index.resolve_active(space)?;
11 for (id, object) in &raw.objects {
12 println!("{id} jcid {:#x}", object.jcid);
13 if let ObjectData::Properties(data) = object.data {
14 let sets = PropertySets::parse(data)?;
15 for (n, set) in sets.sets.iter().enumerate() {
16 for p in set {
17 let v = match &p.value {
18 Value::NoData => "-".to_string(),
19 Value::Bytes(b) => b.iter().map(|x| format!("{x:02x}")).collect(),
20 Value::References { stream, compact_ids } => format!("{stream:?} {}", compact_ids.iter().map(|x| format!("{x:02x}")).collect::<String>()),
21 Value::Sets(r) => format!("sets {r:?}"),
22 };
23 println!(" [{n}] {:#010x} {v}", p.id);
24 }
25 }
26 }
27 }
28 Ok(())
29}
crates/onestore/examples/support/concurrent.rs+110-32
......@@ -318,7 +318,46 @@ pub fn run(
318318#[cfg(test)]
319319mod tests {
320320 use super::*;
321 use onestore::{Insertion, ParagraphJoin, ParagraphSplit, PreparedEdit, TextAttribute};
321 use onestore::{
322 TextAttribute,
323 document::{Format, Layout},
324 op::{Edit, Op, PageOp},
325 page::{Outline, PageObject, PageParagraph, Paragraph, ParagraphContent, TextObject, text::new_id},
326 };
327
328 fn paragraph(text: &str, level: u32) -> PageParagraph {
329 PageParagraph {
330 id: new_id().unwrap(),
331 parent: None,
332 level,
333 style: None,
334 format: Format::default(),
335 content: ParagraphContent::Text(TextObject {
336 id: new_id().unwrap(),
337 date_field: None,
338 text: Paragraph::new(text.into(), Format {
339 font: Some("Calibri".into()),
340 font_size: Some(11.0),
341 language: Some(0x409),
342 ..Format::default()
343 }),
344 tags: Vec::new(),
345 }),
346 lists: Vec::new(),
347 tags: Vec::new(),
348 media: Default::default(),
349 collapsed: false,
350 }
351 }
352
353 fn edited(image: &[u8], space: ExGuid, ops: Vec<PageOp>) -> Vec<u8> {
354 let arena = onestore::Arena::default();
355 let mut section = onestore::Section::open(&arena, image.to_vec()).unwrap();
356 let ops = ops.into_iter().map(|op| Op::Page { space, op }).collect();
357 section.apply("Author", &Edit { at: 134_000_000_000_000_000, ops }).unwrap();
358 section.seal().unwrap();
359 section.image()
360 }
322361
323362 #[test]
324363 fn observation_follows_split_suffixes_and_moved_children_through_active_ancestry() {
......@@ -328,57 +367,96 @@ mod tests {
328367 let index = RevisionIndex::parse(&store).unwrap();
329368 let document = Document::parse(&index).unwrap();
330369 let (space, page) = document.pages().unwrap()[0];
331 let insertion = Insertion::outline(page, 144.0, 216.0, "Document w0:0 🦀", "Author")
332 .unwrap()
333 .with_formatting(0..16, &[TextAttribute::Bold(true)])
334 .unwrap();
335 let inserted = PreparedEdit::insert(&source, space, &insertion).unwrap();
336 let before = document_view(inserted.as_bytes()).unwrap();
370 let first = paragraph("Document w0:0 🦀", 1);
371 let (paragraph_id, text) = (first.id, first.text().unwrap().id);
372 let outline_id = new_id().unwrap();
373 let inserted = edited(
374 &source,
375 space,
376 vec![
377 PageOp::Add {
378 object: PageObject::Outline(Outline {
379 id: outline_id,
380 title: false,
381 min_width: None,
382 layout: Layout {
383 x: Some(144.0),
384 y: Some(216.0),
385 ..Default::default()
386 },
387 indents: Vec::new(),
388 paragraphs: vec![first],
389 unsupported: Vec::new(),
390 }),
391 before: None,
392 },
393 PageOp::Format {
394 text,
395 range: 0..16,
396 set: vec![TextAttribute::Bold(true)],
397 clear: Vec::new(),
398 },
399 ],
400 );
401 let before = document_view(&inserted).unwrap();
337402 assert_eq!(before.texts.len(), 1);
338403 assert_eq!(before.graph.len(), 2);
339 let outline = insertion.object().to_string();
340 let paragraph = before.graph[&outline]["children"][0]
341 .as_str()
342 .unwrap()
343 .to_owned();
344 let child =
345 Insertion::paragraph(paragraph.parse().unwrap(), None, "Unmarked child", "Author")
346 .unwrap();
347 let with_child = PreparedEdit::insert(inserted.as_bytes(), space, &child).unwrap();
348 let before = document_view(with_child.as_bytes()).unwrap();
404 let outline = outline_id.to_string();
405 let paragraph_name = paragraph_id.to_string();
406 assert_eq!(before.graph[&outline]["children"], json!([paragraph_name]));
407 let child = paragraph("Unmarked child", 2);
408 let child_id = child.id;
409 let with_child = edited(
410 &inserted,
411 space,
412 vec![PageOp::Insert {
413 container: paragraph_id,
414 before: None,
415 paragraphs: vec![child],
416 }],
417 );
418 let before = document_view(&with_child).unwrap();
349419 assert_eq!(before.texts.len(), 2);
350420 assert_eq!(before.graph.len(), 3);
351421 assert_eq!(
352422 before.graph[&outline],
353 json!({"parent":page.to_string(), "children":[paragraph], "content":[], "child_level":1, "position":{"x":144.0,"y":216.0}})
423 json!({"parent":page.to_string(), "children":[paragraph_name], "content":[], "child_level":1, "position":{"x":144.0,"y":216.0}})
354424 );
355425 for offset in [14, 16] {
356 let split = ParagraphSplit::new(insertion.text_object(), offset, "Author").unwrap();
357 let split_edit = PreparedEdit::split(with_child.as_bytes(), space, &split).unwrap();
358 let observed = document_view(split_edit.as_bytes()).unwrap();
426 let (split, right) = (new_id().unwrap(), new_id().unwrap());
427 let split_edit = edited(
428 &with_child,
429 space,
430 vec![PageOp::Split {
431 text,
432 at: offset,
433 paragraph: split,
434 right,
435 lists: Vec::new(),
436 }],
437 );
438 let observed = document_view(&split_edit).unwrap();
359439 assert_eq!(observed.texts.len(), 3);
360440 assert_eq!(observed.graph.len(), 4);
361441 assert_eq!(
362 observed.texts[&split.text_object().to_string()]["text"],
442 observed.texts[&right.to_string()]["text"],
363443 if offset == 14 { "🦀" } else { "" }
364444 );
365445 assert_eq!(
366446 observed.graph[&outline]["children"],
367 json!([paragraph, split.object().to_string()])
447 json!([paragraph_name, split.to_string()])
368448 );
369 assert_eq!(observed.graph[&paragraph]["children"], json!([]));
449 assert_eq!(observed.graph[&paragraph_name]["children"], json!([]));
370450 assert_eq!(
371 observed.graph[&split.object().to_string()]["children"],
372 json!([child.object().to_string()])
451 observed.graph[&split.to_string()]["children"],
452 json!([child_id.to_string()])
373453 );
374454 assert_eq!(
375 observed.graph[&child.object().to_string()]["parent"],
376 split.object().to_string()
455 observed.graph[&child_id.to_string()]["parent"],
456 split.to_string()
377457 );
378 let join =
379 ParagraphJoin::new(insertion.text_object(), split.text_object(), "Author").unwrap();
380 let joined = PreparedEdit::join(split_edit.as_bytes(), space, &join).unwrap();
381 let joined = document_view(joined.as_bytes()).unwrap();
458 let joined = edited(&split_edit, space, vec![PageOp::Join { left: text, right }]);
459 let joined = document_view(&joined).unwrap();
382460 assert_eq!(joined.graph, before.graph);
383461 let characters = |view: &DocumentView| {
384462 view.texts
crates/onestore/examples/support/typing.rs created+41
......@@ -0,0 +1,41 @@
1//! Text typed as ops through a `Section`, for the examples that edit a section file.
2#![allow(dead_code)]
3
4use onestore::{
5 Arena, ExGuid, Section, Transaction,
6 op::{Edit, Op, OpError, PageOp},
7};
8use std::{
9 ops::Range,
10 time::{SystemTime, UNIX_EPOCH},
11};
12
13/// FILETIME now, as an edit's `at`.
14pub fn now() -> u64 {
15 let unix = SystemTime::now()
16 .duration_since(UNIX_EPOCH)
17 .unwrap_or_default();
18 (unix.as_secs() + 11_644_473_600) * 10_000_000 + u64::from(unix.subsec_nanos() / 100)
19}
20
21/// The op replacing `range` of the text object `text` in `space` with `with`.
22pub fn text(space: ExGuid, text: ExGuid, range: Range<u32>, with: &str) -> Edit {
23 let op = PageOp::Text {
24 text,
25 range,
26 with: with.to_owned(),
27 };
28 Edit {
29 at: now(),
30 ops: vec![Op::Page { space, op }],
31 }
32}
33
34/// The transaction `edit` seals into on `image`, as `author` made it; none when it stores
35/// nothing.
36pub fn sealed(image: &[u8], author: &str, edit: &Edit) -> Result<Option<Transaction>, OpError> {
37 let arena = Arena::default();
38 let mut section = Section::open(&arena, image.to_vec()).map_err(OpError::Failed)?;
39 section.apply(author, edit)?;
40 section.seal().map_err(OpError::Failed)
41}
crates/onestore/src/active.rs+40-22
......@@ -1,7 +1,13 @@
11use crate::{
2 Error, ExGuid, FileType, Object, RevisionIndex, Store,
3 document::{Document, Element, Kind, Revision, Space},
4 write::{Commit, LiveRevision, PropertyObject, chain_depth, declared, differing},
2 Error, ExGuid, FileType, Object,
3 document::{Element, Kind, Revision},
4 write::{Commit, LiveRevision, PropertyObject, declared, differing},
5};
6#[cfg(test)]
7use crate::{
8 RevisionIndex, Store,
9 document::{Document, Space},
10 write::chain_depth,
511};
612use bumpalo::Bump;
713use std::{
......@@ -35,6 +41,8 @@ pub(crate) struct ActivePage<'a> {
3541}
3642
3743impl<'a> ActivePage<'a> {
44 /// The active page of `space`, for tests comparing writers.
45 #[cfg(test)]
3846 pub(crate) fn parse(index: &'a RevisionIndex<'a>, space: ExGuid) -> Result<Self> {
3947 index.validate_current()?;
4048 let mut document = Document::parse(index)?;
......@@ -131,6 +139,7 @@ impl<'a> ActivePage<'a> {
131139
132140 /// Stores a writer's objects and payloads as the next revision; false when it stores
133141 /// nothing, as appending it would leave the image unchanged.
142 #[cfg(test)]
134143 pub(crate) fn write(
135144 &mut self,
136145 arena: &'a Bump,
......@@ -328,26 +337,19 @@ fn is_title(node: &Element<'_>) -> bool {
328337 .is_some_and(|fields| fields.iter().any(|field| field.id == 0x88001cb4))
329338}
330339
331/// Writes one revision of a page space: `changes` computes its objects from the active page.
332pub(crate) fn write(
333 source: &[u8],
334 space: ExGuid,
335 changes: impl FnOnce(&ActivePage<'_>) -> Result<Changes>,
336) -> Result<Vec<u8>> {
337 let store = Store::parse(source)?;
338 let index = RevisionIndex::parse(&store)?;
339 let changes = changes(&ActivePage::parse(&index, space)?)?;
340 if changes.is_empty() {
341 return Ok(source.to_vec());
342 }
343 crate::write::write_revision_on(&index, space, |_| Ok(changes))
344}
345
346340#[cfg(test)]
347341pub(crate) mod tests {
348342 use super::*;
349343 use crate::{Insertion, ParagraphSplit, TextAttribute, TreeEdit, document::Kind};
350344
345 /// An identity from `fresh_guid`, which seeded tests draw deterministically per thread.
346 fn new_id() -> Result<ExGuid> {
347 Ok(ExGuid {
348 guid: crate::write::fresh_guid()?,
349 n: 1,
350 })
351 }
352
351353 /// A seeded source of the typed writers' changes: insertions, moves, deletions, splits
352354 /// and formatting of what a page holds.
353355 pub(crate) struct Writes(pub u64);
......@@ -392,15 +394,31 @@ pub(crate) mod tests {
392394 let text = *texts.get(self.pick(texts.len())).ok_or(unavailable)?;
393395 let words = ["", "a", "Two words", "東京 🦀", "longer text here"];
394396 let word = words[self.pick(words.len())];
395 match self.pick(8) {
396 0..=3 => Insertion::paragraph(container, anchor, word, "Author")
397 .and_then(|insertion| insertion.changes(active)),
397 // `Section::apply` refuses an edit ending with a table cell emptied.
398 let fills_a_cell = active.parents[&paragraph].iter().any(|parent| {
399 matches!(nodes[parent].kind, Kind::Cell { .. })
400 && nodes[parent].children == [paragraph]
401 });
402 let choice = match self.pick(8) {
403 4 | 5 if fills_a_cell => 0,
404 choice => choice,
405 };
406 match choice {
407 0..=3 => Insertion::paragraph(container, anchor, word, "Author").and_then(
408 |insertion| {
409 let paragraph = new_id()?;
410 insertion.changes_as(active, paragraph, paragraph, new_id()?)
411 },
412 ),
398413 4 => TreeEdit::move_to(paragraph, container, anchor, "Author")
399414 .and_then(|edit| edit.changes(active)),
400415 5 => TreeEdit::delete(paragraph, "Author").and_then(|edit| edit.changes(active)),
401416 6 => {
402417 let at = self.pick(3) as u32;
403 ParagraphSplit::new(text, at, "Author").and_then(|split| split.changes(active))
418 let lists = (0..8).map(|_| new_id()).collect::<Result<Vec<_>>>()?;
419 ParagraphSplit::new(text, at, "Author").and_then(|split| {
420 split.changes_as(active, new_id()?, new_id()?, &lists)
421 })
404422 }
405423 _ => {
406424 let end = self.pick(2) as u32;
crates/onestore/src/commit.rs+20-313
......@@ -1,4 +1,3 @@
1use crate::{ExGuid, replace_property_bytes};
21use std::io::{self, ErrorKind};
32
43#[cfg(any(unix, windows))]
......@@ -33,8 +32,14 @@ impl FileIo {
3332 #[cfg(target_os = "macos")]
3433 {
3534 use std::os::unix::fs::OpenOptionsExt;
36 // SMB can lose exclusion when separate opens race with flock.
37 options.custom_flags(nix::libc::O_EXLOCK | nix::libc::O_NONBLOCK);
35 // SMB can lose exclusion when separate opens race with flock. On smbfs these are
36 // share modes: a shared reader denies only writers, so OneNote's readers proceed.
37 let lock = if write {
38 nix::libc::O_EXLOCK
39 } else {
40 nix::libc::O_SHLOCK
41 };
42 options.custom_flags(lock | nix::libc::O_NONBLOCK);
3843 }
3944 let file = options.open(path)?;
4045 #[cfg(not(target_os = "macos"))]
......@@ -100,14 +105,14 @@ pub fn place_file(path: impl AsRef<Path>, ancestor: [u8; 16], name: &str) -> io:
100105 released
101106}
102107
103/// Reads a snapshot under the same whole-file exclusion used for commits.
104/// Native writers can expose incomplete graphs to unlocked filesystem reads.
108/// Reads a snapshot excluding writers, as commits exclude everyone (macOS shares it with
109/// other readers). Native writers can expose incomplete graphs to unlocked filesystem reads.
105110#[cfg(any(unix, windows))]
106111pub fn read_file(path: impl AsRef<Path>) -> io::Result<Vec<u8>> {
107112 read_file_limited(path, usize::MAX)
108113}
109114
110/// Reads under whole-file exclusion, rejecting a snapshot larger than the byte limit.
115/// `read_file`, rejecting a snapshot larger than the byte limit.
111116/// A size failure returns `FileTooLarge` without a partial snapshot.
112117#[cfg(any(unix, windows))]
113118pub fn read_file_limited(path: impl AsRef<Path>, limit: usize) -> io::Result<Vec<u8>> {
......@@ -147,278 +152,25 @@ impl CommitIo for FileIo {
147152 }
148153}
149154
150/// Commits under an exclusive whole-file lock; contention returns WouldBlock.
151/// Calls are serialized within this process because macOS SMB locks are reentrant.
152/// Durability relies on the filesystem and server honoring their flush contract.
153/// A changed snapshot returns ResourceBusy without publishing the edit.
155/// `confirm` under the same whole-file exclusion `commit_file` uses.
154156#[cfg(any(unix, windows))]
155pub fn commit_file_property(
156 path: impl AsRef<Path>,
157 source: &[u8],
158 space: ExGuid,
159 object: ExGuid,
160 property: u32,
161 value: &[u8],
162) -> Result<(), CommitError> {
157pub fn confirm_file(path: impl AsRef<Path>, base: &Stamp) -> Result<(), CommitError> {
163158 let mut io = FileIo::open(path, true).map_err(|error| CommitError {
164159 state: CommitState::NotCommitted,
165160 error,
166161 })?;
167 let result = commit_property_bytes(&mut io, source, space, object, property, value);
162 let result = confirm(&mut io, base);
168163 io.finish(result)
169164}
170165
171/// Commits a text edit with the same exclusion and snapshot check as scalar edits.
172#[cfg(any(unix, windows))]
173pub fn commit_file_text(
174 path: impl AsRef<Path>,
175 source: &[u8],
176 space: ExGuid,
177 object: ExGuid,
178 range: std::ops::Range<u32>,
179 replacement: &str,
180) -> Result<(), CommitError> {
181 let mut io = FileIo::open(path, true).map_err(|error| CommitError {
182 state: CommitState::NotCommitted,
183 error,
184 })?;
185 let result = commit_text(&mut io, source, space, object, range, replacement);
186 io.finish(result)
187}
188
189/// Atomically publishes text and its dependent run boundaries under caller-held exclusion.
190pub fn commit_text(
191 io: &mut impl CommitIo,
192 source: &[u8],
193 space: ExGuid,
194 object: ExGuid,
195 range: std::ops::Range<u32>,
196 replacement: &str,
197) -> Result<(), CommitError> {
198 PreparedEdit::text(source, space, object, range, replacement)
199 .map_err(|error| CommitError {
200 state: CommitState::NotCommitted,
201 error: io::Error::new(ErrorKind::InvalidData, error),
202 })?
203 .commit(io)
204}
205
206/// An immutable writer-generated transition tied to its original snapshot.
207/// Persist intended revision identities from `as_bytes` before publishing an offline edit.
208/// Missing identities after native maintenance do not prove an edit was never published.
209pub struct PreparedEdit<'a> {
210 source: &'a [u8],
211 written: Vec<u8>,
212}
213
214impl<'a> PreparedEdit<'a> {
215 /// Applies page moves in slice order and publishes final order and indentation atomically.
216 /// Each page occurs once; all pages to move must be explicit, including selected subpages.
217 pub fn pages(source: &'a [u8], edits: &[crate::PageEdit]) -> Result<Self, crate::Error> {
218 Ok(Self {
219 source,
220 written: crate::pages::edit_pages(source, edits, &[])?,
221 })
222 }
223
224 /// Applies table-of-contents edits (sections and section groups: add, rename, colour,
225 /// order, remove) as one revision of a `.onetoc2` file.
226 pub fn table_of_contents(
227 source: &'a [u8],
228 edits: &[crate::TocEdit],
229 ) -> Result<Self, crate::Error> {
230 Ok(Self {
231 source,
232 written: crate::toc::edit_table_of_contents(source, edits)?,
233 })
234 }
235
236 /// Permanently removes explicitly selected pages from the section in one transaction.
237 /// Subpages must be selected explicitly; a surviving first subpage becomes top-level.
238 /// Creates no recycle-bin copies. Prior revisions remain stored; this is not secure erasure.
239 pub fn delete_pages_permanently(
240 source: &'a [u8],
241 pages: &[crate::ExGuid],
242 ) -> Result<Self, crate::Error> {
243 Ok(Self {
244 source,
245 written: crate::pages::edit_pages(source, &[], pages)?,
246 })
247 }
248
249 /// Publishes an edited page model as one revision per changed space.
250 /// The model must come from this snapshot; new paragraphs and outlines carry the identities
251 /// the model assigned, and content outside the model is untouched.
252 pub fn page(
253 source: &'a [u8],
254 space: ExGuid,
255 page: &crate::page::Page,
256 author: &str,
257 ) -> Result<Self, crate::Error> {
258 Ok(Self {
259 source,
260 written: crate::page::write::write_page(source, space, page, author)?,
261 })
262 }
263
264 /// `page` for a password-protected section: the revision is stored under the section's
265 /// key. The model must come from this snapshot unlocked with `password`.
266 #[cfg(feature = "protected")]
267 pub fn page_protected(
268 source: &'a [u8],
269 password: &str,
270 space: ExGuid,
271 page: &crate::page::Page,
272 author: &str,
273 ) -> Result<Self, crate::protected::Error> {
274 Ok(Self {
275 source,
276 written: crate::protected::write_page(source, password, space, page, author)?,
277 })
278 }
279
280 /// Creates a page and its section entry in one transaction, retaining the intent's identities.
281 pub fn create_page(source: &'a [u8], page: &crate::PageCreation) -> Result<Self, crate::Error> {
282 Ok(Self {
283 source,
284 written: page.apply(source)?,
285 })
286 }
287
288 /// Moves or removes a subtree and normalizes its containers in one revision.
289 pub fn tree(
290 source: &'a [u8],
291 space: ExGuid,
292 edit: &crate::TreeEdit,
293 ) -> Result<Self, crate::Error> {
294 Ok(Self {
295 source,
296 written: edit.apply(source, space)?,
297 })
298 }
299
300 /// Changes outline geometry or a paragraph's saved expansion state, preserving content.
301 pub fn outline(
302 source: &'a [u8],
303 space: ExGuid,
304 object: ExGuid,
305 edit: crate::OutlineEdit,
306 ) -> Result<Self, crate::Error> {
307 Ok(Self {
308 source,
309 written: edit.apply(source, space, object)?,
310 })
311 }
312
313 /// Joins adjacent ordinary paragraphs with native left-tag and text-identity semantics.
314 pub fn join(
315 source: &'a [u8],
316 space: ExGuid,
317 join: &crate::ParagraphJoin,
318 ) -> Result<Self, crate::Error> {
319 Ok(Self {
320 source,
321 written: join.apply(source, space)?,
322 })
323 }
324
325 /// Splits a paragraph and updates its children, lists, tags and title metadata atomically.
326 /// Fields and associated run metadata are rejected before I/O.
327 pub fn split(
328 source: &'a [u8],
329 space: ExGuid,
330 split: &crate::ParagraphSplit,
331 ) -> Result<Self, crate::Error> {
332 Ok(Self {
333 source,
334 written: split.apply(source, space)?,
335 })
336 }
337
338 /// Prepares an insertion and its dependent metadata in one revision, without I/O.
339 pub fn insert(
340 source: &'a [u8],
341 space: ExGuid,
342 insertion: &crate::Insertion,
343 ) -> Result<Self, crate::Error> {
344 Ok(Self {
345 source,
346 written: insertion.apply(source, space)?,
347 })
348 }
349
350 /// Validates and prepares a text edit without I/O, with `replace_text` semantics.
351 pub fn text(
352 source: &'a [u8],
353 space: ExGuid,
354 object: ExGuid,
355 range: std::ops::Range<u32>,
356 replacement: &str,
357 ) -> Result<Self, crate::Error> {
358 Ok(Self {
359 source,
360 written: crate::replace_text(source, space, object, range, replacement)?,
361 })
362 }
363
364 /// Changes character formatting over a UTF-16 range, preserving unselected runs and styles.
365 /// A zero-length range sets the insertion style only when the paragraph is empty.
366 /// Fields, associated run objects and boundaries splitting preserved run data are rejected.
367 pub fn format(
368 source: &'a [u8],
369 space: ExGuid,
370 object: ExGuid,
371 range: std::ops::Range<u32>,
372 attributes: &[crate::TextAttribute],
373 ) -> Result<Self, crate::Error> {
374 Ok(Self {
375 source,
376 written: crate::formatting::format_text(source, space, object, range, attributes)?,
377 })
378 }
379
380 /// The exact complete image this edit will publish; identities do not regenerate on commit.
381 /// Do not overwrite a live notebook with this image; use `commit` under exclusion.
382 pub fn as_bytes(&self) -> &[u8] {
383 &self.written
384 }
385
386 /// The bytes publishing this edit writes, against its snapshot's stamp.
387 pub fn transaction(&self) -> Transaction {
388 Transaction::between(self.source, &self.written)
389 }
390
391 /// `Transaction::commit` of this edit.
392 pub fn commit(&self, io: &mut impl CommitIo) -> Result<(), CommitError> {
393 self.transaction().commit(io)
394 }
395
396 /// `Transaction::commit_file` of this edit.
397 #[cfg(any(unix, windows))]
398 pub fn commit_file(&self, path: impl AsRef<Path>) -> Result<(), CommitError> {
399 self.transaction().commit_file(path)
400 }
401}
402
403/// `confirm_snapshot` under the same whole-file exclusion `commit_file` uses.
404#[cfg(any(unix, windows))]
405pub fn confirm_file_snapshot(path: impl AsRef<Path>, source: &[u8]) -> Result<(), CommitError> {
406 let mut io = FileIo::open(path, true).map_err(|error| CommitError {
407 state: CommitState::NotCommitted,
408 error,
409 })?;
410 let result = confirm_snapshot(&mut io, source);
411 io.finish(result)
412}
413
414/// Checks and flushes a snapshot's stamp, then refreshes its header version metadata.
415/// No revision is added; reread before using the snapshot for another physical commit.
166/// Checks that the file still has `base`'s stamp and flushes it, then refreshes its header
167/// version metadata. No revision is added; reread before committing on `base` again.
416168/// The caller must hold OneNote-compatible exclusion and independently establish which
417/// intents the snapshot contains. A successful read alone is not a durable acknowledgement.
418pub fn confirm_snapshot(io: &mut impl CommitIo, source: &[u8]) -> Result<(), CommitError> {
169/// intents the image contains. A successful read alone is not a durable acknowledgement.
170pub fn confirm(io: &mut impl CommitIo, base: &Stamp) -> Result<(), CommitError> {
419171 let mut state = CommitState::NotCommitted;
420172 let result = (|| -> io::Result<()> {
421 let header = crate::Header::parse(source).map_err(io::Error::other)?;
173 let header = crate::Header::parse(&base.header).map_err(io::Error::other)?;
422174 let generation = header
423175 .generation
424176 .checked_add(1)
......@@ -427,7 +179,7 @@ pub fn confirm_snapshot(io: &mut impl CommitIo, source: &[u8]) -> Result<(), Com
427179 version[..16].copy_from_slice(&crate::write::fresh_guid().map_err(io::Error::other)?);
428180 version[16..24].copy_from_slice(&generation.to_le_bytes());
429181 version[24..].copy_from_slice(&crate::write::fresh_guid().map_err(io::Error::other)?);
430 Stamp::of(source).map_err(io::Error::other)?.check(io)?;
182 base.check(io)?;
431183 state = CommitState::Unknown;
432184 io.flush()?;
433185 write_all(io, 212, &version)?;
......@@ -493,26 +245,6 @@ fn write_all(io: &mut impl CommitIo, mut offset: u64, mut bytes: &[u8]) -> io::R
493245 Ok(())
494246}
495247
496/// Publishes a scalar revision after checking the locked file against its snapshot.
497/// Unknown outcomes require rereading; Committed errors affect lock release.
498pub fn commit_property_bytes(
499 io: &mut impl CommitIo,
500 source: &[u8],
501 space: ExGuid,
502 object: ExGuid,
503 property: u32,
504 value: &[u8],
505) -> Result<(), CommitError> {
506 let written =
507 replace_property_bytes(source, space, object, property, value).map_err(|error| {
508 CommitError {
509 state: CommitState::NotCommitted,
510 error: io::Error::new(ErrorKind::InvalidData, error),
511 }
512 })?;
513 Transaction::between(source, &written).commit(io)
514}
515
516248/// What a commit requires unchanged since its snapshot: the header, which every committed
517249/// transaction and every placement rewrites (MS-ONESTORE 2.3.1 `guidFileVersion`), and the
518250/// length appends start from. Equal stamps name the same committed image.
......@@ -619,31 +351,6 @@ impl TryFrom<Wire> for Transaction {
619351}
620352
621353impl Transaction {
622 /// The transaction turning `source`, a parsed image, into `written`, which extends it.
623 pub(crate) fn between(source: &[u8], written: &[u8]) -> Self {
624 let differs = |at: &usize| source[*at] != written[*at];
625 let mut patches = Vec::new();
626 let mut offset = 1024;
627 while let Some(start) = (offset..source.len()).find(differs) {
628 let mut end = start + 1;
629 // Equal gaps shorter than a write request's overhead join the patch.
630 while let Some(next) = (end..source.len().min(end + 64)).find(differs) {
631 end = next + 1;
632 }
633 patches.push((start as u64, written[start..end].to_vec()));
634 offset = end;
635 }
636 Self {
637 base: Stamp {
638 header: *source.first_chunk().unwrap(),
639 length: source.len() as u64,
640 },
641 append: written[source.len()..].to_vec(),
642 patches,
643 header: *written.first_chunk().unwrap(),
644 }
645 }
646
647354 /// The image this transaction applies to.
648355 pub fn base(&self) -> &Stamp {
649356 &self.base
crates/onestore/src/create.rs+55-1
......@@ -17,6 +17,21 @@ pub(crate) fn string(value: &str) -> Vec<u8> {
1717 .collect()
1818}
1919
20/// An author's initials as OneNote 2010 stores them beside the name (`0x1c001df8`, with
21/// no MS-ONE entry): Office's default, the first letter of each word of the name.
22pub(crate) fn initials(author: &str) -> String {
23 author
24 .split_whitespace()
25 .filter_map(|word| word.chars().next())
26 .flat_map(char::to_uppercase)
27 .collect()
28}
29
30/// An author object's properties as OneNote 2010 writes them: initials, then the name.
31pub(crate) fn author_properties(name: &str) -> Vec<(u32, Vec<u8>)> {
32 vec![(0x1c001df8, string(&initials(name))), (0x1c001d75, string(name))]
33}
34
2035pub(crate) fn default_text_style() -> Vec<(u32, Vec<u8>)> {
2136 vec![
2237 (0x14001c3b, 0x409_u32.to_le_bytes().to_vec()),
......@@ -249,6 +264,7 @@ pub fn create_section(file_name: &str, text: &str, author: &str) -> Result<Vec<u
249264 last_modified(),
250265 (0x24001c20, id(23)),
251266 (0x1c001d75, string(author)),
267 (0x1c001df8, string(&initials(author))),
252268 (0x1c001d3c, string(crate::edit::automatic_title(text))),
253269 ],
254270 page_margins(),
......@@ -294,7 +310,7 @@ pub fn create_section(file_name: &str, text: &str, author: &str) -> Result<Vec<u
294310 NewObject {
295311 id: 26,
296312 jcid: 0x120001,
297 properties: vec![(0x1c001d75, string(author))],
313 properties: author_properties(author),
298314 },
299315 NewObject {
300316 id: 27,
......@@ -307,6 +323,44 @@ pub fn create_section(file_name: &str, text: &str, author: &str) -> Result<Vec<u
307323 create(file_name, FileType::Section, spaces)
308324}
309325
326/// Creates a section without pages, coloured `color` (COLORREF, `None` for no colour), as
327/// OneNote 2010 leaves one whose pages are all deleted: its section node lists no pages.
328/// Pages come as `op::SectionOp::Create` edits.
329pub fn create_empty_section(file_name: &str, color: Option<u32>) -> Result<Vec<u8>> {
330 if file_name.contains(['/', '\\', '\0']) || !file_name.to_ascii_lowercase().ends_with(".one") {
331 return Err(Error {
332 offset: 0,
333 message: "Invalid section name",
334 });
335 }
336 let timestamp = current_timestamps()?.1.to_le_bytes().to_vec();
337 let id = |value: u32| value.to_le_bytes().to_vec();
338 let spaces = vec![NewSpace {
339 id: 1,
340 roots: vec![(1, 10), (2, 11)],
341 objects: vec![
342 NewObject {
343 id: 10,
344 jcid: 0x60007,
345 properties: vec![
346 (0x1c001c30, fresh_guid()?.to_vec()),
347 (0x18001c65, timestamp),
348 ],
349 },
350 NewObject {
351 id: 11,
352 jcid: 0x20031,
353 properties: vec![
354 (0x14001d82, id(40)),
355 (0x1400348b, id(40)),
356 (0x14001cbe, id(color.unwrap_or(0xffff_ffff))),
357 ],
358 },
359 ],
360 }];
361 create(file_name, FileType::Section, spaces)
362}
363
310364/// Creates a notebook table of contents from section filenames and file identities.
311365pub fn create_table_of_contents(file_name: &str, sections: &[(&str, [u8; 16])]) -> Result<Vec<u8>> {
312366 if file_name.contains(['/', '\\', '\0'])
crates/onestore/src/edit.rs+4-22
......@@ -28,26 +28,6 @@ pub(crate) fn automatic_title(text: &str) -> &str {
2828/// Insertion at a run boundary uses the following run, except at the end of text.
2929/// The final run retains its insertion style even when emptied.
3030/// Replacing a range with identical text leaves its existing formatting unchanged.
31/// Returns a complete file image without I/O; use `commit_file_text` to update an existing file.
32pub fn replace_text(
33 source: &[u8],
34 space: ExGuid,
35 object: ExGuid,
36 range: Range<u32>,
37 replacement: &str,
38) -> Result<Vec<u8>, Error> {
39 if range.start > range.end || replacement.contains(['\0', '\n', '\r', '\u{fffc}']) {
40 return Err(Error {
41 offset: 0,
42 message: "Use a valid text range and ordinary paragraph text",
43 });
44 }
45 crate::active::write(source, space, |active| {
46 text_changes(active, object, range, replacement)
47 })
48}
49
50/// `replace_text` on an active page.
5131pub(crate) fn text_changes(
5232 active: &ActivePage<'_>,
5333 object: ExGuid,
......@@ -312,7 +292,9 @@ fn rewrite(
312292 .roots
313293 .get(&2)
314294 .ok_or_else(|| invalid("Page title metadata is unavailable"))?;
315 let Kind::Metadata { title, .. } = &revision.nodes[metadata].kind else {
295 let (Kind::Metadata { title, .. } | Kind::ConflictMetadata { title, .. }) =
296 &revision.nodes[metadata].kind
297 else {
316298 return Err(invalid("Page title metadata is unavailable"));
317299 };
318300 let cached: Vec<_> = title_text
......@@ -544,7 +526,7 @@ pub(crate) fn update_title(
544526 .roots
545527 .get(&2)
546528 .ok_or_else(|| invalid("Page title metadata is unavailable"))?;
547 if raw.objects[metadata].jcid != 0x20030 {
529 if ![0x20030, 0x20038].contains(&raw.objects[metadata].jcid) {
548530 return Err(invalid("Page title metadata is unavailable"));
549531 }
550532 let title = string(&title);
crates/onestore/src/formatting.rs+2-14
......@@ -100,20 +100,8 @@ impl TextAttribute {
100100 }
101101}
102102
103pub(crate) fn format_text(
104 source: &[u8],
105 space: ExGuid,
106 object: ExGuid,
107 range: Range<u32>,
108 attributes: &[TextAttribute],
109) -> Result<Vec<u8>, Error> {
110 crate::active::write(source, space, |active| {
111 format_changes(active, object, range, attributes, &[])
112 })
113}
114
115/// `format_text` on an active page, also removing the `cleared` properties from the range's
116/// run styles so the text inherits them.
103/// Character formatting set over a UTF-16 range, also removing the `cleared` properties
104/// from the range's run styles so the text inherits them.
117105pub(crate) fn format_changes(
118106 active: &ActivePage<'_>,
119107 object: ExGuid,
crates/onestore/src/insertion.rs+11-191
......@@ -6,10 +6,8 @@ use crate::{
66 op::content::{NATIVE_INDENTS, measurement_bytes},
77 write::{PropertyObject, fresh_guid},
88};
9use serde::{Deserialize, Serialize};
109use std::{
1110 collections::{BTreeMap, BTreeSet},
12 ops::Range,
1311 sync::Arc,
1412};
1513
......@@ -17,59 +15,29 @@ fn invalid(message: &'static str) -> Error {
1715 Error { offset: 0, message }
1816}
1917
20#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
21#[serde(deny_unknown_fields)]
18#[derive(Debug, Clone, PartialEq)]
2219enum Placement {
2320 Paragraph { before: Option<ExGuid> },
2421 Outline { x: f32, y: f32 },
2522}
2623
27#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
28#[serde(deny_unknown_fields)]
29struct FormatSpan {
30 guid: [u8; 16],
31 range: Range<u32>,
32 attributes: Vec<crate::TextAttribute>,
33}
34
35/// A paragraph or outline insertion with stable object identities and creation time.
36/// Retain this intent across rebases; constructing another intent allocates different identities.
37#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
38#[serde(deny_unknown_fields)]
39pub struct Insertion {
24/// A paragraph or outline insertion with its creation time; `changes_as` names the new
25/// objects.
26#[derive(Debug, Clone, PartialEq)]
27pub(crate) struct Insertion {
4028 guid: [u8; 16],
4129 parent: ExGuid,
4230 placement: Placement,
4331 text: String,
4432 author: String,
4533 created: u32,
46 #[serde(default, skip_serializing_if = "Vec::is_empty")]
47 formats: Vec<FormatSpan>,
4834}
4935
5036impl Insertion {
51 /// Adds a nonoverlapping UTF-16 formatting span, retaining all existing identities.
52 /// Gaps use the ordinary insertion style; only empty text accepts a zero-length span.
53 pub fn with_formatting(
54 &self,
55 range: Range<u32>,
56 attributes: &[crate::TextAttribute],
57 ) -> Result<Self, Error> {
58 let mut intent = self.clone();
59 intent.formats.push(FormatSpan {
60 guid: fresh_guid()?,
61 range,
62 attributes: attributes.to_vec(),
63 });
64 intent.formats.sort_by_key(|span| span.range.start);
65 intent.validate()?;
66 Ok(intent)
67 }
68
6937 /// Inserts before a direct child, or appends when `before` is None.
7038 /// The parent must be an editable outline, paragraph, outline group or table cell.
7139 /// Carriage returns represent soft line breaks; line feeds and embedded-field markers are rejected.
72 pub fn paragraph(
40 pub(crate) fn paragraph(
7341 parent: ExGuid,
7442 before: Option<ExGuid>,
7543 text: &str,
......@@ -79,38 +47,10 @@ impl Insertion {
7947 }
8048
8149 /// Adds an outline to an editable page at coordinates measured in points.
82 pub fn outline(page: ExGuid, x: f32, y: f32, text: &str, author: &str) -> Result<Self, Error> {
50 pub(crate) fn outline(page: ExGuid, x: f32, y: f32, text: &str, author: &str) -> Result<Self, Error> {
8351 Self::new(page, Placement::Outline { x, y }, text, author)
8452 }
8553
86 /// Changes a paragraph intent's placement while retaining its identities, text and author.
87 pub fn reposition_paragraph(
88 &self,
89 parent: ExGuid,
90 before: Option<ExGuid>,
91 ) -> Result<Self, Error> {
92 if !matches!(self.placement, Placement::Paragraph { .. }) {
93 return Err(invalid("An outline intent cannot become a paragraph"));
94 }
95 let mut intent = self.clone();
96 intent.parent = parent;
97 intent.placement = Placement::Paragraph { before };
98 intent.validate()?;
99 Ok(intent)
100 }
101
102 /// Changes an outline intent's placement while retaining its identities, text and author.
103 pub fn reposition_outline(&self, page: ExGuid, x: f32, y: f32) -> Result<Self, Error> {
104 if !matches!(self.placement, Placement::Outline { .. }) {
105 return Err(invalid("A paragraph intent cannot become an outline"));
106 }
107 let mut intent = self.clone();
108 intent.parent = page;
109 intent.placement = Placement::Outline { x, y };
110 intent.validate()?;
111 Ok(intent)
112 }
113
11454 fn new(parent: ExGuid, placement: Placement, text: &str, author: &str) -> Result<Self, Error> {
11555 let intent = Self {
11656 guid: fresh_guid()?,
......@@ -119,28 +59,11 @@ impl Insertion {
11959 text: text.to_owned(),
12060 author: author.to_owned(),
12161 created: current_timestamps()?.0,
122 formats: Vec::new(),
12362 };
12463 intent.validate()?;
12564 Ok(intent)
12665 }
12766
128 /// Identity of the new paragraph, or the new outline for an outline insertion.
129 pub fn object(&self) -> ExGuid {
130 ExGuid {
131 guid: self.guid,
132 n: 1,
133 }
134 }
135
136 /// Identity of the insertion's ordinary rich-text object.
137 pub fn text_object(&self) -> ExGuid {
138 ExGuid {
139 guid: self.guid,
140 n: 2,
141 }
142 }
143
14467 fn validate(&self) -> Result<(), Error> {
14568 if self.guid == [0; 16]
14669 || self.parent.guid == [0; 16]
......@@ -156,69 +79,11 @@ impl Insertion {
15679 {
15780 return Err(invalid("Outline coordinates must be finite"));
15881 }
159 if !self.formats.is_empty() {
160 let units: Vec<_> = self.text.encode_utf16().collect();
161 let length = u32::try_from(units.len())
162 .map_err(|_| invalid("Text exceeds UTF-16 offset range"))?;
163 let mut identities = BTreeSet::from([self.guid]);
164 let mut end = 0;
165 for span in &self.formats {
166 if span.guid == [0; 16] || !identities.insert(span.guid) {
167 return Err(invalid("Formatting identities must be distinct"));
168 }
169 if span.range.start > span.range.end
170 || span.range.end > length
171 || span.range.start < end
172 || (span.range.is_empty() && (!units.is_empty() || self.formats.len() != 1))
173 {
174 return Err(invalid(
175 "Formatting spans must be nonoverlapping ranges within the new text",
176 ));
177 }
178 for boundary in [span.range.start, span.range.end] {
179 if boundary > 0
180 && boundary < length
181 && (0xd800..=0xdbff).contains(&units[boundary as usize - 1])
182 {
183 return Err(invalid("Formatting boundary splits a surrogate pair"));
184 }
185 }
186 crate::formatting::attribute_values(&span.attributes)?;
187 end = span.range.end;
188 }
189 }
19082 Ok(())
19183 }
19284
193 pub(crate) fn apply(&self, source: &[u8], space: ExGuid) -> Result<Vec<u8>, Error> {
194 self.validate()?;
195 crate::active::write(source, space, |active| self.changes(active))
196 }
197
198 pub(crate) fn changes(&self, active: &ActivePage<'_>) -> Result<Changes, Error> {
199 let paragraph = ExGuid {
200 guid: self.guid,
201 n: if matches!(self.placement, Placement::Outline { .. }) {
202 3
203 } else {
204 1
205 },
206 };
207 let changes = self.changes_as(active, self.object(), paragraph, self.text_object())?;
208 let raw = &active.live.revision;
209 if changes
210 .keys()
211 .any(|id| id.guid == self.guid && raw.objects.contains_key(id))
212 {
213 return Err(invalid(
214 "An insertion identity is already present; reconcile the existing edit",
215 ));
216 }
217 Ok(changes)
218 }
219
220 /// `changes` creating the outline or paragraph `object`, the paragraph `paragraph` and its
221 /// text `text` under those identities.
85 /// The objects creating the outline or paragraph `object`, the paragraph `paragraph` and
86 /// its text `text` under those identities.
22287 pub(crate) fn changes_as(
22388 &self,
22489 active: &ActivePage<'_>,
......@@ -321,7 +186,7 @@ impl Insertion {
321186 (0x10001cfe, 0x409_u16.to_le_bytes().to_vec()),
322187 ],
323188 ),
324 (author, 0x120001, vec![(0x1c001d75, string(&self.author))]),
189 (author, 0x120001, crate::create::author_properties(&self.author)),
325190 (default, 0x12004d, default_text_style()),
326191 ] {
327192 new.insert(
......@@ -333,51 +198,6 @@ impl Insertion {
333198 },
334199 );
335200 }
336 if !self.formats.is_empty() {
337 let mut segments = Vec::new();
338 let mut end = 0;
339 for span in &self.formats {
340 if end < span.range.start {
341 segments.push((span.range.start, default));
342 }
343 let id = ExGuid {
344 guid: span.guid,
345 n: 1,
346 };
347 let mut style = PropertyObject {
348 jcid: 0x12004d,
349 bytes: properties(&default_text_style())?,
350 global_ids: Arc::new(BTreeMap::from([(0, span.guid)])),
351 };
352 let values = crate::formatting::attribute_values(&span.attributes)?;
353 style.set(
354 &values
355 .iter()
356 .map(|(id, value)| (*id, value.as_slice()))
357 .collect::<Vec<_>>(),
358 )?;
359 new.insert(id, style);
360 segments.push((span.range.end, id));
361 end = span.range.end;
362 }
363 let length = u32::try_from(self.text.encode_utf16().count())
364 .map_err(|_| invalid("Text exceeds UTF-16 offset range"))?;
365 if end < length {
366 segments.push((length, default));
367 }
368 if !segments.iter().any(|(_, id)| *id == default) {
369 new.remove(&default);
370 }
371 let target = new.get_mut(&text).unwrap();
372 let mut references = Vec::new();
373 let mut ends = Vec::new();
374 for (end, id) in segments {
375 references.extend_from_slice(&target.reference(id)?);
376 ends.extend_from_slice(&end.to_le_bytes());
377 }
378 ends.truncate(ends.len() - 4);
379 target.set(&[(0x24001e13, &references), (0x1c001e12, &ends)])?;
380 }
381201 if let Placement::Outline { x, y } = self.placement {
382202 let children = reference(paragraph)?;
383203 new.insert(
......@@ -449,7 +269,7 @@ impl Insertion {
449269 .roots
450270 .get(&2)
451271 .ok_or_else(|| invalid("Page title metadata is unavailable"))?;
452 if raw.objects[metadata].jcid != 0x20030 {
272 if ![0x20030, 0x20038].contains(&raw.objects[metadata].jcid) {
453273 return Err(invalid("Page title metadata is unavailable"));
454274 }
455275 let title = string(&title);
crates/onestore/src/lib.rs+8-16
......@@ -28,29 +28,21 @@ mod toc;
2828mod tree;
2929mod write;
3030
31pub use commit::{
32 CommitError, CommitIo, CommitState, PreparedEdit, Stamp, Transaction, commit_property_bytes,
33 commit_text, confirm_snapshot, place,
34};
31pub use commit::{CommitError, CommitIo, CommitState, Stamp, Transaction, confirm, place};
3532#[cfg(any(unix, windows))]
36pub use commit::{
37 commit_file_property, commit_file_text, confirm_file_snapshot, place_file, read_file,
38 read_file_limited,
39};
40pub use create::{create_section, create_table_of_contents};
41pub use edit::replace_text;
33pub use commit::{confirm_file, place_file, read_file, read_file_limited};
34pub use create::{create_empty_section, create_section, create_table_of_contents};
4235pub use files::FileDataReference;
4336pub use formatting::{FONT_SIZES, TextAttribute};
44pub use insertion::Insertion;
37pub(crate) use insertion::Insertion;
4538pub use objects::{Object, ObjectData, ObjectReferences, ResolvedRevision};
4639pub use outline::OutlineEdit;
47pub use pages::{PageCreation, PageEdit, PagePosition};
48pub use paragraph::{ParagraphJoin, ParagraphSplit};
40pub use pages::{ConflictPage, PageCreation, PageEdit, PagePosition};
41pub(crate) use paragraph::{ParagraphJoin, ParagraphSplit};
4942pub use properties::{IdStream, Property, PropertySets, Value};
5043pub use revisions::{ExGuid, ObjectSpace, Revision, RevisionIndex};
5144pub use section::{Arena, Section};
5245pub use snapshot::{read_snapshot, read_storage_snapshot};
5346pub use store::{Chunk, Error, FileType, Header, Node, NodeList, Reference, Store};
54pub use toc::TocEdit;
55pub use tree::TreeEdit;
56pub use write::replace_property_bytes;
47pub use toc::{TocEdit, edit_table_of_contents};
48pub(crate) use tree::TreeEdit;
crates/onestore/src/op/apply.rs+278-35
......@@ -49,6 +49,28 @@ impl<'a> Section<'a> {
4949 Op::Section(op) => section.apply_section(author, op)?,
5050 }
5151 }
52 let spaces: BTreeSet<_> = edit
53 .ops
54 .iter()
55 .filter_map(|op| match op {
56 Op::Page { space, .. } => Some(*space),
57 Op::Section(_) => None,
58 })
59 .collect();
60 for space in spaces {
61 // A page the edit went on to delete has no cells left to check.
62 let Ok(page) = section.active(space) else {
63 continue;
64 };
65 if page.view.nodes.values().any(|node| {
66 matches!(node.kind, Kind::Cell { .. }) && node.children.is_empty()
67 }) {
68 return Err(OpError::Unsupported(
69 "A table cell keeps a paragraph; insert its replacement first",
70 )
71 .into());
72 }
73 }
5274 Ok(())
5375 })
5476 })
......@@ -99,28 +121,26 @@ impl<'a> Section<'a> {
99121 }
100122 SectionOp::Import { creation, page } => {
101123 self.apply_section(author, &SectionOp::Create(creation.clone()))?;
124 self.fill(author, creation.space(), page)?;
125 }
126 SectionOp::Conflict {
127 of,
128 creation,
129 page,
130 objects,
131 } => {
132 let level = self
133 .pages()?
134 .into_iter()
135 .find(|(space, ..)| space == of)
136 .map(|(.., level)| level)
137 .ok_or(OpError::TargetUnavailable(*of))?;
102138 let space = creation.space();
103 let before = self.page(space)?;
104 let mut after = page.clone();
105 // The created page's title stays; the imported page brings the rest.
106 after.objects.retain(|object| !matches!(object, PageObject::Title(_)));
107 for object in before.objects.iter().rev() {
108 if let PageObject::Title(_) = object {
109 after.objects.insert(0, object.clone());
110 }
111 }
112 after.title = before.title.clone();
113 after.identity = before.identity;
114 after.created = before.created;
115 after.margin_origin = before.margin_origin;
116 for op in super::lower_page(&before, &after)? {
117 Writer {
118 section: self,
119 space,
120 author,
121 }
122 .apply(&op)?;
123 }
139 let (roots, created, metadata) = creation.conflict(self, &page.title, level)?;
140 self.create_space(space, roots, created)?;
141 self.fill(author, space, page)?;
142 self.mark_conflict(space, creation, page, objects)?;
143 self.link_conflict(*of, space, metadata)?;
124144 }
125145 SectionOp::Pages(edits) => {
126146 for (space, changes) in crate::pages::section_changes(self, edits, &[])? {
......@@ -128,13 +148,235 @@ impl<'a> Section<'a> {
128148 }
129149 }
130150 SectionOp::Delete(pages) => {
131 for (space, changes) in crate::pages::section_changes(self, &[], pages)? {
132 self.apply_changes(space, &[], changes)?;
151 let mut listed = Vec::new();
152 let conflicts = self.conflicts()?;
153 for space in pages {
154 let of = conflicts.iter().find_map(|(of, conflicts)| {
155 conflicts.iter().any(|page| page.space == *space).then_some(*of)
156 });
157 match of {
158 Some(of) => self.unlink_conflict(of, *space)?,
159 None => listed.push(*space),
160 }
161 }
162 if !listed.is_empty() {
163 for (space, changes) in crate::pages::section_changes(self, &[], &listed)? {
164 self.apply_changes(space, &[], changes)?;
165 }
133166 }
134167 }
168 SectionOp::Color(color) => {
169 let root = self.root();
170 let page = self.active(root)?;
171 let metadata = *page
172 .view
173 .roots
174 .get(&2)
175 .filter(|id| page.live.revision.objects.contains_key(id))
176 .ok_or_else(|| invalid("The section has no metadata to colour"))?;
177 let mut object = crate::write::PropertyObject::from_object(
178 &page.live.revision.objects[&metadata],
179 )?;
180 object.set(&[(0x14001cbe, &color.unwrap_or(0xffff_ffff).to_le_bytes())])?;
181 self.apply_changes(root, &[], BTreeMap::from([(metadata, object)]))?;
182 }
135183 }
136184 Ok(())
137185 }
186
187 /// Writes `page`'s content onto the page just created in `space`, whose title stays.
188 fn fill(&mut self, author: &str, space: ExGuid, page: &crate::page::Page) -> Result<(), Failure> {
189 let before = self.page(space)?;
190 let mut after = page.clone();
191 after.objects.retain(|object| !matches!(object, PageObject::Title(_)));
192 for object in &mut after.objects {
193 // Stroke points are page coordinates; new ink takes its position from them.
194 if let PageObject::Ink(ink) = object {
195 ink.layout = Default::default();
196 }
197 }
198 for object in before.objects.iter().rev() {
199 if let PageObject::Title(_) = object {
200 after.objects.insert(0, object.clone());
201 }
202 }
203 after.title = before.title.clone();
204 after.identity = before.identity;
205 after.created = before.created;
206 after.margin_origin = before.margin_origin;
207 for op in super::lower_page(&before, &after)? {
208 Writer {
209 section: self,
210 space,
211 author,
212 }
213 .apply(&op)?;
214 }
215 Ok(())
216 }
217
218 /// Marks a filled conflict page as OneNote 2010 does: the page read-only, deletable and
219 /// a conflict page (MS-ONE 2.1.2), and each of `objects` a conflict object (2.1.1). The
220 /// created title stands for `page`'s, so objects of that title mark its counterparts.
221 fn mark_conflict(
222 &mut self,
223 space: ExGuid,
224 creation: &crate::PageCreation,
225 page: &crate::page::Page,
226 objects: &[ExGuid],
227 ) -> Result<(), Failure> {
228 let created = |n| ExGuid {
229 guid: creation.space().guid,
230 n,
231 };
232 let mut titles = BTreeMap::new();
233 for object in &page.objects {
234 let PageObject::Title(title) = object else {
235 continue;
236 };
237 titles.insert(title.id, created(13));
238 if let Some(outline) = title.outlines.first() {
239 titles.insert(outline.id, created(14));
240 if let Some(paragraph) = outline.paragraphs.first() {
241 titles.insert(paragraph.id, created(15));
242 if let Some(text) = paragraph.text() {
243 titles.insert(text.id, created(16));
244 }
245 }
246 }
247 }
248 let active = self.active(space)?;
249 let mut changes = BTreeMap::new();
250 let mut page_node = crate::write::PropertyObject::from_object(
251 &active.live.revision.objects[&creation.object()],
252 )?;
253 page_node.set(&[(0x88001cde, &[]), (0x88001d0c, &[]), (0x88001d7c, &[])])?;
254 changes.insert(creation.object(), page_node);
255 for id in objects {
256 let id = titles.get(id).copied().unwrap_or(*id);
257 if !active.live.is_reachable(id) {
258 return Err(OpError::TargetUnavailable(id).into());
259 }
260 let mut object =
261 crate::write::PropertyObject::from_object(&active.live.revision.objects[&id])?;
262 // A table is rendered as conflicting but is not selectable as such.
263 if object.jcid == 0x60022 {
264 object.set(&[(0x88001d96, &[])])?;
265 } else {
266 object.set(&[(0x88001ddb, &[]), (0x88001d96, &[])])?;
267 }
268 changes.insert(id, object);
269 }
270 self.apply_changes(space, &[], changes)?;
271 Ok(())
272 }
273
274 /// Lists conflict page `space` under page `of` as OneNote 2010 does: the page's manifest
275 /// references it and keeps a copy of its metadata, and the page's metadata and the
276 /// section's copy of it say the page has conflict pages (MS-ONE 2.2.34, 2.3.71).
277 fn link_conflict(
278 &mut self,
279 of: ExGuid,
280 space: ExGuid,
281 mut metadata: crate::write::PropertyObject,
282 ) -> Result<(), Failure> {
283 use crate::{pages::set_references, write::PropertyObject};
284 let page = self.active(of)?;
285 let manifest_id = page.view.roots[&1];
286 let manifest_node = &page.view.nodes[&manifest_id];
287 let mut spaces = manifest_node.spaces.clone();
288 spaces.push(space);
289 let mut copies = objects(manifest_node, 0x24003442);
290 let copy = crate::pages::metadata_id(space);
291 copies.push(copy);
292 metadata.reference(copy)?;
293 let mut manifest = PropertyObject::from_object(&page.live.revision.objects[&manifest_id])?;
294 set_references(&mut manifest, 0x2c001d63, &spaces)?;
295 set_references(&mut manifest, 0x24003442, &copies)?;
296 self.apply_changes(
297 of,
298 &[],
299 BTreeMap::from([(manifest_id, manifest), (copy, metadata)]),
300 )?;
301 self.has_conflicts(of, true)
302 }
303
304 /// Deletes conflict page `space` of page `of` as OneNote 2010 does: the manifest stops
305 /// referencing it (its metadata copy stays) and the last one's removal clears
306 /// `HasConflictPages`; the conflict page's own space is left as it is.
307 fn unlink_conflict(&mut self, of: ExGuid, space: ExGuid) -> Result<(), Failure> {
308 use crate::{pages::set_references, write::PropertyObject};
309 let page = self.active(of)?;
310 let manifest_id = page.view.roots[&1];
311 let spaces: Vec<ExGuid> = page.view.nodes[&manifest_id]
312 .spaces
313 .iter()
314 .copied()
315 .filter(|listed| *listed != space)
316 .collect();
317 let mut manifest = PropertyObject::from_object(&page.live.revision.objects[&manifest_id])?;
318 if spaces.is_empty() {
319 manifest.remove(&[0x2c001d63])?;
320 } else {
321 set_references(&mut manifest, 0x2c001d63, &spaces)?;
322 }
323 self.apply_changes(of, &[], BTreeMap::from([(manifest_id, manifest)]))?;
324 if spaces.is_empty() {
325 self.has_conflicts(of, false)?;
326 }
327 Ok(())
328 }
329
330 /// Sets or clears `HasConflictPages` on page `of`'s metadata and on the section's copy
331 /// of it, where its series keeps one.
332 fn has_conflicts(&mut self, of: ExGuid, value: bool) -> Result<(), Failure> {
333 use crate::write::PropertyObject;
334 let flag = |object: &crate::Object<'_>| -> Result<PropertyObject, Error> {
335 let mut object = PropertyObject::from_object(object)?;
336 if value {
337 object.set(&[(0x88001d97, &[])])?;
338 } else {
339 object.remove(&[0x88001d97])?;
340 }
341 Ok(object)
342 };
343 let page = self.active(of)?;
344 let stored_id = page.view.roots[&2];
345 let identity = crate::pages::metadata_guid(&page.view.nodes[&stored_id])?;
346 let stored = flag(&page.live.revision.objects[&stored_id])?;
347 self.apply_changes(of, &[], BTreeMap::from([(stored_id, stored)]))?;
348 let root = self.root();
349 let section = self.active(root)?;
350 let copy = section
351 .view
352 .nodes
353 .values()
354 .find(|node| node.spaces.contains(&of))
355 .map(|series| objects(series, 0x24003442))
356 .unwrap_or_default()
357 .into_iter()
358 .find(|id| {
359 section.view.nodes.get(id).is_some_and(|node| {
360 crate::pages::metadata_guid(node).is_ok_and(|guid| guid == identity)
361 })
362 });
363 if let Some(id) = copy {
364 let object = flag(&section.live.revision.objects[&id])?;
365 self.apply_changes(root, &[], BTreeMap::from([(id, object)]))?;
366 }
367 Ok(())
368 }
369}
370
371/// The objects an element's reference property names.
372fn objects(node: &crate::document::Element<'_>, property: u32) -> Vec<ExGuid> {
373 node.extra[0]
374 .iter()
375 .find_map(|field| match &field.value {
376 crate::document::FieldValue::Objects(ids) if field.id == property => Some(ids.clone()),
377 _ => None,
378 })
379 .unwrap_or_default()
138380}
139381
140382/// Whether an op may store more than one revision of a page, so its failure part way needs
......@@ -348,6 +590,19 @@ impl<'a> Writer<'_, 'a> {
348590 Ok(BTreeMap::from([(*metadata, object)]))
349591 })
350592 }
593 PageOp::Color(color) => self.write(|page| {
594 let [node] = crate::active::manifest_pages(&page.view)[..] else {
595 return Err(invalid("The page has no page node to colour"));
596 };
597 let mut object =
598 crate::write::PropertyObject::from_object(&page.live.revision.objects[&node])?;
599 // OneNote's "No color" removes the property.
600 match color {
601 Some(color) => object.set(&[(crate::page::PAGE_COLOR, &color.to_le_bytes())])?,
602 None => object.remove(&[crate::page::PAGE_COLOR])?,
603 }
604 Ok(BTreeMap::from([(node, object)]))
605 }),
351606 PageOp::Insert {
352607 container,
353608 before,
......@@ -386,18 +641,6 @@ impl<'a> Writer<'_, 'a> {
386641 }
387642 PageOp::Delete { object } => {
388643 self.target(*object)?;
389 let page = self.page()?;
390 if matches!(page.view.nodes[object].kind, Kind::Paragraph { .. }) {
391 let container = self.container(*object)?;
392 if matches!(self.page()?.view.nodes[&container].kind, Kind::Cell { .. })
393 && levels::flat_children(&self.page()?.view, container) == [*object]
394 {
395 return Err(OpError::Unsupported(
396 "A table cell keeps a paragraph; insert its replacement first",
397 )
398 .into());
399 }
400 }
401644 let edit = TreeEdit::delete(*object, self.author)?;
402645 self.write(|page| edit.changes(page))
403646 }
crates/onestore/src/op/levels.rs-32
......@@ -14,21 +14,6 @@ fn invalid(message: &'static str) -> Error {
1414 Error { offset: 0, message }
1515}
1616
17/// The children of `container` with its outline groups lifted into it, in order.
18pub(crate) fn flat_children(view: &Revision<'_>, container: ExGuid) -> Vec<ExGuid> {
19 fn flat(view: &Revision<'_>, id: ExGuid, out: &mut Vec<ExGuid>) {
20 for child in &view.nodes[&id].children {
21 match view.nodes[child].kind {
22 Kind::OutlineGroup => flat(view, *child, out),
23 _ => out.push(*child),
24 }
25 }
26 }
27 let mut children = Vec::new();
28 flat(view, container, &mut children);
29 children
30}
31
3217/// How much deeper than `container` each of its children lies, groups lifted.
3318pub(crate) fn stored_depths(view: &Revision<'_>, container: ExGuid) -> Vec<(ExGuid, u32)> {
3419 fn walk(view: &Revision<'_>, id: ExGuid, base: u32, out: &mut Vec<(ExGuid, u32)>) {
......@@ -46,23 +31,6 @@ pub(crate) fn stored_depths(view: &Revision<'_>, container: ExGuid) -> Vec<(ExGu
4631 out
4732}
4833
49/// Lifts the children of `container`'s outline groups into it.
50pub(crate) fn ungroup_changes(active: &ActivePage<'_>, container: ExGuid) -> Result<Changes, Error> {
51 let children = flat_children(&active.view, container);
52 let parents = active.editable_parents(container)?;
53 let modified = crate::create::current_timestamps()?.0.to_le_bytes();
54 let raw = &active.live.revision;
55 let mut holder = PropertyObject::from_object(&raw.objects[&container])?;
56 let mut references = Vec::new();
57 for child in children {
58 references.extend(holder.reference(child)?);
59 }
60 holder.set(&[(0x24001c20, &references), (0x14001d7a, &modified)])?;
61 let mut changed = BTreeMap::from([(container, holder)]);
62 crate::formatting::touch_ancestors(raw, parents, container, &modified, &mut changed)?;
63 Ok(changed)
64}
65
6634/// Children in runs: one child at the container's child level, or a run sharing a deeper
6735/// depth.
6836pub(crate) type Runs = Vec<(u32, Vec<ExGuid>)>;
crates/onestore/src/op/lower.rs+4-15
......@@ -44,6 +44,7 @@ pub fn lower(
4444 identity: None,
4545 created: None,
4646 margin_origin: [0.0; 2],
47 color: None,
4748 objects: vec![PageObject::Outline(Outline {
4849 id: container,
4950 title: false,
......@@ -163,16 +164,6 @@ impl<'a> View<'a> {
163164 self.paragraphs.get(&paragraph).and_then(|p| p.text())
164165 }
165166
166 /// How much deeper than `container` each of its children lies: their outline levels
167 /// less the paragraph's, or less nothing in an outline or cell.
168 pub(crate) fn depths(&self, container: ExGuid, children: &[ExGuid]) -> Option<Vec<u32>> {
169 let base = self.paragraphs.get(&container).map_or(0, |p| p.level);
170 children
171 .iter()
172 .map(|id| self.paragraphs.get(id)?.level.checked_sub(base))
173 .collect()
174 }
175
176167 /// Each outline's paragraphs in document order, a table's cells following its paragraph.
177168 pub(crate) fn documents(&self) -> impl Iterator<Item = Vec<&'a PageParagraph>> + '_ {
178169 fn walk<'a>(list: &'a [PageParagraph], out: &mut Vec<&'a PageParagraph>) {
......@@ -677,11 +668,6 @@ pub(crate) fn paragraph_fields(
677668 })
678669}
679670
680/// `paragraph_fields` encoded, as the page writer stores them.
681pub(crate) fn paragraph_change(stored: &Paragraph, target: &Paragraph) -> Result<Values, Error> {
682 paragraph_fields(stored, target)?.values()
683}
684
685671/// Whether two tag lists store the same tags.
686672pub(crate) fn same_tags(a: &[Tag], b: &[Tag]) -> bool {
687673 let key = |t: &Tag| {
......@@ -1555,6 +1541,9 @@ impl Lowering {
15551541 let shown = fields.iter().all(|(id, shown)| {
15561542 model::paragraph_text(&self.current, *id).is_some_and(|text| text.text.text() == shown)
15571543 });
1544 if before.color != after.color {
1545 self.emit(PageOp::Color(after.color))?;
1546 }
15581547 if before.created == after.created && shown {
15591548 return Ok(());
15601549 }
crates/onestore/src/op/mod.rs+19-4
......@@ -19,7 +19,7 @@ pub(crate) mod model;
1919pub(crate) mod properties;
2020pub(crate) mod table;
2121#[cfg(test)]
22mod tests;
22pub(crate) mod tests;
2323
2424pub use lower::{lower, lower_page};
2525/// `op` applied to `page`, the model of what the section holds: the page `Section::apply`
......@@ -29,7 +29,7 @@ pub use model::apply as predict;
2929/// Property identifiers with their encoded values.
3030pub(crate) type Values = Vec<(u32, Vec<u8>)>;
3131
32/// One user action, applied whole or not at all.
32/// One user action, applied whole or not at all; it leaves every table cell a paragraph.
3333#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
3434#[serde(deny_unknown_fields)]
3535pub struct Edit {
......@@ -80,6 +80,8 @@ pub enum PageOp {
8080 created: u64,
8181 fields: Vec<(ExGuid, String)>,
8282 },
83 /// The page's colour (View, Page Color), COLORREF; `None` is OneNote's "No color".
84 Color(Option<u32>),
8385 /// Inserts paragraphs before a direct child of `container`, or last. Children follow
8486 /// their parents in `paragraphs`; levels are absolute. Text keeps its spans' formats;
8587 /// lists, tags, styles and collapse state are set by their own ops.
......@@ -108,7 +110,7 @@ pub enum PageOp {
108110 parent: Option<ExGuid>,
109111 before: Option<ExGuid>,
110112 },
111 /// Removes a subtree; an emptied table cell receives an empty paragraph.
113 /// Removes a subtree.
112114 Delete { object: ExGuid },
113115 /// Sets a paragraph's absolute outline level, regrouping its container.
114116 Level { paragraph: ExGuid, level: u32 },
......@@ -204,10 +206,23 @@ pub enum SectionOp {
204206 /// A page created with `creation` holding `page`'s content under the identities it
205207 /// carries (`Page::copy` gives fresh ones); the created title stays.
206208 Import { creation: PageCreation, page: Page },
209 /// A read-only conflict page of the listed page `of`, as OneNote 2010 keeps the version of
210 /// a page a merge could not take: created with `creation` (not placed, its author the user
211 /// whose version it is, a title only where `page` has one) and holding `page` as `Import`
212 /// does, with `objects`, identities on `page`, marked as the conflicting ones.
213 Conflict {
214 of: ExGuid,
215 creation: PageCreation,
216 page: Page,
217 objects: Vec<ExGuid>,
218 },
207219 /// Page moves and indentation, in order.
208220 Pages(Vec<PageEdit>),
209 /// Removes pages permanently; their revisions stay stored.
221 /// Removes pages permanently, listed or conflict pages; their revisions stay stored.
210222 Delete(Vec<ExGuid>),
223 /// The section's colour as COLORREF, kept in its own metadata; `None` is OneNote's
224 /// "no colour".
225 Color(Option<u32>),
211226}
212227
213228/// A character property that `PageOp::Format` can clear so the text inherits it.
crates/onestore/src/op/model.rs+2-7
......@@ -603,6 +603,7 @@ fn interpret(page: &mut Page, op: &PageOp) -> Result<(), Error> {
603603 }
604604 page.created = Some(*created);
605605 }
606 PageOp::Color(color) => page.color = *color,
606607 PageOp::Text { text, range, with } => {
607608 let object = text_mut(page, *text)?;
608609 object.text = replace_text(&object.text, range.clone(), with)?;
......@@ -861,13 +862,7 @@ fn interpret(page: &mut Page, op: &PageOp) -> Result<(), Error> {
861862 let (path, index) =
862863 find(page, *object).ok_or_else(|| invalid("The object is not on the page"))?;
863864 let list = list_at(page, &path);
864 let range = subtree(list, index);
865 if !path.cells.is_empty() && range.len() == list.len() {
866 return Err(invalid(
867 "A table cell keeps a paragraph; insert its replacement first",
868 ));
869 }
870 list.drain(range);
865 list.drain(subtree(list, index));
871866 }
872867 drop_emptied(page);
873868 retitle(page, None);
crates/onestore/src/op/table.rs-1
......@@ -35,7 +35,6 @@ pub(crate) fn validate_table(table: &Table) -> Result<(), Error> {
3535 return Err(invalid("Every table row has one cell per column"));
3636 }
3737 for cell in &row.cells {
38 // An emptied cell keeps a replacement paragraph, as the tree writer provides.
3938 for paragraph in &cell.paragraphs {
4039 if let crate::page::ParagraphContent::Table(nested) = &paragraph.content {
4140 validate_table(nested)?;
crates/onestore/src/op/tests.rs+282-219
......@@ -1,5 +1,6 @@
1//! Ops against the page-model writer: for random edits of corpus pages, lowering the edit
2//! and applying the ops through a `Section` stores what `PreparedEdit::page` stores.
1//! Ops against the model: for random edits of corpus pages, lowering the edit and applying
2//! the ops through a `Section` stores the edited page, as the model oracle predicts and as
3//! the sealed image reads back.
34
45use super::*;
56use crate::{
......@@ -11,7 +12,7 @@ use crate::{
1112 },
1213 write::GUIDS,
1314};
14use std::collections::BTreeMap;
15use std::collections::{BTreeMap, BTreeSet};
1516
1617const SOURCES: &[(&str, &[u8])] = &[
1718 ("outline-edit", include_bytes!("../../../../corpus/outline-edit/before/notebook/synthetic.one")),
......@@ -85,20 +86,25 @@ fn read(image: &[u8], space: ExGuid) -> Page {
8586}
8687
8788/// `page` with writer-chosen identities replaced in document order: list nodes, which a
88/// copy for a second owner gets, and tag field indices.
89/// copy for a second owner gets (so equal definitions become one), tag field indices, and ink
90/// coordinates at the resolution they are stored in.
8991pub(super) fn normalize(page: &Page) -> Page {
9092 let mut page = page.clone();
9193 let mut lists = BTreeMap::new();
94 let mut canonical = BTreeMap::new();
9295 let mut paths: Vec<_> = model::lists(&page).into_iter().map(|(path, _, _)| path).collect();
9396 paths.sort_by_key(|path| format!("{path:?}"));
97 let definitions = page.definitions.clone();
9498 for path in &paths {
9599 for paragraph in model::list_at(&mut page, path).iter_mut() {
96100 for list in &mut paragraph.lists {
97101 let next = ExGuid {
98102 guid: [0xee; 16],
99 n: 1 + lists.len() as u32,
103 n: 1 + canonical.len() as u32,
100104 };
101 *list = *lists.entry(*list).or_insert(next);
105 let content = definitions.get(list).map_or(format!("{list:?}"), |d| format!("{d:?}"));
106 let id = *canonical.entry(content).or_insert(next);
107 *list = *lists.entry(*list).or_insert(id);
102108 }
103109 for tag in paragraph.tags.iter_mut() {
104110 tag.extra_set = 0;
......@@ -136,9 +142,113 @@ pub(super) fn normalize(page: &Page) -> Page {
136142 _ => Some((id, definition)),
137143 })
138144 .collect();
145 fn quantize(ink: &mut Ink) {
146 for stroke in &mut ink.strokes {
147 for point in &mut stroke.points {
148 *point = point.map(|value| (value * 10.0).round() / 10.0);
149 }
150 }
151 ink.groups.iter_mut().for_each(quantize);
152 }
153 for object in &mut page.objects {
154 if let PageObject::Ink(ink) = object {
155 quantize(ink);
156 }
157 }
158 for path in &paths {
159 for paragraph in model::list_at(&mut page, path).iter_mut() {
160 if let ParagraphContent::Ink(ink) = &mut paragraph.content {
161 quantize(ink);
162 }
163 }
164 }
139165 page
140166}
141167
168/// `after` with what the writers derive rather than store as given: the title from the
169/// title text, the values they give new objects where the model leaves them unset, and only
170/// the definitions something uses.
171fn as_written(after: &Page, before: &Page, stored: &Page) -> Page {
172 use serde_json::Value;
173 fn ids(value: &Value, out: &mut BTreeSet<String>) {
174 match value {
175 Value::Object(map) => {
176 if let Some(Value::String(id)) = map.get("id") {
177 out.insert(id.clone());
178 }
179 map.values().for_each(|value| ids(value, out));
180 }
181 Value::Array(values) => values.iter().for_each(|value| ids(value, out)),
182 _ => {}
183 }
184 }
185 fn adopt(after: &mut Value, stored: &Value, old: &BTreeSet<String>, new: bool) {
186 match (after, stored) {
187 (Value::Object(after), Value::Object(stored)) => {
188 let new = new
189 || matches!(after.get("id"), Some(Value::String(id)) if !old.contains(id));
190 for (key, value) in after.iter_mut() {
191 let Some(written) = stored.get(key) else {
192 continue;
193 };
194 // Ink coordinates are stored at a coarser resolution.
195 let near = |a: &Value, b: &Value| {
196 a.as_f64().zip(b.as_f64()).is_some_and(|(a, b)| (a - b).abs() < 0.05)
197 };
198 let close_points = key == "points"
199 && value.as_array().zip(written.as_array()).is_some_and(|(a, b)| {
200 a.len() == b.len()
201 && a.iter().zip(b).all(|(a, b)| {
202 a.as_array().zip(b.as_array()).is_some_and(|(a, b)| {
203 a.len() == b.len() && a.iter().zip(b).all(|(a, b)| near(a, b))
204 })
205 })
206 });
207 let unset = new && (value.is_null() || value.as_array().is_some_and(Vec::is_empty));
208 if close_points || unset {
209 *value = written.clone();
210 } else {
211 adopt(value, written, old, new);
212 }
213 }
214 }
215 (Value::Array(after), Value::Array(stored)) if after.len() == stored.len() => {
216 for (value, written) in after.iter_mut().zip(stored) {
217 adopt(value, written, old, new);
218 }
219 }
220 _ => {}
221 }
222 }
223 let mut old = BTreeSet::new();
224 ids(&serde_json::to_value(before).unwrap(), &mut old);
225 let mut value = serde_json::to_value(after).unwrap();
226 adopt(&mut value, &serde_json::to_value(stored).unwrap(), &old, false);
227 let mut page: Page = serde_json::from_value(value).unwrap();
228 page.title = stored.title.clone();
229 unused_dropped(&mut page);
230 page
231}
232
233/// Drops the definitions nothing on `page` uses, as reading a page leaves them out.
234fn unused_dropped(page: &mut Page) {
235 let mut used = BTreeSet::new();
236 for (_, _, list) in model::lists(page) {
237 for paragraph in list {
238 used.extend(paragraph.style);
239 used.extend(paragraph.lists.iter().copied());
240 used.extend(paragraph.tags.iter().filter_map(|tag| tag.definition));
241 if let ParagraphContent::Table(table) = &paragraph.content {
242 used.extend(table.tags.iter().filter_map(|tag| tag.definition));
243 }
244 if let Some(text) = paragraph.text() {
245 used.extend(text.tags.iter().filter_map(|tag| tag.definition));
246 }
247 }
248 }
249 page.definitions.retain(|id, _| used.contains(id));
250}
251
142252#[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord)]
143253pub(super) enum Family {
144254 Text,
......@@ -224,7 +334,17 @@ fn id() -> ExGuid {
224334 new_id().unwrap()
225335}
226336
227fn text_paragraph(text: &str, format: Format) -> PageParagraph {
337/// `ops` on page `space` of `source` as one edit, sealed: the image they leave.
338pub(crate) fn edited(source: &[u8], space: ExGuid, ops: Vec<PageOp>) -> Result<Vec<u8>, OpError> {
339 let arena = Arena::default();
340 let mut section = Section::open(&arena, source.to_vec()).map_err(OpError::Failed)?;
341 let ops = ops.into_iter().map(|op| Op::Page { space, op }).collect();
342 section.apply("Author", &Edit { at: AT, ops })?;
343 section.seal().map_err(OpError::Failed)?;
344 Ok(section.image())
345}
346
347pub(crate) fn text_paragraph(text: &str, format: Format) -> PageParagraph {
228348 PageParagraph {
229349 id: id(),
230350 parent: None,
......@@ -473,8 +593,10 @@ pub(super) fn mutate(page: &mut Page, rng: &mut Rng, family: Family) -> bool {
473593 }
474594 inserted.push(paragraph);
475595 }
476 // Children follow their parent; keep them after the first inserted paragraph.
477 inserted.sort_by_key(|p| p.parent.is_some());
596 // Children follow their parent, the first inserted paragraph, directly.
597 let (children, mut inserted): (Vec<_>, Vec<_>) =
598 inserted.into_iter().partition(|p| p.parent.is_some());
599 inserted.splice(1..1, children);
478600 let cell_level = list.first().map_or(1, |p| p.level);
479601 for paragraph in &mut inserted {
480602 if paragraph.parent.is_none() {
......@@ -993,125 +1115,70 @@ fn stroke(rng: &mut Rng) -> InkStroke {
9931115 }
9941116}
9951117
996/// How one differential case came out.
1118/// How one lowered edit came out.
9971119#[derive(Debug, PartialEq, Eq, PartialOrd, Ord, Clone)]
9981120pub(super) enum Outcome {
9991121 /// The edit made no change the writers store.
10001122 Unchanged,
1001 /// Both refused the edit.
1123 /// Lowering or the section refused the edit; the section is as it was.
10021124 Refused,
1003 /// The page writer refused; the ops were written.
1004 OnlyOps,
1005 /// Identical bytes.
1006 Bytes,
1007 /// Equal pages, both images valid, the op revision no larger.
1008 Model,
1009 /// The op revision declares more than the page writer's.
1010 Larger,
1125 /// The sealed image reads back as the edited page.
1126 Stored,
10111127 Mismatch(String),
10121128}
10131129
1014/// Objects the space's active revision declares in `image`.
1015pub(super) fn declared(image: &[u8], space: ExGuid) -> usize {
1016 let store = Store::parse(image).unwrap();
1017 let index = RevisionIndex::parse(&store).unwrap();
1018 let rid = index.active(space).unwrap();
1019 index.spaces[&space].revisions[&rid]
1020 .nodes
1021 .iter()
1022 .filter_map(|node| match node.reference {
1023 Some(crate::Reference::NodeList(chunk)) if node.id == 0xb0 => Some(chunk),
1024 _ => None,
1025 })
1026 .map(|chunk| {
1027 store
1028 .lists
1029 .values()
1030 .find(|list| list.fragments.first().is_some_and(|f| f.offset == chunk.offset))
1031 .map_or(0, |list| {
1032 list.nodes
1033 .iter()
1034 .filter(|n| {
1035 matches!(n.id, 0x2d | 0x2e | 0x41 | 0x42 | 0xa4 | 0xa5 | 0xc4 | 0xc5 | 0x72 | 0x73)
1036 })
1037 .count()
1038 })
1039 })
1040 .sum()
1041}
1042
1043/// Writes `after` over page `space` of `source` both ways and compares the images.
1044pub(super) fn differential(source: &[u8], space: ExGuid, after: &Page, seed: u64) -> Outcome {
1130/// Lowers `after` over page `space` of `source`, applies and seals the ops, and compares
1131/// the page the section holds with the edited page, with the model oracle's prediction and
1132/// with the page the sealed image reads back as.
1133pub(super) fn stored(source: &[u8], space: ExGuid, after: &Page, seed: u64) -> Outcome {
10451134 let before = read(source, space);
1046 let oracle = seeded(seed, || {
1047 crate::create::at(AT, || crate::PreparedEdit::page(source, space, after, "Author"))
1048 });
10491135 let arena = Arena::default();
10501136 let mut section = Section::open(&arena, source.to_vec()).unwrap();
1051 let ops = lower_page(&before, after);
1052 let written = ops.as_ref().ok().map(|ops| {
1053 seeded(seed, || {
1054 let edit = Edit {
1055 at: AT,
1056 ops: ops
1057 .iter()
1058 .map(|op| Op::Page {
1059 space,
1060 op: op.clone(),
1061 })
1062 .collect(),
1063 };
1064 section.apply("Author", &edit).map(|()| {
1065 let predicted = {
1066 let mut page = before.clone();
1067 for op in ops {
1068 model::apply(&mut page, op).unwrap();
1069 }
1070 page
1071 };
1072 (section.seal().unwrap(), predicted)
1073 })
1074 })
1075 });
1076 let image = section.image();
1077 match (oracle, written) {
1078 (Err(_), None | Some(Err(_))) => Outcome::Refused,
1079 (Err(_), Some(Ok(_))) => Outcome::OnlyOps,
1080 (Ok(_), None) => Outcome::Mismatch(format!("lowering refused: {:?}", ops.unwrap_err())),
1081 (Ok(_), Some(Err(error))) => Outcome::Mismatch(format!("apply refused: {error:?} for {:?}", ops.unwrap())),
1082 (Ok(edit), Some(Ok((transaction, predicted)))) => {
1083 let stored = normalize(&read(&image, space));
1084 if normalize(&predicted) != stored {
1085 return Outcome::Mismatch(format!(
1086 "the model predicts otherwise: {}",
1087 first_difference(&stored, &normalize(&predicted))
1088 ));
1089 }
1090 let expected = edit.as_bytes();
1091 if expected == source && transaction.is_none() {
1092 return Outcome::Unchanged;
1093 }
1094 if expected == image.as_slice() {
1095 return Outcome::Bytes;
1096 }
1097 let store = Store::parse(&image).unwrap();
1098 let index = RevisionIndex::parse(&store).unwrap();
1099 if let Err(error) = index.validate_current() {
1100 return Outcome::Mismatch(format!("invalid image: {error:?}"));
1101 }
1102 let (ours, theirs) = (normalize(&read(&image, space)), normalize(&read(expected, space)));
1103 if ours != theirs {
1104 return Outcome::Mismatch(first_difference(&theirs, &ours));
1105 }
1106 if expected == source {
1107 return Outcome::Model;
1108 }
1109 if declared(&image, space) > declared(expected, space) {
1110 return Outcome::Larger;
1111 }
1112 Outcome::Model
1137 let Ok(ops) = lower_page(&before, after) else {
1138 return Outcome::Refused;
1139 };
1140 let edit = Edit {
1141 at: AT,
1142 ops: ops.iter().map(|op| Op::Page { space, op: op.clone() }).collect(),
1143 };
1144 if seeded(seed, || section.apply("Author", &edit)).is_err() {
1145 if section.page(space).unwrap() != before {
1146 return Outcome::Mismatch("a refused edit changed the page".into());
11131147 }
1148 return Outcome::Refused;
1149 }
1150 let mut predicted = before.clone();
1151 for op in &ops {
1152 model::apply(&mut predicted, op).unwrap();
1153 }
1154 let live = section.page(space).unwrap();
1155 if normalize(&predicted) != normalize(&live) {
1156 return Outcome::Mismatch(format!(
1157 "the model predicts otherwise: {}",
1158 first_difference(&normalize(&live), &normalize(&predicted))
1159 ));
1160 }
1161 let Some(_) = seeded(seed, || section.seal().unwrap()) else {
1162 return if live == before { Outcome::Unchanged } else {
1163 Outcome::Mismatch("the page changed and the seal stored nothing".into())
1164 };
1165 };
1166 let image = section.image();
1167 let store = Store::parse(&image).unwrap();
1168 let index = RevisionIndex::parse(&store).unwrap();
1169 if let Err(error) = index.validate_current() {
1170 return Outcome::Mismatch(format!("invalid image: {error:?}"));
11141171 }
1172 let reread = read(&image, space);
1173 if reread != live {
1174 return Outcome::Mismatch(format!("reads back otherwise: {}", first_difference(&live, &reread)));
1175 }
1176 let written = as_written(after, &before, &reread);
1177 let (stored, wanted) = (normalize(&reread), normalize(&written));
1178 if stored != wanted {
1179 return Outcome::Mismatch(format!("stores another page: {}", first_difference(&wanted, &stored)));
1180 }
1181 Outcome::Stored
11151182}
11161183
11171184fn spans(text: &Paragraph) -> String {
......@@ -1155,7 +1222,7 @@ pub(super) fn first_difference(expected: &Page, actual: &Page) -> String {
11551222
11561223/// Every family on every corpus page it applies to, several seeds each.
11571224#[test]
1158fn ops_store_what_the_page_writer_stores() {
1225fn lowered_edits_store_the_edited_page() {
11591226 let mut tally: BTreeMap<(Family, String), usize> = BTreeMap::new();
11601227 let mut mismatches = Vec::new();
11611228 let seeds = std::env::var("OP_SEEDS").map_or(3, |s| s.parse().unwrap());
......@@ -1170,7 +1237,7 @@ fn ops_store_what_the_page_writer_stores() {
11701237 continue;
11711238 }
11721239 GUIDS.set(None);
1173 let outcome = differential(source, space, &after, 1 << 40 | seed);
1240 let outcome = stored(source, space, &after, 1 << 40 | seed);
11741241 let key = match &outcome {
11751242 Outcome::Mismatch(_) => "Mismatch".to_owned(),
11761243 other => format!("{other:?}"),
......@@ -1192,7 +1259,6 @@ fn ops_store_what_the_page_writer_stores() {
11921259 println!("{mismatch}\n");
11931260 }
11941261 assert!(mismatches.is_empty(), "{} mismatches", mismatches.len());
1195 assert!(!tally.keys().any(|(_, outcome)| outcome == "Larger"));
11961262}
11971263
11981264#[test]
......@@ -1674,42 +1740,34 @@ fn random_ops_read_back_as_the_model_predicts() {
16741740 assert!(applied > refused);
16751741}
16761742
1677/// Page creation, moves, indentation and removal through a section store what the image
1678/// writers store, and the section lists the pages the document does.
1743/// Page creation, moves, indentation and removal through a section leave the page list
1744/// they describe, and the sealed image lists the pages the section does.
16791745#[test]
1680fn section_ops_store_what_the_image_writers_store() {
1746fn section_ops_leave_the_page_list_they_describe() {
16811747 let sources: Vec<Vec<u8>> = vec![
16821748 crate::create_section("pages.one", "First", "Author").unwrap(),
16831749 include_bytes!("../../../../corpus/page-lifecycle/04-nested/notebook/Lifecycle.one").to_vec(),
16841750 SOURCES[3].1.to_vec(),
16851751 ];
1686 let (mut bytes, mut total) = (0, 0);
1752 let mut applied = 0;
16871753 for (n, source) in sources.iter().enumerate() {
16881754 let arena = Arena::default();
16891755 let mut section = Section::open(&arena, source.to_vec()).unwrap();
1690 let mut image = source.clone();
16911756 let mut rng = Rng(n as u64 + 17);
16921757 for step in 0..12u64 {
1693 let spaces = pages(&image);
1758 let listed = section.pages().unwrap();
1759 let spaces: Vec<ExGuid> = listed.iter().map(|(space, ..)| *space).collect();
1760 let anchor = spaces.get(rng.pick(spaces.len()).unwrap_or(0)).copied().filter(|_| rng.coin());
16941761 let op = match step % 4 {
1695 0 => SectionOp::Create(
1696 seeded(1 << 50 | step << 20, || {
1697 crate::PageCreation::new(
1698 spaces.get(rng.pick(spaces.len()).unwrap_or(0)).copied().filter(|_| rng.coin()),
1699 Some("Created"),
1700 "Author",
1701 )
1702 })
1703 .unwrap(),
1704 ),
1762 0 => SectionOp::Create(crate::PageCreation::new(anchor, Some("Created"), "Author").unwrap()),
17051763 1 => {
17061764 let Some(space) = spaces.get(rng.pick(spaces.len()).unwrap_or(0)).copied() else { continue };
17071765 let level = if spaces[0] == space { 1 } else { 1 + (rng.next() % 2) as u32 };
1708 SectionOp::Pages(vec![seeded(1 << 51 | step << 20, || crate::PageEdit::set_level(space, level)).unwrap()])
1766 SectionOp::Pages(vec![crate::PageEdit::set_level(space, level).unwrap()])
17091767 }
17101768 2 => {
17111769 let Some(space) = spaces.get(rng.pick(spaces.len()).unwrap_or(0)).copied() else { continue };
1712 SectionOp::Pages(vec![seeded(1 << 52 | step << 20, || crate::PageEdit::move_to(space, None, 1)).unwrap()])
1770 SectionOp::Pages(vec![crate::PageEdit::move_to(space, None, 1).unwrap()])
17131771 }
17141772 _ => {
17151773 if spaces.len() < 2 {
......@@ -1718,63 +1776,71 @@ fn section_ops_store_what_the_image_writers_store() {
17181776 SectionOp::Delete(vec![spaces[rng.pick(spaces.len()).unwrap()]])
17191777 }
17201778 };
1721 let expected = seeded(1 << 40 | step << 16, || {
1722 crate::create::at(AT, || match &op {
1723 SectionOp::Create(creation) => crate::PreparedEdit::create_page(&image, creation),
1724 SectionOp::Pages(edits) => crate::PreparedEdit::pages(&image, edits),
1725 SectionOp::Delete(spaces) => crate::PreparedEdit::delete_pages_permanently(&image, spaces),
1726 SectionOp::Import { .. } => unreachable!(),
1727 })
1728 .map(|edit| edit.as_bytes().to_vec())
1729 });
1730 let applied = seeded(1 << 40 | step << 16, || {
1731 section
1732 .apply("Author", &Edit { at: AT, ops: vec![Op::Section(op.clone())] })
1733 .map(|()| section.seal().unwrap())
1734 });
1735 match (expected, applied) {
1736 (Err(_), Err(_)) => continue,
1737 (Ok(expected), Ok(_)) => {
1738 total += 1;
1739 let written = section.image();
1740 if written == expected {
1741 bytes += 1;
1779 let edit = Edit { at: AT, ops: vec![Op::Section(op.clone())] };
1780 if section.apply("Author", &edit).is_err() {
1781 assert_eq!(section.pages().unwrap(), listed, "{n} step {step} {op:?}");
1782 continue;
1783 }
1784 section.seal().unwrap();
1785 applied += 1;
1786 let after = section.pages().unwrap();
1787 let order = |list: &[(ExGuid, String, u32)], without: ExGuid| -> Vec<ExGuid> {
1788 list.iter().map(|(space, ..)| *space).filter(|space| *space != without).collect()
1789 };
1790 match &op {
1791 SectionOp::Create(creation) => {
1792 let at = after.iter().position(|(space, ..)| *space == creation.space()).unwrap();
1793 assert_eq!(after[at].1, "Created");
1794 assert_eq!(order(&after, creation.space()), spaces, "{n} step {step}");
1795 match anchor {
1796 Some(before) => assert_eq!(after[at + 1].0, before),
1797 None => assert_eq!(at + 1, after.len()),
1798 }
1799 }
1800 SectionOp::Pages(edits) => {
1801 let space = edits[0].space();
1802 let at = after.iter().position(|(listed, ..)| *listed == space).unwrap();
1803 assert_eq!(after[at].2, edits[0].level(), "{n} step {step} {op:?}");
1804 if edits[0].position() == crate::PagePosition::Keep {
1805 assert_eq!(order(&after, space), order(&listed, space));
1806 assert_eq!(after.iter().map(|(s, ..)| *s).collect::<Vec<_>>(), spaces);
17421807 } else {
1743 let store = Store::parse(&written).unwrap();
1744 RevisionIndex::parse(&store).unwrap().validate_current().unwrap();
1745 let spaces = pages(&expected);
1746 assert_eq!(pages(&written), spaces, "{n} step {step} {op:?}");
1747 for space in spaces {
1748 assert_eq!(read(&written, space), read(&expected, space), "{n} step {step}");
1749 }
1808 assert_eq!(after.last().unwrap().0, space, "{n} step {step}");
17501809 }
1751 image = expected;
1752 let store = Store::parse(&image).unwrap();
1753 let index = RevisionIndex::parse(&store).unwrap();
1754 let document = Document::parse(&index).unwrap();
1755 let listed: Vec<(ExGuid, String, u32)> = document
1756 .pages()
1757 .unwrap()
1758 .into_iter()
1759 .map(|(space, page)| {
1760 let (title, level) = Page::heading(document.active(space).unwrap(), page);
1761 (space, title, level)
1762 })
1763 .collect();
1764 assert_eq!(section.pages().unwrap(), listed, "{n} step {step}");
17651810 }
1766 (expected, applied) => panic!("{n} step {step} {op:?}: {:?} vs {:?}", expected.err(), applied.err()),
1811 SectionOp::Delete(deleted) => {
1812 assert_eq!(after.iter().map(|(s, ..)| *s).collect::<Vec<_>>(), order(&listed, deleted[0]));
1813 }
1814 SectionOp::Import { .. } | SectionOp::Conflict { .. } | SectionOp::Color(_) => {
1815 unreachable!()
1816 }
1817 }
1818 let image = section.image();
1819 let store = Store::parse(&image).unwrap();
1820 let index = RevisionIndex::parse(&store).unwrap();
1821 index.validate_current().unwrap();
1822 let document = Document::parse(&index).unwrap();
1823 let reread: Vec<(ExGuid, String, u32)> = document
1824 .pages()
1825 .unwrap()
1826 .into_iter()
1827 .map(|(space, page)| {
1828 let (title, level) = Page::heading(document.active(space).unwrap(), page);
1829 (space, title, level)
1830 })
1831 .collect();
1832 assert_eq!(reread, after, "{n} step {step}");
1833 for (space, ..) in &after {
1834 assert_eq!(read(&image, *space), section.page(*space).unwrap(), "{n} step {step}");
17671835 }
17681836 }
17691837 }
1770 println!("{bytes} of {total} section edits byte-identical");
1771 assert!(total > 12);
1838 assert!(applied > 12, "{applied} section edits applied");
17721839}
17731840
1774/// Importing a page stores what creating it and writing its copy through the page writer
1775/// does.
1841/// An imported page reads back as its copy under the created page's title.
17761842#[test]
1777fn an_imported_page_reads_as_the_page_writer_leaves_it() {
1843fn an_imported_page_reads_back_as_its_copy() {
17781844 let target = crate::create_section("import.one", "First", "Author").unwrap();
17791845 let mut imported = 0;
17801846 for (name, source) in SOURCES {
......@@ -1783,48 +1849,45 @@ fn an_imported_page_reads_as_the_page_writer_leaves_it() {
17831849 continue;
17841850 };
17851851 let creation = crate::PageCreation::new(None, Some(&copy.title), "Author").unwrap();
1786 let created = crate::PreparedEdit::create_page(&target, &creation).unwrap();
1787 let created = created.as_bytes();
1788 let mut after = read(created, creation.space());
1789 after.objects.retain(|object| matches!(object, PageObject::Title(_)));
1790 after.objects.extend(
1852 let arena = Arena::default();
1853 let mut section = Section::open(&arena, target.clone()).unwrap();
1854 let import = |section: &mut Section<'_>, op: SectionOp| {
1855 section.apply("Author", &Edit { at: AT, ops: vec![Op::Section(op)] })
1856 };
1857 import(&mut section, SectionOp::Create(creation.clone())).unwrap();
1858 let created = section.page(creation.space()).unwrap();
1859 let mut expected = created.clone();
1860 let arena = Arena::default();
1861 let mut section = Section::open(&arena, target.clone()).unwrap();
1862 if import(&mut section, SectionOp::Import { creation: creation.clone(), page: copy.clone() }).is_err() {
1863 continue;
1864 }
1865 section.seal().unwrap();
1866 let image = section.image();
1867 let store = Store::parse(&image).unwrap();
1868 RevisionIndex::parse(&store).unwrap().validate_current().unwrap();
1869 expected.objects.retain(|object| matches!(object, PageObject::Title(_)));
1870 expected.objects.extend(
17911871 copy.objects
17921872 .iter()
17931873 .filter(|object| !matches!(object, PageObject::Title(_)))
17941874 .cloned(),
17951875 );
1796 after.definitions = copy.definitions.clone();
1797 let expected = crate::create::at(AT, || {
1798 crate::PreparedEdit::page(created, creation.space(), &after, "Author")
1799 });
1800 let arena = Arena::default();
1801 let mut section = Section::open(&arena, target.clone()).unwrap();
1802 let applied = section.apply(
1803 "Author",
1804 &Edit {
1805 at: AT,
1806 ops: vec![Op::Section(SectionOp::Import {
1807 creation: creation.clone(),
1808 page: copy.clone(),
1809 })],
1810 },
1811 );
1812 match (expected, applied) {
1813 (Err(_), Err(_)) => continue,
1814 (Ok(expected), Ok(())) => {
1815 section.seal().unwrap();
1816 let image = section.image();
1817 let store = Store::parse(&image).unwrap();
1818 RevisionIndex::parse(&store).unwrap().validate_current().unwrap();
1819 let (ours, theirs) = (
1820 normalize(&read(&image, creation.space())),
1821 normalize(&read(expected.as_bytes(), creation.space())),
1822 );
1823 assert!(ours == theirs, "{name}: {}", first_difference(&theirs, &ours));
1824 imported += 1;
1876 expected.definitions.extend(copy.definitions.clone());
1877 let stored = read(&image, creation.space());
1878 // New page children stack as the writers add them.
1879 let order: Vec<ExGuid> = stored.objects.iter().map(PageObject::id).collect();
1880 expected.objects.sort_by_key(|object| order.iter().position(|id| *id == object.id()));
1881 let mut expected = as_written(&expected, &created, &stored);
1882 // An outline's height follows its content; no op carries it.
1883 for (object, stored) in expected.objects.iter_mut().zip(&stored.objects) {
1884 if let (PageObject::Outline(outline), PageObject::Outline(stored)) = (object, stored) {
1885 outline.layout.max_height = stored.layout.max_height;
18251886 }
1826 (expected, applied) => panic!("{name}: {:?} vs {:?}", expected.err(), applied.err()),
18271887 }
1888 let (ours, theirs) = (normalize(&stored), normalize(&expected));
1889 assert!(ours == theirs, "{name}: {}", first_difference(&theirs, &ours));
1890 imported += 1;
18281891 }
18291892 }
18301893 assert!(imported > 10, "{imported} pages imported");
crates/onestore/src/outline.rs+12-17
......@@ -25,15 +25,6 @@ pub enum OutlineEdit {
2525}
2626
2727impl OutlineEdit {
28 pub(crate) fn apply(
29 self,
30 source: &[u8],
31 space: ExGuid,
32 object: ExGuid,
33 ) -> Result<Vec<u8>, Error> {
34 crate::active::write(source, space, |active| self.changes(active, object))
35 }
36
3728 pub(crate) fn changes(self, active: &ActivePage<'_>, object: ExGuid) -> Result<Changes, Error> {
3829 let invalid = |message| Error { offset: 0, message };
3930 let values = match self {
......@@ -146,8 +137,12 @@ impl OutlineEdit {
146137#[cfg(test)]
147138mod tests {
148139 use super::*;
149 use crate::write::write_revision;
150 use crate::{RevisionIndex, Store, document::Document};
140 use crate::{
141 RevisionIndex, Store,
142 document::Document,
143 op::{PageOp, tests::edited},
144 write::write_revision,
145 };
151146
152147 #[test]
153148 fn protection_on_the_target_or_ancestor_prevents_layout_edits() {
......@@ -184,7 +179,7 @@ mod tests {
184179 (paragraph, OutlineEdit::Collapsed(true)),
185180 ] {
186181 assert_eq!(
187 edit.apply(&bytes, sid, object).is_err(),
182 edited(&bytes, sid, vec![PageOp::Outline { object, edit }]).is_err(),
188183 enabled,
189184 "{target} {property:x} {edit:?}"
190185 );
......@@ -199,10 +194,10 @@ mod tests {
199194 Ok(BTreeMap::from([(outline, object)]))
200195 })
201196 .unwrap();
202 assert!(
203 OutlineEdit::Collapsed(true)
204 .apply(&duplicated, sid, paragraph)
205 .is_err()
206 );
197 let collapse = PageOp::Outline {
198 object: paragraph,
199 edit: OutlineEdit::Collapsed(true),
200 };
201 assert!(edited(&duplicated, sid, vec![collapse]).is_err());
207202 }
208203}
crates/onestore/src/page/mod.rs+49-4
......@@ -17,7 +17,6 @@ pub mod text;
1717
1818pub use ink::{Ink, InkStroke};
1919pub use math::Math;
20pub(crate) mod write;
2120
2221/// Picture and attachment payloads travel with the model (queued intents replay them),
2322/// as base64 text.
......@@ -52,6 +51,10 @@ mod payload {
5251}
5352pub use text::Paragraph;
5453
54/// The page node's colour (View, Page Color) as COLORREF, absent for "No color"; not in
55/// MS-ONE, observed in OneNote 2010's pages (`evidence/notebook-management`).
56pub(crate) const PAGE_COLOR: u32 = 0x14001d2a;
57
5558/// The role of a title-outline paragraph that displays the page's creation date or time.
5659#[derive(Clone, Copy, Debug, PartialEq, serde::Serialize, serde::Deserialize)]
5760pub enum DateField {
......@@ -67,6 +70,9 @@ pub struct Page {
6770 /// FILETIME ticks from the page's TopologyCreationTimeStamp.
6871 pub created: Option<u64>,
6972 pub margin_origin: [f32; 2],
73 /// The page's colour (View, Page Color), COLORREF; `None` is OneNote's "No color".
74 #[serde(default)]
75 pub color: Option<u32>,
7076 pub objects: Vec<PageObject>,
7177 pub definitions: BTreeMap<ExGuid, Definition>,
7278}
......@@ -394,9 +400,14 @@ impl Page {
394400 /// links keep naming the original. Content outside the model has no copy, so an
395401 /// `Unsupported` object is an error.
396402 pub fn copy(&self) -> Result<Self, Error> {
403 self.copy_with(&mut [])
404 }
405
406 /// `copy`, renaming `ids`, identities on this page, to theirs on the copy.
407 pub fn copy_with(&self, ids: &mut [ExGuid]) -> Result<Self, Error> {
397408 let invalid = |message| Error { offset: 0, message };
398 let mut value =
399 serde_json::to_value(self).map_err(|_| invalid("Page is not serializable"))?;
409 let mut value = serde_json::to_value((self, &*ids))
410 .map_err(|_| invalid("Page is not serializable"))?;
400411 let mut fresh: BTreeMap<String, String> = BTreeMap::new();
401412 fn walk(
402413 value: &mut serde_json::Value,
......@@ -444,8 +455,9 @@ impl Page {
444455 Ok(())
445456 }
446457 walk(&mut value, &mut fresh)?;
447 let mut copy: Self =
458 let (mut copy, renamed): (Self, Vec<ExGuid>) =
448459 serde_json::from_value(value).map_err(|_| invalid("Page copy does not deserialize"))?;
460 ids.copy_from_slice(&renamed);
449461 copy.identity = Some(crate::write::fresh_guid()?);
450462 // A paragraph's indent level follows its parent chain; a level beyond that comes
451463 // from an outline group, which the model does not hold.
......@@ -477,6 +489,28 @@ impl Page {
477489 Ok(copy)
478490 }
479491
492 /// The date and time the title's date fields show, when it has both.
493 pub fn date_text(&self) -> Option<[String; 2]> {
494 let title = self.objects.iter().find_map(|object| match object {
495 PageObject::Title(title) => Some(title),
496 _ => None,
497 })?;
498 let outline = title.outlines.iter().find(|outline| Some(outline.id) == title.date)?;
499 let [first, second] = &outline.paragraphs[..] else {
500 return None;
501 };
502 let [first, second] = [first.text()?, second.text()?];
503 let text = |object: &TextObject| object.text.text().to_owned();
504 // OneNote writes the date first; a field marked as the other role says otherwise.
505 Some(
506 if first.date_field == Some(DateField::Time) || second.date_field == Some(DateField::Date) {
507 [text(second), text(first)]
508 } else {
509 [text(first), text(second)]
510 },
511 )
512 }
513
480514 /// The notebook-management identity of the page in an active page revision, which
481515 /// internal links name.
482516 pub fn identity_of(revision: &Revision<'_>) -> Option<[u8; 16]> {
......@@ -562,6 +596,17 @@ impl Page {
562596 margin_origin_x.unwrap_or(1.0_f32 * 36.0),
563597 margin_origin_y.unwrap_or(0.4_f32 * 36.0),
564598 ],
599 color: root
600 .extra
601 .first()
602 .and_then(|fields| fields.iter().find(|field| field.id == PAGE_COLOR))
603 .map(|field| match field.value {
604 FieldValue::Bytes(&[red, green, blue, alpha]) => {
605 Ok(u32::from_le_bytes([red, green, blue, alpha]))
606 }
607 _ => Err(invalid("The page colour is damaged")),
608 })
609 .transpose()?,
565610 objects: Vec::new(),
566611 definitions: BTreeMap::new(),
567612 };
crates/onestore/src/page/write.rs deleted-1646
......@@ -1,1646 +0,0 @@
1//! Publishes an edited page model by lowering the difference from the stored page onto
2//! the typed writers, whose revisions accumulate in memory, then squashing them into one
3//! revision per space. `PreparedEdit::page` keeps it until the op path replaces it.
4
5use super::{Image, Ink, Outline, Page, PageObject, PageParagraph, ParagraphContent};
6use crate::{
7 Error, ExGuid, Insertion, ObjectData, OutlineEdit, ParagraphJoin, ParagraphSplit, PropertySets,
8 RevisionIndex, Store, TreeEdit, Value,
9 active::{ActivePage, Changes},
10 document::Kind,
11 op::{
12 content::{self, AttachmentIds},
13 levels,
14 lower::{View, collect_containers, kept_set, text_edit, validate},
15 properties,
16 table::{self, Structure},
17 },
18 write::{PropertyObject, RevisionEdit},
19};
20use bumpalo::Bump;
21use std::collections::{BTreeMap, BTreeSet};
22
23fn invalid(message: &'static str) -> Error {
24 Error { offset: 0, message }
25}
26
27fn page_images(page: &Page) -> impl Iterator<Item = (ExGuid, &Image)> {
28 page.objects.iter().filter_map(|object| match object {
29 PageObject::Image(image) => Some((image.id, image)),
30 _ => None,
31 })
32}
33
34fn page_ink(page: &Page) -> impl Iterator<Item = (ExGuid, &Ink)> {
35 page.objects.iter().filter_map(|object| match object {
36 PageObject::Ink(ink) => Some((ink.id, ink)),
37 _ => None,
38 })
39}
40
41pub(crate) fn write_page(
42 source: &[u8],
43 space: ExGuid,
44 after: &Page,
45 author: &str,
46) -> Result<Vec<u8>, Error> {
47 if author.contains('\0') {
48 return Err(invalid("Choose an author name without NUL"));
49 }
50 let store = Store::parse(source)?;
51 let index = RevisionIndex::parse(&store)?;
52 let arena = Bump::new();
53 let active = ActivePage::parse(&index, space)?;
54 let [page] = active.pages[..] else {
55 return Err(invalid("Choose an object space containing one active page"));
56 };
57 let before = Page::from_revision(&active.view, page)?;
58 let mut lowering = Lowering {
59 active,
60 arena: &arena,
61 current: Some(before.clone()),
62 edited: false,
63 page,
64 author,
65 alias: BTreeMap::new(),
66 built: BTreeSet::new(),
67 };
68 lowering.run(&before, after)?;
69 if !lowering.edited {
70 return Ok(source.to_vec());
71 }
72 let active = &lowering.active;
73 squash(
74 &index,
75 &[(space, &active.live.revision)],
76 &active.payloads,
77 &lowering.alias,
78 None,
79 )
80}
81
82struct Lowering<'a> {
83 active: ActivePage<'a>,
84 /// Holds the objects the typed writers store for as long as `active` reads them.
85 arena: &'a Bump,
86 /// The page as `active` stores it, until a writer changes it.
87 current: Option<Page>,
88 /// Whether a writer stored anything.
89 edited: bool,
90 page: ExGuid,
91 author: &'a str,
92 /// Model identities of new objects mapped to the identities the typed writers allocated.
93 alias: BTreeMap<ExGuid, ExGuid>,
94 /// Tables whose structure this run has written.
95 built: BTreeSet<ExGuid>,
96}
97
98impl<'a> Lowering<'a> {
99 fn id(&self, model: ExGuid) -> ExGuid {
100 self.alias.get(&model).copied().unwrap_or(model)
101 }
102
103 /// A writer identity for a new object; squash renames it to the model's.
104 fn allocate(&mut self, model: ExGuid) -> Result<ExGuid, Error> {
105 let written = ExGuid {
106 guid: crate::write::fresh_guid()?,
107 n: 1,
108 };
109 self.alias.insert(model, written);
110 Ok(written)
111 }
112
113 /// Stores the next revision a typed writer computes from the page as stored so far.
114 fn write(
115 &mut self,
116 changes: impl FnOnce(&ActivePage<'a>) -> Result<Changes, Error>,
117 ) -> Result<(), Error> {
118 self.write_with(&[], changes)
119 }
120
121 fn write_with(
122 &mut self,
123 payloads: &[([u8; 16], &[u8])],
124 changes: impl FnOnce(&ActivePage<'a>) -> Result<Changes, Error>,
125 ) -> Result<(), Error> {
126 let changes = changes(&self.active)?;
127 if self.active.write(self.arena, payloads, changes)? {
128 self.current = None;
129 self.edited = true;
130 }
131 Ok(())
132 }
133
134 fn current(&mut self) -> Result<Page, Error> {
135 if self.current.is_none() {
136 self.current = Some(Page::from_revision(&self.active.view, self.page)?);
137 }
138 Ok(self.current.clone().unwrap())
139 }
140
141 fn run(&mut self, before: &Page, after: &Page) -> Result<(), Error> {
142 let old = View::new(before, false)?;
143 let new = View::new(after, false)?;
144 validate(before, after, &old, &new)?;
145 if after.created != before.created {
146 return Err(invalid("Page creation time cannot be edited"));
147 }
148 for id in new.outlines.keys().chain(new.paragraphs.keys()) {
149 if !old.outlines.contains_key(id)
150 && !old.paragraphs.contains_key(id)
151 && self.active.live.revision.objects.contains_key(id)
152 {
153 return Err(invalid(
154 "A new model identity already exists in the section",
155 ));
156 }
157 }
158 let mut placed: BTreeMap<ExGuid, Vec<ExGuid>> = BTreeMap::new();
159 for (container, children) in &old.children {
160 placed.insert(*container, children.clone());
161 }
162 let mut page_order: Vec<ExGuid> = old
163 .page_children
164 .iter()
165 .copied()
166 .filter(|id| !old.outlines.contains_key(id) || new.outlines.contains_key(id))
167 .collect();
168 self.insert_outlines(&old, &new, &mut placed, &mut page_order)?;
169 let mut consumed = BTreeSet::new();
170 self.ungroup(&old, &new)?;
171 self.split_and_join(&old, &new, &mut placed, &mut consumed)?;
172 self.place(&old, &new, &placed, &page_order)?;
173 self.delete(&old, &new, &consumed)?;
174 self.edit_levels(&new)?;
175 self.edit_text_identities(&new)?;
176 self.edit_equations(&new)?;
177 self.edit_text(&new)?;
178 self.edit_paragraph_styles(&old, &new, after)?;
179 self.edit_lists(after, &new)?;
180 self.edit_tags(after, &new)?;
181 self.edit_paragraph_formatting(&new)?;
182 self.edit_formatting(&new)?;
183 self.edit_layout(&old, &new)?;
184 Ok(())
185 }
186
187 fn insert_outlines(
188 &mut self,
189 old: &View<'_>,
190 new: &View<'_>,
191 placed: &mut BTreeMap<ExGuid, Vec<ExGuid>>,
192 page_order: &mut Vec<ExGuid>,
193 ) -> Result<(), Error> {
194 for id in &new.page_children {
195 let Some(outline) = new.outlines.get(id) else {
196 continue;
197 };
198 if old.outlines.contains_key(id) {
199 continue;
200 }
201 if new.title_outlines.contains(id) {
202 return Err(invalid("Title outlines cannot be added"));
203 }
204 let (Some(x), Some(y)) = (outline.layout.x, outline.layout.y) else {
205 return Err(invalid("A new outline needs a position"));
206 };
207 let Some(first) = new.children[id].first().copied() else {
208 return Err(invalid("A new outline needs a paragraph"));
209 };
210 // A new outline starts as one text paragraph; other content replaces it in
211 // its own pass, as for a paragraph inserted into an existing outline.
212 let (text, text_id) = match &new.paragraphs[&first].content {
213 ParagraphContent::Text(text) if text.date_field.is_none() => {
214 (text.text.text(), Some(text.id))
215 }
216 ParagraphContent::Table(_)
217 | ParagraphContent::Image(_)
218 | ParagraphContent::Attachment(_)
219 | ParagraphContent::Ink(_) => ("", None),
220 _ => {
221 return Err(invalid(
222 "A new outline starts with a paragraph the writer builds",
223 ));
224 }
225 };
226 let insertion = Insertion::outline(self.page, x, y, text, self.author)?;
227 self.write(|active| insertion.changes(active))?;
228 let outline_id = insertion.object();
229 self.alias.insert(*id, outline_id);
230 self.alias.insert(
231 first,
232 ExGuid {
233 guid: outline_id.guid,
234 n: 3,
235 },
236 );
237 if let Some(text_id) = text_id {
238 self.alias.insert(text_id, insertion.text_object());
239 }
240 placed.insert(*id, vec![first]);
241 placed.insert(first, Vec::new());
242 page_order.push(*id);
243 }
244 Ok(())
245 }
246
247 /// Splits and joins stored paragraphs as OneNote's Enter, Backspace and Delete do. A new
248 /// styled paragraph that is no exact split starts as Enter at the end of the nearest stored
249 /// paragraph that allows one, taking its properties as OneNote's new paragraphs do; later
250 /// passes place it and give it its text.
251 fn split_and_join(
252 &mut self,
253 old: &View<'_>,
254 new: &View<'_>,
255 placed: &mut BTreeMap<ExGuid, Vec<ExGuid>>,
256 consumed: &mut BTreeSet<ExGuid>,
257 ) -> Result<(), Error> {
258 // A new paragraph may be split from the stored paragraph nearest before it, whatever
259 // placement or pasted paragraphs the same save adds between them.
260 let mut splits = Vec::new();
261 for document in new.documents() {
262 let mut left = None;
263 for paragraph in document {
264 if old.paragraphs.contains_key(&paragraph.id) {
265 left = paragraph.text().is_some().then_some(paragraph.id);
266 } else if let Some(left) = left {
267 let container = new.container[&paragraph.id];
268 let at = new.children[&container]
269 .iter()
270 .position(|id| *id == paragraph.id);
271 splits.push(((container, at), left, paragraph.id));
272 }
273 }
274 }
275 splits.sort();
276 let mut split = BTreeSet::new();
277 for (_, left, right) in splits {
278 let (Some(previous), Some(after), Some(next)) =
279 (old.text(left), new.text(left), new.text(right))
280 else {
281 continue;
282 };
283 if split.contains(&left)
284 || after.id != previous.id
285 || new.paragraphs[&right].style != old.paragraphs[&left].style
286 {
287 continue;
288 }
289 let Ok(offset) = after.text.utf16_offset(after.text.text().len()) else {
290 continue;
291 };
292 // Splitting at the end only differs from appending a paragraph by the copied
293 // paragraph style; without one, an appended empty paragraph is an insertion.
294 if offset == previous.text.utf16_offset(previous.text.text().len())?
295 && new.paragraphs[&right].style.is_none()
296 {
297 continue;
298 }
299 let Ok(head) = previous.text.slice(0..offset) else {
300 continue;
301 };
302 let Ok(tail) = previous.text.slice(
303 offset
304 ..previous
305 .text
306 .utf16_offset(previous.text.text().len())
307 .unwrap_or(0),
308 ) else {
309 continue;
310 };
311 // An emptied side takes the writer's insertion style, so only its text must agree.
312 let same = |expected: &super::Paragraph, actual: &super::Paragraph| {
313 expected.text() == actual.text()
314 && (expected.text().is_empty() || expected == actual)
315 };
316 if !same(&head, &after.text) || !same(&tail, &next.text) {
317 continue;
318 }
319 let edit = ParagraphSplit::new(previous.id, offset, self.author)?;
320 if self.split(&edit, left, right, old, new, placed)? {
321 split.insert(left);
322 }
323 }
324 for document in new.documents() {
325 for (at, paragraph) in document.iter().enumerate() {
326 if old.paragraphs.contains_key(&paragraph.id)
327 || self.alias.contains_key(&paragraph.id)
328 || paragraph.style.is_none()
329 || paragraph.text().is_none()
330 {
331 continue;
332 }
333 let stored = |candidate: &&&PageParagraph| {
334 old.paragraphs.contains_key(&candidate.id) && candidate.text().is_some()
335 };
336 let before = document[..at].iter().rev().filter(stored);
337 for template in before.chain(document[at + 1..].iter().filter(stored)) {
338 let text = self.active.view.nodes[&template.id].content[0];
339 let Kind::RichText { text: content, .. } = &self.active.view.nodes[&text].kind
340 else {
341 continue;
342 };
343 let end = u32::try_from(content.encode_utf16().count())
344 .map_err(|_| invalid("Paragraph exceeds the UTF-16 offset range"))?;
345 let edit = ParagraphSplit::new(text, end, self.author)?;
346 if self.split(&edit, template.id, paragraph.id, old, new, placed)? {
347 break;
348 }
349 }
350 }
351 }
352 for (container, children) in &old.children {
353 if !new.children.contains_key(container) {
354 continue;
355 }
356 for pair in children.windows(2) {
357 let [left, right] = [pair[0], pair[1]];
358 let (Some(previous), Some(after), Some(removed)) =
359 (old.text(left), new.text(left), old.text(right))
360 else {
361 continue;
362 };
363 if new.paragraphs.contains_key(&right)
364 || !old.children[&left].is_empty()
365 || !old.children[&right].is_empty()
366 || removed.text.text().is_empty()
367 {
368 continue;
369 }
370 let mut joined = previous.text.clone();
371 if joined.append(removed.text.clone()).is_err() || joined != after.text {
372 continue;
373 }
374 let adopts_right = previous.text.text().is_empty();
375 if after.id
376 != if adopts_right {
377 removed.id
378 } else {
379 previous.id
380 }
381 {
382 continue;
383 }
384 let join = ParagraphJoin::new(previous.id, removed.id, self.author)?;
385 let Ok(changes) = join.changes(&self.active) else {
386 continue;
387 };
388 self.write(|_| Ok(changes))?;
389 consumed.insert(right);
390 placed.get_mut(container).unwrap().retain(|id| *id != right);
391 placed.remove(&right);
392 }
393 }
394 Ok(())
395 }
396
397 /// Stores `edit` of model paragraph `left` as model paragraph `right`, unless the split
398 /// cannot be made there.
399 fn split(
400 &mut self,
401 edit: &ParagraphSplit,
402 left: ExGuid,
403 right: ExGuid,
404 old: &View<'_>,
405 new: &View<'_>,
406 placed: &mut BTreeMap<ExGuid, Vec<ExGuid>>,
407 ) -> Result<bool, Error> {
408 let Ok(changes) = edit.changes(&self.active) else {
409 return Ok(false);
410 };
411 self.write(|_| Ok(changes))?;
412 self.alias.insert(right, edit.object());
413 // A text object a join moved into the paragraph is taken over after placement.
414 if let Some(text) = new.text(right)
415 && !self.active.live.revision.objects.contains_key(&text.id)
416 {
417 self.alias.insert(text.id, edit.text_object());
418 }
419 // The split copies the left paragraph's list nodes in order; a list the model shares
420 // with a stored paragraph stays that paragraph's.
421 if new.paragraphs[&right].lists.len() == old.paragraphs[&left].lists.len() {
422 for (n, list) in (5..).zip(&new.paragraphs[&right].lists) {
423 if !self.active.live.revision.objects.contains_key(list) {
424 let guid = edit.object().guid;
425 self.alias.insert(*list, ExGuid { guid, n });
426 }
427 }
428 }
429 // An earlier split may have carried the left paragraph to its tail.
430 let list = placed
431 .values_mut()
432 .find(|list| list.contains(&left))
433 .unwrap();
434 let at = list.iter().position(|id| *id == left).unwrap();
435 list.insert(at + 1, right);
436 placed.insert(right, placed[&left].clone());
437 placed.insert(left, Vec::new());
438 Ok(true)
439 }
440
441 fn place(
442 &mut self,
443 old: &View<'_>,
444 new: &View<'_>,
445 placed: &BTreeMap<ExGuid, Vec<ExGuid>>,
446 page_order: &[ExGuid],
447 ) -> Result<(), Error> {
448 let title = |page: &Page, id: ExGuid| {
449 page.objects
450 .iter()
451 .any(|object| matches!(object, PageObject::Title(title) if title.id == id))
452 };
453 self.edit_page_images(old, new)?;
454 self.edit_page_ink(old, new)?;
455 let survivors: Vec<ExGuid> = page_order
456 .iter()
457 .copied()
458 .filter(|id| {
459 (new.outlines.contains_key(id) || !old.outlines.contains_key(id))
460 && !title(old.page, *id)
461 })
462 .collect();
463 let after: Vec<ExGuid> = new
464 .page_children
465 .iter()
466 .copied()
467 .filter(|id| !title(new.page, *id))
468 .collect();
469 let kept = kept_set(&survivors, &after, |id| {
470 new.outlines.contains_key(&id) && !new.title_outlines.contains(&id)
471 })?;
472 let mut next = None;
473 for id in after.iter().rev() {
474 if !kept.contains(id) {
475 let edit = TreeEdit::move_to(
476 self.id(*id),
477 self.page,
478 next.map(|n| self.id(n)),
479 self.author,
480 )?;
481 self.write(|active| edit.changes(active))?;
482 }
483 next = Some(*id);
484 }
485 let mut containers: Vec<ExGuid> = Vec::new();
486 for object in &new.page.objects {
487 let outlines: Vec<&Outline> = match object {
488 PageObject::Outline(outline) => vec![outline],
489 PageObject::Title(title) => title.outlines.iter().collect(),
490 PageObject::Image(_) | PageObject::Ink(_) | PageObject::Unsupported(_) => {
491 Vec::new()
492 }
493 };
494 for outline in outlines {
495 containers.push(outline.id);
496 collect_containers(&outline.paragraphs, &mut containers);
497 }
498 }
499 // Cells that do not exist yet, and containers inside them, receive their
500 // paragraphs once the table structure does.
501 let new_cell = |id: &ExGuid| {
502 !(old.children.contains_key(id)
503 || new.paragraphs.contains_key(id)
504 || new.outlines.contains_key(id))
505 };
506 let (existing, deferred): (Vec<ExGuid>, Vec<ExGuid>) =
507 containers.into_iter().partition(|container| {
508 let mut at = *container;
509 loop {
510 if new_cell(&at) {
511 return false;
512 }
513 match new.container.get(&at) {
514 Some(parent) => at = *parent,
515 None => return true,
516 }
517 }
518 });
519 self.place_containers(old, new, placed, &existing)?;
520 // Each round builds the tables whose holders exist and fills their cells, which
521 // may hold further new tables.
522 let mut pending = deferred;
523 loop {
524 self.edit_table_structure(old, new)?;
525 if pending.is_empty() {
526 break;
527 }
528 let (ready, waiting): (Vec<ExGuid>, Vec<ExGuid>) =
529 pending.into_iter().partition(|container| {
530 self.alias.contains_key(container) || old.children.contains_key(container)
531 });
532 if ready.is_empty() {
533 return Err(invalid("Table content has no table to hold it"));
534 }
535 self.place_containers(old, new, placed, &ready)?;
536 pending = waiting;
537 }
538 self.edit_cells(old, new)?;
539 self.edit_images(old, new)?;
540 self.edit_ink_paragraphs(old, new)?;
541 self.edit_attachments(old, new)?;
542 Ok(())
543 }
544
545 /// Gives each new attachment paragraph what OneNote stores for an inserted file: the
546 /// payload embedded in the file-data store, an embedded-file container declaring it,
547 /// and an attachment object naming the file that the paragraph holds as content.
548 fn edit_attachments(&mut self, old: &View<'_>, new: &View<'_>) -> Result<(), Error> {
549 for (paragraph_id, paragraph) in &new.paragraphs {
550 let ParagraphContent::Attachment(attachment) = &paragraph.content else {
551 continue;
552 };
553 if let Some(previous) = old.paragraphs.get(paragraph_id) {
554 let ParagraphContent::Attachment(stored) = &previous.content else {
555 return Err(invalid("Paragraph content type cannot change"));
556 };
557 if stored != attachment {
558 self.write(|active| content::attachment_edit_changes(active, stored, attachment))?;
559 }
560 continue;
561 }
562 if attachment.recording.is_some() {
563 return Err(invalid("Recordings are captured by OneNote, not inserted"));
564 }
565 let Some(bytes) = &attachment.bytes else {
566 return Err(invalid("A new attachment needs its payload"));
567 };
568 let ids = AttachmentIds {
569 object: self.allocate(attachment.id)?,
570 file: ExGuid {
571 guid: crate::write::fresh_guid()?,
572 n: 1,
573 },
574 payload: crate::write::fresh_guid()?,
575 preview: match &attachment.preview {
576 Some(_) => {
577 let payload = crate::write::fresh_guid()?;
578 let icon = ExGuid {
579 guid: crate::write::fresh_guid()?,
580 n: 1,
581 };
582 Some((payload, icon))
583 }
584 None => None,
585 },
586 };
587 let mut payloads: Vec<([u8; 16], &[u8])> = vec![(ids.payload, bytes)];
588 if let (Some((payload, _)), Some(icon)) = (ids.preview, &attachment.preview) {
589 payloads.push((payload, icon));
590 }
591 let holder = self.id(*paragraph_id);
592 self.write_with(&payloads, |active| {
593 content::attachment_changes(active, attachment, &ids, holder)
594 })?;
595 }
596 Ok(())
597 }
598
599 /// Gives each new picture paragraph what OneNote stores for an inserted picture: the
600 /// payload embedded in the section's file-data store, a file-data object declaring it
601 /// by identity and extension, and a picture object the paragraph holds as content.
602 fn edit_images(&mut self, old: &View<'_>, new: &View<'_>) -> Result<(), Error> {
603 for (paragraph_id, paragraph) in &new.paragraphs {
604 let ParagraphContent::Image(image) = &paragraph.content else {
605 continue;
606 };
607 if let Some(previous) = old.paragraphs.get(paragraph_id) {
608 let ParagraphContent::Image(stored) = &previous.content else {
609 return Err(invalid("Paragraph content type cannot change"));
610 };
611 if stored != image {
612 self.edit_image(stored, image)?;
613 }
614 continue;
615 }
616 if image.layout.x.is_some() || image.layout.y.is_some() {
617 return Err(invalid("A paragraph picture has no position of its own"));
618 }
619 let holder = self.id(*paragraph_id);
620 self.insert_image(image, Some(holder))?;
621 }
622 Ok(())
623 }
624
625 /// Page-level pictures are direct page children, as OneNote stores a picture placed
626 /// outside any outline: new ones are appended for the placement pass to order, changed
627 /// ones are moved, resized or described, and removed ones are deleted in the delete pass.
628 fn edit_page_images(&mut self, old: &View<'_>, new: &View<'_>) -> Result<(), Error> {
629 let stored: BTreeMap<ExGuid, &Image> = page_images(old.page).collect();
630 for (id, image) in page_images(new.page) {
631 match stored.get(&id) {
632 Some(previous) => {
633 if *previous != image {
634 self.edit_image(previous, image)?;
635 }
636 }
637 None => {
638 if image.layout.x.is_none() || image.layout.y.is_none() {
639 return Err(invalid("A new page-level picture needs a position"));
640 }
641 self.insert_image(image, None)?;
642 }
643 }
644 }
645 Ok(())
646 }
647
648 /// Ink placed on the page: new drawings are appended for the placement pass to order,
649 /// changed ones have their stroke list rewritten, removed ones go in the delete pass.
650 fn edit_page_ink(&mut self, old: &View<'_>, new: &View<'_>) -> Result<(), Error> {
651 let stored: BTreeMap<ExGuid, &Ink> = page_ink(old.page).collect();
652 for (id, ink) in page_ink(new.page) {
653 match stored.get(&id) {
654 Some(previous) => {
655 if *previous != ink {
656 self.edit_ink(previous, ink)?;
657 }
658 }
659 None => self.insert_ink(ink, None)?,
660 }
661 }
662 Ok(())
663 }
664
665 fn edit_ink_paragraphs(&mut self, old: &View<'_>, new: &View<'_>) -> Result<(), Error> {
666 for (paragraph_id, paragraph) in &new.paragraphs {
667 let ParagraphContent::Ink(ink) = &paragraph.content else {
668 continue;
669 };
670 match old.paragraphs.get(paragraph_id) {
671 Some(previous) => {
672 let ParagraphContent::Ink(stored) = &previous.content else {
673 return Err(invalid("Paragraph content type cannot change"));
674 };
675 if stored != ink {
676 self.edit_ink(stored, ink)?;
677 }
678 }
679 None => {
680 let holder = self.id(*paragraph_id);
681 self.insert_ink(ink, Some(holder))?;
682 }
683 }
684 }
685 Ok(())
686 }
687
688 /// Writes new ink the way OneNote 2010 stores a drawing: a container the page lists as a
689 /// child (or a paragraph holds as content), its data node listing stroke objects, each
690 /// stroke's packet and half-inch origin, and one drawing-attribute object per distinct pen.
691 fn insert_ink(&mut self, ink: &Ink, holder: Option<ExGuid>) -> Result<(), Error> {
692 let container = self.allocate(ink.id)?;
693 let data = ExGuid {
694 guid: crate::write::fresh_guid()?,
695 n: 1,
696 };
697 let mut strokes = Vec::new();
698 for stroke in &ink.strokes {
699 strokes.push((self.allocate(stroke.id)?, stroke));
700 }
701 self.write(|active| content::ink_changes(active, ink, container, data, &strokes, holder))
702 }
703
704 /// Rewrites a drawing's stroke list: stored strokes stay as they are (OneNote erases whole
705 /// strokes rather than editing them), removed ones leave the list, new ones are created.
706 fn edit_ink(&mut self, stored: &Ink, ink: &Ink) -> Result<(), Error> {
707 if stored.id != ink.id {
708 return Err(invalid("Ink identity cannot change"));
709 }
710 if stored.layout != ink.layout || stored.groups != ink.groups {
711 return Err(invalid("Ink position and groups stay as stored"));
712 }
713 let mut kept = Vec::new();
714 let mut added = Vec::new();
715 for stroke in &ink.strokes {
716 match stored.strokes.iter().find(|s| s.id == stroke.id) {
717 Some(previous) if previous == stroke => kept.push(self.id(stroke.id)),
718 Some(_) => return Err(invalid("A stored stroke keeps its path and pen")),
719 None => added.push((self.allocate(stroke.id)?, stroke)),
720 }
721 }
722 let container = self.id(ink.id);
723 self.write(|active| content::strokes_changes(active, container, &kept, &added))
724 }
725
726 /// Gives a new picture what OneNote stores for an inserted one: the payload embedded
727 /// in the section's file-data store, a file-data object declaring it by identity and
728 /// extension, and a picture object that a paragraph holds as content or the page
729 /// lists as a child.
730 fn insert_image(&mut self, image: &Image, holder: Option<ExGuid>) -> Result<(), Error> {
731 let Some(bytes) = &image.bytes else {
732 return Err(invalid("A new picture needs its payload"));
733 };
734 let image_id = self.allocate(image.id)?;
735 let file_id = ExGuid {
736 guid: crate::write::fresh_guid()?,
737 n: 1,
738 };
739 let payload_guid = crate::write::fresh_guid()?;
740 let payload: &[u8] = bytes;
741 self.write_with(&[(payload_guid, payload)], |active| {
742 content::picture_changes(active, image, image_id, file_id, payload_guid, holder)
743 })
744 }
745
746 fn edit_image(&mut self, stored: &Image, image: &Image) -> Result<(), Error> {
747 content::picture_fixed_fields(stored, image)?;
748 let object = self.id(image.id);
749 self.write(|active| {
750 content::picture_edit_changes(
751 active,
752 object,
753 (&stored.layout, &stored.alt),
754 &image.layout,
755 &image.alt,
756 )
757 })
758 }
759
760 fn place_containers(
761 &mut self,
762 old: &View<'_>,
763 new: &View<'_>,
764 placed: &BTreeMap<ExGuid, Vec<ExGuid>>,
765 containers: &[ExGuid],
766 ) -> Result<(), Error> {
767 for container in containers {
768 let after = &new.children[container];
769 let current: Vec<ExGuid> = placed
770 .get(container)
771 .map(|list| {
772 list.iter()
773 .copied()
774 .filter(|id| new.paragraphs.contains_key(id))
775 .collect()
776 })
777 .unwrap_or_default();
778 let kept = kept_set(&current, after, |_| true)?;
779 let mut next = None;
780 for id in after.iter().rev() {
781 let anchor = next.map(|n| self.id(n));
782 if !old.paragraphs.contains_key(id) && !self.alias.contains_key(id) {
783 let paragraph = new.paragraphs[id];
784 // A styled paragraph appended to a stored outline is a split whose
785 // precondition failed under a merge; it stays for review. A new outline
786 // takes styled paragraphs as built (a copied page).
787 let mut root = *container;
788 while let Some(parent) = new.container.get(&root) {
789 root = *parent;
790 }
791 if paragraph.style.is_some() && old.outlines.contains_key(&root) {
792 return Err(invalid(
793 "New paragraphs in a stored outline contain plain text without styles",
794 ));
795 }
796 if paragraph.media != Default::default() {
797 return Err(invalid(
798 "Recording annotations are made by OneNote while it records",
799 ));
800 }
801 // A new table starts as an empty text paragraph whose content the
802 // structure pass replaces with the table.
803 let (text, text_id) = match &paragraph.content {
804 ParagraphContent::Text(text) if text.date_field.is_none() => {
805 (text.text.text(), Some(text.id))
806 }
807 ParagraphContent::Table(_)
808 | ParagraphContent::Image(_)
809 | ParagraphContent::Attachment(_)
810 | ParagraphContent::Ink(_) => ("", None),
811 _ => {
812 return Err(invalid(
813 "New paragraphs contain plain text without fields or styles",
814 ));
815 }
816 };
817 let insertion =
818 Insertion::paragraph(self.id(*container), anchor, text, self.author)?;
819 self.write(|active| insertion.changes(active))?;
820 self.alias.insert(*id, insertion.object());
821 if let Some(text_id) =
822 text_id.filter(|id| !self.active.live.revision.objects.contains_key(id))
823 {
824 self.alias.insert(text_id, insertion.text_object());
825 }
826 } else if !kept.contains(id) {
827 let edit =
828 TreeEdit::move_to(self.id(*id), self.id(*container), anchor, self.author)?;
829 self.write(|active| edit.changes(active))?;
830 }
831 next = Some(*id);
832 }
833 }
834 Ok(())
835 }
836
837 /// Changes the shading and indents of stored cells in place.
838 fn edit_cells(&mut self, old: &View<'_>, new: &View<'_>) -> Result<(), Error> {
839 let stored: BTreeMap<ExGuid, &super::TableCell> = old
840 .paragraphs
841 .values()
842 .filter_map(|paragraph| match &paragraph.content {
843 ParagraphContent::Table(table) => Some(table),
844 _ => None,
845 })
846 .flat_map(|table| table.rows.iter().flat_map(|row| &row.cells))
847 .map(|cell| (cell.id, cell))
848 .collect();
849 for paragraph in new.paragraphs.values() {
850 let ParagraphContent::Table(table) = &paragraph.content else {
851 continue;
852 };
853 for cell in table.rows.iter().flat_map(|row| &row.cells) {
854 let Some(previous) = stored.get(&cell.id) else {
855 continue;
856 };
857 if (cell.shading, &cell.indents) == (previous.shading, &previous.indents) {
858 continue;
859 }
860 let object = self.id(cell.id);
861 self.write(|active| {
862 table::cell_changes(
863 active,
864 object,
865 (previous.shading, &previous.indents),
866 cell.shading,
867 &cell.indents,
868 )
869 })?;
870 }
871 }
872 Ok(())
873 }
874
875 /// Creates tables, rows and cells the model added, rebuilds every table's row and cell
876 /// order to the model's, and writes the column widths, locks and border flag.
877 fn edit_table_structure(&mut self, old: &View<'_>, new: &View<'_>) -> Result<(), Error> {
878 let old_tables: BTreeMap<ExGuid, &super::Table> = old
879 .paragraphs
880 .values()
881 .filter_map(|p| match &p.content {
882 ParagraphContent::Table(table) => Some((table.id, table)),
883 _ => None,
884 })
885 .collect();
886 for (paragraph_id, paragraph) in &new.paragraphs {
887 let ParagraphContent::Table(table) = &paragraph.content else {
888 continue;
889 };
890 let previous = old_tables.get(&table.id).copied();
891 // A table is built once its holder is placed, and only once.
892 if !(old.paragraphs.contains_key(paragraph_id) || self.alias.contains_key(paragraph_id))
893 || self.built.contains(&table.id)
894 {
895 continue;
896 }
897 self.built.insert(table.id);
898 let unchanged = previous.is_some_and(|previous| {
899 previous.columns == table.columns
900 && previous.borders == table.borders
901 && previous.rows.len() == table.rows.len()
902 && previous.rows.iter().zip(&table.rows).all(|(a, b)| {
903 a.id == b.id
904 && a.cells.len() == b.cells.len()
905 && a.cells.iter().zip(&b.cells).all(|(x, y)| x.id == y.id)
906 })
907 });
908 if unchanged {
909 continue;
910 }
911 let table_id = if previous.is_some() {
912 self.id(table.id)
913 } else {
914 self.allocate(table.id)?
915 };
916 let existing_rows: BTreeSet<ExGuid> = previous
917 .map(|p| p.rows.iter().map(|r| r.id).collect())
918 .unwrap_or_default();
919 let existing_cells: BTreeSet<ExGuid> = previous
920 .map(|p| {
921 p.rows
922 .iter()
923 .flat_map(|r| r.cells.iter().map(|c| c.id))
924 .collect()
925 })
926 .unwrap_or_default();
927 let template = previous
928 .and_then(|p| p.rows.first())
929 .and_then(|r| r.cells.first())
930 .map(|cell| cell.indents.as_slice());
931 let mut structure = Structure {
932 rows: Vec::new(),
933 new_rows: BTreeSet::new(),
934 new_cells: BTreeMap::new(),
935 columns: &table.columns,
936 borders: table.borders,
937 };
938 for row in &table.rows {
939 let row_id = if existing_rows.contains(&row.id) {
940 self.id(row.id)
941 } else {
942 let written = self.allocate(row.id)?;
943 structure.new_rows.insert(written);
944 written
945 };
946 let mut cells = Vec::new();
947 for cell in &row.cells {
948 let cell_id = if existing_cells.contains(&cell.id) {
949 self.id(cell.id)
950 } else {
951 let written = self.allocate(cell.id)?;
952 structure.new_cells.insert(
953 written,
954 (table::cell_indents(&cell.indents, template), cell.shading),
955 );
956 written
957 };
958 cells.push(cell_id);
959 }
960 structure.rows.push((row_id, cells));
961 }
962 let holder = previous.is_none().then(|| self.id(*paragraph_id));
963 self.write(|active| table::table_changes(active, table_id, holder, &structure))?;
964 }
965 Ok(())
966 }
967
968 fn delete(
969 &mut self,
970 old: &View<'_>,
971 new: &View<'_>,
972 consumed: &BTreeSet<ExGuid>,
973 ) -> Result<(), Error> {
974 let removed_outline =
975 |id: ExGuid| old.outlines.contains_key(&id) && !new.outlines.contains_key(&id);
976 for id in old.paragraphs.keys() {
977 if new.paragraphs.contains_key(id) || consumed.contains(id) {
978 continue;
979 }
980 let mut ancestor = old.container[id];
981 let mut covered = false;
982 // A surviving ancestor is placed where the model says, carrying this paragraph along.
983 while !new.paragraphs.contains_key(&ancestor) {
984 if removed_outline(ancestor)
985 || (old.paragraphs.contains_key(&ancestor) && !consumed.contains(&ancestor))
986 || (old.children.contains_key(&ancestor)
987 && !old.paragraphs.contains_key(&ancestor)
988 && !old.outlines.contains_key(&ancestor)
989 && !new.children.contains_key(&ancestor))
990 {
991 covered = true;
992 break;
993 }
994 match old.container.get(&ancestor) {
995 Some(parent) => ancestor = *parent,
996 None => break,
997 }
998 }
999 if covered {
1000 continue;
1001 }
1002 let edit = TreeEdit::delete(*id, self.author)?;
1003 self.write(|active| edit.changes(active))?;
1004 }
1005 let kept: BTreeSet<ExGuid> = page_images(new.page)
1006 .map(|(id, _)| id)
1007 .chain(page_ink(new.page).map(|(id, _)| id))
1008 .collect();
1009 for id in page_images(old.page)
1010 .map(|(id, _)| id)
1011 .chain(page_ink(old.page).map(|(id, _)| id))
1012 {
1013 if !kept.contains(&id) {
1014 let edit = TreeEdit::delete(id, self.author)?;
1015 self.write(|active| edit.changes(active))?;
1016 }
1017 }
1018 for id in old.outlines.keys() {
1019 if removed_outline(*id) {
1020 if old.title_outlines.contains(id) {
1021 return Err(invalid("Title outlines cannot be removed"));
1022 }
1023 let edit = TreeEdit::delete(*id, self.author)?;
1024 self.write(|active| edit.changes(active))?;
1025 }
1026 }
1027 Ok(())
1028 }
1029
1030 /// Rewrites every equation paragraph whose stored text object differs from the model.
1031 fn edit_equations(&mut self, new: &View<'_>) -> Result<(), Error> {
1032 let current = self.current()?;
1033 let current = View::new(&current, false)?;
1034 for (id, paragraph) in &new.paragraphs {
1035 let Some(text) = paragraph.text() else {
1036 continue;
1037 };
1038 let stored = current.text(self.id(*id));
1039 // Deleting an equation leaves no math in the model, but its stored runs are still math.
1040 if !super::Math::is_equation(&text.text)
1041 && !stored.is_some_and(|stored| super::Math::is_equation(&stored.text))
1042 {
1043 continue;
1044 }
1045 let stored = stored
1046 .ok_or_else(|| invalid("An equation paragraph is missing after placement"))?;
1047 if stored.text == text.text {
1048 continue;
1049 }
1050 let object = stored.id;
1051 self.write(|active| content::equation_changes(active, object, &text.text))?;
1052 }
1053 Ok(())
1054 }
1055
1056 /// Lifts the paragraphs of outline groups into their container wherever this save changes
1057 /// the container's children or their depths, for placement to see them; `edit_levels`
1058 /// groups them again.
1059 fn ungroup(&mut self, old: &View<'_>, new: &View<'_>) -> Result<(), Error> {
1060 for (container, children) in &old.children {
1061 let Some(after) = new.children.get(container) else {
1062 continue;
1063 };
1064 let grouped = self.active.view.nodes[container]
1065 .children
1066 .iter()
1067 .any(|id| matches!(self.active.view.nodes[id].kind, Kind::OutlineGroup));
1068 if !grouped
1069 || (after == children
1070 && old.depths(*container, children) == new.depths(*container, after))
1071 {
1072 continue;
1073 }
1074 let object = *container;
1075 self.write(|active| levels::ungroup_changes(active, object))?;
1076 }
1077 Ok(())
1078 }
1079
1080 /// Indents each container's children to the depths the model gives them.
1081 fn edit_levels(&mut self, new: &View<'_>) -> Result<(), Error> {
1082 let current = self.current()?;
1083 let stored = View::new(&current, false)?;
1084 for (container, children) in &new.children {
1085 let depths = new
1086 .depths(*container, children)
1087 .ok_or_else(|| invalid("A paragraph lies no deeper than its parent"))?;
1088 let object = self.id(*container);
1089 let written: Vec<ExGuid> = children.iter().map(|id| self.id(*id)).collect();
1090 if children.is_empty() || stored.depths(object, &written) == Some(depths.clone()) {
1091 continue;
1092 }
1093 let is_cell =
1094 !new.paragraphs.contains_key(container) && !new.outlines.contains_key(container);
1095 let (level, runs) = levels::runs(&written, &depths, is_cell)?;
1096 self.write(|active| levels::regroup_changes(active, object, level, &runs))?;
1097 }
1098 Ok(())
1099 }
1100
1101 /// Gives a paragraph the text object its model names when a split and a join in one save
1102 /// moved text between paragraphs: a stored object is taken over, as OneNote moves a lower
1103 /// paragraph's text into an emptied upper one; a new one starts as a copy of the text
1104 /// the paragraph holds.
1105 fn edit_text_identities(&mut self, new: &View<'_>) -> Result<(), Error> {
1106 for (id, paragraph) in &new.paragraphs {
1107 let Some(text) = paragraph.text() else {
1108 continue;
1109 };
1110 let object = self.id(*id);
1111 let stored = self.active.view.nodes[&object].content[0];
1112 if stored == self.id(text.id) {
1113 continue;
1114 }
1115 let target = if self.active.live.revision.objects.contains_key(&text.id) {
1116 text.id
1117 } else {
1118 self.allocate(text.id)?
1119 };
1120 let unstyled = paragraph.style.is_none();
1121 self.write(|active| {
1122 let parents = active.editable_parents(object)?;
1123 let modified = crate::create::current_timestamps()?.0.to_le_bytes();
1124 let raw = &active.live.revision;
1125 let mut changed = BTreeMap::new();
1126 if !raw.objects.contains_key(&target) {
1127 let mut copy = PropertyObject::from_object(&raw.objects[&stored])?;
1128 copy.remove(&[0x1c001c98, 0x14001c99])?;
1129 if unstyled {
1130 copy.remove(&[0x2000342c])?;
1131 }
1132 copy.reference(target)?;
1133 changed.insert(target, copy);
1134 }
1135 let mut holder = PropertyObject::from_object(&raw.objects[&object])?;
1136 let reference = holder.reference(target)?;
1137 holder.set(&[(0x24001c1f, &reference), (0x14001d7a, &modified)])?;
1138 changed.insert(object, holder);
1139 crate::formatting::touch_ancestors(raw, parents, object, &modified, &mut changed)?;
1140 Ok(changed)
1141 })?;
1142 }
1143 Ok(())
1144 }
1145
1146 fn edit_text(&mut self, new: &View<'_>) -> Result<(), Error> {
1147 let current = self.current()?;
1148 let current = View::new(&current, false)?;
1149 for (id, paragraph) in &new.paragraphs {
1150 let Some(text) = paragraph.text() else {
1151 continue;
1152 };
1153 let image_id = self.id(*id);
1154 let Some(stored) = current.text(image_id) else {
1155 return Err(invalid("A paragraph is missing after structural edits"));
1156 };
1157 if stored.id != self.id(text.id) {
1158 return Err(invalid("Text object identities cannot change"));
1159 }
1160 if stored.text.text() == text.text.text() {
1161 continue;
1162 }
1163 let (range, replacement) = text_edit(stored.text.text(), text.text.text())?;
1164 let object = stored.id;
1165 self.write(|active| crate::edit::text_changes(active, object, range, &replacement))?;
1166 }
1167 Ok(())
1168 }
1169
1170 /// Gives each paragraph the list nodes its model references: a definition new to the
1171 /// section becomes a list node carrying the model's identity, a changed one is rewritten
1172 /// in place, a definition another paragraph already owns is copied because native list
1173 /// nodes belong to one paragraph, and a dropped reference leaves the node unreferenced.
1174 fn edit_lists(&mut self, after: &Page, new: &View<'_>) -> Result<(), Error> {
1175 let current = self.current()?;
1176 let stored = View::new(&current, false)?;
1177 let mut owners: BTreeMap<ExGuid, ExGuid> = BTreeMap::new();
1178 for (id, paragraph) in &stored.paragraphs {
1179 for list in &paragraph.lists {
1180 owners.insert(*list, *id);
1181 }
1182 }
1183 for (id, paragraph) in &new.paragraphs {
1184 let image_id = self.id(*id);
1185 let previous = stored
1186 .paragraphs
1187 .get(&image_id)
1188 .ok_or_else(|| invalid("A paragraph is missing after text edits"))?;
1189 let same = paragraph.lists.len() == previous.lists.len()
1190 && paragraph
1191 .lists
1192 .iter()
1193 .zip(&previous.lists)
1194 .all(|(model, node)| {
1195 self.id(*model) == *node
1196 && after.definitions.get(model) == current.definitions.get(node)
1197 });
1198 if same {
1199 continue;
1200 }
1201 let mut nodes = Vec::new();
1202 for list in &paragraph.lists {
1203 // Writer-allocated identities keep a nonzero sequence number so the node's
1204 // compact identity is never the null identity; squash renames them to the
1205 // model's. A copy for a second owner keeps its allocated identity.
1206 let node_id = match owners.get(&self.id(*list)) {
1207 Some(owner) if *owner == image_id => self.id(*list),
1208 Some(_) => ExGuid {
1209 guid: crate::write::fresh_guid()?,
1210 n: 1,
1211 },
1212 // A node its paragraph let go of in this save serves the model again.
1213 None if self.active.live.revision.objects.contains_key(list) => *list,
1214 None => {
1215 let written = ExGuid {
1216 guid: crate::write::fresh_guid()?,
1217 n: 1,
1218 };
1219 self.alias.insert(*list, written);
1220 written
1221 }
1222 };
1223 let definition = after
1224 .definitions
1225 .get(list)
1226 .ok_or_else(|| invalid("A paragraph references a missing list definition"))?;
1227 nodes.push((node_id, properties::list_values(definition)?));
1228 }
1229 let object = self.id(*id);
1230 self.write(|active| properties::list_changes(active, object, &nodes))?;
1231 }
1232 Ok(())
1233 }
1234
1235 /// Rewrites the note tags of paragraphs and text objects whose model tags differ from
1236 /// the stored ones. A tag definition new to the section becomes a definition object
1237 /// carrying the model's identity after squash.
1238 fn edit_tags(&mut self, after: &Page, new: &View<'_>) -> Result<(), Error> {
1239 let current = self.current()?;
1240 let stored = View::new(&current, false)?;
1241 for (id, paragraph) in &new.paragraphs {
1242 let image_id = self.id(*id);
1243 let previous = stored
1244 .paragraphs
1245 .get(&image_id)
1246 .ok_or_else(|| invalid("A paragraph is missing after text edits"))?;
1247 let mut targets = vec![(image_id, &paragraph.tags, &previous.tags)];
1248 if let (Some(text), Some(before)) = (paragraph.text(), previous.text()) {
1249 targets.push((self.id(text.id), &text.tags, &before.tags));
1250 }
1251 for (object, tags, stored_tags) in targets {
1252 if crate::op::lower::same_tags(tags, stored_tags) {
1253 continue;
1254 }
1255 let mut entries = Vec::new();
1256 for tag in tags {
1257 let definition = tag
1258 .definition
1259 .ok_or_else(|| invalid("A note tag names its definition"))?;
1260 let known = self.active.live.revision.objects.contains_key(&definition)
1261 || self.alias.contains_key(&definition);
1262 let written = if known {
1263 self.id(definition)
1264 } else {
1265 let allocated = ExGuid {
1266 guid: crate::write::fresh_guid()?,
1267 n: 1,
1268 };
1269 self.alias.insert(definition, allocated);
1270 allocated
1271 };
1272 entries.push((written, tag, after.definitions.get(&definition)));
1273 }
1274 self.write(|active| properties::tag_changes(active, object, &entries))?;
1275 }
1276 }
1277 Ok(())
1278 }
1279
1280 /// A new paragraph, or one whose style changed, references its paragraph style from
1281 /// its text object; a style the page has not stored yet is created from its
1282 /// definition, as OneNote keeps quick styles.
1283 fn edit_paragraph_styles(
1284 &mut self,
1285 old: &View<'_>,
1286 new: &View<'_>,
1287 after: &Page,
1288 ) -> Result<(), Error> {
1289 for (id, paragraph) in &new.paragraphs {
1290 let Some(definition) = paragraph.style else {
1291 continue;
1292 };
1293 if old
1294 .paragraphs
1295 .get(id)
1296 .is_some_and(|previous| previous.style == paragraph.style)
1297 {
1298 continue;
1299 }
1300 let Some(text) = paragraph.text() else {
1301 return Err(invalid("Only text paragraphs take a paragraph style"));
1302 };
1303 let known = self.active.live.revision.objects.contains_key(&definition)
1304 || self.alias.contains_key(&definition);
1305 let style_id = if known {
1306 self.id(definition)
1307 } else {
1308 let allocated = ExGuid {
1309 guid: crate::write::fresh_guid()?,
1310 n: 1,
1311 };
1312 self.alias.insert(definition, allocated);
1313 allocated
1314 };
1315 let object = self.id(text.id);
1316 let definition = after.definitions.get(&definition);
1317 self.write(|active| properties::style_changes(active, object, style_id, definition))?;
1318 }
1319 Ok(())
1320 }
1321
1322 fn edit_paragraph_formatting(&mut self, new: &View<'_>) -> Result<(), Error> {
1323 let current = self.current()?;
1324 let current = View::new(&current, false)?;
1325 for (id, paragraph) in &new.paragraphs {
1326 let Some(text) = paragraph.text() else {
1327 continue;
1328 };
1329 let stored = current
1330 .text(self.id(*id))
1331 .ok_or_else(|| invalid("A paragraph is missing after text edits"))?;
1332 let values = crate::op::lower::paragraph_change(&stored.text, &text.text)?;
1333 if values.is_empty() {
1334 continue;
1335 }
1336 if text.date_field.is_some() {
1337 return Err(invalid(
1338 "Generated title fields cannot be formatted as ordinary text",
1339 ));
1340 }
1341 let object = stored.id;
1342 self.write(|active| properties::paragraph_format_changes(active, object, &values))?;
1343 }
1344 Ok(())
1345 }
1346
1347 fn edit_formatting(&mut self, new: &View<'_>) -> Result<(), Error> {
1348 let current = self.current()?;
1349 let current = View::new(&current, false)?;
1350 for (id, paragraph) in &new.paragraphs {
1351 let Some(text) = paragraph.text() else {
1352 continue;
1353 };
1354 let stored = current
1355 .text(self.id(*id))
1356 .ok_or_else(|| invalid("A paragraph is missing after text edits"))?;
1357 if stored.text.text() != text.text.text() {
1358 return Err(invalid("Text edits did not converge on the model"));
1359 }
1360 let fresh = self.alias.contains_key(&text.id);
1361 for (range, set, clear) in crate::op::lower::format_edits(&stored.text, &text.text, fresh)? {
1362 let object = stored.id;
1363 self.write(|active| {
1364 crate::formatting::format_changes(active, object, range, &set, &clear)
1365 })?;
1366 }
1367 }
1368 Ok(())
1369 }
1370
1371 fn edit_layout(&mut self, old: &View<'_>, new: &View<'_>) -> Result<(), Error> {
1372 let current = self.current()?;
1373 let current = View::new(&current, false)?;
1374 for (id, paragraph) in &new.paragraphs {
1375 let stored = current
1376 .paragraphs
1377 .get(&self.id(*id))
1378 .ok_or_else(|| invalid("A paragraph is missing after text edits"))?;
1379 if stored.collapsed != paragraph.collapsed {
1380 let object = self.id(*id);
1381 let edit = OutlineEdit::Collapsed(paragraph.collapsed);
1382 self.write(|active| edit.changes(active, object))?;
1383 }
1384 }
1385 for (id, outline) in &new.outlines {
1386 if new.title_outlines.contains(id) {
1387 continue;
1388 }
1389 let stored = current
1390 .outlines
1391 .get(&self.id(*id))
1392 .ok_or_else(|| invalid("An outline is missing after text edits"))?;
1393 let object = self.id(*id);
1394 // A new outline takes the model's indentation table (a copied outline keeps
1395 // its levels' offsets); a stored table stays as it is.
1396 if !old.outlines.contains_key(id)
1397 && !outline.indents.is_empty()
1398 && outline.indents != stored.indents
1399 {
1400 let indents = content::measurement_bytes(&outline.indents, 4)?;
1401 self.write(|active| {
1402 let raw = &active.live.revision;
1403 let mut node = PropertyObject::from_object(&raw.objects[&object])?;
1404 node.set(&[(0x1c001c12, &indents)])?;
1405 Ok(BTreeMap::from([(object, node)]))
1406 })?;
1407 }
1408 if (outline.layout.x, outline.layout.y) != (stored.layout.x, stored.layout.y) {
1409 let (Some(x), Some(y)) = (outline.layout.x, outline.layout.y) else {
1410 return Err(invalid("An outline position needs both coordinates"));
1411 };
1412 let edit = OutlineEdit::Position { x, y };
1413 self.write(|active| edit.changes(active, object))?;
1414 }
1415 if (outline.layout.max_width, outline.layout.width_set_by_user)
1416 != (stored.layout.max_width, stored.layout.width_set_by_user)
1417 && (old.outlines.contains_key(id) || outline.layout.max_width.is_some())
1418 {
1419 let Some(points) = outline.layout.max_width else {
1420 return Err(invalid("An outline width cannot be removed"));
1421 };
1422 let edit = OutlineEdit::Width {
1423 points,
1424 user_set: outline.layout.width_set_by_user == Some(true),
1425 };
1426 self.write(|active| edit.changes(active, object))?;
1427 }
1428 }
1429 Ok(())
1430 }
1431}
1432
1433/// Payload identities the file-data store of `store` declares, in order.
1434pub(crate) fn declared_payloads(store: &Store<'_>) -> Vec<[u8; 16]> {
1435 store
1436 .lists
1437 .values()
1438 .flat_map(|list| &list.nodes)
1439 .filter(|node| node.id == 0x94)
1440 .filter_map(|node| node.payload.get(..16).and_then(|g| g.try_into().ok()))
1441 .collect()
1442}
1443
1444/// Writes the spaces the typed writers `edited` as one transaction on the validated
1445/// `source`, renaming writer-allocated identities to the model's and embedding the
1446/// `payloads` it lacks. A protected `source` takes the revisions its plaintext twin gained.
1447pub(crate) fn squash(
1448 source: &RevisionIndex<'_>,
1449 edited: &[(ExGuid, &crate::ResolvedRevision<'_>)],
1450 payloads: &[([u8; 16], &[u8])],
1451 alias: &BTreeMap<ExGuid, ExGuid>,
1452 protection: Option<&dyn crate::write::Protection>,
1453) -> Result<Vec<u8>, Error> {
1454 let rename: BTreeMap<ExGuid, ExGuid> = alias
1455 .iter()
1456 .map(|(model, image)| (*image, *model))
1457 .collect();
1458 let existing = declared_payloads(source.store);
1459 let payloads: Vec<_> = payloads
1460 .iter()
1461 .filter(|(guid, _)| !existing.contains(guid))
1462 .copied()
1463 .collect();
1464 let edit = |index: &RevisionIndex<'_>| {
1465 let mut changes = BTreeMap::new();
1466 for (sid, after) in edited {
1467 let before = match protection {
1468 Some(protection) => protection.resolve(*sid, index.active(*sid)?)?,
1469 None => index.resolve_active(*sid)?,
1470 };
1471 if before.roots != after.roots {
1472 return Err(invalid("Page edits cannot change revision roots"));
1473 }
1474 // Retired styles and deleted content stay in history; only live objects are written.
1475 let live = after.reachable()?;
1476 let mut changed = BTreeMap::new();
1477 for (id, object) in &after.objects {
1478 if !live.contains(id) {
1479 continue;
1480 }
1481 // Stored tables keep the entries an object names, so those decide.
1482 if let Some(previous) = before.objects.get(id)
1483 && previous.jcid == object.jcid
1484 && previous.data == object.data
1485 {
1486 let entries = match object.data {
1487 ObjectData::Properties(bytes) => crate::write::table_entries(bytes)?,
1488 _ => Vec::new(),
1489 };
1490 if entries
1491 .iter()
1492 .all(|entry| previous.global_ids.get(entry) == object.global_ids.get(entry))
1493 {
1494 continue;
1495 }
1496 }
1497 let id = rename.get(id).copied().unwrap_or(*id);
1498 let mut replacement = match object.data {
1499 ObjectData::Properties(_) => {
1500 let mut replacement = PropertyObject::from_object(object)?;
1501 remap(&mut replacement, &rename)?;
1502 replacement
1503 }
1504 ObjectData::File {
1505 reference,
1506 extension,
1507 } => {
1508 let text = |bytes: &[u8]| {
1509 String::from_utf16(
1510 &bytes
1511 .chunks_exact(2)
1512 .map(|pair| u16::from_le_bytes([pair[0], pair[1]]))
1513 .collect::<Vec<_>>(),
1514 )
1515 .map_err(|_| invalid("Invalid UTF-16 file-data declaration"))
1516 };
1517 let mut replacement =
1518 PropertyObject::file(id, &text(reference)?, &text(extension)?)?;
1519 replacement.jcid = object.jcid;
1520 replacement
1521 }
1522 ObjectData::Encrypted(_) => {
1523 return Err(invalid("Page edits only produce property objects"));
1524 }
1525 };
1526 replacement.reference(id)?;
1527 if before.objects.contains_key(&id) && rename.values().any(|model| *model == id) {
1528 return Err(invalid(
1529 "A new model identity already exists in the section",
1530 ));
1531 }
1532 changed.insert(id, replacement);
1533 }
1534 changes.insert(*sid, RevisionEdit::Update(changed));
1535 }
1536 Ok(changes)
1537 };
1538 let output = crate::write::build_on(source, &payloads, protection, edit)?;
1539 crate::write::check(&output, protection.is_none())?;
1540 Ok(output)
1541}
1542
1543fn remap(object: &mut PropertyObject, rename: &BTreeMap<ExGuid, ExGuid>) -> Result<(), Error> {
1544 let mut remapped = Vec::new();
1545 for property in PropertySets::parse(&object.bytes)?.sets.iter().flatten() {
1546 if let Value::References {
1547 stream: crate::IdStream::Objects,
1548 compact_ids,
1549 } = property.value
1550 {
1551 for bytes in compact_ids.chunks_exact(4) {
1552 let offset = bytes.as_ptr().addr() - object.bytes.as_ptr().addr();
1553 let id = crate::bytes::Cursor { bytes, offset }.compact(&object.global_ids)?;
1554 if let Some(model) = rename.get(&id) {
1555 remapped.push((offset, *model));
1556 }
1557 }
1558 }
1559 }
1560 for (offset, id) in remapped {
1561 let reference = object.reference(id)?;
1562 object.bytes[offset..offset + 4].copy_from_slice(&reference);
1563 }
1564 Ok(())
1565}
1566
1567#[cfg(test)]
1568mod tests {
1569 use super::*;
1570 use crate::{
1571 document::Format,
1572 page::{PageParagraph, Paragraph, TextObject, text::new_id},
1573 };
1574
1575 /// A new page in a new section and the page with `count` paragraphs added in an outline.
1576 fn generated(count: usize) -> (Vec<u8>, ExGuid, Page) {
1577 let section = crate::create_section("big.one", "First", "Author").unwrap();
1578 let creation = crate::PageCreation::new(None, Some("Big"), "Author").unwrap();
1579 let source = crate::PreparedEdit::create_page(&section, &creation)
1580 .unwrap()
1581 .as_bytes()
1582 .to_vec();
1583 let space = creation.space();
1584 let store = Store::parse(&source).unwrap();
1585 let index = RevisionIndex::parse(&store).unwrap();
1586 let mut page =
1587 Page::from_space(&crate::document::Document::parse(&index).unwrap(), space).unwrap();
1588 let paragraph = |i: usize| PageParagraph {
1589 id: new_id().unwrap(),
1590 parent: None,
1591 level: 1,
1592 style: None,
1593 format: Format::default(),
1594 content: ParagraphContent::Text(TextObject {
1595 id: new_id().unwrap(),
1596 date_field: None,
1597 text: Paragraph::new(format!("Paragraph {i}"), Format::default()),
1598 tags: Vec::new(),
1599 }),
1600 lists: Vec::new(),
1601 tags: Vec::new(),
1602 media: Default::default(),
1603 collapsed: false,
1604 };
1605 page.objects.push(PageObject::Outline(Outline {
1606 id: new_id().unwrap(),
1607 title: false,
1608 min_width: None,
1609 layout: crate::document::Layout {
1610 x: Some(36.0),
1611 y: Some(86.4),
1612 ..Default::default()
1613 },
1614 indents: Vec::new(),
1615 paragraphs: (0..count).map(paragraph).collect(),
1616 unsupported: Vec::new(),
1617 }));
1618 (source, space, page)
1619 }
1620
1621 #[test]
1622 fn a_page_write_builds_one_revision_however_many_paragraphs_it_adds() {
1623 for count in [10, 40] {
1624 let (source, space, after) = generated(count);
1625 let builds = crate::write::BUILDS.with(std::cell::Cell::get);
1626 let written = write_page(&source, space, &after, "Author").unwrap();
1627 assert_eq!(crate::write::BUILDS.with(std::cell::Cell::get) - builds, 1);
1628 let store = Store::parse(&written).unwrap();
1629 let index = RevisionIndex::parse(&store).unwrap();
1630 let read = Page::from_space(&crate::document::Document::parse(&index).unwrap(), space)
1631 .unwrap();
1632 let texts = |page: &Page| -> Vec<String> {
1633 page.objects
1634 .iter()
1635 .filter_map(|object| match object {
1636 PageObject::Outline(outline) => Some(outline),
1637 _ => None,
1638 })
1639 .flat_map(|outline| &outline.paragraphs)
1640 .filter_map(|paragraph| Some(paragraph.text()?.text.text().to_owned()))
1641 .collect()
1642 };
1643 assert_eq!(texts(&read), texts(&after));
1644 }
1645 }
1646}
crates/onestore/src/pages.rs+251-76
......@@ -1,9 +1,9 @@
11use crate::{
22 Error, ExGuid,
33 create::{current_timestamps, properties, string},
4 document::{Document, FieldValue, Kind},
4 document::{FieldValue, Kind},
55 op::content::{NATIVE_INDENTS, measurement_bytes},
6 write::{PropertyObject, RevisionEdit, fresh_guid, write_revisions},
6 write::{PropertyObject, fresh_guid},
77};
88use serde::{Deserialize, Serialize};
99use std::{
......@@ -15,7 +15,7 @@ fn invalid(message: &'static str) -> Error {
1515 Error { offset: 0, message }
1616}
1717
18fn metadata_id(space: ExGuid) -> ExGuid {
18pub(crate) fn metadata_id(space: ExGuid) -> ExGuid {
1919 let mut id = ExGuid {
2020 guid: space.guid,
2121 n: 1,
......@@ -28,7 +28,7 @@ fn metadata_id(space: ExGuid) -> ExGuid {
2828 id
2929}
3030
31fn metadata_guid(metadata: &crate::document::Element<'_>) -> Result<[u8; 16], Error> {
31pub(crate) fn metadata_guid(metadata: &crate::document::Element<'_>) -> Result<[u8; 16], Error> {
3232 metadata.extra[0]
3333 .iter()
3434 .find_map(|field| match field.value {
......@@ -38,7 +38,7 @@ fn metadata_guid(metadata: &crate::document::Element<'_>) -> Result<[u8; 16], Er
3838 .ok_or_else(|| invalid("Page metadata needs its page identifier"))
3939}
4040
41fn set_references(object: &mut PropertyObject, property: u32, ids: &[ExGuid]) -> Result<(), Error> {
41pub(crate) fn set_references(object: &mut PropertyObject, property: u32, ids: &[ExGuid]) -> Result<(), Error> {
4242 let mut references = Vec::new();
4343 for id in ids {
4444 references.extend_from_slice(&object.reference(*id)?);
......@@ -51,6 +51,20 @@ fn set_references(object: &mut PropertyObject, property: u32, ids: &[ExGuid]) ->
5151 object.copy_property(&source, property)
5252}
5353
54/// A conflict page (MS-ONE 2.1.2): the version of a page a merge could not take, kept
55/// read-only under the page it conflicts with.
56#[derive(Debug, Clone, PartialEq, Eq)]
57pub struct ConflictPage {
58 pub space: ExGuid,
59 pub title: String,
60 /// Whose version it is (`ConflictingUserName`).
61 pub user: String,
62 /// FILETIME of its `TopologyCreationTimeStamp`, when the merge made it.
63 pub created: Option<u64>,
64 /// The conflict objects on it (`IsConflictObjectForRender`), which OneNote highlights.
65 pub objects: Vec<ExGuid>,
66}
67
5468/// Whether an edit preserves position or moves before a page (None appends).
5569#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
5670#[serde(deny_unknown_fields)]
......@@ -138,42 +152,7 @@ pub(crate) struct PageParts<'a> {
138152 pub stored_metadata: (ExGuid, PropertyObject),
139153}
140154
141pub(crate) fn edit_pages(
142 source: &[u8],
143 edits: &[PageEdit],
144 removals: &[ExGuid],
145) -> Result<Vec<u8>, Error> {
146 write_revisions(source, |index| {
147 let document = Document::parse(index)?;
148 let pages: Vec<ExGuid> = document.pages()?.into_iter().map(|(sid, _)| sid).collect();
149 let root = document.active(document.root)?;
150 let raw = index.resolve_active(document.root)?;
151 let parts = |sid: ExGuid| -> Result<PageParts<'_>, Error> {
152 let page = document.active(sid)?;
153 let revision = index.resolve_active(sid)?;
154 let metadata = page
155 .roots
156 .get(&2)
157 .and_then(|id| page.nodes.get(id))
158 .ok_or_else(|| invalid("Page metadata is unavailable"))?;
159 let (manifest, stored) = (revision.roots[&1], revision.roots[&2]);
160 Ok(PageParts {
161 metadata: metadata.clone(),
162 manifest: (manifest, PropertyObject::from_object(&revision.objects[&manifest])?),
163 stored_metadata: (stored, PropertyObject::from_object(&revision.objects[&stored])?),
164 })
165 };
166 let Some(changes) = page_changes(document.root, root, &raw, &pages, parts, edits, removals)? else {
167 return Ok(BTreeMap::new());
168 };
169 Ok(changes
170 .into_iter()
171 .map(|(space, changes)| (space, RevisionEdit::Update(changes)))
172 .collect())
173 })
174}
175
176/// `edit_pages` on a section kept open: the objects each changed space stores.
155/// Page moves and removals on a section kept open: the objects each changed space stores.
177156pub(crate) fn section_changes(
178157 section: &mut crate::Section<'_>,
179158 edits: &[PageEdit],
......@@ -197,12 +176,18 @@ pub(crate) fn section_changes(
197176 let parts = |sid: ExGuid| -> Result<PageParts<'_>, Error> {
198177 let revision = section.revision(sid)?;
199178 let (manifest, stored) = (revision.roots[&1], revision.roots[&2]);
200 Ok(PageParts {
201 metadata: crate::document::Element::parse_with(
202 &revision.objects[&stored],
179 let element = |id| {
180 crate::document::Element::parse_with(
181 &revision.objects[id],
203182 crate::FileType::Section,
204183 &mut |_| Err(invalid("Page metadata holds no payload")),
205 )?,
184 )
185 };
186 if element(&manifest)?.content.len() != 1 {
187 return Err(invalid("Each page space must contain one page with a valid level"));
188 }
189 Ok(PageParts {
190 metadata: element(&stored)?,
206191 manifest: (manifest, PropertyObject::from_object(&revision.objects[&manifest])?),
207192 stored_metadata: (stored, PropertyObject::from_object(&revision.objects[&stored])?),
208193 })
......@@ -491,6 +476,12 @@ pub struct PageCreation {
491476 title: Option<String>,
492477 author: String,
493478 created: u32,
479 /// The title's date and time fields' text, for a titled page.
480 #[serde(default, skip_serializing_if = "Option::is_none")]
481 date: Option<[String; 2]>,
482 /// The page identity and creation time (FILETIME) a page keeps from elsewhere.
483 #[serde(default, skip_serializing_if = "Option::is_none")]
484 kept: Option<([u8; 16], u64)>,
494485}
495486
496487impl PageCreation {
......@@ -505,11 +496,57 @@ impl PageCreation {
505496 title: title.map(str::to_owned),
506497 author: author.to_owned(),
507498 created: current_timestamps()?.0,
499 date: None,
500 kept: None,
508501 };
509502 page.validate()?;
510503 Ok(page)
511504 }
512505
506 /// Gives a titled page's title OneNote 2010's date and time fields, showing `date` and
507 /// `time`: the creation time (`created`) as the user's long date and short time.
508 pub fn dated(mut self, date: &str, time: &str) -> Result<Self, Error> {
509 if self.title.is_none() {
510 return Err(invalid("Only a titled page shows its date"));
511 }
512 self.date = Some([date.to_owned(), time.to_owned()]);
513 self.validate()?;
514 Ok(self)
515 }
516
517 /// Keeps `identity`, the page identity internal links name, and `created`, as OneNote
518 /// 2010 keeps both for a page it moves to the notebook's recycle bin.
519 pub fn keeping(mut self, identity: [u8; 16], created: u64) -> Result<Self, Error> {
520 if identity == [0; 16] {
521 return Err(invalid("Keep an existing page identity"));
522 }
523 let seconds = (created / 10_000_000)
524 .checked_sub(11_644_473_600 + 315_532_800)
525 .and_then(|seconds| u32::try_from(seconds).ok())
526 .ok_or_else(|| invalid("The kept creation time is outside OneNote's range"))?;
527 self.created = seconds;
528 self.kept = Some((identity, created));
529 Ok(self)
530 }
531
532 /// Who creates the page.
533 pub fn author(&self) -> &str {
534 &self.author
535 }
536
537 /// When the page was created, FILETIME, which its date and time fields show.
538 pub fn created(&self) -> u64 {
539 match self.kept {
540 Some((_, created)) => created,
541 None => (u64::from(self.created) + 315_532_800 + 11_644_473_600) * 10_000_000,
542 }
543 }
544
545 /// The page identity internal links name as `page-id`.
546 fn identity(&self) -> [u8; 16] {
547 self.kept.map_or(self.guid, |(identity, _)| identity)
548 }
549
513550 pub fn space(&self) -> ExGuid {
514551 ExGuid {
515552 guid: self.guid,
......@@ -531,6 +568,11 @@ impl PageCreation {
531568 })
532569 }
533570
571 /// The page the new page goes before; `None` appends it.
572 pub fn before(&self) -> Option<ExGuid> {
573 self.before
574 }
575
534576 /// Changes the insertion anchor while retaining all identities and creation metadata.
535577 pub fn reposition(&self, before: Option<ExGuid>) -> Result<Self, Error> {
536578 let mut page = self.clone();
......@@ -547,8 +589,9 @@ impl PageCreation {
547589 || self.before.is_some_and(|id| id.guid == [0; 16])
548590 || self
549591 .title
550 .as_ref()
551 .is_some_and(|title| title.contains(['\0', '\r', '\n', '\u{fffc}', '\u{fddf}']))
592 .iter()
593 .chain(self.date.iter().flatten())
594 .any(|text| text.contains(['\0', '\r', '\n', '\u{fffc}', '\u{fddf}']))
552595 {
553596 return Err(invalid(
554597 "Use a new page identity, an existing page anchor, and ordinary single-line title text",
......@@ -557,28 +600,7 @@ impl PageCreation {
557600 Ok(())
558601 }
559602
560 pub(crate) fn apply(&self, source: &[u8]) -> Result<Vec<u8>, Error> {
561 self.validate()?;
562 write_revisions(source, |index| {
563 if index.spaces.contains_key(&self.space()) {
564 return Err(invalid(
565 "This page identity already exists; reconcile the original creation",
566 ));
567 }
568 let document = Document::parse(index)?;
569 document.pages()?;
570 let view = document.active(document.root)?;
571 let raw = index.resolve_active(document.root)?;
572 let (root, roots, objects) = self.creation(view, &raw)?;
573 Ok(BTreeMap::from([
574 (document.root, RevisionEdit::Update(root)),
575 (self.space(), RevisionEdit::Create { roots, objects }),
576 ]))
577 })
578 }
579
580 /// `apply` on a section kept open: the root space's changes, and the new space's roots
581 /// and objects.
603 /// The root space's changes creating this page, and the new space's roots and objects.
582604 pub(crate) fn changes(
583605 &self,
584606 section: &mut crate::Section<'_>,
......@@ -594,6 +616,133 @@ impl PageCreation {
594616 self.creation(&page.view, &page.live.revision)
595617 }
596618
619 /// The object space of a conflict page this creation makes, as OneNote 2010 stores one:
620 /// the page's space with `jcidConflictPageMetaData` (MS-ONE 2.2.35) as its metadata,
621 /// naming this creation's author as the conflicting user and `title` as the page's.
622 /// Returns the roots, the objects and the metadata, which the conflicting page's
623 /// manifest also keeps a copy of.
624 pub(crate) fn conflict(
625 &self,
626 section: &mut crate::Section<'_>,
627 title: &str,
628 level: u32,
629 ) -> Result<(BTreeMap<u32, ExGuid>, crate::active::Changes, PropertyObject), Error> {
630 if self.before.is_some() {
631 return Err(invalid("A conflict page is not placed in the page list"));
632 }
633 let (_, roots, mut objects) = self.changes(section)?;
634 let timestamp = self.created();
635 let initials = crate::create::initials(&self.author);
636 let metadata = PropertyObject {
637 jcid: 0x20038,
638 bytes: properties(&[
639 (0x1c001cf3, string(title)),
640 (0x1c001c30, self.identity().to_vec()),
641 (0x14001dff, level.to_le_bytes().to_vec()),
642 (0x14001d82, 40_u32.to_le_bytes().to_vec()),
643 (0x1400348b, 40_u32.to_le_bytes().to_vec()),
644 (0x18001c65, timestamp.to_le_bytes().to_vec()),
645 (0x1c001d9e, string(&self.author)),
646 (0x1c001d9f, string(&initials)),
647 ])?,
648 global_ids: Arc::new(BTreeMap::from([(0, self.guid)])),
649 };
650 objects.insert(roots[&2], metadata.clone());
651 Ok((roots, objects, metadata))
652 }
653
654 /// The title's second outline as OneNote 2010 writes it for a new page (object numbers,
655 /// jcids and properties): a read-only outline of two paragraphs, the date then the
656 /// time, in the `PageDateTime` style.
657 fn date_fields(&self, date: &str, time: &str) -> Vec<(u32, u32, Vec<(u32, Vec<u8>)>)> {
658 let reference = |n: u32| n.to_le_bytes().to_vec();
659 let modified = || (0x14001d7a, self.created.to_le_bytes().to_vec());
660 let yes = |id: u32| (id | 1 << 31, Vec::new());
661 let paragraph = |text: u32| {
662 vec![
663 modified(),
664 (0x20001d79, reference(17)),
665 (0x20001d78, reference(17)),
666 (0x14001d09, self.created.to_le_bytes().to_vec()),
667 (0x0c001c03, vec![1]),
668 (0x24001c1f, reference(text)),
669 yes(0x08001cb2),
670 yes(0x08001d0c),
671 (0x08001c34, Vec::new()),
672 ]
673 };
674 let text = |value: &str, role: u32| {
675 // OneNote keeps Latin-1 text as TextExtendedAscii, other text as Unicode.
676 let content = match value.chars().map(|c| u8::try_from(c).ok()).collect::<Option<Vec<u8>>>() {
677 Some(latin) => (0x1c003498, latin),
678 None => (0x1c001c22, string(value)),
679 };
680 vec![
681 modified(),
682 (0x2000342c, reference(23)),
683 yes(0x08001cde),
684 yes(0x08001c00),
685 yes(0x08001c88),
686 yes(0x08001d0c),
687 yes(role),
688 (0x10001cfe, 0x409_u16.to_le_bytes().to_vec()),
689 (0x1c001cc8, vec![0, 0, 0, 0, 3]),
690 content,
691 (0x24001e13, reference(19)),
692 yes(0x080034dd),
693 ]
694 };
695 let no = |id: u32| (id, Vec::new());
696 vec![
697 (
698 20,
699 0x6000c,
700 vec![
701 modified(),
702 (0x24001c20, [reference(21), reference(24)].concat()),
703 (
704 0x1c001c12,
705 measurement_bytes(&NATIVE_INDENTS, 4).expect("Native indents encode"),
706 ),
707 (0x0c001c13, vec![0]),
708 (0x0c001c03, vec![1]),
709 (0x14001c1c, 0.6_f32.to_le_bytes().to_vec()),
710 yes(0x08001cb5),
711 yes(0x08001c00),
712 yes(0x08001cb2),
713 yes(0x08001cde),
714 yes(0x08001d0c),
715 (0x14001c3e, reference(0)),
716 (0x14001c84, reference(0xc)),
717 ],
718 ),
719 (21, 0x6000d, paragraph(22)),
720 (22, 0x6000e, text(date, 0x08001cb5)),
721 (
722 23,
723 0x12004d,
724 vec![
725 (0x1c00345a, string("PageDateTime")),
726 no(0x08001c04),
727 no(0x08001c05),
728 no(0x08001c06),
729 (0x14001c0c, 0x0080_8080_u32.to_le_bytes().to_vec()),
730 (0x1c001c0a, string("Calibri")),
731 (0x10001c0b, 20_u16.to_le_bytes().to_vec()),
732 (0x1400342e, 0_f32.to_le_bytes().to_vec()),
733 (0x1400342f, 0_f32.to_le_bytes().to_vec()),
734 (0x14003430, 0_f32.to_le_bytes().to_vec()),
735 no(0x08001c07),
736 (0x14001c0d, 0xff00_0000_u32.to_le_bytes().to_vec()),
737 no(0x08001c09),
738 no(0x08001c08),
739 ],
740 ),
741 (24, 0x6000d, paragraph(25)),
742 (25, 0x6000e, text(time, 0x08001c87)),
743 ]
744 }
745
597746 fn creation(
598747 &self,
599748 view: &crate::document::Revision<'_>,
......@@ -621,9 +770,7 @@ impl PageCreation {
621770 };
622771 let id = |n| ExGuid { guid: self.guid, n };
623772 let reference = |n: u32| n.to_le_bytes().to_vec();
624 let timestamp = ((u64::from(self.created) + 315532800 + 11644473600) * 10000000)
625 .to_le_bytes()
626 .to_vec();
773 let timestamp = self.created().to_le_bytes().to_vec();
627774 let modified = || (0x14001d7a, self.created.to_le_bytes().to_vec());
628775 let mut table = BTreeMap::from([(0, self.guid)]);
629776 let metadata_id = metadata_id(self.space());
......@@ -638,7 +785,7 @@ impl PageCreation {
638785 };
639786 let title = self.title.as_deref().unwrap_or_default().trim_start();
640787 let metadata = vec![
641 (0x1c001c30, self.guid.to_vec()),
788 (0x1c001c30, self.identity().to_vec()),
642789 (0x1c001cf3, string(title)),
643790 (0x14001d82, reference(40)),
644791 (0x1400348b, reference(40)),
......@@ -649,6 +796,7 @@ impl PageCreation {
649796 vec![
650797 modified(),
651798 (0x1c001d75, string(&self.author)),
799 (0x1c001df8, string(&crate::create::initials(&self.author))),
652800 (0x1c001d3c, string("")),
653801 ],
654802 crate::create::page_margins(),
......@@ -669,7 +817,11 @@ impl PageCreation {
669817 0x6002c,
670818 vec![
671819 modified(),
672 (0x24001c20, reference(14)),
820 (
821 0x24001c20,
822 [reference(14), self.date.iter().flat_map(|_| reference(20)).collect()]
823 .concat(),
824 ),
673825 (0x14001c14, 0_f32.to_le_bytes().to_vec()),
674826 (0x14001c15, 0_f32.to_le_bytes().to_vec()),
675827 (0x14001c3e, reference(0x0009_000c)),
......@@ -723,7 +875,7 @@ impl PageCreation {
723875 (0x88001cb4, vec![]),
724876 ],
725877 ),
726 (17, 0x120001, vec![(0x1c001d75, string(&self.author))]),
878 (17, 0x120001, crate::create::author_properties(&self.author)),
727879 (
728880 18,
729881 0x12004d,
......@@ -737,6 +889,11 @@ impl PageCreation {
737889 ] {
738890 objects.insert(id(n), object(jcid, values)?);
739891 }
892 if let Some([date, time]) = &self.date {
893 for (n, jcid, values) in self.date_fields(date, time) {
894 objects.insert(id(n), object(jcid, values)?);
895 }
896 }
740897 }
741898 let series = object(
742899 0x60008,
......@@ -774,7 +931,25 @@ impl PageCreation {
774931#[cfg(test)]
775932mod tests {
776933 use super::*;
777 use crate::{RevisionIndex, Store};
934 use crate::{
935 RevisionIndex, Store,
936 document::Document,
937 op::{Edit, Op, OpError, SectionOp},
938 write::{RevisionEdit, write_revisions},
939 };
940
941 /// Page moves and removals through a section, sealed: the image they leave.
942 fn edit_pages(source: &[u8], edits: &[PageEdit], removals: &[ExGuid]) -> Result<Vec<u8>, OpError> {
943 let arena = crate::Arena::default();
944 let mut section = crate::Section::open(&arena, source.to_vec()).map_err(OpError::Failed)?;
945 let mut ops = vec![Op::Section(SectionOp::Pages(edits.to_vec()))];
946 if !removals.is_empty() {
947 ops.push(Op::Section(SectionOp::Delete(removals.to_vec())));
948 }
949 section.apply("Author", &Edit { at: 133_700_000_000_000_000, ops })?;
950 section.seal().map_err(OpError::Failed)?;
951 Ok(section.image())
952 }
778953
779954 #[test]
780955 fn page_movement_restores_an_optional_metadata_cache_from_history() {
crates/onestore/src/paragraph.rs+10-89
......@@ -6,7 +6,6 @@ use crate::{
66 edit::update_title,
77 write::{PropertyObject, fresh_guid},
88};
9use serde::{Deserialize, Serialize};
109use std::{
1110 collections::{BTreeMap, BTreeSet},
1211 ops::Range,
......@@ -17,11 +16,10 @@ fn invalid(message: &'static str) -> Error {
1716 Error { offset: 0, message }
1817}
1918
20/// Splits ordinary paragraph text while retaining the new objects' identities across retries.
21/// The original paragraph/text remain on the left; nested children move to the right.
22#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
23#[serde(deny_unknown_fields)]
24pub struct ParagraphSplit {
19/// Splits ordinary paragraph text. The original paragraph/text remain on the left; nested
20/// children move to the right.
21#[derive(Debug, Clone, PartialEq)]
22pub(crate) struct ParagraphSplit {
2523 guid: [u8; 16],
2624 text: ExGuid,
2725 offset: u32,
......@@ -31,7 +29,7 @@ pub struct ParagraphSplit {
3129
3230impl ParagraphSplit {
3331 /// The offset is measured in UTF-16 code units and must lie between Unicode scalars.
34 pub fn new(text: ExGuid, offset: u32, author: &str) -> Result<Self, Error> {
32 pub(crate) fn new(text: ExGuid, offset: u32, author: &str) -> Result<Self, Error> {
3533 if text.guid == [0; 16] || author.contains('\0') {
3634 return Err(invalid(
3735 "Choose paragraph text and an author name without NUL",
......@@ -46,64 +44,6 @@ impl ParagraphSplit {
4644 })
4745 }
4846
49 /// Identity of the new right paragraph.
50 pub fn object(&self) -> ExGuid {
51 ExGuid {
52 guid: self.guid,
53 n: 1,
54 }
55 }
56
57 /// Identity of the new right paragraph's text.
58 pub fn text_object(&self) -> ExGuid {
59 ExGuid {
60 guid: self.guid,
61 n: 2,
62 }
63 }
64
65 /// Original text identity and UTF-16 split boundary.
66 pub fn position(&self) -> (ExGuid, u32) {
67 (self.text, self.offset)
68 }
69
70 /// Changes the split boundary while retaining allocated identities and creation time.
71 /// Preparing the edit validates the offset against the supplied text.
72 pub fn reposition(&self, offset: u32) -> Self {
73 let mut split = self.clone();
74 split.offset = offset;
75 split
76 }
77
78 pub(crate) fn apply(&self, source: &[u8], space: ExGuid) -> Result<Vec<u8>, Error> {
79 if self.guid == [0; 16] || self.text.guid == [0; 16] || self.author.contains('\0') {
80 return Err(invalid(
81 "Choose paragraph text and an author name without NUL",
82 ));
83 }
84 crate::active::write(source, space, |active| self.changes(active))
85 }
86
87 pub(crate) fn changes(&self, active: &ActivePage<'_>) -> Result<Changes, Error> {
88 let lists = (5..256)
89 .map(|n| ExGuid {
90 guid: self.guid,
91 n,
92 })
93 .collect::<Vec<_>>();
94 let changes = self.changes_as(active, self.object(), self.text_object(), &lists)?;
95 let raw = &active.live.revision;
96 if changes
97 .keys()
98 .any(|id| id.guid == self.guid && raw.objects.contains_key(id))
99 {
100 return Err(invalid(
101 "A split identity already exists; reconcile the existing edit",
102 ));
103 }
104 Ok(changes)
105 }
106
10747 /// `changes` creating the right paragraph `paragraph` with text `right`, and a copy of
10848 /// the left paragraph's list nodes under `lists`, in order.
10949 pub(crate) fn changes_as(
......@@ -244,7 +184,7 @@ impl ParagraphSplit {
244184 author_id,
245185 PropertyObject {
246186 jcid: 0x120001,
247 bytes: properties(&[(0x1c001d75, string(&self.author))])?,
187 bytes: properties(&crate::create::author_properties(&self.author))?,
248188 global_ids: Arc::new(BTreeMap::from([(0, self.guid)])),
249189 },
250190 );
......@@ -411,22 +351,16 @@ fn ordinary_text<'a>(
411351/// The left paragraph survives. Empty left text adopts the right text identity;
412352/// otherwise the left text survives. The left paragraph's tags are retained.
413353/// Right-side tags are removed from active text, including when the left text is empty.
414#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
415#[serde(deny_unknown_fields)]
416pub struct ParagraphJoin {
354#[derive(Debug, Clone, PartialEq)]
355pub(crate) struct ParagraphJoin {
417356 left: ExGuid,
418357 right: ExGuid,
419358 author: String,
420359}
421360
422361impl ParagraphJoin {
423 /// Ordered left and right text identities.
424 pub fn texts(&self) -> [ExGuid; 2] {
425 [self.left, self.right]
426 }
427
428362 /// Select the preceding leaf paragraph's text and the following paragraph's text.
429 pub fn new(left: ExGuid, right: ExGuid, author: &str) -> Result<Self, Error> {
363 pub(crate) fn new(left: ExGuid, right: ExGuid, author: &str) -> Result<Self, Error> {
430364 if left == right || left.guid == [0; 16] || right.guid == [0; 16] || author.contains('\0') {
431365 return Err(invalid(
432366 "Choose two distinct text objects and an author name without NUL",
......@@ -439,19 +373,6 @@ impl ParagraphJoin {
439373 })
440374 }
441375
442 pub(crate) fn apply(&self, source: &[u8], space: ExGuid) -> Result<Vec<u8>, Error> {
443 if self.left == self.right
444 || self.left.guid == [0; 16]
445 || self.right.guid == [0; 16]
446 || self.author.contains('\0')
447 {
448 return Err(invalid(
449 "Choose two distinct text objects and an author name without NUL",
450 ));
451 }
452 crate::active::write(source, space, |active| self.changes(active))
453 }
454
455376 pub(crate) fn changes(&self, active: &ActivePage<'_>) -> Result<Changes, Error> {
456377 let pages = &active.pages;
457378 if pages.len() != 1 {
......@@ -672,7 +593,7 @@ impl ParagraphJoin {
672593 author,
673594 PropertyObject {
674595 jcid: 0x120001,
675 bytes: properties(&[(0x1c001d75, string(&self.author))])?,
596 bytes: properties(&crate::create::author_properties(&self.author))?,
676597 global_ids: Arc::new(BTreeMap::from([(0, author.guid)])),
677598 },
678599 );
crates/onestore/src/protected/crypto.rs+1
......@@ -8,6 +8,7 @@ use zeroize::{Zeroize, Zeroizing};
88const NS: &str = "http://schemas.microsoft.com/office/2006/encryption";
99const PASSWORD_NS: &str = "http://schemas.microsoft.com/office/2006/keyEncryptor/password";
1010
11#[derive(Clone)]
1112pub(super) struct Key {
1213 value: Zeroizing<[u8; 16]>,
1314 file_iv: [u8; 16],
crates/onestore/src/protected/mod.rs+76-56
......@@ -7,7 +7,11 @@
77
88mod crypto;
99
10use crate::{ExGuid, FileDataReference, ObjectData, Reference, RevisionIndex, document::Document};
10use crate::{
11 ExGuid, FileDataReference, ObjectData, Reference, RevisionIndex, Transaction,
12 document::Document,
13 op::{Edit, Op, OpError},
14};
1115use std::{
1216 collections::{BTreeMap, BTreeSet},
1317 fmt,
......@@ -107,7 +111,18 @@ pub struct UnlockedSection<'a> {
107111impl<'a> UnlockedSection<'a> {
108112 /// Verifies the password and every labeled revision before exposing a view.
109113 /// Metadata and password input are each bounded to 64 KiB.
110 pub fn open(index: &'a RevisionIndex<'a>, password: &str, mut limits: Limits) -> Result<Self> {
114 pub fn open(index: &'a RevisionIndex<'a>, password: &str, limits: Limits) -> Result<Self> {
115 Self::unlock(index, limits, |metadata, rounds| {
116 crypto::Key::open(metadata, password, rounds)
117 })
118 }
119
120 /// `open` with each distinct encryption metadata's key from `key`.
121 fn unlock(
122 index: &'a RevisionIndex<'a>,
123 mut limits: Limits,
124 mut key: impl FnMut(&[u8], &mut u64) -> Result<crypto::Key>,
125 ) -> Result<Self> {
111126 if index.store.header.file_type != crate::FileType::Section {
112127 return Err(Error::Unsupported);
113128 }
......@@ -163,10 +178,7 @@ impl<'a> UnlockedSection<'a> {
163178 let metadata =
164179 metadata.ok_or_else(|| invalid("Protected object space has no revision"))?;
165180 if !keys.contains_key(metadata) {
166 keys.insert(
167 metadata,
168 crypto::Key::open(metadata, password, &mut limits.kdf_rounds)?,
169 );
181 keys.insert(metadata, key(metadata, &mut limits.kdf_rounds)?);
170182 }
171183 let key = &keys[metadata];
172184 for rid in space.labels.values().copied().collect::<BTreeSet<_>>() {
......@@ -294,8 +306,10 @@ impl UnlockedSection<'_> {
294306 use crate::write::{PropertyObject, RevisionEdit, append_revisions};
295307 let copy = |space: ExGuid, revision: ExGuid| {
296308 let revision = self.resolve(space, revision)?;
309 // A revision still declares objects its edits detached; a new one holds live ones.
310 let live = revision.reachable()?;
297311 let mut objects = BTreeMap::new();
298 for (id, object) in &revision.objects {
312 for (id, object) in revision.objects.iter().filter(|(id, _)| live.contains(id)) {
299313 let copy = match object.data {
300314 ObjectData::File {
301315 reference,
......@@ -390,60 +404,66 @@ impl UnlockedSection<'_> {
390404 }
391405}
392406
393/// An edited page of a protected section as one stored revision per changed space, the
394/// protected counterpart of [`crate::PreparedEdit::page`].
395pub(crate) fn write_page(
396 source: &[u8],
397 password: &str,
398 space: ExGuid,
399 page: &crate::page::Page,
400 author: &str,
401) -> Result<Vec<u8>> {
402 let store = crate::Store::parse(source)?;
403 let index = RevisionIndex::parse(&store)?;
404 let unlocked = UnlockedSection::open(&index, password, Limits::default())?;
405 if unlocked.keys.len() != 1 {
406 return Err(Error::Unsupported);
407 }
408 let twin = unlocked.twin()?;
409 let applied = Zeroizing::new(crate::page::write::write_page(&twin, space, page, author)?);
410 let applied_store = crate::Store::parse(&applied)?;
411 let applied_index = RevisionIndex::parse(&applied_store)?;
412 let twin_store = crate::Store::parse(&twin)?;
413 if RevisionIndex::parse(&twin_store)?
414 .spaces
415 .keys()
416 .ne(applied_index.spaces.keys())
417 {
418 return Err(invalid("Page edits cannot create object spaces"));
419 }
420 // The twin's scaffold spaces are not the section's.
421 let mut edited = Vec::new();
422 for (sid, space) in &index.spaces {
423 let rid = applied_index.active(*sid)?;
424 if space.labels.get(&(ExGuid::default(), 1)) != Some(&rid) {
425 edited.push((*sid, applied_index.resolve(*sid, rid)?));
407impl UnlockedSection<'_> {
408 /// Applies `edit` to this section as `Section::apply` applies it to an ordinary one and
409 /// seals the result as one revision per changed space, stored under the section's key.
410 /// Page ops only: a protected section's page list is not edited.
411 pub fn apply(&self, author: &str, edit: &Edit) -> std::result::Result<Transaction, OpError> {
412 let failed = |error: crate::Error| OpError::Failed(error);
413 if self.keys.len() != 1 {
414 return Err(OpError::Unsupported(
415 "Only a section under one key takes edits",
416 ));
417 }
418 if edit.ops.iter().any(|op| matches!(op, Op::Section(_))) {
419 return Err(OpError::Unsupported("A protected section's page list is not edited"));
426420 }
421 let twin = self.twin().map_err(failed)?;
422 let arena = crate::Arena::default();
423 let mut section = crate::Section::open(&arena, twin.to_vec()).map_err(failed)?;
424 section.apply(author, edit)?;
425 section.seal().map_err(failed)?;
426 let applied = Zeroizing::new(section.image());
427 drop(section);
428 let applied_store = crate::Store::parse(&applied).map_err(failed)?;
429 let applied_index = RevisionIndex::parse(&applied_store).map_err(failed)?;
430 // The twin's scaffold spaces are not the section's.
431 let edited = self
432 .index
433 .spaces
434 .keys()
435 .map(|space| Ok((*space, applied_index.resolve_active(*space)?)))
436 .collect::<std::result::Result<Vec<_>, crate::Error>>()
437 .map_err(failed)?;
438 let edited: Vec<_> = edited.iter().map(|(space, after)| (*space, after)).collect();
439 let payloads = crate::write::declared_payloads(&applied_store)
440 .into_iter()
441 .map(|guid| Ok((guid, applied_store.file_data(guid)?)))
442 .collect::<std::result::Result<Vec<_>, crate::Error>>()
443 .map_err(failed)?;
444 let transaction = crate::write::squash(self.index, &edited, &payloads, Some(self))
445 .map_err(failed)?
446 .ok_or(OpError::Unsupported("The edit changes nothing"))?;
447 let written = crate::write::applied(self.index.store.data, Some(&transaction))
448 .map_err(failed)?;
449 let store = crate::Store::parse(&written).map_err(failed)?;
450 let index = RevisionIndex::parse(&store).map_err(failed)?;
451 UnlockedSection::unlock(&index, Limits::default(), |metadata, _| {
452 self.keys.get(metadata).cloned().ok_or(Error::PasswordMismatch)
453 })
454 .map_err(|error| match error {
455 Error::Invalid(error) => failed(error),
456 _ => failed(crate::Error {
457 offset: 0,
458 message: "The written section does not unlock under its key",
459 }),
460 })?;
461 Ok(transaction)
427462 }
428 let edited: Vec<_> = edited.iter().map(|(sid, after)| (*sid, after)).collect();
429 let payloads = crate::page::write::declared_payloads(&applied_store)
430 .into_iter()
431 .map(|guid| Ok((guid, applied_store.file_data(guid)?)))
432 .collect::<std::result::Result<Vec<_>, crate::Error>>()?;
433 let written = crate::page::write::squash(
434 &index,
435 &edited,
436 &payloads,
437 &BTreeMap::new(),
438 Some(&unlocked),
439 )?;
440 let store = crate::Store::parse(&written)?;
441 UnlockedSection::open(&RevisionIndex::parse(&store)?, password, Limits::default())?;
442 Ok(written)
443463}
444464
445465impl UnlockedSection<'_> {
446 /// The section key; `write_page` admits sections with exactly one.
466 /// The section key; `apply` admits sections with exactly one.
447467 fn key(&self) -> &crypto::Key {
448468 self.keys.values().next().expect("one section key")
449469 }
crates/onestore/src/section.rs+175-130
......@@ -226,15 +226,108 @@ impl<'a> Section<'a> {
226226 }
227227 }
228228
229 /// The conflict pages of each page that has them, in section order, each page's in the
230 /// order OneNote 2010 lists them under it: the last stored first. Read from the pages'
231 /// manifests and the conflict pages' metadata alone.
232 pub fn conflicts(&mut self) -> Result<Vec<(ExGuid, Vec<crate::ConflictPage>)>> {
233 let listed: Vec<ExGuid> = self.pages()?.into_iter().map(|(space, ..)| space).collect();
234 let element = |object: &crate::Object<'a>| {
235 crate::document::Element::parse_with(object, FileType::Section, &mut |_| {
236 Err(Error {
237 offset: 0,
238 message: "Page metadata holds no payload",
239 })
240 })
241 };
242 let mut conflicts = Vec::new();
243 for page in listed {
244 let revision = self.revision(page)?;
245 let Some(manifest) = revision
246 .roots
247 .get(&1)
248 .and_then(|id| revision.objects.get(id))
249 else {
250 continue;
251 };
252 let mut pages = Vec::new();
253 for space in element(manifest)?.spaces.into_iter().rev() {
254 let Some(metadata) = self.revision(space).ok().and_then(|revision| {
255 revision
256 .roots
257 .get(&2)
258 .and_then(|id| revision.objects.get(id))
259 }) else {
260 continue;
261 };
262 let metadata = element(metadata)?;
263 let crate::document::Kind::ConflictMetadata { title, author, .. } = metadata.kind
264 else {
265 continue;
266 };
267 let created = metadata.extra[0]
268 .iter()
269 .find_map(|field| match field.value {
270 crate::document::FieldValue::Bytes(bytes) if field.id == 0x18001c65 => {
271 bytes.try_into().ok().map(u64::from_le_bytes)
272 }
273 _ => None,
274 });
275 let mut objects = Vec::new();
276 for (id, object) in &self.revision(space)?.objects {
277 let ObjectData::Properties(bytes) = object.data else {
278 continue;
279 };
280 if crate::PropertySets::parse(bytes)?.sets[0]
281 .iter()
282 .any(|property| property.id == 0x88001d96)
283 {
284 objects.push(*id);
285 }
286 }
287 pages.push(crate::ConflictPage {
288 space,
289 title: title.unwrap_or_default(),
290 user: author.unwrap_or_default(),
291 created,
292 objects,
293 });
294 }
295 if !pages.is_empty() {
296 conflicts.push((page, pages));
297 }
298 }
299 Ok(conflicts)
300 }
301
302 /// The page series holding each listed page. Moving a page gives it a series of its
303 /// own, so a page in another series than before was moved.
304 pub fn series(&mut self) -> Result<BTreeMap<ExGuid, ExGuid>> {
305 let view = &self.active(self.root)?.view;
306 let section = view.roots.get(&1).and_then(|id| view.nodes.get(id)).ok_or(Error {
307 offset: 0,
308 message: "Section root is unavailable",
309 })?;
310 Ok(section
311 .children
312 .iter()
313 .filter_map(|series| Some((*series, view.nodes.get(series)?)))
314 .flat_map(|(series, node)| node.spaces.iter().map(move |page| (*page, series)))
315 .collect())
316 }
317
229318 /// Page spaces in section order with the title and outline level (1 at the top) the
230319 /// page list shows, read from each page's metadata alone.
231320 pub fn pages(&mut self) -> Result<Vec<(ExGuid, String, u32)>> {
232321 let root = self.root;
233322 let view = &self.active(root)?.view;
234 let section = view.roots.get(&1).and_then(|id| view.nodes.get(id)).ok_or(Error {
235 offset: 0,
236 message: "Section root is unavailable",
237 })?;
323 let section = view
324 .roots
325 .get(&1)
326 .and_then(|id| view.nodes.get(id))
327 .ok_or(Error {
328 offset: 0,
329 message: "Section root is unavailable",
330 })?;
238331 let spaces: Vec<ExGuid> = section
239332 .children
240333 .iter()
......@@ -253,7 +346,10 @@ impl<'a> Section<'a> {
253346 for space in spaces {
254347 let revision = self.revision(space)?;
255348 let (mut title, mut level) = (None, 1);
256 if let Some(metadata) = revision.roots.get(&2).and_then(|id| revision.objects.get(id))
349 if let Some(metadata) = revision
350 .roots
351 .get(&2)
352 .and_then(|id| revision.objects.get(id))
257353 && let crate::document::Kind::Metadata {
258354 title: stored,
259355 level: stored_level,
......@@ -263,7 +359,10 @@ impl<'a> Section<'a> {
263359 level = stored_level.unwrap_or(1);
264360 }
265361 if title.is_none() {
266 let manifest = revision.roots.get(&1).and_then(|id| revision.objects.get(id));
362 let manifest = revision
363 .roots
364 .get(&1)
365 .and_then(|id| revision.objects.get(id));
267366 if let Some(manifest) = manifest
268367 && let Some(page) = element(manifest)?
269368 .content
......@@ -695,6 +794,17 @@ mod tests {
695794 result
696795 }
697796
797 /// The image the image writer leaves: the `edited` revisions squashed onto the source
798 /// `index` parsed.
799 fn squashed(
800 index: &RevisionIndex<'_>,
801 edited: &[(ExGuid, &ResolvedRevision<'_>)],
802 payloads: &[([u8; 16], &[u8])],
803 ) -> Vec<u8> {
804 let transaction = crate::write::squash(index, edited, payloads, None).unwrap();
805 crate::write::applied(index.store.data, transaction.as_ref()).unwrap()
806 }
807
698808 /// The page spaces of an image, fullest first.
699809 fn page_spaces(image: &[u8]) -> Vec<ExGuid> {
700810 let store = Store::parse(image).unwrap();
......@@ -729,65 +839,62 @@ mod tests {
729839 GUIDS.set(Some(1 << 62 | seed << 40));
730840 let (mut sealed, mut checkpoints) = (0, 0);
731841 for batch in 0..batches {
732 let space = spaces[batch % spaces.len().min(pages)];
733 let store = Store::parse(&image).unwrap();
734 let index = RevisionIndex::parse(&store).unwrap();
735 let bump = Bump::new();
736 let mut legacy = ActivePage::parse(&index, space).unwrap();
737 for _ in 0..1 + (writes.0 as usize >> 40) % largest {
738 let Ok(changes) = writes.next(section.active(space).unwrap()) else {
739 continue;
740 };
741 let stored = legacy.write(&bump, &[], changes.clone()).unwrap();
742 assert_eq!(
743 section.apply_changes(space, &[], changes).unwrap(),
744 stored,
745 "batch {batch}"
746 );
747 }
748 let guids = (batch as u64 + 1) << 32;
749 let expected = seeded(guids, || {
750 crate::page::write::squash(
751 &index,
752 &[(space, &legacy.live.revision)],
753 &legacy.payloads,
754 &BTreeMap::new(),
755 None,
756 )
757 .unwrap()
758 });
759 let before = section.stamp().clone();
760 let chains = depths(&section);
761 let transaction = seeded(guids, || section.seal().unwrap());
762 checkpoints += depths(&section)
763 .iter()
764 .filter(|(id, depth)| chains.get(id).is_some_and(|before| before > depth))
765 .count();
766 let mut written = image.clone();
767 if let Some(transaction) = &transaction {
768 assert_eq!(transaction.base, before);
769 transaction.apply(&mut written).unwrap();
770 sealed += 1;
771 }
772 assert!(written == expected, "batch {batch}");
773 assert_eq!(Stamp::of(&written).unwrap(), *section.stamp());
774 drop(legacy);
775 drop(index);
776 drop(store);
777 image = expected;
778 if batch % 25 == 0 || batch + 1 == batches {
779 assert!(section.image() == image);
842 // The writers stamp modification times; a fixed clock keeps the batches alike.
843 let at = 134_000_000_000_000_000 + batch as u64 * 10_000_000;
844 crate::create::at(at, || {
845 let space = spaces[batch % spaces.len().min(pages)];
780846 let store = Store::parse(&image).unwrap();
781847 let index = RevisionIndex::parse(&store).unwrap();
782 index.validate_current().unwrap();
783 let document = Document::parse(&index).unwrap();
784 for space in &spaces {
848 let bump = Bump::new();
849 let mut legacy = ActivePage::parse(&index, space).unwrap();
850 for _ in 0..1 + (writes.0 as usize >> 40) % largest {
851 let Ok(changes) = writes.next(section.active(space).unwrap()) else {
852 continue;
853 };
854 let stored = legacy.write(&bump, &[], changes.clone()).unwrap();
785855 assert_eq!(
786 section.page(*space).unwrap(),
787 Page::from_space(&document, *space).unwrap()
856 section.apply_changes(space, &[], changes).unwrap(),
857 stored,
858 "batch {batch}"
788859 );
789860 }
790 }
861 let guids = (batch as u64 + 1) << 32;
862 let expected = seeded(guids, || {
863 squashed(&index, &[(space, &legacy.live.revision)], &legacy.payloads)
864 });
865 let before = section.stamp().clone();
866 let chains = depths(&section);
867 let transaction = seeded(guids, || section.seal().unwrap());
868 checkpoints += depths(&section)
869 .iter()
870 .filter(|(id, depth)| chains.get(id).is_some_and(|before| before > depth))
871 .count();
872 let mut written = image.clone();
873 if let Some(transaction) = &transaction {
874 assert_eq!(transaction.base, before);
875 transaction.apply(&mut written).unwrap();
876 sealed += 1;
877 }
878 assert!(written == expected, "batch {batch}");
879 assert_eq!(Stamp::of(&written).unwrap(), *section.stamp());
880 drop(legacy);
881 drop(index);
882 drop(store);
883 image = expected;
884 if batch % 25 == 0 || batch + 1 == batches {
885 assert!(section.image() == image);
886 let store = Store::parse(&image).unwrap();
887 let index = RevisionIndex::parse(&store).unwrap();
888 index.validate_current().unwrap();
889 let document = Document::parse(&index).unwrap();
890 for space in &spaces {
891 assert_eq!(
892 section.page(*space).unwrap(),
893 Page::from_space(&document, *space).unwrap()
894 );
895 }
896 }
897 });
791898 }
792899 GUIDS.set(None);
793900 assert!(sealed * 3 > batches, "{sealed} of {batches} batches sealed");
......@@ -932,14 +1039,7 @@ mod tests {
9321039 legacy.write(&bump, &[], changes).unwrap();
9331040 let expected = seeded(guids, || {
9341041 let revision = &legacy.live.revision;
935 crate::page::write::squash(
936 &index,
937 &[(space, revision)],
938 &[],
939 &BTreeMap::new(),
940 None,
941 )
942 .unwrap()
1042 squashed(&index, &[(space, revision)], &[])
9431043 });
9441044 drop(legacy);
9451045 drop(index);
......@@ -971,14 +1071,7 @@ mod tests {
9711071 let legacy = ActivePage::parse(&index, space).unwrap();
9721072 let expected = seeded(seed, || {
9731073 let revision = &legacy.live.revision;
974 crate::page::write::squash(
975 &index,
976 &[(space, revision)],
977 &[payload],
978 &BTreeMap::new(),
979 None,
980 )
981 .unwrap()
1074 squashed(&index, &[(space, revision)], &[payload])
9821075 });
9831076 let transaction = seeded(seed, || section.seal().unwrap());
9841077 assert_eq!(transaction.is_some(), expected != image);
......@@ -1036,8 +1129,12 @@ mod tests {
10361129 && page.parents.contains_key(id)
10371130 })
10381131 .find_map(|(outline, _)| {
1132 let paragraph = crate::page::text::new_id().ok()?;
1133 let text = crate::page::text::new_id().ok()?;
10391134 Insertion::paragraph(*outline, None, "Tampered text", "Author")
1040 .and_then(|insertion| insertion.changes(page))
1135 .and_then(|insertion| {
1136 insertion.changes_as(page, paragraph, paragraph, text)
1137 })
10411138 .ok()
10421139 })
10431140 .unwrap();
......@@ -1127,8 +1224,7 @@ mod probe {
11271224 (space, texts[texts.len() / 2], texts.len())
11281225 }
11291226
1130 /// One-character edits of a large page through the image writer and through a
1131 /// `Section`: `SECTION_PROBE=path cargo test --release -p onestore --lib probe -- --ignored --nocapture`.
1227 /// One-character edits of a large page through a `Section`: `SECTION_PROBE=path cargo test --release -p onestore --lib probe -- --ignored --nocapture`.
11321228 #[test]
11331229 #[ignore]
11341230 fn keystrokes() {
......@@ -1140,57 +1236,6 @@ mod probe {
11401236 image.len()
11411237 );
11421238
1143 let mut samples = Vec::new();
1144 let mut written = 0;
1145 for _ in 0..9 {
1146 let start = Instant::now();
1147 let edit = crate::PreparedEdit::text(&image, space, text, 0..0, "x").unwrap();
1148 samples.push(start.elapsed());
1149 let transaction = edit.transaction();
1150 written = transaction.append.len()
1151 + transaction
1152 .patches
1153 .iter()
1154 .map(|(_, bytes)| bytes.len())
1155 .sum::<usize>();
1156 }
1157 println!(
1158 "image writer, one keystroke: {:?}, {written} bytes written",
1159 median(samples)
1160 );
1161
1162 let before = {
1163 let store = Store::parse(&image).unwrap();
1164 let index = RevisionIndex::parse(&store).unwrap();
1165 Page::from_space(&crate::document::Document::parse(&index).unwrap(), space).unwrap()
1166 };
1167 let mut after = before.clone();
1168 let edited = after
1169 .objects
1170 .iter_mut()
1171 .filter_map(|object| match object {
1172 crate::page::PageObject::Outline(outline) => Some(outline),
1173 _ => None,
1174 })
1175 .flat_map(|outline| &mut outline.paragraphs)
1176 .find_map(|paragraph| paragraph.text_mut().filter(|object| object.id == text))
1177 .unwrap();
1178 let format = edited.text.format_at(0).unwrap().clone();
1179 edited
1180 .text
1181 .apply(crate::page::text::Edit {
1182 range: 0..0,
1183 replacement: crate::page::Paragraph::new("x".into(), format),
1184 })
1185 .unwrap();
1186 let mut samples = Vec::new();
1187 for _ in 0..9 {
1188 let start = Instant::now();
1189 crate::PreparedEdit::page(&image, space, &after, "Probe").unwrap();
1190 samples.push(start.elapsed());
1191 }
1192 println!("page-model writer, one keystroke: {:?}", median(samples));
1193
11941239 let arena = Arena::default();
11951240 let start = Instant::now();
11961241 let mut section = Section::open(&arena, image.clone()).unwrap();
crates/onestore/src/toc.rs+32-37
......@@ -1,14 +1,15 @@
11//! Edits to a notebook's table of contents: the documented `jcidPersistablePropertyContainerForTOC`
2//! root lists section and section-group entries, each carrying a file identity, an ordering
3//! number, a filename and (for sections) a colour.
2//! root carries the notebook's colour and lists section and section-group entries, each
3//! carrying a file identity, an ordering number and a filename. A section's colour lives in
4//! its own metadata (`op::SectionOp::Color`); its entry keeps 0xffffffff, as OneNote writes.
45
56use crate::{
6 Error, ExGuid, PropertySets, Store, Value,
7 Error, ExGuid, PropertySets, Store, Transaction, Value,
78 document::{Document, Kind},
89 revisions::RevisionIndex,
9 write::{PropertyObject, fresh_guid, write_revision},
10 write::{PropertyObject, RevisionEdit, applied, build_on, check, fresh_guid},
1011};
11use std::collections::BTreeMap;
12use std::collections::{BTreeMap, BTreeSet};
1213
1314fn invalid(message: &'static str) -> Error {
1415 Error { offset: 0, message }
......@@ -27,11 +28,8 @@ pub enum TocEdit {
2728 identity: [u8; 16],
2829 filename: String,
2930 },
30 /// Section colour as COLORREF; `None` restores OneNote's "undefined" 0xffffffff.
31 Color {
32 identity: [u8; 16],
33 color: Option<u32>,
34 },
31 /// The notebook's colour as COLORREF.
32 Color(u32),
3533 /// Every entry, in the wanted order; entries left out keep their relative order after these.
3634 Order(Vec<[u8; 16]>),
3735 Remove {
......@@ -43,15 +41,19 @@ fn component(name: &str) -> bool {
4341 !name.is_empty() && !name.contains(['/', '\\', '\0']) && name != "." && name != ".."
4442}
4543
46/// The table of contents with `edits` applied in order.
47type PropertyChange = (u32, Option<Vec<u8>>);
48
49pub(crate) fn edit_table_of_contents(source: &[u8], edits: &[TocEdit]) -> Result<Vec<u8>, Error> {
44/// Applies table-of-contents edits (sections and section groups: add, rename, order,
45/// remove; the notebook's colour) in order as one revision of a `.onetoc2` file; none when they change
46/// nothing.
47pub fn edit_table_of_contents(
48 source: &[u8],
49 edits: &[TocEdit],
50) -> Result<Option<Transaction>, Error> {
5051 let store = Store::parse(source)?;
5152 if store.header.file_type != crate::FileType::TableOfContents {
5253 return Err(invalid("Choose a table-of-contents file"));
5354 }
5455 let index = RevisionIndex::parse(&store)?;
56 index.validate_current()?;
5557 let document = Document::parse(&index)?;
5658 let space = document.root;
5759 let revision = document.active(space)?;
......@@ -76,8 +78,8 @@ pub(crate) fn edit_table_of_contents(source: &[u8], edits: &[TocEdit]) -> Result
7678 listed.push((*id, *identity, filename.clone().unwrap_or_default()));
7779 }
7880 let mut created: BTreeMap<ExGuid, PropertyObject> = BTreeMap::new();
79 // Property values to set (or remove, `None`) on each existing entry.
80 let mut changes: BTreeMap<ExGuid, Vec<PropertyChange>> = BTreeMap::new();
81 let mut renamed = BTreeSet::new();
82 let mut color = None;
8183 for edit in edits {
8284 let position = |identity: &[u8; 16]| {
8385 listed
......@@ -143,18 +145,9 @@ pub(crate) fn edit_table_of_contents(source: &[u8], edits: &[TocEdit]) -> Result
143145 }
144146 let id = listed[at].0;
145147 listed[at].2 = filename.clone();
146 changes
147 .entry(id)
148 .or_default()
149 .push((0x1c001d6b, Some(crate::create::string(filename))));
150 }
151 TocEdit::Color { identity, color } => {
152 let id = listed[position(identity)?].0;
153 changes.entry(id).or_default().push((
154 0x14001cbe,
155 Some(color.unwrap_or(0xffff_ffff).to_le_bytes().to_vec()),
156 ));
148 renamed.insert(id);
157149 }
150 TocEdit::Color(value) => color = Some(*value),
158151 TocEdit::Order(wanted) => {
159152 let mut ordered: Vec<(ExGuid, [u8; 16], String)> = Vec::new();
160153 for identity in wanted {
......@@ -175,17 +168,17 @@ pub(crate) fn edit_table_of_contents(source: &[u8], edits: &[TocEdit]) -> Result
175168 TocEdit::Remove { identity } => {
176169 let at = position(identity)?;
177170 let (id, _, _) = listed.remove(at);
178 changes.remove(&id);
179171 created.remove(&id);
180172 }
181173 }
182174 }
183175 let listed = listed;
184 write_revision(source, space, |raw| {
176 let transaction = build_on(&index, &[], None, |index| {
177 let raw = index.resolve_active(space)?;
185178 let mut changed = BTreeMap::new();
186179 let mut root_object = PropertyObject::from_object(&raw.objects[&root])?;
187180 let mut references = Vec::new();
188 for (order, (id, _, _)) in listed.iter().enumerate() {
181 for (order, (id, _, filename)) in listed.iter().enumerate() {
189182 let mut object = match created.remove(id) {
190183 Some(object) => object,
191184 None => PropertyObject::from_object(&raw.objects[id])?,
......@@ -201,11 +194,8 @@ pub(crate) fn edit_table_of_contents(source: &[u8], edits: &[TocEdit]) -> Result
201194 if stored_order != Some(order as u32 + 1) {
202195 updates.push((0x14001cb9, (order as u32 + 1).to_le_bytes().to_vec()));
203196 }
204 for (property, value) in changes.get(id).into_iter().flatten() {
205 match value {
206 Some(bytes) => updates.push((*property, bytes.clone())),
207 None => object.remove(&[*property])?,
208 }
197 if renamed.contains(id) {
198 updates.push((0x1c001d6b, crate::create::string(filename)));
209199 }
210200 let updates: Vec<(u32, &[u8])> = updates
211201 .iter()
......@@ -220,7 +210,12 @@ pub(crate) fn edit_table_of_contents(source: &[u8], edits: &[TocEdit]) -> Result
220210 }
221211 }
222212 root_object.set(&[(0x24001cf6, &references)])?;
213 if let Some(color) = color {
214 root_object.set(&[(0x14001cbe, &color.to_le_bytes())])?;
215 }
223216 changed.insert(root, root_object);
224 Ok(changed)
225 })
217 Ok(BTreeMap::from([(space, RevisionEdit::Update(changed))]))
218 })?;
219 check(&applied(source, transaction.as_ref())?, true)?;
220 Ok(transaction)
226221}
crates/onestore/src/tree.rs+66-167
......@@ -1,12 +1,11 @@
11use crate::{
22 Error, ExGuid,
33 active::{ActivePage, Changes},
4 create::{current_timestamps, properties, string},
4 create::{current_timestamps, properties},
55 document::{Kind, Revision},
66 edit::{editable_parents, update_title},
77 write::{PropertyObject, fresh_guid},
88};
9use serde::{Deserialize, Serialize};
109use std::{
1110 collections::{BTreeMap, BTreeSet},
1211 sync::Arc,
......@@ -16,8 +15,7 @@ fn invalid(message: &'static str) -> Error {
1615 Error { offset: 0, message }
1716}
1817
19#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
20#[serde(deny_unknown_fields)]
18#[derive(Debug, Clone, PartialEq)]
2119enum Placement {
2220 Delete,
2321 Move {
......@@ -27,40 +25,25 @@ enum Placement {
2725}
2826
2927/// An atomic subtree move or deletion on one active page.
30/// Retain the intent across retries so an emptied cell's replacement keeps its identity.
31#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
32#[serde(deny_unknown_fields)]
33pub struct TreeEdit {
28#[derive(Debug, Clone, PartialEq)]
29pub(crate) struct TreeEdit {
3430 guid: [u8; 16],
3531 object: ExGuid,
3632 placement: Placement,
3733 author: String,
38 created: u32,
3934}
4035
4136impl TreeEdit {
42 pub fn object(&self) -> ExGuid {
43 self.object
44 }
45
46 /// Move destination and direct sibling anchor; deletion has no destination.
47 pub fn destination(&self) -> Option<(ExGuid, Option<ExGuid>)> {
48 match self.placement {
49 Placement::Delete => None,
50 Placement::Move { parent, before } => Some((parent, before)),
51 }
52 }
53
5437 /// Removes a paragraph or ordinary outline and its descendants from the active tree.
55 /// Historical objects remain available; an emptied table cell receives an empty paragraph.
56 pub fn delete(object: ExGuid, author: &str) -> Result<Self, Error> {
38 /// Historical objects remain available.
39 pub(crate) fn delete(object: ExGuid, author: &str) -> Result<Self, Error> {
5740 Self::new(object, Placement::Delete, author)
5841 }
5942
6043 /// Moves before a direct child, or appends when `before` is None.
6144 /// Paragraph destinations are outlines, groups, paragraphs or cells on the same page.
6245 /// Outlines remain direct page children. Content and explicit list formatting are preserved.
63 pub fn move_to(
46 pub(crate) fn move_to(
6447 object: ExGuid,
6548 parent: ExGuid,
6649 before: Option<ExGuid>,
......@@ -75,7 +58,6 @@ impl TreeEdit {
7558 object,
7659 placement,
7760 author: author.to_owned(),
78 created: current_timestamps()?.0,
7961 };
8062 intent.validate()?;
8163 Ok(intent)
......@@ -95,11 +77,6 @@ impl TreeEdit {
9577 Ok(())
9678 }
9779
98 pub(crate) fn apply(&self, source: &[u8], space: ExGuid) -> Result<Vec<u8>, Error> {
99 self.validate()?;
100 crate::active::write(source, space, |active| self.changes(active))
101 }
102
10380 pub(crate) fn changes(&self, active: &ActivePage<'_>) -> Result<Changes, Error> {
10481 let pages = &active.pages;
10582 let [page] = pages.as_slice() else {
......@@ -164,7 +141,6 @@ impl TreeEdit {
164141 .retain(|id| *id != self.object);
165142 }
166143
167 let mut empty_cell = None;
168144 let mut at = source_parent;
169145 loop {
170146 let node = &view.nodes[&at];
......@@ -174,8 +150,8 @@ impl TreeEdit {
174150 break;
175151 }
176152 if node.children.is_empty() {
153 // `Section::apply` refuses an edit that ends with the cell still empty.
177154 if matches!(node.kind, Kind::Cell { .. }) {
178 empty_cell = Some(at);
179155 break;
180156 }
181157 if !matches!(node.kind, Kind::Outline { .. } | Kind::OutlineGroup) {
......@@ -278,53 +254,14 @@ impl TreeEdit {
278254 guid: self.guid,
279255 n: 3,
280256 };
281 if let Some(cell) = empty_cell {
282 let paragraph = ExGuid {
283 guid: self.guid,
284 n: 1,
285 };
286 let text = ExGuid {
287 guid: self.guid,
288 n: 2,
289 };
290 view.nodes.get_mut(&cell).unwrap().children.push(paragraph);
291 for (id, jcid, values) in [
292 (
293 paragraph,
294 0x6000d,
295 vec![
296 (0x24001c1f, 2_u32.to_le_bytes().to_vec()),
297 (0x0c001c03, vec![1]),
298 (0x14001d09, self.created.to_le_bytes().to_vec()),
299 (0x14001d7a, modified.to_vec()),
300 (0x20001d78, 3_u32.to_le_bytes().to_vec()),
301 (0x20001d79, 3_u32.to_le_bytes().to_vec()),
302 ],
303 ),
304 (
305 text,
306 0x6000e,
307 vec![(0x1c001c22, Vec::new()), (0x14001d7a, modified.to_vec())],
308 ),
309 ] {
310 changed.insert(
311 id,
312 PropertyObject {
313 jcid,
314 bytes: properties(&values)?,
315 global_ids: Arc::new(BTreeMap::from([(0, self.guid)])),
316 },
317 );
318 }
319 }
320257 let moved_paragraph = matches!(self.placement, Placement::Move { .. })
321258 && matches!(view.nodes[&self.object].kind, Kind::Paragraph { .. });
322 if empty_cell.is_some() || moved_paragraph {
259 if moved_paragraph {
323260 changed.insert(
324261 author_id,
325262 PropertyObject {
326263 jcid: 0x120001,
327 bytes: properties(&[(0x1c001d75, string(&self.author))])?,
264 bytes: properties(&crate::create::author_properties(&self.author))?,
328265 global_ids: Arc::new(BTreeMap::from([(0, self.guid)])),
329266 },
330267 );
......@@ -413,47 +350,15 @@ fn checked_path(
413350#[cfg(test)]
414351mod tests {
415352 use super::*;
416 use crate::write::write_revision;
417 use crate::{Insertion, PreparedEdit, RevisionIndex, Store, document::Document};
418
419 #[test]
420 fn a_retained_move_intent_can_reuse_its_immutable_author_after_another_move() {
421 let source = crate::create_section("move.one", "First", "Author").unwrap();
422 let store = Store::parse(&source).unwrap();
423 let index = RevisionIndex::parse(&store).unwrap();
424 let document = Document::parse(&index).unwrap();
425 let (sid, page) = document.pages().unwrap()[0];
426 let view = document.active(sid).unwrap();
427 let outline = *view.nodes[&page]
428 .children
429 .iter()
430 .find(|id| matches!(view.nodes[id].kind, Kind::Outline { .. }))
431 .unwrap();
432 let first = view.nodes[&outline].children[0];
433 let second = Insertion::paragraph(outline, None, "Second", "Author").unwrap();
434 let source = PreparedEdit::insert(&source, sid, &second)
435 .unwrap()
436 .as_bytes()
437 .to_vec();
438 let retained = TreeEdit::move_to(first, outline, None, "Tree author").unwrap();
439 let moved = PreparedEdit::tree(&source, sid, &retained).unwrap();
440 let reversed =
441 TreeEdit::move_to(first, outline, Some(second.object()), "Tree author").unwrap();
442 let reversed = PreparedEdit::tree(moved.as_bytes(), sid, &reversed).unwrap();
443 let repeated = PreparedEdit::tree(reversed.as_bytes(), sid, &retained).unwrap();
444 let store = Store::parse(repeated.as_bytes()).unwrap();
445 let index = RevisionIndex::parse(&store).unwrap();
446 let document = Document::parse(&index).unwrap();
447 let view = document.active(sid).unwrap();
448 assert_eq!(view.nodes[&outline].children, [second.object(), first]);
449 assert_eq!(
450 view.nodes[&first].latest_author,
451 Some(ExGuid {
452 guid: retained.guid,
453 n: 3
454 })
455 );
456 }
353 use crate::{
354 RevisionIndex, Store,
355 document::{Document, Format},
356 op::{
357 PageOp,
358 tests::{edited, text_paragraph},
359 },
360 write::write_revision,
361 };
457362
458363 #[test]
459364 fn group_normalization_preserves_unequal_indentation_and_overlapping_moves() {
......@@ -469,22 +374,23 @@ mod tests {
469374 .find(|id| matches!(view.nodes[id].kind, Kind::Outline { .. }))
470375 .unwrap();
471376 let first = view.nodes[&outline].children[0];
472 let second = Insertion::paragraph(outline, None, "Second", "Author").unwrap();
473 let third = Insertion::paragraph(outline, None, "Third", "Author").unwrap();
474 let source = PreparedEdit::insert(&source, sid, &second)
475 .unwrap()
476 .as_bytes()
477 .to_vec();
478 let source = PreparedEdit::insert(&source, sid, &third)
479 .unwrap()
480 .as_bytes()
481 .to_vec();
377 let paragraphs = vec![
378 text_paragraph("Second", Format::default()),
379 text_paragraph("Third", Format::default()),
380 ];
381 let (second, third) = (paragraphs[0].id, paragraphs[1].id);
382 let insert = PageOp::Insert {
383 container: outline,
384 before: None,
385 paragraphs,
386 };
387 let source = edited(&source, sid, vec![insert]).unwrap();
482388 let guid = fresh_guid().unwrap();
483389 let a = ExGuid { guid, n: 1 };
484390 let b = ExGuid { guid, n: 2 };
485391 let source = write_revision(&source, sid, |raw| {
486392 let mut changed = BTreeMap::new();
487 for (id, child, level) in [(a, first, 2), (b, second.object(), 1)] {
393 for (id, child, level) in [(a, first, 2), (b, second, 1)] {
488394 let mut group = PropertyObject {
489395 jcid: 0x60019,
490396 bytes: properties(&[(0x0c001c03, vec![level])])?,
......@@ -496,7 +402,7 @@ mod tests {
496402 }
497403 let mut object = PropertyObject::from_object(&raw.objects[&outline])?;
498404 let mut references = Vec::new();
499 for id in [a, b, third.object()] {
405 for id in [a, b, third] {
500406 references.extend_from_slice(&object.reference(id)?);
501407 }
502408 object.set(&[(0x24001c20, &references)])?;
......@@ -504,38 +410,37 @@ mod tests {
504410 Ok(changed)
505411 })
506412 .unwrap();
507 for moving in [false, true] {
508 let intent = if moving {
509 TreeEdit::move_to(third.object(), a, Some(first), "Author")
510 } else {
511 TreeEdit::delete(third.object(), "Author")
512 }
513 .unwrap();
514 let edit = PreparedEdit::tree(&source, sid, &intent).unwrap();
515 let store = Store::parse(edit.as_bytes()).unwrap();
413 let page = |image: &[u8]| {
414 let store = Store::parse(image).unwrap();
516415 let index = RevisionIndex::parse(&store).unwrap();
517416 index.validate_current().unwrap();
518 let document = Document::parse(&index).unwrap();
519 let view = document.active(sid).unwrap();
520 assert_eq!(view.nodes[&outline].children, [a, second.object()]);
521 assert_eq!(view.nodes[&outline].child_level, Some(2));
522 assert_eq!(view.nodes[&a].child_level, Some(1));
523 assert_eq!(
524 view.nodes[&a].children,
525 if moving {
526 vec![third.object(), first]
527 } else {
528 vec![first]
529 }
530 );
531 if !moving && let Some(output) = std::env::var_os("ONESTORE_TREE_OUTPUT") {
532 let output = std::path::PathBuf::from(output).join("unequal-groups");
533 assert!(output.is_absolute());
534 std::fs::create_dir_all(output.parent().unwrap()).unwrap();
535 std::fs::create_dir(&output).unwrap();
536 std::fs::write(output.join("groups.one"), edit.as_bytes()).unwrap();
537 }
417 crate::page::Page::from_space(&Document::parse(&index).unwrap(), sid).unwrap()
418 };
419 let written = edited(&source, sid, vec![PageOp::Delete { object: third }]).unwrap();
420 let store = Store::parse(&written).unwrap();
421 let index = RevisionIndex::parse(&store).unwrap();
422 let document = Document::parse(&index).unwrap();
423 let view = document.active(sid).unwrap();
424 assert_eq!(view.nodes[&outline].children, [a, second]);
425 assert_eq!(view.nodes[&outline].child_level, Some(2));
426 assert_eq!(view.nodes[&a].child_level, Some(1));
427 assert_eq!(view.nodes[&a].children, [first]);
428 if let Some(output) = std::env::var_os("ONESTORE_TREE_OUTPUT") {
429 let output = std::path::PathBuf::from(output).join("unequal-groups");
430 assert!(output.is_absolute());
431 std::fs::create_dir_all(output.parent().unwrap()).unwrap();
432 std::fs::create_dir(&output).unwrap();
433 std::fs::write(output.join("groups.one"), &written).unwrap();
538434 }
435 // A move keeps the paragraph's level; the writers regroup the outline around it.
436 let moved = PageOp::Move {
437 object: third,
438 parent: Some(outline),
439 before: Some(first),
440 };
441 let mut predicted = page(&source);
442 crate::op::predict(&mut predicted, &moved).unwrap();
443 assert_eq!(page(&edited(&source, sid, vec![moved]).unwrap()), predicted);
539444 }
540445
541446 #[test]
......@@ -553,7 +458,9 @@ mod tests {
553458 .unwrap();
554459 let paragraph = view.nodes[&outline].children[0];
555460 let text = view.nodes[&paragraph].content[0];
556 let deletion = TreeEdit::delete(paragraph, "Author").unwrap();
461 let deleted = |bytes: &[u8]| {
462 edited(bytes, sid, vec![PageOp::Delete { object: paragraph }]).is_ok()
463 };
557464 for target in [page, outline, paragraph, text] {
558465 for property in [0x08001cde, 0x08001cb4, 0x08001cf9, 0x08001cb2] {
559466 for enabled in [false, true] {
......@@ -563,11 +470,7 @@ mod tests {
563470 Ok(BTreeMap::from([(target, object)]))
564471 })
565472 .unwrap();
566 assert_eq!(
567 PreparedEdit::tree(&bytes, sid, &deletion).is_err(),
568 enabled,
569 "{target}: {property:#x}"
570 );
473 assert_eq!(deleted(&bytes), !enabled, "{target}: {property:#x}");
571474 }
572475 }
573476 }
......@@ -578,7 +481,7 @@ mod tests {
578481 Ok(BTreeMap::from([(protected, object)]))
579482 })
580483 .unwrap();
581 assert!(PreparedEdit::tree(&bytes, sid, &deletion).is_err());
484 assert!(!deleted(&bytes));
582485 }
583486 let bytes = write_revision(&source, sid, |raw| {
584487 let mut object = PropertyObject::from_object(&raw.objects[&outline])?;
......@@ -587,10 +490,6 @@ mod tests {
587490 Ok(BTreeMap::from([(outline, object)]))
588491 })
589492 .unwrap();
590 assert!(PreparedEdit::tree(&bytes, sid, &deletion).is_err());
591 let mut value = serde_json::to_value(&deletion).unwrap();
592 value["guid"] = serde_json::to_value([0_u8; 16]).unwrap();
593 let restored = serde_json::from_value(value).unwrap();
594 assert!(PreparedEdit::tree(&source, sid, &restored).is_err());
493 assert!(!deleted(&bytes));
595494 }
596495}
crates/onestore/src/write.rs+108-73
......@@ -323,45 +323,6 @@ pub(crate) struct ObjectEdit<'a> {
323323 pub inserts: &'a [(u32, &'a [u8])],
324324}
325325
326/// Replaces a root-level scalar byte property in the default active revision.
327/// The caller supplies its encoded value and is responsible for MS-ONE semantics.
328/// Returns a complete file image without I/O; use `commit_file_property` to update an existing file.
329pub fn replace_property_bytes(
330 source: &[u8],
331 space: ExGuid,
332 object_id: ExGuid,
333 property: u32,
334 value: &[u8],
335) -> Result<Vec<u8>> {
336 if !(3..=7).contains(&((property >> 26) & 31)) {
337 return Err(Error {
338 offset: 0,
339 message: "Property does not contain scalar bytes",
340 });
341 }
342 let store = Store::parse(source)?;
343 let index = RevisionIndex::parse(&store)?;
344 index.validate_current()?;
345 replace_objects(
346 &index,
347 space,
348 &[ObjectEdit {
349 object: object_id,
350 updates: &[(property, value)],
351 inserts: &[],
352 }],
353 )
354}
355
356/// Patches objects of a source the caller has parsed and validated.
357pub(crate) fn replace_objects(
358 index: &RevisionIndex<'_>,
359 space: ExGuid,
360 edits: &[ObjectEdit<'_>],
361) -> Result<Vec<u8>> {
362 write_revision_on(index, space, |revision| patched(revision, edits))
363}
364
365326/// Objects of `revision` with `edits` applied.
366327pub(crate) fn patched(
367328 revision: &crate::ResolvedRevision<'_>,
......@@ -511,6 +472,7 @@ pub(crate) struct PropertyObject {
511472
512473pub(crate) enum RevisionEdit {
513474 Update(BTreeMap<ExGuid, PropertyObject>),
475 #[cfg(any(test, feature = "protected"))]
514476 Create {
515477 roots: BTreeMap<u32, ExGuid>,
516478 objects: BTreeMap<ExGuid, PropertyObject>,
......@@ -1269,24 +1231,18 @@ pub(crate) fn chain_depth(index: &RevisionIndex<'_>, space: ExGuid, rid: ExGuid)
12691231 .count()
12701232}
12711233
1234/// A test fixture: `source` with one revision of `space` storing `edit`'s objects.
1235#[cfg(test)]
12721236pub(crate) fn write_revision(
12731237 source: &[u8],
12741238 space: ExGuid,
12751239 edit: impl FnOnce(&crate::ResolvedRevision<'_>) -> Result<BTreeMap<ExGuid, PropertyObject>>,
12761240) -> Result<Vec<u8>> {
1277 write_revision_with_payloads(source, space, &[], edit)
1278}
1279
1280pub(crate) fn write_revision_with_payloads(
1281 source: &[u8],
1282 space: ExGuid,
1283 payloads: &[([u8; 16], &[u8])],
1284 edit: impl FnOnce(&crate::ResolvedRevision<'_>) -> Result<BTreeMap<ExGuid, PropertyObject>>,
1285) -> Result<Vec<u8>> {
1286 write_revisions_with_payloads(source, payloads, update(space, edit))
1241 write_revisions(source, update(space, edit))
12871242}
12881243
12891244/// One space's active revision edited into its update.
1245#[cfg(test)]
12901246fn update(
12911247 space: ExGuid,
12921248 edit: impl FnOnce(&crate::ResolvedRevision<'_>) -> Result<BTreeMap<ExGuid, PropertyObject>>,
......@@ -1300,26 +1256,30 @@ fn update(
13001256 }
13011257}
13021258
1303pub(crate) fn write_revisions(
1259/// A test fixture: the transaction appending `edit`'s revisions to `source`.
1260#[cfg(test)]
1261pub(crate) fn revisions(
13041262 source: &[u8],
13051263 edit: impl FnOnce(&RevisionIndex<'_>) -> Result<BTreeMap<ExGuid, RevisionEdit>>,
1306) -> Result<Vec<u8>> {
1307 write_revisions_with_payloads(source, &[], edit)
1264) -> Result<Option<crate::Transaction>> {
1265 let store = Store::parse(source)?;
1266 let index = RevisionIndex::parse(&store)?;
1267 index.validate_current()?;
1268 build_on(&index, &[], None, edit)
13081269}
13091270
1310/// `write_revisions` that also stores embedded payloads: each becomes a file-data store
1311/// object referenced from the root file node list under its identity, as OneNote embeds
1312/// pictures and attachments.
1313pub(crate) fn write_revisions_with_payloads(
1271/// A test fixture: `source` with `edit`'s revisions appended.
1272#[cfg(test)]
1273pub(crate) fn write_revisions(
13141274 source: &[u8],
1315 payloads: &[([u8; 16], &[u8])],
13161275 edit: impl FnOnce(&RevisionIndex<'_>) -> Result<BTreeMap<ExGuid, RevisionEdit>>,
13171276) -> Result<Vec<u8>> {
1318 publish(source, payloads, None, true, edit)
1277 publish(source, &[], None, true, edit)
13191278}
13201279
1321/// `write_revisions_with_payloads` without validating that current revisions are
1322/// complete: a protected section is validated by unlocking it, through `protection`.
1280/// `source` with `edit`'s revisions and the `payloads` appended, without validating that
1281/// current revisions are complete: a protected section is validated by unlocking it,
1282/// through `protection`.
13231283#[cfg(feature = "protected")]
13241284pub(crate) fn append_revisions(
13251285 source: &[u8],
......@@ -1330,6 +1290,7 @@ pub(crate) fn append_revisions(
13301290 publish(source, payloads, protection, false, edit)
13311291}
13321292
1293#[cfg(any(test, feature = "protected"))]
13331294fn publish(
13341295 source: &[u8],
13351296 payloads: &[([u8; 16], &[u8])],
......@@ -1355,7 +1316,8 @@ pub(crate) fn check(output: &[u8], validate: bool) -> Result<()> {
13551316
13561317/// The written image, unchecked: `check` follows once the caller has released whatever
13571318/// `edit` borrowed.
1358pub(crate) fn build(
1319#[cfg(any(test, feature = "protected"))]
1320fn build(
13591321 source: &[u8],
13601322 payloads: &[([u8; 16], &[u8])],
13611323 protection: Option<&dyn Protection>,
......@@ -1367,16 +1329,20 @@ pub(crate) fn build(
13671329 if validate {
13681330 index.validate_current()?;
13691331 }
1370 build_on(&index, payloads, protection, edit)
1332 applied(source, build_on(&index, payloads, protection, edit)?.as_ref())
13711333}
13721334
13731335/// `write_revision` on a source the caller has parsed and validated.
1336#[cfg(test)]
13741337pub(crate) fn write_revision_on(
13751338 index: &RevisionIndex<'_>,
13761339 space: ExGuid,
13771340 edit: impl FnOnce(&crate::ResolvedRevision<'_>) -> Result<BTreeMap<ExGuid, PropertyObject>>,
13781341) -> Result<Vec<u8>> {
1379 let output = build_on(index, &[], None, update(space, edit))?;
1342 let output = applied(
1343 index.store.data,
1344 build_on(index, &[], None, update(space, edit))?.as_ref(),
1345 )?;
13801346 check(&output, true)?;
13811347 Ok(output)
13821348}
......@@ -1387,12 +1353,14 @@ thread_local! {
13871353 pub(crate) static BUILDS: std::cell::Cell<usize> = const { std::cell::Cell::new(0) };
13881354}
13891355
1356/// The transaction appending `edit`'s revisions and the `payloads` to the parsed store;
1357/// none when nothing changes.
13901358pub(crate) fn build_on(
13911359 index: &RevisionIndex<'_>,
13921360 payloads: &[([u8; 16], &[u8])],
13931361 protection: Option<&dyn Protection>,
13941362 edit: impl FnOnce(&RevisionIndex<'_>) -> Result<BTreeMap<ExGuid, RevisionEdit>>,
1395) -> Result<Vec<u8>> {
1363) -> Result<Option<crate::Transaction>> {
13961364 #[cfg(test)]
13971365 BUILDS.with(|builds| builds.set(builds.get() + 1));
13981366 let store = index.store;
......@@ -1411,13 +1379,13 @@ pub(crate) fn build_on(
14111379 let changes = edit(index)?;
14121380 let mut appending = Appending::new(store.state()?);
14131381 for (space, change) in changes {
1414 let new_space = matches!(change, RevisionEdit::Create { .. });
14151382 let current = (ExGuid::default(), 1_u32);
1416 let (rid, label, revision, replacements) = match change {
1383 let (rid, label, revision, replacements, new_space) = match change {
14171384 RevisionEdit::Update(objects) => {
14181385 let rid = index.active(space)?;
1419 (Some(rid), current, resolve(space, rid)?, objects)
1386 (Some(rid), current, resolve(space, rid)?, objects, false)
14201387 }
1388 #[cfg(any(test, feature = "protected"))]
14211389 RevisionEdit::Create { roots, objects } => {
14221390 if !is_section || space.guid == [0; 16] || index.spaces.contains_key(&space) {
14231391 return Err(Error {
......@@ -1439,6 +1407,7 @@ pub(crate) fn build_on(
14391407 objects: BTreeMap::new(),
14401408 },
14411409 objects,
1410 true,
14421411 )
14431412 }
14441413 #[cfg(feature = "protected")]
......@@ -1462,6 +1431,7 @@ pub(crate) fn build_on(
14621431 objects: BTreeMap::new(),
14631432 },
14641433 objects,
1434 false,
14651435 )
14661436 }
14671437 };
......@@ -1525,13 +1495,78 @@ pub(crate) fn build_on(
15251495 )?;
15261496 }
15271497 appending.payloads(payloads, protection)?;
1528 Ok(match appending.finish()? {
1529 Some((transaction, _)) => {
1530 let mut output = source.to_vec();
1531 transaction.apply(&mut output)?;
1532 output
1498 Ok(appending.finish()?.map(|(transaction, _)| transaction))
1499}
1500
1501/// `source` with `transaction` applied.
1502pub(crate) fn applied(source: &[u8], transaction: Option<&crate::Transaction>) -> Result<Vec<u8>> {
1503 let mut output = source.to_vec();
1504 if let Some(transaction) = transaction {
1505 transaction.apply(&mut output)?;
1506 }
1507 Ok(output)
1508}
1509
1510/// Payload identities the file-data store of `store` declares, in order.
1511#[cfg(any(test, feature = "protected"))]
1512pub(crate) fn declared_payloads(store: &Store<'_>) -> Vec<[u8; 16]> {
1513 store
1514 .lists
1515 .values()
1516 .flat_map(|list| &list.nodes)
1517 .filter(|node| node.id == 0x94)
1518 .filter_map(|node| node.payload.get(..16).and_then(|g| g.try_into().ok()))
1519 .collect()
1520}
1521
1522/// Writes the live objects of each `edited` revision that differ from its space's active
1523/// revision in the validated `source` as one revision per space, embedding the `payloads`
1524/// it lacks. A protected `source` takes the revisions its plaintext twin gained.
1525#[cfg(any(test, feature = "protected"))]
1526pub(crate) fn squash(
1527 source: &RevisionIndex<'_>,
1528 edited: &[(ExGuid, &crate::ResolvedRevision<'_>)],
1529 payloads: &[([u8; 16], &[u8])],
1530 protection: Option<&dyn Protection>,
1531) -> Result<Option<crate::Transaction>> {
1532 let existing = declared_payloads(source.store);
1533 let payloads: Vec<_> = payloads
1534 .iter()
1535 .filter(|(guid, _)| !existing.contains(guid))
1536 .copied()
1537 .collect();
1538 build_on(source, &payloads, protection, |index| {
1539 let mut changes = BTreeMap::new();
1540 for (space, after) in edited {
1541 let before = match protection {
1542 Some(protection) => protection.resolve(*space, index.active(*space)?)?,
1543 None => index.resolve_active(*space)?,
1544 };
1545 if before.roots != after.roots {
1546 return Err(Error {
1547 offset: 0,
1548 message: "Page edits cannot change revision roots",
1549 });
1550 }
1551 // Retired styles and deleted content stay in history; only live objects are written.
1552 let live = after.reachable()?;
1553 let mut changed = BTreeMap::new();
1554 for (id, object) in &after.objects {
1555 if !live.contains(id)
1556 || before
1557 .objects
1558 .get(id)
1559 .map(|previous| unchanged(previous, object))
1560 .transpose()?
1561 == Some(true)
1562 {
1563 continue;
1564 }
1565 changed.insert(*id, replacement(*id, object)?);
1566 }
1567 changes.insert(*space, RevisionEdit::Update(changed));
15331568 }
1534 None => source.to_vec(),
1569 Ok(changes)
15351570 })
15361571}
15371572
crates/onestore/src/write/tests.rs+95-55
......@@ -72,7 +72,15 @@ fn copying_reference_arrays_remaps_every_entry_in_each_stream() {
7272#[test]
7373fn document_insertions_and_formatting_respect_readonly_ancestors() {
7474 use super::{PropertyObject, write_revision};
75 use crate::{ExGuid, Insertion, PreparedEdit, RevisionIndex, Store};
75 use crate::{
76 ExGuid, RevisionIndex, Store, TextAttribute,
77 document::{Format, Layout},
78 op::{
79 PageOp,
80 tests::{edited, text_paragraph},
81 },
82 page::{Outline, PageObject, text::new_id},
83 };
7684 use std::collections::BTreeMap;
7785 let source = crate::create_section("readonly.one", "Original", "Author").unwrap();
7886 let store = Store::parse(&source).unwrap();
......@@ -97,9 +105,23 @@ fn document_insertions_and_formatting_respect_readonly_ancestors() {
97105 ))
98106 })
99107 .unwrap();
100 let right = Insertion::paragraph(outline, None, "Right", "Author").unwrap();
101 let expanded = PreparedEdit::insert(&source, sid, &right).unwrap();
102 let source = expanded.as_bytes();
108 let right = text_paragraph("Right", Format::default());
109 let right_text = right.text().unwrap().id;
110 let insert = |container, paragraph| PageOp::Insert {
111 container,
112 before: None,
113 paragraphs: vec![paragraph],
114 };
115 let expanded = edited(&source, sid, vec![insert(outline, right.clone())]).unwrap();
116 let source = expanded.as_slice();
117 let raw = index
118 .resolve(sid, index.spaces[&sid].labels[&(ExGuid::default(), 1)])
119 .unwrap();
120 let text = raw
121 .objects
122 .iter()
123 .find_map(|(id, object)| (object.jcid == 0x6000e).then_some(*id))
124 .unwrap();
103125 for blocked in [page, outline, paragraph] {
104126 let protected = write_revision(source, sid, |raw| {
105127 let mut object = PropertyObject::from_object(&raw.objects[&blocked])?;
......@@ -107,47 +129,55 @@ fn document_insertions_and_formatting_respect_readonly_ancestors() {
107129 Ok(BTreeMap::from([(blocked, object)]))
108130 })
109131 .unwrap();
110 let child = Insertion::paragraph(paragraph, None, "Nested", "Author").unwrap();
111 assert!(PreparedEdit::insert(&protected, sid, &child).is_err());
112 let raw = index
113 .resolve(sid, index.spaces[&sid].labels[&(ExGuid::default(), 1)])
114 .unwrap();
115 let text = raw
116 .objects
117 .iter()
118 .find_map(|(id, object)| (object.jcid == 0x6000e).then_some(*id))
119 .unwrap();
120 let split = crate::ParagraphSplit::new(text, 1, "Author").unwrap();
121 assert!(PreparedEdit::split(&protected, sid, &split).is_err());
122 let join = crate::ParagraphJoin::new(text, right.text_object(), "Author").unwrap();
123 assert!(PreparedEdit::join(&protected, sid, &join).is_err());
124 assert!(
125 PreparedEdit::format(
126 &protected,
127 sid,
128 text,
129 1..3,
130 &[crate::TextAttribute::Bold(true)]
131 )
132 .is_err()
133 );
132 let refused = |op| edited(&protected, sid, vec![op]).is_err();
133 assert!(refused(insert(paragraph, text_paragraph("Nested", Format::default()))));
134 assert!(refused(PageOp::Split {
135 text,
136 at: 1,
137 paragraph: new_id().unwrap(),
138 right: new_id().unwrap(),
139 lists: Vec::new(),
140 }));
141 assert!(refused(PageOp::Join {
142 left: text,
143 right: right_text,
144 }));
145 assert!(refused(PageOp::Format {
146 text,
147 range: 1..3,
148 set: vec![TextAttribute::Bold(true)],
149 clear: Vec::new(),
150 }));
134151 if blocked == page {
135 let outline = Insertion::outline(page, 36.0, 36.0, "Outline", "Author").unwrap();
136 assert!(PreparedEdit::insert(&protected, sid, &outline).is_err());
152 assert!(refused(PageOp::Add {
153 object: PageObject::Outline(Outline {
154 id: new_id().unwrap(),
155 title: false,
156 min_width: None,
157 layout: Layout {
158 x: Some(36.0),
159 y: Some(36.0),
160 ..Default::default()
161 },
162 indents: Vec::new(),
163 paragraphs: vec![text_paragraph("Outline", Format::default())],
164 unsupported: Vec::new(),
165 }),
166 before: None,
167 }));
137168 }
138169 }
139170 let protected = write_revision(source, sid, |raw| {
140 let mut object = PropertyObject::from_object(&raw.objects[&right.object()])?;
171 let mut object = PropertyObject::from_object(&raw.objects[&right.id])?;
141172 object.set(&[(0x88001cde, &[])])?;
142 Ok(BTreeMap::from([(right.object(), object)]))
173 Ok(BTreeMap::from([(right.id, object)]))
143174 })
144175 .unwrap();
145 let document = crate::document::Document::parse(&index).unwrap();
146 let space = &document.spaces[&sid];
147 let view = space.active().unwrap();
148 let text = view.nodes[&paragraph].content[0];
149 let join = crate::ParagraphJoin::new(text, right.text_object(), "Author").unwrap();
150 assert!(PreparedEdit::join(&protected, sid, &join).is_err());
176 let join = PageOp::Join {
177 left: text,
178 right: right_text,
179 };
180 assert!(edited(&protected, sid, vec![join]).is_err());
151181}
152182
153183fn add_paragraph(source: &[u8], number: u32) -> Vec<u8> {
......@@ -774,8 +804,9 @@ fn invalid_property_splices_are_rejected() {
774804fn character_formatting_preserves_inheritance_and_associated_data() {
775805 use super::{PropertyObject, write_revision};
776806 use crate::{
777 ExGuid, PreparedEdit, RevisionIndex, Store, TextAttribute,
807 ExGuid, RevisionIndex, Store, TextAttribute,
778808 document::{Document, Kind},
809 op::{PageOp, tests::edited},
779810 };
780811 use std::collections::BTreeMap;
781812 let source = crate::create_section("inherited.one", "abcdef", "Author").unwrap();
......@@ -858,13 +889,29 @@ fn character_formatting_preserves_inheritance_and_associated_data() {
858889 Ok(BTreeMap::from([(text, target)]))
859890 })
860891 .unwrap();
892 let text_edit = |range, with: &str| {
893 let with = with.to_owned();
894 edited(&fixture, sid, vec![PageOp::Text { text, range, with }])
895 };
896 let bold = |range| {
897 edited(
898 &fixture,
899 sid,
900 vec![PageOp::Format {
901 text,
902 range,
903 set: vec![TextAttribute::Bold(true)],
904 clear: Vec::new(),
905 }],
906 )
907 };
861908 if variant == 9 {
862 assert!(PreparedEdit::text(&fixture, sid, text, 0..0, "x").is_err());
909 assert!(text_edit(0..0, "x").is_err());
863910 continue;
864911 }
865912 if variant >= 7 {
866 let edit = PreparedEdit::text(&fixture, sid, text, 1..3, "longer").unwrap();
867 let [before, after] = [&fixture, edit.as_bytes()].map(|bytes| {
913 let edit = text_edit(1..3, "longer").unwrap();
914 let [before, after] = [&fixture, &edit].map(|bytes| {
868915 let store = Store::parse(bytes).unwrap();
869916 let index = RevisionIndex::parse(&store).unwrap();
870917 let document = Document::parse(&index).unwrap();
......@@ -877,24 +924,20 @@ fn character_formatting_preserves_inheritance_and_associated_data() {
877924 format!("{:?}", view.nodes[&text].extra)
878925 });
879926 assert_eq!(before, after);
880 assert!(PreparedEdit::text(&fixture, sid, text, 0..4, "x").is_err());
881 assert_eq!(
882 PreparedEdit::text(&fixture, sid, text, 3..5, "x").is_ok(),
883 variant == 7
884 );
927 assert!(text_edit(0..4, "x").is_err());
928 assert_eq!(text_edit(3..5, "x").is_ok(), variant == 7);
885929 continue;
886930 }
887931 // Hidden and hyperlink runs are ordinary text with flags; associated data,
888932 // equations and embedded objects are not.
889 let text_edit = PreparedEdit::text(&fixture, sid, text, 1..5, "x");
890 assert_eq!(text_edit.is_ok(), matches!(variant, 0 | 1 | 4));
891 let edit = PreparedEdit::format(&fixture, sid, text, 0..6, &[TextAttribute::Bold(true)]);
933 assert_eq!(text_edit(1..5, "x").is_ok(), matches!(variant, 0 | 1 | 4));
934 let edit = bold(0..6);
892935 if matches!(variant, 2 | 5 | 6) {
893936 assert!(edit.is_err());
894937 continue;
895938 }
896939 let edit = edit.unwrap();
897 let store = Store::parse(edit.as_bytes()).unwrap();
940 let store = Store::parse(&edit).unwrap();
898941 let index = RevisionIndex::parse(&store).unwrap();
899942 let doc = Document::parse(&index).unwrap();
900943 let s = &doc.spaces[&sid];
......@@ -911,10 +954,7 @@ fn character_formatting_preserves_inheritance_and_associated_data() {
911954 let data = &view.nodes[&text].extra[runs[0].extra_set.unwrap()];
912955 assert_eq!(data.len(), 1);
913956 assert_eq!(data[0].id, 0x14001234);
914 assert!(
915 PreparedEdit::format(&fixture, sid, text, 1..3, &[TextAttribute::Bold(true)])
916 .is_err()
917 );
957 assert!(bold(1..3).is_err());
918958 let previous = Store::parse(&fixture).unwrap();
919959 let previous = RevisionIndex::parse(&previous).unwrap();
920960 let previous_doc = Document::parse(&previous).unwrap();
crates/onestore/src/write/tests/multi.rs+15-19
......@@ -1,7 +1,8 @@
11use crate::{
22 ExGuid, RevisionIndex, Store,
33 document::{Document, Kind},
4 write::{PropertyObject, RevisionEdit, write_revisions},
4 Transaction,
5 write::{PropertyObject, RevisionEdit, applied, revisions, write_revisions},
56};
67use std::collections::BTreeMap;
78
......@@ -18,8 +19,8 @@ mod current {
1819const SOURCE: &[u8] =
1920 include_bytes!("../../../../../corpus/page-lifecycle/03-renamed/notebook/Lifecycle.one");
2021
21fn nest(source: &[u8]) -> Vec<u8> {
22 write_revisions(source, |index| {
22fn nest(source: &[u8]) -> Option<Transaction> {
23 revisions(source, |index| {
2324 let document = Document::parse(index)?;
2425 let pages = document.pages()?;
2526 assert_eq!(pages.len(), 9);
......@@ -100,7 +101,7 @@ fn nest(source: &[u8]) -> Vec<u8> {
100101
101102#[test]
102103fn nesting_publishes_section_order_and_page_levels_in_one_transaction() {
103 let written = nest(SOURCE);
104 let written = applied(SOURCE, nest(SOURCE).as_ref()).unwrap();
104105 let before = Store::parse(SOURCE).unwrap();
105106 let after = Store::parse(&written).unwrap();
106107 assert_eq!(
......@@ -160,7 +161,7 @@ fn nesting_publishes_section_order_and_page_levels_in_one_transaction() {
160161 .collect::<Vec<_>>(),
161162 [1, 1, 1, 3, 1, 1, 1]
162163 );
163 assert_eq!(nest(&written), written);
164 assert_eq!(nest(&written), None);
164165 if let Some(output) = std::env::var_os("ONESTORE_PAGE_BATCH_OUTPUT") {
165166 std::fs::create_dir(&output).unwrap();
166167 std::fs::write(
......@@ -173,7 +174,8 @@ fn nesting_publishes_section_order_and_page_levels_in_one_transaction() {
173174
174175#[test]
175176fn interrupted_multi_space_publication_never_exposes_a_partial_nesting() {
176 let written = nest(SOURCE);
177 let transaction = nest(SOURCE).unwrap();
178 let written = applied(SOURCE, Some(&transaction)).unwrap();
177179 let old = current::current(SOURCE);
178180 let new = current::current(&written);
179181 assert_ne!(old, new);
......@@ -186,9 +188,7 @@ fn interrupted_multi_space_publication_never_exposes_a_partial_nesting() {
186188 write_limit,
187189 random: 1951,
188190 };
189 crate::commit::Transaction::between(SOURCE, &written)
190 .commit(&mut complete)
191 .unwrap();
191 transaction.commit(&mut complete).unwrap();
192192 assert_eq!(complete.durable, written);
193193 for fail_at in 1..=complete.operation {
194194 let mut interrupted = disk::Disk {
......@@ -199,9 +199,7 @@ fn interrupted_multi_space_publication_never_exposes_a_partial_nesting() {
199199 write_limit,
200200 random: 1951 + fail_at as u64,
201201 };
202 crate::commit::Transaction::between(SOURCE, &written)
203 .commit(&mut interrupted)
204 .unwrap_err();
202 transaction.commit(&mut interrupted).unwrap_err();
205203 let observed = current::current(&interrupted.durable);
206204 assert!(
207205 observed == old || observed == new,
......@@ -234,7 +232,7 @@ fn repeated_multi_space_edits_cross_counter_carries_and_checkpoint_each_space()
234232 let initial_index = RevisionIndex::parse(&initial_store).unwrap();
235233 let mut checkpoints = BTreeMap::new();
236234 for step in 1_u32..=514 {
237 let written = write_revisions(&source, |index| {
235 let transaction = revisions(&source, |index| {
238236 let mut changes = BTreeMap::new();
239237 for (sid, space) in &index.spaces {
240238 let raw = index.resolve(*sid, space.labels[&(ExGuid::default(), 1)])?;
......@@ -259,7 +257,9 @@ fn repeated_multi_space_edits_cross_counter_carries_and_checkpoint_each_space()
259257 .map(|(sid, objects)| (sid, RevisionEdit::Update(objects)))
260258 .collect())
261259 })
260 .unwrap()
262261 .unwrap();
262 let written = applied(&source, Some(&transaction)).unwrap();
263263 let store = Store::parse(&written).unwrap();
264264 assert_eq!(
265265 store.header.transaction_count,
......@@ -290,9 +290,7 @@ fn repeated_multi_space_edits_cross_counter_carries_and_checkpoint_each_space()
290290 write_limit: 4096,
291291 random: 1952,
292292 };
293 crate::commit::Transaction::between(&source, &written)
294 .commit(&mut complete)
295 .unwrap();
293 transaction.commit(&mut complete).unwrap();
296294 for fail_at in 1..=complete.operation {
297295 let mut interrupted = disk::Disk {
298296 visible: source.clone(),
......@@ -302,9 +300,7 @@ fn repeated_multi_space_edits_cross_counter_carries_and_checkpoint_each_space()
302300 write_limit: 4096,
303301 random: 1952 + fail_at as u64,
304302 };
305 crate::commit::Transaction::between(&source, &written)
306 .commit(&mut interrupted)
307 .unwrap_err();
303 transaction.commit(&mut interrupted).unwrap_err();
308304 let observed = current::current(&interrupted.durable);
309305 assert!(observed == old || observed == new, "{step}:{fail_at}");
310306 }
crates/onestore/tests/commit.rs+87-137
......@@ -2,104 +2,94 @@
22mod disk;
33use disk::Disk;
44use onestore::{
5 ExGuid, ObjectData, PropertySets, RevisionIndex, Store, Value, replace_property_bytes,
5 Arena, ExGuid, RevisionIndex, Section, Store, Transaction,
6 document::{Document, Kind},
7 op::{Edit, Op, PageOp},
68};
79use std::fs;
810
911#[test]
10fn incomplete_native_save_is_rejected_before_storage_io() {
12fn incomplete_native_save_is_rejected_before_an_edit() {
1113 let source = fs::read("../../corpus/malformed/native-inflight.one").unwrap();
1214 let store = Store::parse(&source).unwrap();
15 let graph_error = RevisionIndex::parse(&store)
16 .unwrap()
17 .validate_current()
18 .unwrap_err();
19 let arena = Arena::default();
20 let error = Section::open(&arena, source.clone()).err().unwrap();
21 assert_eq!(error.to_string(), graph_error.to_string());
22}
23
24/// The text object holding `text` and its page space.
25fn target(source: &[u8], text: &str) -> (ExGuid, ExGuid) {
26 let store = Store::parse(source).unwrap();
1327 let index = RevisionIndex::parse(&store).unwrap();
14 let graph_error = index.validate_current().unwrap_err();
15 let mut checked = 0;
16 for (osid, space) in &index.spaces {
17 let revision = index
18 .resolve(*osid, space.labels[&(ExGuid::default(), 1)])
19 .unwrap();
20 for oid in revision.reachable().unwrap() {
21 let object = &revision.objects[&oid];
22 if object.jcid != 0x6000e {
23 continue;
24 }
25 let ObjectData::Properties(data) = object.data else {
26 continue;
27 };
28 for property in &PropertySets::parse(data).unwrap().sets[0] {
29 if property.value != Value::Bytes(b"Rust same paragraph.") {
30 continue;
31 }
32 let mut disk = Disk {
33 visible: source.clone(),
34 durable: source.clone(),
35 operation: 0,
36 fail_at: None,
37 write_limit: 1024,
38 random: 1,
39 };
40 let error = onestore::commit_property_bytes(
41 &mut disk,
42 &source,
43 *osid,
44 oid,
45 property.id,
46 b"Rust same paragraph.",
47 )
48 .unwrap_err();
49 assert_eq!(error.state, onestore::CommitState::NotCommitted);
50 assert_eq!(error.error.to_string(), graph_error.to_string());
51 assert_eq!(disk.operation, 0);
52 checked += 1;
53 }
54 }
55 }
56 assert_eq!(checked, 1);
28 let document = Document::parse(&index).unwrap();
29 document
30 .spaces
31 .iter()
32 .find_map(|(space, view)| {
33 view.active()?.nodes.iter().find_map(|(id, node)| {
34 matches!(&node.kind, Kind::RichText { text: stored, .. } if stored == text)
35 .then_some((*space, *id))
36 })
37 })
38 .unwrap()
39}
40
41/// The text `object` holds in `image`, which must be valid.
42fn text_of(image: &[u8], space: ExGuid, object: ExGuid) -> String {
43 let store = Store::parse(image).unwrap();
44 assert!(store.checksum_mismatches.is_empty());
45 let index = RevisionIndex::parse(&store).unwrap();
46 index.validate_current().unwrap();
47 let document = Document::parse(&index).unwrap();
48 let Kind::RichText { text, .. } = &document.spaces[&space].active().unwrap().nodes[&object].kind
49 else {
50 panic!()
51 };
52 text.clone()
5753}
5854
5955#[test]
6056fn repeated_appends_cross_log_fragments_and_expose_counter_tears() {
61 let mut source =
57 let initial =
6258 fs::read("../../corpus/native/20260905-05/snapshots/02-text/notebook/synthetic.one")
6359 .unwrap();
64 let store = Store::parse(&source).unwrap();
65 let index = RevisionIndex::parse(&store).unwrap();
66 let mut target = None;
67 for (osid, space) in &index.spaces {
68 let revision = index
69 .resolve(*osid, space.labels[&(ExGuid::default(), 1)])
70 .unwrap();
71 for (oid, object) in revision.objects {
72 if let ObjectData::Properties(data) = object.data {
73 for property in &PropertySets::parse(data).unwrap().sets[0] {
74 if property.value == Value::Bytes(b"Fictitious plain text.") {
75 target = Some((*osid, oid, property.id));
76 }
77 }
78 }
79 }
80 }
81 let (osid, oid, property) = target.unwrap();
60 let store = Store::parse(&initial).unwrap();
61 let (space, object) = target(&initial, "Fictitious plain text.");
8262 let root = store.header.root;
8363 let log = store.header.transaction_log;
8464 let initial_count = store.header.transaction_count;
65 let arena = Arena::default();
66 let mut section = Section::open(&arena, initial.clone()).unwrap();
67 let mut source = initial.clone();
68 let mut length = "Fictitious plain text.".encode_utf16().count() as u32;
8569 let mut saw_counter_tear = false;
86 for transaction in initial_count + 1..=258 {
87 let value = format!("Transaction {transaction}");
88 let written =
89 replace_property_bytes(&source, osid, oid, property, value.as_bytes()).unwrap();
70 for count in initial_count + 1..=258 {
71 let value = format!("Transaction {count}");
72 let op = PageOp::Text {
73 text: object,
74 range: 0..length,
75 with: value.clone(),
76 };
77 length = value.encode_utf16().count() as u32;
78 section
79 .apply("Author", &Edit { at: 133_700_000_000_000_000 + u64::from(count), ops: vec![Op::Page { space, op }] })
80 .unwrap();
81 let transaction = section.seal().unwrap().unwrap();
82 let mut written = source.clone();
83 transaction.apply(&mut written).unwrap();
9084 let store = Store::parse(&written).unwrap();
9185 assert_eq!(store.header.root, root);
9286 assert_eq!(store.header.transaction_log, log);
93 assert_eq!(store.header.transaction_count, transaction);
94 assert!(store.checksum_mismatches.is_empty());
95 RevisionIndex::parse(&store)
96 .unwrap()
97 .validate_current()
98 .unwrap();
87 assert_eq!(store.header.transaction_count, count);
88 assert_eq!(text_of(&written, space, object), value);
9989 let mut previous = written.clone();
10090 previous[..1024].copy_from_slice(&source[..1024]);
10191 let old = Store::parse(&previous).unwrap();
102 assert_eq!(old.header.transaction_count, transaction - 1);
92 assert_eq!(old.header.transaction_count, count - 1);
10393 assert!(old.checksum_mismatches.is_empty());
10494 RevisionIndex::parse(&old)
10595 .unwrap()
......@@ -110,17 +100,17 @@ fn repeated_appends_cross_log_fragments_and_expose_counter_tears() {
110100 torn[prefix..1024].copy_from_slice(&source[prefix..1024]);
111101 let result = Store::parse(&torn)
112102 .and_then(|store| RevisionIndex::parse(&store)?.validate_current());
113 if transaction == 256 && prefix == 97 {
103 if count == 256 && prefix == 97 {
114104 assert!(result.is_err());
115105 saw_counter_tear = true;
116106 } else {
117107 result.unwrap();
118108 }
119109 }
120 if transaction == initial_count + 1 || transaction == 256 {
121 check_crashes(&source, osid, oid, property, value.as_bytes());
110 if count == initial_count + 1 || count == 256 {
111 check_crashes(&source, &transaction, space, object, &value);
122112 }
123 if transaction == initial_count + 1 {
113 if count == initial_count + 1 {
124114 let path = std::env::temp_dir().join(format!(
125115 "onestore-commit-{}-{}.one",
126116 std::process::id(),
......@@ -130,61 +120,29 @@ fn repeated_appends_cross_log_fragments_and_expose_counter_tears() {
130120 .as_nanos()
131121 ));
132122 fs::write(&path, &source).unwrap();
133 onestore::commit_file_property(&path, &source, osid, oid, property, value.as_bytes())
134 .unwrap();
123 transaction.commit_file(&path).unwrap();
135124 let committed = fs::read(&path).unwrap();
136 let parsed = Store::parse(&committed).unwrap();
137 assert_eq!(parsed.header.transaction_count, transaction);
138 RevisionIndex::parse(&parsed)
139 .unwrap()
140 .validate_current()
141 .unwrap();
142 let stale =
143 onestore::commit_file_property(&path, &source, osid, oid, property, b"stale")
144 .unwrap_err();
125 assert_eq!(committed, written);
126 // The file moved past the transaction's base: nothing is written again.
127 let stale = transaction.commit_file(&path).unwrap_err();
145128 assert_eq!(stale.state, onestore::CommitState::NotCommitted);
146129 assert_eq!(fs::read(&path).unwrap(), committed);
147 onestore::commit_file_property(
148 &path,
149 &committed,
150 osid,
151 oid,
152 property,
153 value.as_bytes(),
154 )
155 .unwrap();
156 assert_eq!(fs::read(&path).unwrap(), committed);
157130 fs::remove_file(path).unwrap();
158131 }
159132 source = written;
160133 }
161134 assert!(saw_counter_tear);
135 assert!(section.image() == source);
162136}
163137
164fn check_crashes(source: &[u8], osid: ExGuid, oid: ExGuid, property: u32, value: &[u8]) {
165 let current_value = |data: &[u8]| {
166 let store = Store::parse(data).unwrap();
167 assert!(store.checksum_mismatches.is_empty());
168 let index = RevisionIndex::parse(&store).unwrap();
169 index.validate_current().unwrap();
170 let revision = index
171 .resolve(osid, index.spaces[&osid].labels[&(ExGuid::default(), 1)])
172 .unwrap();
173 let ObjectData::Properties(blob) = revision.objects[&oid].data else {
174 panic!()
175 };
176 let properties = PropertySets::parse(blob).unwrap();
177 let Value::Bytes(value) = properties.sets[0]
178 .iter()
179 .find(|p| p.id == property)
180 .unwrap()
181 .value
182 else {
183 panic!()
184 };
185 value.to_vec()
186 };
187 let before = current_value(source);
138fn check_crashes(
139 source: &[u8],
140 transaction: &Transaction,
141 space: ExGuid,
142 object: ExGuid,
143 value: &str,
144) {
145 let before = text_of(source, space, object);
188146 for limit in [17, 1024] {
189147 let disk = |fail_at, random| Disk {
190148 visible: source.to_vec(),
......@@ -195,22 +153,13 @@ fn check_crashes(source: &[u8], osid: ExGuid, oid: ExGuid, property: u32, value:
195153 random,
196154 };
197155 let mut successful = disk(None, 1);
198 onestore::commit_property_bytes(&mut successful, source, osid, oid, property, value)
199 .unwrap();
200 assert_eq!(current_value(&successful.durable), value);
156 transaction.commit(&mut successful).unwrap();
157 assert_eq!(text_of(&successful.durable, space, object), value);
201158 for at in 1..=successful.operation {
202159 for seed in [0, 1, 42, u64::MAX] {
203160 let mut interrupted = disk(Some(at), seed);
204 let failure = onestore::commit_property_bytes(
205 &mut interrupted,
206 source,
207 osid,
208 oid,
209 property,
210 value,
211 )
212 .unwrap_err();
213 let persisted = current_value(&interrupted.durable);
161 let failure = transaction.commit(&mut interrupted).unwrap_err();
162 let persisted = text_of(&interrupted.durable, space, object);
214163 match failure.state {
215164 onestore::CommitState::NotCommitted => assert_eq!(persisted, before),
216165 onestore::CommitState::Committed => assert_eq!(persisted, value),
......@@ -222,6 +171,7 @@ fn check_crashes(source: &[u8], osid: ExGuid, oid: ExGuid, property: u32, value:
222171 }
223172 }
224173}
174
225175#[test]
226176#[cfg(any(unix, windows))]
227177fn bounded_file_reads_reject_partial_images_and_release_the_owner() {
crates/onestore/tests/conflict.rs created+357
......@@ -0,0 +1,357 @@
1use onestore::{
2 Arena, ConflictPage, ExGuid, ObjectData, PageCreation, PropertySets, RevisionIndex, Section,
3 Store, Value,
4 op::{Edit, Op, PageOp, SectionOp},
5 page::{Page, PageObject},
6};
7use std::collections::BTreeMap;
8
9const AT: u64 = 134_030_000_000_000_000;
10
11fn read(path: &str) -> Vec<u8> {
12 std::fs::read(format!("{}/../../{path}", env!("CARGO_MANIFEST_DIR"))).unwrap()
13}
14
15/// Each object's jcid and root-set property identities, by space and object.
16fn objects(image: &[u8]) -> BTreeMap<ExGuid, BTreeMap<ExGuid, (u32, Vec<u32>)>> {
17 let store = Store::parse(image).unwrap();
18 let index = RevisionIndex::parse(&store).unwrap();
19 index
20 .spaces
21 .keys()
22 .map(|space| {
23 let revision = index.resolve_active(*space).unwrap();
24 let objects = revision
25 .objects
26 .iter()
27 .map(|(id, object)| {
28 let ids = match object.data {
29 ObjectData::Properties(bytes) => PropertySets::parse(bytes).unwrap().sets
30 [0]
31 .iter()
32 .map(|property| property.id)
33 .collect(),
34 _ => Vec::new(),
35 };
36 (*id, (object.jcid, ids))
37 })
38 .collect();
39 (*space, objects)
40 })
41 .collect()
42}
43
44/// The object-space references a property holds.
45fn spaces(image: &[u8], space: ExGuid, object: ExGuid, property: u32) -> usize {
46 let store = Store::parse(image).unwrap();
47 let index = RevisionIndex::parse(&store).unwrap();
48 let revision = index.resolve_active(space).unwrap();
49 let ObjectData::Properties(bytes) = revision.objects[&object].data else {
50 panic!()
51 };
52 PropertySets::parse(bytes).unwrap().sets[0]
53 .iter()
54 .find(|field| field.id == property)
55 .map_or(0, |field| match &field.value {
56 Value::References { compact_ids, .. } => compact_ids.len() / 4,
57 _ => panic!(),
58 })
59}
60
61fn texts(page: &Page) -> Vec<(ExGuid, String)> {
62 page.objects
63 .iter()
64 .flat_map(|object| match object {
65 PageObject::Outline(outline) => outline.paragraphs.clone(),
66 PageObject::Title(title) => title
67 .outlines
68 .iter()
69 .flat_map(|outline| outline.paragraphs.clone())
70 .collect(),
71 _ => Vec::new(),
72 })
73 .filter_map(|paragraph| {
74 paragraph
75 .text()
76 .map(|text| (text.id, text.text.text().to_owned()))
77 })
78 .collect()
79}
80
81#[test]
82fn native_conflict_pages_are_listed_under_their_page_last_first() {
83 let arena = Arena::default();
84 let mut section = Section::open(
85 &arena,
86 read("corpus/collaboration/round-01/offline/notebook/synthetic.one"),
87 )
88 .unwrap();
89 let pages = section.pages().unwrap();
90 let conflicts = section.conflicts().unwrap();
91 assert_eq!(conflicts.len(), 1);
92 let (page, listed) = &conflicts[0];
93 assert_eq!(*page, pages[0].0);
94 let [ConflictPage {
95 space,
96 title,
97 user,
98 created,
99 objects,
100 }] = listed.as_slice()
101 else {
102 panic!("{listed:?}")
103 };
104 assert_eq!(
105 (title.as_str(), user.as_str(), *created),
106 (
107 "Native edited while disconnected.",
108 "snow",
109 Some(0x01dd3d10d7afd9e0)
110 )
111 );
112 let conflict = section.page(*space).unwrap();
113 let marked: Vec<_> = texts(&conflict)
114 .into_iter()
115 .filter(|(id, _)| objects.contains(id))
116 .map(|(_, text)| text)
117 .collect();
118 assert_eq!(marked, ["Native edited while disconnected."]);
119
120 // Three clients' conflict pages: OneNote lists the one stored last first.
121 let arena = Arena::default();
122 let mut section = Section::open(
123 &arena,
124 read("corpus/conflict-page/native-three/notebook/synthetic.one"),
125 )
126 .unwrap();
127 let users: Vec<_> = section.conflicts().unwrap()[0]
128 .1
129 .iter()
130 .map(|page| page.user.clone())
131 .collect();
132 assert_eq!(
133 users,
134 ["ONE-M6-053BD357", "ONE-M6-9451B6C6", "ONE-M6-DDE10E0D"]
135 );
136}
137
138/// The conflict page a merge makes of `local`, the version of `space`'s page it could not
139/// take, marking the text object `conflicting`.
140fn conflict(space: ExGuid, local: &Page, conflicting: ExGuid, user: &str) -> Op {
141 let titled = local
142 .objects
143 .iter()
144 .any(|object| matches!(object, PageObject::Title(_)));
145 let mut objects = [conflicting];
146 let page = local.copy_with(&mut objects).unwrap();
147 Op::Section(SectionOp::Conflict {
148 of: space,
149 creation: PageCreation::new(None, titled.then_some(local.title.as_str()), user).unwrap(),
150 page,
151 objects: objects.to_vec(),
152 })
153}
154
155/// A conflict page written through ops stores what OneNote 2010 writes for one
156/// (`corpus/collaboration/round-01/offline`): its own space under the page's manifest, with
157/// conflict metadata, a read-only deletable conflict page and the conflicting text marked.
158/// `ONESTORE_CONFLICT_EXPORT` names a directory to write the notebook to, for a cold read.
159#[test]
160fn a_conflict_page_is_stored_as_onenote_stores_one() {
161 let notebook = "corpus/native-ink/cold-ui-ink/notebook";
162 let source = read(&format!("{notebook}/synthetic.one"));
163 let arena = Arena::default();
164 let mut section = Section::open(&arena, source.clone()).unwrap();
165 let space = section.pages().unwrap()[0].0;
166 let base = section.page(space).unwrap();
167 let (text, _) = texts(&base)
168 .into_iter()
169 .find(|(_, text)| text.starts_with("Fictitious: café"))
170 .unwrap();
171 // Both machines renamed the paragraph's first word, within its Latin run.
172 let replace = |with: &str| Edit {
173 at: AT,
174 ops: vec![Op::Page {
175 space,
176 op: PageOp::Text {
177 text,
178 range: 0..10,
179 with: with.into(),
180 },
181 }],
182 };
183 section.apply("Remote", &replace("Remote")).unwrap();
184 let scratch = Arena::default();
185 let mut local = Section::open(&scratch, source).unwrap();
186 local.apply("Clover Snow", &replace("Snowbound")).unwrap();
187 let local = local.page(space).unwrap();
188 let op = conflict(space, &local, text, "Clover Snow");
189 let Op::Section(SectionOp::Conflict { creation, .. }) = &op else {
190 unreachable!()
191 };
192 let created = creation.space();
193 section
194 .apply(
195 "Clover Snow",
196 &Edit {
197 at: AT + 10_000_000,
198 ops: vec![op.clone()],
199 },
200 )
201 .unwrap();
202 section.seal().unwrap();
203 let image = section.image();
204
205 let arena = Arena::default();
206 let mut reopened = Section::open(&arena, image.clone()).unwrap();
207 assert_eq!(reopened.pages().unwrap().len(), 1);
208 let listed = reopened.conflicts().unwrap();
209 assert_eq!(listed.len(), 1);
210 assert_eq!(listed[0].0, space);
211 assert_eq!(
212 listed[0].1,
213 [ConflictPage {
214 space: created,
215 title: "Snowbound: café, 東京, مرحبا".into(),
216 user: "Clover Snow".into(),
217 created: listed[0].1[0].created,
218 objects: listed[0].1[0].objects.clone(),
219 }]
220 );
221 let kept = reopened.page(created).unwrap();
222 let marked: Vec<_> = texts(&kept)
223 .into_iter()
224 .filter(|(id, _)| listed[0].1[0].objects.contains(id))
225 .map(|(_, text)| text)
226 .collect();
227 assert_eq!(marked, ["Snowbound: café, 東京, مرحبا"]);
228 assert!(
229 texts(&reopened.page(space).unwrap())
230 .iter()
231 .any(|(_, text)| text == "Remote: café, 東京, مرحبا")
232 );
233
234 // The stored form, property by property, as OneNote writes it.
235 let stored = objects(&image);
236 let conflict_space = &stored[&created];
237 let store = Store::parse(&image).unwrap();
238 let index = RevisionIndex::parse(&store).unwrap();
239 let roots = index.resolve_active(created).unwrap().roots;
240 let (jcid, metadata) = &conflict_space[&roots[&2]];
241 assert_eq!(*jcid, 0x20038);
242 assert_eq!(
243 metadata,
244 &[
245 0x1c001cf3, 0x1c001c30, 0x14001dff, 0x14001d82, 0x1400348b, 0x18001c65, 0x1c001d9e,
246 0x1c001d9f
247 ]
248 );
249 let page_node = conflict_space
250 .values()
251 .find(|(jcid, _)| *jcid == 0x6000b)
252 .unwrap();
253 for flag in [0x88001cde, 0x88001d0c, 0x88001d7c] {
254 assert!(page_node.1.contains(&flag), "{flag:#x}");
255 }
256 let marked: Vec<_> = conflict_space
257 .iter()
258 .filter(|(_, (_, ids))| ids.contains(&0x88001d96))
259 .map(|(_, (jcid, ids))| (*jcid, ids.contains(&0x88001ddb)))
260 .collect();
261 assert_eq!(marked, [(0x6000e, true)]);
262 let main = &stored[&space];
263 let main_roots = index.resolve_active(space).unwrap().roots;
264 assert_eq!(spaces(&image, space, main_roots[&1], 0x2c001d63), 1);
265 assert!(main[&main_roots[&2]].1.contains(&0x88001d97));
266 assert_eq!(
267 main.values()
268 .filter(|(jcid, ids)| *jcid == 0x20038 && ids.contains(&0x1c001d9e))
269 .count(),
270 1
271 );
272 assert_eq!(
273 stored[&index.root]
274 .values()
275 .filter(|(jcid, ids)| *jcid == 0x20030 && ids.contains(&0x88001d97))
276 .count(),
277 1
278 );
279
280 if let Some(output) = std::env::var_os("ONESTORE_CONFLICT_EXPORT") {
281 let output = std::path::Path::new(&output);
282 std::fs::create_dir_all(output).unwrap();
283 std::fs::write(output.join("synthetic.one"), &image).unwrap();
284 std::fs::copy(
285 format!("{}/../../{notebook}/Open Notebook.onetoc2", env!("CARGO_MANIFEST_DIR")),
286 output.join("Open Notebook.onetoc2"),
287 )
288 .unwrap();
289 }
290}
291
292/// Deleting a conflict page stores what OneNote 2010's Delete Conflict Page does
293/// (`corpus/conflict-page/native-delete`): the page's manifest stops listing the conflict
294/// page, keeping its metadata copy, and the page and the section's copy of its metadata no
295/// longer say it has conflict pages; the conflict page's space is left alone.
296/// `ONESTORE_CONFLICT_DELETE_EXPORT` names a directory to write the notebook to.
297#[test]
298fn deleting_a_conflict_page_stores_what_onenote_stores() {
299 let notebook = "corpus/conflict-page/native/conflict/notebook";
300 let arena = Arena::default();
301 let mut section = Section::open(&arena, read(&format!("{notebook}/synthetic.one"))).unwrap();
302 let (page, listed) = section.conflicts().unwrap().remove(0);
303 let conflict = listed[0].space;
304 section
305 .apply(
306 "Clover Snow",
307 &Edit {
308 at: AT,
309 ops: vec![Op::Section(SectionOp::Delete(vec![conflict]))],
310 },
311 )
312 .unwrap();
313 section.seal().unwrap();
314 let image = section.image();
315 let arena = Arena::default();
316 let mut reopened = Section::open(&arena, image.clone()).unwrap();
317 assert!(reopened.conflicts().unwrap().is_empty());
318 assert_eq!(reopened.pages().unwrap().len(), 1);
319
320 // Each space's objects and their property identities, as OneNote left them.
321 let native = read("corpus/conflict-page/native-delete/notebook/synthetic.one");
322 let shapes = |image: &[u8]| {
323 let store = Store::parse(image).unwrap();
324 let index = RevisionIndex::parse(&store).unwrap();
325 let root = index.root;
326 let roots = index.resolve_active(page).unwrap().roots;
327 let objects = objects(image);
328 let manifest = objects[&page][&roots[&1]].clone();
329 let metadata = objects[&page][&roots[&2]].clone();
330 let flagged = objects[&root]
331 .values()
332 .filter(|(_, ids)| ids.contains(&0x88001d97))
333 .count();
334 (manifest, metadata, flagged, objects[&conflict].clone())
335 };
336 let (manifest, metadata, flagged, space) = shapes(&image);
337 let (native_manifest, native_metadata, native_flagged, native_space) = shapes(&native);
338 assert_eq!(manifest.1.iter().filter(|id| **id != 0x3400347b).collect::<Vec<_>>(), [&0x24001c1f, &0x24003442]);
339 assert_eq!(
340 native_manifest.1.iter().filter(|id| **id != 0x3400347b).collect::<Vec<_>>(),
341 [&0x24001c1f, &0x24003442]
342 );
343 assert!(!metadata.1.contains(&0x88001d97) && !native_metadata.1.contains(&0x88001d97));
344 assert_eq!((flagged, native_flagged), (0, 0));
345 assert_eq!(space, native_space);
346
347 if let Some(output) = std::env::var_os("ONESTORE_CONFLICT_DELETE_EXPORT") {
348 let output = std::path::Path::new(&output);
349 std::fs::create_dir_all(output).unwrap();
350 std::fs::write(output.join("synthetic.one"), &image).unwrap();
351 std::fs::copy(
352 format!("{}/../../{notebook}/Open Notebook.onetoc2", env!("CARGO_MANIFEST_DIR")),
353 output.join("Open Notebook.onetoc2"),
354 )
355 .unwrap();
356 }
357}
crates/onestore/tests/document.rs+21-16
......@@ -482,10 +482,6 @@ fn rust_unicode_and_native_latin1_use_utf16_run_offsets() {
482482 };
483483 assert_eq!(actual, text);
484484 assert_eq!(runs[0].end, 7);
485 let bytes = fs::read("../../corpus/m6/ansi/notebook/synthetic.one").unwrap();
486 let store = Store::parse(&bytes).unwrap();
487 let index = RevisionIndex::parse(&store).unwrap();
488 let document = Document::parse(&index).unwrap();
489485 let expected = format!(
490486 "Extended bytes: {}",
491487 (128u8..=255)
......@@ -493,14 +489,23 @@ fn rust_unicode_and_native_latin1_use_utf16_run_offsets() {
493489 .collect::<Vec<_>>()
494490 .join(" ")
495491 );
496 assert!(
497 document
498 .spaces
499 .values()
500 .flat_map(|s| s.revisions.values())
501 .flat_map(|r| r.nodes.values())
502 .any(|n| matches!(&n.kind, Kind::RichText { text, .. } if text == &expected))
503 );
492 // OneNote 2010 reads TextExtendedAscii as Latin-1 whatever the run's language
493 // (corpus/extended-ascii/{en,ru,ja}/cold).
494 for language in ["en", "ru", "ja"] {
495 let path = format!("../../corpus/extended-ascii/{language}/candidate/synthetic.one");
496 let bytes = fs::read(path).unwrap();
497 let store = Store::parse(&bytes).unwrap();
498 let index = RevisionIndex::parse(&store).unwrap();
499 let document = Document::parse(&index).unwrap();
500 assert!(
501 document
502 .spaces
503 .values()
504 .flat_map(|s| s.revisions.values())
505 .flat_map(|r| r.nodes.values())
506 .any(|n| matches!(&n.kind, Kind::RichText { text, .. } if text == &expected))
507 );
508 }
504509}
505510
506511#[test]
......@@ -717,11 +722,11 @@ fn malformed_run_boundaries_fail_after_valid_storage_decoding() {
717722 else {
718723 unreachable!()
719724 };
725 // Damage the stored boundaries where they lie, as no writer would store them.
726 let at = boundaries.as_ptr().addr() - bytes.as_ptr().addr();
720727 for invalid in [5u32, u32::MAX] {
721 let mut changed = boundaries.to_vec();
722 changed[..4].copy_from_slice(&invalid.to_le_bytes());
723 let mutated =
724 onestore::replace_property_bytes(&bytes, sid, oid, 0x1c001e12, &changed).unwrap();
728 let mut mutated = bytes.to_vec();
729 mutated[at..at + 4].copy_from_slice(&invalid.to_le_bytes());
725730 let store = Store::parse(&mutated).unwrap();
726731 let index = RevisionIndex::parse(&store).unwrap();
727732 index.validate_current().unwrap();
crates/onestore/tests/edit.rs+118-58
......@@ -2,14 +2,18 @@
22mod checkpoint;
33#[path = "support/disk.rs"]
44mod disk;
5#[path = "support/ops.rs"]
6mod ops;
57#[path = "support/trace.rs"]
68mod trace;
79
810use disk::Disk;
911use onestore::{
10 CommitState, ExGuid, RevisionIndex, Store,
12 CommitError, CommitIo, CommitState, ExGuid, RevisionIndex, Store, Transaction,
1113 document::{Document, Kind},
14 op::{Op, OpError, PageOp},
1215};
16use std::ops::Range;
1317
1418const SOURCE: &[u8] = include_bytes!(
1519 "../../../corpus/native/20260905-05/snapshots/03-format-unicode/notebook/synthetic.one"
......@@ -31,6 +35,55 @@ fn target(source: &[u8]) -> (ExGuid, ExGuid) {
3135 panic!("Missing native text fixture")
3236}
3337
38fn typed(space: ExGuid, text: ExGuid, range: Range<u32>, with: &str) -> Vec<Op> {
39 let op = PageOp::Text {
40 text,
41 range,
42 with: with.into(),
43 };
44 vec![Op::Page { space, op }]
45}
46
47/// `source` with `range` of the text object replaced.
48fn replace_text(
49 source: &[u8],
50 space: ExGuid,
51 text: ExGuid,
52 range: Range<u32>,
53 with: &str,
54) -> Result<Vec<u8>, OpError> {
55 ops::edited(source, typed(space, text, range, with))
56}
57
58/// The transaction replacing `range` of the text object in `source`.
59fn text_transaction(
60 source: &[u8],
61 space: ExGuid,
62 text: ExGuid,
63 range: Range<u32>,
64 with: &str,
65) -> Result<Transaction, CommitError> {
66 let refused = |error: String| CommitError {
67 state: CommitState::NotCommitted,
68 error: std::io::Error::new(std::io::ErrorKind::InvalidData, error),
69 };
70 ops::transaction(source, "Author", typed(space, text, range, with))
71 .map_err(|error| refused(error.to_string()))?
72 .ok_or_else(|| refused("The edit changes nothing".into()))
73}
74
75/// `text_transaction` committed under caller-held exclusion.
76fn commit_text(
77 io: &mut impl CommitIo,
78 source: &[u8],
79 space: ExGuid,
80 text: ExGuid,
81 range: Range<u32>,
82 with: &str,
83) -> Result<(), CommitError> {
84 text_transaction(source, space, text, range, with)?.commit(io)
85}
86
3487fn text_runs(source: &[u8], sid: ExGuid, oid: ExGuid) -> serde_json::Value {
3588 let store = Store::parse(source).unwrap();
3689 assert!(store.checksum_mismatches.is_empty());
......@@ -61,9 +114,11 @@ fn prepared_publication_preserves_its_identity_through_every_io_failure() {
61114 let source =
62115 onestore::create_section("prepared.one", "Fictitious: café 🦀", "Fixture").unwrap();
63116 let (sid, oid) = target(&source);
64 let source = checkpoint::pending(&source, sid, oid, 0x14001d7a);
65 let edit = onestore::PreparedEdit::text(&source, sid, oid, 0..0, "Prepared 🐈 ").unwrap();
66 let store = Store::parse(edit.as_bytes()).unwrap();
117 let source = checkpoint::pending(&source, sid, oid);
118 let edit = text_transaction(&source, sid, oid, 0..0, "Prepared 🐈 ").unwrap();
119 let mut written = source.clone();
120 edit.apply(&mut written).unwrap();
121 let store = Store::parse(&written).unwrap();
67122 let index = RevisionIndex::parse(&store).unwrap();
68123 let planned = index.spaces[&sid].labels[&(ExGuid::default(), 1)];
69124 let persisted: ExGuid =
......@@ -72,7 +127,7 @@ fn prepared_publication_preserves_its_identity_through_every_io_failure() {
72127 let before_index = RevisionIndex::parse(&before_store).unwrap();
73128 assert!(!before_index.spaces[&sid].revisions.contains_key(&persisted));
74129 let before = text_runs(&source, sid, oid);
75 let after = text_runs(edit.as_bytes(), sid, oid);
130 let after = text_runs(&written, sid, oid);
76131 for write_limit in [17, 4096] {
77132 let disk = |fail_at| Disk {
78133 visible: source.clone(),
......@@ -84,12 +139,12 @@ fn prepared_publication_preserves_its_identity_through_every_io_failure() {
84139 };
85140 let mut success = disk(None);
86141 edit.commit(&mut success).unwrap();
87 assert_eq!(success.durable, edit.as_bytes());
142 assert_eq!(success.durable, written);
88143 let operations = success.operation;
89144 let error = edit.commit(&mut success).unwrap_err();
90145 assert_eq!(error.state, CommitState::NotCommitted);
91146 assert_eq!(error.error.kind(), std::io::ErrorKind::ResourceBusy);
92 assert_eq!(success.durable, edit.as_bytes());
147 assert_eq!(success.durable, written);
93148 for at in 1..=operations {
94149 let mut interrupted = disk(Some(at));
95150 let error = edit.commit(&mut interrupted).unwrap_err();
......@@ -108,7 +163,8 @@ fn prepared_publication_preserves_its_identity_through_every_io_failure() {
108163 interrupted.visible.clone_from(&snapshot);
109164 interrupted.fail_at = None;
110165 interrupted.write_limit = 17;
111 onestore::confirm_snapshot(&mut interrupted, &snapshot).unwrap();
166 onestore::confirm(&mut interrupted, &onestore::Stamp::of(&snapshot).unwrap())
167 .unwrap();
112168 assert_refreshed(&snapshot, &interrupted.durable);
113169 }
114170 }
......@@ -126,9 +182,9 @@ fn snapshot_confirmation_needs_no_surviving_edit_target() {
126182 random: 911,
127183 };
128184 assert!(
129 onestore::replace_text(SOURCE, ExGuid::default(), ExGuid::default(), 0..0, "").is_err()
185 replace_text(SOURCE, ExGuid::default(), ExGuid::default(), 0..0, "").is_err()
130186 );
131 onestore::confirm_snapshot(&mut disk, SOURCE).unwrap();
187 onestore::confirm(&mut disk, &onestore::Stamp::of(SOURCE).unwrap()).unwrap();
132188 assert_refreshed(SOURCE, &disk.durable);
133189 let flush = disk.operation;
134190 for (failure, state) in [
......@@ -140,7 +196,7 @@ fn snapshot_confirmation_needs_no_surviving_edit_target() {
140196 disk.operation = 0;
141197 disk.fail_at = Some(failure);
142198 assert_eq!(
143 onestore::confirm_snapshot(&mut disk, SOURCE)
199 onestore::confirm(&mut disk, &onestore::Stamp::of(SOURCE).unwrap())
144200 .unwrap_err()
145201 .state,
146202 state
......@@ -153,7 +209,7 @@ fn confirmation_notifies_cached_readers_after_interrupted_version_publication()
153209 let source =
154210 onestore::create_section("confirmation.one", "Fictitious: before", "Fixture").unwrap();
155211 let (sid, oid) = target(&source);
156 let mut snapshot = onestore::replace_text(&source, sid, oid, 0..0, "Recovered ").unwrap();
212 let mut snapshot = replace_text(&source, sid, oid, 0..0, "Recovered ").unwrap();
157213 snapshot[212..252].copy_from_slice(&source[212..252]);
158214 let expected = text_runs(&snapshot, sid, oid);
159215 assert_ne!(expected, text_runs(&source, sid, oid));
......@@ -167,11 +223,13 @@ fn confirmation_notifies_cached_readers_after_interrupted_version_publication()
167223 random: 911,
168224 };
169225 let mut success = disk(None);
170 onestore::confirm_snapshot(&mut success, &snapshot).unwrap();
226 onestore::confirm(&mut success, &onestore::Stamp::of(&snapshot).unwrap()).unwrap();
171227 assert_refreshed(&snapshot, &success.durable);
172228 for failure in 1..=success.operation {
173229 let mut interrupted = disk(Some(failure));
174 let error = onestore::confirm_snapshot(&mut interrupted, &snapshot).unwrap_err();
230 let error =
231 onestore::confirm(&mut interrupted, &onestore::Stamp::of(&snapshot).unwrap())
232 .unwrap_err();
175233 assert_ne!(error.state, CommitState::Committed);
176234 assert_eq!(text_runs(&interrupted.durable, sid, oid), expected);
177235 assert_eq!(&interrupted.durable[..212], &snapshot[..212]);
......@@ -183,51 +241,53 @@ fn confirmation_notifies_cached_readers_after_interrupted_version_publication()
183241#[test]
184242fn confirming_visible_text_requires_flush_without_another_revision() {
185243 let (sid, oid) = target(SOURCE);
186 let visible = onestore::replace_text(SOURCE, sid, oid, 0..0, "Recovered ").unwrap();
244 let visible = replace_text(SOURCE, sid, oid, 0..0, "Recovered ").unwrap();
245 let base = onestore::Stamp::of(&visible).unwrap();
187246 let mut disk = Disk {
188247 visible: visible.clone(),
189248 durable: SOURCE.to_vec(),
190249 operation: 0,
191250 fail_at: None,
192 write_limit: 0,
251 write_limit: 64,
193252 random: 1,
194253 };
195 onestore::commit_text(&mut disk, &visible, sid, oid, 0..0, "").unwrap();
196 assert_eq!(disk.visible, visible);
197 assert_eq!(disk.durable, visible);
254 onestore::confirm(&mut disk, &base).unwrap();
255 assert_refreshed(&visible, &disk.durable);
198256 let flush = disk.operation;
199257 for (failure, state) in [
200258 (1, CommitState::NotCommitted),
201259 (flush, CommitState::Unknown),
202260 ] {
261 disk.visible.clone_from(&visible);
203262 disk.durable = SOURCE.to_vec();
204263 disk.operation = 0;
205264 disk.fail_at = Some(failure);
206 let error = onestore::commit_text(&mut disk, &visible, sid, oid, 0..0, "").unwrap_err();
265 let error = onestore::confirm(&mut disk, &base).unwrap_err();
207266 assert_eq!(error.state, state);
208 assert_eq!(disk.visible, visible);
267 assert_eq!(&disk.visible[..212], &visible[..212]);
268 assert_eq!(&disk.visible[252..], &visible[252..]);
209269 }
210270}
211271
212272#[test]
213273fn confirmation_rejects_a_changed_header_or_length_before_any_write() {
214 let (sid, oid) = target(SOURCE);
215274 let mut source = SOURCE.to_vec();
216275 source.resize(3 * 1024 * 1024 + 131, 0);
276 let base = onestore::Stamp::of(&source).unwrap();
217277 let mut disk = trace::Trace {
218278 bytes: source.clone(),
219279 events: Vec::new(),
220280 };
221 onestore::commit_text(&mut disk, &source, sid, oid, 0..0, "").unwrap();
222 assert_eq!(disk.events.len(), 1);
223 if let trace::Event::Write(offset, bytes) = &disk.events[0] {
224 panic!("Confirmation wrote {} bytes at {offset}", bytes.len());
225 }
281 let confirmed = |events: &[trace::Event]| {
282 matches!(events, [trace::Event::Flush, trace::Event::Write(212, version), trace::Event::Flush] if version.len() == 40)
283 };
284 onestore::confirm(&mut disk, &base).unwrap();
285 assert!(confirmed(&disk.events));
226286 for changed in [96, 128, 212, 1023] {
227287 disk.bytes.clone_from(&source);
228288 disk.bytes[changed] ^= 1;
229289 disk.events.clear();
230 let error = onestore::commit_text(&mut disk, &source, sid, oid, 0..0, "").unwrap_err();
290 let error = onestore::confirm(&mut disk, &base).unwrap_err();
231291 assert_eq!(error.state, CommitState::NotCommitted);
232292 assert_eq!(error.error.kind(), std::io::ErrorKind::ResourceBusy);
233293 assert!(disk.events.is_empty());
......@@ -237,14 +297,14 @@ fn confirmation_rejects_a_changed_header_or_length_before_any_write() {
237297 disk.bytes.clone_from(&source);
238298 disk.bytes[changed] ^= 1;
239299 disk.events.clear();
240 onestore::commit_text(&mut disk, &source, sid, oid, 0..0, "").unwrap();
241 assert!(matches!(disk.events[..], [trace::Event::Flush]));
300 onestore::confirm(&mut disk, &base).unwrap();
301 assert!(confirmed(&disk.events));
242302 }
243303 for length in [source.len() - 1, source.len() + 1] {
244304 disk.bytes.clone_from(&source);
245305 disk.bytes.resize(length, 0);
246306 disk.events.clear();
247 let error = onestore::commit_text(&mut disk, &source, sid, oid, 0..0, "").unwrap_err();
307 let error = onestore::confirm(&mut disk, &base).unwrap_err();
248308 assert_eq!(error.state, CommitState::NotCommitted);
249309 assert!(disk.events.is_empty());
250310 }
......@@ -332,7 +392,7 @@ fn text_splices_preserve_formats_objects_and_history_in_one_transaction() {
332392 let old_index = RevisionIndex::parse(&old).unwrap();
333393 let before = text_runs(SOURCE, sid, oid);
334394 for replacement in ["", "🦀", "A longer replacement with 日本語 and e\u{301}"] {
335 let edited = onestore::replace_text(SOURCE, sid, oid, 0..10, replacement).unwrap();
395 let edited = replace_text(SOURCE, sid, oid, 0..10, replacement).unwrap();
336396 let store = Store::parse(&edited).unwrap();
337397 assert_eq!(
338398 store.header.transaction_count,
......@@ -379,7 +439,7 @@ fn text_splices_preserve_formats_objects_and_history_in_one_transaction() {
379439#[test]
380440fn invalid_text_edits_never_touch_storage() {
381441 let (sid, oid) = target(SOURCE);
382 let source = onestore::replace_text(SOURCE, sid, oid, 0..0, "🦀").unwrap();
442 let source = replace_text(SOURCE, sid, oid, 0..0, "🦀").unwrap();
383443 for (range, replacement) in [
384444 (1..1, "x"),
385445 (0..999, "x"),
......@@ -396,7 +456,7 @@ fn invalid_text_edits_never_touch_storage() {
396456 random: 1,
397457 };
398458 let error =
399 onestore::commit_text(&mut disk, &source, sid, oid, range, replacement).unwrap_err();
459 commit_text(&mut disk, &source, sid, oid, range, replacement).unwrap_err();
400460 assert_eq!(error.state, CommitState::NotCommitted);
401461 assert_eq!(disk.operation, 0);
402462 assert_eq!(disk.durable, source);
......@@ -437,7 +497,7 @@ fn cross_run_splices_match_a_character_model_and_preserve_history() {
437497 let mut expected = before.clone();
438498 expected.splice(a..b, replacement.chars().map(|c| (c, before[a].1.clone())));
439499 let edited =
440 onestore::replace_text(SOURCE, sid, oid, offsets[a]..offsets[b], replacement)
500 replace_text(SOURCE, sid, oid, offsets[a]..offsets[b], replacement)
441501 .unwrap();
442502 assert_eq!(characters(&edited), expected, "characters {a}..{b}");
443503 let current_store = Store::parse(&edited).unwrap();
......@@ -460,8 +520,8 @@ fn cross_run_splices_match_a_character_model_and_preserve_history() {
460520 }
461521 }
462522 let cleared =
463 onestore::replace_text(SOURCE, sid, oid, 0..*offsets.last().unwrap(), "").unwrap();
464 let typed = onestore::replace_text(&cleared, sid, oid, 0..0, "new").unwrap();
523 replace_text(SOURCE, sid, oid, 0..*offsets.last().unwrap(), "").unwrap();
524 let typed = replace_text(&cleared, sid, oid, 0..0, "new").unwrap();
465525 let format = &before.last().unwrap().1;
466526 assert_eq!(
467527 characters(&typed),
......@@ -472,7 +532,7 @@ fn cross_run_splices_match_a_character_model_and_preserve_history() {
472532 );
473533 let text: String = before[..before.len() - 1].iter().map(|(c, _)| *c).collect();
474534 assert_eq!(
475 onestore::replace_text(SOURCE, sid, oid, 0..*offsets.last().unwrap(), &text).unwrap(),
535 replace_text(SOURCE, sid, oid, 0..*offsets.last().unwrap(), &text).unwrap(),
476536 SOURCE
477537 );
478538}
......@@ -520,9 +580,9 @@ fn export_native_cross_run_edits() {
520580 .unwrap();
521581 let range = range.start..range.end.min(length);
522582 let mut edited =
523 onestore::replace_text(&source, sid, oid, range.clone(), replacement).unwrap();
583 replace_text(&source, sid, oid, range.clone(), replacement).unwrap();
524584 if retype {
525 edited = onestore::replace_text(&edited, sid, oid, 0..0, "Retyped 🦀").unwrap();
585 edited = replace_text(&edited, sid, oid, 0..0, "Retyped 🦀").unwrap();
526586 }
527587 let candidate = output.join(name).join("candidate");
528588 fs::create_dir_all(&candidate).unwrap();
......@@ -540,7 +600,7 @@ fn export_native_cross_run_edits() {
540600fn insertion_at_a_style_boundary_uses_the_following_style() {
541601 let (sid, oid) = target(SOURCE);
542602 let before = text_runs(SOURCE, sid, oid);
543 let edited = onestore::replace_text(SOURCE, sid, oid, 18..18, "🦀").unwrap();
603 let edited = replace_text(SOURCE, sid, oid, 18..18, "🦀").unwrap();
544604 let mut expected = before.clone();
545605 expected[1]["text"] = format!("🦀{}", before[1]["text"].as_str().unwrap()).into();
546606 assert_eq!(text_runs(&edited, sid, oid), expected);
......@@ -588,10 +648,10 @@ fn automatic_titles_follow_native_line_and_utf16_limits() {
588648 };
589649 let (sid, oid) = check(&source, &expected);
590650 let cleared =
591 onestore::replace_text(&source, sid, oid, 0..text.encode_utf16().count() as u32, "")
651 replace_text(&source, sid, oid, 0..text.encode_utf16().count() as u32, "")
592652 .unwrap();
593653 check(&cleared, "");
594 let written = onestore::replace_text(&cleared, sid, oid, 0..0, " new 🦀 name ").unwrap();
654 let written = replace_text(&cleared, sid, oid, 0..0, " new 🦀 name ").unwrap();
595655 check(&written, "new 🦀 name");
596656 }
597657}
......@@ -641,7 +701,7 @@ fn automatic_titles_follow_native_rtl_and_attachment_order() {
641701 if node.extra[0].iter().any(|field| field.id == 0x88001cb4) {
642702 continue;
643703 }
644 let changed = onestore::replace_text(&source, sid, *oid, 0..0, "Edited ").unwrap();
704 let changed = replace_text(&source, sid, *oid, 0..0, "Edited ").unwrap();
645705 let store = Store::parse(&changed).unwrap();
646706 let current = RevisionIndex::parse(&store).unwrap();
647707 let after = Document::parse(&current).unwrap();
......@@ -675,8 +735,8 @@ fn automatic_titles_follow_native_rtl_and_attachment_order() {
675735fn an_emptied_final_run_retains_its_insertion_style() {
676736 let (sid, oid) = target(SOURCE);
677737 let before = text_runs(SOURCE, sid, oid);
678 let erased = onestore::replace_text(SOURCE, sid, oid, 22..27, "").unwrap();
679 let edited = onestore::replace_text(&erased, sid, oid, 22..22, "a").unwrap();
738 let erased = replace_text(SOURCE, sid, oid, 22..27, "").unwrap();
739 let edited = replace_text(&erased, sid, oid, 22..22, "a").unwrap();
680740 let mut expected = before.clone();
681741 expected[3]["text"] = "a".into();
682742 assert_eq!(text_runs(&edited, sid, oid), expected);
......@@ -739,12 +799,12 @@ fn title_text_and_navigation_caches_publish_together() {
739799 .collect::<std::collections::BTreeMap<_, _>>(),
740800 ])
741801 };
742 let checkpoint = checkpoint::pending(&source, sid, *oid, 0x14001d7a);
802 let checkpoint = checkpoint::pending(&source, sid, *oid);
743803 for (source, write_limit) in [(source.as_slice(), 17), (checkpoint.as_slice(), 257)] {
744804 let before = state(source);
745805 for replacement in ["Renamed 🦀 日本語", ""] {
746806 let edit =
747 onestore::PreparedEdit::text(source, sid, *oid, 0..end, replacement).unwrap();
807 text_transaction(source, sid, *oid, 0..end, replacement).unwrap();
748808 let disk = |fail_at| Disk {
749809 visible: source.to_vec(),
750810 durable: source.to_vec(),
......@@ -790,7 +850,7 @@ fn title_text_and_navigation_caches_publish_together() {
790850 );
791851 }
792852 assert_other_objects_preserved(&old, &current, *oid);
793 for at in source.len().div_ceil(193)..=success.operation {
853 for at in 1..=success.operation {
794854 let mut interrupted = disk(Some(at));
795855 let error = edit.commit(&mut interrupted).unwrap_err();
796856 let observed = state(&interrupted.durable);
......@@ -827,7 +887,7 @@ fn native_conflict_pages_are_readable_but_not_random_edit_targets() {
827887 if !matches!(node.kind, Kind::RichText { .. }) {
828888 continue;
829889 }
830 assert!(onestore::replace_text(&source, *sid, *oid, 0..0, "edit").is_err());
890 assert!(replace_text(&source, *sid, *oid, 0..0, "edit").is_err());
831891 conflicts += 1;
832892 }
833893 }
......@@ -837,7 +897,7 @@ fn native_conflict_pages_are_readable_but_not_random_edit_targets() {
837897#[test]
838898fn interrupted_text_commits_never_publish_mismatched_run_boundaries() {
839899 let (sid, oid) = target(SOURCE);
840 let checkpoint = checkpoint::pending(SOURCE, sid, oid, 0x14001d7a);
900 let checkpoint = checkpoint::pending(SOURCE, sid, oid);
841901 for source in [SOURCE, checkpoint.as_slice()] {
842902 let make_disk = |fail_at, random| Disk {
843903 visible: source.to_vec(),
......@@ -849,14 +909,14 @@ fn interrupted_text_commits_never_publish_mismatched_run_boundaries() {
849909 };
850910 let before = text_runs(source, sid, oid);
851911 let mut success = make_disk(None, 1);
852 onestore::commit_text(&mut success, source, sid, oid, 0..23, "🐈 mixed edit").unwrap();
912 commit_text(&mut success, source, sid, oid, 0..23, "🐈 mixed edit").unwrap();
853913 let after = text_runs(&success.durable, sid, oid);
854914 assert_ne!(before, after);
855 for at in source.len().div_ceil(193)..=success.operation {
915 for at in 1..=success.operation {
856916 for seed in [1, 42] {
857917 let mut disk = make_disk(Some(at), seed);
858918 let error =
859 onestore::commit_text(&mut disk, source, sid, oid, 0..23, "🐈 mixed edit")
919 commit_text(&mut disk, source, sid, oid, 0..23, "🐈 mixed edit")
860920 .unwrap_err();
861921 let observed = text_runs(&disk.durable, sid, oid);
862922 match error.state {
......@@ -906,7 +966,7 @@ fn empty_and_legacy_native_text_gain_unicode_without_losing_existing_properties(
906966 let properties = PropertySets::parse(blob).unwrap();
907967 assert!(!properties.sets[0].iter().any(|p| p.id == 0x1c001c22));
908968 let expected = format!("🦀 café {original}");
909 let written = onestore::replace_text(&source, sid, oid, 0..0, "🦀 café ").unwrap();
969 let written = replace_text(&source, sid, oid, 0..0, "🦀 café ").unwrap();
910970 let after_store = Store::parse(&written).unwrap();
911971 assert!(after_store.checksum_mismatches.is_empty());
912972 let after = RevisionIndex::parse(&after_store).unwrap();
......@@ -953,12 +1013,12 @@ fn empty_and_legacy_native_text_gain_unicode_without_losing_existing_properties(
9531013 random,
9541014 };
9551015 let mut success = disk(None, 1);
956 onestore::commit_text(&mut success, &source, sid, oid, 0..0, "🦀 café ").unwrap();
957 for at in source.len().div_ceil(193)..=success.operation {
1016 commit_text(&mut success, &source, sid, oid, 0..0, "🦀 café ").unwrap();
1017 for at in 1..=success.operation {
9581018 for seed in [1, 42] {
9591019 let mut interrupted = disk(Some(at), seed);
9601020 let error =
961 onestore::commit_text(&mut interrupted, &source, sid, oid, 0..0, "🦀 café ")
1021 commit_text(&mut interrupted, &source, sid, oid, 0..0, "🦀 café ")
9621022 .unwrap_err();
9631023 let actual = text_runs(&interrupted.durable, sid, oid);
9641024 match error.state {
crates/onestore/tests/formatting.rs+18-63
......@@ -1,10 +1,13 @@
1#[path = "support/ops.rs"]
2mod ops;
13#[path = "support/current.rs"]
24mod current;
35#[path = "support/disk.rs"]
46mod disk;
57use onestore::{
6 ExGuid, PreparedEdit, RevisionIndex, Store, TextAttribute as A,
8 ExGuid, RevisionIndex, Store, TextAttribute as A,
79 document::{Document, Kind},
10 op::PageOp,
811};
912use serde_json::{Value, json};
1013fn target(source: &[u8]) -> (ExGuid, ExGuid) {
......@@ -85,13 +88,7 @@ fn overlapping_unicode_format_edits_match_an_independent_character_model() {
8588 json!(0x00ff00_u32),
8689 ),
8790 };
88 let edit = PreparedEdit::format(
89 &source,
90 sid,
91 id,
92 offsets[start]..offsets[end],
93 std::slice::from_ref(&attribute),
94 )
91 let edit = ops::page_op(&source, sid, PageOp::Format { text: id, range: offsets[start]..offsets[end], set: std::slice::from_ref(&attribute).to_vec(), clear: Vec::new() })
9592 .unwrap();
9693 for (_, style) in &mut expected[start..end] {
9794 style[key] = value.clone();
......@@ -102,13 +99,7 @@ fn overlapping_unicode_format_edits_match_an_independent_character_model() {
10299 "seed {seed}, step {step}"
103100 );
104101 assert_eq!(
105 PreparedEdit::format(
106 edit.as_bytes(),
107 sid,
108 id,
109 offsets[start]..offsets[end],
110 &[attribute]
111 )
102 ops::page_op(edit.as_bytes(), sid, PageOp::Format { text: id, range: offsets[start]..offsets[end], set: vec![attribute], clear: Vec::new() })
112103 .unwrap()
113104 .as_bytes(),
114105 edit.as_bytes()
......@@ -130,21 +121,9 @@ fn overlapping_unicode_format_edits_match_an_independent_character_model() {
130121fn script_positions_are_exclusive_and_explicit_false_overrides_true() {
131122 let source = onestore::create_section("script.one", "abc", "Author").unwrap();
132123 let (sid, id) = target(&source);
133 let superscript = PreparedEdit::format(
134 &source,
135 sid,
136 id,
137 0..3,
138 &[A::Superscript(true), A::Bold(true)],
139 )
124 let superscript = ops::page_op(&source, sid, PageOp::Format { text: id, range: 0..3, set: vec![A::Superscript(true), A::Bold(true)], clear: Vec::new() })
140125 .unwrap();
141 let subscript = PreparedEdit::format(
142 superscript.as_bytes(),
143 sid,
144 id,
145 1..2,
146 &[A::Subscript(true), A::Bold(false)],
147 )
126 let subscript = ops::page_op(superscript.as_bytes(), sid, PageOp::Format { text: id, range: 1..2, set: vec![A::Subscript(true), A::Bold(false)], clear: Vec::new() })
148127 .unwrap();
149128 let chars = characters(subscript.as_bytes(), sid, id);
150129 assert_eq!(chars[0].1["superscript"], true);
......@@ -159,15 +138,9 @@ fn script_positions_are_exclusive_and_explicit_false_overrides_true() {
159138fn empty_paragraph_style_is_used_by_later_text_edits() {
160139 let source = onestore::create_section("empty.one", "", "Author").unwrap();
161140 let (sid, id) = target(&source);
162 let formatted = PreparedEdit::format(
163 &source,
164 sid,
165 id,
166 0..0,
167 &[A::Italic(true), A::FontSize(18.0)],
168 )
141 let formatted = ops::page_op(&source, sid, PageOp::Format { text: id, range: 0..0, set: vec![A::Italic(true), A::FontSize(18.0)], clear: Vec::new() })
169142 .unwrap();
170 let filled = PreparedEdit::text(formatted.as_bytes(), sid, id, 0..0, "Added 🦀").unwrap();
143 let filled = ops::page_op(formatted.as_bytes(), sid, PageOp::Text { text: id, range: 0..0, with: "Added 🦀".into() }).unwrap();
171144 for (_, format) in characters(filled.as_bytes(), sid, id) {
172145 assert_eq!(format["italic"], true);
173146 assert_eq!(format["font_size"], 18.0);
......@@ -179,7 +152,7 @@ fn invalid_ranges_attributes_and_fields_are_rejected() {
179152 let (sid, id) = target(&source);
180153 for (start, end) in [(2, 3), (1, 2), (0, 8), (2, 2), (3, 1), (1, 1)] {
181154 let range = start..end;
182 assert!(PreparedEdit::format(&source, sid, id, range, &[A::Bold(true)]).is_err());
155 assert!(ops::page_op(&source, sid, PageOp::Format { text: id, range, set: vec![A::Bold(true)], clear: Vec::new() }).is_err());
183156 }
184157 for attributes in [
185158 vec![],
......@@ -188,24 +161,18 @@ fn invalid_ranges_attributes_and_fields_are_rejected() {
188161 vec![A::Font("".into())],
189162 vec![A::Font("a\0b".into())],
190163 ] {
191 assert!(PreparedEdit::format(&source, sid, id, 0..4, &attributes).is_err());
164 assert!(ops::page_op(&source, sid, PageOp::Format { text: id, range: 0..4, set: attributes.to_vec(), clear: Vec::new() }).is_err());
192165 }
193166 for size in [f32::NAN, f32::INFINITY, 0.0, 5.5, 130.5, 144.0, 144.5, 12.1] {
194 assert!(PreparedEdit::format(&source, sid, id, 0..4, &[A::FontSize(size)]).is_err());
167 assert!(ops::page_op(&source, sid, PageOp::Format { text: id, range: 0..4, set: vec![A::FontSize(size)], clear: Vec::new() }).is_err());
195168 }
196 assert!(PreparedEdit::format(&source, sid, ExGuid::default(), 0..4, &[A::Bold(true)]).is_err());
169 assert!(ops::page_op(&source, sid, PageOp::Format { text: ExGuid::default(), range: 0..4, set: vec![A::Bold(true)], clear: Vec::new() }).is_err());
197170}
198171#[test]
199172fn formatting_publication_faults_preserve_complete_old_or_new_styles() {
200173 let source = onestore::create_section("atomic.one", "Before 🦀 after", "Author").unwrap();
201174 let (sid, id) = target(&source);
202 let edit = PreparedEdit::format(
203 &source,
204 sid,
205 id,
206 2..10,
207 &[A::Bold(true), A::Color(Some([8, 64, 128]))],
208 )
175 let edit = ops::page_op(&source, sid, PageOp::Format { text: id, range: 2..10, set: vec![A::Bold(true), A::Color(Some([8, 64, 128]))], clear: Vec::new() })
209176 .unwrap();
210177 let before = current::current(&source);
211178 let after = current::current(edit.as_bytes());
......@@ -251,7 +218,7 @@ fn export_native_formatting_candidates() {
251218 let mut save =
252219 |name: &str, source: &[u8], sid, id, range: std::ops::Range<u32>, attributes: &[A]| {
253220 let prepared =
254 PreparedEdit::format(source, sid, id, range.clone(), attributes).unwrap();
221 ops::page_op(source, sid, PageOp::Format { text: id, range: range.clone(), set: attributes.to_vec(), clear: Vec::new() }).unwrap();
255222 fs::write(output.join(format!("{name}.one")), prepared.as_bytes()).unwrap();
256223 manifest.push(
257224 json!({"name":name,"space":sid,"object":id,"range":range,"attributes":attributes}),
......@@ -329,21 +296,9 @@ fn export_native_formatting_candidates() {
329296 );
330297 let empty = onestore::create_section("empty.one", "", "Author").unwrap();
331298 let (empty_sid, empty_id) = target(&empty);
332 let formatted = PreparedEdit::format(
333 &empty,
334 empty_sid,
335 empty_id,
336 0..0,
337 &[A::FontSize(18.0), A::Italic(true)],
338 )
299 let formatted = ops::page_op(&empty, empty_sid, PageOp::Format { text: empty_id, range: 0..0, set: vec![A::FontSize(18.0), A::Italic(true)], clear: Vec::new() })
339300 .unwrap();
340 let typed = PreparedEdit::text(
341 formatted.as_bytes(),
342 empty_sid,
343 empty_id,
344 0..0,
345 "Typed café 🦀",
346 )
301 let typed = ops::page_op(formatted.as_bytes(), empty_sid, PageOp::Text { text: empty_id, range: 0..0, with: "Typed café 🦀".into() })
347302 .unwrap();
348303 fs::write(output.join("empty-then-type.one"), typed.as_bytes()).unwrap();
349304 let native = include_bytes!("../../../corpus/native-ink/20260905-ui/notebook/synthetic.one");
crates/onestore/tests/insertion.rs+231-309
......@@ -4,11 +4,65 @@ mod checkpoint;
44mod current;
55#[path = "support/disk.rs"]
66mod disk;
7#[path = "support/ops.rs"]
8mod ops;
79
810use onestore::{
9 ExGuid, Insertion, PreparedEdit, RevisionIndex, Store, TextAttribute as A,
11 ExGuid, RevisionIndex, Store, TextAttribute as A,
1012 document::{Document, Kind},
13 op::{Edit, Op, PageOp},
1114};
15use std::ops::Range;
16
17/// Where an insertion goes: a paragraph in a container before a child of it, or last; or a
18/// new outline at a point on the page.
19#[derive(Clone, Copy)]
20enum Place {
21 In(ExGuid, Option<ExGuid>),
22 Outline(f32, f32),
23}
24
25/// The ops inserting `text` at `place` with `formats` set over UTF-16 ranges of it, and the
26/// identities of the new paragraph (or outline) and text.
27fn insertion(
28 space: ExGuid,
29 place: Place,
30 text: &str,
31 formats: &[(Range<u32>, Vec<A>)],
32) -> (Vec<Op>, ExGuid, ExGuid) {
33 let paragraph = ops::paragraph(text);
34 let text = paragraph.text().unwrap().id;
35 let (op, object) = match place {
36 Place::In(container, before) => {
37 let id = paragraph.id;
38 let paragraphs = vec![paragraph];
39 (
40 PageOp::Insert {
41 container,
42 before,
43 paragraphs,
44 },
45 id,
46 )
47 }
48 Place::Outline(x, y) => {
49 let object = ops::outline(x, y, vec![paragraph]);
50 let id = object.id();
51 (PageOp::Add { object, before: None }, id)
52 }
53 };
54 let mut result = vec![Op::Page { space, op }];
55 for (range, set) in formats {
56 let op = PageOp::Format {
57 text,
58 range: range.clone(),
59 set: set.clone(),
60 clear: Vec::new(),
61 };
62 result.push(Op::Page { space, op });
63 }
64 (result, object, text)
65}
1266
1367fn targets(source: &[u8]) -> (ExGuid, ExGuid, ExGuid, ExGuid, ExGuid) {
1468 let store = Store::parse(source).unwrap();
......@@ -34,23 +88,15 @@ fn atomic_formatted_insertions_match_a_character_model_and_preserve_history() {
3488 let source = onestore::create_section("rich.one", "Original", "Author").unwrap();
3589 let (sid, page, outline, _, _) = targets(&source);
3690 let text = "ab🦀 e\u{301} 東京\rEnd\t!";
37 let plain = Insertion::paragraph(outline, None, text, "Author").unwrap();
38 assert!(
39 serde_json::to_value(&plain)
40 .unwrap()
41 .get("formats")
42 .is_none()
43 );
44 let restored: Insertion = serde_json::from_slice(&serde_json::to_vec(&plain).unwrap()).unwrap();
45 assert_eq!(plain, restored);
46 let base = PreparedEdit::insert(&source, sid, &plain).unwrap();
47 let base_store = Store::parse(base.as_bytes()).unwrap();
91 let (plain, _, plain_text) = insertion(sid, Place::In(outline, None), text, &[]);
92 let base = ops::edited(&source, plain).unwrap();
93 let base_store = Store::parse(&base).unwrap();
4894 let base_index = RevisionIndex::parse(&base_store).unwrap();
4995 let base_doc = Document::parse(&base_index).unwrap();
5096 let base_space = &base_doc.spaces[&sid];
5197 let base_view = &base_space.revisions[&base_space.contexts[&ExGuid::default()]];
5298 let default =
53 serde_json::to_value(&base_view.text_runs(plain.text_object()).unwrap()[0].format).unwrap();
99 serde_json::to_value(&base_view.text_runs(plain_text).unwrap()[0].format).unwrap();
54100 let offsets: Vec<u32> = std::iter::once(0)
55101 .chain(text.chars().scan(0, |n, c| {
56102 *n += c.len_utf16() as u32;
......@@ -62,12 +108,8 @@ fn atomic_formatted_insertions_match_a_character_model_and_preserve_history() {
62108 for seed in 1..=48_u64 {
63109 let mut random = seed;
64110 let mut expected: Vec<_> = text.chars().map(|c| (c, default.clone())).collect();
65 let mut intent = if seed % 2 == 0 {
66 Insertion::outline(page, 72.0, 144.0, text, "Author").unwrap()
67 } else {
68 plain.clone()
69 };
70 // Descending construction also exercises persisted ordering independently of call order.
111 let mut formats = Vec::new();
112 // Descending construction also exercises the order formats apply in.
71113 for i in (0..expected.len()).rev() {
72114 random ^= random << 13;
73115 random ^= random >> 7;
......@@ -106,18 +148,22 @@ fn atomic_formatted_insertions_match_a_character_model_and_preserve_history() {
106148 json!({"bold":false,"color":0xff000000_u32,"highlight":0xff000000_u32}),
107149 ),
108150 };
109 intent = intent
110 .with_formatting(offsets[i]..offsets[i + 1], &attributes)
111 .unwrap();
151 formats.push((offsets[i]..offsets[i + 1], attributes));
112152 for (key, value) in values.as_object().unwrap() {
113153 expected[i].1[key] = value.clone();
114154 }
115155 }
116 let restored: Insertion =
117 serde_json::from_slice(&serde_json::to_vec(&intent).unwrap()).unwrap();
118 assert_eq!(restored, intent);
119 let edited = PreparedEdit::insert(&source, sid, &restored).unwrap();
120 let store = Store::parse(edited.as_bytes()).unwrap();
156 let place = if seed % 2 == 0 {
157 Place::Outline(72.0, 144.0)
158 } else {
159 Place::In(outline, None)
160 };
161 let (ops, _, inserted) = insertion(sid, place, text, &formats);
162 let edit = Edit { at: ops::now(), ops };
163 let restored: Edit = serde_json::from_slice(&serde_json::to_vec(&edit).unwrap()).unwrap();
164 assert_eq!(restored, edit);
165 let edited = ops::edited(&source, restored.ops).unwrap();
166 let store = Store::parse(&edited).unwrap();
121167 assert_eq!(
122168 store.header.transaction_count,
123169 old_store.header.transaction_count + 1
......@@ -139,8 +185,11 @@ fn atomic_formatted_insertions_match_a_character_model_and_preserve_history() {
139185 let document = Document::parse(&index).unwrap();
140186 let space = &document.spaces[&sid];
141187 let view = &space.revisions[&space.contexts[&ExGuid::default()]];
188 if let Place::Outline(..) = place {
189 assert!(view.nodes[&page].children.len() > 1);
190 }
142191 let actual: Vec<_> = view
143 .text_runs(intent.text_object())
192 .text_runs(inserted)
144193 .unwrap()
145194 .into_iter()
146195 .flat_map(|run| {
......@@ -152,88 +201,34 @@ fn atomic_formatted_insertions_match_a_character_model_and_preserve_history() {
152201 }
153202}
154203
155#[test]
156fn invalid_or_forged_insertion_formats_fail_before_source_parsing() {
157 use serde_json::json;
158 let source = onestore::create_section("invalid-rich.one", "Original", "Author").unwrap();
159 let (sid, _, outline, _, _) = targets(&source);
160 let plain = Insertion::paragraph(outline, None, "a🦀e\u{301}b", "Author").unwrap();
161 for (start, end) in [(0, 0), (2, 3), (1, 2), (4, 3), (0, 7)] {
162 let range = start..end;
163 assert!(plain.with_formatting(range, &[A::Bold(true)]).is_err());
164 }
165 for attributes in [
166 vec![],
167 vec![A::Bold(true), A::Bold(false)],
168 vec![A::FontSize(f32::NAN)],
169 vec![A::FontSize(5.5)],
170 vec![A::Font("a\0b".into())],
171 vec![A::Superscript(true), A::Subscript(true)],
172 ] {
173 assert!(plain.with_formatting(1..3, &attributes).is_err());
174 }
175 let formatted = plain.with_formatting(1..3, &[A::Bold(true)]).unwrap();
176 for range in [0..3, 1..3, 1..4] {
177 assert!(
178 formatted
179 .with_formatting(range, &[A::Italic(true)])
180 .is_err()
181 );
182 }
183 let encoded = serde_json::to_value(&formatted).unwrap();
184 for mutate in 0..6 {
185 let mut value = encoded.clone();
186 match mutate {
187 0 => value["formats"][0]["guid"] = serde_json::to_value([0; 16]).unwrap(),
188 1 => value["formats"][0]["guid"] = value["guid"].clone(),
189 2 => {
190 let duplicate = value["formats"][0].clone();
191 value["formats"].as_array_mut().unwrap().push(duplicate);
192 }
193 3 => value["formats"][0]["range"] = json!({"start":2,"end":3}),
194 4 => value["formats"][0]["attributes"] = json!([]),
195 _ => value["formats"][0]["range"] = json!({"start":0,"end":u32::MAX}),
196 }
197 let forged: Insertion = serde_json::from_value(value).unwrap();
198 let error = PreparedEdit::insert(&[], sid, &forged).err().unwrap();
199 assert_eq!(
200 error,
201 PreparedEdit::insert(&source, sid, &forged).err().unwrap()
202 );
203 }
204}
205
206204#[test]
207205fn formatted_empty_insertions_preserve_the_insertion_style_for_later_typing() {
208206 let source = onestore::create_section("empty-rich.one", "Original", "Author").unwrap();
209 let (sid, page, outline, _, _) = targets(&source);
210 for plain in [
211 Insertion::outline(page, 72.0, 144.0, "", "Author").unwrap(),
212 Insertion::paragraph(outline, None, "", "Author").unwrap(),
213 ] {
214 let intent = plain
215 .with_formatting(0..0, &[A::Italic(true), A::FontSize(18.0)])
216 .unwrap();
217 assert!(intent.with_formatting(0..0, &[A::Bold(true)]).is_err());
218 let inserted = PreparedEdit::insert(&source, sid, &intent).unwrap();
219 let typed = PreparedEdit::text(
220 inserted.as_bytes(),
207 let (sid, _, outline, _, _) = targets(&source);
208 for place in [Place::Outline(72.0, 144.0), Place::In(outline, None)] {
209 let style = [(0..0, vec![A::Italic(true), A::FontSize(18.0)])];
210 let (ops, _, text) = insertion(sid, place, "", &style);
211 let inserted = ops::edited(&source, ops).unwrap();
212 let typed = ops::page_edited(
213 &inserted,
221214 sid,
222 intent.text_object(),
223 0..0,
224 "Typed 🦀",
215 vec![PageOp::Text {
216 text,
217 range: 0..0,
218 with: "Typed 🦀".into(),
219 }],
225220 )
226221 .unwrap();
227 for (bytes, text) in [(inserted.as_bytes(), ""), (typed.as_bytes(), "Typed 🦀")] {
222 for (bytes, expected) in [(&inserted, ""), (&typed, "Typed 🦀")] {
228223 let store = Store::parse(bytes).unwrap();
229224 let index = RevisionIndex::parse(&store).unwrap();
230225 index.validate_current().unwrap();
231226 let document = Document::parse(&index).unwrap();
232227 let space = &document.spaces[&sid];
233228 let view = &space.revisions[&space.contexts[&ExGuid::default()]];
234 let runs = view.text_runs(intent.text_object()).unwrap();
229 let runs = view.text_runs(text).unwrap();
235230 assert_eq!(runs.len(), 1);
236 assert_eq!(runs[0].text, text);
231 assert_eq!(runs[0].text, expected);
237232 assert_eq!(runs[0].format.italic, Some(true));
238233 assert_eq!(runs[0].format.font_size, Some(18.0));
239234 }
......@@ -263,11 +258,9 @@ fn export_native_formatted_insertions() {
263258 ("empty", generated.as_slice(), true),
264259 ("empty-typed", generated.as_slice(), true),
265260 ] {
266 let (sid, page, outline, paragraph, _) = targets(source);
267 let insertion = match name {
268 "native-paragraph" => {
269 Insertion::paragraph(outline, Some(paragraph), text, "Rich author")
270 }
261 let (sid, _, outline, paragraph, _) = targets(source);
262 let place = match name {
263 "native-paragraph" => Place::In(outline, Some(paragraph)),
271264 "native-cell" => {
272265 let store = Store::parse(source).unwrap();
273266 let index = RevisionIndex::parse(&store).unwrap();
......@@ -280,24 +273,14 @@ fn export_native_formatted_insertions() {
280273 .find(|(_, node)| matches!(node.kind, Kind::Cell { .. }))
281274 .unwrap()
282275 .0;
283 Insertion::paragraph(cell, None, text, "Rich author")
276 Place::In(cell, None)
284277 }
285 _ => Insertion::outline(
286 page,
287 72.0,
288 144.0,
289 if empty { "" } else { text },
290 "Rich author",
291 ),
292 }
293 .unwrap();
294 let mut intent = insertion;
295 if empty {
296 intent = intent
297 .with_formatting(0..0, &[A::Italic(true), A::FontSize(18.0)])
298 .unwrap();
278 _ => Place::Outline(72.0, 144.0),
279 };
280 let formats = if empty {
281 vec![(0..0, vec![A::Italic(true), A::FontSize(18.0)])]
299282 } else {
300 for (label, attributes) in [
283 [
301284 ("Bold", vec![A::Bold(true), A::Underline(true)]),
302285 ("🦀", vec![A::Font("Arial".into()), A::FontSize(13.5)]),
303286 ("italic", vec![A::Italic(true), A::Strike(true)]),
......@@ -319,36 +302,33 @@ fn export_native_formatted_insertions() {
319302 A::Highlight(None),
320303 ],
321304 ),
322 ] {
305 ]
306 .into_iter()
307 .map(|(label, attributes)| {
323308 let start = text[..text.find(label).unwrap()].encode_utf16().count() as u32;
324 intent = intent
325 .with_formatting(
326 start..start + label.encode_utf16().count() as u32,
327 &attributes,
328 )
329 .unwrap();
330 }
331 }
332 let inserted = PreparedEdit::insert(source, sid, &intent).unwrap();
333 let bytes = if name == "empty-typed" {
334 PreparedEdit::text(
335 inserted.as_bytes(),
336 sid,
337 intent.text_object(),
338 0..0,
339 "Typed café 🦀",
340 )
341 .unwrap()
342 .as_bytes()
343 .to_vec()
344 } else {
345 inserted.as_bytes().to_vec()
309 (start..start + label.encode_utf16().count() as u32, attributes)
310 })
311 .collect()
346312 };
313 let (mut ops, object, inserted) =
314 insertion(sid, place, if empty { "" } else { text }, &formats);
315 if name == "empty-typed" {
316 let op = PageOp::Text {
317 text: inserted,
318 range: 0..0,
319 with: "Typed café 🦀".into(),
320 };
321 ops.push(Op::Page { space: sid, op });
322 }
323 let edit = Edit { at: ops::now(), ops };
324 let bytes = ops::edited(source, edit.ops.clone()).unwrap();
347325 current::current(&bytes);
348326 let notebook = output.join(name);
349327 fs::create_dir(&notebook).unwrap();
350328 fs::write(notebook.join("synthetic.one"), &bytes).unwrap();
351 manifest.push(serde_json::json!({"name":name,"space":sid,"intent":intent}));
329 manifest.push(serde_json::json!({
330 "name": name, "space": sid, "object": object, "text": inserted, "edit": edit
331 }));
352332 }
353333 fs::write(
354334 output.join("manifest.json"),
......@@ -361,23 +341,15 @@ fn export_native_formatted_insertions() {
361341fn paragraph_and_outline_insertions_publish_metadata_and_references_together() {
362342 let source = onestore::create_section("insertion.one", "Original", "Original author").unwrap();
363343 let (sid, page, outline, paragraph, original_text) = targets(&source);
364 for (intent, expected_parent, expected_x, expected_y) in [
365 (
366 Insertion::paragraph(outline, Some(paragraph), "First 🦀\rSecond", "New author")
367 .unwrap(),
368 outline,
369 None,
370 None,
371 ),
372 (
373 Insertion::outline(page, 144.0, 18.0, "First 🦀\rSecond", "New author").unwrap(),
374 page,
375 Some(144.0),
376 Some(18.0),
377 ),
344 for (place, expected_parent, expected_x, expected_y) in [
345 (Place::In(outline, Some(paragraph)), outline, None, None),
346 (Place::Outline(144.0, 18.0), page, Some(144.0), Some(18.0)),
378347 ] {
379 let prepared = PreparedEdit::insert(&source, sid, &intent).unwrap();
380 let store = Store::parse(prepared.as_bytes()).unwrap();
348 let (ops, object, text) = insertion(sid, place, "First 🦀\rSecond", &[]);
349 let transaction = ops::transaction(&source, "New author", ops).unwrap().unwrap();
350 let mut written = source.clone();
351 transaction.apply(&mut written).unwrap();
352 let store = Store::parse(&written).unwrap();
381353 assert_eq!(
382354 store.header.transaction_count,
383355 Store::parse(&source).unwrap().header.transaction_count + 1
......@@ -387,18 +359,14 @@ fn paragraph_and_outline_insertions_publish_metadata_and_references_together() {
387359 let document = Document::parse(&index).unwrap();
388360 let space = &document.spaces[&sid];
389361 let view = &space.revisions[&space.contexts[&ExGuid::default()]];
390 assert!(
391 view.nodes[&expected_parent]
392 .children
393 .contains(&intent.object())
394 );
395 assert_eq!(view.nodes[&intent.object()].layout.x, expected_x);
396 assert_eq!(view.nodes[&intent.object()].layout.y, expected_y);
362 assert!(view.nodes[&expected_parent].children.contains(&object));
363 assert_eq!(view.nodes[&object].layout.x, expected_x);
364 assert_eq!(view.nodes[&object].layout.y, expected_y);
397365 assert!(
398366 matches!(&view.nodes[&original_text].kind, Kind::RichText { text, .. } if text == "Original")
399367 );
400368 assert!(
401 matches!(&view.nodes[&intent.text_object()].kind, Kind::RichText { text, .. } if text == "First 🦀\rSecond")
369 matches!(&view.nodes[&text].kind, Kind::RichText { text, .. } if text == "First 🦀\rSecond")
402370 );
403371 assert!(
404372 matches!(&view.nodes[&page].kind, Kind::Page { alternate_title, .. } if alternate_title.as_deref() == Some("First 🦀"))
......@@ -411,7 +379,7 @@ fn paragraph_and_outline_insertions_publish_metadata_and_references_together() {
411379 .values()
412380 .any(|node| matches!(&node.kind, Kind::Author { name } if name.as_deref() == Some("New author")))
413381 );
414 let runs = view.text_runs(intent.text_object()).unwrap();
382 let runs = view.text_runs(text).unwrap();
415383 assert_eq!(runs.len(), 1);
416384 assert_eq!(runs[0].format.font.as_deref(), Some("Calibri"));
417385 assert_eq!(runs[0].format.font_size, Some(11.0));
......@@ -434,26 +402,20 @@ fn repeated_insertions_and_formatting_share_immutable_objects() {
434402 let (sid, _, outline, _, _) = targets(&source);
435403 let mut texts = Vec::new();
436404 for _ in 0..12 {
437 let insertion =
438 Insertion::paragraph(outline, None, "Repeated paragraph", "Same author").unwrap();
439 source = PreparedEdit::insert(&source, sid, &insertion)
405 let (ops, _, text) = insertion(sid, Place::In(outline, None), "Repeated paragraph", &[]);
406 let transaction = ops::transaction(&source, "Same author", ops).unwrap().unwrap();
407 transaction.apply(&mut source).unwrap();
408 let bold = PageOp::Format {
409 text,
410 range: 1..8,
411 set: vec![A::Bold(true), A::FontSize(20.0)],
412 clear: Vec::new(),
413 };
414 let transaction = ops::transaction(&source, "Same author", vec![Op::Page { space: sid, op: bold }])
440415 .unwrap()
441 .as_bytes()
442 .to_vec();
443 source = PreparedEdit::format(
444 &source,
445 sid,
446 insertion.text_object(),
447 1..8,
448 &[
449 onestore::TextAttribute::Bold(true),
450 onestore::TextAttribute::FontSize(20.0),
451 ],
452 )
453 .unwrap()
454 .as_bytes()
455 .to_vec();
456 texts.push(insertion.text_object());
416 .unwrap();
417 transaction.apply(&mut source).unwrap();
418 texts.push(text);
457419 }
458420 let store = Store::parse(&source).unwrap();
459421 let index = RevisionIndex::parse(&store).unwrap();
......@@ -494,103 +456,54 @@ fn repeated_insertions_and_formatting_share_immutable_objects() {
494456fn serialized_insertions_rebase_with_the_same_objects_and_preserve_remote_edits() {
495457 let source = onestore::create_section("rebase.one", "Original", "Author").unwrap();
496458 let (sid, _, outline, paragraph, text) = targets(&source);
497 let intent = Insertion::paragraph(outline, Some(paragraph), "Inserted", "Author")
498 .unwrap()
499 .with_formatting(0..3, &[A::Bold(true), A::Color(Some([12, 34, 56]))])
500 .unwrap();
501 let encoded = serde_json::to_vec(&intent).unwrap();
502 let restored: Insertion = serde_json::from_slice(&encoded).unwrap();
503 assert_eq!(intent.object(), restored.object());
504 assert_eq!(intent.text_object(), restored.text_object());
505 assert_eq!(intent, restored);
506 let original_preparation = PreparedEdit::insert(&source, sid, &intent).unwrap();
507 let remote = PreparedEdit::text(&source, sid, text, 0..0, "Remote ").unwrap();
508 let updated = PreparedEdit::insert(remote.as_bytes(), sid, &restored).unwrap();
509 let store = Store::parse(updated.as_bytes()).unwrap();
459 let formats = [(0..3, vec![A::Bold(true), A::Color(Some([12, 34, 56]))])];
460 let (ops, object, inserted) =
461 insertion(sid, Place::In(outline, Some(paragraph)), "Inserted", &formats);
462 let encoded = serde_json::to_vec(&ops).unwrap();
463 let restored: Vec<Op> = serde_json::from_slice(&encoded).unwrap();
464 assert_eq!(ops, restored);
465 let original = ops::edited(&source, ops).unwrap();
466 let remote = ops::page_edited(
467 &source,
468 sid,
469 vec![PageOp::Text {
470 text,
471 range: 0..0,
472 with: "Remote ".into(),
473 }],
474 )
475 .unwrap();
476 let updated = ops::edited(&remote, restored.clone()).unwrap();
477 let store = Store::parse(&updated).unwrap();
510478 let index = RevisionIndex::parse(&store).unwrap();
511479 let document = Document::parse(&index).unwrap();
512480 let space = &document.spaces[&sid];
513481 let view = &space.revisions[&space.contexts[&ExGuid::default()]];
514 assert_eq!(
515 view.nodes[&outline].children,
516 [restored.object(), paragraph]
517 );
482 assert_eq!(view.nodes[&outline].children, [object, paragraph]);
518483 assert!(
519484 matches!(&view.nodes[&text].kind, Kind::RichText { text, .. } if text == "Remote Original")
520485 );
521486 assert!(
522 matches!(&view.nodes[&restored.text_object()].kind, Kind::RichText { text, .. } if text == "Inserted")
487 matches!(&view.nodes[&inserted].kind, Kind::RichText { text, .. } if text == "Inserted")
523488 );
524 let runs = view.text_runs(restored.text_object()).unwrap();
489 let runs = view.text_runs(inserted).unwrap();
525490 assert_eq!(runs[0].text, "Ins");
526491 assert_eq!(runs[0].format.bold, Some(true));
527492 assert_eq!(runs[0].format.color, Some(0x38220c));
528493 assert_eq!(runs[1].text, "erted");
529494 assert_ne!(runs[1].format.bold, Some(true));
530 assert!(PreparedEdit::insert(original_preparation.as_bytes(), sid, &restored).is_err());
531 assert!(PreparedEdit::insert(updated.as_bytes(), sid, &restored).is_err());
532}
533
534#[test]
535fn repositioning_preserves_intent_identity_and_kind() {
536 let source = onestore::create_section("placement.one", "Original", "Author").unwrap();
537 let (sid, page, outline, paragraph, _) = targets(&source);
538 let p = Insertion::paragraph(outline, Some(paragraph), "Inserted", "Author")
539 .unwrap()
540 .with_formatting(1..4, &[A::Italic(true)])
541 .unwrap();
542 let o = Insertion::outline(page, 144.0, 144.0, "Inserted", "Author")
543 .unwrap()
544 .with_formatting(0..8, &[A::FontSize(18.0)])
545 .unwrap();
546 assert!(p.reposition_outline(page, 72.0, 72.0).is_err());
547 assert!(o.reposition_paragraph(outline, None).is_err());
548 assert!(p.reposition_paragraph(ExGuid::default(), None).is_err());
549 assert!(o.reposition_outline(page, f32::NAN, 72.0).is_err());
550 for (original, moved) in [
551 (&p, p.reposition_paragraph(outline, None).unwrap()),
552 (&o, o.reposition_outline(page, 288.0, 360.0).unwrap()),
553 ] {
554 let mut before = serde_json::to_value(original).unwrap();
555 let mut after = serde_json::to_value(&moved).unwrap();
556 for name in ["parent", "placement"] {
557 before.as_object_mut().unwrap().remove(name);
558 after.as_object_mut().unwrap().remove(name);
559 }
560 assert_eq!(before, after);
561 assert_eq!(original.object(), moved.object());
562 assert_eq!(original.text_object(), moved.text_object());
563 let restored: Insertion =
564 serde_json::from_value(serde_json::to_value(&moved).unwrap()).unwrap();
565 let prepared = PreparedEdit::insert(&source, sid, &restored).unwrap();
566 let store = Store::parse(prepared.as_bytes()).unwrap();
567 let index = RevisionIndex::parse(&store).unwrap();
568 index.validate_current().unwrap();
569 let doc = Document::parse(&index).unwrap();
570 let space = &doc.spaces[&sid];
571 let view = &space.revisions[&space.contexts[&ExGuid::default()]];
572 if original == &p {
573 assert_eq!(view.nodes[&outline].children, [paragraph, moved.object()]);
574 } else {
575 assert_eq!(
576 (
577 view.nodes[&moved.object()].layout.x,
578 view.nodes[&moved.object()].layout.y
579 ),
580 (Some(288.0), Some(360.0))
581 );
582 }
583 }
495 assert!(ops::edited(&original, restored.clone()).is_err());
496 assert!(ops::edited(&updated, restored).is_err());
584497}
585498
586499#[test]
587500#[cfg(any(unix, windows))]
588fn filesystem_insertion_uses_the_prepared_identity_and_rejects_stale_replay() {
501fn filesystem_insertion_commits_once_and_rejects_stale_replay() {
589502 use std::{fs, io::Write};
590503 let source = onestore::create_section("file.one", "Original", "Author").unwrap();
591504 let (sid, _, outline, _, _) = targets(&source);
592 let intent = Insertion::paragraph(outline, None, "File insertion", "Author").unwrap();
593 let path = std::env::temp_dir().join(format!("onestore-insertion-{}.one", intent.object()));
505 let (ops, object, _) = insertion(sid, Place::In(outline, None), "File insertion", &[]);
506 let path = std::env::temp_dir().join(format!("onestore-insertion-{object}.one"));
594507 let mut file = fs::OpenOptions::new()
595508 .write(true)
596509 .create_new(true)
......@@ -599,60 +512,71 @@ fn filesystem_insertion_uses_the_prepared_identity_and_rejects_stale_replay() {
599512 file.write_all(&source).unwrap();
600513 file.sync_all().unwrap();
601514 drop(file);
602 let edit = PreparedEdit::insert(&source, sid, &intent).unwrap();
603 let result = edit.commit_file(&path);
515 let transaction = ops::transaction(&source, "Author", ops).unwrap().unwrap();
516 let result = transaction.commit_file(&path);
604517 let written = onestore::read_file(&path).unwrap();
605 let repeated = edit.commit_file(&path).unwrap_err();
518 let repeated = transaction.commit_file(&path).unwrap_err();
606519 fs::remove_file(path).unwrap();
607520 result.unwrap();
608 assert_eq!(written, edit.as_bytes());
521 let mut expected = source.clone();
522 transaction.apply(&mut expected).unwrap();
523 assert_eq!(written, expected);
609524 assert_eq!(repeated.state, onestore::CommitState::NotCommitted);
610525 assert_eq!(repeated.error.kind(), std::io::ErrorKind::ResourceBusy);
611526}
612527
613528#[test]
614fn anchors_targets_serialized_identities_and_text_are_validated_before_publication() {
529fn anchors_targets_identities_and_text_are_validated_before_publication() {
615530 let source = onestore::create_section("invalid.one", "Original", "Author").unwrap();
616 let (sid, page, outline, paragraph, text) = targets(&source);
617 assert!(Insertion::outline(page, f32::NAN, 0.0, "Text", "Author").is_err());
618 assert!(Insertion::outline(page, 0.0, f32::INFINITY, "Text", "Author").is_err());
531 let (sid, page, outline, _, text) = targets(&source);
532 let refused = |source: &[u8], place, content: &str| {
533 ops::edited(source, insertion(sid, place, content, &[]).0).is_err()
534 };
535 assert!(refused(&source, Place::Outline(f32::NAN, 0.0), "Text"));
536 assert!(refused(&source, Place::Outline(0.0, f32::INFINITY), "Text"));
619537 for content in ["a\0b", "a\nb", "a\u{fffc}b"] {
620 assert!(Insertion::paragraph(outline, None, content, "Author").is_err());
538 assert!(refused(&source, Place::In(outline, None), content));
621539 }
622 assert!(Insertion::paragraph(outline, None, "Text", "a\0b").is_err());
623 for intent in [
624 Insertion::paragraph(outline, Some(text), "Text", "Author").unwrap(),
625 Insertion::paragraph(text, None, "Text", "Author").unwrap(),
626 Insertion::paragraph(page, None, "Text", "Author").unwrap(),
627 Insertion::outline(paragraph, 0.0, 0.0, "Text", "Author").unwrap(),
540 assert!(
541 ops::transaction(&source, "a\0b", insertion(sid, Place::In(outline, None), "Text", &[]).0)
542 .is_err()
543 );
544 for place in [
545 Place::In(outline, Some(text)),
546 Place::In(text, None),
547 Place::In(page, None),
628548 ] {
629 assert!(PreparedEdit::insert(&source, sid, &intent).is_err());
549 assert!(refused(&source, place, "Text"));
630550 }
631 let intent = Insertion::paragraph(outline, None, "Text", "Author").unwrap();
632 let created = PreparedEdit::insert(&source, sid, &intent).unwrap();
633 let before_missing =
634 Insertion::paragraph(outline, Some(intent.object()), "Anchored", "Author").unwrap();
635 assert!(PreparedEdit::insert(&source, sid, &before_missing).is_err());
636 assert!(PreparedEdit::insert(created.as_bytes(), sid, &before_missing).is_ok());
637 let mut encoded = serde_json::to_value(&intent).unwrap();
638 encoded["parent"] = serde_json::to_value(intent.object()).unwrap();
639 let collision: Insertion = serde_json::from_value(encoded).unwrap();
640 assert!(PreparedEdit::insert(created.as_bytes(), sid, &collision).is_err());
551 let (ops, object, _) = insertion(sid, Place::In(outline, None), "Text", &[]);
552 let created = ops::edited(&source, ops.clone()).unwrap();
553 let before_created = Place::In(outline, Some(object));
554 assert!(refused(&source, before_created, "Anchored"));
555 assert!(!refused(&created, before_created, "Anchored"));
556 // A new object's identity may not already be on the page.
557 assert!(ops::edited(&created, ops).is_err());
641558}
642559
643560#[test]
644561fn insertions_into_nested_paragraphs_and_native_table_cells_preserve_structure() {
645562 let source = onestore::create_section("nested.one", "Original", "Author").unwrap();
646563 let (sid, _, outline, paragraph, _) = targets(&source);
647 let child = Insertion::paragraph(paragraph, None, "Nested", "Author").unwrap();
648 let changed = PreparedEdit::insert(&source, sid, &child).unwrap();
649 let store = Store::parse(changed.as_bytes()).unwrap();
564 let mut nested = ops::paragraph("Nested");
565 nested.level = 2;
566 let child = nested.id;
567 let insert = PageOp::Insert {
568 container: paragraph,
569 before: None,
570 paragraphs: vec![nested],
571 };
572 let changed = ops::page_edited(&source, sid, vec![insert]).unwrap();
573 let store = Store::parse(&changed).unwrap();
650574 let index = RevisionIndex::parse(&store).unwrap();
651575 let document = Document::parse(&index).unwrap();
652576 let space = &document.spaces[&sid];
653577 let view = &space.revisions[&space.contexts[&ExGuid::default()]];
654578 assert_eq!(view.nodes[&outline].children, [paragraph]);
655 assert_eq!(view.nodes[&paragraph].children, [child.object()]);
579 assert_eq!(view.nodes[&paragraph].children, [child]);
656580 let source = include_bytes!(
657581 "../../../corpus/native/20260905-05/snapshots/07-table/notebook/synthetic.one"
658582 );
......@@ -669,34 +593,32 @@ fn insertions_into_nested_paragraphs_and_native_table_cells_preserve_structure()
669593 })
670594 })
671595 .unwrap();
672 let insertion = Insertion::paragraph(cell, None, "Added to cell", "Author").unwrap();
673 let changed = PreparedEdit::insert(source, sid, &insertion).unwrap();
674 current::current(changed.as_bytes());
596 let (ops, _, _) = insertion(sid, Place::In(cell, None), "Added to cell", &[]);
597 current::current(&ops::edited(source, ops).unwrap());
675598}
676599
677600#[test]
678601fn insertion_publication_faults_expose_only_complete_graphs_and_title_caches() {
679602 let original = onestore::create_section("atomic.one", "Original", "Author").unwrap();
680603 let (sid, _, _, _, text) = targets(&original);
681 let checkpoint = checkpoint::pending(&original, sid, text, 0x14001d7a);
604 let checkpoint = checkpoint::pending(&original, sid, text);
682605 for source in [
683606 original.as_slice(),
684607 include_bytes!("../../../corpus/append/round-01/tx-255/notebook/synthetic.one"),
685608 &checkpoint,
686609 ] {
687 let (sid, page, outline, paragraph, _) = targets(source);
688 for intent in [
689 Insertion::paragraph(outline, Some(paragraph), "First 🦀", "New author").unwrap(),
690 Insertion::outline(page, 0.0, 0.0, "First 🦀", "New author").unwrap(),
691 ] {
692 let intent = intent
693 .with_formatting(0..3, &[A::Bold(true), A::FontSize(18.0)])
694 .unwrap()
695 .with_formatting(6..8, &[A::Italic(true), A::Highlight(Some([255, 255, 0]))])
696 .unwrap();
697 let edit = PreparedEdit::insert(source, sid, &intent).unwrap();
610 let (sid, _, outline, paragraph, _) = targets(source);
611 for place in [Place::In(outline, Some(paragraph)), Place::Outline(0.0, 0.0)] {
612 let formats = [
613 (0..3, vec![A::Bold(true), A::FontSize(18.0)]),
614 (6..8, vec![A::Italic(true), A::Highlight(Some([255, 255, 0]))]),
615 ];
616 let (ops, _, _) = insertion(sid, place, "First 🦀", &formats);
617 let edit = ops::transaction(source, "New author", ops).unwrap().unwrap();
618 let mut written = source.to_vec();
619 edit.apply(&mut written).unwrap();
698620 let before = current::current(source);
699 let after = current::current(edit.as_bytes());
621 let after = current::current(&written);
700622 for write_limit in [17, 4096] {
701623 let disk = |fail_at| disk::Disk {
702624 visible: source.to_vec(),
......@@ -708,7 +630,7 @@ fn insertion_publication_faults_expose_only_complete_graphs_and_title_caches() {
708630 };
709631 let mut successful = disk(None);
710632 edit.commit(&mut successful).unwrap();
711 assert_eq!(successful.durable, edit.as_bytes());
633 assert_eq!(successful.durable, written);
712634 for at in 1..=successful.operation {
713635 let mut interrupted = disk(Some(at));
714636 let failure = edit.commit(&mut interrupted).unwrap_err();
crates/onestore/tests/outline.rs+29-51
......@@ -1,6 +1,7 @@
11use onestore::{
2 ExGuid, OutlineEdit as Edit, PreparedEdit, RevisionIndex, Store,
2 ExGuid, OutlineEdit as Edit, RevisionIndex, Store,
33 document::{Document, Kind},
4 op::PageOp,
45};
56use std::collections::BTreeSet;
67
......@@ -8,6 +9,8 @@ use std::collections::BTreeSet;
89mod current;
910#[path = "support/disk.rs"]
1011mod disk;
12#[path = "support/ops.rs"]
13mod ops;
1114
1215const SOURCE: &[u8] = include_bytes!("../../../corpus/outline-edit/before/notebook/synthetic.one");
1316
......@@ -62,7 +65,7 @@ fn geometry_and_expansion_preserve_unrelated_properties_objects_and_history() {
6265 for (name, sid, object, edit) in cases(&document) {
6366 let restored = serde_json::from_value(serde_json::to_value(edit).unwrap()).unwrap();
6467 assert_eq!(edit, restored);
65 let prepared = PreparedEdit::outline(SOURCE, sid, object, restored).unwrap();
68 let prepared = ops::page_op(SOURCE, sid, PageOp::Outline { object, edit: restored }).unwrap();
6669 let after_store = Store::parse(prepared.as_bytes()).unwrap();
6770 assert!(after_store.checksum_mismatches.is_empty());
6871 let after_index = RevisionIndex::parse(&after_store).unwrap();
......@@ -139,7 +142,7 @@ fn geometry_and_expansion_preserve_unrelated_properties_objects_and_history() {
139142 assert_eq!(old_space.labels, after_index.spaces[space_id].labels);
140143 }
141144 }
142 let again = PreparedEdit::outline(prepared.as_bytes(), sid, object, edit).unwrap();
145 let again = ops::page_op(prepared.as_bytes(), sid, PageOp::Outline { object, edit }).unwrap();
143146 assert_eq!(again.as_bytes(), prepared.as_bytes());
144147 }
145148}
......@@ -159,20 +162,15 @@ fn invalid_geometry_and_non_outline_targets_are_rejected() {
159162 user_set: true,
160163 },
161164 ] {
162 assert!(PreparedEdit::outline(SOURCE, sid, object, edit).is_err());
165 assert!(ops::page_op(SOURCE, sid, PageOp::Outline { object, edit }).is_err());
163166 }
164167 }
165168 for points in [-1.0, 0.0, 35.999] {
166169 assert!(
167 PreparedEdit::outline(
168 SOURCE,
169 sid,
170 object,
171 Edit::Width {
170 ops::page_op(SOURCE, sid, PageOp::Outline { object, edit: Edit::Width {
172171 points,
173172 user_set: false
174 }
175 )
173 } })
176174 .is_err()
177175 );
178176 }
......@@ -181,7 +179,7 @@ fn invalid_geometry_and_non_outline_targets_are_rejected() {
181179 for (id, node) in &view.nodes {
182180 if !matches!(node.kind, Kind::Outline { .. }) {
183181 assert!(
184 PreparedEdit::outline(SOURCE, sid, *id, Edit::Position { x: 72.0, y: 72.0 })
182 ops::page_op(SOURCE, sid, PageOp::Outline { object: *id, edit: Edit::Position { x: 72.0, y: 72.0 } })
185183 .is_err()
186184 );
187185 }
......@@ -197,20 +195,20 @@ fn invalid_geometry_and_non_outline_targets_are_rejected() {
197195 },
198196 Edit::Collapsed(true),
199197 ] {
200 assert!(PreparedEdit::outline(SOURCE, sid, child, edit).is_err());
198 assert!(ops::page_op(SOURCE, sid, PageOp::Outline { object: child, edit }).is_err());
201199 }
202200 }
203201 }
204202 if !matches!(node.kind, Kind::Paragraph { .. }) {
205 assert!(PreparedEdit::outline(SOURCE, sid, *id, Edit::Collapsed(true)).is_err());
203 assert!(ops::page_op(SOURCE, sid, PageOp::Outline { object: *id, edit: Edit::Collapsed(true) }).is_err());
206204 }
207205 }
208206 assert!(
209 PreparedEdit::outline(SOURCE, ExGuid::default(), object, Edit::Collapsed(true))
207 ops::page_op(SOURCE, ExGuid::default(), PageOp::Outline { object, edit: Edit::Collapsed(true) })
210208 .is_err()
211209 );
212210 assert!(
213 PreparedEdit::outline(SOURCE, sid, ExGuid::default(), Edit::Collapsed(true)).is_err()
211 ops::page_op(SOURCE, sid, PageOp::Outline { object: ExGuid::default(), edit: Edit::Collapsed(true) }).is_err()
214212 );
215213 }
216214}
......@@ -228,15 +226,10 @@ fn resizing_a_native_reserved_width_preserves_content() {
228226 let space = &document.spaces[&sid];
229227 let before = &space.revisions[&space.contexts[&ExGuid::default()]];
230228 assert!(before.nodes[&object].layout.reserved_width.is_some());
231 let edit = PreparedEdit::outline(
232 source,
233 sid,
234 object,
235 Edit::Width {
229 let edit = ops::page_op(source, sid, PageOp::Outline { object, edit: Edit::Width {
236230 points: 144.0,
237231 user_set: true,
238 },
239 )
232 } })
240233 .unwrap();
241234 let after_store = Store::parse(edit.as_bytes()).unwrap();
242235 let after_index = RevisionIndex::parse(&after_store).unwrap();
......@@ -270,12 +263,8 @@ fn outline_movement_updates_both_automatic_title_fields_without_changing_content
270263 let index = RevisionIndex::parse(&store).unwrap();
271264 let document = Document::parse(&index).unwrap();
272265 let (sid, page) = document.pages().unwrap()[0];
273 let insertion =
274 onestore::Insertion::outline(page, 144.0, 144.0, "Second 🦋 é", "Author").unwrap();
275 let source = PreparedEdit::insert(&source, sid, &insertion)
276 .unwrap()
277 .as_bytes()
278 .to_vec();
266 let (add, insertion, _) = ops::new_outline(144.0, 144.0, "Second 🦋 é");
267 let source = ops::page_op(&source, sid, add).unwrap().image;
279268 let store = Store::parse(&source).unwrap();
280269 let index = RevisionIndex::parse(&store).unwrap();
281270 let document = Document::parse(&index).unwrap();
......@@ -288,7 +277,7 @@ fn outline_movement_updates_both_automatic_title_fields_without_changing_content
288277 (144.0, 36.0, "First"),
289278 (0.0, 144.0, "First"),
290279 ] {
291 let edit = PreparedEdit::outline(&source, sid, insertion.object(), Edit::Position { x, y })
280 let edit = ops::page_op(&source, sid, PageOp::Outline { object: insertion, edit: Edit::Position { x, y } })
292281 .unwrap();
293282 let saved = Store::parse(edit.as_bytes()).unwrap();
294283 let saved_index = RevisionIndex::parse(&saved).unwrap();
......@@ -308,7 +297,7 @@ fn outline_movement_updates_both_automatic_title_fields_without_changing_content
308297 after.nodes.keys().collect::<Vec<_>>()
309298 );
310299 for (oid, node) in &before.nodes {
311 if ![page, metadata, insertion.object()].contains(oid) {
300 if ![page, metadata, insertion].contains(oid) {
312301 assert_eq!(
313302 serde_json::to_value(node).unwrap(),
314303 serde_json::to_value(&after.nodes[oid]).unwrap()
......@@ -320,12 +309,7 @@ fn outline_movement_updates_both_automatic_title_fields_without_changing_content
320309 format!("{:?}", saved_index.resolve(sid, rid).unwrap())
321310 );
322311 assert_eq!(
323 PreparedEdit::outline(
324 edit.as_bytes(),
325 sid,
326 insertion.object(),
327 Edit::Position { x, y }
328 )
312 ops::page_op(edit.as_bytes(), sid, PageOp::Outline { object: insertion, edit: Edit::Position { x, y } })
329313 .unwrap()
330314 .as_bytes(),
331315 edit.as_bytes()
......@@ -366,7 +350,7 @@ fn native_automatic_and_explicit_titles_follow_outline_movement() {
366350 72.0
367351 },
368352 };
369 candidate = PreparedEdit::outline(&candidate, sid, outline, change)
353 candidate = ops::page_op(&candidate, sid, PageOp::Outline { object: outline, edit: change })
370354 .unwrap()
371355 .as_bytes()
372356 .to_vec();
......@@ -418,11 +402,8 @@ fn outline_publication_interruptions_reopen_as_complete_old_or_new_layout() {
418402 .find(|id| matches!(view.nodes[id].kind, Kind::Outline { .. }))
419403 .unwrap();
420404 let paragraph = view.nodes[&outline].children[0];
421 let second = onestore::Insertion::outline(page, 144.0, 36.0, "Second", "Author").unwrap();
422 let source = PreparedEdit::insert(&source, sid, &second)
423 .unwrap()
424 .as_bytes()
425 .to_vec();
405 let (add, _, _) = ops::new_outline(144.0, 36.0, "Second");
406 let source = ops::page_op(&source, sid, add).unwrap().image;
426407 for (object, operation) in [
427408 (outline, Edit::Position { x: 216.0, y: 72.0 }),
428409 (
......@@ -434,7 +415,7 @@ fn outline_publication_interruptions_reopen_as_complete_old_or_new_layout() {
434415 ),
435416 (paragraph, Edit::Collapsed(true)),
436417 ] {
437 let edit = PreparedEdit::outline(&source, sid, object, operation).unwrap();
418 let edit = ops::page_op(&source, sid, PageOp::Outline { object, edit: operation }).unwrap();
438419 let before = current::current(&source);
439420 let after = current::current(edit.as_bytes());
440421 for write_limit in [17, 4096] {
......@@ -481,11 +462,8 @@ fn repeated_geometry_changes_and_expansion_match_an_independent_model() {
481462 .unwrap();
482463 let paragraph = view.nodes[&outline].children[0];
483464 let text = view.nodes[&paragraph].content[0];
484 let second = onestore::Insertion::outline(page, 144.0, 36.0, "Second", "Author").unwrap();
485 let original = PreparedEdit::insert(&original, sid, &second)
486 .unwrap()
487 .as_bytes()
488 .to_vec();
465 let (add, _, _) = ops::new_outline(144.0, 36.0, "Second");
466 let original = ops::page_op(&original, sid, add).unwrap().image;
489467 for seed in 1..=16_u64 {
490468 let mut rng = seed;
491469 let mut source = original.clone();
......@@ -520,7 +498,7 @@ fn repeated_geometry_changes_and_expansion_match_an_independent_model() {
520498 (paragraph, Edit::Collapsed(rng & 1 != 0))
521499 }
522500 };
523 source = PreparedEdit::outline(&source, sid, object, edit)
501 source = ops::page_op(&source, sid, PageOp::Outline { object, edit })
524502 .unwrap()
525503 .as_bytes()
526504 .to_vec();
......@@ -574,7 +552,7 @@ fn export_native_outline_candidates() {
574552 let document = Document::parse(&index).unwrap();
575553 let mut bytes = SOURCE.to_vec();
576554 for (_, sid, object, edit) in cases(&document) {
577 bytes = PreparedEdit::outline(&bytes, sid, object, edit)
555 bytes = ops::page_op(&bytes, sid, PageOp::Outline { object, edit })
578556 .unwrap()
579557 .as_bytes()
580558 .to_vec();
crates/onestore/tests/page.rs+94-24
......@@ -1,6 +1,7 @@
11use onestore::{
2 ExGuid, Insertion, PageCreation, PreparedEdit, RevisionIndex, Store,
2 ExGuid, PageCreation, RevisionIndex, Store,
33 document::{Document, Kind},
4 op::{PageOp, SectionOp},
45 page::{Page, PageObject},
56};
67
......@@ -8,6 +9,8 @@ use onestore::{
89mod current;
910#[path = "support/disk.rs"]
1011mod disk;
12#[path = "support/ops.rs"]
13mod ops;
1114
1215const SOURCE: &[u8] =
1316 include_bytes!("../../../corpus/page-lifecycle/03-renamed/notebook/Lifecycle.one");
......@@ -28,7 +31,7 @@ fn native_section_accepts_empty_and_titled_pages_with_preserved_history() {
2831 let serialized = serde_json::to_vec(&intent).unwrap();
2932 let retained = serde_json::from_slice(&serialized).unwrap();
3033 assert_eq!(intent, retained);
31 let prepared = PreparedEdit::create_page(&source, &retained).unwrap();
34 let prepared = ops::section_op(&source, SectionOp::Create(retained.clone())).unwrap();
3235 let store = Store::parse(prepared.as_bytes()).unwrap();
3336 assert_eq!(
3437 store.header.transaction_count,
......@@ -78,7 +81,7 @@ fn native_section_accepts_empty_and_titled_pages_with_preserved_history() {
7881 assert_eq!(space.labels, index.spaces[sid].labels);
7982 }
8083 }
81 assert!(PreparedEdit::create_page(prepared.as_bytes(), &intent).is_err());
84 assert!(ops::section_op(prepared.as_bytes(), SectionOp::Create(intent.clone())).is_err());
8285 source = prepared.as_bytes().to_vec();
8386 }
8487 if let Some(output) = std::env::var_os("ONESTORE_PAGE_CREATION_OUTPUT") {
......@@ -92,13 +95,12 @@ fn created_pages_support_title_edits_and_body_insertion() {
9295 let mut source = onestore::create_section("pages.one", "Original", "Author").unwrap();
9396 for title in [None, Some(""), Some("Explicit 🦋 é")] {
9497 let intent = PageCreation::new(None, title, "Author").unwrap();
95 source = PreparedEdit::create_page(&source, &intent)
98 source = ops::section_op(&source, SectionOp::Create(intent.clone()))
9699 .unwrap()
97100 .as_bytes()
98101 .to_vec();
99 let insertion =
100 Insertion::outline(intent.object(), 36.0, 36.0, "Body 🦀 é", "Author").unwrap();
101 source = PreparedEdit::insert(&source, intent.space(), &insertion)
102 let (add, ..) = ops::new_outline(36.0, 36.0, "Body 🦀 é");
103 source = ops::page_op(&source, intent.space(), add)
102104 .unwrap()
103105 .as_bytes()
104106 .to_vec();
......@@ -115,13 +117,7 @@ fn created_pages_support_title_edits_and_body_insertion() {
115117 Some(title.filter(|s| !s.is_empty()).unwrap_or("Body 🦀 é"))
116118 );
117119 if let Some(text) = intent.title_object() {
118 source = PreparedEdit::text(
119 &source,
120 intent.space(),
121 text,
122 0..title.unwrap().encode_utf16().count() as u32,
123 "Renamed",
124 )
120 source = ops::page_op(&source, intent.space(), PageOp::Text { text, range: 0..title.unwrap().encode_utf16().count() as u32, with: "Renamed".into() })
125121 .unwrap()
126122 .as_bytes()
127123 .to_vec();
......@@ -149,13 +145,13 @@ fn created_body_outlines_carry_the_indentation_table_onenote_writes() {
149145 let [first] = document.active(space).unwrap().nodes[&page].children[..] else {
150146 panic!()
151147 };
152 let insertion = Insertion::outline(page, 72.0, 144.0, "Second", "Author").unwrap();
153 let written = PreparedEdit::insert(&source, space, &insertion).unwrap();
148 let (add, insertion, _) = ops::new_outline(72.0, 144.0, "Second");
149 let written = ops::page_op(&source, space, add).unwrap();
154150 let store = Store::parse(written.as_bytes()).unwrap();
155151 let index = RevisionIndex::parse(&store).unwrap();
156152 let document = Document::parse(&index).unwrap();
157153 let view = document.active(space).unwrap();
158 for outline in [first, insertion.object()] {
154 for outline in [first, insertion] {
159155 let Kind::Outline { indents } = &view.nodes[&outline].kind else {
160156 panic!()
161157 };
......@@ -167,7 +163,7 @@ fn created_body_outlines_carry_the_indentation_table_onenote_writes() {
167163fn a_created_title_outline_is_stored_and_read_as_onenote_stores_it() {
168164 let source = onestore::create_section("pages.one", "Original", "Author").unwrap();
169165 let intent = PageCreation::new(None, Some("Title"), "Author").unwrap();
170 let prepared = PreparedEdit::create_page(&source, &intent).unwrap();
166 let prepared = ops::section_op(&source, SectionOp::Create(intent.clone())).unwrap();
171167 let store = Store::parse(prepared.as_bytes()).unwrap();
172168 let index = RevisionIndex::parse(&store).unwrap();
173169 let document = Document::parse(&index).unwrap();
......@@ -222,11 +218,11 @@ fn invalid_page_intents_and_nonleading_anchors_reject_before_publication() {
222218 let index = RevisionIndex::parse(&store).unwrap();
223219 let pages = Document::parse(&index).unwrap().pages().unwrap();
224220 let intent = PageCreation::new(Some(pages[4].0), Some("Child"), "Author").unwrap();
225 assert!(PreparedEdit::create_page(source, &intent).is_err());
221 assert!(ops::section_op(source, SectionOp::Create(intent.clone())).is_err());
226222 let mut value = serde_json::to_value(PageCreation::new(None, None, "Author").unwrap()).unwrap();
227223 value["guid"] = serde_json::to_value([0_u8; 16]).unwrap();
228224 let invalid: PageCreation = serde_json::from_value(value.clone()).unwrap();
229 assert!(PreparedEdit::create_page(source, &invalid).is_err());
225 assert!(ops::section_op(source, SectionOp::Create(invalid.clone())).is_err());
230226 value["unknown"] = true.into();
231227 assert!(serde_json::from_value::<PageCreation>(value).is_err());
232228}
......@@ -235,7 +231,7 @@ fn invalid_page_intents_and_nonleading_anchors_reject_before_publication() {
235231fn new_space_and_section_entry_publish_as_one_complete_action() {
236232 let source = onestore::create_section("pages.one", "Original", "Author").unwrap();
237233 let intent = PageCreation::new(None, Some("New 🦋 é"), "Author").unwrap();
238 let prepared = PreparedEdit::create_page(&source, &intent).unwrap();
234 let prepared = ops::section_op(&source, SectionOp::Create(intent.clone())).unwrap();
239235 let old = current::current(&source);
240236 let new = current::current(prepared.as_bytes());
241237 assert_eq!(new.len(), old.len() + 1);
......@@ -285,7 +281,7 @@ fn repeated_page_creation_crosses_root_fragments_counters_and_section_checkpoint
285281 "Stress author",
286282 )
287283 .unwrap();
288 let prepared = PreparedEdit::create_page(&source, &intent).unwrap();
284 let prepared = ops::section_op(&source, SectionOp::Create(intent.clone())).unwrap();
289285 let mut unpublished = prepared.as_bytes().to_vec();
290286 unpublished[96..100].copy_from_slice(&source[96..100]);
291287 let old_store = Store::parse(&unpublished).unwrap();
......@@ -363,7 +359,7 @@ fn native_changes_on_created_pages_accept_rust_followups() {
363359 let mut source = initial.to_vec();
364360 for (sid, title, body) in targets {
365361 for (object, prefix) in [(body, "Rust + "), (title, "Reviewed ")] {
366 source = PreparedEdit::text(&source, sid, object, 0..0, prefix)
362 source = ops::page_op(&source, sid, PageOp::Text { text: object, range: 0..0, with: prefix.into() })
367363 .unwrap()
368364 .as_bytes()
369365 .to_vec();
......@@ -400,7 +396,7 @@ fn page_creation_preserves_native_features_and_file_data() {
400396 let old = RevisionIndex::parse(&before).unwrap();
401397 let original = Document::parse(&old).unwrap();
402398 let intent = PageCreation::new(None, Some("Preserved features"), "Author").unwrap();
403 let prepared = PreparedEdit::create_page(source, &intent).unwrap();
399 let prepared = ops::section_op(source, SectionOp::Create(intent.clone())).unwrap();
404400 let after = Store::parse(prepared.as_bytes()).unwrap();
405401 let new = RevisionIndex::parse(&after).unwrap();
406402 let document = Document::parse(&new).unwrap();
......@@ -427,3 +423,77 @@ fn page_creation_preserves_native_features_and_file_data() {
427423 }
428424 }
429425}
426
427/// A page created into an empty section as OneNote 2010 creates one (its title's date and
428/// time fields, then a colour) reads back with both; a page kept from elsewhere keeps its
429/// identity and creation time.
430#[test]
431fn dated_pages_in_an_empty_section_read_back_with_their_date_and_colour() {
432 let source = onestore::create_empty_section("New Section 1.one", Some(0x00e4a88a)).unwrap();
433 let store = Store::parse(&source).unwrap();
434 let index = RevisionIndex::parse(&store).unwrap();
435 index.validate_current().unwrap();
436 assert!(Document::parse(&index).unwrap().pages().unwrap().is_empty());
437 let creation = PageCreation::new(None, Some(""), "Rust Author")
438 .unwrap()
439 .dated("Sunday, September 27, 2026", "2:23 AM")
440 .unwrap();
441 let retained: PageCreation =
442 serde_json::from_slice(&serde_json::to_vec(&creation).unwrap()).unwrap();
443 assert_eq!(retained, creation);
444 let space = creation.space();
445 let identity = [7; 16];
446 let kept = PageCreation::new(None, Some("Kept"), "Rust Author")
447 .unwrap()
448 .keeping(identity, 133_700_000_001_234_567)
449 .unwrap();
450 let image = ops::edited(
451 &source,
452 vec![
453 onestore::op::Op::Section(SectionOp::Create(creation.clone())),
454 onestore::op::Op::Page {
455 space,
456 op: PageOp::Color(Some(0x00f2f9d4)),
457 },
458 onestore::op::Op::Section(SectionOp::Create(kept.clone())),
459 ],
460 )
461 .unwrap();
462 let store = Store::parse(&image).unwrap();
463 let index = RevisionIndex::parse(&store).unwrap();
464 index.validate_current().unwrap();
465 let document = Document::parse(&index).unwrap();
466 let page = Page::from_space(&document, space).unwrap();
467 assert_eq!(page.color, Some(0x00f2f9d4));
468 assert_eq!(page.created, Some(creation.created()));
469 let [PageObject::Title(title)] = &page.objects[..] else {
470 panic!("{:?}", page.objects)
471 };
472 let shown: Vec<_> = title.outlines[1]
473 .paragraphs
474 .iter()
475 .map(|paragraph| {
476 let text = paragraph.text().unwrap();
477 (text.date_field, text.text.text().to_owned())
478 })
479 .collect();
480 assert_eq!(title.date, Some(title.outlines[1].id));
481 assert_eq!(
482 shown,
483 [
484 // OneNote's own new pages read the same: the fields' order names their roles.
485 (None, "Sunday, September 27, 2026".to_owned()),
486 (None, "2:23 AM".to_owned()),
487 ]
488 );
489 let kept_page = Page::from_space(&document, kept.space()).unwrap();
490 assert_eq!(kept_page.identity, Some(identity));
491 assert_eq!(kept_page.created, Some(133_700_000_001_234_567));
492 assert_eq!(kept_page.color, None);
493 // Clearing the colour stores OneNote's "No color".
494 let cleared = ops::page_edited(&image, space, vec![PageOp::Color(None)]).unwrap();
495 let store = Store::parse(&cleared).unwrap();
496 let index = RevisionIndex::parse(&store).unwrap();
497 let document = Document::parse(&index).unwrap();
498 assert_eq!(Page::from_space(&document, space).unwrap().color, None);
499}
crates/onestore/tests/page_attachments.rs+16-15
......@@ -1,5 +1,7 @@
1#[path = "support/ops.rs"]
2mod ops;
13use onestore::{
2 ExGuid, PreparedEdit, RevisionIndex, Store,
4 ExGuid, RevisionIndex, Store,
35 document::{Document, Format},
46 page::{
57 Attachment, Page, PageObject, PageParagraph, ParagraphContent, TextObject, text::new_id,
......@@ -9,7 +11,6 @@ use std::sync::Arc;
911
1012const NATIVE: &[u8] =
1113 include_bytes!("../../../corpus/native/cold-05-06-attachment/notebook/synthetic.one");
12const AUTHOR: &str = "Attachment author";
1314const PAYLOAD: &[u8] = b"Rust wrote this attachment.\n";
1415
1516fn first_page(bytes: &[u8]) -> (ExGuid, Page) {
......@@ -117,8 +118,8 @@ fn insert_attachment(section: &str, preview: Option<Arc<[u8]>>) -> Vec<u8> {
117118 });
118119 body_paragraphs(&mut after).push(holder.clone());
119120 body_paragraphs(&mut after).push(plain_paragraph(&template, "After the file"));
120 let written = PreparedEdit::page(&source, space, &after, AUTHOR).unwrap();
121 let stored = page_in(written.as_bytes(), space);
121 let written = ops::saved(&source, space, &after).unwrap();
122 let stored = page_in(written.as_slice(), space);
122123 let mut expected = after.clone();
123124 expected.title = stored.title.clone();
124125 assert_eq!(stored, expected);
......@@ -129,16 +130,16 @@ fn insert_attachment(section: &str, preview: Option<Arc<[u8]>>) -> Vec<u8> {
129130 assert_eq!(stored_attachment.bytes.as_deref(), Some(PAYLOAD));
130131 assert_eq!(stored_attachment.preview, preview);
131132 assert_eq!(
132 PreparedEdit::page(written.as_bytes(), space, &stored, AUTHOR)
133 ops::saved(written.as_slice(), space, &stored)
133134 .unwrap()
134 .as_bytes(),
135 written.as_bytes()
135 .as_slice(),
136 written.as_slice()
136137 );
137138 let mut removed = stored.clone();
138139 body_paragraphs(&mut removed).retain(|p| p.id != holder.id);
139 let again = PreparedEdit::page(written.as_bytes(), space, &removed, AUTHOR).unwrap();
140 assert!(attachments(&page_in(again.as_bytes(), space)).is_empty());
141 written.as_bytes().to_vec()
140 let again = ops::saved(written.as_slice(), space, &removed).unwrap();
141 assert!(attachments(&page_in(again.as_slice(), space)).is_empty());
142 written.as_slice().to_vec()
142143}
143144
144145/// `ONESTORE_ATTACHMENT_EXPORT` names a directory receiving both candidates for a cold reopen:
......@@ -186,7 +187,7 @@ fn attachments_need_a_file_name_and_stored_ones_are_renamed_in_place() {
186187 recording: None,
187188 });
188189 body_paragraphs(&mut after).push(holder);
189 assert!(PreparedEdit::page(&source, space, &after, AUTHOR).is_err());
190 assert!(ops::saved(&source, space, &after).is_err());
190191 let (space, native) = first_page(NATIVE);
191192 let mut renamed = native.clone();
192193 let mut found = 0;
......@@ -202,8 +203,8 @@ fn attachments_need_a_file_name_and_stored_ones_are_renamed_in_place() {
202203 }
203204 }
204205 assert_eq!(found, 1);
205 let written = PreparedEdit::page(NATIVE, space, &renamed, AUTHOR).unwrap();
206 let stored = page_in(written.as_bytes(), space);
206 let written = ops::saved(NATIVE, space, &renamed).unwrap();
207 let stored = page_in(written.as_slice(), space);
207208 let (before, after) = (attachments(&native)[0], attachments(&stored)[0]);
208209 assert_eq!(after.filename, "renamed \u{1f980}.txt");
209210 assert_eq!(
......@@ -216,8 +217,8 @@ fn attachments_need_a_file_name_and_stored_ones_are_renamed_in_place() {
216217 if let Some(directory) = std::env::var_os("ONESTORE_ATTACHMENT_RENAME_EXPORT") {
217218 let directory = std::path::PathBuf::from(directory);
218219 std::fs::create_dir_all(&directory).unwrap();
219 std::fs::write(directory.join("synthetic.one"), written.as_bytes()).unwrap();
220 let file_id = Store::parse(written.as_bytes()).unwrap().header.file_id;
220 std::fs::write(directory.join("synthetic.one"), written.as_slice()).unwrap();
221 let file_id = Store::parse(written.as_slice()).unwrap().header.file_id;
221222 std::fs::write(
222223 directory.join("Open Notebook.onetoc2"),
223224 onestore::create_table_of_contents(
crates/onestore/tests/page_deletion.rs+23-13
......@@ -1,6 +1,7 @@
11use onestore::{
2 ExGuid, FileDataReference, ObjectData, PreparedEdit, PropertySets, RevisionIndex, Store,
2 ExGuid, FileDataReference, ObjectData, PropertySets, RevisionIndex, Store,
33 document::{Document, Kind},
4 op::{PageOp, SectionOp},
45};
56use std::collections::BTreeSet;
67
......@@ -130,7 +131,7 @@ fn removal_matches_all_native_cases_and_retains_every_old_revision() {
130131 .into_iter()
131132 .map(|ordinal| pages[ordinal].0)
132133 .collect();
133 let prepared = PreparedEdit::delete_pages_permanently(&original, &removed).unwrap();
134 let prepared = ops::section_op(&original, SectionOp::Delete(removed.to_vec())).unwrap();
134135 if let Some(output) = std::env::var_os("ONESTORE_PAGE_REMOVAL_OUTPUT") {
135136 let output = std::path::Path::new(&output);
136137 assert!(output.is_absolute());
......@@ -140,7 +141,7 @@ fn removal_matches_all_native_cases_and_retains_every_old_revision() {
140141 std::fs::write(case.join("Lifecycle.one"), prepared.as_bytes()).unwrap();
141142 }
142143 verify(&original, prepared.as_bytes(), &native, &removed);
143 assert!(PreparedEdit::delete_pages_permanently(prepared.as_bytes(), &removed).is_err());
144 assert!(ops::section_op(prepared.as_bytes(), SectionOp::Delete(removed.to_vec())).is_err());
144145 }
145146}
146147
......@@ -160,22 +161,22 @@ fn removal_rejects_duplicate_missing_and_non_page_spaces_without_publication() {
160161 }],
161162 vec![pages[0].0, index.root],
162163 ] {
163 assert!(PreparedEdit::delete_pages_permanently(SOURCE, &selected).is_err());
164 assert!(ops::section_op(SOURCE, SectionOp::Delete(selected.to_vec())).is_err());
164165 }
165166 assert_eq!(
166 PreparedEdit::delete_pages_permanently(SOURCE, &[])
167 ops::section_op(SOURCE, SectionOp::Delete([].to_vec()))
167168 .unwrap()
168169 .as_bytes(),
169170 SOURCE
170171 );
171 let written = PreparedEdit::delete_pages_permanently(SOURCE, &[pages[0].0]).unwrap();
172 let written = ops::section_op(SOURCE, SectionOp::Delete([pages[0].0].to_vec())).unwrap();
172173 assert!(
173 PreparedEdit::delete_pages_permanently(written.as_bytes(), &[pages[1].0, pages[0].0])
174 ops::section_op(written.as_bytes(), SectionOp::Delete([pages[1].0, pages[0].0].to_vec()))
174175 .is_err()
175176 );
176177}
177178
178use page_schedule::{current, disk};
179use page_schedule::{current, disk, ops};
179180
180181/// OneNote created a page in each section after Rust permanently removed pages from it;
181182/// the surviving pages keep their revisions through the native save and a cold reopen.
......@@ -267,7 +268,7 @@ fn tombstones_and_first_page_promotion_survive_each_storage_interruption() {
267268 .take(if all { pages.len() } else { 1 })
268269 .map(|page| page.0)
269270 .collect();
270 let prepared = PreparedEdit::delete_pages_permanently(source, &removed).unwrap();
271 let prepared = ops::section_op(source, SectionOp::Delete(removed.to_vec())).unwrap();
271272 let old = current::current(source);
272273 let new = current::current(prepared.as_bytes());
273274 assert_ne!(old, new);
......@@ -331,8 +332,17 @@ fn stale_removal_cannot_delete_a_newer_page_edit() {
331332 _ => None,
332333 })
333334 .unwrap();
334 let removal = PreparedEdit::delete_pages_permanently(SOURCE, &[sid]).unwrap();
335 let remote = onestore::replace_text(SOURCE, sid, text, 0..0, "New remote content ").unwrap();
335 let removal = ops::section_op(SOURCE, SectionOp::Delete([sid].to_vec())).unwrap();
336 let remote = ops::page_edited(
337 SOURCE,
338 sid,
339 vec![PageOp::Text {
340 text,
341 range: 0..0,
342 with: "New remote content ".into(),
343 }],
344 )
345 .unwrap();
336346 let mut disk = disk::Disk {
337347 visible: remote.clone(),
338348 durable: remote.clone(),
......@@ -380,7 +390,7 @@ fn external_payloads_and_their_historical_references_survive_file_removal() {
380390 let pages = Document::parse(&index).unwrap().pages().unwrap();
381391 assert_eq!(pages.len(), 3);
382392 let selected: Vec<_> = pages.iter().map(|page| page.0).collect();
383 let prepared = PreparedEdit::delete_pages_permanently(&original, &selected).unwrap();
393 let prepared = ops::section_op(&original, SectionOp::Delete(selected.to_vec())).unwrap();
384394 let output = std::env::var_os("ONESTORE_PAGE_REMOVAL_EXTERNAL_OUTPUT");
385395 let root = output
386396 .as_ref()
......@@ -409,7 +419,7 @@ fn external_payloads_and_their_historical_references_survive_file_removal() {
409419 assert_eq!(payloads.len(), 3);
410420 let path = root.join("synthetic.one");
411421 std::fs::write(&path, &original).unwrap();
412 prepared.commit_file(&path).unwrap();
422 prepared.transaction.as_ref().unwrap().commit_file(&path).unwrap();
413423 let written = onestore::read_file(&path).unwrap();
414424 assert_eq!(written, prepared.as_bytes());
415425 let current_store = Store::parse(&written).unwrap();
crates/onestore/tests/page_images.rs+39-38
......@@ -1,5 +1,7 @@
1#[path = "support/ops.rs"]
2mod ops;
13use onestore::{
2 ExGuid, PreparedEdit, RevisionIndex, Store,
4 ExGuid, RevisionIndex, Store,
35 document::{Document, Format, Layout},
46 page::{Image, Page, PageObject, PageParagraph, ParagraphContent, TextObject, text::new_id},
57};
......@@ -11,7 +13,6 @@ const NATIVE_RESIZE: &[u8] =
1113 include_bytes!("../../../corpus/picture-edit/native-resize/notebook/pictures.one");
1214const NATIVE_PAGE_LEVEL: &[u8] =
1315 include_bytes!("../../../corpus/picture-edit/native-page-level/notebook/pictures.one");
14const AUTHOR: &str = "Picture author";
1516/// A one-pixel PNG.
1617const PNG: &[u8] = &[
1718 0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a, 0x00, 0x00, 0x00, 0x0d, 0x49, 0x48, 0x44, 0x52,
......@@ -120,8 +121,8 @@ fn a_picture_inserted_on_a_fresh_page_reads_back_and_can_be_removed() {
120121 holder.content = ParagraphContent::Image(image.clone());
121122 body_paragraphs(&mut after).push(holder.clone());
122123 body_paragraphs(&mut after).push(plain_paragraph(&template, "After the picture"));
123 let written = PreparedEdit::page(&source, space, &after, AUTHOR).unwrap();
124 let stored = page_in(written.as_bytes(), space);
124 let written = ops::saved(&source, space, &after).unwrap();
125 let stored = page_in(written.as_slice(), space);
125126 let mut expected = after.clone();
126127 expected.title = stored.title.clone();
127128 assert_eq!(stored, expected);
......@@ -131,16 +132,16 @@ fn a_picture_inserted_on_a_fresh_page_reads_back_and_can_be_removed() {
131132 };
132133 assert_eq!(picture.bytes.as_deref(), Some(PNG));
133134 assert_eq!(
134 PreparedEdit::page(written.as_bytes(), space, &stored, AUTHOR)
135 ops::saved(written.as_slice(), space, &stored)
135136 .unwrap()
136 .as_bytes(),
137 written.as_bytes()
137 .as_slice(),
138 written.as_slice()
138139 );
139140 if let Some(directory) = std::env::var_os("ONESTORE_IMAGE_EXPORT") {
140141 let directory = std::path::PathBuf::from(directory);
141142 std::fs::create_dir(&directory).unwrap();
142 std::fs::write(directory.join("pictures.one"), written.as_bytes()).unwrap();
143 let written_store = Store::parse(written.as_bytes()).unwrap();
143 std::fs::write(directory.join("pictures.one"), written.as_slice()).unwrap();
144 let written_store = Store::parse(written.as_slice()).unwrap();
144145 std::fs::write(
145146 directory.join("Open Notebook.onetoc2"),
146147 onestore::create_table_of_contents(
......@@ -153,8 +154,8 @@ fn a_picture_inserted_on_a_fresh_page_reads_back_and_can_be_removed() {
153154 }
154155 let mut removed = stored.clone();
155156 body_paragraphs(&mut removed).retain(|p| p.id != holder.id);
156 let again = PreparedEdit::page(written.as_bytes(), space, &removed, AUTHOR).unwrap();
157 let stored = page_in(again.as_bytes(), space);
157 let again = ops::saved(written.as_slice(), space, &removed).unwrap();
158 let stored = page_in(again.as_slice(), space);
158159 assert!(pictures(&stored).is_empty());
159160 assert_eq!(body_paragraphs(&mut stored.clone()).len(), 2);
160161}
......@@ -187,20 +188,20 @@ fn a_native_picture_is_resized_and_described_then_reset() {
187188 ..Default::default()
188189 };
189190 resize(&mut resized, layout, Some("Resized in Rust"));
190 let written = PreparedEdit::page(NATIVE, space, &resized, AUTHOR).unwrap();
191 let stored = page_in(written.as_bytes(), space);
191 let written = ops::saved(NATIVE, space, &resized).unwrap();
192 let stored = page_in(written.as_slice(), space);
192193 assert_eq!(stored, resized);
193194 assert_eq!(
194 PreparedEdit::page(written.as_bytes(), space, &stored, AUTHOR)
195 ops::saved(written.as_slice(), space, &stored)
195196 .unwrap()
196 .as_bytes(),
197 written.as_bytes()
197 .as_slice(),
198 written.as_slice()
198199 );
199200 if let Some(directory) = std::env::var_os("ONESTORE_IMAGE_RESIZE_EXPORT") {
200201 let directory = std::path::PathBuf::from(directory);
201202 std::fs::create_dir(&directory).unwrap();
202 std::fs::write(directory.join("synthetic.one"), written.as_bytes()).unwrap();
203 let written_store = Store::parse(written.as_bytes()).unwrap();
203 std::fs::write(directory.join("synthetic.one"), written.as_slice()).unwrap();
204 let written_store = Store::parse(written.as_slice()).unwrap();
204205 std::fs::write(
205206 directory.join("Open Notebook.onetoc2"),
206207 onestore::create_table_of_contents(
......@@ -213,8 +214,8 @@ fn a_native_picture_is_resized_and_described_then_reset() {
213214 }
214215 let mut reset = stored.clone();
215216 resize(&mut reset, Layout::default(), None);
216 let again = PreparedEdit::page(written.as_bytes(), space, &reset, AUTHOR).unwrap();
217 assert_eq!(page_in(again.as_bytes(), space), native);
217 let again = ops::saved(written.as_slice(), space, &reset).unwrap();
218 assert_eq!(page_in(again.as_slice(), space), native);
218219}
219220
220221#[test]
......@@ -234,8 +235,8 @@ fn a_rust_picture_resized_natively_reads_back_with_its_size_and_description() {
234235 let mut layout = picture.layout.clone();
235236 layout.max_width = Some(200.0);
236237 resize(&mut wider, layout, Some("Wider in Rust"));
237 let written = PreparedEdit::page(NATIVE_RESIZE, space, &wider, AUTHOR).unwrap();
238 assert_eq!(page_in(written.as_bytes(), space), wider);
238 let written = ops::saved(NATIVE_RESIZE, space, &wider).unwrap();
239 assert_eq!(page_in(written.as_slice(), space), wider);
239240}
240241
241242fn page_pictures(page: &Page) -> Vec<&Image> {
......@@ -270,8 +271,8 @@ fn a_page_level_picture_placed_natively_reads_as_a_page_object() {
270271 image.layout.y = Some(400.0);
271272 }
272273 }
273 let written = PreparedEdit::page(NATIVE_PAGE_LEVEL, space, &moved, AUTHOR).unwrap();
274 assert_eq!(page_in(written.as_bytes(), space), moved);
274 let written = ops::saved(NATIVE_PAGE_LEVEL, space, &moved).unwrap();
275 assert_eq!(page_in(written.as_slice(), space), moved);
275276}
276277
277278/// `ONESTORE_PAGE_IMAGE_EXPORT` names a new directory receiving the candidate for a cold reopen.
......@@ -296,23 +297,23 @@ fn a_page_level_picture_is_inserted_moved_and_removed() {
296297 background: false,
297298 };
298299 after.objects.push(PageObject::Image(image.clone()));
299 let written = PreparedEdit::page(&source, space, &after, AUTHOR).unwrap();
300 let stored = page_in(written.as_bytes(), space);
300 let written = ops::saved(&source, space, &after).unwrap();
301 let stored = page_in(written.as_slice(), space);
301302 let mut expected = after.clone();
302303 expected.title = stored.title.clone();
303304 assert_eq!(stored, expected);
304305 assert_eq!(page_pictures(&stored)[0].bytes.as_deref(), Some(PNG));
305306 assert_eq!(
306 PreparedEdit::page(written.as_bytes(), space, &stored, AUTHOR)
307 ops::saved(written.as_slice(), space, &stored)
307308 .unwrap()
308 .as_bytes(),
309 written.as_bytes()
309 .as_slice(),
310 written.as_slice()
310311 );
311312 if let Some(directory) = std::env::var_os("ONESTORE_PAGE_IMAGE_EXPORT") {
312313 let directory = std::path::PathBuf::from(directory);
313314 std::fs::create_dir(&directory).unwrap();
314 std::fs::write(directory.join("pictures.one"), written.as_bytes()).unwrap();
315 let written_store = Store::parse(written.as_bytes()).unwrap();
315 std::fs::write(directory.join("pictures.one"), written.as_slice()).unwrap();
316 let written_store = Store::parse(written.as_slice()).unwrap();
316317 std::fs::write(
317318 directory.join("Open Notebook.onetoc2"),
318319 onestore::create_table_of_contents(
......@@ -330,19 +331,19 @@ fn a_page_level_picture_is_inserted_moved_and_removed() {
330331 image.alt = None;
331332 }
332333 }
333 let again = PreparedEdit::page(written.as_bytes(), space, &moved, AUTHOR).unwrap();
334 assert_eq!(page_in(again.as_bytes(), space), moved);
334 let again = ops::saved(written.as_slice(), space, &moved).unwrap();
335 assert_eq!(page_in(again.as_slice(), space), moved);
335336 let mut removed = moved.clone();
336337 removed.objects.retain(|object| object.id() != image.id);
337 let last = PreparedEdit::page(again.as_bytes(), space, &removed, AUTHOR).unwrap();
338 let stored = page_in(last.as_bytes(), space);
338 let last = ops::saved(again.as_slice(), space, &removed).unwrap();
339 let stored = page_in(last.as_slice(), space);
339340 assert!(page_pictures(&stored).is_empty());
340341 assert_eq!(stored, removed);
341342 let mut unplaced = stored.clone();
342343 let mut nowhere = image.clone();
343344 nowhere.layout = Layout::default();
344345 unplaced.objects.push(PageObject::Image(nowhere));
345 assert!(PreparedEdit::page(last.as_bytes(), space, &unplaced, AUTHOR).is_err());
346 assert!(ops::saved(last.as_slice(), space, &unplaced).is_err());
346347}
347348
348349#[test]
......@@ -361,7 +362,7 @@ fn pictures_need_a_recognised_payload_and_stored_ones_stay_fixed() {
361362 background: false,
362363 });
363364 body_paragraphs(&mut after).push(holder);
364 assert!(PreparedEdit::page(&source, space, &after, AUTHOR).is_err());
365 assert!(ops::saved(&source, space, &after).is_err());
365366 let (space, native) = first_page(NATIVE);
366367 let mut resized = native.clone();
367368 let mut found = 0;
......@@ -376,5 +377,5 @@ fn pictures_need_a_recognised_payload_and_stored_ones_stay_fixed() {
376377 }
377378 }
378379 assert_eq!(found, 1);
379 assert!(PreparedEdit::page(NATIVE, space, &resized, AUTHOR).is_err());
380 assert!(ops::saved(NATIVE, space, &resized).is_err());
380381}
crates/onestore/tests/page_ink.rs+17-16
......@@ -1,10 +1,11 @@
1#[path = "support/ops.rs"]
2mod ops;
13use onestore::{
2 ExGuid, PreparedEdit, RevisionIndex, Store,
4 ExGuid, RevisionIndex, Store,
35 document::Document,
46 page::{Ink, InkStroke, Page, PageObject, text::new_id},
57};
68
7const AUTHOR: &str = "Ink author";
89
910/// Two drawings made with the mouse in OneNote 2010 (`tools/native/ink.ahk`): a diamond and a
1011/// diagonal line, whose native read reports their positions and sizes.
......@@ -142,8 +143,8 @@ fn an_ink_drawing_is_written_erased_stroke_by_stroke_and_removed() {
142143 let mut after = before.clone();
143144 let ink = drawing();
144145 after.objects.push(PageObject::Ink(ink.clone()));
145 let written = PreparedEdit::page(&source, space, &after, AUTHOR).unwrap();
146 let stored = page_in(written.as_bytes(), space);
146 let written = ops::saved(&source, space, &after).unwrap();
147 let stored = page_in(written.as_slice(), space);
147148 let mut expected = after.clone();
148149 expected.title = stored.title.clone();
149150 assert_eq!(stored, expected);
......@@ -157,12 +158,12 @@ fn an_ink_drawing_is_written_erased_stroke_by_stroke_and_removed() {
157158 ]
158159 ));
159160 assert_eq!(
160 PreparedEdit::page(written.as_bytes(), space, &stored, AUTHOR)
161 ops::saved(written.as_slice(), space, &stored)
161162 .unwrap()
162 .as_bytes(),
163 written.as_bytes()
163 .as_slice(),
164 written.as_slice()
164165 );
165 export("ONESTORE_INK_EXPORT", "ink.one", written.as_bytes());
166 export("ONESTORE_INK_EXPORT", "ink.one", written.as_slice());
166167 let mut erased = stored.clone();
167168 for object in &mut erased.objects {
168169 if let PageObject::Ink(ink) = object {
......@@ -171,13 +172,13 @@ fn an_ink_drawing_is_written_erased_stroke_by_stroke_and_removed() {
171172 .push(stroke(&[[330.0, 100.0], [330.0, 200.0]], Some(0xff0000)));
172173 }
173174 }
174 let again = PreparedEdit::page(written.as_bytes(), space, &erased, AUTHOR).unwrap();
175 let stored = page_in(again.as_bytes(), space);
175 let again = ops::saved(written.as_slice(), space, &erased).unwrap();
176 let stored = page_in(again.as_slice(), space);
176177 assert_eq!(stored, erased);
177178 let mut removed = stored.clone();
178179 removed.objects.retain(|object| object.id() != ink.id);
179 let last = PreparedEdit::page(again.as_bytes(), space, &removed, AUTHOR).unwrap();
180 let stored = page_in(last.as_bytes(), space);
180 let last = ops::saved(again.as_slice(), space, &removed).unwrap();
181 let stored = page_in(last.as_slice(), space);
181182 assert!(drawings(&stored).is_empty());
182183 assert_eq!(stored, removed);
183184}
......@@ -209,15 +210,15 @@ fn handwriting_is_written_as_paragraph_content() {
209210 paragraph.format = Default::default();
210211 paragraph.content = ParagraphContent::Ink(drawing());
211212 outline.paragraphs.push(paragraph);
212 let written = PreparedEdit::page(&source, space, &after, AUTHOR).unwrap();
213 let stored = page_in(written.as_bytes(), space);
213 let written = ops::saved(&source, space, &after).unwrap();
214 let stored = page_in(written.as_slice(), space);
214215 let mut expected = after.clone();
215216 expected.title = stored.title.clone();
216217 assert_eq!(stored, expected);
217218 export(
218219 "ONESTORE_INK_PARAGRAPH_EXPORT",
219220 "ink.one",
220 written.as_bytes(),
221 written.as_slice(),
221222 );
222223}
223224
......@@ -230,5 +231,5 @@ fn stored_strokes_keep_their_paths() {
230231 ink.strokes[0].points[0][0] += 1.0;
231232 }
232233 }
233 assert!(PreparedEdit::page(NATIVE, space, &moved, AUTHOR).is_err());
234 assert!(ops::saved(NATIVE, space, &moved).is_err());
234235}
crates/onestore/tests/page_links.rs+17-15
......@@ -1,5 +1,7 @@
1#[path = "support/ops.rs"]
2mod ops;
13use onestore::{
2 ExGuid, PreparedEdit, RevisionIndex, Store,
4 ExGuid, RevisionIndex, Store,
35 document::Document,
46 page::{
57 Page, PageObject, PageParagraph, Paragraph,
......@@ -13,7 +15,6 @@ const OUTLINES: &[u8] =
1315const NATIVE_LINKS: &[u8] =
1416 include_bytes!("../../../corpus/link-edit/native-links/notebook/links.one");
1517const NATIVE_BASE_PATH: &str = r"C:\one-tests\runs\capture\notebook\links.one";
16const AUTHOR: &str = "Link author";
1718const CODE: &str = "\u{fddf}HYPERLINK \"https://example.invalid/rust\"";
1819
1920fn page_by_title(bytes: &[u8], title: &str) -> (ExGuid, Page) {
......@@ -133,8 +134,8 @@ fn a_native_link_is_removed_leaving_plain_label_text() {
133134 replacement: Paragraph::new(content[end..].to_owned(), plain),
134135 })
135136 .unwrap();
136 let written = PreparedEdit::page(OUTLINES, space, &after, AUTHOR).unwrap();
137 let stored = assert_same(written.as_bytes(), space, &after);
137 let written = ops::saved(OUTLINES, space, &after).unwrap();
138 let stored = assert_same(written.as_slice(), space, &after);
138139 let mut stored = stored;
139140 let text = &body_paragraphs(&mut stored)[at].text().unwrap().text;
140141 assert!(text.text().ends_with(" Link label"));
......@@ -159,8 +160,8 @@ fn a_link_is_added_to_a_fresh_page_and_reads_back() {
159160 &mut body_paragraphs(&mut after)[0].text_mut().unwrap().text,
160161 "the Rust site",
161162 );
162 let written = PreparedEdit::page(&source, space, &after, AUTHOR).unwrap();
163 let stored = assert_same(written.as_bytes(), space, &after);
163 let written = ops::saved(&source, space, &after).unwrap();
164 let stored = assert_same(written.as_slice(), space, &after);
164165 let mut stored = stored;
165166 let text = &body_paragraphs(&mut stored)[0].text().unwrap().text;
166167 let flags: Vec<(Option<bool>, Option<bool>, Option<bool>)> = text
......@@ -185,8 +186,8 @@ fn a_link_is_added_to_a_fresh_page_and_reads_back() {
185186 if let Some(directory) = std::env::var_os("ONESTORE_LINK_EXPORT") {
186187 let directory = std::path::PathBuf::from(directory);
187188 std::fs::create_dir(&directory).unwrap();
188 std::fs::write(directory.join("links.one"), written.as_bytes()).unwrap();
189 let written_store = Store::parse(written.as_bytes()).unwrap();
189 std::fs::write(directory.join("links.one"), written.as_slice()).unwrap();
190 let written_store = Store::parse(written.as_slice()).unwrap();
190191 std::fs::write(
191192 directory.join("Open Notebook.onetoc2"),
192193 onestore::create_table_of_contents(
......@@ -292,10 +293,11 @@ fn internal_links_match_what_onenote_stores() {
292293fn a_page_links_to_another_page_and_its_paragraph() {
293294 let source = onestore::create_section("links.one", "Linking page", "Author").unwrap();
294295 let creation = onestore::PageCreation::new(None, Some("Link target"), "Author").unwrap();
295 let with_target = onestore::PreparedEdit::create_page(&source, &creation)
296 .unwrap()
297 .as_bytes()
298 .to_vec();
296 let with_target = ops::edited(
297 &source,
298 vec![onestore::op::Op::Section(onestore::op::SectionOp::Create(creation))],
299 )
300 .unwrap();
299301 let section = Store::parse(&with_target).unwrap().header.file_id;
300302 let (_, target) = page_by_title(&with_target, "Link target");
301303 let (space, before) = page_by_title(&with_target, "Linking page");
......@@ -351,12 +353,12 @@ fn a_page_links_to_another_page_and_its_paragraph() {
351353 replacement: Paragraph::from_runs(runs),
352354 })
353355 .unwrap();
354 let written = PreparedEdit::page(&with_target, space, &after, AUTHOR).unwrap();
355 assert_same(written.as_bytes(), space, &after);
356 let written = ops::saved(&with_target, space, &after).unwrap();
357 assert_same(written.as_slice(), space, &after);
356358 if let Some(directory) = std::env::var_os("ONESTORE_INTERNAL_LINK_EXPORT") {
357359 let directory = std::path::PathBuf::from(directory);
358360 std::fs::create_dir(&directory).unwrap();
359 std::fs::write(directory.join("links.one"), written.as_bytes()).unwrap();
361 std::fs::write(directory.join("links.one"), written.as_slice()).unwrap();
360362 std::fs::write(
361363 directory.join("Open Notebook.onetoc2"),
362364 onestore::create_table_of_contents("Open Notebook.onetoc2", &[("links.one", section)])
crates/onestore/tests/page_lists.rs+25-24
......@@ -1,5 +1,7 @@
1#[path = "support/ops.rs"]
2mod ops;
13use onestore::{
2 ExGuid, PreparedEdit, RevisionIndex, Store,
4 ExGuid, RevisionIndex, Store,
35 document::{Document, Format, Kind},
46 page::{
57 Definition, Page, PageObject, PageParagraph, ParagraphContent, TextObject, text::new_id,
......@@ -9,7 +11,6 @@ use std::collections::BTreeMap;
911
1012const TREES: &[u8] =
1113 include_bytes!("../../../corpus/outline-edit/tree/before/notebook/synthetic.one");
12const AUTHOR: &str = "List author";
1314
1415fn page_by_title(bytes: &[u8], title: &str) -> (ExGuid, Page) {
1516 let store = Store::parse(bytes).unwrap();
......@@ -114,8 +115,8 @@ fn a_numbered_paragraph_becomes_plain_while_its_neighbours_keep_their_nodes() {
114115 assert!(numbered.len() >= 2, "{lists:?}");
115116 let mut after = before.clone();
116117 body_paragraphs(&mut after)[numbered[0]].lists.clear();
117 let written = PreparedEdit::page(TREES, space, &after, AUTHOR).unwrap();
118 let stored = page_in(written.as_bytes(), space);
118 let written = ops::saved(TREES, space, &after).unwrap();
119 let stored = page_in(written.as_slice(), space);
119120 let mut expected = after.clone();
120121 expected.title = stored.title.clone();
121122 expected.definitions.remove(&lists[numbered[0]][0]);
......@@ -123,16 +124,16 @@ fn a_numbered_paragraph_becomes_plain_while_its_neighbours_keep_their_nodes() {
123124 for other in &numbered[1..] {
124125 for node in &lists[*other] {
125126 assert_eq!(
126 raw_object(written.as_bytes(), space, *node),
127 raw_object(written.as_slice(), space, *node),
127128 raw_object(TREES, space, *node)
128129 );
129130 }
130131 }
131132 assert_eq!(
132 PreparedEdit::page(written.as_bytes(), space, &stored, AUTHOR)
133 ops::saved(written.as_slice(), space, &stored)
133134 .unwrap()
134 .as_bytes(),
135 written.as_bytes()
135 .as_slice(),
136 written.as_slice()
136137 );
137138}
138139
......@@ -157,16 +158,16 @@ fn bullets_and_numbering_write_from_definitions_and_read_back() {
157158 let mut fresh = plain_paragraph(&template, "Numbered from three");
158159 fresh.lists = vec![number];
159160 body_paragraphs(&mut after).push(fresh);
160 let written = PreparedEdit::page(TREES, space, &after, AUTHOR).unwrap();
161 let stored = page_in(written.as_bytes(), space);
161 let written = ops::saved(TREES, space, &after).unwrap();
162 let stored = page_in(written.as_slice(), space);
162163 let mut expected = after.clone();
163164 expected.title = stored.title.clone();
164165 assert_eq!(stored, expected);
165166 assert_eq!(
166 PreparedEdit::page(written.as_bytes(), space, &stored, AUTHOR)
167 ops::saved(written.as_slice(), space, &stored)
167168 .unwrap()
168 .as_bytes(),
169 written.as_bytes()
169 .as_slice(),
170 written.as_slice()
170171 );
171172}
172173
......@@ -183,8 +184,8 @@ fn a_list_definition_changes_in_place() {
183184 };
184185 *restart = Some(7);
185186 definition.format.bold = Some(true);
186 let written = PreparedEdit::page(TREES, space, &after, AUTHOR).unwrap();
187 let stored = page_in(written.as_bytes(), space);
187 let written = ops::saved(TREES, space, &after).unwrap();
188 let stored = page_in(written.as_slice(), space);
188189 let mut expected = after.clone();
189190 expected.title = stored.title.clone();
190191 assert_eq!(stored, expected);
......@@ -199,8 +200,8 @@ fn a_definition_shared_between_paragraphs_becomes_a_node_per_paragraph() {
199200 let plain = (0..lists.len()).find(|i| lists[*i].is_empty()).unwrap();
200201 let mut shared = before.clone();
201202 body_paragraphs(&mut shared)[plain].lists = lists[numbered].clone();
202 let written = PreparedEdit::page(TREES, space, &shared, AUTHOR).unwrap();
203 let stored = page_in(written.as_bytes(), space);
203 let written = ops::saved(TREES, space, &shared).unwrap();
204 let stored = page_in(written.as_slice(), space);
204205 let after = list_ids(&stored);
205206 assert_eq!(after[numbered], lists[numbered]);
206207 assert_eq!(after[plain].len(), 1);
......@@ -218,12 +219,12 @@ fn missing_and_foreign_list_definitions_are_refused() {
218219 let plain = (0..lists.len()).find(|i| lists[*i].is_empty()).unwrap();
219220 let mut missing = before.clone();
220221 body_paragraphs(&mut missing)[plain].lists = vec![new_id().unwrap()];
221 assert!(PreparedEdit::page(TREES, space, &missing, AUTHOR).is_err());
222 assert!(ops::saved(TREES, space, &missing).is_err());
222223 let mut foreign = before.clone();
223224 let paragraph = body_paragraphs(&mut foreign)[plain].id;
224225 foreign.definitions.insert(paragraph, bullet_definition());
225226 body_paragraphs(&mut foreign)[plain].lists = vec![paragraph];
226 assert!(PreparedEdit::page(TREES, space, &foreign, AUTHOR).is_err());
227 assert!(ops::saved(TREES, space, &foreign).is_err());
227228 let mut style: BTreeMap<ExGuid, Definition> = before.definitions.clone();
228229 if let Some((id, definition)) = style
229230 .iter_mut()
......@@ -232,7 +233,7 @@ fn missing_and_foreign_list_definitions_are_refused() {
232233 definition.format.bold = Some(true);
233234 let mut edited = before.clone();
234235 edited.definitions.insert(*id, definition.clone());
235 assert!(PreparedEdit::page(TREES, space, &edited, AUTHOR).is_err());
236 assert!(ops::saved(TREES, space, &edited).is_err());
236237 }
237238}
238239
......@@ -299,16 +300,16 @@ fn bullets_numbering_nesting_and_restarts_publish_on_a_fresh_page() {
299300 body_paragraphs(&mut after).push(paragraph);
300301 }
301302 body_paragraphs(&mut after).push(plain_paragraph(&template, "Plain again"));
302 let written = PreparedEdit::page(&source, space, &after, AUTHOR).unwrap();
303 let stored = page_in(written.as_bytes(), space);
303 let written = ops::saved(&source, space, &after).unwrap();
304 let stored = page_in(written.as_slice(), space);
304305 let mut expected = after.clone();
305306 expected.title = stored.title.clone();
306307 assert_eq!(stored, expected);
307308 if let Some(directory) = std::env::var_os("ONESTORE_LIST_EXPORT") {
308309 let directory = std::path::PathBuf::from(directory);
309310 std::fs::create_dir(&directory).unwrap();
310 std::fs::write(directory.join("lists.one"), written.as_bytes()).unwrap();
311 let written_store = Store::parse(written.as_bytes()).unwrap();
311 std::fs::write(directory.join("lists.one"), written.as_slice()).unwrap();
312 let written_store = Store::parse(written.as_slice()).unwrap();
312313 std::fs::write(
313314 directory.join("Open Notebook.onetoc2"),
314315 onestore::create_table_of_contents(
crates/onestore/tests/page_math.rs+8-6
......@@ -1,3 +1,5 @@
1#[path = "support/ops.rs"]
2mod ops;
13use onestore::{
24 RevisionIndex, Store,
35 document::Document,
......@@ -272,8 +274,8 @@ fn equations_are_written_and_read_back() {
272274 }
273275 outline.paragraphs.push(paragraph);
274276 }
275 let written = onestore::PreparedEdit::page(&source, space, &after, "Math author").unwrap();
276 let store = Store::parse(written.as_bytes()).unwrap();
277 let written = ops::saved(&source, space, &after).unwrap();
278 let store = Store::parse(written.as_slice()).unwrap();
277279 let index = RevisionIndex::parse(&store).unwrap();
278280 let document = Document::parse(&index).unwrap();
279281 let stored = Page::from_space(&document, space).unwrap();
......@@ -305,13 +307,13 @@ fn equations_are_written_and_read_back() {
305307 }
306308 assert_eq!(stored, expected);
307309 assert_eq!(
308 rendered_equations(written.as_bytes(), &stored.title),
310 rendered_equations(written.as_slice(), &stored.title),
309311 expected_mathml
310312 );
311313 if let Some(directory) = std::env::var_os("ONESTORE_MATH_EXPORT") {
312314 let directory = std::path::PathBuf::from(directory);
313315 std::fs::create_dir(&directory).unwrap();
314 std::fs::write(directory.join("math.one"), written.as_bytes()).unwrap();
316 std::fs::write(directory.join("math.one"), written.as_slice()).unwrap();
315317 std::fs::write(
316318 directory.join("Open Notebook.onetoc2"),
317319 onestore::create_table_of_contents(
......@@ -361,8 +363,8 @@ fn a_paragraph_emptied_of_its_inline_math_saves() {
361363 let plain = text.text.spans()[0].format.clone();
362364 assert_ne!(plain.math, Some(true));
363365 text.text = onestore::page::Paragraph::new(String::new(), plain);
364 let written = onestore::PreparedEdit::page(NATIVE, space, &after, "Author").unwrap();
365 let store = Store::parse(written.as_bytes()).unwrap();
366 let written = ops::saved(NATIVE, space, &after).unwrap();
367 let store = Store::parse(written.as_slice()).unwrap();
366368 let index = RevisionIndex::parse(&store).unwrap();
367369 let document = Document::parse(&index).unwrap();
368370 assert_eq!(Page::from_space(&document, space).unwrap(), after);
crates/onestore/tests/page_media.rs+10-8
......@@ -1,5 +1,7 @@
1#[path = "support/ops.rs"]
2mod ops;
13use onestore::{
2 ExGuid, PreparedEdit, RevisionIndex, Store,
4 ExGuid, RevisionIndex, Store,
35 document::Document,
46 page::{MediaIndex, Page, PageObject, ParagraphContent, Recording},
57};
......@@ -96,8 +98,8 @@ fn editing_a_recorded_page_keeps_the_recording_and_its_annotation() {
9698 tags: Vec::new(),
9799 });
98100 outline.paragraphs.push(added.clone());
99 let written = PreparedEdit::page(NATIVE, space, &after, "Media author").unwrap();
100 let (_, stored) = page(written.as_bytes());
101 let written = ops::saved(NATIVE, space, &after).unwrap();
102 let (_, stored) = page(written.as_slice());
101103 assert_eq!(recording(&stored), recording(&before));
102104 assert_eq!(stored.objects.len(), before.objects.len());
103105
......@@ -112,7 +114,7 @@ fn editing_a_recorded_page_keeps_the_recording_and_its_annotation() {
112114 .unwrap();
113115 let last = outline.paragraphs.last_mut().unwrap();
114116 last.media = recording(&before).1;
115 assert!(PreparedEdit::page(written.as_bytes(), space, &annotated, "x").is_err());
117 assert!(ops::saved(written.as_slice(), space, &annotated).is_err());
116118 let mut recaptured = after.clone();
117119 for object in &mut recaptured.objects {
118120 if let PageObject::Outline(outline) = object {
......@@ -123,13 +125,13 @@ fn editing_a_recorded_page_keeps_the_recording_and_its_annotation() {
123125 }
124126 }
125127 }
126 assert!(PreparedEdit::page(written.as_bytes(), space, &recaptured, "x").is_err());
128 assert!(ops::saved(written.as_slice(), space, &recaptured).is_err());
127129
128130 if let Some(directory) = std::env::var_os("ONESTORE_MEDIA_EXPORT") {
129131 let directory = std::path::PathBuf::from(directory);
130132 std::fs::create_dir_all(&directory).unwrap();
131 std::fs::write(directory.join("Features.one"), written.as_bytes()).unwrap();
132 let file_id = Store::parse(written.as_bytes()).unwrap().header.file_id;
133 std::fs::write(directory.join("Features.one"), written.as_slice()).unwrap();
134 let file_id = Store::parse(written.as_slice()).unwrap().header.file_id;
133135 std::fs::write(
134136 directory.join("Open Notebook.onetoc2"),
135137 onestore::create_table_of_contents(
......@@ -139,7 +141,7 @@ fn editing_a_recorded_page_keeps_the_recording_and_its_annotation() {
139141 .unwrap(),
140142 )
141143 .unwrap();
142 let store = Store::parse(written.as_bytes()).unwrap();
144 let store = Store::parse(written.as_slice()).unwrap();
143145 let index = RevisionIndex::parse(&store).unwrap();
144146 let count = Document::parse(&index).unwrap().pages().unwrap().len();
145147 std::fs::write(directory.join("expected-count.txt"), count.to_string()).unwrap();
crates/onestore/tests/page_movement.rs+25-24
......@@ -1,6 +1,10 @@
1#[path = "support/ops.rs"]
2mod ops;
3
14use onestore::{
2 ExGuid, PageEdit, PreparedEdit, RevisionIndex, Store,
5 ExGuid, PageEdit, RevisionIndex, Store,
36 document::{Document, Kind},
7 op::{PageOp, SectionOp},
48};
59
610#[path = "support/current.rs"]
......@@ -175,10 +179,10 @@ fn explicit_page_edits_match_native_selection_and_indentation() {
175179 let serialized = serde_json::to_vec(&edits).unwrap();
176180 let retained: Vec<PageEdit> = serde_json::from_slice(&serialized).unwrap();
177181 assert_eq!(edits, retained);
178 let prepared = PreparedEdit::pages(&source, &retained).unwrap();
182 let prepared = ops::section_op(&source, SectionOp::Pages(retained.to_vec())).unwrap();
179183 verify(&source, prepared.as_bytes(), native);
180184 assert_eq!(
181 PreparedEdit::pages(prepared.as_bytes(), &retained)
185 ops::section_op(prepared.as_bytes(), SectionOp::Pages(retained.to_vec()))
182186 .unwrap()
183187 .as_bytes(),
184188 prepared.as_bytes()
......@@ -187,7 +191,7 @@ fn explicit_page_edits_match_native_selection_and_indentation() {
187191 source = prepared.as_bytes().to_vec();
188192 }
189193 let edit = PageEdit::move_to(ids[3], None, 1).unwrap();
190 let single = PreparedEdit::pages(SOURCE, &[edit]).unwrap();
194 let single = ops::section_op(SOURCE, SectionOp::Pages([edit].to_vec())).unwrap();
191195 verify(
192196 SOURCE,
193197 single.as_bytes(),
......@@ -206,32 +210,29 @@ fn invalid_and_empty_page_edits_do_not_publish() {
206210 }
207211 assert!(PageEdit::move_to(ids[0], Some(ids[0]), 1).is_err());
208212 assert!(PageEdit::set_level(ExGuid::default(), 1).is_err());
209 assert_eq!(PreparedEdit::pages(SOURCE, &[]).unwrap().as_bytes(), SOURCE);
213 assert_eq!(ops::section_op(SOURCE, SectionOp::Pages([].to_vec())).unwrap().as_bytes(), SOURCE);
210214 assert_eq!(
211 PreparedEdit::pages(SOURCE, &[PageEdit::set_level(ids[0], 1).unwrap()])
215 ops::section_op(SOURCE, SectionOp::Pages([PageEdit::set_level(ids[0], 1).unwrap()].to_vec()))
212216 .unwrap()
213217 .as_bytes(),
214218 SOURCE
215219 );
216 assert!(PreparedEdit::pages(SOURCE, &[PageEdit::set_level(ids[0], 2).unwrap()]).is_err());
220 assert!(ops::section_op(SOURCE, SectionOp::Pages([PageEdit::set_level(ids[0], 2).unwrap()].to_vec())).is_err());
217221 let duplicate = PageEdit::set_level(ids[3], 2).unwrap();
218 assert!(PreparedEdit::pages(SOURCE, &[duplicate.clone(), duplicate]).is_err());
222 assert!(ops::section_op(SOURCE, SectionOp::Pages([duplicate.clone(), duplicate].to_vec())).is_err());
219223 let missing = ExGuid {
220224 guid: [0x77; 16],
221225 n: 1,
222226 };
223 assert!(PreparedEdit::pages(SOURCE, &[PageEdit::set_level(missing, 1).unwrap()]).is_err());
227 assert!(ops::section_op(SOURCE, SectionOp::Pages([PageEdit::set_level(missing, 1).unwrap()].to_vec())).is_err());
224228 assert!(
225 PreparedEdit::pages(
226 SOURCE,
227 &[PageEdit::move_to(ids[3], Some(missing), 1).unwrap()]
228 )
229 ops::section_op(SOURCE, SectionOp::Pages([PageEdit::move_to(ids[3], Some(missing), 1).unwrap()].to_vec()))
229230 .is_err()
230231 );
231232 let mut value = serde_json::to_value(PageEdit::set_level(ids[3], 2).unwrap()).unwrap();
232233 value["level"] = 4.into();
233234 let invalid: PageEdit = serde_json::from_value(value.clone()).unwrap();
234 assert!(PreparedEdit::pages(SOURCE, &[invalid]).is_err());
235 assert!(ops::section_op(SOURCE, SectionOp::Pages([invalid].to_vec())).is_err());
235236 value["unknown"] = true.into();
236237 assert!(serde_json::from_value::<PageEdit>(value).is_err());
237238}
......@@ -246,12 +247,12 @@ fn moving_pages_preserves_ink_file_data_and_native_feature_objects() {
246247 let first = pages(source)[0];
247248 let created =
248249 onestore::PageCreation::new(Some(first), Some("Movement anchor"), "Author").unwrap();
249 let source = PreparedEdit::create_page(source, &created)
250 let source = ops::section_op(source, SectionOp::Create(created.clone()))
250251 .unwrap()
251252 .as_bytes()
252253 .to_vec();
253254 let moved =
254 PreparedEdit::pages(&source, &[PageEdit::move_to(first, None, 1).unwrap()]).unwrap();
255 ops::section_op(&source, SectionOp::Pages([PageEdit::move_to(first, None, 1).unwrap()].to_vec())).unwrap();
255256 assert_eq!(pages(moved.as_bytes()).last(), Some(&first));
256257 let stores = [&source, moved.as_bytes()].map(|bytes| Store::parse(bytes).unwrap());
257258 let indexes = stores
......@@ -295,7 +296,7 @@ fn section_reordering_refreshes_stale_native_metadata_levels() {
295296 .enumerate()
296297 .map(|(i, sid)| PageEdit::move_to(*sid, None, u32::try_from(i + 1).unwrap()).unwrap())
297298 .collect();
298 let prepared = PreparedEdit::pages(source, &edits).unwrap();
299 let prepared = ops::section_op(source, SectionOp::Pages(edits.to_vec())).unwrap();
299300 let stores = [source.as_slice(), prepared.as_bytes()].map(|b| Store::parse(b).unwrap());
300301 let indexes = stores.each_ref().map(|s| RevisionIndex::parse(s).unwrap());
301302 let documents = indexes.each_ref().map(|i| Document::parse(i).unwrap());
......@@ -335,7 +336,7 @@ fn metadata_copy_order_does_not_change_page_identity() {
335336 "../../../corpus/page-lifecycle/page-edits/optional-cache/source-cold/notebook/Lifecycle.one"
336337 );
337338 let sid = pages(source)[3];
338 let edited = PreparedEdit::pages(source, &[PageEdit::set_level(sid, 2).unwrap()]).unwrap();
339 let edited = ops::section_op(source, SectionOp::Pages([PageEdit::set_level(sid, 2).unwrap()].to_vec())).unwrap();
339340 let expected = include_bytes!(
340341 "../../../corpus/page-lifecycle/page-edits/optional-cache/candidate/Lifecycle.one"
341342 );
......@@ -371,8 +372,8 @@ fn native_edits_on_moved_pages_support_more_rust_changes() {
371372 let bodies: Vec<_> = view.nodes.iter().filter_map(|(id, node)|
372373 matches!(&node.kind, Kind::RichText { text, .. } if text == "Native body after a Rust page move.").then_some(*id)).collect();
373374 let [body] = bodies.as_slice() else { panic!() };
374 let moved = PreparedEdit::pages(source, &[PageEdit::set_level(sid, 2).unwrap()]).unwrap();
375 let edited = PreparedEdit::text(moved.as_bytes(), sid, *body, 0..0, "Rust + ").unwrap();
375 let moved = ops::section_op(source, SectionOp::Pages([PageEdit::set_level(sid, 2).unwrap()].to_vec())).unwrap();
376 let edited = ops::page_op(moved.as_bytes(), sid, PageOp::Text { text: *body, range: 0..0, with: "Rust + ".into() }).unwrap();
376377 let after_store = Store::parse(edited.as_bytes()).unwrap();
377378 let after_index = RevisionIndex::parse(&after_store).unwrap();
378379 let after = Document::parse(&after_index).unwrap();
......@@ -401,7 +402,7 @@ fn grouped_page_levels_and_membership_survive_each_storage_interruption() {
401402 let mut source = onestore::create_section("movement.one", "Original", "Author").unwrap();
402403 for _ in 0..2 {
403404 let page = onestore::PageCreation::new(None, Some("Same title"), "Author").unwrap();
404 source = PreparedEdit::create_page(&source, &page)
405 source = ops::section_op(&source, SectionOp::Create(page.clone()))
405406 .unwrap()
406407 .as_bytes()
407408 .to_vec();
......@@ -421,7 +422,7 @@ fn grouped_page_levels_and_membership_survive_each_storage_interruption() {
421422 PageEdit::set_level(ids[first], 2).unwrap(),
422423 PageEdit::set_level(ids[first + 1], 3).unwrap(),
423424 ];
424 let prepared = PreparedEdit::pages(source, &edits).unwrap();
425 let prepared = ops::section_op(source, SectionOp::Pages(edits.to_vec())).unwrap();
425426 let old = current::current(source);
426427 let new = current::current(prepared.as_bytes());
427428 assert_eq!(new.len(), old.len());
......@@ -460,7 +461,7 @@ fn grouped_page_levels_and_membership_survive_each_storage_interruption() {
460461fn repeated_nesting_retains_series_history_across_revision_checkpoints() {
461462 let mut source = onestore::create_section("movement.one", "Original", "Author").unwrap();
462463 let page = onestore::PageCreation::new(None, Some("Child"), "Author").unwrap();
463 source = PreparedEdit::create_page(&source, &page)
464 source = ops::section_op(&source, SectionOp::Create(page.clone()))
464465 .unwrap()
465466 .as_bytes()
466467 .to_vec();
......@@ -468,7 +469,7 @@ fn repeated_nesting_retains_series_history_across_revision_checkpoints() {
468469 for step in 0..520 {
469470 let level = if step % 2 == 0 { 2 } else { 1 };
470471 let edit = PageEdit::set_level(page.space(), level).unwrap();
471 let written = PreparedEdit::pages(&source, &[edit])
472 let written = ops::section_op(&source, SectionOp::Pages([edit].to_vec()))
472473 .unwrap()
473474 .as_bytes()
474475 .to_vec();
crates/onestore/tests/page_removal.rs+12-1
......@@ -1,3 +1,5 @@
1#[path = "support/ops.rs"]
2mod ops;
13use onestore::{
24 ExGuid, FileDataReference, IdStream, ObjectData, PropertySets, RevisionIndex, Store, Value,
35 document::Document,
......@@ -525,7 +527,16 @@ fn copy_comparison_rejects_text_and_payload_changes() {
525527 .then_some(*oid)
526528 })
527529 .unwrap();
528 let changed = onestore::replace_text(copied, copy_sid, text, 0..0, "Changed ").unwrap();
530 let changed = ops::page_edited(
531 copied,
532 copy_sid,
533 vec![onestore::op::PageOp::Text {
534 text,
535 range: 0..0,
536 with: "Changed ".into(),
537 }],
538 )
539 .unwrap();
529540 let changed_store = Store::parse(&changed).unwrap();
530541 RevisionIndex::parse(&changed_store)
531542 .unwrap()
crates/onestore/tests/page_tables.rs+34-36
......@@ -1,5 +1,7 @@
1#[path = "support/ops.rs"]
2mod ops;
13use onestore::{
2 ExGuid, PreparedEdit, RevisionIndex, Store,
4 ExGuid, RevisionIndex, Store,
35 document::{Document, Format},
46 page::{
57 Page, PageObject, PageParagraph, ParagraphContent, Table, TableCell, TableColumn, TableRow,
......@@ -9,7 +11,6 @@ use onestore::{
911
1012const TREES: &[u8] =
1113 include_bytes!("../../../corpus/outline-edit/tree/before/notebook/synthetic.one");
12const AUTHOR: &str = "Table author";
1314
1415fn page_by_title(bytes: &[u8], title: &str) -> (ExGuid, Page) {
1516 let store = Store::parse(bytes).unwrap();
......@@ -174,19 +175,19 @@ fn a_row_and_a_column_are_added_to_a_native_table_and_removed_again() {
174175 new_cell(&template, "Second row, third column"),
175176 ],
176177 });
177 let written = PreparedEdit::page(TREES, space, &after, AUTHOR).unwrap();
178 let stored = assert_same(written.as_bytes(), space, &after);
178 let written = ops::saved(TREES, space, &after).unwrap();
179 let stored = assert_same(written.as_slice(), space, &after);
179180 assert_eq!(
180 PreparedEdit::page(written.as_bytes(), space, &stored, AUTHOR)
181 ops::saved(written.as_slice(), space, &stored)
181182 .unwrap()
182 .as_bytes(),
183 written.as_bytes()
183 .as_slice(),
184 written.as_slice()
184185 );
185186 if let Some(directory) = std::env::var_os("ONESTORE_TABLE_EDIT_EXPORT") {
186187 let directory = std::path::PathBuf::from(directory);
187188 std::fs::create_dir(&directory).unwrap();
188 std::fs::write(directory.join("synthetic.one"), written.as_bytes()).unwrap();
189 let written_store = Store::parse(written.as_bytes()).unwrap();
189 std::fs::write(directory.join("synthetic.one"), written.as_slice()).unwrap();
190 let written_store = Store::parse(written.as_slice()).unwrap();
190191 std::fs::write(
191192 directory.join("Open Notebook.onetoc2"),
192193 onestore::create_table_of_contents(
......@@ -204,8 +205,8 @@ fn a_row_and_a_column_are_added_to_a_native_table_and_removed_again() {
204205 row.cells.remove(1);
205206 }
206207 table.rows.remove(0);
207 let again = PreparedEdit::page(written.as_bytes(), space, &smaller, AUTHOR).unwrap();
208 let mut stored = assert_same(again.as_bytes(), space, &smaller);
208 let again = ops::saved(written.as_slice(), space, &smaller).unwrap();
209 let mut stored = assert_same(again.as_slice(), space, &smaller);
209210 let table = table_mut(&mut stored);
210211 assert_eq!((table.rows.len(), table.columns.len()), (1, 2));
211212}
......@@ -218,8 +219,8 @@ fn column_widths_locks_and_borders_change_in_place() {
218219 table.columns[0].width = 300.0;
219220 table.columns[1].locked = false;
220221 table.borders = Some(false);
221 let written = PreparedEdit::page(TREES, space, &after, AUTHOR).unwrap();
222 assert_same(written.as_bytes(), space, &after);
222 let written = ops::saved(TREES, space, &after).unwrap();
223 assert_same(written.as_slice(), space, &after);
223224}
224225
225226#[test]
......@@ -232,17 +233,17 @@ fn cell_shading_and_indents_change_in_place() {
232233 cell.shading = Some(0x0000ffff);
233234 let original_indents = cell.indents.clone();
234235 cell.indents.clear();
235 assert!(PreparedEdit::page(TREES, space, &after, AUTHOR).is_err());
236 assert!(ops::saved(TREES, space, &after).is_err());
236237 let cell = &mut table_mut(&mut after).rows[0].cells[0];
237238 cell.indents = vec![18.0, 0.0, 27.0, 27.0];
238 let written = PreparedEdit::page(TREES, space, &after, AUTHOR).unwrap();
239 let stored = assert_same(written.as_bytes(), space, &after);
239 let written = ops::saved(TREES, space, &after).unwrap();
240 let stored = assert_same(written.as_slice(), space, &after);
240241 let mut reset = stored.clone();
241242 let cell = &mut table_mut(&mut reset).rows[0].cells[0];
242243 cell.shading = None;
243244 cell.indents = original_indents;
244 let again = PreparedEdit::page(written.as_bytes(), space, &reset, AUTHOR).unwrap();
245 assert_eq!(assert_same(again.as_bytes(), space, &reset), before);
245 let again = ops::saved(written.as_slice(), space, &reset).unwrap();
246 assert_eq!(assert_same(again.as_slice(), space, &reset), before);
246247}
247248
248249/// `ONESTORE_NESTED_TABLE_EXPORT` names a new directory receiving the candidate for a cold reopen.
......@@ -286,7 +287,7 @@ fn a_table_nests_inside_a_cell_and_is_removed_again() {
286287 table_mut(&mut after).rows[0].cells[1]
287288 .paragraphs
288289 .push(holder);
289 let written = PreparedEdit::page(TREES, space, &after, AUTHOR).unwrap();
290 let written = ops::saved(TREES, space, &after).unwrap();
290291 let ParagraphContent::Table(nested) = &mut table_mut(&mut after).rows[0].cells[1]
291292 .paragraphs
292293 .last_mut()
......@@ -296,12 +297,12 @@ fn a_table_nests_inside_a_cell_and_is_removed_again() {
296297 panic!()
297298 };
298299 with_native_indents(nested);
299 let stored = assert_same(written.as_bytes(), space, &after);
300 let stored = assert_same(written.as_slice(), space, &after);
300301 if let Some(directory) = std::env::var_os("ONESTORE_NESTED_TABLE_EXPORT") {
301302 let directory = std::path::PathBuf::from(directory);
302303 std::fs::create_dir(&directory).unwrap();
303 std::fs::write(directory.join("synthetic.one"), written.as_bytes()).unwrap();
304 let written_store = Store::parse(written.as_bytes()).unwrap();
304 std::fs::write(directory.join("synthetic.one"), written.as_slice()).unwrap();
305 let written_store = Store::parse(written.as_slice()).unwrap();
305306 std::fs::write(
306307 directory.join("Open Notebook.onetoc2"),
307308 onestore::create_table_of_contents(
......@@ -316,8 +317,8 @@ fn a_table_nests_inside_a_cell_and_is_removed_again() {
316317 table_mut(&mut removed).rows[0].cells[1]
317318 .paragraphs
318319 .retain(|paragraph| paragraph.id != holder_id);
319 let again = PreparedEdit::page(written.as_bytes(), space, &removed, AUTHOR).unwrap();
320 assert_eq!(assert_same(again.as_bytes(), space, &removed), before);
320 let again = ops::saved(written.as_slice(), space, &removed).unwrap();
321 assert_eq!(assert_same(again.as_slice(), space, &removed), before);
321322}
322323
323324#[test]
......@@ -328,20 +329,17 @@ fn inconsistent_tables_are_refused() {
328329 table_mut(&mut ragged).rows[0]
329330 .cells
330331 .push(new_cell(&template, "Extra"));
331 assert!(PreparedEdit::page(TREES, space, &ragged, AUTHOR).is_err());
332 assert!(ops::saved(TREES, space, &ragged).is_err());
333 // A cell keeps a paragraph.
332334 let mut empty = before.clone();
333335 table_mut(&mut empty).rows[0].cells[0].paragraphs.clear();
334 let written = PreparedEdit::page(TREES, space, &empty, AUTHOR).unwrap();
335 let mut stored = page_in(written.as_bytes(), space);
336 let cell = &table_mut(&mut stored).rows[0].cells[0];
337 assert_eq!(cell.paragraphs.len(), 1);
338 assert_eq!(cell.paragraphs[0].text().unwrap().text.text(), "");
336 assert!(ops::saved(TREES, space, &empty).is_err());
339337 let mut narrow = before.clone();
340338 table_mut(&mut narrow).columns[0].width = 10.0;
341 assert!(PreparedEdit::page(TREES, space, &narrow, AUTHOR).is_err());
339 assert!(ops::saved(TREES, space, &narrow).is_err());
342340 let mut rowless = before.clone();
343341 table_mut(&mut rowless).rows.clear();
344 assert!(PreparedEdit::page(TREES, space, &rowless, AUTHOR).is_err());
342 assert!(ops::saved(TREES, space, &rowless).is_err());
345343}
346344
347345/// `ONESTORE_TABLE_EXPORT` names a new directory receiving the candidate for a cold reopen.
......@@ -394,14 +392,14 @@ fn a_new_table_is_created_on_a_fresh_page() {
394392 holder.format = Format::default();
395393 body_paragraphs(&mut after).push(holder);
396394 body_paragraphs(&mut after).push(cell_paragraph(&template, "After the table"));
397 let written = PreparedEdit::page(&source, space, &after, AUTHOR).unwrap();
395 let written = ops::saved(&source, space, &after).unwrap();
398396 with_native_indents(table_mut(&mut after));
399 assert_same(written.as_bytes(), space, &after);
397 assert_same(written.as_slice(), space, &after);
400398 if let Some(directory) = std::env::var_os("ONESTORE_TABLE_EXPORT") {
401399 let directory = std::path::PathBuf::from(directory);
402400 std::fs::create_dir(&directory).unwrap();
403 std::fs::write(directory.join("tables.one"), written.as_bytes()).unwrap();
404 let written_store = Store::parse(written.as_bytes()).unwrap();
401 std::fs::write(directory.join("tables.one"), written.as_slice()).unwrap();
402 let written_store = Store::parse(written.as_slice()).unwrap();
405403 std::fs::write(
406404 directory.join("Open Notebook.onetoc2"),
407405 onestore::create_table_of_contents(
crates/onestore/tests/page_tags.rs+20-26
......@@ -1,5 +1,7 @@
1#[path = "support/ops.rs"]
2mod ops;
13use onestore::{
2 ExGuid, PreparedEdit, RevisionIndex, Store,
4 ExGuid, RevisionIndex, Store,
35 document::{Document, Format, Kind, Tag},
46 page::{
57 Definition, Page, PageObject, PageParagraph, ParagraphContent, TextObject, text::new_id,
......@@ -8,7 +10,6 @@ use onestore::{
810
911const TREES: &[u8] =
1012 include_bytes!("../../../corpus/outline-edit/tree/before/notebook/synthetic.one");
11const AUTHOR: &str = "Tag author";
1213
1314fn page_by_title(bytes: &[u8], title: &str) -> (ExGuid, Page) {
1415 let store = Store::parse(bytes).unwrap();
......@@ -161,13 +162,13 @@ fn removing_a_tag_leaves_its_definition_and_the_other_paragraphs_untouched() {
161162 .unwrap()
162163 .tags
163164 .clear();
164 let written = PreparedEdit::page(TREES, space, &after, AUTHOR).unwrap();
165 let written = ops::saved(TREES, space, &after).unwrap();
165166 let mut expected = after.clone();
166167 expected.definitions.remove(&definition);
167 let stored = assert_same(written.as_bytes(), space, &expected);
168 let stored = assert_same(written.as_slice(), space, &expected);
168169 assert!(!stored.definitions.contains_key(&definition));
169170 assert_eq!(
170 raw_object(written.as_bytes(), space, definition),
171 raw_object(written.as_slice(), space, definition),
171172 raw_object(TREES, space, definition)
172173 );
173174 let count = body_paragraphs(&mut after).len();
......@@ -176,7 +177,7 @@ fn removing_a_tag_leaves_its_definition_and_the_other_paragraphs_untouched() {
176177 .unwrap()
177178 .id;
178179 assert_eq!(
179 raw_object(written.as_bytes(), space, other),
180 raw_object(written.as_slice(), space, other),
180181 raw_object(TREES, space, other)
181182 );
182183}
......@@ -195,24 +196,17 @@ fn an_existing_definition_tags_another_paragraph_and_a_task_completes_in_place()
195196 .unwrap()
196197 .tags
197198 .push(tag(definition, false));
198 let written = PreparedEdit::page(TREES, space, &after, AUTHOR).unwrap();
199 let stored = assert_same(written.as_bytes(), space, &after);
199 let written = ops::saved(TREES, space, &after).unwrap();
200 let stored = assert_same(written.as_slice(), space, &after);
200201 let mut completed = stored.clone();
201202 let text = body_paragraphs(&mut completed)[other].text_mut().unwrap();
202203 text.tags[0].status = 1;
203204 text.tags[0].completed = Some(1_262_405_106);
204 let again = PreparedEdit::page(written.as_bytes(), space, &completed, AUTHOR).unwrap();
205 assert_same(again.as_bytes(), space, &completed);
205 let again = ops::saved(written.as_slice(), space, &completed).unwrap();
206 assert_same(again.as_slice(), space, &completed);
206207 assert_eq!(
207 PreparedEdit::page(
208 again.as_bytes(),
209 space,
210 &page_in(again.as_bytes(), space),
211 AUTHOR
212 )
213 .unwrap()
214 .as_bytes(),
215 again.as_bytes()
208 ops::saved(&again, space, &page_in(&again, space)).unwrap(),
209 again
216210 );
217211}
218212
......@@ -225,7 +219,7 @@ fn tags_without_a_known_definition_are_refused() {
225219 .unwrap()
226220 .tags
227221 .push(tag(new_id().unwrap(), false));
228 assert!(PreparedEdit::page(TREES, space, &missing, AUTHOR).is_err());
222 assert!(ops::saved(TREES, space, &missing).is_err());
229223 let mut repeated = before.clone();
230224 let at = tagged(&repeated);
231225 let existing = body_paragraphs(&mut repeated)[at].text().unwrap().tags[0].clone();
......@@ -234,7 +228,7 @@ fn tags_without_a_known_definition_are_refused() {
234228 .unwrap()
235229 .tags
236230 .push(tag(existing.definition.unwrap(), false));
237 assert!(PreparedEdit::page(TREES, space, &repeated, AUTHOR).is_err());
231 assert!(ops::saved(TREES, space, &repeated).is_err());
238232 let mut wrong = before.clone();
239233 let list = wrong
240234 .definitions
......@@ -247,7 +241,7 @@ fn tags_without_a_known_definition_are_refused() {
247241 .unwrap()
248242 .tags
249243 .push(tag(list, false));
250 assert!(PreparedEdit::page(TREES, space, &wrong, AUTHOR).is_err());
244 assert!(ops::saved(TREES, space, &wrong).is_err());
251245 }
252246}
253247
......@@ -293,13 +287,13 @@ fn new_definitions_and_tags_publish_on_a_fresh_page() {
293287 both.text_mut().unwrap().tags.push(tag(task, false));
294288 body_paragraphs(&mut after).push(both);
295289 body_paragraphs(&mut after).push(plain_paragraph(&template, "Untagged"));
296 let written = PreparedEdit::page(&source, space, &after, AUTHOR).unwrap();
297 assert_same(written.as_bytes(), space, &after);
290 let written = ops::saved(&source, space, &after).unwrap();
291 assert_same(written.as_slice(), space, &after);
298292 if let Some(directory) = std::env::var_os("ONESTORE_TAG_EXPORT") {
299293 let directory = std::path::PathBuf::from(directory);
300294 std::fs::create_dir(&directory).unwrap();
301 std::fs::write(directory.join("tags.one"), written.as_bytes()).unwrap();
302 let written_store = Store::parse(written.as_bytes()).unwrap();
295 std::fs::write(directory.join("tags.one"), written.as_slice()).unwrap();
296 let written_store = Store::parse(written.as_slice()).unwrap();
303297 std::fs::write(
304298 directory.join("Open Notebook.onetoc2"),
305299 onestore::create_table_of_contents(
crates/onestore/tests/page_writer.rs+155-107
......@@ -1,15 +1,34 @@
1//! The page writer must publish the same graph the typed writers publish for the same edit,
2//! in one transaction, and must round-trip the model it was given.
1//! A page edit lowered whole (`op::lower_page`) must publish the same graph its ops applied
2//! one by one publish, in one transaction, and must round-trip the model it was given.
3
4#[path = "support/ops.rs"]
5mod ops;
36
47use onestore::{
5 ExGuid, Insertion, OutlineEdit, ParagraphJoin, ParagraphSplit, PreparedEdit, RevisionIndex,
6 Store, TextAttribute, TreeEdit,
8 ExGuid, OutlineEdit, RevisionIndex, Store, TextAttribute,
79 document::{Document, Element, Kind, Revision},
10 op::PageOp,
811 page::{Page, PageObject, ParagraphContent, text::new_id},
912};
1013use std::collections::BTreeSet;
1114
12const AUTHOR: &str = "Page author";
15/// `source` after `ops` on the page in `space`.
16fn edited(source: &[u8], space: ExGuid, ops: Vec<PageOp>) -> Vec<u8> {
17 ops::page_edited(source, space, ops).unwrap()
18}
19
20/// A plain paragraph of `text` inserted in `container` before `before`, at `level`.
21fn insert(container: ExGuid, before: Option<ExGuid>, text: &str, level: u32) -> PageOp {
22 let mut paragraph = ops::paragraph(text);
23 paragraph.level = level;
24 PageOp::Insert {
25 container,
26 before,
27 paragraphs: vec![paragraph],
28 }
29}
30
31const AUTHOR: &str = "Author";
1332const PARAGRAPHS: &[u8] =
1433 include_bytes!("../../../corpus/paragraph-edit/before/notebook/synthetic.one");
1534const OUTLINES: &[u8] =
......@@ -144,7 +163,7 @@ fn same_element(revision_a: &Revision<'_>, revision_b: &Revision<'_>, id: ExGuid
144163 }
145164}
146165
147/// Every active revision the typed writers produced is reproduced by the page writer,
166/// Every active revision the ops produced one by one is reproduced by the page lowered whole,
148167/// ignoring writer-allocated immutable styles and author records. Modification times are
149168/// not compared: a reorder can be attributed to either sibling, and the model cannot say which.
150169fn equivalent(reference: &[u8], written: &[u8]) {
......@@ -182,11 +201,11 @@ fn equivalent(reference: &[u8], written: &[u8]) {
182201 }
183202}
184203
185/// Runs the page writer on the model the typed writers produced and checks both contracts.
204/// Lowers the page the ops produced whole and checks both contracts.
186205fn oracle(source: &[u8], space: ExGuid, reference: &[u8]) -> Vec<u8> {
187206 let after = page_in(reference, space);
188 let prepared = PreparedEdit::page(source, space, &after, AUTHOR).unwrap();
189 let written = prepared.as_bytes().to_vec();
207 let prepared = ops::saved(source, space, &after).unwrap();
208 let written = prepared.as_slice().to_vec();
190209 assert_eq!(page_in(&written, space), after, "model round trip");
191210 equivalent(reference, &written);
192211 let transactions = |bytes: &[u8]| Store::parse(bytes).unwrap().header.transaction_count;
......@@ -209,9 +228,12 @@ fn text_replacement_matches_replace_text() {
209228 (end..end, " end"),
210229 (0..end, ""),
211230 ] {
212 let reference =
213 onestore::replace_text(PARAGRAPHS, space, text, range, replacement).unwrap();
214 oracle(PARAGRAPHS, space, &reference);
231 let op = PageOp::Text {
232 text,
233 range,
234 with: replacement.into(),
235 };
236 oracle(PARAGRAPHS, space, &edited(PARAGRAPHS, space, vec![op]));
215237 }
216238}
217239
......@@ -220,28 +242,16 @@ fn formatting_matches_format_text() {
220242 let (space, page) = page_by_title(PARAGRAPHS, "Split style boundary");
221243 let (_, text, content) = text_ids(body(&page)[0])[0].clone();
222244 let end = u32::try_from(content.encode_utf16().count()).unwrap();
223 let reference = PreparedEdit::format(
224 PARAGRAPHS,
225 space,
226 text,
227 1..end.min(4),
228 &[
245 let reference = ops::page_op(PARAGRAPHS, space, PageOp::Format { text, range: 1..end.min(4), set: vec![
229246 TextAttribute::Bold(true),
230247 TextAttribute::Color(Some([255, 0, 0])),
231248 TextAttribute::FontSize(14.0),
232 ],
233 )
249 ], clear: Vec::new() })
234250 .unwrap();
235 let reference = PreparedEdit::format(
236 reference.as_bytes(),
237 space,
238 text,
239 0..1,
240 &[
251 let reference = ops::page_op(reference.as_bytes(), space, PageOp::Format { text, range: 0..1, set: vec![
241252 TextAttribute::Italic(true),
242253 TextAttribute::Font("Consolas".into()),
243 ],
244 )
254 ], clear: Vec::new() })
245255 .unwrap();
246256 oracle(PARAGRAPHS, space, reference.as_bytes());
247257}
......@@ -257,14 +267,13 @@ fn paragraph_insertions_match_insertion() {
257267 .find(|p| p.parent.is_none())
258268 .unwrap()
259269 .id;
260 for intent in [
261 Insertion::paragraph(outline.id, None, "Appended 🦋 é", AUTHOR).unwrap(),
262 Insertion::paragraph(outline.id, Some(paragraphs[0].0), "First", AUTHOR).unwrap(),
263 Insertion::paragraph(parent, None, "Nested child", AUTHOR).unwrap(),
264 Insertion::paragraph(outline.id, None, "", AUTHOR).unwrap(),
270 for op in [
271 insert(outline.id, None, "Appended 🦋 é", 1),
272 insert(outline.id, Some(paragraphs[0].0), "First", 1),
273 insert(parent, None, "Nested child", 2),
274 insert(outline.id, None, "", 1),
265275 ] {
266 let reference = PreparedEdit::insert(OUTLINES, space, &intent).unwrap();
267 oracle(OUTLINES, space, reference.as_bytes());
276 oracle(OUTLINES, space, &edited(OUTLINES, space, vec![op]));
268277 }
269278}
270279
......@@ -281,27 +290,23 @@ fn cell_insertion_and_deletion_match() {
281290 })
282291 .unwrap();
283292 let cell = &table.rows[0].cells[0];
284 let intent = Insertion::paragraph(cell.id, None, "Cell text", AUTHOR).unwrap();
285 let reference = PreparedEdit::insert(TREES, space, &intent).unwrap();
286 oracle(TREES, space, reference.as_bytes());
293 let reference = edited(TREES, space, vec![insert(cell.id, None, "Cell text", 1)]);
294 oracle(TREES, space, &reference);
287295 let victim = cell
288296 .paragraphs
289297 .iter()
290298 .find(|p| p.parent.is_none())
291299 .unwrap()
292300 .id;
293 let reference =
294 PreparedEdit::tree(TREES, space, &TreeEdit::delete(victim, AUTHOR).unwrap()).unwrap();
295 oracle(TREES, space, reference.as_bytes());
301 let reference = edited(&reference, space, vec![PageOp::Delete { object: victim }]);
302 oracle(TREES, space, &reference);
296303}
297304
298305#[test]
299306fn outline_insertion_matches_insertion() {
300307 let (space, _) = page_by_title(OUTLINES, "Move leaf down");
301 let page = page_id(OUTLINES, space);
302 let intent = Insertion::outline(page, 144.0, 200.0, "New outline 東京", AUTHOR).unwrap();
303 let reference = PreparedEdit::insert(OUTLINES, space, &intent).unwrap();
304 oracle(OUTLINES, space, reference.as_bytes());
308 let (add, ..) = ops::new_outline(144.0, 200.0, "New outline 東京");
309 oracle(OUTLINES, space, &edited(OUTLINES, space, vec![add]));
305310}
306311
307312#[test]
......@@ -325,16 +330,24 @@ fn splits_and_joins_match_their_writers() {
325330 .is_none_or(|u| !(0xd800..=0xdbff).contains(&u))
326331 })
327332 .unwrap();
328 let split = ParagraphSplit::new(text, offset, AUTHOR).unwrap();
329 let reference = PreparedEdit::split(PARAGRAPHS, space, &split).unwrap();
330 oracle(PARAGRAPHS, space, reference.as_bytes());
333 let lists = body(&page)[0].paragraphs[0].lists.len();
334 let split = PageOp::Split {
335 text,
336 at: offset,
337 paragraph: new_id().unwrap(),
338 right: new_id().unwrap(),
339 lists: (0..lists).map(|_| new_id().unwrap()).collect(),
340 };
341 oracle(PARAGRAPHS, space, &edited(PARAGRAPHS, space, vec![split]));
331342 }
332343 for title in ["Split middle", "Split empty"] {
333344 let (space, page) = page_by_title(PARAGRAPHS, title);
334345 let paragraphs = text_ids(body(&page)[0]);
335 let join = ParagraphJoin::new(paragraphs[0].1, paragraphs[1].1, AUTHOR).unwrap();
336 let reference = PreparedEdit::join(PARAGRAPHS, space, &join).unwrap();
337 oracle(PARAGRAPHS, space, reference.as_bytes());
346 let join = PageOp::Join {
347 left: paragraphs[0].1,
348 right: paragraphs[1].1,
349 };
350 oracle(PARAGRAPHS, space, &edited(PARAGRAPHS, space, vec![join]));
338351 }
339352}
340353
......@@ -357,18 +370,28 @@ fn subtree_moves_and_deletions_match_tree_edits() {
357370 .filter(|p| p.parent.is_none())
358371 .map(|p| p.id)
359372 .collect();
360 let edit = match build {
361 0 | 1 => TreeEdit::move_to(top[0], outline.id, top.get(2).copied(), AUTHOR).unwrap(),
362 2 | 3 => TreeEdit::delete(top[0], AUTHOR).unwrap(),
363 4 => TreeEdit::move_to(outline.id, page_id(OUTLINES, space), None, AUTHOR).unwrap(),
364 _ => TreeEdit::delete(outline.id, AUTHOR).unwrap(),
373 let op = match build {
374 0 | 1 => PageOp::Move {
375 object: top[0],
376 parent: Some(outline.id),
377 before: top.get(2).copied(),
378 },
379 2 | 3 => PageOp::Delete { object: top[0] },
380 4 => PageOp::Move {
381 object: outline.id,
382 parent: None,
383 before: None,
384 },
385 _ => PageOp::Delete { object: outline.id },
365386 };
366 let reference = PreparedEdit::tree(OUTLINES, space, &edit).unwrap();
367 oracle(OUTLINES, space, reference.as_bytes());
387 oracle(OUTLINES, space, &edited(OUTLINES, space, vec![op]));
368388 if outlines.len() > 1 && build == 0 {
369 let across = TreeEdit::move_to(top[1], outlines[1].id, None, AUTHOR).unwrap();
370 let reference = PreparedEdit::tree(OUTLINES, space, &across).unwrap();
371 oracle(OUTLINES, space, reference.as_bytes());
389 let across = PageOp::Move {
390 object: top[1],
391 parent: Some(outlines[1].id),
392 before: None,
393 };
394 oracle(OUTLINES, space, &edited(OUTLINES, space, vec![across]));
372395 }
373396 }
374397}
......@@ -401,7 +424,7 @@ fn outline_layout_and_collapse_match_outline_edits() {
401424 ),
402425 (parent, OutlineEdit::Collapsed(true)),
403426 ] {
404 let reference = PreparedEdit::outline(OUTLINES, space, object, edit).unwrap();
427 let reference = ops::page_op(OUTLINES, space, PageOp::Outline { object, edit }).unwrap();
405428 oracle(OUTLINES, space, reference.as_bytes());
406429 }
407430}
......@@ -411,38 +434,41 @@ fn composed_edits_publish_one_transaction() {
411434 let (space, page) = page_by_title(OUTLINES, "Move subtree up");
412435 let outline = body(&page)[0];
413436 let paragraphs = text_ids(outline);
414 let mut reference =
415 onestore::replace_text(OUTLINES, space, paragraphs[0].1, 0..0, "Lead ").unwrap();
416 let insertion =
417 Insertion::paragraph(outline.id, Some(paragraphs[1].0), "Inserted", AUTHOR).unwrap();
418 reference = PreparedEdit::insert(&reference, space, &insertion)
419 .unwrap()
420 .as_bytes()
421 .to_vec();
422 reference = PreparedEdit::format(
437 let mut reference = edited(
438 OUTLINES,
439 space,
440 vec![PageOp::Text {
441 text: paragraphs[0].1,
442 range: 0..0,
443 with: "Lead ".into(),
444 }],
445 );
446 let insertion = insert(outline.id, Some(paragraphs[1].0), "Inserted", 1);
447 let PageOp::Insert { paragraphs: inserted, .. } = &insertion else {
448 unreachable!()
449 };
450 let inserted = inserted[0].text().unwrap().id;
451 reference = edited(&reference, space, vec![insertion]);
452 reference = edited(
423453 &reference,
424454 space,
425 insertion.text_object(),
426 0..3,
427 &[TextAttribute::Bold(true)],
428 )
429 .unwrap()
430 .as_bytes()
431 .to_vec();
455 vec![PageOp::Format {
456 text: inserted,
457 range: 0..3,
458 set: vec![TextAttribute::Bold(true)],
459 clear: Vec::new(),
460 }],
461 );
432462 let last = paragraphs.last().unwrap().0;
433 reference = PreparedEdit::tree(&reference, space, &TreeEdit::delete(last, AUTHOR).unwrap())
434 .unwrap()
435 .as_bytes()
436 .to_vec();
437 reference = PreparedEdit::outline(
463 reference = edited(&reference, space, vec![PageOp::Delete { object: last }]);
464 reference = edited(
438465 &reference,
439466 space,
440 outline.id,
441 OutlineEdit::Position { x: 60.0, y: 60.0 },
442 )
443 .unwrap()
444 .as_bytes()
445 .to_vec();
467 vec![PageOp::Outline {
468 object: outline.id,
469 edit: OutlineEdit::Position { x: 60.0, y: 60.0 },
470 }],
471 );
446472 let written = oracle(OUTLINES, space, &reference);
447473 assert_eq!(
448474 Store::parse(&reference).unwrap().header.transaction_count,
......@@ -535,8 +561,8 @@ fn model_edits_round_trip_and_leave_other_objects_untouched() {
535561 );
536562 outline.paragraphs.push(paragraph);
537563 after.objects.push(PageObject::Outline(outline));
538 let written = PreparedEdit::page(TREES, space, &after, AUTHOR).unwrap();
539 let stored = page_in(written.as_bytes(), space);
564 let written = ops::saved(TREES, space, &after).unwrap();
565 let stored = page_in(written.as_slice(), space);
540566 let normalize = |page: &Page| -> Vec<(ExGuid, Vec<(ExGuid, String)>)> {
541567 body(page)
542568 .iter()
......@@ -565,7 +591,7 @@ fn model_edits_round_trip_and_leave_other_objects_untouched() {
565591 let store = Store::parse(TREES).unwrap();
566592 let index = RevisionIndex::parse(&store).unwrap();
567593 let raw_before = index.resolve_active(space).unwrap();
568 let store = Store::parse(written.as_bytes()).unwrap();
594 let store = Store::parse(written.as_slice()).unwrap();
569595 let index = RevisionIndex::parse(&store).unwrap();
570596 let raw_after = index.resolve_active(space).unwrap();
571597 let touched: BTreeSet<ExGuid> = body(&after)
......@@ -604,7 +630,7 @@ fn unsupported_model_edits_are_rejected_before_writing() {
604630 }
605631 }
606632 }
607 assert!(PreparedEdit::page(TREES, space, &widened, AUTHOR).is_ok());
633 assert!(ops::saved(TREES, space, &widened).is_ok());
608634 let mut relisted = page_in(TREES, space);
609635 let outline = body(&page)[0];
610636 for object in &mut relisted.objects {
......@@ -614,11 +640,33 @@ fn unsupported_model_edits_are_rejected_before_writing() {
614640 o.paragraphs[0].lists.push(new_id().unwrap());
615641 }
616642 }
617 assert!(PreparedEdit::page(TREES, space, &relisted, AUTHOR).is_err());
618 let mut renamed = page_in(TREES, space);
619 renamed.created = Some(1);
620 assert!(PreparedEdit::page(TREES, space, &renamed, AUTHOR).is_err());
621 assert!(PreparedEdit::page(TREES, space, &page, "a\0b").is_err());
643 assert!(ops::saved(TREES, space, &relisted).is_err());
644 // A page's date is an op of its own now.
645 let mut redated = page_in(TREES, space);
646 redated.created = Some(1);
647 let written = ops::saved(TREES, space, &redated).unwrap();
648 assert_eq!(page_in(&written, space).created, Some(1));
649 let text = page
650 .objects
651 .iter()
652 .find_map(|object| match object {
653 PageObject::Title(title) => title
654 .outlines
655 .iter()
656 .flat_map(|outline| &outline.paragraphs)
657 .find_map(|paragraph| paragraph.text().map(|text| text.id)),
658 _ => None,
659 })
660 .unwrap();
661 let op = onestore::op::Op::Page {
662 space,
663 op: PageOp::Text {
664 text,
665 range: 0..0,
666 with: "x".into(),
667 },
668 };
669 assert!(ops::transaction(TREES, "a\0b", vec![op]).is_err());
622670}
623671
624672/// Unset run values stay unset; only the language, which MS-ONE requires on every run, comes
......@@ -655,8 +703,8 @@ fn a_new_paragraph_with_unspecified_formatting_rereads_with_only_the_required_la
655703 unsupported: Vec::new(),
656704 };
657705 page.objects.insert(0, PageObject::Outline(outline));
658 let written = PreparedEdit::page(OUTLINES, space, &page, AUTHOR).unwrap();
659 let stored = page_in(written.as_bytes(), space);
706 let written = ops::saved(OUTLINES, space, &page).unwrap();
707 let stored = page_in(written.as_slice(), space);
660708 let read_back = body(&stored)
661709 .iter()
662710 .flat_map(|outline| outline.paragraphs.iter())
......@@ -673,18 +721,18 @@ fn a_new_paragraph_with_unspecified_formatting_rereads_with_only_the_required_la
673721 }
674722 );
675723 assert_eq!(
676 PreparedEdit::page(written.as_bytes(), space, &stored, AUTHOR)
724 ops::saved(written.as_slice(), space, &stored)
677725 .unwrap()
678 .as_bytes(),
679 written.as_bytes()
726 .as_slice(),
727 written.as_slice()
680728 );
681729}
682730
683731#[test]
684732fn an_unchanged_model_publishes_nothing() {
685733 let (space, page) = page_by_title(OUTLINES, "Resize outline");
686 let prepared = PreparedEdit::page(OUTLINES, space, &page, AUTHOR).unwrap();
687 assert_eq!(prepared.as_bytes(), OUTLINES);
734 let prepared = ops::saved(OUTLINES, space, &page).unwrap();
735 assert_eq!(prepared.as_slice(), OUTLINES);
688736}
689737
690738const JOIN_PAGE: &str = "Bold 🦀 italic e\u{301} color 東京";
......@@ -1001,8 +1049,8 @@ fn export_native_page_model_candidates() {
10011049 });
10021050 }
10031051 }
1004 let edit = PreparedEdit::page(&bytes, space, &after, "Rust page writer").unwrap();
1005 bytes = edit.as_bytes().to_vec();
1052 let edit = ops::saved(&bytes, space, &after).unwrap();
1053 bytes = edit.as_slice().to_vec();
10061054 let stored = page_in(&bytes, space);
10071055 let index = {
10081056 let store = Store::parse(&bytes).unwrap();
crates/onestore/tests/paragraph.rs+164-104
......@@ -1,6 +1,8 @@
11use onestore::{
2 ExGuid, ParagraphJoin, ParagraphSplit, PreparedEdit, RevisionIndex, Store,
2 ExGuid, RevisionIndex, Store,
33 document::{Document, Kind, Revision},
4 op::{Op, OpError, PageOp},
5 page::text::new_id,
46};
57use serde_json::Value;
68
......@@ -10,6 +12,40 @@ mod checkpoint;
1012mod current;
1113#[path = "support/disk.rs"]
1214mod disk;
15#[path = "support/ops.rs"]
16mod ops;
17
18/// Enter at `at` in `text`, with fresh identities for the new paragraph, its text and a
19/// copy of each list node of the paragraph: the op, the new paragraph and its text.
20fn split(source: &[u8], space: ExGuid, text: ExGuid, at: u32) -> (Op, ExGuid, ExGuid) {
21 let store = Store::parse(source).unwrap();
22 let index = RevisionIndex::parse(&store).unwrap();
23 let document = Document::parse(&index).unwrap();
24 let lists = document.spaces[&space]
25 .active()
26 .unwrap()
27 .nodes
28 .values()
29 .find_map(|node| match &node.kind {
30 Kind::Paragraph { lists, .. } if node.content == [text] => Some(lists.len()),
31 _ => None,
32 })
33 .unwrap_or(0);
34 let (paragraph, right) = (new_id().unwrap(), new_id().unwrap());
35 let op = PageOp::Split {
36 text,
37 at,
38 paragraph,
39 right,
40 lists: (0..lists).map(|_| new_id().unwrap()).collect(),
41 };
42 (Op::Page { space, op }, paragraph, right)
43}
44
45/// `source` after joining the paragraph of text `right` to the one of text `left`.
46fn join(source: &[u8], space: ExGuid, left: ExGuid, right: ExGuid) -> Result<Vec<u8>, OpError> {
47 ops::edited(source, vec![Op::Page { space, op: PageOp::Join { left, right } }])
48}
1349
1450const SOURCE: &[u8] =
1551 include_bytes!("../../../corpus/paragraph-edit/before/notebook/synthetic.one");
......@@ -114,12 +150,9 @@ fn joins_match_native_graphs_tags_and_inherited_character_styles() {
114150 .contains_key(&left)
115151 })
116152 .unwrap();
117 let intent = ParagraphJoin::new(left, right, "Join author").unwrap();
118 let restored = serde_json::from_value(serde_json::to_value(&intent).unwrap()).unwrap();
119 assert_eq!(intent, restored);
120 let edited = PreparedEdit::join(source, *sid, &restored)
121 .unwrap_or_else(|error| panic!("{name}: {error}"));
122 let current_store = Store::parse(edited.as_bytes()).unwrap();
153 let edited =
154 join(source, *sid, left, right).unwrap_or_else(|error| panic!("{name}: {error}"));
155 let current_store = Store::parse(&edited).unwrap();
123156 assert_eq!(
124157 current_store.header.transaction_count,
125158 store.header.transaction_count + 1
......@@ -276,7 +309,7 @@ fn joins_match_native_graphs_tags_and_inherited_character_styles() {
276309 }
277310 }
278311 }
279 assert!(PreparedEdit::join(edited.as_bytes(), *sid, &intent).is_err());
312 assert!(join(&edited, *sid, left, right).is_err());
280313 assert_eq!(
281314 space.contexts.len(),
282315 current_document.spaces[sid].contexts.len()
......@@ -321,10 +354,8 @@ fn splits_partition_native_paragraphs_at_every_scalar_boundary() {
321354 }))
322355 .collect();
323356 for (position, offset) in offsets.iter().enumerate() {
324 let intent = ParagraphSplit::new(text, *offset, "Split author").unwrap();
325 let restored = serde_json::from_value(serde_json::to_value(&intent).unwrap()).unwrap();
326 assert_eq!(intent, restored);
327 let edited = PreparedEdit::split(SOURCE, *sid, &restored);
357 let (op, new_paragraph, new_text) = split(SOURCE, *sid, text, *offset);
358 let edited = ops::edited(SOURCE, vec![op.clone()]);
328359 // OneNote 2010 splits before or after a hyperlink and ignores Enter inside one.
329360 let link = |at: usize| {
330361 expected
......@@ -341,7 +372,7 @@ fn splits_partition_native_paragraphs_at_every_scalar_boundary() {
341372 }
342373 let edited =
343374 edited.unwrap_or_else(|error| panic!("{} at {offset}: {error}", case["case"]));
344 let current_store = Store::parse(edited.as_bytes()).unwrap();
375 let current_store = Store::parse(&edited).unwrap();
345376 assert_eq!(
346377 current_store.header.transaction_count,
347378 store.header.transaction_count + 1
......@@ -384,7 +415,7 @@ fn splits_partition_native_paragraphs_at_every_scalar_boundary() {
384415 case["case"]
385416 );
386417 assert_eq!(
387 characters(after, intent.text_object()),
418 characters(after, new_text),
388419 expected[position..],
389420 "{} suffix at {offset}",
390421 case["case"]
......@@ -392,7 +423,7 @@ fn splits_partition_native_paragraphs_at_every_scalar_boundary() {
392423 if case["offset_utf16"] == *offset {
393424 let native = &native_document.spaces[sid];
394425 let native = &native.revisions[&native.contexts[&ExGuid::default()]];
395 let new_text = serde_json::from_value(case["new_text"].clone()).unwrap();
426 let native_text = serde_json::from_value(case["new_text"].clone()).unwrap();
396427 assert_eq!(
397428 characters(after, text),
398429 characters(native, text),
......@@ -400,8 +431,8 @@ fn splits_partition_native_paragraphs_at_every_scalar_boundary() {
400431 case["case"]
401432 );
402433 assert_eq!(
403 characters(after, intent.text_object()),
404 characters(native, new_text),
434 characters(after, new_text),
435 characters(native, native_text),
405436 "{}",
406437 case["case"]
407438 );
......@@ -409,31 +440,31 @@ fn splits_partition_native_paragraphs_at_every_scalar_boundary() {
409440 let mut children = parent_node.children.clone();
410441 children.insert(
411442 children.iter().position(|id| *id == paragraph).unwrap() + 1,
412 intent.object(),
443 new_paragraph,
413444 );
414445 assert_eq!(after.nodes[parent].children, children);
415446 assert_eq!(after.nodes[&paragraph].content, [text]);
416447 assert!(after.nodes[&paragraph].children.is_empty());
417448 assert_eq!(
418 after.nodes[&intent.object()].content,
419 [intent.text_object()]
449 after.nodes[&new_paragraph].content,
450 [new_text]
420451 );
421452 assert_eq!(
422 after.nodes[&intent.object()].children,
453 after.nodes[&new_paragraph].children,
423454 before.nodes[&paragraph].children
424455 );
425456 assert_eq!(
426457 serde_json::to_value(&after.nodes[&text].tags).unwrap(),
427458 serde_json::to_value(&before.nodes[&text].tags).unwrap()
428459 );
429 assert!(after.nodes[&intent.text_object()].tags.is_empty());
460 assert!(after.nodes[&new_text].tags.is_empty());
430461 let Kind::Paragraph {
431462 lists: old_lists, ..
432463 } = &before.nodes[&paragraph].kind
433464 else {
434465 panic!()
435466 };
436 let Kind::Paragraph { lists, .. } = &after.nodes[&intent.object()].kind else {
467 let Kind::Paragraph { lists, .. } = &after.nodes[&new_paragraph].kind else {
437468 panic!()
438469 };
439470 assert_eq!(old_lists.len(), lists.len());
......@@ -453,26 +484,23 @@ fn splits_partition_native_paragraphs_at_every_scalar_boundary() {
453484 }
454485 }
455486 }
456 assert!(PreparedEdit::split(edited.as_bytes(), *sid, &intent).is_err());
487 assert!(ops::edited(&edited, vec![op]).is_err());
457488 }
458489 for offset in 0..=*offsets.last().unwrap() + 1 {
459490 if offsets.contains(&offset) {
460491 continue;
461492 }
462 let intent = ParagraphSplit::new(text, offset, "Author").unwrap();
463 assert!(PreparedEdit::split(SOURCE, *sid, &intent).is_err());
493 assert!(ops::edited(SOURCE, vec![split(SOURCE, *sid, text, offset).0]).is_err());
464494 }
465495 }
466496}
467497
468498#[test]
469499fn invalid_split_identities_and_title_targets_are_rejected() {
470 assert!(ParagraphSplit::new(ExGuid::default(), 0, "Author").is_err());
471500 let manifest: Value =
472501 serde_json::from_str(include_str!("../../../corpus/paragraph-edit/manifest.json")).unwrap();
473502 let text: ExGuid =
474503 serde_json::from_value(manifest["cases"][0]["original_text"].clone()).unwrap();
475 assert!(ParagraphSplit::new(text, 0, "a\0b").is_err());
476504 let store = Store::parse(SOURCE).unwrap();
477505 let index = RevisionIndex::parse(&store).unwrap();
478506 let document = Document::parse(&index).unwrap();
......@@ -486,18 +514,19 @@ fn invalid_split_identities_and_title_targets_are_rejected() {
486514 })
487515 .unwrap()
488516 .0;
489 let intent = ParagraphSplit::new(text, 1, "Author").unwrap();
490 for (field, value) in [
491 ("guid", serde_json::to_value([0_u8; 16]).unwrap()),
492 ("guid", serde_json::to_value(text.guid).unwrap()),
493 ("text", serde_json::to_value(ExGuid::default()).unwrap()),
494 ("author", serde_json::json!("a\0b")),
495 ("offset", serde_json::json!(u32::MAX)),
517 let (op, ..) = split(SOURCE, sid, text, 1);
518 assert!(ops::transaction(SOURCE, "a\0b", vec![op.clone()]).is_err());
519 let Op::Page { op: PageOp::Split { paragraph, right, lists, .. }, .. } = op else {
520 unreachable!()
521 };
522 for (text, at, paragraph) in [
523 (ExGuid::default(), 1, paragraph),
524 (text, u32::MAX, paragraph),
525 (text, 1, ExGuid::default()),
526 (text, 1, text),
496527 ] {
497 let mut encoded = serde_json::to_value(&intent).unwrap();
498 encoded[field] = value;
499 let forged = serde_json::from_value(encoded).unwrap();
500 assert!(PreparedEdit::split(SOURCE, sid, &forged).is_err());
528 let op = PageOp::Split { text, at, paragraph, right, lists: lists.clone() };
529 assert!(ops::page_edited(SOURCE, sid, vec![op]).is_err());
501530 }
502531 let mut titles = 0;
503532 for (sid, _) in document.pages().unwrap() {
......@@ -516,8 +545,7 @@ fn invalid_split_identities_and_title_targets_are_rejected() {
516545 let node = &view.nodes[&id];
517546 pending.extend(node.children.iter().chain(&node.content).copied());
518547 if matches!(node.kind, Kind::RichText { .. }) {
519 let intent = ParagraphSplit::new(id, 0, "Author").unwrap();
520 assert!(PreparedEdit::split(SOURCE, sid, &intent).is_err());
548 assert!(ops::edited(SOURCE, vec![split(SOURCE, sid, id, 0).0]).is_err());
521549 titles += 1;
522550 }
523551 }
......@@ -551,16 +579,14 @@ fn export_native_paragraph_splits() {
551579 })
552580 .unwrap()
553581 .0;
554 let intent = ParagraphSplit::new(
555 text,
556 serde_json::from_value(case["offset_utf16"].clone()).unwrap(),
557 "Rust split author",
558 )
559 .unwrap();
560 let edited = PreparedEdit::split(&source, sid, &intent).unwrap();
561 written.push(serde_json::json!({"case": case["case"], "intent": intent,
562 "new_paragraph": intent.object(), "new_text": intent.text_object()}));
563 source = edited.as_bytes().to_vec();
582 let at = serde_json::from_value(case["offset_utf16"].clone()).unwrap();
583 let (op, new_paragraph, new_text) = split(&source, sid, text, at);
584 let transaction = ops::transaction(&source, "Rust split author", vec![op.clone()])
585 .unwrap()
586 .unwrap();
587 written.push(serde_json::json!({"case": case["case"], "op": op,
588 "new_paragraph": new_paragraph, "new_text": new_text}));
589 transaction.apply(&mut source).unwrap();
564590 }
565591 let candidate = output.join("candidate");
566592 fs::create_dir(&candidate).unwrap();
......@@ -587,18 +613,36 @@ fn interrupted_splits_publish_a_complete_graph_or_retain_the_original() {
587613 .find(|id| matches!(view.nodes[id].kind, Kind::Outline { .. }))
588614 .unwrap();
589615 let paragraph = view.nodes[outline].children[0];
590 let insertion = onestore::Insertion::paragraph(paragraph, None, "a🦀b", "Author")
591 .unwrap()
592 .with_formatting(1..3, &[onestore::TextAttribute::Bold(true)])
593 .unwrap();
594 let inserted = PreparedEdit::insert(&original, sid, &insertion).unwrap();
595 let source = inserted.as_bytes();
596 let checkpoint = checkpoint::pending(source, sid, insertion.text_object(), 0x14001d7a);
616 let mut child = ops::paragraph("a🦀b");
617 child.level = 2;
618 let text = child.text().unwrap().id;
619 let inserted = ops::page_edited(
620 &original,
621 sid,
622 vec![
623 PageOp::Insert {
624 container: paragraph,
625 before: None,
626 paragraphs: vec![child],
627 },
628 PageOp::Format {
629 text,
630 range: 1..3,
631 set: vec![onestore::TextAttribute::Bold(true)],
632 clear: Vec::new(),
633 },
634 ],
635 )
636 .unwrap();
637 let source = inserted.as_slice();
638 let checkpoint = checkpoint::pending(source, sid, text);
597639 for source in [source, &checkpoint] {
598 let intent = ParagraphSplit::new(insertion.text_object(), 1, "Author").unwrap();
599 let edit = PreparedEdit::split(source, sid, &intent).unwrap();
640 let (op, ..) = split(source, sid, text, 1);
641 let edit = ops::transaction(source, "Author", vec![op]).unwrap().unwrap();
642 let mut written = source.to_vec();
643 edit.apply(&mut written).unwrap();
600644 let before = current::current(source);
601 let after = current::current(edit.as_bytes());
645 let after = current::current(&written);
602646 for write_limit in [17, 4096] {
603647 let disk = |fail_at| disk::Disk {
604648 visible: source.to_vec(),
......@@ -610,7 +654,7 @@ fn interrupted_splits_publish_a_complete_graph_or_retain_the_original() {
610654 };
611655 let mut successful = disk(None);
612656 edit.commit(&mut successful).unwrap();
613 assert_eq!(successful.durable, edit.as_bytes());
657 assert_eq!(successful.durable, written);
614658 for at in 1..=successful.operation {
615659 let mut interrupted = disk(Some(at));
616660 let error = edit.commit(&mut interrupted).unwrap_err();
......@@ -654,10 +698,12 @@ fn export_native_paragraph_joins() {
654698 })
655699 .unwrap()
656700 .0;
657 let intent = ParagraphJoin::new(left, right, "Rust join author").unwrap();
658 let edit = PreparedEdit::join(&source, sid, &intent).unwrap();
659 source = edit.as_bytes().to_vec();
660 manifest.push(serde_json::json!({"case":name,"space":sid,"intent":intent}));
701 let op = Op::Page { space: sid, op: PageOp::Join { left, right } };
702 let transaction = ops::transaction(&source, "Rust join author", vec![op.clone()])
703 .unwrap()
704 .unwrap();
705 transaction.apply(&mut source).unwrap();
706 manifest.push(serde_json::json!({"case":name,"space":sid,"op":op}));
661707 }
662708 let folder = output.join(["split", "inheritance", "tags"][number]);
663709 fs::create_dir_all(folder.join("candidate")).unwrap();
......@@ -685,22 +731,48 @@ fn interrupted_joins_preserve_complete_graphs_and_empty_text_adoption() {
685731 .find(|id| matches!(view.nodes[id].kind, Kind::Outline { .. }))
686732 .unwrap();
687733 let left = view.nodes[&view.nodes[&outline].children[0]].content[0];
688 let right = onestore::Insertion::paragraph(outline, None, "Right 🦀", "Author")
689 .unwrap()
690 .with_formatting(0..5, &[onestore::TextAttribute::Bold(true)])
691 .unwrap();
692 let inserted = PreparedEdit::insert(&source, sid, &right).unwrap();
693 let child =
694 onestore::Insertion::paragraph(right.object(), None, "Retained child", "Author").unwrap();
695 let original = PreparedEdit::insert(inserted.as_bytes(), sid, &child).unwrap();
696 let original = original.as_bytes();
697 let empty = onestore::replace_text(original, sid, left, 0..4, "").unwrap();
698 let checkpoint = checkpoint::pending(original, sid, left, 0x14001d7a);
734 let right = ops::paragraph("Right 🦀");
735 let right_text = right.text().unwrap().id;
736 let mut child = ops::paragraph("Retained child");
737 child.parent = Some(right.id);
738 child.level = 2;
739 let original = ops::page_edited(
740 &source,
741 sid,
742 vec![
743 PageOp::Insert {
744 container: outline,
745 before: None,
746 paragraphs: vec![right, child],
747 },
748 PageOp::Format {
749 text: right_text,
750 range: 0..5,
751 set: vec![onestore::TextAttribute::Bold(true)],
752 clear: Vec::new(),
753 },
754 ],
755 )
756 .unwrap();
757 let original = original.as_slice();
758 let empty = ops::page_edited(
759 original,
760 sid,
761 vec![PageOp::Text {
762 text: left,
763 range: 0..4,
764 with: String::new(),
765 }],
766 )
767 .unwrap();
768 let checkpoint = checkpoint::pending(original, sid, left);
699769 for source in [original, &empty, &checkpoint] {
700 let intent = ParagraphJoin::new(left, right.text_object(), "Join author").unwrap();
701 let edit = PreparedEdit::join(source, sid, &intent).unwrap();
770 let op = Op::Page { space: sid, op: PageOp::Join { left, right: right_text } };
771 let edit = ops::transaction(source, "Join author", vec![op]).unwrap().unwrap();
772 let mut written = source.to_vec();
773 edit.apply(&mut written).unwrap();
702774 let before = current::current(source);
703 let after = current::current(edit.as_bytes());
775 let after = current::current(&written);
704776 for write_limit in [17, 4096] {
705777 let disk = |fail_at| disk::Disk {
706778 visible: source.to_vec(),
......@@ -712,8 +784,8 @@ fn interrupted_joins_preserve_complete_graphs_and_empty_text_adoption() {
712784 };
713785 let mut successful = disk(None);
714786 edit.commit(&mut successful).unwrap();
715 assert_eq!(successful.durable, edit.as_bytes());
716 for at in std::iter::once(1).chain(source.len().div_ceil(193)..=successful.operation) {
787 assert_eq!(successful.durable, written);
788 for at in 1..=successful.operation {
717789 let mut interrupted = disk(Some(at));
718790 let error = edit.commit(&mut interrupted).unwrap_err();
719791 let recovered = current::current(&interrupted.durable);
......@@ -748,26 +820,16 @@ fn joins_reject_invalid_identities_wrong_order_and_unrelated_pages() {
748820 .contains_key(left)
749821 })
750822 .unwrap();
751 assert!(ParagraphJoin::new(*left, *left, "Author").is_err());
752 assert!(ParagraphJoin::new(ExGuid::default(), *right, "Author").is_err());
753 assert!(ParagraphJoin::new(*left, *right, "a\0b").is_err());
754 let intent = ParagraphJoin::new(*left, *right, "Author").unwrap();
755 for (field, value) in [
756 ("left", serde_json::to_value(ExGuid::default()).unwrap()),
757 ("right", serde_json::to_value(left).unwrap()),
758 ("author", serde_json::json!("a\0b")),
759 ] {
760 let mut encoded = serde_json::to_value(&intent).unwrap();
761 encoded[field] = value;
762 let invalid = serde_json::from_value(encoded).unwrap();
763 assert!(PreparedEdit::join(source, *sid, &invalid).is_err());
764 }
765 for intent in [
766 ParagraphJoin::new(*right, *left, "Author").unwrap(),
767 ParagraphJoin::new(*left, cases[1].2, "Author").unwrap(),
823 for (a, b) in [
824 (*left, *left),
825 (ExGuid::default(), *right),
826 (*right, *left),
827 (*left, cases[1].2),
768828 ] {
769 assert!(PreparedEdit::join(source, *sid, &intent).is_err());
829 assert!(join(source, *sid, a, b).is_err());
770830 }
831 let op = Op::Page { space: *sid, op: PageOp::Join { left: *left, right: *right } };
832 assert!(ops::transaction(source, "a\0b", vec![op]).is_err());
771833 let view = &space.revisions[&space.contexts[&ExGuid::default()]];
772834 let paragraph = *view
773835 .nodes
......@@ -781,8 +843,7 @@ fn joins_reject_invalid_identities_wrong_order_and_unrelated_pages() {
781843 .find(|node| node.children.contains(&paragraph))
782844 .unwrap();
783845 let last = view.nodes[parent.children.last().unwrap()].content[0];
784 let nonadjacent = ParagraphJoin::new(*left, last, "Author").unwrap();
785 assert!(PreparedEdit::join(source, *sid, &nonadjacent).is_err());
846 assert!(join(source, *sid, *left, last).is_err());
786847 let title = view
787848 .nodes
788849 .iter()
......@@ -794,8 +855,7 @@ fn joins_reject_invalid_identities_wrong_order_and_unrelated_pages() {
794855 let node = &view.nodes[&id];
795856 pending.extend(node.children.iter().chain(&node.content).copied());
796857 if matches!(node.kind, Kind::RichText { .. }) {
797 let intent = ParagraphJoin::new(id, *right, "Author").unwrap();
798 assert!(PreparedEdit::join(source, *sid, &intent).is_err());
858 assert!(join(source, *sid, id, *right).is_err());
799859 rejected += 1;
800860 }
801861 }
crates/onestore/tests/paragraph_formatting.rs+9-7
......@@ -1,5 +1,7 @@
1#[path = "support/ops.rs"]
2mod ops;
13use onestore::{
2 ExGuid, PreparedEdit, RevisionIndex, Store,
4 ExGuid, RevisionIndex, Store,
35 document::{Document, Kind},
46 page::{Page, PageObject, Paragraph},
57};
......@@ -48,12 +50,12 @@ fn paragraph_properties_preserve_native_mixed_runs_and_shared_styles() {
4850 }
4951 }
5052 assert_eq!(count, 6);
51 let prepared = PreparedEdit::page(SOURCE, space, &after, "Format author").unwrap();
52 assert_eq!(page(prepared.as_bytes()).1, after);
53 let prepared = ops::saved(SOURCE, space, &after).unwrap();
54 assert_eq!(page(prepared.as_slice()).1, after);
5355 let before_store = Store::parse(SOURCE).unwrap();
5456 let before_index = RevisionIndex::parse(&before_store).unwrap();
5557 let before_doc = Document::parse(&before_index).unwrap();
56 let after_store = Store::parse(prepared.as_bytes()).unwrap();
58 let after_store = Store::parse(prepared.as_slice()).unwrap();
5759 let after_index = RevisionIndex::parse(&after_store).unwrap();
5860 let old = before_index.resolve_active(space).unwrap();
5961 let new = after_index.resolve_active(space).unwrap();
......@@ -65,7 +67,7 @@ fn paragraph_properties_preserve_native_mixed_runs_and_shared_styles() {
6567 if let Some(directory) = std::env::var_os("ONESTORE_PARAGRAPH_FORMAT_EXPORT") {
6668 let directory = std::path::PathBuf::from(directory);
6769 std::fs::create_dir(&directory).unwrap();
68 std::fs::write(directory.join("synthetic.one"), prepared.as_bytes()).unwrap();
70 std::fs::write(directory.join("synthetic.one"), prepared.as_slice()).unwrap();
6971 std::fs::write(
7072 directory.join("Open Notebook.onetoc2"),
7173 onestore::create_table_of_contents(
......@@ -103,7 +105,7 @@ fn paragraph_formatting_rejects_invalid_values_and_conflicting_run_settings() {
103105 let mut format = text.text.format_at(0).unwrap().clone();
104106 change(&mut format);
105107 text.text = Paragraph::new(text.text.text().to_owned(), format);
106 assert!(PreparedEdit::page(SOURCE, space, &after, "Author").is_err());
108 assert!(ops::saved(SOURCE, space, &after).is_err());
107109 }
108110 let mut after = before.clone();
109111 let text = after
......@@ -121,5 +123,5 @@ fn paragraph_formatting_rejects_invalid_values_and_conflicting_run_settings() {
121123 left.alignment = Some(1);
122124 right.alignment = Some(2);
123125 text.text = Paragraph::from_runs([("Left".into(), left), ("Right".into(), right)]);
124 assert!(PreparedEdit::page(SOURCE, space, &after, "Author").is_err());
126 assert!(ops::saved(SOURCE, space, &after).is_err());
125127}
crates/onestore/tests/protected.rs+27-9
......@@ -3,6 +3,7 @@
33use onestore::{
44 RevisionIndex, Store,
55 document::{Document, Kind},
6 op::{Edit, Op, PageOp},
67 protected::{Error, Limits, UnlockedSection},
78};
89use std::{fs, path::Path};
......@@ -33,10 +34,19 @@ fn known_passwords_open_independent_native_fixtures() {
3334 .iter()
3435 .find(|(_, node)| matches!(node.kind, Kind::RichText { .. }))
3536 .unwrap();
36 assert!(
37 onestore::PreparedEdit::text(&bytes, space, *object, 0..0, "Must remain protected")
38 .is_err()
39 );
37 // The ordinary section writer refuses a protected section.
38 let arena = onestore::Arena::default();
39 assert!(onestore::Section::open(&arena, bytes.clone()).is_err());
40 let op = PageOp::Text {
41 text: *object,
42 range: 0..0,
43 with: "Must remain protected".into(),
44 };
45 let edit = Edit {
46 at: 134_000_000_000_000_000,
47 ops: vec![Op::Page { space, op }],
48 };
49 assert!(unlocked.apply("Rust", &edit).is_ok());
4050 assert!(
4151 document
4252 .spaces
......@@ -139,6 +149,7 @@ fn a_protected_page_edit_is_stored_under_the_section_key() {
139149 let mut expected = pages(&bytes);
140150 let mut edited = 0;
141151 for (space, page) in &mut expected {
152 let before = page.clone();
142153 let paragraphs = page.objects.iter_mut().find_map(|object| match object {
143154 PageObject::Outline(outline)
144155 if outline.paragraphs.iter().any(|p| p.text().is_some()) =>
......@@ -174,14 +185,21 @@ fn a_protected_page_edit_is_stored_under_the_section_key() {
174185 text.text
175186 .append(Paragraph::new(" still protected".to_owned(), format))
176187 .unwrap();
177 let edit =
178 onestore::PreparedEdit::page_protected(&bytes, password, *space, page, "Rust")
179 .unwrap();
188 let ops = onestore::op::lower_page(&before, page).unwrap();
189 let edit = Edit {
190 at: 134_000_000_000_000_000,
191 ops: ops.into_iter().map(|op| Op::Page { space: *space, op }).collect(),
192 };
193 let store = Store::parse(&bytes).unwrap();
194 let index = RevisionIndex::parse(&store).unwrap();
180195 assert!(matches!(
181 onestore::PreparedEdit::page_protected(&bytes, "wrong", *space, page, "Rust"),
196 UnlockedSection::open(&index, "wrong", Limits::default()),
182197 Err(Error::PasswordMismatch)
183198 ));
184 let written = edit.as_bytes().to_vec();
199 let unlocked = UnlockedSection::open(&index, password, Limits::default()).unwrap();
200 let transaction = unlocked.apply("Rust", &edit).unwrap();
201 let mut written = bytes.clone();
202 transaction.apply(&mut written).unwrap();
185203 for clear in [&b" still protected"[..], b"stays sealed"] {
186204 assert!(!written.windows(clear.len()).any(|w| w == clear));
187205 }
crates/onestore/tests/shared_snapshot.rs+27-5
......@@ -4,6 +4,8 @@ use current::current;
44
55#[path = "support/checkpoint.rs"]
66mod checkpoint;
7#[path = "support/ops.rs"]
8mod ops;
79#[path = "support/trace.rs"]
810mod trace;
911
......@@ -15,6 +17,26 @@ use onestore::{
1517use std::{fs, io};
1618use trace::{Event, Trace};
1719
20/// Types `with` over `range` of the text object in `source` and commits it to `io`.
21fn commit_text(
22 io: &mut impl onestore::CommitIo,
23 source: &[u8],
24 space: ExGuid,
25 text: ExGuid,
26 range: std::ops::Range<u32>,
27 with: &str,
28) -> Result<(), onestore::CommitError> {
29 let op = onestore::op::PageOp::Text {
30 text,
31 range,
32 with: with.into(),
33 };
34 ops::transaction(source, "Author", vec![onestore::op::Op::Page { space, op }])
35 .unwrap()
36 .unwrap()
37 .commit(io)
38}
39
1840#[test]
1941fn storage_inspection_preserves_opaque_images_without_claiming_edit_readiness() {
2042 for path in [
......@@ -111,7 +133,7 @@ fn version_cached_readers_cannot_miss_a_completed_publication() {
111133 bytes: source.clone(),
112134 events: Vec::new(),
113135 };
114 onestore::commit_text(&mut trace, &source, sid, oid, end..end, " [cached reader]").unwrap();
136 commit_text(&mut trace, &source, sid, oid, end..end, " [cached reader]").unwrap();
115137 let final_content = current(&trace.bytes);
116138 let mut visible = source;
117139 for event in &trace.events {
......@@ -168,7 +190,7 @@ fn published_snapshots_survive_interleaved_commit_io() {
168190 let mut source = fs::read(format!("../../corpus/{path}")).unwrap();
169191 let (sid, oid, _) = target(&source);
170192 if name == "checkpoint" {
171 source = checkpoint::pending(&source, sid, oid, 0x14001d7a);
193 source = checkpoint::pending(&source, sid, oid);
172194 }
173195 let (_, _, end) = target(&source);
174196 let before = current(&source);
......@@ -176,7 +198,7 @@ fn published_snapshots_survive_interleaved_commit_io() {
176198 bytes: source.clone(),
177199 events: Vec::new(),
178200 };
179 onestore::commit_text(&mut trace, &source, sid, oid, end..end, " [reader café 🦀]")
201 commit_text(&mut trace, &source, sid, oid, end..end, " [reader café 🦀]")
180202 .unwrap();
181203 let after = current(&trace.bytes);
182204 assert_ne!(before, after);
......@@ -407,7 +429,7 @@ fn unpublished_append_remains_available_for_retry() {
407429 bytes: source.clone(),
408430 events: Vec::new(),
409431 };
410 onestore::commit_text(&mut trace, &source, sid, oid, end..end, " abandoned").unwrap();
432 commit_text(&mut trace, &source, sid, oid, end..end, " abandoned").unwrap();
411433 let Event::Write(offset, append) = &trace.events[0] else {
412434 panic!()
413435 };
......@@ -432,7 +454,7 @@ fn unpublished_append_remains_available_for_retry() {
432454 bytes: persisted,
433455 events: Vec::new(),
434456 };
435 onestore::commit_text(&mut retry, &snapshot, sid, oid, end..end, " retry").unwrap();
457 commit_text(&mut retry, &snapshot, sid, oid, end..end, " retry").unwrap();
436458 assert_ne!(current(&retry.bytes), current(&source));
437459 }
438460}
crates/onestore/tests/storage.rs+1-1
......@@ -23,7 +23,7 @@ fn native_file_data_alignment_is_relative_to_the_object() {
2323 assert_eq!(payload.len(), 10056);
2424 assert_eq!(
2525 format!("{:x}", md5::compute(payload)),
26 "e1b57a8851177dd25dc05b50b904656a"
26 "fec8f53bd6db9305eabac4cd4b71fb08"
2727 );
2828 let blob = store.chunk_data(chunk).unwrap();
2929 for alignment in 0..8 {
crates/onestore/tests/support/checkpoint.rs+34-17
......@@ -1,21 +1,38 @@
1use onestore::{ExGuid, RevisionIndex, Store};
1use onestore::{
2 Arena, ExGuid, RevisionIndex, Section, Store,
3 op::{Edit, Op, PageOp},
4};
25
3pub fn pending(source: &[u8], sid: ExGuid, oid: ExGuid, property: u32) -> Vec<u8> {
4 let mut source = source.to_vec();
5 for value in 1_000_000_u32.. {
6 let store = Store::parse(&source).unwrap();
6/// `source` with revisions of `space` appended until its chain is 512 deep, so its next
7/// revision is a checkpoint. Each retouches the text object `text`, typing a character and
8/// removing it again.
9pub fn pending(source: &[u8], space: ExGuid, text: ExGuid) -> Vec<u8> {
10 let depth = |image: &[u8]| {
11 let store = Store::parse(image).unwrap();
712 let index = RevisionIndex::parse(&store).unwrap();
8 let rid = index.spaces[&sid].labels[&(ExGuid::default(), 1)];
9 let depth =
10 std::iter::successors(Some(rid), |id| index.spaces[&sid].revisions[id].dependency)
11 .count();
12 assert!(depth <= 512);
13 if depth == 512 {
14 return source;
15 }
16 source =
17 onestore::replace_property_bytes(&source, sid, oid, property, &value.to_le_bytes())
18 .unwrap();
13 let rid = index.spaces[&space].labels[&(ExGuid::default(), 1)];
14 std::iter::successors(Some(rid), |id| {
15 index.spaces[&space].revisions[id].dependency
16 })
17 .count()
18 };
19 let arena = Arena::default();
20 let mut section = Section::open(&arena, source.to_vec()).unwrap();
21 let retouch = |range, with: &str| Op::Page {
22 space,
23 op: PageOp::Text {
24 text,
25 range,
26 with: with.into(),
27 },
28 };
29 for at in 0..512 - depth(source) as u64 {
30 let ops = vec![retouch(0..0, "x"), retouch(0..1, "")];
31 let at = 134_000_000_000_000_000 + at * 10_000_000;
32 section.apply("Author", &Edit { at, ops }).unwrap();
33 section.seal().unwrap().unwrap();
1934 }
20 unreachable!()
35 let image = section.image();
36 assert_eq!(depth(&image), 512);
37 image
2138}
crates/onestore/tests/support/ops.rs created+187
......@@ -0,0 +1,187 @@
1//! Edits as ops through a `Section`, sealed: the image an edit leaves, for tests that read
2//! it back.
3#![allow(dead_code)]
4
5use onestore::{
6 Arena, ExGuid, Section, Transaction,
7 op::{Edit, Op, OpError, PageOp},
8 page::Page,
9};
10
11/// FILETIME now, as an edit's `at`.
12pub fn now() -> u64 {
13 let unix = std::time::SystemTime::now()
14 .duration_since(std::time::UNIX_EPOCH)
15 .unwrap();
16 (unix.as_secs() + 11_644_473_600) * 10_000_000 + u64::from(unix.subsec_nanos() / 100)
17}
18
19/// The transaction `ops` seal into on `source`, as one edit by `author`; none when they
20/// store nothing.
21pub fn transaction(
22 source: &[u8],
23 author: &str,
24 ops: Vec<Op>,
25) -> Result<Option<Transaction>, OpError> {
26 let arena = Arena::default();
27 let mut section = Section::open(&arena, source.to_vec()).map_err(OpError::Failed)?;
28 section.apply(author, &Edit { at: now(), ops })?;
29 section.seal().map_err(OpError::Failed)
30}
31
32/// `source` after `ops`, applied as one edit and sealed.
33pub fn edited(source: &[u8], ops: Vec<Op>) -> Result<Vec<u8>, OpError> {
34 let mut image = source.to_vec();
35 if let Some(transaction) = transaction(source, "Author", ops)? {
36 transaction.apply(&mut image).map_err(OpError::Failed)?;
37 }
38 Ok(image)
39}
40
41/// `source` after `ops` on the page in `space`.
42pub fn page_edited(source: &[u8], space: ExGuid, ops: Vec<PageOp>) -> Result<Vec<u8>, OpError> {
43 edited(
44 source,
45 ops.into_iter().map(|op| Op::Page { space, op }).collect(),
46 )
47}
48
49/// `source` with the page in `space` taken to `after` by the ops `op::lower_page` finds,
50/// as a test states an edit by the page it leaves.
51pub fn saved(source: &[u8], space: ExGuid, after: &Page) -> Result<Vec<u8>, String> {
52 let arena = Arena::default();
53 let section = Section::open(&arena, source.to_vec()).map_err(|error| error.to_string())?;
54 let before = section.page(space).map_err(|error| error.to_string())?;
55 let ops = onestore::op::lower_page(&before, after).map_err(|error| error.to_string())?;
56 page_edited(source, space, ops).map_err(|error| error.to_string())
57}
58
59/// A plain paragraph holding `text` in OneNote's default text style, with fresh identities.
60pub fn paragraph(text: &str) -> onestore::page::PageParagraph {
61 use onestore::page::{PageParagraph, Paragraph, ParagraphContent, TextObject, text::new_id};
62 let format = onestore::document::Format {
63 font: Some("Calibri".into()),
64 font_size: Some(11.0),
65 language: Some(0x409),
66 ..Default::default()
67 };
68 PageParagraph {
69 id: new_id().unwrap(),
70 parent: None,
71 level: 1,
72 style: None,
73 format: Default::default(),
74 content: ParagraphContent::Text(TextObject {
75 id: new_id().unwrap(),
76 date_field: None,
77 text: Paragraph::new(text.into(), format),
78 tags: Vec::new(),
79 }),
80 lists: Vec::new(),
81 tags: Vec::new(),
82 media: Default::default(),
83 collapsed: false,
84 }
85}
86
87/// A new outline at `(x, y)` in points holding `paragraphs`.
88pub fn outline(
89 x: f32,
90 y: f32,
91 paragraphs: Vec<onestore::page::PageParagraph>,
92) -> onestore::page::PageObject {
93 use onestore::page::{Outline, PageObject, text::new_id};
94 PageObject::Outline(Outline {
95 id: new_id().unwrap(),
96 title: false,
97 min_width: None,
98 layout: onestore::document::Layout {
99 x: Some(x),
100 y: Some(y),
101 ..Default::default()
102 },
103 indents: Vec::new(),
104 paragraphs,
105 unsupported: Vec::new(),
106 })
107}
108
109/// An edit applied through a `Section`: the image it leaves and the transaction publishing
110/// it, none when it stores nothing.
111pub struct Applied {
112 pub image: Vec<u8>,
113 pub transaction: Option<Transaction>,
114}
115
116impl Applied {
117 pub fn as_bytes(&self) -> &[u8] {
118 &self.image
119 }
120
121 /// Publishes the transaction under the filesystem adapter's exclusion.
122 #[cfg(any(unix, windows))]
123 pub fn commit_file(
124 &self,
125 path: impl AsRef<std::path::Path>,
126 ) -> Result<(), onestore::CommitError> {
127 self.transaction
128 .as_ref()
129 .expect("the edit stores a revision")
130 .commit_file(path)
131 }
132
133 /// Publishes the transaction under caller-held exclusion.
134 pub fn commit(&self, io: &mut impl onestore::CommitIo) -> Result<(), onestore::CommitError> {
135 self.transaction
136 .as_ref()
137 .expect("the edit stores a revision")
138 .commit(io)
139 }
140}
141
142/// `ops` applied to `source` as one edit by `author`.
143pub fn apply(source: &[u8], author: &str, ops: Vec<Op>) -> Result<Applied, OpError> {
144 let transaction = transaction(source, author, ops)?;
145 let mut image = source.to_vec();
146 if let Some(transaction) = &transaction {
147 transaction.apply(&mut image).map_err(OpError::Failed)?;
148 }
149 Ok(Applied { image, transaction })
150}
151
152/// `saved` as an `Applied`, to publish it.
153pub fn save(source: &[u8], space: ExGuid, after: &Page) -> Result<Applied, String> {
154 let arena = Arena::default();
155 let section = Section::open(&arena, source.to_vec()).map_err(|error| error.to_string())?;
156 let before = section.page(space).map_err(|error| error.to_string())?;
157 let ops = onestore::op::lower_page(&before, after).map_err(|error| error.to_string())?;
158 let ops = ops.into_iter().map(|op| Op::Page { space, op }).collect();
159 apply(source, "Author", ops).map_err(|error| error.to_string())
160}
161
162/// `op` on the page in `space`, applied to `source`.
163pub fn page_op(source: &[u8], space: ExGuid, op: PageOp) -> Result<Applied, OpError> {
164 apply(source, "Author", vec![Op::Page { space, op }])
165}
166
167/// `op` on the section, applied to `source`.
168pub fn section_op(source: &[u8], op: onestore::op::SectionOp) -> Result<Applied, OpError> {
169 apply(source, "Author", vec![Op::Section(op)])
170}
171
172/// Adds an outline at `(x, y)` holding one paragraph of `text`: the op and the identities
173/// of the outline and its text.
174pub fn new_outline(x: f32, y: f32, text: &str) -> (PageOp, ExGuid, ExGuid) {
175 let paragraph = paragraph(text);
176 let text = paragraph.text().unwrap().id;
177 let object = outline(x, y, vec![paragraph]);
178 let id = object.id();
179 (
180 PageOp::Add {
181 object,
182 before: None,
183 },
184 id,
185 text,
186 )
187}
crates/onestore/tests/support/page_edits.rs+29-7
......@@ -1,8 +1,11 @@
1//! Drives random page-model mutations through the page writer and checks its contract:
2//! a published model reads back as written, and objects outside the edit keep their bytes.
1//! Drives random page-model mutations through `op::lower_page` and a `Section` and checks
2//! the contract: the page the section holds is what the model oracle predicts from the ops,
3//! the sealed image reads back as the edited model, and objects outside the edit keep their
4//! bytes.
35
46use onestore::{
5 ExGuid, PreparedEdit, RevisionIndex, Store,
7 Arena, ExGuid, RevisionIndex, Section, Store,
8 op::{self, Op},
69 document::{Document, Format, Layout},
710 page::{
811 Outline, Page, PageObject, PageParagraph, Paragraph, ParagraphContent,
......@@ -564,13 +567,31 @@ pub fn run(input: &[u8]) {
564567 let Some(mut after) = model(source, space) else {
565568 return;
566569 };
570 let before = after.clone();
567571 mutate(&mut after, &mut bytes);
568 let Ok(edit) = PreparedEdit::page(source, space, &after, "Fuzz author") else {
572 let Ok(ops) = onestore::op::lower_page(&before, &after) else {
569573 return;
570574 };
571 let written = edit.as_bytes();
572 if written == source.as_slice() {
573 let before = model(source, space).unwrap();
575 let arena = Arena::default();
576 let mut section = Section::open(&arena, source.clone()).unwrap();
577 let edit = op::Edit {
578 at: 134_000_000_000_000_000,
579 ops: ops.iter().map(|op| Op::Page { space, op: op.clone() }).collect(),
580 };
581 if section.apply("Fuzz author", &edit).is_err() {
582 assert_eq!(section.page(space).unwrap(), before, "a refused edit changes nothing");
583 return;
584 }
585 let mut predicted = before.clone();
586 for op in &ops {
587 onestore::op::predict(&mut predicted, op).unwrap();
588 }
589 assert_eq!(
590 projection(&section.page(space).unwrap()),
591 projection(&predicted),
592 "the model oracle"
593 );
594 if section.seal().unwrap().is_none() {
574595 assert_eq!(
575596 projection(&before),
576597 projection(&after),
......@@ -578,6 +599,7 @@ pub fn run(input: &[u8]) {
578599 );
579600 return;
580601 }
602 let written = &section.image();
581603 current::current(written);
582604 let stored = model(written, space).expect("the written page reads back");
583605 let (stored, expected) = (projection(&stored), projection(&after));
crates/onestore/tests/support/page_schedule.rs+19-19
......@@ -1,6 +1,7 @@
11use onestore::{
2 CommitState, ExGuid, Insertion, PageCreation, PageEdit, PreparedEdit, RevisionIndex, Store,
2 CommitState, ExGuid, PageCreation, PageEdit, RevisionIndex, Store,
33 document::{Document, FieldValue, Kind},
4 op::{PageOp, SectionOp},
45};
56use std::{collections::BTreeMap, sync::LazyLock};
67
......@@ -8,6 +9,8 @@ use std::{collections::BTreeMap, sync::LazyLock};
89pub(crate) mod current;
910#[path = "disk.rs"]
1011pub(crate) mod disk;
12#[path = "ops.rs"]
13pub(crate) mod ops;
1114
1215static SOURCE: LazyLock<Vec<u8>> = LazyLock::new(|| {
1316 onestore::create_section("pages.one", "Original 🦀 é 東京", "Author").unwrap()
......@@ -24,17 +27,17 @@ pub fn run(input: &[u8]) {
2427 _ => SOURCE.as_slice(),
2528 };
2629 if let Ok(intent) = serde_json::from_slice::<PageCreation>(input)
27 && let Ok(prepared) = PreparedEdit::create_page(source, &intent)
30 && let Ok(prepared) = ops::section_op(source, SectionOp::Create(intent.clone()))
2831 {
2932 current::current(prepared.as_bytes());
3033 }
3134 if let Ok(edits) = serde_json::from_slice::<Vec<PageEdit>>(input)
32 && let Ok(prepared) = PreparedEdit::pages(source, &edits)
35 && let Ok(prepared) = ops::section_op(source, SectionOp::Pages(edits.to_vec()))
3336 {
3437 current::current(prepared.as_bytes());
3538 }
3639 if let Ok(pages) = serde_json::from_slice::<Vec<ExGuid>>(input)
37 && let Ok(prepared) = PreparedEdit::delete_pages_permanently(source, &pages)
40 && let Ok(prepared) = ops::section_op(source, SectionOp::Delete(pages.to_vec()))
3841 {
3942 current::current(prepared.as_bytes());
4043 }
......@@ -65,7 +68,7 @@ pub fn run(input: &[u8]) {
6568 .get(usize::from(step[2]) % pages.len().max(1))
6669 .copied();
6770 let text = ["", "Same title", "é 🦋 東京", " spaces "][usize::from(step[3]) % 4];
68 let existing = if let Some((sid, page)) = selected {
71 let existing = if let Some((sid, _)) = selected {
6972 let space = &document.spaces[&sid];
7073 let view = &space.revisions[&space.contexts[&ExGuid::default()]];
7174 let titles: Vec<_> = view
......@@ -91,7 +94,7 @@ pub fn run(input: &[u8]) {
9194 let removed: Vec<_> = (0..count)
9295 .map(|i| pages[(usize::from(step[2]) + i) % pages.len()].0)
9396 .collect();
94 let prepared = PreparedEdit::delete_pages_permanently(source, &removed).unwrap();
97 let prepared = ops::section_op(source, SectionOp::Delete(removed.to_vec())).unwrap();
9598 pages.retain(|page| !removed.contains(&page.0));
9699 for sid in removed {
97100 levels.remove(&sid);
......@@ -128,7 +131,7 @@ pub fn run(input: &[u8]) {
128131 let restored: Vec<PageEdit> =
129132 serde_json::from_value(serde_json::to_value(&edits).unwrap()).unwrap();
130133 assert_eq!(edits, restored);
131 let prepared = PreparedEdit::pages(source, &restored);
134 let prepared = ops::section_op(source, SectionOp::Pages(restored.to_vec()));
132135 if levels[&pages[0].0] != 1 {
133136 assert!(prepared.is_err());
134137 continue;
......@@ -140,21 +143,15 @@ pub fn run(input: &[u8]) {
140143 unreachable!()
141144 };
142145 Some((
143 PreparedEdit::text(
144 source,
145 sid,
146 object,
147 0..u32::try_from(original.encode_utf16().count()).unwrap(),
148 text,
149 )
146 ops::page_op(source, sid, PageOp::Text { text: object, range: 0..u32::try_from(original.encode_utf16().count()).unwrap(), with: text.into() })
150147 .unwrap(),
151148 Some((sid, object, text)),
152149 ))
153150 } else if step[1] & 16 != 0 {
154 let intent = Insertion::outline(page, 36.0, 36.0, text, "Page fuzz").unwrap();
151 let (add, _, inserted) = ops::new_outline(36.0, 36.0, text);
155152 Some((
156 PreparedEdit::insert(source, sid, &intent).unwrap(),
157 Some((sid, intent.text_object(), text)),
153 ops::page_op(source, sid, add).unwrap(),
154 Some((sid, inserted, text)),
158155 ))
159156 } else {
160157 None
......@@ -182,7 +179,7 @@ pub fn run(input: &[u8]) {
182179 let intent = PageCreation::new(before, title, "Page fuzz").unwrap();
183180 let restored = serde_json::from_value(serde_json::to_value(&intent).unwrap()).unwrap();
184181 assert_eq!(intent, restored);
185 let edit = PreparedEdit::create_page(source, &restored).unwrap();
182 let edit = ops::section_op(source, SectionOp::Create(restored.clone())).unwrap();
186183 let position = before.map_or(pages.len(), |sid| {
187184 pages.iter().position(|p| p.0 == sid).unwrap()
188185 });
......@@ -257,7 +254,10 @@ pub fn run(input: &[u8]) {
257254 write_limit: if step[6] & 1 == 0 { 17 } else { 4096 },
258255 random: u64::from(step[7]) + 1,
259256 };
260 let result = edit.commit(&mut disk);
257 let Some(transaction) = &edit.transaction else {
258 continue;
259 };
260 let result = transaction.commit(&mut disk);
261261 let observed = current::current(&disk.durable);
262262 match result {
263263 Ok(()) => assert_eq!(observed, after),
crates/onestore/tests/support/tree_model.rs+69-208
......@@ -1,16 +1,18 @@
11use onestore::{
2 CommitState, ExGuid, Insertion, PreparedEdit, RevisionIndex, Store, TreeEdit,
2 CommitState, ExGuid, RevisionIndex, Store,
33 document::{Document, Kind, Revision},
4 op::{Op, PageOp},
5 page::Page,
46};
57use std::{
6 collections::{BTreeMap, BTreeSet},
8 collections::BTreeMap,
79 sync::LazyLock,
810};
911
10use crate::{current, disk};
12use crate::{current, disk, ops};
1113
1214static SOURCE: LazyLock<Vec<u8>> = LazyLock::new(|| {
13 let mut source = onestore::create_section("tree.one", "Original", "Author").unwrap();
15 let source = onestore::create_section("tree.one", "Original", "Author").unwrap();
1416 let store = Store::parse(&source).unwrap();
1517 let index = RevisionIndex::parse(&store).unwrap();
1618 let document = Document::parse(&index).unwrap();
......@@ -22,26 +24,24 @@ static SOURCE: LazyLock<Vec<u8>> = LazyLock::new(|| {
2224 .iter()
2325 .find(|id| matches!(view.nodes[id].kind, Kind::Outline { .. }))
2426 .unwrap();
25 let other = Insertion::outline(page, 360.0, 72.0, "Other", "Author").unwrap();
26 source = PreparedEdit::insert(&source, sid, &other)
27 .unwrap()
28 .as_bytes()
29 .to_vec();
30 for parent in [outline, other.object()] {
31 for text in ["First 🦀 é", "Second 東京", "Third"] {
32 let insertion = Insertion::paragraph(parent, None, text, "Author").unwrap();
33 source = PreparedEdit::insert(&source, sid, &insertion)
34 .unwrap()
35 .as_bytes()
36 .to_vec();
37 let child = Insertion::paragraph(insertion.object(), None, "Child", "Author").unwrap();
38 source = PreparedEdit::insert(&source, sid, &child)
39 .unwrap()
40 .as_bytes()
41 .to_vec();
27 let (add, other, _) = ops::new_outline(360.0, 72.0, "Other");
28 let mut edits = vec![add];
29 for container in [outline, other] {
30 let mut paragraphs = Vec::new();
31 for text in ["First 🦀 é", "Second 東京", "Third"] {
32 let paragraph = ops::paragraph(text);
33 let mut child = ops::paragraph("Child");
34 child.parent = Some(paragraph.id);
35 child.level = 2;
36 paragraphs.extend([paragraph, child]);
4237 }
38 edits.push(PageOp::Insert {
39 container,
40 before: None,
41 paragraphs,
42 });
4343 }
44 source
44 ops::page_edited(&source, sid, edits).unwrap()
4545});
4646
4747fn forest(view: &Revision<'_>, root: ExGuid) -> BTreeMap<ExGuid, (ExGuid, u32, usize)> {
......@@ -69,6 +69,9 @@ fn forest(view: &Revision<'_>, root: ExGuid) -> BTreeMap<ExGuid, (ExGuid, u32, u
6969 positions
7070}
7171
72/// Random subtree moves and deletions by twelve clients, each on its own possibly stale
73/// image, committed under injected storage faults: an edit stores the page the model
74/// predicts, and the file always holds a complete old or new graph.
7275pub fn run(input: &[u8]) {
7376 let Some((&selector, input)) = input.split_first() else {
7477 return;
......@@ -110,194 +113,74 @@ pub fn run(input: &[u8]) {
110113 }
111114 let target = targets[usize::from(step[3]) % targets.len()];
112115 let subtree = forest(view, target);
113 let parents: BTreeMap<_, _> = before
114 .keys()
115 .flat_map(|id| view.nodes[id].children.iter().map(|child| (*child, *id)))
116 .collect();
117 let parent = parents[&target];
116 let outline = matches!(view.nodes[&target].kind, Kind::Outline { .. });
118117 let deleting = step[1] & 6 == 0;
119 let mut expected: BTreeMap<_, _> = before
120 .iter()
121 .filter_map(|(id, pos)| {
122 matches!(view.nodes[id].kind, Kind::Paragraph { .. })
123 .then_some((*id, (pos.0, pos.1)))
124 })
125 .collect();
126 let mut ordered: Vec<_> = expected.keys().copied().collect();
127 ordered.sort_by_key(|id| before[id].2);
128 let mut expected_order = BTreeMap::<_, Vec<_>>::new();
129 for id in ordered {
130 expected_order.entry(before[&id].0).or_default().push(id);
131 }
132 let intent;
133 let mut reversal = None;
134 let edit = if deleting {
135 expected.retain(|id, _| !subtree.contains_key(id));
136 for ids in expected_order.values_mut() {
137 ids.retain(|id| !subtree.contains_key(id));
138 }
139 intent = TreeEdit::delete(target, "Tree schedule").unwrap();
140 PreparedEdit::tree(source, sid, &intent)
118 let (op, cycle) = if deleting {
119 (PageOp::Delete { object: target }, false)
141120 } else {
142121 let mut destinations: Vec<_> = before
143122 .keys()
144123 .copied()
145124 .filter(|id| {
146 if matches!(view.nodes[&target].kind, Kind::Outline { .. }) {
125 if outline {
147126 *id == page
148127 } else {
149128 matches!(
150129 view.nodes[id].kind,
151 Kind::Outline { .. }
152 | Kind::OutlineGroup
153 | Kind::Paragraph { .. }
154 | Kind::Cell { .. }
130 Kind::Outline { .. } | Kind::Paragraph { .. } | Kind::Cell { .. }
155131 )
156132 }
157133 })
158134 .collect();
159135 destinations.sort_by_key(|id| before[id].2);
160 if destinations.is_empty() {
161 continue;
162 }
163136 let destination = destinations[usize::from(step[4]) % destinations.len()];
164 let children = &view.nodes[&destination].children;
165 let anchor = children
166 .get(usize::from(step[5]) % (children.len() + 1))
167 .copied();
168 intent = TreeEdit::move_to(target, destination, anchor, "Tree schedule").unwrap();
169 let edit = PreparedEdit::tree(source, sid, &intent);
170 if subtree.contains_key(&destination) {
171 assert!(edit.is_err());
172 continue;
173 }
174 if destination == parent && step[9] & 3 == 0 {
175 let original = &view.nodes[&parent].children;
176 let next = original.iter().position(|id| *id == target).unwrap() + 1;
177 reversal = Some(
178 TreeEdit::move_to(target, parent, original.get(next).copied(), "Tree schedule")
179 .unwrap(),
180 );
181 }
182 if matches!(view.nodes[&target].kind, Kind::Paragraph { .. }) {
183 let (scope, base, _) = before[&destination];
184 let level = base + u32::from(view.nodes[&destination].child_level.unwrap_or(1));
185 let (old_scope, old_level, _) = before[&target];
186 if anchor != Some(target) {
187 let moved: Vec<_> = expected_order[&old_scope]
188 .iter()
189 .copied()
190 .filter(|id| subtree.contains_key(id))
191 .collect();
192 expected_order
193 .get_mut(&old_scope)
194 .unwrap()
195 .retain(|id| !subtree.contains_key(id));
196 let boundary = anchor.map(|id| before[&id].2).unwrap_or_else(|| {
197 forest(view, destination)
198 .keys()
199 .map(|id| before[id].2)
200 .max()
201 .unwrap()
202 + 1
203 });
204 let sequence = expected_order.entry(scope).or_default();
205 let position = sequence
206 .iter()
207 .position(|id| before[id].2 >= boundary)
208 .unwrap_or(sequence.len());
209 sequence.splice(position..position, moved);
210 }
211 for (id, pos) in &mut expected {
212 if subtree.contains_key(id) && pos.0 == old_scope {
213 *pos = (scope, level + pos.1 - old_level);
214 }
137 // Outline groups are the writers' business: anchors are the paragraphs in them.
138 let mut children = Vec::new();
139 let mut pending: Vec<ExGuid> = view.nodes[&destination].children.iter().rev().copied().collect();
140 while let Some(id) = pending.pop() {
141 match view.nodes[&id].kind {
142 Kind::OutlineGroup => pending.extend(view.nodes[&id].children.iter().rev()),
143 _ if id == target => {}
144 _ => children.push(id),
215145 }
216146 }
217 edit
147 let before = children
148 .get(usize::from(step[5]) % (children.len() + 1))
149 .copied();
150 let op = PageOp::Move {
151 object: target,
152 parent: (!outline).then_some(destination),
153 before,
154 };
155 (op, subtree.contains_key(&destination))
218156 };
219 let restored = serde_json::from_value(serde_json::to_value(&intent).unwrap()).unwrap();
220 assert_eq!(intent, restored);
221 let edit = match edit {
222 Ok(edit) => edit,
223 Err(error)
224 if error.message
225 == "Removing this group would exceed 31 child indentation levels" =>
226 {
227 continue;
228 }
229 Err(error) => panic!("{intent:?}: {error:?}"),
157 let model = Page::from_space(&document, sid).unwrap();
158 let ops = vec![Op::Page { space: sid, op: op.clone() }];
159 let Ok(edit) = ops::apply(source, "Tree schedule", ops) else {
160 continue;
230161 };
162 assert!(!cycle, "{op:?}");
231163 let after_store = Store::parse(edit.as_bytes()).unwrap();
232164 let after_index = RevisionIndex::parse(&after_store).unwrap();
233165 after_index.validate_current().unwrap();
234166 let after_document = Document::parse(&after_index).unwrap();
167 let mut predicted = model.clone();
168 onestore::op::predict(&mut predicted, &op).unwrap();
169 let stored = Page::from_space(&after_document, sid).unwrap();
170 if stored != predicted {
171 let (a, b) = (format!("{stored:?}"), format!("{predicted:?}"));
172 let at = a.bytes().zip(b.bytes()).take_while(|(x, y)| x == y).count();
173 panic!("{op:?}\n stored …{}\n predicted …{}", &a[a.floor_char_boundary(at.saturating_sub(300))..a.floor_char_boundary((at + 200).min(a.len()))], &b[b.floor_char_boundary(at.saturating_sub(300))..b.floor_char_boundary((at + 200).min(b.len()))]);
174 }
235175 let space = &after_document.spaces[&sid];
236176 let after = &space.revisions[&space.contexts[&ExGuid::default()]];
237 let actual = forest(after, page);
238 let mut added = BTreeSet::new();
239 for (id, pos) in &actual {
240 let node = &after.nodes[id];
241 if let Kind::Paragraph { .. } = node.kind {
242 if let Some(wanted) = expected.remove(id) {
243 assert_eq!((pos.0, pos.1), wanted, "{intent:?}: {id}");
244 } else {
245 assert!(!before.contains_key(id));
246 assert!(matches!(view.nodes[&parent].kind, Kind::Cell { .. }));
247 assert_eq!(view.nodes[&parent].children, [target]);
248 assert_eq!(node.children.len(), 0);
249 assert_eq!(node.content.len(), 1);
250 assert!(node.tags.is_empty());
251 assert!(
252 matches!(&after.nodes[&node.content[0]].kind,Kind::RichText{text,..} if text.is_empty())
253 );
254 added.insert(*id);
255 }
256 }
257 if before.contains_key(id) {
258 assert_eq!(node.content, view.nodes[id].content);
259 let mut expected_kind = serde_json::to_value(&view.nodes[id].kind).unwrap();
260 if *id == page && expected_kind["alternate_title"].is_string() {
261 let mut roots = after.nodes[&page].children.clone();
262 roots.sort_by(|a, b| {
263 let a = &after.nodes[a].layout;
264 let b = &after.nodes[b].layout;
265 a.y.unwrap_or(0.0)
266 .total_cmp(&b.y.unwrap_or(0.0))
267 .then_with(|| a.x.unwrap_or(0.0).total_cmp(&b.x.unwrap_or(0.0)))
268 });
269 let mut title = String::new();
270 for root in roots {
271 let mut nodes: Vec<_> = forest(after, root).into_iter().collect();
272 nodes.sort_by_key(|(_, pos)| pos.2);
273 for (id, _) in nodes {
274 if let Kind::RichText { text, .. } = &after.nodes[&id].kind {
275 title = text
276 .trim_start()
277 .split('\r')
278 .next()
279 .unwrap()
280 .trim_end()
281 .to_owned();
282 if !title.is_empty() {
283 break;
284 }
285 }
286 }
287 if !title.is_empty() {
288 break;
289 }
290 }
291 expected_kind["alternate_title"] = title.into();
292 let Kind::Metadata { title, .. } = &after.nodes[&after.roots[&2]].kind else {
293 panic!()
294 };
295 assert_eq!(title.as_deref(), expected_kind["alternate_title"].as_str());
296 }
297 assert_eq!(serde_json::to_value(&node.kind).unwrap(), expected_kind);
177 for (id, _) in forest(after, page) {
178 let node = &after.nodes[&id];
179 if before.contains_key(&id) {
180 assert_eq!(node.content, view.nodes[&id].content);
298181 assert_eq!(
299182 serde_json::to_value(&node.tags).unwrap(),
300 serde_json::to_value(&view.nodes[id].tags).unwrap()
183 serde_json::to_value(&view.nodes[&id].tags).unwrap()
301184 );
302185 }
303186 if matches!(
......@@ -311,31 +194,6 @@ pub fn run(input: &[u8]) {
311194 );
312195 }
313196 }
314 assert!(expected.is_empty());
315 assert!(added.len() <= 1);
316 let mut ordered: Vec<_> = actual
317 .iter()
318 .filter(|(id, _)| {
319 matches!(after.nodes[id].kind, Kind::Paragraph { .. }) && !added.contains(id)
320 })
321 .collect();
322 ordered.sort_by_key(|(_, pos)| pos.2);
323 let mut actual_order = BTreeMap::<_, Vec<_>>::new();
324 for (id, pos) in ordered {
325 actual_order.entry(pos.0).or_default().push(*id);
326 }
327 expected_order.retain(|_, ids| !ids.is_empty());
328 assert_eq!(actual_order, expected_order, "{intent:?}");
329 if let Some(reversal) = reversal {
330 let reversed = PreparedEdit::tree(edit.as_bytes(), sid, &reversal).unwrap();
331 let replayed = PreparedEdit::tree(reversed.as_bytes(), sid, &intent).unwrap();
332 let store = Store::parse(replayed.as_bytes()).unwrap();
333 let index = RevisionIndex::parse(&store).unwrap();
334 let document = Document::parse(&index).unwrap();
335 let space = &document.spaces[&sid];
336 let replayed = &space.revisions[&space.contexts[&ExGuid::default()]];
337 assert_eq!(forest(replayed, page), actual);
338 }
339197 let before = current::current(&persisted);
340198 let after = current::current(edit.as_bytes());
341199 let mut disk = disk::Disk {
......@@ -346,7 +204,10 @@ pub fn run(input: &[u8]) {
346204 write_limit: if step[8] & 1 == 0 { 17 } else { 4096 },
347205 random: u64::from(step[9]) + 1,
348206 };
349 let result = edit.commit(&mut disk);
207 let Some(transaction) = &edit.transaction else {
208 continue;
209 };
210 let result = transaction.commit(&mut disk);
350211 let actual = current::current(&disk.durable);
351212 match result {
352213 Ok(()) => assert_eq!(actual, after),
crates/onestore/tests/table_growth.rs+8-5
......@@ -1,8 +1,11 @@
11//! A rewritten object stores the global id table entries it names, not the table of the
22//! revision it was read from.
33
4#[path = "support/ops.rs"]
5mod ops;
6
47use onestore::{
5 PreparedEdit, RevisionIndex, Store,
8 RevisionIndex, Store,
69 document::Document,
710 page::{Page, PageObject, Paragraph},
811};
......@@ -64,17 +67,17 @@ fn a_text_edit_stores_the_table_entries_it_names() {
6467 text.text
6568 .append(Paragraph::new(" and a few more words".to_owned(), format))
6669 .unwrap();
67 let written = PreparedEdit::page(&bytes, space, &page, "Rust")
70 let written = ops::saved(&bytes, space, &page)
6871 .unwrap()
69 .as_bytes()
72 .as_slice()
7073 .to_vec();
7174 for (entries, span) in tables(&written, bytes.len()) {
7275 sparse |= (entries as u32) < span;
7376 }
7477 assert_eq!(
75 PreparedEdit::page(&written, space, &page, "Rust")
78 ops::saved(&written, space, &page)
7679 .unwrap()
77 .as_bytes(),
80 .as_slice(),
7881 written
7982 );
8083 bytes = written;
crates/onestore/tests/toc.rs+34-18
......@@ -1,4 +1,25 @@
1use onestore::{PreparedEdit, RevisionIndex, Store, TocEdit, document::Document};
1use onestore::{RevisionIndex, Store, TocEdit, document::Document};
2
3/// `source` with `edits` applied.
4fn edited(source: &[u8], edits: &[TocEdit]) -> Result<Vec<u8>, onestore::Error> {
5 let mut image = source.to_vec();
6 if let Some(transaction) = onestore::edit_table_of_contents(source, edits)? {
7 transaction.apply(&mut image)?;
8 }
9 Ok(image)
10}
11
12/// The notebook's colour.
13fn color(bytes: &[u8]) -> Option<u32> {
14 let store = Store::parse(bytes).unwrap();
15 let index = RevisionIndex::parse(&store).unwrap();
16 let document = Document::parse(&index).unwrap();
17 let revision = document.active(document.root).unwrap();
18 match &revision.nodes[&revision.roots[&1]].kind {
19 onestore::document::Kind::Toc { color, .. } => *color,
20 _ => panic!(),
21 }
22}
223
324fn entries(bytes: &[u8]) -> Vec<(String, [u8; 16], u32, Option<u32>)> {
425 let store = Store::parse(bytes).unwrap();
......@@ -31,7 +52,7 @@ fn entries(bytes: &[u8]) -> Vec<(String, [u8; 16], u32, Option<u32>)> {
3152}
3253
3354#[test]
34fn sections_and_groups_are_added_renamed_coloured_ordered_and_removed() {
55fn sections_and_groups_are_added_renamed_ordered_and_removed_and_the_notebook_coloured() {
3556 let a = [1; 16];
3657 let b = [2; 16];
3758 let toc = onestore::create_table_of_contents(
......@@ -41,7 +62,7 @@ fn sections_and_groups_are_added_renamed_coloured_ordered_and_removed() {
4162 .unwrap();
4263 let c = [3; 16];
4364 let group = [4; 16];
44 let written = PreparedEdit::table_of_contents(
65 let written = edited(
4566 &toc,
4667 &[
4768 TocEdit::Add {
......@@ -58,7 +79,7 @@ fn sections_and_groups_are_added_renamed_coloured_ordered_and_removed() {
5879 )
5980 .unwrap();
6081 assert_eq!(
61 entries(written.as_bytes()),
82 entries(&written),
6283 [
6384 ("Alpha.one".to_owned(), a, 1, Some(0xffff_ffff)),
6485 ("Beta.one".to_owned(), b, 2, Some(0xffff_ffff)),
......@@ -66,33 +87,31 @@ fn sections_and_groups_are_added_renamed_coloured_ordered_and_removed() {
6687 ("Archive".to_owned(), group, 4, None),
6788 ]
6889 );
69 let again = PreparedEdit::table_of_contents(
70 written.as_bytes(),
90 let again = edited(
91 &written,
7192 &[
7293 TocEdit::Rename {
7394 identity: b,
7495 filename: "Renamed.one".into(),
7596 },
76 TocEdit::Color {
77 identity: c,
78 color: Some(0x00d7ff),
79 },
97 TocEdit::Color(0x00d7ff),
8098 TocEdit::Order(vec![group, c]),
8199 TocEdit::Remove { identity: a },
82100 ],
83101 )
84102 .unwrap();
85103 assert_eq!(
86 entries(again.as_bytes()),
104 entries(&again),
87105 [
88106 ("Archive".to_owned(), group, 1, None),
89 ("Gamma.one".to_owned(), c, 2, Some(0x00d7ff)),
107 ("Gamma.one".to_owned(), c, 2, Some(0xffff_ffff)),
90108 ("Renamed.one".to_owned(), b, 3, Some(0xffff_ffff)),
91109 ]
92110 );
111 assert_eq!(color(&again), Some(0x00d7ff));
93112 assert!(
94 PreparedEdit::table_of_contents(
95 again.as_bytes(),
113 edited(
114 &again,
96115 &[TocEdit::Add {
97116 filename: "renamed.one".into(),
98117 identity: [9; 16],
......@@ -101,8 +120,5 @@ fn sections_and_groups_are_added_renamed_coloured_ordered_and_removed() {
101120 )
102121 .is_err()
103122 );
104 assert!(
105 PreparedEdit::table_of_contents(again.as_bytes(), &[TocEdit::Remove { identity: a }])
106 .is_err()
107 );
123 assert!(edited(&again, &[TocEdit::Remove { identity: a }]).is_err());
108124}
crates/onestore/tests/transaction.rs+27-9
......@@ -1,6 +1,10 @@
1#[path = "support/ops.rs"]
2mod ops;
3
14use onestore::{
2 CommitIo, CommitState, ExGuid, PreparedEdit, RevisionIndex, Stamp, Store,
5 CommitIo, CommitState, ExGuid, RevisionIndex, Stamp, Store, Transaction,
36 document::{Document, Kind},
7 op::{Op, PageOp},
48};
59use std::io;
610
......@@ -23,6 +27,21 @@ fn text(source: &[u8]) -> (ExGuid, ExGuid) {
2327 panic!("Missing text fixture")
2428}
2529
30/// The transaction typing `with` at the start of the text object, and the image it leaves.
31fn typed(space: ExGuid, text: ExGuid, with: &str) -> (Transaction, Vec<u8>) {
32 let op = PageOp::Text {
33 text,
34 range: 0..0,
35 with: with.into(),
36 };
37 let transaction = ops::transaction(SOURCE, "Author", vec![Op::Page { space, op }])
38 .unwrap()
39 .unwrap();
40 let mut image = SOURCE.to_vec();
41 transaction.apply(&mut image).unwrap();
42 (transaction, image)
43}
44
2645/// A file that records how many bytes a commit reads and writes.
2746struct Counted {
2847 bytes: Vec<u8>,
......@@ -54,20 +73,19 @@ impl CommitIo for Counted {
5473#[test]
5574fn a_transaction_reads_the_header_and_writes_only_its_edit() {
5675 let (sid, oid) = text(SOURCE);
57 let edit = PreparedEdit::text(SOURCE, sid, oid, 0..0, "é").unwrap();
58 let transaction = edit.transaction();
76 let (transaction, written) = typed(sid, oid, "é");
5977 let mut image = SOURCE.to_vec();
6078 transaction.apply(&mut image).unwrap();
61 assert_eq!(image, edit.as_bytes());
79 assert_eq!(image, written);
6280 let mut file = Counted {
6381 bytes: SOURCE.to_vec(),
6482 read: 0,
6583 written: 0,
6684 };
6785 transaction.commit(&mut file).unwrap();
68 assert_eq!(file.bytes, edit.as_bytes());
86 assert_eq!(file.bytes, written);
6987 assert!(file.read <= 1024 + 2, "read {} bytes", file.read);
70 let appended = edit.as_bytes().len() - SOURCE.len();
88 let appended = written.len() - SOURCE.len();
7189 assert!(
7290 file.written < appended + 2048,
7391 "wrote {} bytes for {appended} appended",
......@@ -86,9 +104,9 @@ fn a_transaction_reads_the_header_and_writes_only_its_edit() {
86104#[test]
87105fn stamps_name_the_committed_image() {
88106 let (sid, oid) = text(SOURCE);
89 let edit = PreparedEdit::text(SOURCE, sid, oid, 0..0, "x").unwrap();
107 let (transaction, written) = typed(sid, oid, "x");
90108 let before = Stamp::of(SOURCE).unwrap();
91 let after = Stamp::of(edit.as_bytes()).unwrap();
109 let after = Stamp::of(&written).unwrap();
92110 assert_ne!(before, after);
93111 assert_eq!(before.length, SOURCE.len() as u64);
94112 // An unpublished tail changes the length, so a commit built without it is refused.
......@@ -100,7 +118,7 @@ fn stamps_name_the_committed_image() {
100118 read: 0,
101119 written: 0,
102120 };
103 let error = edit.commit(&mut file).unwrap_err();
121 let error = transaction.commit(&mut file).unwrap_err();
104122 assert_eq!(error.error.kind(), io::ErrorKind::ResourceBusy);
105123 assert_eq!(file.written, 0);
106124 assert!(Stamp::of(&SOURCE[..1023]).is_err());
crates/onestore/tests/tree.rs+121-55
......@@ -1,6 +1,7 @@
11use onestore::{
2 ExGuid, PreparedEdit, RevisionIndex, Store, TreeEdit,
2 ExGuid, RevisionIndex, Store,
33 document::{Document, Kind, Revision},
4 op::{Op, PageOp},
45};
56use std::collections::{BTreeMap, BTreeSet};
67
......@@ -8,6 +9,8 @@ use std::collections::{BTreeMap, BTreeSet};
89mod current;
910#[path = "support/disk.rs"]
1011mod disk;
12#[path = "support/ops.rs"]
13mod ops;
1114#[path = "support/tree_model.rs"]
1215mod tree_model;
1316
......@@ -55,7 +58,35 @@ fn active(view: &Revision<'_>, page: ExGuid) -> BTreeSet<ExGuid> {
5558 ids
5659}
5760
58fn operation(view: &Revision<'_>, page: ExGuid) -> Option<(String, ExGuid, TreeEdit)> {
61/// The container `id` lies in, outline groups passed over.
62fn container(view: &Revision<'_>, ids: &BTreeSet<ExGuid>, id: ExGuid) -> ExGuid {
63 let mut at = *ids
64 .iter()
65 .find(|parent| view.nodes[parent].children.contains(&id))
66 .unwrap();
67 while matches!(view.nodes[&at].kind, Kind::OutlineGroup) {
68 at = container(view, ids, at);
69 }
70 at
71}
72
73/// The outline level of each paragraph in `outline`.
74fn levels(view: &Revision<'_>, outline: ExGuid) -> BTreeMap<ExGuid, u32> {
75 let mut levels = BTreeMap::new();
76 let mut pending = vec![(outline, 0)];
77 while let Some((id, level)) = pending.pop() {
78 let node = &view.nodes[&id];
79 if matches!(node.kind, Kind::Paragraph { .. }) {
80 levels.insert(id, level);
81 }
82 let child = level + u32::from(node.child_level.unwrap_or(1));
83 pending.extend(node.children.iter().map(|id| (*id, child)));
84 }
85 levels
86}
87
88/// The named edit of a fixture page as ops, with the object it acts on.
89fn operation(view: &Revision<'_>, page: ExGuid) -> Option<(String, ExGuid, Vec<PageOp>)> {
5990 let Kind::Metadata {
6091 title: Some(name), ..
6192 } = &view.nodes[&view.roots[&2]].kind
......@@ -85,32 +116,46 @@ fn operation(view: &Revision<'_>, page: ExGuid) -> Option<(String, ExGuid, TreeE
85116 .iter()
86117 .find(|id| matches!(view.nodes[id].kind, Kind::Outline { .. }))
87118 .unwrap();
88 let parent = *ids
89 .iter()
90 .find(|id| view.nodes[id].children.contains(&target))
91 .unwrap();
119 let parent = container(view, &ids, target);
120 let move_to = |parent, before| PageOp::Move {
121 object: target,
122 parent: Some(parent),
123 before,
124 };
92125 let edit = match name.as_str() {
93 "Delete outline" => TreeEdit::delete(outline, "Tree author"),
94 name if name.starts_with("Delete") => TreeEdit::delete(target, "Tree author"),
126 "Delete outline" => vec![PageOp::Delete { object: outline }],
127 "Delete sole cell paragraph" => vec![
128 PageOp::Insert {
129 container: parent,
130 before: None,
131 paragraphs: vec![ops::paragraph("")],
132 },
133 PageOp::Delete { object: target },
134 ],
135 name if name.starts_with("Delete") => vec![PageOp::Delete { object: target }],
95136 "Move leaf down"
96137 | "Move subtree down"
97138 | "Move numbered subtree down"
98 | "Move cell subtree down" => TreeEdit::move_to(target, parent, None, "Tree author"),
99 "Move subtree up" => {
100 TreeEdit::move_to(target, parent, Some(paragraphs["Anchor"]), "Tree author")
101 }
102 "Indent subtree" | "Indent bullet subtree" => {
103 TreeEdit::move_to(target, paragraphs["Anchor"], None, "Tree author")
139 | "Move cell subtree down" => vec![move_to(parent, None)],
140 "Move subtree up" => vec![move_to(parent, Some(paragraphs["Anchor"]))],
141 "Indent subtree" | "Indent bullet subtree" => vec![move_to(paragraphs["Anchor"], None)],
142 "Outdent subtree" | "Outdent first group" => {
143 // Levels are absolute: the subtree moves up to the outline's first level.
144 let levels = levels(view, outline);
145 let shift = levels[&target] - 1;
146 let mut ops = vec![move_to(outline, Some(paragraphs["Trailing sibling"]))];
147 let mut pending = vec![target];
148 while let Some(paragraph) = pending.pop() {
149 ops.push(PageOp::Level {
150 paragraph,
151 level: levels[&paragraph] - shift,
152 });
153 pending.extend(view.nodes[&paragraph].children.iter().rev());
154 }
155 ops
104156 }
105 "Outdent subtree" | "Outdent first group" => TreeEdit::move_to(
106 target,
107 outline,
108 Some(paragraphs["Trailing sibling"]),
109 "Tree author",
110 ),
111157 _ => return None,
112 }
113 .unwrap();
158 };
114159 Some((
115160 name.clone(),
116161 if name == "Delete outline" {
......@@ -139,9 +184,9 @@ fn native_subtree_controls_match_with_preserved_fields_and_history() {
139184 let Some((name, object, edit)) = operation(before, page) else {
140185 continue;
141186 };
142 let restored = serde_json::from_value(serde_json::to_value(&edit).unwrap()).unwrap();
143 assert_eq!(edit, restored);
144 let prepared = PreparedEdit::tree(source, sid, &restored).unwrap();
187 let edit: Vec<Op> = edit.into_iter().map(|op| Op::Page { space: sid, op }).collect();
188 let prepared = ops::apply(source, "Tree author", edit.clone())
189 .unwrap_or_else(|error| panic!("{name}: {error}"));
145190 let after_store = Store::parse(prepared.as_bytes()).unwrap();
146191 assert!(after_store.checksum_mismatches.is_empty());
147192 let after_index = RevisionIndex::parse(&after_store).unwrap();
......@@ -267,11 +312,19 @@ fn native_subtree_controls_match_with_preserved_fields_and_history() {
267312 serde_json::to_value(&old.tags).unwrap(),
268313 "{name}: tags {id}"
269314 );
270 } else if matches!(node.kind, Kind::RichText { .. }) {
271 assert_eq!(
272 serde_json::to_value(&node.kind).unwrap(),
273 serde_json::to_value(&wanted.kind).unwrap()
274 );
315 } else if let (
316 Kind::RichText {
317 text, boilerplate, ..
318 },
319 Kind::RichText {
320 text: native,
321 boilerplate: native_boilerplate,
322 ..
323 },
324 ) = (&node.kind, &wanted.kind)
325 {
326 // A new run carries the language MS-ONE requires, which OneNote omits.
327 assert_eq!((text, boilerplate), (native, native_boilerplate));
275328 }
276329 }
277330 for (sid, space) in &index.spaces {
......@@ -284,7 +337,7 @@ fn native_subtree_controls_match_with_preserved_fields_and_history() {
284337 }
285338 }
286339 }
287 let another = PreparedEdit::tree(source, sid, &edit).unwrap();
340 let another = ops::apply(source, "Tree author", edit.clone()).unwrap();
288341 let second_store = Store::parse(another.as_bytes()).unwrap();
289342 let second_index = RevisionIndex::parse(&second_store).unwrap();
290343 let second_doc = Document::parse(&second_index).unwrap();
......@@ -293,10 +346,7 @@ fn native_subtree_controls_match_with_preserved_fields_and_history() {
293346 actual,
294347 active(&space.revisions[&space.contexts[&ExGuid::default()]], page)
295348 );
296 candidate = PreparedEdit::tree(&candidate, sid, &edit)
297 .unwrap()
298 .as_bytes()
299 .to_vec();
349 candidate = ops::apply(&candidate, "Tree author", edit).unwrap().image;
300350 count += 1;
301351 }
302352 if let Some(output) = std::env::var_os("ONESTORE_TREE_OUTPUT") {
......@@ -329,22 +379,24 @@ fn moves_reject_cycles_wrong_parents_and_stale_siblings() {
329379 .unwrap();
330380 let paragraph = view.nodes[&outline].children[0];
331381 let text = view.nodes[&paragraph].content[0];
332 for intent in [
333 TreeEdit::delete(page, "Author"),
334 TreeEdit::delete(text, "Author"),
335 TreeEdit::move_to(outline, paragraph, None, "Author"),
336 TreeEdit::move_to(paragraph, paragraph, None, "Author"),
337 TreeEdit::move_to(paragraph, text, None, "Author"),
338 TreeEdit::move_to(paragraph, outline, Some(page), "Author"),
382 let move_to = |object, parent, before| PageOp::Move {
383 object,
384 parent: Some(parent),
385 before,
386 };
387 for op in [
388 PageOp::Delete { object: page },
389 PageOp::Delete { object: text },
390 move_to(outline, paragraph, None),
391 move_to(paragraph, paragraph, None),
392 move_to(paragraph, text, None),
393 move_to(paragraph, outline, Some(page)),
339394 ] {
340 assert!(PreparedEdit::tree(&source, sid, &intent.unwrap()).is_err());
395 assert!(ops::page_op(&source, sid, op).is_err());
341396 }
342397 for before in [None, Some(paragraph)] {
343 let edit = TreeEdit::move_to(paragraph, outline, before, "Author").unwrap();
344 assert_eq!(
345 PreparedEdit::tree(&source, sid, &edit).unwrap().as_bytes(),
346 source
347 );
398 let op = move_to(paragraph, outline, before);
399 assert_eq!(ops::page_op(&source, sid, op).unwrap().as_bytes(), source);
348400 }
349401}
350402
......@@ -370,7 +422,8 @@ fn publication_interruptions_expose_complete_old_or_new_trees() {
370422 ) {
371423 continue;
372424 }
373 let edit = PreparedEdit::tree(source, sid, &intent).unwrap();
425 let intent = intent.into_iter().map(|op| Op::Page { space: sid, op }).collect();
426 let edit = ops::apply(source, "Tree author", intent).unwrap();
374427 let after = current::current(edit.as_bytes());
375428 for write_limit in [17, 4096] {
376429 let disk = |fail_at| disk::Disk {
......@@ -453,8 +506,24 @@ fn cross_container_moves_keep_tables_and_replace_emptied_cells() {
453506 "Move cell subtree down" => (target, outlines[1]),
454507 _ => (view.nodes[&outlines[0]].children[0], outlines[1]),
455508 };
456 let intent = TreeEdit::move_to(object, destination, None, "Tree author").unwrap();
457 let edit = PreparedEdit::tree(source, sid, &intent).unwrap();
509 let mut intent = vec![Op::Page {
510 space: sid,
511 op: PageOp::Move {
512 object,
513 parent: Some(destination),
514 before: None,
515 },
516 }];
517 // A cell keeps a paragraph: its replacement goes in first.
518 if name == "Delete sole cell paragraph" {
519 let op = PageOp::Insert {
520 container: parents[&target],
521 before: None,
522 paragraphs: vec![ops::paragraph("")],
523 };
524 intent.insert(0, Op::Page { space: sid, op });
525 }
526 let edit = ops::apply(source, "Tree author", intent.clone()).unwrap();
458527 let store = Store::parse(edit.as_bytes()).unwrap();
459528 let index = RevisionIndex::parse(&store).unwrap();
460529 let document = Document::parse(&index).unwrap();
......@@ -503,10 +572,7 @@ fn cross_container_moves_keep_tables_and_replace_emptied_cells() {
503572 "Delete cell subtree" => assert!(!actual.contains(&outlines[0])),
504573 _ => assert_eq!(after.nodes[&parents[&target]].children.len(), 1),
505574 }
506 candidate = PreparedEdit::tree(&candidate, sid, &intent)
507 .unwrap()
508 .as_bytes()
509 .to_vec();
575 candidate = ops::apply(&candidate, "Tree author", intent).unwrap().image;
510576 count += 1;
511577 }
512578 assert_eq!(count, 4);
crates/onestore/tests/writer.rs+112-83
......@@ -1,39 +1,63 @@
1#[path = "support/ops.rs"]
2mod ops;
3
14use onestore::{
2 ExGuid, ObjectData, PropertySets, RevisionIndex, Store, Value, replace_property_bytes,
5 ExGuid, ObjectData, PropertySets, RevisionIndex, Store, TocEdit, Value,
6 document::{Document, Kind},
7 op::{Edit, Op, PageOp},
38};
49use std::fs;
510
11/// The text object holding `text` in `source` and its page space.
12fn text_object(source: &[u8], text: &str) -> (ExGuid, ExGuid) {
13 let store = Store::parse(source).unwrap();
14 let index = RevisionIndex::parse(&store).unwrap();
15 let document = Document::parse(&index).unwrap();
16 let mut found = document.spaces.iter().flat_map(|(space, view)| {
17 view.active()
18 .unwrap()
19 .nodes
20 .iter()
21 .filter(|(_, node)| matches!(&node.kind, Kind::RichText { text: stored, .. } if stored == text))
22 .map(|(id, _)| (*space, *id))
23 .collect::<Vec<_>>()
24 });
25 let target = found.next().unwrap();
26 assert!(found.next().is_none());
27 target
28}
29
30/// `source`, a table of contents, with the notebook recoloured.
31fn recolored(source: &[u8], color: u32) -> Vec<u8> {
32 let mut image = source.to_vec();
33 let edit = TocEdit::Color(color);
34 if let Some(transaction) = onestore::edit_table_of_contents(source, &[edit]).unwrap() {
35 transaction.apply(&mut image).unwrap();
36 }
37 image
38}
39
640#[test]
7fn scalar_edit_appends_a_revision_and_preserves_every_prior_object() {
41fn text_edit_appends_a_revision_and_preserves_every_prior_object() {
842 let source =
943 fs::read("../../corpus/native/20260905-05/snapshots/02-text/notebook/synthetic.one")
1044 .unwrap();
1145 let store = Store::parse(&source).unwrap();
1246 let index = RevisionIndex::parse(&store).unwrap();
13 let mut selected = None;
14 for (osid, space) in &index.spaces {
15 let revision = index
16 .resolve(*osid, space.labels[&(ExGuid::default(), 1)])
17 .unwrap();
18 for oid in revision.reachable().unwrap() {
19 let object = &revision.objects[&oid];
20 if let ObjectData::Properties(bytes) = object.data {
21 for property in &PropertySets::parse(bytes).unwrap().sets[0] {
22 if property.value == Value::Bytes(b"Fictitious plain text.") {
23 assert!(selected.replace((*osid, oid, property.id)).is_none());
24 }
25 }
26 }
27 }
28 }
29 let (osid, oid, property) = selected.unwrap();
30 assert_eq!(
31 replace_property_bytes(&source, osid, oid, property, b"Fictitious plain text.").unwrap(),
32 source
33 );
47 let (osid, oid) = text_object(&source, "Fictitious plain text.");
48 let whole = 0.."Fictitious plain text.".encode_utf16().count() as u32;
49 let typed = |with: &str| {
50 let op = PageOp::Text {
51 text: oid,
52 range: whole.clone(),
53 with: with.into(),
54 };
55 ops::page_edited(&source, osid, vec![op]).unwrap()
56 };
57 assert_eq!(typed("Fictitious plain text."), source);
3458 for length in 0..=40 {
35 let value = vec![b'x'; length];
36 let written = replace_property_bytes(&source, osid, oid, property, &value).unwrap();
59 let value = "x".repeat(length);
60 let written = typed(&value);
3761 assert_preserved_committed_bytes(&source, &written);
3862 let parsed = Store::parse(&written).unwrap();
3963 assert!(parsed.checksum_mismatches.is_empty());
......@@ -64,22 +88,11 @@ fn scalar_edit_appends_a_revision_and_preserves_every_prior_object() {
6488 }
6589 }
6690 }
67 let space = &after.spaces[&osid];
68 let active = after
69 .resolve(osid, space.labels[&(ExGuid::default(), 1)])
70 .unwrap();
71 let ObjectData::Properties(data) = active.objects[&oid].data else {
91 let document = Document::parse(&after).unwrap();
92 let Kind::RichText { text, .. } = &document.active(osid).unwrap().nodes[&oid].kind else {
7293 panic!()
7394 };
74 let properties = PropertySets::parse(data).unwrap();
75 assert_eq!(
76 properties.sets[0]
77 .iter()
78 .find(|candidate| candidate.id == property)
79 .unwrap()
80 .value,
81 Value::Bytes(&value)
82 );
95 assert_eq!(*text, value);
8396 }
8497}
8598
......@@ -93,26 +106,33 @@ fn toc_color_edit_uses_its_native_revision_encoding_and_crc() {
93106 let index = RevisionIndex::parse(&store).unwrap();
94107 let rid = index.spaces[&index.root].labels[&(ExGuid::default(), 1)];
95108 let revision = index.resolve(index.root, rid).unwrap();
96 let oid = revision.roots[&1];
97 let written =
98 replace_property_bytes(&source, index.root, oid, 0x14001cbe, &[0x33, 0x66, 0x99, 0])
99 .unwrap();
109 let written = recolored(&source, 0x996633);
100110 let after = Store::parse(&written).unwrap();
101111 assert!(after.checksum_mismatches.is_empty());
102112 assert_preserved_committed_bytes(&source, &written);
103113 let current = RevisionIndex::parse(&after).unwrap();
104114 current.validate_current().unwrap();
105 assert_eq!(
106 current.resolve(index.root, rid).unwrap().objects[&oid].data,
107 revision.objects[&oid].data
108 );
109115 let active = current
110116 .resolve(
111117 index.root,
112118 current.spaces[&index.root].labels[&(ExGuid::default(), 1)],
113119 )
114120 .unwrap();
115 let ObjectData::Properties(blob) = active.objects[&oid].data else {
121 let recoloured: Vec<_> = active
122 .objects
123 .iter()
124 .filter(|(id, object)| {
125 revision
126 .objects
127 .get(id)
128 .is_none_or(|old| old.data != object.data)
129 })
130 .map(|(_, object)| object)
131 .collect();
132 let [entry] = recoloured[..] else {
133 panic!("{} objects changed", recoloured.len())
134 };
135 let ObjectData::Properties(blob) = entry.data else {
116136 panic!()
117137 };
118138 assert!(
......@@ -302,13 +322,8 @@ fn toc_writes_cross_fragment_and_counter_boundaries_without_crc_drift() {
302322 let store = Store::parse(&bytes).unwrap();
303323 assert!(store.checksum_mismatches.is_empty(), "edit {i}");
304324 let index = RevisionIndex::parse(&store).unwrap();
305 let root = index.root;
306 let revision = index
307 .resolve(root, index.spaces[&root].labels[&(ExGuid::default(), 1)])
308 .unwrap();
309 let oid = revision.roots[&1];
310 let written =
311 replace_property_bytes(&bytes, root, oid, 0x14001cbe, &i.to_le_bytes()).unwrap();
325 index.validate_current().unwrap();
326 let written = recolored(&bytes, i);
312327 assert_preserved_committed_bytes(&bytes, &written);
313328 bytes = written;
314329 }
......@@ -332,31 +347,31 @@ fn checkpoints_bound_dependencies_and_preserve_native_objects_and_history() {
332347 let mut bytes = fs::read(path).unwrap();
333348 let initial = Store::parse(&bytes).unwrap();
334349 let index = RevisionIndex::parse(&initial).unwrap();
350 // A section retypes text objects of one page; a table of contents recolours the notebook.
351 let document = Document::parse(&index).unwrap();
335352 let mut candidates = Vec::new();
336 for (sid, space) in &index.spaces {
337 let revision = index
338 .resolve(*sid, space.labels[&(ExGuid::default(), 1)])
339 .unwrap();
340 for oid in revision.reachable().unwrap() {
341 let object = &revision.objects[&oid];
342 if object.jcid != if section { 0x6000e } else { 0x20001 } {
343 continue;
344 }
345 let ObjectData::Properties(blob) = object.data else {
346 panic!()
347 };
348 let property = if section { 0x14001d7a } else { 0x14001cbe };
349 if PropertySets::parse(blob).unwrap().sets[0]
350 .iter()
351 .any(|p| p.id == property)
352 {
353 candidates.push((*sid, oid, property));
353 for (sid, view) in &document.spaces {
354 for (oid, node) in &view.active().unwrap().nodes {
355 match &node.kind {
356 Kind::RichText {
357 text,
358 boilerplate: false,
359 ..
360 } if section && !text.is_empty() => {
361 candidates.push((*sid, *oid));
362 }
363 Kind::Toc { entries, .. } if !section && !entries.is_empty() => {
364 candidates.push((*sid, *oid));
365 }
366 _ => {}
354367 }
355368 }
356369 }
357370 assert!(!candidates.is_empty());
358371 let sid = candidates[0].0;
359372 candidates.retain(|candidate| candidate.0 == sid);
373 // What each edit rewrites: modification times, or the notebook's colour.
374 let property = if section { 0x14001d7a } else { 0x14001cbe };
360375 if section {
361376 let revision = index
362377 .resolve(sid, index.spaces[&sid].labels[&(ExGuid::default(), 1)])
......@@ -371,12 +386,30 @@ fn checkpoints_bound_dependencies_and_preserve_native_objects_and_history() {
371386 }
372387 let mut checkpoints = 0;
373388 for operation in 0_u32..1025 {
374 let (_, oid, property) = candidates[operation as usize % candidates.len()];
389 let (_, oid) = candidates[operation as usize % candidates.len()];
375390 let store = Store::parse(&bytes).unwrap();
376391 let before = RevisionIndex::parse(&store).unwrap();
377392 let rid = before.spaces[&sid].labels[&(ExGuid::default(), 1)];
378 let value = (1_000_000 + operation).to_le_bytes();
379 let written = replace_property_bytes(&bytes, sid, oid, property, &value).unwrap();
393 let written = if section {
394 let typed = |range, with: &str| Op::Page {
395 space: sid,
396 op: PageOp::Text {
397 text: oid,
398 range,
399 with: with.into(),
400 },
401 };
402 // Modification times count seconds: each edit is a second later.
403 let at = 134_000_000_000_000_000 + u64::from(operation) * 10_000_000;
404 let ops = vec![typed(0..0, "x"), typed(0..1, "")];
405 let arena = onestore::Arena::default();
406 let mut section = onestore::Section::open(&arena, bytes.clone()).unwrap();
407 section.apply("Author", &Edit { at, ops }).unwrap();
408 section.seal().unwrap().unwrap();
409 section.image()
410 } else {
411 recolored(&bytes, 1_000_000 + operation)
412 };
380413 let updated = Store::parse(&written).unwrap();
381414 assert!(updated.checksum_mismatches.is_empty());
382415 assert_eq!(
......@@ -412,9 +445,6 @@ fn checkpoints_bound_dependencies_and_preserve_native_objects_and_history() {
412445 .iter()
413446 .all(|(entry, guid)| object.global_ids.get(entry) == Some(guid))
414447 );
415 if *id != oid {
416 assert_eq!(object.data, same.data);
417 }
418448 }
419449 if let (ObjectData::Properties(a), ObjectData::Properties(b)) =
420450 (object.data, same.data)
......@@ -426,10 +456,9 @@ fn checkpoints_bound_dependencies_and_preserve_native_objects_and_history() {
426456 assert_eq!(a.len(), b.len());
427457 for (a, b) in a.iter().zip(b) {
428458 assert_eq!(a.id, b.id);
429 if *id == oid && a.id == property {
430 assert_eq!(b.value, Value::Bytes(&value));
431 } else if !matches!(a.value, Value::References { .. }) {
432 assert_eq!(a.value, b.value);
459 if a.id != property && !matches!(a.value, Value::References { .. })
460 {
461 assert_eq!(a.value, b.value, "{id} {:#x}", a.id);
433462 }
434463 }
435464 }
tools/canvas/README.md+4-2
......@@ -82,9 +82,11 @@ Superscript and subscript runs draw at two thirds of their size, a third of the
8282
8383A paragraph stored as an equation draws in two dimensions from the tree `onestore::page::Math` parses: sub- and superscripts at 70%, fractions over a rule on the math axis, radicals drawn with the pen, n-ary operators at display size with limits above and below (integrals take them as scripts), stretched fences, accents and bars above, boxes, matrices and equation arrays. Latin letters draw in mathematical italic. Cambria Math falls back to STIX Two Math on macOS, and atoms are placed by their glyph ink rather than the math font's tall line box; an equation line is at least as tall as a line of text. The linear text stays the editable source, laid out in the body font so the caret keeps a text height; editing equations in two dimensions is a follow-up.
8484
85## Optional font substitution
85## Font substitution
8686
87The host and `layout-probe` accept repeated `--substitute-font FONT_FILE` options. Supply Arimo regular/italic variable fonts or Carlito regular/bold/italic/bold-italic files. Each file is registered only in this process, as Arial or Calibri respectively; this explicitly replaces that family even when a system copy exists. Stored `Format.font` values remain unchanged. No font files are installed or bundled by this option.
87`TextEngine::default()` lays out Calibri, Arial, Times New Roman and Courier New in the bundled Carlito 1.104, Arimo 1.341, Tinos 1.340 and Cousine 1.241 (`crates/canvas/assets/fonts`, SIL Open Font License) wherever the system lacks them. The font box names such a family with its substitute, as "Carlito (Calibri)"; stored `Format.font` values keep the document's name.
88
89The host and `layout-probe` also accept repeated `--substitute-font FONT_FILE` options for another Arimo, Carlito, Tinos or Cousine file. Each file is registered only in this process, as its family's counterpart; this explicitly replaces that family even when a system copy exists. No font files are installed.
8890
8991```sh
9092"target/Snowbound.app/Contents/MacOS/Snowbound" TEXT_FILE 240 --substitute-font ARIMO_FILE
tools/document_model.py+17-1
......@@ -1,8 +1,24 @@
11"""Traversal of the exported document's referenced revisions."""
22import os
33from pathlib import Path
4import shutil
5import subprocess
46
5EXPORTER = Path(os.environ.get('ONESTORE_DOCUMENT', Path(__file__).resolve().parent.parent / 'target/debug/examples/document'))
7
8def _exporter():
9 """The document exporter built with protected sections, copied aside because a workspace
10 build without the feature overwrites target/debug/examples/document."""
11 if 'ONESTORE_DOCUMENT' in os.environ:
12 return Path(os.environ['ONESTORE_DOCUMENT'])
13 root = Path(__file__).resolve().parent.parent
14 subprocess.run(['cargo', 'build', '--quiet', '-p', 'notebook', '--features', 'protected',
15 '--example', 'document'], cwd=root, check=True)
16 protected = root / 'target/debug/examples/document-protected'
17 shutil.copy2(root / 'target/debug/examples/document', protected)
18 return protected
19
20
21EXPORTER = _exporter()
622BRIDGE = Path(__file__).resolve().parent.parent / 'target/debug/onestore-diagnostic'
723
824DEFAULT_CONTEXT = '{00000000-0000-0000-0000-000000000000},0'
tools/native/collaborate.ps1+40
......@@ -47,6 +47,46 @@ try {
4747 Invoke-Stress $app $section $command $output $shared
4848 }
4949 elseif ($command.action -eq 'sync') { $app.SyncHierarchy($notebook) }
50 elseif ($command.action -in @('pages', 'move', 'delete-page')) {
51 $namespace = 'http://schemas.microsoft.com/office/onenote/2010/onenote'
52 $hierarchy = ''
53 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
54 [xml]$tree = $hierarchy
55 $listed = @($tree.SelectNodes('//*[local-name()="Page"]'))
56 $named = { param($title) @($listed | Where-Object { $_.GetAttribute('name') -eq $title }) }
57 if ($command.action -eq 'pages') {
58 # New titled pages with one body paragraph each, appended in order.
59 foreach ($title in $command.titles) {
60 $page = ''
61 $app.CreateNewPage($section, [ref]$page, 2)
62 $body = [Security.SecurityElement]::Escape('Body ' + $title + '.')
63 $name = [Security.SecurityElement]::Escape($title)
64 $xml = '<one:Page xmlns:one="' + $namespace + '" ID="' + $page + '"><one:Title><one:OE><one:T>' + $name + '</one:T></one:OE></one:Title><one:Outline><one:OEChildren><one:OE><one:T>' + $body + '</one:T></one:OE></one:OEChildren></one:Outline></one:Page>'
65 $app.UpdatePageContent($xml, [DateTime]::MinValue, 1, $false)
66 }
67 } elseif ($command.action -eq 'move') {
68 # Moves the page titled `page` before the one titled `before`, or last.
69 $moved = @(& $named $command.page)
70 if ($moved.Count -ne 1) { throw 'Expected one page to move.' }
71 $order = [Collections.ArrayList]@($listed | Where-Object { $_ -ne $moved[0] })
72 $at = $order.Count
73 if ($command.before) {
74 $anchor = @(& $named $command.before)
75 if ($anchor.Count -ne 1) { throw 'Expected one anchor page.' }
76 $at = $order.IndexOf($anchor[0])
77 }
78 $order.Insert($at, $moved[0])
79 $xml = '<one:Section xmlns:one="' + $namespace + '" ID="' + $section + '">'
80 foreach ($node in $order) {
81 $xml += '<one:Page ID="' + $node.GetAttribute('ID') + '" pageLevel="' + $node.GetAttribute('pageLevel') + '"/>'
82 }
83 $app.UpdateHierarchy($xml + '</one:Section>', 1)
84 } else {
85 $doomed = @(& $named $command.page)
86 if ($doomed.Count -ne 1) { throw 'Expected one page to delete.' }
87 $app.DeleteHierarchy($doomed[0].GetAttribute('ID'), [DateTime]::MinValue, $false)
88 }
89 }
5090 elseif ($command.action -eq 'kill-process') {
5191 Get-Process ONENOTE | Stop-Process -Force
5292 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
tools/native_collaboration.py+4-4
......@@ -18,7 +18,7 @@ import xml.etree.ElementTree as ET
1818
1919from native_runner import ROOT, clone, command, windows, vm, collect_artifacts
2020from native_xml import texts
21from document_model import ordered_pages, view, walk
21from document_model import EXPORTER, ordered_pages, view, walk
2222import linux_vm
2323
2424
......@@ -260,7 +260,7 @@ def replay(output, server, stress_clients=0, stress_operations=30, sync_every=1,
260260 target = output / label / 'notebook'
261261 target.mkdir(parents=True)
262262 for name, data in snapshot.items(): (target / name).write_bytes(data)
263 result = subprocess.run([ROOT / 'target/debug/examples/document', target / 'synthetic.one', target.parent / 'model'], capture_output=True, text=True)
263 result = subprocess.run([EXPORTER, target / 'synthetic.one', target.parent / 'model'], capture_output=True, text=True)
264264 if result.returncode == 0:
265265 (target.parent / 'locks.txt').write_text(ssh('sudo smbstatus --locks'))
266266 print('Captured collaboration state:', label, flush=True)
......@@ -318,7 +318,7 @@ def replay(output, server, stress_clients=0, stress_operations=30, sync_every=1,
318318 recovered.mkdir()
319319 with (recovered / 'synthetic.one').open('wb') as stream:
320320 subprocess.run(linux_vm.ssh_argv(server, 'cat /srv/agent/m6-collaboration/synthetic.one'), stdout=stream, check=True, timeout=60)
321 subprocess.run([ROOT / 'target/debug/examples/document', recovered / 'synthetic.one', recovered / 'model'], check=True)
321 subprocess.run([EXPORTER, recovered / 'synthetic.one', recovered / 'model'], check=True)
322322 model = json.loads((recovered / 'model/document.json').read_text())
323323 text = active_text(model)
324324 retained = verify_text('Concurrent edits:', text, logs)
......@@ -459,7 +459,7 @@ def replay(output, server, stress_clients=0, stress_operations=30, sync_every=1,
459459 if temporary.exists(): shutil.rmtree(temporary)
460460 snapshot = output / 'conflict.one'
461461 snapshot.write_bytes(snapshot_file('synthetic.one'))
462 subprocess.run([ROOT / 'target/debug/examples/document', snapshot, temporary], check=True)
462 subprocess.run([EXPORTER, snapshot, temporary], check=True)
463463 model = json.loads((temporary / 'document.json').read_text())
464464 values = [node['kind'].get('text') for space in model['spaces'].values() for revision in space['revisions'].values() for node in revision['nodes'].values()]
465465 if 'Client A concurrent edit.' in values and 'Client B concurrent edit.' in values: break
tools/native_conflict.py created+362
......@@ -0,0 +1,362 @@
1#!/usr/bin/env python3
2"""OneNote 2010 conflict pages: how they arise, what they store and how OneNote shows them.
3
4`sync LINUX OUTPUT` runs two disposable OneNote clients on one Samba notebook: client A edits
5two paragraphs offline while client B edits one of them and a table cell online; A then
6reconnects, which makes OneNote write a conflict page. The server file is kept before and
7after, A and B are screenshotted with the information bar, the page list shown, the conflict
8page open and its menu open, and A then deletes the conflict page through that menu.
9
10`pages LINUX OUTPUT` runs the same two clients on page-list changes: client A, offline, moves
11a page that client B also moves, moves another page, and edits a page B deletes; A then
12reconnects. The server's page order and pages, with the recycle bin, are kept at each step.
13
14`restore LINUX OUTPUT` runs them on a page in the middle of the list: client A, offline, edits
15it and moves a page B leaves alone while B deletes it; A then reconnects.
16
17`cold NOTEBOOK OUTPUT [--delete]` cold-opens a notebook holding a conflict page in a fresh
18clone and screenshots the same views; `--delete` deletes the conflict page through the menu
19and captures the notebook OneNote leaves.
20"""
21import argparse
22from contextlib import ExitStack
23from concurrent.futures import ThreadPoolExecutor
24import base64
25import json
26from pathlib import Path
27import shutil
28import subprocess
29import sys
30import time
31import xml.etree.ElementTree as ET
32
33sys.path.insert(0, str(Path(__file__).resolve().parent))
34sys.path.insert(0, str(Path(__file__).resolve().parent / 'w7'))
35import native_runner as runner
36from native_runner import ROOT, clone, command, windows, vm
37import linux_vm
38
39FIXTURE = ROOT / 'corpus/native-ink/cold-ui-ink/notebook'
40SHARE = 'm6-conflict'
41ORIGINAL = 'Fictitious: café, 東京, مرحبا'
42OTHER_OUTLINE = 'Fictitious positioned outline.'
43CELL = 'Left cell'
44
45# Coordinates on OneNote 2010 maximized at 800x600, from the stage-5, m8 and sync-01 captures:
46# the information bar's text, and the first conflict page listed under its page.
47BAR = (332, 100)
48FIRST_CONFLICT = (725, 142)
49# Delete Conflict Page, first in the menu the bar opens on a conflict page.
50DELETE = (420, 113)
51
52
53def shot(name, output, label, script):
54 """Runs an AutoHotkey step on the clone, keeping its screenshot as `label.png`."""
55 (output / f'{label}.ahk').write_text(script)
56 result = windows.do_exec(script, target=name, timeout_ms=120000, shot_delay_ms=1500)
57 screenshot = result.pop('png_b64', None)
58 if screenshot:
59 (output / f'{label}.png').write_bytes(base64.b64decode(screenshot))
60 (output / f'{label}.json').write_text(json.dumps(result, indent=2))
61 if result.get('error') or result.get('exit') != 0:
62 raise RuntimeError(f'{label}: {result}')
63 return result
64
65
66def click(x, y):
67 return ('WinActivate("ahk_exe ONENOTE.EXE")\nCoordMode("Mouse", "Screen")\n'
68 f'Click({x}, {y})\nSleep(1200)\n')
69
70
71# Reports any dialog OneNote raised, so a confirmation shows in the step's record.
72DIALOG = ('if WinExist("ahk_class #32770 ahk_exe ONENOTE.EXE") {\n'
73 ' FileAppend("dialog: " WinGetTitle() " | " WinGetText() "`n", "*", "UTF-8")\n'
74 '}\n')
75
76
77def conflict_views(name, output, page, prefix, delete=False):
78 """Screenshots a page with a conflict page: the bar, the list shown, the conflict page and
79 its menu; `delete` then chooses Delete Conflict Page."""
80 shot(name, output, f'{prefix}-main', runner.navigation_script(page) + 'Sleep(2000)\n')
81 shot(name, output, f'{prefix}-shown', click(*BAR))
82 shot(name, output, f'{prefix}-conflict', click(*FIRST_CONFLICT))
83 shot(name, output, f'{prefix}-menu', click(*BAR))
84 if delete:
85 shot(name, output, f'{prefix}-delete', click(*DELETE) + DIALOG)
86 else:
87 shot(name, output, f'{prefix}-closed', 'WinActivate("ahk_exe ONENOTE.EXE")\nSend("{Esc}")\nSleep(800)\n')
88
89
90def cold(notebook, output, delete):
91 def interaction(name, output):
92 page = ET.parse(sorted((output / 'read').glob('page-*.xml'))[0]).getroot().attrib['ID']
93 conflict_views(name, output, page, 'cold', delete)
94 if delete:
95 # OneNote writes the deletion in the background; give it time before the re-read.
96 time.sleep(20)
97 runner.capture(notebook, output, collect_notebook=True, interaction=interaction)
98
99
100class Lab:
101 """Two OneNote clients on one Samba notebook: `action` runs a controller command,
102 `state` copies the share and exports its section, `settle` syncs until a check holds."""
103
104
105def page_list(model):
106 """The section's pages in order, each as its sorted texts."""
107 from document_model import ordered_pages, walk
108 return [sorted(node['kind']['text'] for _, node in walk(page, oid) if node['kind']['type'] == 'RichText')
109 for _, _, page, oid in ordered_pages(model)]
110
111
112def edits(lab):
113 from native_collaboration import reachable_page_text
114 a, b, output, action, settle = lab.a, lab.b, lab.output, lab.action, lab.settle
115 state = lambda label: reachable_page_text(lab.state(label))
116 texts = lambda state: {text for values in state.values() for text in values}
117 state('initial')
118 vm.qmp(a['name'], 'set_link', {'name': 'lab', 'up': False})
119 try:
120 action(a, 'edit', expected=ORIGINAL, text='Client A conflicting edit.')
121 action(a, 'edit', expected=OTHER_OUTLINE, text='Client A disjoint edit.')
122 action(a, 'snapshot')
123 action(b, 'edit', expected=ORIGINAL, text='Client B conflicting edit.')
124 action(b, 'edit', expected=CELL, text='Client B disjoint cell.')
125 settle(b, lambda label: {'Client B conflicting edit.', 'Client B disjoint cell.'} <= texts(state(label)), 'b-published')
126 finally:
127 vm.qmp(a['name'], 'set_link', {'name': 'lab', 'up': True})
128 settle(a, lambda label: len(found := state(label)) > 1 and 'Client A disjoint edit.' in texts(found), 'conflict')
129 (output / 'conflict.json').write_text(json.dumps(state('conflict'), indent=2, ensure_ascii=False))
130 page = action(a, 'snapshot')[0].attrib['ID']
131 conflict_views(a['name'], output / 'a', page, 'a')
132 action(b, 'sync')
133 time.sleep(5)
134 page_b = action(b, 'snapshot')[0].attrib['ID']
135 conflict_views(b['name'], output / 'b', page_b, 'b')
136 conflict_views(a['name'], output / 'a', page, 'a-again', delete=True)
137 settle(a, lambda label: len(state(label)) == 1, 'deleted')
138 action(b, 'sync')
139 time.sleep(5)
140 shot(b['name'], output / 'b', 'b-after-delete', runner.navigation_script(page_b) + 'Sleep(2000)\n')
141
142
143PAGES = ['One', 'Two', 'Three', 'Four', 'Target']
144
145
146def titled(model, titles):
147 """The section's pages in order, each named by the title among `titles` it holds."""
148 return [next((text for text in texts if text in titles), '') for texts in page_list(model)]
149
150
151def offline(lab, titles, a_actions, b_actions):
152 """Creates the titled pages; A runs its actions offline while B runs and publishes its
153 own; A then reconnects. Keeps the merged order and pages, and what each client lists."""
154 a, b, output, action = lab.a, lab.b, lab.output, lab.action
155 action(a, 'pages', titles=titles)
156 lab.settle(a, lambda label: 'Body Target.' in str(page_list(lab.state(label))), 'initial')
157 deadline = time.monotonic() + 300
158 while not any(node.get('name') == 'Target' for node in action(b, 'snapshot')):
159 action(b, 'sync')
160 if time.monotonic() > deadline: raise TimeoutError('B never listed the new pages')
161 vm.qmp(a['name'], 'set_link', {'name': 'lab', 'up': False})
162 try:
163 for kind, parameters in a_actions:
164 action(a, kind, **parameters)
165 action(a, 'snapshot')
166 for kind, parameters in b_actions:
167 action(b, kind, **parameters)
168 action(b, 'snapshot')
169 lab.settle(b, lambda label: 'Body Target.' not in str(page_list(lab.state(label))), 'b-published')
170 finally:
171 vm.qmp(a['name'], 'set_link', {'name': 'lab', 'up': True})
172 # A's changes reach the server when its sync writes; wait until the file stops changing.
173 previous, steady = None, 0
174 deadline = time.monotonic() + 300
175 while steady < 3 and time.monotonic() < deadline:
176 action(a, 'sync')
177 time.sleep(10)
178 current = page_list(lab.state('merged'))
179 steady = steady + 1 if current == previous else 0
180 previous = current
181 (output / 'merged.json').write_text(json.dumps({'order': titled(lab.state('merged'), titles), 'pages': previous}, indent=2, ensure_ascii=False))
182 for client, label in ((a, 'a'), (b, 'b')):
183 action(client, 'sync')
184 time.sleep(5)
185 listed = action(client, 'snapshot')
186 (output / label / 'listed.json').write_text(json.dumps([node.get('name') for node in listed], ensure_ascii=False))
187 if listed:
188 shot(client['name'], output / label, f'{label}-merged', runner.navigation_script(listed[0].get('ID')) + 'Sleep(2000)\n')
189
190
191def pages(lab):
192 """Competing page moves, and an offline edit of a page the other client deletes."""
193 offline(lab, PAGES, [
194 ('move', {'page': 'Four', 'before': 'One'}),
195 ('move', {'page': 'Two', 'before': None}),
196 ('edit', {'expected': 'Body Target.', 'text': 'Body Target edited offline.'}),
197 ], [
198 ('move', {'page': 'Four', 'before': None}),
199 ('move', {'page': 'Three', 'before': 'One'}),
200 ('delete-page', {'page': 'Target'}),
201 ])
202
203
204def restore(lab):
205 """An offline edit of a page in the middle of the list the other client deletes, with an
206 offline move of a page the other client leaves alone."""
207 offline(lab, ['One', 'Two', 'Target', 'Three', 'Four'], [
208 ('edit', {'expected': 'Body Target.', 'text': 'Body Target edited offline.'}),
209 ('move', {'page': 'Four', 'before': 'One'}),
210 ], [
211 ('delete-page', {'page': 'Target'}),
212 ])
213
214
215def sync(server, output, scenario):
216 from document_model import EXPORTER
217
218 if linux_vm.instance_path(server).exists():
219 raise ValueError('Choose a new Linux VM name; existing machines are not owned by this run.')
220 output = output.resolve()
221 output.mkdir(parents=True, exist_ok=False)
222 scripts = output / 'scripts'
223 scripts.mkdir()
224 for name in ('cold.ps1', 'collaborate.ps1', 'network.ps1', 'text.ps1', 'stress.ps1'):
225 shutil.copyfile(ROOT / 'tools/native' / name, scripts / name)
226 shutil.copyfile(Path(__file__), output / 'native_conflict.py')
227
228 def ssh(text, timeout=90):
229 result = linux_vm.run_ssh(server, text, timeout=timeout)
230 with (output / 'server.jsonl').open('a') as log:
231 log.write(json.dumps({'command': text, 'exit': result.returncode, 'stdout': result.stdout, 'stderr': result.stderr}) + '\n')
232 result.check_returncode()
233 return result.stdout
234
235 def server_state(label):
236 """Copies the server's notebook, recycle bin included, until its section parses,
237 keeping it as `label/notebook`; returns the section's exported model."""
238 deadline = time.monotonic() + 120
239 while True:
240 target = output / label
241 shutil.rmtree(target, ignore_errors=True)
242 (target / 'notebook').mkdir(parents=True)
243 archive = subprocess.run(linux_vm.ssh_argv(server, f"tar cf - -C '/srv/agent/{SHARE}' ."), capture_output=True, check=True, timeout=60).stdout
244 subprocess.run(['tar', 'xf', '-', '-C', target / 'notebook'], input=archive, check=True)
245 result = subprocess.run([EXPORTER, target / 'notebook/synthetic.one', target / 'model'], capture_output=True, text=True)
246 if result.returncode == 0:
247 return json.loads((target / 'model/document.json').read_text())
248 if time.monotonic() > deadline:
249 raise RuntimeError(f'{label}: {result.stderr}')
250 time.sleep(1)
251
252 linux_vm.create_instance(server)
253 try:
254 linux_vm.launch(server)
255 linux_vm.wait_instance(server, 600)
256 (output / 'linux.json').write_text(json.dumps(linux_vm.load_instance(server), indent=2))
257 ssh(f'mkdir /srv/agent/{SHARE}')
258 for name in ('synthetic.one', 'Open Notebook.onetoc2'):
259 with (FIXTURE / name).open('rb') as stream:
260 subprocess.run(linux_vm.ssh_argv(server, f"cat > '/srv/agent/{SHARE}/{name}'"), stdin=stream, check=True)
261 ssh(f'chmod u+w /srv/agent/{SHARE}/*')
262 with ExitStack() as stack:
263 labels = ['a', 'b']
264 for label in labels:
265 (output / label).mkdir()
266
267 def start(label):
268 manager = clone(output / label)
269 name = manager.__enter__()
270 return label, name, manager
271 with ThreadPoolExecutor(max_workers=2) as pool:
272 started = list(pool.map(start, labels))
273 for _, _, manager in started:
274 stack.push(manager)
275 clients = {}
276 for label, name, _ in started:
277 folder = output / label
278 for local in scripts.iterdir():
279 remote = 'cold-current.ps1' if local.name == 'cold.ps1' else local.name
280 result = windows.do_put(local, 'C:\\one-tests\\' + remote, name)
281 if result.get('error'): raise RuntimeError(result['error'])
282 command(name, 'mkdir C:\\one-tests\\runs\\capture', folder)
283 command(name, 'powershell -NoProfile -ExecutionPolicy Bypass -File C:\\one-tests\\network.ps1 -LabMac ' + vm.lab_mac(name), folder)
284 result = windows.do_spawn('cmd /c powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\collaborate.ps1 -Root C:\\one-tests\\runs\\capture -SharedPath \\\\192.168.77.1\\agent\\' + SHARE + ' -CloneHost ONE-' + name.upper() + ' > C:\\one-tests\\runs\\capture\\controller.log 2>&1', name)
285 if result.get('error'): raise RuntimeError(result['error'])
286 deadline = time.monotonic() + 180
287 while windows.do_cmd('if exist C:\\one-tests\\runs\\capture\\ready (echo ready)', target=name).get('stdout', '').find('ready') < 0:
288 if time.monotonic() > deadline: raise RuntimeError('The collaboration controller did not open the notebook')
289 time.sleep(1)
290 clients[label] = {'name': name, 'folder': folder, 'sequence': 0}
291 (output / 'clients.json').write_text(json.dumps({label: client['name'] for label, client in clients.items()}, indent=2))
292
293 def action(client, kind, **parameters):
294 client['sequence'] += 1
295 sequence = client['sequence']
296 local = client['folder'] / f'command-{sequence:04}.json'
297 local.write_text(json.dumps({'action': kind, **parameters}, ensure_ascii=False))
298 inbox = f'C:\\one-tests\\runs\\capture\\inbox\\{sequence:04}'
299 result = windows.do_put(local, inbox + '.tmp', client['name'])
300 if result.get('error'): raise RuntimeError(result['error'])
301 command(client['name'], f'move {inbox}.tmp {inbox}.json', client['folder'])
302 remote = f'C:\\one-tests\\runs\\capture\\outbox\\{sequence}'
303 deadline = time.monotonic() + 600
304 while True:
305 result = windows.do_cmd(f'if exist {remote}\\error (type {remote}\\error & exit /b 1) else (if exist {remote}\\done (echo complete))', target=client['name'])
306 if result.get('exit') != 0: raise RuntimeError(str(result))
307 if 'complete' in result.get('stdout', ''): break
308 if time.monotonic() > deadline: raise TimeoutError(f'{kind} did not complete')
309 time.sleep(.5)
310 if kind != 'snapshot':
311 return None
312 hierarchy = client['folder'] / f'hierarchy-{sequence:04}.xml'
313 windows.do_get(remote + '\\hierarchy.xml', hierarchy, client['name'])
314 pages = [node for node in ET.fromstring(hierarchy.read_text(encoding='utf-8-sig').strip()).iter() if node.tag.endswith('}Page')]
315 for i in range(len(pages)):
316 windows.do_get(remote + f'\\page-{i}.xml', client['folder'] / f'snapshot-{sequence:04}-{i}.xml', client['name'])
317 return pages
318
319 def settle(client, check, label):
320 deadline = time.monotonic() + 300
321 while True:
322 action(client, 'sync')
323 if check(label): return
324 if time.monotonic() > deadline: raise TimeoutError(label)
325 time.sleep(2)
326
327 lab = Lab()
328 lab.a, lab.b, lab.output = clients['a'], clients['b'], output
329 lab.action, lab.state, lab.settle = action, server_state, settle
330 scenario(lab)
331
332 for client in clients.values():
333 try:
334 action(client, 'close')
335 except Exception as error:
336 (client['folder'] / 'close-failure.txt').write_text(str(error))
337 server_state('final')
338 finally:
339 if linux_vm.running(server):
340 linux_vm.shutdown(server, 120)
341 linux_vm.delete_instance(server)
342 (output / 'linux-teardown.json').write_text(json.dumps({'absent': not linux_vm.instance_path(server).exists()}) + '\n')
343
344
345if __name__ == '__main__':
346 parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
347 commands = parser.add_subparsers(dest='mode', required=True)
348 scenarios = {'sync': edits, 'pages': pages, 'restore': restore}
349 for mode in scenarios:
350 lab = commands.add_parser(mode)
351 lab.add_argument('linux')
352 lab.add_argument('output', type=Path)
353 reopen = commands.add_parser('cold')
354 reopen.add_argument('notebook', type=Path)
355 reopen.add_argument('output', type=Path)
356 reopen.add_argument('--delete', action='store_true')
357 args = parser.parse_args()
358 if args.mode in scenarios:
359 sync(args.linux, args.output, scenarios[args.mode])
360 else:
361 cold(args.notebook, args.output, args.delete)
362 print('Captured', args.output)
tools/smb-proxy.py+28-4
......@@ -8,6 +8,18 @@ import struct
88import time
99
1010
11def lease_state(frame, at, contexts):
12 """The lease state of a CREATE's `RqLs` context, if any (MS-SMB2 2.2.13.2.8, 2.2.14.2.10)."""
13 offset, length = struct.unpack_from('<II', frame, at + contexts)
14 cursor = at + offset if length else None
15 while cursor is not None:
16 following, name_offset, name_length, _, data_offset = struct.unpack_from('<IHHHH', frame, cursor)
17 if frame[cursor + name_offset:cursor + name_offset + name_length] == b'RqLs':
18 return struct.unpack_from('<I', frame, cursor + data_offset + 16)[0]
19 cursor = cursor + following if following else None
20 return None
21
22
1123def header_fields(offset, data):
1224 result = {}
1325 for name, start, length in [('transactions', 96, 4), ('version', 212, 16),
......@@ -77,7 +89,7 @@ async def main():
7789 at = 0
7890 while frame[at:at+4] == b'\xfeSMB':
7991 command = struct.unpack_from('<H', frame, at+12)[0]
80 entry = {'connection': connection_id, 'direction': direction, 'command': command,
92 entry = {'connection': connection_id, 'direction': direction, 'command': command, 'frame': frames[direction],
8193 'message': struct.unpack_from('<Q', frame, at+24)[0],
8294 'credit_charge': struct.unpack_from('<H', frame, at+6)[0],
8395 'credits': struct.unpack_from('<H', frame, at+14)[0]}
......@@ -110,12 +122,16 @@ async def main():
110122 elif command == 5:
111123 entry['oplock'] = frame[at+67]
112124 entry['access'] = struct.unpack_from('<I', frame, at+88)[0]
113 entry['share'] = struct.unpack_from('<I', frame, at+96)[0]
125 entry['share'], entry['disposition'], entry['options'] = struct.unpack_from('<III', frame, at+96)
114126 offset, length = struct.unpack_from('<HH', frame, at+108)
115127 entry['path'] = frame[at+offset:at+offset+length].decode('utf-16-le')
128 if entry['oplock'] == 0xff:
129 entry['lease'] = lease_state(frame, at, 112)
116130 elif command == 5 and entry['status'] == '0x0':
117131 entry['oplock'] = frame[at+66]
118132 entry['file_id'] = frame[at+128:at+144].hex()
133 if entry['oplock'] == 0xff:
134 entry['lease'] = lease_state(frame, at, 144)
119135 elif command == 9 and entry['status'] == '0x0':
120136 entry['written'] = struct.unpack_from('<I', frame, at+68)[0]
121137 if direction == 'request':
......@@ -145,12 +161,20 @@ async def main():
145161 if not next_command:
146162 break
147163 at += next_command
164 frames[direction] += 1
148165 if frame[:4] == b'\xfdSMB':
149166 record(encrypted=True, direction=direction)
150 writer.write(prefix + frame)
151 await writer.drain()
167 if state.get('delay_ms'):
168 # Half the round trip each way, as on a wide-area link: every frame
169 # arrives that much later, in order, without queueing behind others.
170 loop = asyncio.get_running_loop()
171 loop.call_at(loop.time() + state['delay_ms'] / 2000, writer.write, prefix + frame)
172 else:
173 writer.write(prefix + frame)
174 await writer.drain()
152175
153176 requests = {}
177 frames = {'request': 0, 'response': 0}
154178 tasks = [asyncio.create_task(forward(client, server_writer, 'request')),
155179 asyncio.create_task(forward(server, client_writer, 'response'))]
156180 done, pending = await asyncio.wait(tasks, return_when=asyncio.FIRST_COMPLETED)
tools/smb_lock_race.py+21-11
......@@ -13,13 +13,17 @@ import sys
1313import time
1414
1515
16def worker(file, marker, mode, operations, atomic):
16def worker(file, marker, mode, operations, atomic, shared):
1717 counts = Counter()
1818 for i in range(operations):
1919 access = ('r+b' if i % 2 else 'rb') if mode == 'mixed' else mode
20 # With shared reads, readers open O_SHLOCK and may overlap each other, never a writer.
21 reading = shared and access == 'rb'
22 held = marker.with_name(f'{marker.name}-reader-{os.getpid()}') if reading else marker
2023 stage = 'open'
2124 try:
22 with (os.fdopen(os.open(file, (os.O_RDWR if access == 'r+b' else os.O_RDONLY) | os.O_EXLOCK | os.O_NONBLOCK), access) if atomic else file.open(access)) as stream:
25 lock = os.O_SHLOCK if reading else os.O_EXLOCK
26 with (os.fdopen(os.open(file, (os.O_RDWR if access == 'r+b' else os.O_RDONLY) | lock | os.O_NONBLOCK), access) if atomic else file.open(access)) as stream:
2327 stage = 'lock'
2428 if not atomic:
2529 fcntl.flock(stream, fcntl.LOCK_EX | fcntl.LOCK_NB)
......@@ -27,8 +31,13 @@ def worker(file, marker, mode, operations, atomic):
2731 owned = False
2832 try:
2933 try:
30 marker.mkdir()
34 held.mkdir()
3135 owned = True
36 others = [p for p in marker.parent.glob(marker.name + '*') if p != held]
37 if others and not (reading and all('-reader-' in p.name for p in others)):
38 counts['overlap'] += 1
39 elif others:
40 counts['shared'] += 1
3241 except FileExistsError:
3342 counts['overlap'] += 1
3443 time.sleep(.002)
......@@ -38,7 +47,7 @@ def worker(file, marker, mode, operations, atomic):
3847 counts[f'read:{error.errno}'] += 1
3948 finally:
4049 if owned:
41 marker.rmdir()
50 held.rmdir()
4251 try:
4352 fcntl.flock(stream, fcntl.LOCK_UN)
4453 except OSError as error:
......@@ -50,10 +59,10 @@ def worker(file, marker, mode, operations, atomic):
5059 print(json.dumps(counts), flush=True)
5160
5261
53def run(output, directory, operations, atomic, clients):
62def run(output, directory, operations, atomic, clients, shared):
5463 output.mkdir(parents=True, exist_ok=False)
5564 directory.mkdir(exist_ok=False)
56 (output / 'run.json').write_text(json.dumps({'operations': operations, 'clients': clients, 'atomic': atomic,
65 (output / 'run.json').write_text(json.dumps({'operations': operations, 'clients': clients, 'atomic': atomic, 'shared_reads': shared,
5766 'platform': platform.platform(), 'script_sha256': hashlib.sha256(Path(__file__).read_bytes()).hexdigest()}, indent=2))
5867 results = {}
5968 for mode in ('rb', 'r+b', 'mixed'):
......@@ -63,7 +72,7 @@ def run(output, directory, operations, atomic, clients):
6372 processes = []
6473 try:
6574 for i in range(clients):
66 processes.append(subprocess.Popen([sys.executable, __file__, '--worker', str(file), str(marker), mode, str(operations), str(int(atomic))],
75 processes.append(subprocess.Popen([sys.executable, __file__, '--worker', str(file), str(marker), mode, str(operations), str(int(atomic)), str(int(shared))],
6776 stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True))
6877 results[mode] = []
6978 for process in processes:
......@@ -83,13 +92,13 @@ def run(output, directory, operations, atomic, clients):
8392 (output / 'result.json').write_text(json.dumps(results, indent=2))
8493 print(mode, results[mode], flush=True)
8594 assert all(c.get('acquired', 0) and not c.get('overlap', 0) and
86 all(k in ('acquired', 'open:35', 'lock:35') for k in c)
95 all(k in ('acquired', 'shared', 'open:35', 'lock:35') for k in c)
8796 for counters in results.values() for c in counters), 'Lock exclusion or progress failed'
8897
8998
9099if __name__ == '__main__':
91 if len(sys.argv) == 7 and sys.argv[1] == '--worker':
92 worker(Path(sys.argv[2]), Path(sys.argv[3]), sys.argv[4], int(sys.argv[5]), bool(int(sys.argv[6])))
100 if len(sys.argv) == 8 and sys.argv[1] == '--worker':
101 worker(Path(sys.argv[2]), Path(sys.argv[3]), sys.argv[4], int(sys.argv[5]), bool(int(sys.argv[6])), bool(int(sys.argv[7])))
93102 else:
94103 parser = argparse.ArgumentParser(description=__doc__)
95104 parser.add_argument('output', type=Path)
......@@ -97,5 +106,6 @@ if __name__ == '__main__':
97106 parser.add_argument('--operations', type=int, default=500)
98107 parser.add_argument('--atomic', action='store_true')
99108 parser.add_argument('--clients', type=int, default=4)
109 parser.add_argument('--shared-reads', action='store_true', help='With --atomic, readers open O_SHLOCK.')
100110 args = parser.parse_args()
101 run(args.output.resolve(), args.directory.resolve(), args.operations, args.atomic, args.clients)
111 run(args.output.resolve(), args.directory.resolve(), args.operations, args.atomic, args.clients, args.shared_reads)
tools/test_conflict_page.py created+166
......@@ -0,0 +1,166 @@
1"""Conflict pages (corpus/conflict-page): what OneNote 2010 stores and shows for one, and
2that it reads the ones Rust writes and deletes."""
3import json
4from pathlib import Path
5import runpy
6import shutil
7import subprocess
8from tempfile import TemporaryDirectory
9import unittest
10import hashlib
11
12from native_xml import pages, texts
13from document_model import EXPORTER, ordered_pages, view
14
15ROOT = Path(__file__).resolve().parent.parent
16FIXTURE = ROOT / 'corpus/conflict-page'
17compare = runpy.run_path(str(ROOT / 'tools/verify-document.py'))['compare']
18DEFAULT = '{00000000-0000-0000-0000-000000000000},0'
19RENDER, SELECT = 0x88001D96, 0x88001DDB
20
21
22def model(notebook, section='synthetic.one'):
23 with TemporaryDirectory() as temporary:
24 output = Path(temporary) / 'model'
25 subprocess.run([EXPORTER, notebook / section, output], check=True, capture_output=True)
26 return json.loads((output / 'document.json').read_text())
27
28
29def conflicts(document):
30 """Each listed page's conflict pages: whose version, its texts and the marked texts."""
31 listed = {}
32 for sid, _, page, _ in ordered_pages(document):
33 manifest = page['nodes'][page['roots']['1']]
34 versions = []
35 for space in manifest['spaces']:
36 _, revision = view(document, space)
37 metadata = revision['nodes'][revision['roots']['2']]['kind']
38 rich = [node for node in revision['nodes'].values() if node['kind']['type'] == 'RichText']
39 flagged = lambda node, flag: any(field['id'] == flag for fields in node['extra'] for field in fields)
40 versions.append({'user': metadata['author'], 'texts': sorted(node['kind']['text'] for node in rich),
41 'marked': sorted(node['kind']['text'] for node in rich if flagged(node, RENDER) and flagged(node, SELECT))})
42 listed[sid] = versions
43 return listed
44
45
46def page_texts(document, sid):
47 _, page = view(document, sid)
48 return sorted(node['kind']['text'] for node in page['nodes'].values() if node['kind']['type'] == 'RichText')
49
50
51TITLES = {'One', 'Two', 'Three', 'Four', 'Target'}
52
53
54def series(document):
55 """The section's page series in order, each as the titles of its pages."""
56 _, root = view(document, document['root'])
57 section = root['nodes'][root['roots']['1']]
58 return {oid: [next((text for text in page_texts(document, space) if text in TITLES), '')
59 for space in root['nodes'][oid]['spaces']] for oid in section['children']}
60
61
62def spaces(document):
63 """Each listed page's space by its title."""
64 return {next((text for text in page_texts(document, sid) if text in TITLES), ''): sid
65 for sid, *_ in ordered_pages(document)}
66
67
68class ConflictPageTest(unittest.TestCase):
69 def test_onenote_keeps_the_remote_version_and_the_local_one_as_a_conflict_page(self):
70 document = model(FIXTURE / 'native/conflict/notebook')
71 (sid, versions), = conflicts(document).items()
72 # Client A's disjoint edit merged; the paragraph both changed keeps client B's text.
73 self.assertIn('Client A disjoint edit.', page_texts(document, sid))
74 self.assertIn('Client B conflicting edit.', page_texts(document, sid))
75 self.assertEqual(versions, [{'user': 'snow',
76 'texts': ['', 'Client A conflicting edit.', 'Client A disjoint edit.', 'Left cell', 'Right cell'],
77 'marked': ['Client A conflicting edit.']}])
78 # COM lists the page alone; the conflict page shows only through its bar.
79 for client in 'ab':
80 self.assertTrue((FIXTURE / f'native/{client}/{client}-shown.png').is_file())
81
82 def test_onenote_deletes_a_conflict_page_from_its_page_only(self):
83 document = model(FIXTURE / 'native-delete/notebook')
84 self.assertEqual(conflicts(document), {sid: [] for sid in conflicts(document)})
85 self.assertEqual(len(pages(FIXTURE / 'native-delete/read')), 1)
86
87 def test_three_clients_conflict_pages_list_the_last_stored_first(self):
88 document = model(FIXTURE / 'native-three/notebook')
89 (_, versions), = conflicts(document).items()
90 self.assertEqual([version['user'] for version in versions],
91 ['ONE-M6-DDE10E0D', 'ONE-M6-9451B6C6', 'ONE-M6-053BD357'])
92
93 def test_onenote_reads_a_rust_conflict_page_shows_its_bar_and_deletes_it(self):
94 source = json.loads((FIXTURE / 'cold/source.json').read_text())
95 written = hashlib.sha256((FIXTURE / 'candidate/synthetic.one').read_bytes()).hexdigest()
96 self.assertIn(written, [item['sha256'] for item in source])
97 with TemporaryDirectory() as temporary:
98 read = Path(temporary) / 'read'
99 shutil.copytree(FIXTURE / 'cold/before-read', read)
100 compare(FIXTURE / 'candidate', read)
101 (page,) = pages(FIXTURE / 'cold/before-read')
102 self.assertIn('Remote: café, 東京, مرحبا', texts(page))
103 document = model(FIXTURE / 'candidate')
104 (_, versions), = conflicts(document).items()
105 self.assertEqual(versions[0]['user'], 'Clover Snow')
106 self.assertEqual(versions[0]['marked'], ['Snowbound: café, 東京, مرحبا'])
107 # The bar on the page opens the conflict page, as OneNote's own do, and its menu
108 # deletes it, leaving the page without a bar.
109 titles = {name: json.loads((FIXTURE / f'cold/cold-{name}.json').read_text())['win']['title']
110 for name in ('main', 'shown', 'delete')}
111 self.assertEqual(titles, {'main': 'Remote: café, 東京, مرحبا - Microsoft OneNote',
112 'shown': 'Snowbound: café, 東京, مرحبا - Microsoft OneNote',
113 'delete': 'Remote: café, 東京, مرحبا - Microsoft OneNote'})
114 deleted = model(FIXTURE / 'cold/notebook')
115 self.assertEqual(conflicts(deleted), {sid: [] for sid in conflicts(deleted)})
116
117 def test_onenote_reads_a_conflict_page_rust_deleted(self):
118 document = model(FIXTURE / 'candidate-delete')
119 self.assertEqual(conflicts(document), {sid: [] for sid in conflicts(document)})
120 with TemporaryDirectory() as temporary:
121 read = Path(temporary) / 'read'
122 shutil.copytree(FIXTURE / 'cold-delete/read', read)
123 compare(FIXTURE / 'candidate-delete', read)
124 (page,) = pages(FIXTURE / 'cold-delete/read')
125 self.assertIn('Client B conflicting edit.', texts(page))
126
127 def test_page_moves_merge_as_page_series_do(self):
128 """Each client gives a page it moves a series of its own; the merge keeps each page
129 in the series of the client that gave it one, the server's where both did, and
130 leaves the other client's series empty. A page the server deleted comes back in a
131 new series where the offline client had it."""
132 for capture, order, empty, server, kept, client in (
133 ('native-pages', ['One', 'Two', 'Four', 'Three', 'Target'], 2, ['One', 'Two', 'Four'], [], ['Three', 'Target']),
134 ('native-restore', ['Four', 'One', 'Two', 'Target', 'Three'], 0, [], ['Four'], ['One', 'Two', 'Three', 'Target'])):
135 root = FIXTURE / capture
136 initial, published, merged = (series(model(root / phase / 'notebook'))
137 for phase in ('initial', 'b-published', 'merged'))
138 self.assertEqual([titles for titles in merged.values() if titles][1:], [[title] for title in order])
139 self.assertEqual(sum(not titles for titles in merged.values()), empty, capture)
140 self.assertEqual(json.loads((root / 'merged.json').read_text())['order'][1:], order)
141 for listed in 'ab':
142 self.assertEqual(json.loads((root / listed / 'listed.json').read_text())[1:], order)
143 owner = lambda state, title: next((oid for oid, titles in state.items() if title in titles), None)
144 for title in server:
145 self.assertEqual(owner(merged, title), owner(published, title), title)
146 self.assertNotIn(owner(merged, title), initial, title)
147 for title in kept:
148 self.assertEqual(owner(merged, title), owner(initial, title), title)
149 for title in client:
150 self.assertNotIn(owner(merged, title), {**initial, **published}, title)
151
152 def test_a_page_edited_offline_and_deleted_online_comes_back_as_a_new_page(self):
153 for capture in ('native-pages', 'native-restore'):
154 root = FIXTURE / capture
155 before = spaces(model(root / 'initial/notebook'))
156 document = model(root / 'merged/notebook')
157 target = spaces(document)['Target']
158 self.assertNotEqual(target, before['Target'], capture)
159 self.assertIn('Body Target edited offline.', page_texts(document, target))
160 # The deleting client's version stays in the recycle bin.
161 deleted = model(root / 'merged/notebook/OneNote_RecycleBin', 'OneNote_DeletedPages.one')
162 self.assertIn('Body Target.', page_texts(deleted, spaces(deleted)['Target']), capture)
163
164
165if __name__ == '__main__':
166 unittest.main()
tools/test_extended_ascii.py created+31
......@@ -0,0 +1,31 @@
1from pathlib import Path
2import runpy
3import shutil
4from tempfile import TemporaryDirectory
5import unittest
6import xml.etree.ElementTree as ET
7
8from native_xml import ns
9
10ROOT = Path(__file__).resolve().parent.parent
11FIXTURE = ROOT / 'corpus/extended-ascii'
12compare = runpy.run_path(str(ROOT / 'tools/verify-document.py'))['compare']
13
14
15class ExtendedAsciiTest(unittest.TestCase):
16 def test_onenote_reads_extended_ascii_as_latin_1_whatever_the_language(self):
17 expected = 'Extended bytes: ' + ' '.join(chr(byte) for byte in range(128, 256))
18 for language in ('en', 'ru', 'ja'):
19 with self.subTest(language=language):
20 capture = FIXTURE / language / 'cold'
21 with TemporaryDirectory() as temporary:
22 read = Path(temporary) / 'read'
23 shutil.copytree(capture / 'read', read)
24 compare(FIXTURE / language / 'candidate', read)
25 page = ET.parse(capture / 'read/page-000.xml').getroot()
26 texts = [t.text for t in page.iterfind('.//one:Outline//one:T', ns)]
27 self.assertEqual(texts, [expected.replace('\xa0', '&nbsp;')])
28
29
30if __name__ == '__main__':
31 unittest.main()
tools/test_native_stress.py+2-2
......@@ -9,7 +9,7 @@ from unittest.mock import patch
99import native_stress
1010from native_stress import edit_history, native_history, verify_capture
1111from native_collaboration import reachable_page_text
12from document_model import DEFAULT_CONTEXT
12from document_model import DEFAULT_CONTEXT, EXPORTER
1313
1414
1515class NativeHistoryTests(unittest.TestCase):
......@@ -17,7 +17,7 @@ class NativeHistoryTests(unittest.TestCase):
1717 root = Path(__file__).resolve().parent.parent
1818 with tempfile.TemporaryDirectory() as folder:
1919 output = Path(folder) / 'model'
20 subprocess.run([root / 'target/debug/examples/document',
20 subprocess.run([EXPORTER,
2121 root / 'corpus/collaboration/round-01/offline/notebook/synthetic.one', output], check=True)
2222 model = json.loads((output / 'document.json').read_text())
2323 retained = reachable_page_text(model)