authorgravatar for git@paperclover.netclover caruso <git@paperclover.net> 2026-09-26 14:32:33-07:00
committergravatar for git@paperclover.netclover caruso <git@paperclover.net> 2026-09-27 04:23:49-07:00
logb305654ab5ef9281fe63f30857a5ae23fa611558
treee462cf86d464276cd529148fdf369876f76b5ec3
parented9e69d6d927206decb398664df5f135e7244603
signature Signed by SSH key SHA256:52mNGHRsVFBDED9IAX5pe+LRWUefqTbxEReunq21QvU

feat(data): conflict pages, SMB like OneNote, and no whole-image edit API left

Conflicts publish as OneNote 2010 conflict pages (one-sided merge, conflict page under the page, OneNote's delete) instead of a review; cache schema 16 drops the dead columns. SMB publishes in compound requests with the spec's write order, the remote stamp is required, and our locks match OneNote's (three OneNote and three Rust clients converge). Every whole-page and whole-image edit API is gone from onestore and notebook: tests, examples, fuzz targets and evidence generators use ops, and protected sections have an ops path. Edits carry the author's name and initials. New corpus rows: conflict-page, page-date, extended-ascii. Assisted-by: claude-opus-5.5 Assisted-by: claude-opus-5

472 files changed, 12831 insertions(+), 8187 deletions(-)

corpus/conflict-page/README.md created+77
...@@ -0,0 +1,77 @@
1# Conflict pages
2
3What OneNote 2010 does when a sync finds that two clients changed the same object, and
4that it reads and deletes the conflict pages Rust writes. `tools/native_conflict.py`
5made every capture; `tools/test_conflict_page.py` checks them without a VM.
6
7`native` is two OneNote clients on one Samba notebook (`native_conflict.py sync`, fixture
8`native-ink/cold-ui-ink`): client A, offline, rewrote the first outline's paragraph and the
9second outline's; client B rewrote the same first paragraph and the left table cell and
10published (`b-published`); A reconnected and synced (`conflict`). A kept B's version of the
11paragraph both changed and merged its own disjoint edit into the page; its own version of the
12page became a conflict page:
13
14- its own object space, listed only in the page manifest's `ChildGraphSpaceElementNodes`
15 (`0x2c001d63`), never in the page series, with a copy of its metadata in the manifest's
16 `0x24003442` whose identity is the space's XOR the metadata salt (MS-ONE 2.2.81);
17- metadata root `jcidConflictPageMetaData` (`0x20038`): `CachedTitleString`, a fresh
18 `NotebookManagementEntityGuid`, `PageLevel`, both schema revisions (`0x28`),
19 `TopologyCreationTimeStamp` of the merge, `ConflictingUserName` and
20 `ConflictingUserInitials` of the client whose version it is;
21- a page node with `IsReadOnly`, `Deletable` and `IsConflictPage`, holding A's whole page;
22 only the rich text of the paragraph both changed has `IsConflictObjectForSelection` and
23 `IsConflictObjectForRender`;
24- the page's metadata and the section's copy of it gain `HasConflictPages`.
25
26Both clients show the same UI (`a/`, `b/`): the page's tab carries a conflict mark and a
27yellow bar reads "This page has changes that could not be merged during synchronization.
28Click here to show versions of the page with unmerged changes." Clicking it lists the
29conflict pages under the page, each as its date and user ("9/26/2026 snow"), and opens the
30first; that page's bar reads "Conflicting changes are highlighted in red. This page cannot be
31edited, but you can copy changes to the primary page. Click here for more options." over a
32light red highlight (RGB 255, 214, 214), and its menu offers Delete Conflict Page, Copy Page
33To..., Select Previous/Next Conflicting Change and Collapse Conflict Pages.
34
35`native-three` is `evidence/m8/conflict-abrupt-cold-01`: three clients' conflict pages under
36one page. OneNote appends each to `ChildGraphSpaceElementNodes` and lists them last first
37(`conflicts.png`); the metadata copies are in their own order.
38
39`native-delete` is OneNote's Delete Conflict Page on `native/conflict` in a fresh clone
40(`native_conflict.py cold --delete`): the manifest loses `ChildGraphSpaceElementNodes` (its
41last entry) but keeps the metadata copy, the page's metadata and the section's copy lose
42`HasConflictPages`, and the conflict page's space is left as it was.
43
44`native-pages` and `native-restore` are page-list merges (`native_conflict.py pages`,
45`restore`): client A, offline, moved pages through COM and edited `Target`'s body while
46client B moved pages and deleted `Target`; A reconnected (`merged`, both clients' `listed.json`
47and screenshots). OneNote merges page series, not page positions. A page a client moves
48gets a series of its own, and a COM move also re-series every page it jumps over (a drag
49re-series only the dragged pages, `page-lifecycle/movement`). The merge keeps each page in
50the series of the client that re-seriesed it, the server's where both did, leaving the
51other's series empty (`native-pages` keeps two, which MS-ONE's one-page minimum does not
52allow); series only A made go after the server's. `Target` comes back as a new page space
53holding A's edit, in a new series where A had it; B's version stays in the recycle bin.
54
55- `native-pages`: from One, Two, Three, Four, Target, A moved Four before One and Two last
56 (re-seriesing One, Two, Three), B moved Four last and Three before One (re-seriesing One,
57 Two, Four): One, Two, Four, Three, Target.
58- `native-restore`: from One, Two, Target, Three, Four, A moved Four before One
59 (re-seriesing One, Two, Target, Three) and B deleted Target: Four, One, Two, Target, Three.
60
61`page_moves_and_a_removed_page_merge_as_onenote_merges_them` in
62`crates/notebook/tests/sync_pages.rs` replays both with Snowbound's queue, A's moves as
63moves of the pages A re-seriesed, and reaches OneNote's order.
64
65`candidate` is `a_conflict_page_is_stored_as_onenote_stores_one` in
66`crates/onestore/tests/conflict.rs` (`ONESTORE_CONFLICT_EXPORT`): the remote renamed the
67first word of the fixture's first paragraph, this machine did too, and `SectionOp::Conflict`
68kept this machine's page for "Clover Snow", whose author objects carry her initials beside
69her name as OneNote's do. `cold` is OneNote's fresh read and the same UI
70walk, ending in Delete Conflict Page on the Rust-written page: its bar and list entry show,
71the conflict page opens (window title) with the paragraph highlighted, and the deletion
72stores what `native-delete` did.
73
74`candidate-delete` is `deleting_a_conflict_page_stores_what_onenote_stores`
75(`ONESTORE_CONFLICT_DELETE_EXPORT`): `SectionOp::Delete` of `native/conflict`'s conflict page,
76which stores the objects OneNote's deletion did; `cold-delete` is its fresh read, without a
77bar.
corpus/conflict-page/candidate-delete/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/candidate-delete/Open Notebook.onetoc2 differ
corpus/conflict-page/candidate-delete/synthetic.one created
Binary files /dev/null and b/corpus/conflict-page/candidate-delete/synthetic.one differ
corpus/conflict-page/candidate/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/candidate/Open Notebook.onetoc2 differ
corpus/conflict-page/candidate/synthetic.one created
Binary files /dev/null and b/corpus/conflict-page/candidate/synthetic.one differ
corpus/conflict-page/cold-delete/commands.jsonl created+3
...@@ -0,0 +1,3 @@
1{"command": "powershell -NoProfile -Command \"Expand-Archive -LiteralPath C:\\one-tests\\transfer.zip -DestinationPath C:\\one-tests\\runs\\capture\\notebook\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-4D3235A0 -ExpectedPages 1 -KeepOpen", "exit": 0, "stdout": "Read 1 sections and 1 pages.\r\n", "stderr": "", "error": null}
3{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\read -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
corpus/conflict-page/cold-delete/machine.json created+1
...@@ -0,0 +1 @@
1{"name": "m6-4d3235a0", "hostname": "ONE-M6-4D3235A0"}
corpus/conflict-page/cold-delete/read/af1af4b8987109ce1e335e538910fb3860b7c57eb3b6110a06aa2114e06d69a7.attachment created+1
...@@ -0,0 +1 @@
1Fictitious attachment for native corpus.
\ No newline at end of file
corpus/conflict-page/cold-delete/read/environment.json created+7
...@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-4D3235A0",
5 "cold": true,
6 "onenote": "14.0.4763.1000"
7}
corpus/conflict-page/cold-delete/read/hierarchy.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{1421A7BE-8880-40AC-A90F-B8EBEDAA41C7}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-27T01:42:28.000Z" color="#9595AA"><one:Section name="synthetic" ID="{F27BEF77-EE9A-0A3A-12FB-EA2DDBE6F411}{1}{B0}" path="C:\one-tests\runs\capture\notebook\synthetic.one" lastModifiedTime="2026-09-27T01:42:28.000Z" color="#8AA8E4"><one:Page ID="{DE321CF1-6751-492B-959C-42BC27DEE80E}{1}{B0}" name="Client B conflicting edit." dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1"/></one:Section></one:Notebook>
corpus/conflict-page/cold-delete/read/page-000.navigation.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Client B conflicting edit. - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/cold-delete/read/page-000.png created
Binary files /dev/null and b/corpus/conflict-page/cold-delete/read/page-000.png differ
corpus/conflict-page/cold-delete/read/page-000.xml created+21
...@@ -0,0 +1,21 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{DE321CF1-6751-492B-959C-42BC27DEE80E}{1}{B0}" name="Client B conflicting edit." dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="ONE-M6-D63654A2" authorInitials="4A2" lastModifiedBy="ONE-M6-D63654A2" lastModifiedByInitials="4A2" lastModifiedTime="2026-09-27T01:00:05.000Z" objectID="{B9FEB44D-2C24-4A9D-9A6E-CE67D8420D6F}{10}{B0}"><one:Position x="36.0" y="14.40000057220459" z="1"/><one:Size width="127.5136947631836" height="13.42771625518799"/><one:OEChildren><one:OE creationTime="2026-09-27T01:00:05.000Z" lastModifiedTime="2026-09-27T01:00:05.000Z" objectID="{B9FEB44D-2C24-4A9D-9A6E-CE67D8420D6F}{11}{B0}" alignment="left" quickStyleIndex="0" style="font-family:Calibri;font-size:11.0pt;color:#1F4E79"><one:T><![CDATA[Client B conflicting edit.]]></one:T></one:OE></one:OEChildren></one:Outline><one:InkDrawing lastModifiedTime="2026-09-05T05:11:40.000Z" objectID="{9B1398DF-74CC-442E-87F6-6CEF18C260BB}{15}{B0}"><one:Position x="421.5118103027344" y="84.75590515136718" z="5"/><one:Size width="60.00945281982422" height="60.00944519042969"/><one:Data>AK4FHQSiAaIBAWAqABofi0JwRYR1gL7QRdkE+oeVMvLsF0G4wBa/8yv0et67E4vlXrNInJuhJVq3
3URK/7HHMzCKxQYkOuxMs7nqhWM9UiuaXxU+PBvi60uGbIsRmbacNslhJmh1OQDQGbBADBkgQRSNG
4IwUIC2VPZmdoZGkZFDIIAIAeAiHi4kEzCADwEAKLxeJBZAMVRgBACpUE/QGH81d3mq7mkzgM0mcB
5mk1gEBgEBgEBgE1mUAgMzmUAmsygE1mUzmkBmc0gMzmk1mUAmsyTOaTWZQCazKZgALSmiazKATWZ
6QCAwCAwBAZpM4DAJlNYBMprAJlNZpM4DNJmmU1mkzgM0mcBmkzmU1gEyAAFpTSAzOaQGZzRAYBAY
7BAYBMprAIDNJnAZpM0yms0mcBmkzgM0mcymsAmU1gEyms0maZAACZTSawCZTWATKawCAoBAYBAZn
8NIDAJrMoBNZlAJrMpnMoBNZlAJrMpnNIDM5pAZnNJrMkzmk1AACH8C7XgWnmkzgM0msAmU1gEBgE
9BgEBTOaQGATWZQCazKATWZTOaTWZTNNJrMpmmUzTSaplM00mqZTOaTVMpmmk1TKATKazSawCAzSZ
10wGAQFAIDAJrMoBAZnNIDM5pAZnMpnNJrMpmmk1mUzTSaplM00mqZTNNJrMpmmk1TKZplhuAzKATW
11ZQCazKAIDAIDAIDNJnAYBMprAJlM4DNJnMprNJnMk1mkzmSazRM5kms0TOZJnMprNEzmSazQtKaQ
12GZzRNZlAIDAIDAIDAJlNYAmU1gEymsAmU1mkzmU1miZzKazRM5kmcyTWaJnMk1mkzmSazRM5kAUU
13RgBAAAAJBAUURgAAAAASABEgXhT+65lz/EiPi+8SFvjqCAk+gEcDAa/89Qd0AxVGAEAKABEg2R87
14QQrSFUA=</one:Data></one:InkDrawing><one:InkDrawing lastModifiedTime="2026-09-05T05:11:41.000Z" objectID="{9B1398DF-74CC-442E-87F6-6CEF18C260BB}{23}{B0}"><one:Position x="504.0" y="92.23936462402343" z="6"/><one:Size width="30.01890563964844" height="45.04251861572266"/><one:Data>AOECHQJSegFgKgAaH4tCcEWEdYC+0EXZBPqHlTLy7BdBuMAWv/Mr9HreuxOL5V6zSJyboSVat1ES
15v+xxzMwisUGJDrsTLO56oVjPVIrml8VPjwb4utLhmyLEZm2nDbJYSZodTkA0BmwQAwZIEEUjRiMF
16CAtlT2ZnaGRpGRQyCACAHgIh4uJBMwgA8BACi8XiQWQDFUYAQArKAUuH88s3nkzmkzgM0mcBmkzg
17KAQGAQGATWZQCAzOaQGZzKATWZTOaQGZzSAzOaTWZQCazJM5pNZlAJrMpmAAAACH8D9XgeBJlM0z
18mRNZpNZpM4DAJlNYBAUAgMAgMzmkBgE1mUAmsyTOaTWZTOaTVMpnNJqmUzTSaplM00mqZTOZTNNJ
19qmUzTSagBRRGAIAAAAkEBRRGAAAAABIAESDPTQ4xwIZyTqQ8FVDbHxeKCj9AI5rARvPPUDugAxVG
20AEAKABEgLMlfQdkfI0A=</one:Data></one:InkDrawing><one:Outline author="ONE-M6-8A93780A" authorInitials="80A" lastModifiedBy="ONE-M6-8A93780A" lastModifiedByInitials="80A" lastModifiedTime="2026-09-27T00:59:59.000Z" objectID="{EF4E6858-9F71-40C0-80D8-4CEF7C4EE926}{10}{B0}"><one:Position x="144.0" y="96.0" z="2"/><one:Size width="132.69140625" height="13.42771339416504"/><one:OEChildren><one:OE creationTime="2026-09-27T00:59:59.000Z" lastModifiedTime="2026-09-27T00:59:59.000Z" objectID="{EF4E6858-9F71-40C0-80D8-4CEF7C4EE926}{11}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Client A disjoint edit.]]></one:T></one:OE></one:OEChildren></one:Outline><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-05T05:06:58.000Z" objectID="{D26700D7-79EB-496E-B6A8-E71B9023A1A8}{10}{B0}"><one:Position x="144.0" y="192.0" z="3"/><one:Size width="72.0" height="0.750005722045898"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:58.000Z" lastModifiedTime="2026-09-05T05:06:58.000Z" objectID="{D26700D7-79EB-496E-B6A8-E71B9023A1A8}{11}{B0}" alignment="left"><one:Image format="png" originalPageNumber="0"><one:Data>iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAusB9Y9J1uoA
21AAAASUVORK5CYII=</one:Data></one:Image></one:OE></one:OEChildren></one:Outline><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-05T05:06:59.000Z" objectID="{577D18B7-BF93-4AA8-9305-80E5A4F26E9B}{10}{B0}"><one:Position x="264.0" y="192.0" z="4"/><one:Size width="72.00001525878906" height="63.0"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:59.000Z" lastModifiedTime="2026-09-05T05:06:59.000Z" objectID="{577D18B7-BF93-4AA8-9305-80E5A4F26E9B}{11}{B0}" alignment="left"><one:InsertedFile pathCache="C:\Users\clover\AppData\Local\Microsoft\OneNote\14.0\OneNoteOfflineCache_Files\458001d8-0f1e-451f-81cb-621828fc0b39.txt" pathSource="C:\one-tests\runs\20260905-05\assets\fictitious-attachment.txt" preferredName="fictitious-attachment.txt"/></one:OE></one:OEChildren></one:Outline><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-27T01:00:54.000Z" objectID="{79A261E7-DE03-4128-8C0D-62B08CA195DD}{10}{B0}"><one:Position x="144.0" y="312.0" z="0"/><one:Size width="92.42181396484374" height="48.80315780639648"/><one:OEChildren><one:OE creationTime="2026-09-27T01:00:05.000Z" lastModifiedTime="2026-09-27T01:00:54.000Z" objectID="{81219965-9957-428D-81B5-197CB0D3C873}{27}{B0}" alignment="left"><one:Table bordersVisible="true" lastModifiedTime="2026-09-27T01:00:54.000Z" objectID="{81219965-9957-428D-81B5-197CB0D3C873}{28}{B0}"><one:Columns><one:Column index="0" width="45.947021484375"/><one:Column index="1" width="38.34479141235352"/></one:Columns><one:Row objectID="{81219965-9957-428D-81B5-197CB0D3C873}{29}{B0}" lastModifiedTime="2026-09-27T01:00:54.000Z"><one:Cell lastModifiedTime="2026-09-27T01:00:54.000Z" objectID="{81219965-9957-428D-81B5-197CB0D3C873}{30}{B0}" lastModifiedByInitials="4A2"><one:OEChildren><one:OE creationTime="2026-09-27T01:00:05.000Z" lastModifiedTime="2026-09-27T01:00:05.000Z" objectID="{79A261E7-DE03-4128-8C0D-62B08CA195DD}{17}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Client B disjoint cell.]]></one:T></one:OE></one:OEChildren></one:Cell><one:Cell lastModifiedTime="2026-09-27T01:00:54.000Z" objectID="{81219965-9957-428D-81B5-197CB0D3C873}{37}{B0}" lastModifiedByInitials="4A2"><one:OEChildren><one:OE creationTime="2026-09-27T01:00:05.000Z" lastModifiedTime="2026-09-27T01:00:05.000Z" objectID="{79A261E7-DE03-4128-8C0D-62B08CA195DD}{25}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Right cell]]></one:T></one:OE></one:OEChildren></one:Cell></one:Row></one:Table></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/conflict-page/cold-delete/read/payloads.json created+10
...@@ -0,0 +1,10 @@
1[
2 {
3 "sha256": "af1af4b8987109ce1e335e538910fb3860b7c57eb3b6110a06aa2114e06d69a7",
4 "name": "fictitious-attachment.txt",
5 "object": "{577D18B7-BF93-4AA8-9305-80E5A4F26E9B}{11}{B0}",
6 "kind": "InsertedFile",
7 "page": "{DE321CF1-6751-492B-959C-42BC27DEE80E}{1}{B0}",
8 "bytes": 43
9 }
10]
\ No newline at end of file
corpus/conflict-page/cold-delete/run.json created+18
...@@ -0,0 +1,18 @@
1{
2 "notebook": "/private/tmp/gate/candidate-delete",
3 "expected_pages": 1,
4 "author": null,
5 "author_timeout_seconds": 600,
6 "inspect": false,
7 "collect_notebook": false,
8 "base": {
9 "file": "win7-office-base.qcow2",
10 "format": "qcow2",
11 "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346",
12 "virtual_size": 68719476736
13 },
14 "scripts": {
15 "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331",
16 "read.ps1": "04013bfcccee40a17e2a225f9b9e96f40a3a8daad9e350609654f8eda3ccbb41"
17 }
18}
corpus/conflict-page/cold-delete/scripts/cold.ps1 created+27
...@@ -0,0 +1,27 @@
1param([Parameter(Mandatory=$true)][string]$Root, [string]$CloneHost = '')
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
5if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
6 throw 'Choose a run directly below C:\one-tests\runs.'
7}
8if (Get-Process ONENOTE -ErrorAction SilentlyContinue) { throw 'Close OneNote before resetting its test cache.' }
9$key = 'HKCU:\Software\Microsoft\Office\14.0\OneNote'
10if ($CloneHost) {
11 if ($CloneHost -notmatch '^ONE-[A-Z0-9-]+$' -or [Environment]::MachineName -ne $CloneHost) {
12 throw 'The disposable clone hostname does not match this machine.'
13 }
14 New-Item "$key\Options\Paths" -Force | Out-Null
15 New-ItemProperty "$key\Options\Paths" -Name UnfiledNotesSection -PropertyType ExpandString -Value 'C:\one-tests\Loose.one' -Force | Out-Null
16} elseif ((Get-ItemProperty "$key\Options\Paths").UnfiledNotesSection -ne 'C:\one-tests\Loose.one' -or
17 -not (Test-Path 'C:\one-tests\profile-original-cache')) {
18 throw 'Park the personal OneNote profile before resetting the test cache.'
19}
20$cache = Join-Path $env:LOCALAPPDATA 'Microsoft\OneNote\14.0'
21$parked = Join-Path 'C:\one-tests\caches' ([IO.Path]::GetFileName($root))
22if (Test-Path $parked) { throw 'Choose a new run; its parked cache already exists.' }
23New-Item -ItemType Directory -Path 'C:\one-tests\caches' -Force | Out-Null
24if (Test-Path $cache) { Move-Item -LiteralPath $cache -Destination $parked }
25if (Test-Path "$key\OpenNotebooks") { Remove-Item "$key\OpenNotebooks" -Recurse }
26New-Item "$key\OpenNotebooks" | Out-Null
27New-ItemProperty "$key\OpenNotebooks" -Name '1' -PropertyType String -Value "$root\notebook" | Out-Null
corpus/conflict-page/cold-delete/scripts/read.ps1 created+142
...@@ -0,0 +1,142 @@
1param(
2 [Parameter(Mandatory=$true)][string]$Root,
3 [int]$ExpectedPages = -1,
4 [switch]$UseCurrentCache,
5 [switch]$Pdf,
6 [switch]$KeepOpen,
7 [string]$CloneHost = ''
8)
9Set-StrictMode -Version Latest
10$ErrorActionPreference = 'Stop'
11$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
12if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
13 throw 'Choose a run directly below C:\one-tests\runs.'
14}
15$notebook = Join-Path $root 'notebook'
16$output = Join-Path $root 'read'
17if (Test-Path $output) { throw 'Choose a new read destination.' }
18if ($UseCurrentCache) {
19 if ((Get-ItemProperty 'HKCU:\Software\Microsoft\Office\14.0\OneNote\Options\Paths').UnfiledNotesSection -ne 'C:\one-tests\Loose.one') {
20 throw 'Park the personal OneNote profile before reading test notebooks.'
21 }
22} else {
23 & "$PSScriptRoot\cold-current.ps1" -Root $root -CloneHost $CloneHost
24}
25New-Item -ItemType Directory -Path $output | Out-Null
26$app = New-Object -ComObject OneNote.Application
27$notebookId = ''
28$failure = $null
29try {
30 $app.OpenHierarchy($notebook, '', [ref]$notebookId, 0)
31 $process = Get-Process ONENOTE
32 @{ hostname = [Environment]::MachineName; onenote = $process.MainModule.FileVersionInfo.FileVersion;
33 powershell = $PSVersionTable.PSVersion.ToString(); schema = 'xs2010'; cold = (-not $UseCurrentCache.IsPresent) } |
34 ConvertTo-Json | Set-Content (Join-Path $output 'environment.json') -Encoding UTF8
35 $sections = @()
36 foreach ($file in @(Get-ChildItem $notebook -Recurse | Where-Object { $_.Extension -eq '.one' })) {
37 $id = ''
38 $app.OpenHierarchy($file.FullName, '', [ref]$id, 0)
39 $sections += $id
40 }
41 $deadline = [DateTime]::UtcNow.AddSeconds(300)
42 $previous = ''
43 $lastChange = ''
44 $stableSince = [DateTime]::UtcNow
45 $settled = $false
46 do {
47 $pages = @{}
48 foreach ($section in $sections) {
49 $hierarchy = ''
50 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
51 [xml]$xml = $hierarchy
52 foreach ($node in $xml.SelectNodes('//*[@path]')) {
53 if (-not $node.GetAttribute('path').StartsWith("$notebook\", [StringComparison]::OrdinalIgnoreCase)) {
54 throw 'OneNote opened a section outside the copied notebook.'
55 }
56 }
57 foreach ($node in $xml.SelectNodes('//*[local-name()="Page"]')) {
58 $id = $node.GetAttribute('ID')
59 $content = ''
60 $app.GetPageContent($id, [ref]$content, 1, 1)
61 $pages[$id] = $content
62 }
63 }
64 $signature = [String]::Join('|', @($pages.Keys | Sort-Object | ForEach-Object { $_ + $pages[$_] }))
65 if ($signature -ne $previous) {
66 $lastChange = $previous
67 $previous = $signature
68 $stableSince = [DateTime]::UtcNow
69 }
70 if ((($ExpectedPages -ge 0 -and $pages.Count -eq $ExpectedPages) -or
71 ($ExpectedPages -lt 0 -and $pages.Count -gt 0)) -and
72 ([DateTime]::UtcNow - $stableSince).TotalSeconds -ge 2) { $settled = $true; break }
73 Start-Sleep -Milliseconds 250
74 } while ([DateTime]::UtcNow -lt $deadline)
75 if (-not $settled -or ($ExpectedPages -ge 0 -and $pages.Count -ne $ExpectedPages) -or ($ExpectedPages -lt 0 -and $pages.Count -eq 0)) {
76 [IO.File]::WriteAllText((Join-Path $output 'previous-signature.txt'), $lastChange, [Text.Encoding]::UTF8)
77 $index = 0
78 foreach ($id in @($pages.Keys | Sort-Object)) {
79 [IO.File]::WriteAllText((Join-Path $output ('unsettled-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
80 $index++
81 }
82 $hierarchy = ''
83 $app.GetHierarchy($notebookId, 4, [ref]$hierarchy, 1)
84 [IO.File]::WriteAllText((Join-Path $output 'unsettled-hierarchy.xml'), $hierarchy, [Text.Encoding]::UTF8)
85 throw "Expected $ExpectedPages stable pages; OneNote returned $($pages.Count), settled=$settled."
86 }
87 $index = 0
88 $payloads = @()
89 foreach ($id in @($pages.Keys | Sort-Object)) {
90 [IO.File]::WriteAllText((Join-Path $output ('page-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
91 if ($Pdf) {
92 $pdfPath = Join-Path $output ('page-{0:d3}.pdf' -f $index)
93 $app.NavigateTo($id, '', $false)
94 $app.Publish($id, $pdfPath, 3, '')
95 if (-not (Test-Path $pdfPath) -or (Get-Item $pdfPath).Length -eq 0) { throw 'OneNote did not publish the page PDF.' }
96 }
97 [xml]$page = $pages[$id]
98 foreach ($file in $page.SelectNodes('//*[local-name()="InsertedFile" or local-name()="MediaFile"]')) {
99 $bytes = [IO.File]::ReadAllBytes($file.GetAttribute('pathCache'))
100 $hash = [BitConverter]::ToString([Security.Cryptography.SHA256]::Create().ComputeHash($bytes)).Replace('-', '').ToLowerInvariant()
101 [IO.File]::WriteAllBytes((Join-Path $output ($hash + '.attachment')), $bytes)
102 $payloads += @{ page = $id; object = $file.ParentNode.GetAttribute('objectID');
103 kind = $file.LocalName; name = $file.GetAttribute('preferredName');
104 sha256 = $hash; bytes = $bytes.Length }
105 }
106 $index++
107 }
108 [IO.File]::WriteAllText((Join-Path $output 'payloads.json'), (ConvertTo-Json -InputObject $payloads -Depth 4), [Text.Encoding]::UTF8)
109 $all = ''
110 $app.GetHierarchy($notebookId, 4, [ref]$all, 1)
111 [xml]$finalTree = $all
112 $finalIds = @($finalTree.SelectNodes('//*[local-name()="Page"]') | ForEach-Object { $_.GetAttribute('ID') } | Sort-Object -Unique)
113 if ($finalIds.Count -ne $pages.Count -or @($finalIds | Where-Object { -not $pages.ContainsKey($_) }).Count -ne 0) {
114 throw 'The notebook hierarchy changed while collecting page evidence; repeat the cold read.'
115 }
116 [IO.File]::WriteAllText((Join-Path $output 'hierarchy.xml'), $all, [Text.Encoding]::UTF8)
117 Write-Output "Read $($sections.Count) sections and $($pages.Count) pages."
118} catch {
119 $failure = $_
120 [IO.File]::WriteAllText((Join-Path $output 'failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
121 throw
122} finally {
123 try {
124 try {
125 if ($notebookId -and $CloneHost) { $app.SyncHierarchy($notebookId) }
126 if ($notebookId -and -not $KeepOpen -and (-not $UseCurrentCache -or $CloneHost)) {
127 $app.CloseNotebook($notebookId, $false)
128 }
129 } catch {
130 if ($null -eq $failure) { throw }
131 [IO.File]::WriteAllText((Join-Path $output 'cleanup-failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
132 }
133 } finally {
134 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
135 }
136 $app = $null
137 [GC]::Collect()
138 [GC]::WaitForPendingFinalizers()
139 if (-not $UseCurrentCache -and -not $CloneHost) {
140 Get-Process ONENOTE -ErrorAction SilentlyContinue | Wait-Process -Timeout 10
141 }
142}
corpus/conflict-page/cold-delete/source.json created+14
...@@ -0,0 +1,14 @@
1[
2 {
3 "path": "Open Notebook.onetoc2",
4 "bytes": 4944,
5 "sha256": "36274ad5cc077c6b1ad51ee4c4b90d3b6798a45ce446cbfd7c9d7ac504776b28",
6 "mtime_ns": 1790473161246592184
7 },
8 {
9 "path": "synthetic.one",
10 "bytes": 41728,
11 "sha256": "78e247a18c1fe2c8d4de66dd8bf79bf918636bfb42933a5533a91c8f21939de0",
12 "mtime_ns": 1790473248313861507
13 }
14]
corpus/conflict-page/cold-delete/teardown.json created+1
...@@ -0,0 +1 @@
1{"absent": true}
corpus/conflict-page/cold/before-read/af1af4b8987109ce1e335e538910fb3860b7c57eb3b6110a06aa2114e06d69a7.attachment created+1
...@@ -0,0 +1 @@
1Fictitious attachment for native corpus.
\ No newline at end of file
corpus/conflict-page/cold/before-read/environment.json created+7
...@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-FA8F1E62",
5 "cold": true,
6 "onenote": "14.0.4763.1000"
7}
corpus/conflict-page/cold/before-read/hierarchy.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{7CB92080-B95E-49CC-89A9-E352D88FAECF}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-27T06:13:14.000Z" color="#9595AA"><one:Section name="synthetic" ID="{4BA2718A-BD46-084F-151A-CF885CE1A6EE}{1}{B0}" path="C:\one-tests\runs\capture\notebook\synthetic.one" lastModifiedTime="2026-09-27T06:13:14.000Z" color="#8AA8E4"><one:Page ID="{67EB820C-348D-4B5E-927D-6719A0D9BAF1}{1}{B0}" name="Remote: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1"/></one:Section></one:Notebook>
corpus/conflict-page/cold/before-read/page-000.xml created+26
...@@ -0,0 +1,26 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{67EB820C-348D-4B5E-927D-6719A0D9BAF1}{1}{B0}" name="Remote: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2025-09-22T07:33:20.000Z" objectID="{00272AB0-7FF8-48E8-9D8F-EBC25F455F90}{10}{B0}"><one:Position x="36.0" y="14.40000057220459" z="0"/><one:Size width="127.5136947631836" height="14.30000114440918"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:55.000Z" lastModifiedTime="2025-09-22T07:33:20.000Z" objectID="{00272AB0-7FF8-48E8-9D8F-EBC25F455F90}{11}{B0}" alignment="left" quickStyleIndex="0" style="font-size:11.0pt;color:#1F4E79"><one:T><![CDATA[<span
3style='font-weight:bold;font-family:Calibri' lang=en-US>Remote: café, </span><span
4style='font-weight:bold;font-family:SimSun' lang=en-US>東京</span><span
5style='font-weight:bold;font-family:Calibri' lang=en-US>, </span><span
6style='font-weight:bold;font-family:Arial;direction:rtl;unicode-bidi:embed'
7lang=ar-SA>مرحبا</span>]]></one:T></one:OE></one:OEChildren></one:Outline><one:InkDrawing lastModifiedTime="2026-09-05T05:11:40.000Z" objectID="{22CA0622-2710-465B-8017-494A9FC53244}{15}{B0}"><one:Position x="421.5118103027344" y="84.75590515136718" z="5"/><one:Size width="60.00945281982422" height="60.00944519042969"/><one:Data>AK4FHQSiAaIBAWAqABofi0JwRYR1gL7QRdkE+oeVMvLsF0G4wBa/8yv0et67E4vlXrNInJuhJVq3
8URK/7HHMzCKxQYkOuxMs7nqhWM9UiuaXxU+PBvi60uGbIsRmbacNslhJmh1OQDQGbBADBkgQRSNG
9IwUIC2VPZmdoZGkZFDIIAIAeAiHi4kEzCADwEAKLxeJBZAMVRgBACpUE/QGH81d3mq7mkzgM0mcB
10mk1gEBgEBgEBgE1mUAgMzmUAmsygE1mUzmkBmc0gMzmk1mUAmsyTOaTWZQCazKZgALSmiazKATWZ
11QCAwCAwBAZpM4DAJlNYBMprAJlNZpM4DNJmmU1mkzgM0mcBmkzmU1gEyAAFpTSAzOaQGZzRAYBAY
12BAYBMprAIDNJnAZpM0yms0mcBmkzgM0mcymsAmU1gEyms0maZAACZTSawCZTWATKawCAoBAYBAZn
13NIDAJrMoBNZlAJrMpnMoBNZlAJrMpnNIDM5pAZnNJrMkzmk1AACH8C7XgWnmkzgM0msAmU1gEBgE
14BgEBTOaQGATWZQCazKATWZTOaTWZTNNJrMpmmUzTSaplM00mqZTOaTVMpmmk1TKATKazSawCAzSZ
15wGAQFAIDAJrMoBAZnNIDM5pAZnMpnNJrMpmmk1mUzTSaplM00mqZTNNJrMpmmk1TKZplhuAzKATW
16ZQCazKAIDAIDAIDNJnAYBMprAJlM4DNJnMprNJnMk1mkzmSazRM5kms0TOZJnMprNEzmSazQtKaQ
17GZzRNZlAIDAIDAIDAJlNYAmU1gEymsAmU1mkzmU1miZzKazRM5kmcyTWaJnMk1mkzmSazRM5kAUU
18RgBAAAAJBAUURgAAAAASABEgXhT+65lz/EiPi+8SFvjqCAk+gEcDAa/89Qd0AxVGAEAKABEg2R87
19QQrSFUA=</one:Data></one:InkDrawing><one:InkDrawing lastModifiedTime="2026-09-05T05:11:41.000Z" objectID="{22CA0622-2710-465B-8017-494A9FC53244}{23}{B0}"><one:Position x="504.0" y="92.23936462402343" z="7"/><one:Size width="30.01890563964844" height="45.04251861572266"/><one:Data>AOECHQJSegFgKgAaH4tCcEWEdYC+0EXZBPqHlTLy7BdBuMAWv/Mr9HreuxOL5V6zSJyboSVat1ES
20v+xxzMwisUGJDrsTLO56oVjPVIrml8VPjwb4utLhmyLEZm2nDbJYSZodTkA0BmwQAwZIEEUjRiMF
21CAtlT2ZnaGRpGRQyCACAHgIh4uJBMwgA8BACi8XiQWQDFUYAQArKAUuH88s3nkzmkzgM0mcBmkzg
22KAQGAQGATWZQCAzOaQGZzKATWZTOaQGZzSAzOaTWZQCazJM5pNZlAJrMpmAAAACH8D9XgeBJlM0z
23mRNZpNZpM4DAJlNYBAUAgMAgMzmkBgE1mUAmsyTOaTWZTOaTVMpnNJqmUzTSaplM00mqZTOZTNNJ
24qmUzTSagBRRGAIAAAAkEBRRGAAAAABIAESDPTQ4xwIZyTqQ8FVDbHxeKCj9AI5rARvPPUDugAxVG
25AEAKABEgLMlfQdkfI0A=</one:Data></one:InkDrawing><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-05T05:06:57.000Z" objectID="{5697F6A5-CCAD-42B5-8739-694AFB49BBD9}{10}{B0}"><one:Position x="144.0" y="96.0" z="1"/><one:Size width="167.0449523925781" height="13.42771339416504"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:57.000Z" lastModifiedTime="2026-09-05T05:06:57.000Z" objectID="{5697F6A5-CCAD-42B5-8739-694AFB49BBD9}{11}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Fictitious positioned outline.]]></one:T></one:OE></one:OEChildren></one:Outline><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-05T05:06:58.000Z" objectID="{6BBE9E2A-2A37-4B1B-B149-C2BE1724F357}{10}{B0}"><one:Position x="144.0" y="192.0" z="2"/><one:Size width="72.0" height="0.750005722045898"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:58.000Z" lastModifiedTime="2026-09-05T05:06:58.000Z" objectID="{6BBE9E2A-2A37-4B1B-B149-C2BE1724F357}{11}{B0}" alignment="left"><one:Image format="png" originalPageNumber="0"><one:Data>iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAusB9Y9J1uoA
26AAAASUVORK5CYII=</one:Data></one:Image></one:OE></one:OEChildren></one:Outline><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-05T05:06:59.000Z" objectID="{EEA4864A-EC4F-48DD-94E4-A54023F53C64}{10}{B0}"><one:Position x="264.0" y="192.0" z="3"/><one:Size width="72.00001525878906" height="63.0"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:59.000Z" lastModifiedTime="2026-09-05T05:06:59.000Z" objectID="{EEA4864A-EC4F-48DD-94E4-A54023F53C64}{11}{B0}" alignment="left"><one:InsertedFile pathCache="C:\Users\clover\AppData\Local\Microsoft\OneNote\14.0\OneNoteOfflineCache_Files\458001d8-0f1e-451f-81cb-621828fc0b39.txt" pathSource="C:\one-tests\runs\20260905-05\assets\fictitious-attachment.txt" preferredName="fictitious-attachment.txt"/></one:OE></one:OEChildren></one:Outline><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-05T05:07:00.000Z" objectID="{C07BFF1A-8DDF-435D-8BEC-47150BA6C722}{10}{B0}"><one:Position x="144.0" y="312.0" z="4"/><one:Size width="92.42181396484374" height="21.94773101806641"/><one:OEChildren><one:OE creationTime="2026-09-05T05:07:00.000Z" lastModifiedTime="2026-09-05T05:07:00.000Z" objectID="{C07BFF1A-8DDF-435D-8BEC-47150BA6C722}{11}{B0}" alignment="left"><one:Table bordersVisible="true" lastModifiedTime="2026-09-05T05:07:00.000Z" objectID="{C07BFF1A-8DDF-435D-8BEC-47150BA6C722}{12}{B0}"><one:Columns><one:Column index="0" width="39.14614105224609"/><one:Column index="1" width="45.14567184448242"/></one:Columns><one:Row objectID="{C07BFF1A-8DDF-435D-8BEC-47150BA6C722}{13}{B0}" lastModifiedTime="2026-09-05T05:07:00.000Z"><one:Cell lastModifiedTime="2026-09-05T05:07:00.000Z" objectID="{C07BFF1A-8DDF-435D-8BEC-47150BA6C722}{14}{B0}" lastModifiedByInitials="S"><one:OEChildren><one:OE creationTime="2026-09-05T05:07:00.000Z" lastModifiedTime="2026-09-05T05:07:00.000Z" objectID="{C07BFF1A-8DDF-435D-8BEC-47150BA6C722}{17}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Left cell]]></one:T></one:OE></one:OEChildren></one:Cell><one:Cell lastModifiedTime="2026-09-05T05:07:00.000Z" objectID="{C07BFF1A-8DDF-435D-8BEC-47150BA6C722}{22}{B0}" lastModifiedByInitials="S"><one:OEChildren><one:OE creationTime="2026-09-05T05:07:00.000Z" lastModifiedTime="2026-09-05T05:07:00.000Z" objectID="{C07BFF1A-8DDF-435D-8BEC-47150BA6C722}{25}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Right cell]]></one:T></one:OE></one:OEChildren></one:Cell></one:Row></one:Table></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/conflict-page/cold/before-read/payloads.json created+10
...@@ -0,0 +1,10 @@
1[
2 {
3 "sha256": "af1af4b8987109ce1e335e538910fb3860b7c57eb3b6110a06aa2114e06d69a7",
4 "name": "fictitious-attachment.txt",
5 "object": "{EEA4864A-EC4F-48DD-94E4-A54023F53C64}{11}{B0}",
6 "kind": "InsertedFile",
7 "page": "{67EB820C-348D-4B5E-927D-6719A0D9BAF1}{1}{B0}",
8 "bytes": 43
9 }
10]
\ No newline at end of file
corpus/conflict-page/cold/cold-conflict.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Snowbound: caf\u00e9, \u6771\u4eac, \u0645\u0631\u062d\u0628\u0627 - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/cold/cold-conflict.png created
Binary files /dev/null and b/corpus/conflict-page/cold/cold-conflict.png differ
corpus/conflict-page/cold/cold-delete.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Remote: caf\u00e9, \u6771\u4eac, \u0645\u0631\u062d\u0628\u0627 - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/cold/cold-delete.png created
Binary files /dev/null and b/corpus/conflict-page/cold/cold-delete.png differ
corpus/conflict-page/cold/cold-main.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Remote: caf\u00e9, \u6771\u4eac, \u0645\u0631\u062d\u0628\u0627 - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/cold/cold-main.png created
Binary files /dev/null and b/corpus/conflict-page/cold/cold-main.png differ
corpus/conflict-page/cold/cold-menu.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "",
10 "class": "Net UI Tool Window",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/cold/cold-menu.png created
Binary files /dev/null and b/corpus/conflict-page/cold/cold-menu.png differ
corpus/conflict-page/cold/cold-shown.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Snowbound: caf\u00e9, \u6771\u4eac, \u0645\u0631\u062d\u0628\u0627 - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/cold/cold-shown.png created
Binary files /dev/null and b/corpus/conflict-page/cold/cold-shown.png differ
corpus/conflict-page/cold/commands.jsonl created+6
...@@ -0,0 +1,6 @@
1{"command": "powershell -NoProfile -Command \"Expand-Archive -LiteralPath C:\\one-tests\\transfer.zip -DestinationPath C:\\one-tests\\runs\\capture\\notebook\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-FA8F1E62 -ExpectedPages -1 -KeepOpen", "exit": 0, "stdout": "Read 1 sections and 1 pages.\r\n", "stderr": "", "error": null}
3{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
4{"command": "powershell -NoProfile -Command \"Rename-Item C:\\one-tests\\runs\\capture\\read before-read\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
5{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-FA8F1E62 -UseCurrentCache -ExpectedPages -1", "exit": 0, "stdout": "Read 2 sections and 1 pages.\r\n", "stderr": "", "error": null}
6{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
corpus/conflict-page/cold/machine.json created+1
...@@ -0,0 +1 @@
1{"name": "m6-fa8f1e62", "hostname": "ONE-M6-FA8F1E62"}
corpus/conflict-page/cold/notebook/OneNote_RecycleBin/OneNote_DeletedPages.one created
Binary files /dev/null and b/corpus/conflict-page/cold/notebook/OneNote_RecycleBin/OneNote_DeletedPages.one differ
corpus/conflict-page/cold/notebook/OneNote_RecycleBin/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/cold/notebook/OneNote_RecycleBin/Open Notebook.onetoc2 differ
corpus/conflict-page/cold/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/cold/notebook/Open Notebook.onetoc2 differ
corpus/conflict-page/cold/notebook/synthetic.one created
Binary files /dev/null and b/corpus/conflict-page/cold/notebook/synthetic.one differ
corpus/conflict-page/cold/read/af1af4b8987109ce1e335e538910fb3860b7c57eb3b6110a06aa2114e06d69a7.attachment created+1
...@@ -0,0 +1 @@
1Fictitious attachment for native corpus.
\ No newline at end of file
corpus/conflict-page/cold/read/environment.json created+7
...@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-FA8F1E62",
5 "cold": false,
6 "onenote": "14.0.4763.1000"
7}
corpus/conflict-page/cold/read/hierarchy.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{7CB92080-B95E-49CC-89A9-E352D88FAECF}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-27T06:15:23.000Z" color="#9595AA" isCurrentlyViewed="true"><one:Section name="synthetic" ID="{4BA2718A-BD46-084F-151A-CF885CE1A6EE}{1}{B0}" path="C:\one-tests\runs\capture\notebook\synthetic.one" lastModifiedTime="2026-09-27T06:15:19.000Z" color="#8AA8E4" isCurrentlyViewed="true"><one:Page ID="{67EB820C-348D-4B5E-927D-6719A0D9BAF1}{1}{B0}" name="Remote: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1" isCurrentlyViewed="true"/></one:Section><one:SectionGroup name="OneNote_RecycleBin" ID="{62F4F7E5-1E54-4653-9B7D-BF1D873540AC}{1}{B0}" path="C:\one-tests\runs\capture\notebook\OneNote_RecycleBin" lastModifiedTime="2026-09-27T06:15:23.000Z" isRecycleBin="true"><one:Section name="Deleted Pages" ID="{C113CD25-3D86-450F-B646-6FD80F1E279B}{1}{B0}" path="C:\one-tests\runs\capture\notebook\OneNote_RecycleBin\OneNote_DeletedPages.one" lastModifiedTime="2026-09-27T06:15:19.000Z" color="#E1E1E1" isInRecycleBin="true" isDeletedPages="true"/></one:SectionGroup></one:Notebook>
corpus/conflict-page/cold/read/page-000.xml created+26
...@@ -0,0 +1,26 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{67EB820C-348D-4B5E-927D-6719A0D9BAF1}{1}{B0}" name="Remote: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1" isCurrentlyViewed="true" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2025-09-22T07:33:20.000Z" objectID="{00272AB0-7FF8-48E8-9D8F-EBC25F455F90}{10}{B0}"><one:Position x="36.0" y="14.40000057220459" z="0"/><one:Size width="127.5136947631836" height="14.30000114440918"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:55.000Z" lastModifiedTime="2025-09-22T07:33:20.000Z" objectID="{00272AB0-7FF8-48E8-9D8F-EBC25F455F90}{11}{B0}" alignment="left" quickStyleIndex="0" style="font-size:11.0pt;color:#1F4E79"><one:T><![CDATA[<span
3style='font-weight:bold;font-family:Calibri' lang=en-US>Remote: café, </span><span
4style='font-weight:bold;font-family:SimSun' lang=en-US>東京</span><span
5style='font-weight:bold;font-family:Calibri' lang=en-US>, </span><span
6style='font-weight:bold;font-family:Arial;direction:rtl;unicode-bidi:embed'
7lang=ar-SA>مرحبا</span>]]></one:T></one:OE></one:OEChildren></one:Outline><one:InkDrawing lastModifiedTime="2026-09-05T05:11:40.000Z" objectID="{22CA0622-2710-465B-8017-494A9FC53244}{15}{B0}"><one:Position x="421.5118103027344" y="84.75590515136718" z="5"/><one:Size width="60.00945281982422" height="60.00944519042969"/><one:Data>AK4FHQSiAaIBAWAqABofi0JwRYR1gL7QRdkE+oeVMvLsF0G4wBa/8yv0et67E4vlXrNInJuhJVq3
8URK/7HHMzCKxQYkOuxMs7nqhWM9UiuaXxU+PBvi60uGbIsRmbacNslhJmh1OQDQGbBADBkgQRSNG
9IwUIC2VPZmdoZGkZFDIIAIAeAiHi4kEzCADwEAKLxeJBZAMVRgBACpUE/QGH81d3mq7mkzgM0mcB
10mk1gEBgEBgEBgE1mUAgMzmUAmsygE1mUzmkBmc0gMzmk1mUAmsyTOaTWZQCazKZgALSmiazKATWZ
11QCAwCAwBAZpM4DAJlNYBMprAJlNZpM4DNJmmU1mkzgM0mcBmkzmU1gEyAAFpTSAzOaQGZzRAYBAY
12BAYBMprAIDNJnAZpM0yms0mcBmkzgM0mcymsAmU1gEyms0maZAACZTSawCZTWATKawCAoBAYBAZn
13NIDAJrMoBNZlAJrMpnMoBNZlAJrMpnNIDM5pAZnNJrMkzmk1AACH8C7XgWnmkzgM0msAmU1gEBgE
14BgEBTOaQGATWZQCazKATWZTOaTWZTNNJrMpmmUzTSaplM00mqZTOaTVMpmmk1TKATKazSawCAzSZ
15wGAQFAIDAJrMoBAZnNIDM5pAZnMpnNJrMpmmk1mUzTSaplM00mqZTNNJrMpmmk1TKZplhuAzKATW
16ZQCazKAIDAIDAIDNJnAYBMprAJlM4DNJnMprNJnMk1mkzmSazRM5kms0TOZJnMprNEzmSazQtKaQ
17GZzRNZlAIDAIDAIDAJlNYAmU1gEymsAmU1mkzmU1miZzKazRM5kmcyTWaJnMk1mkzmSazRM5kAUU
18RgBAAAAJBAUURgAAAAASABEgXhT+65lz/EiPi+8SFvjqCAk+gEcDAa/89Qd0AxVGAEAKABEg2R87
19QQrSFUA=</one:Data></one:InkDrawing><one:InkDrawing lastModifiedTime="2026-09-05T05:11:41.000Z" objectID="{22CA0622-2710-465B-8017-494A9FC53244}{23}{B0}"><one:Position x="504.0" y="92.23936462402343" z="7"/><one:Size width="30.01890563964844" height="45.04251861572266"/><one:Data>AOECHQJSegFgKgAaH4tCcEWEdYC+0EXZBPqHlTLy7BdBuMAWv/Mr9HreuxOL5V6zSJyboSVat1ES
20v+xxzMwisUGJDrsTLO56oVjPVIrml8VPjwb4utLhmyLEZm2nDbJYSZodTkA0BmwQAwZIEEUjRiMF
21CAtlT2ZnaGRpGRQyCACAHgIh4uJBMwgA8BACi8XiQWQDFUYAQArKAUuH88s3nkzmkzgM0mcBmkzg
22KAQGAQGATWZQCAzOaQGZzKATWZTOaQGZzSAzOaTWZQCazJM5pNZlAJrMpmAAAACH8D9XgeBJlM0z
23mRNZpNZpM4DAJlNYBAUAgMAgMzmkBgE1mUAmsyTOaTWZTOaTVMpnNJqmUzTSaplM00mqZTOZTNNJ
24qmUzTSagBRRGAIAAAAkEBRRGAAAAABIAESDPTQ4xwIZyTqQ8FVDbHxeKCj9AI5rARvPPUDugAxVG
25AEAKABEgLMlfQdkfI0A=</one:Data></one:InkDrawing><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-27T06:14:58.000Z" objectID="{5697F6A5-CCAD-42B5-8739-694AFB49BBD9}{10}{B0}"><one:Position x="144.0" y="96.0" z="1"/><one:Size width="167.0449523925781" height="13.42771339416504"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:57.000Z" lastModifiedTime="2026-09-27T06:14:58.000Z" objectID="{5697F6A5-CCAD-42B5-8739-694AFB49BBD9}{11}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Fictitious positioned outline.]]></one:T></one:OE></one:OEChildren></one:Outline><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-05T05:06:58.000Z" objectID="{6BBE9E2A-2A37-4B1B-B149-C2BE1724F357}{10}{B0}"><one:Position x="144.0" y="192.0" z="2"/><one:Size width="72.0" height="0.750005722045898"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:58.000Z" lastModifiedTime="2026-09-05T05:06:58.000Z" objectID="{6BBE9E2A-2A37-4B1B-B149-C2BE1724F357}{11}{B0}" alignment="left"><one:Image format="png" originalPageNumber="0"><one:Data>iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAusB9Y9J1uoA
26AAAASUVORK5CYII=</one:Data></one:Image></one:OE></one:OEChildren></one:Outline><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-05T05:06:59.000Z" objectID="{EEA4864A-EC4F-48DD-94E4-A54023F53C64}{10}{B0}"><one:Position x="264.0" y="192.0" z="3"/><one:Size width="72.00001525878906" height="63.0"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:59.000Z" lastModifiedTime="2026-09-05T05:06:59.000Z" objectID="{EEA4864A-EC4F-48DD-94E4-A54023F53C64}{11}{B0}" alignment="left"><one:InsertedFile pathCache="C:\Users\clover\AppData\Local\Microsoft\OneNote\14.0\OneNoteOfflineCache_Files\458001d8-0f1e-451f-81cb-621828fc0b39.txt" pathSource="C:\one-tests\runs\20260905-05\assets\fictitious-attachment.txt" preferredName="fictitious-attachment.txt"/></one:OE></one:OEChildren></one:Outline><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-27T06:14:58.000Z" objectID="{C07BFF1A-8DDF-435D-8BEC-47150BA6C722}{10}{B0}"><one:Position x="144.0" y="312.0" z="4"/><one:Size width="92.42181396484374" height="21.94773101806641"/><one:OEChildren><one:OE creationTime="2026-09-05T05:07:00.000Z" lastModifiedTime="2026-09-27T06:14:58.000Z" objectID="{C07BFF1A-8DDF-435D-8BEC-47150BA6C722}{11}{B0}" alignment="left"><one:Table bordersVisible="true" lastModifiedTime="2026-09-27T06:14:58.000Z" objectID="{C07BFF1A-8DDF-435D-8BEC-47150BA6C722}{12}{B0}"><one:Columns><one:Column index="0" width="39.14614105224609"/><one:Column index="1" width="45.14567184448242"/></one:Columns><one:Row objectID="{C07BFF1A-8DDF-435D-8BEC-47150BA6C722}{13}{B0}" lastModifiedTime="2026-09-27T06:14:58.000Z"><one:Cell lastModifiedTime="2026-09-27T06:14:58.000Z" objectID="{C07BFF1A-8DDF-435D-8BEC-47150BA6C722}{14}{B0}" lastModifiedByInitials="S"><one:OEChildren><one:OE creationTime="2026-09-05T05:07:00.000Z" lastModifiedTime="2026-09-27T06:14:58.000Z" objectID="{C07BFF1A-8DDF-435D-8BEC-47150BA6C722}{17}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Left cell]]></one:T></one:OE></one:OEChildren></one:Cell><one:Cell lastModifiedTime="2026-09-27T06:14:58.000Z" objectID="{C07BFF1A-8DDF-435D-8BEC-47150BA6C722}{22}{B0}" lastModifiedByInitials="S"><one:OEChildren><one:OE creationTime="2026-09-05T05:07:00.000Z" lastModifiedTime="2026-09-27T06:14:58.000Z" objectID="{C07BFF1A-8DDF-435D-8BEC-47150BA6C722}{25}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Right cell]]></one:T></one:OE></one:OEChildren></one:Cell></one:Row></one:Table></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/conflict-page/cold/read/payloads.json created+10
...@@ -0,0 +1,10 @@
1[
2 {
3 "sha256": "af1af4b8987109ce1e335e538910fb3860b7c57eb3b6110a06aa2114e06d69a7",
4 "name": "fictitious-attachment.txt",
5 "object": "{EEA4864A-EC4F-48DD-94E4-A54023F53C64}{11}{B0}",
6 "kind": "InsertedFile",
7 "page": "{67EB820C-348D-4B5E-927D-6719A0D9BAF1}{1}{B0}",
8 "bytes": 43
9 }
10]
\ No newline at end of file
corpus/conflict-page/cold/run.json created+18
...@@ -0,0 +1,18 @@
1{
2 "notebook": "/private/tmp/gate3/candidate",
3 "expected_pages": -1,
4 "author": null,
5 "author_timeout_seconds": 600,
6 "inspect": false,
7 "collect_notebook": true,
8 "base": {
9 "file": "win7-office-base.qcow2",
10 "format": "qcow2",
11 "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346",
12 "virtual_size": 68719476736
13 },
14 "scripts": {
15 "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331",
16 "read.ps1": "04013bfcccee40a17e2a225f9b9e96f40a3a8daad9e350609654f8eda3ccbb41"
17 }
18}
corpus/conflict-page/cold/scripts/cold.ps1 created+27
...@@ -0,0 +1,27 @@
1param([Parameter(Mandatory=$true)][string]$Root, [string]$CloneHost = '')
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
5if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
6 throw 'Choose a run directly below C:\one-tests\runs.'
7}
8if (Get-Process ONENOTE -ErrorAction SilentlyContinue) { throw 'Close OneNote before resetting its test cache.' }
9$key = 'HKCU:\Software\Microsoft\Office\14.0\OneNote'
10if ($CloneHost) {
11 if ($CloneHost -notmatch '^ONE-[A-Z0-9-]+$' -or [Environment]::MachineName -ne $CloneHost) {
12 throw 'The disposable clone hostname does not match this machine.'
13 }
14 New-Item "$key\Options\Paths" -Force | Out-Null
15 New-ItemProperty "$key\Options\Paths" -Name UnfiledNotesSection -PropertyType ExpandString -Value 'C:\one-tests\Loose.one' -Force | Out-Null
16} elseif ((Get-ItemProperty "$key\Options\Paths").UnfiledNotesSection -ne 'C:\one-tests\Loose.one' -or
17 -not (Test-Path 'C:\one-tests\profile-original-cache')) {
18 throw 'Park the personal OneNote profile before resetting the test cache.'
19}
20$cache = Join-Path $env:LOCALAPPDATA 'Microsoft\OneNote\14.0'
21$parked = Join-Path 'C:\one-tests\caches' ([IO.Path]::GetFileName($root))
22if (Test-Path $parked) { throw 'Choose a new run; its parked cache already exists.' }
23New-Item -ItemType Directory -Path 'C:\one-tests\caches' -Force | Out-Null
24if (Test-Path $cache) { Move-Item -LiteralPath $cache -Destination $parked }
25if (Test-Path "$key\OpenNotebooks") { Remove-Item "$key\OpenNotebooks" -Recurse }
26New-Item "$key\OpenNotebooks" | Out-Null
27New-ItemProperty "$key\OpenNotebooks" -Name '1' -PropertyType String -Value "$root\notebook" | Out-Null
corpus/conflict-page/cold/scripts/read.ps1 created+142
...@@ -0,0 +1,142 @@
1param(
2 [Parameter(Mandatory=$true)][string]$Root,
3 [int]$ExpectedPages = -1,
4 [switch]$UseCurrentCache,
5 [switch]$Pdf,
6 [switch]$KeepOpen,
7 [string]$CloneHost = ''
8)
9Set-StrictMode -Version Latest
10$ErrorActionPreference = 'Stop'
11$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
12if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
13 throw 'Choose a run directly below C:\one-tests\runs.'
14}
15$notebook = Join-Path $root 'notebook'
16$output = Join-Path $root 'read'
17if (Test-Path $output) { throw 'Choose a new read destination.' }
18if ($UseCurrentCache) {
19 if ((Get-ItemProperty 'HKCU:\Software\Microsoft\Office\14.0\OneNote\Options\Paths').UnfiledNotesSection -ne 'C:\one-tests\Loose.one') {
20 throw 'Park the personal OneNote profile before reading test notebooks.'
21 }
22} else {
23 & "$PSScriptRoot\cold-current.ps1" -Root $root -CloneHost $CloneHost
24}
25New-Item -ItemType Directory -Path $output | Out-Null
26$app = New-Object -ComObject OneNote.Application
27$notebookId = ''
28$failure = $null
29try {
30 $app.OpenHierarchy($notebook, '', [ref]$notebookId, 0)
31 $process = Get-Process ONENOTE
32 @{ hostname = [Environment]::MachineName; onenote = $process.MainModule.FileVersionInfo.FileVersion;
33 powershell = $PSVersionTable.PSVersion.ToString(); schema = 'xs2010'; cold = (-not $UseCurrentCache.IsPresent) } |
34 ConvertTo-Json | Set-Content (Join-Path $output 'environment.json') -Encoding UTF8
35 $sections = @()
36 foreach ($file in @(Get-ChildItem $notebook -Recurse | Where-Object { $_.Extension -eq '.one' })) {
37 $id = ''
38 $app.OpenHierarchy($file.FullName, '', [ref]$id, 0)
39 $sections += $id
40 }
41 $deadline = [DateTime]::UtcNow.AddSeconds(300)
42 $previous = ''
43 $lastChange = ''
44 $stableSince = [DateTime]::UtcNow
45 $settled = $false
46 do {
47 $pages = @{}
48 foreach ($section in $sections) {
49 $hierarchy = ''
50 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
51 [xml]$xml = $hierarchy
52 foreach ($node in $xml.SelectNodes('//*[@path]')) {
53 if (-not $node.GetAttribute('path').StartsWith("$notebook\", [StringComparison]::OrdinalIgnoreCase)) {
54 throw 'OneNote opened a section outside the copied notebook.'
55 }
56 }
57 foreach ($node in $xml.SelectNodes('//*[local-name()="Page"]')) {
58 $id = $node.GetAttribute('ID')
59 $content = ''
60 $app.GetPageContent($id, [ref]$content, 1, 1)
61 $pages[$id] = $content
62 }
63 }
64 $signature = [String]::Join('|', @($pages.Keys | Sort-Object | ForEach-Object { $_ + $pages[$_] }))
65 if ($signature -ne $previous) {
66 $lastChange = $previous
67 $previous = $signature
68 $stableSince = [DateTime]::UtcNow
69 }
70 if ((($ExpectedPages -ge 0 -and $pages.Count -eq $ExpectedPages) -or
71 ($ExpectedPages -lt 0 -and $pages.Count -gt 0)) -and
72 ([DateTime]::UtcNow - $stableSince).TotalSeconds -ge 2) { $settled = $true; break }
73 Start-Sleep -Milliseconds 250
74 } while ([DateTime]::UtcNow -lt $deadline)
75 if (-not $settled -or ($ExpectedPages -ge 0 -and $pages.Count -ne $ExpectedPages) -or ($ExpectedPages -lt 0 -and $pages.Count -eq 0)) {
76 [IO.File]::WriteAllText((Join-Path $output 'previous-signature.txt'), $lastChange, [Text.Encoding]::UTF8)
77 $index = 0
78 foreach ($id in @($pages.Keys | Sort-Object)) {
79 [IO.File]::WriteAllText((Join-Path $output ('unsettled-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
80 $index++
81 }
82 $hierarchy = ''
83 $app.GetHierarchy($notebookId, 4, [ref]$hierarchy, 1)
84 [IO.File]::WriteAllText((Join-Path $output 'unsettled-hierarchy.xml'), $hierarchy, [Text.Encoding]::UTF8)
85 throw "Expected $ExpectedPages stable pages; OneNote returned $($pages.Count), settled=$settled."
86 }
87 $index = 0
88 $payloads = @()
89 foreach ($id in @($pages.Keys | Sort-Object)) {
90 [IO.File]::WriteAllText((Join-Path $output ('page-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
91 if ($Pdf) {
92 $pdfPath = Join-Path $output ('page-{0:d3}.pdf' -f $index)
93 $app.NavigateTo($id, '', $false)
94 $app.Publish($id, $pdfPath, 3, '')
95 if (-not (Test-Path $pdfPath) -or (Get-Item $pdfPath).Length -eq 0) { throw 'OneNote did not publish the page PDF.' }
96 }
97 [xml]$page = $pages[$id]
98 foreach ($file in $page.SelectNodes('//*[local-name()="InsertedFile" or local-name()="MediaFile"]')) {
99 $bytes = [IO.File]::ReadAllBytes($file.GetAttribute('pathCache'))
100 $hash = [BitConverter]::ToString([Security.Cryptography.SHA256]::Create().ComputeHash($bytes)).Replace('-', '').ToLowerInvariant()
101 [IO.File]::WriteAllBytes((Join-Path $output ($hash + '.attachment')), $bytes)
102 $payloads += @{ page = $id; object = $file.ParentNode.GetAttribute('objectID');
103 kind = $file.LocalName; name = $file.GetAttribute('preferredName');
104 sha256 = $hash; bytes = $bytes.Length }
105 }
106 $index++
107 }
108 [IO.File]::WriteAllText((Join-Path $output 'payloads.json'), (ConvertTo-Json -InputObject $payloads -Depth 4), [Text.Encoding]::UTF8)
109 $all = ''
110 $app.GetHierarchy($notebookId, 4, [ref]$all, 1)
111 [xml]$finalTree = $all
112 $finalIds = @($finalTree.SelectNodes('//*[local-name()="Page"]') | ForEach-Object { $_.GetAttribute('ID') } | Sort-Object -Unique)
113 if ($finalIds.Count -ne $pages.Count -or @($finalIds | Where-Object { -not $pages.ContainsKey($_) }).Count -ne 0) {
114 throw 'The notebook hierarchy changed while collecting page evidence; repeat the cold read.'
115 }
116 [IO.File]::WriteAllText((Join-Path $output 'hierarchy.xml'), $all, [Text.Encoding]::UTF8)
117 Write-Output "Read $($sections.Count) sections and $($pages.Count) pages."
118} catch {
119 $failure = $_
120 [IO.File]::WriteAllText((Join-Path $output 'failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
121 throw
122} finally {
123 try {
124 try {
125 if ($notebookId -and $CloneHost) { $app.SyncHierarchy($notebookId) }
126 if ($notebookId -and -not $KeepOpen -and (-not $UseCurrentCache -or $CloneHost)) {
127 $app.CloseNotebook($notebookId, $false)
128 }
129 } catch {
130 if ($null -eq $failure) { throw }
131 [IO.File]::WriteAllText((Join-Path $output 'cleanup-failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
132 }
133 } finally {
134 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
135 }
136 $app = $null
137 [GC]::Collect()
138 [GC]::WaitForPendingFinalizers()
139 if (-not $UseCurrentCache -and -not $CloneHost) {
140 Get-Process ONENOTE -ErrorAction SilentlyContinue | Wait-Process -Timeout 10
141 }
142}
corpus/conflict-page/cold/source.json created+14
...@@ -0,0 +1,14 @@
1[
2 {
3 "path": "Open Notebook.onetoc2",
4 "bytes": 4944,
5 "sha256": "36274ad5cc077c6b1ad51ee4c4b90d3b6798a45ce446cbfd7c9d7ac504776b28",
6 "mtime_ns": 1788585164198018949
7 },
8 {
9 "path": "synthetic.one",
10 "bytes": 43112,
11 "sha256": "ab763188e885724957e2440a78eb29c457f47d538f78c3cc40e9c1a396be5c3b",
12 "mtime_ns": 1790489509247358867
13 }
14]
corpus/conflict-page/cold/teardown.json created+1
...@@ -0,0 +1 @@
1{"absent": true}
corpus/conflict-page/native-delete/before-read/af1af4b8987109ce1e335e538910fb3860b7c57eb3b6110a06aa2114e06d69a7.attachment created+1
...@@ -0,0 +1 @@
1Fictitious attachment for native corpus.
\ No newline at end of file
corpus/conflict-page/native-delete/before-read/environment.json created+7
...@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-7691E04E",
5 "cold": true,
6 "onenote": "14.0.4763.1000"
7}
corpus/conflict-page/native-delete/before-read/hierarchy.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{D55F46CA-9FD0-412D-8ED0-322FCEAD2763}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-27T01:30:09.000Z" color="#9595AA"><one:Section name="synthetic" ID="{BC6E92D1-2D86-0BD5-3D78-3FFC33523E50}{1}{B0}" path="C:\one-tests\runs\capture\notebook\synthetic.one" lastModifiedTime="2026-09-27T01:30:09.000Z" color="#8AA8E4"><one:Page ID="{90276157-A44D-48C4-BA1F-976DCF6A224F}{1}{B0}" name="Client B conflicting edit." dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1"/></one:Section></one:Notebook>
corpus/conflict-page/native-delete/before-read/page-000.xml created+21
...@@ -0,0 +1,21 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{90276157-A44D-48C4-BA1F-976DCF6A224F}{1}{B0}" name="Client B conflicting edit." dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="ONE-M6-D63654A2" authorInitials="4A2" lastModifiedBy="ONE-M6-D63654A2" lastModifiedByInitials="4A2" lastModifiedTime="2026-09-27T01:00:05.000Z" objectID="{F7EBC9EB-EF38-4B72-B5ED-1BB630F6C72E}{10}{B0}"><one:Position x="36.0" y="14.40000057220459" z="1"/><one:Size width="127.5136947631836" height="13.42771625518799"/><one:OEChildren><one:OE creationTime="2026-09-27T01:00:05.000Z" lastModifiedTime="2026-09-27T01:00:05.000Z" objectID="{F7EBC9EB-EF38-4B72-B5ED-1BB630F6C72E}{11}{B0}" alignment="left" quickStyleIndex="0" style="font-family:Calibri;font-size:11.0pt;color:#1F4E79"><one:T><![CDATA[Client B conflicting edit.]]></one:T></one:OE></one:OEChildren></one:Outline><one:InkDrawing lastModifiedTime="2026-09-05T05:11:40.000Z" objectID="{D506E579-B7D0-45C1-A875-B93EF076AAFA}{15}{B0}"><one:Position x="421.5118103027344" y="84.75590515136718" z="5"/><one:Size width="60.00945281982422" height="60.00944519042969"/><one:Data>AK4FHQSiAaIBAWAqABofi0JwRYR1gL7QRdkE+oeVMvLsF0G4wBa/8yv0et67E4vlXrNInJuhJVq3
3URK/7HHMzCKxQYkOuxMs7nqhWM9UiuaXxU+PBvi60uGbIsRmbacNslhJmh1OQDQGbBADBkgQRSNG
4IwUIC2VPZmdoZGkZFDIIAIAeAiHi4kEzCADwEAKLxeJBZAMVRgBACpUE/QGH81d3mq7mkzgM0mcB
5mk1gEBgEBgEBgE1mUAgMzmUAmsygE1mUzmkBmc0gMzmk1mUAmsyTOaTWZQCazKZgALSmiazKATWZ
6QCAwCAwBAZpM4DAJlNYBMprAJlNZpM4DNJmmU1mkzgM0mcBmkzmU1gEyAAFpTSAzOaQGZzRAYBAY
7BAYBMprAIDNJnAZpM0yms0mcBmkzgM0mcymsAmU1gEyms0maZAACZTSawCZTWATKawCAoBAYBAZn
8NIDAJrMoBNZlAJrMpnMoBNZlAJrMpnNIDM5pAZnNJrMkzmk1AACH8C7XgWnmkzgM0msAmU1gEBgE
9BgEBTOaQGATWZQCazKATWZTOaTWZTNNJrMpmmUzTSaplM00mqZTOaTVMpmmk1TKATKazSawCAzSZ
10wGAQFAIDAJrMoBAZnNIDM5pAZnMpnNJrMpmmk1mUzTSaplM00mqZTNNJrMpmmk1TKZplhuAzKATW
11ZQCazKAIDAIDAIDNJnAYBMprAJlM4DNJnMprNJnMk1mkzmSazRM5kms0TOZJnMprNEzmSazQtKaQ
12GZzRNZlAIDAIDAIDAJlNYAmU1gEymsAmU1mkzmU1miZzKazRM5kmcyTWaJnMk1mkzmSazRM5kAUU
13RgBAAAAJBAUURgAAAAASABEgXhT+65lz/EiPi+8SFvjqCAk+gEcDAa/89Qd0AxVGAEAKABEg2R87
14QQrSFUA=</one:Data></one:InkDrawing><one:InkDrawing lastModifiedTime="2026-09-05T05:11:41.000Z" objectID="{D506E579-B7D0-45C1-A875-B93EF076AAFA}{23}{B0}"><one:Position x="504.0" y="92.23936462402343" z="6"/><one:Size width="30.01890563964844" height="45.04251861572266"/><one:Data>AOECHQJSegFgKgAaH4tCcEWEdYC+0EXZBPqHlTLy7BdBuMAWv/Mr9HreuxOL5V6zSJyboSVat1ES
15v+xxzMwisUGJDrsTLO56oVjPVIrml8VPjwb4utLhmyLEZm2nDbJYSZodTkA0BmwQAwZIEEUjRiMF
16CAtlT2ZnaGRpGRQyCACAHgIh4uJBMwgA8BACi8XiQWQDFUYAQArKAUuH88s3nkzmkzgM0mcBmkzg
17KAQGAQGATWZQCAzOaQGZzKATWZTOaQGZzSAzOaTWZQCazJM5pNZlAJrMpmAAAACH8D9XgeBJlM0z
18mRNZpNZpM4DAJlNYBAUAgMAgMzmkBgE1mUAmsyTOaTWZTOaTVMpnNJqmUzTSaplM00mqZTOZTNNJ
19qmUzTSagBRRGAIAAAAkEBRRGAAAAABIAESDPTQ4xwIZyTqQ8FVDbHxeKCj9AI5rARvPPUDugAxVG
20AEAKABEgLMlfQdkfI0A=</one:Data></one:InkDrawing><one:Outline author="ONE-M6-8A93780A" authorInitials="80A" lastModifiedBy="ONE-M6-8A93780A" lastModifiedByInitials="80A" lastModifiedTime="2026-09-27T00:59:59.000Z" objectID="{A15B15FE-5C6D-412F-AF5B-993E94FA2367}{10}{B0}"><one:Position x="144.0" y="96.0" z="2"/><one:Size width="132.69140625" height="13.42771339416504"/><one:OEChildren><one:OE creationTime="2026-09-27T00:59:59.000Z" lastModifiedTime="2026-09-27T00:59:59.000Z" objectID="{A15B15FE-5C6D-412F-AF5B-993E94FA2367}{11}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Client A disjoint edit.]]></one:T></one:OE></one:OEChildren></one:Outline><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-05T05:06:58.000Z" objectID="{9C727D71-BAF7-4881-992B-32CA78976BE9}{10}{B0}"><one:Position x="144.0" y="192.0" z="3"/><one:Size width="72.0" height="0.750005722045898"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:58.000Z" lastModifiedTime="2026-09-05T05:06:58.000Z" objectID="{9C727D71-BAF7-4881-992B-32CA78976BE9}{11}{B0}" alignment="left"><one:Image format="png" originalPageNumber="0"><one:Data>iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAusB9Y9J1uoA
21AAAASUVORK5CYII=</one:Data></one:Image></one:OE></one:OEChildren></one:Outline><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-05T05:06:59.000Z" objectID="{19686511-7C8F-4B47-BC86-55344C46A4DA}{10}{B0}"><one:Position x="264.0" y="192.0" z="4"/><one:Size width="72.00001525878906" height="63.0"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:59.000Z" lastModifiedTime="2026-09-05T05:06:59.000Z" objectID="{19686511-7C8F-4B47-BC86-55344C46A4DA}{11}{B0}" alignment="left"><one:InsertedFile pathCache="C:\Users\clover\AppData\Local\Microsoft\OneNote\14.0\OneNoteOfflineCache_Files\458001d8-0f1e-451f-81cb-621828fc0b39.txt" pathSource="C:\one-tests\runs\20260905-05\assets\fictitious-attachment.txt" preferredName="fictitious-attachment.txt"/></one:OE></one:OEChildren></one:Outline><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-27T01:00:54.000Z" objectID="{37B71C41-1D1F-40C7-A38E-B76164155F9C}{10}{B0}"><one:Position x="144.0" y="312.0" z="0"/><one:Size width="92.42181396484374" height="48.80315780639648"/><one:OEChildren><one:OE creationTime="2026-09-27T01:00:05.000Z" lastModifiedTime="2026-09-27T01:00:54.000Z" objectID="{CF34E4C3-5A4B-4362-AE36-CCAD58670232}{27}{B0}" alignment="left"><one:Table bordersVisible="true" lastModifiedTime="2026-09-27T01:00:54.000Z" objectID="{CF34E4C3-5A4B-4362-AE36-CCAD58670232}{28}{B0}"><one:Columns><one:Column index="0" width="45.947021484375"/><one:Column index="1" width="38.34479141235352"/></one:Columns><one:Row objectID="{CF34E4C3-5A4B-4362-AE36-CCAD58670232}{29}{B0}" lastModifiedTime="2026-09-27T01:00:54.000Z"><one:Cell lastModifiedTime="2026-09-27T01:00:54.000Z" objectID="{CF34E4C3-5A4B-4362-AE36-CCAD58670232}{30}{B0}" lastModifiedByInitials="4A2"><one:OEChildren><one:OE creationTime="2026-09-27T01:00:05.000Z" lastModifiedTime="2026-09-27T01:00:05.000Z" objectID="{37B71C41-1D1F-40C7-A38E-B76164155F9C}{17}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Client B disjoint cell.]]></one:T></one:OE></one:OEChildren></one:Cell><one:Cell lastModifiedTime="2026-09-27T01:00:54.000Z" objectID="{CF34E4C3-5A4B-4362-AE36-CCAD58670232}{37}{B0}" lastModifiedByInitials="4A2"><one:OEChildren><one:OE creationTime="2026-09-27T01:00:05.000Z" lastModifiedTime="2026-09-27T01:00:05.000Z" objectID="{37B71C41-1D1F-40C7-A38E-B76164155F9C}{25}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Right cell]]></one:T></one:OE></one:OEChildren></one:Cell></one:Row></one:Table></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/conflict-page/native-delete/before-read/payloads.json created+10
...@@ -0,0 +1,10 @@
1[
2 {
3 "sha256": "af1af4b8987109ce1e335e538910fb3860b7c57eb3b6110a06aa2114e06d69a7",
4 "name": "fictitious-attachment.txt",
5 "object": "{19686511-7C8F-4B47-BC86-55344C46A4DA}{11}{B0}",
6 "kind": "InsertedFile",
7 "page": "{90276157-A44D-48C4-BA1F-976DCF6A224F}{1}{B0}",
8 "bytes": 43
9 }
10]
\ No newline at end of file
corpus/conflict-page/native-delete/cold-conflict.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Client A conflicting edit. - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/native-delete/cold-conflict.png created
Binary files /dev/null and b/corpus/conflict-page/native-delete/cold-conflict.png differ
corpus/conflict-page/native-delete/cold-delete.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Client B conflicting edit. - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/native-delete/cold-delete.png created
Binary files /dev/null and b/corpus/conflict-page/native-delete/cold-delete.png differ
corpus/conflict-page/native-delete/cold-main.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Client B conflicting edit. - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/native-delete/cold-main.png created
Binary files /dev/null and b/corpus/conflict-page/native-delete/cold-main.png differ
corpus/conflict-page/native-delete/cold-menu.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "",
10 "class": "Net UI Tool Window",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/native-delete/cold-menu.png created
Binary files /dev/null and b/corpus/conflict-page/native-delete/cold-menu.png differ
corpus/conflict-page/native-delete/cold-shown.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Client A conflicting edit. - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/native-delete/cold-shown.png created
Binary files /dev/null and b/corpus/conflict-page/native-delete/cold-shown.png differ
corpus/conflict-page/native-delete/commands.jsonl created+6
...@@ -0,0 +1,6 @@
1{"command": "powershell -NoProfile -Command \"Expand-Archive -LiteralPath C:\\one-tests\\transfer.zip -DestinationPath C:\\one-tests\\runs\\capture\\notebook\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-7691E04E -ExpectedPages -1 -KeepOpen", "exit": 0, "stdout": "Read 1 sections and 1 pages.\r\n", "stderr": "", "error": null}
3{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
4{"command": "powershell -NoProfile -Command \"Rename-Item C:\\one-tests\\runs\\capture\\read before-read\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
5{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-7691E04E -UseCurrentCache -ExpectedPages -1", "exit": 0, "stdout": "Read 2 sections and 1 pages.\r\n", "stderr": "", "error": null}
6{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
corpus/conflict-page/native-delete/machine.json created+1
...@@ -0,0 +1 @@
1{"name": "m6-7691e04e", "hostname": "ONE-M6-7691E04E"}
corpus/conflict-page/native-delete/notebook/OneNote_RecycleBin/OneNote_DeletedPages.one created
Binary files /dev/null and b/corpus/conflict-page/native-delete/notebook/OneNote_RecycleBin/OneNote_DeletedPages.one differ
corpus/conflict-page/native-delete/notebook/OneNote_RecycleBin/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/native-delete/notebook/OneNote_RecycleBin/Open Notebook.onetoc2 differ
corpus/conflict-page/native-delete/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/native-delete/notebook/Open Notebook.onetoc2 differ
corpus/conflict-page/native-delete/notebook/synthetic.one created
Binary files /dev/null and b/corpus/conflict-page/native-delete/notebook/synthetic.one differ
corpus/conflict-page/native-delete/read/af1af4b8987109ce1e335e538910fb3860b7c57eb3b6110a06aa2114e06d69a7.attachment created+1
...@@ -0,0 +1 @@
1Fictitious attachment for native corpus.
\ No newline at end of file
corpus/conflict-page/native-delete/read/environment.json created+7
...@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-7691E04E",
5 "cold": false,
6 "onenote": "14.0.4763.1000"
7}
corpus/conflict-page/native-delete/read/hierarchy.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{D55F46CA-9FD0-412D-8ED0-322FCEAD2763}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-27T01:32:11.000Z" color="#9595AA" isCurrentlyViewed="true"><one:Section name="synthetic" ID="{BC6E92D1-2D86-0BD5-3D78-3FFC33523E50}{1}{B0}" path="C:\one-tests\runs\capture\notebook\synthetic.one" lastModifiedTime="2026-09-27T01:32:07.000Z" color="#8AA8E4" isCurrentlyViewed="true"><one:Page ID="{90276157-A44D-48C4-BA1F-976DCF6A224F}{1}{B0}" name="Client B conflicting edit." dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1" isCurrentlyViewed="true"/></one:Section><one:SectionGroup name="OneNote_RecycleBin" ID="{1FBDA5C2-882E-488F-98BC-34FC12A65A9E}{1}{B0}" path="C:\one-tests\runs\capture\notebook\OneNote_RecycleBin" lastModifiedTime="2026-09-27T01:32:11.000Z" isRecycleBin="true"><one:Section name="Deleted Pages" ID="{07CEB579-3338-4C03-8027-1BBBF7B2DF88}{1}{B0}" path="C:\one-tests\runs\capture\notebook\OneNote_RecycleBin\OneNote_DeletedPages.one" lastModifiedTime="2026-09-27T01:32:07.000Z" color="#E1E1E1" isInRecycleBin="true" isDeletedPages="true"/></one:SectionGroup></one:Notebook>
corpus/conflict-page/native-delete/read/page-000.xml created+21
...@@ -0,0 +1,21 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{90276157-A44D-48C4-BA1F-976DCF6A224F}{1}{B0}" name="Client B conflicting edit." dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1" isCurrentlyViewed="true" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="ONE-M6-D63654A2" authorInitials="4A2" lastModifiedBy="ONE-M6-D63654A2" lastModifiedByInitials="4A2" lastModifiedTime="2026-09-27T01:31:45.000Z" objectID="{F7EBC9EB-EF38-4B72-B5ED-1BB630F6C72E}{10}{B0}"><one:Position x="36.0" y="14.40000057220459" z="1"/><one:Size width="127.5136947631836" height="13.42771625518799"/><one:OEChildren><one:OE creationTime="2026-09-27T01:00:05.000Z" lastModifiedTime="2026-09-27T01:31:45.000Z" objectID="{F7EBC9EB-EF38-4B72-B5ED-1BB630F6C72E}{11}{B0}" alignment="left" quickStyleIndex="0" style="font-family:Calibri;font-size:11.0pt;color:#1F4E79"><one:T><![CDATA[Client B conflicting edit.]]></one:T></one:OE></one:OEChildren></one:Outline><one:InkDrawing lastModifiedTime="2026-09-05T05:11:40.000Z" objectID="{D506E579-B7D0-45C1-A875-B93EF076AAFA}{15}{B0}"><one:Position x="421.5118103027344" y="84.75590515136718" z="5"/><one:Size width="60.00945281982422" height="60.00944519042969"/><one:Data>AK4FHQSiAaIBAWAqABofi0JwRYR1gL7QRdkE+oeVMvLsF0G4wBa/8yv0et67E4vlXrNInJuhJVq3
3URK/7HHMzCKxQYkOuxMs7nqhWM9UiuaXxU+PBvi60uGbIsRmbacNslhJmh1OQDQGbBADBkgQRSNG
4IwUIC2VPZmdoZGkZFDIIAIAeAiHi4kEzCADwEAKLxeJBZAMVRgBACpUE/QGH81d3mq7mkzgM0mcB
5mk1gEBgEBgEBgE1mUAgMzmUAmsygE1mUzmkBmc0gMzmk1mUAmsyTOaTWZQCazKZgALSmiazKATWZ
6QCAwCAwBAZpM4DAJlNYBMprAJlNZpM4DNJmmU1mkzgM0mcBmkzmU1gEyAAFpTSAzOaQGZzRAYBAY
7BAYBMprAIDNJnAZpM0yms0mcBmkzgM0mcymsAmU1gEyms0maZAACZTSawCZTWATKawCAoBAYBAZn
8NIDAJrMoBNZlAJrMpnMoBNZlAJrMpnNIDM5pAZnNJrMkzmk1AACH8C7XgWnmkzgM0msAmU1gEBgE
9BgEBTOaQGATWZQCazKATWZTOaTWZTNNJrMpmmUzTSaplM00mqZTOaTVMpmmk1TKATKazSawCAzSZ
10wGAQFAIDAJrMoBAZnNIDM5pAZnMpnNJrMpmmk1mUzTSaplM00mqZTNNJrMpmmk1TKZplhuAzKATW
11ZQCazKAIDAIDAIDNJnAYBMprAJlM4DNJnMprNJnMk1mkzmSazRM5kms0TOZJnMprNEzmSazQtKaQ
12GZzRNZlAIDAIDAIDAJlNYAmU1gEymsAmU1mkzmU1miZzKazRM5kmcyTWaJnMk1mkzmSazRM5kAUU
13RgBAAAAJBAUURgAAAAASABEgXhT+65lz/EiPi+8SFvjqCAk+gEcDAa/89Qd0AxVGAEAKABEg2R87
14QQrSFUA=</one:Data></one:InkDrawing><one:InkDrawing lastModifiedTime="2026-09-05T05:11:41.000Z" objectID="{D506E579-B7D0-45C1-A875-B93EF076AAFA}{23}{B0}"><one:Position x="504.0" y="92.23936462402343" z="6"/><one:Size width="30.01890563964844" height="45.04251861572266"/><one:Data>AOECHQJSegFgKgAaH4tCcEWEdYC+0EXZBPqHlTLy7BdBuMAWv/Mr9HreuxOL5V6zSJyboSVat1ES
15v+xxzMwisUGJDrsTLO56oVjPVIrml8VPjwb4utLhmyLEZm2nDbJYSZodTkA0BmwQAwZIEEUjRiMF
16CAtlT2ZnaGRpGRQyCACAHgIh4uJBMwgA8BACi8XiQWQDFUYAQArKAUuH88s3nkzmkzgM0mcBmkzg
17KAQGAQGATWZQCAzOaQGZzKATWZTOaQGZzSAzOaTWZQCazJM5pNZlAJrMpmAAAACH8D9XgeBJlM0z
18mRNZpNZpM4DAJlNYBAUAgMAgMzmkBgE1mUAmsyTOaTWZTOaTVMpnNJqmUzTSaplM00mqZTOZTNNJ
19qmUzTSagBRRGAIAAAAkEBRRGAAAAABIAESDPTQ4xwIZyTqQ8FVDbHxeKCj9AI5rARvPPUDugAxVG
20AEAKABEgLMlfQdkfI0A=</one:Data></one:InkDrawing><one:Outline author="ONE-M6-8A93780A" authorInitials="80A" lastModifiedBy="ONE-M6-8A93780A" lastModifiedByInitials="80A" lastModifiedTime="2026-09-27T01:31:45.000Z" objectID="{A15B15FE-5C6D-412F-AF5B-993E94FA2367}{10}{B0}"><one:Position x="144.0" y="96.0" z="2"/><one:Size width="132.69140625" height="13.42771339416504"/><one:OEChildren><one:OE creationTime="2026-09-27T00:59:59.000Z" lastModifiedTime="2026-09-27T01:31:45.000Z" objectID="{A15B15FE-5C6D-412F-AF5B-993E94FA2367}{11}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Client A disjoint edit.]]></one:T></one:OE></one:OEChildren></one:Outline><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-05T05:06:58.000Z" objectID="{9C727D71-BAF7-4881-992B-32CA78976BE9}{10}{B0}"><one:Position x="144.0" y="192.0" z="3"/><one:Size width="72.0" height="0.750005722045898"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:58.000Z" lastModifiedTime="2026-09-05T05:06:58.000Z" objectID="{9C727D71-BAF7-4881-992B-32CA78976BE9}{11}{B0}" alignment="left"><one:Image format="png" originalPageNumber="0"><one:Data>iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAusB9Y9J1uoA
21AAAASUVORK5CYII=</one:Data></one:Image></one:OE></one:OEChildren></one:Outline><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-05T05:06:59.000Z" objectID="{19686511-7C8F-4B47-BC86-55344C46A4DA}{10}{B0}"><one:Position x="264.0" y="192.0" z="4"/><one:Size width="72.00001525878906" height="63.0"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:59.000Z" lastModifiedTime="2026-09-05T05:06:59.000Z" objectID="{19686511-7C8F-4B47-BC86-55344C46A4DA}{11}{B0}" alignment="left"><one:InsertedFile pathCache="C:\Users\clover\AppData\Local\Microsoft\OneNote\14.0\OneNoteOfflineCache_Files\458001d8-0f1e-451f-81cb-621828fc0b39.txt" pathSource="C:\one-tests\runs\20260905-05\assets\fictitious-attachment.txt" preferredName="fictitious-attachment.txt"/></one:OE></one:OEChildren></one:Outline><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-27T01:31:45.000Z" objectID="{37B71C41-1D1F-40C7-A38E-B76164155F9C}{10}{B0}"><one:Position x="144.0" y="312.0" z="0"/><one:Size width="92.42181396484374" height="48.80315780639648"/><one:OEChildren><one:OE creationTime="2026-09-27T01:00:05.000Z" lastModifiedTime="2026-09-27T01:31:45.000Z" objectID="{CF34E4C3-5A4B-4362-AE36-CCAD58670232}{27}{B0}" alignment="left"><one:Table bordersVisible="true" lastModifiedTime="2026-09-27T01:31:45.000Z" objectID="{CF34E4C3-5A4B-4362-AE36-CCAD58670232}{28}{B0}"><one:Columns><one:Column index="0" width="45.947021484375"/><one:Column index="1" width="38.34479141235352"/></one:Columns><one:Row objectID="{CF34E4C3-5A4B-4362-AE36-CCAD58670232}{29}{B0}" lastModifiedTime="2026-09-27T01:31:45.000Z"><one:Cell lastModifiedTime="2026-09-27T01:31:45.000Z" objectID="{CF34E4C3-5A4B-4362-AE36-CCAD58670232}{30}{B0}" lastModifiedByInitials="4A2"><one:OEChildren><one:OE creationTime="2026-09-27T01:00:05.000Z" lastModifiedTime="2026-09-27T01:31:45.000Z" objectID="{37B71C41-1D1F-40C7-A38E-B76164155F9C}{17}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Client B disjoint cell.]]></one:T></one:OE></one:OEChildren></one:Cell><one:Cell lastModifiedTime="2026-09-27T01:31:45.000Z" objectID="{CF34E4C3-5A4B-4362-AE36-CCAD58670232}{37}{B0}" lastModifiedByInitials="4A2"><one:OEChildren><one:OE creationTime="2026-09-27T01:00:05.000Z" lastModifiedTime="2026-09-27T01:31:45.000Z" objectID="{37B71C41-1D1F-40C7-A38E-B76164155F9C}{25}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Right cell]]></one:T></one:OE></one:OEChildren></one:Cell></one:Row></one:Table></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/conflict-page/native-delete/read/payloads.json created+10
...@@ -0,0 +1,10 @@
1[
2 {
3 "sha256": "af1af4b8987109ce1e335e538910fb3860b7c57eb3b6110a06aa2114e06d69a7",
4 "name": "fictitious-attachment.txt",
5 "object": "{19686511-7C8F-4B47-BC86-55344C46A4DA}{11}{B0}",
6 "kind": "InsertedFile",
7 "page": "{90276157-A44D-48C4-BA1F-976DCF6A224F}{1}{B0}",
8 "bytes": 43
9 }
10]
\ No newline at end of file
corpus/conflict-page/native-delete/run.json created+18
...@@ -0,0 +1,18 @@
1{
2 "notebook": "/private/tmp/gate/native-conflict",
3 "expected_pages": -1,
4 "author": null,
5 "author_timeout_seconds": 600,
6 "inspect": false,
7 "collect_notebook": true,
8 "base": {
9 "file": "win7-office-base.qcow2",
10 "format": "qcow2",
11 "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346",
12 "virtual_size": 68719476736
13 },
14 "scripts": {
15 "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331",
16 "read.ps1": "04013bfcccee40a17e2a225f9b9e96f40a3a8daad9e350609654f8eda3ccbb41"
17 }
18}
corpus/conflict-page/native-delete/scripts/cold.ps1 created+27
...@@ -0,0 +1,27 @@
1param([Parameter(Mandatory=$true)][string]$Root, [string]$CloneHost = '')
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
5if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
6 throw 'Choose a run directly below C:\one-tests\runs.'
7}
8if (Get-Process ONENOTE -ErrorAction SilentlyContinue) { throw 'Close OneNote before resetting its test cache.' }
9$key = 'HKCU:\Software\Microsoft\Office\14.0\OneNote'
10if ($CloneHost) {
11 if ($CloneHost -notmatch '^ONE-[A-Z0-9-]+$' -or [Environment]::MachineName -ne $CloneHost) {
12 throw 'The disposable clone hostname does not match this machine.'
13 }
14 New-Item "$key\Options\Paths" -Force | Out-Null
15 New-ItemProperty "$key\Options\Paths" -Name UnfiledNotesSection -PropertyType ExpandString -Value 'C:\one-tests\Loose.one' -Force | Out-Null
16} elseif ((Get-ItemProperty "$key\Options\Paths").UnfiledNotesSection -ne 'C:\one-tests\Loose.one' -or
17 -not (Test-Path 'C:\one-tests\profile-original-cache')) {
18 throw 'Park the personal OneNote profile before resetting the test cache.'
19}
20$cache = Join-Path $env:LOCALAPPDATA 'Microsoft\OneNote\14.0'
21$parked = Join-Path 'C:\one-tests\caches' ([IO.Path]::GetFileName($root))
22if (Test-Path $parked) { throw 'Choose a new run; its parked cache already exists.' }
23New-Item -ItemType Directory -Path 'C:\one-tests\caches' -Force | Out-Null
24if (Test-Path $cache) { Move-Item -LiteralPath $cache -Destination $parked }
25if (Test-Path "$key\OpenNotebooks") { Remove-Item "$key\OpenNotebooks" -Recurse }
26New-Item "$key\OpenNotebooks" | Out-Null
27New-ItemProperty "$key\OpenNotebooks" -Name '1' -PropertyType String -Value "$root\notebook" | Out-Null
corpus/conflict-page/native-delete/scripts/read.ps1 created+142
...@@ -0,0 +1,142 @@
1param(
2 [Parameter(Mandatory=$true)][string]$Root,
3 [int]$ExpectedPages = -1,
4 [switch]$UseCurrentCache,
5 [switch]$Pdf,
6 [switch]$KeepOpen,
7 [string]$CloneHost = ''
8)
9Set-StrictMode -Version Latest
10$ErrorActionPreference = 'Stop'
11$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
12if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
13 throw 'Choose a run directly below C:\one-tests\runs.'
14}
15$notebook = Join-Path $root 'notebook'
16$output = Join-Path $root 'read'
17if (Test-Path $output) { throw 'Choose a new read destination.' }
18if ($UseCurrentCache) {
19 if ((Get-ItemProperty 'HKCU:\Software\Microsoft\Office\14.0\OneNote\Options\Paths').UnfiledNotesSection -ne 'C:\one-tests\Loose.one') {
20 throw 'Park the personal OneNote profile before reading test notebooks.'
21 }
22} else {
23 & "$PSScriptRoot\cold-current.ps1" -Root $root -CloneHost $CloneHost
24}
25New-Item -ItemType Directory -Path $output | Out-Null
26$app = New-Object -ComObject OneNote.Application
27$notebookId = ''
28$failure = $null
29try {
30 $app.OpenHierarchy($notebook, '', [ref]$notebookId, 0)
31 $process = Get-Process ONENOTE
32 @{ hostname = [Environment]::MachineName; onenote = $process.MainModule.FileVersionInfo.FileVersion;
33 powershell = $PSVersionTable.PSVersion.ToString(); schema = 'xs2010'; cold = (-not $UseCurrentCache.IsPresent) } |
34 ConvertTo-Json | Set-Content (Join-Path $output 'environment.json') -Encoding UTF8
35 $sections = @()
36 foreach ($file in @(Get-ChildItem $notebook -Recurse | Where-Object { $_.Extension -eq '.one' })) {
37 $id = ''
38 $app.OpenHierarchy($file.FullName, '', [ref]$id, 0)
39 $sections += $id
40 }
41 $deadline = [DateTime]::UtcNow.AddSeconds(300)
42 $previous = ''
43 $lastChange = ''
44 $stableSince = [DateTime]::UtcNow
45 $settled = $false
46 do {
47 $pages = @{}
48 foreach ($section in $sections) {
49 $hierarchy = ''
50 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
51 [xml]$xml = $hierarchy
52 foreach ($node in $xml.SelectNodes('//*[@path]')) {
53 if (-not $node.GetAttribute('path').StartsWith("$notebook\", [StringComparison]::OrdinalIgnoreCase)) {
54 throw 'OneNote opened a section outside the copied notebook.'
55 }
56 }
57 foreach ($node in $xml.SelectNodes('//*[local-name()="Page"]')) {
58 $id = $node.GetAttribute('ID')
59 $content = ''
60 $app.GetPageContent($id, [ref]$content, 1, 1)
61 $pages[$id] = $content
62 }
63 }
64 $signature = [String]::Join('|', @($pages.Keys | Sort-Object | ForEach-Object { $_ + $pages[$_] }))
65 if ($signature -ne $previous) {
66 $lastChange = $previous
67 $previous = $signature
68 $stableSince = [DateTime]::UtcNow
69 }
70 if ((($ExpectedPages -ge 0 -and $pages.Count -eq $ExpectedPages) -or
71 ($ExpectedPages -lt 0 -and $pages.Count -gt 0)) -and
72 ([DateTime]::UtcNow - $stableSince).TotalSeconds -ge 2) { $settled = $true; break }
73 Start-Sleep -Milliseconds 250
74 } while ([DateTime]::UtcNow -lt $deadline)
75 if (-not $settled -or ($ExpectedPages -ge 0 -and $pages.Count -ne $ExpectedPages) -or ($ExpectedPages -lt 0 -and $pages.Count -eq 0)) {
76 [IO.File]::WriteAllText((Join-Path $output 'previous-signature.txt'), $lastChange, [Text.Encoding]::UTF8)
77 $index = 0
78 foreach ($id in @($pages.Keys | Sort-Object)) {
79 [IO.File]::WriteAllText((Join-Path $output ('unsettled-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
80 $index++
81 }
82 $hierarchy = ''
83 $app.GetHierarchy($notebookId, 4, [ref]$hierarchy, 1)
84 [IO.File]::WriteAllText((Join-Path $output 'unsettled-hierarchy.xml'), $hierarchy, [Text.Encoding]::UTF8)
85 throw "Expected $ExpectedPages stable pages; OneNote returned $($pages.Count), settled=$settled."
86 }
87 $index = 0
88 $payloads = @()
89 foreach ($id in @($pages.Keys | Sort-Object)) {
90 [IO.File]::WriteAllText((Join-Path $output ('page-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
91 if ($Pdf) {
92 $pdfPath = Join-Path $output ('page-{0:d3}.pdf' -f $index)
93 $app.NavigateTo($id, '', $false)
94 $app.Publish($id, $pdfPath, 3, '')
95 if (-not (Test-Path $pdfPath) -or (Get-Item $pdfPath).Length -eq 0) { throw 'OneNote did not publish the page PDF.' }
96 }
97 [xml]$page = $pages[$id]
98 foreach ($file in $page.SelectNodes('//*[local-name()="InsertedFile" or local-name()="MediaFile"]')) {
99 $bytes = [IO.File]::ReadAllBytes($file.GetAttribute('pathCache'))
100 $hash = [BitConverter]::ToString([Security.Cryptography.SHA256]::Create().ComputeHash($bytes)).Replace('-', '').ToLowerInvariant()
101 [IO.File]::WriteAllBytes((Join-Path $output ($hash + '.attachment')), $bytes)
102 $payloads += @{ page = $id; object = $file.ParentNode.GetAttribute('objectID');
103 kind = $file.LocalName; name = $file.GetAttribute('preferredName');
104 sha256 = $hash; bytes = $bytes.Length }
105 }
106 $index++
107 }
108 [IO.File]::WriteAllText((Join-Path $output 'payloads.json'), (ConvertTo-Json -InputObject $payloads -Depth 4), [Text.Encoding]::UTF8)
109 $all = ''
110 $app.GetHierarchy($notebookId, 4, [ref]$all, 1)
111 [xml]$finalTree = $all
112 $finalIds = @($finalTree.SelectNodes('//*[local-name()="Page"]') | ForEach-Object { $_.GetAttribute('ID') } | Sort-Object -Unique)
113 if ($finalIds.Count -ne $pages.Count -or @($finalIds | Where-Object { -not $pages.ContainsKey($_) }).Count -ne 0) {
114 throw 'The notebook hierarchy changed while collecting page evidence; repeat the cold read.'
115 }
116 [IO.File]::WriteAllText((Join-Path $output 'hierarchy.xml'), $all, [Text.Encoding]::UTF8)
117 Write-Output "Read $($sections.Count) sections and $($pages.Count) pages."
118} catch {
119 $failure = $_
120 [IO.File]::WriteAllText((Join-Path $output 'failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
121 throw
122} finally {
123 try {
124 try {
125 if ($notebookId -and $CloneHost) { $app.SyncHierarchy($notebookId) }
126 if ($notebookId -and -not $KeepOpen -and (-not $UseCurrentCache -or $CloneHost)) {
127 $app.CloseNotebook($notebookId, $false)
128 }
129 } catch {
130 if ($null -eq $failure) { throw }
131 [IO.File]::WriteAllText((Join-Path $output 'cleanup-failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
132 }
133 } finally {
134 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
135 }
136 $app = $null
137 [GC]::Collect()
138 [GC]::WaitForPendingFinalizers()
139 if (-not $UseCurrentCache -and -not $CloneHost) {
140 Get-Process ONENOTE -ErrorAction SilentlyContinue | Wait-Process -Timeout 10
141 }
142}
corpus/conflict-page/native-delete/source.json created+14
...@@ -0,0 +1,14 @@
1[
2 {
3 "path": "Open Notebook.onetoc2",
4 "bytes": 4944,
5 "sha256": "36274ad5cc077c6b1ad51ee4c4b90d3b6798a45ce446cbfd7c9d7ac504776b28",
6 "mtime_ns": 1790472520423354873
7 },
8 {
9 "path": "synthetic.one",
10 "bytes": 40600,
11 "sha256": "e8d43dd57e02fe76e88c8ef45a7e18487bbf28928d0baf077288a4bb1415692c",
12 "mtime_ns": 1790472520423873581
13 }
14]
corpus/conflict-page/native-delete/teardown.json created+1
...@@ -0,0 +1 @@
1{"absent": true}
corpus/conflict-page/native-pages/a/a-merged.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Fictitious: caf\u00e9, \u6771\u4eac, \u0645\u0631\u062d\u0628\u0627 - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/native-pages/a/a-merged.png created
Binary files /dev/null and b/corpus/conflict-page/native-pages/a/a-merged.png differ
corpus/conflict-page/native-pages/a/command-0001.json created+1
...@@ -0,0 +1 @@
1{"action": "pages", "titles": ["One", "Two", "Three", "Four", "Target"]}
\ No newline at end of file
corpus/conflict-page/native-pages/a/command-0002.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-pages/a/command-0003.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-pages/a/command-0004.json created+1
...@@ -0,0 +1 @@
1{"action": "move", "page": "Four", "before": "One"}
\ No newline at end of file
corpus/conflict-page/native-pages/a/command-0005.json created+1
...@@ -0,0 +1 @@
1{"action": "move", "page": "Two", "before": null}
\ No newline at end of file
corpus/conflict-page/native-pages/a/command-0006.json created+1
...@@ -0,0 +1 @@
1{"action": "edit", "expected": "Body Target.", "text": "Body Target edited offline."}
\ No newline at end of file
corpus/conflict-page/native-pages/a/command-0007.json created+1
...@@ -0,0 +1 @@
1{"action": "snapshot"}
\ No newline at end of file
corpus/conflict-page/native-pages/a/command-0008.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-pages/a/command-0009.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-pages/a/command-0010.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-pages/a/command-0011.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-pages/a/command-0012.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-pages/a/command-0013.json created+1
...@@ -0,0 +1 @@
1{"action": "snapshot"}
\ No newline at end of file
corpus/conflict-page/native-pages/a/command-0014.json created+1
...@@ -0,0 +1 @@
1{"action": "close"}
\ No newline at end of file
corpus/conflict-page/native-pages/a/commands.jsonl created+16
...@@ -0,0 +1,16 @@
1{"command": "mkdir C:\\one-tests\\runs\\capture", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -ExecutionPolicy Bypass -File C:\\one-tests\\network.ps1 -LabMac 52:54:02:a8:e9:55", "exit": 0, "stdout": "{\"renewalResult\":0,\"releaseResult\":0,\"mac\":\"52:54:02:a8:e9:55\",\"address\":\"192.168.77.42\"}\r\n", "stderr": "", "error": null}
3{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0001.tmp C:\\one-tests\\runs\\capture\\inbox\\0001.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
4{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0002.tmp C:\\one-tests\\runs\\capture\\inbox\\0002.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
5{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0003.tmp C:\\one-tests\\runs\\capture\\inbox\\0003.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
6{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0004.tmp C:\\one-tests\\runs\\capture\\inbox\\0004.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
7{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0005.tmp C:\\one-tests\\runs\\capture\\inbox\\0005.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
8{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0006.tmp C:\\one-tests\\runs\\capture\\inbox\\0006.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
9{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0007.tmp C:\\one-tests\\runs\\capture\\inbox\\0007.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
10{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0008.tmp C:\\one-tests\\runs\\capture\\inbox\\0008.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
11{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0009.tmp C:\\one-tests\\runs\\capture\\inbox\\0009.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
12{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0010.tmp C:\\one-tests\\runs\\capture\\inbox\\0010.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
13{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0011.tmp C:\\one-tests\\runs\\capture\\inbox\\0011.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
14{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0012.tmp C:\\one-tests\\runs\\capture\\inbox\\0012.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
15{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0013.tmp C:\\one-tests\\runs\\capture\\inbox\\0013.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
16{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0014.tmp C:\\one-tests\\runs\\capture\\inbox\\0014.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
corpus/conflict-page/native-pages/a/hierarchy-0007.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Section xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="synthetic" ID="{63142E76-6879-0236-0E9B-223917E9FE4E}{1}{B0}" path="\\192.168.77.1\agent\m6-conflict\synthetic.one" lastModifiedTime="2026-09-27T09:22:37.000Z" color="#8AA8E4"><one:Page ID="{4F5DDDF0-E1B2-4127-89FC-8AA8EBD1E251}{1}{B0}" name="Fictitious: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1"/><one:Page ID="{FE831102-E2A2-47B2-A9AB-FA1A7ED19095}{1}{B0}" name="Four" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:20:19.000Z" pageLevel="1"/><one:Page ID="{5DBAC284-CA61-451B-B041-C2719A268F2A}{1}{B0}" name="One" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:20:19.000Z" pageLevel="1"/><one:Page ID="{540C9694-2DCE-4E68-8CCE-0B8D44BA9AAB}{1}{B0}" name="Three" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:20:19.000Z" pageLevel="1"/><one:Page ID="{4A9CC72B-5F8D-4666-BC1D-7E8CEF0D5351}{1}{B0}" name="Target" dateTime="2026-09-27T09:20:18.000Z" lastModifiedTime="2026-09-27T09:20:19.000Z" pageLevel="1"/><one:Page ID="{DB35495A-2D25-485D-B817-34DF84D8C5BF}{1}{B0}" name="Two" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:20:19.000Z" pageLevel="1"/></one:Section>
corpus/conflict-page/native-pages/a/hierarchy-0013.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Section xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="synthetic" ID="{63142E76-6879-0236-0E9B-223917E9FE4E}{1}{B0}" path="\\192.168.77.1\agent\m6-conflict\synthetic.one" lastModifiedTime="2026-09-27T09:22:59.000Z" color="#8AA8E4"><one:Page ID="{4F5DDDF0-E1B2-4127-89FC-8AA8EBD1E251}{1}{B0}" name="Fictitious: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1"/><one:Page ID="{5DBAC284-CA61-451B-B041-C2719A268F2A}{1}{B0}" name="One" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:20:19.000Z" pageLevel="1"/><one:Page ID="{DB35495A-2D25-485D-B817-34DF84D8C5BF}{1}{B0}" name="Two" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:20:19.000Z" pageLevel="1"/><one:Page ID="{FE831102-E2A2-47B2-A9AB-FA1A7ED19095}{1}{B0}" name="Four" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:20:19.000Z" pageLevel="1"/><one:Page ID="{540C9694-2DCE-4E68-8CCE-0B8D44BA9AAB}{1}{B0}" name="Three" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:20:19.000Z" pageLevel="1"/><one:Page ID="{B32E58B6-6903-44FC-A2CF-974EDF7A1EF2}{1}{B0}" name="Target" dateTime="2026-09-27T09:20:18.000Z" lastModifiedTime="2026-09-27T09:22:49.000Z" pageLevel="1"/></one:Section>
corpus/conflict-page/native-pages/a/listed.json created+1
...@@ -0,0 +1 @@
1["Fictitious: café, 東京, مرحبا", "One", "Two", "Four", "Three", "Target"]
\ No newline at end of file
corpus/conflict-page/native-pages/a/machine.json created+1
...@@ -0,0 +1 @@
1{"name": "m6-327953db", "hostname": "ONE-M6-327953DB"}
corpus/conflict-page/native-pages/a/teardown.json created+1
...@@ -0,0 +1 @@
1{"absent": true}
corpus/conflict-page/native-pages/b-published/notebook/OneNote_RecycleBin/OneNote_DeletedPages.one created
Binary files /dev/null and b/corpus/conflict-page/native-pages/b-published/notebook/OneNote_RecycleBin/OneNote_DeletedPages.one differ
corpus/conflict-page/native-pages/b-published/notebook/OneNote_RecycleBin/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/native-pages/b-published/notebook/OneNote_RecycleBin/Open Notebook.onetoc2 differ
corpus/conflict-page/native-pages/b-published/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/native-pages/b-published/notebook/Open Notebook.onetoc2 differ
corpus/conflict-page/native-pages/b-published/notebook/synthetic.one created
Binary files /dev/null and b/corpus/conflict-page/native-pages/b-published/notebook/synthetic.one differ
corpus/conflict-page/native-pages/b/b-merged.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Fictitious: caf\u00e9, \u6771\u4eac, \u0645\u0631\u062d\u0628\u0627 - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/native-pages/b/b-merged.png created
Binary files /dev/null and b/corpus/conflict-page/native-pages/b/b-merged.png differ
corpus/conflict-page/native-pages/b/command-0001.json created+1
...@@ -0,0 +1 @@
1{"action": "snapshot"}
\ No newline at end of file
corpus/conflict-page/native-pages/b/command-0002.json created+1
...@@ -0,0 +1 @@
1{"action": "move", "page": "Four", "before": null}
\ No newline at end of file
corpus/conflict-page/native-pages/b/command-0003.json created+1
...@@ -0,0 +1 @@
1{"action": "move", "page": "Three", "before": "One"}
\ No newline at end of file
corpus/conflict-page/native-pages/b/command-0004.json created+1
...@@ -0,0 +1 @@
1{"action": "delete-page", "page": "Target"}
\ No newline at end of file
corpus/conflict-page/native-pages/b/command-0005.json created+1
...@@ -0,0 +1 @@
1{"action": "snapshot"}
\ No newline at end of file
corpus/conflict-page/native-pages/b/command-0006.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-pages/b/command-0007.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-pages/b/command-0008.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-pages/b/command-0009.json created+1
...@@ -0,0 +1 @@
1{"action": "snapshot"}
\ No newline at end of file
corpus/conflict-page/native-pages/b/command-0010.json created+1
...@@ -0,0 +1 @@
1{"action": "close"}
\ No newline at end of file
corpus/conflict-page/native-pages/b/commands.jsonl created+12
...@@ -0,0 +1,12 @@
1{"command": "mkdir C:\\one-tests\\runs\\capture", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -ExecutionPolicy Bypass -File C:\\one-tests\\network.ps1 -LabMac 52:54:02:19:c7:6d", "exit": 0, "stdout": "{\"renewalResult\":0,\"releaseResult\":0,\"mac\":\"52:54:02:19:c7:6d\",\"address\":\"192.168.77.210\"}\r\n", "stderr": "", "error": null}
3{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0001.tmp C:\\one-tests\\runs\\capture\\inbox\\0001.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
4{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0002.tmp C:\\one-tests\\runs\\capture\\inbox\\0002.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
5{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0003.tmp C:\\one-tests\\runs\\capture\\inbox\\0003.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
6{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0004.tmp C:\\one-tests\\runs\\capture\\inbox\\0004.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
7{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0005.tmp C:\\one-tests\\runs\\capture\\inbox\\0005.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
8{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0006.tmp C:\\one-tests\\runs\\capture\\inbox\\0006.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
9{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0007.tmp C:\\one-tests\\runs\\capture\\inbox\\0007.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
10{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0008.tmp C:\\one-tests\\runs\\capture\\inbox\\0008.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
11{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0009.tmp C:\\one-tests\\runs\\capture\\inbox\\0009.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
12{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0010.tmp C:\\one-tests\\runs\\capture\\inbox\\0010.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
corpus/conflict-page/native-pages/b/hierarchy-0001.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Section xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="synthetic" ID="{D7EAB610-D407-0FFC-036F-02FDAAF1BCEF}{1}{B0}" path="\\192.168.77.1\agent\m6-conflict\synthetic.one" lastModifiedTime="2026-09-27T09:21:21.000Z" color="#8AA8E4" isUnread="true"><one:Page ID="{FBA34596-5DCC-4CED-8408-AA6C56C9A0F0}{1}{B0}" name="Fictitious: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1"/><one:Page ID="{E9445AE2-761F-48D1-BDB5-E2B5273ECD8B}{1}{B0}" name="One" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:21:21.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{6FCBD13C-915B-4597-B5E3-141B39C0871E}{1}{B0}" name="Two" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:21:21.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{E0F20EF2-91B0-43A2-813A-2B49F9A2D80A}{1}{B0}" name="Three" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:21:21.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{4A7D8964-5EDC-4A78-A45F-DADEC3C9D234}{1}{B0}" name="Four" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:21:21.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{FE625F4D-E3F3-4BAC-B1E9-5E48521511F0}{1}{B0}" name="Target" dateTime="2026-09-27T09:20:18.000Z" lastModifiedTime="2026-09-27T09:21:21.000Z" pageLevel="1" isUnread="true"/></one:Section>
corpus/conflict-page/native-pages/b/hierarchy-0005.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Section xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="synthetic" ID="{D7EAB610-D407-0FFC-036F-02FDAAF1BCEF}{1}{B0}" path="\\192.168.77.1\agent\m6-conflict\synthetic.one" lastModifiedTime="2026-09-27T09:22:44.000Z" color="#8AA8E4" isUnread="true"><one:Page ID="{FBA34596-5DCC-4CED-8408-AA6C56C9A0F0}{1}{B0}" name="Fictitious: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1"/><one:Page ID="{E0F20EF2-91B0-43A2-813A-2B49F9A2D80A}{1}{B0}" name="Three" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:21:21.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{E9445AE2-761F-48D1-BDB5-E2B5273ECD8B}{1}{B0}" name="One" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:21:21.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{6FCBD13C-915B-4597-B5E3-141B39C0871E}{1}{B0}" name="Two" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:21:21.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{4A7D8964-5EDC-4A78-A45F-DADEC3C9D234}{1}{B0}" name="Four" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:21:21.000Z" pageLevel="1" isUnread="true"/></one:Section>
corpus/conflict-page/native-pages/b/hierarchy-0009.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Section xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="synthetic" ID="{D7EAB610-D407-0FFC-036F-02FDAAF1BCEF}{1}{B0}" path="\\192.168.77.1\agent\m6-conflict\synthetic.one" lastModifiedTime="2026-09-27T09:23:22.000Z" color="#8AA8E4" isUnread="true"><one:Page ID="{FBA34596-5DCC-4CED-8408-AA6C56C9A0F0}{1}{B0}" name="Fictitious: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1"/><one:Page ID="{E9445AE2-761F-48D1-BDB5-E2B5273ECD8B}{1}{B0}" name="One" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:21:21.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{6FCBD13C-915B-4597-B5E3-141B39C0871E}{1}{B0}" name="Two" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:21:21.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{4A7D8964-5EDC-4A78-A45F-DADEC3C9D234}{1}{B0}" name="Four" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:21:21.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{E0F20EF2-91B0-43A2-813A-2B49F9A2D80A}{1}{B0}" name="Three" dateTime="2026-09-27T09:20:17.000Z" lastModifiedTime="2026-09-27T09:21:21.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{07D0C0D0-D57D-4936-AF3B-B78A62625C53}{1}{B0}" name="Target" dateTime="2026-09-27T09:20:18.000Z" lastModifiedTime="2026-09-27T09:23:22.000Z" pageLevel="1" isUnread="true"/></one:Section>
corpus/conflict-page/native-pages/b/listed.json created+1
...@@ -0,0 +1 @@
1["Fictitious: café, 東京, مرحبا", "One", "Two", "Four", "Three", "Target"]
\ No newline at end of file
corpus/conflict-page/native-pages/b/machine.json created+1
...@@ -0,0 +1 @@
1{"name": "m6-39da335d", "hostname": "ONE-M6-39DA335D"}
corpus/conflict-page/native-pages/b/teardown.json created+1
...@@ -0,0 +1 @@
1{"absent": true}
corpus/conflict-page/native-pages/clients.json created+4
...@@ -0,0 +1,4 @@
1{
2 "a": "m6-327953db",
3 "b": "m6-39da335d"
4}
\ No newline at end of file
corpus/conflict-page/native-pages/initial/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/native-pages/initial/notebook/Open Notebook.onetoc2 differ
corpus/conflict-page/native-pages/initial/notebook/synthetic.one created
Binary files /dev/null and b/corpus/conflict-page/native-pages/initial/notebook/synthetic.one differ
corpus/conflict-page/native-pages/linux-teardown.json created+1
...@@ -0,0 +1 @@
1{"absent": true}
corpus/conflict-page/native-pages/linux.json created+11
...@@ -0,0 +1,11 @@
1{
2 "cpus": 2,
3 "disk_gb": 16,
4 "hostname": "one-conflict2",
5 "lab_address": "192.168.77.1",
6 "lab_mac": "52:54:04:00:03:8c",
7 "memory_mb": 2048,
8 "samba_port": 14450,
9 "ssh_port": 22000,
10 "wan_mac": "52:54:03:c3:23:a2"
11}
\ No newline at end of file
corpus/conflict-page/native-pages/merged.json created+49
...@@ -0,0 +1,49 @@
1{
2 "order": [
3 "",
4 "One",
5 "Two",
6 "Four",
7 "Three",
8 "Target"
9 ],
10 "pages": [
11 [
12 "",
13 "Fictitious positioned outline.",
14 "Fictitious: café, 東京, مرحبا",
15 "Left cell",
16 "Right cell"
17 ],
18 [
19 "2:20 AM",
20 "Body One.",
21 "One",
22 "Sunday, September 27, 2026"
23 ],
24 [
25 "2:20 AM",
26 "Body Two.",
27 "Sunday, September 27, 2026",
28 "Two"
29 ],
30 [
31 "2:20 AM",
32 "Body Four.",
33 "Four",
34 "Sunday, September 27, 2026"
35 ],
36 [
37 "2:20 AM",
38 "Body Three.",
39 "Sunday, September 27, 2026",
40 "Three"
41 ],
42 [
43 "2:20 AM",
44 "Body Target edited offline.",
45 "Sunday, September 27, 2026",
46 "Target"
47 ]
48 ]
49}
\ No newline at end of file
corpus/conflict-page/native-pages/merged/notebook/OneNote_RecycleBin/OneNote_DeletedPages.one created
Binary files /dev/null and b/corpus/conflict-page/native-pages/merged/notebook/OneNote_RecycleBin/OneNote_DeletedPages.one differ
corpus/conflict-page/native-pages/merged/notebook/OneNote_RecycleBin/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/native-pages/merged/notebook/OneNote_RecycleBin/Open Notebook.onetoc2 differ
corpus/conflict-page/native-pages/merged/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/native-pages/merged/notebook/Open Notebook.onetoc2 differ
corpus/conflict-page/native-pages/merged/notebook/synthetic.one created
Binary files /dev/null and b/corpus/conflict-page/native-pages/merged/notebook/synthetic.one differ
corpus/conflict-page/native-pages/native_conflict.py created+362
...@@ -0,0 +1,362 @@
1#!/usr/bin/env python3
2"""OneNote 2010 conflict pages: how they arise, what they store and how OneNote shows them.
3
4`sync LINUX OUTPUT` runs two disposable OneNote clients on one Samba notebook: client A edits
5two paragraphs offline while client B edits one of them and a table cell online; A then
6reconnects, which makes OneNote write a conflict page. The server file is kept before and
7after, A and B are screenshotted with the information bar, the page list shown, the conflict
8page open and its menu open, and A then deletes the conflict page through that menu.
9
10`pages LINUX OUTPUT` runs the same two clients on page-list changes: client A, offline, moves
11a page that client B also moves, moves another page, and edits a page B deletes; A then
12reconnects. The server's page order and pages, with the recycle bin, are kept at each step.
13
14`restore LINUX OUTPUT` runs them on a page in the middle of the list: client A, offline, edits
15it and moves a page B leaves alone while B deletes it; A then reconnects.
16
17`cold NOTEBOOK OUTPUT [--delete]` cold-opens a notebook holding a conflict page in a fresh
18clone and screenshots the same views; `--delete` deletes the conflict page through the menu
19and captures the notebook OneNote leaves.
20"""
21import argparse
22from contextlib import ExitStack
23from concurrent.futures import ThreadPoolExecutor
24import base64
25import json
26from pathlib import Path
27import shutil
28import subprocess
29import sys
30import time
31import xml.etree.ElementTree as ET
32
33sys.path.insert(0, str(Path(__file__).resolve().parent))
34sys.path.insert(0, str(Path(__file__).resolve().parent / 'w7'))
35import native_runner as runner
36from native_runner import ROOT, clone, command, windows, vm
37import linux_vm
38
39FIXTURE = ROOT / 'corpus/native-ink/cold-ui-ink/notebook'
40SHARE = 'm6-conflict'
41ORIGINAL = 'Fictitious: café, 東京, مرحبا'
42OTHER_OUTLINE = 'Fictitious positioned outline.'
43CELL = 'Left cell'
44
45# Coordinates on OneNote 2010 maximized at 800x600, from the stage-5, m8 and sync-01 captures:
46# the information bar's text, and the first conflict page listed under its page.
47BAR = (332, 100)
48FIRST_CONFLICT = (725, 142)
49# Delete Conflict Page, first in the menu the bar opens on a conflict page.
50DELETE = (420, 113)
51
52
53def shot(name, output, label, script):
54 """Runs an AutoHotkey step on the clone, keeping its screenshot as `label.png`."""
55 (output / f'{label}.ahk').write_text(script)
56 result = windows.do_exec(script, target=name, timeout_ms=120000, shot_delay_ms=1500)
57 screenshot = result.pop('png_b64', None)
58 if screenshot:
59 (output / f'{label}.png').write_bytes(base64.b64decode(screenshot))
60 (output / f'{label}.json').write_text(json.dumps(result, indent=2))
61 if result.get('error') or result.get('exit') != 0:
62 raise RuntimeError(f'{label}: {result}')
63 return result
64
65
66def click(x, y):
67 return ('WinActivate("ahk_exe ONENOTE.EXE")\nCoordMode("Mouse", "Screen")\n'
68 f'Click({x}, {y})\nSleep(1200)\n')
69
70
71# Reports any dialog OneNote raised, so a confirmation shows in the step's record.
72DIALOG = ('if WinExist("ahk_class #32770 ahk_exe ONENOTE.EXE") {\n'
73 ' FileAppend("dialog: " WinGetTitle() " | " WinGetText() "`n", "*", "UTF-8")\n'
74 '}\n')
75
76
77def conflict_views(name, output, page, prefix, delete=False):
78 """Screenshots a page with a conflict page: the bar, the list shown, the conflict page and
79 its menu; `delete` then chooses Delete Conflict Page."""
80 shot(name, output, f'{prefix}-main', runner.navigation_script(page) + 'Sleep(2000)\n')
81 shot(name, output, f'{prefix}-shown', click(*BAR))
82 shot(name, output, f'{prefix}-conflict', click(*FIRST_CONFLICT))
83 shot(name, output, f'{prefix}-menu', click(*BAR))
84 if delete:
85 shot(name, output, f'{prefix}-delete', click(*DELETE) + DIALOG)
86 else:
87 shot(name, output, f'{prefix}-closed', 'WinActivate("ahk_exe ONENOTE.EXE")\nSend("{Esc}")\nSleep(800)\n')
88
89
90def cold(notebook, output, delete):
91 def interaction(name, output):
92 page = ET.parse(sorted((output / 'read').glob('page-*.xml'))[0]).getroot().attrib['ID']
93 conflict_views(name, output, page, 'cold', delete)
94 if delete:
95 # OneNote writes the deletion in the background; give it time before the re-read.
96 time.sleep(20)
97 runner.capture(notebook, output, collect_notebook=True, interaction=interaction)
98
99
100class Lab:
101 """Two OneNote clients on one Samba notebook: `action` runs a controller command,
102 `state` copies the share and exports its section, `settle` syncs until a check holds."""
103
104
105def page_list(model):
106 """The section's pages in order, each as its sorted texts."""
107 from document_model import ordered_pages, walk
108 return [sorted(node['kind']['text'] for _, node in walk(page, oid) if node['kind']['type'] == 'RichText')
109 for _, _, page, oid in ordered_pages(model)]
110
111
112def edits(lab):
113 from native_collaboration import reachable_page_text
114 a, b, output, action, settle = lab.a, lab.b, lab.output, lab.action, lab.settle
115 state = lambda label: reachable_page_text(lab.state(label))
116 texts = lambda state: {text for values in state.values() for text in values}
117 state('initial')
118 vm.qmp(a['name'], 'set_link', {'name': 'lab', 'up': False})
119 try:
120 action(a, 'edit', expected=ORIGINAL, text='Client A conflicting edit.')
121 action(a, 'edit', expected=OTHER_OUTLINE, text='Client A disjoint edit.')
122 action(a, 'snapshot')
123 action(b, 'edit', expected=ORIGINAL, text='Client B conflicting edit.')
124 action(b, 'edit', expected=CELL, text='Client B disjoint cell.')
125 settle(b, lambda label: {'Client B conflicting edit.', 'Client B disjoint cell.'} <= texts(state(label)), 'b-published')
126 finally:
127 vm.qmp(a['name'], 'set_link', {'name': 'lab', 'up': True})
128 settle(a, lambda label: len(found := state(label)) > 1 and 'Client A disjoint edit.' in texts(found), 'conflict')
129 (output / 'conflict.json').write_text(json.dumps(state('conflict'), indent=2, ensure_ascii=False))
130 page = action(a, 'snapshot')[0].attrib['ID']
131 conflict_views(a['name'], output / 'a', page, 'a')
132 action(b, 'sync')
133 time.sleep(5)
134 page_b = action(b, 'snapshot')[0].attrib['ID']
135 conflict_views(b['name'], output / 'b', page_b, 'b')
136 conflict_views(a['name'], output / 'a', page, 'a-again', delete=True)
137 settle(a, lambda label: len(state(label)) == 1, 'deleted')
138 action(b, 'sync')
139 time.sleep(5)
140 shot(b['name'], output / 'b', 'b-after-delete', runner.navigation_script(page_b) + 'Sleep(2000)\n')
141
142
143PAGES = ['One', 'Two', 'Three', 'Four', 'Target']
144
145
146def titled(model, titles):
147 """The section's pages in order, each named by the title among `titles` it holds."""
148 return [next((text for text in texts if text in titles), '') for texts in page_list(model)]
149
150
151def offline(lab, titles, a_actions, b_actions):
152 """Creates the titled pages; A runs its actions offline while B runs and publishes its
153 own; A then reconnects. Keeps the merged order and pages, and what each client lists."""
154 a, b, output, action = lab.a, lab.b, lab.output, lab.action
155 action(a, 'pages', titles=titles)
156 lab.settle(a, lambda label: 'Body Target.' in str(page_list(lab.state(label))), 'initial')
157 deadline = time.monotonic() + 300
158 while not any(node.get('name') == 'Target' for node in action(b, 'snapshot')):
159 action(b, 'sync')
160 if time.monotonic() > deadline: raise TimeoutError('B never listed the new pages')
161 vm.qmp(a['name'], 'set_link', {'name': 'lab', 'up': False})
162 try:
163 for kind, parameters in a_actions:
164 action(a, kind, **parameters)
165 action(a, 'snapshot')
166 for kind, parameters in b_actions:
167 action(b, kind, **parameters)
168 action(b, 'snapshot')
169 lab.settle(b, lambda label: 'Body Target.' not in str(page_list(lab.state(label))), 'b-published')
170 finally:
171 vm.qmp(a['name'], 'set_link', {'name': 'lab', 'up': True})
172 # A's changes reach the server when its sync writes; wait until the file stops changing.
173 previous, steady = None, 0
174 deadline = time.monotonic() + 300
175 while steady < 3 and time.monotonic() < deadline:
176 action(a, 'sync')
177 time.sleep(10)
178 current = page_list(lab.state('merged'))
179 steady = steady + 1 if current == previous else 0
180 previous = current
181 (output / 'merged.json').write_text(json.dumps({'order': titled(lab.state('merged'), titles), 'pages': previous}, indent=2, ensure_ascii=False))
182 for client, label in ((a, 'a'), (b, 'b')):
183 action(client, 'sync')
184 time.sleep(5)
185 listed = action(client, 'snapshot')
186 (output / label / 'listed.json').write_text(json.dumps([node.get('name') for node in listed], ensure_ascii=False))
187 if listed:
188 shot(client['name'], output / label, f'{label}-merged', runner.navigation_script(listed[0].get('ID')) + 'Sleep(2000)\n')
189
190
191def pages(lab):
192 """Competing page moves, and an offline edit of a page the other client deletes."""
193 offline(lab, PAGES, [
194 ('move', {'page': 'Four', 'before': 'One'}),
195 ('move', {'page': 'Two', 'before': None}),
196 ('edit', {'expected': 'Body Target.', 'text': 'Body Target edited offline.'}),
197 ], [
198 ('move', {'page': 'Four', 'before': None}),
199 ('move', {'page': 'Three', 'before': 'One'}),
200 ('delete-page', {'page': 'Target'}),
201 ])
202
203
204def restore(lab):
205 """An offline edit of a page in the middle of the list the other client deletes, with an
206 offline move of a page the other client leaves alone."""
207 offline(lab, ['One', 'Two', 'Target', 'Three', 'Four'], [
208 ('edit', {'expected': 'Body Target.', 'text': 'Body Target edited offline.'}),
209 ('move', {'page': 'Four', 'before': 'One'}),
210 ], [
211 ('delete-page', {'page': 'Target'}),
212 ])
213
214
215def sync(server, output, scenario):
216 from document_model import EXPORTER
217
218 if linux_vm.instance_path(server).exists():
219 raise ValueError('Choose a new Linux VM name; existing machines are not owned by this run.')
220 output = output.resolve()
221 output.mkdir(parents=True, exist_ok=False)
222 scripts = output / 'scripts'
223 scripts.mkdir()
224 for name in ('cold.ps1', 'collaborate.ps1', 'network.ps1', 'text.ps1', 'stress.ps1'):
225 shutil.copyfile(ROOT / 'tools/native' / name, scripts / name)
226 shutil.copyfile(Path(__file__), output / 'native_conflict.py')
227
228 def ssh(text, timeout=90):
229 result = linux_vm.run_ssh(server, text, timeout=timeout)
230 with (output / 'server.jsonl').open('a') as log:
231 log.write(json.dumps({'command': text, 'exit': result.returncode, 'stdout': result.stdout, 'stderr': result.stderr}) + '\n')
232 result.check_returncode()
233 return result.stdout
234
235 def server_state(label):
236 """Copies the server's notebook, recycle bin included, until its section parses,
237 keeping it as `label/notebook`; returns the section's exported model."""
238 deadline = time.monotonic() + 120
239 while True:
240 target = output / label
241 shutil.rmtree(target, ignore_errors=True)
242 (target / 'notebook').mkdir(parents=True)
243 archive = subprocess.run(linux_vm.ssh_argv(server, f"tar cf - -C '/srv/agent/{SHARE}' ."), capture_output=True, check=True, timeout=60).stdout
244 subprocess.run(['tar', 'xf', '-', '-C', target / 'notebook'], input=archive, check=True)
245 result = subprocess.run([EXPORTER, target / 'notebook/synthetic.one', target / 'model'], capture_output=True, text=True)
246 if result.returncode == 0:
247 return json.loads((target / 'model/document.json').read_text())
248 if time.monotonic() > deadline:
249 raise RuntimeError(f'{label}: {result.stderr}')
250 time.sleep(1)
251
252 linux_vm.create_instance(server)
253 try:
254 linux_vm.launch(server)
255 linux_vm.wait_instance(server, 600)
256 (output / 'linux.json').write_text(json.dumps(linux_vm.load_instance(server), indent=2))
257 ssh(f'mkdir /srv/agent/{SHARE}')
258 for name in ('synthetic.one', 'Open Notebook.onetoc2'):
259 with (FIXTURE / name).open('rb') as stream:
260 subprocess.run(linux_vm.ssh_argv(server, f"cat > '/srv/agent/{SHARE}/{name}'"), stdin=stream, check=True)
261 ssh(f'chmod u+w /srv/agent/{SHARE}/*')
262 with ExitStack() as stack:
263 labels = ['a', 'b']
264 for label in labels:
265 (output / label).mkdir()
266
267 def start(label):
268 manager = clone(output / label)
269 name = manager.__enter__()
270 return label, name, manager
271 with ThreadPoolExecutor(max_workers=2) as pool:
272 started = list(pool.map(start, labels))
273 for _, _, manager in started:
274 stack.push(manager)
275 clients = {}
276 for label, name, _ in started:
277 folder = output / label
278 for local in scripts.iterdir():
279 remote = 'cold-current.ps1' if local.name == 'cold.ps1' else local.name
280 result = windows.do_put(local, 'C:\\one-tests\\' + remote, name)
281 if result.get('error'): raise RuntimeError(result['error'])
282 command(name, 'mkdir C:\\one-tests\\runs\\capture', folder)
283 command(name, 'powershell -NoProfile -ExecutionPolicy Bypass -File C:\\one-tests\\network.ps1 -LabMac ' + vm.lab_mac(name), folder)
284 result = windows.do_spawn('cmd /c powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\collaborate.ps1 -Root C:\\one-tests\\runs\\capture -SharedPath \\\\192.168.77.1\\agent\\' + SHARE + ' -CloneHost ONE-' + name.upper() + ' > C:\\one-tests\\runs\\capture\\controller.log 2>&1', name)
285 if result.get('error'): raise RuntimeError(result['error'])
286 deadline = time.monotonic() + 180
287 while windows.do_cmd('if exist C:\\one-tests\\runs\\capture\\ready (echo ready)', target=name).get('stdout', '').find('ready') < 0:
288 if time.monotonic() > deadline: raise RuntimeError('The collaboration controller did not open the notebook')
289 time.sleep(1)
290 clients[label] = {'name': name, 'folder': folder, 'sequence': 0}
291 (output / 'clients.json').write_text(json.dumps({label: client['name'] for label, client in clients.items()}, indent=2))
292
293 def action(client, kind, **parameters):
294 client['sequence'] += 1
295 sequence = client['sequence']
296 local = client['folder'] / f'command-{sequence:04}.json'
297 local.write_text(json.dumps({'action': kind, **parameters}, ensure_ascii=False))
298 inbox = f'C:\\one-tests\\runs\\capture\\inbox\\{sequence:04}'
299 result = windows.do_put(local, inbox + '.tmp', client['name'])
300 if result.get('error'): raise RuntimeError(result['error'])
301 command(client['name'], f'move {inbox}.tmp {inbox}.json', client['folder'])
302 remote = f'C:\\one-tests\\runs\\capture\\outbox\\{sequence}'
303 deadline = time.monotonic() + 600
304 while True:
305 result = windows.do_cmd(f'if exist {remote}\\error (type {remote}\\error & exit /b 1) else (if exist {remote}\\done (echo complete))', target=client['name'])
306 if result.get('exit') != 0: raise RuntimeError(str(result))
307 if 'complete' in result.get('stdout', ''): break
308 if time.monotonic() > deadline: raise TimeoutError(f'{kind} did not complete')
309 time.sleep(.5)
310 if kind != 'snapshot':
311 return None
312 hierarchy = client['folder'] / f'hierarchy-{sequence:04}.xml'
313 windows.do_get(remote + '\\hierarchy.xml', hierarchy, client['name'])
314 pages = [node for node in ET.fromstring(hierarchy.read_text(encoding='utf-8-sig').strip()).iter() if node.tag.endswith('}Page')]
315 for i in range(len(pages)):
316 windows.do_get(remote + f'\\page-{i}.xml', client['folder'] / f'snapshot-{sequence:04}-{i}.xml', client['name'])
317 return pages
318
319 def settle(client, check, label):
320 deadline = time.monotonic() + 300
321 while True:
322 action(client, 'sync')
323 if check(label): return
324 if time.monotonic() > deadline: raise TimeoutError(label)
325 time.sleep(2)
326
327 lab = Lab()
328 lab.a, lab.b, lab.output = clients['a'], clients['b'], output
329 lab.action, lab.state, lab.settle = action, server_state, settle
330 scenario(lab)
331
332 for client in clients.values():
333 try:
334 action(client, 'close')
335 except Exception as error:
336 (client['folder'] / 'close-failure.txt').write_text(str(error))
337 server_state('final')
338 finally:
339 if linux_vm.running(server):
340 linux_vm.shutdown(server, 120)
341 linux_vm.delete_instance(server)
342 (output / 'linux-teardown.json').write_text(json.dumps({'absent': not linux_vm.instance_path(server).exists()}) + '\n')
343
344
345if __name__ == '__main__':
346 parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
347 commands = parser.add_subparsers(dest='mode', required=True)
348 scenarios = {'sync': edits, 'pages': pages, 'restore': restore}
349 for mode in scenarios:
350 lab = commands.add_parser(mode)
351 lab.add_argument('linux')
352 lab.add_argument('output', type=Path)
353 reopen = commands.add_parser('cold')
354 reopen.add_argument('notebook', type=Path)
355 reopen.add_argument('output', type=Path)
356 reopen.add_argument('--delete', action='store_true')
357 args = parser.parse_args()
358 if args.mode in scenarios:
359 sync(args.linux, args.output, scenarios[args.mode])
360 else:
361 cold(args.notebook, args.output, args.delete)
362 print('Captured', args.output)
corpus/conflict-page/native-pages/scripts/cold.ps1 created+27
...@@ -0,0 +1,27 @@
1param([Parameter(Mandatory=$true)][string]$Root, [string]$CloneHost = '')
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
5if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
6 throw 'Choose a run directly below C:\one-tests\runs.'
7}
8if (Get-Process ONENOTE -ErrorAction SilentlyContinue) { throw 'Close OneNote before resetting its test cache.' }
9$key = 'HKCU:\Software\Microsoft\Office\14.0\OneNote'
10if ($CloneHost) {
11 if ($CloneHost -notmatch '^ONE-[A-Z0-9-]+$' -or [Environment]::MachineName -ne $CloneHost) {
12 throw 'The disposable clone hostname does not match this machine.'
13 }
14 New-Item "$key\Options\Paths" -Force | Out-Null
15 New-ItemProperty "$key\Options\Paths" -Name UnfiledNotesSection -PropertyType ExpandString -Value 'C:\one-tests\Loose.one' -Force | Out-Null
16} elseif ((Get-ItemProperty "$key\Options\Paths").UnfiledNotesSection -ne 'C:\one-tests\Loose.one' -or
17 -not (Test-Path 'C:\one-tests\profile-original-cache')) {
18 throw 'Park the personal OneNote profile before resetting the test cache.'
19}
20$cache = Join-Path $env:LOCALAPPDATA 'Microsoft\OneNote\14.0'
21$parked = Join-Path 'C:\one-tests\caches' ([IO.Path]::GetFileName($root))
22if (Test-Path $parked) { throw 'Choose a new run; its parked cache already exists.' }
23New-Item -ItemType Directory -Path 'C:\one-tests\caches' -Force | Out-Null
24if (Test-Path $cache) { Move-Item -LiteralPath $cache -Destination $parked }
25if (Test-Path "$key\OpenNotebooks") { Remove-Item "$key\OpenNotebooks" -Recurse }
26New-Item "$key\OpenNotebooks" | Out-Null
27New-ItemProperty "$key\OpenNotebooks" -Name '1' -PropertyType String -Value "$root\notebook" | Out-Null
corpus/conflict-page/native-pages/scripts/collaborate.ps1 created+145
...@@ -0,0 +1,145 @@
1param([Parameter(Mandatory=$true)][string]$Root,
2 [Parameter(Mandatory=$true)][string]$SharedPath,
3 [Parameter(Mandatory=$true)][string]$CloneHost,
4 [switch]$ResumeCache,
5 [int]$StartSequence = 1)
6function Keep-Container($page, $target) {
7 $container = $target
8 while ($container.ParentNode -ne $page.DocumentElement) { $container = $container.ParentNode }
9 foreach ($sibling in @($page.DocumentElement.ChildNodes)) {
10 if ($sibling -ne $container -and $sibling.LocalName -in @('Outline', 'Title', 'Image', 'InkDrawing')) {
11 [void]$page.DocumentElement.RemoveChild($sibling)
12 }
13 }
14}
15Set-StrictMode -Version Latest
16$ErrorActionPreference = 'Stop'
17. "$PSScriptRoot\text.ps1"
18if ($SharedPath -notmatch '^\\\\192\.168\.77\.1\\agent\\m6-[a-z0-9-]+$') { throw 'Choose a disposable Linux lab notebook.' }
19if ($env:COMPUTERNAME -ne $CloneHost) { throw 'The collaboration controller requires its owned clone.' }
20if (-not $ResumeCache) { & "$PSScriptRoot\cold-current.ps1" -Root $Root -CloneHost $CloneHost }
21$shared = $SharedPath
22if (-not (Test-Path "$shared\synthetic.one")) { throw 'Create the shared fixture first.' }
23New-Item -ItemType Directory "$Root\inbox", "$Root\outbox" -Force | Out-Null
24$user = 'HKCU:\Software\Microsoft\Office\Common\UserInfo'
25New-Item $user -Force | Out-Null
26New-ItemProperty $user -Name UserName -Value $CloneHost -PropertyType String -Force | Out-Null
27New-ItemProperty $user -Name UserInitials -Value $CloneHost.Substring($CloneHost.Length - 3) -PropertyType String -Force | Out-Null
28$app = New-Object -ComObject OneNote.Application
29$notebook = ''
30try {
31 $app.OpenHierarchy($shared, '', [ref]$notebook, 0)
32 $section = ''
33 $app.OpenHierarchy('synthetic.one', $notebook, [ref]$section, 0)
34 [IO.File]::WriteAllText("$Root\ready", 'ready')
35 $deadline = [DateTime]::UtcNow.AddMinutes(30)
36 $sequence = $StartSequence
37 while ([DateTime]::UtcNow -lt $deadline) {
38 $name = '{0:d4}.json' -f $sequence
39 $input = "$Root\inbox\$name"
40 if (-not (Test-Path $input)) { Start-Sleep -Milliseconds 100; continue }
41 $command = Get-Content -Raw -Encoding UTF8 $input | ConvertFrom-Json
42 $output = "$Root\outbox\$sequence"
43 New-Item -ItemType Directory $output | Out-Null
44 try {
45 if ($command.action -in @('prepare-stress', 'stress')) {
46 . "$PSScriptRoot\stress.ps1"
47 Invoke-Stress $app $section $command $output $shared
48 }
49 elseif ($command.action -eq 'sync') { $app.SyncHierarchy($notebook) }
50 elseif ($command.action -in @('pages', 'move', 'delete-page')) {
51 $namespace = 'http://schemas.microsoft.com/office/onenote/2010/onenote'
52 $hierarchy = ''
53 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
54 [xml]$tree = $hierarchy
55 $listed = @($tree.SelectNodes('//*[local-name()="Page"]'))
56 $named = { param($title) @($listed | Where-Object { $_.GetAttribute('name') -eq $title }) }
57 if ($command.action -eq 'pages') {
58 # New titled pages with one body paragraph each, appended in order.
59 foreach ($title in $command.titles) {
60 $page = ''
61 $app.CreateNewPage($section, [ref]$page, 2)
62 $body = [Security.SecurityElement]::Escape('Body ' + $title + '.')
63 $name = [Security.SecurityElement]::Escape($title)
64 $xml = '<one:Page xmlns:one="' + $namespace + '" ID="' + $page + '"><one:Title><one:OE><one:T>' + $name + '</one:T></one:OE></one:Title><one:Outline><one:OEChildren><one:OE><one:T>' + $body + '</one:T></one:OE></one:OEChildren></one:Outline></one:Page>'
65 $app.UpdatePageContent($xml, [DateTime]::MinValue, 1, $false)
66 }
67 } elseif ($command.action -eq 'move') {
68 # Moves the page titled `page` before the one titled `before`, or last.
69 $moved = @(& $named $command.page)
70 if ($moved.Count -ne 1) { throw 'Expected one page to move.' }
71 $order = [Collections.ArrayList]@($listed | Where-Object { $_ -ne $moved[0] })
72 $at = $order.Count
73 if ($command.before) {
74 $anchor = @(& $named $command.before)
75 if ($anchor.Count -ne 1) { throw 'Expected one anchor page.' }
76 $at = $order.IndexOf($anchor[0])
77 }
78 $order.Insert($at, $moved[0])
79 $xml = '<one:Section xmlns:one="' + $namespace + '" ID="' + $section + '">'
80 foreach ($node in $order) {
81 $xml += '<one:Page ID="' + $node.GetAttribute('ID') + '" pageLevel="' + $node.GetAttribute('pageLevel') + '"/>'
82 }
83 $app.UpdateHierarchy($xml + '</one:Section>', 1)
84 } else {
85 $doomed = @(& $named $command.page)
86 if ($doomed.Count -ne 1) { throw 'Expected one page to delete.' }
87 $app.DeleteHierarchy($doomed[0].GetAttribute('ID'), [DateTime]::MinValue, $false)
88 }
89 }
90 elseif ($command.action -eq 'kill-process') {
91 Get-Process ONENOTE | Stop-Process -Force
92 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
93 $app = New-Object -ComObject OneNote.Application
94 $app.OpenHierarchy($shared, '', [ref]$notebook, 0)
95 $app.OpenHierarchy('synthetic.one', $notebook, [ref]$section, 0)
96 }
97 elseif ($command.action -eq 'edit' -or $command.action -eq 'snapshot') {
98 $hierarchy = ''
99 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
100 [IO.File]::WriteAllText("$output\hierarchy.xml", $hierarchy, [Text.Encoding]::UTF8)
101 [xml]$tree = $hierarchy
102 $matches = 0
103 $editedPage = $null
104 $i = 0
105 foreach ($node in $tree.SelectNodes('//*[local-name()="Page"]')) {
106 $content = ''
107 $app.GetPageContent($node.GetAttribute('ID'), [ref]$content, 1, 1)
108 [IO.File]::WriteAllText("$output\page-$i.xml", $content, [Text.Encoding]::UTF8)
109 [xml]$page = $content
110 if ($command.action -eq 'edit') {
111 $targets = @($page.SelectNodes('//*[local-name()="T"]') | Where-Object {
112 (Get-ParagraphText $_.InnerText) -eq $command.expected
113 })
114 if ($targets.Count -gt 1) { throw 'Expected one matching text object.' }
115 if ($targets.Count -eq 1) {
116 $matches++
117 if ($matches -gt 1) { throw 'The text occurs on multiple pages.' }
118 $targets[0].InnerText = [Security.SecurityElement]::Escape($command.text)
119 Keep-Container $page $targets[0]
120 $editedPage = $page
121 }
122 }
123 $i++
124 }
125 if ($command.action -eq 'edit' -and $matches -ne 1) { throw 'Expected one matching text object.' }
126 if ($editedPage) { $app.UpdatePageContent($editedPage.OuterXml, [DateTime]::MinValue, 1, $false) }
127 } elseif ($command.action -ne 'close') { throw 'Unknown collaboration command.' }
128 [IO.File]::WriteAllText("$output\done", 'ok')
129 } catch {
130 [IO.File]::WriteAllText("$output\error", $_.Exception.ToString())
131 }
132 if ($command.action -eq 'close') { break }
133 $sequence++
134 }
135} finally {
136 try { if ($notebook) { $app.CloseNotebook($notebook, $false) } }
137 finally {
138 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
139 $app = $null
140 [GC]::Collect()
141 [GC]::WaitForPendingFinalizers()
142 Get-Process ONENOTE -ErrorAction SilentlyContinue | Wait-Process -Timeout 10
143 [IO.File]::WriteAllText("$Root\closed", 'closed')
144 }
145}
corpus/conflict-page/native-pages/scripts/network.ps1 created+20
...@@ -0,0 +1,20 @@
1param([Parameter(Mandatory=$true)][string]$LabMac)
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$adapter = @(Get-WmiObject Win32_NetworkAdapterConfiguration | Where-Object { $_.MACAddress -eq $LabMac })
5if ($adapter.Count -ne 1) { throw 'The clone lab adapter is ambiguous.' }
6$enabled = $adapter[0].EnableDHCP()
7if ($enabled.ReturnValue -notin @(0, 1)) {
8 throw ('Unable to enable DHCP on the clone lab adapter; result ' + $enabled.ReturnValue)
9}
10$release = $adapter[0].ReleaseDHCPLease()
11$result = $adapter[0].RenewDHCPLease()
12$deadline = [DateTime]::UtcNow.AddSeconds(60)
13do {
14 $adapter = @(Get-WmiObject Win32_NetworkAdapterConfiguration | Where-Object { $_.MACAddress -eq $LabMac })
15 $addresses = @($adapter[0].IPAddress | Where-Object { $_ -match '^192\.168\.77\.\d+$' -and $_ -ne '192.168.77.1' })
16 if ($addresses.Count -eq 1) { break }
17 Start-Sleep -Milliseconds 250
18} while ([DateTime]::UtcNow -lt $deadline)
19if ($addresses.Count -ne 1) { throw ('The clone did not receive a lab IPv4 address; renewal result ' + $result.ReturnValue + '; adapter ' + ($adapter[0] | Select-Object MACAddress,DHCPEnabled,IPEnabled,IPAddress | ConvertTo-Json -Compress)) }
20@{mac=$LabMac;address=$addresses[0];releaseResult=$release.ReturnValue;renewalResult=$result.ReturnValue} | ConvertTo-Json -Compress
corpus/conflict-page/native-pages/scripts/stress.ps1 created+83
...@@ -0,0 +1,83 @@
1function Invoke-Stress($app, $section, $command, $output, $shared) {
2 $hierarchy = ''
3 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
4 [IO.File]::WriteAllText("$output\hierarchy.xml", $hierarchy, [Text.Encoding]::UTF8)
5 [xml]$tree = $hierarchy
6 $pages = @($tree.SelectNodes('//*[local-name()="Page"]'))
7 if ($pages.Count -ne 1) { throw 'Expected one stress-test page.' }
8 $pageId = $pages[0].GetAttribute('ID')
9 if ($command.action -eq 'prepare-stress') {
10 for ($i = 0; $i -lt $command.clients; $i++) {
11 $xml = '<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="' + $pageId + '"><one:Outline><one:Position x="48" y="' + (150 + 75 * $i) + '" z="' + ($i + 1) + '"/><one:OEChildren><one:OE><one:T><![CDATA[Native ' + $i + ':]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>'
12 $app.UpdatePageContent($xml, [DateTime]::MinValue, 1, $false)
13 }
14 return
15 }
16 $random = New-Object Random([int]$command.seed)
17 $events = New-Object IO.StreamWriter("$output\events.jsonl", $false, [Text.Encoding]::UTF8)
18 $events.AutoFlush = $true
19 try {
20 [IO.File]::WriteAllText("$output\ready", 'ready')
21 $deadline = [DateTime]::UtcNow.AddMinutes(10)
22 while (-not (Test-Path "$shared\stress-start")) {
23 if ([DateTime]::UtcNow -gt $deadline) { throw 'Stress start timed out.' }
24 Start-Sleep -Milliseconds 20
25 }
26 for ($i = 0; $i -lt $command.operations; $i++) {
27 if (($command.PSObject.Properties.Name -contains 'maintenance') -and $command.maintenance -and $i -eq [Math]::Floor($command.operations / 2)) {
28 [IO.File]::WriteAllText("$shared\maintenance-paused-n$($command.actor)", 'paused')
29 while (-not (Test-Path "$shared\maintenance-resume")) {
30 if ([DateTime]::UtcNow -gt $deadline) { throw 'Maintenance pause timed out.' }
31 Start-Sleep -Milliseconds 100
32 }
33 }
34 Start-Sleep -Milliseconds $random.Next(10, 100)
35 $started = [DateTime]::UtcNow.Ticks
36 $content = ''
37 $app.GetPageContent($pageId, [ref]$content, 1, 1)
38 [xml]$page = $content
39 $matches = @($page.SelectNodes('//*[local-name()="T"]') | Where-Object {
40 (Get-ParagraphText $_.InnerText).StartsWith($command.prefix)
41 })
42 if ($matches.Count -ne 1) { throw 'Expected one stress-test paragraph.' }
43 $before = Get-ParagraphText $matches[0].InnerText
44 $token = ' [n' + $command.actor + ':' + $i + ']'
45 $start = $end = $before.Length
46 $replacement = $token
47 $bold = $italic = $false
48 if ($command.edit) {
49 $boundaries = New-Object 'Collections.Generic.List[int]'
50 for ($at = $command.prefix.Length; $at -le $before.Length; $at++) {
51 $boundaries.Add($at)
52 if ($at -lt $before.Length -and [char]::IsHighSurrogate($before[$at])) { $at++ }
53 }
54 $first = $boundaries[$random.Next($boundaries.Count)]
55 $second = $boundaries[$random.Next($boundaries.Count)]
56 $start = [Math]::Min($first, $second)
57 $end = [Math]::Max($first, $second)
58 $replacement = ' caf' + [char]0xe9 + ' ' + [char]::ConvertFromUtf32(0x1f980) + $token
59 $bold = $random.Next(2) -eq 1
60 $italic = $random.Next(2) -eq 1
61 }
62 $after = $before.Substring(0, $start) + $replacement + $before.Substring($end)
63 $html = [Security.SecurityElement]::Escape($after)
64 if ($command.edit) {
65 $weight = if ($bold) { 'bold' } else { 'normal' }
66 $style = if ($italic) { 'italic' } else { 'normal' }
67 $html = '<span style="font-weight:' + $weight + ';font-style:' + $style + '">' + $html + '</span>'
68 }
69 $matches[0].InnerText = $html
70 Keep-Container $page $matches[0]
71 $updateStarted = [DateTime]::UtcNow.Ticks
72 $app.UpdatePageContent($page.OuterXml, [DateTime]::MinValue, 1, $false)
73 $updated = [DateTime]::UtcNow.Ticks
74 $events.WriteLine((@{ operation=$i; before=$before; token=$token; started_ticks=$started;
75 range=@($start, $end); replacement=$replacement; bold=$bold; italic=$italic;
76 update_started_ticks=$updateStarted; updated_ticks=$updated } | ConvertTo-Json -Compress))
77 if ($i -eq 0) { [IO.File]::WriteAllText("$output\editing", 'editing') }
78 if ($command.sync_every -gt 0 -and (($i + 1) % $command.sync_every) -eq 0) {
79 $app.SyncHierarchy($section)
80 }
81 }
82 } finally { $events.Dispose() }
83}
corpus/conflict-page/native-pages/scripts/text.ps1 created+13
...@@ -0,0 +1,13 @@
1function Get-ParagraphText([string]$html) {
2 $plain = [regex]::Replace($html, '<[^>]*>', '')
3 $plain = [regex]::Replace($plain, '&#(?:x([0-9a-fA-F]+)|([0-9]+));', {
4 param($match)
5 $value = if ($match.Groups[1].Success) {
6 [Convert]::ToInt32($match.Groups[1].Value, 16)
7 } else { [int]$match.Groups[2].Value }
8 # The Win7 framework leaves supplementary numeric entities undecoded.
9 if ($value -gt 0xffff -and $value -le 0x10ffff) { [char]::ConvertFromUtf32($value) }
10 else { $match.Value }
11 })
12 [Net.WebUtility]::HtmlDecode($plain)
13}
corpus/conflict-page/native-pages/server.jsonl created+2
...@@ -0,0 +1,2 @@
1{"command": "mkdir /srv/agent/m6-conflict", "exit": 0, "stdout": "", "stderr": ""}
2{"command": "chmod u+w /srv/agent/m6-conflict/*", "exit": 0, "stdout": "", "stderr": ""}
corpus/conflict-page/native-restore/a/a-merged.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Fictitious: caf\u00e9, \u6771\u4eac, \u0645\u0631\u062d\u0628\u0627 - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/native-restore/a/a-merged.png created
Binary files /dev/null and b/corpus/conflict-page/native-restore/a/a-merged.png differ
corpus/conflict-page/native-restore/a/command-0001.json created+1
...@@ -0,0 +1 @@
1{"action": "pages", "titles": ["One", "Two", "Target", "Three", "Four"]}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0002.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0003.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0004.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0005.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0006.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0007.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0008.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0009.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0010.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0011.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0012.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0013.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0014.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0015.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0016.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0017.json created+1
...@@ -0,0 +1 @@
1{"action": "edit", "expected": "Body Target.", "text": "Body Target edited offline."}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0018.json created+1
...@@ -0,0 +1 @@
1{"action": "move", "page": "Four", "before": "One"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0019.json created+1
...@@ -0,0 +1 @@
1{"action": "snapshot"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0020.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0021.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0022.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0023.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0024.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0025.json created+1
...@@ -0,0 +1 @@
1{"action": "snapshot"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/command-0026.json created+1
...@@ -0,0 +1 @@
1{"action": "close"}
\ No newline at end of file
corpus/conflict-page/native-restore/a/commands.jsonl created+28
...@@ -0,0 +1,28 @@
1{"command": "mkdir C:\\one-tests\\runs\\capture", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -ExecutionPolicy Bypass -File C:\\one-tests\\network.ps1 -LabMac 52:54:02:b6:9a:97", "exit": 0, "stdout": "{\"renewalResult\":0,\"releaseResult\":0,\"mac\":\"52:54:02:b6:9a:97\",\"address\":\"192.168.77.119\"}\r\n", "stderr": "", "error": null}
3{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0001.tmp C:\\one-tests\\runs\\capture\\inbox\\0001.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
4{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0002.tmp C:\\one-tests\\runs\\capture\\inbox\\0002.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
5{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0003.tmp C:\\one-tests\\runs\\capture\\inbox\\0003.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
6{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0004.tmp C:\\one-tests\\runs\\capture\\inbox\\0004.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
7{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0005.tmp C:\\one-tests\\runs\\capture\\inbox\\0005.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
8{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0006.tmp C:\\one-tests\\runs\\capture\\inbox\\0006.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
9{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0007.tmp C:\\one-tests\\runs\\capture\\inbox\\0007.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
10{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0008.tmp C:\\one-tests\\runs\\capture\\inbox\\0008.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
11{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0009.tmp C:\\one-tests\\runs\\capture\\inbox\\0009.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
12{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0010.tmp C:\\one-tests\\runs\\capture\\inbox\\0010.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
13{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0011.tmp C:\\one-tests\\runs\\capture\\inbox\\0011.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
14{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0012.tmp C:\\one-tests\\runs\\capture\\inbox\\0012.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
15{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0013.tmp C:\\one-tests\\runs\\capture\\inbox\\0013.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
16{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0014.tmp C:\\one-tests\\runs\\capture\\inbox\\0014.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
17{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0015.tmp C:\\one-tests\\runs\\capture\\inbox\\0015.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
18{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0016.tmp C:\\one-tests\\runs\\capture\\inbox\\0016.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
19{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0017.tmp C:\\one-tests\\runs\\capture\\inbox\\0017.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
20{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0018.tmp C:\\one-tests\\runs\\capture\\inbox\\0018.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
21{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0019.tmp C:\\one-tests\\runs\\capture\\inbox\\0019.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
22{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0020.tmp C:\\one-tests\\runs\\capture\\inbox\\0020.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
23{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0021.tmp C:\\one-tests\\runs\\capture\\inbox\\0021.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
24{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0022.tmp C:\\one-tests\\runs\\capture\\inbox\\0022.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
25{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0023.tmp C:\\one-tests\\runs\\capture\\inbox\\0023.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
26{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0024.tmp C:\\one-tests\\runs\\capture\\inbox\\0024.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
27{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0025.tmp C:\\one-tests\\runs\\capture\\inbox\\0025.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
28{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0026.tmp C:\\one-tests\\runs\\capture\\inbox\\0026.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
corpus/conflict-page/native-restore/a/hierarchy-0019.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Section xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="synthetic" ID="{FA5C2511-AB32-007B-27C3-C7F92431FBB5}{1}{B0}" path="\\192.168.77.1\agent\m6-conflict\synthetic.one" lastModifiedTime="2026-09-27T09:56:46.000Z" color="#8AA8E4"><one:Page ID="{D615D697-22F9-436A-A0A4-6F68D809E7AA}{1}{B0}" name="Fictitious: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1"/><one:Page ID="{E269A741-C23A-4D0D-9598-77B0FEECEE98}{1}{B0}" name="Four" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:55:48.000Z" pageLevel="1"/><one:Page ID="{64456FBD-67B9-4F61-88FD-16458A15913B}{1}{B0}" name="One" dateTime="2026-09-27T09:55:47.000Z" lastModifiedTime="2026-09-27T09:55:48.000Z" pageLevel="1"/><one:Page ID="{B3F46589-DF47-43C8-9EA9-0AEDC9806EF7}{1}{B0}" name="Two" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:55:48.000Z" pageLevel="1"/><one:Page ID="{8443300C-6402-4B3D-969D-D6F992F2BF2C}{1}{B0}" name="Target" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:56:46.000Z" pageLevel="1"/><one:Page ID="{F81D581F-A816-4C4B-960F-E41C28A79ED3}{1}{B0}" name="Three" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:55:48.000Z" pageLevel="1"/></one:Section>
corpus/conflict-page/native-restore/a/hierarchy-0025.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Section xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="synthetic" ID="{FA5C2511-AB32-007B-27C3-C7F92431FBB5}{1}{B0}" path="\\192.168.77.1\agent\m6-conflict\synthetic.one" lastModifiedTime="2026-09-27T09:57:04.000Z" color="#8AA8E4"><one:Page ID="{D615D697-22F9-436A-A0A4-6F68D809E7AA}{1}{B0}" name="Fictitious: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1"/><one:Page ID="{E269A741-C23A-4D0D-9598-77B0FEECEE98}{1}{B0}" name="Four" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:55:48.000Z" pageLevel="1"/><one:Page ID="{64456FBD-67B9-4F61-88FD-16458A15913B}{1}{B0}" name="One" dateTime="2026-09-27T09:55:47.000Z" lastModifiedTime="2026-09-27T09:55:48.000Z" pageLevel="1"/><one:Page ID="{B3F46589-DF47-43C8-9EA9-0AEDC9806EF7}{1}{B0}" name="Two" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:55:48.000Z" pageLevel="1"/><one:Page ID="{77E5F974-8FC1-41FC-BAC4-EE995A884F9A}{1}{B0}" name="Target" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:56:55.000Z" pageLevel="1"/><one:Page ID="{F81D581F-A816-4C4B-960F-E41C28A79ED3}{1}{B0}" name="Three" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:55:48.000Z" pageLevel="1"/></one:Section>
corpus/conflict-page/native-restore/a/listed.json created+1
...@@ -0,0 +1 @@
1["Fictitious: café, 東京, مرحبا", "Four", "One", "Two", "Target", "Three"]
\ No newline at end of file
corpus/conflict-page/native-restore/a/machine.json created+1
...@@ -0,0 +1 @@
1{"name": "m6-d281b38e", "hostname": "ONE-M6-D281B38E"}
corpus/conflict-page/native-restore/a/teardown.json created+1
...@@ -0,0 +1 @@
1{"absent": true}
corpus/conflict-page/native-restore/b-published/notebook/OneNote_RecycleBin/OneNote_DeletedPages.one created
Binary files /dev/null and b/corpus/conflict-page/native-restore/b-published/notebook/OneNote_RecycleBin/OneNote_DeletedPages.one differ
corpus/conflict-page/native-restore/b-published/notebook/OneNote_RecycleBin/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/native-restore/b-published/notebook/OneNote_RecycleBin/Open Notebook.onetoc2 differ
corpus/conflict-page/native-restore/b-published/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/native-restore/b-published/notebook/Open Notebook.onetoc2 differ
corpus/conflict-page/native-restore/b-published/notebook/synthetic.one created
Binary files /dev/null and b/corpus/conflict-page/native-restore/b-published/notebook/synthetic.one differ
corpus/conflict-page/native-restore/b/b-merged.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Fictitious: caf\u00e9, \u6771\u4eac, \u0645\u0631\u062d\u0628\u0627 - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/native-restore/b/b-merged.png created
Binary files /dev/null and b/corpus/conflict-page/native-restore/b/b-merged.png differ
corpus/conflict-page/native-restore/b/command-0001.json created+1
...@@ -0,0 +1 @@
1{"action": "snapshot"}
\ No newline at end of file
corpus/conflict-page/native-restore/b/command-0002.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/b/command-0003.json created+1
...@@ -0,0 +1 @@
1{"action": "snapshot"}
\ No newline at end of file
corpus/conflict-page/native-restore/b/command-0004.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/b/command-0005.json created+1
...@@ -0,0 +1 @@
1{"action": "snapshot"}
\ No newline at end of file
corpus/conflict-page/native-restore/b/command-0006.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/b/command-0007.json created+1
...@@ -0,0 +1 @@
1{"action": "snapshot"}
\ No newline at end of file
corpus/conflict-page/native-restore/b/command-0008.json created+1
...@@ -0,0 +1 @@
1{"action": "delete-page", "page": "Target"}
\ No newline at end of file
corpus/conflict-page/native-restore/b/command-0009.json created+1
...@@ -0,0 +1 @@
1{"action": "snapshot"}
\ No newline at end of file
corpus/conflict-page/native-restore/b/command-0010.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/b/command-0011.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/b/command-0012.json created+1
...@@ -0,0 +1 @@
1{"action": "sync"}
\ No newline at end of file
corpus/conflict-page/native-restore/b/command-0013.json created+1
...@@ -0,0 +1 @@
1{"action": "snapshot"}
\ No newline at end of file
corpus/conflict-page/native-restore/b/command-0014.json created+1
...@@ -0,0 +1 @@
1{"action": "close"}
\ No newline at end of file
corpus/conflict-page/native-restore/b/commands.jsonl created+16
...@@ -0,0 +1,16 @@
1{"command": "mkdir C:\\one-tests\\runs\\capture", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -ExecutionPolicy Bypass -File C:\\one-tests\\network.ps1 -LabMac 52:54:02:7b:1a:4d", "exit": 0, "stdout": "{\"renewalResult\":0,\"releaseResult\":0,\"mac\":\"52:54:02:7b:1a:4d\",\"address\":\"192.168.77.40\"}\r\n", "stderr": "", "error": null}
3{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0001.tmp C:\\one-tests\\runs\\capture\\inbox\\0001.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
4{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0002.tmp C:\\one-tests\\runs\\capture\\inbox\\0002.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
5{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0003.tmp C:\\one-tests\\runs\\capture\\inbox\\0003.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
6{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0004.tmp C:\\one-tests\\runs\\capture\\inbox\\0004.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
7{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0005.tmp C:\\one-tests\\runs\\capture\\inbox\\0005.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
8{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0006.tmp C:\\one-tests\\runs\\capture\\inbox\\0006.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
9{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0007.tmp C:\\one-tests\\runs\\capture\\inbox\\0007.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
10{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0008.tmp C:\\one-tests\\runs\\capture\\inbox\\0008.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
11{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0009.tmp C:\\one-tests\\runs\\capture\\inbox\\0009.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
12{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0010.tmp C:\\one-tests\\runs\\capture\\inbox\\0010.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
13{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0011.tmp C:\\one-tests\\runs\\capture\\inbox\\0011.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
14{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0012.tmp C:\\one-tests\\runs\\capture\\inbox\\0012.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
15{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0013.tmp C:\\one-tests\\runs\\capture\\inbox\\0013.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
16{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0014.tmp C:\\one-tests\\runs\\capture\\inbox\\0014.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
corpus/conflict-page/native-restore/b/hierarchy-0001.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Section xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="synthetic" ID="{E9F59793-A58A-0A1E-0F4D-7805A7589855}{1}{B0}" path="\\192.168.77.1\agent\m6-conflict\synthetic.one" lastModifiedTime="2026-09-27T09:56:24.000Z" color="#8AA8E4" areAllPagesAvailable="false"><one:Page ID="{C5BC6415-2C41-490F-882A-D0945B60844A}{1}{B0}" name="Fictitious: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1"/></one:Section>
corpus/conflict-page/native-restore/b/hierarchy-0003.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Section xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="synthetic" ID="{E9F59793-A58A-0A1E-0F4D-7805A7589855}{1}{B0}" path="\\192.168.77.1\agent\m6-conflict\synthetic.one" lastModifiedTime="2026-09-27T09:56:36.000Z" color="#8AA8E4" areAllPagesAvailable="false"><one:Page ID="{C5BC6415-2C41-490F-882A-D0945B60844A}{1}{B0}" name="Fictitious: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1"/></one:Section>
corpus/conflict-page/native-restore/b/hierarchy-0005.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Section xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="synthetic" ID="{E9F59793-A58A-0A1E-0F4D-7805A7589855}{1}{B0}" path="\\192.168.77.1\agent\m6-conflict\synthetic.one" lastModifiedTime="2026-09-27T09:56:37.000Z" color="#8AA8E4" areAllPagesAvailable="false"><one:Page ID="{C5BC6415-2C41-490F-882A-D0945B60844A}{1}{B0}" name="Fictitious: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1"/><one:Page ID="{77ECDD3F-6901-4504-A073-A9B9097CF2DB}{1}{B0}" name="One" dateTime="2026-09-27T09:55:47.000Z" lastModifiedTime="2026-09-27T09:56:37.000Z" pageLevel="1"/></one:Section>
corpus/conflict-page/native-restore/b/hierarchy-0007.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Section xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="synthetic" ID="{E9F59793-A58A-0A1E-0F4D-7805A7589855}{1}{B0}" path="\\192.168.77.1\agent\m6-conflict\synthetic.one" lastModifiedTime="2026-09-27T09:56:39.000Z" color="#8AA8E4"><one:Page ID="{C5BC6415-2C41-490F-882A-D0945B60844A}{1}{B0}" name="Fictitious: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1"/><one:Page ID="{77ECDD3F-6901-4504-A073-A9B9097CF2DB}{1}{B0}" name="One" dateTime="2026-09-27T09:55:47.000Z" lastModifiedTime="2026-09-27T09:56:37.000Z" pageLevel="1"/><one:Page ID="{A05DD70B-D1FF-49AD-B627-B5114AE90D17}{1}{B0}" name="Two" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:56:38.000Z" pageLevel="1"/><one:Page ID="{97EA828E-6ABA-4158-BE13-6905119BDCCC}{1}{B0}" name="Target" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:56:39.000Z" pageLevel="1"/><one:Page ID="{EBB4EA9D-A6AE-462E-BE81-5BE0ABCEFD33}{1}{B0}" name="Three" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:56:39.000Z" pageLevel="1"/><one:Page ID="{F1C015C3-CC82-4768-BD16-C84C7D858D78}{1}{B0}" name="Four" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:56:39.000Z" pageLevel="1"/></one:Section>
corpus/conflict-page/native-restore/b/hierarchy-0009.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Section xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="synthetic" ID="{E9F59793-A58A-0A1E-0F4D-7805A7589855}{1}{B0}" path="\\192.168.77.1\agent\m6-conflict\synthetic.one" lastModifiedTime="2026-09-27T09:56:49.000Z" color="#8AA8E4" isUnread="true"><one:Page ID="{C5BC6415-2C41-490F-882A-D0945B60844A}{1}{B0}" name="Fictitious: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1"/><one:Page ID="{77ECDD3F-6901-4504-A073-A9B9097CF2DB}{1}{B0}" name="One" dateTime="2026-09-27T09:55:47.000Z" lastModifiedTime="2026-09-27T09:56:37.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{A05DD70B-D1FF-49AD-B627-B5114AE90D17}{1}{B0}" name="Two" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:56:38.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{EBB4EA9D-A6AE-462E-BE81-5BE0ABCEFD33}{1}{B0}" name="Three" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:56:39.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{F1C015C3-CC82-4768-BD16-C84C7D858D78}{1}{B0}" name="Four" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:56:39.000Z" pageLevel="1" isUnread="true"/></one:Section>
corpus/conflict-page/native-restore/b/hierarchy-0013.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Section xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="synthetic" ID="{E9F59793-A58A-0A1E-0F4D-7805A7589855}{1}{B0}" path="\\192.168.77.1\agent\m6-conflict\synthetic.one" lastModifiedTime="2026-09-27T09:57:25.000Z" color="#8AA8E4" isUnread="true"><one:Page ID="{C5BC6415-2C41-490F-882A-D0945B60844A}{1}{B0}" name="Fictitious: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-05T05:11:41.000Z" pageLevel="1"/><one:Page ID="{F1C015C3-CC82-4768-BD16-C84C7D858D78}{1}{B0}" name="Four" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:56:39.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{77ECDD3F-6901-4504-A073-A9B9097CF2DB}{1}{B0}" name="One" dateTime="2026-09-27T09:55:47.000Z" lastModifiedTime="2026-09-27T09:56:37.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{A05DD70B-D1FF-49AD-B627-B5114AE90D17}{1}{B0}" name="Two" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:56:38.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{644C4BF6-8179-4B99-924A-5165D9E12C7A}{1}{B0}" name="Target" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:57:25.000Z" pageLevel="1" isUnread="true"/><one:Page ID="{EBB4EA9D-A6AE-462E-BE81-5BE0ABCEFD33}{1}{B0}" name="Three" dateTime="2026-09-27T09:55:48.000Z" lastModifiedTime="2026-09-27T09:56:39.000Z" pageLevel="1" isUnread="true"/></one:Section>
corpus/conflict-page/native-restore/b/listed.json created+1
...@@ -0,0 +1 @@
1["Fictitious: café, 東京, مرحبا", "Four", "One", "Two", "Target", "Three"]
\ No newline at end of file
corpus/conflict-page/native-restore/b/machine.json created+1
...@@ -0,0 +1 @@
1{"name": "m6-a205cc61", "hostname": "ONE-M6-A205CC61"}
corpus/conflict-page/native-restore/b/teardown.json created+1
...@@ -0,0 +1 @@
1{"absent": true}
corpus/conflict-page/native-restore/clients.json created+4
...@@ -0,0 +1,4 @@
1{
2 "a": "m6-d281b38e",
3 "b": "m6-a205cc61"
4}
\ No newline at end of file
corpus/conflict-page/native-restore/initial/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/native-restore/initial/notebook/Open Notebook.onetoc2 differ
corpus/conflict-page/native-restore/initial/notebook/synthetic.one created
Binary files /dev/null and b/corpus/conflict-page/native-restore/initial/notebook/synthetic.one differ
corpus/conflict-page/native-restore/linux-teardown.json created+1
...@@ -0,0 +1 @@
1{"absent": true}
corpus/conflict-page/native-restore/linux.json created+11
...@@ -0,0 +1,11 @@
1{
2 "cpus": 2,
3 "disk_gb": 16,
4 "hostname": "one-conflict3",
5 "lab_address": "192.168.77.1",
6 "lab_mac": "52:54:04:08:6d:03",
7 "memory_mb": 2048,
8 "samba_port": 14450,
9 "ssh_port": 22000,
10 "wan_mac": "52:54:03:bb:9c:87"
11}
\ No newline at end of file
corpus/conflict-page/native-restore/merged.json created+49
...@@ -0,0 +1,49 @@
1{
2 "order": [
3 "",
4 "Four",
5 "One",
6 "Two",
7 "Target",
8 "Three"
9 ],
10 "pages": [
11 [
12 "",
13 "Fictitious positioned outline.",
14 "Fictitious: café, 東京, مرحبا",
15 "Left cell",
16 "Right cell"
17 ],
18 [
19 "2:55 AM",
20 "Body Four.",
21 "Four",
22 "Sunday, September 27, 2026"
23 ],
24 [
25 "2:55 AM",
26 "Body One.",
27 "One",
28 "Sunday, September 27, 2026"
29 ],
30 [
31 "2:55 AM",
32 "Body Two.",
33 "Sunday, September 27, 2026",
34 "Two"
35 ],
36 [
37 "2:55 AM",
38 "Body Target edited offline.",
39 "Sunday, September 27, 2026",
40 "Target"
41 ],
42 [
43 "2:55 AM",
44 "Body Three.",
45 "Sunday, September 27, 2026",
46 "Three"
47 ]
48 ]
49}
\ No newline at end of file
corpus/conflict-page/native-restore/merged/notebook/OneNote_RecycleBin/OneNote_DeletedPages.one created
Binary files /dev/null and b/corpus/conflict-page/native-restore/merged/notebook/OneNote_RecycleBin/OneNote_DeletedPages.one differ
corpus/conflict-page/native-restore/merged/notebook/OneNote_RecycleBin/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/native-restore/merged/notebook/OneNote_RecycleBin/Open Notebook.onetoc2 differ
corpus/conflict-page/native-restore/merged/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/native-restore/merged/notebook/Open Notebook.onetoc2 differ
corpus/conflict-page/native-restore/merged/notebook/synthetic.one created
Binary files /dev/null and b/corpus/conflict-page/native-restore/merged/notebook/synthetic.one differ
corpus/conflict-page/native-restore/native_conflict.py created+362
...@@ -0,0 +1,362 @@
1#!/usr/bin/env python3
2"""OneNote 2010 conflict pages: how they arise, what they store and how OneNote shows them.
3
4`sync LINUX OUTPUT` runs two disposable OneNote clients on one Samba notebook: client A edits
5two paragraphs offline while client B edits one of them and a table cell online; A then
6reconnects, which makes OneNote write a conflict page. The server file is kept before and
7after, A and B are screenshotted with the information bar, the page list shown, the conflict
8page open and its menu open, and A then deletes the conflict page through that menu.
9
10`pages LINUX OUTPUT` runs the same two clients on page-list changes: client A, offline, moves
11a page that client B also moves, moves another page, and edits a page B deletes; A then
12reconnects. The server's page order and pages, with the recycle bin, are kept at each step.
13
14`restore LINUX OUTPUT` runs them on a page in the middle of the list: client A, offline, edits
15it and moves a page B leaves alone while B deletes it; A then reconnects.
16
17`cold NOTEBOOK OUTPUT [--delete]` cold-opens a notebook holding a conflict page in a fresh
18clone and screenshots the same views; `--delete` deletes the conflict page through the menu
19and captures the notebook OneNote leaves.
20"""
21import argparse
22from contextlib import ExitStack
23from concurrent.futures import ThreadPoolExecutor
24import base64
25import json
26from pathlib import Path
27import shutil
28import subprocess
29import sys
30import time
31import xml.etree.ElementTree as ET
32
33sys.path.insert(0, str(Path(__file__).resolve().parent))
34sys.path.insert(0, str(Path(__file__).resolve().parent / 'w7'))
35import native_runner as runner
36from native_runner import ROOT, clone, command, windows, vm
37import linux_vm
38
39FIXTURE = ROOT / 'corpus/native-ink/cold-ui-ink/notebook'
40SHARE = 'm6-conflict'
41ORIGINAL = 'Fictitious: café, 東京, مرحبا'
42OTHER_OUTLINE = 'Fictitious positioned outline.'
43CELL = 'Left cell'
44
45# Coordinates on OneNote 2010 maximized at 800x600, from the stage-5, m8 and sync-01 captures:
46# the information bar's text, and the first conflict page listed under its page.
47BAR = (332, 100)
48FIRST_CONFLICT = (725, 142)
49# Delete Conflict Page, first in the menu the bar opens on a conflict page.
50DELETE = (420, 113)
51
52
53def shot(name, output, label, script):
54 """Runs an AutoHotkey step on the clone, keeping its screenshot as `label.png`."""
55 (output / f'{label}.ahk').write_text(script)
56 result = windows.do_exec(script, target=name, timeout_ms=120000, shot_delay_ms=1500)
57 screenshot = result.pop('png_b64', None)
58 if screenshot:
59 (output / f'{label}.png').write_bytes(base64.b64decode(screenshot))
60 (output / f'{label}.json').write_text(json.dumps(result, indent=2))
61 if result.get('error') or result.get('exit') != 0:
62 raise RuntimeError(f'{label}: {result}')
63 return result
64
65
66def click(x, y):
67 return ('WinActivate("ahk_exe ONENOTE.EXE")\nCoordMode("Mouse", "Screen")\n'
68 f'Click({x}, {y})\nSleep(1200)\n')
69
70
71# Reports any dialog OneNote raised, so a confirmation shows in the step's record.
72DIALOG = ('if WinExist("ahk_class #32770 ahk_exe ONENOTE.EXE") {\n'
73 ' FileAppend("dialog: " WinGetTitle() " | " WinGetText() "`n", "*", "UTF-8")\n'
74 '}\n')
75
76
77def conflict_views(name, output, page, prefix, delete=False):
78 """Screenshots a page with a conflict page: the bar, the list shown, the conflict page and
79 its menu; `delete` then chooses Delete Conflict Page."""
80 shot(name, output, f'{prefix}-main', runner.navigation_script(page) + 'Sleep(2000)\n')
81 shot(name, output, f'{prefix}-shown', click(*BAR))
82 shot(name, output, f'{prefix}-conflict', click(*FIRST_CONFLICT))
83 shot(name, output, f'{prefix}-menu', click(*BAR))
84 if delete:
85 shot(name, output, f'{prefix}-delete', click(*DELETE) + DIALOG)
86 else:
87 shot(name, output, f'{prefix}-closed', 'WinActivate("ahk_exe ONENOTE.EXE")\nSend("{Esc}")\nSleep(800)\n')
88
89
90def cold(notebook, output, delete):
91 def interaction(name, output):
92 page = ET.parse(sorted((output / 'read').glob('page-*.xml'))[0]).getroot().attrib['ID']
93 conflict_views(name, output, page, 'cold', delete)
94 if delete:
95 # OneNote writes the deletion in the background; give it time before the re-read.
96 time.sleep(20)
97 runner.capture(notebook, output, collect_notebook=True, interaction=interaction)
98
99
100class Lab:
101 """Two OneNote clients on one Samba notebook: `action` runs a controller command,
102 `state` copies the share and exports its section, `settle` syncs until a check holds."""
103
104
105def page_list(model):
106 """The section's pages in order, each as its sorted texts."""
107 from document_model import ordered_pages, walk
108 return [sorted(node['kind']['text'] for _, node in walk(page, oid) if node['kind']['type'] == 'RichText')
109 for _, _, page, oid in ordered_pages(model)]
110
111
112def edits(lab):
113 from native_collaboration import reachable_page_text
114 a, b, output, action, settle = lab.a, lab.b, lab.output, lab.action, lab.settle
115 state = lambda label: reachable_page_text(lab.state(label))
116 texts = lambda state: {text for values in state.values() for text in values}
117 state('initial')
118 vm.qmp(a['name'], 'set_link', {'name': 'lab', 'up': False})
119 try:
120 action(a, 'edit', expected=ORIGINAL, text='Client A conflicting edit.')
121 action(a, 'edit', expected=OTHER_OUTLINE, text='Client A disjoint edit.')
122 action(a, 'snapshot')
123 action(b, 'edit', expected=ORIGINAL, text='Client B conflicting edit.')
124 action(b, 'edit', expected=CELL, text='Client B disjoint cell.')
125 settle(b, lambda label: {'Client B conflicting edit.', 'Client B disjoint cell.'} <= texts(state(label)), 'b-published')
126 finally:
127 vm.qmp(a['name'], 'set_link', {'name': 'lab', 'up': True})
128 settle(a, lambda label: len(found := state(label)) > 1 and 'Client A disjoint edit.' in texts(found), 'conflict')
129 (output / 'conflict.json').write_text(json.dumps(state('conflict'), indent=2, ensure_ascii=False))
130 page = action(a, 'snapshot')[0].attrib['ID']
131 conflict_views(a['name'], output / 'a', page, 'a')
132 action(b, 'sync')
133 time.sleep(5)
134 page_b = action(b, 'snapshot')[0].attrib['ID']
135 conflict_views(b['name'], output / 'b', page_b, 'b')
136 conflict_views(a['name'], output / 'a', page, 'a-again', delete=True)
137 settle(a, lambda label: len(state(label)) == 1, 'deleted')
138 action(b, 'sync')
139 time.sleep(5)
140 shot(b['name'], output / 'b', 'b-after-delete', runner.navigation_script(page_b) + 'Sleep(2000)\n')
141
142
143PAGES = ['One', 'Two', 'Three', 'Four', 'Target']
144
145
146def titled(model, titles):
147 """The section's pages in order, each named by the title among `titles` it holds."""
148 return [next((text for text in texts if text in titles), '') for texts in page_list(model)]
149
150
151def offline(lab, titles, a_actions, b_actions):
152 """Creates the titled pages; A runs its actions offline while B runs and publishes its
153 own; A then reconnects. Keeps the merged order and pages, and what each client lists."""
154 a, b, output, action = lab.a, lab.b, lab.output, lab.action
155 action(a, 'pages', titles=titles)
156 lab.settle(a, lambda label: 'Body Target.' in str(page_list(lab.state(label))), 'initial')
157 deadline = time.monotonic() + 300
158 while not any(node.get('name') == 'Target' for node in action(b, 'snapshot')):
159 action(b, 'sync')
160 if time.monotonic() > deadline: raise TimeoutError('B never listed the new pages')
161 vm.qmp(a['name'], 'set_link', {'name': 'lab', 'up': False})
162 try:
163 for kind, parameters in a_actions:
164 action(a, kind, **parameters)
165 action(a, 'snapshot')
166 for kind, parameters in b_actions:
167 action(b, kind, **parameters)
168 action(b, 'snapshot')
169 lab.settle(b, lambda label: 'Body Target.' not in str(page_list(lab.state(label))), 'b-published')
170 finally:
171 vm.qmp(a['name'], 'set_link', {'name': 'lab', 'up': True})
172 # A's changes reach the server when its sync writes; wait until the file stops changing.
173 previous, steady = None, 0
174 deadline = time.monotonic() + 300
175 while steady < 3 and time.monotonic() < deadline:
176 action(a, 'sync')
177 time.sleep(10)
178 current = page_list(lab.state('merged'))
179 steady = steady + 1 if current == previous else 0
180 previous = current
181 (output / 'merged.json').write_text(json.dumps({'order': titled(lab.state('merged'), titles), 'pages': previous}, indent=2, ensure_ascii=False))
182 for client, label in ((a, 'a'), (b, 'b')):
183 action(client, 'sync')
184 time.sleep(5)
185 listed = action(client, 'snapshot')
186 (output / label / 'listed.json').write_text(json.dumps([node.get('name') for node in listed], ensure_ascii=False))
187 if listed:
188 shot(client['name'], output / label, f'{label}-merged', runner.navigation_script(listed[0].get('ID')) + 'Sleep(2000)\n')
189
190
191def pages(lab):
192 """Competing page moves, and an offline edit of a page the other client deletes."""
193 offline(lab, PAGES, [
194 ('move', {'page': 'Four', 'before': 'One'}),
195 ('move', {'page': 'Two', 'before': None}),
196 ('edit', {'expected': 'Body Target.', 'text': 'Body Target edited offline.'}),
197 ], [
198 ('move', {'page': 'Four', 'before': None}),
199 ('move', {'page': 'Three', 'before': 'One'}),
200 ('delete-page', {'page': 'Target'}),
201 ])
202
203
204def restore(lab):
205 """An offline edit of a page in the middle of the list the other client deletes, with an
206 offline move of a page the other client leaves alone."""
207 offline(lab, ['One', 'Two', 'Target', 'Three', 'Four'], [
208 ('edit', {'expected': 'Body Target.', 'text': 'Body Target edited offline.'}),
209 ('move', {'page': 'Four', 'before': 'One'}),
210 ], [
211 ('delete-page', {'page': 'Target'}),
212 ])
213
214
215def sync(server, output, scenario):
216 from document_model import EXPORTER
217
218 if linux_vm.instance_path(server).exists():
219 raise ValueError('Choose a new Linux VM name; existing machines are not owned by this run.')
220 output = output.resolve()
221 output.mkdir(parents=True, exist_ok=False)
222 scripts = output / 'scripts'
223 scripts.mkdir()
224 for name in ('cold.ps1', 'collaborate.ps1', 'network.ps1', 'text.ps1', 'stress.ps1'):
225 shutil.copyfile(ROOT / 'tools/native' / name, scripts / name)
226 shutil.copyfile(Path(__file__), output / 'native_conflict.py')
227
228 def ssh(text, timeout=90):
229 result = linux_vm.run_ssh(server, text, timeout=timeout)
230 with (output / 'server.jsonl').open('a') as log:
231 log.write(json.dumps({'command': text, 'exit': result.returncode, 'stdout': result.stdout, 'stderr': result.stderr}) + '\n')
232 result.check_returncode()
233 return result.stdout
234
235 def server_state(label):
236 """Copies the server's notebook, recycle bin included, until its section parses,
237 keeping it as `label/notebook`; returns the section's exported model."""
238 deadline = time.monotonic() + 120
239 while True:
240 target = output / label
241 shutil.rmtree(target, ignore_errors=True)
242 (target / 'notebook').mkdir(parents=True)
243 archive = subprocess.run(linux_vm.ssh_argv(server, f"tar cf - -C '/srv/agent/{SHARE}' ."), capture_output=True, check=True, timeout=60).stdout
244 subprocess.run(['tar', 'xf', '-', '-C', target / 'notebook'], input=archive, check=True)
245 result = subprocess.run([EXPORTER, target / 'notebook/synthetic.one', target / 'model'], capture_output=True, text=True)
246 if result.returncode == 0:
247 return json.loads((target / 'model/document.json').read_text())
248 if time.monotonic() > deadline:
249 raise RuntimeError(f'{label}: {result.stderr}')
250 time.sleep(1)
251
252 linux_vm.create_instance(server)
253 try:
254 linux_vm.launch(server)
255 linux_vm.wait_instance(server, 600)
256 (output / 'linux.json').write_text(json.dumps(linux_vm.load_instance(server), indent=2))
257 ssh(f'mkdir /srv/agent/{SHARE}')
258 for name in ('synthetic.one', 'Open Notebook.onetoc2'):
259 with (FIXTURE / name).open('rb') as stream:
260 subprocess.run(linux_vm.ssh_argv(server, f"cat > '/srv/agent/{SHARE}/{name}'"), stdin=stream, check=True)
261 ssh(f'chmod u+w /srv/agent/{SHARE}/*')
262 with ExitStack() as stack:
263 labels = ['a', 'b']
264 for label in labels:
265 (output / label).mkdir()
266
267 def start(label):
268 manager = clone(output / label)
269 name = manager.__enter__()
270 return label, name, manager
271 with ThreadPoolExecutor(max_workers=2) as pool:
272 started = list(pool.map(start, labels))
273 for _, _, manager in started:
274 stack.push(manager)
275 clients = {}
276 for label, name, _ in started:
277 folder = output / label
278 for local in scripts.iterdir():
279 remote = 'cold-current.ps1' if local.name == 'cold.ps1' else local.name
280 result = windows.do_put(local, 'C:\\one-tests\\' + remote, name)
281 if result.get('error'): raise RuntimeError(result['error'])
282 command(name, 'mkdir C:\\one-tests\\runs\\capture', folder)
283 command(name, 'powershell -NoProfile -ExecutionPolicy Bypass -File C:\\one-tests\\network.ps1 -LabMac ' + vm.lab_mac(name), folder)
284 result = windows.do_spawn('cmd /c powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\collaborate.ps1 -Root C:\\one-tests\\runs\\capture -SharedPath \\\\192.168.77.1\\agent\\' + SHARE + ' -CloneHost ONE-' + name.upper() + ' > C:\\one-tests\\runs\\capture\\controller.log 2>&1', name)
285 if result.get('error'): raise RuntimeError(result['error'])
286 deadline = time.monotonic() + 180
287 while windows.do_cmd('if exist C:\\one-tests\\runs\\capture\\ready (echo ready)', target=name).get('stdout', '').find('ready') < 0:
288 if time.monotonic() > deadline: raise RuntimeError('The collaboration controller did not open the notebook')
289 time.sleep(1)
290 clients[label] = {'name': name, 'folder': folder, 'sequence': 0}
291 (output / 'clients.json').write_text(json.dumps({label: client['name'] for label, client in clients.items()}, indent=2))
292
293 def action(client, kind, **parameters):
294 client['sequence'] += 1
295 sequence = client['sequence']
296 local = client['folder'] / f'command-{sequence:04}.json'
297 local.write_text(json.dumps({'action': kind, **parameters}, ensure_ascii=False))
298 inbox = f'C:\\one-tests\\runs\\capture\\inbox\\{sequence:04}'
299 result = windows.do_put(local, inbox + '.tmp', client['name'])
300 if result.get('error'): raise RuntimeError(result['error'])
301 command(client['name'], f'move {inbox}.tmp {inbox}.json', client['folder'])
302 remote = f'C:\\one-tests\\runs\\capture\\outbox\\{sequence}'
303 deadline = time.monotonic() + 600
304 while True:
305 result = windows.do_cmd(f'if exist {remote}\\error (type {remote}\\error & exit /b 1) else (if exist {remote}\\done (echo complete))', target=client['name'])
306 if result.get('exit') != 0: raise RuntimeError(str(result))
307 if 'complete' in result.get('stdout', ''): break
308 if time.monotonic() > deadline: raise TimeoutError(f'{kind} did not complete')
309 time.sleep(.5)
310 if kind != 'snapshot':
311 return None
312 hierarchy = client['folder'] / f'hierarchy-{sequence:04}.xml'
313 windows.do_get(remote + '\\hierarchy.xml', hierarchy, client['name'])
314 pages = [node for node in ET.fromstring(hierarchy.read_text(encoding='utf-8-sig').strip()).iter() if node.tag.endswith('}Page')]
315 for i in range(len(pages)):
316 windows.do_get(remote + f'\\page-{i}.xml', client['folder'] / f'snapshot-{sequence:04}-{i}.xml', client['name'])
317 return pages
318
319 def settle(client, check, label):
320 deadline = time.monotonic() + 300
321 while True:
322 action(client, 'sync')
323 if check(label): return
324 if time.monotonic() > deadline: raise TimeoutError(label)
325 time.sleep(2)
326
327 lab = Lab()
328 lab.a, lab.b, lab.output = clients['a'], clients['b'], output
329 lab.action, lab.state, lab.settle = action, server_state, settle
330 scenario(lab)
331
332 for client in clients.values():
333 try:
334 action(client, 'close')
335 except Exception as error:
336 (client['folder'] / 'close-failure.txt').write_text(str(error))
337 server_state('final')
338 finally:
339 if linux_vm.running(server):
340 linux_vm.shutdown(server, 120)
341 linux_vm.delete_instance(server)
342 (output / 'linux-teardown.json').write_text(json.dumps({'absent': not linux_vm.instance_path(server).exists()}) + '\n')
343
344
345if __name__ == '__main__':
346 parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
347 commands = parser.add_subparsers(dest='mode', required=True)
348 scenarios = {'sync': edits, 'pages': pages, 'restore': restore}
349 for mode in scenarios:
350 lab = commands.add_parser(mode)
351 lab.add_argument('linux')
352 lab.add_argument('output', type=Path)
353 reopen = commands.add_parser('cold')
354 reopen.add_argument('notebook', type=Path)
355 reopen.add_argument('output', type=Path)
356 reopen.add_argument('--delete', action='store_true')
357 args = parser.parse_args()
358 if args.mode in scenarios:
359 sync(args.linux, args.output, scenarios[args.mode])
360 else:
361 cold(args.notebook, args.output, args.delete)
362 print('Captured', args.output)
corpus/conflict-page/native-restore/scripts/cold.ps1 created+27
...@@ -0,0 +1,27 @@
1param([Parameter(Mandatory=$true)][string]$Root, [string]$CloneHost = '')
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
5if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
6 throw 'Choose a run directly below C:\one-tests\runs.'
7}
8if (Get-Process ONENOTE -ErrorAction SilentlyContinue) { throw 'Close OneNote before resetting its test cache.' }
9$key = 'HKCU:\Software\Microsoft\Office\14.0\OneNote'
10if ($CloneHost) {
11 if ($CloneHost -notmatch '^ONE-[A-Z0-9-]+$' -or [Environment]::MachineName -ne $CloneHost) {
12 throw 'The disposable clone hostname does not match this machine.'
13 }
14 New-Item "$key\Options\Paths" -Force | Out-Null
15 New-ItemProperty "$key\Options\Paths" -Name UnfiledNotesSection -PropertyType ExpandString -Value 'C:\one-tests\Loose.one' -Force | Out-Null
16} elseif ((Get-ItemProperty "$key\Options\Paths").UnfiledNotesSection -ne 'C:\one-tests\Loose.one' -or
17 -not (Test-Path 'C:\one-tests\profile-original-cache')) {
18 throw 'Park the personal OneNote profile before resetting the test cache.'
19}
20$cache = Join-Path $env:LOCALAPPDATA 'Microsoft\OneNote\14.0'
21$parked = Join-Path 'C:\one-tests\caches' ([IO.Path]::GetFileName($root))
22if (Test-Path $parked) { throw 'Choose a new run; its parked cache already exists.' }
23New-Item -ItemType Directory -Path 'C:\one-tests\caches' -Force | Out-Null
24if (Test-Path $cache) { Move-Item -LiteralPath $cache -Destination $parked }
25if (Test-Path "$key\OpenNotebooks") { Remove-Item "$key\OpenNotebooks" -Recurse }
26New-Item "$key\OpenNotebooks" | Out-Null
27New-ItemProperty "$key\OpenNotebooks" -Name '1' -PropertyType String -Value "$root\notebook" | Out-Null
corpus/conflict-page/native-restore/scripts/collaborate.ps1 created+145
...@@ -0,0 +1,145 @@
1param([Parameter(Mandatory=$true)][string]$Root,
2 [Parameter(Mandatory=$true)][string]$SharedPath,
3 [Parameter(Mandatory=$true)][string]$CloneHost,
4 [switch]$ResumeCache,
5 [int]$StartSequence = 1)
6function Keep-Container($page, $target) {
7 $container = $target
8 while ($container.ParentNode -ne $page.DocumentElement) { $container = $container.ParentNode }
9 foreach ($sibling in @($page.DocumentElement.ChildNodes)) {
10 if ($sibling -ne $container -and $sibling.LocalName -in @('Outline', 'Title', 'Image', 'InkDrawing')) {
11 [void]$page.DocumentElement.RemoveChild($sibling)
12 }
13 }
14}
15Set-StrictMode -Version Latest
16$ErrorActionPreference = 'Stop'
17. "$PSScriptRoot\text.ps1"
18if ($SharedPath -notmatch '^\\\\192\.168\.77\.1\\agent\\m6-[a-z0-9-]+$') { throw 'Choose a disposable Linux lab notebook.' }
19if ($env:COMPUTERNAME -ne $CloneHost) { throw 'The collaboration controller requires its owned clone.' }
20if (-not $ResumeCache) { & "$PSScriptRoot\cold-current.ps1" -Root $Root -CloneHost $CloneHost }
21$shared = $SharedPath
22if (-not (Test-Path "$shared\synthetic.one")) { throw 'Create the shared fixture first.' }
23New-Item -ItemType Directory "$Root\inbox", "$Root\outbox" -Force | Out-Null
24$user = 'HKCU:\Software\Microsoft\Office\Common\UserInfo'
25New-Item $user -Force | Out-Null
26New-ItemProperty $user -Name UserName -Value $CloneHost -PropertyType String -Force | Out-Null
27New-ItemProperty $user -Name UserInitials -Value $CloneHost.Substring($CloneHost.Length - 3) -PropertyType String -Force | Out-Null
28$app = New-Object -ComObject OneNote.Application
29$notebook = ''
30try {
31 $app.OpenHierarchy($shared, '', [ref]$notebook, 0)
32 $section = ''
33 $app.OpenHierarchy('synthetic.one', $notebook, [ref]$section, 0)
34 [IO.File]::WriteAllText("$Root\ready", 'ready')
35 $deadline = [DateTime]::UtcNow.AddMinutes(30)
36 $sequence = $StartSequence
37 while ([DateTime]::UtcNow -lt $deadline) {
38 $name = '{0:d4}.json' -f $sequence
39 $input = "$Root\inbox\$name"
40 if (-not (Test-Path $input)) { Start-Sleep -Milliseconds 100; continue }
41 $command = Get-Content -Raw -Encoding UTF8 $input | ConvertFrom-Json
42 $output = "$Root\outbox\$sequence"
43 New-Item -ItemType Directory $output | Out-Null
44 try {
45 if ($command.action -in @('prepare-stress', 'stress')) {
46 . "$PSScriptRoot\stress.ps1"
47 Invoke-Stress $app $section $command $output $shared
48 }
49 elseif ($command.action -eq 'sync') { $app.SyncHierarchy($notebook) }
50 elseif ($command.action -in @('pages', 'move', 'delete-page')) {
51 $namespace = 'http://schemas.microsoft.com/office/onenote/2010/onenote'
52 $hierarchy = ''
53 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
54 [xml]$tree = $hierarchy
55 $listed = @($tree.SelectNodes('//*[local-name()="Page"]'))
56 $named = { param($title) @($listed | Where-Object { $_.GetAttribute('name') -eq $title }) }
57 if ($command.action -eq 'pages') {
58 # New titled pages with one body paragraph each, appended in order.
59 foreach ($title in $command.titles) {
60 $page = ''
61 $app.CreateNewPage($section, [ref]$page, 2)
62 $body = [Security.SecurityElement]::Escape('Body ' + $title + '.')
63 $name = [Security.SecurityElement]::Escape($title)
64 $xml = '<one:Page xmlns:one="' + $namespace + '" ID="' + $page + '"><one:Title><one:OE><one:T>' + $name + '</one:T></one:OE></one:Title><one:Outline><one:OEChildren><one:OE><one:T>' + $body + '</one:T></one:OE></one:OEChildren></one:Outline></one:Page>'
65 $app.UpdatePageContent($xml, [DateTime]::MinValue, 1, $false)
66 }
67 } elseif ($command.action -eq 'move') {
68 # Moves the page titled `page` before the one titled `before`, or last.
69 $moved = @(& $named $command.page)
70 if ($moved.Count -ne 1) { throw 'Expected one page to move.' }
71 $order = [Collections.ArrayList]@($listed | Where-Object { $_ -ne $moved[0] })
72 $at = $order.Count
73 if ($command.before) {
74 $anchor = @(& $named $command.before)
75 if ($anchor.Count -ne 1) { throw 'Expected one anchor page.' }
76 $at = $order.IndexOf($anchor[0])
77 }
78 $order.Insert($at, $moved[0])
79 $xml = '<one:Section xmlns:one="' + $namespace + '" ID="' + $section + '">'
80 foreach ($node in $order) {
81 $xml += '<one:Page ID="' + $node.GetAttribute('ID') + '" pageLevel="' + $node.GetAttribute('pageLevel') + '"/>'
82 }
83 $app.UpdateHierarchy($xml + '</one:Section>', 1)
84 } else {
85 $doomed = @(& $named $command.page)
86 if ($doomed.Count -ne 1) { throw 'Expected one page to delete.' }
87 $app.DeleteHierarchy($doomed[0].GetAttribute('ID'), [DateTime]::MinValue, $false)
88 }
89 }
90 elseif ($command.action -eq 'kill-process') {
91 Get-Process ONENOTE | Stop-Process -Force
92 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
93 $app = New-Object -ComObject OneNote.Application
94 $app.OpenHierarchy($shared, '', [ref]$notebook, 0)
95 $app.OpenHierarchy('synthetic.one', $notebook, [ref]$section, 0)
96 }
97 elseif ($command.action -eq 'edit' -or $command.action -eq 'snapshot') {
98 $hierarchy = ''
99 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
100 [IO.File]::WriteAllText("$output\hierarchy.xml", $hierarchy, [Text.Encoding]::UTF8)
101 [xml]$tree = $hierarchy
102 $matches = 0
103 $editedPage = $null
104 $i = 0
105 foreach ($node in $tree.SelectNodes('//*[local-name()="Page"]')) {
106 $content = ''
107 $app.GetPageContent($node.GetAttribute('ID'), [ref]$content, 1, 1)
108 [IO.File]::WriteAllText("$output\page-$i.xml", $content, [Text.Encoding]::UTF8)
109 [xml]$page = $content
110 if ($command.action -eq 'edit') {
111 $targets = @($page.SelectNodes('//*[local-name()="T"]') | Where-Object {
112 (Get-ParagraphText $_.InnerText) -eq $command.expected
113 })
114 if ($targets.Count -gt 1) { throw 'Expected one matching text object.' }
115 if ($targets.Count -eq 1) {
116 $matches++
117 if ($matches -gt 1) { throw 'The text occurs on multiple pages.' }
118 $targets[0].InnerText = [Security.SecurityElement]::Escape($command.text)
119 Keep-Container $page $targets[0]
120 $editedPage = $page
121 }
122 }
123 $i++
124 }
125 if ($command.action -eq 'edit' -and $matches -ne 1) { throw 'Expected one matching text object.' }
126 if ($editedPage) { $app.UpdatePageContent($editedPage.OuterXml, [DateTime]::MinValue, 1, $false) }
127 } elseif ($command.action -ne 'close') { throw 'Unknown collaboration command.' }
128 [IO.File]::WriteAllText("$output\done", 'ok')
129 } catch {
130 [IO.File]::WriteAllText("$output\error", $_.Exception.ToString())
131 }
132 if ($command.action -eq 'close') { break }
133 $sequence++
134 }
135} finally {
136 try { if ($notebook) { $app.CloseNotebook($notebook, $false) } }
137 finally {
138 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
139 $app = $null
140 [GC]::Collect()
141 [GC]::WaitForPendingFinalizers()
142 Get-Process ONENOTE -ErrorAction SilentlyContinue | Wait-Process -Timeout 10
143 [IO.File]::WriteAllText("$Root\closed", 'closed')
144 }
145}
corpus/conflict-page/native-restore/scripts/network.ps1 created+20
...@@ -0,0 +1,20 @@
1param([Parameter(Mandatory=$true)][string]$LabMac)
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$adapter = @(Get-WmiObject Win32_NetworkAdapterConfiguration | Where-Object { $_.MACAddress -eq $LabMac })
5if ($adapter.Count -ne 1) { throw 'The clone lab adapter is ambiguous.' }
6$enabled = $adapter[0].EnableDHCP()
7if ($enabled.ReturnValue -notin @(0, 1)) {
8 throw ('Unable to enable DHCP on the clone lab adapter; result ' + $enabled.ReturnValue)
9}
10$release = $adapter[0].ReleaseDHCPLease()
11$result = $adapter[0].RenewDHCPLease()
12$deadline = [DateTime]::UtcNow.AddSeconds(60)
13do {
14 $adapter = @(Get-WmiObject Win32_NetworkAdapterConfiguration | Where-Object { $_.MACAddress -eq $LabMac })
15 $addresses = @($adapter[0].IPAddress | Where-Object { $_ -match '^192\.168\.77\.\d+$' -and $_ -ne '192.168.77.1' })
16 if ($addresses.Count -eq 1) { break }
17 Start-Sleep -Milliseconds 250
18} while ([DateTime]::UtcNow -lt $deadline)
19if ($addresses.Count -ne 1) { throw ('The clone did not receive a lab IPv4 address; renewal result ' + $result.ReturnValue + '; adapter ' + ($adapter[0] | Select-Object MACAddress,DHCPEnabled,IPEnabled,IPAddress | ConvertTo-Json -Compress)) }
20@{mac=$LabMac;address=$addresses[0];releaseResult=$release.ReturnValue;renewalResult=$result.ReturnValue} | ConvertTo-Json -Compress
corpus/conflict-page/native-restore/scripts/stress.ps1 created+83
...@@ -0,0 +1,83 @@
1function Invoke-Stress($app, $section, $command, $output, $shared) {
2 $hierarchy = ''
3 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
4 [IO.File]::WriteAllText("$output\hierarchy.xml", $hierarchy, [Text.Encoding]::UTF8)
5 [xml]$tree = $hierarchy
6 $pages = @($tree.SelectNodes('//*[local-name()="Page"]'))
7 if ($pages.Count -ne 1) { throw 'Expected one stress-test page.' }
8 $pageId = $pages[0].GetAttribute('ID')
9 if ($command.action -eq 'prepare-stress') {
10 for ($i = 0; $i -lt $command.clients; $i++) {
11 $xml = '<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="' + $pageId + '"><one:Outline><one:Position x="48" y="' + (150 + 75 * $i) + '" z="' + ($i + 1) + '"/><one:OEChildren><one:OE><one:T><![CDATA[Native ' + $i + ':]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>'
12 $app.UpdatePageContent($xml, [DateTime]::MinValue, 1, $false)
13 }
14 return
15 }
16 $random = New-Object Random([int]$command.seed)
17 $events = New-Object IO.StreamWriter("$output\events.jsonl", $false, [Text.Encoding]::UTF8)
18 $events.AutoFlush = $true
19 try {
20 [IO.File]::WriteAllText("$output\ready", 'ready')
21 $deadline = [DateTime]::UtcNow.AddMinutes(10)
22 while (-not (Test-Path "$shared\stress-start")) {
23 if ([DateTime]::UtcNow -gt $deadline) { throw 'Stress start timed out.' }
24 Start-Sleep -Milliseconds 20
25 }
26 for ($i = 0; $i -lt $command.operations; $i++) {
27 if (($command.PSObject.Properties.Name -contains 'maintenance') -and $command.maintenance -and $i -eq [Math]::Floor($command.operations / 2)) {
28 [IO.File]::WriteAllText("$shared\maintenance-paused-n$($command.actor)", 'paused')
29 while (-not (Test-Path "$shared\maintenance-resume")) {
30 if ([DateTime]::UtcNow -gt $deadline) { throw 'Maintenance pause timed out.' }
31 Start-Sleep -Milliseconds 100
32 }
33 }
34 Start-Sleep -Milliseconds $random.Next(10, 100)
35 $started = [DateTime]::UtcNow.Ticks
36 $content = ''
37 $app.GetPageContent($pageId, [ref]$content, 1, 1)
38 [xml]$page = $content
39 $matches = @($page.SelectNodes('//*[local-name()="T"]') | Where-Object {
40 (Get-ParagraphText $_.InnerText).StartsWith($command.prefix)
41 })
42 if ($matches.Count -ne 1) { throw 'Expected one stress-test paragraph.' }
43 $before = Get-ParagraphText $matches[0].InnerText
44 $token = ' [n' + $command.actor + ':' + $i + ']'
45 $start = $end = $before.Length
46 $replacement = $token
47 $bold = $italic = $false
48 if ($command.edit) {
49 $boundaries = New-Object 'Collections.Generic.List[int]'
50 for ($at = $command.prefix.Length; $at -le $before.Length; $at++) {
51 $boundaries.Add($at)
52 if ($at -lt $before.Length -and [char]::IsHighSurrogate($before[$at])) { $at++ }
53 }
54 $first = $boundaries[$random.Next($boundaries.Count)]
55 $second = $boundaries[$random.Next($boundaries.Count)]
56 $start = [Math]::Min($first, $second)
57 $end = [Math]::Max($first, $second)
58 $replacement = ' caf' + [char]0xe9 + ' ' + [char]::ConvertFromUtf32(0x1f980) + $token
59 $bold = $random.Next(2) -eq 1
60 $italic = $random.Next(2) -eq 1
61 }
62 $after = $before.Substring(0, $start) + $replacement + $before.Substring($end)
63 $html = [Security.SecurityElement]::Escape($after)
64 if ($command.edit) {
65 $weight = if ($bold) { 'bold' } else { 'normal' }
66 $style = if ($italic) { 'italic' } else { 'normal' }
67 $html = '<span style="font-weight:' + $weight + ';font-style:' + $style + '">' + $html + '</span>'
68 }
69 $matches[0].InnerText = $html
70 Keep-Container $page $matches[0]
71 $updateStarted = [DateTime]::UtcNow.Ticks
72 $app.UpdatePageContent($page.OuterXml, [DateTime]::MinValue, 1, $false)
73 $updated = [DateTime]::UtcNow.Ticks
74 $events.WriteLine((@{ operation=$i; before=$before; token=$token; started_ticks=$started;
75 range=@($start, $end); replacement=$replacement; bold=$bold; italic=$italic;
76 update_started_ticks=$updateStarted; updated_ticks=$updated } | ConvertTo-Json -Compress))
77 if ($i -eq 0) { [IO.File]::WriteAllText("$output\editing", 'editing') }
78 if ($command.sync_every -gt 0 -and (($i + 1) % $command.sync_every) -eq 0) {
79 $app.SyncHierarchy($section)
80 }
81 }
82 } finally { $events.Dispose() }
83}
corpus/conflict-page/native-restore/scripts/text.ps1 created+13
...@@ -0,0 +1,13 @@
1function Get-ParagraphText([string]$html) {
2 $plain = [regex]::Replace($html, '<[^>]*>', '')
3 $plain = [regex]::Replace($plain, '&#(?:x([0-9a-fA-F]+)|([0-9]+));', {
4 param($match)
5 $value = if ($match.Groups[1].Success) {
6 [Convert]::ToInt32($match.Groups[1].Value, 16)
7 } else { [int]$match.Groups[2].Value }
8 # The Win7 framework leaves supplementary numeric entities undecoded.
9 if ($value -gt 0xffff -and $value -le 0x10ffff) { [char]::ConvertFromUtf32($value) }
10 else { $match.Value }
11 })
12 [Net.WebUtility]::HtmlDecode($plain)
13}
corpus/conflict-page/native-restore/server.jsonl created+2
...@@ -0,0 +1,2 @@
1{"command": "mkdir /srv/agent/m6-conflict", "exit": 0, "stdout": "", "stderr": ""}
2{"command": "chmod u+w /srv/agent/m6-conflict/*", "exit": 0, "stdout": "", "stderr": ""}
corpus/conflict-page/native-three/commands.jsonl created+6
...@@ -0,0 +1,6 @@
1{"command": "powershell -NoProfile -Command \"Expand-Archive -LiteralPath C:\\one-tests\\transfer.zip -DestinationPath C:\\one-tests\\runs\\capture\\notebook\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-C6F6C14F -ExpectedPages -1 -Pdf -KeepOpen", "exit": 0, "stdout": "Read 1 sections and 1 pages.\r\n", "stderr": "", "error": null}
3{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\read -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
4{"command": "powershell -NoProfile -Command \"Rename-Item C:\\one-tests\\runs\\capture\\read before-read\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
5{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-C6F6C14F -UseCurrentCache -ExpectedPages -1 -Pdf", "exit": 0, "stdout": "Read 1 sections and 1 pages.\r\n", "stderr": "", "error": null}
6{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
corpus/conflict-page/native-three/conflicts.png created
Binary files /dev/null and b/corpus/conflict-page/native-three/conflicts.png differ
corpus/conflict-page/native-three/expand.png created
Binary files /dev/null and b/corpus/conflict-page/native-three/expand.png differ
corpus/conflict-page/native-three/machine.json created+1
...@@ -0,0 +1 @@
1{"name": "m6-c6f6c14f", "hostname": "ONE-M6-C6F6C14F"}
corpus/conflict-page/native-three/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/native-three/notebook/Open Notebook.onetoc2 differ
corpus/conflict-page/native-three/notebook/synthetic.one created
Binary files /dev/null and b/corpus/conflict-page/native-three/notebook/synthetic.one differ
corpus/conflict-page/native-three/run.json created+17
...@@ -0,0 +1,17 @@
1{
2 "notebook": "/Users/clo/dev/one/evidence/m8/conflict-abrupt-01/conflict-closed/notebook",
3 "expected_pages": -1,
4 "author": null,
5 "author_timeout_seconds": 600,
6 "inspect": true,
7 "base": {
8 "file": "win7-office-base.qcow2",
9 "format": "qcow2",
10 "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346",
11 "virtual_size": 68719476736
12 },
13 "scripts": {
14 "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331",
15 "read.ps1": "086789e467779add0fc071ad8c524c2b02d3d0d609aea4a41f3128cadba371e8"
16 }
17}
corpus/conflict-page/native-three/source.json created+14
...@@ -0,0 +1,14 @@
1[
2 {
3 "path": "Open Notebook.onetoc2",
4 "bytes": 3780,
5 "sha256": "f21c6d4eefc25ce78aee0420fdee0b409da52fdf3119b66302ac6ca4690c55ae",
6 "mtime_ns": 1788733155312083279
7 },
8 {
9 "path": "synthetic.one",
10 "bytes": 46416,
11 "sha256": "7ac7a42932848380b90aa478800ab141fce4fb591aae2bf1e167f1e516e5edc0",
12 "mtime_ns": 1788733155311971612
13 }
14]
corpus/conflict-page/native-three/teardown.json created+1
...@@ -0,0 +1 @@
1{"absent": true}
corpus/conflict-page/native/a/a-conflict.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Client A conflicting edit. - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/native/a/a-conflict.png created
Binary files /dev/null and b/corpus/conflict-page/native/a/a-conflict.png differ
corpus/conflict-page/native/a/a-main.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Client B conflicting edit. - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/native/a/a-main.png created
Binary files /dev/null and b/corpus/conflict-page/native/a/a-main.png differ
corpus/conflict-page/native/a/a-menu.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "",
10 "class": "Net UI Tool Window",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/native/a/a-menu.png created
Binary files /dev/null and b/corpus/conflict-page/native/a/a-menu.png differ
corpus/conflict-page/native/a/a-shown.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Client A conflicting edit. - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/native/a/a-shown.png created
Binary files /dev/null and b/corpus/conflict-page/native/a/a-shown.png differ
corpus/conflict-page/native/a/commands.jsonl created+124
...@@ -0,0 +1,124 @@
1{"command": "mkdir C:\\one-tests\\runs\\capture", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -ExecutionPolicy Bypass -File C:\\one-tests\\network.ps1 -LabMac 52:54:02:a3:5c:e3", "exit": 0, "stdout": "{\"renewalResult\":0,\"releaseResult\":0,\"mac\":\"52:54:02:a3:5c:e3\",\"address\":\"192.168.77.180\"}\r\n", "stderr": "", "error": null}
3{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0001.tmp C:\\one-tests\\runs\\capture\\inbox\\0001.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
4{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0002.tmp C:\\one-tests\\runs\\capture\\inbox\\0002.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
5{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0003.tmp C:\\one-tests\\runs\\capture\\inbox\\0003.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
6{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0004.tmp C:\\one-tests\\runs\\capture\\inbox\\0004.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
7{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0005.tmp C:\\one-tests\\runs\\capture\\inbox\\0005.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
8{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0006.tmp C:\\one-tests\\runs\\capture\\inbox\\0006.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
9{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0007.tmp C:\\one-tests\\runs\\capture\\inbox\\0007.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
10{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0008.tmp C:\\one-tests\\runs\\capture\\inbox\\0008.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
11{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0009.tmp C:\\one-tests\\runs\\capture\\inbox\\0009.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
12{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0010.tmp C:\\one-tests\\runs\\capture\\inbox\\0010.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
13{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0011.tmp C:\\one-tests\\runs\\capture\\inbox\\0011.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
14{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0012.tmp C:\\one-tests\\runs\\capture\\inbox\\0012.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
15{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0013.tmp C:\\one-tests\\runs\\capture\\inbox\\0013.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
16{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0014.tmp C:\\one-tests\\runs\\capture\\inbox\\0014.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
17{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0015.tmp C:\\one-tests\\runs\\capture\\inbox\\0015.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
18{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0016.tmp C:\\one-tests\\runs\\capture\\inbox\\0016.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
19{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0017.tmp C:\\one-tests\\runs\\capture\\inbox\\0017.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
20{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0018.tmp C:\\one-tests\\runs\\capture\\inbox\\0018.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
21{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0019.tmp C:\\one-tests\\runs\\capture\\inbox\\0019.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
22{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0020.tmp C:\\one-tests\\runs\\capture\\inbox\\0020.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
23{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0021.tmp C:\\one-tests\\runs\\capture\\inbox\\0021.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
24{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0022.tmp C:\\one-tests\\runs\\capture\\inbox\\0022.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
25{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0023.tmp C:\\one-tests\\runs\\capture\\inbox\\0023.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
26{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0024.tmp C:\\one-tests\\runs\\capture\\inbox\\0024.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
27{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0025.tmp C:\\one-tests\\runs\\capture\\inbox\\0025.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
28{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0026.tmp C:\\one-tests\\runs\\capture\\inbox\\0026.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
29{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0027.tmp C:\\one-tests\\runs\\capture\\inbox\\0027.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
30{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0028.tmp C:\\one-tests\\runs\\capture\\inbox\\0028.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
31{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0029.tmp C:\\one-tests\\runs\\capture\\inbox\\0029.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
32{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0030.tmp C:\\one-tests\\runs\\capture\\inbox\\0030.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
33{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0031.tmp C:\\one-tests\\runs\\capture\\inbox\\0031.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
34{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0032.tmp C:\\one-tests\\runs\\capture\\inbox\\0032.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
35{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0033.tmp C:\\one-tests\\runs\\capture\\inbox\\0033.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
36{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0034.tmp C:\\one-tests\\runs\\capture\\inbox\\0034.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
37{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0035.tmp C:\\one-tests\\runs\\capture\\inbox\\0035.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
38{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0036.tmp C:\\one-tests\\runs\\capture\\inbox\\0036.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
39{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0037.tmp C:\\one-tests\\runs\\capture\\inbox\\0037.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
40{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0038.tmp C:\\one-tests\\runs\\capture\\inbox\\0038.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
41{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0039.tmp C:\\one-tests\\runs\\capture\\inbox\\0039.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
42{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0040.tmp C:\\one-tests\\runs\\capture\\inbox\\0040.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
43{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0041.tmp C:\\one-tests\\runs\\capture\\inbox\\0041.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
44{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0042.tmp C:\\one-tests\\runs\\capture\\inbox\\0042.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
45{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0043.tmp C:\\one-tests\\runs\\capture\\inbox\\0043.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
46{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0044.tmp C:\\one-tests\\runs\\capture\\inbox\\0044.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
47{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0045.tmp C:\\one-tests\\runs\\capture\\inbox\\0045.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
48{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0046.tmp C:\\one-tests\\runs\\capture\\inbox\\0046.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
49{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0047.tmp C:\\one-tests\\runs\\capture\\inbox\\0047.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
50{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0048.tmp C:\\one-tests\\runs\\capture\\inbox\\0048.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
51{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0049.tmp C:\\one-tests\\runs\\capture\\inbox\\0049.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
52{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0050.tmp C:\\one-tests\\runs\\capture\\inbox\\0050.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
53{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0051.tmp C:\\one-tests\\runs\\capture\\inbox\\0051.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
54{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0052.tmp C:\\one-tests\\runs\\capture\\inbox\\0052.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
55{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0053.tmp C:\\one-tests\\runs\\capture\\inbox\\0053.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
56{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0054.tmp C:\\one-tests\\runs\\capture\\inbox\\0054.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
57{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0055.tmp C:\\one-tests\\runs\\capture\\inbox\\0055.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
58{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0056.tmp C:\\one-tests\\runs\\capture\\inbox\\0056.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
59{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0057.tmp C:\\one-tests\\runs\\capture\\inbox\\0057.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
60{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0058.tmp C:\\one-tests\\runs\\capture\\inbox\\0058.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
61{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0059.tmp C:\\one-tests\\runs\\capture\\inbox\\0059.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
62{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0060.tmp C:\\one-tests\\runs\\capture\\inbox\\0060.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
63{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0061.tmp C:\\one-tests\\runs\\capture\\inbox\\0061.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
64{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0062.tmp C:\\one-tests\\runs\\capture\\inbox\\0062.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
65{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0063.tmp C:\\one-tests\\runs\\capture\\inbox\\0063.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
66{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0064.tmp C:\\one-tests\\runs\\capture\\inbox\\0064.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
67{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0065.tmp C:\\one-tests\\runs\\capture\\inbox\\0065.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
68{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0066.tmp C:\\one-tests\\runs\\capture\\inbox\\0066.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
69{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0067.tmp C:\\one-tests\\runs\\capture\\inbox\\0067.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
70{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0068.tmp C:\\one-tests\\runs\\capture\\inbox\\0068.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
71{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0069.tmp C:\\one-tests\\runs\\capture\\inbox\\0069.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
72{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0070.tmp C:\\one-tests\\runs\\capture\\inbox\\0070.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
73{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0071.tmp C:\\one-tests\\runs\\capture\\inbox\\0071.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
74{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0072.tmp C:\\one-tests\\runs\\capture\\inbox\\0072.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
75{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0073.tmp C:\\one-tests\\runs\\capture\\inbox\\0073.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
76{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0074.tmp C:\\one-tests\\runs\\capture\\inbox\\0074.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
77{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0075.tmp C:\\one-tests\\runs\\capture\\inbox\\0075.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
78{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0076.tmp C:\\one-tests\\runs\\capture\\inbox\\0076.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
79{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0077.tmp C:\\one-tests\\runs\\capture\\inbox\\0077.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
80{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0078.tmp C:\\one-tests\\runs\\capture\\inbox\\0078.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
81{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0079.tmp C:\\one-tests\\runs\\capture\\inbox\\0079.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
82{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0080.tmp C:\\one-tests\\runs\\capture\\inbox\\0080.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
83{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0081.tmp C:\\one-tests\\runs\\capture\\inbox\\0081.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
84{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0082.tmp C:\\one-tests\\runs\\capture\\inbox\\0082.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
85{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0083.tmp C:\\one-tests\\runs\\capture\\inbox\\0083.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
86{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0084.tmp C:\\one-tests\\runs\\capture\\inbox\\0084.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
87{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0085.tmp C:\\one-tests\\runs\\capture\\inbox\\0085.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
88{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0086.tmp C:\\one-tests\\runs\\capture\\inbox\\0086.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
89{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0087.tmp C:\\one-tests\\runs\\capture\\inbox\\0087.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
90{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0088.tmp C:\\one-tests\\runs\\capture\\inbox\\0088.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
91{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0089.tmp C:\\one-tests\\runs\\capture\\inbox\\0089.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
92{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0090.tmp C:\\one-tests\\runs\\capture\\inbox\\0090.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
93{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0091.tmp C:\\one-tests\\runs\\capture\\inbox\\0091.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
94{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0092.tmp C:\\one-tests\\runs\\capture\\inbox\\0092.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
95{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0093.tmp C:\\one-tests\\runs\\capture\\inbox\\0093.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
96{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0094.tmp C:\\one-tests\\runs\\capture\\inbox\\0094.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
97{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0095.tmp C:\\one-tests\\runs\\capture\\inbox\\0095.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
98{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0096.tmp C:\\one-tests\\runs\\capture\\inbox\\0096.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
99{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0097.tmp C:\\one-tests\\runs\\capture\\inbox\\0097.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
100{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0098.tmp C:\\one-tests\\runs\\capture\\inbox\\0098.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
101{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0099.tmp C:\\one-tests\\runs\\capture\\inbox\\0099.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
102{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0100.tmp C:\\one-tests\\runs\\capture\\inbox\\0100.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
103{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0101.tmp C:\\one-tests\\runs\\capture\\inbox\\0101.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
104{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0102.tmp C:\\one-tests\\runs\\capture\\inbox\\0102.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
105{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0103.tmp C:\\one-tests\\runs\\capture\\inbox\\0103.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
106{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0104.tmp C:\\one-tests\\runs\\capture\\inbox\\0104.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
107{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0105.tmp C:\\one-tests\\runs\\capture\\inbox\\0105.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
108{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0106.tmp C:\\one-tests\\runs\\capture\\inbox\\0106.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
109{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0107.tmp C:\\one-tests\\runs\\capture\\inbox\\0107.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
110{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0108.tmp C:\\one-tests\\runs\\capture\\inbox\\0108.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
111{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0109.tmp C:\\one-tests\\runs\\capture\\inbox\\0109.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
112{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0110.tmp C:\\one-tests\\runs\\capture\\inbox\\0110.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
113{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0111.tmp C:\\one-tests\\runs\\capture\\inbox\\0111.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
114{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0112.tmp C:\\one-tests\\runs\\capture\\inbox\\0112.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
115{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0113.tmp C:\\one-tests\\runs\\capture\\inbox\\0113.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
116{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0114.tmp C:\\one-tests\\runs\\capture\\inbox\\0114.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
117{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0115.tmp C:\\one-tests\\runs\\capture\\inbox\\0115.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
118{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0116.tmp C:\\one-tests\\runs\\capture\\inbox\\0116.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
119{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0117.tmp C:\\one-tests\\runs\\capture\\inbox\\0117.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
120{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0118.tmp C:\\one-tests\\runs\\capture\\inbox\\0118.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
121{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0119.tmp C:\\one-tests\\runs\\capture\\inbox\\0119.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
122{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0120.tmp C:\\one-tests\\runs\\capture\\inbox\\0120.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
123{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0121.tmp C:\\one-tests\\runs\\capture\\inbox\\0121.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
124{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "ZipArchiveHelper : The process cannot access the file \r\n'C:\\one-tests\\runs\\capture\\controller.log' because it is being used by another \r\nprocess.\r\nAt C:\\Windows\\system32\\WindowsPowerShell\\v1.0\\Modules\\Microsoft.PowerShell.Arch\r\nive\\Microsoft.PowerShell.Archive.psm1:631 char:30\r\n+ ... sArchived = ZipArchiveHelper $sourceFilePaths $destinationPath $compr ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (C:\\one-tests\\runs\\capture\\con \r\n troller.log:String) [Write-Error], IOException\r\n + FullyQualifiedErrorId : CompressArchiveUnauthorizedAccessError,ZipArchiv \r\n eHelper\r\n \r\n", "error": null}
corpus/conflict-page/native/a/machine.json created+1
...@@ -0,0 +1 @@
1{"name": "m6-8a93780a", "hostname": "ONE-M6-8A93780A"}
corpus/conflict-page/native/a/teardown.json created+1
...@@ -0,0 +1 @@
1{"absent": true}
corpus/conflict-page/native/b-published/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/native/b-published/notebook/Open Notebook.onetoc2 differ
corpus/conflict-page/native/b-published/notebook/synthetic.one created
Binary files /dev/null and b/corpus/conflict-page/native/b-published/notebook/synthetic.one differ
corpus/conflict-page/native/b/b-main.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Client B conflicting edit. - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/native/b/b-main.png created
Binary files /dev/null and b/corpus/conflict-page/native/b/b-main.png differ
corpus/conflict-page/native/b/b-shown.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Client A conflicting edit. - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/conflict-page/native/b/b-shown.png created
Binary files /dev/null and b/corpus/conflict-page/native/b/b-shown.png differ
corpus/conflict-page/native/b/commands.jsonl created+21
...@@ -0,0 +1,21 @@
1{"command": "mkdir C:\\one-tests\\runs\\capture", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -ExecutionPolicy Bypass -File C:\\one-tests\\network.ps1 -LabMac 52:54:02:e7:d0:a7", "exit": 0, "stdout": "{\"renewalResult\":0,\"releaseResult\":0,\"mac\":\"52:54:02:e7:d0:a7\",\"address\":\"192.168.77.42\"}\r\n", "stderr": "", "error": null}
3{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0001.tmp C:\\one-tests\\runs\\capture\\inbox\\0001.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
4{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0002.tmp C:\\one-tests\\runs\\capture\\inbox\\0002.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
5{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0003.tmp C:\\one-tests\\runs\\capture\\inbox\\0003.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
6{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0004.tmp C:\\one-tests\\runs\\capture\\inbox\\0004.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
7{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0005.tmp C:\\one-tests\\runs\\capture\\inbox\\0005.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
8{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0006.tmp C:\\one-tests\\runs\\capture\\inbox\\0006.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
9{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0007.tmp C:\\one-tests\\runs\\capture\\inbox\\0007.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
10{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0008.tmp C:\\one-tests\\runs\\capture\\inbox\\0008.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
11{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0009.tmp C:\\one-tests\\runs\\capture\\inbox\\0009.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
12{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0010.tmp C:\\one-tests\\runs\\capture\\inbox\\0010.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
13{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0011.tmp C:\\one-tests\\runs\\capture\\inbox\\0011.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
14{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0012.tmp C:\\one-tests\\runs\\capture\\inbox\\0012.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
15{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0013.tmp C:\\one-tests\\runs\\capture\\inbox\\0013.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
16{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0014.tmp C:\\one-tests\\runs\\capture\\inbox\\0014.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
17{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0015.tmp C:\\one-tests\\runs\\capture\\inbox\\0015.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
18{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0016.tmp C:\\one-tests\\runs\\capture\\inbox\\0016.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
19{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0017.tmp C:\\one-tests\\runs\\capture\\inbox\\0017.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
20{"command": "move C:\\one-tests\\runs\\capture\\inbox\\0018.tmp C:\\one-tests\\runs\\capture\\inbox\\0018.json", "exit": 0, "stdout": " 1 file(s) moved.\r\n", "stderr": "", "error": null}
21{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "ZipArchiveHelper : The process cannot access the file \r\n'C:\\one-tests\\runs\\capture\\controller.log' because it is being used by another \r\nprocess.\r\nAt C:\\Windows\\system32\\WindowsPowerShell\\v1.0\\Modules\\Microsoft.PowerShell.Arch\r\nive\\Microsoft.PowerShell.Archive.psm1:631 char:30\r\n+ ... sArchived = ZipArchiveHelper $sourceFilePaths $destinationPath $compr ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : PermissionDenied: (C:\\one-tests\\runs\\capture\\con \r\n troller.log:String) [Write-Error], IOException\r\n + FullyQualifiedErrorId : CompressArchiveUnauthorizedAccessError,ZipArchiv \r\n eHelper\r\n \r\n", "error": null}
corpus/conflict-page/native/b/machine.json created+1
...@@ -0,0 +1 @@
1{"name": "m6-d63654a2", "hostname": "ONE-M6-D63654A2"}
corpus/conflict-page/native/b/teardown.json created+1
...@@ -0,0 +1 @@
1{"absent": true}
corpus/conflict-page/native/clients.json created+4
...@@ -0,0 +1,4 @@
1{
2 "a": "m6-8a93780a",
3 "b": "m6-d63654a2"
4}
\ No newline at end of file
corpus/conflict-page/native/conflict.json created+15
...@@ -0,0 +1,15 @@
1{
2 "{64183EF4-4E19-0CBE-3D3B-88FF17D58B1C},1": [
3 "Client A disjoint edit.",
4 "Client B conflicting edit.",
5 "Client B disjoint cell.",
6 "Right cell"
7 ],
8 "{9C2E0DE1-2E23-094F-3F9F-A919195FC40D},1": [
9 "",
10 "Client A conflicting edit.",
11 "Client A disjoint edit.",
12 "Left cell",
13 "Right cell"
14 ]
15}
\ No newline at end of file
corpus/conflict-page/native/conflict/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/native/conflict/notebook/Open Notebook.onetoc2 differ
corpus/conflict-page/native/conflict/notebook/synthetic.one created
Binary files /dev/null and b/corpus/conflict-page/native/conflict/notebook/synthetic.one differ
corpus/conflict-page/native/initial/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/conflict-page/native/initial/notebook/Open Notebook.onetoc2 differ
corpus/conflict-page/native/initial/notebook/synthetic.one created
Binary files /dev/null and b/corpus/conflict-page/native/initial/notebook/synthetic.one differ
corpus/conflict-page/native/linux-teardown.json created+1
...@@ -0,0 +1 @@
1{"absent": true}
corpus/conflict-page/native/linux.json created+11
...@@ -0,0 +1,11 @@
1{
2 "cpus": 2,
3 "disk_gb": 16,
4 "hostname": "one-conflict",
5 "lab_address": "192.168.77.1",
6 "lab_mac": "52:54:04:69:39:00",
7 "memory_mb": 2048,
8 "samba_port": 14450,
9 "ssh_port": 22000,
10 "wan_mac": "52:54:03:76:a2:ae"
11}
\ No newline at end of file
corpus/conflict-page/native/native_conflict.py created+261
...@@ -0,0 +1,261 @@
1#!/usr/bin/env python3
2"""OneNote 2010 conflict pages: how they arise, what they store and how OneNote shows them.
3
4`sync LINUX OUTPUT` runs two disposable OneNote clients on one Samba notebook: client A edits
5two paragraphs offline while client B edits one of them and a table cell online; A then
6reconnects, which makes OneNote write a conflict page. The server file is kept before and
7after, A and B are screenshotted with the information bar, the page list shown, the conflict
8page open and its menu open, and A then deletes the conflict page through that menu.
9
10`cold NOTEBOOK OUTPUT [--delete]` cold-opens a notebook holding a conflict page in a fresh
11clone and screenshots the same views; `--delete` deletes the conflict page through the menu
12and captures the notebook OneNote leaves.
13"""
14import argparse
15from contextlib import ExitStack
16from concurrent.futures import ThreadPoolExecutor
17import base64
18import json
19from pathlib import Path
20import shutil
21import subprocess
22import sys
23import time
24import xml.etree.ElementTree as ET
25
26sys.path.insert(0, str(Path(__file__).resolve().parent))
27sys.path.insert(0, str(Path(__file__).resolve().parent / 'w7'))
28import native_runner as runner
29from native_runner import ROOT, clone, command, windows, vm
30import linux_vm
31
32FIXTURE = ROOT / 'corpus/native-ink/cold-ui-ink/notebook'
33SHARE = 'm6-conflict'
34ORIGINAL = 'Fictitious: café, 東京, مرحبا'
35OTHER_OUTLINE = 'Fictitious positioned outline.'
36CELL = 'Left cell'
37
38# Coordinates on OneNote 2010 maximized at 800x600, from the stage-5 and m8 captures: the
39# information bar's text, and the first conflict page listed under its page.
40BAR = (332, 100)
41FIRST_CONFLICT = (725, 142)
42
43
44def shot(name, output, label, script):
45 """Runs an AutoHotkey step on the clone, keeping its screenshot as `label.png`."""
46 (output / f'{label}.ahk').write_text(script)
47 result = windows.do_exec(script, target=name, timeout_ms=120000, shot_delay_ms=1500)
48 screenshot = result.pop('png_b64', None)
49 if screenshot:
50 (output / f'{label}.png').write_bytes(base64.b64decode(screenshot))
51 (output / f'{label}.json').write_text(json.dumps(result, indent=2))
52 if result.get('error') or result.get('exit') != 0:
53 raise RuntimeError(f'{label}: {result}')
54 return result
55
56
57def click(x, y):
58 return ('WinActivate("ahk_exe ONENOTE.EXE")\nCoordMode("Mouse", "Screen")\n'
59 f'Click({x}, {y})\nSleep(1200)\n')
60
61
62# Reports any dialog OneNote raised, so a confirmation shows in the step's record.
63DIALOG = ('if WinExist("ahk_class #32770 ahk_exe ONENOTE.EXE") {\n'
64 ' FileAppend("dialog: " WinGetTitle() " | " WinGetText() "`n", "*", "UTF-8")\n'
65 '}\n')
66
67
68def conflict_views(name, output, page, prefix, delete=False):
69 """Screenshots a page with a conflict page: the bar, the list shown, the conflict page and
70 its menu; `delete` then chooses Delete Conflict Page."""
71 shot(name, output, f'{prefix}-main', runner.navigation_script(page) + 'Sleep(2000)\n')
72 shot(name, output, f'{prefix}-shown', click(*BAR))
73 shot(name, output, f'{prefix}-conflict', click(*FIRST_CONFLICT))
74 shot(name, output, f'{prefix}-menu', click(*BAR))
75 if delete:
76 shot(name, output, f'{prefix}-delete', 'WinActivate("ahk_exe ONENOTE.EXE")\nSend("d")\nSleep(1500)\n' + DIALOG)
77 else:
78 shot(name, output, f'{prefix}-closed', 'WinActivate("ahk_exe ONENOTE.EXE")\nSend("{Esc}")\nSleep(800)\n')
79
80
81def cold(notebook, output, delete):
82 def interaction(name, output):
83 page = ET.parse(sorted((output / 'read').glob('page-*.xml'))[0]).getroot().attrib['ID']
84 conflict_views(name, output, page, 'cold', delete)
85 if delete:
86 # OneNote writes the deletion in the background; give it time before the re-read.
87 time.sleep(20)
88 runner.capture(notebook, output, collect_notebook=True, interaction=interaction)
89
90
91def sync(server, output):
92 from native_collaboration import reachable_page_text
93 from document_model import EXPORTER
94
95 if linux_vm.instance_path(server).exists():
96 raise ValueError('Choose a new Linux VM name; existing machines are not owned by this run.')
97 output = output.resolve()
98 output.mkdir(parents=True, exist_ok=False)
99 scripts = output / 'scripts'
100 scripts.mkdir()
101 for name in ('cold.ps1', 'collaborate.ps1', 'network.ps1', 'text.ps1', 'stress.ps1'):
102 shutil.copyfile(ROOT / 'tools/native' / name, scripts / name)
103 shutil.copyfile(Path(__file__), output / 'native_conflict.py')
104
105 def ssh(text, timeout=90):
106 result = linux_vm.run_ssh(server, text, timeout=timeout)
107 with (output / 'server.jsonl').open('a') as log:
108 log.write(json.dumps({'command': text, 'exit': result.returncode, 'stdout': result.stdout, 'stderr': result.stderr}) + '\n')
109 result.check_returncode()
110 return result.stdout
111
112 def server_state(label):
113 """Copies the server's notebook until it parses, keeping it as `label/notebook`."""
114 deadline = time.monotonic() + 120
115 while True:
116 target = output / label
117 shutil.rmtree(target, ignore_errors=True)
118 (target / 'notebook').mkdir(parents=True)
119 for name in ('synthetic.one', 'Open Notebook.onetoc2'):
120 with (target / 'notebook' / name).open('wb') as stream:
121 subprocess.run(linux_vm.ssh_argv(server, f"cat '/srv/agent/{SHARE}/{name}'"), stdout=stream, check=True, timeout=60)
122 result = subprocess.run([EXPORTER, target / 'notebook/synthetic.one', target / 'model'], capture_output=True, text=True)
123 if result.returncode == 0:
124 model = json.loads((target / 'model/document.json').read_text())
125 return reachable_page_text(model)
126 if time.monotonic() > deadline:
127 raise RuntimeError(f'{label}: {result.stderr}')
128 time.sleep(1)
129
130 linux_vm.create_instance(server)
131 try:
132 linux_vm.launch(server)
133 linux_vm.wait_instance(server, 600)
134 (output / 'linux.json').write_text(json.dumps(linux_vm.load_instance(server), indent=2))
135 ssh(f'mkdir /srv/agent/{SHARE}')
136 for name in ('synthetic.one', 'Open Notebook.onetoc2'):
137 with (FIXTURE / name).open('rb') as stream:
138 subprocess.run(linux_vm.ssh_argv(server, f"cat > '/srv/agent/{SHARE}/{name}'"), stdin=stream, check=True)
139 ssh(f'chmod u+w /srv/agent/{SHARE}/*')
140 with ExitStack() as stack:
141 labels = ['a', 'b']
142 for label in labels:
143 (output / label).mkdir()
144
145 def start(label):
146 manager = clone(output / label)
147 name = manager.__enter__()
148 return label, name, manager
149 with ThreadPoolExecutor(max_workers=2) as pool:
150 started = list(pool.map(start, labels))
151 for _, _, manager in started:
152 stack.push(manager)
153 clients = {}
154 for label, name, _ in started:
155 folder = output / label
156 for local in scripts.iterdir():
157 remote = 'cold-current.ps1' if local.name == 'cold.ps1' else local.name
158 result = windows.do_put(local, 'C:\\one-tests\\' + remote, name)
159 if result.get('error'): raise RuntimeError(result['error'])
160 command(name, 'mkdir C:\\one-tests\\runs\\capture', folder)
161 command(name, 'powershell -NoProfile -ExecutionPolicy Bypass -File C:\\one-tests\\network.ps1 -LabMac ' + vm.lab_mac(name), folder)
162 result = windows.do_spawn('cmd /c powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\collaborate.ps1 -Root C:\\one-tests\\runs\\capture -SharedPath \\\\192.168.77.1\\agent\\' + SHARE + ' -CloneHost ONE-' + name.upper() + ' > C:\\one-tests\\runs\\capture\\controller.log 2>&1', name)
163 if result.get('error'): raise RuntimeError(result['error'])
164 deadline = time.monotonic() + 180
165 while windows.do_cmd('if exist C:\\one-tests\\runs\\capture\\ready (echo ready)', target=name).get('stdout', '').find('ready') < 0:
166 if time.monotonic() > deadline: raise RuntimeError('The collaboration controller did not open the notebook')
167 time.sleep(1)
168 clients[label] = {'name': name, 'folder': folder, 'sequence': 0}
169 (output / 'clients.json').write_text(json.dumps({label: client['name'] for label, client in clients.items()}, indent=2))
170
171 def action(client, kind, **parameters):
172 client['sequence'] += 1
173 sequence = client['sequence']
174 local = client['folder'] / f'command-{sequence:04}.json'
175 local.write_text(json.dumps({'action': kind, **parameters}, ensure_ascii=False))
176 inbox = f'C:\\one-tests\\runs\\capture\\inbox\\{sequence:04}'
177 result = windows.do_put(local, inbox + '.tmp', client['name'])
178 if result.get('error'): raise RuntimeError(result['error'])
179 command(client['name'], f'move {inbox}.tmp {inbox}.json', client['folder'])
180 remote = f'C:\\one-tests\\runs\\capture\\outbox\\{sequence}'
181 deadline = time.monotonic() + 600
182 while True:
183 result = windows.do_cmd(f'if exist {remote}\\error (type {remote}\\error & exit /b 1) else (if exist {remote}\\done (echo complete))', target=client['name'])
184 if result.get('exit') != 0: raise RuntimeError(str(result))
185 if 'complete' in result.get('stdout', ''): break
186 if time.monotonic() > deadline: raise TimeoutError(f'{kind} did not complete')
187 time.sleep(.5)
188 if kind != 'snapshot':
189 return None
190 hierarchy = client['folder'] / f'hierarchy-{sequence:04}.xml'
191 windows.do_get(remote + '\\hierarchy.xml', hierarchy, client['name'])
192 pages = [node for node in ET.fromstring(hierarchy.read_text(encoding='utf-8-sig').strip()).iter() if node.tag.endswith('}Page')]
193 for i in range(len(pages)):
194 windows.do_get(remote + f'\\page-{i}.xml', client['folder'] / f'snapshot-{sequence:04}-{i}.xml', client['name'])
195 return pages
196
197 def settle(client, check, label):
198 deadline = time.monotonic() + 300
199 while True:
200 action(client, 'sync')
201 state = server_state(label)
202 if check(state): return state
203 if time.monotonic() > deadline: raise TimeoutError(f'{label}: {state}')
204 time.sleep(2)
205
206 a, b = clients['a'], clients['b']
207 texts = lambda state: {text for values in state.values() for text in values}
208 server_state('initial')
209 vm.qmp(a['name'], 'set_link', {'name': 'lab', 'up': False})
210 try:
211 action(a, 'edit', expected=ORIGINAL, text='Client A conflicting edit.')
212 action(a, 'edit', expected=OTHER_OUTLINE, text='Client A disjoint edit.')
213 action(a, 'snapshot')
214 action(b, 'edit', expected=ORIGINAL, text='Client B conflicting edit.')
215 action(b, 'edit', expected=CELL, text='Client B disjoint cell.')
216 settle(b, lambda state: {'Client B conflicting edit.', 'Client B disjoint cell.'} <= texts(state), 'b-published')
217 finally:
218 vm.qmp(a['name'], 'set_link', {'name': 'lab', 'up': True})
219 state = settle(a, lambda state: len(state) > 1 and 'Client A disjoint edit.' in texts(state), 'conflict')
220 (output / 'conflict.json').write_text(json.dumps(state, indent=2, ensure_ascii=False))
221 page = action(a, 'snapshot')[0].attrib['ID']
222 conflict_views(a['name'], output / 'a', page, 'a')
223 action(b, 'sync')
224 time.sleep(5)
225 page_b = action(b, 'snapshot')[0].attrib['ID']
226 conflict_views(b['name'], output / 'b', page_b, 'b')
227 conflict_views(a['name'], output / 'a', page, 'a-again', delete=True)
228 state = settle(a, lambda state: len(state) == 1, 'deleted')
229 (output / 'deleted.json').write_text(json.dumps(state, indent=2, ensure_ascii=False))
230 action(b, 'sync')
231 time.sleep(5)
232 shot(b['name'], output / 'b', 'b-after-delete', runner.navigation_script(page_b) + 'Sleep(2000)\n')
233 for client in clients.values():
234 try:
235 action(client, 'close')
236 except Exception as error:
237 (client['folder'] / 'close-failure.txt').write_text(str(error))
238 server_state('final')
239 finally:
240 if linux_vm.running(server):
241 linux_vm.shutdown(server, 120)
242 linux_vm.delete_instance(server)
243 (output / 'linux-teardown.json').write_text(json.dumps({'absent': not linux_vm.instance_path(server).exists()}) + '\n')
244
245
246if __name__ == '__main__':
247 parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
248 commands = parser.add_subparsers(dest='mode', required=True)
249 lab = commands.add_parser('sync')
250 lab.add_argument('linux')
251 lab.add_argument('output', type=Path)
252 reopen = commands.add_parser('cold')
253 reopen.add_argument('notebook', type=Path)
254 reopen.add_argument('output', type=Path)
255 reopen.add_argument('--delete', action='store_true')
256 args = parser.parse_args()
257 if args.mode == 'sync':
258 sync(args.linux, args.output)
259 else:
260 cold(args.notebook, args.output, args.delete)
261 print('Captured', args.output)
corpus/conflict-page/native/scripts/cold.ps1 created+27
...@@ -0,0 +1,27 @@
1param([Parameter(Mandatory=$true)][string]$Root, [string]$CloneHost = '')
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
5if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
6 throw 'Choose a run directly below C:\one-tests\runs.'
7}
8if (Get-Process ONENOTE -ErrorAction SilentlyContinue) { throw 'Close OneNote before resetting its test cache.' }
9$key = 'HKCU:\Software\Microsoft\Office\14.0\OneNote'
10if ($CloneHost) {
11 if ($CloneHost -notmatch '^ONE-[A-Z0-9-]+$' -or [Environment]::MachineName -ne $CloneHost) {
12 throw 'The disposable clone hostname does not match this machine.'
13 }
14 New-Item "$key\Options\Paths" -Force | Out-Null
15 New-ItemProperty "$key\Options\Paths" -Name UnfiledNotesSection -PropertyType ExpandString -Value 'C:\one-tests\Loose.one' -Force | Out-Null
16} elseif ((Get-ItemProperty "$key\Options\Paths").UnfiledNotesSection -ne 'C:\one-tests\Loose.one' -or
17 -not (Test-Path 'C:\one-tests\profile-original-cache')) {
18 throw 'Park the personal OneNote profile before resetting the test cache.'
19}
20$cache = Join-Path $env:LOCALAPPDATA 'Microsoft\OneNote\14.0'
21$parked = Join-Path 'C:\one-tests\caches' ([IO.Path]::GetFileName($root))
22if (Test-Path $parked) { throw 'Choose a new run; its parked cache already exists.' }
23New-Item -ItemType Directory -Path 'C:\one-tests\caches' -Force | Out-Null
24if (Test-Path $cache) { Move-Item -LiteralPath $cache -Destination $parked }
25if (Test-Path "$key\OpenNotebooks") { Remove-Item "$key\OpenNotebooks" -Recurse }
26New-Item "$key\OpenNotebooks" | Out-Null
27New-ItemProperty "$key\OpenNotebooks" -Name '1' -PropertyType String -Value "$root\notebook" | Out-Null
corpus/conflict-page/native/scripts/collaborate.ps1 created+105
...@@ -0,0 +1,105 @@
1param([Parameter(Mandatory=$true)][string]$Root,
2 [Parameter(Mandatory=$true)][string]$SharedPath,
3 [Parameter(Mandatory=$true)][string]$CloneHost,
4 [switch]$ResumeCache,
5 [int]$StartSequence = 1)
6function Keep-Container($page, $target) {
7 $container = $target
8 while ($container.ParentNode -ne $page.DocumentElement) { $container = $container.ParentNode }
9 foreach ($sibling in @($page.DocumentElement.ChildNodes)) {
10 if ($sibling -ne $container -and $sibling.LocalName -in @('Outline', 'Title', 'Image', 'InkDrawing')) {
11 [void]$page.DocumentElement.RemoveChild($sibling)
12 }
13 }
14}
15Set-StrictMode -Version Latest
16$ErrorActionPreference = 'Stop'
17. "$PSScriptRoot\text.ps1"
18if ($SharedPath -notmatch '^\\\\192\.168\.77\.1\\agent\\m6-[a-z0-9-]+$') { throw 'Choose a disposable Linux lab notebook.' }
19if ($env:COMPUTERNAME -ne $CloneHost) { throw 'The collaboration controller requires its owned clone.' }
20if (-not $ResumeCache) { & "$PSScriptRoot\cold-current.ps1" -Root $Root -CloneHost $CloneHost }
21$shared = $SharedPath
22if (-not (Test-Path "$shared\synthetic.one")) { throw 'Create the shared fixture first.' }
23New-Item -ItemType Directory "$Root\inbox", "$Root\outbox" -Force | Out-Null
24$user = 'HKCU:\Software\Microsoft\Office\Common\UserInfo'
25New-Item $user -Force | Out-Null
26New-ItemProperty $user -Name UserName -Value $CloneHost -PropertyType String -Force | Out-Null
27New-ItemProperty $user -Name UserInitials -Value $CloneHost.Substring($CloneHost.Length - 3) -PropertyType String -Force | Out-Null
28$app = New-Object -ComObject OneNote.Application
29$notebook = ''
30try {
31 $app.OpenHierarchy($shared, '', [ref]$notebook, 0)
32 $section = ''
33 $app.OpenHierarchy('synthetic.one', $notebook, [ref]$section, 0)
34 [IO.File]::WriteAllText("$Root\ready", 'ready')
35 $deadline = [DateTime]::UtcNow.AddMinutes(30)
36 $sequence = $StartSequence
37 while ([DateTime]::UtcNow -lt $deadline) {
38 $name = '{0:d4}.json' -f $sequence
39 $input = "$Root\inbox\$name"
40 if (-not (Test-Path $input)) { Start-Sleep -Milliseconds 100; continue }
41 $command = Get-Content -Raw -Encoding UTF8 $input | ConvertFrom-Json
42 $output = "$Root\outbox\$sequence"
43 New-Item -ItemType Directory $output | Out-Null
44 try {
45 if ($command.action -in @('prepare-stress', 'stress')) {
46 . "$PSScriptRoot\stress.ps1"
47 Invoke-Stress $app $section $command $output $shared
48 }
49 elseif ($command.action -eq 'sync') { $app.SyncHierarchy($notebook) }
50 elseif ($command.action -eq 'kill-process') {
51 Get-Process ONENOTE | Stop-Process -Force
52 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
53 $app = New-Object -ComObject OneNote.Application
54 $app.OpenHierarchy($shared, '', [ref]$notebook, 0)
55 $app.OpenHierarchy('synthetic.one', $notebook, [ref]$section, 0)
56 }
57 elseif ($command.action -eq 'edit' -or $command.action -eq 'snapshot') {
58 $hierarchy = ''
59 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
60 [IO.File]::WriteAllText("$output\hierarchy.xml", $hierarchy, [Text.Encoding]::UTF8)
61 [xml]$tree = $hierarchy
62 $matches = 0
63 $editedPage = $null
64 $i = 0
65 foreach ($node in $tree.SelectNodes('//*[local-name()="Page"]')) {
66 $content = ''
67 $app.GetPageContent($node.GetAttribute('ID'), [ref]$content, 1, 1)
68 [IO.File]::WriteAllText("$output\page-$i.xml", $content, [Text.Encoding]::UTF8)
69 [xml]$page = $content
70 if ($command.action -eq 'edit') {
71 $targets = @($page.SelectNodes('//*[local-name()="T"]') | Where-Object {
72 (Get-ParagraphText $_.InnerText) -eq $command.expected
73 })
74 if ($targets.Count -gt 1) { throw 'Expected one matching text object.' }
75 if ($targets.Count -eq 1) {
76 $matches++
77 if ($matches -gt 1) { throw 'The text occurs on multiple pages.' }
78 $targets[0].InnerText = [Security.SecurityElement]::Escape($command.text)
79 Keep-Container $page $targets[0]
80 $editedPage = $page
81 }
82 }
83 $i++
84 }
85 if ($command.action -eq 'edit' -and $matches -ne 1) { throw 'Expected one matching text object.' }
86 if ($editedPage) { $app.UpdatePageContent($editedPage.OuterXml, [DateTime]::MinValue, 1, $false) }
87 } elseif ($command.action -ne 'close') { throw 'Unknown collaboration command.' }
88 [IO.File]::WriteAllText("$output\done", 'ok')
89 } catch {
90 [IO.File]::WriteAllText("$output\error", $_.Exception.ToString())
91 }
92 if ($command.action -eq 'close') { break }
93 $sequence++
94 }
95} finally {
96 try { if ($notebook) { $app.CloseNotebook($notebook, $false) } }
97 finally {
98 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
99 $app = $null
100 [GC]::Collect()
101 [GC]::WaitForPendingFinalizers()
102 Get-Process ONENOTE -ErrorAction SilentlyContinue | Wait-Process -Timeout 10
103 [IO.File]::WriteAllText("$Root\closed", 'closed')
104 }
105}
corpus/conflict-page/native/scripts/network.ps1 created+20
...@@ -0,0 +1,20 @@
1param([Parameter(Mandatory=$true)][string]$LabMac)
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$adapter = @(Get-WmiObject Win32_NetworkAdapterConfiguration | Where-Object { $_.MACAddress -eq $LabMac })
5if ($adapter.Count -ne 1) { throw 'The clone lab adapter is ambiguous.' }
6$enabled = $adapter[0].EnableDHCP()
7if ($enabled.ReturnValue -notin @(0, 1)) {
8 throw ('Unable to enable DHCP on the clone lab adapter; result ' + $enabled.ReturnValue)
9}
10$release = $adapter[0].ReleaseDHCPLease()
11$result = $adapter[0].RenewDHCPLease()
12$deadline = [DateTime]::UtcNow.AddSeconds(60)
13do {
14 $adapter = @(Get-WmiObject Win32_NetworkAdapterConfiguration | Where-Object { $_.MACAddress -eq $LabMac })
15 $addresses = @($adapter[0].IPAddress | Where-Object { $_ -match '^192\.168\.77\.\d+$' -and $_ -ne '192.168.77.1' })
16 if ($addresses.Count -eq 1) { break }
17 Start-Sleep -Milliseconds 250
18} while ([DateTime]::UtcNow -lt $deadline)
19if ($addresses.Count -ne 1) { throw ('The clone did not receive a lab IPv4 address; renewal result ' + $result.ReturnValue + '; adapter ' + ($adapter[0] | Select-Object MACAddress,DHCPEnabled,IPEnabled,IPAddress | ConvertTo-Json -Compress)) }
20@{mac=$LabMac;address=$addresses[0];releaseResult=$release.ReturnValue;renewalResult=$result.ReturnValue} | ConvertTo-Json -Compress
corpus/conflict-page/native/scripts/stress.ps1 created+83
...@@ -0,0 +1,83 @@
1function Invoke-Stress($app, $section, $command, $output, $shared) {
2 $hierarchy = ''
3 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
4 [IO.File]::WriteAllText("$output\hierarchy.xml", $hierarchy, [Text.Encoding]::UTF8)
5 [xml]$tree = $hierarchy
6 $pages = @($tree.SelectNodes('//*[local-name()="Page"]'))
7 if ($pages.Count -ne 1) { throw 'Expected one stress-test page.' }
8 $pageId = $pages[0].GetAttribute('ID')
9 if ($command.action -eq 'prepare-stress') {
10 for ($i = 0; $i -lt $command.clients; $i++) {
11 $xml = '<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="' + $pageId + '"><one:Outline><one:Position x="48" y="' + (150 + 75 * $i) + '" z="' + ($i + 1) + '"/><one:OEChildren><one:OE><one:T><![CDATA[Native ' + $i + ':]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>'
12 $app.UpdatePageContent($xml, [DateTime]::MinValue, 1, $false)
13 }
14 return
15 }
16 $random = New-Object Random([int]$command.seed)
17 $events = New-Object IO.StreamWriter("$output\events.jsonl", $false, [Text.Encoding]::UTF8)
18 $events.AutoFlush = $true
19 try {
20 [IO.File]::WriteAllText("$output\ready", 'ready')
21 $deadline = [DateTime]::UtcNow.AddMinutes(10)
22 while (-not (Test-Path "$shared\stress-start")) {
23 if ([DateTime]::UtcNow -gt $deadline) { throw 'Stress start timed out.' }
24 Start-Sleep -Milliseconds 20
25 }
26 for ($i = 0; $i -lt $command.operations; $i++) {
27 if (($command.PSObject.Properties.Name -contains 'maintenance') -and $command.maintenance -and $i -eq [Math]::Floor($command.operations / 2)) {
28 [IO.File]::WriteAllText("$shared\maintenance-paused-n$($command.actor)", 'paused')
29 while (-not (Test-Path "$shared\maintenance-resume")) {
30 if ([DateTime]::UtcNow -gt $deadline) { throw 'Maintenance pause timed out.' }
31 Start-Sleep -Milliseconds 100
32 }
33 }
34 Start-Sleep -Milliseconds $random.Next(10, 100)
35 $started = [DateTime]::UtcNow.Ticks
36 $content = ''
37 $app.GetPageContent($pageId, [ref]$content, 1, 1)
38 [xml]$page = $content
39 $matches = @($page.SelectNodes('//*[local-name()="T"]') | Where-Object {
40 (Get-ParagraphText $_.InnerText).StartsWith($command.prefix)
41 })
42 if ($matches.Count -ne 1) { throw 'Expected one stress-test paragraph.' }
43 $before = Get-ParagraphText $matches[0].InnerText
44 $token = ' [n' + $command.actor + ':' + $i + ']'
45 $start = $end = $before.Length
46 $replacement = $token
47 $bold = $italic = $false
48 if ($command.edit) {
49 $boundaries = New-Object 'Collections.Generic.List[int]'
50 for ($at = $command.prefix.Length; $at -le $before.Length; $at++) {
51 $boundaries.Add($at)
52 if ($at -lt $before.Length -and [char]::IsHighSurrogate($before[$at])) { $at++ }
53 }
54 $first = $boundaries[$random.Next($boundaries.Count)]
55 $second = $boundaries[$random.Next($boundaries.Count)]
56 $start = [Math]::Min($first, $second)
57 $end = [Math]::Max($first, $second)
58 $replacement = ' caf' + [char]0xe9 + ' ' + [char]::ConvertFromUtf32(0x1f980) + $token
59 $bold = $random.Next(2) -eq 1
60 $italic = $random.Next(2) -eq 1
61 }
62 $after = $before.Substring(0, $start) + $replacement + $before.Substring($end)
63 $html = [Security.SecurityElement]::Escape($after)
64 if ($command.edit) {
65 $weight = if ($bold) { 'bold' } else { 'normal' }
66 $style = if ($italic) { 'italic' } else { 'normal' }
67 $html = '<span style="font-weight:' + $weight + ';font-style:' + $style + '">' + $html + '</span>'
68 }
69 $matches[0].InnerText = $html
70 Keep-Container $page $matches[0]
71 $updateStarted = [DateTime]::UtcNow.Ticks
72 $app.UpdatePageContent($page.OuterXml, [DateTime]::MinValue, 1, $false)
73 $updated = [DateTime]::UtcNow.Ticks
74 $events.WriteLine((@{ operation=$i; before=$before; token=$token; started_ticks=$started;
75 range=@($start, $end); replacement=$replacement; bold=$bold; italic=$italic;
76 update_started_ticks=$updateStarted; updated_ticks=$updated } | ConvertTo-Json -Compress))
77 if ($i -eq 0) { [IO.File]::WriteAllText("$output\editing", 'editing') }
78 if ($command.sync_every -gt 0 -and (($i + 1) % $command.sync_every) -eq 0) {
79 $app.SyncHierarchy($section)
80 }
81 }
82 } finally { $events.Dispose() }
83}
corpus/conflict-page/native/scripts/text.ps1 created+13
...@@ -0,0 +1,13 @@
1function Get-ParagraphText([string]$html) {
2 $plain = [regex]::Replace($html, '<[^>]*>', '')
3 $plain = [regex]::Replace($plain, '&#(?:x([0-9a-fA-F]+)|([0-9]+));', {
4 param($match)
5 $value = if ($match.Groups[1].Success) {
6 [Convert]::ToInt32($match.Groups[1].Value, 16)
7 } else { [int]$match.Groups[2].Value }
8 # The Win7 framework leaves supplementary numeric entities undecoded.
9 if ($value -gt 0xffff -and $value -le 0x10ffff) { [char]::ConvertFromUtf32($value) }
10 else { $match.Value }
11 })
12 [Net.WebUtility]::HtmlDecode($plain)
13}
corpus/conflict-page/native/server.jsonl created+2
...@@ -0,0 +1,2 @@
1{"command": "mkdir /srv/agent/m6-conflict", "exit": 0, "stdout": "", "stderr": ""}
2{"command": "chmod u+w /srv/agent/m6-conflict/*", "exit": 0, "stdout": "", "stderr": ""}
corpus/extended-ascii/README.md created+9
...@@ -0,0 +1,9 @@
1# Extended ASCII text
2
3`TextExtendedAscii` (`0x1c003498`, MS-ONE 2.2.89) names no code page. Each `candidate`
4is `m6/ansi/notebook` with the paragraph's `RichEditTextLangID` (`0x10001cfe`) set to
50x0409 (`en`), 0x0419 (`ru`, ANSI code page 1251) or 0x0411 (`ja`, code page 932); its
6text holds the bytes 0x80–0xFF. `cold` is each one's fresh OneNote 2010 read: all three
7export U+0080–U+00FF, so OneNote reads the bytes as Latin-1 whatever the language, and
8`read/page-000.png` shows 0x80–0x9F as nothing and 0xC0 as À rather than Cyrillic А.
9OneNote left the section files unchanged.
corpus/extended-ascii/en/candidate/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/extended-ascii/en/candidate/Open Notebook.onetoc2 differ
corpus/extended-ascii/en/candidate/synthetic.one created
Binary files /dev/null and b/corpus/extended-ascii/en/candidate/synthetic.one differ
corpus/extended-ascii/en/cold/commands.jsonl created+3
...@@ -0,0 +1,3 @@
1{"command": "powershell -NoProfile -Command \"Expand-Archive -LiteralPath C:\\one-tests\\transfer.zip -DestinationPath C:\\one-tests\\runs\\capture\\notebook\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-A375E105 -ExpectedPages 1 -KeepOpen", "exit": 0, "stdout": "Read 1 sections and 1 pages.\r\n", "stderr": "", "error": null}
3{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
corpus/extended-ascii/en/cold/machine.json created+1
...@@ -0,0 +1 @@
1{"name": "m6-a375e105", "hostname": "ONE-M6-A375E105"}
corpus/extended-ascii/en/cold/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/extended-ascii/en/cold/notebook/Open Notebook.onetoc2 differ
corpus/extended-ascii/en/cold/notebook/synthetic.one created
Binary files /dev/null and b/corpus/extended-ascii/en/cold/notebook/synthetic.one differ
corpus/extended-ascii/en/cold/read/environment.json created+7
...@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-A375E105",
5 "cold": true,
6 "onenote": "14.0.4763.1000"
7}
corpus/extended-ascii/en/cold/read/hierarchy.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{EE076CAD-41AC-4A4A-8929-025A57318F7D}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-27T05:33:41.000Z" color="#9595AA"><one:Section name="synthetic" ID="{AF80AC8F-943F-0D15-0824-72674F116D2B}{1}{B0}" path="C:\one-tests\runs\capture\notebook\synthetic.one" lastModifiedTime="2026-09-27T05:33:41.000Z" color="#8AA8E4"><one:Page ID="{83C95F09-1DF4-4E04-8F43-DAF6B3297134}{1}{B0}" name="Fictitious plain text." dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-27T05:33:41.000Z" pageLevel="1"/></one:Section></one:Notebook>
corpus/extended-ascii/en/cold/read/page-000.navigation.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Extended bytes: \u0080 \u0081 \u0082 \u0083 \u0084 \u0085 \u0086 \u0087 \u0088 \u0089 \u008a \u008b \u008c \u008d \u008e \u008f \u0090 - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/extended-ascii/en/cold/read/page-000.png created
Binary files /dev/null and b/corpus/extended-ascii/en/cold/read/page-000.png differ
corpus/extended-ascii/en/cold/read/page-000.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{83C95F09-1DF4-4E04-8F43-DAF6B3297134}{1}{B0}" name="Fictitious plain text." dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-27T05:33:41.000Z" pageLevel="1" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-05T05:06:55.000Z" objectID="{E405F7B5-5681-4DB2-80B1-562D4CB59455}{10}{B0}"><one:Position x="36.0" y="14.40000057220459" z="0"/><one:Size width="468.0" height="40.28314971923828"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:55.000Z" lastModifiedTime="2026-09-05T05:06:55.000Z" objectID="{E405F7B5-5681-4DB2-80B1-562D4CB59455}{11}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Extended bytes: €  ‚ ƒ „ … † ‡ ˆ ‰ Š ‹ Œ  Ž   ‘ ’ “ ” • – — ˜ ™ š › œ  ž Ÿ &nbsp; ¡ ¢ £ ¤ ¥ ¦ § ¨ © ª « ¬ ­ ® ¯ ° ± ² ³ ´ µ ¶ · ¸ ¹ º » ¼ ½ ¾ ¿ À Á Â Ã Ä Å Æ Ç È É Ê Ë Ì Í Î Ï Ð Ñ Ò Ó Ô Õ Ö × Ø Ù Ú Û Ü Ý Þ ß à á â ã ä å æ ç è é ê ë ì í î ï ð ñ ò ó ô õ ö ÷ ø ù ú û ü ý þ ÿ]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/extended-ascii/en/cold/read/payloads.json created+3
...@@ -0,0 +1,3 @@
1[
2
3]
\ No newline at end of file
corpus/extended-ascii/en/cold/run.json created+18
...@@ -0,0 +1,18 @@
1{
2 "notebook": "/private/tmp/ansi-lang/en/notebook",
3 "expected_pages": 1,
4 "author": null,
5 "author_timeout_seconds": 600,
6 "inspect": false,
7 "collect_notebook": true,
8 "base": {
9 "file": "win7-office-base.qcow2",
10 "format": "qcow2",
11 "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346",
12 "virtual_size": 68719476736
13 },
14 "scripts": {
15 "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331",
16 "read.ps1": "04013bfcccee40a17e2a225f9b9e96f40a3a8daad9e350609654f8eda3ccbb41"
17 }
18}
corpus/extended-ascii/en/cold/scripts/cold.ps1 created+27
...@@ -0,0 +1,27 @@
1param([Parameter(Mandatory=$true)][string]$Root, [string]$CloneHost = '')
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
5if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
6 throw 'Choose a run directly below C:\one-tests\runs.'
7}
8if (Get-Process ONENOTE -ErrorAction SilentlyContinue) { throw 'Close OneNote before resetting its test cache.' }
9$key = 'HKCU:\Software\Microsoft\Office\14.0\OneNote'
10if ($CloneHost) {
11 if ($CloneHost -notmatch '^ONE-[A-Z0-9-]+$' -or [Environment]::MachineName -ne $CloneHost) {
12 throw 'The disposable clone hostname does not match this machine.'
13 }
14 New-Item "$key\Options\Paths" -Force | Out-Null
15 New-ItemProperty "$key\Options\Paths" -Name UnfiledNotesSection -PropertyType ExpandString -Value 'C:\one-tests\Loose.one' -Force | Out-Null
16} elseif ((Get-ItemProperty "$key\Options\Paths").UnfiledNotesSection -ne 'C:\one-tests\Loose.one' -or
17 -not (Test-Path 'C:\one-tests\profile-original-cache')) {
18 throw 'Park the personal OneNote profile before resetting the test cache.'
19}
20$cache = Join-Path $env:LOCALAPPDATA 'Microsoft\OneNote\14.0'
21$parked = Join-Path 'C:\one-tests\caches' ([IO.Path]::GetFileName($root))
22if (Test-Path $parked) { throw 'Choose a new run; its parked cache already exists.' }
23New-Item -ItemType Directory -Path 'C:\one-tests\caches' -Force | Out-Null
24if (Test-Path $cache) { Move-Item -LiteralPath $cache -Destination $parked }
25if (Test-Path "$key\OpenNotebooks") { Remove-Item "$key\OpenNotebooks" -Recurse }
26New-Item "$key\OpenNotebooks" | Out-Null
27New-ItemProperty "$key\OpenNotebooks" -Name '1' -PropertyType String -Value "$root\notebook" | Out-Null
corpus/extended-ascii/en/cold/scripts/read.ps1 created+142
...@@ -0,0 +1,142 @@
1param(
2 [Parameter(Mandatory=$true)][string]$Root,
3 [int]$ExpectedPages = -1,
4 [switch]$UseCurrentCache,
5 [switch]$Pdf,
6 [switch]$KeepOpen,
7 [string]$CloneHost = ''
8)
9Set-StrictMode -Version Latest
10$ErrorActionPreference = 'Stop'
11$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
12if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
13 throw 'Choose a run directly below C:\one-tests\runs.'
14}
15$notebook = Join-Path $root 'notebook'
16$output = Join-Path $root 'read'
17if (Test-Path $output) { throw 'Choose a new read destination.' }
18if ($UseCurrentCache) {
19 if ((Get-ItemProperty 'HKCU:\Software\Microsoft\Office\14.0\OneNote\Options\Paths').UnfiledNotesSection -ne 'C:\one-tests\Loose.one') {
20 throw 'Park the personal OneNote profile before reading test notebooks.'
21 }
22} else {
23 & "$PSScriptRoot\cold-current.ps1" -Root $root -CloneHost $CloneHost
24}
25New-Item -ItemType Directory -Path $output | Out-Null
26$app = New-Object -ComObject OneNote.Application
27$notebookId = ''
28$failure = $null
29try {
30 $app.OpenHierarchy($notebook, '', [ref]$notebookId, 0)
31 $process = Get-Process ONENOTE
32 @{ hostname = [Environment]::MachineName; onenote = $process.MainModule.FileVersionInfo.FileVersion;
33 powershell = $PSVersionTable.PSVersion.ToString(); schema = 'xs2010'; cold = (-not $UseCurrentCache.IsPresent) } |
34 ConvertTo-Json | Set-Content (Join-Path $output 'environment.json') -Encoding UTF8
35 $sections = @()
36 foreach ($file in @(Get-ChildItem $notebook -Recurse | Where-Object { $_.Extension -eq '.one' })) {
37 $id = ''
38 $app.OpenHierarchy($file.FullName, '', [ref]$id, 0)
39 $sections += $id
40 }
41 $deadline = [DateTime]::UtcNow.AddSeconds(300)
42 $previous = ''
43 $lastChange = ''
44 $stableSince = [DateTime]::UtcNow
45 $settled = $false
46 do {
47 $pages = @{}
48 foreach ($section in $sections) {
49 $hierarchy = ''
50 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
51 [xml]$xml = $hierarchy
52 foreach ($node in $xml.SelectNodes('//*[@path]')) {
53 if (-not $node.GetAttribute('path').StartsWith("$notebook\", [StringComparison]::OrdinalIgnoreCase)) {
54 throw 'OneNote opened a section outside the copied notebook.'
55 }
56 }
57 foreach ($node in $xml.SelectNodes('//*[local-name()="Page"]')) {
58 $id = $node.GetAttribute('ID')
59 $content = ''
60 $app.GetPageContent($id, [ref]$content, 1, 1)
61 $pages[$id] = $content
62 }
63 }
64 $signature = [String]::Join('|', @($pages.Keys | Sort-Object | ForEach-Object { $_ + $pages[$_] }))
65 if ($signature -ne $previous) {
66 $lastChange = $previous
67 $previous = $signature
68 $stableSince = [DateTime]::UtcNow
69 }
70 if ((($ExpectedPages -ge 0 -and $pages.Count -eq $ExpectedPages) -or
71 ($ExpectedPages -lt 0 -and $pages.Count -gt 0)) -and
72 ([DateTime]::UtcNow - $stableSince).TotalSeconds -ge 2) { $settled = $true; break }
73 Start-Sleep -Milliseconds 250
74 } while ([DateTime]::UtcNow -lt $deadline)
75 if (-not $settled -or ($ExpectedPages -ge 0 -and $pages.Count -ne $ExpectedPages) -or ($ExpectedPages -lt 0 -and $pages.Count -eq 0)) {
76 [IO.File]::WriteAllText((Join-Path $output 'previous-signature.txt'), $lastChange, [Text.Encoding]::UTF8)
77 $index = 0
78 foreach ($id in @($pages.Keys | Sort-Object)) {
79 [IO.File]::WriteAllText((Join-Path $output ('unsettled-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
80 $index++
81 }
82 $hierarchy = ''
83 $app.GetHierarchy($notebookId, 4, [ref]$hierarchy, 1)
84 [IO.File]::WriteAllText((Join-Path $output 'unsettled-hierarchy.xml'), $hierarchy, [Text.Encoding]::UTF8)
85 throw "Expected $ExpectedPages stable pages; OneNote returned $($pages.Count), settled=$settled."
86 }
87 $index = 0
88 $payloads = @()
89 foreach ($id in @($pages.Keys | Sort-Object)) {
90 [IO.File]::WriteAllText((Join-Path $output ('page-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
91 if ($Pdf) {
92 $pdfPath = Join-Path $output ('page-{0:d3}.pdf' -f $index)
93 $app.NavigateTo($id, '', $false)
94 $app.Publish($id, $pdfPath, 3, '')
95 if (-not (Test-Path $pdfPath) -or (Get-Item $pdfPath).Length -eq 0) { throw 'OneNote did not publish the page PDF.' }
96 }
97 [xml]$page = $pages[$id]
98 foreach ($file in $page.SelectNodes('//*[local-name()="InsertedFile" or local-name()="MediaFile"]')) {
99 $bytes = [IO.File]::ReadAllBytes($file.GetAttribute('pathCache'))
100 $hash = [BitConverter]::ToString([Security.Cryptography.SHA256]::Create().ComputeHash($bytes)).Replace('-', '').ToLowerInvariant()
101 [IO.File]::WriteAllBytes((Join-Path $output ($hash + '.attachment')), $bytes)
102 $payloads += @{ page = $id; object = $file.ParentNode.GetAttribute('objectID');
103 kind = $file.LocalName; name = $file.GetAttribute('preferredName');
104 sha256 = $hash; bytes = $bytes.Length }
105 }
106 $index++
107 }
108 [IO.File]::WriteAllText((Join-Path $output 'payloads.json'), (ConvertTo-Json -InputObject $payloads -Depth 4), [Text.Encoding]::UTF8)
109 $all = ''
110 $app.GetHierarchy($notebookId, 4, [ref]$all, 1)
111 [xml]$finalTree = $all
112 $finalIds = @($finalTree.SelectNodes('//*[local-name()="Page"]') | ForEach-Object { $_.GetAttribute('ID') } | Sort-Object -Unique)
113 if ($finalIds.Count -ne $pages.Count -or @($finalIds | Where-Object { -not $pages.ContainsKey($_) }).Count -ne 0) {
114 throw 'The notebook hierarchy changed while collecting page evidence; repeat the cold read.'
115 }
116 [IO.File]::WriteAllText((Join-Path $output 'hierarchy.xml'), $all, [Text.Encoding]::UTF8)
117 Write-Output "Read $($sections.Count) sections and $($pages.Count) pages."
118} catch {
119 $failure = $_
120 [IO.File]::WriteAllText((Join-Path $output 'failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
121 throw
122} finally {
123 try {
124 try {
125 if ($notebookId -and $CloneHost) { $app.SyncHierarchy($notebookId) }
126 if ($notebookId -and -not $KeepOpen -and (-not $UseCurrentCache -or $CloneHost)) {
127 $app.CloseNotebook($notebookId, $false)
128 }
129 } catch {
130 if ($null -eq $failure) { throw }
131 [IO.File]::WriteAllText((Join-Path $output 'cleanup-failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
132 }
133 } finally {
134 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
135 }
136 $app = $null
137 [GC]::Collect()
138 [GC]::WaitForPendingFinalizers()
139 if (-not $UseCurrentCache -and -not $CloneHost) {
140 Get-Process ONENOTE -ErrorAction SilentlyContinue | Wait-Process -Timeout 10
141 }
142}
corpus/extended-ascii/en/cold/source.json created+14
...@@ -0,0 +1,14 @@
1[
2 {
3 "path": "Open Notebook.onetoc2",
4 "bytes": 4816,
5 "sha256": "b2f857961b76258a1582b402c048b340e89962257681426b3bfc58df8e0adbfa",
6 "mtime_ns": 1790487072902156985
7 },
8 {
9 "path": "synthetic.one",
10 "bytes": 12456,
11 "sha256": "aa6f1188773f1cc41cc5c619f213ce04b30592320c6b2e32b2b4c91639423ab4",
12 "mtime_ns": 1790487072902335569
13 }
14]
corpus/extended-ascii/en/cold/teardown.json created+1
...@@ -0,0 +1 @@
1{"absent": true}
corpus/extended-ascii/ja/candidate/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/extended-ascii/ja/candidate/Open Notebook.onetoc2 differ
corpus/extended-ascii/ja/candidate/synthetic.one created
Binary files /dev/null and b/corpus/extended-ascii/ja/candidate/synthetic.one differ
corpus/extended-ascii/ja/cold/commands.jsonl created+3
...@@ -0,0 +1,3 @@
1{"command": "powershell -NoProfile -Command \"Expand-Archive -LiteralPath C:\\one-tests\\transfer.zip -DestinationPath C:\\one-tests\\runs\\capture\\notebook\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-8DC7DCC7 -ExpectedPages 1 -KeepOpen", "exit": 0, "stdout": "Read 1 sections and 1 pages.\r\n", "stderr": "", "error": null}
3{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
corpus/extended-ascii/ja/cold/machine.json created+1
...@@ -0,0 +1 @@
1{"name": "m6-8dc7dcc7", "hostname": "ONE-M6-8DC7DCC7"}
corpus/extended-ascii/ja/cold/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/extended-ascii/ja/cold/notebook/Open Notebook.onetoc2 differ
corpus/extended-ascii/ja/cold/notebook/synthetic.one created
Binary files /dev/null and b/corpus/extended-ascii/ja/cold/notebook/synthetic.one differ
corpus/extended-ascii/ja/cold/read/environment.json created+7
...@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-8DC7DCC7",
5 "cold": true,
6 "onenote": "14.0.4763.1000"
7}
corpus/extended-ascii/ja/cold/read/hierarchy.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{5B6DFD6A-ECAF-406E-B615-008B44758AA3}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-27T05:37:38.000Z" color="#9595AA"><one:Section name="synthetic" ID="{3B11134F-882C-06AE-07C4-197CA488888D}{1}{B0}" path="C:\one-tests\runs\capture\notebook\synthetic.one" lastModifiedTime="2026-09-27T05:37:38.000Z" color="#8AA8E4"><one:Page ID="{1758E0C9-01E7-45BF-80A3-B1ED58B09492}{1}{B0}" name="Fictitious plain text." dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-27T05:37:38.000Z" pageLevel="1"/></one:Section></one:Notebook>
corpus/extended-ascii/ja/cold/read/page-000.navigation.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Extended bytes: \u0080 \u0081 \u0082 \u0083 \u0084 \u0085 \u0086 \u0087 \u0088 \u0089 \u008a \u008b \u008c \u008d \u008e \u008f \u0090 - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/extended-ascii/ja/cold/read/page-000.png created
Binary files /dev/null and b/corpus/extended-ascii/ja/cold/read/page-000.png differ
corpus/extended-ascii/ja/cold/read/page-000.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{1758E0C9-01E7-45BF-80A3-B1ED58B09492}{1}{B0}" name="Fictitious plain text." dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-27T05:37:38.000Z" pageLevel="1" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-05T05:06:55.000Z" objectID="{70944875-4A92-4609-8F51-3D36A72C71F3}{10}{B0}"><one:Position x="36.0" y="14.40000057220459" z="0"/><one:Size width="468.0" height="40.28314971923828"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:55.000Z" lastModifiedTime="2026-09-05T05:06:55.000Z" objectID="{70944875-4A92-4609-8F51-3D36A72C71F3}{11}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Extended bytes: €  ‚ ƒ „ … † ‡ ˆ ‰ Š ‹ Œ  Ž   ‘ ’ “ ” • – — ˜ ™ š › œ  ž Ÿ &nbsp; ¡ ¢ £ ¤ ¥ ¦ § ¨ © ª « ¬ ­ ® ¯ ° ± ² ³ ´ µ ¶ · ¸ ¹ º » ¼ ½ ¾ ¿ À Á Â Ã Ä Å Æ Ç È É Ê Ë Ì Í Î Ï Ð Ñ Ò Ó Ô Õ Ö × Ø Ù Ú Û Ü Ý Þ ß à á â ã ä å æ ç è é ê ë ì í î ï ð ñ ò ó ô õ ö ÷ ø ù ú û ü ý þ ÿ]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/extended-ascii/ja/cold/read/payloads.json created+3
...@@ -0,0 +1,3 @@
1[
2
3]
\ No newline at end of file
corpus/extended-ascii/ja/cold/run.json created+18
...@@ -0,0 +1,18 @@
1{
2 "notebook": "/private/tmp/ansi-lang/ja/notebook",
3 "expected_pages": 1,
4 "author": null,
5 "author_timeout_seconds": 600,
6 "inspect": false,
7 "collect_notebook": true,
8 "base": {
9 "file": "win7-office-base.qcow2",
10 "format": "qcow2",
11 "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346",
12 "virtual_size": 68719476736
13 },
14 "scripts": {
15 "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331",
16 "read.ps1": "04013bfcccee40a17e2a225f9b9e96f40a3a8daad9e350609654f8eda3ccbb41"
17 }
18}
corpus/extended-ascii/ja/cold/scripts/cold.ps1 created+27
...@@ -0,0 +1,27 @@
1param([Parameter(Mandatory=$true)][string]$Root, [string]$CloneHost = '')
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
5if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
6 throw 'Choose a run directly below C:\one-tests\runs.'
7}
8if (Get-Process ONENOTE -ErrorAction SilentlyContinue) { throw 'Close OneNote before resetting its test cache.' }
9$key = 'HKCU:\Software\Microsoft\Office\14.0\OneNote'
10if ($CloneHost) {
11 if ($CloneHost -notmatch '^ONE-[A-Z0-9-]+$' -or [Environment]::MachineName -ne $CloneHost) {
12 throw 'The disposable clone hostname does not match this machine.'
13 }
14 New-Item "$key\Options\Paths" -Force | Out-Null
15 New-ItemProperty "$key\Options\Paths" -Name UnfiledNotesSection -PropertyType ExpandString -Value 'C:\one-tests\Loose.one' -Force | Out-Null
16} elseif ((Get-ItemProperty "$key\Options\Paths").UnfiledNotesSection -ne 'C:\one-tests\Loose.one' -or
17 -not (Test-Path 'C:\one-tests\profile-original-cache')) {
18 throw 'Park the personal OneNote profile before resetting the test cache.'
19}
20$cache = Join-Path $env:LOCALAPPDATA 'Microsoft\OneNote\14.0'
21$parked = Join-Path 'C:\one-tests\caches' ([IO.Path]::GetFileName($root))
22if (Test-Path $parked) { throw 'Choose a new run; its parked cache already exists.' }
23New-Item -ItemType Directory -Path 'C:\one-tests\caches' -Force | Out-Null
24if (Test-Path $cache) { Move-Item -LiteralPath $cache -Destination $parked }
25if (Test-Path "$key\OpenNotebooks") { Remove-Item "$key\OpenNotebooks" -Recurse }
26New-Item "$key\OpenNotebooks" | Out-Null
27New-ItemProperty "$key\OpenNotebooks" -Name '1' -PropertyType String -Value "$root\notebook" | Out-Null
corpus/extended-ascii/ja/cold/scripts/read.ps1 created+142
...@@ -0,0 +1,142 @@
1param(
2 [Parameter(Mandatory=$true)][string]$Root,
3 [int]$ExpectedPages = -1,
4 [switch]$UseCurrentCache,
5 [switch]$Pdf,
6 [switch]$KeepOpen,
7 [string]$CloneHost = ''
8)
9Set-StrictMode -Version Latest
10$ErrorActionPreference = 'Stop'
11$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
12if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
13 throw 'Choose a run directly below C:\one-tests\runs.'
14}
15$notebook = Join-Path $root 'notebook'
16$output = Join-Path $root 'read'
17if (Test-Path $output) { throw 'Choose a new read destination.' }
18if ($UseCurrentCache) {
19 if ((Get-ItemProperty 'HKCU:\Software\Microsoft\Office\14.0\OneNote\Options\Paths').UnfiledNotesSection -ne 'C:\one-tests\Loose.one') {
20 throw 'Park the personal OneNote profile before reading test notebooks.'
21 }
22} else {
23 & "$PSScriptRoot\cold-current.ps1" -Root $root -CloneHost $CloneHost
24}
25New-Item -ItemType Directory -Path $output | Out-Null
26$app = New-Object -ComObject OneNote.Application
27$notebookId = ''
28$failure = $null
29try {
30 $app.OpenHierarchy($notebook, '', [ref]$notebookId, 0)
31 $process = Get-Process ONENOTE
32 @{ hostname = [Environment]::MachineName; onenote = $process.MainModule.FileVersionInfo.FileVersion;
33 powershell = $PSVersionTable.PSVersion.ToString(); schema = 'xs2010'; cold = (-not $UseCurrentCache.IsPresent) } |
34 ConvertTo-Json | Set-Content (Join-Path $output 'environment.json') -Encoding UTF8
35 $sections = @()
36 foreach ($file in @(Get-ChildItem $notebook -Recurse | Where-Object { $_.Extension -eq '.one' })) {
37 $id = ''
38 $app.OpenHierarchy($file.FullName, '', [ref]$id, 0)
39 $sections += $id
40 }
41 $deadline = [DateTime]::UtcNow.AddSeconds(300)
42 $previous = ''
43 $lastChange = ''
44 $stableSince = [DateTime]::UtcNow
45 $settled = $false
46 do {
47 $pages = @{}
48 foreach ($section in $sections) {
49 $hierarchy = ''
50 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
51 [xml]$xml = $hierarchy
52 foreach ($node in $xml.SelectNodes('//*[@path]')) {
53 if (-not $node.GetAttribute('path').StartsWith("$notebook\", [StringComparison]::OrdinalIgnoreCase)) {
54 throw 'OneNote opened a section outside the copied notebook.'
55 }
56 }
57 foreach ($node in $xml.SelectNodes('//*[local-name()="Page"]')) {
58 $id = $node.GetAttribute('ID')
59 $content = ''
60 $app.GetPageContent($id, [ref]$content, 1, 1)
61 $pages[$id] = $content
62 }
63 }
64 $signature = [String]::Join('|', @($pages.Keys | Sort-Object | ForEach-Object { $_ + $pages[$_] }))
65 if ($signature -ne $previous) {
66 $lastChange = $previous
67 $previous = $signature
68 $stableSince = [DateTime]::UtcNow
69 }
70 if ((($ExpectedPages -ge 0 -and $pages.Count -eq $ExpectedPages) -or
71 ($ExpectedPages -lt 0 -and $pages.Count -gt 0)) -and
72 ([DateTime]::UtcNow - $stableSince).TotalSeconds -ge 2) { $settled = $true; break }
73 Start-Sleep -Milliseconds 250
74 } while ([DateTime]::UtcNow -lt $deadline)
75 if (-not $settled -or ($ExpectedPages -ge 0 -and $pages.Count -ne $ExpectedPages) -or ($ExpectedPages -lt 0 -and $pages.Count -eq 0)) {
76 [IO.File]::WriteAllText((Join-Path $output 'previous-signature.txt'), $lastChange, [Text.Encoding]::UTF8)
77 $index = 0
78 foreach ($id in @($pages.Keys | Sort-Object)) {
79 [IO.File]::WriteAllText((Join-Path $output ('unsettled-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
80 $index++
81 }
82 $hierarchy = ''
83 $app.GetHierarchy($notebookId, 4, [ref]$hierarchy, 1)
84 [IO.File]::WriteAllText((Join-Path $output 'unsettled-hierarchy.xml'), $hierarchy, [Text.Encoding]::UTF8)
85 throw "Expected $ExpectedPages stable pages; OneNote returned $($pages.Count), settled=$settled."
86 }
87 $index = 0
88 $payloads = @()
89 foreach ($id in @($pages.Keys | Sort-Object)) {
90 [IO.File]::WriteAllText((Join-Path $output ('page-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
91 if ($Pdf) {
92 $pdfPath = Join-Path $output ('page-{0:d3}.pdf' -f $index)
93 $app.NavigateTo($id, '', $false)
94 $app.Publish($id, $pdfPath, 3, '')
95 if (-not (Test-Path $pdfPath) -or (Get-Item $pdfPath).Length -eq 0) { throw 'OneNote did not publish the page PDF.' }
96 }
97 [xml]$page = $pages[$id]
98 foreach ($file in $page.SelectNodes('//*[local-name()="InsertedFile" or local-name()="MediaFile"]')) {
99 $bytes = [IO.File]::ReadAllBytes($file.GetAttribute('pathCache'))
100 $hash = [BitConverter]::ToString([Security.Cryptography.SHA256]::Create().ComputeHash($bytes)).Replace('-', '').ToLowerInvariant()
101 [IO.File]::WriteAllBytes((Join-Path $output ($hash + '.attachment')), $bytes)
102 $payloads += @{ page = $id; object = $file.ParentNode.GetAttribute('objectID');
103 kind = $file.LocalName; name = $file.GetAttribute('preferredName');
104 sha256 = $hash; bytes = $bytes.Length }
105 }
106 $index++
107 }
108 [IO.File]::WriteAllText((Join-Path $output 'payloads.json'), (ConvertTo-Json -InputObject $payloads -Depth 4), [Text.Encoding]::UTF8)
109 $all = ''
110 $app.GetHierarchy($notebookId, 4, [ref]$all, 1)
111 [xml]$finalTree = $all
112 $finalIds = @($finalTree.SelectNodes('//*[local-name()="Page"]') | ForEach-Object { $_.GetAttribute('ID') } | Sort-Object -Unique)
113 if ($finalIds.Count -ne $pages.Count -or @($finalIds | Where-Object { -not $pages.ContainsKey($_) }).Count -ne 0) {
114 throw 'The notebook hierarchy changed while collecting page evidence; repeat the cold read.'
115 }
116 [IO.File]::WriteAllText((Join-Path $output 'hierarchy.xml'), $all, [Text.Encoding]::UTF8)
117 Write-Output "Read $($sections.Count) sections and $($pages.Count) pages."
118} catch {
119 $failure = $_
120 [IO.File]::WriteAllText((Join-Path $output 'failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
121 throw
122} finally {
123 try {
124 try {
125 if ($notebookId -and $CloneHost) { $app.SyncHierarchy($notebookId) }
126 if ($notebookId -and -not $KeepOpen -and (-not $UseCurrentCache -or $CloneHost)) {
127 $app.CloseNotebook($notebookId, $false)
128 }
129 } catch {
130 if ($null -eq $failure) { throw }
131 [IO.File]::WriteAllText((Join-Path $output 'cleanup-failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
132 }
133 } finally {
134 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
135 }
136 $app = $null
137 [GC]::Collect()
138 [GC]::WaitForPendingFinalizers()
139 if (-not $UseCurrentCache -and -not $CloneHost) {
140 Get-Process ONENOTE -ErrorAction SilentlyContinue | Wait-Process -Timeout 10
141 }
142}
corpus/extended-ascii/ja/cold/source.json created+14
...@@ -0,0 +1,14 @@
1[
2 {
3 "path": "Open Notebook.onetoc2",
4 "bytes": 4816,
5 "sha256": "b2f857961b76258a1582b402c048b340e89962257681426b3bfc58df8e0adbfa",
6 "mtime_ns": 1790487072903313695
7 },
8 {
9 "path": "synthetic.one",
10 "bytes": 12456,
11 "sha256": "b792b045a793ef97ded2a9d6ab456070fb5d9d3f48763067ed5e24bde482358a",
12 "mtime_ns": 1790487072903438737
13 }
14]
corpus/extended-ascii/ja/cold/teardown.json created+1
...@@ -0,0 +1 @@
1{"absent": true}
corpus/extended-ascii/ru/candidate/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/extended-ascii/ru/candidate/Open Notebook.onetoc2 differ
corpus/extended-ascii/ru/candidate/synthetic.one created
Binary files /dev/null and b/corpus/extended-ascii/ru/candidate/synthetic.one differ
corpus/extended-ascii/ru/cold/commands.jsonl created+3
...@@ -0,0 +1,3 @@
1{"command": "powershell -NoProfile -Command \"Expand-Archive -LiteralPath C:\\one-tests\\transfer.zip -DestinationPath C:\\one-tests\\runs\\capture\\notebook\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-2B8202A3 -ExpectedPages 1 -KeepOpen", "exit": 0, "stdout": "Read 1 sections and 1 pages.\r\n", "stderr": "", "error": null}
3{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
corpus/extended-ascii/ru/cold/machine.json created+1
...@@ -0,0 +1 @@
1{"name": "m6-2b8202a3", "hostname": "ONE-M6-2B8202A3"}
corpus/extended-ascii/ru/cold/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/extended-ascii/ru/cold/notebook/Open Notebook.onetoc2 differ
corpus/extended-ascii/ru/cold/notebook/synthetic.one created
Binary files /dev/null and b/corpus/extended-ascii/ru/cold/notebook/synthetic.one differ
corpus/extended-ascii/ru/cold/read/environment.json created+7
...@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-2B8202A3",
5 "cold": true,
6 "onenote": "14.0.4763.1000"
7}
corpus/extended-ascii/ru/cold/read/hierarchy.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{6A0D391B-07F4-4CA2-B500-0760660A4E33}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-27T05:33:41.000Z" color="#9595AA"><one:Section name="synthetic" ID="{7392C207-21C6-0CE2-3853-9AD75F71833E}{1}{B0}" path="C:\one-tests\runs\capture\notebook\synthetic.one" lastModifiedTime="2026-09-27T05:33:41.000Z" color="#8AA8E4"><one:Page ID="{5FDB3181-A80D-4FF3-BF34-3246A3499F21}{1}{B0}" name="Fictitious plain text." dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-27T05:33:41.000Z" pageLevel="1"/></one:Section></one:Notebook>
corpus/extended-ascii/ru/cold/read/page-000.navigation.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Extended bytes: \u0080 \u0081 \u0082 \u0083 \u0084 \u0085 \u0086 \u0087 \u0088 \u0089 \u008a \u008b \u008c \u008d \u008e \u008f \u0090 - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/extended-ascii/ru/cold/read/page-000.png created
Binary files /dev/null and b/corpus/extended-ascii/ru/cold/read/page-000.png differ
corpus/extended-ascii/ru/cold/read/page-000.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{5FDB3181-A80D-4FF3-BF34-3246A3499F21}{1}{B0}" name="Fictitious plain text." dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-27T05:33:41.000Z" pageLevel="1" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-05T05:06:55.000Z" objectID="{3817993D-E378-4C45-B0C6-BE9D5CD57A40}{10}{B0}"><one:Position x="36.0" y="14.40000057220459" z="0"/><one:Size width="468.0" height="40.28314971923828"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:55.000Z" lastModifiedTime="2026-09-05T05:06:55.000Z" objectID="{3817993D-E378-4C45-B0C6-BE9D5CD57A40}{11}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Extended bytes: €  ‚ ƒ „ … † ‡ ˆ ‰ Š ‹ Œ  Ž   ‘ ’ “ ” • – — ˜ ™ š › œ  ž Ÿ &nbsp; ¡ ¢ £ ¤ ¥ ¦ § ¨ © ª « ¬ ­ ® ¯ ° ± ² ³ ´ µ ¶ · ¸ ¹ º » ¼ ½ ¾ ¿ À Á Â Ã Ä Å Æ Ç È É Ê Ë Ì Í Î Ï Ð Ñ Ò Ó Ô Õ Ö × Ø Ù Ú Û Ü Ý Þ ß à á â ã ä å æ ç è é ê ë ì í î ï ð ñ ò ó ô õ ö ÷ ø ù ú û ü ý þ ÿ]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/extended-ascii/ru/cold/read/payloads.json created+3
...@@ -0,0 +1,3 @@
1[
2
3]
\ No newline at end of file
corpus/extended-ascii/ru/cold/run.json created+18
...@@ -0,0 +1,18 @@
1{
2 "notebook": "/private/tmp/ansi-lang/ru/notebook",
3 "expected_pages": 1,
4 "author": null,
5 "author_timeout_seconds": 600,
6 "inspect": false,
7 "collect_notebook": true,
8 "base": {
9 "file": "win7-office-base.qcow2",
10 "format": "qcow2",
11 "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346",
12 "virtual_size": 68719476736
13 },
14 "scripts": {
15 "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331",
16 "read.ps1": "04013bfcccee40a17e2a225f9b9e96f40a3a8daad9e350609654f8eda3ccbb41"
17 }
18}
corpus/extended-ascii/ru/cold/scripts/cold.ps1 created+27
...@@ -0,0 +1,27 @@
1param([Parameter(Mandatory=$true)][string]$Root, [string]$CloneHost = '')
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
5if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
6 throw 'Choose a run directly below C:\one-tests\runs.'
7}
8if (Get-Process ONENOTE -ErrorAction SilentlyContinue) { throw 'Close OneNote before resetting its test cache.' }
9$key = 'HKCU:\Software\Microsoft\Office\14.0\OneNote'
10if ($CloneHost) {
11 if ($CloneHost -notmatch '^ONE-[A-Z0-9-]+$' -or [Environment]::MachineName -ne $CloneHost) {
12 throw 'The disposable clone hostname does not match this machine.'
13 }
14 New-Item "$key\Options\Paths" -Force | Out-Null
15 New-ItemProperty "$key\Options\Paths" -Name UnfiledNotesSection -PropertyType ExpandString -Value 'C:\one-tests\Loose.one' -Force | Out-Null
16} elseif ((Get-ItemProperty "$key\Options\Paths").UnfiledNotesSection -ne 'C:\one-tests\Loose.one' -or
17 -not (Test-Path 'C:\one-tests\profile-original-cache')) {
18 throw 'Park the personal OneNote profile before resetting the test cache.'
19}
20$cache = Join-Path $env:LOCALAPPDATA 'Microsoft\OneNote\14.0'
21$parked = Join-Path 'C:\one-tests\caches' ([IO.Path]::GetFileName($root))
22if (Test-Path $parked) { throw 'Choose a new run; its parked cache already exists.' }
23New-Item -ItemType Directory -Path 'C:\one-tests\caches' -Force | Out-Null
24if (Test-Path $cache) { Move-Item -LiteralPath $cache -Destination $parked }
25if (Test-Path "$key\OpenNotebooks") { Remove-Item "$key\OpenNotebooks" -Recurse }
26New-Item "$key\OpenNotebooks" | Out-Null
27New-ItemProperty "$key\OpenNotebooks" -Name '1' -PropertyType String -Value "$root\notebook" | Out-Null
corpus/extended-ascii/ru/cold/scripts/read.ps1 created+142
...@@ -0,0 +1,142 @@
1param(
2 [Parameter(Mandatory=$true)][string]$Root,
3 [int]$ExpectedPages = -1,
4 [switch]$UseCurrentCache,
5 [switch]$Pdf,
6 [switch]$KeepOpen,
7 [string]$CloneHost = ''
8)
9Set-StrictMode -Version Latest
10$ErrorActionPreference = 'Stop'
11$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
12if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
13 throw 'Choose a run directly below C:\one-tests\runs.'
14}
15$notebook = Join-Path $root 'notebook'
16$output = Join-Path $root 'read'
17if (Test-Path $output) { throw 'Choose a new read destination.' }
18if ($UseCurrentCache) {
19 if ((Get-ItemProperty 'HKCU:\Software\Microsoft\Office\14.0\OneNote\Options\Paths').UnfiledNotesSection -ne 'C:\one-tests\Loose.one') {
20 throw 'Park the personal OneNote profile before reading test notebooks.'
21 }
22} else {
23 & "$PSScriptRoot\cold-current.ps1" -Root $root -CloneHost $CloneHost
24}
25New-Item -ItemType Directory -Path $output | Out-Null
26$app = New-Object -ComObject OneNote.Application
27$notebookId = ''
28$failure = $null
29try {
30 $app.OpenHierarchy($notebook, '', [ref]$notebookId, 0)
31 $process = Get-Process ONENOTE
32 @{ hostname = [Environment]::MachineName; onenote = $process.MainModule.FileVersionInfo.FileVersion;
33 powershell = $PSVersionTable.PSVersion.ToString(); schema = 'xs2010'; cold = (-not $UseCurrentCache.IsPresent) } |
34 ConvertTo-Json | Set-Content (Join-Path $output 'environment.json') -Encoding UTF8
35 $sections = @()
36 foreach ($file in @(Get-ChildItem $notebook -Recurse | Where-Object { $_.Extension -eq '.one' })) {
37 $id = ''
38 $app.OpenHierarchy($file.FullName, '', [ref]$id, 0)
39 $sections += $id
40 }
41 $deadline = [DateTime]::UtcNow.AddSeconds(300)
42 $previous = ''
43 $lastChange = ''
44 $stableSince = [DateTime]::UtcNow
45 $settled = $false
46 do {
47 $pages = @{}
48 foreach ($section in $sections) {
49 $hierarchy = ''
50 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
51 [xml]$xml = $hierarchy
52 foreach ($node in $xml.SelectNodes('//*[@path]')) {
53 if (-not $node.GetAttribute('path').StartsWith("$notebook\", [StringComparison]::OrdinalIgnoreCase)) {
54 throw 'OneNote opened a section outside the copied notebook.'
55 }
56 }
57 foreach ($node in $xml.SelectNodes('//*[local-name()="Page"]')) {
58 $id = $node.GetAttribute('ID')
59 $content = ''
60 $app.GetPageContent($id, [ref]$content, 1, 1)
61 $pages[$id] = $content
62 }
63 }
64 $signature = [String]::Join('|', @($pages.Keys | Sort-Object | ForEach-Object { $_ + $pages[$_] }))
65 if ($signature -ne $previous) {
66 $lastChange = $previous
67 $previous = $signature
68 $stableSince = [DateTime]::UtcNow
69 }
70 if ((($ExpectedPages -ge 0 -and $pages.Count -eq $ExpectedPages) -or
71 ($ExpectedPages -lt 0 -and $pages.Count -gt 0)) -and
72 ([DateTime]::UtcNow - $stableSince).TotalSeconds -ge 2) { $settled = $true; break }
73 Start-Sleep -Milliseconds 250
74 } while ([DateTime]::UtcNow -lt $deadline)
75 if (-not $settled -or ($ExpectedPages -ge 0 -and $pages.Count -ne $ExpectedPages) -or ($ExpectedPages -lt 0 -and $pages.Count -eq 0)) {
76 [IO.File]::WriteAllText((Join-Path $output 'previous-signature.txt'), $lastChange, [Text.Encoding]::UTF8)
77 $index = 0
78 foreach ($id in @($pages.Keys | Sort-Object)) {
79 [IO.File]::WriteAllText((Join-Path $output ('unsettled-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
80 $index++
81 }
82 $hierarchy = ''
83 $app.GetHierarchy($notebookId, 4, [ref]$hierarchy, 1)
84 [IO.File]::WriteAllText((Join-Path $output 'unsettled-hierarchy.xml'), $hierarchy, [Text.Encoding]::UTF8)
85 throw "Expected $ExpectedPages stable pages; OneNote returned $($pages.Count), settled=$settled."
86 }
87 $index = 0
88 $payloads = @()
89 foreach ($id in @($pages.Keys | Sort-Object)) {
90 [IO.File]::WriteAllText((Join-Path $output ('page-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
91 if ($Pdf) {
92 $pdfPath = Join-Path $output ('page-{0:d3}.pdf' -f $index)
93 $app.NavigateTo($id, '', $false)
94 $app.Publish($id, $pdfPath, 3, '')
95 if (-not (Test-Path $pdfPath) -or (Get-Item $pdfPath).Length -eq 0) { throw 'OneNote did not publish the page PDF.' }
96 }
97 [xml]$page = $pages[$id]
98 foreach ($file in $page.SelectNodes('//*[local-name()="InsertedFile" or local-name()="MediaFile"]')) {
99 $bytes = [IO.File]::ReadAllBytes($file.GetAttribute('pathCache'))
100 $hash = [BitConverter]::ToString([Security.Cryptography.SHA256]::Create().ComputeHash($bytes)).Replace('-', '').ToLowerInvariant()
101 [IO.File]::WriteAllBytes((Join-Path $output ($hash + '.attachment')), $bytes)
102 $payloads += @{ page = $id; object = $file.ParentNode.GetAttribute('objectID');
103 kind = $file.LocalName; name = $file.GetAttribute('preferredName');
104 sha256 = $hash; bytes = $bytes.Length }
105 }
106 $index++
107 }
108 [IO.File]::WriteAllText((Join-Path $output 'payloads.json'), (ConvertTo-Json -InputObject $payloads -Depth 4), [Text.Encoding]::UTF8)
109 $all = ''
110 $app.GetHierarchy($notebookId, 4, [ref]$all, 1)
111 [xml]$finalTree = $all
112 $finalIds = @($finalTree.SelectNodes('//*[local-name()="Page"]') | ForEach-Object { $_.GetAttribute('ID') } | Sort-Object -Unique)
113 if ($finalIds.Count -ne $pages.Count -or @($finalIds | Where-Object { -not $pages.ContainsKey($_) }).Count -ne 0) {
114 throw 'The notebook hierarchy changed while collecting page evidence; repeat the cold read.'
115 }
116 [IO.File]::WriteAllText((Join-Path $output 'hierarchy.xml'), $all, [Text.Encoding]::UTF8)
117 Write-Output "Read $($sections.Count) sections and $($pages.Count) pages."
118} catch {
119 $failure = $_
120 [IO.File]::WriteAllText((Join-Path $output 'failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
121 throw
122} finally {
123 try {
124 try {
125 if ($notebookId -and $CloneHost) { $app.SyncHierarchy($notebookId) }
126 if ($notebookId -and -not $KeepOpen -and (-not $UseCurrentCache -or $CloneHost)) {
127 $app.CloseNotebook($notebookId, $false)
128 }
129 } catch {
130 if ($null -eq $failure) { throw }
131 [IO.File]::WriteAllText((Join-Path $output 'cleanup-failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
132 }
133 } finally {
134 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
135 }
136 $app = $null
137 [GC]::Collect()
138 [GC]::WaitForPendingFinalizers()
139 if (-not $UseCurrentCache -and -not $CloneHost) {
140 Get-Process ONENOTE -ErrorAction SilentlyContinue | Wait-Process -Timeout 10
141 }
142}
corpus/extended-ascii/ru/cold/source.json created+14
...@@ -0,0 +1,14 @@
1[
2 {
3 "path": "Open Notebook.onetoc2",
4 "bytes": 4816,
5 "sha256": "b2f857961b76258a1582b402c048b340e89962257681426b3bfc58df8e0adbfa",
6 "mtime_ns": 1790487072902780320
7 },
8 {
9 "path": "synthetic.one",
10 "bytes": 12456,
11 "sha256": "3d9562625271091ff5c85c24325ab2bf12a670e81aab2caffb6b00c724d7e19f",
12 "mtime_ns": 1790487072903016820
13 }
14]
corpus/extended-ascii/ru/cold/teardown.json created+1
...@@ -0,0 +1 @@
1{"absent": true}
corpus/m6/native-template-controls-01/ORACLE.md+4-2
...@@ -1,7 +1,9 @@...@@ -1,7 +1,9 @@
1# Native default page template and unaligned image container1# Native default page template and unaligned image container
22
3The captured UI operations selected Hearts from the native Templates pane, selected Hearts - Decorative as the section default, and created a new page. The default background appeared automatically; `default-template.png` and `ui-operations.json` record that result. `read/` contains the three native page exports and PDFs. The clone was deleted.3The captured UI operations selected Hearts from the native Templates pane, selected Hearts - Decorative as the section default, and created a new page. The default background appeared automatically; `default-template.png` and `ui-operations.json` record that result. The clone was deleted.
44
5The section property 0x2c001d62 refers to a separate page object space whose role-2 metadata has JCID 0x2003e. Property 0x1c001ca5 contains the UTF-16 template name Hearts. These native template fields are not in the published MS-ONE object list; the typed model identifies them as observed native metadata. The original personal notebook uses the same relationship with the name Ivy.5The section property 0x2c001d62 refers to a separate page object space whose role-2 metadata has JCID 0x2003e. Property 0x1c001ca5 contains the UTF-16 template name Hearts. These native template fields are not in the published MS-ONE object list; the typed model identifies them as observed native metadata. The original personal notebook uses the same relationship with the name Ivy.
66
7The embedded image object starts at byte 11396, four bytes past an eight-byte file boundary. Its payload is 10056 bytes; the footer begins at 21492 after four padding bytes. Native XML exports the same image bytes (SHA-256 2035407a0540e1c4f7934db08ba4add750fcb9a62863ddd9553e7871c81a99e3). MS-ONESTORE 2.6.13 pads the object length, independent of its file position. This fixture exposed the reader's incorrect absolute-offset alignment. The regression relocates the object through all eight offsets and separately corrupts its footer.7The embedded image object starts at byte 11396, four bytes past an eight-byte file boundary. Its payload is 10056 bytes; the footer begins at 21492 after four padding bytes. MS-ONESTORE 2.6.13 pads the object length, independent of its file position. This fixture exposed the reader's incorrect absolute-offset alignment. The regression relocates the object through all eight offsets and separately corrupts its footer.
8
9The stored picture is Snowbound's rendering of Hearts (SHA-256 6a0593db927c1a34a94cf11093e5bd8700843323ed4fe8c905df0c595cb09c26), written over Microsoft's at the same length with a PNG comment as padding, so the file is otherwise native. `read/` is a cold OneNote read of that notebook, whose XML exports the same image bytes; `default-template.png` still shows Microsoft's art as OneNote drew it during the capture.
corpus/m6/native-template-controls-01/notebook/synthetic.one
Binary files a/corpus/m6/native-template-controls-01/notebook/synthetic.one and b/corpus/m6/native-template-controls-01/notebook/synthetic.one differ
corpus/m6/native-template-controls-01/read/environment.json+2-2
...@@ -1,7 +1,7 @@...@@ -1,7 +1,7 @@
1{ 1{
2 "powershell": "5.1.14409.1005", 2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010", 3 "schema": "xs2010",
4 "hostname": "ONE-M6-A33C1CD2", 4 "hostname": "ONE-M6-F782B029",
5 "cold": false, 5 "cold": true,
6 "onenote": "14.0.4763.1000" 6 "onenote": "14.0.4763.1000"
7} 7}
corpus/m6/native-template-controls-01/read/hierarchy.xml+1-1
...@@ -1,2 +1,2 @@...@@ -1,2 +1,2 @@
1<?xml version="1.0"?> 1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{BF109DC2-6C00-44C4-B8B2-9892971AF8B9}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-06T04:25:01.000Z" color="#B49EDE" isCurrentlyViewed="true"><one:Section name="synthetic" ID="{9C3D193E-0294-060F-08C2-1BB16FEA3244}{1}{B0}" path="C:\one-tests\runs\capture\notebook\synthetic.one" lastModifiedTime="2026-09-06T04:25:01.000Z" color="#8AA8E4" isCurrentlyViewed="true"><one:Page ID="{F3B3040C-0089-05FF-2C9F-3BFB91647BCD}{1}{B0}" name="A notebook created in Rust." dateTime="2026-09-05T07:11:28.000Z" lastModifiedTime="2026-09-06T04:18:00.000Z" pageLevel="1" isCurrentlyViewed="true"/><one:Page ID="{CEEEB7B7-FAC8-4A77-917B-DBF7BAF6813C}{1}{B0}" name="Hearts template control" dateTime="2026-09-06T04:21:25.000Z" lastModifiedTime="2026-09-06T04:23:18.000Z" pageLevel="1" stationeryName="Hearts"><one:Meta name="PageLocID" content="00040"/><one:Meta name="LocVer" content="1"/><one:Meta name="PageVerComment" content=""/><one:Meta name="PageOveride" content="0"/><one:Meta name="PageName" content="Untitled page"/></one:Page><one:Page ID="{92E09529-B4AC-4953-828F-1D1072EB4782}{1}{B0}" name="Default template page" dateTime="2026-09-06T04:24:38.000Z" lastModifiedTime="2026-09-06T04:24:51.000Z" pageLevel="1" stationeryName="Hearts"/></one:Section></one:Notebook> 2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{B9AA08C0-7C3A-40B1-910B-631014E2286E}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-27T08:50:30.000Z" color="#B49EDE" isCurrentlyViewed="true"><one:Section name="synthetic" ID="{E2261F82-C226-039C-33BA-1D20ADCA3D7B}{1}{B0}" path="C:\one-tests\runs\capture\notebook\synthetic.one" lastModifiedTime="2026-09-27T08:50:30.000Z" color="#8AA8E4" isCurrentlyViewed="true"><one:Page ID="{8DA802B0-C03B-006C-17E7-3D6A534474F2}{1}{B0}" name="A notebook created in Rust." dateTime="2026-09-05T07:11:28.000Z" lastModifiedTime="2026-09-27T08:50:30.000Z" pageLevel="1"/><one:Page ID="{B0F5B10B-3A7A-4FE4-AA03-DD6678D68E03}{1}{B0}" name="Hearts template control" dateTime="2026-09-06T04:21:25.000Z" lastModifiedTime="2026-09-06T04:23:18.000Z" pageLevel="1" stationeryName="Hearts"><one:Meta name="PageLocID" content="00040"/><one:Meta name="LocVer" content="1"/><one:Meta name="PageVerComment" content=""/><one:Meta name="PageOveride" content="0"/><one:Meta name="PageName" content="Untitled page"/></one:Page><one:Page ID="{ECFB9395-741E-4CC0-B9F7-1B81B0CB48BD}{1}{B0}" name="Default template page" dateTime="2026-09-06T04:24:38.000Z" lastModifiedTime="2026-09-06T04:24:51.000Z" pageLevel="1" stationeryName="Hearts" isCurrentlyViewed="true"/></one:Section></one:Notebook>
corpus/m6/native-template-controls-01/read/page-000.pdf
Binary files a/corpus/m6/native-template-controls-01/read/page-000.pdf and b/corpus/m6/native-template-controls-01/read/page-000.pdf differ
corpus/m6/native-template-controls-01/read/page-000.xml+1-178
...@@ -1,179 +1,2 @@...@@ -1,179 +1,2 @@
1<?xml version="1.0"?> 1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{92E09529-B4AC-4953-828F-1D1072EB4782}{1}{B0}" name="Default template page" dateTime="2026-09-06T04:24:38.000Z" lastModifiedTime="2026-09-06T04:24:52.000Z" pageLevel="1" stationeryName="Hearts" isCurrentlyViewed="true"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Title lang="en-US"><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-06T04:24:51.000Z" lastModifiedTime="2026-09-06T04:24:51.000Z" objectID="{40E51230-3D0A-4B15-8CFD-067DA64C81AA}{18}{B0}" alignment="left" quickStyleIndex="0" style="font-family:Calibri;font-size:17.0pt"><one:Meta name="oeLocID" content="0001"/><one:Meta name="oeLocComment" content=""/><one:T><![CDATA[Default template page]]></one:T></one:OE></one:Title><one:Image format="png" backgroundImage="true" originalPageNumber="0" lastModifiedTime="2009-07-28T18:49:14.000Z" objectID="{40E51230-3D0A-4B15-8CFD-067DA64C81AA}{14}{B0}"><one:Position x="-90.0" y="-3.599996089935303" z="0"/><one:Size width="89.21739196777343" height="287.9999694824219" isSetByUser="true"/><one:Data>iVBORw0KGgoAAAANSUhEUgAAAKsAAAIoCAMAAAAhjet0AAADAFBMVEX////d0dncvM/S6+W+4qyw 2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{8DA802B0-C03B-006C-17E7-3D6A534474F2}{1}{B0}" name="A notebook created in Rust." dateTime="2026-09-05T07:11:28.000Z" lastModifiedTime="2026-09-27T08:50:30.000Z" pageLevel="1" style="font-family:Calibri;font-size:11.0pt" lang="en-US"><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="Fixture Author" lastModifiedBy="Fixture Author" lastModifiedTime="2026-09-05T07:11:28.000Z" objectID="{E2261F82-C226-039C-33BA-1D20ADCA3D7B}{23}{B0}"><one:Position x="36.0" y="36.0" z="0"/><one:Size width="191.1661224365234" height="13.4277172088623"/><one:Indents><one:Indent level="0" indent="0.0"/></one:Indents><one:OEChildren><one:OE creationTime="2026-09-05T07:11:28.000Z" lastModifiedTime="2026-09-05T07:11:28.000Z" objectID="{E2261F82-C226-039C-33BA-1D20ADCA3D7B}{24}{B0}" alignment="left"><one:T><![CDATA[A notebook created in Rust.]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
33t7akbqZ06ir2qrewNLB5OHcxtTr9/fp1eH9+/z9+fvcmL7tyN274avlzdu44d/cvc/jydjP6uTE
45ub25O779/n58/bbscncw9Lbuc2f1q3Y7++03quy3t6i1qncztez3r7gxNXm9Oq14ODu3ufbpsTJ
55+Pcz9jak7v6/f327vPcwdHcydXam7/F5uLjrcvy5u35/PrcvtDcy9bL6OPO6uq/5OT1+/vczdf7
68ve24N/cv9D46/L06vDcxdPw4uqf1aiq0rS84uDbtsyb06jfn8Os27jw+fm64uLhpsew3KrM6dTa
7lrzaocHD5eGm2Knamb20397outTZ79/x1uXbq8fbs8vs9+/n0d7z+vTi8/Pqwdjr2eTcytXcwtK/
85MnN6eTT7Ozd8fHansCy3aq64uC54Kvcx9S336um2bPao8Ok16n03erbrsjbqcWp2anG5s7H5uK7
90sDJ6Oid1Kjf8eTp9fLn9fW+4+Gt26q54cPvz+HmtNDM0cyd0qvZvs/U0dLJobXQ0c+y293Y0dW/
100cPOxtOu0rey0rr0+vnTpbbA0djYlrnd8OzT7NnRnLfLyNS12dy419u71drH0cni8u7SwtHRxNKh
110q7Gzda20r3m9PDY7ujD0cb8/v22yK7h1t3KvrLg8vDWwNDDz9el0rG9rbLNnbb7/frexNTIw7HP
12r7TMubPV7Of7+fq909m2yLnq4uepzbHPpMPWm7i/3Ky6srHFza/d2eSxuq6dzqnn9OC+17LGyLDb
1379Du4Oju5+vKsMi8yrzIytXL6Mru+PW1tq/Gq7Ot267A47r9/P273cTewtPk0dzf1Nvh4uDfq8Xy
147vHp2OLy6e7M6eiq1Kr6/fzFytXQ6sXz+vDr9vTVqsbZ3dm02KvS7OnY1+HD0q7B16339PbBqbPa
157+rCxMDXm77T7Oeu3LjYydLL6ejQpLvKuc3G5snXp7za1dfx+ffgzdne0dq94rCy2dvE2t+lxqvA
16ysDRqrXNtLPU7OkRzzSdAAAAAWJLR0QAiAUdSAAAAAxjbVBQSkNtcDA3MTIAAAADSABzvAAAI95J
17REFUeF7NnXdgFMe9x/eW050k1AtCgBoCgYSECiAJ05Gw6YhmejXVgOk2YGxckzixndhOc3p7qS8v
18eekvyeu9l7zee++9vzd9Z+vM7MzsZf4B3e3dfva33/m1mbtzHJujZrC3urHCJWPGkcnZNs+m8d7r
19T/RQSu/fnt5ZGm9p56Wzq5k5g8AnauycMuW7DjaGLeo9UrE+5dtaeNnsRFLIfMLCWVO9ZW+STclz
201ane2fSLamZIoLrutwPs7NgpFbiEXtNGUn4/aVTXLbWrVUB1ZygbwugLaiKcf7x6lxk9t+qbCX2V
21D7xH9e1NHj8p5QG8g0po2BpZF8ByGZOGUnuvakWzum7J0phZyqjupJoxzB2tbla30dzZld6pJs6s
22Q0PT4ZgYGgsfoXQGcwdHOYG1Eyfn5bzRf7ItgDto7vwq7xRKWcYu8pyEuH96M49bGsEGZ9bQOGdQ
23/r/zhjjY0uSxy3x3dyyOFFJP9w4tzeQ6wrNOxNiUPDzMji1N/sJlLc0nk1FzuQkGqzIjTB3LybV5
24lQg1l2OaNXV+7n1mD072wjE5GJMhDzJLyaDm5lFvYJZ19mS13x3NOBFRMnv14LDYqtz8Msg6GNU6
25cd2K6qATP0HtKphW9EL6iWFNsdb0JqT5M/zJJ82yx/qlzJrLkQhmhrWmV5COzuBtS1mT/KrvKlbh
26G2GEdb1E4lztNaaIpIckrQoOw5mMAdYauQSvh5mWyFXarFQE+qySvRMASJMPzDomb9YcDl7arPKo
27rNeDWS8qsM6DL9COsSqoFBazRiSB8fTwBdq5C/OVLBYl/gd10ZQlgOPsEU0NeOFSjhW1/NChknGA
28mBqy6jbglPo8iBHAon/lwitVBUy2NOuC9ZLW5A9b5iBvLJFgcfKFKbdmveVLmgPcQxPD43j+zBu/
292MbVpbjrruAFSPqit8oRWzqPTYQc/byLFLcHxq21yqwVelMrRgIPxeT6J7lKTyHAkrJL02VFLkwM
30JTjOYSaFh5TtqlnGRrRQBfVTP62epiuzaq5zhVnXCqPRSZw4q7Nq9glDHmutRPa8CsEqsg65uo3t
31IKsMai7XvzYNq+4aV5BV0r9DWEW7rnV1e/CBCCt9flBFSx+L56B+W9s/t5olxEpm/6pmNdZ52nJ1
32/KwqmdOwysG53Lj+Yrc/FkiqFZtW2Mby+d/prqZ3dRxfjG1W8+5KR7dpJgMglfD1fBUjvBLrWl2P
33BWB5R6A2W5RQc/oScBy+M2CRdVxfAn7BWmSdbkACDi5H8LDIOqRZvuAsnROBRVbdvAWzzvbsqubd
34VebWSd1qm1Q/Xhvbns9q00xdaaHmLVXZiwW/plcVeq/2XKywJFC579yxq4zMLEjsGVatkyIP/kum
35zMrFrjb50ysd+evmWJlhLQn2W+ZQHYe5ArU8T9a2v2qSlfU1FfsTcrDf+nmTrA7rwJnyBC+8/hgY
36r78Ar+YrRlEdttfKTOh6/qUiGS8+k8t90Swrqw8UqsN4AXzkWYoK/r3/DwyjOkwFBvIXZlRCfNM0
37LFVBs7ZiP8gZFf93y1HDtNQX6MaDj4RQi8WOw4ZhafmtsHAZIdnnea162PsNw5K+hmJvPcB7f4RZ
384UOGYek2YZ3p9UwMqnFYuv1aqf3it+v7Y1mLhjVLujDpVfB6PGqxw7A3IBlX6ugVp1bsugzPLwKb
39Mt96PsGs4CnTQQGX4FK7rsIe67Fk1uICw5bFsHLrBkFafmZtKAfjyiYfvWkVkOZGmvnFSaD7Qh6N
40hh23eVrDXtZx8OaHFLH2zQzrSUyKRjkH+4hhEdDCVh32RUq1g0PN5y9tsBdrU8NSpHIfKhBCN4M1
41rljQ5ELdQ0XLvpEQbQiggj/3MFjD0QtqahYqbdVgqVwvhVk9WNM+FsLirXBKsCTJ3hOBms9fIZY1
42P7tQ6wjqYEh+dS5HAizxVkHN0glmOCsgjmU22ikin3ThLLs70qz5/AUSFoy7WEILSwXx1nESvV7A
43d/l8DGt+B37ehmBx0xuaVjL3xmn2pjjUfB57rvnGwwF7w0mgWvFWDWhazBo9s9AFXEAH2JlcmBc6
44hGaZHZmPIRR/yPIbGXtZ3q5HX7v3CDL2zTujRuwNd45JTDHM2hCvAWJYj2n/FixhNDpOmfEQg0C2
45EyLvhVhvJ6ASxdLIdQdZlB+nzNh2WY9QCChzSZAruArkCnDCPXovSApte8eIEJz1je5Y4noCCgV8
46Mhg2cQNjXdARgQoeOmUG1hmsTqRFrF2JGsjD4AXtuj+aFDy6xYwOQEbT2zM2HKdbxBodYNkF3ACH
47ANRTsagGYcG6+JHmiehWImJNNmv+ScSahGq2UVezrHEoyriQNSJ19dF3QdZkVBAtTMkAaX/W5Iy2
48UAsBssYlLhQYhK75ryEB3L7RBV3xpR00WeRkYUyzZKKGcWVY8+AWQ6hubg42lPsLc/C0+ZwhgAtZ
49g5VWSL7IepsCuVhDyLY2krHZ1d5Mg8sEUqxXwnH4fNC0hoKC31uDz1I9hNPxxyRZI6/nkq/lAZRi
50dH4x5FlH3DbocyVZYxKxBq6JAHViQwW4MhsDpoVlrEgDl+JzxiCsmawrHLJBO2EY5doi1q6EfKHB
51LwNTmUGIFnxW6aQMa1J2m7/kn2C2DAsaNc2rJOyaHILP+9KEKUMpV/htBt21oOa+IcgHBE/D1IYN
52i4VZr/tVcYwVsPola82uYHPHl7VZaQ8B29Yi6+BfisotCYHwvsAiq/PPwBYSOImHkOaMbbs6s8AZ
53BHWB8FJQRUaGTbs684X1lpCVNJKs29W5KQ5cQlhuJcSqXe+I+gNC0nyeiwdWWUfFBZeQFlZkWejV
54Af0pIYzggMxYgWAFzQzhpXisHVY14CzQn1yeXk0nhbWt9ceX4lF/tdZxOoRVt8iwnh8wuih+9fjm
55gm883QqafyIYwfNsidHgCmNtfQAUUf/9f8WvbUhdhBe3zJm1vs5vUvbX9+mlsCwfMNbOWLc9hhQ8
56/DNS9os7iOZZxjbyHIgnBc98XAOWmdXUil0yaqHwzdSwDbQ6NOWvRKiF70nNSvtapnzAukQBoCc/
57lxIWdpLhMNXTrI1yVQH8lIalYjWF6hwXm7VQ+KE0hqWo80213VbLoBY+n4KVopqaVo6zT4r1FXVW
58kgd0mHJWjlMrhVoo/IkiLO1sbzG4decpP+veywfnbAQ7DNrnHNw1wD+lGA9oW9toC2spBzRysJ3/
59RoD2gxzt/6rY9QJxq/ONdgY5CYwc4kHR/9vPsSt5qzwrXYZ5xFxihQqKqxRm4FqIFD6wiz4vzcpI
60zc0pUvrUE5a9vrvPYTNYObte2IMTgPkpl13WHahfuhRmp6AsObDaX58Rucaiuu07ydVIlIgNO/By
61RsfNdDq9ui+QQ29/ClRRbOC0dSDOqsDAcwiraOHgUjkBPZXOpKuPRyb7+zzjYpA5kVolD+7Fx+yJ
62F8Gl8+WkqNpyM+V8qo0tS+rqqWERxq4kVPcgZmWLyF1Pwr3QZOzp7iYZ6iP3phakjvtJZUlhOzYt
63Tgeh848f7TxrF/Kc833j3tTUnZTmJCZbF1fr4VPXXYXHtYrN6rpIBG+HdgWB895+g0FTDhWc/QBl
64DccAv5EvQ9Y3AL1e2DCV+i4ntIdqk62KTAtgoV13JioAPIkE+4byfMPfpHNFoi6WVKZ3ALFeFrEi
65r/WZroZ/FJ003fNyCXRhHWSNDq7cBRyCrN/I/2Q6FOGros26d1cfGudoLKqDfkAkV9eFrB/7HeFJ
66Ux4QVuvOy7zD33gd48IQK5IAZv3Bz6ZEEb0sVEWPhG/0QWJc4dRCrK/8gOicaZ9nmR72pQPXI23X
67h54ckbLrr6RFEb6OZnoYdW+cIg/BvF/MuhEcxYKy8NyqB/hYd8YnUe0gJIlZoc9qVUWQPt7HmjTP
68AawcK59HSmNIHcizJrv69p1i1uvAX0idNtVBfOMvOYlyD50Tzi0wB+3JFadPeAhRxKFgpFBYl8pk
69ci/yWPuEZhMeMFDYLHfWdEd53f/E8kSICQ8ALut4Ogq5V3l9Sn3Wa3YlwAlWn/WyXQk4Dkte9Fl3
702vQCUCZMBMLsVKRZINdA/0NOhvJHsWRb2w9ctxkI8AU9LetfRXY9V0AVr81Bw8GAiEXwfLvtmQWN
71QJvAmoK9jmpzy4PWBrv0DLu3zl6K5VmA1IdJPUDxZRyy7bAwL21nHBQTxR+xKxOzOg5ZYpEo/mJZ
722wcsZoO+SUCml4Zh+zIyq+OsxpE2oeASOayBpyx7AO/tSe2dOnYdtJq4BsxA0gJx7h9p4PYBe+Vr
73xP3Ckt2bzhX0Pe2srFqZmQrIRoZUKmj/sf8sg2PFmYx4SSs+RRr7lnciUjTuy4YWww4ISu+QSJ57
74m0cK/9eZiRRwxhW/Mhip5efe5UcFSsjEtLizIe6vcNBh1LKyhVuzMG0rkoFKwhUQALbxwmNZwGLN
75ysP+RFAA5O9sYNEeLFnYT74phrUsExnUok7MuYT1bE6t745DzUizuAqX8gbvi0UFml2chWadq1C0
76AxKpARcDwtQrMmF1VqPkQBhu35Jg1syCguMcgKYdEYSweLXii6jKxrJOLSwYBxJN+95ks5aVbcuI
77FexugELYmZDKRIYBH382qQGySCv0tSNxtM9xvvWBR6eBsfyJB/2mzsgX4Nt3IIH2HYyr6TQkRWPJ
78XB/tfZmpANkWKmEkqsD9EqV6gJGC/7Q87IPNxskyi6yG+7Z2Xg75hEhUQLubh81QsaQvcwD2Pfde
799+FS5+qzKtYBB5udK/DUtvopmCbsve6lCZ/CRGtaeAUQ0XKwdzNVLD0Zxj1H3QIpBxaFUX2WzXZ2
80cYZBO/nw7tfnsO3CCkDsnmYXlsSunheD0QzLdW6EAhDsA0wGmSSycQYBS+MgFf8AgqmMUgB8rGUN
81hT1bQsOC4FsHdmrhvMULAkHo5ZS1ZIIlPmx74RoKsDFqRdyVBHZmSe0KW6B/hUginQCxcAvNDUrM
826rR+DaI+GKdWX0goNavzx5D1iUTWacSwJXUE0FCdkHV5MitxsllVB7H37xZ0rsmo01rw7Co560wA
838bCAdRpOD/2sK6uOdXZWZcq/AkDsFrHihIvjOnZrIXW7M89mltpKyHXatNM+1pWdDBQDZ9RWduC5
84RGadhj0B7sYuDpLCp7LJxMGJmsSsSLAI9e42evN9/67IwKGtBGcUeFcaZ2FSuPhMJCl4MINObZXM
851MI5N8gHVsKZGDes52GQNSkZIPJYBGeQszUwpwLUthMxyBqfDzIhQ9azx5JRrc8wyCqeWtOQXRPu
86P3nKbsNegRXzzH24chEYj1ayeoG/BKvFLmCVcFnIrnA0LfFuwulKfxsJPm91kUmJtSkwC1u8wpEa
87d4XFgLtYzq6PwrsfkTYsCpn2jMWMPLnWoncc1FxrIt2FV+ZSy1rMvBLKbc4/PFD2QEwDoaUp4B62
882VOBHGtTbP9g2rSgQ7CXx8yMb2NwduWmf8gbB2Ww0NrquBxrYrhgRTlRg7VYe0bKrsmhbXnAG9hS
89bGdiz0Ui/MJD+KYyzBws+a27UnFLxOxfW7C1KFplhLXFrwJbs8sIq38dxJoItok6GaL7j5/3Ld7d
90siTYmTL5q5jXN71sCbZTpi4Qs07zKdaSXY/J1FsSrE/weYEl1q3iFpEE6TSWjiNkS6zOwoS8RIqS
91HJSBXZ2ZMkWMBDSfHNqya2dyB16CEh/CFTTW1u2qyuLW4aQx0YF0vQZowd6SjSFHwLFaTLfNTC6O
921V7N1SnswUuJwWO1uNdgq5mMwPMD9iTgOAsFa0ZSZuWCrK2UEHrC+3bL0SQexdaYy2y2M5xjJgTL
938gG7e7kWm0gLWQJrtVfoOLfQbjetwfJXaxU3idtnJZY3kq+EFVy20myWYazUTgloMmCzp0lwV2h6
94LaqADFCdTr0wSxsvWaA6W9foTCyKestWc8ifDm+TWDiKuxyKajO08rQaoetRXMFus5dcBaqM9KGL
95pFdnsrn/EJvmWsslFjx5LZzG2dXMzIwKYfH+nJakTWUResVLXNvsrheGCs0VS1B9pxRsl6AcIGtS
96x5kJPeyjKvELk97K9O5jE98HuwRN0ons6UpIuu2szbQ6tsnQCViXS1bfLUtgH3tFZwb7RiJ5zwDW
97J2Ry7uW7wdRfceZudj4qxDsTzK0HoQRe/dDjcLzn1b8IzftFSyqbts08c7YEEuV5P/rXcBfW8i88
98/lH+0Zdffvn3EPnjn/jEJ85WVVWV0JiM66e+AIy4pOxDPlJbDbSk962q6uz8u9/4o/9bsmgRTlM/
99/Oqr73n88X95+bfwq17+0IfRw5UWexDi615c1XlrW1lT5aLY7tqHX/VUWWmvZSZC3doJdlPNVYhE
100TSVivXsf3KCkAAo1UArWlWfgpr+5lYp91RKwVsGtwmUPolxEaSzJem5VwY3C6jaFF7Xc2vJJ5NTC
101pGXcZ/FULJvNZ7cx+GK8jS5qc5IU8sPZ5cx38d7ENan7Ertt96KoFohRIz/fJmVVkBBkNLm2km20
102igWT/yoezCR1riJ7U/XWAJ+wtVOFdwTHyGJo1AccJQWAvFYGkYuizk09rcgFzbWeoMINt2hIl3Zx
103tn7AttdaSfdRazX7EP6jtnaqUMGyjfSK/Z0I47ZYW6HGsOgTWChdUZhFcYc2WV1FgZ+8MKRWcAF2
104QxfOVuDQaPcyMy+3Gbo8s5pZAZ5rsZnifUJBL2RRyz5sMXR5n/uQafCIZ5/FQoaFAbl912LWRfaK
105A+awTLG22Pu4pucFzOwNBQ24TlHjIe3z3Ie/zMytafY6Gsy5gv+IxShzRKW1MMuz6iaE+EqWWKtm
106eVbtTQCIdVGZrbyQ/1ididwFLHFZ21vD+QGpDzlKKNbabkDfhwDNeII1tibXWV6wZjYxNpVZKrq2
107+lj1ixigkSZrkcv/8Wrt4hCwVlpzBP4PWGsX3YjVVpT1cm2kBs1mBnQTlWXWilmf15L7pGOy46q0
1085rQcLoNFlk3cvdQiUZNVWvyKv4Bhy5bEm61FxlGADSxpsz7h6wKKLYuHbVlTalavmUF9bYxlwT4q
109mZoM2NViLRv6toXIjAtuTpLZigeOs7joHv6+hfCawSL0kdvSszq0/8oF3Cd8i4anyZ5PmW1CYNuF
110RbuCnTW+tAD/wRZjT+9mH2OW6STaZnWCjgvjrmkCg35gDapahhVcmFW7OouTvs0Eca+4K8kKDrPL
111KoRdsRimuhJ6hV9AZZlVAHvfYgdmZBJ+AH41hm3WRFjY/IOSlmCFW6+tszrIclFjG4pDMCuXqMjg
1125y8yYHXI6nGAtxPXT/BRibIcbmjNgjXqe5jYF3GhKxBvK4FHCfMlMwesPMO3N7Zxv5uBWBMSRpxK
113oo8LmUGReZeqzpnQ28685dtICb7JBQzhMjj6fj+Zs9g8BlfnwgUQuFs0gxVkwZXiGScSAYzI3y6s
114Aq8Fv8zHWs0tLxviyZLTF5Q7/kMWv2uWCE46NMmGRUnZ156Wt4CdI2namJS/4M82/X5hnx0E6Xel
115EfjBhHiAPoXxJndvqWFZ+zPex55GZn23u3GgsNTyr1wmm9hrf8b6LVyVfcB1D+0s1GX1s3GR1Cz4
116xnUTycdx3wK+dbv9XKGwub50/sArIGMkSz6R/Rz63Zg54DePC/us/zBnjBa4yjyy9Um+6vdd9Cdu
117Nl4Gvz9fVyJcLgOLgKUfxfsU93M813aB33Sv25fpb8hhS/NdhNBXzbGvF3uv/9e5EO7m+qzdgq+j
118ONcfbBnqm0I/JNZ+EPwGfeHpjI3r73jwbaTlrDXztqgfPZsD3EJhcwY/1epNtUAPfO5uEsJauO+6
119gB4rYmzclTUt2iLPj4crweC/VSwsAQqOnNjm7JTAth4GmdnfvBcImvca8GEZBl9uf0w07ieTfm+y
120/TL82anMwll0Q5FxR84sDn8ONG1WQhCo4H2in/Fsh3OscDybTAG2NmPHl0So4PnrEHb7Oum8WedA
121/zK+HztRrcwhgEBWqMvG2cZ/STLIXGXGIQhbOK5jMOnXxsG+E2eD4oFhsynKYmADWUsCM4bdnskM
122i4R9h9ig7IgsYSMmmAoqqMigYTOybFXwu/3VUF33YIaw/q/3f6cwCIQEguJtIaMWzUr0WU84/uO3
123ZT0ATwwz8Ky8AfBxdztvgZby96b7/XnQ8YDjKWlfaeLAzSl/fh5F20Ihu5TWcepT/u6468LsG4Tb
1247OrG2rqUZnXdjdiwS03cXKn3OJ5OrWiWYV9QyKr1tbqQ2qyg7YWnVyGbDNHZN6AQWkOH9mHW7VJ3
125UPeg1eJf7U26FqLYbBzXvoLqz8770WGTA/qCDFKu1YVzOhJw3WuYNYvMu15nZqGrJKwF6062tq4g
12691vj8cYnIrBfJRzQlYDrEk9gX7HbtSXgzqEisJzDrCtoegEgDca6Wdd9Jr/+eCF12sIE3E7tajl4
127wd8O1R6M1WrD4GqhcE0bFReJ1gMt+PV2vaAFL5TptVCwGbvqCuK8Reh+qc8ClrVYIAAvMCKUwGUR
128LK4Q0bDotfBvCAtGn+AQmmhBVospN/jla/jjzMmjT3AMLrnwsMdaC95dXBIAOV5PuBpSymJWe32N
129VvDu9PfE42ng1ImXAW4U2bcrkKsEK1oeGImZYJwPsKqBz/6TFCvW40CUDtDiHD+steV/+BUF1kJh
13058GAbQ8GSe1Vs7/7h9AiYr2Cg45j2w3sOoh+gB5U2nP6aIbNm9VWojX67x+TYoXus5XAIq6REdIS
131CNx/8Kcll3X05/I/LcUKS79W5+kwWPgRS6Xs4Z/Nd30Dni3JdaLbDd1Aq1MLwoZwWDHr6NSefP7J
132P4cnF6avcPkVpCQSsDYacEenOm7k8/nyNyD1CSIsSvhg+lQrkoH++tHoAt/YP3VqS7G7C6DmryBW
133UcWNpjtO9WDoiB9LNXLXo3em7s3vKIZG9w1EyliT6wKcR5O0tHVzPGpqrS6YIpS3u8vLu7oIHEZk
134o/w70ZmTO0TY3dOOSm19XTTtvnQ9l9H997Atu8u7GgJ4flZy2qQihiRRXi0cRbv5eCpSCrppz/lE
135TgjdhfWaZFjSVvOHo6v7OCnULa1P1yU+egrrs3tHkj3Zc4Q1vpKdQwJUyBnVtrbWg3G1Ne18WjAf
1363/s9F6RI8/m3EhEMkCAf9F4si049b2L6KJS0XHjv2ZVQ1sJAVAKz0cuizG4rPUpsekXWphD4u705
1373RdMpdEmNzrS3ukos45O4bt/O9o5xUni4xzOQB/nDub0cWW02SJqwSMY9Yb87Uf43/T7yp0jl/v6
138+i6P+DjhIQaTfWLUTeclp5R3WFLI9J6rM9apHN1C7v8lZdT8/0jBGvMCh0nE36B4/9GFfU6G1VgC
139rYWa/zosDkXDlFop6qY0VgWG/bwI1NwKNkUtptAqEsGfCg272ZBvZajl6tOKvIILB5EmrkuXmIQ8
140B/UAxQ2pUb2cIFoMplCdUzTlV4tW/gv7+tsTJKtTlvhMu4CidmuYNZ//TVwdRAyDHzMigbVYlEtW
1414y7o/Pd/Jho1ZVkSFeT2s6JPy6z5hmLxx5eGaLc/ZWj+I3RmVj0J5PPdxQ5n3XGurbJ934FU9VNs
1422nCHmTW9w8I3pLxYPAxP0wrqkqutrWYxET+pVQHxk3oayHcVi1PGMqnIN/KaFDoeC11msbjFLqvX
143TtFm3VAsjlqF9Vh1NZDfUyzeyYhVd27lnywWb2bEquuz4OSanxFrUdMPwGhQrLEJy7Uq1WvCwNWB
1449/rljFi1RdBdLP63TcPyLWBdrwVYv1xt0bA8q65hbxSLX3UH7cGy1AVCayoWZATF5h57KrjJG3aT
145Ssct7DYg69+6R6wZ9jDPqlVxoUyr+Eyba0+ypIlJkPckONkGkUQg6+s5i7BeAotw42EbNoj8xHnw
146+sdyAPaILc36DRubxjbsKUqy5oabeyx5A79i4yzbADI+ESvUwIs5MOYBHcyyMsV8rgDKIKKpdQmg
147FkVeArLeD1kt0pLOK/MIG0JtrR2b4JOixhzHmsv1D689st68bY+GVlv9XfgLIHjCIaMBYlds3Ivf
148VW0cd4HPycI/Nt1gtj1/BT7QAaagKqsdXK+h4VHfvlJeXn6jG939YsdhCVZoft6uRLoXf9GsdaNg
149OWN3HHaAqEVVTjQrIB5vqzgx25x2E2EBqgPIbwgqB3gLwnZF1u2/ODZj0liESICFqJBV1J+FNyKG
150FeAOj7nVpoy7IGLvBZLBFtT6ETstUBsmsSLaRkN+ga7FBpzCKdyggI8mdz1BzZ3Mimh7lpkR7lTY
151tI/Q9gQEuZIoWNDLELHmctObTdEeZf14bNyOKdb1QX8nRi4YhhP0ij1Yrr/NdQ2lNqOveRH33n7u
152diHWJK91AR0BckLRGB9z3UZDs2x0wRQYry3w6wqljklrdTtkWXP9E2DnxgljHiysf5zmJhgWheLi
153R0RWRc+PN7tuhSGXEDFVyYp97Moi7BCB8YwUa65/CJi20U6O6zgkVsTGAyyB4vNyrMAhANiKSTP+
154K/guRzFLbKBFXqBYlEUFvhbowFZlRrse0QEBBS0Jl8VdyyoI22PIIfhNm7wUStLxl+TtmsshWNeG
155DlhxvilifhG1Ft+swprrXwthbfQ+vNZXSAYNOCMvFt+oxEpgZ5h3tcRrRblZMrGKz6qhgoiLLDvD
156uPMa5TKbbj4zAG0OMlB3QGlgzVYYn2GcYYubvCWmC9SqRUW5oouyBMsblpW6l5hRVSIBZ/txtK3T
157uGVfo/ea/nvbMyl86NNKt58ePGwHNtihCbCreSx2YSCjtWBZtuMoaGD89wup7EqcgVth2BskluZK
158QYu/rHnIGbim/WywqcgbWDEQcLRYsu4Mw2lXoCLjWONbA2JpPIRhTYfb2PklnbpGoPdjFZhOZEa9
159bSe+GSZRFSbY9yJmNb6UFymDT6d0ApQfJQYgJpjOY/hgS2z7bPqJhWlx+AJFmOH55YR6X9qouRyZ
160XqYl6zijv+DTqgHU3DxiWNd4zvUdz9zv0b6oqVWsAmpY0172z8B7P//BT0Pc97+kK1UyvahiTavg
161K2L3rn4EcQWGE4MvqoNIvIJEWsPh60clzqx+SD+dXSYjQs2/qYPIvILOLpNO9kdkTpziGCYCg4b9
1621xQcMi9hLtacYZetkjlxmmNItmUwh2lMgyH1GtiVxcPQDp9ZE1LnTXMQbMqSYab4OnEyDYbUazjW
163E0byrQqp06Y66KRn1x4TrOuHUmFIvYilBIDZRLpVPV3qtKkO4lkNiKDGHU+FIfUiWnVBKRgQwTK3
164X+q0qQ7i5pbr6nuCI2tTUci9iCUE0LD6a3XuQ3KnTXWUF7cAa6+uJ1jvqk2ti/MUoFd5LstERVut
165OLWmr1WQN25u0qFdd/W4aonLdLdN2rBemoVxNTUwy3WlT40OBF5IGtY3s/RZl6myQu8uCetl2kQF
166mnatdhUjLIpEUrD+iWVAAz2qrDgSPSSeYHD7Q2Do2RXIVdGuJBKtFc1IProS5Ao91vXARGpzi0XN
167RLc8zysIPNtqdgx7XcVQgLZf4DE2HHeVYMt01NBMtBqVWaEf6mG0UbIdDroqyq2ZD1Qos8KbWzFJ
168T9/cNuyLueMX40DBKzQ73HBKq+kVUTpHuHvcPNQ2HY6JIdpri1SA7prMIHhXNT+Aw6ZTMyOaJ+lR
169zY8ogKilyHoSszqzgHrUhm6DALgBd0xJA3A3HkpCZivCai/OnoAnVmIF2xwxqyqs9i5Z4LJcpZyQ
170xHgUgJQsq+lbwekQa6xLjzA4lgBJRGfJTzAD67LoZCrdLJKPkMBew7uupJlmYnsZen+FyUUzUpaE
171TMrMMDO7jrEt5Ks9PLP4YmQWklHi6NWMV8Qw+BzShSzNW3xF3mAyrbHPfWFWWRGQnSGh5eB4WoMf
1727XDIvZP0BCzch2b1rMmwS6g4MqnfFOKyc8IqZ1gvKY1qoNSs7z1C1dDY2LvMRAfTV0fQOSGjWC5/
1731kxE05Uy7M6J25p8qm/03sqSsyncLCj1+vkUWrPGk4ULHOe5m2RY+MkMb+hmd+lgYU5IRnO8DOYF
1746hLtlCkVLMy12WiL7k+Mh4rSksjVgTUMZ9qJULQdn/DdfXSsZpmfyqjwRcFAPjYxTLSwavziRFRD
175wnVLIwGSwCanHsFnS+MFgF25ySVLrN31TyuC2bKE7Djje+7k0Wm7R5q5ZGZVF4GBxT95QwaOrJE2
176KD7QePqkQl6tBFtCBYCLqpGp7uj1GCidVewYOpb1J8UG1l5M0wKFLxZXouQqGs0UpDrAsiootQDk
177G1MVpUoDArdBootm7aOvyoqoEWi2pyTVYNxlJDWmGr9Nbj9jr+mNdrQ9J0pTBiRro2bZkQDujOpl
178NkH/HxtpiBH77U/NAAAAAElFTkSuQmCC
179</one:Data></one:Image></one:Page>
corpus/m6/native-template-controls-01/read/page-001.pdf
Binary files a/corpus/m6/native-template-controls-01/read/page-001.pdf and b/corpus/m6/native-template-controls-01/read/page-001.pdf differ
corpus/m6/native-template-controls-01/read/page-001.xml+177-177
...@@ -1,179 +1,179 @@...@@ -1,179 +1,179 @@
1<?xml version="1.0"?> 1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{CEEEB7B7-FAC8-4A77-917B-DBF7BAF6813C}{1}{B0}" name="Hearts template control" dateTime="2026-09-06T04:21:25.000Z" lastModifiedTime="2026-09-06T04:23:18.000Z" pageLevel="1" stationeryName="Hearts"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:Meta name="PageLocID" content="00040"/><one:Meta name="LocVer" content="1"/><one:Meta name="PageVerComment" content=""/><one:Meta name="PageOveride" content="0"/><one:Meta name="PageName" content="Untitled page"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Title lang="en-US"><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-06T04:23:18.000Z" lastModifiedTime="2026-09-06T04:23:18.000Z" objectID="{37F0B5A1-103E-4EE9-9AA9-94166A9D8F91}{18}{B0}" alignment="left" quickStyleIndex="0" style="font-family:Calibri;font-size:17.0pt"><one:Meta name="oeLocID" content="0001"/><one:Meta name="oeLocComment" content=""/><one:T><![CDATA[Hearts template control]]></one:T></one:OE></one:Title><one:Image format="png" backgroundImage="true" originalPageNumber="0" lastModifiedTime="2009-07-28T18:49:14.000Z" objectID="{37F0B5A1-103E-4EE9-9AA9-94166A9D8F91}{14}{B0}"><one:Position x="-90.0" y="-3.599996089935303" z="0"/><one:Size width="89.21739196777343" height="287.9999694824219" isSetByUser="true"/><one:Data>iVBORw0KGgoAAAANSUhEUgAAAKsAAAIoCAMAAAAhjet0AAADAFBMVEX////d0dncvM/S6+W+4qyw 2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{B0F5B10B-3A7A-4FE4-AA03-DD6678D68E03}{1}{B0}" name="Hearts template control" dateTime="2026-09-06T04:21:25.000Z" lastModifiedTime="2026-09-06T04:23:18.000Z" pageLevel="1" stationeryName="Hearts"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:Meta name="PageLocID" content="00040"/><one:Meta name="LocVer" content="1"/><one:Meta name="PageVerComment" content=""/><one:Meta name="PageOveride" content="0"/><one:Meta name="PageName" content="Untitled page"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Title lang="en-US"><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-06T04:23:18.000Z" lastModifiedTime="2026-09-06T04:23:18.000Z" objectID="{49EBB31D-D08C-4B7A-A1D1-9287A8BD80AE}{18}{B0}" alignment="left" quickStyleIndex="0" style="font-family:Calibri;font-size:17.0pt"><one:Meta name="oeLocID" content="0001"/><one:Meta name="oeLocComment" content=""/><one:T><![CDATA[Hearts template control]]></one:T></one:OE></one:Title><one:Image format="png" backgroundImage="true" originalPageNumber="0" lastModifiedTime="2009-07-28T18:49:14.000Z" objectID="{49EBB31D-D08C-4B7A-A1D1-9287A8BD80AE}{14}{B0}"><one:Position x="-90.0" y="-3.599996089935303" z="0"/><one:Size width="89.21739196777343" height="287.9999694824219" isSetByUser="true"/><one:Data>iVBORw0KGgoAAAANSUhEUgAAAKsAAAIoCAMAAAAhjet0AAADAFBMVEUAAAABAQECAgIDAwMXHhna
33t7akbqZ06ir2qrewNLB5OHcxtTr9/fp1eH9+/z9+fvcmL7tyN274avlzdu44d/cvc/jydjP6uTE 3kbrakbrakbrakbrakbrakbrakbrakbrakbrakrvakrvblr3bmr/TnLjbnMDcn8LdosPPo7bdpsXd
45ub25O779/n58/bbscncw9Lbuc2f1q3Y7++03quy3t6i1qncztez3r7gxNXm9Oq14ODu3ufbpsTJ 4q8jdr8ncssqEtpHdt83duc7du8/evNDdvNDcvM/cvM/cvM/cvM/cvM/cvM/cvM/cvM/cvM/cvM/c
55+Pcz9jak7v6/f327vPcwdHcydXam7/F5uLjrcvy5u35/PrcvtDcy9bL6OPO6uq/5OT1+/vczdf7 5vM/cvc/cvc/avc/TvtDTv9DTv9DUv9DYwNHbwNHcwNHcwdLewdLcwtLdw9PdxNPdxtS6yK7dyNXd
68ve24N/cv9D46/L06vDcxdPw4uqf1aiq0rS84uDbtsyb06jfn8Os27jw+fm64uLhpsew3KrM6dTa 6ydXeydbgydfizNrnzd3lztzd0Nnd0Nnd0Njd0Nnd0Nnd0Nnd0Nno0d/d0dnd0dnd0dnd0dnd0dnd
7lrzaocHD5eGm2Knamb20397outTZ79/x1uXbq8fbs8vs9+/n0d7z+vTi8/Pqwdjr2eTcytXcwtK/ 70dnd0dnd0dnd0dnd0dnd0dmY0qeh0q2Z06iZ06iZ06iZ06iZ06iZ06iZ06iZ06iZ06ia06ie06mZ
85MnN6eTT7Ozd8fHansCy3aq64uC54Kvcx9S336um2bPao8Ok16n03erbrsjbqcWp2anG5s7H5uK7 806ig06uk1LTq1OHh1Nyc1Kib1Kie1ana1duq1sHt1+Tl1+Dr2+Ss287u3Oex3Kux3Ku03auz3auw
90sDJ6Oid1Kjf8eTp9fLn9fW+4+Gt26q54cPvz+HmtNDM0cyd0qvZvs/U0dLJobXQ0c+y293Y0dW/ 93t6w3t6w3t7v3uiw3t6w3t6w3t6x3t6x3t6w3t6w3t6w3t6w3t7V3uC938i737+937m336uy396z
100cPOxtOu0rey0rr0+vnTpbbA0djYlrnd8OzT7NnRnLfLyNS12dy419u71drH0cni8u7SwtHRxNKh 1039+939W9387U3+C24N/y4Ory4Oq64Ku54Ku/4La/4LS94Ni44d+84ay94ay64uC+4qy+4qy74uC8
110q7Gzda20r3m9PDY7ujD0cb8/v22yK7h1t3KvrLg8vDWwNDDz9el0rG9rbLNnbb7/frexNTIw7HP 114uC94q694q++4qy+4qy+4qy+4qy+4qy+4qzy4uu+4ty+4qy+4q2+4qy+4qy+4+C+4+K+49+94+G/
12r7TMubPV7Of7+fq909m2yLnq4uepzbHPpMPWm7i/3Ky6srHFza/d2eSxuq6dzqnn9OC+17LGyLDb 125OHQ5OPN6eXN6eLN6ebP6uXR6+TS6+LS6+XS6+XS6+XS6+XS6+XS6+XS6+XS6+XS6+XT7OHT7N/T
1379Du4Oju5+vKsMi8yrzIytXL6Mru+PW1tq/Gq7Ot267A47r9/P273cTewtPk0dzf1Nvh4uDfq8Xy 137N/T7ODT7OLT7OLT7OTT7OLX7ujc8Ovl8/Hm9PHn9PPp9fTs9vXy9vfw9/fv9/f09/j39/j5+vr7
147vHp2OLy6e7M6eiq1Kr6/fzFytXQ6sXz+vDr9vTVqsbZ3dm02KvS7OnY1+HD0q7B16339PbBqbPa 14/fz9/f39/f39/f78/v79/v79/v79/v79/v79/v79/v78/v79/v79/v79/v7/////////////////
157+rCxMDXm77T7Oeu3LjYydLL6ejQpLvKuc3G5snXp7za1dfx+ffgzdne0dq94rCy2dvE2t+lxqvA 15////////////////////////////////////////////////////////////////////////////
16ysDRqrXNtLPU7OkRzzSdAAAAAWJLR0QAiAUdSAAAAAxjbVBQSkNtcDA3MTIAAAADSABzvAAAI95J 16//////////////+3cRl2AAAAGXRFWHRTbm93Ym91bmQgdGVtcGxhdGUAaGVhcnRzVrgCNgAAGoNJ
17REFUeF7NnXdgFMe9x/eW050k1AtCgBoCgYSECiAJ05Gw6YhmejXVgOk2YGxckzixndhOc3p7qS8v 17REFUeNrt3btPI1uaAHBSR+AxWhsEInLoXd0AjMwFDNeZI0czEzkCjahtQFhNYMlkyyKkJeAO9BJc
18eekvyeu9l7zee++9vzd9Z+vM7MzsZf4B3e3dfva33/m1mbtzHJujZrC3urHCJWPGkcnZNs+m8d7r 18jeSgpI26hCEiWQHSCkPftmzKnmz+h/oLth4uux6nqs7j+6rbmv6kkWbm8vjdw3e+8yzXlMYYg2I2
19T/RQSu/fnt5ZGm9p56Wzq5k5g8AnauycMuW7DjaGLeo9UrE+5dtaeNnsRFLIfMLCWVO9ZW+STclz 19OuoaRkxhULPZYu87sJazlCF/c+uRBcmt6LEcji0Pvq21pxtWNzYrwyhtrq3kgvNg8E2tv+Q2ShV3
201ane2fSLamZIoLrutwPs7NgpFbiEXtNGUn4/aVTXLbWrVUB1ZygbwugLaiKcf7x6lxk9t+qbCX2V 20lDZWgrD53je0djYqpCgV4sGyWDuVoCitxoGdgqDqsUnuafnBN7GGUgMTAbKDTQFR9dggloTyN7Ae
21D7xH9e1NHj8p5QG8g0po2BpZF8ByGZOGUnuvakWzum7J0phZyqjupJoxzB2tbla30dzZld6pJs6s 21RForJWIeHMVulSsVXmwnZuvf/1LhxhYGmNaeXP9Fj0P5fUCdrSHYQzSreujs0WVrDnJYocXm8LLA
22Q0PT4ZgYGgsfoXQGcwdHOYG1Eyfn5bzRf7ItgDto7vwq7xRKWcYu8pyEuH96M49bGsEGZ9bQOGdQ 22a1UPfCWyPtD+XqGOTbws8FjreVJBlzv01soaWi1wWXtBE+ktBmtlBWv4clrlvD07Lazp4ZjurbFY
23/r/zhjjY0uSxy3x3dyyOFFJP9w4tzeQ6wrNOxNiUPDzMji1N/sJlLc0nk1FzuQkGqzIjTB3LybV5 23SSl7CGyVh8m1Oc69Qo7DWlkn/GVUUKtFXXHPT0trpnaDyVpBatgpFzW3SZ7usbUrqRZAZKxt7Rg/
24lQg1l2OaNXV+7n1mD072wjE5GJMhDzJLyaDm5lFvYJZ19mS13x3NOBFRMnv14LDYqtz8Msg6GNU6 24cLkU0LEZrZVVlFIwtKpGt1otBXWWAqO1RKixYFajWK0E/+rVinjDdoCsR4EJMMSyWgkZewBjNTIg
25cd2K6qATP0HtKphW9EL6iWFNsdb0JqT5M/zJJ82yx/qlzJrLkQhmhrWmV5COzuBtS1mT/KrvKlbh 25x+xhLAV5GKsxB9gEpZJqbAfCatSAVVgqqXcdQlgPwDNAD/9EtghgVdmLPUUQplsDceshRrOSkkAW
26G2GEdb1E4lztNaaIpIckrQoOw5mMAdYauQSvh5mWyFXarFQE+qySvRMASJMPzDomb9YcDl7arPKo 26t+pFoFCBjxz40DVlTgRKCFb/cLAlbDV6FgKVkLB5YWsBJwVIQ5dg55rqsU9P6WIDfDSYMtK1ghLg
27rNeDWS8qsM6DL9COsSqoFBazRiSB8fTwBdq5C/OVLBYl/gd10ZQlgOPsEU0NeOFSjhW1/NChknGA 27hWBKr1jLOFbwQjC1hZSupPWsqBVl0Aqwbv1zWTdwrIUJsq79sMZkzf5zWdfj6lsFcesk1azCBFlX
28mBqy6jbglPo8iBHAon/lwitVBUy2NOuC9ZLW5A9b5iBvLJFgcfKFKbdmveVLmgPcQxPD43j+zBu/ 28JsianRTrFtIShjTPEjzvNOaESEULfF/TnGujdK4SvNVYw6CsDTfgrcb2APTGmxnr8GsYc5MQIwnW
292MbVpbjrruAFSPqit8oRWzqPTYQc/byLFLcHxq21yqwVelMrRgIPxeT6J7lKTyHAkrJL02VFLkwM 294NeG5l4Gwh4RaWtbdM1tbr1hVIJl6KmLsZ9VxOldWYQ9IutkC7x3baBYB1mMOcE6/BGXsa9dxsjY
30JTjOYSaFh5TtqlnGRrRQBfVTP62epiuzaq5zhVnXCqPRSZw4q7Nq9glDHmutRPa8CsEqsg65uo3t 30Ao7VOt7E71odAKtmtgHw6iCLsgdvNyxojd1AOOW0zg0L4N2LMGqVYazWdWHIddcKfNcaWgdWrwXM
31IKsMai7XvzYNq+4aV5BV0r9DWEW7rnV1e/CBCCt9flBFSx+L56B+W9s/t5olxEpm/6pmNdZ52nJ1 31gizSuaGdsYApu4HQtUZ3HbbMH7cMhSWkK9DZsWbdGgfEriB0rfF9l0PrJwKdIWfhTw0dVvuBjFWI
32/KwqmdOwysG53Lj+Yrc/FkiqFZtW2Mby+d/prqZ3dRxfjG1W8+5KR7dpJgMglfD1fBUjvBLrWl2P 32liVd1OvBWbVeHi4NCjTp2nt7ejTi6a3ParXvkkFgc1ETwv7z1W51HLtXX9msmn1FMyc65yIst10j
33BWB5R6A2W5RQc/oScBy+M2CRdVxfAn7BWmSdbkACDi5H8LDIOqRZvuAsnROBRVbdvAWzzvbsqubd 33wduVZBIlSarp/7G4z0zW8TNEgvOY9dCR4O3UdNbOmsP4dLxjeHfbDFYHVqyHkZ46GWWpKd25bLqj
34VebWSd1qm1Q/Xhvbns9q00xdaaHmLVXZiwW/plcVeq/2XKywJFC579yxq4zMLEjsGVatkyIP/kum 34ZmhP+/RW60qZcB6QUsBO1yeJJDW1+j/40Ke3joqBMYXjbtqNwOraNxpV+tgkxvGengd9eqsTm+Nd
35zMrFrjb50ysd+evmWJlhLQn2W+ZQHYe5ArU8T9a2v2qSlfU1FfsTcrDf+nmTrA7rwJnyBC+8/hgY 35L64GVdevRt/faQbFmY79wGB1Yr0XTAVSwJxnfzX+/rVmcBzr//yRwaqpzovQPIlAGrSMC3ptnboX
36r78Ar+YrRlEdttfKTOh6/qUiGS8+k8t90Swrqw8UqsN4AXzkWYoK/r3/DwyjOkwFBvIXZlRCfNM0 36Rm02d3Rsj8Hqfvwxt8asLZDTta079s5Cqc1L/WuuWKzawPUAJHNFyBHnrpEJYIb+RRKTdfQI5Ehb
37LFVBs7ZiP8gZFf93y1HDtNQX6MaDj4RQi8WOw4ZhafmtsHAZIdnnea162PsNw5K+hmJvPcB7f4RZ 37EkyBstbfpaGaheuNzap13M8ZsGQCOQXMYhVJbX4M611Bz+6onkcNctS9jJQC6pWRrJ/orFesVm8e
384UOGYek2YZ3p9UwMqnFYuv1aqf3it+v7Y1mLhjVLujDpVfB6PGqxw7A3IBlX6ugVp1bsugzPLwKb 38GBWMqt6SBoLC1ypVBljWU3Yr4SkOmlQgDQSHlBnAbyU0rfEkL/tAkH00mvUjjbXGbdV6W/5fHJ65
39Mt96PsGs4CnTQQGX4FK7rsIe67Fk1uICw5bFsHLrBkFafmZtKAfjyiYfvWkVkOZGmvnFSaD7Qh6N 39xBTYpW1W0/rIZx0/I0PLJaXAf9Bmq1lfq8+8Vm1wRPqrZpfXS9QpYDTrXpMuXfdqx39rNv/n16cO
40hh23eVrDXtZx8OaHFLH2zQzrSUyKRjkH+4hhEdDCVh32RUq1g0PN5y9tsBdrU8NSpHIfKhBCN4M1 40h5X0+Flw65JWBD/RN+uO48t+7XBY9ZFhKxvE9XS15YBmrR43meO///Pxjd0aotW5m6XQFDCbda/J
41rljQ5ELdQ0XLvpEQbQiggj/3MFjD0QtqahYqbdVgqVwvhVk9WNM+FsLirXBKsCTJ3hOBms9fIZY1 41EztV6bHPbA3T6oXMTl7S83s/V0Mn2JFTmWd2q17ADoK12WUzeXNBKVDjs36SnGMu0+cPqAf5EG6u
42P7tQ6wjqYEh+dS5HAizxVkHN0glmOCsgjmU22ikin3ThLLs70qz5/AUSFoy7WEILSwXx1nESvV7A 42sBaUAtVLPqu5UhhhGT8vYyAXsmxhpoDU5A3JgWX9HBJ9dCgzWT+IWWvmn+WZ02o8mZynt1YF0tWI
43d/l8DGt+B37ehmBx0xuaVjL3xmn2pjjUfB57rvnGwwF7w0mgWvFWDWhazBo9s9AFXEAH2JlcmBc6 43PfPftc9rZWlcK12Pm2JJYGUBp9XIXKpP0DHnAnRzrLAkMFe33FajiB0WKK38VHNGO2xYEasxQhyE
44hGaZHZmPIRR/yPIbGXtZ3q5HX7v3CDL2zTujRuwNd45JTDHM2hCvAWJYj2n/FixhNDpOmfEQg0C2 44p27hg6j10rJK4lYjdcO4R1WxMqDHcDu5DWEN5bbBrFdAVpNLyt0inHUXzmp2NV9lkB9Fy+vIWgW1
45EyLvhVhvJ6ASxdLIdQdZlB+nzNh2WY9QCChzSZAruArkCnDCPXovSApte8eIEJz1je5Y4noCCgV8 45GqOE7J6NDa6ErWe29Q3YatbdcfKWze0WMesxotWsZMNskAGsNdv6jGM1s6FsPMdfFbZKtvURzWp2
46Mhg2cQNjXdARgQoeOmUG1hmsTqRFrF2JGsjD4AXtuj+aFDy6xYwOQEbT2zM2HKdbxBodYNkF3ACH 46Nm1oFRhiL6vxWDUA6ygFsPJ13LLCVik265uoddys+kzrO7eOmxV8LIC2Opr1NCYr77Lgcq/qKAPx
47ANRTsagGYcG6+JHmiehWImJNNmv+ScSahGq2UVezrHEoyriQNSJ19dF3QdZkVBAtTMkAaX/W5Iy2 47WGviGWCceOBan4WsNedtCA3b+ihidVIh1jCIVhdVeG2IanVTzTMEYatc3p4zYmm7/g5o3XFRrbNE
48UAsBssYlLhQYhK75ryEB3L7RBV3xpR00WeRkYUyzZKKGcWVY8+AWQ6hubg42lPsLc/C0+ZwhgAtZ 48Iau6v5BwxuyS58N/n6zfxLyj+Umq+ptVxPq+lPDH7Lbqr1ms663anpsq9cSsKklqRNKh5bKeeRp1
49g5VWSL7IepsCuVhDyLY2krHZ1d5Mg8sEUqxXwnH4fNC0hoKC31uDz1I9hNPxxyRZI6/nkq/lAZRi 49dIrEa60nE4GRLAtYz3a80tG9Ek7rdiI05t75rJ9qvjYdZQCvdSkREUnZaa1SQnf2qoQYXTHjspYT
50dH4x5FlH3DbocyVZYxKxBq6JAHViQwW4MhsDpoVlrEgDl+JzxiCsmawrHLJBO2EY5doi1q6EfKHB 500VFmsn76SGpRxz4xr1W2PdMpPYISd4nOevnxuCYFOd23CTis6qz5V07P29sA85lUANa21qz4aIf1
51LwNTmUGIFnxW6aQMa1J2m7/kn2C2DAsaNc2rJOyaHILP+9KEKUMpV/htBt21oOa+IcgHBE/D1IYN 51PyVJ2qtGhPPiA4fV6FezC+7NoIX0tB9bHFm540kTs84mUgv+bbZFgna/LyaV3Efe7NZ6IkPev1z0
52i4VZr/tVcYwVsPola82uYHPHl7VZaQ8B29Yi6+BfisotCYHwvsAiq/PPwBYSOImHkOaMbbs6s8AZ 52Z4IsRN31XOBltxbnA3eGM77S9UcB6pX3HiSzdbAYsuU+782Df+GWfvBf02K2HoaeD/iwP3P++UmX
53BHWB8FJQRUaGTbs684X1lpCVNJKs29W5KQ5cQlhuJcSqXe+I+gNC0nyeiwdWWUfFBZeQFlZkWejV 53M1itnYhzl4UZt/UPYFKEdy14W5Y9C06DLukxWtXo07c5oYY9fQ6+WcxolSmOCtO8Dbt79cxyXzsq
54Af0pIYzggMxYgWAFzQzhpXisHVY14CzQn1yeXk0nhbWt9ceX4lF/tdZxOoRVt8iwnh8wuih+9fjm 54jiisizMc1tOr58iHUBitWzRnsPPuhv1TeG06vXp8o3tWht4qy7TWrHsAm7869YXxHMzz2xvb8zxU
55gm883QqafyIYwfNsidHgCmNtfQAUUf/9f8WvbUhdhBe3zJm1vs5vUvbX9+mlsCwfMNbOWLc9hhQ8 55VnnbysHkHN3FAXfDJjWgoLDKjp6dznI0rByX1b1eTdFZ3ROD7Zis8rACJVMps8jTWRddA0I6Hqts
56/DNS9os7iOZZxjbyHIgnBc98XAOWmdXUil0yaqHwzdSwDbQ6NOWvRKiF70nNSvtapnzAukQBoCc/ 56rFCmh0uABX3+n6HDzroaVo3DqiYNqWNmtZDhSQI5DqueAMn5LEcsYiRsqFVfWSUXs1zhWt1m8K36
57lxIWdpLhMNXTrI1yVQH8lIalYjWF6hwXm7VQ+KE0hqWo80213VbLoBY+n4KVopqaVo6zT4r1FXVW 572nqak+qeFCTxrXpdneekehIW3SpTl/7IoUvGti7xZ4AeLmsd2aqKNGs2OwNfCIKt+4mEQLO6pwRF
58kgd0mHJWjlMrhVoo/IkiLO1sbzG4decpP+veywfnbAQ7DNrnHNw1wD+lGA9oW9toC2spBzRysJ3/ 58ZOsC7eBPUQgyyNYk7Xj67a3vAgXLZ03jWmXaORVV0cK1bieSE2RN/bD+sE6QdWZirPUJqlmy4FgQ
59RoD2gxzt/6rY9QJxq/ONdgY5CYwc4kHR/9vPsSt5qzwrXYZ5xFxihQqKqxRm4FqIFD6wiz4vzcpI 59p1VLiI2xsVrTYp0rVmtRbE4Yq7UuNteO1arNCFWteK1Fod6VjtX6nhBZHMZrNW5fpGGsaXTru3Fz
60zc0pUvrUE5a9vrvPYTNYObte2IMTgPkpl13WHahfuhRmp6AsObDaX58Rucaiuu07ydVIlIgNO/By 60BcSaQbcaGTu9wGlNxWxVZ/g334Ktave13W6/drt9SKtRY3mxSe/dFzO6baUxDuXldzirkQWcWMIG
61RsfNdDq9ui+QQ29/ClRRbOC0dSDOqsDAcwiraOHgUjkBPZXOpKuPRyb7+zzjYpA5kVolD+7Fx+yJ 61rPpy0/DGRVuFsqpGF5meh7B27yycotw+6HHbUn47N/+flz6M1UzZxDR7NVjwXNZSTem55RxFy2zo
62F8Gl8+WkqNpyM+V8qo0tS+rqqWERxq4kVPcgZmWLyF1Pwr3QZOzp7iYZ6iP3phakjvtJZUlhOzYt 62iy8wVk02N9FSrOXAs6fZvtBN160HX9xeG9o7GOsQm0gv8lv/8JvRpgSpEcoJLZbm3NDCTqdZ2tY5
63Tgeh848f7TxrF/Kc833j3tTUnZTmJCZbF1fr4VPXXYXHtYrN6rpIBG+HdgWB895+g0FTDhWc/QBl 63FKQNjfIQFLfGP36BsWrv9i9OzfFY53XKSeshOFpGGnRhrI5zzulUhi4XxodxPxnU24ewMGquAmTV
64DccAv5EvQ9Y3AL1e2DCV+i4ntIdqk62KTAtgoV13JioAPIkE+4byfMPfpHNFoi6WVKZ3ALFeFrEi 64BwXHNnVyNhNdxFJO6sX9Q3gYxasNZfVdeU6m0unM/HzksEVFNRv2BsyqZ20mQY5kyhnuCyTz0Qlg
65r/WZroZ/FJ003fNyCXRhHWSNDq7cBRyCrN/I/2Q6FOGros26d1cfGudoLKqDfkAkV9eFrB/7HeFJ 65xL2RsSqcNeDie2hQUh8eLmiSgO2+y3t5loWaaoQWK9bexXxHTy5Sc2eMwnnz8O2sBrfsvYlHjn8z
66Ux4QVuvOy7zD33gd48IQK5IAZv3Bz6ZEEb0sVEWPhG/0QWJc4dRCrK/8gOicaZ9nmR72pQPXI23X 66MuCe2nqBYTWzYX87EwFOU2eAZW1gWYctXN/ezmQCksLIgPOH78bqUJuxv21FXf/v/2X8+hal1Zht
67h54ckbLrr6RFEb6OZnoYdW+cIg/BvF/MuhEcxYKy8NyqB/hYd8YnUe0gJIlZoc9qVUWQPt7HmjTP 67nes/pvePGKy+6F8wNOuD/je4uGub/7X9ez9u6wtDtj7cuue2L+1uN07rBX0R0NPVkyv3+ncrr3FZ
68AawcK59HSmNIHcizJrv69p1i1uvAX0idNtVBfOMvOYlyD50Tzi0wB+3JFadPeAhRxKFgpFBYl8pk 68X41m/fzAHZ/NdWM8VoUlBYJG3YaixmDtmivUewHrvbnIvejiW82edf0gEq2GF4tkFU6BYcq6sDhW
69ci/yWPuEZhMeMFDYLHfWdEd53f/E8kSICQ8ALut4Ogq5V3l9Sn3Wa3YlwAlWn/WyXQk4Dkte9Fl3 69tcEyEAQWMgvbx7W2hdN13LDj9TiO1dpkEaQOM7bR+IJqvRDvWsORt+FYiaFYrXRVhK32tuIrovUL
702vQCUCZMBMLsVKRZINdA/0NOhvJHsWRb2w9ctxkI8AU9LetfRXY9V0AVr81Bw8GAiEXwfLvtmQWN 70kNXeq71BtL5AlAFHwg43ZVCsCpD11rbeff/Wh9HuN561AVJeHVYrCTCsVhloPABaX7CsXXjr3QRZ
71QJvAmoK9jmpzy4PWBrv0DLu3zl6K5VmA1IdJPUDxZRyy7bAwL21nHBQTxR+xKxOzOg5ZYpEo/mJZ 71FSzrC5S1NT5b+u6tyg8rivUC36oAWW8bsVlvH+BSANvaAkyB8+/c2nKe3n/nVucB/h2yVXCu7exZ
722wcsZoO+SUCml4Zh+zIyq+OsxpE2oeASOayBpyx7AO/tSe2dOnYdtJq4BsxA0gJx7h9p4PYBe+Vr 721rnH92u9dlq7yPVVzNpyXd/QkK2fRaj3Jw1vumJahdbc7psxr9jWC6BhQI8+trUBRh3uEn2X1tsT
73xP3Ckt2bzhX0Pe2srFqZmQrIRoZUKmj/sf8sg2PFmYx4SSs+RRr7lnciUjTuy4YWww4ISu+QSJ57 73t7WLZ31tiA0GXqp9+Im4huG1tjzU0fUSTCtfgVW8V+NGZ8oY1r6A9d5/ja+LaR3uZfAU2Na5jzq+
74m0cK/9eZiRRwxhW/Mhip5efe5UcFSsjEtLizIe6vcNBh1LKyhVuzMG0rkoFKwhUQALbxwmNZwGLN 74DYViHf5C5s3iW3+jNm76uFaFq2jdKyd+qvPeFqr1Vljqug+HuP/KUrRanxvEcN0zRLVSFoJb5Zws
75ysP+RFAA5O9sYNEeLFnYT74phrUsExnUok7MuYT1bE6t745DzUizuAqX8gbvi0UFml2chWadq1C0 75bSh9dGu7QT8rvFUuGg0qKo61S1m0bpXPJ43gOO9q8VlDCsF9S7kJc/pbFet8qxFSCHSlcn3eiAwf
76AxKpARcDwtQrMmF1VqPkQBhu35Jg1syCguMcgKYdEYSweLXii6jKxrJOLSwYBxJN+95ks5aVbcuI 76Fcl6QygE9y0Ded2gjDv/rRccq11gFR2oCz9fX580mOL8lfBTcawvDbFQiHc2v0frecCNTRxrV4R6
77FexugELYmZDKRIYBH382qQGySCv0tSNxtM9xvvWBR6eBsfyJB/2mzsgX4Nt3IIH2HYyr6TQkRWPJ 77F+/9LAGr0g38qUh3SLjbtBvyQ5GsN1x5ehd+DRrzbg7jH/816hkezDtPLFCaR6OQrG1wqLC1q4fx
78XB/tfZmpANkWKmEkqsD9EqV6gJGC/7Q87IPNxskyi6yG+7Z2Xg75hEhUQLubh81QsaQvcwD2Pfde 78QJ4Zd4ojqJ13X6gf5eOxdn83YQ3RUF6+MD1yyGTtd3VjAyDOlZfXLnMbUVt15g2AUjGUfY0r6J4z
799+FS5+qzKtYBB5udK/DUtvopmCbsve6lCZ/CRGtaeAUQ0XKwdzNVLD0Zxj1H3QIpBxaFUX2WzXZ2 79er27EDe+fOFFUlvVtsIP1IXtbjeWzyfU+q+8UKWvQUf4c8cCf/oLLU6r9Zgod8RoFZTGaW2LdvzY
80cYZBO/nw7tfnsO3CCkDsnmYXlsSunheD0QzLdW6EAhDsA0wGmSSycQYBS+MgFf8AgqmMUgB8rGUN 80rF2Ait+Nx9puNCbE2r9rTIq1DzLix2LtUo36J9fXv+mjUssM906QvbPS1rCt3Yj+f36tC0P3q23r
81hT1bQsOC4FsHdmrhvMULAkHo5ZS1ZIIlPmx74RoKsDFqRdyVBHZmSe0KW6B/hUginQCxcAvNDUrM 81i4Zs7YdQz28Umo1qJSZrcK5eK7Sb/3FZ74KgDPduY7ISV58nCtulsHisXaKU9SqzfVFF0RCt/RsA
826rR+DaI+GKdWX0goNavzx5D1iUTWacSwJXUE0FCdkHV5MitxsllVB7H37xZ0rsmo01rw7Co560wA 82aUxWfwZc81wJjMPa9TfqA0/cxmD1lqtz3nuW+FZvs15wPx6Ab1WgqPhWT7NeCzx0gW69g2rV0RVb
838bCAdRpOD/2sK6uOdXZWZcq/AkDsFrHihIvjOnZrIXW7M89mltpKyHXatNM+1pWdDBQDZ9RWduC5 83PKtrznIidH35Gtn6xdWsYrdssa2uFIC5uIpmdQ6vos9fIltVuAwYWe+QrM6KJfyo6DXunPAFsFlj
84RGadhj0B7sYuDpLCp7LJxMGJmsSsSLAI9e42evN9/67IwKGtBGcUeFcaZ2FSuPhMJCl4MINObZXM 84tIo/gYtsVcCKQKzWeyhrF9168zA5VmWCrC0wq4ZkHdeBh8mxngNYz5GG2KH1C1x1tefaaFYV3vqC
851MI5N8gHVsKZGDes52GQNSkZIPJYBGeQszUwpwLUthMxyBqfDzIhQ9azx5JRrc8wyCqeWtOQXRPu 85ZR3NswCs91jbr975K4C1hVWyvOuCCzBrH81qX10He1j4RsOz2lXrFsh6h2jtn8NOs9qIVpAH7xxD
86P3nKbsNegRXzzH24chEYj1ayeoG/BKvFLmCVcFnIrnA0LfFuwulKfxsJPm91kUmJtSkwC1u8wpEa 86QRfTOixbJ0DlVUO1dmEmLy20dHXuFb+CVFira70iW2E+7sZKJRXbamHPxVYxJ0gTF69V+3IuWgpu
87d4XFgLtYzq6PwrsfkTYsCpn2jMWMPLnWoncc1FxrIt2FV+ZSy1rMvBLKbc4/PFD2QEwDoaUp4B62 870SqW74zTvDkiUmMVvBTwn3OrL4oI9gKtCpDvZajdtqex7tlSoBufVTPukP2f/evvlfPPbClwo8Vq
882VOBHGtTbP9g2rSgQ7CXx8yMb2NwduWmf8gbB2Ww0NrquBxrYrhgRTlRg7VYe0bKrsmhbXnAG9hS 881bR/9L4Yl1r/95qpjJ2gFVct+j6hwjTuojZrpPWCqYidYzZrpJVpnqjgjQO01hZTs3YnwqogrbUR
89bGdiz0Ui/MJD+KYyzBws+a27UnFLxOxfW7C1KFplhLXFrwJbs8sIq38dxJoItok6GaL7j5/3Ld7d 89rLeuDKhv78tQL5JnqAOU+Wp+8HPX/sbhiyHSc0vbZVmOxUo/P/jsqgGq98UBunlfxrW+0m4aeJK1
90siTYmTL5q5jXN71sCbZTpi4Qs07zKdaSXY/J1FsSrE/weYEl1q3iFpEE6TSWjiNkS6zOwoS8RIqS 90Tv4s89m5Ir84ytqnXIgrnjlLMfST4pfKMoKVMgm8VC0d+Sn8c9sytFWlaVgfVaV7w8Fc+R3Sai5s
91HJSBXZ2ZMkWMBDSfHNqya2dyB16CEh/CFTTW1u2qyuLW4aQx0YF0vQZowd6SjSFHwLFaTLfNTC6O 91Iu4T+AtrnfrdEbNFGc5qlq2wKcG9+SH57llAkeVVF7OUrUthNS9EBl8qMT/j4Kbr/h73O0/m5tJp
921V7N1SnswUuJwWO1uNdgq5mMwPMD9iTgOAsFa0ZSZuWCrK2UEHrC+3bL0SQexdaYy2y2M5xjJgTL 927xtO3LEgw1gtbMBbflrm9pXvFVru927YL41ZnDfQZPPsPoh1eIP7phUg9T8mKrsZvpdIzaVnk+xa
938gG7e7kWm0gLWQJrtVfoOLfQbjetwfJXaxU3idtnJZY3kq+EFVy20myWYazUTgloMmCzp0lwV2h6 93ymeizMW45xLs8OMYbgjz1W0XIuDtk/MZL3i2DmHV+i/W/t+5dUtbUYZPkyvEmbU7XUPehbU4l3Z5
94LaqADFCdTr0wSxsvWaA6W9foTCyKestWc8ifDm+TWDiKuxyKajO08rQaoetRXMFus5dcBaqM9KGL 94F1QAK/GRo5uXgB+dJKZrkDfjeL9gsg5hNbkv9sdwGE/hBbZBRLqGc5dUEOswiC/DcMY+Tbp6X4aX
95pFdnsrn/EJvmWsslFjx5LZzG2dXMzIwKYfH+nJakTWUResVLXNvsrheGCs0VS1B9pxRsl6AcIGtS 95nrZnC++A1shwjwS0bxxctF+Ok5RjtLomLtMsb5mbDsOiWN0vi2J5fd9iOgSLYXV3LcZX0M6biZBU
96x5kJPeyjKvELk97K9O5jE98HuwRN0ons6UpIuu2szbQ6tsnQCViXS1bfLUtgH3tFZwb7RiJ5zwDW 96Y7KWudJ11LRmSUjHZHW//oz59ZjW+waX4rG6ulaS3Zo1h716LFbWkYCMnVVjsMocIwHpVY7FGKx1
97J2Ry7uW7wdRfceZudj4qxDsTzK0HoQRe/dDjcLzn1b8IzftFSyqbts08c7YEEuV5P/rXcBfW8i88 97oa7leD/iO751W7BrWdVg2t+9EKwLol1rlLLv6NYM96jlipQvYxGsIqOWcwDT/yjxWrnfm242bB3Z
98/lH+0Zdffvn3EPnjn/jEJ85WVVWV0JiM66e+AIy4pOxDPlJbDbSk962q6uz8u9/4o/9bsmgRTlM/ 98KkOUgWHGLsVqzQrEdGIW2brNO3klDQhyjNaUiHXOk7Dw1iUw66L9cnc0awamZBkxk1iYGGsqkZkY
99/Oqr73n88X95+bfwq17+0IfRw5UWexDi615c1XlrW1lT5aLY7tqHX/VUWWmvZSZC3doJdlPNVYhE 99axrbmgQaCuKwQg0FP6xo1lnsOgBoTWHXV0DrdGJ/UqwL6POBtPgqdhiZxIyGa4UbC1Lo81cw6wL+
100TSVivXsf3KCkAAo1UArWlWfgpr+5lYp91RKwVsGtwmUPolxEaSzJem5VwY3C6jaFF7Xc2vJJ5NTC 100umABap4165lqf8fz1wXPjBDDug+0hkklZlRsK9DaUE/7soZtVRMQRWuesF3MYu29vb09Pw7j6lSP
101pGXcZ/FULJvNZ7cx+GK8jS5qc5IU8sPZ5cx38d7ENan7Ertt96KoFohRIz/fJmVVkBBkNLm2km20 101q8dRvI2+bFZ4S1OnTnvGAUpr/+1Jl32ohsfj6OuX2Bey3mnunE71nySHW78+X51W6eJ09E1l9o2X
102igWT/yoezCR1riJ7U/XWAJ+wtVOFdwTHyGJo1AccJQWAvFYGkYuizk09rcgFzbWeoMINt2hIl3Zx 102dNKZ1+ZhTFFjsH59pGV6rO/s+9p6dqbsfynrSI5wXBBkbV9JVbaQNHLChp1yO2upUYzTelhLoLJG
103tn7AttdaSfdRazX7EP6jtnaqUMGyjfSK/Z0I47ZYW6HGsOgTWChdUZhFcYc2WV1FgZ+8MKRWcAF2 103af3KDDVi/P1F9p1Czz0C2nPD5w9VrvgaUGGnFxitM9sBJ91e6/NulTPGRctz1YFinM2/l4fTiJnM
104QxfOVuDQaPcyMy+3Gbo8s5pZAZ5rsZnifUJBL2RRyz5sMXR5n/uQafCIZ5/FQoaFAbl912LWRfaK 104PuWZ/Bu3tFp9Gv8Yz+2s6LFryxxEZDn8QqfT2j+tCsSjFtS7phejrIdUY5HD2pZEqNWrwPlL9AFH
105A+awTLG22Pu4pucFzOwNBQ24TlHjIe3z3Ie/zMytafY6Gsy5gv+IxShzRKW1MMuz6iaE+EqWWKtm 105j9H6WBWLU+ePzSRYimxRY7KK/f19VjnBgu0wWfsMhWpPcgSxwGreA7lw7IHGYo2iStJOrfbx48cm
106eVbtTQCIdVGZrbyQ/1ididwFLHFZ21vD+QGpDzlKKNbabkDfhwDNeII1tibXWV6wZjYxNpVZKrq2 106KWokq+8CbGpRMANs62mIsnZ81gyNj77BwKxA3muD0wGla2HAZL0KgtYimC7rm/sny74LgynCCLaY
107+lj1ixigkSZrkcv/8Wrt4hCwVlpzBP4PWGsX3YjVVpT1cm2kBs1mBnQTlWXWilmf15L7pGOy46q0 107Lmss1jYRunPcpIzhN3gf3Nv3YROzHu1iZnpGZbH2JFKLXjabjNZHLRqbSKZHqbA4NzsdMKMKtPqT
1085rQcLoNFlk3cvdQiUZNVWvyKv4Bhy5bEm61FxlGADSxpsz7h6wKKLYuHbVlTalavmUF9bYxlwT4q 108VTpusoQUYNX2yVddk6nU6OpuRmOx+jOg1mzCWDV5NhEeaZXJ6p2uSJeMVLtonRJ+vJoJpzJd2596
109mZoM2NViLRv6toXIjAtuTpLZigeOs7joHv6+hfCawSL0kdvSszq0/8oF3Cd8i4anyZ5PmW1CYNuF 109FmzUcGtgHlgJwPZs6tSuMDXcqqlLAdKZssYWU+7h8yMH1S6wu0G/471IaNuZIvMDvy7r3llTwFoN
110RbuCnTW+tAD/wRZjT+9mH2OW6STaZnWCjgvjrmkCg35gDapahhVcmFW7OouTvs0Eca+4K8kKDrPL 110/i3q/oKbm9nX2MNlPW7yRaTVrAnlYiaTSWcyC9ucTxpNCeYqvVU8HFaJl9rci916xm2V4rZyZ0D8
111KoRdsRimuhJ6hV9AZZlVAHvfYgdmZBJ+AH41hm3WRFjY/IOSlmCFW6+tszrIclFjG4pDMCuXqMjg 1111r1PwtZ+XFaBZrUHg7e4rJeTY5Wak2OtAVi1mKxnE2QVodoTgpis0g8rirUGYJUmyHo6EdZarFYJ
1125y8yYHXI6nGAtxPXT/BRibIcbmjNgjXqe5jYF3GhKxBvK4FHCfMlMwesPMO3N7Zxv5uBWBMSRpxK 112wnooT4z137PZwmFPQ7fuAVg3zJ2fwqGKZt0Vn7pY1j+ON1RlpM+kagN0LnP++rNzA7AsDxCsw603
113oo8LmUGReZeqzpnQ28685dtICb7JBQzhMjj6fj+Zs9g8BlfnwgUQuFs0gxVkwZXiGScSAYzI3y6s 113SdT653/17qzL8NbhlualoPVn/z4wdE+bsreKdwSsxvf/RD4LgMwFY1/7WbBhz/Tv/lPwKUsH0mph
114Aq8Fv8zHWs0tLxviyZLTF5Q7/kMWv2uWCE46NMmGRUnZ156Wt4CdI2namJS/4M82/X5hnx0E6Xel 114+TO2Rk6BcS7UVTir9iaJlALd+ueIQ0yQxrXP4k4F1gZSeLMOG1c8c0fnsb0r3m3CZmDP8jauCmQ1
115EfjBhHiAPoXxJndvqWFZ+zPex55GZn23u3GgsNTyr1wmm9hrf8b6LVyVfcB1D+0s1GX1s3GR1Cz4 115MuHxry4DZW87pmlWK36Roax6fP11LK1RlrEaZbMKp4Lvvkvn6ddf//brX2sS9Xas5JgKUES+PoCy
116xnUTycdx3wK+dbv9XKGwub50/sArIGMkSz6R/Rz63Zg54DePC/us/zBnjBa4yjyy9Um+6vdd9Cdu 116WvFMv7YNSoFlcG2A9ZG+4O6QUyBXKq0FcvNc3UzYehnQrIWKHqW1XBCXo21DrDWRZs2WKlaUCjmo
117Nl4Gvz9fVyJcLgOLgKUfxfsU93M813aB33Sv25fpb8hhS/NdhNBXzbGvF3uv/9e5EO7m+qzdgq+j 117TAjJ1xplsxLHrJXKODYDuPk6iPWNduYVNGZtVpxR2lgJqGAA1j5lvh4HjFm5ijcCUvcXVdhqrhai
118ONcfbBnqm0I/JNZ+EPwGfeHpjI3r73jwbaTlrDXztqgfPZsD3EJhcwY/1epNtUAPfO5uEsJauO+6 118d+Uv96rk4rpWIcRmgfSlhwNR6xNNwn6SqhE9i6Zx87KgtS9RJIFeA6of8kEFixykxi0PhKw0lcCk
119gB4rYmzclTUt2iLPj4crweC/VSwsAQqOnNjm7JTAth4GmdnfvBcImvca8GEZBl9uf0w07ieTfm+y 119ytE9K7px6Zo2+A70h4izZDMBPvTLVD3LExs+7S8DEauRBSHYS4N62h+QmnW9Ehmb3iKW7whYta9S
120/TL82anMwll0Q5FxR84sDn8ONG1WQhCo4H2in/Fsh3OscDybTAG2NmPHl0So4PnrEHb7Oum8WedA 120yIWC2p51NVNm6FkR2rqA1cSSc9ZsVMm4kFVm61nuxC2w5UHo8wXmKkzyac92zC1M40hTZe5Zbq27
121/zK+HztRrcwhgEBWqMvG2cZ/STLIXGXGIQhbOK5jMOnXxsG+E2eD4oFhsynKYmADWUsCM4bdnskM 121bfM9fuvw+vZezZEJZzVzn+bUOniTeXpWcCbkZQGr1n/cte9AGiFZVwVO7SPCIu2YFaJ11QRZwKr5
122i4R9h9ig7IgsYSMmmAoqqMigYTOybFXwu/3VUF33YIaw/q/3f6cwCIQEguJtIaMWzUr0WU84/uO3 122n4s4fRodaKvcPcsZrnp7IGQ1etnz46kZV0+uQ1dSCqxWmKO0SoUVe36LlAIbFY5wJsIBilUV7Vnk
123ZT0ATwwz8Ky8AfBxdztvgZby96b7/XnQ8YDjKWlfaeLAzSl/fh5F20Ihu5TWcepT/u6468LsG4Tb 123+nWAYZUFimtY0x4gWMsQPYtUbI/grXApMCwI4aVLxAqaAlbT5sJGMBHrIWgKWFh7+6MwgLUWgFPA
1247OrG2rqUZnXdjdiwS03cXKn3OJ5OrWiWYV9QyKr1tbqQ2qyg7YWnVyGbDNHZN6AQWkOH9mHW7VJ3 124hS2CWnvgKWBGIbB/CVhl+BRwYXuA1i2EFHBiIT/zDycFHNgjMGtHaJZN08HyYM/Jk9K1AhTDOnsA
125UPeg1eJf7U26FqLYbBzXvoLqz8770WGTA/qCDFKu1YVzOhJw3WuYNYvMu15nZqGrJKwF6062tq4g 125ZS2DTAeDsNYPVIGseaDpIDnWCQ3LbVWRKpYdK/6G5bYS0nUZkDpM2QMQ6yFaxXJlwQDCWkSrWHbk
12691vj8cYnIrBfJRzQlYDrEk9gX7HbtSXgzqEisJzDrCtoegEgDca6Wdd9Jr/+eCF12sIE3E7tajl4 126vJNDbityuuqx4R28eK09rAHW27164lYZO1312PT0Ll7rEddOJmMYQ21B3HqAnq52KVCFrVt4kwFH
127wd8O1R6M1WrD4GqhcE0bFReJ1gMt+PV2vaAFL5TptVCwGbvqCuK8Reh+qc8ClrVYIAAvMCKUwGUR 127jXVXAl5rHnckGIYxOSyLWgeYExd378qLWjvCW5l0YfzgDrx1A8O66kxYTutRTNZ151IGzlrCsBqV
128LK4Q0bDotfBvCAtGn+AQmmhBVospN/jla/jjzMmjT3AMLrnwsMdaC95dXBIAOV5PuBpSymJWe32N 128YAveWkEJ52gAZl3BsRpDl6D1IC7rqrh1Ky7rmmO+PQnWDrR1Fce6gWFd+2GdNOtggqzgNauAY10X
129VvDu9PfE42ng1ImXAW4U2bcrkKsEK1oeGImZYJwPsKqBz/6TFCvW40CUDtDiHD+steV/+BUF1kJh 129tx7FWLPyk2ItiM9d5FiWMBVzj+BI0NqJcZ4lOtdWY5y/qoJWLaZ1wQbA2pCwTYi13joQtpZjKbAl
13058GAbQ8GSe1Vs7/7h9AiYr2Cg45j2w3sOoh+gB5U2nP6aIbNm9VWojX67x+TYoXus5XAIq6REdIS 130kL0MGX+b0JsCkHuaCJ1rGWSfkLBJtIHRrM6NbW5rHAm74trSBD2HKeE2K+j5FnASGKdGWxqElVBh
131CNx/8Kcll3X05/I/LcUKS79W5+kwWPgRS6Xs4Z/Nd30Dni3JdaLbDd1Aq1MLwoZwWDHr6NSefP7J 131gacEq1nPJ22Cnh2zJkHo129mPc0qYB2In24UIjIgPwCyAhwZrBRCqd5bWgLWjnDDrgR+g0n1XtUD
132P4cnF6avcPkVpCQSsDYacEenOm7k8/nyNyD1CSIsSvhg+lQrkoH++tHoAt/YP3VqS7G7C6DmryBW 132vvPE1rD6AmWF+B3mnf7iANBK6F1sKxmjp+f8p7jWnU0fVew+YUHw/Ni6I7BaIjQqgQp+pzRXYrbq
133UcWNpjtO9WDoiB9LNXLXo3em7s3vKIZG9w1EyliT6wKcR5O0tHVzPGpqrS6YIpS3u8vLu7oIHEZk 1332vG/YWnduupSBL6jRzyRY9mDcyZPYW1jfa1g39A7ID1qIGYlXdOjP5Y9kEnPKGWzQbfhBd+3Qbqq
134o/w70ZmTO0TY3dOOSm19XTTtvnQ9l9H997Atu8u7GgJ4flZy2qQihiRRXi0cRbv5eCpSCrppz/lE 134ST0tkDV1iyTdovzsetbBKy+QssZg3/FptwIf4hF9jwnxwQ067F+GeXToKCf5sA8qEH7nCikLlku0
135TgjdhfWaZFjSVvOHo6v7OCnULa1P1yU+egrrs3tHkj3Zc4Q1vpKdQwJUyBnVtrbWg3G1Ne18WjAf 135zWqFKh8dbG1tHcpCz8NQZEGRs2UPmH+V+LtseqTOnIvuYO/fwEp+3CyydHF8gAbEO4LI2JVSeL1i
1363/s9F6RI8/m3EhEMkCAf9F4si049b2L6KJS0XHjv2ZVQ1sJAVAKz0cuizG4rPUpsekXWphD4u705 136j/8H9g/ABkdaki4AAAlPdEVYdENvbW1lbnQAICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
1373RdMpdEmNzrS3ukos45O4bt/O9o5xUni4xzOQB/nDub0cWW02SJqwSMY9Yb87Uf43/T7yp0jl/v6 137ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
138+i6P+DjhIQaTfWLUTeclp5R3WFLI9J6rM9apHN1C7v8lZdT8/0jBGvMCh0nE36B4/9GFfU6G1VgC 138ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
139rYWa/zosDkXDlFop6qY0VgWG/bwI1NwKNkUtptAqEsGfCg272ZBvZajl6tOKvIILB5EmrkuXmIQ8 139ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
140B/UAxQ2pUb2cIFoMplCdUzTlV4tW/gv7+tsTJKtTlvhMu4CidmuYNZ//TVwdRAyDHzMigbVYlEtW 140ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
1414y7o/Pd/Jho1ZVkSFeT2s6JPy6z5hmLxx5eGaLc/ZWj+I3RmVj0J5PPdxQ5n3XGurbJ934FU9VNs 141ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
1422nCHmTW9w8I3pLxYPAxP0wrqkqutrWYxET+pVQHxk3oayHcVi1PGMqnIN/KaFDoeC11msbjFLqvX 142ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
143TtFm3VAsjlqF9Vh1NZDfUyzeyYhVd27lnywWb2bEquuz4OSanxFrUdMPwGhQrLEJy7Uq1WvCwNWB 143ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
1449/rljFi1RdBdLP63TcPyLWBdrwVYv1xt0bA8q65hbxSLX3UH7cGy1AVCayoWZATF5h57KrjJG3aT 144ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
145Ssct7DYg69+6R6wZ9jDPqlVxoUyr+Eyba0+ypIlJkPckONkGkUQg6+s5i7BeAotw42EbNoj8xHnw 145ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
146+sdyAPaILc36DRubxjbsKUqy5oabeyx5A79i4yzbADI+ESvUwIs5MOYBHcyyMsV8rgDKIKKpdQmg 146ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
147FkVeArLeD1kt0pLOK/MIG0JtrR2b4JOixhzHmsv1D689st68bY+GVlv9XfgLIHjCIaMBYlds3Ivf 147ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
148VW0cd4HPycI/Nt1gtj1/BT7QAaagKqsdXK+h4VHfvlJeXn6jG939YsdhCVZoft6uRLoXf9GsdaNg 148ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
149OWN3HHaAqEVVTjQrIB5vqzgx25x2E2EBqgPIbwgqB3gLwnZF1u2/ODZj0liESICFqJBV1J+FNyKG 149ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
150FeAOj7nVpoy7IGLvBZLBFtT6ETstUBsmsSLaRkN+ga7FBpzCKdyggI8mdz1BzZ3Mimh7lpkR7lTY 150ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
151tI/Q9gQEuZIoWNDLELHmctObTdEeZf14bNyOKdb1QX8nRi4YhhP0ij1Yrr/NdQ2lNqOveRH33n7u 151ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
152diHWJK91AR0BckLRGB9z3UZDs2x0wRQYry3w6wqljklrdTtkWXP9E2DnxgljHiysf5zmJhgWheLi 152ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
153R0RWRc+PN7tuhSGXEDFVyYp97Moi7BCB8YwUa65/CJi20U6O6zgkVsTGAyyB4vNyrMAhANiKSTP+ 153ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
154K/guRzFLbKBFXqBYlEUFvhbowFZlRrse0QEBBS0Jl8VdyyoI22PIIfhNm7wUStLxl+TtmsshWNeG 154ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
155DlhxvilifhG1Ft+swprrXwthbfQ+vNZXSAYNOCMvFt+oxEpgZ5h3tcRrRblZMrGKz6qhgoiLLDvD 155ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
156uPMa5TKbbj4zAG0OMlB3QGlgzVYYn2GcYYubvCWmC9SqRUW5oouyBMsblpW6l5hRVSIBZ/txtK3T 156ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
157uGVfo/ea/nvbMyl86NNKt58ePGwHNtihCbCreSx2YSCjtWBZtuMoaGD89wup7EqcgVth2BskluZK 157ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
158QYu/rHnIGbim/WywqcgbWDEQcLRYsu4Mw2lXoCLjWONbA2JpPIRhTYfb2PklnbpGoPdjFZhOZEa9 158ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
159bSe+GSZRFSbY9yJmNb6UFymDT6d0ApQfJQYgJpjOY/hgS2z7bPqJhWlx+AJFmOH55YR6X9qouRyZ 159ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
160XqYl6zijv+DTqgHU3DxiWNd4zvUdz9zv0b6oqVWsAmpY0172z8B7P//BT0Pc97+kK1UyvahiTavg 160ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
161K2L3rn4EcQWGE4MvqoNIvIJEWsPh60clzqx+SD+dXSYjQs2/qYPIvILOLpNO9kdkTpziGCYCg4b9 161ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
1621xQcMi9hLtacYZetkjlxmmNItmUwh2lMgyH1GtiVxcPQDp9ZE1LnTXMQbMqSYab4OnEyDYbUazjW 162ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
163E0byrQqp06Y66KRn1x4TrOuHUmFIvYilBIDZRLpVPV3qtKkO4lkNiKDGHU+FIfUiWnVBKRgQwTK3 163ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
164X+q0qQ7i5pbr6nuCI2tTUci9iCUE0LD6a3XuQ3KnTXWUF7cAa6+uJ1jvqk2ti/MUoFd5LstERVut 164ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
165OLWmr1WQN25u0qFdd/W4aonLdLdN2rBemoVxNTUwy3WlT40OBF5IGtY3s/RZl6myQu8uCetl2kQF 165ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
166mnatdhUjLIpEUrD+iWVAAz2qrDgSPSSeYHD7Q2Do2RXIVdGuJBKtFc1IProS5Ao91vXARGpzi0XN 166ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
167RLc8zysIPNtqdgx7XcVQgLZf4DE2HHeVYMt01NBMtBqVWaEf6mG0UbIdDroqyq2ZD1Qos8KbWzFJ 167ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
168T9/cNuyLueMX40DBKzQ73HBKq+kVUTpHuHvcPNQ2HY6JIdpri1SA7prMIHhXNT+Aw6ZTMyOaJ+lR 168ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
169zY8ogKilyHoSszqzgHrUhm6DALgBd0xJA3A3HkpCZivCai/OnoAnVmIF2xwxqyqs9i5Z4LJcpZyQ 169ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
170xHgUgJQsq+lbwekQa6xLjzA4lgBJRGfJTzAD67LoZCrdLJKPkMBew7uupJlmYnsZen+FyUUzUpaE 170ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
171TMrMMDO7jrEt5Ks9PLP4YmQWklHi6NWMV8Qw+BzShSzNW3xF3mAyrbHPfWFWWRGQnSGh5eB4WoMf 171ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
1727XDIvZP0BCzch2b1rMmwS6g4MqnfFOKyc8IqZ1gvKY1qoNSs7z1C1dDY2LvMRAfTV0fQOSGjWC5/ 172ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
1731kxE05Uy7M6J25p8qm/03sqSsyncLCj1+vkUWrPGk4ULHOe5m2RY+MkMb+hmd+lgYU5IRnO8DOYF 173ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
1746hLtlCkVLMy12WiL7k+Mh4rSksjVgTUMZ9qJULQdn/DdfXSsZpmfyqjwRcFAPjYxTLSwavziRFRD 174ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
175wnVLIwGSwCanHsFnS+MFgF25ySVLrN31TyuC2bKE7Djje+7k0Wm7R5q5ZGZVF4GBxT95QwaOrJE2 175ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
176KD7QePqkQl6tBFtCBYCLqpGp7uj1GCidVewYOpb1J8UG1l5M0wKFLxZXouQqGs0UpDrAsiootQDk 176ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
177G1MVpUoDArdBootm7aOvyoqoEWi2pyTVYNxlJDWmGr9Nbj9jr+mNdrQ9J0pTBiRro2bZkQDujOpl 177ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
178NkH/HxtpiBH77U/NAAAAAElFTkSuQmCC 178ICAgICAgICBmyrzgAAAAAElFTkSuQmCC
179</one:Data></one:Image></one:Page> 179</one:Data></one:Image></one:Page>
corpus/m6/native-template-controls-01/read/page-002.pdf
Binary files a/corpus/m6/native-template-controls-01/read/page-002.pdf and b/corpus/m6/native-template-controls-01/read/page-002.pdf differ
corpus/m6/native-template-controls-01/read/page-002.xml+178-1
...@@ -1,2 +1,179 @@...@@ -1,2 +1,179 @@
1<?xml version="1.0"?> 1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{F3B3040C-0089-05FF-2C9F-3BFB91647BCD}{1}{B0}" name="A notebook created in Rust." dateTime="2026-09-05T07:11:28.000Z" lastModifiedTime="2026-09-06T04:21:25.000Z" pageLevel="1" style="font-family:Calibri;font-size:11.0pt" lang="en-US"><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="Fixture Author" lastModifiedBy="Fixture Author" lastModifiedTime="2026-09-05T07:11:28.000Z" objectID="{9C3D193E-0294-060F-08C2-1BB16FEA3244}{23}{B0}"><one:Position x="36.0" y="36.0" z="0"/><one:Size width="191.1661224365234" height="13.4277172088623"/><one:Indents><one:Indent level="0" indent="0.0"/></one:Indents><one:OEChildren><one:OE creationTime="2026-09-05T07:11:28.000Z" lastModifiedTime="2026-09-05T07:11:28.000Z" objectID="{9C3D193E-0294-060F-08C2-1BB16FEA3244}{24}{B0}" alignment="left"><one:T><![CDATA[A notebook created in Rust.]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page> 2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{ECFB9395-741E-4CC0-B9F7-1B81B0CB48BD}{1}{B0}" name="Default template page" dateTime="2026-09-06T04:24:38.000Z" lastModifiedTime="2026-09-06T04:24:51.000Z" pageLevel="1" stationeryName="Hearts"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Title lang="en-US"><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-06T04:24:51.000Z" lastModifiedTime="2026-09-06T04:24:51.000Z" objectID="{3EFE148C-FDB8-4E86-B785-00EC646C8E95}{18}{B0}" alignment="left" quickStyleIndex="0" style="font-family:Calibri;font-size:17.0pt"><one:Meta name="oeLocID" content="0001"/><one:Meta name="oeLocComment" content=""/><one:T><![CDATA[Default template page]]></one:T></one:OE></one:Title><one:Image format="png" backgroundImage="true" originalPageNumber="0" lastModifiedTime="2009-07-28T18:49:14.000Z" objectID="{3EFE148C-FDB8-4E86-B785-00EC646C8E95}{14}{B0}"><one:Position x="-90.0" y="-3.599996089935303" z="0"/><one:Size width="89.21739196777343" height="287.9999694824219" isSetByUser="true"/><one:Data>iVBORw0KGgoAAAANSUhEUgAAAKsAAAIoCAMAAAAhjet0AAADAFBMVEUAAAABAQECAgIDAwMXHhna
3kbrakbrakbrakbrakbrakbrakbrakbrakbrakrvakrvblr3bmr/TnLjbnMDcn8LdosPPo7bdpsXd
4q8jdr8ncssqEtpHdt83duc7du8/evNDdvNDcvM/cvM/cvM/cvM/cvM/cvM/cvM/cvM/cvM/cvM/c
5vM/cvc/cvc/avc/TvtDTv9DTv9DUv9DYwNHbwNHcwNHcwdLewdLcwtLdw9PdxNPdxtS6yK7dyNXd
6ydXeydbgydfizNrnzd3lztzd0Nnd0Nnd0Njd0Nnd0Nnd0Nnd0Nno0d/d0dnd0dnd0dnd0dnd0dnd
70dnd0dnd0dnd0dnd0dnd0dmY0qeh0q2Z06iZ06iZ06iZ06iZ06iZ06iZ06iZ06iZ06ia06ie06mZ
806ig06uk1LTq1OHh1Nyc1Kib1Kie1ana1duq1sHt1+Tl1+Dr2+Ss287u3Oex3Kux3Ku03auz3auw
93t6w3t6w3t7v3uiw3t6w3t6w3t6x3t6x3t6w3t6w3t6w3t6w3t7V3uC938i737+937m336uy396z
1039+939W9387U3+C24N/y4Ory4Oq64Ku54Ku/4La/4LS94Ni44d+84ay94ay64uC+4qy+4qy74uC8
114uC94q694q++4qy+4qy+4qy+4qy+4qy+4qzy4uu+4ty+4qy+4q2+4qy+4qy+4+C+4+K+49+94+G/
125OHQ5OPN6eXN6eLN6ebP6uXR6+TS6+LS6+XS6+XS6+XS6+XS6+XS6+XS6+XS6+XS6+XT7OHT7N/T
137N/T7ODT7OLT7OLT7OTT7OLX7ujc8Ovl8/Hm9PHn9PPp9fTs9vXy9vfw9/fv9/f09/j39/j5+vr7
14/fz9/f39/f39/f78/v79/v79/v79/v79/v79/v79/v78/v79/v79/v79/v7/////////////////
15////////////////////////////////////////////////////////////////////////////
16//////////////+3cRl2AAAAGXRFWHRTbm93Ym91bmQgdGVtcGxhdGUAaGVhcnRzVrgCNgAAGoNJ
17REFUeNrt3btPI1uaAHBSR+AxWhsEInLoXd0AjMwFDNeZI0czEzkCjahtQFhNYMlkyyKkJeAO9BJc
18jeSgpI26hCEiWQHSCkPftmzKnmz+h/oLth4uux6nqs7j+6rbmv6kkWbm8vjdw3e+8yzXlMYYg2I2
19OuoaRkxhULPZYu87sJazlCF/c+uRBcmt6LEcji0Pvq21pxtWNzYrwyhtrq3kgvNg8E2tv+Q2ShV3
20lDZWgrD53je0djYqpCgV4sGyWDuVoCitxoGdgqDqsUnuafnBN7GGUgMTAbKDTQFR9dggloTyN7Ae
21RForJWIeHMVulSsVXmwnZuvf/1LhxhYGmNaeXP9Fj0P5fUCdrSHYQzSreujs0WVrDnJYocXm8LLA
22a1UPfCWyPtD+XqGOTbws8FjreVJBlzv01soaWi1wWXtBE+ktBmtlBWv4clrlvD07Lazp4ZjurbFY
23SSl7CGyVh8m1Oc69Qo7DWlkn/GVUUKtFXXHPT0trpnaDyVpBatgpFzW3SZ7usbUrqRZAZKxt7Rg/
24cLkU0LEZrZVVlFIwtKpGt1otBXWWAqO1RKixYFajWK0E/+rVinjDdoCsR4EJMMSyWgkZewBjNTIg
25x+xhLAV5GKsxB9gEpZJqbAfCatSAVVgqqXcdQlgPwDNAD/9EtghgVdmLPUUQplsDceshRrOSkkAW
26t+pFoFCBjxz40DVlTgRKCFb/cLAlbDV6FgKVkLB5YWsBJwVIQ5dg55rqsU9P6WIDfDSYMtK1ghLg
27hWBKr1jLOFbwQjC1hZSupPWsqBVl0Aqwbv1zWTdwrIUJsq79sMZkzf5zWdfj6lsFcesk1azCBFlX
28JsianRTrFtIShjTPEjzvNOaESEULfF/TnGujdK4SvNVYw6CsDTfgrcb2APTGmxnr8GsYc5MQIwnW
294NeG5l4Gwh4RaWtbdM1tbr1hVIJl6KmLsZ9VxOldWYQ9IutkC7x3baBYB1mMOcE6/BGXsa9dxsjY
30Ao7VOt7E71odAKtmtgHw6iCLsgdvNyxojd1AOOW0zg0L4N2LMGqVYazWdWHIddcKfNcaWgdWrwXM
31gizSuaGdsYApu4HQtUZ3HbbMH7cMhSWkK9DZsWbdGgfEriB0rfF9l0PrJwKdIWfhTw0dVvuBjFWI
32liVd1OvBWbVeHi4NCjTp2nt7ejTi6a3ParXvkkFgc1ETwv7z1W51HLtXX9msmn1FMyc65yIst10j
33wduVZBIlSarp/7G4z0zW8TNEgvOY9dCR4O3UdNbOmsP4dLxjeHfbDFYHVqyHkZ46GWWpKd25bLqj
34ZmhP+/RW60qZcB6QUsBO1yeJJDW1+j/40Ke3joqBMYXjbtqNwOraNxpV+tgkxvGengd9eqsTm+Nd
35L64GVdevRt/faQbFmY79wGB1Yr0XTAVSwJxnfzX+/rVmcBzr//yRwaqpzovQPIlAGrSMC3ptnboX
36Rm02d3Rsj8Hqfvwxt8asLZDTta079s5Cqc1L/WuuWKzawPUAJHNFyBHnrpEJYIb+RRKTdfQI5Ehb
37EkyBstbfpaGaheuNzap13M8ZsGQCOQXMYhVJbX4M611Bz+6onkcNctS9jJQC6pWRrJ/orFesVm8e
38GBWMqt6SBoLC1ypVBljWU3Yr4SkOmlQgDQSHlBnAbyU0rfEkL/tAkH00mvUjjbXGbdV6W/5fHJ65
39xBTYpW1W0/rIZx0/I0PLJaXAf9Bmq1lfq8+8Vm1wRPqrZpfXS9QpYDTrXpMuXfdqx39rNv/n16cO
40h5X0+Flw65JWBD/RN+uO48t+7XBY9ZFhKxvE9XS15YBmrR43meO///Pxjd0aotW5m6XQFDCbda/J
41EztV6bHPbA3T6oXMTl7S83s/V0Mn2JFTmWd2q17ADoK12WUzeXNBKVDjs36SnGMu0+cPqAf5EG6u
42sBaUAtVLPqu5UhhhGT8vYyAXsmxhpoDU5A3JgWX9HBJ9dCgzWT+IWWvmn+WZ02o8mZynt1YF0tWI
43PfPftc9rZWlcK12Pm2JJYGUBp9XIXKpP0DHnAnRzrLAkMFe33FajiB0WKK38VHNGO2xYEasxQhyE
44p27hg6j10rJK4lYjdcO4R1WxMqDHcDu5DWEN5bbBrFdAVpNLyt0inHUXzmp2NV9lkB9Fy+vIWgW1
45GqOE7J6NDa6ErWe29Q3YatbdcfKWze0WMesxotWsZMNskAGsNdv6jGM1s6FsPMdfFbZKtvURzWp2
46Nm1oFRhiL6vxWDUA6ygFsPJ13LLCVik265uoddys+kzrO7eOmxV8LIC2Opr1NCYr77Lgcq/qKAPx
47WGviGWCceOBan4WsNedtCA3b+ihidVIh1jCIVhdVeG2IanVTzTMEYatc3p4zYmm7/g5o3XFRrbNE
48Iau6v5BwxuyS58N/n6zfxLyj+Umq+ptVxPq+lPDH7Lbqr1ms663anpsq9cSsKklqRNKh5bKeeRp1
49dIrEa60nE4GRLAtYz3a80tG9Ek7rdiI05t75rJ9qvjYdZQCvdSkREUnZaa1SQnf2qoQYXTHjspYT
500VFmsn76SGpRxz4xr1W2PdMpPYISd4nOevnxuCYFOd23CTis6qz5V07P29sA85lUANa21qz4aIf1
51PyVJ2qtGhPPiA4fV6FezC+7NoIX0tB9bHFm540kTs84mUgv+bbZFgna/LyaV3Efe7NZ6IkPev1z0
52Z4IsRN31XOBltxbnA3eGM77S9UcB6pX3HiSzdbAYsuU+782Df+GWfvBf02K2HoaeD/iwP3P++UmX
53M1itnYhzl4UZt/UPYFKEdy14W5Y9C06DLukxWtXo07c5oYY9fQ6+WcxolSmOCtO8Dbt79cxyXzsq
54jiisizMc1tOr58iHUBitWzRnsPPuhv1TeG06vXp8o3tWht4qy7TWrHsAm7869YXxHMzz2xvb8zxU
55VnnbysHkHN3FAXfDJjWgoLDKjp6dznI0rByX1b1eTdFZ3ROD7Zis8rACJVMps8jTWRddA0I6Hqts
56rFCmh0uABX3+n6HDzroaVo3DqiYNqWNmtZDhSQI5DqueAMn5LEcsYiRsqFVfWSUXs1zhWt1m8K36
572nqak+qeFCTxrXpdneekehIW3SpTl/7IoUvGti7xZ4AeLmsd2aqKNGs2OwNfCIKt+4mEQLO6pwRF
58ZOsC7eBPUQgyyNYk7Xj67a3vAgXLZ03jWmXaORVV0cK1bieSE2RN/bD+sE6QdWZirPUJqlmy4FgQ
59p1VLiI2xsVrTYp0rVmtRbE4Yq7UuNteO1arNCFWteK1Fod6VjtX6nhBZHMZrNW5fpGGsaXTru3Fz
60BcSaQbcaGTu9wGlNxWxVZ/g334Ktave13W6/drt9SKtRY3mxSe/dFzO6baUxDuXldzirkQWcWMIG
61rPpy0/DGRVuFsqpGF5meh7B27yycotw+6HHbUn47N/+flz6M1UzZxDR7NVjwXNZSTem55RxFy2zo
62iy8wVk02N9FSrOXAs6fZvtBN160HX9xeG9o7GOsQm0gv8lv/8JvRpgSpEcoJLZbm3NDCTqdZ2tY5
63FKQNjfIQFLfGP36BsWrv9i9OzfFY53XKSeshOFpGGnRhrI5zzulUhi4XxodxPxnU24ewMGquAmTV
64BwXHNnVyNhNdxFJO6sX9Q3gYxasNZfVdeU6m0unM/HzksEVFNRv2BsyqZ20mQY5kyhnuCyTz0Qlg
65xL2RsSqcNeDie2hQUh8eLmiSgO2+y3t5loWaaoQWK9bexXxHTy5Sc2eMwnnz8O2sBrfsvYlHjn8z
66MuCe2nqBYTWzYX87EwFOU2eAZW1gWYctXN/ezmQCksLIgPOH78bqUJuxv21FXf/v/2X8+hal1Zht
67nes/pvePGKy+6F8wNOuD/je4uGub/7X9ez9u6wtDtj7cuue2L+1uN07rBX0R0NPVkyv3+ncrr3FZ
68X41m/fzAHZ/NdWM8VoUlBYJG3YaixmDtmivUewHrvbnIvejiW82edf0gEq2GF4tkFU6BYcq6sDhW
69tcEyEAQWMgvbx7W2hdN13LDj9TiO1dpkEaQOM7bR+IJqvRDvWsORt+FYiaFYrXRVhK32tuIrovUL
70kNXeq71BtL5AlAFHwg43ZVCsCpD11rbeff/Wh9HuN561AVJeHVYrCTCsVhloPABaX7CsXXjr3QRZ
71FSzrC5S1NT5b+u6tyg8rivUC36oAWW8bsVlvH+BSANvaAkyB8+/c2nKe3n/nVucB/h2yVXCu7exZ
721rnH92u9dlq7yPVVzNpyXd/QkK2fRaj3Jw1vumJahdbc7psxr9jWC6BhQI8+trUBRh3uEn2X1tsT
73t7WLZ31tiA0GXqp9+Im4huG1tjzU0fUSTCtfgVW8V+NGZ8oY1r6A9d5/ja+LaR3uZfAU2Na5jzq+
74DYViHf5C5s3iW3+jNm76uFaFq2jdKyd+qvPeFqr1Vljqug+HuP/KUrRanxvEcN0zRLVSFoJb5Zws
75bSh9dGu7QT8rvFUuGg0qKo61S1m0bpXPJ43gOO9q8VlDCsF9S7kJc/pbFet8qxFSCHSlcn3eiAwf
76Fcl6QygE9y0Ded2gjDv/rRccq11gFR2oCz9fX580mOL8lfBTcawvDbFQiHc2v0frecCNTRxrV4R6
77F+/9LAGr0g38qUh3SLjbtBvyQ5GsN1x5ehd+DRrzbg7jH/816hkezDtPLFCaR6OQrG1wqLC1q4fx
78QJ4Zd4ojqJ13X6gf5eOxdn83YQ3RUF6+MD1yyGTtd3VjAyDOlZfXLnMbUVt15g2AUjGUfY0r6J4z
79er27EDe+fOFFUlvVtsIP1IXtbjeWzyfU+q+8UKWvQUf4c8cCf/oLLU6r9Zgod8RoFZTGaW2LdvzY
80rF2Ait+Nx9puNCbE2r9rTIq1DzLix2LtUo36J9fXv+mjUssM906QvbPS1rCt3Yj+f36tC0P3q23r
81i4Zs7YdQz28Umo1qJSZrcK5eK7Sb/3FZ74KgDPduY7ISV58nCtulsHisXaKU9SqzfVFF0RCt/RsA
82aUxWfwZc81wJjMPa9TfqA0/cxmD1lqtz3nuW+FZvs15wPx6Ab1WgqPhWT7NeCzx0gW69g2rV0RVb
83PKtrznIidH35Gtn6xdWsYrdssa2uFIC5uIpmdQ6vos9fIltVuAwYWe+QrM6KJfyo6DXunPAFsFlj
84tIo/gYtsVcCKQKzWeyhrF9168zA5VmWCrC0wq4ZkHdeBh8mxngNYz5GG2KH1C1x1tefaaFYV3vqC
85ZR3NswCs91jbr975K4C1hVWyvOuCCzBrH81qX10He1j4RsOz2lXrFsh6h2jtn8NOs9qIVpAH7xxD
86QRfTOixbJ0DlVUO1dmEmLy20dHXuFb+CVFira70iW2E+7sZKJRXbamHPxVYxJ0gTF69V+3IuWgpu
870SqW74zTvDkiUmMVvBTwn3OrL4oI9gKtCpDvZajdtqex7tlSoBufVTPukP2f/evvlfPPbClwo8Vq
881bR/9L4Yl1r/95qpjJ2gFVct+j6hwjTuojZrpPWCqYidYzZrpJVpnqjgjQO01hZTs3YnwqogrbUR
89rLeuDKhv78tQL5JnqAOU+Wp+8HPX/sbhiyHSc0vbZVmOxUo/P/jsqgGq98UBunlfxrW+0m4aeJK1
90Tv4s89m5Ir84ytqnXIgrnjlLMfST4pfKMoKVMgm8VC0d+Sn8c9sytFWlaVgfVaV7w8Fc+R3Sai5s
91Iu4T+AtrnfrdEbNFGc5qlq2wKcG9+SH57llAkeVVF7OUrUthNS9EBl8qMT/j4Kbr/h73O0/m5tJp
927xtO3LEgw1gtbMBbflrm9pXvFVru927YL41ZnDfQZPPsPoh1eIP7phUg9T8mKrsZvpdIzaVnk+xa
93ymeizMW45xLs8OMYbgjz1W0XIuDtk/MZL3i2DmHV+i/W/t+5dUtbUYZPkyvEmbU7XUPehbU4l3Z5
94F1QAK/GRo5uXgB+dJKZrkDfjeL9gsg5hNbkv9sdwGE/hBbZBRLqGc5dUEOswiC/DcMY+Tbp6X4aX
95nrZnC++A1shwjwS0bxxctF+Ok5RjtLomLtMsb5mbDsOiWN0vi2J5fd9iOgSLYXV3LcZX0M6biZBU
96Y7KWudJ11LRmSUjHZHW//oz59ZjW+waX4rG6ulaS3Zo1h716LFbWkYCMnVVjsMocIwHpVY7FGKx1
97oa7leD/iO751W7BrWdVg2t+9EKwLol1rlLLv6NYM96jlipQvYxGsIqOWcwDT/yjxWrnfm242bB3Z
98KkOUgWHGLsVqzQrEdGIW2brNO3klDQhyjNaUiHXOk7Dw1iUw66L9cnc0awamZBkxk1iYGGsqkZkY
99axrbmgQaCuKwQg0FP6xo1lnsOgBoTWHXV0DrdGJ/UqwL6POBtPgqdhiZxIyGa4UbC1Lo81cw6wL+
100umABap4165lqf8fz1wXPjBDDug+0hkklZlRsK9DaUE/7soZtVRMQRWuesF3MYu29vb09Pw7j6lSP
101q8dRvI2+bFZ4S1OnTnvGAUpr/+1Jl32ohsfj6OuX2Bey3mnunE71nySHW78+X51W6eJ09E1l9o2X
102dNKZ1+ZhTFFjsH59pGV6rO/s+9p6dqbsfynrSI5wXBBkbV9JVbaQNHLChp1yO2upUYzTelhLoLJG
103af3KDDVi/P1F9p1Czz0C2nPD5w9VrvgaUGGnFxitM9sBJ91e6/NulTPGRctz1YFinM2/l4fTiJnM
104PuWZ/Bu3tFp9Gv8Yz+2s6LFryxxEZDn8QqfT2j+tCsSjFtS7phejrIdUY5HD2pZEqNWrwPlL9AFH
105j9H6WBWLU+ePzSRYimxRY7KK/f19VjnBgu0wWfsMhWpPcgSxwGreA7lw7IHGYo2iStJOrfbx48cm
106KWokq+8CbGpRMANs62mIsnZ81gyNj77BwKxA3muD0wGla2HAZL0KgtYimC7rm/sny74LgynCCLaY
107Lmss1jYRunPcpIzhN3gf3Nv3YROzHu1iZnpGZbH2JFKLXjabjNZHLRqbSKZHqbA4NzsdMKMKtPqT
108VTpusoQUYNX2yVddk6nU6OpuRmOx+jOg1mzCWDV5NhEeaZXJ6p2uSJeMVLtonRJ+vJoJpzJd2596
109FmzUcGtgHlgJwPZs6tSuMDXcqqlLAdKZssYWU+7h8yMH1S6wu0G/471IaNuZIvMDvy7r3llTwFoN
110/i3q/oKbm9nX2MNlPW7yRaTVrAnlYiaTSWcyC9ucTxpNCeYqvVU8HFaJl9rci916xm2V4rZyZ0D8
1111r1PwtZ+XFaBZrUHg7e4rJeTY5Wak2OtAVi1mKxnE2QVodoTgpis0g8rirUGYJUmyHo6EdZarFYJ
112wnooT4z137PZwmFPQ7fuAVg3zJ2fwqGKZt0Vn7pY1j+ON1RlpM+kagN0LnP++rNzA7AsDxCsw603
113SdT653/17qzL8NbhlualoPVn/z4wdE+bsreKdwSsxvf/RD4LgMwFY1/7WbBhz/Tv/lPwKUsH0mph
114+TO2Rk6BcS7UVTir9iaJlALd+ueIQ0yQxrXP4k4F1gZSeLMOG1c8c0fnsb0r3m3CZmDP8jauCmQ1
115MuHxry4DZW87pmlWK36Roax6fP11LK1RlrEaZbMKp4Lvvkvn6ddf//brX2sS9Xas5JgKUES+PoCy
116WvFMv7YNSoFlcG2A9ZG+4O6QUyBXKq0FcvNc3UzYehnQrIWKHqW1XBCXo21DrDWRZs2WKlaUCjmo
117TAjJ1xplsxLHrJXKODYDuPk6iPWNduYVNGZtVpxR2lgJqGAA1j5lvh4HjFm5ijcCUvcXVdhqrhai
118d+Uv96rk4rpWIcRmgfSlhwNR6xNNwn6SqhE9i6Zx87KgtS9RJIFeA6of8kEFixykxi0PhKw0lcCk
119ytE9K7px6Zo2+A70h4izZDMBPvTLVD3LExs+7S8DEauRBSHYS4N62h+QmnW9Ehmb3iKW7whYta9S
120yIWC2p51NVNm6FkR2rqA1cSSc9ZsVMm4kFVm61nuxC2w5UHo8wXmKkzyac92zC1M40hTZe5Zbq27
121bfM9fuvw+vZezZEJZzVzn+bUOniTeXpWcCbkZQGr1n/cte9AGiFZVwVO7SPCIu2YFaJ11QRZwKr5
122n4s4fRodaKvcPcsZrnp7IGQ1etnz46kZV0+uQ1dSCqxWmKO0SoUVe36LlAIbFY5wJsIBilUV7Vnk
123+nWAYZUFimtY0x4gWMsQPYtUbI/grXApMCwI4aVLxAqaAlbT5sJGMBHrIWgKWFh7+6MwgLUWgFPA
124hS2CWnvgKWBGIbB/CVhl+BRwYXuA1i2EFHBiIT/zDycFHNgjMGtHaJZN08HyYM/Jk9K1AhTDOnsA
125ZS2DTAeDsNYPVIGseaDpIDnWCQ3LbVWRKpYdK/6G5bYS0nUZkDpM2QMQ6yFaxXJlwQDCWkSrWHbk
126vJNDbityuuqx4R28eK09rAHW27164lYZO1312PT0Ll7rEddOJmMYQ21B3HqAnq52KVCFrVt4kwFH
127jXVXAl5rHnckGIYxOSyLWgeYExd378qLWjvCW5l0YfzgDrx1A8O66kxYTutRTNZ151IGzlrCsBqV
128YAveWkEJ52gAZl3BsRpDl6D1IC7rqrh1Ky7rmmO+PQnWDrR1Fce6gWFd+2GdNOtggqzgNauAY10X
129tx7FWLPyk2ItiM9d5FiWMBVzj+BI0NqJcZ4lOtdWY5y/qoJWLaZ1wQbA2pCwTYi13joQtpZjKbAl
130kL0MGX+b0JsCkHuaCJ1rGWSfkLBJtIHRrM6NbW5rHAm74trSBD2HKeE2K+j5FnASGKdGWxqElVBh
131gacEq1nPJ22Cnh2zJkHo129mPc0qYB2In24UIjIgPwCyAhwZrBRCqd5bWgLWjnDDrgR+g0n1XtUD
132vvPE1rD6AmWF+B3mnf7iANBK6F1sKxmjp+f8p7jWnU0fVew+YUHw/Ni6I7BaIjQqgQp+pzRXYrbq
1332vG/YWnduupSBL6jRzyRY9mDcyZPYW1jfa1g39A7ID1qIGYlXdOjP5Y9kEnPKGWzQbfhBd+3Qbqq
134ST0tkDV1iyTdovzsetbBKy+QssZg3/FptwIf4hF9jwnxwQ067F+GeXToKCf5sA8qEH7nCikLlku0
135zWqFKh8dbG1tHcpCz8NQZEGRs2UPmH+V+LtseqTOnIvuYO/fwEp+3CyydHF8gAbEO4LI2JVSeL1i
136j/8H9g/ABkdaki4AAAlPdEVYdENvbW1lbnQAICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
137ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
138ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
139ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
140ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
141ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
142ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
143ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
144ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
145ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
146ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
147ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
148ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
149ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
150ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
151ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
152ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
153ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
154ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
155ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
156ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
157ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
158ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
159ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
160ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
161ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
162ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
163ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
164ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
165ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
166ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
167ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
168ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
169ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
170ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
171ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
172ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
173ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
174ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
175ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
176ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
177ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
178ICAgICAgICBmyrzgAAAAAElFTkSuQmCC
179</one:Data></one:Image></one:Page>
corpus/outline-edit/README.md+1-1
...@@ -46,7 +46,7 @@ The controller selects the target again after activating the native window and...@@ -46,7 +46,7 @@ The controller selects the target again after activating the native window and
46transfers XML as UTF-8 data. Captured authoring scripts retain their exact source;46transfers XML as UTF-8 data. Captured authoring scripts retain their exact source;
47identical immutable artifacts link to one canonical corpus copy.47identical immutable artifacts link to one canonical corpus copy.
4848
49`layout/candidate` applies `PreparedEdit::outline` to the five geometry and saved49`layout/candidate` applies Rust outline edits to the five geometry and saved
50collapse cases in `before`. `layout/cold` captures that Rust output through a fresh50collapse cases in `before`. `layout/cold` captures that Rust output through a fresh
51OneNote cache. Every active identity, child/content/structure reference, child level,51OneNote cache. Every active identity, child/content/structure reference, child level,
52and collapse default survives; text, fields, tags and explicit formatting compare52and collapse default survives; text, fields, tags and explicit formatting compare
corpus/page-model/README.md+1-1
...@@ -1,6 +1,6 @@...@@ -1,6 +1,6 @@
1# Page-model edits through a cold native reopen1# Page-model edits through a cold native reopen
22
3`candidate` is `corpus/outline-edit/before` after eight compound `PreparedEdit::page`3`candidate` is `corpus/outline-edit/before` after eight compound page-model
4publications, one per edited page, each combining several kinds of change in a single4publications, one per edited page, each combining several kinds of change in a single
5transaction: paragraph reorder and Unicode replacement with a bold range; a subtree5transaction: paragraph reorder and Unicode replacement with a bold range; a subtree
6move with a new nested child and an italic/colored/resized range; a subtree deletion6move with a new nested child and an italic/colored/resized range; a subtree deletion
corpus/paragraph-edit/README.md+1-1
...@@ -26,7 +26,7 @@ notebook before completing inspection. Cold-open the split notebook for the join...@@ -26,7 +26,7 @@ notebook before completing inspection. Cold-open the split notebook for the join
26pass. Captured COM IDs belong to their originating sessions; discover fresh IDs26pass. Captured COM IDs belong to their originating sessions; discover fresh IDs
27when regenerating the corpus.27when regenerating the corpus.
2828
29`rust-split` retains twelve `ParagraphSplit` outputs, their cold native captures,29`rust-split` retains twelve Rust paragraph-split outputs, their cold native captures,
30and a second native session typing into four empty boundary paragraphs. The30and a second native session typing into four empty boundary paragraphs. The
31hyperlink case has no output: the operation rejected fields when this row was31hyperlink case has no output: the operation rejected fields when this row was
32captured, and the writer now splits before a link as OneNote does. The Rust32captured, and the writer now splits before a link as OneNote does. The Rust
corpus/protected-edit/README.md+1-1
...@@ -1,7 +1,7 @@...@@ -1,7 +1,7 @@
1# Protected page edit1# Protected page edit
22
3`candidate/notebook` is `native-encrypted/encrypted-01` after3`candidate/notebook` is `native-encrypted/encrypted-01` after
4`Notebook::save_unlocked` appended text to the first paragraph and added a4a Rust edit of the unlocked page appended text to the first paragraph and added a
5paragraph, exported by `an_unlocked_page_is_saved_under_the_section_key`5paragraph, exported by `an_unlocked_page_is_saved_under_the_section_key`
6(`ONESTORE_PROTECTED_EXPORT`). `candidate/read` is OneNote 2010's COM read from a6(`ONESTORE_PROTECTED_EXPORT`). `candidate/read` is OneNote 2010's COM read from a
7fresh clone with an empty cache after unlocking the section in its UI with the7fresh clone with an empty cache after unlocking the section in its UI with the
corpus/table-edit/README.md+1-1
...@@ -27,7 +27,7 @@ modification time, the indent array copied from a sibling cell (OneNote's...@@ -27,7 +27,7 @@ modification time, the indent array copied from a sibling cell (OneNote's
27default without one), and the flags every native cell carries, rebuilds each table's row and cell order to the27default without one), and the flags every native cell carries, rebuilds each table's row and cell order to the
28model's, and writes the column count, widths, locks and border flag. New28model's, and writes the column count, widths, locks and border flag. New
29tables start as an empty paragraph whose content the structure pass replaces.29tables start as an empty paragraph whose content the structure pass replaces.
30An emptied cell keeps a replacement paragraph, as the tree writer provides.30An edit that ends with an empty cell is refused; its ops refill the cell themselves.
31`tools/test_table_edit.py` checks all three captures without a VM. Regenerate31`tools/test_table_edit.py` checks all three captures without a VM. Regenerate
32with `ONESTORE_TABLE_EXPORT`, `ONESTORE_TABLE_EDIT_EXPORT` and32with `ONESTORE_TABLE_EXPORT`, `ONESTORE_TABLE_EDIT_EXPORT` and
33`ONESTORE_NESTED_TABLE_EXPORT` set to new absolute directories while running33`ONESTORE_NESTED_TABLE_EXPORT` set to new absolute directories while running
crates/notebook/Cargo.toml+4
...@@ -36,3 +36,7 @@ required-features = ["smb"]...@@ -36,3 +36,7 @@ required-features = ["smb"]
36[[example]]36[[example]]
37name = "smb_concurrent_client"37name = "smb_concurrent_client"
38required-features = ["smb"]38required-features = ["smb"]
39
40[[example]]
41name = "smb_publish_probe"
42required-features = ["smb"]
crates/notebook/README.md+37-29
...@@ -11,7 +11,7 @@ each by hash. Local success does not acknowledge publication to a shared noteboo...@@ -11,7 +11,7 @@ each by hash. Local success does not acknowledge publication to a shared noteboo
11editor ── Edit ──► section thread ─ Section::apply, INSERT edit ─┐11editor ── Edit ──► section thread ─ Section::apply, INSERT edit ─┐
12 (the parsed section: base + sealed + open) │ one fsync per burst12 (the parsed section: base + sealed + open) │ one fsync per burst
13sync thread: remote.stamp() == base ? publish(sealed batch) ──────┤ base chunks += transaction13sync thread: remote.stamp() == base ? publish(sealed batch) ──────┤ base chunks += transaction
14 otherwise read once, replay the queue on it ─────────┘ conflict, or base := remote14 otherwise read once, replay the queue on it ─────────┘ base := remote, conflict pages queued
15```15```
1616
17## Edits17## Edits
...@@ -56,15 +56,11 @@ same queue after a relaunch. A section is `Send + Sync`: `apply(author, edit)`...@@ -56,15 +56,11 @@ same queue after a relaunch. A section is `Send + Sync`: `apply(author, edit)`
56returns at once and reports a refusal as `Event::Rejected`; `events()` drains56returns at once and reports a refusal as `Event::Rejected`; `events()` drains
57remote changes (`Changed(spaces)`), publication outcomes, unreachable files and57remote changes (`Changed(spaces)`), publication outcomes, unreachable files and
58failures; `notify` runs on a background thread whenever an event waits.58failures; `notify` runs on a background thread whenever an event waits.
59`conflict()`, `remote_page(space)` and `resolve(id, Resolution)` present and end a59`conflicts()` lists each page's conflict pages, which `page` reads and `delete_pages`
60conflict; `release(id, archive, Resolution)` ends an uncertain attempt.60removes; `release(id, archive, Resolution)` ends an uncertain attempt. The author an
61edit names is the host's: the app passes the account's full name.
61`import_page` creates a page holding a copy of another; `delete_pages` removes pages.62`import_page` creates a page holding a copy of another; `delete_pages` removes pages.
6263
63Until the application emits ops, `save`, `queue_save`, `saved`, `saving`,
64`conflicts`, `queue`, `review` and `Event::Refreshed` keep the whole-page API: a
65save lowers the page it is given against the stored page (`onestore::op::lower_page`)
66and applies the ops.
67
68`Section::resume(file, replica, notify)` starts from an owned `Replica` without64`Section::resume(file, replica, notify)` starts from an owned `Replica` without
69consulting the remote file; with the `smb` feature, `Section::resume_smb(path,65consulting the remote file; with the `smb` feature, `Section::resume_smb(path,
70replica, limit, connect, notify)` binds a share-relative path, `connect` running on66replica, limit, connect, notify)` binds a share-relative path, `connect` running on
...@@ -81,10 +77,15 @@ queue replays on it (`merge.rs`): an op whose objects the remote left alone appl...@@ -81,10 +77,15 @@ queue replays on it (`merge.rs`): an op whose objects the remote left alone appl
81as it is; a text op shifts past the remote's changes to the same text when its77as it is; a text op shifts past the remote's changes to the same text when its
82range stays clear of them; a move, deletion or property the remote already made is78range stays clear of them; a move, deletion or property the remote already made is
83done; a page the remote already holds as the local edits leave it drops their ops.79done; a page the remote already holds as the local edits leave it drops their ops.
84Anything else is a `Conflict` on the batch: the queue stays on its base and the80Anything else conflicts, as in OneNote 2010: the op is dropped with every later op naming
85remote image is kept for review. `resolve(id, Mine)` rewrites the remote page to the81what it named, the remote's version stays the page, and the local version becomes a
86local one (`lower_page`, once), `Theirs` drops the local ops on that page; either82conflict page under it (`SectionOp::Conflict`, queued as one more edit and published with
87applies from the conflicted batch on.83the rest), its conflicting objects marked, named for the author of the first edit that did
84not replay. Page lists merge as OneNote 2010 merges page series: a page the remote moved
85keeps the remote's place, a page moved here goes before the next page in the local order
86the remote left in place, or last, and a page the remote removed comes back as a new copy
87of the local one, placed the same way (`corpus/conflict-page/native-pages`,
88`native-restore`). Nothing blocks the queue.
8889
89Publication attempts are recorded before network I/O. An attempt confirms only90Publication attempts are recorded before network I/O. An attempt confirms only
90when the remote holds its revisions, or every page it changed as it changed them,91when the remote holds its revisions, or every page it changed as it changed them,
...@@ -96,7 +97,9 @@ A durable `Published` receipt survives reopening....@@ -96,7 +97,9 @@ A durable `Published` receipt survives reopening.
96A schema-14 cache is converted when opened: it is exported to97A schema-14 cache is converted when opened: it is exported to
97`<cache>.v14-recovery`, each queued page is lowered against the page the conversion98`<cache>.v14-recovery`, each queued page is lowered against the page the conversion
98has so far, and every converted page must equal the page in the old working image,99has so far, and every converted page must equal the page in the old working image,
99or the conversion rolls back and the open fails naming the archive.100or the conversion rolls back and the open fails naming the archive; a page it held in
101conflict becomes a conflict page. A schema-15 cache's batches lose the review conflict
102they recorded when opened.
100103
101With the optional `smb` feature, `SmbRemote::new(client, path, limit)` binds an104With the optional `smb` feature, `SmbRemote::new(client, path, limit)` binds an
102`notebook::smb::Client` to one share-relative file and snapshot limit. Remote identity uses the logical root105`notebook::smb::Client` to one share-relative file and snapshot limit. Remote identity uses the logical root
...@@ -161,7 +164,7 @@ conflict policy lives in the client....@@ -161,7 +164,7 @@ conflict policy lives in the client.
161## Recovery archives164## Recovery archives
162165
163`export_recovery(new_path)` captures the base and remote images, the edit queue,166`export_recovery(new_path)` captures the base and remote images, the edit queue,
164uncertain attempts, conflicts, receipts, downloaded media and the edit-ID sequence in one167uncertain attempts, receipts, downloaded media and the edit-ID sequence in one
165SQLite snapshot. It refuses existing destinations and leaves the live queue168SQLite snapshot. It refuses existing destinations and leaves the live queue
166unchanged. Export to a local directory from a background thread: copying holds169unchanged. Export to a local directory from a background thread: copying holds
167the cache mutex while capturing the database. Failure after the final rename can170the cache mutex while capturing the database. Failure after the final rename can
...@@ -261,16 +264,16 @@ targets are `None`....@@ -261,16 +264,16 @@ targets are `None`.
261264
262With the optional `protected` feature, `Notebook::unlock(path, password)`265With the optional `protected` feature, `Notebook::unlock(path, password)`
263reads a `Locked` section as `Unlocked { pages, .. }`, and266reads a `Locked` section as `Unlocked { pages, .. }`, and
264`Notebook::save_unlocked(path, password, &mut unlocked, space, page, author)`267`Notebook::apply_unlocked(path, password, &mut unlocked, author, edit)`
265saves an edited page under the section's key, straight to the file: nothing of268applies page ops and stores them under the section's key, straight to the file:
266a protected section is cached or queued, a section written since `unlocked`269nothing of a protected section is cached or queued, a section written since
267was read fails the save, and a wrong password is270`unlocked` was read refuses the edit, and a wrong password is
268`Error::Protected(PasswordMismatch)`.271`Error::Protected(PasswordMismatch)`.
269272
270`Section::import_page(page, author)` copies a page, usually read from another273`Section::import_page(page, author)` copies a page, usually read from another
271section, to the end of this one as a page creation and a save queued like the274section, to the end of this one as one `SectionOp::Import` queued like the
272user's own edits, under fresh identities (`Page::copy`); payloads travel with275user's own edits, under fresh identities (`Page::copy`); payloads travel with
273the model, so the copy publishes offline later like any save. Content outside276the model, so the copy publishes offline later like any edit. Content outside
274the model refuses to copy. A move is an import here followed by277the model refuses to copy. A move is an import here followed by
275`Section::delete_pages` there.278`Section::delete_pages` there.
276279
...@@ -309,15 +312,20 @@ let document = onestore::document::Document::parse(&revisions)?;...@@ -309,15 +312,20 @@ let document = onestore::document::Document::parse(&revisions)?;
309```312```
310313
311Paths are relative to the share. The read limit bounds the complete physical314Paths are relative to the share. The read limit bounds the complete physical
312snapshot. Call from a background thread outside a Tokio runtime. Use identities315snapshot. Call from a background thread outside a Tokio runtime.
313from the document and the same snapshot with `Client::commit_text` or316`Client::commit_transaction` publishes a `Transaction` from `Section::seal`; its errors
314`Client::commit_property_bytes`; their errors retain `onestore::CommitState`.317retain `onestore::CommitState`. `Client::stamp` reads a file's header and length in one
315`PreparedEdit::page` separates preparation from I/O: inspect the immutable image318round trip without coordination, for polling. `Client::confirm` checks and flushes a
316and persist the intended revision identity before `Client::commit_transaction` publishes319stamp, then refreshes its header version metadata without adding a revision. The caller
317its `transaction()`. `Client::stamp` reads a file's header and length without coordination,320must first establish which intents that image contains and reread before another commit.
318for polling. `Client::confirm_snapshot` checks and flushes an observed image's stamp, then refreshes321
319its header version metadata without adding a revision. The caller must first322A commit is eight round trips while its appended bytes and patches fit fifteen 64 KiB
320establish which intents that image contains and reread before another commit.323writes: open; the coordination locks, identity queries and the stamp check's reads as one
324compound request; the path's identity; the appended bytes and patches, the header, the
325counter (twice at a carry) and the version, each as its writes and a flush in one
326compound; close. The server
327performs a compound in order and answers its flush once the writes are durable, so the
328writes carry no write-through flag.
321329
322Readers use shared native guards while writers publish under native write-open330Readers use shared native guards while writers publish under native write-open
323and byte-lock exclusion. Maintenance is excluded during each operation; pathname331and byte-lock exclusion. Maintenance is excluded during each operation; pathname
crates/notebook/examples/cache_probe.rs+13-4
...@@ -32,6 +32,17 @@ fn content(bytes: &[u8]) -> (ExGuid, ExGuid, String) {...@@ -32,6 +32,17 @@ fn content(bytes: &[u8]) -> (ExGuid, ExGuid, String) {
32 .unwrap()32 .unwrap()
33}33}
3434
35/// The images the queue leaves and started from, as a recovery archive records them.
36fn images(cache: &Replica, path: &Path) -> Result<(Vec<u8>, Vec<u8>), Box<dyn std::error::Error>> {
37 let archive = path.with_extension(format!("probe-{}", std::process::id()));
38 cache.export_recovery(&archive)?;
39 let recovery = notebook::Recovery::open(&archive)?;
40 let images = (recovery.snapshot()?, recovery.remote_snapshot()?);
41 drop(recovery);
42 std::fs::remove_file(&archive)?;
43 Ok(images)
44}
45
35fn payload(operation: u64, size: usize) -> String {46fn payload(operation: u64, size: usize) -> String {
36 format!("{operation}:🦀{}", "x".repeat(size))47 format!("{operation}:🦀{}", "x".repeat(size))
37}48}
...@@ -73,8 +84,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -73,8 +84,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
73 }84 }
74 let cache = Replica::open(path)?;85 let cache = Replica::open(path)?;
75 if mode == "read" {86 if mode == "read" {
76 let snapshot = cache.snapshot()?;87 let (snapshot, base) = images(&cache, path)?;
77 let base = cache.remote_snapshot()?;
78 let (sid, target, mut expected) = content(&base);88 let (sid, target, mut expected) = content(&base);
79 let store = Store::parse(&snapshot)?;89 let store = Store::parse(&snapshot)?;
80 assert!(store.checksum_mismatches.is_empty());90 assert!(store.checksum_mismatches.is_empty());
...@@ -164,8 +174,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -164,8 +174,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
164 let instruction = lines.next().unwrap()?;174 let instruction = lines.next().unwrap()?;
165 let (operation, acknowledgement) = instruction.split_once(' ').unwrap();175 let (operation, acknowledgement) = instruction.split_once(' ').unwrap();
166 let operation: u64 = operation.parse()?;176 let operation: u64 = operation.parse()?;
167 let source = cache.snapshot()?;177 let (sid, oid, text) = content(&images(&cache, path)?.0);
168 let (sid, oid, text) = content(&source);
169 let replacement = payload(operation, size);178 let replacement = payload(operation, size);
170 println!("editing {operation}");179 println!("editing {operation}");
171 io::stdout().flush()?;180 io::stdout().flush()?;
crates/notebook/examples/keystroke_probe.rs+5-4
...@@ -66,15 +66,15 @@ impl Remote for Memory {...@@ -66,15 +66,15 @@ impl Remote for Memory {
66 self.read += self.bytes.len() as u64;66 self.read += self.bytes.len() as u64;
67 Ok(self.bytes.clone())67 Ok(self.bytes.clone())
68 }68 }
69 fn stamp(&mut self) -> io::Result<Option<Stamp>> {69 fn stamp(&mut self) -> io::Result<Stamp> {
70 self.read += 1024;70 self.read += 1024;
71 Ok(Stamp::of(&self.bytes).ok())71 Stamp::of(&self.bytes).map_err(io::Error::other)
72 }72 }
73 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {73 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
74 transaction.commit(self)74 transaction.commit(self)
75 }75 }
76 fn confirm(&mut self, snapshot: &[u8]) -> Result<(), CommitError> {76 fn confirm(&mut self, base: &Stamp) -> Result<(), CommitError> {
77 onestore::confirm_snapshot(self, snapshot)77 onestore::confirm(self, base)
78 }78 }
79}79}
8080
...@@ -135,6 +135,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -135,6 +135,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
135 identity: None,135 identity: None,
136 created: None,136 created: None,
137 margin_origin: [0.0; 2],137 margin_origin: [0.0; 2],
138 color: None,
138 objects: vec![PageObject::Outline(Outline {139 objects: vec![PageObject::Outline(Outline {
139 id: new_id()?,140 id: new_id()?,
140 title: false,141 title: false,
crates/notebook/examples/migrate_probe.rs+13-9
...@@ -28,14 +28,14 @@ impl Remote for Memory {...@@ -28,14 +28,14 @@ impl Remote for Memory {
28 fn read(&mut self) -> io::Result<Vec<u8>> {28 fn read(&mut self) -> io::Result<Vec<u8>> {
29 Ok(self.0.clone())29 Ok(self.0.clone())
30 }30 }
31 fn stamp(&mut self) -> io::Result<Option<Stamp>> {31 fn stamp(&mut self) -> io::Result<Stamp> {
32 Ok(Stamp::of(&self.0).ok())32 Stamp::of(&self.0).map_err(io::Error::other)
33 }33 }
34 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {34 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
35 transaction.commit(self)35 transaction.commit(self)
36 }36 }
37 fn confirm(&mut self, snapshot: &[u8]) -> Result<(), CommitError> {37 fn confirm(&mut self, base: &Stamp) -> Result<(), CommitError> {
38 onestore::confirm_snapshot(self, snapshot)38 onestore::confirm(self, base)
39 }39 }
40}40}
4141
...@@ -72,11 +72,11 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -72,11 +72,11 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
72 let pending = replica.pending()?;72 let pending = replica.pending()?;
73 let summary = replica.recovery_summary()?;73 let summary = replica.recovery_summary()?;
74 println!(74 println!(
75 "{path}: converted in {:.1} ms; {} edits; {:?}; conflict {:?}",75 "{path}: converted in {:.1} ms; {} edits; {:?}; conflict pages {:?}",
76 elapsed.as_secs_f64() * 1e3,76 elapsed.as_secs_f64() * 1e3,
77 pending.len(),77 pending.len(),
78 summary,78 summary,
79 replica.conflict()?79 replica.conflicts()?
80 );80 );
81 for edit in &pending {81 for edit in &pending {
82 println!(82 println!(
...@@ -89,11 +89,15 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -89,11 +89,15 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
89 for (space, title, level) in replica.pages()? {89 for (space, title, level) in replica.pages()? {
90 println!(" page {space} {level} {title:?}");90 println!(" page {space} {level} {title:?}");
91 }91 }
92 let blocked = replica.conflict()?.is_some()92 let blocked = replica.recovery_summary()?.uncertain_edits > 0;
93 || replica.recovery_summary()?.uncertain_edits > 0;
94 if !pending.is_empty() && !blocked {93 if !pending.is_empty() && !blocked {
95 let local = pages(&replica)?;94 let local = pages(&replica)?;
96 let mut remote = Memory(replica.remote_snapshot()?);95 // The remote the queue was made on, as a recovery archive records it.
96 let archive = copy.with_extension("probe-recovery");
97 replica.export_recovery(&archive)?;
98 let mut remote =
99 Memory(notebook::Recovery::open(&archive)?.remote_snapshot()?);
100 std::fs::remove_file(&archive)?;
97 let synced = replica.sync_once(&mut remote)?;101 let synced = replica.sync_once(&mut remote)?;
98 let published = {102 let published = {
99 let arena = onestore::Arena::default();103 let arena = onestore::Arena::default();
crates/notebook/examples/open_probe.rs+5-4
...@@ -44,6 +44,7 @@ fn generated(count: usize) -> Result<Vec<u8>, Box<dyn std::error::Error>> {...@@ -44,6 +44,7 @@ fn generated(count: usize) -> Result<Vec<u8>, Box<dyn std::error::Error>> {
44 identity: None,44 identity: None,
45 created: None,45 created: None,
46 margin_origin: [0.0; 2],46 margin_origin: [0.0; 2],
47 color: None,
47 objects: vec![PageObject::Outline(Outline {48 objects: vec![PageObject::Outline(Outline {
48 id: new_id()?,49 id: new_id()?,
49 title: false,50 title: false,
...@@ -106,14 +107,14 @@ impl Remote for Memory {...@@ -106,14 +107,14 @@ impl Remote for Memory {
106 fn read(&mut self) -> io::Result<Vec<u8>> {107 fn read(&mut self) -> io::Result<Vec<u8>> {
107 Ok(self.0.clone())108 Ok(self.0.clone())
108 }109 }
109 fn stamp(&mut self) -> io::Result<Option<Stamp>> {110 fn stamp(&mut self) -> io::Result<Stamp> {
110 Ok(Stamp::of(&self.0).ok())111 Stamp::of(&self.0).map_err(io::Error::other)
111 }112 }
112 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {113 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
113 transaction.commit(self)114 transaction.commit(self)
114 }115 }
115 fn confirm(&mut self, snapshot: &[u8]) -> Result<(), CommitError> {116 fn confirm(&mut self, base: &Stamp) -> Result<(), CommitError> {
116 onestore::confirm_snapshot(self, snapshot)117 onestore::confirm(self, base)
117 }118 }
118}119}
119120
crates/notebook/examples/queue_scale.rs+45-47
...@@ -1,8 +1,8 @@...@@ -1,8 +1,8 @@
1use notebook::{EditStatus, Remote, Replica};1use notebook::{EditStatus, Remote, Replica};
2use onestore::{2use onestore::{
3 CommitError, PageCreation, PreparedEdit, RevisionIndex, Store, Transaction,3 Arena, CommitError, ExGuid, PageCreation, Section, Stamp, Transaction,
4 document::Document,4 op::{Edit, Op, PageOp, SectionOp},
5 page::{Page, PageObject, Paragraph, text::Edit},5 page::PageObject,
6};6};
7use std::{io, path::PathBuf, time::Instant};7use std::{io, path::PathBuf, time::Instant};
88
...@@ -12,11 +12,14 @@ impl Remote for FileRemote {...@@ -12,11 +12,14 @@ impl Remote for FileRemote {
12 fn read(&mut self) -> io::Result<Vec<u8>> {12 fn read(&mut self) -> io::Result<Vec<u8>> {
13 onestore::read_file(&self.0)13 onestore::read_file(&self.0)
14 }14 }
15 fn stamp(&mut self) -> io::Result<Stamp> {
16 Stamp::of(&self.read()?).map_err(io::Error::other)
17 }
15 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {18 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
16 transaction.commit_file(&self.0)19 transaction.commit_file(&self.0)
17 }20 }
18 fn confirm(&mut self, snapshot: &[u8]) -> Result<(), CommitError> {21 fn confirm(&mut self, base: &Stamp) -> Result<(), CommitError> {
19 onestore::confirm_file_snapshot(&self.0, snapshot)22 onestore::confirm_file(&self.0, base)
20 }23 }
21}24}
2225
...@@ -29,13 +32,14 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -29,13 +32,14 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
29 std::fs::create_dir(&directory)?;32 std::fs::create_dir(&directory)?;
30 let source = onestore::create_section("queue.one", "First", "Fixture")?;33 let source = onestore::create_section("queue.one", "First", "Fixture")?;
31 let creation = PageCreation::new(None, Some("Second"), "Fixture")?;34 let creation = PageCreation::new(None, Some("Second"), "Fixture")?;
32 let source = PreparedEdit::create_page(&source, &creation)?35 let (source, pages) = {
33 .as_bytes()36 let arena = Arena::default();
34 .to_vec();37 let mut section = Section::open(&arena, source)?;
35 let pages = {38 let ops = vec![Op::Section(SectionOp::Create(creation))];
36 let store = Store::parse(&source)?;39 section.apply("Fixture", &Edit { at: 133_000_000_000_000_000, ops })?;
37 let index = RevisionIndex::parse(&store)?;40 section.seal()?;
38 Document::parse(&index)?.pages()?41 let pages: Vec<ExGuid> = section.pages()?.into_iter().map(|(space, ..)| space).collect();
42 (section.image(), pages)
39 };43 };
40 assert_eq!(pages.len(), 2);44 assert_eq!(pages.len(), 2);
41 let path = directory.join("cache.sqlite");45 let path = directory.join("cache.sqlite");
...@@ -48,40 +52,32 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -48,40 +52,32 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
48 for n in 0..count {52 for n in 0..count {
49 let edit_start = Instant::now();53 let edit_start = Instant::now();
50 let slot = n % 2;54 let slot = n % 2;
51 let space = pages[slot].0;55 let space = pages[slot];
52 let mut page = cache.page(space)?;56 let page = cache.page(space)?;
53 let text = page57 let text = page
54 .objects58 .objects
55 .iter_mut()59 .iter()
56 .find_map(|object| match object {60 .find_map(|object| match object {
57 PageObject::Outline(outline) => {
58 outline.paragraphs.iter_mut().find_map(|p| p.text_mut())
59 }
60 PageObject::Title(title) => title61 PageObject::Title(title) => title
61 .outlines62 .outlines
62 .iter_mut()63 .iter()
63 .flat_map(|outline| &mut outline.paragraphs)64 .flat_map(|outline| &outline.paragraphs)
64 .find_map(|p| p.text_mut().filter(|text| text.date_field.is_none())),65 .find_map(|p| p.text().filter(|text| text.date_field.is_none())),
65 _ => None,66 _ => None,
66 })67 })
67 .unwrap();68 .unwrap();
68 expected[slot] = format!("Edit {n} 🦀 e\u{301}");69 expected[slot] = format!("Edit {n} 🦀 e\u{301}");
69 let end = u32::try_from(text.text.text().encode_utf16().count())?;70 let end = u32::try_from(text.text.text().encode_utf16().count())?;
70 let format = text.text.format_at(0)?.clone();71 let op = PageOp::Text {
71 text.text.apply(Edit {72 text: text.id,
72 range: 0..end,73 range: 0..end,
73 replacement: Paragraph::new(expected[slot].clone(), format),74 with: expected[slot].clone(),
74 })?;75 };
75 let before = cache.page(space)?;
76 let ops = onestore::op::lower_page(&before, &page)?;
77 let id = cache.apply(76 let id = cache.apply(
78 "Fixture",77 "Fixture",
79 onestore::op::Edit {78 Edit {
80 at: 133_000_000_000_000_000,79 at: 133_000_000_000_000_000,
81 ops: ops80 ops: vec![Op::Page { space, op }],
82 .into_iter()
83 .map(|op| onestore::op::Op::Page { space, op })
84 .collect(),
85 },81 },
86 )?;82 )?;
87 assert!(ids.last().is_none_or(|previous| *previous < id));83 assert!(ids.last().is_none_or(|previous| *previous < id));
...@@ -108,8 +104,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -108,8 +104,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
108 .collect::<Vec<_>>(),104 .collect::<Vec<_>>(),
109 ids105 ids
110 );106 );
111 let source = cache.snapshot()?;107 verify(&cache, &pages, &expected)?;
112 verify(&source, &pages, &expected)?;
113 println!(108 println!(
114 "{}",109 "{}",
115 serde_json::json!({"phase":"reopen","seconds":start.elapsed().as_secs_f64()})110 serde_json::json!({"phase":"reopen","seconds":start.elapsed().as_secs_f64()})
...@@ -132,9 +127,13 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -132,9 +127,13 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
132 serde_json::json!({"phase":"publish","count":ids.len(),"ms":step.elapsed().as_secs_f64()*1000.0})127 serde_json::json!({"phase":"publish","count":ids.len(),"ms":step.elapsed().as_secs_f64()*1000.0})
133 );128 );
134 assert!(cache.pending()?.is_empty());129 assert!(cache.pending()?.is_empty());
135 let published = cache.snapshot()?;130 let published = onestore::read_file(&remote.0)?;
136 assert_eq!(onestore::read_file(&remote.0)?, published);131 verify(&cache, &pages, &expected)?;
137 verify(&published, &pages, &expected)?;132 let arena = Arena::default();
133 let section = Section::open(&arena, published.clone())?;
134 for space in &pages {
135 assert_eq!(section.page(*space)?, cache.page(*space)?);
136 }
138 drop(cache);137 drop(cache);
139 let cache = Replica::open(&path)?;138 let cache = Replica::open(&path)?;
140 for id in &ids {139 for id in &ids {
...@@ -152,7 +151,11 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -152,7 +151,11 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
152 .collect::<Vec<_>>(),151 .collect::<Vec<_>>(),
153 ids152 ids
154 );153 );
155 verify(&archive.snapshot()?, &pages, &expected)?;154 let arena = Arena::default();
155 let section = Section::open(&arena, archive.snapshot()?)?;
156 for (slot, space) in pages.iter().enumerate() {
157 assert_eq!(section.page(*space)?.title, expected[slot]);
158 }
156 println!(159 println!(
157 "{}",160 "{}",
158 serde_json::json!({"phase":"complete","count":count,161 serde_json::json!({"phase":"complete","count":count,
...@@ -163,17 +166,12 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -163,17 +166,12 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
163}166}
164167
165fn verify(168fn verify(
166 bytes: &[u8],169 cache: &Replica,
167 pages: &[(onestore::ExGuid, onestore::ExGuid)],170 pages: &[ExGuid],
168 expected: &[String; 2],171 expected: &[String; 2],
169) -> Result<(), Box<dyn std::error::Error>> {172) -> Result<(), Box<dyn std::error::Error>> {
170 let store = Store::parse(bytes)?;173 for (slot, space) in pages.iter().enumerate() {
171 assert!(store.checksum_mismatches.is_empty());174 assert_eq!(cache.page(*space)?.title, expected[slot]);
172 let index = RevisionIndex::parse(&store)?;
173 index.validate_current()?;
174 let document = Document::parse(&index)?;
175 for (slot, (space, _)) in pages.iter().enumerate() {
176 assert_eq!(Page::from_space(&document, *space)?.title, expected[slot]);
177 }175 }
178 Ok(())176 Ok(())
179}177}
crates/notebook/examples/recovery_probe.rs+9-7
...@@ -1,10 +1,10 @@...@@ -1,10 +1,10 @@
1mod support {1mod support {
2 pub mod view;2 pub mod view;
3}3}
4use support::view::view;4use support::view::{cached, view};
55
6use notebook::{EditStatus, Remote, Replica};6use notebook::{EditStatus, Remote, Replica};
7use onestore::{CommitError, CommitIo, Transaction};7use onestore::{CommitError, CommitIo, Stamp, Transaction};
8use serde_json::json;8use serde_json::json;
9use std::{9use std::{
10 env,10 env,
...@@ -73,16 +73,19 @@ impl Remote for Disk {...@@ -73,16 +73,19 @@ impl Remote for Disk {
73 phase("publish-after");73 phase("publish-after");
74 result74 result
75 }75 }
76 fn confirm(&mut self, snapshot: &[u8]) -> Result<(), CommitError> {76 fn stamp(&mut self) -> io::Result<Stamp> {
77 Stamp::of(&self.read()?).map_err(io::Error::other)
78 }
79 fn confirm(&mut self, base: &Stamp) -> Result<(), CommitError> {
77 phase("confirm-before");80 phase("confirm-before");
78 let result = onestore::confirm_snapshot(self, snapshot);81 let result = onestore::confirm(self, base);
79 phase("confirm-after");82 phase("confirm-after");
80 result83 result
81 }84 }
82}85}
8386
84fn report(cache: &Replica, root: &Path) -> Result<(), Box<dyn std::error::Error>> {87fn report(cache: &Replica, root: &Path) -> Result<(), Box<dyn std::error::Error>> {
85 let local = view(&cache.snapshot()?)?;88 let (_, _, local) = cached(cache)?;
86 let remote = view(&fs::read(root.join("remote.one"))?)?;89 let remote = view(&fs::read(root.join("remote.one"))?)?;
87 let (status, revision) = match cache.status(1)? {90 let (status, revision) = match cache.status(1)? {
88 Some(EditStatus::Pending) => ("pending", None),91 Some(EditStatus::Pending) => ("pending", None),
...@@ -90,14 +93,13 @@ fn report(cache: &Replica, root: &Path) -> Result<(), Box<dyn std::error::Error>...@@ -90,14 +93,13 @@ fn report(cache: &Replica, root: &Path) -> Result<(), Box<dyn std::error::Error>
90 ("uncertain", Some(revision.to_string()))93 ("uncertain", Some(revision.to_string()))
91 }94 }
92 Some(EditStatus::Published { revision }) => ("published", Some(revision.to_string())),95 Some(EditStatus::Published { revision }) => ("published", Some(revision.to_string())),
93 Some(EditStatus::Conflict(_)) => ("conflict", None),
94 Some(EditStatus::Archived { archive }) => ("archived", Some(archive)),96 Some(EditStatus::Archived { archive }) => ("archived", Some(archive)),
95 None => ("missing", None),97 None => ("missing", None),
96 };98 };
97 println!(99 println!(
98 "{}",100 "{}",
99 json!({"event":"state", "status":status, "revision":revision,101 json!({"event":"state", "status":status, "revision":revision,
100 "local_text":local.text, "remote_text":remote.text, "remote_revision":remote.revision.to_string(),102 "local_text":local, "remote_text":remote.text, "remote_revision":remote.revision.to_string(),
101 "pending":cache.pending()?.iter().map(|pending| match &pending.edit.ops[..] {103 "pending":cache.pending()?.iter().map(|pending| match &pending.edit.ops[..] {
102 [onestore::op::Op::Page { op: onestore::op::PageOp::Text { range, with, .. }, .. }] =>104 [onestore::op::Op::Page { op: onestore::op::PageOp::Text { range, with, .. }, .. }] =>
103 json!({"id":pending.id,"replacement":with,"range":[range.start,range.end]}),105 json!({"id":pending.id,"replacement":with,"range":[range.start,range.end]}),
crates/notebook/examples/save_probe.rs+32-30
...@@ -1,12 +1,13 @@...@@ -1,12 +1,13 @@
1//! Times importing a generated page into a copy of a section, then saving a one-character1//! Times importing a generated page into a copy of a section, then typing one character
2//! edit of it: `save_probe SECTION PARAGRAPHS`.2//! into it until durable and queued: `save_probe SECTION PARAGRAPHS`.
33
4use notebook::session::Section;4use notebook::session::Section;
5use onestore::{5use onestore::{
6 document::{Format, Layout},6 document::{Format, Layout},
7 op::{Edit, Op, PageOp},
7 page::{8 page::{
8 Outline, Page, PageObject, PageParagraph, Paragraph, ParagraphContent, TextObject,9 Outline, Page, PageObject, PageParagraph, Paragraph, ParagraphContent, TextObject,
9 text::{Edit, new_id},10 text::new_id,
10 },11 },
11};12};
12use std::time::{Duration, Instant};13use std::time::{Duration, Instant};
...@@ -49,6 +50,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -49,6 +50,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
49 identity: None,50 identity: None,
50 created: None,51 created: None,
51 margin_origin: [0.0; 2],52 margin_origin: [0.0; 2],
53 color: None,
52 objects: vec![PageObject::Outline(Outline {54 objects: vec![PageObject::Outline(Outline {
53 id: new_id()?,55 id: new_id()?,
54 title: false,56 title: false,
...@@ -67,38 +69,38 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -67,38 +69,38 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
67 let start = Instant::now();69 let start = Instant::now();
68 let space = section.import_page(&imported, "Probe")?;70 let space = section.import_page(&imported, "Probe")?;
69 println!("import {count}: {:?}", start.elapsed());71 println!("import {count}: {:?}", start.elapsed());
70 let before = section.page(space)?;72 let text = section
71 let edit = |page: &Page, word: &str| -> Result<Page, Box<dyn std::error::Error>> {73 .page(space)?
72 let mut after = page.clone();74 .objects
73 let text = after75 .iter()
74 .objects76 .find_map(|object| match object {
75 .iter_mut()77 PageObject::Outline(outline) if outline.paragraphs.len() == count => {
76 .find_map(|object| match object {78 outline.paragraphs[count / 2].text().map(|text| text.id)
77 PageObject::Outline(outline) if outline.paragraphs.len() == count => {79 }
78 outline.paragraphs[count / 2].text_mut()80 _ => None,
79 }81 })
80 _ => None,82 .ok_or("the imported outline is missing")?;
81 })83 let typed = |word: &str| Edit {
82 .ok_or("the imported outline is missing")?;84 at: 133_000_000_000_000_000,
83 let format = text.text.format_at(0)?.clone();85 ops: vec![Op::Page {
84 text.text.apply(Edit {86 space,
85 range: 0..0,87 op: PageOp::Text {
86 replacement: Paragraph::new(word.into(), format),88 text,
87 })?;89 range: 0..0,
88 Ok(after)90 with: word.into(),
91 },
92 }],
89 };93 };
90 let after = edit(&before, "x")?;
91 let start = Instant::now();94 let start = Instant::now();
92 section.save(space, &before, &after, "Probe")?;95 section.replica().apply("Probe", typed("x"))?;
93 println!("save: {:?}", start.elapsed());96 println!("typed, durable: {:?}", start.elapsed());
94 let next = edit(&after, "y")?;
95 let start = Instant::now();97 let start = Instant::now();
96 section.queue_save(space, after, next, "Probe")?;98 section.apply("Probe", typed("y"))?;
97 println!("queue_save: {:?}", start.elapsed());99 println!("typed, queued: {:?}", start.elapsed());
98 let deadline = start + Duration::from_secs(600);100 let deadline = start + Duration::from_secs(600);
99 while Instant::now() < deadline {101 while Instant::now() < deadline {
100 if let Some((_, save)) = section.saved().pop() {102 if section.replica().pending()?.len() == 2 {
101 println!("queued save done: {:?} {save:?}", start.elapsed());103 println!("queued edit durable: {:?}", start.elapsed());
102 break;104 break;
103 }105 }
104 std::thread::sleep(Duration::from_millis(1));106 std::thread::sleep(Duration::from_millis(1));
crates/notebook/examples/session_client.rs+36-37
...@@ -1,15 +1,16 @@...@@ -1,15 +1,16 @@
1//! Drives a section the way the application does: open it through a session, save1//! Drives a section the way the application does: open it through a session, apply
2//! page-model edits, wait for their publication and report the page texts.2//! edits, wait for their publication and report the page texts.
3//! `session_client SECTION CACHE_DIR LABEL` edits the first body paragraph of every3//! `session_client SECTION CACHE_DIR LABEL` edits the first body paragraph of every
4//! page whose title starts with "Move" and adds one outline per page; each launch4//! page whose title starts with "Move" and adds one outline per page; each launch
5//! reports the pending queue it found and the receipts it obtained.5//! reports the pending queue it found and the receipts it obtained.
66
7use notebook::{7use notebook::{
8 EditStatus,8 EditStatus,
9 session::{Event, Save, Section},9 session::{Event, Section},
10};10};
11use onestore::{11use onestore::{
12 ExGuid,12 ExGuid,
13 op::PageOp,
13 page::{Outline, Page, PageObject, PageParagraph, ParagraphContent, TextObject, text::new_id},14 page::{Outline, Page, PageObject, PageParagraph, ParagraphContent, TextObject, text::new_id},
14};15};
15use std::{16use std::{
...@@ -46,28 +47,9 @@ fn texts(page: &Page) -> Vec<String> {...@@ -46,28 +47,9 @@ fn texts(page: &Page) -> Vec<String> {
46 out47 out
47}48}
4849
49fn edit(page: &mut Page, label: &str) {50/// Types `label` before the first body paragraph and adds an outline below the body.
51fn edit(page: &Page, label: &str) -> Vec<PageOp> {
50 let text = body_text(page).expect("a body paragraph");52 let text = body_text(page).expect("a body paragraph");
51 for object in &mut page.objects {
52 let PageObject::Outline(outline) = object else {
53 continue;
54 };
55 if let Some(paragraph) = outline
56 .paragraphs
57 .iter_mut()
58 .find(|p| p.text().is_some_and(|t| t.id == text))
59 {
60 let target = paragraph.text_mut().unwrap();
61 let format = target.text.format_at(0).unwrap().clone();
62 target
63 .text
64 .apply(onestore::page::text::Edit {
65 range: 0..0,
66 replacement: onestore::page::Paragraph::new(format!("{label} "), format),
67 })
68 .unwrap();
69 }
70 }
71 let template = page53 let template = page
72 .objects54 .objects
73 .iter()55 .iter()
...@@ -110,12 +92,21 @@ fn edit(page: &mut Page, label: &str) {...@@ -110,12 +92,21 @@ fn edit(page: &mut Page, label: &str) {
110 paragraphs: vec![paragraph],92 paragraphs: vec![paragraph],
111 unsupported: Vec::new(),93 unsupported: Vec::new(),
112 };94 };
113 let at = page95 let title = page.objects.iter().find_map(|object| match object {
114 .objects96 PageObject::Title(title) => Some(title.id),
115 .iter()97 _ => None,
116 .position(|o| matches!(o, PageObject::Title(_)))98 });
117 .unwrap_or(page.objects.len());99 vec![
118 page.objects.insert(at, PageObject::Outline(outline));100 PageOp::Text {
101 text,
102 range: 0..0,
103 with: format!("{label} "),
104 },
105 PageOp::Add {
106 object: PageObject::Outline(outline),
107 before: title,
108 },
109 ]
119}110}
120111
121fn main() -> Result<(), Box<dyn std::error::Error>> {112fn main() -> Result<(), Box<dyn std::error::Error>> {
...@@ -138,13 +129,21 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -138,13 +129,21 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
138 .iter()129 .iter()
139 .filter(|(_, title, _)| title.starts_with("Move"))130 .filter(|(_, title, _)| title.starts_with("Move"))
140 {131 {
141 let before = section.page(*space)?;132 let mut page = section.page(*space)?;
142 let mut after = before.clone();133 let ops = edit(&page, label);
143 edit(&mut after, label);134 for op in &ops {
144 match section.save(*space, &before, &after, "session client")? {135 onestore::op::predict(&mut page, op)?;
145 Save::Queued(id) => queued.push((id, *space, title.clone(), texts(&after))),
146 other => return Err(format!("{title}: {other:?}").into()),
147 }136 }
137 let unix = std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH)?;
138 let edit = onestore::op::Edit {
139 at: (unix.as_secs() + 11_644_473_600) * 10_000_000,
140 ops: ops
141 .into_iter()
142 .map(|op| onestore::op::Op::Page { space: *space, op })
143 .collect(),
144 };
145 let id = section.replica().apply("session client", edit)?;
146 queued.push((id, *space, title.clone(), texts(&page)));
148 }147 }
149 let deadline = Instant::now() + Duration::from_secs(120);148 let deadline = Instant::now() + Duration::from_secs(120);
150 let mut receipts = Vec::new();149 let mut receipts = Vec::new();
...@@ -176,7 +175,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -176,7 +175,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
176 }175 }
177 Event::Failed(error) => return Err(error.into()),176 Event::Failed(error) => return Err(error.into()),
178 Event::Rejected { error, .. } => return Err(error.into()),177 Event::Rejected { error, .. } => return Err(error.into()),
179 Event::Refreshed | Event::Changed(_) => {}178 Event::Changed(_) => {}
180 }179 }
181 }180 }
182 // A batch reports its newest edit; each edit's receipt is its own.181 // A batch reports its newest edit; each edit's receipt is its own.
crates/notebook/examples/smb_concurrent_client.rs+6-1
...@@ -1,5 +1,7 @@...@@ -1,5 +1,7 @@
1#[path = "../../onestore/examples/support/concurrent.rs"]1#[path = "../../onestore/examples/support/concurrent.rs"]
2mod concurrent;2mod concurrent;
3#[path = "support/edit.rs"]
4mod edit;
35
4use notebook::smb::{Client, Credentials};6use notebook::smb::{Client, Credentials};
5use std::{env, time::Duration};7use std::{env, time::Duration};
...@@ -16,7 +18,10 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -16,7 +18,10 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
16 &args,18 &args,
17 |path| client.read(path, 256 * 1024 * 1024),19 |path| client.read(path, 256 * 1024 * 1024),
18 |path, source, space, object, range, replacement| {20 |path, source, space, object, range, replacement| {
19 client.commit_text(path, source, space, object, range, replacement)21 client.commit_transaction(
22 path,
23 &edit::replaced(source, space, object, range, replacement)?,
24 )
20 },25 },
21 )26 )
22}27}
crates/notebook/examples/smb_offline_client.rs+47-43
...@@ -3,7 +3,7 @@ mod concurrent;...@@ -3,7 +3,7 @@ mod concurrent;
33
4use notebook::smb::{Client, Credentials};4use notebook::smb::{Client, Credentials};
5use notebook::{EditStatus, Error, Remote, Replica, SmbRemote};5use notebook::{EditStatus, Error, Remote, Replica, SmbRemote};
6use onestore::{CommitError, CommitState, ExGuid, RevisionIndex, Store, Transaction};6use onestore::{CommitError, CommitState, ExGuid, RevisionIndex, Stamp, Store, Transaction};
7use serde_json::json;7use serde_json::json;
8use std::{8use std::{
9 env,9 env,
...@@ -18,7 +18,7 @@ mod support {...@@ -18,7 +18,7 @@ mod support {
18 pub mod view;18 pub mod view;
19}19}
20use concurrent::{DocumentView, document_view};20use concurrent::{DocumentView, document_view};
21use support::view::view;21use support::view::{cached, view};
2222
23impl DocumentView {23impl DocumentView {
24 fn changes(&self, before: &Self) -> Self {24 fn changes(&self, before: &Self) -> Self {
...@@ -79,6 +79,13 @@ impl<R: Remote> Remote for Traced<R> {...@@ -79,6 +79,13 @@ impl<R: Remote> Remote for Traced<R> {
79 Ok(bytes)79 Ok(bytes)
80 }80 }
81 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {81 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
82 // Publication reads nothing while the stamp holds; observe an unseen base once.
83 if Stamp::of(&self.read).ok().as_ref() != Some(transaction.base()) {
84 self.read().map_err(|error| CommitError {
85 state: CommitState::NotCommitted,
86 error,
87 })?;
88 }
82 let mut image = self.read.clone();89 let mut image = self.read.clone();
83 transaction.apply(&mut image).map_err(|error| CommitError {90 transaction.apply(&mut image).map_err(|error| CommitError {
84 state: CommitState::NotCommitted,91 state: CommitState::NotCommitted,
...@@ -193,10 +200,22 @@ impl<R: Remote> Remote for Traced<R> {...@@ -193,10 +200,22 @@ impl<R: Remote> Remote for Traced<R> {
193 thread::sleep(Duration::from_millis(10));200 thread::sleep(Duration::from_millis(10));
194 }201 }
195 }202 }
203 if result.is_ok() {
204 self.read = image;
205 self.before = Some((after.text, documents));
206 }
196 result207 result
197 }208 }
198 fn confirm(&mut self, snapshot: &[u8]) -> Result<(), CommitError> {209 fn stamp(&mut self) -> io::Result<Stamp> {
210 self.remote.stamp()
211 }
212 fn confirm(&mut self, base: &Stamp) -> Result<(), CommitError> {
199 let captured = (|| -> Result<_, Box<dyn std::error::Error>> {213 let captured = (|| -> Result<_, Box<dyn std::error::Error>> {
214 // A confirmation follows the read of its image, except for edits that changed nothing.
215 if Stamp::of(&self.read).ok().as_ref() != Some(base) {
216 self.read = self.remote.read()?;
217 }
218 let snapshot = &self.read[..];
200 let store = Store::parse(snapshot)?;219 let store = Store::parse(snapshot)?;
201 let index = RevisionIndex::parse(&store)?;220 let index = RevisionIndex::parse(&store)?;
202 let revisions = index221 let revisions = index
...@@ -240,7 +259,7 @@ impl<R: Remote> Remote for Traced<R> {...@@ -240,7 +259,7 @@ impl<R: Remote> Remote for Traced<R> {
240 error: io::Error::other(error.to_string()),259 error: io::Error::other(error.to_string()),
241 })?;260 })?;
242 let started = now();261 let started = now();
243 let result = self.remote.confirm(snapshot);262 let result = self.remote.confirm(base);
244 println!(263 println!(
245 "{}",264 "{}",
246 json!({"event":"remote_confirm", "started_us":started, "finished_us":now(), "revisions":revisions, "current_revisions":current, "capture":capture.as_ref().and_then(|path| path.file_name()).map(|name| name.to_string_lossy()), "text":self.before.as_ref().map(|(text,_)|text),265 json!({"event":"remote_confirm", "started_us":started, "finished_us":now(), "revisions":revisions, "current_revisions":current, "capture":capture.as_ref().and_then(|path| path.file_name()).map(|name| name.to_string_lossy()), "text":self.before.as_ref().map(|(text,_)|text),
...@@ -343,6 +362,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -343,6 +362,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
343 }362 }
344 })?;363 })?;
345 let mut ids = Vec::new();364 let mut ids = Vec::new();
365 let mut tokens = Vec::new();
346 let result = (|| -> Result<(), Box<dyn std::error::Error>> {366 let result = (|| -> Result<(), Box<dyn std::error::Error>> {
347 let mut generated = 0;367 let mut generated = 0;
348 let mut received = 0;368 let mut received = 0;
...@@ -380,62 +400,46 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -380,62 +400,46 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
380 8400 8
381 };401 };
382 if generated < operations && pending.len() < capacity {402 if generated < operations && pending.len() < capacity {
383 let source = cache.snapshot()?;403 let (space, object, text) = cached(&cache)?;
384 let target = view(&source)?;404 let at = u32::try_from(text.encode_utf16().count())?;
385 let at = u32::try_from(target.text.encode_utf16().count())?;
386 let token = format!(" [{}:{}]", args[2], generated);405 let token = format!(" [{}:{}]", args[2], generated);
387 let started = now();406 let started = now();
388 match cache.apply(407 match cache.apply(
389 "Offline document writer",408 "Offline document writer",
390 append(target.space, target.object, at, &token),409 append(space, object, at, &token),
391 ) {410 ) {
392 Ok(id) => {411 Ok(id) => {
393 println!(412 println!(
394 "{}",413 "{}",
395 json!({"event":"local_commit", "id":id, "operation":generated, "space":target.space.to_string(), "object":target.object.to_string(), "before":target.text, "token":token, "started_us":started, "finished_us":now()})414 json!({"event":"local_commit", "id":id, "operation":generated, "space":space.to_string(), "object":object.to_string(), "before":text, "token":token, "started_us":started, "finished_us":now()})
396 );415 );
397 ids.push(id);416 ids.push(id);
417 tokens.push(token);
398 generated += 1;418 generated += 1;
399 }419 }
400 other => return Err(format!("Unexpected local result: {other:?}").into()),420 other => return Err(format!("Unexpected local result: {other:?}").into()),
401 }421 }
402 }422 }
403 if let Some(conflict) = cache.conflict()? {423 if received == ids.len() && cache.pending()?.is_empty() {
404 // Concurrent appends meet at the end of the text: take the remote text and424 // Concurrent appends meet at the end of the text: a merge keeps the remote's
405 // append the queued tokens it lacks after it again.425 // text and this writer's on a conflict page. Append the published tokens
406 let remote = view(&cache.remote_snapshot()?)?;426 // the text lacks after it again.
407 let tokens: Vec<String> = cache427 let (space, object, text) = cached(&cache)?;
408 .pending()?428 let missing: Vec<String> = tokens
409 .iter()429 .iter()
410 .flat_map(|queued| queued.edit.ops.iter())430 .filter(|token| !text.contains(token.as_str()))
411 .filter_map(|op| match op {431 .cloned()
412 onestore::op::Op::Page {
413 op: onestore::op::PageOp::Text { with, .. },
414 ..
415 } => Some(with.clone()),
416 _ => None,
417 })
418 .filter(|token| !remote.text.contains(token.as_str()))
419 .collect();432 .collect();
420 match cache.resolve(conflict.id, notebook::Resolution::Theirs) {433 let mut at = u32::try_from(text.encode_utf16().count())?;
421 Ok(()) => {434 for token in &missing {
422 let mut at = u32::try_from(remote.text.encode_utf16().count())?;435 ids.push(cache.apply("Offline document writer", append(space, object, at, token))?);
423 for token in &tokens {436 at += u32::try_from(token.encode_utf16().count())?;
424 cache.apply(437 }
425 "Offline document writer",438 if !missing.is_empty() {
426 append(remote.space, remote.object, at, token),439 println!(
427 )?;440 "{}",
428 at += u32::try_from(token.encode_utf16().count())?;441 json!({"event":"reviewed_append", "remote":text, "tokens":missing, "at_us":now()})
429 }442 );
430 println!(
431 "{}",
432 json!({"event":"reviewed_append", "id":conflict.id, "remote":remote.text, "tokens":tokens, "at_us":now()})
433 );
434 }
435 Err(Error::Io(error))
436 if [io::ErrorKind::WouldBlock, io::ErrorKind::InvalidInput]
437 .contains(&error.kind()) => {}
438 Err(error) => return Err(error.into()),
439 }443 }
440 }444 }
441 if generated == operations && received == ids.len() {445 if generated == operations && received == ids.len() {
crates/notebook/examples/smb_publish_probe.rs created+110
...@@ -0,0 +1,110 @@
1//! Publishes typed keystrokes to a Samba lab one at a time and reports each publication's
2//! wall time: `smb_publish_probe ADDRESS KEYSTROKES [CONTROL]`. With `CONTROL`, the
3//! `tools/smb-proxy.py` control file, each publication is marked in the proxy's trace.
4
5use notebook::{
6 EditStatus, Replica, SmbRemote,
7 smb::{Client, Credentials},
8};
9use onestore::op::{Edit, Op, PageOp};
10use std::{
11 env, fs,
12 time::{Duration, Instant, SystemTime, UNIX_EPOCH},
13};
14
15fn main() -> Result<(), Box<dyn std::error::Error>> {
16 let args: Vec<_> = env::args().skip(1).collect();
17 let [address, count, control @ ..] = &args[..] else {
18 return Err("smb_publish_probe ADDRESS KEYSTROKES [CONTROL]".into());
19 };
20 let count: u32 = count.parse()?;
21 let mark = |phase: String| -> std::io::Result<()> {
22 if let [control] = control {
23 fs::write(
24 format!("{control}.tmp"),
25 format!("{{\"phase\":\"{phase}\"}}"),
26 )?;
27 fs::rename(format!("{control}.tmp"), control)?;
28 // The proxy polls its control file every 50 ms.
29 std::thread::sleep(Duration::from_millis(120));
30 }
31 Ok(())
32 };
33 let connect = || {
34 Client::connect(
35 address,
36 "agent",
37 Credentials::default(),
38 Duration::from_secs(5),
39 )
40 };
41 let path = format!(
42 "probe-{}.one",
43 SystemTime::now().duration_since(UNIX_EPOCH)?.as_nanos()
44 );
45 let source = onestore::create_section(&path, "Body", "Probe")?;
46 connect()?.create(&path, &source)?;
47 let directory = tempfile::tempdir()?;
48 let replica = Replica::create(directory.path().join("cache.sqlite"), &source)?;
49 let space = replica.pages()?[0].0;
50 let text = replica
51 .page(space)?
52 .objects
53 .iter()
54 .find_map(|object| match object {
55 onestore::page::PageObject::Outline(outline) => outline
56 .paragraphs
57 .iter()
58 .find_map(|p| p.text().map(|t| t.id)),
59 _ => None,
60 })
61 .ok_or("no text")?;
62 let mut remote = SmbRemote::new(connect()?, path.clone(), 1 << 24);
63 mark("idle".into())?;
64 let mut idle = Vec::new();
65 for _ in 0..count {
66 let started = Instant::now();
67 assert_eq!(replica.sync_once(&mut remote)?.edit, None);
68 idle.push(started.elapsed());
69 }
70 let mut published = Vec::new();
71 for n in 0..count {
72 replica.apply(
73 "Probe",
74 Edit {
75 at: 133_000_000_000_000_000,
76 ops: vec![Op::Page {
77 space,
78 op: PageOp::Text {
79 text,
80 range: 4 + n..4 + n,
81 with: "x".into(),
82 },
83 }],
84 },
85 )?;
86 mark(format!("publish-{n}"))?;
87 let started = Instant::now();
88 let synced = replica.sync_once(&mut remote)?;
89 published.push(started.elapsed());
90 assert!(matches!(
91 synced.edit,
92 Some((_, EditStatus::Published { .. }))
93 ));
94 }
95 mark("done".into())?;
96 let quantiles = |mut times: Vec<Duration>| {
97 times.sort();
98 let at = |q: f64| times[((times.len() - 1) as f64 * q) as usize].as_secs_f64() * 1000.0;
99 format!(
100 "p50 {:.2} ms, p90 {:.2} ms, max {:.2} ms",
101 at(0.5),
102 at(0.9),
103 at(1.0)
104 )
105 };
106 println!("idle poll: {}", quantiles(idle));
107 println!("publish: {}", quantiles(published));
108 connect()?.delete(&path)?;
109 Ok(())
110}
crates/notebook/examples/smb_reconnect_client.rs+17-17
...@@ -1,9 +1,11 @@...@@ -1,9 +1,11 @@
1#[path = "../../onestore/examples/support/concurrent.rs"]1#[path = "../../onestore/examples/support/concurrent.rs"]
2mod concurrent;2mod concurrent;
3#[path = "support/edit.rs"]
4mod edit;
35
4use notebook::smb::{Client, Credentials};6use notebook::smb::{Client, Credentials};
5use onestore::{7use onestore::{
6 CommitError, CommitState, ExGuid, RevisionIndex, Store,8 CommitError, CommitState, ExGuid, RevisionIndex, Stamp, Store,
7 document::{Document, Kind},9 document::{Document, Kind},
8};10};
9use serde_json::json;11use serde_json::json;
...@@ -155,14 +157,14 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -155,14 +157,14 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
155 &args,157 &args,
156 read,158 read,
157 |path, source, space, object, range, replacement| {159 |path, source, space, object, range, replacement| {
158 let outcome = client.borrow().as_ref().unwrap().commit_text(160 let outcome =
159 path,161 edit::replaced(source, space, object, range, replacement).and_then(|transaction| {
160 source,162 client
161 space,163 .borrow()
162 object,164 .as_ref()
163 range,165 .unwrap()
164 replacement,166 .commit_transaction(path, &transaction)
165 );167 });
166 let Err(error) = outcome else {168 let Err(error) = outcome else {
167 return Ok(());169 return Ok(());
168 };170 };
...@@ -209,14 +211,12 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -209,14 +211,12 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
209 error: io::Error::other(failure.to_string()),211 error: io::Error::other(failure.to_string()),
210 })?;212 })?;
211 if published {213 if published {
212 let confirmation = client.borrow().as_ref().unwrap().commit_text(214 let confirmation = Stamp::of(&current)
213 path,215 .map_err(|failure| CommitError {
214 &current,216 state: CommitState::NotCommitted,
215 space,217 error: io::Error::other(failure.to_string()),
216 object,218 })
217 0..0,219 .and_then(|base| client.borrow().as_ref().unwrap().confirm(path, &base));
218 "",
219 );
220 if let Err(failure) = confirmation220 if let Err(failure) = confirmation
221 && failure.state != CommitState::Committed221 && failure.state != CommitState::Committed
222 {222 {
crates/notebook/examples/support/edit.rs created+49
...@@ -0,0 +1,49 @@
1use onestore::{
2 Arena, CommitError, CommitState, ExGuid, Section, Transaction,
3 op::{Edit, Op, PageOp},
4};
5use std::{
6 io,
7 ops::Range,
8 time::{SystemTime, UNIX_EPOCH},
9};
10
11/// The transaction replacing `range` of `text` in `image`, typed now.
12pub fn replaced(
13 image: &[u8],
14 space: ExGuid,
15 text: ExGuid,
16 range: Range<u32>,
17 with: &str,
18) -> Result<Transaction, CommitError> {
19 let failed = |error: String| CommitError {
20 state: CommitState::NotCommitted,
21 error: io::Error::new(io::ErrorKind::InvalidData, error),
22 };
23 let arena = Arena::default();
24 let mut section = Section::open(&arena, image.to_vec()).map_err(|e| failed(e.to_string()))?;
25 let at = SystemTime::now()
26 .duration_since(UNIX_EPOCH)
27 .map_err(|e| failed(e.to_string()))?
28 .as_nanos() as u64
29 / 100
30 + 116_444_736_000_000_000;
31 let edit = Edit {
32 at,
33 ops: vec![Op::Page {
34 space,
35 op: PageOp::Text {
36 text,
37 range,
38 with: with.to_owned(),
39 },
40 }],
41 };
42 section
43 .apply("Concurrent client", &edit)
44 .map_err(|e| failed(format!("{e:?}")))?;
45 section
46 .seal()
47 .map_err(|e| failed(e.to_string()))?
48 .ok_or_else(|| failed("The edit changed nothing".into()))
49}
crates/notebook/examples/support/view.rs+41
...@@ -1,3 +1,5 @@...@@ -1,3 +1,5 @@
1#![allow(dead_code)]
2
1use onestore::{3use onestore::{
2 ExGuid, RevisionIndex, Store,4 ExGuid, RevisionIndex, Store,
3 document::{Document, Kind},5 document::{Document, Kind},
...@@ -55,3 +57,42 @@ pub fn view(bytes: &[u8]) -> Result<View, Box<dyn std::error::Error>> {...@@ -55,3 +57,42 @@ pub fn view(bytes: &[u8]) -> Result<View, Box<dyn std::error::Error>> {
55 }57 }
56 Ok(found.pop().unwrap())58 Ok(found.pop().unwrap())
57}59}
60
61/// The concurrent-edit paragraph of `page`: its text object and text.
62fn in_page(page: &onestore::page::Page) -> Option<(ExGuid, String)> {
63 let mut pending: Vec<&onestore::page::PageParagraph> = page
64 .objects
65 .iter()
66 .flat_map(|object| match object {
67 onestore::page::PageObject::Outline(outline) => outline.paragraphs.iter().collect(),
68 _ => Vec::new(),
69 })
70 .collect();
71 pending.reverse();
72 while let Some(paragraph) = pending.pop() {
73 if let Some(text) = paragraph.text()
74 && text.text.text().starts_with("Concurrent edits:")
75 {
76 return Some((text.id, text.text.text().to_owned()));
77 }
78 if let onestore::page::ParagraphContent::Table(table) = &paragraph.content {
79 pending.extend(table.rows.iter().flat_map(|row| &row.cells).flat_map(|cell| &cell.paragraphs).rev());
80 }
81 }
82 None
83}
84
85/// The concurrent-edit paragraph as the queued edits leave it: its page, text object and
86/// text.
87pub fn cached(cache: &notebook::Replica) -> Result<(ExGuid, ExGuid, String), Box<dyn std::error::Error>> {
88 let mut found = Vec::new();
89 for (space, ..) in cache.pages()? {
90 if let Some((object, text)) = in_page(&cache.page(space)?) {
91 found.push((space, object, text));
92 }
93 }
94 match <[_; 1]>::try_from(found) {
95 Ok([found]) => Ok(found),
96 Err(_) => Err("Expected one concurrent-edit paragraph".into()),
97 }
98}
crates/notebook/src/bin/onestore-diagnostic.rs+104-11
...@@ -1,4 +1,11 @@...@@ -1,4 +1,11 @@
1use onestore::{ExGuid, Insertion, PreparedEdit, TextAttribute};1use onestore::{
2 Arena, ExGuid, Section, TextAttribute,
3 document::{Format, Layout},
4 op::{Op, PageOp},
5 page::{
6 Outline, PageObject, PageParagraph, Paragraph, ParagraphContent, TextObject, text::new_id,
7 },
8};
2use serde::Deserialize;9use serde::Deserialize;
3use serde_json::{Value, json};10use serde_json::{Value, json};
4use std::{11use std::{
...@@ -40,6 +47,33 @@ enum Action {...@@ -40,6 +47,33 @@ enum Action {
40 },47 },
41}48}
4249
50/// A new paragraph holding `text` in OneNote's default text style.
51fn paragraph(text: String) -> Result<PageParagraph, Box<dyn std::error::Error>> {
52 let format = Format {
53 font: Some("Calibri".into()),
54 font_size: Some(11.0),
55 language: Some(0x409),
56 ..Default::default()
57 };
58 Ok(PageParagraph {
59 id: new_id()?,
60 parent: None,
61 level: 1,
62 style: None,
63 format: Format::default(),
64 content: ParagraphContent::Text(TextObject {
65 id: new_id()?,
66 date_field: None,
67 text: Paragraph::new(text, format),
68 tags: Vec::new(),
69 }),
70 lists: Vec::new(),
71 tags: Vec::new(),
72 media: Default::default(),
73 collapsed: false,
74 })
75}
76
43fn run(args: &[std::ffi::OsString]) -> Result<Value, Box<dyn std::error::Error>> {77fn run(args: &[std::ffi::OsString]) -> Result<Value, Box<dyn std::error::Error>> {
44 if let [mode, root] = args78 if let [mode, root] = args
45 && mode == "catalog"79 && mode == "catalog"
...@@ -79,34 +113,93 @@ fn run(args: &[std::ffi::OsString]) -> Result<Value, Box<dyn std::error::Error>>...@@ -79,34 +113,93 @@ fn run(args: &[std::ffi::OsString]) -> Result<Value, Box<dyn std::error::Error>>
79 let edit: Edit = serde_json::from_reader(io::stdin().lock())?;113 let edit: Edit = serde_json::from_reader(io::stdin().lock())?;
80 let sid = edit.space;114 let sid = edit.space;
81 let oid = edit.object;115 let oid = edit.object;
82 let prepared = match edit.action {116 let (op, author) = match edit.action {
83 Action::Text {117 Action::Text {
84 start,118 start,
85 end,119 end,
86 replacement,120 replacement,
87 } => PreparedEdit::text(&bytes, sid, oid, start..end, &replacement)?,121 } => (
122 PageOp::Text {
123 text: oid,
124 range: start..end,
125 with: replacement,
126 },
127 String::new(),
128 ),
88 Action::Format {129 Action::Format {
89 start,130 start,
90 end,131 end,
91 attributes,132 attributes,
92 } => PreparedEdit::format(&bytes, sid, oid, start..end, &attributes)?,133 } => (
134 PageOp::Format {
135 text: oid,
136 range: start..end,
137 set: attributes,
138 clear: Vec::new(),
139 },
140 String::new(),
141 ),
93 Action::Paragraph {142 Action::Paragraph {
94 before,143 before,
95 text,144 text,
96 author,145 author,
97 } => PreparedEdit::insert(146 } => (
98 &bytes,147 PageOp::Insert {
99 sid,148 container: oid,
100 &Insertion::paragraph(oid, before, &text, &author)?,149 before,
101 )?,150 paragraphs: vec![paragraph(text)?],
151 },
152 author,
153 ),
102 Action::Outline { x, y, text, author } => {154 Action::Outline { x, y, text, author } => {
103 PreparedEdit::insert(&bytes, sid, &Insertion::outline(oid, x, y, &text, &author)?)?155 let store = onestore::Store::parse(&bytes)?;
156 let index = onestore::RevisionIndex::parse(&store)?;
157 if !onestore::document::Document::parse(&index)?
158 .pages()?
159 .contains(&(sid, oid))
160 {
161 return Err("Choose the page to hold the outline".into());
162 }
163 (
164 PageOp::Add {
165 object: PageObject::Outline(Outline {
166 id: new_id()?,
167 title: false,
168 min_width: None,
169 layout: Layout {
170 x: Some(x),
171 y: Some(y),
172 ..Default::default()
173 },
174 indents: Vec::new(),
175 paragraphs: vec![paragraph(text)?],
176 unsupported: Vec::new(),
177 }),
178 before: None,
179 },
180 author,
181 )
104 }182 }
105 };183 };
184 let arena = Arena::default();
185 let mut section = Section::open(&arena, bytes)?;
186 let unix = std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH)?;
187 let at = (unix.as_secs() + 11_644_473_600) * 10_000_000 + u64::from(unix.subsec_nanos() / 100);
188 section.apply(
189 &author,
190 &onestore::op::Edit {
191 at,
192 ops: vec![Op::Page { space: sid, op }],
193 },
194 )?;
195 let transaction = section.seal()?;
106 if check {196 if check {
107 return Ok(json!({"ok": true}));197 return Ok(json!({"ok": true}));
108 }198 }
109 Ok(match prepared.commit_file(&args[1]) {199 let Some(transaction) = transaction else {
200 return Ok(json!({"ok": true, "state": "Unchanged"}));
201 };
202 Ok(match transaction.commit_file(&args[1]) {
110 Ok(()) => json!({"ok": true, "state": "Committed"}),203 Ok(()) => json!({"ok": true, "state": "Committed"}),
111 Err(error) => json!({"ok": false, "state": format!("{:?}", error.state),204 Err(error) => json!({"ok": false, "state": format!("{:?}", error.state),
112 "kind": format!("{:?}", error.error.kind()), "error": error.error.to_string()}),205 "kind": format!("{:?}", error.error.kind()), "error": error.error.to_string()}),
crates/notebook/src/lib.rs+11-16
...@@ -29,7 +29,7 @@ mod schema;...@@ -29,7 +29,7 @@ mod schema;
29pub mod session;29pub mod session;
30pub use recovery::{Recovery, RecoverySummary};30pub use recovery::{Recovery, RecoverySummary};
31mod sync;31mod sync;
32pub use sync::{ConflictKind, EditStatus, Remote, Synced};32pub use sync::{EditStatus, Remote, Synced};
33mod worker;33mod worker;
34mod working;34mod working;
35#[cfg(feature = "smb")]35#[cfg(feature = "smb")]
...@@ -72,19 +72,8 @@ pub struct PendingEdit {...@@ -72,19 +72,8 @@ pub struct PendingEdit {
72 pub edit: Edit,72 pub edit: Edit,
73}73}
7474
75/// Unpublished work the remote section no longer accepts: the batch holding edit `id`75/// How an uncertain attempt ends after review: `Mine` publishes it again, `Theirs` drops
76/// changed page `space` where the remote changed it too.76/// the whole unpublished branch.
77#[derive(Debug, Clone, PartialEq, Eq)]
78pub struct Conflict {
79 pub id: u64,
80 pub space: ExGuid,
81 pub kind: ConflictKind,
82}
83
84/// How a conflict or an uncertain attempt ends. `Mine` keeps the local pages: a conflicted
85/// page is rewritten from the remote page to the local one; a released attempt publishes
86/// again. `Theirs` drops the local edits to the conflicted page, or the whole unpublished
87/// branch of a released attempt.
88#[derive(Debug, Clone, Copy, PartialEq, Eq)]77#[derive(Debug, Clone, Copy, PartialEq, Eq)]
89pub enum Resolution {78pub enum Resolution {
90 Mine,79 Mine,
...@@ -161,6 +150,7 @@ impl Replica {...@@ -161,6 +150,7 @@ impl Replica {
161 // the file as it was.150 // the file as it was.
162 match version {151 match version {
163 migrate::VERSION => migrate::migrate(&mut connection, path)?,152 migrate::VERSION => migrate::migrate(&mut connection, path)?,
153 schema::PREVIOUS => schema::upgrade(&mut connection)?,
164 schema::VERSION => {}154 schema::VERSION => {}
165 _ => {155 _ => {
166 return Err(io::Error::new(156 return Err(io::Error::new(
...@@ -248,9 +238,14 @@ impl Replica {...@@ -248,9 +238,14 @@ impl Replica {
248 self.ask(|reply| working::Request::Pages { reply })238 self.ask(|reply| working::Request::Pages { reply })
249 }239 }
250240
241 /// The conflict pages of each page that has them (`onestore::Section::conflicts`).
242 pub fn conflicts(&self) -> Result<Vec<(ExGuid, Vec<onestore::ConflictPage>)>> {
243 self.ask(|reply| working::Request::Conflicts { reply })
244 }
245
251 /// The section image the queued edits leave, the unsealed ones sealed as one more246 /// The section image the queued edits leave, the unsealed ones sealed as one more
252 /// revision whose identities differ per call: O(section), for tests and diagnostics.247 /// revision whose identities differ per call: O(section).
253 pub fn snapshot(&self) -> Result<Vec<u8>> {248 pub(crate) fn snapshot(&self) -> Result<Vec<u8>> {
254 self.ask(|reply| working::Request::Flush { reply })?;249 self.ask(|reply| working::Request::Flush { reply })?;
255 working::image(&*self.lock()?)250 working::image(&*self.lock()?)
256 }251 }
crates/notebook/src/merge.rs+354-296
...@@ -2,11 +2,16 @@...@@ -2,11 +2,16 @@
2//! section when the objects it reads are as the base image had them; text ops on a text2//! section when the objects it reads are as the base image had them; text ops on a text
3//! the remote also changed shift past the remote's changes when they stay clear of them;3//! the remote also changed shift past the remote's changes when they stay clear of them;
4//! a move, deletion or outline setting the remote already made is dropped as done.4//! a move, deletion or outline setting the remote already made is dropped as done.
5//! Anything else is a conflict: the queue stays on its base until it is resolved.5//! An op on a page the remote changed where it did is dropped, with every later op naming
6//! the objects it named, and the page conflicts: as OneNote 2010 does, the remote's version
7//! stays the page and the local one becomes a conflict page under it (`conflict_page`).
8//! Page-list edits merge as OneNote 2010 merges page series: a page the remote moved keeps
9//! the remote's place, and a page the queue placed goes before the next page in the queue's
10//! order the remote left in place (`corpus/conflict-page/native-pages`, `native-restore`).
6//! Objects the queue itself created are not in the base, so nothing remote can have11//! Objects the queue itself created are not in the base, so nothing remote can have
7//! changed them.12//! changed them.
813
9use crate::{ConflictKind, PendingEdit, Result};14use crate::{PendingEdit, Result};
10use onestore::{15use onestore::{
11 ExGuid, OutlineEdit, PageCreation, PageEdit, PagePosition, Section,16 ExGuid, OutlineEdit, PageCreation, PageEdit, PagePosition, Section,
12 document::{Format, Layout, Tag},17 document::{Format, Layout, Tag},
...@@ -21,143 +26,113 @@ use std::{...@@ -21,143 +26,113 @@ use std::{
21 ops::Range,26 ops::Range,
22};27};
2328
24/// A conflict being resolved: from edit `first` on, ops on `space` are dropped. Keeping29/// The remote changed what an op reads: the op does not replay.
25/// mine replaces the first of them by the ops rewriting the remote page to `target` (or30struct Conflicting;
26/// putting `target` back as a page where the remote removed it), or, for the section's
27/// page list, keeps what of each section op still applies.
28pub(crate) struct Resolving<'p> {
29 pub first: u64,
30 pub space: ExGuid,
31 pub mine: bool,
32 pub target: Option<&'p Page>,
33}
3431
35pub(crate) enum Outcome {32/// The queue replayed on the remote section.
36 /// Every edit applied; these edits' ops changed. An edit whose ops were all dropped33pub(crate) struct Merged {
37 /// stays, empty, to be published and acknowledged with its batch.34 /// Edits whose ops changed. An edit whose ops were all dropped stays, empty, to be
38 Applied(Vec<(u64, Edit)>),35 /// published and acknowledged with its batch.
39 Conflict {36 pub rewritten: Vec<(u64, Edit)>,
40 id: u64,37 /// Pages whose local version the remote could not take, with who made the first edit
41 space: ExGuid,38 /// that did not replay and the objects the dropped ops named.
42 kind: ConflictKind,39 pub conflicts: BTreeMap<ExGuid, (String, BTreeSet<ExGuid>)>,
43 },
44}40}
4541
46/// Replays `edits` on `new`, the remote section; `old` is the base they were made on.42/// Replays `edits` on `new`, the remote section; `old` is the base they were made on.
43/// Ops on the pages in `converged`, which the remote holds as the edits leave them, are done.
47pub(crate) fn rebase(44pub(crate) fn rebase(
48 old: &mut Section<'_>,45 old: &mut Section<'_>,
49 new: &mut Section<'_>,46 new: &mut Section<'_>,
50 edits: &[PendingEdit],47 edits: &[PendingEdit],
51 converged: &BTreeSet<ExGuid>,48 converged: &BTreeSet<ExGuid>,
52 resolving: Option<Resolving<'_>>,49) -> Result<Merged> {
53) -> Result<Outcome> {
54 let root = new.root();
55 let before: BTreeMap<ExGuid, ExGuid> = old.revisions().collect();50 let before: BTreeMap<ExGuid, ExGuid> = old.revisions().collect();
56 let after: BTreeMap<ExGuid, ExGuid> = new.revisions().collect();51 let after: BTreeMap<ExGuid, ExGuid> = new.revisions().collect();
57 let orders = (order(old)?, order(new)?);52 let orders = (order(old)?, order(new)?);
58 let mut state = Replay {53 let mut state = Replay {
59 before,54 before,
60 after,55 after,
56 local: orders.0.clone(),
57 unmoved: unmoved(old, new)?,
61 orders,58 orders,
62 created: BTreeSet::new(),59 created: BTreeSet::new(),
63 diffs: BTreeMap::new(),60 diffs: BTreeMap::new(),
64 };61 };
65 let mut rewritten = Vec::new();62 let mut merged = Merged {
66 let mut resolved = false;63 rewritten: Vec::new(),
64 conflicts: BTreeMap::new(),
65 };
67 for queued in edits {66 for queued in edits {
68 let branch = resolving
69 .as_ref()
70 .filter(|resolving| queued.id >= resolving.first);
71 let mut ops = Vec::new();67 let mut ops = Vec::new();
72 let mut changed = false;68 let mut changed = false;
73 for op in &queued.edit.ops {69 for op in &queued.edit.ops {
74 let conflict = |kind, space| Outcome::Conflict {70 let apply = |state: &mut Replay, new: &mut Section<'_>, op: &Op| {
75 id: queued.id,71 state.apply(new, &queued.author, queued.edit.at, op)
76 space,
77 kind,
78 };72 };
79 if let Some(resolving) = branch {73 match op {
80 let dropped = match op {74 Op::Page { space, op: page_op } => {
81 Op::Page { space, .. } => *space == resolving.space,75 if converged.contains(space) {
82 Op::Section(_) => resolving.space == root,76 changed = true;
83 };77 continue;
84 if dropped {78 }
85 changed = true;79 // An op naming what a dropped op named would build on it.
86 let rewrite = match op {80 let named = named(page_op);
87 Op::Page { space, .. } if !resolved && resolving.mine => {81 if let Some((_, dropped)) = merged.conflicts.get_mut(space)
88 resolving.target.map(|target| (*space, target))82 && named.iter().any(|id| dropped.contains(id))
83 {
84 dropped.extend(named);
85 changed = true;
86 continue;
87 }
88 let replayed = match state.check(old, new, *space, page_op)? {
89 Ok(None) => {
90 changed = true;
91 continue;
92 }
93 Ok(Some(mapped)) => {
94 let mapped = Op::Page {
95 space: *space,
96 op: mapped,
97 };
98 apply(&mut state, new, &mapped)?.then_some(mapped)
89 }99 }
90 _ => None,100 Err(Conflicting) => None,
91 };101 };
92 resolved |= matches!(op, Op::Page { .. });102 match replayed {
93 if let Some((space, target)) = rewrite {103 Some(mapped) => {
94 if !state.listed(space) {104 changed |= mapped != *op;
95 // Kept while the remote removed it: the page comes back.105 ops.push(mapped);
96 match state.restore(
97 new,
98 space,
99 target,
100 &queued.author,
101 queued.edit.at,
102 )? {
103 Ok(restored) => ops.extend(restored),
104 Err(kind) => return Ok(conflict(kind, space)),
105 }
106 continue;
107 }106 }
108 let current = new.page(space)?;107 None => {
109 for op in onestore::op::lower_page(&current, target)108 changed = true;
110 .map_err(|error| OpError::Unsupported(error.message))?109 merged
111 {110 .conflicts
112 let op = Op::Page { space, op };111 .entry(*space)
113 if let Some(kind) =112 .or_insert_with(|| (queued.author.clone(), BTreeSet::new()))
114 state.apply(new, &queued.author, queued.edit.at, &op)?113 .1
115 {114 .extend(named);
116 return Ok(conflict(kind, space));
117 }
118 ops.push(op);
119 }115 }
120 }116 }
121 if let (Op::Section(op), true) = (op, resolving.mine) {117 }
122 // Keeping mine for the section list: the first form that applies.118 Op::Section(section_op) => {
123 for op in state.salvage(new, op) {119 advance(&mut state.local, section_op);
124 let op = Op::Section(op);120 // The first form of the edit the remote's page list takes, or none.
125 if state121 let mut kept = None;
126 .apply(new, &queued.author, queued.edit.at, &op)?122 for form in state.forms(new, section_op)? {
127 .is_none()123 let form = Op::Section(form);
128 {124 if apply(&mut state, new, &form)? {
129 ops.push(op);125 kept = Some(form);
130 break;126 break;
131 }
132 }127 }
133 }128 }
134 continue;129 changed |= kept.as_ref() != Some(op);
135 }130 ops.extend(kept);
136 }
137 let space = match op {
138 Op::Page { space, .. } => *space,
139 Op::Section(_) => root,
140 };
141 if matches!(op, Op::Page { .. }) && converged.contains(&space) {
142 changed = true;
143 continue;
144 }
145 let mapped = match state.check(old, new, op)? {
146 Ok(Some(mapped)) => mapped,
147 Ok(None) => {
148 changed = true;
149 continue;
150 }131 }
151 Err(kind) => return Ok(conflict(kind, space)),
152 };
153 changed |= mapped != *op;
154 if let Some(kind) = state.apply(new, &queued.author, queued.edit.at, &mapped)? {
155 return Ok(conflict(kind, space));
156 }132 }
157 ops.push(mapped);
158 }133 }
159 if changed {134 if changed {
160 rewritten.push((135 merged.rewritten.push((
161 queued.id,136 queued.id,
162 Edit {137 Edit {
163 at: queued.edit.at,138 at: queued.edit.at,
...@@ -166,7 +141,176 @@ pub(crate) fn rebase(...@@ -166,7 +141,176 @@ pub(crate) fn rebase(
166 ));141 ));
167 }142 }
168 }143 }
169 Ok(Outcome::Applied(rewritten))144 Ok(merged)
145}
146
147/// The stored objects a page op names; a conflict page marks those it holds.
148fn named(op: &PageOp) -> Vec<ExGuid> {
149 match op {
150 PageOp::Text { text, .. }
151 | PageOp::Format { text, .. }
152 | PageOp::Link { text, .. }
153 | PageOp::Equation { text, .. }
154 | PageOp::Split { text, .. } => vec![*text],
155 PageOp::Date { fields, .. } => fields.iter().map(|(text, _)| *text).collect(),
156 PageOp::Color(_) => Vec::new(),
157 PageOp::Join { left, right } => vec![*left, *right],
158 PageOp::Insert { container, .. } => vec![*container],
159 PageOp::Move { object, .. } | PageOp::Delete { object } | PageOp::Outline { object, .. } => {
160 vec![*object]
161 }
162 PageOp::Level { paragraph, .. }
163 | PageOp::Paragraph { paragraph, .. }
164 | PageOp::Style { paragraph, .. }
165 | PageOp::List { paragraph, .. } => vec![*paragraph],
166 PageOp::Tags { target, .. } => vec![*target],
167 PageOp::Add { object, .. } => vec![object.id()],
168 PageOp::Picture { picture: object, .. }
169 | PageOp::Attachment {
170 attachment: object, ..
171 }
172 | PageOp::Strokes { ink: object, .. } => vec![*object],
173 PageOp::Table { table, edit } => match edit {
174 TableEdit::Cell { cell, .. } => vec![*cell],
175 TableEdit::DeleteRow(row) => vec![*row],
176 _ => vec![*table],
177 },
178 }
179}
180
181/// The edit keeping `page`, the version of page `space` the remote could not take, and
182/// applies it to `new`: a conflict page under the remote's page marking `objects` where
183/// `page` holds them, or, where the remote removed the page, the page put back as a copy
184/// where `local`, the section as the queue leaves it, has it. Content outside the page
185/// model stays on the remote's page only; `at` is now.
186#[allow(clippy::too_many_arguments)]
187pub(crate) fn conflict_page(
188 new: &mut Section<'_>,
189 old: &mut Section<'_>,
190 local: &mut Section<'_>,
191 space: ExGuid,
192 page: &Page,
193 author: &str,
194 objects: &BTreeSet<ExGuid>,
195 at: u64,
196) -> Result<Edit> {
197 let mut page = page.clone();
198 page.objects.retain(|object| !matches!(object, PageObject::Unsupported(_)));
199 let listed = order(new)?;
200 let ops = if listed.iter().any(|(listed, _)| *listed == space) {
201 let index = Index::of(&page);
202 let mut marked: Vec<ExGuid> = objects
203 .iter()
204 .copied()
205 .filter(|id| index.nodes.contains_key(id))
206 .collect();
207 let copy = page.copy_with(&mut marked)?;
208 let titled = page
209 .objects
210 .iter()
211 .any(|object| matches!(object, PageObject::Title(_)));
212 vec![Op::Section(SectionOp::Conflict {
213 of: space,
214 creation: PageCreation::new(None, titled.then_some(page.title.as_str()), author)?,
215 page: copy,
216 objects: marked,
217 })]
218 } else {
219 // A copy under fresh identities, as OneNote 2010 puts a removed page back.
220 let (base, unmoved) = (order(old)?, unmoved(old, new)?);
221 let queued = order(local)?;
222 let at = queued.iter().position(|(queued, _)| *queued == space);
223 let before = at
224 .map_or(&[][..], |at| &queued[at + 1..])
225 .iter()
226 .map(|(next, _)| *next)
227 .find(|next| {
228 listed.iter().any(|(listed, _)| listed == next)
229 && (unmoved.contains(next) || base.iter().all(|(page, _)| page != next))
230 });
231 let creation = PageCreation::new(before, Some(&page.title), author)?;
232 let mut ops = vec![Op::Section(SectionOp::Import {
233 creation: creation.clone(),
234 page: page.copy()?,
235 })];
236 if let Some(level) = at.map(|at| queued[at].1).filter(|level| *level > 1) {
237 ops.push(Op::Section(SectionOp::Pages(vec![PageEdit::set_level(
238 creation.space(),
239 level,
240 )?])));
241 }
242 ops
243 };
244 let edit = Edit { at, ops };
245 new.apply(author, &edit)?;
246 Ok(edit)
247}
248
249/// Pages the base and the remote both list that the remote left in place: in the same
250/// series at the same level, in base order among themselves. OneNote 2010 gives a page it
251/// moves a series of its own; a page moved within its series is out of base order.
252fn unmoved(old: &mut Section<'_>, new: &mut Section<'_>) -> Result<BTreeSet<ExGuid>> {
253 let series = (old.series()?, new.series()?);
254 let base: BTreeMap<ExGuid, (usize, u32)> = order(old)?
255 .into_iter()
256 .enumerate()
257 .map(|(at, (space, level))| (space, (at, level)))
258 .collect();
259 let kept: Vec<(ExGuid, usize)> = order(new)?
260 .into_iter()
261 .filter_map(|(space, level)| {
262 let (at, before) = base.get(&space)?;
263 (*before == level && series.0.get(&space) == series.1.get(&space)).then_some((space, *at))
264 })
265 .collect();
266 // A longest subsequence of `kept` in base order (patience sorting).
267 let mut tails: Vec<usize> = Vec::new();
268 let mut previous = vec![None; kept.len()];
269 for (i, (_, at)) in kept.iter().enumerate() {
270 let k = tails.partition_point(|tail| kept[*tail].1 < *at);
271 previous[i] = k.checked_sub(1).map(|k| tails[k]);
272 if k == tails.len() {
273 tails.push(i);
274 } else {
275 tails[k] = i;
276 }
277 }
278 let mut unmoved = BTreeSet::new();
279 let mut next = tails.last().copied();
280 while let Some(i) = next {
281 unmoved.insert(kept[i].0);
282 next = previous[i];
283 }
284 Ok(unmoved)
285}
286
287/// Applies a queued page-list edit to `local`, the page order as the queue leaves it.
288fn advance(local: &mut Order, op: &SectionOp) {
289 let insert = |local: &mut Order, page: (ExGuid, u32), before: Option<ExGuid>| {
290 let at = before
291 .and_then(|before| local.iter().position(|(space, _)| *space == before))
292 .unwrap_or(local.len());
293 local.insert(at, page);
294 };
295 match op {
296 SectionOp::Create(creation) | SectionOp::Import { creation, .. } => {
297 insert(local, (creation.space(), 1), creation.before());
298 }
299 SectionOp::Pages(edits) => {
300 for edit in edits {
301 let Some(at) = local.iter().position(|(space, _)| *space == edit.space()) else {
302 continue;
303 };
304 local[at].1 = edit.level();
305 if let PagePosition::Before(before) = edit.position() {
306 let page = local.remove(at);
307 insert(local, page, before);
308 }
309 }
310 }
311 SectionOp::Delete(spaces) => local.retain(|(space, _)| !spaces.contains(space)),
312 SectionOp::Conflict { .. } | SectionOp::Color(_) => {}
313 }
170}314}
171315
172fn order(section: &mut Section<'_>) -> Result<Order> {316fn order(section: &mut Section<'_>) -> Result<Order> {
...@@ -186,6 +330,10 @@ struct Replay {...@@ -186,6 +330,10 @@ struct Replay {
186 after: BTreeMap<ExGuid, ExGuid>,330 after: BTreeMap<ExGuid, ExGuid>,
187 /// Page order of the base and the remote.331 /// Page order of the base and the remote.
188 orders: (Order, Order),332 orders: (Order, Order),
333 /// Page order as the queue leaves it, up to the op replaying.
334 local: Order,
335 /// Pages the remote left in place (`unmoved`).
336 unmoved: BTreeSet<ExGuid>,
189 /// Page spaces the replayed edits created.337 /// Page spaces the replayed edits created.
190 created: BTreeSet<ExGuid>,338 created: BTreeSet<ExGuid>,
191 /// Per page the remote changed, what it changed; `None` when it left the page alone.339 /// Per page the remote changed, what it changed; `None` when it left the page alone.
...@@ -193,14 +341,8 @@ struct Replay {...@@ -193,14 +341,8 @@ struct Replay {
193}341}
194342
195impl Replay {343impl Replay {
196 /// Applies one op to the remote section; `Some` names why it did not apply.344 /// Applies one op to the remote section; false when the section refused it.
197 fn apply(345 fn apply(&mut self, new: &mut Section<'_>, author: &str, at: u64, op: &Op) -> Result<bool> {
198 &mut self,
199 new: &mut Section<'_>,
200 author: &str,
201 at: u64,
202 op: &Op,
203 ) -> Result<Option<ConflictKind>> {
204 match new.apply(346 match new.apply(
205 author,347 author,
206 &Edit {348 &Edit {
...@@ -210,64 +352,49 @@ impl Replay {...@@ -210,64 +352,49 @@ impl Replay {
210 ) {352 ) {
211 Ok(()) => {353 Ok(()) => {
212 if let Op::Section(354 if let Op::Section(
213 SectionOp::Create(creation) | SectionOp::Import { creation, .. },355 SectionOp::Create(creation)
356 | SectionOp::Import { creation, .. }
357 | SectionOp::Conflict { creation, .. },
214 ) = op358 ) = op
215 {359 {
216 self.created.insert(creation.space());360 self.created.insert(creation.space());
217 }361 }
218 Ok(None)362 Ok(true)
219 }363 }
220 Err(OpError::Failed(error)) => Err(error.into()),364 Err(OpError::Failed(error)) => Err(error.into()),
221 Err(error) => Ok(Some(match op {365 Err(_) => Ok(false),
222 // A page list the remote reordered refuses the op, however it says so.
223 Op::Section(_) => ConflictKind::StructureChanged,
224 Op::Page { .. } if matches!(error, OpError::Unsupported(_)) => {
225 ConflictKind::UnsupportedEdit
226 }
227 Op::Page { space, .. } if !self.listed(*space) => ConflictKind::TargetUnavailable,
228 Op::Page { .. } => ConflictKind::ContentChanged,
229 })),
230 }366 }
231 }367 }
232368
233 /// The op as it replays on the remote section, or why it cannot.369 /// The page op as it replays on the remote section; `None` when the remote made it.
234 fn check(370 fn check(
235 &mut self,371 &mut self,
236 old: &mut Section<'_>,372 old: &mut Section<'_>,
237 new: &mut Section<'_>,373 new: &mut Section<'_>,
238 op: &Op,374 space: ExGuid,
239 ) -> Result<std::result::Result<Option<Op>, ConflictKind>> {375 op: &PageOp,
240 match op {376 ) -> Result<std::result::Result<Option<PageOp>, Conflicting>> {
241 Op::Page { space, op } => {377 if !self.listed(space) {
242 if !self.listed(*space) {378 return Ok(Err(Conflicting));
243 return Ok(Err(ConflictKind::TargetUnavailable));379 }
244 }380 let diff = match self.diffs.get_mut(&space) {
245 let diff = match self.diffs.get_mut(space) {381 Some(diff) => diff,
246 Some(diff) => diff,382 None => {
247 None => {383 let diff = match (self.before.get(&space), self.after.get(&space)) {
248 let diff = match (self.before.get(space), self.after.get(space)) {384 (Some(before), Some(after)) if before != after => Some(Diff {
249 (Some(before), Some(after)) if before != after => Some(Diff {385 old: Index::of(&old.page(space)?),
250 old: Index::of(&old.page(*space)?),386 new: Index::of(&new.page(space)?),
251 new: Index::of(&new.page(*space)?),387 regions: BTreeMap::new(),
252 regions: BTreeMap::new(),388 }),
253 }),389 _ => None,
254 _ => None,
255 };
256 self.diffs.entry(*space).or_insert(diff)
257 }
258 };390 };
259 Ok(match diff {391 self.diffs.entry(space).or_insert(diff)
260 None => Ok(Some(Op::Page {
261 space: *space,
262 op: op.clone(),
263 })),
264 Some(diff) => diff
265 .check(op)
266 .map(|op| op.map(|op| Op::Page { space: *space, op })),
267 })
268 }392 }
269 Op::Section(section) => Ok(self.section(section).map(|()| Some(op.clone()))),393 };
270 }394 Ok(match diff {
395 None => Ok(Some(op.clone())),
396 Some(diff) => diff.check(op),
397 })
271 }398 }
272399
273 /// Whether the remote section lists the page, or the replayed edits created it; a page400 /// Whether the remote section lists the page, or the replayed edits created it; a page
...@@ -276,151 +403,82 @@ impl Replay {...@@ -276,151 +403,82 @@ impl Replay {
276 self.created.contains(&space) || self.orders.1.iter().any(|(page, _)| *page == space)403 self.created.contains(&space) || self.orders.1.iter().any(|(page, _)| *page == space)
277 }404 }
278405
279 /// Puts `page`, which the remote removed, back into the section and returns the ops406 /// Whether the remote moved or re-leveled a page the base listed.
280 /// that did: a copy under fresh identities at its level, before the first page after it407 fn placed(&self, page: ExGuid) -> bool {
281 /// that the remote still has and that can take a page before it, or last.408 !self.unmoved.contains(&page) && self.orders.0.iter().any(|(listed, _)| *listed == page)
282 fn restore(
283 &mut self,
284 new: &mut Section<'_>,
285 space: ExGuid,
286 page: &Page,
287 author: &str,
288 at: u64,
289 ) -> Result<std::result::Result<Vec<Op>, ConflictKind>> {
290 let old = self.orders.0.clone();
291 let position = old.iter().position(|(listed, _)| *listed == space);
292 let anchors = position
293 .map_or(&[][..], |position| &old[position + 1..])
294 .iter()
295 .map(|(next, _)| Some(*next))
296 .filter(|next| next.is_some_and(|next| self.listed(next)))
297 .chain([None])
298 .collect::<Vec<_>>();
299 let copy = page.copy()?;
300 let mut refusal = ConflictKind::StructureChanged;
301 for before in anchors {
302 let creation = PageCreation::new(before, Some(&page.title), author)?;
303 let import = Op::Section(SectionOp::Import {
304 creation: creation.clone(),
305 page: copy.clone(),
306 });
307 match self.apply(new, author, at, &import)? {
308 Some(kind) => refusal = kind,
309 None => {
310 let mut ops = vec![import];
311 if let Some(level) = position.map(|at| old[at].1).filter(|level| *level > 1) {
312 let indent = Op::Section(SectionOp::Pages(vec![PageEdit::set_level(
313 creation.space(),
314 level,
315 )?]));
316 if let Some(kind) = self.apply(new, author, at, &indent)? {
317 return Ok(Err(kind));
318 }
319 ops.push(indent);
320 }
321 return Ok(Ok(ops));
322 }
323 }
324 }
325 Ok(Err(refusal))
326 }409 }
327410
328 fn section(&self, op: &SectionOp) -> std::result::Result<(), ConflictKind> {411 /// Where a page the queue put before `before` goes on the remote: before the first page
329 let (old, new) = &self.orders;412 /// from `before` on, in the queue's order, that the remote lists where the base had it,
330 let find =413 /// or last, as OneNote 2010 places a page series only one side changed.
331 |list: &[(ExGuid, u32)], space: ExGuid| list.iter().position(|(s, _)| *s == space);414 fn anchor(&self, listed: &Order, page: ExGuid, before: Option<ExGuid>) -> Option<ExGuid> {
332 match op {415 let from = self.local.iter().position(|(space, _)| Some(*space) == before)?;
333 SectionOp::Create(_) | SectionOp::Import { .. } => Ok(()),416 self.local[from..].iter().map(|(space, _)| *space).find(|space| {
334 SectionOp::Delete(pages) => {417 *space != page && !self.placed(*space) && listed.iter().any(|(listed, _)| listed == space)
335 for page in pages {418 })
336 if find(new, *page).is_none() && !self.created.contains(page) {419 }
337 return Err(ConflictKind::TargetUnavailable);420
338 }421 /// Forms of a section op to try on the remote section, in order: a page placed by
339 if self.before.get(page) != self.after.get(page) {422 /// `anchor`, the page edits of pages the remote still has and did not move, removals of
340 return Err(ConflictKind::ContentChanged);423 /// pages the remote still has (a page it changed is not removed), a conflict page's
341 }424 /// content as a page of its own where the remote removed its page.
425 fn forms(&self, new: &mut Section<'_>, op: &SectionOp) -> Result<Vec<SectionOp>> {
426 let conflicts: BTreeSet<ExGuid> = new
427 .conflicts()?
428 .into_iter()
429 .flat_map(|(_, pages)| pages.into_iter().map(|page| page.space))
430 .collect();
431 let listed = order(new)?;
432 let placed = |creation: &PageCreation| {
433 creation.reposition(self.anchor(&listed, creation.space(), creation.before()))
434 };
435 Ok(match op {
436 SectionOp::Create(creation) => vec![SectionOp::Create(placed(creation)?)],
437 SectionOp::Import { creation, page } => vec![SectionOp::Import {
438 creation: placed(creation)?,
439 page: page.clone(),
440 }],
441 SectionOp::Color(_) => vec![op.clone()],
442 SectionOp::Conflict {
443 of, creation, page, ..
444 } => vec![if self.listed(*of) {
445 op.clone()
446 } else {
447 SectionOp::Import {
448 creation: creation.clone(),
449 page: page.clone(),
342 }450 }
343 Ok(())451 }],
344 }452 // A page the remote moved or re-leveled keeps the remote's placement.
345 SectionOp::Pages(edits) => {453 SectionOp::Pages(edits) => {
346 // Where the batch puts each page, from the base order.454 let mut kept = Vec::new();
347 let mut wanted: Vec<ExGuid> = old.iter().map(|(space, _)| *space).collect();
348 for edit in edits {455 for edit in edits {
349 if let PagePosition::Before(before) = edit.position()456 if self.placed(edit.space())
350 && let Some(at) = wanted.iter().position(|space| *space == edit.space())457 || listed.iter().all(|(listed, _)| *listed != edit.space())
351 {458 {
352 let space = wanted.remove(at);
353 let at = before
354 .and_then(|before| wanted.iter().position(|space| *space == before))
355 .unwrap_or(wanted.len());
356 wanted.insert(at, space);
357 }
358 }
359 let wanted: Vec<(ExGuid, u32)> =
360 wanted.into_iter().map(|space| (space, 0)).collect();
361 for edit in edits {
362 let space = edit.space();
363 let Some(at) = find(old, space) else {
364 continue;459 continue;
365 };
366 let Some(now) = find(new, space) else {
367 return Err(ConflictKind::TargetUnavailable);
368 };
369 if old[at].1 != new[now].1 && new[now].1 != edit.level() {
370 return Err(ConflictKind::StructureChanged);
371 }
372 if edit.position() != PagePosition::Keep {
373 // Each page the remote also has must keep its side of the moved page,
374 // or already be on the side the batch wants.
375 let side = |list: &[(ExGuid, u32)], peer: ExGuid| {
376 Some(find(list, peer)? < find(list, space)?)
377 };
378 let conflicting = old.iter().any(|(peer, _)| {
379 *peer != space
380 && find(new, *peer).is_some()
381 && side(new, *peer) != side(old, *peer)
382 && side(new, *peer) != side(&wanted, *peer)
383 });
384 if conflicting {
385 return Err(ConflictKind::StructureChanged);
386 }
387 }460 }
461 kept.push(match edit.position() {
462 PagePosition::Keep => edit.clone(),
463 PagePosition::Before(before) => edit.reposition(
464 PagePosition::Before(self.anchor(&listed, edit.space(), before)),
465 edit.level(),
466 )?,
467 });
388 }468 }
389 Ok(())469 if kept.is_empty() {
390 }
391 }
392 }
393
394 /// Forms of a section op to try on the remote section list, in order: as it is, a
395 /// creation appended instead, moves and removals of the pages still there.
396 fn salvage(&self, new: &mut Section<'_>, op: &SectionOp) -> Vec<SectionOp> {
397 let pages: BTreeSet<ExGuid> = new
398 .pages()
399 .map(|pages| pages.into_iter().map(|(space, ..)| space).collect())
400 .unwrap_or_default();
401 let present = |space: &ExGuid| pages.contains(space) || self.created.contains(space);
402 match op {
403 SectionOp::Create(creation) => std::iter::once(creation.clone())
404 .chain(creation.reposition(None).ok())
405 .map(SectionOp::Create)
406 .collect(),
407 SectionOp::Import { .. } => vec![op.clone()],
408 SectionOp::Pages(edits) => {
409 let edits: Vec<PageEdit> = edits
410 .iter()
411 .filter(|edit| present(&edit.space()))
412 .cloned()
413 .collect();
414 if edits.is_empty() {
415 Vec::new()470 Vec::new()
416 } else {471 } else {
417 vec![SectionOp::Pages(edits)]472 vec![SectionOp::Pages(kept)]
418 }473 }
419 }474 }
420 SectionOp::Delete(deleted) => {475 SectionOp::Delete(deleted) => {
421 let deleted: Vec<ExGuid> = deleted476 let deleted: Vec<ExGuid> = deleted
422 .iter()477 .iter()
423 .filter(|space| present(space))478 .filter(|space| {
479 self.listed(**space) && self.before.get(space) == self.after.get(space)
480 || conflicts.contains(space)
481 })
424 .copied()482 .copied()
425 .collect();483 .collect();
426 if deleted.is_empty() {484 if deleted.is_empty() {
...@@ -429,7 +487,7 @@ impl Replay {...@@ -429,7 +487,7 @@ impl Replay {
429 vec![SectionOp::Delete(deleted)]487 vec![SectionOp::Delete(deleted)]
430 }488 }
431 }489 }
432 }490 })
433 }491 }
434}492}
435493
...@@ -491,7 +549,7 @@ impl Diff {...@@ -491,7 +549,7 @@ impl Diff {
491549
492 /// Where the remote changed a text's characters and formats; its tags and date field550 /// Where the remote changed a text's characters and formats; its tags and date field
493 /// are not positions an op names.551 /// are not positions an op names.
494 fn region(&mut self, text: ExGuid) -> std::result::Result<Option<&mut Changes>, ConflictKind> {552 fn region(&mut self, text: ExGuid) -> std::result::Result<Option<&mut Changes>, Conflicting> {
495 if !self.regions.contains_key(&text) {553 if !self.regions.contains_key(&text) {
496 if self.ours(text) {554 if self.ours(text) {
497 return Ok(None);555 return Ok(None);
...@@ -500,7 +558,7 @@ impl Diff {...@@ -500,7 +558,7 @@ impl Diff {
500 self.old.nodes.get(&text).map(|node| &node.own),558 self.old.nodes.get(&text).map(|node| &node.own),
501 self.new.nodes.get(&text).map(|node| &node.own),559 self.new.nodes.get(&text).map(|node| &node.own),
502 ) else {560 ) else {
503 return Err(ConflictKind::ContentChanged);561 return Err(Conflicting);
504 };562 };
505 if a.text == b.text {563 if a.text == b.text {
506 return Ok(None);564 return Ok(None);
...@@ -512,21 +570,20 @@ impl Diff {...@@ -512,21 +570,20 @@ impl Diff {
512 }570 }
513571
514 /// The op as it replays on the remote page; `None` when the remote already made it.572 /// The op as it replays on the remote page; `None` when the remote already made it.
515 fn check(&mut self, op: &PageOp) -> std::result::Result<Option<PageOp>, ConflictKind> {573 fn check(&mut self, op: &PageOp) -> std::result::Result<Option<PageOp>, Conflicting> {
516 use ConflictKind::ContentChanged as Changed;574 let require = |ok: bool| if ok { Ok(()) } else { Err(Conflicting) };
517 let require = |ok: bool| if ok { Ok(()) } else { Err(Changed) };
518 let mut op = op.clone();575 let mut op = op.clone();
519 match &mut op {576 match &mut op {
520 PageOp::Text { text, range, with } => {577 PageOp::Text { text, range, with } => {
521 if let Some(region) = self.region(*text)? {578 if let Some(region) = self.region(*text)? {
522 let mapped = region.map(range).ok_or(Changed)?;579 let mapped = region.map(range).ok_or(Conflicting)?;
523 region.replaced(range, with.encode_utf16().count() as u32);580 region.replaced(range, with.encode_utf16().count() as u32);
524 *range = mapped;581 *range = mapped;
525 }582 }
526 }583 }
527 PageOp::Format { text, range, .. } => {584 PageOp::Format { text, range, .. } => {
528 if let Some(region) = self.region(*text)? {585 if let Some(region) = self.region(*text)? {
529 *range = region.map(range).ok_or(Changed)?;586 *range = region.map(range).ok_or(Conflicting)?;
530 }587 }
531 }588 }
532 PageOp::Link { text, .. } | PageOp::Equation { text, .. } => {589 PageOp::Link { text, .. } | PageOp::Equation { text, .. } => {
...@@ -547,7 +604,7 @@ impl Diff {...@@ -547,7 +604,7 @@ impl Diff {
547 if let Some(region) = self.region(*text)? {604 if let Some(region) = self.region(*text)? {
548 // The cut moves with the character after it: text the remote typed at605 // The cut moves with the character after it: text the remote typed at
549 // the cut stays left of it.606 // the cut stays left of it.
550 let cut = region.map(&(*at..*at + 1)).ok_or(Changed)?;607 let cut = region.map(&(*at..*at + 1)).ok_or(Conflicting)?;
551 let right_changes = region.split(*at);608 let right_changes = region.split(*at);
552 if !right_changes.0.is_empty() {609 if !right_changes.0.is_empty() {
553 self.regions.insert(*right, right_changes);610 self.regions.insert(*right, right_changes);
...@@ -565,7 +622,8 @@ impl Diff {...@@ -565,7 +622,8 @@ impl Diff {
565 self.region(text)?;622 self.region(text)?;
566 }623 }
567 }624 }
568 PageOp::Insert { .. } | PageOp::Add { .. } => {}625 // A page's colour is the latest one set, whichever side set it.
626 PageOp::Insert { .. } | PageOp::Add { .. } | PageOp::Color(_) => {}
569 PageOp::Move {627 PageOp::Move {
570 object,628 object,
571 parent,629 parent,
...@@ -633,7 +691,7 @@ impl Diff {...@@ -633,7 +691,7 @@ impl Diff {
633 }691 }
634 _ => false,692 _ => false,
635 };693 };
636 return if done { Ok(None) } else { Err(Changed) };694 return if done { Ok(None) } else { Err(Conflicting) };
637 }695 }
638 require(before.is_some())?696 require(before.is_some())?
639 }697 }
crates/notebook/src/migrate.rs+58-17
...@@ -1,7 +1,8 @@...@@ -1,7 +1,8 @@
1//! Converts a schema-14 cache, whose queue holds whole-page edits over a working image,1//! Converts a schema-14 cache, whose queue holds whole-page edits over a working image,
2//! to this schema's op queue. The cache is exported first; each queued page is lowered to2//! to this schema's op queue. The cache is exported first; each queued page is lowered to
3//! ops against the page the conversion has so far, and every converted page must equal3//! ops against the page the conversion has so far, and every converted page must equal
4//! the page in the old working image, or nothing changes.4//! the page in the old working image, or nothing changes. A page edit schema 14 held in
5//! conflict becomes the conflict page a merge makes now, the remote's page staying.
56
6use crate::{Result, base, queue, schema};7use crate::{Result, base, queue, schema};
7use onestore::{8use onestore::{
...@@ -87,7 +88,6 @@ struct Converted {...@@ -87,7 +88,6 @@ struct Converted {
87 edit: Edit,88 edit: Edit,
88 /// Schema 14's publication evidence, when this edit was attempted.89 /// Schema 14's publication evidence, when this edit was attempted.
89 attempted: Option<String>,90 attempted: Option<String>,
90 conflict: Option<(i64, ExGuid)>,
91}91}
9292
93pub(crate) fn migrate(connection: &mut Connection, path: &Path) -> Result<()> {93pub(crate) fn migrate(connection: &mut Connection, path: &Path) -> Result<()> {
...@@ -107,6 +107,53 @@ pub(crate) fn migrate(connection: &mut Connection, path: &Path) -> Result<()> {...@@ -107,6 +107,53 @@ pub(crate) fn migrate(connection: &mut Connection, path: &Path) -> Result<()> {
107 Ok(())107 Ok(())
108}108}
109109
110/// The conflict page keeping `local`, the version of page `space` its merge with `remote`
111/// did not take, marking the texts `remote` does not hold as they are.
112fn conflict(space: ExGuid, remote: &Page, local: &Page, author: &str) -> Result<Op> {
113 let texts = |page: &Page| -> BTreeMap<ExGuid, String> {
114 let mut texts = BTreeMap::new();
115 let mut pending: Vec<&onestore::page::PageParagraph> = Vec::new();
116 for object in &page.objects {
117 match object {
118 onestore::page::PageObject::Outline(outline) => pending.extend(&outline.paragraphs),
119 onestore::page::PageObject::Title(title) => {
120 pending.extend(title.outlines.iter().flat_map(|outline| &outline.paragraphs))
121 }
122 _ => {}
123 }
124 }
125 while let Some(paragraph) = pending.pop() {
126 match &paragraph.content {
127 onestore::page::ParagraphContent::Text(text) => {
128 texts.insert(text.id, format!("{:?}", text.text));
129 }
130 onestore::page::ParagraphContent::Table(table) => pending.extend(
131 table.rows.iter().flat_map(|row| &row.cells).flat_map(|cell| &cell.paragraphs),
132 ),
133 _ => {}
134 }
135 }
136 texts
137 };
138 let kept = texts(remote);
139 let mut objects: Vec<ExGuid> = texts(local)
140 .into_iter()
141 .filter(|(id, text)| kept.get(id) != Some(text))
142 .map(|(id, _)| id)
143 .collect();
144 let page = local.copy_with(&mut objects)?;
145 let titled = local
146 .objects
147 .iter()
148 .any(|object| matches!(object, onestore::page::PageObject::Title(_)));
149 Ok(Op::Section(SectionOp::Conflict {
150 of: space,
151 creation: PageCreation::new(None, titled.then_some(local.title.as_str()), author)?,
152 page,
153 objects,
154 }))
155}
156
110fn convert(transaction: &rusqlite::Transaction<'_>) -> Result<()> {157fn convert(transaction: &rusqlite::Transaction<'_>) -> Result<()> {
111 let (base, patch): (Vec<u8>, Vec<u8>) =158 let (base, patch): (Vec<u8>, Vec<u8>) =
112 transaction.query_row("SELECT base, working FROM replica WHERE id=1", [], |row| {159 transaction.query_row("SELECT base, working FROM replica WHERE id=1", [], |row| {
...@@ -118,9 +165,9 @@ fn convert(transaction: &rusqlite::Transaction<'_>) -> Result<()> {...@@ -118,9 +165,9 @@ fn convert(transaction: &rusqlite::Transaction<'_>) -> Result<()> {
118 Ok((row.get(0)?, row.get(1)?))165 Ok((row.get(0)?, row.get(1)?))
119 })166 })
120 .optional()?;167 .optional()?;
121 let conflicts: BTreeMap<i64, i64> = transaction168 let conflicts: std::collections::BTreeSet<i64> = transaction
122 .prepare("SELECT edit_id, kind FROM conflicts")?169 .prepare("SELECT edit_id FROM conflicts")?
123 .query_map([], |row| Ok((row.get(0)?, row.get(1)?)))?170 .query_map([], |row| row.get(0))?
124 .collect::<rusqlite::Result<_>>()?;171 .collect::<rusqlite::Result<_>>()?;
125 let queued: Vec<(i64, String, String)> = transaction172 let queued: Vec<(i64, String, String)> = transaction
126 .prepare("SELECT id, space, operation FROM edits ORDER BY id")?173 .prepare("SELECT id, space, operation FROM edits ORDER BY id")?
...@@ -146,6 +193,11 @@ fn convert(transaction: &rusqlite::Transaction<'_>) -> Result<()> {...@@ -146,6 +193,11 @@ fn convert(transaction: &rusqlite::Transaction<'_>) -> Result<()> {
146 let operation: Operation = serde_json::from_str(&operation)193 let operation: Operation = serde_json::from_str(&operation)
147 .map_err(|error| invalid(format!("Queued edit {id} is unreadable: {error}")))?;194 .map_err(|error| invalid(format!("Queued edit {id} is unreadable: {error}")))?;
148 let (author, ops) = match operation {195 let (author, ops) = match operation {
196 Operation::Page(intent) if conflicts.contains(&id) => {
197 let current = section.page(space)?;
198 let op = conflict(space, &current, &intent.after, &intent.author)?;
199 (intent.author, vec![op])
200 }
149 Operation::Page(intent) => {201 Operation::Page(intent) => {
150 let current = section.page(space)?;202 let current = section.page(space)?;
151 let ops = onestore::op::lower_page(&current, &intent.after)?;203 let ops = onestore::op::lower_page(&current, &intent.after)?;
...@@ -185,17 +237,12 @@ fn convert(transaction: &rusqlite::Transaction<'_>) -> Result<()> {...@@ -185,17 +237,12 @@ fn convert(transaction: &rusqlite::Transaction<'_>) -> Result<()> {
185 }237 }
186 sealed = Some(section.seal()?);238 sealed = Some(section.seal()?);
187 }239 }
188 // A conflicting page-list edit is the section's conflict, whichever page it names.240 // A schema-14 conflict is queued work: the next rebase keeps both versions.
189 let conflicted = match edit.ops.first() {
190 Some(Op::Section(_)) => section.root(),
191 _ => space,
192 };
193 converted.push(Converted {241 converted.push(Converted {
194 id,242 id,
195 author,243 author,
196 edit,244 edit,
197 attempted,245 attempted,
198 conflict: conflicts.get(&id).map(|kind| (*kind, conflicted)),
199 });246 });
200 }247 }
201 let listed = |image: &[u8]| -> Result<Vec<ExGuid>> {248 let listed = |image: &[u8]| -> Result<Vec<ExGuid>> {
...@@ -258,12 +305,6 @@ fn convert(transaction: &rusqlite::Transaction<'_>) -> Result<()> {...@@ -258,12 +305,6 @@ fn convert(transaction: &rusqlite::Transaction<'_>) -> Result<()> {
258 } else {305 } else {
259 batch = Some(current);306 batch = Some(current);
260 }307 }
261 if let Some((kind, space)) = edit.conflict {
262 transaction.execute(
263 "UPDATE batches SET conflict=?1, space=?2 WHERE id=?3",
264 params![kind, space.to_string(), current],
265 )?;
266 }
267 queue::insert(308 queue::insert(
268 transaction,309 transaction,
269 Some(crate::unsigned(edit.id)?),310 Some(crate::unsigned(edit.id)?),
crates/notebook/src/queue.rs+5-1
...@@ -74,7 +74,7 @@ fn slots(edit: &mut Edit, mut visit: impl FnMut(&mut Payload)) {...@@ -74,7 +74,7 @@ fn slots(edit: &mut Edit, mut visit: impl FnMut(&mut Payload)) {
74 } => cells(list, &mut visit),74 } => cells(list, &mut visit),
75 _ => {}75 _ => {}
76 },76 },
77 Op::Section(SectionOp::Import { page, .. }) => {77 Op::Section(SectionOp::Import { page, .. } | SectionOp::Conflict { page, .. }) => {
78 for added in &mut page.objects {78 for added in &mut page.objects {
79 object(added, &mut visit);79 object(added, &mut visit);
80 }80 }
...@@ -235,8 +235,12 @@ pub(crate) fn spaces(edit: &Edit, root: onestore::ExGuid) -> Vec<onestore::ExGui...@@ -235,8 +235,12 @@ pub(crate) fn spaces(edit: &Edit, root: onestore::ExGuid) -> Vec<onestore::ExGui
235 SectionOp::Create(creation) | SectionOp::Import { creation, .. } => {235 SectionOp::Create(creation) | SectionOp::Import { creation, .. } => {
236 spaces.push(creation.space());236 spaces.push(creation.space());
237 }237 }
238 SectionOp::Conflict { of, creation, .. } => {
239 spaces.extend([*of, creation.space()]);
240 }
238 SectionOp::Pages(edits) => spaces.extend(edits.iter().map(|edit| edit.space())),241 SectionOp::Pages(edits) => spaces.extend(edits.iter().map(|edit| edit.space())),
239 SectionOp::Delete(pages) => spaces.extend(pages),242 SectionOp::Delete(pages) => spaces.extend(pages),
243 SectionOp::Color(_) => {}
240 }244 }
241 }245 }
242 }246 }
crates/notebook/src/recovery.rs+4-6
...@@ -8,7 +8,6 @@ const RECOVERY_ID: u32 = 0x4f4e4552;...@@ -8,7 +8,6 @@ const RECOVERY_ID: u32 = 0x4f4e4552;
8#[derive(Debug, Clone, Copy, PartialEq, Eq, serde::Serialize)]8#[derive(Debug, Clone, Copy, PartialEq, Eq, serde::Serialize)]
9pub struct RecoverySummary {9pub struct RecoverySummary {
10 pub queued_edits: u64,10 pub queued_edits: u64,
11 pub conflicts: u64,
12 pub uncertain_edits: u64,11 pub uncertain_edits: u64,
13 pub published_receipts: u64,12 pub published_receipts: u64,
14 /// Bytes of the image the queued edits apply to.13 /// Bytes of the image the queued edits apply to.
...@@ -38,7 +37,8 @@ impl Recovery {...@@ -38,7 +37,8 @@ impl Recovery {
38 io::Error::new(io::ErrorKind::InvalidData, "Not a recovery archive").into(),37 io::Error::new(io::ErrorKind::InvalidData, "Not a recovery archive").into(),
39 );38 );
40 }39 }
41 if version != schema::VERSION {40 // A schema-15 archive differs only in columns nothing reads.
41 if ![schema::PREVIOUS, schema::VERSION].contains(&version) {
42 return Err(io::Error::new(42 return Err(io::Error::new(
43 io::ErrorKind::InvalidData,43 io::ErrorKind::InvalidData,
44 format!(44 format!(
...@@ -166,18 +166,16 @@ fn summary(connection: &Connection) -> Result<RecoverySummary> {...@@ -166,18 +166,16 @@ fn summary(connection: &Connection) -> Result<RecoverySummary> {
166 let length = |image| -> Result<u64> {166 let length = |image| -> Result<u64> {
167 Ok(base::stamp(connection, image)?.map_or(0, |stamp| stamp.length))167 Ok(base::stamp(connection, image)?.map_or(0, |stamp| stamp.length))
168 };168 };
169 let (queued_edits, conflicts, uncertain_edits, published_receipts): (i64, i64, i64, i64) =169 let (queued_edits, uncertain_edits, published_receipts): (i64, i64, i64) =
170 connection.query_row(170 connection.query_row(
171 "SELECT (SELECT count(*) FROM edits),171 "SELECT (SELECT count(*) FROM edits),
172 (SELECT count(*) FROM edits JOIN batches ON batches.id=edits.batch WHERE conflict IS NOT NULL),
173 (SELECT count(*) FROM edits JOIN batches ON batches.id=edits.batch WHERE attempted=1),172 (SELECT count(*) FROM edits JOIN batches ON batches.id=edits.batch WHERE attempted=1),
174 (SELECT count(*) FROM receipts)",173 (SELECT count(*) FROM receipts)",
175 [],174 [],
176 |row| Ok((row.get(0)?, row.get(1)?, row.get(2)?, row.get(3)?)),175 |row| Ok((row.get(0)?, row.get(1)?, row.get(2)?)),
177 )?;176 )?;
178 Ok(RecoverySummary {177 Ok(RecoverySummary {
179 queued_edits: unsigned(queued_edits)?,178 queued_edits: unsigned(queued_edits)?,
180 conflicts: unsigned(conflicts)?,
181 uncertain_edits: unsigned(uncertain_edits)?,179 uncertain_edits: unsigned(uncertain_edits)?,
182 published_receipts: unsigned(published_receipts)?,180 published_receipts: unsigned(published_receipts)?,
183 base_bytes: length(base::Image::Base)?,181 base_bytes: length(base::Image::Base)?,
crates/notebook/src/schema.rs+59-6
...@@ -1,7 +1,9 @@...@@ -1,7 +1,9 @@
1use super::*;1use super::*;
2use rusqlite::Transaction;2use rusqlite::{OptionalExtension, Transaction};
33
4pub(crate) const VERSION: u32 = 15;4pub(crate) const VERSION: u32 = 16;
5/// The schema whose batches still recorded review conflicts.
6pub(crate) const PREVIOUS: u32 = 15;
57
6/// Tables a schema-14 cache shares with this one.8/// Tables a schema-14 cache shares with this one.
7pub(crate) const KEPT: &str = "9pub(crate) const KEPT: &str = "
...@@ -28,11 +30,8 @@ pub(crate) const QUEUE: &str = "...@@ -28,11 +30,8 @@ pub(crate) const QUEUE: &str = "
28 sealed TEXT,30 sealed TEXT,
29 revisions TEXT,31 revisions TEXT,
30 attempted INTEGER NOT NULL DEFAULT 0 CHECK(attempted IN (0,1)),32 attempted INTEGER NOT NULL DEFAULT 0 CHECK(attempted IN (0,1)),
31 conflict INTEGER CHECK(conflict BETWEEN 0 AND 3),
32 space TEXT,
33 CHECK((sealed IS NULL) = (revisions IS NULL)),33 CHECK((sealed IS NULL) = (revisions IS NULL)),
34 CHECK(attempted=0 OR sealed IS NOT NULL),34 CHECK(attempted=0 OR sealed IS NOT NULL)
35 CHECK((conflict IS NULL) = (space IS NULL))
36 ) STRICT;35 ) STRICT;
37 CREATE TABLE edits (36 CREATE TABLE edits (
38 id INTEGER PRIMARY KEY AUTOINCREMENT CHECK(id>0),37 id INTEGER PRIMARY KEY AUTOINCREMENT CHECK(id>0),
...@@ -51,3 +50,57 @@ pub(crate) fn create(transaction: &Transaction<'_>) -> Result<()> {...@@ -51,3 +50,57 @@ pub(crate) fn create(transaction: &Transaction<'_>) -> Result<()> {
51 transaction.execute_batch(KEPT)?;50 transaction.execute_batch(KEPT)?;
52 Ok(())51 Ok(())
53}52}
53
54/// Converts a schema-15 cache: its batches lose the review conflict and its page, which
55/// a merge no longer records. A conflict it held is queued work the next rebase keeps as
56/// conflict pages. Foreign keys stay off while `edits` briefly references the old table.
57pub(crate) fn upgrade(connection: &mut Connection) -> Result<()> {
58 connection.execute_batch("PRAGMA foreign_keys=OFF")?;
59 let upgraded = (|| -> Result<()> {
60 let transaction = connection.transaction_with_behavior(TransactionBehavior::Exclusive)?;
61 let sequence: Option<i64> = transaction
62 .query_row(
63 "SELECT seq FROM sqlite_sequence WHERE name='batches'",
64 [],
65 |row| row.get(0),
66 )
67 .optional()?;
68 transaction.execute_batch(
69 "CREATE TABLE upgraded (
70 id INTEGER PRIMARY KEY AUTOINCREMENT CHECK(id>0),
71 sealed TEXT,
72 revisions TEXT,
73 attempted INTEGER NOT NULL DEFAULT 0 CHECK(attempted IN (0,1)),
74 CHECK((sealed IS NULL) = (revisions IS NULL)),
75 CHECK(attempted=0 OR sealed IS NOT NULL)
76 ) STRICT;
77 INSERT INTO upgraded(id, sealed, revisions, attempted)
78 SELECT id, sealed, revisions, attempted FROM batches;
79 DROP TABLE batches;
80 ALTER TABLE upgraded RENAME TO batches;",
81 )?;
82 if let Some(sequence) = sequence {
83 transaction.execute(
84 "UPDATE sqlite_sequence SET seq=max(seq, ?1) WHERE name='batches'",
85 [sequence],
86 )?;
87 }
88 let dangling: bool = transaction.query_row(
89 "SELECT EXISTS(SELECT 1 FROM pragma_foreign_key_check)",
90 [],
91 |row| row.get(0),
92 )?;
93 if dangling {
94 return Err(io::Error::new(
95 io::ErrorKind::InvalidData,
96 "The schema-15 cache holds edits without their batch",
97 )
98 .into());
99 }
100 transaction.pragma_update(None, "user_version", VERSION)?;
101 transaction.commit()?;
102 Ok(())
103 })();
104 connection.execute_batch("PRAGMA foreign_keys=ON")?;
105 upgraded
106}
crates/notebook/src/session.rs+263-280
...@@ -2,11 +2,11 @@...@@ -2,11 +2,11 @@
2//! publishes their edits to the section file in the background.2//! publishes their edits to the section file in the background.
33
4use crate::{4use crate::{
5 Conflict, ConflictKind, EditStatus, Error, PendingEdit, Remote, Replica, Resolution, Result,5 EditStatus, Error, PendingEdit, Remote, Replica, Resolution, Result,
6 SyncWorker, discover,6 SyncWorker, discover,
7};7};
8use onestore::{8use onestore::{
9 CommitError, ExGuid, PageCreation, PreparedEdit, RevisionIndex, Stamp, Store, Transaction,9 CommitError, ExGuid, PageCreation, RevisionIndex, Stamp, Store, Transaction,
10 document::Document,10 document::Document,
11 op::{Edit, Op, SectionOp},11 op::{Edit, Op, SectionOp},
12 page::Page,12 page::Page,
...@@ -18,7 +18,6 @@ use std::{...@@ -18,7 +18,6 @@ use std::{
18 path::{Path, PathBuf},18 path::{Path, PathBuf},
19 sync::{19 sync::{
20 Arc, Mutex,20 Arc, Mutex,
21 atomic::{AtomicUsize, Ordering},
22 mpsc::{self, Receiver, Sender},21 mpsc::{self, Receiver, Sender},
23 },22 },
24 time::Duration,23 time::Duration,
...@@ -81,25 +80,19 @@ pub trait Storage: Send + Sync {...@@ -81,25 +80,19 @@ pub trait Storage: Send + Sync {
81 fn create_directory(&self, path: &str) -> Result<()>;80 fn create_directory(&self, path: &str) -> Result<()>;
82 /// Renames or moves a file or directory; an existing target is an error.81 /// Renames or moves a file or directory; an existing target is an error.
83 fn rename(&self, from: &str, to: &str) -> Result<()>;82 fn rename(&self, from: &str, to: &str) -> Result<()>;
83 /// Deletes a file or an empty directory.
84 fn delete(&self, path: &str) -> Result<()>;
84 /// Names a section or TOC file for its notebook, as `onestore::place`.85 /// Names a section or TOC file for its notebook, as `onestore::place`.
85 fn place(&self, path: &str, ancestor: [u8; 16], name: &str) -> Result<()>;86 fn place(&self, path: &str, ancestor: [u8; 16], name: &str) -> Result<()>;
86 fn commit(&self, path: &str, edit: &PreparedEdit<'_>) -> Result<()>;87 /// Publishes a transaction made on the file's current image.
87 fn set_property(88 fn commit(&self, path: &str, transaction: &Transaction) -> Result<()>;
88 &self,
89 path: &str,
90 source: &[u8],
91 space: ExGuid,
92 object: ExGuid,
93 property: u32,
94 value: &[u8],
95 ) -> Result<()>;
96}89}
9790
98/// A password-protected section as `Notebook::unlock` read it.91/// A password-protected section as `Notebook::unlock` read it.
99#[cfg(feature = "protected")]92#[cfg(feature = "protected")]
100pub struct Unlocked {93pub struct Unlocked {
101 pub pages: Vec<(ExGuid, Page)>,94 pub pages: Vec<(ExGuid, Page)>,
102 /// The stored section the pages came from, which a save must still find in place.95 /// The stored section the pages came from, which an edit must still find in place.
103 snapshot: Vec<u8>,96 snapshot: Vec<u8>,
104}97}
10598
...@@ -145,31 +138,21 @@ impl Storage for Directory {...@@ -145,31 +138,21 @@ impl Storage for Directory {
145 Ok(std::fs::rename(self.path(from), self.path(to))?)138 Ok(std::fs::rename(self.path(from), self.path(to))?)
146 }139 }
147140
141 fn delete(&self, path: &str) -> Result<()> {
142 let path = self.path(path);
143 Ok(if path.is_dir() {
144 std::fs::remove_dir(path)?
145 } else {
146 std::fs::remove_file(path)?
147 })
148 }
149
148 fn place(&self, path: &str, ancestor: [u8; 16], name: &str) -> Result<()> {150 fn place(&self, path: &str, ancestor: [u8; 16], name: &str) -> Result<()> {
149 Ok(onestore::place_file(self.path(path), ancestor, name)?)151 Ok(onestore::place_file(self.path(path), ancestor, name)?)
150 }152 }
151153
152 fn commit(&self, path: &str, edit: &PreparedEdit<'_>) -> Result<()> {154 fn commit(&self, path: &str, transaction: &Transaction) -> Result<()> {
153 Ok(edit.commit_file(self.path(path))?)155 Ok(transaction.commit_file(self.path(path))?)
154 }
155
156 fn set_property(
157 &self,
158 path: &str,
159 source: &[u8],
160 space: ExGuid,
161 object: ExGuid,
162 property: u32,
163 value: &[u8],
164 ) -> Result<()> {
165 Ok(onestore::commit_file_property(
166 self.path(path),
167 source,
168 space,
169 object,
170 property,
171 value,
172 )?)
173 }156 }
174}157}
175158
...@@ -225,33 +208,18 @@ impl Storage for Share {...@@ -225,33 +208,18 @@ impl Storage for Share {
225 Ok(self.client.rename(&self.path(from), &self.path(to))?)208 Ok(self.client.rename(&self.path(from), &self.path(to))?)
226 }209 }
227210
211 fn delete(&self, path: &str) -> Result<()> {
212 Ok(self.client.delete(&self.path(path))?)
213 }
214
228 fn place(&self, path: &str, ancestor: [u8; 16], name: &str) -> Result<()> {215 fn place(&self, path: &str, ancestor: [u8; 16], name: &str) -> Result<()> {
229 Ok(self.client.place(&self.path(path), ancestor, name)?)216 Ok(self.client.place(&self.path(path), ancestor, name)?)
230 }217 }
231218
232 fn commit(&self, path: &str, edit: &PreparedEdit<'_>) -> Result<()> {219 fn commit(&self, path: &str, transaction: &Transaction) -> Result<()> {
233 Ok(self220 Ok(self
234 .client221 .client
235 .commit_transaction(&self.path(path), &edit.transaction())?)222 .commit_transaction(&self.path(path), transaction)?)
236 }
237
238 fn set_property(
239 &self,
240 path: &str,
241 source: &[u8],
242 space: ExGuid,
243 object: ExGuid,
244 property: u32,
245 value: &[u8],
246 ) -> Result<()> {
247 Ok(self.client.commit_property_bytes(
248 &self.path(path),
249 source,
250 space,
251 object,
252 property,
253 value,
254 )?)
255 }223 }
256}224}
257225
...@@ -279,6 +247,22 @@ impl Notebook {...@@ -279,6 +247,22 @@ impl Notebook {
279 Self::with(Box::new(Directory(root.clone())), Some(root), cache)247 Self::with(Box::new(Directory(root.clone())), Some(root), cache)
280 }248 }
281249
250 /// Creates a notebook in the new folder `root`, as OneNote 2010 creates one: a table of
251 /// contents in the notebook colour `color` (COLORREF) and "New Section 1" holding the
252 /// page `page` creates.
253 pub fn create(
254 root: impl AsRef<Path>,
255 cache: impl AsRef<Path>,
256 color: u32,
257 page: &PageCreation,
258 ) -> Result<Self> {
259 std::fs::create_dir(root.as_ref())?;
260 let mut notebook = Self::open(root, cache)?;
261 notebook.edit_toc("", &[onestore::TocEdit::Color(color)])?;
262 notebook.create_section("", "New Section 1", page)?;
263 Ok(notebook)
264 }
265
282 /// Opens the notebook at `root` on the share `client` is connected to. Sections open266 /// Opens the notebook at `root` on the share `client` is connected to. Sections open
283 /// through `Section::resume_smb` with the catalog's paths.267 /// through `Section::resume_smb` with the catalog's paths.
284 #[cfg(feature = "smb")]268 #[cfg(feature = "smb")]
...@@ -393,18 +377,38 @@ impl Notebook {...@@ -393,18 +377,38 @@ impl Notebook {
393377
394 fn edit_toc(&self, folder: &str, edits: &[onestore::TocEdit]) -> Result<()> {378 fn edit_toc(&self, folder: &str, edits: &[onestore::TocEdit]) -> Result<()> {
395 let (toc, _) = self.toc(folder)?;379 let (toc, _) = self.toc(folder)?;
396 let source = self.storage.read(&toc)?;380 self.commit_toc(&toc, edits)
397 self.storage
398 .commit(&toc, &PreparedEdit::table_of_contents(&source, edits)?)
399 }381 }
400382
401 /// Creates `name.one` in `folder` with one empty page and lists it last in the folder's383 fn commit_toc(&self, toc: &str, edits: &[onestore::TocEdit]) -> Result<()> {
402 /// TOC, as OneNote creates a section. Returns the new catalog path.384 let source = self.storage.read(toc)?;
403 pub fn create_section(&mut self, folder: &str, name: &str, author: &str) -> Result<String> {385 match onestore::edit_table_of_contents(&source, edits)? {
386 Some(transaction) => self.storage.commit(toc, &transaction),
387 None => Ok(()),
388 }
389 }
390
391 /// Creates `name.one` in `folder` holding the page `page` creates, in the next of
392 /// OneNote's section colours, and lists it last in the folder's TOC, as OneNote creates
393 /// a section. Returns the new catalog path.
394 pub fn create_section(&mut self, folder: &str, name: &str, page: &PageCreation) -> Result<String> {
404 let filename = format!("{name}.one");395 let filename = format!("{name}.one");
405 self.folder(folder)?;396 let color = next_color(self.folder(folder)?);
406 let (_, ancestor) = self.toc(folder)?;397 let (_, ancestor) = self.toc(folder)?;
407 let bytes = onestore::create_section(&filename, "", author)?;398 let mut bytes = onestore::create_empty_section(&filename, Some(color))?;
399 let transaction = {
400 let arena = onestore::Arena::default();
401 let mut section = onestore::Section::open(&arena, bytes.clone())?;
402 let edit = Edit {
403 at: crate::now(),
404 ops: vec![Op::Section(SectionOp::Create(page.clone()))],
405 };
406 section.apply(page.author(), &edit)?;
407 section.seal()?
408 };
409 if let Some(transaction) = transaction {
410 transaction.apply(&mut bytes)?;
411 }
408 let path = catalog_path(folder, &filename);412 let path = catalog_path(folder, &filename);
409 self.storage.create(&path, &bytes)?;413 self.storage.create(&path, &bytes)?;
410 self.storage.place(&path, ancestor, &filename)?;414 self.storage.place(&path, ancestor, &filename)?;
...@@ -483,23 +487,16 @@ impl Notebook {...@@ -483,23 +487,16 @@ impl Notebook {
483 /// Sets a section's colour (COLORREF) in its own metadata, where OneNote keeps it.487 /// Sets a section's colour (COLORREF) in its own metadata, where OneNote keeps it.
484 pub fn set_section_color(&mut self, path: &str, color: Option<u32>) -> Result<()> {488 pub fn set_section_color(&mut self, path: &str, color: Option<u32>) -> Result<()> {
485 let path = self.section_path(path)?.path.clone();489 let path = self.section_path(path)?.path.clone();
486 let source = self.storage.read(&path)?;490 let arena = onestore::Arena::default();
487 let store = onestore::Store::parse(&source)?;491 let mut section = onestore::Section::open(&arena, self.storage.read(&path)?)?;
488 let index = onestore::RevisionIndex::parse(&store)?;492 let edit = Edit {
489 let document = Document::parse(&index)?;493 at: crate::now(),
490 let revision = document.active(document.root)?;494 ops: vec![Op::Section(SectionOp::Color(color))],
491 let metadata = *revision495 };
492 .roots496 section.apply("", &edit)?;
493 .get(&2)497 if let Some(transaction) = section.seal()? {
494 .ok_or_else(|| io::Error::from(io::ErrorKind::InvalidData))?;498 self.storage.commit(&path, &transaction)?;
495 self.storage.set_property(499 }
496 &path,
497 &source,
498 document.root,
499 metadata,
500 0x14001cbe,
501 &color.unwrap_or(0xffff_ffff).to_le_bytes(),
502 )?;
503 self.refresh()?;500 self.refresh()?;
504 Ok(())501 Ok(())
505 }502 }
...@@ -518,15 +515,51 @@ impl Notebook {...@@ -518,15 +515,51 @@ impl Notebook {
518 self.refresh().map(drop)515 self.refresh().map(drop)
519 }516 }
520517
521 /// Deletes a section the way OneNote does: the file moves into the notebook's518 /// Deletes a section or section group the way OneNote 2010 does. A section's file moves
522 /// `OneNote_RecycleBin` folder, a section group with its own TOC that lists it (and that519 /// into the notebook's `OneNote_RecycleBin` folder, a section group with its own TOC that
523 /// the root TOC lists), and its own folder's TOC entry goes.520 /// lists it (and that the root TOC lists). A group's sections, in its groups too, move
521 /// there the same way, then its folders go. Either way its folder's TOC entry goes.
524 pub fn delete(&mut self, path: &str) -> Result<()> {522 pub fn delete(&mut self, path: &str) -> Result<()> {
525 let (folder, entry) = split(path);523 let (folder, entry) = split(path);
526 let (filename, identity) = self.entry(folder, entry)?;524 let (filename, identity) = self.entry(folder, entry)?;
527 if !filename.to_ascii_lowercase().ends_with(".one") {525 if path == RECYCLE_BIN {
528 return Err(io::Error::from(io::ErrorKind::InvalidInput).into());526 return Err(io::Error::from(io::ErrorKind::InvalidInput).into());
529 }527 }
528 if filename.to_ascii_lowercase().ends_with(".one") {
529 self.bin_section(path, &filename, identity)?;
530 } else {
531 // Sections first, then each folder once it is empty, deepest first.
532 let mut sections = Vec::new();
533 let mut folders = Vec::new();
534 let mut pending = vec![self.folder(path)?];
535 while let Some(group) = pending.pop() {
536 sections.extend(
537 group
538 .sections
539 .iter()
540 .map(|section| (section.path.clone(), section.file_id)),
541 );
542 folders.push((group.path.clone(), group.toc.as_ref().map(|toc| toc.filename.clone())));
543 pending.extend(&group.groups);
544 }
545 for (section, identity) in sections {
546 let (_, name) = split(&section);
547 self.bin_section(&section, name, identity)?;
548 }
549 for (group, toc) in folders.into_iter().rev() {
550 if let Some(toc) = toc {
551 self.storage.delete(&catalog_path(&group, &toc))?;
552 }
553 self.storage.delete(&group)?;
554 }
555 }
556 self.edit_toc(folder, &[onestore::TocEdit::Remove { identity }])?;
557 self.refresh().map(drop)
558 }
559
560 /// The recycle bin's TOC path and file identity, creating the bin as OneNote does: a
561 /// section group the root TOC lists.
562 fn bin(&self) -> Result<(String, [u8; 16])> {
530 let bin_toc = catalog_path(RECYCLE_BIN, TOC);563 let bin_toc = catalog_path(RECYCLE_BIN, TOC);
531 if !self.storage.exists(&bin_toc) {564 if !self.storage.exists(&bin_toc) {
532 if !self.storage.exists(RECYCLE_BIN) {565 if !self.storage.exists(RECYCLE_BIN) {
...@@ -545,37 +578,116 @@ impl Notebook {...@@ -545,37 +578,116 @@ impl Notebook {
545 }],578 }],
546 )?;579 )?;
547 }580 }
548 let mut target = filename.clone();581 let source = self.storage.read(&bin_toc)?;
582 Ok((bin_toc, onestore::Store::parse(&source)?.header.file_id))
583 }
584
585 /// Moves the section file at `path` into the recycle bin, under a name no binned
586 /// section has, and lists it there; its own folder's TOC is the caller's.
587 fn bin_section(&self, path: &str, filename: &str, identity: [u8; 16]) -> Result<()> {
588 let (bin_toc, bin_identity) = self.bin()?;
589 let mut target = filename.to_owned();
549 let mut attempt = 1;590 let mut attempt = 1;
550 while self.storage.exists(&catalog_path(RECYCLE_BIN, &target)) {591 while self.storage.exists(&catalog_path(RECYCLE_BIN, &target)) {
551 attempt += 1;592 attempt += 1;
552 let (stem, extension) = filename.rsplit_once('.').unwrap_or((&filename, ""));593 let (stem, extension) = filename.rsplit_once('.').unwrap_or((filename, ""));
553 target = format!("{stem} ({attempt}).{extension}");594 target = format!("{stem} ({attempt}).{extension}");
554 }595 }
555 let binned = catalog_path(RECYCLE_BIN, &target);596 let binned = catalog_path(RECYCLE_BIN, &target);
556 self.storage597 self.storage.rename(path, &binned)?;
557 .rename(&catalog_path(folder, &filename), &binned)?;598 self.storage.place(&binned, bin_identity, &target)?;
558 let source = self.storage.read(&bin_toc)?;599 self.commit_toc(
559 self.storage.place(
560 &binned,
561 onestore::Store::parse(&source)?.header.file_id,
562 &target,
563 )?;
564 self.storage.commit(
565 &bin_toc,600 &bin_toc,
566 &PreparedEdit::table_of_contents(601 &[onestore::TocEdit::Add {
567 &source,602 filename: target,
603 identity,
604 group: false,
605 }],
606 )
607 }
608
609 /// Keeps copies of `pages` in the notebook's recycle bin, as OneNote 2010 does with a
610 /// page it deletes: in `OneNote_RecycleBin/OneNote_DeletedPages.one`, each at the top
611 /// level with its identity, title, date and creation time. The caller then deletes the
612 /// pages from their section.
613 pub fn recycle_pages(&mut self, pages: &[Page], author: &str) -> Result<()> {
614 const DELETED: &str = "OneNote_DeletedPages.one";
615 let (bin_toc, bin_identity) = self.bin()?;
616 let path = catalog_path(RECYCLE_BIN, DELETED);
617 if !self.storage.exists(&path) {
618 // OneNote's "Deleted Pages" section is grey.
619 let bytes = onestore::create_empty_section(DELETED, Some(0x00e1e1e1))?;
620 self.storage.create(&path, &bytes)?;
621 self.storage.place(&path, bin_identity, DELETED)?;
622 self.commit_toc(
623 &bin_toc,
568 &[onestore::TocEdit::Add {624 &[onestore::TocEdit::Add {
569 filename: target,625 filename: DELETED.into(),
570 identity,626 identity: onestore::Store::parse(&bytes)?.header.file_id,
571 group: false,627 group: false,
572 }],628 }],
573 )?,629 )?;
574 )?;630 }
575 self.edit_toc(folder, &[onestore::TocEdit::Remove { identity }])?;631 let arena = onestore::Arena::default();
632 let mut section = onestore::Section::open(&arena, self.storage.read(&path)?)?;
633 let mut ops = Vec::new();
634 for page in pages {
635 let mut creation = PageCreation::new(None, Some(&page.title), author)?;
636 if let (Some(identity), Some(created)) = (page.identity, page.created) {
637 creation = creation.keeping(identity, created)?;
638 }
639 if let Some([date, time]) = page.date_text() {
640 creation = creation.dated(&date, &time)?;
641 }
642 ops.push(Op::Section(SectionOp::Import {
643 creation,
644 page: page.copy()?,
645 }));
646 }
647 section.apply(author, &Edit { at: crate::now(), ops })?;
648 if let Some(transaction) = section.seal()? {
649 self.storage.commit(&path, &transaction)?;
650 }
576 self.refresh().map(drop)651 self.refresh().map(drop)
577 }652 }
578653
654 /// Moves a section or section group into the folder at catalog path `folder`, last, as
655 /// OneNote moves one dragged onto a group: the file or folder moves and each TOC's
656 /// entry follows it. Returns its new catalog path.
657 pub fn move_entry(&mut self, path: &str, folder: &str) -> Result<String> {
658 let (from, entry) = split(path);
659 let (filename, identity) = self.entry(from, entry)?;
660 self.folder(folder)?;
661 let group = !filename.to_ascii_lowercase().ends_with(".one");
662 let target = catalog_path(folder, &filename);
663 if from == folder
664 || path == RECYCLE_BIN
665 || group && (folder == path || folder.starts_with(&format!("{path}/")))
666 || self.storage.exists(&target)
667 {
668 return Err(io::Error::from(io::ErrorKind::InvalidInput).into());
669 }
670 let (_, ancestor) = self.toc(folder)?;
671 self.storage.rename(path, &target)?;
672 let placed = if group {
673 catalog_path(&target, TOC)
674 } else {
675 target.clone()
676 };
677 self.storage.place(&placed, ancestor, &filename)?;
678 self.edit_toc(
679 folder,
680 &[onestore::TocEdit::Add {
681 filename,
682 identity,
683 group,
684 }],
685 )?;
686 self.edit_toc(from, &[onestore::TocEdit::Remove { identity }])?;
687 self.refresh()?;
688 Ok(target)
689 }
690
579 /// The stored filename and TOC identity of a folder's section or group.691 /// The stored filename and TOC identity of a folder's section or group.
580 fn entry(&self, folder: &str, name: &str) -> Result<(String, [u8; 16])> {692 fn entry(&self, folder: &str, name: &str) -> Result<(String, [u8; 16])> {
581 let parent = self.folder(folder)?;693 let parent = self.folder(folder)?;
...@@ -660,30 +772,31 @@ impl Notebook {...@@ -660,30 +772,31 @@ impl Notebook {
660 Ok(Unlocked { pages, snapshot })772 Ok(Unlocked { pages, snapshot })
661 }773 }
662774
663 /// Saves an edited page of `unlocked` under the section's key, straight to the file:775 /// Applies `edit` to pages of `unlocked` and stores it under the section's key, straight
664 /// nothing of a protected section is cached or queued. A section written since776 /// to the file: nothing of a protected section is cached or queued. A section written
665 /// `unlocked` was read fails the save; unlock again for its pages.777 /// since `unlocked` was read refuses the edit; unlock again for its pages.
666 #[cfg(feature = "protected")]778 #[cfg(feature = "protected")]
667 pub fn save_unlocked(779 pub fn apply_unlocked(
668 &self,780 &self,
669 path: &str,781 path: &str,
670 password: &str,782 password: &str,
671 unlocked: &mut Unlocked,783 unlocked: &mut Unlocked,
672 space: ExGuid,
673 page: &Page,
674 author: &str,784 author: &str,
785 edit: &Edit,
675 ) -> Result<()> {786 ) -> Result<()> {
676 let path = self.section_path(path)?.path.clone();787 let path = self.section_path(path)?.path.clone();
677 let edit = onestore::PreparedEdit::page_protected(788 let transaction = {
678 &unlocked.snapshot,789 let store = Store::parse(&unlocked.snapshot)?;
679 password,790 let index = RevisionIndex::parse(&store)?;
680 space,791 onestore::protected::UnlockedSection::open(
681 page,792 &index,
682 author,793 password,
683 )?;794 onestore::protected::Limits::default(),
684 self.storage.commit(&path, &edit)?;795 )?
685 let written = edit.as_bytes().to_vec();796 .apply(author, edit)?
686 unlocked.snapshot = written;797 };
798 self.storage.commit(&path, &transaction)?;
799 transaction.apply(&mut unlocked.snapshot)?;
687 Ok(())800 Ok(())
688 }801 }
689802
...@@ -706,6 +819,27 @@ impl Notebook {...@@ -706,6 +819,27 @@ impl Notebook {
706 }819 }
707}820}
708821
822/// The section colours OneNote 2010 gives new sections, COLORREF, in the order it gives
823/// them (`evidence/notebook-management/new-notebook`).
824const SECTION_COLORS: [u32; 3] = [0x00e4a88a, 0x0078b0f6, 0x00bba4d5];
825
826/// The colour OneNote gives a new section in `folder`: the first of its colours none of
827/// the folder's sections has, or the next in turn when all are taken.
828fn next_color(folder: &discover::Folder) -> u32 {
829 let taken: Vec<u32> = folder
830 .sections
831 .iter()
832 .filter_map(|section| match section.state {
833 discover::SectionState::Readable { color, .. } => color,
834 _ => None,
835 })
836 .collect();
837 SECTION_COLORS
838 .into_iter()
839 .find(|color| !taken.contains(color))
840 .unwrap_or(SECTION_COLORS[folder.sections.len() % SECTION_COLORS.len()])
841}
842
709fn component(name: &str) -> bool {843fn component(name: &str) -> bool {
710 !name.is_empty() && !name.contains(['/', '\\', '\0']) && name != "." && name != ".."844 !name.is_empty() && !name.contains(['/', '\\', '\0']) && name != "." && name != ".."
711}845}
...@@ -733,8 +867,6 @@ pub enum Event {...@@ -733,8 +867,6 @@ pub enum Event {
733 /// The section refused an edit `apply` handed it; the pages it names are as they were867 /// The section refused an edit `apply` handed it; the pages it names are as they were
734 /// before it, so an editor showing them should reload them.868 /// before it, so an editor showing them should reload them.
735 Rejected { spaces: Vec<ExGuid>, error: String },869 Rejected { spaces: Vec<ExGuid>, error: String },
736 /// Old session API, until the application reloads on `Changed`: pages changed remotely.
737 Refreshed,
738 /// A publication attempt finished with this durable state.870 /// A publication attempt finished with this durable state.
739 Attempt { id: u64, status: EditStatus },871 Attempt { id: u64, status: EditStatus },
740 /// The section file could not be reached; the replica keeps its state.872 /// The section file could not be reached; the replica keeps its state.
...@@ -743,25 +875,6 @@ pub enum Event {...@@ -743,25 +875,6 @@ pub enum Event {
743 Failed(String),875 Failed(String),
744}876}
745877
746/// The outcome of saving an edited page (old session API).
747#[derive(Debug, PartialEq, Eq)]
748pub enum Save {
749 /// The model equals the stored page.
750 Unchanged,
751 /// The edit is durable locally under this id.
752 Queued(u64),
753 /// The stored page no longer matches `before`; reload it before saving again.
754 Stale,
755}
756
757/// A queued edit and its durable state.
758#[derive(Debug, Clone, PartialEq, Eq)]
759pub struct QueuedEdit {
760 pub id: u64,
761 pub space: ExGuid,
762 pub status: EditStatus,
763}
764
765type Notify = Arc<dyn Fn() + Send + Sync>;878type Notify = Arc<dyn Fn() + Send + Sync>;
766879
767/// A section file with its replica and background publication. `Send + Sync`: edits880/// A section file with its replica and background publication. `Send + Sync`: edits
...@@ -773,10 +886,6 @@ pub struct Section {...@@ -773,10 +886,6 @@ pub struct Section {
773 events: Mutex<Receiver<Event>>,886 events: Mutex<Receiver<Event>>,
774 sender: Sender<Event>,887 sender: Sender<Event>,
775 notify: Notify,888 notify: Notify,
776 saved: Mutex<Receiver<(ExGuid, std::result::Result<Save, String>)>>,
777 saves: Sender<(ExGuid, std::result::Result<Save, String>)>,
778 /// Saves queued and not yet reported.
779 unsaved: Arc<AtomicUsize>,
780}889}
781890
782impl Section {891impl Section {
...@@ -850,7 +959,6 @@ impl Section {...@@ -850,7 +959,6 @@ impl Section {
850 ) -> Result<Self> {959 ) -> Result<Self> {
851 let replica = Arc::new(replica);960 let replica = Arc::new(replica);
852 let (sender, events) = mpsc::channel();961 let (sender, events) = mpsc::channel();
853 let (saves, saved) = mpsc::channel();
854 let notify = Mutex::new(notify);962 let notify = Mutex::new(notify);
855 let notify: Notify = Arc::new(move || {963 let notify: Notify = Arc::new(move || {
856 if let Ok(notify) = notify.lock() {964 if let Ok(notify) = notify.lock() {
...@@ -865,7 +973,6 @@ impl Section {...@@ -865,7 +973,6 @@ impl Section {
865 Ok(synced) => {973 Ok(synced) => {
866 if !synced.changed.is_empty() {974 if !synced.changed.is_empty() {
867 events.push(Event::Changed(synced.changed.clone()));975 events.push(Event::Changed(synced.changed.clone()));
868 events.push(Event::Refreshed);
869 }976 }
870 if let Some((id, status)) = &synced.edit {977 if let Some((id, status)) = &synced.edit {
871 events.push(Event::Attempt {978 events.push(Event::Attempt {
...@@ -899,9 +1006,6 @@ impl Section {...@@ -899,9 +1006,6 @@ impl Section {
899 events: Mutex::new(events),1006 events: Mutex::new(events),
900 sender,1007 sender,
901 notify,1008 notify,
902 saved: Mutex::new(saved),
903 saves,
904 unsaved: Arc::new(AtomicUsize::new(0)),
905 })1009 })
906 }1010 }
9071011
...@@ -971,8 +1075,8 @@ impl Section {...@@ -971,8 +1075,8 @@ impl Section {
971 Ok(space)1075 Ok(space)
972 }1076 }
9731077
974 /// Removes pages permanently, queued like the user's own edits (a move across1078 /// Removes pages or conflict pages permanently, queued like the user's own edits (a
975 /// sections is `import_page` there, then this here).1079 /// move across sections is `import_page` there, then this here).
976 pub fn delete_pages(&self, pages: &[ExGuid]) -> Result<u64> {1080 pub fn delete_pages(&self, pages: &[ExGuid]) -> Result<u64> {
977 self.replica.apply(1081 self.replica.apply(
978 "",1082 "",
...@@ -991,23 +1095,10 @@ impl Section {...@@ -991,23 +1095,10 @@ impl Section {
991 self.replica.pending()1095 self.replica.pending()
992 }1096 }
9931097
994 /// The unpublished batch the remote no longer accepts, if any. Show `remote_page`1098 /// The conflict pages of each page that has them, as the local edits leave them
995 /// against `page` for its space, then `resolve`.1099 /// (`onestore::Section::conflicts`); `page` reads one, `delete_pages` removes it.
996 pub fn conflict(&self) -> Result<Option<Conflict>> {1100 pub fn conflicts(&self) -> Result<Vec<(ExGuid, Vec<onestore::ConflictPage>)>> {
997 self.replica.conflict()1101 self.replica.conflicts()
998 }
999
1000 /// The page as last observed in the section file, for reviewing a conflict.
1001 pub fn remote_page(&self, space: ExGuid) -> Result<Page> {
1002 let snapshot = self.replica.remote_snapshot()?;
1003 let arena = onestore::Arena::default();
1004 Ok(onestore::Section::open(&arena, snapshot)?.page(space)?)
1005 }
1006
1007 /// Ends a conflict: `Mine` publishes the local page over the remote change, `Theirs`
1008 /// drops the local edits to the conflicted page (and its later ones).
1009 pub fn resolve(&self, id: u64, resolution: Resolution) -> Result<()> {
1010 self.replica.resolve(id, resolution)
1011 }1102 }
10121103
1013 /// Retires an uncertain attempt after review, exporting the queue to `archive` first:1104 /// Retires an uncertain attempt after review, exporting the queue to `archive` first:
...@@ -1056,114 +1147,6 @@ impl Section {...@@ -1056,114 +1147,6 @@ impl Section {
1056 None => Ok(()),1147 None => Ok(()),
1057 }1148 }
1058 }1149 }
1059
1060 // Old whole-page session API, kept until the application emits ops.
1061
1062 /// Saves an edited page. `before` is the model the edit started from; a stored page
1063 /// that differs from it means the section changed underneath the editor.
1064 pub fn save(&self, space: ExGuid, before: &Page, after: &Page, author: &str) -> Result<Save> {
1065 let outcome = self.replica.ask(|reply| crate::working::Request::Save {
1066 space,
1067 before: before.clone(),
1068 after: after.clone(),
1069 author: author.to_owned(),
1070 reply,
1071 })?;
1072 Ok(outcome.unwrap_or(Save::Unchanged))
1073 }
1074
1075 /// `save` on the section thread, returning at once; `saved` reports the outcome.
1076 /// Saves queue in order, and one whose `before` is the previous save's `after`
1077 /// continues it: pass each save's `after` as the next one's `before`.
1078 pub fn queue_save(&self, space: ExGuid, before: Page, after: Page, author: &str) -> Result<()> {
1079 let (saves, notify, unsaved) = (
1080 self.saves.clone(),
1081 Arc::clone(&self.notify),
1082 Arc::clone(&self.unsaved),
1083 );
1084 self.unsaved.fetch_add(1, Ordering::Relaxed);
1085 self.replica.send(crate::working::Request::Save {
1086 space,
1087 before,
1088 after,
1089 author: author.to_owned(),
1090 reply: Box::new(move |outcome| {
1091 unsaved.fetch_sub(1, Ordering::Release);
1092 let outcome = match outcome {
1093 Ok(None) => return,
1094 Ok(Some(save)) => Ok(save),
1095 Err(error) => Err(error.to_string()),
1096 };
1097 if saves.send((space, outcome)).is_ok() {
1098 notify();
1099 }
1100 }),
1101 })
1102 }
1103
1104 /// Whether a queued save has yet to report, so the stored page may trail the editor's.
1105 pub fn saving(&self) -> bool {
1106 self.unsaved.load(Ordering::Acquire) > 0
1107 }
1108
1109 /// Outcomes of queued saves since the last poll, oldest first; `notify` runs for each.
1110 /// Consecutive saves continuing one another finish as one, reported once.
1111 pub fn saved(&self) -> Vec<(ExGuid, std::result::Result<Save, String>)> {
1112 self.saved
1113 .lock()
1114 .map(|saved| saved.try_iter().collect())
1115 .unwrap_or_default()
1116 }
1117
1118 /// Every queued edit with its state: pending, awaiting confirmation of a retained
1119 /// attempt, or a conflict awaiting review. `space` is the first page it edits.
1120 pub fn queue(&self) -> Result<Vec<QueuedEdit>> {
1121 self.replica
1122 .pending()?
1123 .into_iter()
1124 .map(|edit| {
1125 Ok(QueuedEdit {
1126 id: edit.id,
1127 space: edit
1128 .edit
1129 .ops
1130 .iter()
1131 .find_map(|op| match op {
1132 Op::Page { space, .. } => Some(*space),
1133 Op::Section(_) => None,
1134 })
1135 .unwrap_or(self.replica.root),
1136 status: self.replica.status(edit.id)?.unwrap_or(EditStatus::Pending),
1137 })
1138 })
1139 .collect()
1140 }
1141
1142 /// The conflict as a queue entry, for the review dialog.
1143 pub fn conflicts(&self) -> Result<Vec<(QueuedEdit, ConflictKind)>> {
1144 Ok(self
1145 .replica
1146 .conflict()?
1147 .map(|conflict| {
1148 (
1149 QueuedEdit {
1150 id: conflict.id,
1151 space: conflict.space,
1152 status: EditStatus::Conflict(conflict.kind),
1153 },
1154 conflict.kind,
1155 )
1156 })
1157 .into_iter()
1158 .collect())
1159 }
1160
1161 /// Resolves the conflict holding `id` with a page reviewed against `remote_page`: the
1162 /// remote page is rewritten to `after`.
1163 pub fn review(&self, id: u64, after: &Page) -> Result<()> {
1164 self.replica
1165 .resolve_with(id, Resolution::Mine, Some(after.clone()))
1166 }
1167}1150}
11681151
1169/// The section file itself as the publication target, under OneNote-compatible exclusion.1152/// The section file itself as the publication target, under OneNote-compatible exclusion.
...@@ -1176,21 +1159,21 @@ impl Remote for FileRemote {...@@ -1176,21 +1159,21 @@ impl Remote for FileRemote {
11761159
1177 /// Read without the whole-file lock, which would block OneNote's readers: a change1160 /// Read without the whole-file lock, which would block OneNote's readers: a change
1178 /// detector, never a snapshot to edit.1161 /// detector, never a snapshot to edit.
1179 fn stamp(&mut self) -> io::Result<Option<Stamp>> {1162 fn stamp(&mut self) -> io::Result<Stamp> {
1180 use std::io::Read;1163 use std::io::Read;
1181 let mut file = std::fs::File::open(&self.0)?;1164 let mut file = std::fs::File::open(&self.0)?;
1182 let mut header = [0; 1024];1165 let mut header = [0; 1024];
1183 file.read_exact(&mut header)?;1166 file.read_exact(&mut header)?;
1184 let length = file.metadata()?.len();1167 let length = file.metadata()?.len();
1185 Ok(Some(Stamp { header, length }))1168 Ok(Stamp { header, length })
1186 }1169 }
11871170
1188 fn publish(&mut self, transaction: &Transaction) -> std::result::Result<(), CommitError> {1171 fn publish(&mut self, transaction: &Transaction) -> std::result::Result<(), CommitError> {
1189 transaction.commit_file(&self.0)1172 transaction.commit_file(&self.0)
1190 }1173 }
11911174
1192 fn confirm(&mut self, snapshot: &[u8]) -> std::result::Result<(), CommitError> {1175 fn confirm(&mut self, base: &Stamp) -> std::result::Result<(), CommitError> {
1193 onestore::confirm_file_snapshot(&self.0, snapshot)1176 onestore::confirm_file(&self.0, base)
1194 }1177 }
1195}1178}
11961179
crates/notebook/src/smb/directory.rs+1-5
...@@ -35,11 +35,7 @@ impl Client {...@@ -35,11 +35,7 @@ impl Client {
35 create_contexts: Vec::new(),35 create_contexts: Vec::new(),
36 },36 },
37 )?;37 )?;
38 let file = File {38 let file = File::new(self, &response);
39 client: self,
40 id: Some(response.file_id),
41 length: response.end_of_file,
42 };
43 let mut entries = BTreeMap::new();39 let mut entries = BTreeMap::new();
44 loop {40 loop {
45 let response: io::Result<QueryDirectoryResponse> =41 let response: io::Result<QueryDirectoryResponse> =
crates/notebook/src/smb/mod.rs+335-150
...@@ -1,9 +1,9 @@...@@ -1,9 +1,9 @@
1//! Blocking SMB access to OneNote sections, table-of-contents files and payloads.1//! Blocking SMB access to OneNote sections, table-of-contents files and payloads.
22
3use onestore::{CommitError, CommitIo, CommitState, ExGuid};3use onestore::{CommitError, CommitIo, CommitState};
4use smb2::{4use smb2::{
5 Session, Tree,5 Session, Tree,
6 client::connection::{Connection, NegotiatedParams},6 client::connection::{CompoundOp, Connection, Frame, NegotiatedParams},
7 msg::{7 msg::{
8 close::{CloseRequest, CloseResponse},8 close::{CloseRequest, CloseResponse},
9 create::{9 create::{
...@@ -22,7 +22,7 @@ use smb2::{...@@ -22,7 +22,7 @@ use smb2::{
22 flags::{Capabilities, FileAccessMask},22 flags::{Capabilities, FileAccessMask},
23 },23 },
24};24};
25use std::{io, ops::Range, sync::Mutex, time::Duration};25use std::{io, sync::Mutex, time::Duration};
26use tokio::runtime::{Handle, Runtime};26use tokio::runtime::{Handle, Runtime};
2727
28mod directory;28mod directory;
...@@ -149,6 +149,54 @@ impl Client {...@@ -149,6 +149,54 @@ impl Client {
149 .map_err(|_| io::Error::from(io::ErrorKind::TimedOut))149 .map_err(|_| io::Error::from(io::ErrorKind::TimedOut))
150 .and_then(|result| result.map_err(io::Error::other))150 .and_then(|result| result.map_err(io::Error::other))
151 .inspect_err(|_| self.retire())?;151 .inspect_err(|_| self.retire())?;
152 self.unpack(&self.body(command, frame)?)
153 }
154
155 /// Sends `requests` as one related compound request, which the server performs in order,
156 /// and returns each response body or status error. A request after a CREATE names the
157 /// file it opened as `FileId::SENTINEL`; each request fits one credit.
158 fn compound(&self, requests: &[(Command, &dyn Pack)]) -> io::Result<Vec<io::Result<Vec<u8>>>> {
159 if Handle::try_current().is_ok() {
160 return Err(io::ErrorKind::InvalidInput.into());
161 }
162 let connection = self
163 .connection
164 .lock()
165 .map_err(|_| io::ErrorKind::Other)?
166 .clone()
167 .ok_or(io::ErrorKind::NotConnected)?;
168 let operations: Vec<_> = requests
169 .iter()
170 .map(|(command, body)| CompoundOp::new(*command, *body, Some(self.tree.tree_id)))
171 .collect();
172 let frames = {
173 let runtime = self.runtime.lock().map_err(|_| io::ErrorKind::Other)?;
174 runtime
175 .as_ref()
176 .ok_or(io::ErrorKind::NotConnected)?
177 .block_on(async {
178 tokio::time::timeout(self.timeout, connection.execute_compound(&operations))
179 .await
180 })
181 };
182 let frames = frames
183 .map_err(|_| io::Error::from(io::ErrorKind::TimedOut))
184 .and_then(|result| result.map_err(io::Error::other))
185 .inspect_err(|_| self.retire())?;
186 Ok(frames
187 .into_iter()
188 .zip(requests)
189 .map(|(frame, (command, _))| {
190 let frame = frame
191 .map_err(io::Error::other)
192 .inspect_err(|_| self.retire())?;
193 self.body(*command, frame)
194 })
195 .collect())
196 }
197
198 /// The body of a response to `command`, or its status as an error.
199 fn body(&self, command: Command, frame: Frame) -> io::Result<Vec<u8>> {
152 if frame.header.command != command {200 if frame.header.command != command {
153 self.retire();201 self.retire();
154 return Err(io::ErrorKind::InvalidData.into());202 return Err(io::ErrorKind::InvalidData.into());
...@@ -170,12 +218,17 @@ impl Client {...@@ -170,12 +218,17 @@ impl Client {
170 },218 },
171 ));219 ));
172 }220 }
173 T::unpack(&mut ReadCursor::new(&frame.body)).map_err(|error| {221 Ok(frame.body)
222 }
223
224 fn unpack<T: Unpack>(&self, body: &[u8]) -> io::Result<T> {
225 T::unpack(&mut ReadCursor::new(body)).map_err(|error| {
174 self.retire();226 self.retire();
175 io::Error::new(io::ErrorKind::InvalidData, error)227 io::Error::new(io::ErrorKind::InvalidData, error)
176 })228 })
177 }229 }
178230
231 /// Opens as OneNote 2010 does: readers share everything, a writer denies other writers.
179 fn open(&self, path: &str, write: bool) -> io::Result<File<'_>> {232 fn open(&self, path: &str, write: bool) -> io::Result<File<'_>> {
180 self.open_shared(path, write, if write { 5 } else { 7 })233 self.open_shared(path, write, if write { 5 } else { 7 })
181 }234 }
...@@ -186,7 +239,7 @@ impl Client {...@@ -186,7 +239,7 @@ impl Client {
186 if write { 0xc0000000 } else { 0x80000000 },239 if write { 0xc0000000 } else { 0x80000000 },
187 sharing,240 sharing,
188 CreateDisposition::FileOpen,241 CreateDisposition::FileOpen,
189 0x42,242 0x40,
190 )243 )
191 }244 }
192245
...@@ -198,28 +251,11 @@ impl Client {...@@ -198,28 +251,11 @@ impl Client {
198 disposition: CreateDisposition,251 disposition: CreateDisposition,
199 options: u32,252 options: u32,
200 ) -> io::Result<File<'_>> {253 ) -> io::Result<File<'_>> {
201 if path.is_empty() || path.contains('\0') || path.encode_utf16().count() > 32767 {
202 return Err(io::ErrorKind::InvalidInput.into());
203 }
204 let response: CreateResponse = self.request(254 let response: CreateResponse = self.request(
205 Command::Create,255 Command::Create,
206 CreateRequest {256 create_request(path, access, sharing, disposition, options)?,
207 requested_oplock_level: OplockLevel::None,
208 impersonation_level: ImpersonationLevel::Impersonation,
209 desired_access: FileAccessMask::new(access),
210 file_attributes: 0,
211 share_access: ShareAccess(sharing),
212 create_disposition: disposition,
213 create_options: options,
214 name: smb2::encode_path(&path.replace('\\', "/")),
215 create_contexts: Vec::new(),
216 },
217 )?;257 )?;
218 Ok(File {258 Ok(File::new(self, &response))
219 client: self,
220 id: Some(response.file_id),
221 length: response.end_of_file,
222 })
223 }259 }
224260
225 /// Creates a file holding `bytes`; an existing file is an error.261 /// Creates a file holding `bytes`; an existing file is an error.
...@@ -286,7 +322,7 @@ impl Client {...@@ -286,7 +322,7 @@ impl Client {
286 /// Names a section or TOC file for its notebook (`onestore::place`) under native322 /// Names a section or TOC file for its notebook (`onestore::place`) under native
287 /// writer coordination.323 /// writer coordination.
288 pub fn place(&self, path: &str, ancestor: [u8; 16], name: &str) -> Result<(), CommitError> {324 pub fn place(&self, path: &str, ancestor: [u8; 16], name: &str) -> Result<(), CommitError> {
289 self.commit(path, |file| {325 self.commit(path, &[(0, 1024)], |file| {
290 onestore::place(file, ancestor, name).map_err(|error| CommitError {326 onestore::place(file, ancestor, name).map_err(|error| CommitError {
291 state: CommitState::NotCommitted,327 state: CommitState::NotCommitted,
292 error,328 error,
...@@ -338,7 +374,9 @@ impl Client {...@@ -338,7 +374,9 @@ impl Client {
338 path: &str,374 path: &str,
339 snapshot: impl FnOnce(&mut File<'_>) -> io::Result<Option<Vec<u8>>>,375 snapshot: impl FnOnce(&mut File<'_>) -> io::Result<Option<Vec<u8>>>,
340 ) -> io::Result<Vec<u8>> {376 ) -> io::Result<Vec<u8>> {
341 let mut file = self.open(path, false)?.coordinate(path, false)?;377 let mut file = self
378 .open(path, false)?
379 .coordinate(path, false, &[(0, 1024)])?;
342 let result = snapshot(&mut file)380 let result = snapshot(&mut file)
343 .and_then(|snapshot| snapshot.ok_or_else(|| io::ErrorKind::WouldBlock.into()));381 .and_then(|snapshot| snapshot.ok_or_else(|| io::ErrorKind::WouldBlock.into()));
344 let closed = file.close();382 let closed = file.close();
...@@ -347,79 +385,67 @@ impl Client {...@@ -347,79 +385,67 @@ impl Client {
347 Ok(snapshot)385 Ok(snapshot)
348 }386 }
349387
350 pub fn commit_text(
351 &self,
352 path: &str,
353 source: &[u8],
354 space: ExGuid,
355 object: ExGuid,
356 range: Range<u32>,
357 replacement: &str,
358 ) -> Result<(), CommitError> {
359 self.commit(path, |file| {
360 onestore::commit_text(file, source, space, object, range, replacement)
361 })
362 }
363
364 pub fn commit_property_bytes(
365 &self,
366 path: &str,
367 source: &[u8],
368 space: ExGuid,
369 object: ExGuid,
370 property: u32,
371 value: &[u8],
372 ) -> Result<(), CommitError> {
373 self.commit(path, |file| {
374 onestore::commit_property_bytes(file, source, space, object, property, value)
375 })
376 }
377
378 /// Publishes a transaction using the same native writer coordination as text commits.388 /// Publishes a transaction using the same native writer coordination as text commits.
379 pub fn commit_transaction(389 pub fn commit_transaction(
380 &self,390 &self,
381 path: &str,391 path: &str,
382 transaction: &onestore::Transaction,392 transaction: &onestore::Transaction,
383 ) -> Result<(), CommitError> {393 ) -> Result<(), CommitError> {
384 self.commit(path, |file| transaction.commit(file))394 self.commit(path, &checked(transaction.base()), |file| {
395 transaction.commit(file)
396 })
385 }397 }
386398
387 /// The header and length of a revision store, read without writer coordination or path399 /// The header and length of a revision store, in one round trip, without writer
388 /// identity checks: a change detector for polling, never a snapshot to edit.400 /// coordination or path identity checks: a change detector for polling, never a snapshot
401 /// to edit.
389 pub fn stamp(&self, path: &str) -> io::Result<onestore::Stamp> {402 pub fn stamp(&self, path: &str) -> io::Result<onestore::Stamp> {
390 let mut file = self.open(path, false)?;403 let create = create_request(path, 0x80000000, 7, CreateDisposition::FileOpen, 0x40)?;
391 let mut header = [0; 1024];404 let read = read_request(FileId::SENTINEL, 0, 1024);
392 let mut read = 0;405 let close = CloseRequest {
393 let result = loop {406 file_id: FileId::SENTINEL,
394 match file.read_at(read as u64, &mut header[read..]) {407 flags: 0,
395 Ok(0) => break Err(io::ErrorKind::UnexpectedEof.into()),
396 Ok(count) => read += count,
397 Err(error) => break Err(error),
398 }
399 if read == header.len() {
400 break Ok(());
401 }
402 };408 };
403 let length = file.length;409 let [created, read, closed]: [_; 3] = self
404 let closed = file.close();410 .compound(&[
405 result?;411 (Command::Create, &create),
412 (Command::Read, &read),
413 (Command::Close, &close),
414 ])?
415 .try_into()
416 .map_err(|_| io::ErrorKind::InvalidData)?;
417 let created: CreateResponse = self.unpack(&created?)?;
418 if closed.is_err() {
419 self.retire();
420 }
421 let read: ReadResponse = self.unpack(&read?)?;
406 closed?;422 closed?;
407 Ok(onestore::Stamp { header, length })423 let header = read
424 .data
425 .try_into()
426 .map_err(|_| io::ErrorKind::UnexpectedEof)?;
427 Ok(onestore::Stamp {
428 header,
429 length: created.end_of_file,
430 })
408 }431 }
409432
410 /// Confirms an observed snapshot's durability under native writer coordination.433 /// Confirms that the file still has `base`'s stamp and is durable, then refreshes its
411 pub fn confirm_snapshot(&self, path: &str, source: &[u8]) -> Result<(), CommitError> {434 /// version, under native writer coordination.
412 self.commit(path, |file| onestore::confirm_snapshot(file, source))435 pub fn confirm(&self, path: &str, base: &onestore::Stamp) -> Result<(), CommitError> {
436 self.commit(path, &checked(base), |file| onestore::confirm(file, base))
413 }437 }
414438
439 /// Opens `path` for writing under OneNote's coordination, making `reads` with the locks.
415 fn commit(440 fn commit(
416 &self,441 &self,
417 path: &str,442 path: &str,
443 reads: &[(u64, usize)],
418 operation: impl FnOnce(&mut File<'_>) -> Result<(), CommitError>,444 operation: impl FnOnce(&mut File<'_>) -> Result<(), CommitError>,
419 ) -> Result<(), CommitError> {445 ) -> Result<(), CommitError> {
420 let mut file = self446 let mut file = self
421 .open(path, true)447 .open(path, true)
422 .and_then(|file| file.coordinate(path, true))448 .and_then(|file| file.coordinate(path, true, reads))
423 .map_err(|error| CommitError {449 .map_err(|error| CommitError {
424 state: CommitState::NotCommitted,450 state: CommitState::NotCommitted,
425 error,451 error,
...@@ -443,24 +469,103 @@ impl Drop for Client {...@@ -443,24 +469,103 @@ impl Drop for Client {
443struct File<'a> {469struct File<'a> {
444 client: &'a Client,470 client: &'a Client,
445 id: Option<FileId>,471 id: Option<FileId>,
446 /// The end of file when opened.472 /// Writes `flush` sends ahead of its flush, in order.
447 length: u64,473 writes: Vec<(u64, Vec<u8>)>,
474 /// Reads made under the coordination locks, each returned once to the same `read_at`.
475 prefetched: Vec<(u64, usize, Vec<u8>)>,
448}476}
449impl File<'_> {477impl<'a> File<'a> {
450 fn coordinate(self, path: &str, write: bool) -> io::Result<Self> {478 fn new(client: &'a Client, created: &CreateResponse) -> Self {
451 self.lock(0xfffffffb, 0x11)?;479 Self {
480 client,
481 id: Some(created.file_id),
482 writes: Vec::new(),
483 prefetched: Vec::new(),
484 }
485 }
486
487 /// Takes OneNote 2010's coordination locks in one request (the shared reader byte, and
488 /// to write the exclusive writer byte), makes `reads` under them, then checks that the
489 /// handle is still the file at `path`: maintenance replaces a file while holding its
490 /// locks.
491 fn coordinate(mut self, path: &str, write: bool, reads: &[(u64, usize)]) -> io::Result<Self> {
492 let id = self.id.unwrap();
493 let mut locks = vec![LockElement {
494 offset: 0xfffffffb,
495 length: 1,
496 flags: 0x11,
497 }];
452 if write {498 if write {
453 self.lock(0xfffffffd, 0x12)?;499 locks.push(LockElement {
500 offset: 0xfffffffd,
501 length: 1,
502 flags: 0x12,
503 });
504 }
505 let lock = LockRequest {
506 file_id: id,
507 lock_sequence: 0,
508 locks,
509 };
510 let [index, volume] = identity_requests(id);
511 let reading: Vec<_> = reads
512 .iter()
513 .map(|(offset, length)| read_request(id, *offset, *length))
514 .collect();
515 let mut requests: Vec<(Command, &dyn Pack)> = vec![
516 (Command::Lock, &lock),
517 (Command::QueryInfo, &index),
518 (Command::QueryInfo, &volume),
519 ];
520 requests.extend(
521 reading
522 .iter()
523 .map(|read| (Command::Read, read as &dyn Pack)),
524 );
525 let mut responses = self.client.compound(&requests)?.into_iter();
526 let mut next = || responses.next().ok_or(io::ErrorKind::InvalidData);
527 let _: LockResponse = self.client.unpack(&next()??)?;
528 let own = self.client.identity(next()?, next()?)?;
529 for (offset, length) in reads {
530 let data = match next()? {
531 Err(error) if error.kind() == io::ErrorKind::UnexpectedEof => Vec::new(),
532 body => self.client.unpack::<ReadResponse>(&body?)?.data,
533 };
534 if data.len() > *length {
535 self.client.retire();
536 return Err(io::ErrorKind::InvalidData.into());
537 }
538 self.prefetched.push((*offset, *length, data));
539 }
540 let create = create_request(path, 0x80, 7, CreateDisposition::FileOpen, 0)?;
541 let [index, volume] = identity_requests(FileId::SENTINEL);
542 let close = CloseRequest {
543 file_id: FileId::SENTINEL,
544 flags: 0,
545 };
546 let [created, index, volume, closed]: [_; 4] = self
547 .client
548 .compound(&[
549 (Command::Create, &create),
550 (Command::QueryInfo, &index),
551 (Command::QueryInfo, &volume),
552 (Command::Close, &close),
553 ])?
554 .try_into()
555 .map_err(|_| io::ErrorKind::InvalidData)?;
556 created?;
557 if closed.is_err() {
558 self.client.retire();
454 }559 }
455 let current = self.client.open(path, false)?;560 let current = self.client.identity(index, volume)?;
456 let same = self.identity()? == current.identity()?;561 closed?;
457 current.close()?;562 if own != current {
458 if !same {
459 return Err(io::ErrorKind::ResourceBusy.into());563 return Err(io::ErrorKind::ResourceBusy.into());
460 }564 }
461 Ok(self)565 Ok(self)
462 }566 }
463567
568 #[cfg(test)]
464 fn lock(&self, offset: u64, flags: u32) -> io::Result<()> {569 fn lock(&self, offset: u64, flags: u32) -> io::Result<()> {
465 let _: LockResponse = self.client.request(570 let _: LockResponse = self.client.request(
466 Command::Lock,571 Command::Lock,
...@@ -477,47 +582,6 @@ impl File<'_> {...@@ -477,47 +582,6 @@ impl File<'_> {
477 Ok(())582 Ok(())
478 }583 }
479584
480 fn identity(&self) -> io::Result<(u64, u32)> {
481 let index: QueryInfoResponse = self.client.request(
482 Command::QueryInfo,
483 QueryInfoRequest {
484 info_type: InfoType::File,
485 file_info_class: 6,
486 output_buffer_length: 8,
487 additional_information: 0,
488 flags: 0,
489 file_id: self.id.unwrap(),
490 input_buffer: Vec::new(),
491 },
492 )?;
493 let index = u64::from_le_bytes(
494 index
495 .output_buffer
496 .try_into()
497 .map_err(|_| io::ErrorKind::InvalidData)?,
498 );
499 if index == 0 {
500 return Err(io::ErrorKind::Unsupported.into());
501 }
502 let volume: QueryInfoResponse = self.client.request(
503 Command::QueryInfo,
504 QueryInfoRequest {
505 info_type: InfoType::Filesystem,
506 file_info_class: 1,
507 output_buffer_length: 1024,
508 additional_information: 0,
509 flags: 0,
510 file_id: self.id.unwrap(),
511 input_buffer: Vec::new(),
512 },
513 )?;
514 let serial = volume
515 .output_buffer
516 .get(8..12)
517 .ok_or(io::ErrorKind::InvalidData)?;
518 Ok((index, u32::from_le_bytes(serial.try_into().unwrap())))
519 }
520
521 fn close(mut self) -> io::Result<()> {585 fn close(mut self) -> io::Result<()> {
522 self.release()586 self.release()
523 }587 }
...@@ -540,6 +604,87 @@ impl Drop for File<'_> {...@@ -540,6 +604,87 @@ impl Drop for File<'_> {
540 let _ = self.release();604 let _ = self.release();
541 }605 }
542}606}
607
608impl Client {
609 /// A file's volume serial number and index, from the responses to `identity_requests`.
610 fn identity(
611 &self,
612 index: io::Result<Vec<u8>>,
613 volume: io::Result<Vec<u8>>,
614 ) -> io::Result<(u64, u32)> {
615 let index: QueryInfoResponse = self.unpack(&index?)?;
616 let index = u64::from_le_bytes(
617 index
618 .output_buffer
619 .try_into()
620 .map_err(|_| io::ErrorKind::InvalidData)?,
621 );
622 if index == 0 {
623 return Err(io::ErrorKind::Unsupported.into());
624 }
625 let volume: QueryInfoResponse = self.unpack(&volume?)?;
626 let serial = volume
627 .output_buffer
628 .get(8..12)
629 .ok_or(io::ErrorKind::InvalidData)?;
630 Ok((index, u32::from_le_bytes(serial.try_into().unwrap())))
631 }
632}
633
634/// FileInternalInformation and FileFsVolumeInformation queries.
635fn identity_requests(file_id: FileId) -> [QueryInfoRequest; 2] {
636 let query = |info_type, file_info_class, output_buffer_length| QueryInfoRequest {
637 info_type,
638 file_info_class,
639 output_buffer_length,
640 additional_information: 0,
641 flags: 0,
642 file_id,
643 input_buffer: Vec::new(),
644 };
645 [
646 query(InfoType::File, 6, 8),
647 query(InfoType::Filesystem, 1, 1024),
648 ]
649}
650
651fn create_request(
652 path: &str,
653 access: u32,
654 sharing: u32,
655 disposition: CreateDisposition,
656 options: u32,
657) -> io::Result<CreateRequest> {
658 if path.is_empty() || path.contains('\0') || path.encode_utf16().count() > 32767 {
659 return Err(io::ErrorKind::InvalidInput.into());
660 }
661 Ok(CreateRequest {
662 requested_oplock_level: OplockLevel::None,
663 impersonation_level: ImpersonationLevel::Impersonation,
664 desired_access: FileAccessMask::new(access),
665 file_attributes: 0,
666 share_access: ShareAccess(sharing),
667 create_disposition: disposition,
668 create_options: options,
669 name: smb2::encode_path(&path.replace('\\', "/")),
670 create_contexts: Vec::new(),
671 })
672}
673
674fn read_request(file_id: FileId, offset: u64, length: usize) -> ReadRequest {
675 ReadRequest {
676 file_id,
677 offset,
678 length: length as u32,
679 minimum_count: 1,
680 flags: 0,
681 padding: 0,
682 channel: 0,
683 remaining_bytes: 0,
684 read_channel_info: Vec::new(),
685 }
686}
687
543fn read_size(params: &NegotiatedParams, credits: u16, requested: usize) -> usize {688fn read_size(params: &NegotiatedParams, credits: u16, requested: usize) -> usize {
544 let budget = if params.dialect != Dialect::Smb2_0_2689 let budget = if params.dialect != Dialect::Smb2_0_2
545 && params.capabilities.contains(Capabilities::LARGE_MTU)690 && params.capabilities.contains(Capabilities::LARGE_MTU)
...@@ -559,6 +704,15 @@ impl CommitIo for File<'_> {...@@ -559,6 +704,15 @@ impl CommitIo for File<'_> {
559 if output.is_empty() {704 if output.is_empty() {
560 return Ok(0);705 return Ok(0);
561 }706 }
707 if let Some(at) = self
708 .prefetched
709 .iter()
710 .position(|(start, length, _)| *start == offset && *length == output.len())
711 {
712 let (_, _, data) = self.prefetched.swap_remove(at);
713 output[..data.len()].copy_from_slice(&data);
714 return Ok(data.len());
715 }
562 let mut size = 0;716 let mut size = 0;
563 let response: io::Result<ReadResponse> =717 let response: io::Result<ReadResponse> =
564 self.client.request_with(Command::Read, |connection| {718 self.client.request_with(Command::Read, |connection| {
...@@ -595,40 +749,71 @@ impl CommitIo for File<'_> {...@@ -595,40 +749,71 @@ impl CommitIo for File<'_> {
595 output[..response.data.len()].copy_from_slice(&response.data);749 output[..response.data.len()].copy_from_slice(&response.data);
596 Ok(response.data.len())750 Ok(response.data.len())
597 }751 }
752 /// Queues at most one request's worth of `bytes`; `flush` sends it.
598 fn write_at(&mut self, offset: u64, bytes: &[u8]) -> io::Result<usize> {753 fn write_at(&mut self, offset: u64, bytes: &[u8]) -> io::Result<usize> {
599 if bytes.is_empty() {
600 return Ok(0);
601 }
602 let size = bytes.len().min(65536);754 let size = bytes.len().min(65536);
603 let response: WriteResponse = self.client.request(755 if size > 0 {
604 Command::Write,756 self.writes.push((offset, bytes[..size].to_vec()));
605 WriteRequest {757 }
606 file_id: self.id.unwrap(),758 Ok(size)
759 }
760
761 /// Sends the queued writes and then the flush in related compound requests of at most
762 /// sixteen, awaiting each: the server performs a compound in order, and answers the flush
763 /// once every write before it is durable.
764 fn flush(&mut self) -> io::Result<()> {
765 let file_id = self.id.unwrap();
766 let writes: Vec<_> = self
767 .writes
768 .drain(..)
769 .map(|(offset, data)| WriteRequest {
770 file_id,
607 offset,771 offset,
608 data: bytes[..size].to_vec(),772 data,
609 data_offset: 112,773 data_offset: 112,
610 flags: 1,774 flags: 0,
611 channel: 0,775 channel: 0,
612 remaining_bytes: 0,776 remaining_bytes: 0,
613 write_channel_info_offset: 0,777 write_channel_info_offset: 0,
614 write_channel_info_length: 0,778 write_channel_info_length: 0,
615 },779 })
616 )?;780 .collect();
617 if response.count as usize > size {781 let flush = FlushRequest { file_id };
618 return Err(io::ErrorKind::InvalidData.into());782 let mut requests: Vec<(Command, &dyn Pack)> = writes
783 .iter()
784 .map(|write| (Command::Write, write as &dyn Pack))
785 .collect();
786 requests.push((Command::Flush, &flush));
787 let mut sizes = writes.iter().map(|write| write.data.len());
788 for compound in requests.chunks(16) {
789 for (response, (command, _)) in
790 self.client.compound(compound)?.into_iter().zip(compound)
791 {
792 let body = response?;
793 if *command == Command::Flush {
794 let _: FlushResponse = self.client.unpack(&body)?;
795 continue;
796 }
797 let written: WriteResponse = self.client.unpack(&body)?;
798 let size = sizes.next().ok_or(io::ErrorKind::InvalidData)?;
799 if written.count as usize != size {
800 return Err(if (written.count as usize) < size {
801 io::ErrorKind::WriteZero
802 } else {
803 io::ErrorKind::InvalidData
804 }
805 .into());
806 }
807 }
619 }808 }
620 Ok(response.count as usize)
621 }
622 fn flush(&mut self) -> io::Result<()> {
623 let _: FlushResponse = self.client.request(
624 Command::Flush,
625 FlushRequest {
626 file_id: self.id.unwrap(),
627 },
628 )?;
629 Ok(())809 Ok(())
630 }810 }
631}811}
632812
813/// The reads `Stamp::check` makes: the header and a probe of the last byte.
814fn checked(stamp: &onestore::Stamp) -> [(u64, usize); 2] {
815 [(0, 1024), (stamp.length.saturating_sub(1), 2)]
816}
817
633#[cfg(test)]818#[cfg(test)]
634mod tests;819mod tests;
crates/notebook/src/smb/remote.rs+4-4
...@@ -26,15 +26,15 @@ impl Remote for SmbRemote {...@@ -26,15 +26,15 @@ impl Remote for SmbRemote {
26 self.client.read(&self.path, self.limit)26 self.client.read(&self.path, self.limit)
27 }27 }
2828
29 fn stamp(&mut self) -> io::Result<Option<Stamp>> {29 fn stamp(&mut self) -> io::Result<Stamp> {
30 self.client.stamp(&self.path).map(Some)30 self.client.stamp(&self.path)
31 }31 }
3232
33 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {33 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
34 self.client.commit_transaction(&self.path, transaction)34 self.client.commit_transaction(&self.path, transaction)
35 }35 }
3636
37 fn confirm(&mut self, snapshot: &[u8]) -> Result<(), CommitError> {37 fn confirm(&mut self, base: &Stamp) -> Result<(), CommitError> {
38 self.client.confirm_snapshot(&self.path, snapshot)38 self.client.confirm(&self.path, base)
39 }39 }
40}40}
crates/notebook/src/smb/tests.rs+67-38
...@@ -1,7 +1,8 @@...@@ -1,7 +1,8 @@
1use super::*;1use super::*;
2use onestore::{2use onestore::{
3 RevisionIndex, Store,3 ExGuid, RevisionIndex, Store, Transaction,
4 document::{Document, Kind},4 document::{Document, Kind},
5 op::{Edit, Op, PageOp},
5};6};
6use std::{7use std::{
7 fs,8 fs,
...@@ -117,11 +118,7 @@ fn create(client: &Client, path: &str, bytes: &[u8]) {...@@ -117,11 +118,7 @@ fn create(client: &Client, path: &str, bytes: &[u8]) {
117 },118 },
118 )119 )
119 .unwrap();120 .unwrap();
120 let mut file = File {121 let mut file = File::new(client, &response);
121 client,
122 id: Some(response.file_id),
123 length: response.end_of_file,
124 };
125 let mut offset = 0;122 let mut offset = 0;
126 while offset < bytes.len() {123 while offset < bytes.len() {
127 offset += file.write_at(offset as u64, &bytes[offset..]).unwrap();124 offset += file.write_at(offset as u64, &bytes[offset..]).unwrap();
...@@ -129,6 +126,43 @@ fn create(client: &Client, path: &str, bytes: &[u8]) {...@@ -129,6 +126,43 @@ fn create(client: &Client, path: &str, bytes: &[u8]) {
129 file.flush().unwrap();126 file.flush().unwrap();
130 file.close().unwrap();127 file.close().unwrap();
131}128}
129/// The transaction replacing `range` of the first text in `image` with `with`, as typed now.
130fn replaced(image: &[u8], range: std::ops::Range<u32>, with: &str) -> Transaction {
131 let (space, text, _) = text(image);
132 let arena = onestore::Arena::default();
133 let mut section = onestore::Section::open(&arena, image.to_vec()).unwrap();
134 let at = SystemTime::now()
135 .duration_since(UNIX_EPOCH)
136 .unwrap()
137 .as_nanos() as u64
138 / 100
139 + 116_444_736_000_000_000;
140 section
141 .apply(
142 "Fixture",
143 &Edit {
144 at,
145 ops: vec![Op::Page {
146 space,
147 op: PageOp::Text {
148 text,
149 range,
150 with: with.to_owned(),
151 },
152 }],
153 },
154 )
155 .unwrap();
156 section.seal().unwrap().unwrap()
157}
158
159/// `image` after `transaction`.
160fn applied(image: &[u8], transaction: &Transaction) -> Vec<u8> {
161 let mut image = image.to_vec();
162 transaction.apply(&mut image).unwrap();
163 image
164}
165
132fn text(bytes: &[u8]) -> (ExGuid, ExGuid, String) {166fn text(bytes: &[u8]) -> (ExGuid, ExGuid, String) {
133 let store = Store::parse(bytes).unwrap();167 let store = Store::parse(bytes).unwrap();
134 assert!(store.checksum_mismatches.is_empty());168 assert!(store.checksum_mismatches.is_empty());
...@@ -171,21 +205,15 @@ fn live_coordination() {...@@ -171,21 +205,15 @@ fn live_coordination() {
171 client205 client
172 .open(&path, false)206 .open(&path, false)
173 .unwrap()207 .unwrap()
174 .coordinate(&path, false)208 .coordinate(&path, false, &[])
175 .unwrap()209 .unwrap()
176 })210 })
177 .collect();211 .collect();
178 let (sid, oid, before) = text(&source);212 let (_, _, before) = text(&source);
179 let replacement = "After café 🦀";213 let replacement = "After café 🦀";
214 let everything = 0..before.encode_utf16().count() as u32;
180 writer215 writer
181 .commit_text(216 .commit_transaction(&path, &replaced(&source, everything.clone(), replacement))
182 &path,
183 &source,
184 sid,
185 oid,
186 0..before.encode_utf16().count() as u32,
187 replacement,
188 )
189 .unwrap();217 .unwrap();
190 let after = writer.read(&path, 1 << 20).unwrap();218 let after = writer.read(&path, 1 << 20).unwrap();
191 assert_eq!(text(&after).2, replacement);219 assert_eq!(text(&after).2, replacement);
...@@ -196,7 +224,7 @@ fn live_coordination() {...@@ -196,7 +224,7 @@ fn live_coordination() {
196 assert_eq!(snapshot, after);224 assert_eq!(snapshot, after);
197 }225 }
198 let error = writer226 let error = writer
199 .commit_text(&path, &source, sid, oid, 0..1, "X")227 .commit_transaction(&path, &replaced(&source, 0..1, "X"))
200 .unwrap_err();228 .unwrap_err();
201 assert_eq!(error.state, CommitState::NotCommitted);229 assert_eq!(error.state, CommitState::NotCommitted);
202 assert_eq!(error.error.kind(), io::ErrorKind::ResourceBusy);230 assert_eq!(error.error.kind(), io::ErrorKind::ResourceBusy);
...@@ -262,7 +290,7 @@ fn live_coordination() {...@@ -262,7 +290,7 @@ fn live_coordination() {
262 .unwrap();290 .unwrap();
263 drop(connection);291 drop(connection);
264 guard.close().unwrap();292 guard.close().unwrap();
265 let error = stale.coordinate(&path, true).err().unwrap();293 let error = stale.coordinate(&path, true, &[]).err().unwrap();
266 assert_eq!(error.kind(), io::ErrorKind::ResourceBusy);294 assert_eq!(error.kind(), io::ErrorKind::ResourceBusy);
267 assert_eq!(writer.read(&path, 1 << 20).unwrap(), source);295 assert_eq!(writer.read(&path, 1 << 20).unwrap(), source);
268296
...@@ -270,7 +298,7 @@ fn live_coordination() {...@@ -270,7 +298,7 @@ fn live_coordination() {
270 let file = retiring298 let file = retiring
271 .open(&path, true)299 .open(&path, true)
272 .unwrap()300 .unwrap()
273 .coordinate(&path, true)301 .coordinate(&path, true, &[])
274 .unwrap();302 .unwrap();
275 retiring.retire();303 retiring.retire();
276 assert_eq!(304 assert_eq!(
...@@ -282,7 +310,7 @@ fn live_coordination() {...@@ -282,7 +310,7 @@ fn live_coordination() {
282 loop {310 loop {
283 match writer311 match writer
284 .open(&path, true)312 .open(&path, true)
285 .and_then(|file| file.coordinate(&path, true))313 .and_then(|file| file.coordinate(&path, true, &[]))
286 {314 {
287 Ok(file) => {315 Ok(file) => {
288 file.close().unwrap();316 file.close().unwrap();
...@@ -300,7 +328,7 @@ fn live_coordination() {...@@ -300,7 +328,7 @@ fn live_coordination() {
300 let mut unfinished = writer328 let mut unfinished = writer
301 .open(&path, true)329 .open(&path, true)
302 .unwrap()330 .unwrap()
303 .coordinate(&path, true)331 .coordinate(&path, true, &[])
304 .unwrap();332 .unwrap();
305 assert_eq!(333 assert_eq!(
306 unfinished334 unfinished
...@@ -314,14 +342,7 @@ fn live_coordination() {...@@ -314,14 +342,7 @@ fn live_coordination() {
314 assert_eq!(snapshot.len(), source.len() + 11);342 assert_eq!(snapshot.len(), source.len() + 11);
315 assert_eq!(text(&snapshot).2, before);343 assert_eq!(text(&snapshot).2, before);
316 writer344 writer
317 .commit_text(345 .commit_transaction(&path, &replaced(&snapshot, everything, "Recovered café 🦀"))
318 &path,
319 &snapshot,
320 sid,
321 oid,
322 0..before.encode_utf16().count() as u32,
323 "Recovered café 🦀",
324 )
325 .unwrap();346 .unwrap();
326 let recovered = writer.read(&path, 1 << 20).unwrap();347 let recovered = writer.read(&path, 1 << 20).unwrap();
327 assert_eq!(text(&recovered).2, "Recovered café 🦀");348 assert_eq!(text(&recovered).2, "Recovered café 🦀");
...@@ -376,19 +397,27 @@ fn live_stamp_and_transaction() {...@@ -376,19 +397,27 @@ fn live_stamp_and_transaction() {
376 onestore::Stamp::of(&source).unwrap()397 onestore::Stamp::of(&source).unwrap()
377 );398 );
378 guard.close().unwrap();399 guard.close().unwrap();
379 let (sid, oid, _) = text(&source);400 let transaction = replaced(&source, 0..0, "Changed ");
380 let edit = onestore::PreparedEdit::text(&source, sid, oid, 0..0, "Changed ").unwrap();401 let edited = applied(&source, &transaction);
381 let transaction = edit.transaction();
382 poller.commit_transaction(&path, &transaction).unwrap();402 poller.commit_transaction(&path, &transaction).unwrap();
383 assert_eq!(403 assert_eq!(
384 poller.stamp(&path).unwrap(),404 poller.stamp(&path).unwrap(),
385 onestore::Stamp::of(edit.as_bytes()).unwrap()405 onestore::Stamp::of(&edited).unwrap()
386 );406 );
387 assert_eq!(poller.read(&path, 1 << 20).unwrap(), edit.as_bytes());407 assert_eq!(poller.read(&path, 1 << 20).unwrap(), edited);
388 let stale = poller.commit_transaction(&path, &transaction).unwrap_err();408 let stale = poller.commit_transaction(&path, &transaction).unwrap_err();
389 assert_eq!(stale.state, CommitState::NotCommitted);409 assert_eq!(stale.state, CommitState::NotCommitted);
390 assert_eq!(stale.error.kind(), io::ErrorKind::ResourceBusy);410 assert_eq!(stale.error.kind(), io::ErrorKind::ResourceBusy);
391 assert_eq!(poller.read(&path, 1 << 20).unwrap(), edit.as_bytes());411 assert_eq!(poller.read(&path, 1 << 20).unwrap(), edited);
412 let base = onestore::Stamp::of(&edited).unwrap();
413 poller.confirm(&path, &base).unwrap();
414 let confirmed = poller.read(&path, 1 << 20).unwrap();
415 assert_eq!(confirmed[..212], edited[..212]);
416 assert_ne!(confirmed[212..252], edited[212..252]);
417 assert_eq!(confirmed[252..], edited[252..]);
418 let stale = poller.confirm(&path, &base).unwrap_err();
419 assert_eq!(stale.state, CommitState::NotCommitted);
420 assert_eq!(stale.error.kind(), io::ErrorKind::ResourceBusy);
392 poller.delete(&path).unwrap();421 poller.delete(&path).unwrap();
393}422}
394423
...@@ -488,7 +517,7 @@ fn live_reader_hold() {...@@ -488,7 +517,7 @@ fn live_reader_hold() {
488 let mut file = client517 let mut file = client
489 .open(&path, false)518 .open(&path, false)
490 .unwrap()519 .unwrap()
491 .coordinate(&path, false)520 .coordinate(&path, false, &[])
492 .unwrap();521 .unwrap();
493 fs::write(output.join("ready"), b"held").unwrap();522 fs::write(output.join("ready"), b"held").unwrap();
494 let deadline = Instant::now() + Duration::from_secs(300);523 let deadline = Instant::now() + Duration::from_secs(300);
...@@ -580,13 +609,13 @@ fn live_structure() {...@@ -580,13 +609,13 @@ fn live_structure() {
580 let mut notebook =609 let mut notebook =
581 Notebook::open_smb(std::sync::Arc::clone(&client), &root, cache.path()).unwrap();610 Notebook::open_smb(std::sync::Arc::clone(&client), &root, cache.path()).unwrap();
582 assert_eq!(611 assert_eq!(
583 notebook.create_section("", "Second", "Author").unwrap(),612 notebook.create_section("", "Second", &onestore::PageCreation::new(None, Some(""), "Author").unwrap()).unwrap(),
584 "Second.one"613 "Second.one"
585 );614 );
586 assert_eq!(notebook.create_group("", "Archive").unwrap(), "Archive");615 assert_eq!(notebook.create_group("", "Archive").unwrap(), "Archive");
587 assert_eq!(616 assert_eq!(
588 notebook617 notebook
589 .create_section("Archive", "Inner", "Author")618 .create_section("Archive", "Inner", &onestore::PageCreation::new(None, Some(""), "Author").unwrap())
590 .unwrap(),619 .unwrap(),
591 "Archive/Inner.one"620 "Archive/Inner.one"
592 );621 );
crates/notebook/src/smb/tests/faults.rs+12-33
...@@ -297,7 +297,7 @@ fn live_message_loss() {...@@ -297,7 +297,7 @@ fn live_message_loss() {
297 &source,297 &source,
298 )298 )
299 .unwrap();299 .unwrap();
300 let (sid, oid, before) = text(&source);300 let (_, _, before) = text(&source);
301 let after = if fixture == "chunked" {301 let after = if fixture == "chunked" {
302 "After café 🦀 ".repeat(6000)302 "After café 🦀 ".repeat(6000)
303 } else {303 } else {
...@@ -318,15 +318,11 @@ fn live_message_loss() {...@@ -318,15 +318,11 @@ fn live_message_loss() {
318 );318 );
319 std::thread::sleep(Duration::from_millis(5));319 std::thread::sleep(Duration::from_millis(5));
320 }320 }
321 let expected = onestore::replace_text(321 let everything = 0..before.encode_utf16().count() as u32;
322 let new = snapshot(&applied(
322 &source,323 &source,
323 sid,324 &replaced(&source, everything.clone(), &after),
324 oid,325 ));
325 0..before.encode_utf16().count() as u32,
326 &after,
327 )
328 .unwrap();
329 let new = snapshot(&expected);
330 let baseline_path = format!("fault-{prefix}-{fixture}-baseline.one");326 let baseline_path = format!("fault-{prefix}-{fixture}-baseline.one");
331 create(&observer, &baseline_path, &source);327 create(&observer, &baseline_path, &source);
332 configure(&output, json!({"phase":format!("{fixture}-setup")}));328 configure(&output, json!({"phase":format!("{fixture}-setup")}));
...@@ -340,13 +336,9 @@ fn live_message_loss() {...@@ -340,13 +336,9 @@ fn live_message_loss() {
340 let start = configure(&output, json!({"phase":format!("{fixture}-baseline")}));336 let start = configure(&output, json!({"phase":format!("{fixture}-baseline")}));
341 let began = stamp();337 let began = stamp();
342 baseline338 baseline
343 .commit_text(339 .commit_transaction(
344 &baseline_path,340 &baseline_path,
345 &source,341 &replaced(&source, everything.clone(), &after),
346 sid,
347 oid,
348 0..before.encode_utf16().count() as u32,
349 &after,
350 )342 )
351 .unwrap();343 .unwrap();
352 let events = records(&output);344 let events = records(&output);
...@@ -397,15 +389,9 @@ fn live_message_loss() {...@@ -397,15 +389,9 @@ fn live_message_loss() {
397 json!({"phase":name, "direction":direction, "cut":command, "status":status, "occurrence":occurrence}),389 json!({"phase":name, "direction":direction, "cut":command, "status":status, "occurrence":occurrence}),
398 );390 );
399 let began = stamp();391 let began = stamp();
392 let transaction = replaced(&source, everything.clone(), &after);
400 let error = interrupted393 let error = interrupted
401 .commit_text(394 .commit_transaction(&path, &transaction)
402 &path,
403 &source,
404 sid,
405 oid,
406 0..before.encode_utf16().count() as u32,
407 &after,
408 )
409 .unwrap_err();395 .unwrap_err();
410 assert_eq!(396 assert_eq!(
411 interrupted.read(&path, 1 << 20).unwrap_err().kind(),397 interrupted.read(&path, 1 << 20).unwrap_err().kind(),
...@@ -429,7 +415,7 @@ fn live_message_loss() {...@@ -429,7 +415,7 @@ fn live_message_loss() {
429 loop {415 loop {
430 match fresh416 match fresh
431 .open(&path, true)417 .open(&path, true)
432 .and_then(|file| file.coordinate(&path, true))418 .and_then(|file| file.coordinate(&path, true, &[]))
433 {419 {
434 Ok(file) => {420 Ok(file) => {
435 file.close().unwrap();421 file.close().unwrap();
...@@ -458,14 +444,7 @@ fn live_message_loss() {...@@ -458,14 +444,7 @@ fn live_message_loss() {
458 }444 }
459 if !visible {445 if !visible {
460 fresh446 fresh
461 .commit_text(447 .commit_transaction(&path, &replaced(&saved, everything.clone(), &after))
462 &path,
463 &saved,
464 sid,
465 oid,
466 0..before.encode_utf16().count() as u32,
467 &after,
468 )
469 .unwrap();448 .unwrap();
470 }449 }
471 let saved = fresh.read(&path, 1 << 20).unwrap();450 let saved = fresh.read(&path, 1 << 20).unwrap();
...@@ -475,7 +454,7 @@ fn live_message_loss() {...@@ -475,7 +454,7 @@ fn live_message_loss() {
475 );454 );
476 let end = after.encode_utf16().count() as u32;455 let end = after.encode_utf16().count() as u32;
477 fresh456 fresh
478 .commit_text(&path, &saved, sid, oid, end..end, suffix)457 .commit_transaction(&path, &replaced(&saved, end..end, suffix))
479 .unwrap();458 .unwrap();
480 let recovered = fresh.read(&path, 1 << 20).unwrap();459 let recovered = fresh.read(&path, 1 << 20).unwrap();
481 assert_eq!(text(&recovered).2, format!("{after}{suffix}"));460 assert_eq!(text(&recovered).2, format!("{after}{suffix}"));
crates/notebook/src/sync.rs+40-193
...@@ -1,5 +1,5 @@...@@ -1,5 +1,5 @@
1use super::*;1use super::*;
2use crate::working::{Rebased, Request, Resolve, Sealed};2use crate::working::{Request, Sealed};
3use onestore::{CommitError, CommitState, RevisionIndex, Stamp, Store, Transaction};3use onestore::{CommitError, CommitState, RevisionIndex, Stamp, Store, Transaction};
4use rusqlite::OptionalExtension;4use rusqlite::OptionalExtension;
5use std::{5use std::{
...@@ -12,44 +12,12 @@ use std::{...@@ -12,44 +12,12 @@ use std::{
12/// cached readers.12/// cached readers.
13pub trait Remote {13pub trait Remote {
14 fn read(&mut self) -> io::Result<Vec<u8>>;14 fn read(&mut self) -> io::Result<Vec<u8>>;
15 /// The file's stamp without reading its body, when the remote can tell. While it is the15 /// The file's stamp without reading its body or coordinating with writers. While it is
16 /// last observed image's, synchronization neither reads nor revalidates the file.16 /// the last observed image's, synchronization neither reads nor revalidates the file.
17 fn stamp(&mut self) -> io::Result<Option<Stamp>> {17 fn stamp(&mut self) -> io::Result<Stamp>;
18 Ok(None)
19 }
20 fn publish(&mut self, transaction: &Transaction) -> std::result::Result<(), CommitError>;18 fn publish(&mut self, transaction: &Transaction) -> std::result::Result<(), CommitError>;
21 fn confirm(&mut self, snapshot: &[u8]) -> std::result::Result<(), CommitError>;19 /// Confirms that the file still has `base`'s stamp and is durable (`onestore::confirm`).
22}20 fn confirm(&mut self, base: &Stamp) -> std::result::Result<(), CommitError>;
23
24#[derive(Debug, Clone, Copy, PartialEq, Eq)]
25#[repr(i64)]
26pub enum ConflictKind {
27 /// The edited page or a selected page no longer exists remotely.
28 TargetUnavailable = 0,
29 /// The remote section no longer accepts the edit.
30 UnsupportedEdit = 1,
31 /// Page order or indentation changed remotely in a way the batch cannot absorb.
32 StructureChanged = 2,
33 /// The remote page changed where the local edits changed it too; review is required.
34 ContentChanged = 3,
35}
36
37impl ConflictKind {
38 fn from_stored(kind: i64) -> Result<Self> {
39 Ok(match kind {
40 0 => Self::TargetUnavailable,
41 1 => Self::UnsupportedEdit,
42 2 => Self::StructureChanged,
43 3 => Self::ContentChanged,
44 _ => {
45 return Err(io::Error::new(
46 io::ErrorKind::InvalidData,
47 "Unknown cached conflict kind",
48 )
49 .into());
50 }
51 })
52 }
53}21}
5422
55#[derive(Debug, Clone, PartialEq, Eq)]23#[derive(Debug, Clone, PartialEq, Eq)]
...@@ -59,7 +27,6 @@ pub enum EditStatus {...@@ -59,7 +27,6 @@ pub enum EditStatus {
59 AwaitingConfirmation {27 AwaitingConfirmation {
60 revision: ExGuid,28 revision: ExGuid,
61 },29 },
62 Conflict(ConflictKind),
63 /// Historical confirmation; later remote edits or restores may remove the effect.30 /// Historical confirmation; later remote edits or restores may remove the effect.
64 Published {31 Published {
65 revision: ExGuid,32 revision: ExGuid,
...@@ -84,15 +51,13 @@ struct State {...@@ -84,15 +51,13 @@ struct State {
84 base: Stamp,51 base: Stamp,
85 /// The observed remote image's stamp while it is not the base.52 /// The observed remote image's stamp while it is not the base.
86 remote: Option<Stamp>,53 remote: Option<Stamp>,
87 /// The first batch waiting on a remote answer: sealed, attempted, or in conflict.54 /// The first batch waiting on a remote answer to an attempt.
88 blocked: Option<Blocked>,55 blocked: Option<Blocked>,
89 queued: bool,56 queued: bool,
90}57}
9158
92struct Blocked {59struct Blocked {
93 batch: i64,60 batch: i64,
94 attempted: bool,
95 conflict: Option<ConflictKind>,
96 revisions: Option<BTreeMap<ExGuid, ExGuid>>,61 revisions: Option<BTreeMap<ExGuid, ExGuid>>,
97}62}
9863
...@@ -101,24 +66,14 @@ fn state(connection: &Connection) -> Result<State> {...@@ -101,24 +66,14 @@ fn state(connection: &Connection) -> Result<State> {
101 .ok_or_else(|| io::Error::new(io::ErrorKind::InvalidData, "The cache has no base image"))?;66 .ok_or_else(|| io::Error::new(io::ErrorKind::InvalidData, "The cache has no base image"))?;
102 let blocked = connection67 let blocked = connection
103 .query_row(68 .query_row(
104 "SELECT id, attempted, conflict, revisions FROM batches69 "SELECT id, revisions FROM batches WHERE attempted=1 ORDER BY id LIMIT 1",
105 WHERE attempted=1 OR conflict IS NOT NULL ORDER BY id LIMIT 1",
106 [],70 [],
107 |row| {71 |row| Ok((row.get::<_, i64>(0)?, row.get::<_, Option<String>>(1)?)),
108 Ok((
109 row.get::<_, i64>(0)?,
110 row.get::<_, bool>(1)?,
111 row.get::<_, Option<i64>>(2)?,
112 row.get::<_, Option<String>>(3)?,
113 ))
114 },
115 )72 )
116 .optional()?73 .optional()?
117 .map(|(batch, attempted, conflict, revisions)| {74 .map(|(batch, revisions)| {
118 Ok::<_, Error>(Blocked {75 Ok::<_, Error>(Blocked {
119 batch,76 batch,
120 attempted,
121 conflict: conflict.map(ConflictKind::from_stored).transpose()?,
122 revisions: revisions77 revisions: revisions
123 .map(|revisions| decode_revisions(&revisions))78 .map(|revisions| decode_revisions(&revisions))
124 .transpose()?,79 .transpose()?,
...@@ -186,7 +141,7 @@ impl Replica {...@@ -186,7 +141,7 @@ impl Replica {
186141
187 /// The last observed remote image. Observation alone does not acknowledge any pending142 /// The last observed remote image. Observation alone does not acknowledge any pending
188 /// edit's remote durability.143 /// edit's remote durability.
189 pub fn remote_snapshot(&self) -> Result<Vec<u8>> {144 pub(crate) fn remote_snapshot(&self) -> Result<Vec<u8>> {
190 let connection = self.lock()?;145 let connection = self.lock()?;
191 let image = match base::read(&connection, base::Image::Remote)? {146 let image = match base::read(&connection, base::Image::Remote)? {
192 Some(image) => Some(image),147 Some(image) => Some(image),
...@@ -197,26 +152,6 @@ impl Replica {...@@ -197,26 +152,6 @@ impl Replica {
197 })?)152 })?)
198 }153 }
199154
200 /// The unpublished batch the remote no longer accepts, if any.
201 pub fn conflict(&self) -> Result<Option<Conflict>> {
202 let connection = self.lock()?;
203 let row: Option<(i64, i64, String)> = connection
204 .query_row(
205 "SELECT id, conflict, space FROM batches WHERE conflict IS NOT NULL ORDER BY id LIMIT 1",
206 [],
207 |row| Ok((row.get(0)?, row.get(1)?, row.get(2)?)),
208 )
209 .optional()?;
210 row.map(|(batch, kind, space)| {
211 Ok(Conflict {
212 id: newest(&connection, batch)?,
213 space: space.parse()?,
214 kind: ConflictKind::from_stored(kind)?,
215 })
216 })
217 .transpose()
218 }
219
220 /// Publishes the oldest unpublished batch, rebasing the queue first when the remote155 /// Publishes the oldest unpublished batch, rebasing the queue first when the remote
221 /// changed. Reads the remote image only when its stamp moved; network I/O holds156 /// changed. Reads the remote image only when its stamp moved; network I/O holds
222 /// synchronization ownership without holding the cache mutex. Uncertain attempts are157 /// synchronization ownership without holding the cache mutex. Uncertain attempts are
...@@ -225,8 +160,8 @@ impl Replica {...@@ -225,8 +160,8 @@ impl Replica {
225 let _owner = self.sync_owner()?;160 let _owner = self.sync_owner()?;
226 let state = state(&*self.lock()?)?;161 let state = state(&*self.lock()?)?;
227 let observed = remote.stamp().map_err(Error::RemoteIo)?;162 let observed = remote.stamp().map_err(Error::RemoteIo)?;
228 if let (Some(observed), Some(blocked)) = (&observed, &state.blocked)163 if let Some(blocked) = &state.blocked
229 && Some(observed) == state.remote.as_ref().or(Some(&state.base))164 && observed == *state.remote.as_ref().unwrap_or(&state.base)
230 {165 {
231 // Nothing new to decide: the remote is as it was when the batch blocked.166 // Nothing new to decide: the remote is as it was when the batch blocked.
232 let id = newest(&*self.lock()?, blocked.batch)?;167 let id = newest(&*self.lock()?, blocked.batch)?;
...@@ -239,7 +174,7 @@ impl Replica {...@@ -239,7 +174,7 @@ impl Replica {
239 });174 });
240 }175 }
241 let image = match observed {176 let image = match observed {
242 Some(observed) if observed == state.base => None,177 observed if observed == state.base => None,
243 _ => {178 _ => {
244 let image = remote.read().map_err(Error::RemoteIo)?;179 let image = remote.read().map_err(Error::RemoteIo)?;
245 // A read image is compared whole: a stamp stands for it only when read alone.180 // A read image is compared whole: a stamp stands for it only when read alone.
...@@ -249,13 +184,7 @@ impl Replica {...@@ -249,13 +184,7 @@ impl Replica {
249 };184 };
250 let mut changed = Vec::new();185 let mut changed = Vec::new();
251 if let Some(image) = image {186 if let Some(image) = image {
252 if let Some(Blocked {187 if let Some(Blocked { batch, revisions }) = &state.blocked {
253 batch,
254 attempted: true,
255 revisions,
256 ..
257 }) = &state.blocked
258 {
259 let id = newest(&*self.lock()?, *batch)?;188 let id = newest(&*self.lock()?, *batch)?;
260 let revisions = revisions.clone().unwrap_or_default();189 let revisions = revisions.clone().unwrap_or_default();
261 let store = Store::parse(&image)?;190 let store = Store::parse(&image)?;
...@@ -308,7 +237,7 @@ impl Replica {...@@ -308,7 +237,7 @@ impl Replica {
308 .collect(),237 .collect(),
309 )238 )
310 };239 };
311 if let Err(error) = remote.confirm(&image) {240 if let Err(error) = remote.confirm(&Stamp::of(&image)?) {
312 if error.state == CommitState::Committed {241 if error.state == CommitState::Committed {
313 self.acknowledge(*batch, sealed.as_ref(), receipts.as_ref())?;242 self.acknowledge(*batch, sealed.as_ref(), receipts.as_ref())?;
314 }243 }
...@@ -316,49 +245,26 @@ impl Replica {...@@ -316,49 +245,26 @@ impl Replica {
316 }245 }
317 self.acknowledge(*batch, sealed.as_ref(), receipts.as_ref())?;246 self.acknowledge(*batch, sealed.as_ref(), receipts.as_ref())?;
318 let revision = self.receipt(id)?;247 let revision = self.receipt(id)?;
319 changed = self.rebase(Some(image), None)?.unwrap_or_default();248 changed = self.rebase(Some(image))?;
320 return Ok(Synced {249 return Ok(Synced {
321 edit: Some((id, EditStatus::Published { revision })),250 edit: Some((id, EditStatus::Published { revision })),
322 changed,251 changed,
323 });252 });
324 }253 }
325 match self.rebase(Some(image), None)? {254 changed = self.rebase(Some(image))?;
326 Ok(spaces) => changed = spaces,
327 Err(conflict) => {
328 return Ok(Synced {
329 edit: Some((conflict.id, EditStatus::Conflict(conflict.kind))),
330 changed,
331 });
332 }
333 }
334 } else if let Some(blocked) = &state.blocked {255 } else if let Some(blocked) = &state.blocked {
335 if state.remote.is_some() {256 if state.remote.is_some() {
336 // The remote is back at the base: what blocked the queue is gone.257 // The remote is back at the base: what blocked the queue is gone.
337 let connection = self.lock()?;258 base::clear(&*self.lock()?, base::Image::Remote)?;
338 base::clear(&connection, base::Image::Remote)?;
339 if blocked.conflict.is_some() {
340 connection.execute("UPDATE batches SET conflict=NULL, space=NULL", [])?;
341 }
342 }
343 if blocked.attempted {
344 let id = newest(&*self.lock()?, blocked.batch)?;
345 return Ok(Synced {
346 edit: Some((
347 id,
348 status(&*self.lock()?, id)?.unwrap_or(EditStatus::Pending),
349 )),
350 changed,
351 });
352 }
353 if let Some(kind) = blocked.conflict
354 && state.remote.is_none()
355 {
356 let id = newest(&*self.lock()?, blocked.batch)?;
357 return Ok(Synced {
358 edit: Some((id, EditStatus::Conflict(kind))),
359 changed,
360 });
361 }259 }
260 let id = newest(&*self.lock()?, blocked.batch)?;
261 return Ok(Synced {
262 edit: Some((
263 id,
264 status(&*self.lock()?, id)?.unwrap_or(EditStatus::Pending),
265 )),
266 changed,
267 });
362 } else if !state.queued {268 } else if !state.queued {
363 return Ok(Synced::default());269 return Ok(Synced::default());
364 }270 }
...@@ -383,7 +289,9 @@ impl Replica {...@@ -383,7 +289,9 @@ impl Replica {
383 let id = newest(&*self.lock()?, batch)?;289 let id = newest(&*self.lock()?, batch)?;
384 let Some(transaction) = transaction else {290 let Some(transaction) = transaction else {
385 // Edits that changed nothing are published once the remote's image is durable.291 // Edits that changed nothing are published once the remote's image is durable.
386 let base = base::read(&*self.lock()?, base::Image::Base)?.unwrap_or_default();292 let base = base::stamp(&*self.lock()?, base::Image::Base)?.ok_or_else(|| {
293 io::Error::new(io::ErrorKind::InvalidData, "The cache holds no base image")
294 })?;
387 if let Err(error) = remote.confirm(&base) {295 if let Err(error) = remote.confirm(&base) {
388 if error.state == CommitState::Committed {296 if error.state == CommitState::Committed {
389 self.acknowledge(batch, None, None)?;297 self.acknowledge(batch, None, None)?;
...@@ -439,22 +347,10 @@ impl Replica {...@@ -439,22 +347,10 @@ impl Replica {
439 ))347 ))
440 }348 }
441349
442 /// Asks the section thread to replay the queue on `image`; `Err` is the conflict found.350 /// Asks the section thread to replay the queue on `image`, returning the pages the
443 fn rebase(351 /// remote changed.
444 &self,352 fn rebase(&self, image: Option<Vec<u8>>) -> Result<Vec<ExGuid>> {
445 image: Option<Vec<u8>>,353 self.ask(|reply| Request::Rebase { image, reply })
446 resolve: Option<Resolve>,
447 ) -> Result<std::result::Result<Vec<ExGuid>, Conflict>> {
448 Ok(
449 match self.ask(|reply| Request::Rebase {
450 image,
451 resolve,
452 reply,
453 })? {
454 Rebased::Applied { changed } => Ok(changed),
455 Rebased::Conflict { id, space, kind } => Err(Conflict { id, space, kind }),
456 },
457 )
458 }354 }
459355
460 fn receipt(&self, id: u64) -> Result<ExGuid> {356 fn receipt(&self, id: u64) -> Result<ExGuid> {
...@@ -501,49 +397,6 @@ impl Replica {...@@ -501,49 +397,6 @@ impl Replica {
501 Ok(())397 Ok(())
502 }398 }
503399
504 /// Resolves the conflict holding edit `id`: the batch and everything queued after it
505 /// drop their ops on the conflicted page; `Mine` rewrites the remote page to the local
506 /// one first. A conflict on another page may follow.
507 pub fn resolve(&self, id: u64, resolution: Resolution) -> Result<()> {
508 self.resolve_with(id, resolution, None)
509 }
510
511 pub(crate) fn resolve_with(&self, id: u64, keep: Resolution, page: Option<Page>) -> Result<()> {
512 let owner = self.sync_owner()?;
513 let (first, space) = {
514 let connection = self.lock()?;
515 let row: Option<(i64, String)> = connection
516 .query_row(
517 "SELECT (SELECT min(id) FROM edits WHERE batch=batches.id), space FROM batches
518 WHERE conflict IS NOT NULL AND id=(SELECT batch FROM edits WHERE id=?1)",
519 [signed(id)?],
520 |row| Ok((row.get(0)?, row.get(1)?)),
521 )
522 .optional()?;
523 let Some((first, space)) = row else {
524 return Err(io::Error::new(
525 io::ErrorKind::InvalidInput,
526 "The edit is not in conflict",
527 )
528 .into());
529 };
530 (unsigned(first)?, space.parse()?)
531 };
532 // A conflict on another page is recorded like any other.
533 let _ = self.rebase(
534 None,
535 Some(Resolve {
536 first,
537 space,
538 keep,
539 page,
540 }),
541 )?;
542 drop(owner);
543 self.wake_sync();
544 Ok(())
545 }
546
547 /// Retires the uncertain attempt of the batch holding edit `id` after review. The400 /// Retires the uncertain attempt of the batch holding edit `id` after review. The
548 /// queue is first exported to `archive` (a new file), which is the record: no receipt401 /// queue is first exported to `archive` (a new file), which is the record: no receipt
549 /// is written. `Mine` publishes the batch again against the current remote; `Theirs`402 /// is written. `Mine` publishes the batch again against the current remote; `Theirs`
...@@ -620,10 +473,7 @@ impl Replica {...@@ -620,10 +473,7 @@ impl Replica {
620 None if !state.queued => state.base,473 None if !state.queued => state.base,
621 None => return Ok(false),474 None => return Ok(false),
622 };475 };
623 let Some(stamp) = remote.stamp().map_err(Error::RemoteIo)? else {476 Ok(remote.stamp().map_err(Error::RemoteIo)? == expected)
624 return Ok(false);
625 };
626 Ok(stamp == expected)
627 }477 }
628}478}
629479
...@@ -651,26 +501,23 @@ pub(crate) fn status(connection: &Connection, id: u64) -> Result<Option<EditStat...@@ -651,26 +501,23 @@ pub(crate) fn status(connection: &Connection, id: u64) -> Result<Option<EditStat
651 {501 {
652 return Ok(Some(EditStatus::Archived { archive }));502 return Ok(Some(EditStatus::Archived { archive }));
653 }503 }
654 let record: Option<(bool, Option<i64>, Option<String>, String)> = connection504 let record: Option<(bool, Option<String>, String)> = connection
655 .query_row(505 .query_row(
656 "SELECT batches.attempted, batches.conflict, batches.revisions, edits.edit506 "SELECT batches.attempted, batches.revisions, edits.edit
657 FROM edits JOIN batches ON batches.id=edits.batch WHERE edits.id=?1",507 FROM edits JOIN batches ON batches.id=edits.batch WHERE edits.id=?1",
658 [id],508 [id],
659 |row| Ok((row.get(0)?, row.get(1)?, row.get(2)?, row.get(3)?)),509 |row| Ok((row.get(0)?, row.get(1)?, row.get(2)?)),
660 )510 )
661 .optional()?;511 .optional()?;
662 Ok(match record {512 Ok(match record {
663 None => None,513 None => None,
664 Some((true, _, revisions, edit)) => {514 Some((true, revisions, edit)) => {
665 let revisions = decode_revisions(revisions.as_deref().unwrap_or("{}"))?;515 let revisions = decode_revisions(revisions.as_deref().unwrap_or("{}"))?;
666 let edit: onestore::op::Edit = serde_json::from_str(&edit)516 let edit: onestore::op::Edit = serde_json::from_str(&edit)
667 .map_err(|error| io::Error::new(io::ErrorKind::InvalidData, error))?;517 .map_err(|error| io::Error::new(io::ErrorKind::InvalidData, error))?;
668 let revision = revision_of(&edit, &revisions)?;518 let revision = revision_of(&edit, &revisions)?;
669 Some(EditStatus::AwaitingConfirmation { revision })519 Some(EditStatus::AwaitingConfirmation { revision })
670 }520 }
671 Some((false, Some(kind), ..)) => {521 Some((false, ..)) => Some(EditStatus::Pending),
672 Some(EditStatus::Conflict(ConflictKind::from_stored(kind)?))
673 }
674 Some((false, None, ..)) => Some(EditStatus::Pending),
675 })522 })
676}523}
crates/notebook/src/working.rs+120-327
...@@ -4,12 +4,11 @@...@@ -4,12 +4,11 @@
4//! and, when the remote changed, to rebase the queue.4//! and, when the remote changed, to rebase the queue.
55
6use crate::{6use crate::{
7 ConflictKind, Resolution, Result, base, lock, merge, queue, session::Save, signed,7 Result, base, lock, merge, queue, worker::Signal,
8 worker::Signal,
9};8};
10use onestore::{9use onestore::{
11 Arena, ExGuid, Section, Transaction,10 Arena, ExGuid, Section, Transaction,
12 op::{Edit, Op, OpError},11 op::{Edit, OpError},
13 page::Page,12 page::Page,
14};13};
15use rusqlite::{Connection, OptionalExtension, TransactionBehavior, params};14use rusqlite::{Connection, OptionalExtension, TransactionBehavior, params};
...@@ -28,15 +27,6 @@ pub(crate) enum Request {...@@ -28,15 +27,6 @@ pub(crate) enum Request {
28 edit: Edit,27 edit: Edit,
29 reply: Reply<u64>,28 reply: Reply<u64>,
30 },29 },
31 /// A whole-page save of the old session API: `None` answers a save a later one
32 /// continuing it absorbed.
33 Save {
34 space: ExGuid,
35 before: Page,
36 after: Page,
37 author: String,
38 reply: Reply<Option<Save>>,
39 },
40 Page {30 Page {
41 space: ExGuid,31 space: ExGuid,
42 reply: Reply<Page>,32 reply: Reply<Page>,
...@@ -44,6 +34,9 @@ pub(crate) enum Request {...@@ -44,6 +34,9 @@ pub(crate) enum Request {
44 Pages {34 Pages {
45 reply: Reply<Vec<(ExGuid, String, u32)>>,35 reply: Reply<Vec<(ExGuid, String, u32)>>,
46 },36 },
37 Conflicts {
38 reply: Reply<Vec<(ExGuid, Vec<onestore::ConflictPage>)>>,
39 },
47 /// Answers once the edits before it are written.40 /// Answers once the edits before it are written.
48 Flush {41 Flush {
49 reply: Reply<()>,42 reply: Reply<()>,
...@@ -53,11 +46,11 @@ pub(crate) enum Request {...@@ -53,11 +46,11 @@ pub(crate) enum Request {
53 Seal {46 Seal {
54 reply: Reply<Option<Sealed>>,47 reply: Reply<Option<Sealed>>,
55 },48 },
56 /// Replays the queue on `image` (the stored remote image, or the base, when `None`).49 /// Replays the queue on `image` (the stored remote image, or the base, when `None`);
50 /// answers with the pages the remote changed.
57 Rebase {51 Rebase {
58 image: Option<Vec<u8>>,52 image: Option<Vec<u8>>,
59 resolve: Option<Resolve>,53 reply: Reply<Vec<ExGuid>>,
60 reply: Reply<Rebased>,
61 },54 },
62 /// Rereads the queue after the sync thread replaced it.55 /// Rereads the queue after the sync thread replaced it.
63 Reopen {56 Reopen {
...@@ -78,34 +71,14 @@ pub(crate) struct Sealed {...@@ -78,34 +71,14 @@ pub(crate) struct Sealed {
78 pub transaction: Option<Transaction>,71 pub transaction: Option<Transaction>,
79}72}
8073
81/// A conflict's resolution: the batch it holds and every later edit drop their ops on
82/// `space`; `Mine` first rewrites the rebased page to `page`, the local one unless given.
83pub(crate) struct Resolve {
84 pub first: u64,
85 pub space: ExGuid,
86 pub keep: Resolution,
87 pub page: Option<Page>,
88}
89
90pub(crate) enum Rebased {
91 /// The queue now applies to the image, which is the base; these pages changed.
92 Applied { changed: Vec<ExGuid> },
93 /// The batch holding edit `id` no longer applies; nothing changed but the record.
94 Conflict {
95 id: u64,
96 space: ExGuid,
97 kind: ConflictKind,
98 },
99}
100
101impl Request {74impl Request {
102 fn fail(self, message: &str) {75 fn fail(self, message: &str) {
103 let error = || io::Error::other(message.to_owned()).into();76 let error = || io::Error::other(message.to_owned()).into();
104 match self {77 match self {
105 Self::Apply { reply, .. } => reply(Err(error())),78 Self::Apply { reply, .. } => reply(Err(error())),
106 Self::Save { reply, .. } => reply(Err(error())),
107 Self::Page { reply, .. } => reply(Err(error())),79 Self::Page { reply, .. } => reply(Err(error())),
108 Self::Pages { reply } => reply(Err(error())),80 Self::Pages { reply } => reply(Err(error())),
81 Self::Conflicts { reply } => reply(Err(error())),
109 Self::Flush { reply } => reply(Err(error())),82 Self::Flush { reply } => reply(Err(error())),
110 Self::Seal { reply } => reply(Err(error())),83 Self::Seal { reply } => reply(Err(error())),
111 Self::Rebase { reply, .. } => reply(Err(error())),84 Self::Rebase { reply, .. } => reply(Err(error())),
...@@ -244,8 +217,7 @@ fn fail(error: crate::Error, backlog: VecDeque<Request>, requests: mpsc::Receive...@@ -244,8 +217,7 @@ fn fail(error: crate::Error, backlog: VecDeque<Request>, requests: mpsc::Receive
244enum Job {217enum Job {
245 Rebase {218 Rebase {
246 image: Option<Vec<u8>>,219 image: Option<Vec<u8>>,
247 resolve: Option<Resolve>,220 reply: Reply<Vec<ExGuid>>,
248 reply: Reply<Rebased>,
249 },221 },
250 Reopen {222 Reopen {
251 reply: Reply<()>,223 reply: Reply<()>,
...@@ -253,18 +225,14 @@ enum Job {...@@ -253,18 +225,14 @@ enum Job {
253}225}
254226
255/// Runs `job`, rereads the section and takes the requests over from the thread that227/// Runs `job`, rereads the section and takes the requests over from the thread that
256/// started it; a rebase that found a conflict changed nothing that thread serves.228/// started it; a failed rebase changed nothing that thread serves.
257fn build(shared: Arc<Thread>, job: Job, signal: mpsc::Sender<Request>) {229fn build(shared: Arc<Thread>, job: Job, signal: mpsc::Sender<Request>) {
258 let answer: Box<dyn FnOnce() + Send> = match job {230 let answer: Box<dyn FnOnce() + Send> = match job {
259 Job::Reopen { reply } => Box::new(move || reply(Ok(()))),231 Job::Reopen { reply } => Box::new(move || reply(Ok(()))),
260 Job::Rebase {232 Job::Rebase { image, reply } => match rebase(&shared.connection, image) {
261 image,233 Ok(changed) => Box::new(move || reply(Ok(changed))),
262 resolve,234 Err(error) => {
263 reply,235 reply(Err(error));
264 } => match rebase(&shared.connection, image, resolve) {
265 Ok(applied @ Rebased::Applied { .. }) => Box::new(move || reply(Ok(applied))),
266 other => {
267 reply(other);
268 let _ = signal.send(Request::Resume);236 let _ = signal.send(Request::Resume);
269 return;237 return;
270 }238 }
...@@ -298,21 +266,7 @@ fn build(shared: Arc<Thread>, job: Job, signal: mpsc::Sender<Request>) {...@@ -298,21 +266,7 @@ fn build(shared: Arc<Thread>, job: Job, signal: mpsc::Sender<Request>) {
298struct Accepted {266struct Accepted {
299 author: String,267 author: String,
300 edit: Edit,268 edit: Edit,
301 done: Done,269 reply: Reply<u64>,
302}
303
304enum Done {
305 Apply(Reply<u64>),
306 Save(Reply<Option<Save>>),
307}
308
309impl Done {
310 fn reply(self, written: Result<u64>) {
311 match self {
312 Self::Apply(reply) => reply(written),
313 Self::Save(reply) => reply(written.map(|id| Some(Save::Queued(id)))),
314 }
315 }
316}270}
317271
318struct Working<'a> {272struct Working<'a> {
...@@ -321,8 +275,6 @@ struct Working<'a> {...@@ -321,8 +275,6 @@ struct Working<'a> {
321 open: Option<i64>,275 open: Option<i64>,
322 /// Spaces the open batch's edits change.276 /// Spaces the open batch's edits change.
323 touched: BTreeSet<ExGuid>,277 touched: BTreeSet<ExGuid>,
324 /// The last whole-page save: its space, the page it was given, and the page as stored.
325 saved: Option<(ExGuid, Page, Page)>,
326}278}
327279
328impl<'a> Working<'a> {280impl<'a> Working<'a> {
...@@ -332,7 +284,6 @@ impl<'a> Working<'a> {...@@ -332,7 +284,6 @@ impl<'a> Working<'a> {
332 section,284 section,
333 open,285 open,
334 touched,286 touched,
335 saved: None,
336 })287 })
337 }288 }
338289
...@@ -367,6 +318,9 @@ impl<'a> Working<'a> {...@@ -367,6 +318,9 @@ impl<'a> Working<'a> {
367 reply(self.section.page(space).map_err(Into::into))318 reply(self.section.page(space).map_err(Into::into))
368 }319 }
369 Request::Pages { reply } => reply(self.section.pages().map_err(Into::into)),320 Request::Pages { reply } => reply(self.section.pages().map_err(Into::into)),
321 Request::Conflicts { reply } => {
322 reply(self.section.conflicts().map_err(Into::into))
323 }
370 Request::Handover(to) => {324 Request::Handover(to) => {
371 let mut waiting = held.take().unwrap_or_default();325 let mut waiting = held.take().unwrap_or_default();
372 waiting.extend(burst.drain(..).chain(later.drain(..)));326 waiting.extend(burst.drain(..).chain(later.drain(..)));
...@@ -386,86 +340,9 @@ impl<'a> Working<'a> {...@@ -386,86 +340,9 @@ impl<'a> Working<'a> {
386 edit,340 edit,
387 reply,341 reply,
388 } => self342 } => self
389 .accept(343 .accept(author, edit, reply, &mut accepted, connection, worker)
390 author,
391 edit,
392 Done::Apply(reply),
393 &mut accepted,
394 connection,
395 worker,
396 )
397 .err()344 .err()
398 .unwrap_or(false),345 .unwrap_or(false),
399 Request::Save {
400 space,
401 before,
402 mut after,
403 author,
404 reply,
405 } => {
406 // Saves that continue one another are written once.
407 let mut replies = vec![reply];
408 while let Some(Request::Save {
409 space: next,
410 before: continued,
411 author: by,
412 ..
413 }) = burst.front()
414 && (*next, continued, by) == (space, &after, &author)
415 {
416 let Some(Request::Save {
417 after: later,
418 reply,
419 ..
420 }) = burst.pop_front()
421 else {
422 unreachable!()
423 };
424 after = later;
425 replies.push(reply);
426 }
427 let last = replies.pop().unwrap();
428 for folded in replies {
429 folded(Ok(None));
430 }
431 match self.save(space, &before, &after) {
432 Ok(Some(edit)) => match self.accept(
433 author,
434 edit,
435 Done::Save(last),
436 &mut accepted,
437 connection,
438 worker,
439 ) {
440 Ok(()) => {
441 self.saved = self
442 .section
443 .page(space)
444 .ok()
445 .map(|stored| (space, after, stored));
446 false
447 }
448 Err(broken) => broken,
449 },
450 Ok(None) => {
451 self.saved = self
452 .section
453 .page(space)
454 .ok()
455 .map(|stored| (space, after, stored));
456 last(Ok(Some(Save::Unchanged)));
457 false
458 }
459 Err(Stale::Stale) => {
460 last(Ok(Some(Save::Stale)));
461 false
462 }
463 Err(Stale::Error(error)) => {
464 last(Err(error));
465 false
466 }
467 }
468 }
469 Request::Page { space, reply } => {346 Request::Page { space, reply } => {
470 reply(self.section.page(space).map_err(Into::into));347 reply(self.section.page(space).map_err(Into::into));
471 false348 false
...@@ -474,6 +351,10 @@ impl<'a> Working<'a> {...@@ -474,6 +351,10 @@ impl<'a> Working<'a> {
474 reply(self.section.pages().map_err(Into::into));351 reply(self.section.pages().map_err(Into::into));
475 false352 false
476 }353 }
354 Request::Conflicts { reply } => {
355 reply(self.section.conflicts().map_err(Into::into));
356 false
357 }
477 request @ (Request::Flush { .. } | Request::Seal { .. })358 request @ (Request::Flush { .. } | Request::Seal { .. })
478 if !burst.is_empty() =>359 if !burst.is_empty() =>
479 {360 {
...@@ -502,50 +383,15 @@ impl<'a> Working<'a> {...@@ -502,50 +383,15 @@ impl<'a> Working<'a> {
502 failed383 failed
503 }384 }
504 }385 }
505 Request::Rebase {386 Request::Rebase { image, reply } => {
506 image,387 if self.flush(connection, worker, &mut accepted) {
507 mut resolve,388 held = self
508 reply,389 .rebuild(shared, Job::Rebase { image, reply })
509 } => {390 .then(VecDeque::new);
510 let written = self.flush(connection, worker, &mut accepted);391 false
511 // The page a `Mine` resolution keeps is read here, before the392 } else {
512 // rebuild; the root space lists pages and holds none.393 reply(Err(io::Error::other("The queue could not be written").into()));
513 let root = self.section.root();394 true
514 let kept = match &mut resolve {
515 Some(Resolve {
516 keep: Resolution::Mine,
517 page: page @ None,
518 space,
519 ..
520 }) if *space != root => {
521 self.section.page(*space).map(|local| *page = Some(local))
522 }
523 _ => Ok(()),
524 };
525 match (written, kept) {
526 (true, Ok(())) => {
527 held = self
528 .rebuild(
529 shared,
530 Job::Rebase {
531 image,
532 resolve,
533 reply,
534 },
535 )
536 .then(VecDeque::new);
537 false
538 }
539 (_, Err(error)) => {
540 reply(Err(error.into()));
541 !written
542 }
543 (false, _) => {
544 reply(Err(
545 io::Error::other("The queue could not be written").into()
546 ));
547 true
548 }
549 }395 }
550 }396 }
551 Request::Reopen { reply } => {397 Request::Reopen { reply } => {
...@@ -598,7 +444,7 @@ impl<'a> Working<'a> {...@@ -598,7 +444,7 @@ impl<'a> Working<'a> {
598 &mut self,444 &mut self,
599 author: String,445 author: String,
600 edit: Edit,446 edit: Edit,
601 done: Done,447 reply: Reply<u64>,
602 accepted: &mut Vec<Accepted>,448 accepted: &mut Vec<Accepted>,
603 connection: &Mutex<Connection>,449 connection: &Mutex<Connection>,
604 worker: &Worker,450 worker: &Worker,
...@@ -607,7 +453,11 @@ impl<'a> Working<'a> {...@@ -607,7 +453,11 @@ impl<'a> Working<'a> {
607 Ok(()) => {453 Ok(()) => {
608 self.touched454 self.touched
609 .extend(queue::spaces(&edit, self.section.root()));455 .extend(queue::spaces(&edit, self.section.root()));
610 accepted.push(Accepted { author, edit, done });456 accepted.push(Accepted {
457 author,
458 edit,
459 reply,
460 });
611 Ok(())461 Ok(())
612 }462 }
613 Err(error) => {463 Err(error) => {
...@@ -615,40 +465,12 @@ impl<'a> Working<'a> {...@@ -615,40 +465,12 @@ impl<'a> Working<'a> {
615 if broken {465 if broken {
616 self.flush(connection, worker, accepted);466 self.flush(connection, worker, accepted);
617 }467 }
618 done.reply(Err(error.into()));468 reply(Err(error.into()));
619 Err(broken)469 Err(broken)
620 }470 }
621 }471 }
622 }472 }
623473
624 /// The edit that takes the stored page from `before` to `after`. A save continuing
625 /// the previous one finds the page as that one stored it, which may read back
626 /// normalized.
627 fn save(
628 &mut self,
629 space: ExGuid,
630 before: &Page,
631 after: &Page,
632 ) -> std::result::Result<Option<Edit>, Stale> {
633 let current = self
634 .section
635 .page(space)
636 .map_err(|error| Stale::Error(error.into()))?;
637 let expected = match &self.saved {
638 Some((saved, given, stored)) if *saved == space && given == before => stored,
639 _ => before,
640 };
641 if current != *expected {
642 return Err(Stale::Stale);
643 }
644 let ops = onestore::op::lower_page(&current, after)
645 .map_err(|error| Stale::Error(OpError::Unsupported(error.message).into()))?;
646 Ok((!ops.is_empty()).then(|| Edit {
647 at: crate::now(),
648 ops: ops.into_iter().map(|op| Op::Page { space, op }).collect(),
649 }))
650 }
651
652 /// Writes the accepted edits in one transaction, then answers their senders; false474 /// Writes the accepted edits in one transaction, then answers their senders; false
653 /// when the write failed and the section holds edits the queue lacks.475 /// when the write failed and the section holds edits the queue lacks.
654 fn flush(476 fn flush(
...@@ -689,15 +511,14 @@ impl<'a> Working<'a> {...@@ -689,15 +511,14 @@ impl<'a> Working<'a> {
689 match written {511 match written {
690 Ok(ids) => {512 Ok(ids) => {
691 for (edit, id) in accepted.drain(..).zip(ids) {513 for (edit, id) in accepted.drain(..).zip(ids) {
692 edit.done.reply(Ok(id));514 (edit.reply)(Ok(id));
693 }515 }
694 crate::wake(worker);516 crate::wake(worker);
695 }517 }
696 Err(error) => {518 Err(error) => {
697 let message = error.to_string();519 let message = error.to_string();
698 for edit in accepted.drain(..) {520 for edit in accepted.drain(..) {
699 edit.done521 (edit.reply)(Err(io::Error::other(message.clone()).into()));
700 .reply(Err(io::Error::other(message.clone()).into()));
701 }522 }
702 }523 }
703 }524 }
...@@ -712,7 +533,7 @@ impl<'a> Working<'a> {...@@ -712,7 +533,7 @@ impl<'a> Working<'a> {
712 {533 {
713 let connection = lock(connection)?;534 let connection = lock(connection)?;
714 let waiting: bool = connection.query_row(535 let waiting: bool = connection.query_row(
715 "SELECT EXISTS(SELECT 1 FROM batches WHERE sealed IS NOT NULL OR conflict IS NOT NULL)",536 "SELECT EXISTS(SELECT 1 FROM batches WHERE sealed IS NOT NULL)",
716 [],537 [],
717 |row| row.get(0),538 |row| row.get(0),
718 )?;539 )?;
...@@ -746,12 +567,10 @@ impl<'a> Working<'a> {...@@ -746,12 +567,10 @@ impl<'a> Working<'a> {
746 }567 }
747}568}
748569
749/// Replays every queued edit on `image`, keeping it as the base when all apply.570/// Replays every queued edit on `image`, which becomes the base; each page whose local
750fn rebase(571/// version the remote could not take gains a conflict page holding it, queued as a new edit.
751 connection: &Mutex<Connection>,572/// Returns the pages the remote changed.
752 image: Option<Vec<u8>>,573fn rebase(connection: &Mutex<Connection>, image: Option<Vec<u8>>) -> Result<Vec<ExGuid>> {
753 resolve: Option<Resolve>,
754) -> Result<Rebased> {
755 let (base_image, image, edits) = {574 let (base_image, image, edits) = {
756 let connection = lock(connection)?;575 let connection = lock(connection)?;
757 if connection.query_row(576 if connection.query_row(
...@@ -777,117 +596,91 @@ fn rebase(...@@ -777,117 +596,91 @@ fn rebase(
777 (base_image, image, queue::load(&connection, None)?)596 (base_image, image, queue::load(&connection, None)?)
778 };597 };
779 let old_arena = Arena::default();598 let old_arena = Arena::default();
780 let mut old = Section::open(&old_arena, base_image.clone())?;599 let mut old = Section::open(&old_arena, base_image)?;
781 let before: BTreeMap<ExGuid, ExGuid> = old.revisions().collect();600 let before: BTreeMap<ExGuid, ExGuid> = old.revisions().collect();
782 let mut after: BTreeMap<ExGuid, ExGuid>;601 let remote_arena = Arena::default();
602 let remote = Section::open(&remote_arena, image.clone())?;
603 if old.root() != remote.root() {
604 return Err(io::Error::new(
605 io::ErrorKind::InvalidInput,
606 "Remote snapshot belongs to another document",
607 )
608 .into());
609 }
610 let after: BTreeMap<ExGuid, ExGuid> = remote.revisions().collect();
611 // The pages as the queue leaves them, read once a page conflicts: O(section).
612 let local_arena = Arena::default();
613 let mut local = None;
783 // Pages the remote already holds as the local edits leave them: their ops are done.614 // Pages the remote already holds as the local edits leave them: their ops are done.
784 let mut converged = BTreeSet::new();615 let mut converged = BTreeSet::new();
785 let outcome = loop {616 let (rewritten, added) = loop {
786 let arena = Arena::default();617 let arena = Arena::default();
787 let mut new = Section::open(&arena, image.clone())?;618 let mut new = Section::open(&arena, image.clone())?;
788 if old.root() != new.root() {619 let merged = merge::rebase(&mut old, &mut new, &edits, &converged)?;
789 return Err(io::Error::new(620 if !merged.conflicts.is_empty() && local.is_none() {
790 io::ErrorKind::InvalidInput,621 local = Some(replay(&local_arena, &*lock(connection)?)?.0);
791 "Remote snapshot belongs to another document",
792 )
793 .into());
794 }622 }
795 after = new.revisions().collect();623 let pages: BTreeMap<ExGuid, Page> = merged
796 let outcome = merge::rebase(624 .conflicts
797 &mut old,625 .keys()
798 &mut new,626 .filter_map(|space| Some((*space, local.as_ref()?.page(*space).ok()?)))
799 &edits,627 .collect();
800 &converged,628 let settled: Vec<ExGuid> = pages
801 resolve.as_ref().map(|resolve| merge::Resolving {629 .iter()
802 first: resolve.first,630 .filter(|(space, page)| remote.page(**space).is_ok_and(|remote| remote == **page))
803 space: resolve.space,631 .map(|(space, _)| *space)
804 mine: resolve.keep == Resolution::Mine,632 .collect();
805 target: resolve.page.as_ref(),633 if !settled.is_empty() {
806 }),634 converged.extend(settled);
807 )?;
808 if let merge::Outcome::Conflict { space, .. } = &outcome
809 && !converged.contains(space)
810 && let Some(local) = local_page(connection, *space)?
811 && Section::open(&Arena::default(), image.clone())?
812 .page(*space)
813 .is_ok_and(|remote| remote == local)
814 {
815 converged.insert(*space);
816 continue;635 continue;
817 }636 }
818 break outcome;637 let mut added = Vec::new();
819 };638 for (space, (author, objects)) in &merged.conflicts {
820 let mut connection = lock(connection)?;639 // A page the queue went on to delete keeps no version of its own.
821 let transaction = connection.transaction_with_behavior(TransactionBehavior::Immediate)?;640 let (Some(page), Some(local)) = (pages.get(space), local.as_mut()) else {
822 let rebased = match outcome {641 continue;
823 merge::Outcome::Conflict { id, space, kind } => {642 };
824 transaction.execute("UPDATE batches SET conflict=NULL, space=NULL", [])?;643 let edit = merge::conflict_page(
825 transaction.execute(644 &mut new,
826 "UPDATE batches SET conflict=?1, space=?2 WHERE id=(SELECT batch FROM edits WHERE id=?3)",645 &mut old,
827 params![kind as i64, space.to_string(), signed(id)?],646 local,
647 *space,
648 page,
649 author,
650 objects,
651 crate::now(),
828 )?;652 )?;
829 if image == base_image {653 added.push((author.clone(), edit));
830 base::clear(&transaction, base::Image::Remote)?;
831 } else {
832 base::write(&transaction, base::Image::Remote, &image)?;
833 }
834 // A conflict is reported under its batch's newest edit.
835 let id = crate::unsigned(transaction.query_row(
836 "SELECT max(id) FROM edits WHERE batch=(SELECT batch FROM edits WHERE id=?1)",
837 [signed(id)?],
838 |row| row.get(0),
839 )?)?;
840 Rebased::Conflict { id, space, kind }
841 }
842 merge::Outcome::Applied(rewritten) => {
843 base::write(&transaction, base::Image::Base, &image)?;
844 base::clear(&transaction, base::Image::Remote)?;
845 transaction.execute("INSERT INTO batches DEFAULT VALUES", [])?;
846 let batch = transaction.last_insert_rowid();
847 transaction.execute("UPDATE edits SET batch=?1", [batch])?;
848 transaction.execute("DELETE FROM batches WHERE id<>?1", [batch])?;
849 for (id, edit) in rewritten {
850 queue::rewrite(&transaction, id, &edit)?;
851 }
852 if transaction
853 .query_row("SELECT count(*) FROM edits", [], |row| row.get::<_, i64>(0))?
854 == 0
855 {
856 transaction.execute("DELETE FROM batches", [])?;
857 }
858 queue::collect(&transaction)?;
859 let mut changed: BTreeSet<ExGuid> = before
860 .iter()
861 .filter(|(space, rid)| after.get(space) != Some(rid))
862 .map(|(space, _)| *space)
863 .chain(
864 after
865 .keys()
866 .filter(|space| !before.contains_key(space))
867 .copied(),
868 )
869 .collect();
870 changed.extend(resolve.as_ref().map(|resolve| resolve.space));
871 changed.extend(converged);
872 Rebased::Applied {
873 changed: changed.into_iter().collect(),
874 }
875 }654 }
655 break (merged.rewritten, added);
876 };656 };
657 let mut connection = lock(connection)?;
658 let transaction = connection.transaction_with_behavior(TransactionBehavior::Immediate)?;
659 base::write(&transaction, base::Image::Base, &image)?;
660 base::clear(&transaction, base::Image::Remote)?;
661 transaction.execute("INSERT INTO batches DEFAULT VALUES", [])?;
662 let batch = transaction.last_insert_rowid();
663 transaction.execute("UPDATE edits SET batch=?1", [batch])?;
664 transaction.execute("DELETE FROM batches WHERE id<>?1", [batch])?;
665 for (id, edit) in rewritten {
666 queue::rewrite(&transaction, id, &edit)?;
667 }
668 for (author, edit) in &added {
669 queue::insert(&transaction, None, batch, author, edit)?;
670 }
671 if transaction.query_row("SELECT count(*) FROM edits", [], |row| row.get::<_, i64>(0))? == 0 {
672 transaction.execute("DELETE FROM batches", [])?;
673 }
674 queue::collect(&transaction)?;
877 transaction.commit()?;675 transaction.commit()?;
878 Ok(rebased)676 let mut changed: BTreeSet<ExGuid> = before
879}677 .iter()
880678 .filter(|(space, rid)| after.get(space) != Some(rid))
881/// A page as the queued edits leave it, read from the cache: O(section).679 .map(|(space, _)| *space)
882fn local_page(connection: &Mutex<Connection>, space: ExGuid) -> Result<Option<Page>> {680 .chain(after.keys().filter(|space| !before.contains_key(space)).copied())
883 let arena = Arena::default();681 .collect();
884 let (section, ..) = replay(&arena, &*lock(connection)?)?;682 changed.extend(converged);
885 Ok(section.page(space).ok())683 Ok(changed.into_iter().collect())
886}
887
888enum Stale {
889 Stale,
890 Error(crate::Error),
891}684}
892685
893/// Batches in order with their sealed transactions: `None` while open, `Some(None)` when686/// Batches in order with their sealed transactions: `None` while open, `Some(None)` when
crates/notebook/tests/assets.rs+16-9
...@@ -15,6 +15,7 @@ use std::{...@@ -15,6 +15,7 @@ use std::{
15mod model_ops;15mod model_ops;
16#[path = "support/server.rs"]16#[path = "support/server.rs"]
17mod server;17mod server;
18use server::{remote_snapshot, snapshot};
1819
19const FIXTURE: &str = "../../corpus/native-external-assets/notebook";20const FIXTURE: &str = "../../corpus/native-external-assets/notebook";
2021
...@@ -86,8 +87,8 @@ impl notebook::discover::Source for Payload {...@@ -86,8 +87,8 @@ impl notebook::discover::Source for Payload {
86fn downloaded_media_survives_reopen_and_recovery_with_the_queue_intact() {87fn downloaded_media_survives_reopen_and_recovery_with_the_queue_intact() {
87 let directory = tempfile::tempdir().unwrap();88 let directory = tempfile::tempdir().unwrap();
88 let cache = queued_cache(directory.path());89 let cache = queued_cache(directory.path());
89 let working = cache.snapshot().unwrap();90 let working = snapshot(&cache);
90 let remote = cache.remote_snapshot().unwrap();91 let remote = remote_snapshot(&cache);
91 let pending = cache.pending().unwrap();92 let pending = cache.pending().unwrap();
92 assert_eq!(93 assert_eq!(
93 pending.iter().map(|edit| edit.id).collect::<Vec<_>>(),94 pending.iter().map(|edit| edit.id).collect::<Vec<_>>(),
...@@ -118,10 +119,10 @@ fn downloaded_media_survives_reopen_and_recovery_with_the_queue_intact() {...@@ -118,10 +119,10 @@ fn downloaded_media_survives_reopen_and_recovery_with_the_queue_intact() {
118 (2, 1024)119 (2, 1024)
119 );120 );
120 assert_eq!(121 assert_eq!(
121 server::pages(&cache.snapshot().unwrap()),122 server::pages(&snapshot(&cache)),
122 server::pages(&working)123 server::pages(&working)
123 );124 );
124 assert_eq!(cache.remote_snapshot().unwrap(), remote);125 assert_eq!(remote_snapshot(&cache), remote);
125 assert_eq!(cache.pending().unwrap(), pending);126 assert_eq!(cache.pending().unwrap(), pending);
126 assert_eq!(cache.status(1).unwrap(), uncertain);127 assert_eq!(cache.status(1).unwrap(), uncertain);
127 cache128 cache
...@@ -227,7 +228,7 @@ fn download_network_wait_does_not_block_local_edits() {...@@ -227,7 +228,7 @@ fn download_network_wait_does_not_block_local_edits() {
227 .unwrap()228 .unwrap()
228 });229 });
229 waiting.recv_timeout(Duration::from_secs(5)).unwrap();230 waiting.recv_timeout(Duration::from_secs(5)).unwrap();
230 let text = first_text(&cache.snapshot().unwrap());231 let text = first_text(&snapshot(&cache));
231 let id = model_ops::save(&cache, text, |page| {232 let id = model_ops::save(&cache, text, |page| {
232 model_ops::replace_text(page, text, 0..0, "during download ")233 model_ops::replace_text(page, text, 0..0, "during download ")
233 })234 })
...@@ -272,12 +273,18 @@ fn a_native_refresh_removing_the_reference_rejects_an_inflight_download() {...@@ -272,12 +273,18 @@ fn a_native_refresh_removing_the_reference_rejects_an_inflight_download() {
272 struct Native;273 struct Native;
273 impl notebook::Remote for Native {274 impl notebook::Remote for Native {
274 fn read(&mut self) -> io::Result<Vec<u8>> {275 fn read(&mut self) -> io::Result<Vec<u8>> {
275 fs::read("../../corpus/native-external-assets/native/synthetic.one")276 let mut image = fs::read("../../corpus/native-external-assets/native/synthetic.one")?;
277 // The fixture keeps the old header; a native commit writes a new file version.
278 image[212] ^= 1;
279 Ok(image)
276 }280 }
277 fn publish(&mut self, _: &onestore::Transaction) -> Result<(), onestore::CommitError> {281 fn publish(&mut self, _: &onestore::Transaction) -> Result<(), onestore::CommitError> {
278 panic!("Unexpected publication")282 panic!("Unexpected publication")
279 }283 }
280 fn confirm(&mut self, _: &[u8]) -> Result<(), onestore::CommitError> {284 fn stamp(&mut self) -> io::Result<onestore::Stamp> {
285 onestore::Stamp::of(&self.read()?).map_err(io::Error::other)
286 }
287 fn confirm(&mut self, _: &onestore::Stamp) -> Result<(), onestore::CommitError> {
281 panic!("Unexpected confirmation")288 panic!("Unexpected confirmation")
282 }289 }
283 }290 }
...@@ -301,8 +308,8 @@ fn a_native_refresh_removing_the_reference_rejects_an_inflight_download() {...@@ -301,8 +308,8 @@ fn a_native_refresh_removing_the_reference_rejects_an_inflight_download() {
301 );308 );
302 assert!(cache.cached_asset(&name, 1024).unwrap().is_none());309 assert!(cache.cached_asset(&name, 1024).unwrap().is_none());
303 assert_eq!(310 assert_eq!(
304 cache.snapshot().unwrap(),311 snapshot(&cache),
305 fs::read("../../corpus/native-external-assets/native/synthetic.one").unwrap()312 notebook::Remote::read(&mut Native).unwrap()
306 );313 );
307}314}
308315
crates/notebook/tests/cache.rs+22-22
...@@ -1,3 +1,5 @@...@@ -1,3 +1,5 @@
1#[path = "../../onestore/tests/support/ops.rs"]
2mod ops;
1use notebook::{Error, Replica};3use notebook::{Error, Replica};
2use onestore::{4use onestore::{
3 ExGuid, RevisionIndex, Store,5 ExGuid, RevisionIndex, Store,
...@@ -17,6 +19,7 @@ use std::{...@@ -17,6 +19,7 @@ use std::{
17mod model_ops;19mod model_ops;
18#[path = "support/server.rs"]20#[path = "support/server.rs"]
19mod server;21mod server;
22use server::snapshot;
2023
21fn target(source: &[u8]) -> (ExGuid, ExGuid, String) {24fn target(source: &[u8]) -> (ExGuid, ExGuid, String) {
22 let store = Store::parse(source).unwrap();25 let store = Store::parse(source).unwrap();
...@@ -60,7 +63,7 @@ fn cache_reopen_preserves_the_base_and_queued_edits() {...@@ -60,7 +63,7 @@ fn cache_reopen_preserves_the_base_and_queued_edits() {
60 let path = dir.path().join("section.sqlite");63 let path = dir.path().join("section.sqlite");
61 let source = onestore::create_section("section.one", "café 🦀", "Fixture").unwrap();64 let source = onestore::create_section("section.one", "café 🦀", "Fixture").unwrap();
62 let replica = Replica::create(&path, &source).unwrap();65 let replica = Replica::create(&path, &source).unwrap();
63 assert_eq!(replica.snapshot().unwrap(), source);66 assert_eq!(snapshot(&replica), source);
64 assert!(replica.pending().unwrap().is_empty());67 assert!(replica.pending().unwrap().is_empty());
65 let (sid, oid, _) = target(&source);68 let (sid, oid, _) = target(&source);
66 let unchanged = |page: &mut onestore::page::Page| {69 let unchanged = |page: &mut onestore::page::Page| {
...@@ -73,7 +76,7 @@ fn cache_reopen_preserves_the_base_and_queued_edits() {...@@ -73,7 +76,7 @@ fn cache_reopen_preserves_the_base_and_queued_edits() {
73 })76 })
74 .unwrap()77 .unwrap()
75 .unwrap();78 .unwrap();
76 assert_eq!(target(&replica.snapshot().unwrap()).2, "café 🐈 日本語");79 assert_eq!(target(&snapshot(&replica)).2, "café 🐈 日本語");
77 let edits = replica.pending().unwrap();80 let edits = replica.pending().unwrap();
78 assert_eq!(edits.len(), 1);81 assert_eq!(edits.len(), 1);
79 assert_eq!(82 assert_eq!(
...@@ -99,14 +102,14 @@ fn cache_reopen_preserves_the_base_and_queued_edits() {...@@ -99,14 +102,14 @@ fn cache_reopen_preserves_the_base_and_queued_edits() {
99 "closing checkpoints and removes the log"102 "closing checkpoints and removes the log"
100 );103 );
101 let replica = Replica::open(&path).unwrap();104 let replica = Replica::open(&path).unwrap();
102 assert_eq!(target(&replica.snapshot().unwrap()).2, "café 🐈 日本語");105 assert_eq!(target(&snapshot(&replica)).2, "café 🐈 日本語");
103 assert_eq!(replica.pending().unwrap(), edits);106 assert_eq!(replica.pending().unwrap(), edits);
104 let second = replica107 let second = replica
105 .apply(model_ops::AUTHOR, typed(sid, oid, "Recovered "))108 .apply(model_ops::AUTHOR, typed(sid, oid, "Recovered "))
106 .unwrap();109 .unwrap();
107 assert!(second > first);110 assert!(second > first);
108 assert_eq!(111 assert_eq!(
109 target(&replica.snapshot().unwrap()).2,112 target(&snapshot(&replica)).2,
110 "Recovered café 🐈 日本語"113 "Recovered café 🐈 日本語"
111 );114 );
112 assert_eq!(replica.pending().unwrap().len(), 2);115 assert_eq!(replica.pending().unwrap().len(), 2);
...@@ -160,7 +163,7 @@ fn refused_edits_and_failed_writes_leave_the_queue_as_it_was() {...@@ -160,7 +163,7 @@ fn refused_edits_and_failed_writes_leave_the_queue_as_it_was() {
160 replica.apply("Author", typed(ExGuid::default(), oid, "Elsewhere ")),163 replica.apply("Author", typed(ExGuid::default(), oid, "Elsewhere ")),
161 Err(Error::Rejected(_))164 Err(Error::Rejected(_))
162 ));165 ));
163 assert_eq!(replica.snapshot().unwrap(), source);166 assert_eq!(snapshot(&replica), source);
164 assert!(replica.pending().unwrap().is_empty());167 assert!(replica.pending().unwrap().is_empty());
165 drop(replica);168 drop(replica);
166 let connection = rusqlite::Connection::open(&path).unwrap();169 let connection = rusqlite::Connection::open(&path).unwrap();
...@@ -168,10 +171,10 @@ fn refused_edits_and_failed_writes_leave_the_queue_as_it_was() {...@@ -168,10 +171,10 @@ fn refused_edits_and_failed_writes_leave_the_queue_as_it_was() {
168 drop(connection);171 drop(connection);
169 let replica = Replica::open(&path).unwrap();172 let replica = Replica::open(&path).unwrap();
170 assert!(replica.apply("Author", typed(sid, oid, "lost? ")).is_err());173 assert!(replica.apply("Author", typed(sid, oid, "lost? ")).is_err());
171 assert_eq!(replica.snapshot().unwrap(), source);174 assert_eq!(snapshot(&replica), source);
172 drop(replica);175 drop(replica);
173 let replica = Replica::open(&path).unwrap();176 let replica = Replica::open(&path).unwrap();
174 assert_eq!(replica.snapshot().unwrap(), source);177 assert_eq!(snapshot(&replica), source);
175 assert!(replica.pending().unwrap().is_empty());178 assert!(replica.pending().unwrap().is_empty());
176}179}
177180
...@@ -230,7 +233,7 @@ fn concurrent_recovery_exports_capture_one_complete_acknowledged_queue() {...@@ -230,7 +233,7 @@ fn concurrent_recovery_exports_capture_one_complete_acknowledged_queue() {
230 }233 }
231 });234 });
232 assert_eq!(replica.pending().unwrap().len(), 60);235 assert_eq!(replica.pending().unwrap().len(), 60);
233 let content = target(&replica.snapshot().unwrap()).2;236 let content = target(&snapshot(&replica)).2;
234 for writer in 0..3 {237 for writer in 0..3 {
235 for edit in 0..20 {238 for edit in 0..20 {
236 assert_eq!(content.matches(&format!("[{writer}:{edit}] ")).count(), 1);239 assert_eq!(content.matches(&format!("[{writer}:{edit}] ")).count(), 1);
...@@ -255,7 +258,7 @@ fn ownership_and_foreign_file_rejection_preserve_existing_data() {...@@ -255,7 +258,7 @@ fn ownership_and_foreign_file_rejection_preserve_existing_data() {
255 assert!(258 assert!(
256 matches!(Replica::create(&path, &source), Err(Error::Io(error)) if error.kind() == ErrorKind::AlreadyExists)259 matches!(Replica::create(&path, &source), Err(Error::Io(error)) if error.kind() == ErrorKind::AlreadyExists)
257 );260 );
258 assert_eq!(replica.snapshot().unwrap(), source);261 assert_eq!(snapshot(&replica), source);
259 }262 }
260 drop(replica);263 drop(replica);
261 let current: u32 = rusqlite::Connection::open(&path)264 let current: u32 = rusqlite::Connection::open(&path)
...@@ -264,10 +267,10 @@ fn ownership_and_foreign_file_rejection_preserve_existing_data() {...@@ -264,10 +267,10 @@ fn ownership_and_foreign_file_rejection_preserve_existing_data() {
264 .unwrap();267 .unwrap();
265 for sql in [268 for sql in [
266 "PRAGMA application_id=0".to_owned(),269 "PRAGMA application_id=0".to_owned(),
267 // Schema 14 converts; older caches do not.270 // Schemas 14 and 15 convert; older caches do not.
268 format!(271 format!(
269 "PRAGMA application_id=1330529615; PRAGMA user_version={}",272 "PRAGMA application_id=1330529615; PRAGMA user_version={}",
270 current - 2273 current - 3
271 ),274 ),
272 format!("PRAGMA user_version={}", current + 1),275 format!("PRAGMA user_version={}", current + 1),
273 ] {276 ] {
...@@ -348,7 +351,7 @@ fn twelve_local_editors_queue_every_edit_once_in_order() {...@@ -348,7 +351,7 @@ fn twelve_local_editors_queue_every_edit_once_in_order() {
348 }351 }
349 let ids: BTreeSet<_> = outcomes.into_iter().flatten().collect();352 let ids: BTreeSet<_> = outcomes.into_iter().flatten().collect();
350 assert_eq!(ids.len(), 240);353 assert_eq!(ids.len(), 240);
351 let content = target(&replica.snapshot().unwrap()).2;354 let content = target(&snapshot(&replica)).2;
352 assert!(content.ends_with("Shared café 🦀"));355 assert!(content.ends_with("Shared café 🦀"));
353 for writer in 0..12 {356 for writer in 0..12 {
354 for edit in 0..20 {357 for edit in 0..20 {
...@@ -362,7 +365,7 @@ fn twelve_local_editors_queue_every_edit_once_in_order() {...@@ -362,7 +365,7 @@ fn twelve_local_editors_queue_every_edit_once_in_order() {
362 );365 );
363 drop(replica);366 drop(replica);
364 let reopened = Replica::open(&path).unwrap();367 let reopened = Replica::open(&path).unwrap();
365 assert_eq!(target(&reopened.snapshot().unwrap()).2, content);368 assert_eq!(target(&snapshot(&reopened)).2, content);
366 assert_eq!(reopened.pending().unwrap(), pending);369 assert_eq!(reopened.pending().unwrap(), pending);
367}370}
368371
...@@ -410,7 +413,7 @@ fn seeded_unicode_edits_and_restarts_match_an_independent_text_model() {...@@ -410,7 +413,7 @@ fn seeded_unicode_edits_and_restarts_match_an_independent_text_model() {
410 }413 }
411 text = expected;414 text = expected;
412 assert_eq!(415 assert_eq!(
413 target(&replica.snapshot().unwrap()).2,416 target(&snapshot(&replica)).2,
414 text,417 text,
415 "seed 911, step {step}"418 "seed 911, step {step}"
416 );419 );
...@@ -418,7 +421,7 @@ fn seeded_unicode_edits_and_restarts_match_an_independent_text_model() {...@@ -418,7 +421,7 @@ fn seeded_unicode_edits_and_restarts_match_an_independent_text_model() {
418 let pending = replica.pending().unwrap();421 let pending = replica.pending().unwrap();
419 drop(replica);422 drop(replica);
420 replica = Replica::open(&path).unwrap();423 replica = Replica::open(&path).unwrap();
421 assert_eq!(target(&replica.snapshot().unwrap()).2, text);424 assert_eq!(target(&snapshot(&replica)).2, text);
422 assert_eq!(replica.pending().unwrap(), pending);425 assert_eq!(replica.pending().unwrap(), pending);
423 }426 }
424 }427 }
...@@ -508,13 +511,10 @@ fn twelve_local_clients_preserve_inserted_identities_and_dependent_edits() {...@@ -508,13 +511,10 @@ fn twelve_local_clients_preserve_inserted_identities_and_dependent_edits() {
508 .flat_map(|(ids, _)| ids.iter().copied())511 .flat_map(|(ids, _)| ids.iter().copied())
509 .collect();512 .collect();
510 assert_eq!(ids.len(), 84);513 assert_eq!(ids.len(), 84);
511 let snapshot = cache.snapshot().unwrap();514 let image = snapshot(&cache);
512 drop(cache);515 drop(cache);
513 let cache = Replica::open(&path).unwrap();516 let cache = Replica::open(&path).unwrap();
514 assert_eq!(517 assert_eq!(server::pages(&snapshot(&cache)), server::pages(&image));
515 server::pages(&cache.snapshot().unwrap()),
516 server::pages(&snapshot)
517 );
518 assert_eq!(518 assert_eq!(
519 cache519 cache
520 .pending()520 .pending()
...@@ -524,7 +524,7 @@ fn twelve_local_clients_preserve_inserted_identities_and_dependent_edits() {...@@ -524,7 +524,7 @@ fn twelve_local_clients_preserve_inserted_identities_and_dependent_edits() {
524 .collect::<BTreeSet<_>>(),524 .collect::<BTreeSet<_>>(),
525 ids525 ids
526 );526 );
527 let store = Store::parse(&snapshot).unwrap();527 let store = Store::parse(&image).unwrap();
528 let index = RevisionIndex::parse(&store).unwrap();528 let index = RevisionIndex::parse(&store).unwrap();
529 let doc = Document::parse(&index).unwrap();529 let doc = Document::parse(&index).unwrap();
530 let s = &doc.spaces[&sid];530 let s = &doc.spaces[&sid];
...@@ -559,7 +559,7 @@ fn unrecognized_persisted_ops_are_rejected_without_dropping_fields() {...@@ -559,7 +559,7 @@ fn unrecognized_persisted_ops_are_rejected_without_dropping_fields() {
559 let path = directory.path().join("unknown.sqlite");559 let path = directory.path().join("unknown.sqlite");
560 let first = onestore::create_section("unknown.one", "Original", "Author").unwrap();560 let first = onestore::create_section("unknown.one", "Original", "Author").unwrap();
561 let second = onestore::PageCreation::new(None, Some("Second"), "Author").unwrap();561 let second = onestore::PageCreation::new(None, Some("Second"), "Author").unwrap();
562 let source = onestore::PreparedEdit::create_page(&first, &second)562 let source = ops::section_op(&first, SectionOp::Create(second.clone()))
563 .unwrap()563 .unwrap()
564 .as_bytes()564 .as_bytes()
565 .to_vec();565 .to_vec();
crates/notebook/tests/changes.rs+2-2
...@@ -24,9 +24,9 @@ fn external_structure_changes_are_reported_by_identity() {...@@ -24,9 +24,9 @@ fn external_structure_changes_are_reported_by_identity() {
24 let mut other = Notebook::open(&root, temporary.path().join("other")).unwrap();24 let mut other = Notebook::open(&root, temporary.path().join("other")).unwrap();
25 assert_eq!(watcher.refresh().unwrap(), []);25 assert_eq!(watcher.refresh().unwrap(), []);
2626
27 other.create_section("", "Second", "Author").unwrap();27 other.create_section("", "Second", &onestore::PageCreation::new(None, Some(""), "Author").unwrap()).unwrap();
28 other.create_group("", "Group").unwrap();28 other.create_group("", "Group").unwrap();
29 other.create_section("Group", "Inner", "Author").unwrap();29 other.create_section("Group", "Inner", &onestore::PageCreation::new(None, Some(""), "Author").unwrap()).unwrap();
30 assert_eq!(30 assert_eq!(
31 watcher.refresh().unwrap(),31 watcher.refresh().unwrap(),
32 [32 [
crates/notebook/tests/copy.rs+4-13
...@@ -1,7 +1,4 @@...@@ -1,7 +1,4 @@
1use notebook::{1use notebook::session::{Event, Section};
2 EditStatus,
3 session::{Event, Section},
4};
5use onestore::{2use onestore::{
6 RevisionIndex, Store,3 RevisionIndex, Store,
7 document::Document,4 document::Document,
...@@ -197,13 +194,7 @@ fn pages_copy_into_another_section_with_their_content_and_fresh_identities() {...@@ -197,13 +194,7 @@ fn pages_copy_into_another_section_with_their_content_and_fresh_identities() {
197 ]194 ]
198 );195 );
199 let deadline = Instant::now() + Duration::from_secs(120);196 let deadline = Instant::now() + Duration::from_secs(120);
200 while section197 while !section.pending().unwrap().is_empty() {
201 .queue()
202 .unwrap()
203 .iter()
204 .any(|edit| !matches!(edit.status, EditStatus::Published { .. }))
205 || !section.queue().unwrap().is_empty()
206 {
207 for event in section.events() {198 for event in section.events() {
208 if let Event::Failed(message) = event {199 if let Event::Failed(message) = event {
209 panic!("{message}");200 panic!("{message}");
...@@ -212,7 +203,7 @@ fn pages_copy_into_another_section_with_their_content_and_fresh_identities() {...@@ -212,7 +203,7 @@ fn pages_copy_into_another_section_with_their_content_and_fresh_identities() {
212 assert!(203 assert!(
213 Instant::now() < deadline,204 Instant::now() < deadline,
214 "publication did not finish: {:?}",205 "publication did not finish: {:?}",
215 section.queue().unwrap()206 section.pending().unwrap()
216 );207 );
217 std::thread::sleep(Duration::from_millis(50));208 std::thread::sleep(Duration::from_millis(50));
218 }209 }
...@@ -233,7 +224,7 @@ fn pages_copy_into_another_section_with_their_content_and_fresh_identities() {...@@ -233,7 +224,7 @@ fn pages_copy_into_another_section_with_their_content_and_fresh_identities() {
233 }224 }
234 section.delete_pages(&[expected.last().unwrap().0]).unwrap();225 section.delete_pages(&[expected.last().unwrap().0]).unwrap();
235 let deadline = Instant::now() + Duration::from_secs(60);226 let deadline = Instant::now() + Duration::from_secs(60);
236 while !section.queue().unwrap().is_empty() {227 while !section.pending().unwrap().is_empty() {
237 assert!(Instant::now() < deadline);228 assert!(Instant::now() < deadline);
238 std::thread::sleep(Duration::from_millis(50));229 std::thread::sleep(Duration::from_millis(50));
239 }230 }
crates/notebook/tests/migrate.rs+112-37
...@@ -1,8 +1,12 @@...@@ -1,8 +1,12 @@
1//! Converting schema-14 caches: whole-page edits become ops, verified page by page against1//! Converting schema-14 caches: whole-page edits become ops, verified page by page against
2//! the old working image, with a recovery archive first and nothing changed on a mismatch.2//! the old working image, with a recovery archive first and nothing changed on a mismatch;
3//! and schema-15 caches, whose batches lose the review conflict.
34
4use notebook::{ConflictKind, EditStatus, Replica, Resolution};5#[path = "../../onestore/tests/support/ops.rs"]
5use onestore::{ExGuid, PageCreation, PageEdit, PreparedEdit, page::Page};6mod ops;
7
8use notebook::{EditStatus, Replica};
9use onestore::{ExGuid, PageCreation, PageEdit, op::SectionOp, page::Page};
6use rusqlite::{Connection, params};10use rusqlite::{Connection, params};
7use serde_json::json;11use serde_json::json;
8use std::path::Path;12use std::path::Path;
...@@ -48,17 +52,17 @@ fn v14(path: &Path, base: &[u8], queue: &[Queued], working: Option<&[u8]>) -> Ve...@@ -48,17 +52,17 @@ fn v14(path: &Path, base: &[u8], queue: &[Queued], working: Option<&[u8]>) -> Ve
48 let (space, operation, next) = match entry {52 let (space, operation, next) = match entry {
49 Queued::Page(space, after) => {53 Queued::Page(space, after) => {
50 let before = model_ops::page_of(&image, *space);54 let before = model_ops::page_of(&image, *space);
51 let next = PreparedEdit::page(&image, *space, after, "Author").unwrap();55 let next = ops::saved(&image, *space, after).unwrap();
52 (56 (
53 *space,57 *space,
54 json!({"Page": {"before": before, "after": after, "author": "Author"}}),58 json!({"Page": {"before": before, "after": after, "author": "Author"}}),
55 next.as_bytes().to_vec(),59 next.as_slice().to_vec(),
56 )60 )
57 }61 }
58 Queued::Create(creation) => (62 Queued::Create(creation) => (
59 creation.space(),63 creation.space(),
60 json!({"CreatePage": creation}),64 json!({"CreatePage": creation}),
61 PreparedEdit::create_page(&image, creation)65 ops::section_op(&image, SectionOp::Create(creation.clone()))
62 .unwrap()66 .unwrap()
63 .as_bytes()67 .as_bytes()
64 .to_vec(),68 .to_vec(),
...@@ -71,7 +75,7 @@ fn v14(path: &Path, base: &[u8], queue: &[Queued], working: Option<&[u8]>) -> Ve...@@ -71,7 +75,7 @@ fn v14(path: &Path, base: &[u8], queue: &[Queued], working: Option<&[u8]>) -> Ve
71 (75 (
72 root(&image),76 root(&image),
73 json!({"Pages": {"edits": batch, "observed": observed}}),77 json!({"Pages": {"edits": batch, "observed": observed}}),
74 PreparedEdit::pages(&image, batch)78 ops::section_op(&image, SectionOp::Pages(batch.to_vec()))
75 .unwrap()79 .unwrap()
76 .as_bytes()80 .as_bytes()
77 .to_vec(),81 .to_vec(),
...@@ -80,7 +84,7 @@ fn v14(path: &Path, base: &[u8], queue: &[Queued], working: Option<&[u8]>) -> Ve...@@ -80,7 +84,7 @@ fn v14(path: &Path, base: &[u8], queue: &[Queued], working: Option<&[u8]>) -> Ve
80 Queued::Delete(spaces) => (84 Queued::Delete(spaces) => (
81 root(&image),85 root(&image),
82 json!({"DeletePages": spaces}),86 json!({"DeletePages": spaces}),
83 PreparedEdit::delete_pages_permanently(&image, spaces)87 ops::section_op(&image, SectionOp::Delete(spaces.to_vec()))
84 .unwrap()88 .unwrap()
85 .as_bytes()89 .as_bytes()
86 .to_vec(),90 .to_vec(),
...@@ -187,9 +191,9 @@ fn a_schema_14_queue_converts_to_ops_that_reach_its_working_pages_and_publish()...@@ -187,9 +191,9 @@ fn a_schema_14_queue_converts_to_ops_that_reach_its_working_pages_and_publish()
187 Queued::Page(191 Queued::Page(
188 a,192 a,
189 appended(193 appended(
190 PreparedEdit::page(SOURCE, a, &first, "Author")194 ops::saved(SOURCE, a, &first)
191 .unwrap()195 .unwrap()
192 .as_bytes(),196 .as_slice(),
193 a,197 a,
194 a_text,198 a_text,
195 " two",199 " two",
...@@ -216,12 +220,12 @@ fn a_schema_14_queue_converts_to_ops_that_reach_its_working_pages_and_publish()...@@ -216,12 +220,12 @@ fn a_schema_14_queue_converts_to_ops_that_reach_its_working_pages_and_publish()
216 }220 }
217 })221 })
218 );222 );
219 let local = pages(&cache.snapshot().unwrap());223 let local = pages(&snapshot(&cache));
220 assert_eq!(local, pages(&working));224 assert_eq!(local, pages(&working));
221 drop(cache);225 drop(cache);
222 // A converted cache opens as it is.226 // A converted cache opens as it is.
223 let cache = Replica::open(&path).unwrap();227 let cache = Replica::open(&path).unwrap();
224 assert_eq!(pages(&cache.snapshot().unwrap()), local);228 assert_eq!(pages(&snapshot(&cache)), local);
225 let next = cache229 let next = cache
226 .apply(230 .apply(
227 "Author",231 "Author",
...@@ -245,7 +249,7 @@ fn a_schema_14_queue_converts_to_ops_that_reach_its_working_pages_and_publish()...@@ -245,7 +249,7 @@ fn a_schema_14_queue_converts_to_ops_that_reach_its_working_pages_and_publish()
245 Some((id, EditStatus::Published { .. })) if id == next249 Some((id, EditStatus::Published { .. })) if id == next
246 ));250 ));
247 let published = pages(&server.durable);251 let published = pages(&server.durable);
248 assert_eq!(published, pages(&cache.snapshot().unwrap()));252 assert_eq!(published, pages(&snapshot(&cache)));
249 assert!(253 assert!(
250 published254 published
251 .iter()255 .iter()
...@@ -255,7 +259,7 @@ fn a_schema_14_queue_converts_to_ops_that_reach_its_working_pages_and_publish()...@@ -255,7 +259,7 @@ fn a_schema_14_queue_converts_to_ops_that_reach_its_working_pages_and_publish()
255}259}
256260
257#[test]261#[test]
258fn an_uncertain_attempt_and_a_conflict_keep_their_states() {262fn an_uncertain_attempt_keeps_its_state_and_a_conflict_becomes_a_conflict_page() {
259 let body = body_pages(SOURCE);263 let body = body_pages(SOURCE);
260 let (a, a_text) = body[0];264 let (a, a_text) = body[0];
261 let directory = tempfile::tempdir().unwrap();265 let directory = tempfile::tempdir().unwrap();
...@@ -263,9 +267,9 @@ fn an_uncertain_attempt_and_a_conflict_keep_their_states() {...@@ -263,9 +267,9 @@ fn an_uncertain_attempt_and_a_conflict_keep_their_states() {
263 // The oldest edit was attempted: its schema-14 evidence names the revision it published.267 // The oldest edit was attempted: its schema-14 evidence names the revision it published.
264 let path = directory.path().join("attempted.sqlite");268 let path = directory.path().join("attempted.sqlite");
265 let after = appended(SOURCE, a, a_text, " attempted");269 let after = appended(SOURCE, a, a_text, " attempted");
266 let published = PreparedEdit::page(SOURCE, a, &after, "Author").unwrap();270 let published = ops::saved(SOURCE, a, &after).unwrap();
267 let revision =271 let revision =
268 onestore::RevisionIndex::parse(&onestore::Store::parse(published.as_bytes()).unwrap())272 onestore::RevisionIndex::parse(&onestore::Store::parse(published.as_slice()).unwrap())
269 .unwrap()273 .unwrap()
270 .active(a)274 .active(a)
271 .unwrap();275 .unwrap();
...@@ -286,7 +290,7 @@ fn an_uncertain_attempt_and_a_conflict_keep_their_states() {...@@ -286,7 +290,7 @@ fn an_uncertain_attempt_and_a_conflict_keep_their_states() {
286 Some((2, awaiting))290 Some((2, awaiting))
287 );291 );
288 assert_eq!(unchanged.publications, 0);292 assert_eq!(unchanged.publications, 0);
289 let mut landed = Server::new(published.as_bytes());293 let mut landed = Server::new(published.as_slice());
290 assert_eq!(294 assert_eq!(
291 cache.sync_once(&mut landed).unwrap().edit,295 cache.sync_once(&mut landed).unwrap().edit,
292 Some((2, EditStatus::Published { revision }))296 Some((2, EditStatus::Published { revision }))
...@@ -294,17 +298,17 @@ fn an_uncertain_attempt_and_a_conflict_keep_their_states() {...@@ -294,17 +298,17 @@ fn an_uncertain_attempt_and_a_conflict_keep_their_states() {
294 assert_eq!(landed.publications, 0);298 assert_eq!(landed.publications, 0);
295 drop(cache);299 drop(cache);
296300
297 // The oldest edit was in conflict: the conversion lowered it onto the remote page the301 // The oldest edit was in conflict: the conversion keeps the remote page the conflict was
298 // conflict was recorded against, and keeping mine publishes it.302 // recorded against and queues the local version as its conflict page.
299 let path = directory.path().join("conflicted.sqlite");303 let path = directory.path().join("conflicted.sqlite");
300 let remote = PreparedEdit::page(SOURCE, a, &appended(SOURCE, a, a_text, " remote"), "Native")304 let remote = ops::saved(SOURCE, a, &appended(SOURCE, a, a_text, " remote"))
301 .unwrap()305 .unwrap()
302 .as_bytes()306 .as_slice()
303 .to_vec();307 .to_vec();
304 let local = appended(SOURCE, a, a_text, " local");308 let local = appended(SOURCE, a, a_text, " local");
305 let working = PreparedEdit::page(SOURCE, a, &local, "Author")309 let working = ops::saved(SOURCE, a, &local)
306 .unwrap()310 .unwrap()
307 .as_bytes()311 .as_slice()
308 .to_vec();312 .to_vec();
309 {313 {
310 // Schema 14 stored the remote as the base and kept the local working image.314 // Schema 14 stored the remote as the base and kept the local working image.
...@@ -324,27 +328,26 @@ fn an_uncertain_attempt_and_a_conflict_keep_their_states() {...@@ -324,27 +328,26 @@ fn an_uncertain_attempt_and_a_conflict_keep_their_states() {
324 .unwrap();328 .unwrap();
325 }329 }
326 let cache = Replica::open(&path).unwrap();330 let cache = Replica::open(&path).unwrap();
331 assert_eq!(cache.status(2).unwrap(), Some(EditStatus::Pending));
327 assert_eq!(332 assert_eq!(
328 cache.status(2).unwrap(),333 model_ops::page_of(&snapshot(&cache), a),
329 Some(EditStatus::Conflict(ConflictKind::ContentChanged))334 model_ops::page_of(&remote, a)
330 );
331 assert_eq!(
332 model_ops::page_of(&cache.snapshot().unwrap(), a),
333 model_ops::page_of(&working, a)
334 );335 );
335 let mut server = Server::new(&remote);336 let mut server = Server::new(&remote);
336 assert_eq!(
337 cache.sync_once(&mut server).unwrap().edit,
338 Some((2, EditStatus::Conflict(ConflictKind::ContentChanged)))
339 );
340 cache.resolve(2, Resolution::Mine).unwrap();
341 assert!(matches!(337 assert!(matches!(
342 cache.sync_once(&mut server).unwrap().edit,338 cache.sync_once(&mut server).unwrap().edit,
343 Some((2, EditStatus::Published { .. }))339 Some((2, EditStatus::Published { .. }))
344 ));340 ));
345 assert_eq!(341 assert_eq!(
346 model_ops::page_of(&server.durable, a),342 model_ops::page_of(&server.durable, a),
347 model_ops::page_of(&working, a)343 model_ops::page_of(&remote, a)
344 );
345 assert_eq!(
346 conflicts(&server.durable),
347 [(
348 a,
349 vec![("Author".to_owned(), page_texts(&model_ops::page_of(&working, a)))]
350 )]
348 );351 );
349}352}
350353
...@@ -355,9 +358,9 @@ fn a_page_the_conversion_cannot_reproduce_leaves_the_cache_untouched() {...@@ -355,9 +358,9 @@ fn a_page_the_conversion_cannot_reproduce_leaves_the_cache_untouched() {
355 let directory = tempfile::tempdir().unwrap();358 let directory = tempfile::tempdir().unwrap();
356 let path = directory.path().join("cache.sqlite");359 let path = directory.path().join("cache.sqlite");
357 // The working image holds a page the queued edit does not reach.360 // The working image holds a page the queued edit does not reach.
358 let stray = PreparedEdit::page(SOURCE, a, &appended(SOURCE, a, a_text, " stray"), "Author")361 let stray = ops::saved(SOURCE, a, &appended(SOURCE, a, a_text, " stray"))
359 .unwrap()362 .unwrap()
360 .as_bytes()363 .as_slice()
361 .to_vec();364 .to_vec();
362 v14(365 v14(
363 &path,366 &path,
...@@ -382,3 +385,75 @@ fn a_page_the_conversion_cannot_reproduce_leaves_the_cache_untouched() {...@@ -382,3 +385,75 @@ fn a_page_the_conversion_cannot_reproduce_leaves_the_cache_untouched() {
382 assert!(Replica::open(&path).is_err());385 assert!(Replica::open(&path).is_err());
383 assert_eq!(std::fs::read(&path).unwrap(), before);386 assert_eq!(std::fs::read(&path).unwrap(), before);
384}387}
388
389/// A schema-15 cache upgrades in place: its queue, receipts and recovery archives stay, and
390/// a conflict it recorded is queued work the next sync keeps as a conflict page.
391#[test]
392fn a_schema_15_cache_upgrades_in_place_and_its_conflict_becomes_a_conflict_page() {
393 let (space, text) = body_pages(SOURCE)[0];
394 let directory = tempfile::tempdir().unwrap();
395 let path = directory.path().join("cache.sqlite");
396 let cache = Replica::create(&path, SOURCE).unwrap();
397 let id = model_ops::save(&cache, text, |page| {
398 model_ops::replace_text(page, text, 0..0, "Local ")
399 })
400 .unwrap()
401 .unwrap();
402 let pending = cache.pending().unwrap();
403 let archive = directory.path().join("before.sqlite");
404 drop(cache);
405 {
406 // Schema 15 recorded the review conflict and its page on the batch.
407 let connection = Connection::open(&path).unwrap();
408 connection
409 .execute_batch(&format!(
410 "PRAGMA foreign_keys=OFF;
411 BEGIN;
412 CREATE TABLE old (
413 id INTEGER PRIMARY KEY AUTOINCREMENT CHECK(id>0),
414 sealed TEXT,
415 revisions TEXT,
416 attempted INTEGER NOT NULL DEFAULT 0 CHECK(attempted IN (0,1)),
417 conflict INTEGER CHECK(conflict BETWEEN 0 AND 3),
418 space TEXT,
419 CHECK((sealed IS NULL) = (revisions IS NULL)),
420 CHECK(attempted=0 OR sealed IS NOT NULL),
421 CHECK((conflict IS NULL) = (space IS NULL))
422 ) STRICT;
423 INSERT INTO old SELECT id, sealed, revisions, attempted, 3, '{space}' FROM batches;
424 DROP TABLE batches;
425 ALTER TABLE old RENAME TO batches;
426 PRAGMA user_version=15;
427 COMMIT;"
428 ))
429 .unwrap();
430 }
431 let cache = Replica::open(&path).unwrap();
432 assert_eq!(cache.pending().unwrap(), pending);
433 assert_eq!(cache.status(id).unwrap(), Some(EditStatus::Pending));
434 cache.export_recovery(&archive).unwrap();
435 drop(cache);
436 let connection = Connection::open(&path).unwrap();
437 let version: u32 = connection
438 .pragma_query_value(None, "user_version", |row| row.get(0))
439 .unwrap();
440 let columns: Vec<String> = connection
441 .prepare("SELECT name FROM pragma_table_info('batches')")
442 .unwrap()
443 .query_map([], |row| row.get(0))
444 .unwrap()
445 .collect::<Result<_, _>>()
446 .unwrap();
447 assert_eq!(version, 16);
448 assert_eq!(columns, ["id", "sealed", "revisions", "attempted"]);
449 drop(connection);
450 assert_eq!(notebook::Recovery::open(&archive).unwrap().pending().unwrap(), pending);
451 let cache = Replica::open(&path).unwrap();
452 let remote = typed(SOURCE, space, text, 0..0, "Remote ");
453 let mut server = Server::new(&remote);
454 assert!(matches!(
455 cache.sync_once(&mut server).unwrap().edit,
456 Some((_, EditStatus::Published { .. }))
457 ));
458 assert!(conflicted(&server.durable, space));
459}
crates/notebook/tests/protected.rs+15-6
...@@ -28,8 +28,8 @@ fn a_locked_section_unlocks_for_reading() {...@@ -28,8 +28,8 @@ fn a_locked_section_unlocks_for_reading() {
28 ));28 ));
29}29}
3030
31/// An unlocked page is edited and saved under the section's key, then reads back with the31/// An unlocked page is edited as ops stored under the section's key, then reads back with
32/// same password. `ONESTORE_PROTECTED_EXPORT` names a directory receiving the notebook for32/// the same password. `ONESTORE_PROTECTED_EXPORT` names a directory receiving the notebook for
33/// a cold reopen in OneNote.33/// a cold reopen in OneNote.
34#[test]34#[test]
35fn an_unlocked_page_is_saved_under_the_section_key() {35fn an_unlocked_page_is_saved_under_the_section_key() {
...@@ -59,6 +59,7 @@ fn an_unlocked_page_is_saved_under_the_section_key() {...@@ -59,6 +59,7 @@ fn an_unlocked_page_is_saved_under_the_section_key() {
59 .clone();59 .clone();
60 let mut unlocked = notebook.unlock(&section, password).unwrap();60 let mut unlocked = notebook.unlock(&section, password).unwrap();
61 let (space, mut page) = unlocked.pages[0].clone();61 let (space, mut page) = unlocked.pages[0].clone();
62 let before = page.clone();
62 let paragraphs = page63 let paragraphs = page
63 .objects64 .objects
64 .iter_mut()65 .iter_mut()
...@@ -95,20 +96,28 @@ fn an_unlocked_page_is_saved_under_the_section_key() {...@@ -95,20 +96,28 @@ fn an_unlocked_page_is_saved_under_the_section_key() {
95 format,96 format,
96 ))97 ))
97 .unwrap();98 .unwrap();
99 let edit = onestore::op::Edit {
100 at: 134_000_000_000_000_000,
101 ops: onestore::op::lower_page(&before, &page)
102 .unwrap()
103 .into_iter()
104 .map(|op| onestore::op::Op::Page { space, op })
105 .collect(),
106 };
98 assert!(matches!(107 assert!(matches!(
99 notebook.save_unlocked(&section, "wrong", &mut unlocked, space, &page, "Rust"),108 notebook.apply_unlocked(&section, "wrong", &mut unlocked, "Rust", &edit),
100 Err(notebook::Error::Protected(109 Err(notebook::Error::Protected(
101 onestore::protected::Error::PasswordMismatch110 onestore::protected::Error::PasswordMismatch
102 ))111 ))
103 ));112 ));
104 let mut stale = notebook.unlock(&section, password).unwrap();113 let mut stale = notebook.unlock(&section, password).unwrap();
105 notebook114 notebook
106 .save_unlocked(&section, password, &mut unlocked, space, &page, "Rust")115 .apply_unlocked(&section, password, &mut unlocked, "Rust", &edit)
107 .unwrap();116 .unwrap();
108 // A view read before that save no longer matches the file.117 // A view read before that edit no longer matches the file.
109 assert!(118 assert!(
110 notebook119 notebook
111 .save_unlocked(&section, password, &mut stale, space, &page, "Rust")120 .apply_unlocked(&section, password, &mut stale, "Rust", &edit)
112 .is_err()121 .is_err()
113 );122 );
114 let (_, stored) = notebook.unlock(&section, password).unwrap().pages.remove(0);123 let (_, stored) = notebook.unlock(&section, password).unwrap().pages.remove(0);
crates/notebook/tests/session.rs+98-125
...@@ -1,8 +1,14 @@...@@ -1,8 +1,14 @@
1#[path = "../../onestore/tests/support/ops.rs"]
2mod ops;
1use notebook::{3use notebook::{
2 ConflictKind, EditStatus, Recovery, Resolution,4 EditStatus, Recovery, Resolution,
3 session::{Event, Notebook, QueuedEdit, Save, Section},5 session::{Event, Notebook, Section},
6};
7use onestore::{
8 ExGuid,
9 op::{Edit, Op, PageOp},
10 page::Page,
4};11};
5use onestore::{ExGuid, PreparedEdit, page::Page};
6use std::{12use std::{
7 path::Path,13 path::Path,
8 sync::{14 sync::{
...@@ -35,6 +41,21 @@ fn edited(page: &Page, text: &str) -> Page {...@@ -35,6 +41,21 @@ fn edited(page: &Page, text: &str) -> Page {
35 after41 after
36}42}
3743
44/// Replaces `range` of the page's first body text as the editor does: the edit's id once
45/// it is durable.
46fn typed(section: &Section, space: ExGuid, page: &Page, range: std::ops::Range<u32>, text: &str) -> u64 {
47 let op = PageOp::Text {
48 text: first_text(page),
49 range,
50 with: text.into(),
51 };
52 let edit = Edit {
53 at: model_ops::now(),
54 ops: vec![Op::Page { space, op }],
55 };
56 section.replica().apply("Editor", edit).unwrap()
57}
58
38fn open(file: &Path, cache: &Path) -> (Section, Arc<AtomicUsize>) {59fn open(file: &Path, cache: &Path) -> (Section, Arc<AtomicUsize>) {
39 let notified = Arc::new(AtomicUsize::new(0));60 let notified = Arc::new(AtomicUsize::new(0));
40 let counter = Arc::clone(&notified);61 let counter = Arc::clone(&notified);
...@@ -105,14 +126,8 @@ fn a_section_opens_through_its_replica_and_a_save_reaches_the_file_and_survives_...@@ -105,14 +126,8 @@ fn a_section_opens_through_its_replica_and_a_save_reaches_the_file_and_survives_
105 assert_eq!(pages.len(), 1);126 assert_eq!(pages.len(), 1);
106 let space = pages[0].0;127 let space = pages[0].0;
107 let before = section.page(space).unwrap();128 let before = section.page(space).unwrap();
108 assert_eq!(
109 section.save(space, &before, &before, "Editor").unwrap(),
110 Save::Unchanged
111 );
112 let after = edited(&before, "Saved ");129 let after = edited(&before, "Saved ");
113 let Save::Queued(id) = section.save(space, &before, &after, "Editor").unwrap() else {130 let id = typed(&section, space, &before, 0..0, "Saved ");
114 panic!()
115 };
116 assert_same(section.page(space).unwrap(), &after);131 assert_same(section.page(space).unwrap(), &after);
117 published(&section, id);132 published(&section, id);
118 // The worker reports the step after writing its receipt.133 // The worker reports the step after writing its receipt.
...@@ -167,9 +182,7 @@ fn saves_wait_for_an_unreachable_file_and_publish_after_relaunch() {...@@ -167,9 +182,7 @@ fn saves_wait_for_an_unreachable_file_and_publish_after_relaunch() {
167 let permissions = std::fs::metadata(&file).unwrap().permissions();182 let permissions = std::fs::metadata(&file).unwrap().permissions();
168 std::fs::set_permissions(&file, std::fs::Permissions::from_mode(0o444)).unwrap();183 std::fs::set_permissions(&file, std::fs::Permissions::from_mode(0o444)).unwrap();
169 let after = edited(&before, "Offline ");184 let after = edited(&before, "Offline ");
170 let Save::Queued(id) = section.save(space, &before, &after, "Editor").unwrap() else {185 let id = typed(&section, space, &before, 0..0, "Offline ");
171 panic!()
172 };
173 wait(&section, |event| matches!(event, Event::Unreachable(_)));186 wait(&section, |event| matches!(event, Event::Unreachable(_)));
174 section.close().unwrap();187 section.close().unwrap();
175 assert_eq!(188 assert_eq!(
...@@ -198,7 +211,7 @@ fn saves_wait_for_an_unreachable_file_and_publish_after_relaunch() {...@@ -198,7 +211,7 @@ fn saves_wait_for_an_unreachable_file_and_publish_after_relaunch() {
198}211}
199212
200#[test]213#[test]
201fn an_external_change_refreshes_the_page_and_stales_a_save_from_the_old_model() {214fn an_external_change_reloads_the_page_and_later_edits_apply_to_it() {
202 let directory = tempfile::tempdir().unwrap();215 let directory = tempfile::tempdir().unwrap();
203 let file = directory.path().join("notes.one");216 let file = directory.path().join("notes.one");
204 let cache = directory.path().join("cache");217 let cache = directory.path().join("cache");
...@@ -214,7 +227,7 @@ fn an_external_change_refreshes_the_page_and_stales_a_save_from_the_old_model()...@@ -214,7 +227,7 @@ fn an_external_change_refreshes_the_page_and_stales_a_save_from_the_old_model()
214 let deadline = Instant::now() + Duration::from_secs(20);227 let deadline = Instant::now() + Duration::from_secs(20);
215 loop {228 loop {
216 let bytes = onestore::read_file(&file).unwrap();229 let bytes = onestore::read_file(&file).unwrap();
217 match PreparedEdit::page(&bytes, space, &native, "Native")230 match ops::save(&bytes, space, &native)
218 .unwrap()231 .unwrap()
219 .commit_file(&file)232 .commit_file(&file)
220 {233 {
...@@ -227,29 +240,18 @@ fn an_external_change_refreshes_the_page_and_stales_a_save_from_the_old_model()...@@ -227,29 +240,18 @@ fn an_external_change_refreshes_the_page_and_stales_a_save_from_the_old_model()
227 }240 }
228 }241 }
229 section.wake();242 section.wake();
230 let deadline = Instant::now() + Duration::from_secs(20);243 wait(&section, |event| matches!(event, Event::Changed(spaces) if spaces.contains(&space)));
231 while section.page(space).unwrap().title == before.title {
232 assert!(Instant::now() < deadline, "the refresh did not arrive");
233 wait(&section, |event| matches!(event, Event::Refreshed));
234 }
235 assert_same(section.page(space).unwrap(), &native);244 assert_same(section.page(space).unwrap(), &native);
236 let native = section.page(space).unwrap();245 // An edit made on the page shown before the change applies to the page as it is now.
237 let stale = edited(&before, "Local ");246 let id = typed(&section, space, &before, 0..0, "Local ");
238 assert_eq!(
239 section.save(space, &before, &stale, "Editor").unwrap(),
240 Save::Stale
241 );
242 let after = edited(&native, "Local ");247 let after = edited(&native, "Local ");
243 let Save::Queued(id) = section.save(space, &native, &after, "Editor").unwrap() else {
244 panic!()
245 };
246 published(&section, id);248 published(&section, id);
247 assert_same(stored_page(&file, space), &after);249 assert_same(stored_page(&file, space), &after);
248 section.close().unwrap();250 section.close().unwrap();
249}251}
250252
251#[test]253#[test]
252fn queued_saves_continue_each_other_detect_stale_pages_and_finish_before_close() {254fn edits_apply_in_order_without_waiting_and_finish_before_close() {
253 let directory = tempfile::tempdir().unwrap();255 let directory = tempfile::tempdir().unwrap();
254 let file = directory.path().join("notes.one");256 let file = directory.path().join("notes.one");
255 let cache = directory.path().join("cache");257 let cache = directory.path().join("cache");
...@@ -260,36 +262,44 @@ fn queued_saves_continue_each_other_detect_stale_pages_and_finish_before_close()...@@ -260,36 +262,44 @@ fn queued_saves_continue_each_other_detect_stale_pages_and_finish_before_close()
260 .unwrap();262 .unwrap();
261 let (section, _) = open(&file, &cache);263 let (section, _) = open(&file, &cache);
262 let space = section.pages().unwrap()[0].0;264 let space = section.pages().unwrap()[0].0;
263 let mut before = section.page(space).unwrap();265 let original = section.page(space).unwrap();
266 let text = first_text(&original);
267 let apply = |word: &str| {
268 let op = PageOp::Text {
269 text,
270 range: 0..0,
271 with: word.into(),
272 };
273 let edit = Edit {
274 at: model_ops::now(),
275 ops: vec![Op::Page { space, op }],
276 };
277 section.apply("Editor", edit).unwrap();
278 };
279 let mut expected = original.clone();
264 for word in ["one ", "two ", "three ", "four ", "five "] {280 for word in ["one ", "two ", "three ", "four ", "five "] {
265 let after = edited(&before, word);281 apply(word);
266 section282 expected = edited(&expected, word);
267 .queue_save(space, before, after.clone(), "Editor")
268 .unwrap();
269 before = after;
270 }283 }
271 let deadline = Instant::now() + Duration::from_secs(20);284 let deadline = Instant::now() + Duration::from_secs(20);
272 loop {285 loop {
273 let mut stored = section.page(space).unwrap();286 let mut stored = section.page(space).unwrap();
274 stored.title = before.title.clone();287 stored.title = expected.title.clone();
275 if stored == before {288 if stored == expected {
276 break;289 break;
277 }290 }
278 assert!(Instant::now() < deadline, "the saves did not arrive");291 assert!(Instant::now() < deadline, "the edits did not arrive");
279 for (_, save) in section.saved() {
280 assert!(matches!(save, Ok(Save::Queued(_))), "{save:?}");
281 }
282 std::thread::sleep(Duration::from_millis(20));292 std::thread::sleep(Duration::from_millis(20));
283 }293 }
284 while !section.pending().unwrap().is_empty() {294 while !section.pending().unwrap().is_empty() {
285 assert!(Instant::now() < deadline, "the saves were not published");295 assert!(Instant::now() < deadline, "the edits were not published");
286 std::thread::sleep(Duration::from_millis(20));296 std::thread::sleep(Duration::from_millis(20));
287 }297 }
288 let native = edited(&section.page(space).unwrap(), "Native ");298 let native = edited(&section.page(space).unwrap(), "Native ");
289 let deadline = Instant::now() + Duration::from_secs(20);299 let deadline = Instant::now() + Duration::from_secs(20);
290 loop {300 loop {
291 let bytes = onestore::read_file(&file).unwrap();301 let bytes = onestore::read_file(&file).unwrap();
292 match PreparedEdit::page(&bytes, space, &native, "Native")302 match ops::save(&bytes, space, &native)
293 .unwrap()303 .unwrap()
294 .commit_file(&file)304 .commit_file(&file)
295 {305 {
...@@ -302,29 +312,11 @@ fn queued_saves_continue_each_other_detect_stale_pages_and_finish_before_close()...@@ -302,29 +312,11 @@ fn queued_saves_continue_each_other_detect_stale_pages_and_finish_before_close()
302 }312 }
303 }313 }
304 section.wake();314 section.wake();
305 while !section.page(space).unwrap().objects.eq(&native.objects) {315 wait(&section, |event| matches!(event, Event::Changed(spaces) if spaces.contains(&space)));
306 assert!(Instant::now() < deadline, "the refresh did not arrive");316 apply("Local ");
307 wait(&section, |event| matches!(event, Event::Refreshed));
308 }
309 let stale = edited(&before, "Local ");
310 section.queue_save(space, before, stale, "Editor").unwrap();
311 // Outcomes of the earlier saves may still be arriving.
312 while !section
313 .saved()
314 .into_iter()
315 .any(|saved| saved == (space, Ok(Save::Stale)))
316 {
317 assert!(Instant::now() < deadline, "the stale save was not reported");
318 std::thread::sleep(Duration::from_millis(20));
319 }
320 let native = section.page(space).unwrap();
321 let after = edited(&native, "Local ");
322 section
323 .queue_save(space, native, after.clone(), "Editor")
324 .unwrap();
325 section.close().unwrap();317 section.close().unwrap();
326 let (section, _) = open(&file, &cache);318 let (section, _) = open(&file, &cache);
327 assert_same(section.page(space).unwrap(), &after);319 assert_same(section.page(space).unwrap(), &edited(&native, "Local "));
328 section.close().unwrap();320 section.close().unwrap();
329}321}
330322
...@@ -426,9 +418,7 @@ fn an_absent_remote_does_not_prevent_local_relaunch_or_further_saves() {...@@ -426,9 +418,7 @@ fn an_absent_remote_does_not_prevent_local_relaunch_or_further_saves() {
426 let space = section.pages().unwrap()[0].0;418 let space = section.pages().unwrap()[0].0;
427 let before = section.page(space).unwrap();419 let before = section.page(space).unwrap();
428 let after = edited(&before, "First ");420 let after = edited(&before, "First ");
429 let Save::Queued(id) = section.save(space, &before, &after, "Editor").unwrap() else {421 let id = typed(&section, space, &before, 0..0, "First ");
430 panic!()
431 };
432 wait(422 wait(
433 &section,423 &section,
434 |event| matches!(event, Event::Unreachable(error) if error.kind() == std::io::ErrorKind::NotFound),424 |event| matches!(event, Event::Unreachable(error) if error.kind() == std::io::ErrorKind::NotFound),
...@@ -442,9 +432,7 @@ fn an_absent_remote_does_not_prevent_local_relaunch_or_further_saves() {...@@ -442,9 +432,7 @@ fn an_absent_remote_does_not_prevent_local_relaunch_or_further_saves() {
442 assert_eq!(section.status(id).unwrap(), Some(EditStatus::Pending));432 assert_eq!(section.status(id).unwrap(), Some(EditStatus::Pending));
443 let before = section.page(space).unwrap();433 let before = section.page(space).unwrap();
444 let after = edited(&before, "Second ");434 let after = edited(&before, "Second ");
445 let Save::Queued(next) = section.save(space, &before, &after, "Editor").unwrap() else {435 let next = typed(&section, space, &before, 0..0, "Second ");
446 panic!()
447 };
448 assert!(!file.exists());436 assert!(!file.exists());
449 let restored = directory.path().join("restored.one");437 let restored = directory.path().join("restored.one");
450 std::fs::write(&restored, &source).unwrap();438 std::fs::write(&restored, &source).unwrap();
...@@ -470,9 +458,7 @@ fn resuming_against_another_document_preserves_both_remote_and_pending_edit() {...@@ -470,9 +458,7 @@ fn resuming_against_another_document_preserves_both_remote_and_pending_edit() {
470 let space = section.pages().unwrap()[0].0;458 let space = section.pages().unwrap()[0].0;
471 let before = section.page(space).unwrap();459 let before = section.page(space).unwrap();
472 let after = edited(&before, "Local ");460 let after = edited(&before, "Local ");
473 let Save::Queued(id) = section.save(space, &before, &after, "Editor").unwrap() else {461 let id = typed(&section, space, &before, 0..0, "Local ");
474 panic!()
475 };
476 wait(&section, |event| matches!(event, Event::Unreachable(_)));462 wait(&section, |event| matches!(event, Event::Unreachable(_)));
477 section.close().unwrap();463 section.close().unwrap();
478464
...@@ -499,10 +485,7 @@ fn offline_save_process() {...@@ -499,10 +485,7 @@ fn offline_save_process() {
499 let section = Section::resume(root.join("absent.one"), replica, || {}).unwrap();485 let section = Section::resume(root.join("absent.one"), replica, || {}).unwrap();
500 let space = section.pages().unwrap()[0].0;486 let space = section.pages().unwrap()[0].0;
501 let before = section.page(space).unwrap();487 let before = section.page(space).unwrap();
502 let after = edited(&before, "Durable ");488 let id = typed(&section, space, &before, 0..0, "Durable ");
503 let Save::Queued(id) = section.save(space, &before, &after, "Editor").unwrap() else {
504 panic!()
505 };
506 std::fs::write(root.join("acknowledged"), id.to_string()).unwrap();489 std::fs::write(root.join("acknowledged"), id.to_string()).unwrap();
507 // Terminate without running Session or SQLite destructors after acknowledgement.490 // Terminate without running Session or SQLite destructors after acknowledgement.
508 std::process::exit(0);491 std::process::exit(0);
...@@ -562,9 +545,7 @@ fn smb_connection_failure_keeps_the_session_locally_editable_and_retries() {...@@ -562,9 +545,7 @@ fn smb_connection_failure_keeps_the_session_locally_editable_and_retries() {
562 let space = section.pages().unwrap()[0].0;545 let space = section.pages().unwrap()[0].0;
563 let before = section.page(space).unwrap();546 let before = section.page(space).unwrap();
564 let after = edited(&before, "Offline SMB ");547 let after = edited(&before, "Offline SMB ");
565 let Save::Queued(id) = section.save(space, &before, &after, "Editor").unwrap() else {548 let id = typed(&section, space, &before, 0..0, "Offline SMB ");
566 panic!()
567 };
568 section.wake();549 section.wake();
569 wait(550 wait(
570 &section,551 &section,
...@@ -611,9 +592,7 @@ fn live_smb_session_save_publishes_and_reopens() {...@@ -611,9 +592,7 @@ fn live_smb_session_save_publishes_and_reopens() {
611 let space = section.pages().unwrap()[0].0;592 let space = section.pages().unwrap()[0].0;
612 let before = section.page(space).unwrap();593 let before = section.page(space).unwrap();
613 let after = edited(&before, "Session SMB ");594 let after = edited(&before, "Session SMB ");
614 let Save::Queued(id) = section.save(space, &before, &after, "Editor").unwrap() else {595 let id = typed(&section, space, &before, 0..0, "Session SMB ");
615 panic!()
616 };
617 published(&section, id);596 published(&section, id);
618 let remote = client.read("session.one", 1024 * 1024).unwrap();597 let remote = client.read("session.one", 1024 * 1024).unwrap();
619 assert_same(model_ops::page_of(&remote, space), &after);598 assert_same(model_ops::page_of(&remote, space), &after);
...@@ -661,9 +640,7 @@ fn dropping_during_connection_keeps_cache_owned_until_the_worker_finishes() {...@@ -661,9 +640,7 @@ fn dropping_during_connection_keeps_cache_owned_until_the_worker_finishes() {
661 let space = section.pages().unwrap()[0].0;640 let space = section.pages().unwrap()[0].0;
662 let before = section.page(space).unwrap();641 let before = section.page(space).unwrap();
663 let after = edited(&before, "Saved during connection ");642 let after = edited(&before, "Saved during connection ");
664 let Save::Queued(id) = section.save(space, &before, &after, "Editor").unwrap() else {643 let id = typed(&section, space, &before, 0..0, "Saved during connection ");
665 panic!()
666 };
667 let (dropped, finished) = mpsc::channel();644 let (dropped, finished) = mpsc::channel();
668 let owner = std::thread::spawn(move || {645 let owner = std::thread::spawn(move || {
669 drop(section);646 drop(section);
...@@ -692,14 +669,14 @@ fn dropping_during_connection_keeps_cache_owned_until_the_worker_finishes() {...@@ -692,14 +669,14 @@ fn dropping_during_connection_keeps_cache_owned_until_the_worker_finishes() {
692 };669 };
693 assert_eq!(replica.status(id).unwrap(), Some(EditStatus::Pending));670 assert_eq!(replica.status(id).unwrap(), Some(EditStatus::Pending));
694 assert_same(671 assert_same(
695 model_ops::page_of(&replica.snapshot().unwrap(), space),672 model_ops::page_of(&server::snapshot(&replica), space),
696 &after,673 &after,
697 );674 );
698 assert!(connecting.try_recv().is_err());675 assert!(connecting.try_recv().is_err());
699}676}
700677
701#[test]678#[test]
702fn a_conflicting_save_is_reviewed_against_the_remote_page_and_archived_for_recovery() {679fn a_conflicting_save_keeps_the_native_page_and_a_conflict_page_the_session_deletes() {
703 let directory = tempfile::tempdir().unwrap();680 let directory = tempfile::tempdir().unwrap();
704 let file = directory.path().join("notes.one");681 let file = directory.path().join("notes.one");
705 let cache = directory.path().join("cache");682 let cache = directory.path().join("cache");
...@@ -717,9 +694,7 @@ fn a_conflicting_save_is_reviewed_against_the_remote_page_and_archived_for_recov...@@ -717,9 +694,7 @@ fn a_conflicting_save_is_reviewed_against_the_remote_page_and_archived_for_recov
717 std::fs::set_permissions(&file, std::fs::Permissions::from_mode(0o444)).unwrap();694 std::fs::set_permissions(&file, std::fs::Permissions::from_mode(0o444)).unwrap();
718 let mut local = before.clone();695 let mut local = before.clone();
719 model_ops::replace_text(&mut local, text, 0..8, "Local");696 model_ops::replace_text(&mut local, text, 0..8, "Local");
720 let Save::Queued(id) = section.save(space, &before, &local, "Editor").unwrap() else {697 let id = typed(&section, space, &before, 0..8, "Local");
721 panic!()
722 };
723 wait(&section, |event| matches!(event, Event::Unreachable(_)));698 wait(&section, |event| matches!(event, Event::Unreachable(_)));
724 std::fs::set_permissions(&file, permissions).unwrap();699 std::fs::set_permissions(&file, permissions).unwrap();
725 let mut native = before.clone();700 let mut native = before.clone();
...@@ -727,7 +702,7 @@ fn a_conflicting_save_is_reviewed_against_the_remote_page_and_archived_for_recov...@@ -727,7 +702,7 @@ fn a_conflicting_save_is_reviewed_against_the_remote_page_and_archived_for_recov
727 let deadline = Instant::now() + Duration::from_secs(20);702 let deadline = Instant::now() + Duration::from_secs(20);
728 loop {703 loop {
729 let bytes = onestore::read_file(&file).unwrap();704 let bytes = onestore::read_file(&file).unwrap();
730 match PreparedEdit::page(&bytes, space, &native, "Native")705 match ops::save(&bytes, space, &native)
731 .unwrap()706 .unwrap()
732 .commit_file(&file)707 .commit_file(&file)
733 {708 {
...@@ -740,40 +715,38 @@ fn a_conflicting_save_is_reviewed_against_the_remote_page_and_archived_for_recov...@@ -740,40 +715,38 @@ fn a_conflicting_save_is_reviewed_against_the_remote_page_and_archived_for_recov
740 }715 }
741 }716 }
742 section.wake();717 section.wake();
743 wait(718 wait(&section, |event| {
744 &section,719 matches!(event, Event::Attempt { id: n, status: EditStatus::Published { .. } } if *n > id)
745 |event| matches!(event, Event::Attempt { id: n, status: EditStatus::Conflict(ConflictKind::ContentChanged) } if *n == id),720 });
746 );721 assert!(matches!(
747 let conflicts = section.conflicts().unwrap();722 section.status(id).unwrap(),
723 Some(EditStatus::Published { .. })
724 ));
725 // As OneNote does, the native version stays the page and the local one is kept beside it.
726 assert_same(section.page(space).unwrap(), &native);
727 assert_same(stored_page(&file, space), &native);
728 let listed = section.conflicts().unwrap();
729 assert_eq!(listed.len(), 1);
730 assert_eq!(listed[0].0, space);
731 let conflict = &listed[0].1[0];
732 assert_eq!(conflict.user, "Editor");
733 let kept = section.page(conflict.space).unwrap();
748 assert_eq!(734 assert_eq!(
749 conflicts,735 model_ops::paragraph_with(&kept, first_text(&kept))
750 [(736 .and_then(|paragraph| paragraph.text())
751 QueuedEdit {737 .map(|text| text.text.text().to_owned()),
752 id,738 Some("Local".to_owned())
753 space,
754 status: EditStatus::Conflict(ConflictKind::ContentChanged)
755 },
756 ConflictKind::ContentChanged
757 )]
758 );739 );
759 assert_same(section.page(space).unwrap(), &local);
760 let remote = section.remote_page(space).unwrap();
761 assert_same(remote.clone(), &native);
762 let archive = directory.path().join("review.sqlite");
763 section.export_recovery(&archive).unwrap();
764 let recovery = Recovery::open(&archive).unwrap();
765 assert_eq!(recovery.pending().unwrap().len(), 1);
766 assert_eq!(740 assert_eq!(
767 recovery.status(id).unwrap(),741 server::conflicts(&onestore::read_file(&file).unwrap())[0].1,
768 Some(EditStatus::Conflict(ConflictKind::ContentChanged))742 [("Editor".to_owned(), vec!["Local".to_owned()])]
769 );743 );
770 let mut reviewed = remote.clone();744 // Merged by hand, the conflict page is deleted.
771 model_ops::replace_text(&mut reviewed, text, 0..6, "Native and local");745 let deleted = section.delete_pages(&[conflict.space]).unwrap();
772 section.review(id, &reviewed).unwrap();746 published(&section, deleted);
773 assert_eq!(section.status(id).unwrap(), Some(EditStatus::Pending));747 assert!(section.conflicts().unwrap().is_empty());
774 published(&section, id);748 assert!(server::conflicts(&onestore::read_file(&file).unwrap()).is_empty());
775 assert_same(stored_page(&file, space), &reviewed);749 assert!(section.pending().unwrap().is_empty());
776 assert!(section.queue().unwrap().is_empty());
777 section.close().unwrap();750 section.close().unwrap();
778}751}
779752
...@@ -819,7 +792,7 @@ fn an_uncertain_attempt_is_released_after_restart_by_review() {...@@ -819,7 +792,7 @@ fn an_uncertain_attempt_is_released_after_restart_by_review() {
819 );792 );
820 assert_eq!(793 assert_eq!(
821 section794 section
822 .queue()795 .pending()
823 .unwrap()796 .unwrap()
824 .iter()797 .iter()
825 .map(|edit| edit.id)798 .map(|edit| edit.id)
...@@ -850,7 +823,7 @@ fn an_uncertain_attempt_is_released_after_restart_by_review() {...@@ -850,7 +823,7 @@ fn an_uncertain_attempt_is_released_after_restart_by_review() {
850 recovery.status(id).unwrap(),823 recovery.status(id).unwrap(),
851 Some(EditStatus::AwaitingConfirmation { .. })824 Some(EditStatus::AwaitingConfirmation { .. })
852 ));825 ));
853 assert!(section.queue().unwrap().is_empty());826 assert!(section.pending().unwrap().is_empty());
854 assert!(827 assert!(
855 section828 section
856 .release(829 .release(
crates/notebook/tests/smb_queue.rs+10-6
...@@ -17,6 +17,10 @@ use std::{...@@ -17,6 +17,10 @@ use std::{
17 time::{Duration, Instant, SystemTime, UNIX_EPOCH},17 time::{Duration, Instant, SystemTime, UNIX_EPOCH},
18};18};
1919
20#[path = "support/server.rs"]
21mod server;
22use server::snapshot;
23
20fn client() -> Client {24fn client() -> Client {
21 Client::connect(25 Client::connect(
22 &std::env::var("ONESTORE_SMB_LAB").unwrap(),26 &std::env::var("ONESTORE_SMB_LAB").unwrap(),
...@@ -49,15 +53,15 @@ impl Remote for Counted {...@@ -49,15 +53,15 @@ impl Remote for Counted {
49 self.reads += 1;53 self.reads += 1;
50 self.remote.read()54 self.remote.read()
51 }55 }
52 fn stamp(&mut self) -> io::Result<Option<Stamp>> {56 fn stamp(&mut self) -> io::Result<Stamp> {
53 self.stamps += 1;57 self.stamps += 1;
54 self.remote.stamp()58 self.remote.stamp()
55 }59 }
56 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {60 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
57 self.remote.publish(transaction)61 self.remote.publish(transaction)
58 }62 }
59 fn confirm(&mut self, snapshot: &[u8]) -> Result<(), CommitError> {63 fn confirm(&mut self, base: &Stamp) -> Result<(), CommitError> {
60 self.remote.confirm(snapshot)64 self.remote.confirm(base)
61 }65 }
62}66}
6367
...@@ -140,7 +144,7 @@ fn live_keystrokes_publish_reading_only_the_header_and_a_native_change_merges()...@@ -140,7 +144,7 @@ fn live_keystrokes_publish_reading_only_the_header_and_a_native_change_merges()
140 assert_eq!(remote.stamps, 20);144 assert_eq!(remote.stamps, 20);
141 assert_eq!(145 assert_eq!(
142 lab.read(&path, 1 << 24).unwrap(),146 lab.read(&path, 1 << 24).unwrap(),
143 replica.snapshot().unwrap()147 snapshot(&replica)
144 );148 );
145 for _ in 0..5 {149 for _ in 0..5 {
146 assert_eq!(replica.sync_once(&mut remote).unwrap().edit, None);150 assert_eq!(replica.sync_once(&mut remote).unwrap().edit, None);
...@@ -173,7 +177,7 @@ fn live_keystrokes_publish_reading_only_the_header_and_a_native_change_merges()...@@ -173,7 +177,7 @@ fn live_keystrokes_publish_reading_only_the_header_and_a_native_change_merges()
173 assert!(merged.ends_with(" local"), "{merged}");177 assert!(merged.ends_with(" local"), "{merged}");
174 assert_eq!(178 assert_eq!(
175 lab.read(&path, 1 << 24).unwrap(),179 lab.read(&path, 1 << 24).unwrap(),
176 replica.snapshot().unwrap()180 snapshot(&replica)
177 );181 );
178 lab.delete(&path).unwrap();182 lab.delete(&path).unwrap();
179}183}
...@@ -244,6 +248,6 @@ fn live_session_applies_edits_through_the_embedded_client_and_reopens() {...@@ -244,6 +248,6 @@ fn live_session_applies_edits_through_the_embedded_client_and_reopens() {
244 section.close().unwrap();248 section.close().unwrap();
245 let replica = Replica::open(&cache).unwrap();249 let replica = Replica::open(&cache).unwrap();
246 assert_eq!(text_of(&replica, space, text), "0123456789Session");250 assert_eq!(text_of(&replica, space, text), "0123456789Session");
247 assert_eq!(replica.snapshot().unwrap(), published);251 assert_eq!(snapshot(&replica), published);
248 lab.delete(&path).unwrap();252 lab.delete(&path).unwrap();
249}253}
crates/notebook/tests/structure.rs+2-2
...@@ -53,13 +53,13 @@ fn sections_and_groups_are_created_renamed_coloured_ordered_and_deleted() {...@@ -53,13 +53,13 @@ fn sections_and_groups_are_created_renamed_coloured_ordered_and_deleted() {
53 onestore::place_file(root.join("First.one"), toc_id, "First.one").unwrap();53 onestore::place_file(root.join("First.one"), toc_id, "First.one").unwrap();
54 let mut notebook = Notebook::open(&root, temporary.path().join("cache")).unwrap();54 let mut notebook = Notebook::open(&root, temporary.path().join("cache")).unwrap();
55 assert_eq!(55 assert_eq!(
56 notebook.create_section("", "Second", "Author").unwrap(),56 notebook.create_section("", "Second", &onestore::PageCreation::new(None, Some(""), "Author").unwrap()).unwrap(),
57 "Second.one"57 "Second.one"
58 );58 );
59 assert_eq!(notebook.create_group("", "Archive").unwrap(), "Archive");59 assert_eq!(notebook.create_group("", "Archive").unwrap(), "Archive");
60 assert_eq!(60 assert_eq!(
61 notebook61 notebook
62 .create_section("Archive", "Inner", "Author")62 .create_section("Archive", "Inner", &onestore::PageCreation::new(None, Some(""), "Author").unwrap())
63 .unwrap(),63 .unwrap(),
64 "Archive/Inner.one"64 "Archive/Inner.one"
65 );65 );
crates/notebook/tests/support/model_ops.rs+12-13
...@@ -287,17 +287,16 @@ pub fn save(...@@ -287,17 +287,16 @@ pub fn save(
287 text: ExGuid,287 text: ExGuid,
288 edit: impl FnOnce(&mut Page),288 edit: impl FnOnce(&mut Page),
289) -> Result<Option<u64>, notebook::Error> {289) -> Result<Option<u64>, notebook::Error> {
290 let (space, _) = locate(&cache.snapshot()?, text);290 let mut space = None;
291 save_page(cache, space, edit)291 for (listed, ..) in cache.pages()? {
292}292 if paragraph_with(&cache.page(listed)?, text).is_some() {
293293 space = Some(listed);
294/// Resolves a conflict by taking the remote page, then queues `edit` of it.294 break;
295pub fn review(295 }
296 cache: &Replica,296 }
297 id: u64,297 save_page(
298 text: ExGuid,298 cache,
299 edit: impl FnOnce(&mut Page),299 space.expect("text object belongs to an active page"),
300) -> Result<Option<u64>, notebook::Error> {300 edit,
301 cache.resolve(id, notebook::Resolution::Theirs)?;301 )
302 save(cache, text, edit)
303}302}
crates/notebook/tests/support/model_schedule.rs+68-39
...@@ -1,16 +1,20 @@...@@ -1,16 +1,20 @@
1//! A deterministic multi-actor schedule over page-model saves: several replicas edit one1//! A deterministic multi-actor schedule over page-model saves: several replicas edit one
2//! page offline, publish through a fault-injecting remote, reopen, and review conflicts.2//! page offline, publish through a fault-injecting remote, reopen, and delete the conflict
3//! pages the merges leave.
34
4use crate::model_ops::{self, AUTHOR};5use crate::model_ops::{self, AUTHOR};
5use crate::server::{Fault, Server};6use crate::server::{Fault, Server, remote_snapshot, snapshot};
6use notebook::{EditStatus, Remote, Replica, Resolution};7use notebook::{EditStatus, Remote, Replica};
7use onestore::{8use onestore::{
8 CommitError, ExGuid, PreparedEdit, RevisionIndex, Store, Transaction,9 CommitError, ExGuid, RevisionIndex, Stamp, Store, Transaction,
9 document::Document,10 document::Document,
10 page::{Outline, Page, PageObject},11 page::{Outline, Page, PageObject},
11};12};
12use std::{io, sync::LazyLock};13use std::{io, sync::LazyLock};
1314
15#[path = "../../../onestore/tests/support/ops.rs"]
16mod ops;
17
14static SOURCE: LazyLock<(Vec<u8>, ExGuid)> = LazyLock::new(|| {18static SOURCE: LazyLock<(Vec<u8>, ExGuid)> = LazyLock::new(|| {
15 let source =19 let source =
16 onestore::create_section("model-schedule.one", "Original 🦀 é 0", "Author").unwrap();20 onestore::create_section("model-schedule.one", "Original 🦀 é 0", "Author").unwrap();
...@@ -23,9 +27,9 @@ static SOURCE: LazyLock<(Vec<u8>, ExGuid)> = LazyLock::new(|| {...@@ -23,9 +27,9 @@ static SOURCE: LazyLock<(Vec<u8>, ExGuid)> = LazyLock::new(|| {
23 for at in 1..8 {27 for at in 1..8 {
24 anchor = model_ops::insert_after(&mut page, anchor, &format!("Original 🦀 é {at}")).1;28 anchor = model_ops::insert_after(&mut page, anchor, &format!("Original 🦀 é {at}")).1;
25 }29 }
26 let source = PreparedEdit::page(&source, space, &page, "Author")30 let source = ops::saved(&source, space, &page)
27 .unwrap()31 .unwrap()
28 .as_bytes()32 .as_slice()
29 .to_vec();33 .to_vec();
30 (source, space)34 (source, space)
31});35});
...@@ -139,6 +143,10 @@ impl Remote for Session<'_> {...@@ -139,6 +143,10 @@ impl Remote for Session<'_> {
139 self.server.read()143 self.server.read()
140 }144 }
141145
146 fn stamp(&mut self) -> io::Result<Stamp> {
147 self.server.stamp()
148 }
149
142 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {150 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
143 assert!(!self.retired, "a retired attempt was replayed");151 assert!(!self.retired, "a retired attempt was replayed");
144 self.publications += 1;152 self.publications += 1;
...@@ -146,8 +154,8 @@ impl Remote for Session<'_> {...@@ -146,8 +154,8 @@ impl Remote for Session<'_> {
146 self.server.publish(transaction)154 self.server.publish(transaction)
147 }155 }
148156
149 fn confirm(&mut self, snapshot: &[u8]) -> Result<(), CommitError> {157 fn confirm(&mut self, base: &Stamp) -> Result<(), CommitError> {
150 self.server.confirm(snapshot)158 self.server.confirm(base)
151 }159 }
152}160}
153161
...@@ -165,11 +173,17 @@ fn statuses(cache: &Replica) -> Vec<(u64, Option<EditStatus>)> {...@@ -165,11 +173,17 @@ fn statuses(cache: &Replica) -> Vec<(u64, Option<EditStatus>)> {
165 .collect()173 .collect()
166}174}
167175
168/// Fails unless the durable remote holds `revision` in the edited page's space.176/// Fails unless the durable remote holds `revision`: in the edited page's space, or, for
177/// the edit adding a conflict page, in the space it changed first.
169fn durable(server: &Server, revision: ExGuid) {178fn durable(server: &Server, revision: ExGuid) {
170 let store = Store::parse(&server.durable).unwrap();179 let store = Store::parse(&server.durable).unwrap();
171 let index = RevisionIndex::parse(&store).unwrap();180 let index = RevisionIndex::parse(&store).unwrap();
172 assert!(index.spaces[&SOURCE.1].revisions.contains_key(&revision));181 assert!(
182 index
183 .spaces
184 .values()
185 .any(|space| space.revisions.contains_key(&revision))
186 );
173}187}
174188
175pub fn run(input: &[u8]) {189pub fn run(input: &[u8]) {
...@@ -181,10 +195,10 @@ pub fn run(input: &[u8]) {...@@ -181,10 +195,10 @@ pub fn run(input: &[u8]) {
181 let actor = usize::from(step[0]) % replicas.len();195 let actor = usize::from(step[0]) % replicas.len();
182 let path = directory.path().join(format!("{actor}.sqlite"));196 let path = directory.path().join(format!("{actor}.sqlite"));
183 let cache = replicas[actor].get_or_insert_with(|| Replica::create(&path, source).unwrap());197 let cache = replicas[actor].get_or_insert_with(|| Replica::create(&path, source).unwrap());
184 let snapshot = cache.snapshot().unwrap();198 let image = snapshot(cache);
185 let before = local(cache);199 let before = local(cache);
186 let pending = cache.pending().unwrap();200 let pending = cache.pending().unwrap();
187 let rows_now = rows(&snapshot);201 let rows_now = rows(&image);
188 match step[1] % 8 {202 match step[1] % 8 {
189 0..=2 => {203 0..=2 => {
190 let row = rows_now[usize::from(step[2]) % rows_now.len()].clone();204 let row = rows_now[usize::from(step[2]) % rows_now.len()].clone();
...@@ -266,12 +280,17 @@ pub fn run(input: &[u8]) {...@@ -266,12 +280,17 @@ pub fn run(input: &[u8]) {
266 };280 };
267 server.fault = Fault::None;281 server.fault = Fault::None;
268 let next = cache.pending().unwrap();282 let next = cache.pending().unwrap();
269 // Edits leave the queue oldest first, each with a durable receipt.283 // Edits leave the queue oldest first, each with a durable receipt; a rebase
270 let left = pending.len() - next.len();284 // queues the conflict pages it makes after them.
271 assert!(285 let queued = |edit: &notebook::PendingEdit| pending.iter().any(|kept| kept.id == edit.id);
272 next.iter()286 let left = pending.len() - next.iter().filter(|edit| queued(edit)).count();
273 .all(|edit| pending[left..].iter().any(|kept| kept.id == edit.id))287 assert!(next.iter().all(|edit| {
274 );288 pending[left..].iter().any(|kept| kept.id == edit.id)
289 || matches!(
290 &edit.edit.ops[..],
291 [onestore::op::Op::Section(onestore::op::SectionOp::Conflict { .. })]
292 )
293 }));
275 for edit in &pending[..left] {294 for edit in &pending[..left] {
276 let Some(EditStatus::Published { revision }) = cache.status(edit.id).unwrap()295 let Some(EditStatus::Published { revision }) = cache.status(edit.id).unwrap()
277 else {296 else {
...@@ -283,7 +302,7 @@ pub fn run(input: &[u8]) {...@@ -283,7 +302,7 @@ pub fn run(input: &[u8]) {
283 assert_eq!(302 assert_eq!(
284 local(cache),303 local(cache),
285 shape(&model_ops::page_of(304 shape(&model_ops::page_of(
286 &cache.remote_snapshot().unwrap(),305 &remote_snapshot(cache),
287 *space306 *space
288 ))307 ))
289 );308 );
...@@ -301,31 +320,41 @@ pub fn run(input: &[u8]) {...@@ -301,31 +320,41 @@ pub fn run(input: &[u8]) {
301 &char::from(b'a' + step[3] % 26).to_string(),320 &char::from(b'a' + step[3] % 26).to_string(),
302 );321 );
303 let visible = server.visible.clone();322 let visible = server.visible.clone();
304 PreparedEdit::page(&visible, *space, &page, "Remote")323 ops::save(&visible, *space, &page)
305 .unwrap()324 .unwrap()
306 .commit(&mut server)325 .commit(&mut server)
307 .unwrap();326 .unwrap();
308 }327 }
309 }328 }
310 6 => {329 6 => {
311 if let Some(conflict) = cache.conflict().unwrap() {330 // The writer merged a conflict page's version by hand and deletes it.
312 let keep = if step[2] & 1 == 0 {331 let listed = cache.conflicts().unwrap();
313 Resolution::Mine332 if let Some(conflict) = listed
314 } else {333 .iter()
315 Resolution::Theirs334 .find(|(page, _)| page == space)
316 };335 .map(|(_, pages)| pages[usize::from(step[2]) % pages.len()].space)
317 let remote = model_ops::page_of(&cache.remote_snapshot().unwrap(), *space);336 {
318 cache.resolve(conflict.id, keep).unwrap();337 let delete = onestore::op::Op::Section(onestore::op::SectionOp::Delete(vec![
319 match cache.conflict().unwrap() {338 conflict,
320 Some(next) => assert_ne!(next.space, conflict.space),339 ]));
321 None => assert_eq!(340 cache
322 local(cache),341 .apply(
323 match keep {342 AUTHOR,
324 Resolution::Mine => before.clone(),343 onestore::op::Edit {
325 Resolution::Theirs => shape(&remote),344 at: model_ops::now(),
326 }345 ops: vec![delete],
327 ),346 },
328 }347 )
348 .unwrap();
349 assert!(
350 cache
351 .conflicts()
352 .unwrap()
353 .iter()
354 .flat_map(|(_, pages)| pages)
355 .all(|page| page.space != conflict)
356 );
357 assert_eq!(local(cache), before);
329 let (result, publications) = {358 let (result, publications) = {
330 let mut session = Session {359 let mut session = Session {
331 retired: false,360 retired: false,
...@@ -356,6 +385,6 @@ pub fn run(input: &[u8]) {...@@ -356,6 +385,6 @@ pub fn run(input: &[u8]) {
356 }385 }
357 }386 }
358 rows(&server.durable);387 rows(&server.durable);
359 rows(&replicas[actor].as_ref().unwrap().snapshot().unwrap());388 rows(&snapshot(replicas[actor].as_ref().unwrap()));
360 }389 }
361}390}
crates/notebook/tests/support/page_schedule.rs+34-40
...@@ -1,8 +1,12 @@...@@ -1,8 +1,12 @@
1use crate::{disk, model_ops};1use crate::{
2use notebook::{EditStatus, Remote, Replica, Resolution};2 disk, model_ops,
3 server::{remote_snapshot, snapshot},
4};
5use notebook::{Remote, Replica};
3use onestore::{6use onestore::{
4 CommitError, ExGuid, PageEdit, PreparedEdit, RevisionIndex, Store, Transaction,7 CommitError, ExGuid, PageEdit, RevisionIndex, Stamp, Store, Transaction,
5 document::{Document, Format, Kind, Layout},8 document::{Document, Format, Kind, Layout},
9 op::SectionOp,
6 page::{10 page::{
7 Outline, Page, PageObject, PageParagraph, Paragraph, ParagraphContent, TextObject,11 Outline, Page, PageObject, PageParagraph, Paragraph, ParagraphContent, TextObject,
8 text::new_id,12 text::new_id,
...@@ -10,6 +14,9 @@ use onestore::{...@@ -10,6 +14,9 @@ use onestore::{
10};14};
11use std::{io, sync::LazyLock};15use std::{io, sync::LazyLock};
1216
17#[path = "../../../onestore/tests/support/ops.rs"]
18mod ops;
19
13const BODY: &str = "Body 🦋 é";20const BODY: &str = "Body 🦋 é";
1421
15#[path = "../../../onestore/tests/support/current.rs"]22#[path = "../../../onestore/tests/support/current.rs"]
...@@ -19,7 +26,7 @@ static SOURCE: LazyLock<Vec<u8>> = LazyLock::new(|| {...@@ -19,7 +26,7 @@ static SOURCE: LazyLock<Vec<u8>> = LazyLock::new(|| {
19 let mut source = onestore::create_section("page-schedule.one", "Original", "Author").unwrap();26 let mut source = onestore::create_section("page-schedule.one", "Original", "Author").unwrap();
20 for _ in 0..3 {27 for _ in 0..3 {
21 let page = onestore::PageCreation::new(None, Some("Same title"), "Author").unwrap();28 let page = onestore::PageCreation::new(None, Some("Same title"), "Author").unwrap();
22 source = PreparedEdit::create_page(&source, &page)29 source = ops::section_op(&source, SectionOp::Create(page.clone()))
23 .unwrap()30 .unwrap()
24 .as_bytes()31 .as_bytes()
25 .to_vec();32 .to_vec();
...@@ -107,6 +114,10 @@ impl Remote for Session<'_> {...@@ -107,6 +114,10 @@ impl Remote for Session<'_> {
107 Ok(self.disk.visible.clone())114 Ok(self.disk.visible.clone())
108 }115 }
109116
117 fn stamp(&mut self) -> io::Result<Stamp> {
118 Stamp::of(&self.disk.visible).map_err(io::Error::other)
119 }
120
110 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {121 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
111 let mut image = self.disk.visible.clone();122 let mut image = self.disk.visible.clone();
112 transaction.apply(&mut image).unwrap();123 transaction.apply(&mut image).unwrap();
...@@ -121,8 +132,8 @@ impl Remote for Session<'_> {...@@ -121,8 +132,8 @@ impl Remote for Session<'_> {
121 result132 result
122 }133 }
123134
124 fn confirm(&mut self, snapshot: &[u8]) -> Result<(), CommitError> {135 fn confirm(&mut self, base: &Stamp) -> Result<(), CommitError> {
125 onestore::confirm_snapshot(self.disk, snapshot)136 onestore::confirm(self.disk, base)
126 }137 }
127}138}
128139
...@@ -152,8 +163,8 @@ pub fn run(input: &[u8]) {...@@ -152,8 +163,8 @@ pub fn run(input: &[u8]) {
152 let actor = usize::from(step[0]) % replicas.len();163 let actor = usize::from(step[0]) % replicas.len();
153 let path = directory.path().join(format!("{actor}.sqlite"));164 let path = directory.path().join(format!("{actor}.sqlite"));
154 let cache = replicas[actor].get_or_insert_with(|| Replica::create(&path, &SOURCE).unwrap());165 let cache = replicas[actor].get_or_insert_with(|| Replica::create(&path, &SOURCE).unwrap());
155 let snapshot = cache.snapshot().unwrap();166 let image = snapshot(cache);
156 let listed = pages(&snapshot);167 let listed = pages(&image);
157 let pending = cache.pending().unwrap();168 let pending = cache.pending().unwrap();
158 match step[1] % 8 {169 match step[1] % 8 {
159 0 | 1 => {170 0 | 1 => {
...@@ -183,7 +194,7 @@ pub fn run(input: &[u8]) {...@@ -183,7 +194,7 @@ pub fn run(input: &[u8]) {
183 .collect();194 .collect();
184 replicas[actor] = None;195 replicas[actor] = None;
185 let cache = Replica::open(&path).unwrap();196 let cache = Replica::open(&path).unwrap();
186 assert_eq!(pages(&cache.snapshot().unwrap()), listed);197 assert_eq!(pages(&snapshot(&cache)), listed);
187 assert_eq!(cache.pending().unwrap(), pending);198 assert_eq!(cache.pending().unwrap(), pending);
188 assert_eq!(199 assert_eq!(
189 pending200 pending
...@@ -200,24 +211,9 @@ pub fn run(input: &[u8]) {...@@ -200,24 +211,9 @@ pub fn run(input: &[u8]) {
200 disk.fail_at =211 disk.fail_at =
201 (step[2] != 0).then_some(usize::from(u16::from_le_bytes([step[2], step[3]])));212 (step[2] != 0).then_some(usize::from(u16::from_le_bytes([step[2], step[3]])));
202 disk.random = u64::from(step[4]) + 1;213 disk.random = u64::from(step[4]) + 1;
203 let prior = pending
204 .first()
205 .and_then(|edit| cache.status(edit.id).unwrap());
206 let result = cache.sync_once(&mut Session { disk: &mut disk });214 let result = cache.sync_once(&mut Session { disk: &mut disk });
207 if matches!(prior, Some(EditStatus::AwaitingConfirmation { .. })) {
208 assert!(
209 result.is_err()
210 || !matches!(
211 result.as_ref().unwrap().edit,
212 Some((_, EditStatus::Conflict(_)))
213 )
214 );
215 }
216 if result.is_ok() && cache.pending().unwrap().is_empty() {215 if result.is_ok() && cache.pending().unwrap().is_empty() {
217 assert_eq!(216 assert_eq!(pages(&snapshot(cache)), pages(&remote_snapshot(cache)));
218 pages(&cache.snapshot().unwrap()),
219 pages(&cache.remote_snapshot().unwrap())
220 );
221 }217 }
222 }218 }
223 5 => {219 5 => {
...@@ -244,36 +240,34 @@ pub fn run(input: &[u8]) {...@@ -244,36 +240,34 @@ pub fn run(input: &[u8]) {
244 }240 }
245 })241 })
246 .collect();242 .collect();
247 let expected = PreparedEdit::pages(&snapshot, &edits);243 let expected = ops::section_op(&image, SectionOp::Pages(edits.to_vec()));
248 match section_op(cache, onestore::op::SectionOp::Pages(edits)) {244 match section_op(cache, onestore::op::SectionOp::Pages(edits)) {
249 Ok(_) => assert_eq!(245 Ok(_) => assert_eq!(
250 pages(&cache.snapshot().unwrap()),246 pages(&snapshot(cache)),
251 pages(expected.unwrap().as_bytes())247 pages(expected.unwrap().as_bytes())
252 ),248 ),
253 Err(_) => {249 Err(_) => {
254 assert!(expected.is_err());250 assert!(expected.is_err());
255 assert_eq!(pages(&cache.snapshot().unwrap()), listed);251 assert_eq!(pages(&snapshot(cache)), listed);
256 assert_eq!(cache.pending().unwrap(), pending);252 assert_eq!(cache.pending().unwrap(), pending);
257 }253 }
258 }254 }
259 }255 }
260 7 => {256 7 => {
261 if let Some(conflict) = cache.conflict().unwrap() {257 // The actor merged a conflict page's version by hand and deletes it.
262 // Taking theirs for the page list would drop the pages this actor made.258 if let Some(conflict) = cache
263 let root = listed.iter().all(|(sid, ..)| *sid != conflict.space);259 .conflicts()
264 let keep = if root || step[2] & 1 == 0 {260 .unwrap()
265 Resolution::Mine261 .first()
266 } else {262 .map(|(_, pages)| pages[usize::from(step[2]) % pages.len()].space)
267 Resolution::Theirs263 {
268 };264 section_op(cache, onestore::op::SectionOp::Delete(vec![conflict])).unwrap();
269 if cache.resolve(conflict.id, keep).is_err() {265 assert_eq!(pages(&snapshot(cache)), listed);
270 assert_eq!(cache.pending().unwrap(), pending);
271 }
272 }266 }
273 }267 }
274 _ => unreachable!(),268 _ => unreachable!(),
275 }269 }
276 let local = pages(&replicas[actor].as_ref().unwrap().snapshot().unwrap());270 let local = pages(&snapshot(replicas[actor].as_ref().unwrap()));
277 for page in &owned[actor] {271 for page in &owned[actor] {
278 assert!(local.iter().any(|(sid, oid, _)| (*sid, *oid) == *page));272 assert!(local.iter().any(|(sid, oid, _)| (*sid, *oid) == *page));
279 }273 }
crates/notebook/tests/support/server.rs+101-4
...@@ -75,8 +75,8 @@ impl Remote for Server {...@@ -75,8 +75,8 @@ impl Remote for Server {
75 fn read(&mut self) -> io::Result<Vec<u8>> {75 fn read(&mut self) -> io::Result<Vec<u8>> {
76 Ok(self.visible.clone())76 Ok(self.visible.clone())
77 }77 }
78 fn stamp(&mut self) -> io::Result<Option<Stamp>> {78 fn stamp(&mut self) -> io::Result<Stamp> {
79 Ok(Stamp::of(&self.visible).ok())79 Stamp::of(&self.visible).map_err(io::Error::other)
80 }80 }
81 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {81 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
82 self.publications += 1;82 self.publications += 1;
...@@ -99,13 +99,13 @@ impl Remote for Server {...@@ -99,13 +99,13 @@ impl Remote for Server {
99 _ => Ok(()),99 _ => Ok(()),
100 }100 }
101 }101 }
102 fn confirm(&mut self, snapshot: &[u8]) -> Result<(), CommitError> {102 fn confirm(&mut self, base: &Stamp) -> Result<(), CommitError> {
103 self.confirmations += 1;103 self.confirmations += 1;
104 if matches!(self.fault, Fault::Confirm) {104 if matches!(self.fault, Fault::Confirm) {
105 self.fault = Fault::None;105 self.fault = Fault::None;
106 return Err(failure(CommitState::Unknown));106 return Err(failure(CommitState::Unknown));
107 }107 }
108 onestore::confirm_snapshot(self, snapshot)?;108 onestore::confirm(self, base)?;
109 if matches!(self.fault, Fault::ConfirmCommitted) {109 if matches!(self.fault, Fault::ConfirmCommitted) {
110 self.fault = Fault::None;110 self.fault = Fault::None;
111 return Err(failure(CommitState::Committed));111 return Err(failure(CommitState::Committed));
...@@ -165,3 +165,100 @@ pub fn section_op(cache: &notebook::Replica, op: onestore::op::SectionOp) -> u64...@@ -165,3 +165,100 @@ pub fn section_op(cache: &notebook::Replica, op: onestore::op::SectionOp) -> u64
165 )165 )
166 .unwrap()166 .unwrap()
167}167}
168
169/// The image the queue leaves, as a recovery archive of the cache records it.
170pub fn snapshot(cache: &notebook::Replica) -> Vec<u8> {
171 archived(cache, |recovery| recovery.snapshot())
172}
173
174/// The remote image the cache last observed, as a recovery archive records it.
175pub fn remote_snapshot(cache: &notebook::Replica) -> Vec<u8> {
176 archived(cache, |recovery| recovery.remote_snapshot())
177}
178
179fn archived(
180 cache: &notebook::Replica,
181 image: impl FnOnce(&notebook::Recovery) -> Result<Vec<u8>, notebook::Error>,
182) -> Vec<u8> {
183 let directory = tempfile::tempdir().unwrap();
184 let path = directory.path().join("recovery.sqlite");
185 cache.export_recovery(&path).unwrap();
186 image(&notebook::Recovery::open(&path).unwrap()).unwrap()
187}
188
189/// `image` after another writer replaces `range` of the text object `text` with `with`.
190pub fn typed(
191 image: &[u8],
192 space: ExGuid,
193 text: ExGuid,
194 range: std::ops::Range<u32>,
195 with: &str,
196) -> Vec<u8> {
197 let op = onestore::op::PageOp::Text {
198 text,
199 range,
200 with: with.into(),
201 };
202 edited(image, vec![onestore::op::Op::Page { space, op }]).unwrap()
203}
204
205/// `image` after another writer's `ops`, sealed as one revision per space they change.
206pub fn edited(image: &[u8], ops: Vec<onestore::op::Op>) -> Result<Vec<u8>, onestore::op::OpError> {
207 let arena = onestore::Arena::default();
208 let mut section =
209 onestore::Section::open(&arena, image.to_vec()).map_err(onestore::op::OpError::Failed)?;
210 let edit = onestore::op::Edit {
211 at: 134_000_000_000_000_000,
212 ops,
213 };
214 section.apply("Other writer", &edit)?;
215 section.seal().map_err(onestore::op::OpError::Failed)?;
216 Ok(section.image())
217}
218
219/// A conflict page as the tests compare it: whose version it is and its texts.
220pub type Version = (String, Vec<String>);
221
222/// Each page's conflict pages in list order.
223pub fn conflicts(image: &[u8]) -> Vec<(ExGuid, Vec<Version>)> {
224 let arena = onestore::Arena::default();
225 let mut section = onestore::Section::open(&arena, image.to_vec()).unwrap();
226 section
227 .conflicts()
228 .unwrap()
229 .into_iter()
230 .map(|(page, conflicts)| {
231 let conflicts = conflicts
232 .into_iter()
233 .map(|conflict| {
234 let page = section.page(conflict.space).unwrap();
235 (conflict.user, page_texts(&page))
236 })
237 .collect();
238 (page, conflicts)
239 })
240 .collect()
241}
242
243/// The texts of a page's outlines and title, in page order.
244pub fn page_texts(page: &onestore::page::Page) -> Vec<String> {
245 use onestore::page::PageObject;
246 page.objects
247 .iter()
248 .flat_map(|object| match object {
249 PageObject::Outline(outline) => outline.paragraphs.clone(),
250 PageObject::Title(title) => title
251 .outlines
252 .iter()
253 .flat_map(|outline| outline.paragraphs.clone())
254 .collect(),
255 _ => Vec::new(),
256 })
257 .filter_map(|paragraph| Some(paragraph.text()?.text.text().to_owned()))
258 .collect()
259}
260
261/// Whether `image` lists a conflict page under page `space`.
262pub fn conflicted(image: &[u8], space: ExGuid) -> bool {
263 conflicts(image).iter().any(|(page, _)| *page == space)
264}
crates/notebook/tests/sync.rs+148-270
...@@ -1,6 +1,6 @@...@@ -1,6 +1,6 @@
1//! Replica protocol: durable receipts, retired attempts, conflict resolution and contention.1//! Replica protocol: durable receipts, retired attempts, conflict pages and contention.
22
3use notebook::{ConflictKind, EditStatus, Error, Remote, Replica, Resolution};3use notebook::{EditStatus, Error, Remote, Replica};
4use onestore::{CommitError, CommitState, ExGuid, Transaction};4use onestore::{CommitError, CommitState, ExGuid, Transaction};
5use std::{io, ops::Range};5use std::{io, ops::Range};
66
...@@ -29,14 +29,14 @@ fn an_unchanged_stamp_publishes_and_settles_without_reading_the_remote() {...@@ -29,14 +29,14 @@ fn an_unchanged_stamp_publishes_and_settles_without_reading_the_remote() {
29 self.reads += 1;29 self.reads += 1;
30 self.server.read()30 self.server.read()
31 }31 }
32 fn stamp(&mut self) -> io::Result<Option<onestore::Stamp>> {32 fn stamp(&mut self) -> io::Result<onestore::Stamp> {
33 self.server.stamp()33 self.server.stamp()
34 }34 }
35 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {35 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
36 self.server.publish(transaction)36 self.server.publish(transaction)
37 }37 }
38 fn confirm(&mut self, snapshot: &[u8]) -> Result<(), CommitError> {38 fn confirm(&mut self, base: &onestore::Stamp) -> Result<(), CommitError> {
39 self.server.confirm(snapshot)39 self.server.confirm(base)
40 }40 }
41 }41 }
42 let directory = tempfile::tempdir().unwrap();42 let directory = tempfile::tempdir().unwrap();
...@@ -56,11 +56,11 @@ fn an_unchanged_stamp_publishes_and_settles_without_reading_the_remote() {...@@ -56,11 +56,11 @@ fn an_unchanged_stamp_publishes_and_settles_without_reading_the_remote() {
56 assert_eq!(remote.reads, 0);56 assert_eq!(remote.reads, 0);
57 // Another writer's commit moves the header, so the next step reads the file.57 // Another writer's commit moves the header, so the next step reads the file.
58 let native =58 let native =
59 onestore::replace_text(&remote.server.visible, sid, object, 0..0, "Native ").unwrap();59 typed(&remote.server.visible, sid, object, 0..0, "Native ");
60 remote.server.visible.clone_from(&native);60 remote.server.visible.clone_from(&native);
61 assert_eq!(cache.sync_once(&mut remote).unwrap().edit, None);61 assert_eq!(cache.sync_once(&mut remote).unwrap().edit, None);
62 assert_eq!(remote.reads, 1);62 assert_eq!(remote.reads, 1);
63 assert_eq!(cache.snapshot().unwrap(), native);63 assert_eq!(snapshot(&cache), native);
64}64}
6565
66#[test]66#[test]
...@@ -89,14 +89,13 @@ fn recovery_archive_preserves_the_queue_uncertainty_and_receipts_without_becomin...@@ -89,14 +89,13 @@ fn recovery_archive_preserves_the_queue_uncertainty_and_receipts_without_becomin
89 let created = onestore::PageCreation::new(None, Some("Recovery 🦀"), "Fixture").unwrap();89 let created = onestore::PageCreation::new(None, Some("Recovery 🦀"), "Fixture").unwrap();
90 section_op(&cache, onestore::op::SectionOp::Create(created));90 section_op(&cache, onestore::op::SectionOp::Create(created));
91 save(&cache, object, 0..0, "Last ").unwrap();91 save(&cache, object, 0..0, "Last ").unwrap();
92 let working = cache.snapshot().unwrap();92 let working = snapshot(&cache);
93 let remote = cache.remote_snapshot().unwrap();93 let remote = remote_snapshot(&cache);
94 let pending = cache.pending().unwrap();94 let pending = cache.pending().unwrap();
95 let uncertain = cache.status(attempted).unwrap();95 let uncertain = cache.status(attempted).unwrap();
96 let source_file = std::fs::read(&path).unwrap();96 let source_file = std::fs::read(&path).unwrap();
97 let summary = RecoverySummary {97 let summary = RecoverySummary {
98 queued_edits: 4,98 queued_edits: 4,
99 conflicts: 0,
100 uncertain_edits: 1,99 uncertain_edits: 1,
101 published_receipts: 1,100 published_receipts: 1,
102 base_bytes: remote.len() as u64,101 base_bytes: remote.len() as u64,
...@@ -173,28 +172,21 @@ fn recovery_archive_preserves_the_queue_uncertainty_and_receipts_without_becomin...@@ -173,28 +172,21 @@ fn recovery_archive_preserves_the_queue_uncertainty_and_receipts_without_becomin
173}172}
174173
175#[test]174#[test]
176fn recovery_archive_retains_conflict_images_and_rejects_foreign_or_future_archives() {175fn recovery_archive_retains_the_queue_and_rejects_foreign_or_future_archives() {
177 use notebook::Recovery;176 use notebook::Recovery;
178177
179 let directory = tempfile::tempdir().unwrap();178 let directory = tempfile::tempdir().unwrap();
180 let source = onestore::create_section("recovery.one", "Original", "Fixture").unwrap();179 let source = onestore::create_section("recovery.one", "Original", "Fixture").unwrap();
181 let (space, object, _) = text(&source);180 let (_, object, _) = text(&source);
182 let cache = Replica::create(directory.path().join("live.sqlite"), &source).unwrap();181 let cache = Replica::create(directory.path().join("live.sqlite"), &source).unwrap();
183 let id = save(&cache, object, 0..8, "Local").unwrap();182 let id = save(&cache, object, 0..8, "Local").unwrap();
184 let changed = onestore::replace_text(&source, space, object, 0..8, "Remote").unwrap();183 let path = directory.path().join("queued.sqlite");
185 let mut server = Server::new(&changed);
186 let outcome = cache.sync_once(&mut server).unwrap().edit.unwrap();
187 assert_eq!(
188 outcome,
189 (id, EditStatus::Conflict(ConflictKind::ContentChanged))
190 );
191 let path = directory.path().join("conflict.sqlite");
192 cache.export_recovery(&path).unwrap();184 cache.export_recovery(&path).unwrap();
193 let archive = Recovery::open(&path).unwrap();185 let archive = Recovery::open(&path).unwrap();
194 assert_eq!(text(&archive.snapshot().unwrap()).2, "Local");186 assert_eq!(text(&archive.snapshot().unwrap()).2, "Local");
195 assert_eq!(archive.remote_snapshot().unwrap(), changed);187 assert_eq!(archive.remote_snapshot().unwrap(), source);
196 assert_eq!(archive.status(id).unwrap(), Some(outcome.1.clone()));188 assert_eq!(archive.status(id).unwrap(), Some(EditStatus::Pending));
197 assert_eq!(archive.summary().unwrap().conflicts, 1);189 assert_eq!(archive.summary().unwrap().queued_edits, 1);
198 assert_eq!(archive.summary().unwrap().uncertain_edits, 0);190 assert_eq!(archive.summary().unwrap().uncertain_edits, 0);
199 drop(archive);191 drop(archive);
200 for sql in [192 for sql in [
...@@ -208,8 +200,8 @@ fn recovery_archive_retains_conflict_images_and_rejects_foreign_or_future_archiv...@@ -208,8 +200,8 @@ fn recovery_archive_retains_conflict_images_and_rejects_foreign_or_future_archiv
208 assert!(Recovery::open(&path).is_err());200 assert!(Recovery::open(&path).is_err());
209 assert_eq!(std::fs::read(&path).unwrap(), before);201 assert_eq!(std::fs::read(&path).unwrap(), before);
210 }202 }
211 assert_eq!(cache.status(id).unwrap(), Some(outcome.1.clone()));203 assert_eq!(cache.status(id).unwrap(), Some(EditStatus::Pending));
212 assert_eq!(text(&cache.snapshot().unwrap()).2, "Local");204 assert_eq!(text(&snapshot(&cache)).2, "Local");
213}205}
214206
215#[test]207#[test]
...@@ -220,13 +212,13 @@ fn disjoint_remote_changes_merge_and_persist_the_remote_receipt() {...@@ -220,13 +212,13 @@ fn disjoint_remote_changes_merge_and_persist_the_remote_receipt() {
220 let (sid, oid, _) = text(&source);212 let (sid, oid, _) = text(&source);
221 let cache = Replica::create(&path, &source).unwrap();213 let cache = Replica::create(&path, &source).unwrap();
222 let id = save(&cache, oid, 2..4, "🐈").unwrap();214 let id = save(&cache, oid, 2..4, "🐈").unwrap();
223 let remote = onestore::replace_text(&source, sid, oid, 0..6, "Xab🦀cYd").unwrap();215 let remote = typed(&source, sid, oid, 0..6, "Xab🦀cYd");
224 let mut server = Server::new(&remote);216 let mut server = Server::new(&remote);
225 let outcome = cache.sync_once(&mut server).unwrap().edit.unwrap();217 let outcome = cache.sync_once(&mut server).unwrap().edit.unwrap();
226 assert_eq!(outcome.0, id);218 assert_eq!(outcome.0, id);
227 assert!(matches!(outcome.1, EditStatus::Published { .. }));219 assert!(matches!(outcome.1, EditStatus::Published { .. }));
228 assert_eq!(text(&server.durable).2, "Xab🐈cYd");220 assert_eq!(text(&server.durable).2, "Xab🐈cYd");
229 assert_eq!(cache.snapshot().unwrap(), server.durable);221 assert_eq!(snapshot(&cache), server.durable);
230 assert!(cache.pending().unwrap().is_empty());222 assert!(cache.pending().unwrap().is_empty());
231 drop(cache);223 drop(cache);
232 let cache = Replica::open(&path).unwrap();224 let cache = Replica::open(&path).unwrap();
...@@ -238,31 +230,38 @@ fn disjoint_remote_changes_merge_and_persist_the_remote_receipt() {...@@ -238,31 +230,38 @@ fn disjoint_remote_changes_merge_and_persist_the_remote_receipt() {
238 assert!(next > id);230 assert!(next > id);
239}231}
240232
233/// As OneNote 2010 does, the remote's version stays the page and the local one becomes a
234/// conflict page under it, published with the rest of the queue.
241#[test]235#[test]
242fn overlapping_changes_preserve_both_images_and_survive_restart() {236fn overlapping_changes_keep_both_versions_and_survive_restart() {
243 let dir = tempfile::tempdir().unwrap();237 let dir = tempfile::tempdir().unwrap();
244 let path = dir.path().join("cache.sqlite");238 let path = dir.path().join("cache.sqlite");
245 let source = onestore::create_section("sync.one", "abc", "Fixture").unwrap();239 let source = onestore::create_section("sync.one", "abc", "Fixture").unwrap();
246 let (sid, oid, _) = text(&source);240 let (sid, oid, _) = text(&source);
247 let cache = Replica::create(&path, &source).unwrap();241 let cache = Replica::create(&path, &source).unwrap();
248 let id = save(&cache, oid, 1..2, "L").unwrap();242 let id = save(&cache, oid, 1..2, "L").unwrap();
249 let local = cache.snapshot().unwrap();243 let remote = typed(&source, sid, oid, 1..2, "R");
250 let remote = onestore::replace_text(&source, sid, oid, 1..2, "R").unwrap();
251 let mut server = Server::new(&remote);244 let mut server = Server::new(&remote);
252 assert_eq!(245 assert!(matches!(
253 cache.sync_once(&mut server).unwrap().edit,246 cache.sync_once(&mut server).unwrap().edit,
254 Some((id, EditStatus::Conflict(ConflictKind::ContentChanged)))247 Some((newest, EditStatus::Published { .. })) if newest > id
248 ));
249 assert_eq!(server.publications, 1);
250 assert_eq!(content(&server.durable, oid), "aRc");
251 assert_eq!(
252 conflicts(&server.durable),
253 [(sid, vec![(model_ops::AUTHOR.to_owned(), vec!["aLc".to_owned()])])]
255 );254 );
256 assert_eq!(server.publications, 0);255 assert_eq!(snapshot(&cache), server.durable);
256 assert!(cache.pending().unwrap().is_empty());
257 drop(cache);257 drop(cache);
258 let cache = Replica::open(&path).unwrap();258 let cache = Replica::open(&path).unwrap();
259 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));259 assert!(matches!(
260 assert_eq!(cache.remote_snapshot().unwrap(), remote);
261 assert_eq!(cache.pending().unwrap().len(), 1);
262 assert_eq!(
263 cache.status(id).unwrap(),260 cache.status(id).unwrap(),
264 Some(EditStatus::Conflict(ConflictKind::ContentChanged))261 Some(EditStatus::Published { .. })
265 );262 ));
263 assert_eq!(cache.conflicts().unwrap()[0].0, sid);
264 assert_eq!(cache.sync_once(&mut server).unwrap().edit, None);
266}265}
267266
268#[test]267#[test]
...@@ -452,16 +451,19 @@ fn twelve_local_editors_progress_during_remote_reads_publication_and_confirmatio...@@ -452,16 +451,19 @@ fn twelve_local_editors_progress_during_remote_reads_publication_and_confirmatio
452 }451 }
453 impl Remote for Paused {452 impl Remote for Paused {
454 fn read(&mut self) -> io::Result<Vec<u8>> {453 fn read(&mut self) -> io::Result<Vec<u8>> {
455 self.wait("read");
456 self.server.read()454 self.server.read()
457 }455 }
456 fn stamp(&mut self) -> io::Result<onestore::Stamp> {
457 self.wait("read");
458 self.server.stamp()
459 }
458 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {460 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
459 self.wait("publish");461 self.wait("publish");
460 self.server.publish(transaction)462 self.server.publish(transaction)
461 }463 }
462 fn confirm(&mut self, source: &[u8]) -> Result<(), CommitError> {464 fn confirm(&mut self, base: &onestore::Stamp) -> Result<(), CommitError> {
463 self.wait("confirm");465 self.wait("confirm");
464 self.server.confirm(source)466 self.server.confirm(base)
465 }467 }
466 }468 }
467 for phase in ["read", "publish", "confirm"] {469 for phase in ["read", "publish", "confirm"] {
...@@ -505,9 +507,6 @@ fn twelve_local_editors_progress_during_remote_reads_publication_and_confirmatio...@@ -505,9 +507,6 @@ fn twelve_local_editors_progress_during_remote_reads_publication_and_confirmatio
505 assert!(507 assert!(
506 matches!(cache.sync_once(&mut Server::new(&source)),Err(Error::Io(error)) if error.kind()==io::ErrorKind::WouldBlock)508 matches!(cache.sync_once(&mut Server::new(&source)),Err(Error::Io(error)) if error.kind()==io::ErrorKind::WouldBlock)
507 );509 );
508 assert!(
509 matches!(cache.resolve(first, Resolution::Theirs), Err(Error::Io(error)) if error.kind() == io::ErrorKind::WouldBlock)
510 );
511 let started = std::time::Instant::now();510 let started = std::time::Instant::now();
512 let handles: Vec<_> = (0..12)511 let handles: Vec<_> = (0..12)
513 .map(|writer| {512 .map(|writer| {
...@@ -550,7 +549,7 @@ fn twelve_local_editors_progress_during_remote_reads_publication_and_confirmatio...@@ -550,7 +549,7 @@ fn twelve_local_editors_progress_during_remote_reads_publication_and_confirmatio
550 cache.pending().unwrap().len(),549 cache.pending().unwrap().len(),
551 if phase == "read" { 0 } else { 12 }550 if phase == "read" { 0 } else { 12 }
552 );551 );
553 let expected = text(&cache.snapshot().unwrap()).2;552 let expected = text(&snapshot(&cache)).2;
554 for writer in 0..12 {553 for writer in 0..12 {
555 assert_eq!(expected.matches(&format!("[{writer}] ")).count(), 1);554 assert_eq!(expected.matches(&format!("[{writer}] ")).count(), 1);
556 }555 }
...@@ -562,7 +561,7 @@ fn twelve_local_editors_progress_during_remote_reads_publication_and_confirmatio...@@ -562,7 +561,7 @@ fn twelve_local_editors_progress_during_remote_reads_publication_and_confirmatio
562 }561 }
563 assert_eq!(server.publications, if phase == "read" { 1 } else { 2 });562 assert_eq!(server.publications, if phase == "read" { 1 } else { 2 });
564 assert_eq!(text(&server.durable).2, expected);563 assert_eq!(text(&server.durable).2, expected);
565 assert_eq!(cache.snapshot().unwrap(), server.durable);564 assert_eq!(snapshot(&cache), server.durable);
566 }565 }
567}566}
568567
...@@ -578,12 +577,12 @@ fn unrelated_remote_files_never_replace_a_local_cache() {...@@ -578,12 +577,12 @@ fn unrelated_remote_files_never_replace_a_local_cache() {
578 if pending {577 if pending {
579 save(&cache, oid, 0..0, "Local ").unwrap();578 save(&cache, oid, 0..0, "Local ").unwrap();
580 }579 }
581 let before = pages(&cache.snapshot().unwrap());580 let before = pages(&snapshot(&cache));
582 assert!(581 assert!(
583 matches!(cache.sync_once(&mut server),Err(Error::Io(error)) if error.kind()==io::ErrorKind::InvalidInput)582 matches!(cache.sync_once(&mut server),Err(Error::Io(error)) if error.kind()==io::ErrorKind::InvalidInput)
584 );583 );
585 assert_eq!(pages(&cache.snapshot().unwrap()), before);584 assert_eq!(pages(&snapshot(&cache)), before);
586 assert_eq!(cache.remote_snapshot().unwrap(), source);585 assert_eq!(remote_snapshot(&cache), source);
587 assert_eq!(server.publications, 0);586 assert_eq!(server.publications, 0);
588 }587 }
589}588}
...@@ -600,165 +599,76 @@ fn content(bytes: &[u8], text: ExGuid) -> String {...@@ -600,165 +599,76 @@ fn content(bytes: &[u8], text: ExGuid) -> String {
600 .to_owned()599 .to_owned()
601}600}
602601
602/// A conflict never holds the queue: later edits of the page apply to the remote's version.
603#[test]603#[test]
604fn a_conflict_holds_later_edits_until_resolved_either_way_and_survives_reopen() {604fn edits_after_a_conflict_apply_to_the_remote_version_and_survive_reopen() {
605 for resolution in [Resolution::Mine, Resolution::Theirs] {
606 let dir = tempfile::tempdir().unwrap();
607 let path = dir.path().join("cache.sqlite");
608 let source = onestore::create_section("resolve.one", "abc", "Fixture").unwrap();
609 let (sid, oid, _) = text(&source);
610 let cache = Replica::create(&path, &source).unwrap();
611 let first = save(&cache, oid, 1..2, "L").unwrap();
612 let remote = onestore::replace_text(&source, sid, oid, 0..3, "aRcZ").unwrap();
613 let mut server = Server::new(&remote);
614 let conflict = EditStatus::Conflict(ConflictKind::ContentChanged);
615 assert_eq!(
616 cache.sync_once(&mut server).unwrap().edit,
617 Some((first, conflict.clone()))
618 );
619 let mut dependent = None;
620 for n in 0..12 {
621 dependent = save(&cache, oid, 0..0, &format!("[{n}] "));
622 }
623 let dependent = dependent.unwrap();
624 assert_ne!(dependent, first);
625 assert_eq!(cache.status(dependent).unwrap(), Some(conflict.clone()));
626 let local = content(&cache.snapshot().unwrap(), oid);
627 assert!(local.ends_with("[0] aLc"), "{local}");
628 let pending = cache.pending().unwrap();
629 assert_eq!(pending.len(), 13);
630 assert_eq!(
631 cache.sync_once(&mut server).unwrap().edit,
632 Some((dependent, conflict.clone())),
633 "an unchanged remote leaves the conflict as it was"
634 );
635 assert_eq!(server.publications, 0);
636 drop(cache);
637 let cache = Replica::open(&path).unwrap();
638 assert_eq!(cache.pending().unwrap(), pending);
639 assert_eq!(content(&cache.snapshot().unwrap(), oid), local);
640 assert_eq!(
641 cache.conflict().unwrap(),
642 Some(notebook::Conflict {
643 id: dependent,
644 space: sid,
645 kind: ConflictKind::ContentChanged
646 })
647 );
648 cache.resolve(first, resolution).unwrap();
649 assert_eq!(cache.conflict().unwrap(), None);
650 assert_eq!(cache.remote_snapshot().unwrap(), remote);
651 let expected = match resolution {
652 Resolution::Mine => local.clone(),
653 Resolution::Theirs => "aRcZ".to_owned(),
654 };
655 assert_eq!(content(&cache.snapshot().unwrap(), oid), expected);
656 drop(cache);
657 let cache = Replica::open(&path).unwrap();
658 match resolution {
659 Resolution::Mine => {
660 assert!(matches!(
661 cache.sync_once(&mut server).unwrap().edit,
662 Some((_, EditStatus::Published { .. }))
663 ));
664 assert_eq!(server.publications, 1);
665 for edit in &pending {
666 assert!(matches!(
667 cache.status(edit.id).unwrap(),
668 None | Some(EditStatus::Published { .. })
669 ));
670 }
671 assert!(matches!(
672 cache.status(first).unwrap(),
673 Some(EditStatus::Published { .. })
674 ));
675 }
676 Resolution::Theirs => {
677 // The emptied edits are acknowledged once the remote image is flushed.
678 assert!(matches!(
679 cache.sync_once(&mut server).unwrap().edit,
680 Some((id, EditStatus::Published { .. })) if id == dependent
681 ));
682 assert_eq!((server.publications, server.confirmations), (0, 1));
683 assert!(matches!(
684 cache.status(first).unwrap(),
685 Some(EditStatus::Published { .. })
686 ));
687 }
688 }
689 assert_eq!(content(&server.durable, oid), expected);
690 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&server.durable));
691 assert!(cache.pending().unwrap().is_empty());
692 }
693}
694
695#[test]
696fn resolution_refuses_edits_that_are_not_in_conflict() {
697 let dir = tempfile::tempdir().unwrap();605 let dir = tempfile::tempdir().unwrap();
606 let path = dir.path().join("cache.sqlite");
698 let source = onestore::create_section("resolve.one", "abc", "Fixture").unwrap();607 let source = onestore::create_section("resolve.one", "abc", "Fixture").unwrap();
699 let (sid, oid, _) = text(&source);608 let (sid, oid, _) = text(&source);
700 let cache = Replica::create(dir.path().join("cache.sqlite"), &source).unwrap();609 let cache = Replica::create(&path, &source).unwrap();
701 let id = save(&cache, oid, 1..2, "L").unwrap();610 save(&cache, oid, 1..2, "L").unwrap();
702 let refused = |cache: &Replica, id| matches!(cache.resolve(id, Resolution::Mine), Err(Error::Io(error)) if error.kind() == io::ErrorKind::InvalidInput);611 let remote = typed(&source, sid, oid, 0..3, "aRcZ");
703 assert!(refused(&cache, id));
704 let remote = onestore::replace_text(&source, sid, oid, 0..3, "🦀Rc").unwrap();
705 let mut server = Server::new(&remote);612 let mut server = Server::new(&remote);
706 assert!(matches!(613 assert!(matches!(
707 cache.sync_once(&mut server).unwrap().edit,614 cache.sync_once(&mut server).unwrap().edit,
708 Some((_, EditStatus::Conflict(_)))615 Some((_, EditStatus::Published { .. }))
709 ));616 ));
617 assert_eq!(content(&snapshot(&cache), oid), "aRcZ");
618 let mut expected = "aRcZ".to_owned();
619 for n in 0..12 {
620 save(&cache, oid, 0..0, &format!("[{n}] ")).unwrap();
621 expected.insert_str(0, &format!("[{n}] "));
622 }
710 let pending = cache.pending().unwrap();623 let pending = cache.pending().unwrap();
711 assert!(refused(&cache, id + 1));624 drop(cache);
625 let cache = Replica::open(&path).unwrap();
712 assert_eq!(cache.pending().unwrap(), pending);626 assert_eq!(cache.pending().unwrap(), pending);
713 cache.resolve(id, Resolution::Mine).unwrap();
714 server.fault = Fault::UnknownBefore;
715 assert!(
716 matches!(cache.sync_once(&mut server), Err(Error::Remote(error)) if error.state == CommitState::Unknown)
717 );
718 let attempted = cache.status(id).unwrap();
719 assert!(matches!(627 assert!(matches!(
720 attempted,628 cache.sync_once(&mut server).unwrap().edit,
721 Some(EditStatus::AwaitingConfirmation { .. })629 Some((id, EditStatus::Published { .. })) if id == pending.last().unwrap().id
722 ));630 ));
723 assert!(refused(&cache, id));631 assert_eq!(server.publications, 2);
724 assert_eq!(cache.status(id).unwrap(), attempted);632 assert_eq!(content(&server.durable, oid), expected);
725 assert_eq!(server.publications, 1);633 assert_eq!(
634 conflicts(&server.durable),
635 [(sid, vec![(model_ops::AUTHOR.to_owned(), vec!["aLc".to_owned()])])]
636 );
637 assert!(cache.pending().unwrap().is_empty());
726}638}
727639
640/// A rebase that cannot be stored leaves the queue, and every page, as it was.
728#[test]641#[test]
729fn a_failed_resolution_leaves_the_conflict_as_it_was() {642fn a_failed_rebase_leaves_the_queue_as_it_was() {
730 let dir = tempfile::tempdir().unwrap();643 let dir = tempfile::tempdir().unwrap();
731 let path = dir.path().join("cache.sqlite");644 let path = dir.path().join("cache.sqlite");
732 let source = onestore::create_section("resolve.one", "abc", "Fixture").unwrap();645 let source = onestore::create_section("resolve.one", "abc", "Fixture").unwrap();
733 let (sid, oid, _) = text(&source);646 let (sid, oid, _) = text(&source);
734 let cache = Replica::create(&path, &source).unwrap();647 let cache = Replica::create(&path, &source).unwrap();
735 let id = save(&cache, oid, 1..2, "L").unwrap();648 let id = save(&cache, oid, 1..2, "L").unwrap();
736 let remote = onestore::replace_text(&source, sid, oid, 0..3, "XaRc").unwrap();649 let remote = typed(&source, sid, oid, 0..3, "XaRc");
737 let mut server = Server::new(&remote);650 let mut server = Server::new(&remote);
738 cache.sync_once(&mut server).unwrap();651 let local = pages(&snapshot(&cache));
739 let local = pages(&cache.snapshot().unwrap());
740 let pending = cache.pending().unwrap();652 let pending = cache.pending().unwrap();
741 let status = cache.status(id).unwrap();
742 drop(cache);653 drop(cache);
743 let connection = rusqlite::Connection::open(&path).unwrap();654 let connection = rusqlite::Connection::open(&path).unwrap();
744 connection.execute_batch("CREATE TRIGGER fail_clear BEFORE DELETE ON batches BEGIN SELECT RAISE(ABORT, 'test resolution failure'); END;").unwrap();655 connection.execute_batch("CREATE TRIGGER fail_clear BEFORE DELETE ON batches BEGIN SELECT RAISE(ABORT, 'test rebase failure'); END;").unwrap();
745 drop(connection);656 drop(connection);
746 let cache = Replica::open(&path).unwrap();657 let cache = Replica::open(&path).unwrap();
747 assert!(matches!(658 assert!(matches!(
748 cache.resolve(id, Resolution::Mine),659 cache.sync_once(&mut server),
749 Err(Error::Database(_))660 Err(Error::Database(_))
750 ));661 ));
751 drop(cache);662 drop(cache);
752 let cache = Replica::open(&path).unwrap();663 let cache = Replica::open(&path).unwrap();
753 assert_eq!(cache.pending().unwrap(), pending);664 assert_eq!(cache.pending().unwrap(), pending);
754 assert_eq!(cache.status(id).unwrap(), status);665 assert_eq!(cache.status(id).unwrap(), Some(EditStatus::Pending));
755 assert_eq!(pages(&cache.snapshot().unwrap()), local);666 assert_eq!(pages(&snapshot(&cache)), local);
756 assert_eq!(cache.remote_snapshot().unwrap(), remote);
757 assert_eq!(server.publications, 0);667 assert_eq!(server.publications, 0);
758}668}
759669
760#[test]670#[test]
761fn seeded_conflicts_resolve_to_the_remote_text_and_later_edits_publish_once() {671fn seeded_conflicts_keep_the_remote_text_and_later_edits_publish_once() {
762 let dir = tempfile::tempdir().unwrap();672 let dir = tempfile::tempdir().unwrap();
763 let original: Vec<_> = "abcdefghij🦀klmnop".chars().collect();673 let original: Vec<_> = "abcdefghij🦀klmnop".chars().collect();
764 let mut seed = 911_u64;674 let mut seed = 911_u64;
...@@ -779,70 +689,71 @@ fn seeded_conflicts_resolve_to_the_remote_text_and_later_edits_publish_once() {...@@ -779,70 +689,71 @@ fn seeded_conflicts_resolve_to_the_remote_text_and_later_edits_publish_once() {
779 .unwrap();689 .unwrap();
780 let (sid, oid, _) = text(&source);690 let (sid, oid, _) = text(&source);
781 let cache = Replica::create(dir.path().join(format!("{case}.sqlite")), &source).unwrap();691 let cache = Replica::create(dir.path().join(format!("{case}.sqlite")), &source).unwrap();
782 let id = save(&cache, oid, start..end, "λ🦊μ").unwrap();692 save(&cache, oid, start..end, "λ🦊μ").unwrap();
783 save(&cache, oid, start + 1..start + 3, "🐕").unwrap();693 save(&cache, oid, start + 1..start + 3, "🐕").unwrap();
694 let local = content(&snapshot(&cache), oid);
784 let mut remote_text = original.clone();695 let mut remote_text = original.clone();
785 remote_text.splice(at..at + 1, "Ω🐈π".chars());696 remote_text.splice(at..at + 1, "Ω🐈π".chars());
786 let remote_text = prefix.clone() + &remote_text.iter().collect::<String>() + &suffix;697 let remote_text = prefix.clone() + &remote_text.iter().collect::<String>() + &suffix;
787 let remote = onestore::replace_text(698 let remote = typed(
788 &source,699 &source,
789 sid,700 sid,
790 oid,701 oid,
791 0..original.iter().map(|ch| ch.len_utf16() as u32).sum(),702 0..original.iter().map(|ch| ch.len_utf16() as u32).sum(),
792 &remote_text,703 &remote_text,
793 )704 );
794 .unwrap();
795 let mut server = Server::new(&remote);705 let mut server = Server::new(&remote);
796 assert!(706 assert!(
797 matches!(707 matches!(
798 cache.sync_once(&mut server).unwrap().edit,708 cache.sync_once(&mut server).unwrap().edit,
799 Some((_, EditStatus::Conflict(ConflictKind::ContentChanged)))709 Some((_, EditStatus::Published { .. }))
800 ),710 ),
801 "case {case}, seed {seed}"711 "case {case}, seed {seed}"
802 );712 );
713 assert_eq!(content(&server.durable, oid), remote_text, "case {case}, seed {seed}");
714 assert_eq!(
715 conflicts(&server.durable),
716 [(sid, vec![(model_ops::AUTHOR.to_owned(), vec![local])])],
717 "case {case}, seed {seed}"
718 );
803 let at_remote = start + prefix.len() as u32;719 let at_remote = start + prefix.len() as u32;
804 let reviewed = model_ops::review(&cache, id, oid, |page| {720 let later = save(&cache, oid, at_remote..at_remote + 4, "λ🐕μ").unwrap();
805 model_ops::replace_text(page, oid, at_remote..at_remote + 4, "λ🐕μ")
806 })
807 .unwrap()
808 .unwrap();
809 assert!(721 assert!(
810 matches!(cache.sync_once(&mut server).unwrap().edit, Some((actual, EditStatus::Published { .. })) if actual == reviewed),722 matches!(cache.sync_once(&mut server).unwrap().edit, Some((actual, EditStatus::Published { .. })) if actual == later),
811 "case {case}, seed {seed}"723 "case {case}, seed {seed}"
812 );724 );
813 let mut expected = original.clone();725 let mut expected = original.clone();
814 expected.splice(at..at + 1, "λ🐕μ".chars());726 expected.splice(at..at + 1, "λ🐕μ".chars());
815 let expected = prefix + &expected.iter().collect::<String>() + &suffix;727 let expected = prefix + &expected.iter().collect::<String>() + &suffix;
816 assert_eq!(728 assert_eq!(content(&server.durable, oid), expected, "case {case}, seed {seed}");
817 text(&server.durable).2,729 assert_eq!(server.publications, 2);
818 expected,
819 "case {case}, seed {seed}"
820 );
821 assert_eq!(server.publications, 1);
822 assert!(cache.pending().unwrap().is_empty());730 assert!(cache.pending().unwrap().is_empty());
823 }731 }
824}732}
825733
826#[test]734#[test]
827fn remote_changes_after_resolution_are_never_overwritten() {735fn remote_changes_after_a_conflict_are_never_overwritten() {
828 struct ChangedAfterRead {736 struct ChangedAfterStamp {
829 server: Server,737 server: Server,
830 change: Option<Vec<u8>>,738 change: Option<Vec<u8>>,
831 }739 }
832 impl Remote for ChangedAfterRead {740 impl Remote for ChangedAfterStamp {
833 fn read(&mut self) -> io::Result<Vec<u8>> {741 fn read(&mut self) -> io::Result<Vec<u8>> {
834 let snapshot = self.server.read()?;742 self.server.read()
743 }
744 fn stamp(&mut self) -> io::Result<onestore::Stamp> {
745 let stamp = self.server.stamp()?;
835 if let Some(changed) = self.change.take() {746 if let Some(changed) = self.change.take() {
836 self.server.visible = changed.clone();747 self.server.visible = changed.clone();
837 self.server.durable = changed;748 self.server.durable = changed;
838 }749 }
839 Ok(snapshot)750 Ok(stamp)
840 }751 }
841 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {752 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
842 self.server.publish(transaction)753 self.server.publish(transaction)
843 }754 }
844 fn confirm(&mut self, snapshot: &[u8]) -> Result<(), CommitError> {755 fn confirm(&mut self, base: &onestore::Stamp) -> Result<(), CommitError> {
845 self.server.confirm(snapshot)756 self.server.confirm(base)
846 }757 }
847 }758 }
848 for after_read in [false, true] {759 for after_read in [false, true] {
...@@ -850,50 +761,41 @@ fn remote_changes_after_resolution_are_never_overwritten() {...@@ -850,50 +761,41 @@ fn remote_changes_after_resolution_are_never_overwritten() {
850 let source = onestore::create_section("resolve.one", "abc", "Fixture").unwrap();761 let source = onestore::create_section("resolve.one", "abc", "Fixture").unwrap();
851 let (sid, oid, _) = text(&source);762 let (sid, oid, _) = text(&source);
852 let cache = Replica::create(dir.path().join("cache.sqlite"), &source).unwrap();763 let cache = Replica::create(dir.path().join("cache.sqlite"), &source).unwrap();
853 let id = save(&cache, oid, 1..2, "L").unwrap();764 save(&cache, oid, 1..2, "L").unwrap();
854 let remote = onestore::replace_text(&source, sid, oid, 1..2, "R").unwrap();765 let remote = typed(&source, sid, oid, 1..2, "R");
855 let mut remote = ChangedAfterRead {766 let mut remote = ChangedAfterStamp {
856 server: Server::new(&remote),767 server: Server::new(&remote),
857 change: None,768 change: None,
858 };769 };
859 assert!(matches!(770 assert!(matches!(
860 cache.sync_once(&mut remote).unwrap().edit,771 cache.sync_once(&mut remote).unwrap().edit,
861 Some((_, EditStatus::Conflict(_)))772 Some((_, EditStatus::Published { .. }))
862 ));773 ));
863 let id = model_ops::review(&cache, id, oid, |page| {774 // The local version again, typed over the remote's.
864 model_ops::replace_text(page, oid, 1..2, "L")775 let id = save(&cache, oid, 1..2, "L").unwrap();
865 })776 assert_eq!(content(&snapshot(&cache), oid), "aLc");
866 .unwrap()777 let changed = typed(&remote.server.visible, sid, oid, 1..2, "Q");
867 .unwrap();
868 let local = content(&cache.snapshot().unwrap(), oid);
869 assert_eq!(local, "aLc");
870 let changed = onestore::replace_text(&remote.server.visible, sid, oid, 1..2, "Q").unwrap();
871 if after_read {778 if after_read {
872 remote.change = Some(changed.clone());779 remote.change = Some(changed.clone());
873 } else {
874 remote.server.visible = changed.clone();
875 remote.server.durable = changed.clone();
876 }
877 if after_read {
878 assert!(780 assert!(
879 matches!(cache.sync_once(&mut remote), Err(Error::Remote(error)) if error.state == CommitState::NotCommitted)781 matches!(cache.sync_once(&mut remote), Err(Error::Remote(error)) if error.state == CommitState::NotCommitted)
880 );782 );
881 assert_eq!(cache.status(id).unwrap(), Some(EditStatus::Pending));783 assert_eq!(cache.status(id).unwrap(), Some(EditStatus::Pending));
784 } else {
785 remote.server.visible = changed.clone();
786 remote.server.durable = changed.clone();
882 }787 }
883 assert_eq!(
884 cache.sync_once(&mut remote).unwrap().edit,
885 Some((id, EditStatus::Conflict(ConflictKind::ContentChanged)))
886 );
887 assert_eq!(remote.server.durable, changed);
888 assert_eq!(remote.server.visible, changed);
889 assert_eq!(remote.server.publications, usize::from(after_read));
890 assert_eq!(content(&cache.snapshot().unwrap(), oid), local);
891 assert!(cache.pending().unwrap()[0].edit.ops.is_empty());
892 assert!(matches!(788 assert!(matches!(
893 &cache.pending().unwrap()[1].edit.ops[..],789 cache.sync_once(&mut remote).unwrap().edit,
894 [onestore::op::Op::Page { op: onestore::op::PageOp::Text { range, with, .. }, .. }]790 Some((_, EditStatus::Published { .. }))
895 if *range == (1..2) && with == "L"
896 ));791 ));
792 assert_eq!(content(&remote.server.durable, oid), "aQc");
793 let versions = [(model_ops::AUTHOR.to_owned(), vec!["aLc".to_owned()])];
794 assert_eq!(
795 conflicts(&remote.server.durable),
796 [(sid, [versions.clone(), versions].concat())]
797 );
798 assert!(cache.pending().unwrap().is_empty());
897 }799 }
898}800}
899801
...@@ -908,7 +810,7 @@ fn remote_restore_retains_historical_receipts_without_replaying_them() {...@@ -908,7 +810,7 @@ fn remote_restore_retains_historical_receipts_without_replaying_them() {
908 let published = save(&cache, object, 0..0, "Published ").unwrap();810 let published = save(&cache, object, 0..0, "Published ").unwrap();
909 let (_, receipt) = cache.sync_once(&mut server).unwrap().edit.unwrap();811 let (_, receipt) = cache.sync_once(&mut server).unwrap().edit.unwrap();
910 assert!(matches!(receipt, EditStatus::Published { .. }));812 assert!(matches!(receipt, EditStatus::Published { .. }));
911 let published_image = cache.snapshot().unwrap();813 let published_image = snapshot(&cache);
912 cache814 cache
913 .export_recovery(directory.path().join("published.sqlite"))815 .export_recovery(directory.path().join("published.sqlite"))
914 .unwrap();816 .unwrap();
...@@ -918,8 +820,8 @@ fn remote_restore_retains_historical_receipts_without_replaying_them() {...@@ -918,8 +820,8 @@ fn remote_restore_retains_historical_receipts_without_replaying_them() {
918 server.durable.clone_from(&source);820 server.durable.clone_from(&source);
919 let cache = Replica::open(&path).unwrap();821 let cache = Replica::open(&path).unwrap();
920 assert_eq!(cache.sync_once(&mut server).unwrap().edit, None);822 assert_eq!(cache.sync_once(&mut server).unwrap().edit, None);
921 assert_eq!(cache.snapshot().unwrap(), source);823 assert_eq!(snapshot(&cache), source);
922 assert_eq!(cache.remote_snapshot().unwrap(), source);824 assert_eq!(remote_snapshot(&cache), source);
923 assert_eq!(cache.status(published).unwrap(), Some(receipt.clone()));825 assert_eq!(cache.status(published).unwrap(), Some(receipt.clone()));
924 assert_eq!(server.publications, 1);826 assert_eq!(server.publications, 1);
925 assert!(cache.pending().unwrap().is_empty());827 assert!(cache.pending().unwrap().is_empty());
...@@ -929,7 +831,7 @@ fn remote_restore_retains_historical_receipts_without_replaying_them() {...@@ -929,7 +831,7 @@ fn remote_restore_retains_historical_receipts_without_replaying_them() {
929 assert_eq!(text(&archive.snapshot().unwrap()).2, "Published Original");831 assert_eq!(text(&archive.snapshot().unwrap()).2, "Published Original");
930 drop(cache);832 drop(cache);
931 let cache = Replica::open(path).unwrap();833 let cache = Replica::open(path).unwrap();
932 assert_eq!(text(&cache.snapshot().unwrap()).2, "Original");834 assert_eq!(text(&snapshot(&cache)).2, "Original");
933 assert_eq!(cache.status(published).unwrap(), Some(receipt.clone()));835 assert_eq!(cache.status(published).unwrap(), Some(receipt.clone()));
934}836}
935837
...@@ -945,14 +847,14 @@ fn remote_restore_rebases_unsent_work_but_never_replays_an_uncertain_attempt() {...@@ -945,14 +847,14 @@ fn remote_restore_rebases_unsent_work_but_never_replays_an_uncertain_attempt() {
945 let mut server = Server::new(&source);847 let mut server = Server::new(&source);
946 let published = save(&cache, object, 0..0, "Published ").unwrap();848 let published = save(&cache, object, 0..0, "Published ").unwrap();
947 let (_, receipt) = cache.sync_once(&mut server).unwrap().edit.unwrap();849 let (_, receipt) = cache.sync_once(&mut server).unwrap().edit.unwrap();
948 let end = u32::try_from(text(&cache.snapshot().unwrap()).2.encode_utf16().count()).unwrap();850 let end = u32::try_from(text(&snapshot(&cache)).2.encode_utf16().count()).unwrap();
949 let queued = save(&cache, object, end..end, " Local").unwrap();851 let queued = save(&cache, object, end..end, " Local").unwrap();
950 if uncertain {852 if uncertain {
951 server.fault = fault;853 server.fault = fault;
952 assert!(cache.sync_once(&mut server).is_err());854 assert!(cache.sync_once(&mut server).is_err());
953 }855 }
954 let prior_status = cache.status(queued).unwrap();856 let prior_status = cache.status(queued).unwrap();
955 let local = cache.snapshot().unwrap();857 let local = snapshot(&cache);
956 let pending = cache.pending().unwrap();858 let pending = cache.pending().unwrap();
957 cache859 cache
958 .export_recovery(directory.path().join("before-restore.sqlite"))860 .export_recovery(directory.path().join("before-restore.sqlite"))
...@@ -969,7 +871,7 @@ fn remote_restore_rebases_unsent_work_but_never_replays_an_uncertain_attempt() {...@@ -969,7 +871,7 @@ fn remote_restore_rebases_unsent_work_but_never_replays_an_uncertain_attempt() {
969 assert_eq!(Some(status.clone()), prior_status);871 assert_eq!(Some(status.clone()), prior_status);
970 assert_eq!(server.publications, attempts);872 assert_eq!(server.publications, attempts);
971 assert_eq!(server.visible, source);873 assert_eq!(server.visible, source);
972 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));874 assert_eq!(pages(&snapshot(&cache)), pages(&local));
973 assert_eq!(cache.pending().unwrap(), pending);875 assert_eq!(cache.pending().unwrap(), pending);
974 } else {876 } else {
975 assert!(matches!(status, EditStatus::Published { .. }));877 assert!(matches!(status, EditStatus::Published { .. }));
...@@ -989,8 +891,10 @@ fn remote_restore_rebases_unsent_work_but_never_replays_an_uncertain_attempt() {...@@ -989,8 +891,10 @@ fn remote_restore_rebases_unsent_work_but_never_replays_an_uncertain_attempt() {
989 }891 }
990}892}
991893
894/// A remote restored to an older image conflicts with local work on what it undid: the
895/// restored page stays and the local version becomes its conflict page.
992#[test]896#[test]
993fn restore_conflicts_keep_dependent_work_until_mine_publishes_it() {897fn a_remote_restore_keeps_local_work_on_a_conflict_page() {
994 let directory = tempfile::tempdir().unwrap();898 let directory = tempfile::tempdir().unwrap();
995 let path = directory.path().join("cache.sqlite");899 let path = directory.path().join("cache.sqlite");
996 let source = onestore::create_section("restore.one", "Original", "Fixture").unwrap();900 let source = onestore::create_section("restore.one", "Original", "Fixture").unwrap();
...@@ -1002,52 +906,26 @@ fn restore_conflicts_keep_dependent_work_until_mine_publishes_it() {...@@ -1002,52 +906,26 @@ fn restore_conflicts_keep_dependent_work_until_mine_publishes_it() {
1002 let head = save(&cache, object, 0..9, "Revised").unwrap();906 let head = save(&cache, object, 0..9, "Revised").unwrap();
1003 server.visible.clone_from(&source);907 server.visible.clone_from(&source);
1004 server.durable.clone_from(&source);908 server.durable.clone_from(&source);
1005 let conflict = EditStatus::Conflict(ConflictKind::ContentChanged);909 assert!(matches!(
1006 assert_eq!(
1007 cache.sync_once(&mut server).unwrap().edit,910 cache.sync_once(&mut server).unwrap().edit,
1008 Some((head, conflict.clone()))911 Some((_, EditStatus::Published { .. }))
1009 );912 ));
1010 let end = u32::try_from(text(&cache.snapshot().unwrap()).2.encode_utf16().count()).unwrap();913 assert_eq!(content(&server.durable, object), "Original");
1011 let dependent = save(&cache, object, end..end, " Later").unwrap();
1012 assert_ne!(head, dependent);
1013 let local = text(&cache.snapshot().unwrap()).2;
1014 assert_eq!(local, "Revised Original Later");
1015 let pending = cache.pending().unwrap();
1016 assert_eq!(pending.len(), 2);
1017 cache
1018 .export_recovery(directory.path().join("conflicted.sqlite"))
1019 .unwrap();
1020 drop(cache);
1021 let cache = Replica::open(&path).unwrap();
1022 assert_eq!(cache.status(head).unwrap(), Some(conflict.clone()));
1023 assert_eq!(cache.status(dependent).unwrap(), Some(conflict.clone()));
1024 assert_eq!(cache.pending().unwrap(), pending);
1025 let changed = onestore::replace_text(&source, space, object, 8..8, " Remote").unwrap();
1026 server.visible.clone_from(&changed);
1027 server.durable.clone_from(&changed);
1028 assert_eq!(914 assert_eq!(
1029 cache.sync_once(&mut server).unwrap().edit,915 conflicts(&server.durable),
1030 Some((dependent, conflict.clone()))916 [(
917 space,
918 vec![(model_ops::AUTHOR.to_owned(), vec!["Revised Original".to_owned()])]
919 )]
1031 );920 );
1032 assert_eq!(cache.remote_snapshot().unwrap(), changed);
1033 assert_eq!(server.publications, 1);
1034 cache.resolve(head, Resolution::Mine).unwrap();
1035 drop(cache);921 drop(cache);
1036 let cache = Replica::open(&path).unwrap();922 let cache = Replica::open(&path).unwrap();
1037 assert!(matches!(
1038 cache.sync_once(&mut server).unwrap().edit,
1039 Some((_, EditStatus::Published { .. }))
1040 ));
1041 assert_eq!(text(&server.durable).2, local);
1042 assert!(matches!(923 assert!(matches!(
1043 cache.status(head).unwrap(),924 cache.status(head).unwrap(),
1044 Some(EditStatus::Published { .. })925 Some(EditStatus::Published { .. })
1045 ));926 ));
1046 assert_eq!(cache.status(published).unwrap(), Some(receipt.clone()));927 assert_eq!(cache.status(published).unwrap(), Some(receipt));
1047 let archive = notebook::Recovery::open(directory.path().join("conflicted.sqlite")).unwrap();928 assert!(cache.pending().unwrap().is_empty());
1048 assert_eq!(text(&archive.snapshot().unwrap()).2, local);
1049 assert_eq!(archive.pending().unwrap(), pending);
1050 assert_eq!(archive.status(head).unwrap(), Some(conflict.clone()));
1051}929}
1052930
1053#[test]931#[test]
crates/notebook/tests/sync/tree.rs+41-49
...@@ -34,9 +34,9 @@ fn fixture() -> (Vec<u8>, ExGuid, ExGuid, [ExGuid; 4], [ExGuid; 4]) {...@@ -34,9 +34,9 @@ fn fixture() -> (Vec<u8>, ExGuid, ExGuid, [ExGuid; 4], [ExGuid; 4]) {
34 for at in 1..4 {34 for at in 1..4 {
35 anchor = model_ops::insert_after(&mut page, anchor, &format!("Sibling {at}")).1;35 anchor = model_ops::insert_after(&mut page, anchor, &format!("Sibling {at}")).1;
36 }36 }
37 let source = PreparedEdit::page(&source, space, &page, "Author")37 let source = ops::saved(&source, space, &page)
38 .unwrap()38 .unwrap()
39 .as_bytes()39 .as_slice()
40 .to_vec();40 .to_vec();
41 let stored = page_of(&source, space);41 let stored = page_of(&source, space);
42 let siblings = &outline_of(&stored, outline).paragraphs;42 let siblings = &outline_of(&stored, outline).paragraphs;
...@@ -87,12 +87,12 @@ fn move_preserves_remote_content_and_dependent_edits_through_reopen() {...@@ -87,12 +87,12 @@ fn move_preserves_remote_content_and_dependent_edits_through_reopen() {
87 })87 })
88 .unwrap()88 .unwrap()
89 .unwrap();89 .unwrap();
90 let local = cache.snapshot().unwrap();90 let local = snapshot(&cache);
91 let queue = cache.pending().unwrap();91 let queue = cache.pending().unwrap();
92 drop(cache);92 drop(cache);
93 let cache = Replica::open(&path).unwrap();93 let cache = Replica::open(&path).unwrap();
94 assert_eq!(cache.pending().unwrap(), queue);94 assert_eq!(cache.pending().unwrap(), queue);
95 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));95 assert_eq!(pages(&snapshot(&cache)), pages(&local));
96 let mut server = Server::new(&remote_with(&source, space, |page| {96 let mut server = Server::new(&remote_with(&source, space, |page| {
97 insert_child(page, paragraphs[0], "New remote child");97 insert_child(page, paragraphs[0], "New remote child");
98 restyle(page, texts[1], 0..7, |format| format.bold = Some(true));98 restyle(page, texts[1], 0..7, |format| format.bold = Some(true));
...@@ -185,14 +185,16 @@ fn native_empty_child_list_normalization_merges_with_a_local_deletion() {...@@ -185,14 +185,16 @@ fn native_empty_child_list_normalization_merges_with_a_local_deletion() {
185 }185 }
186}186}
187187
188/// Deleting what the remote changed keeps the remote's content, and the local version on a
189/// conflict page; the edits that do not depend on the deletion merge.
188#[test]190#[test]
189fn deletion_requires_review_of_remote_content_and_preserves_later_work() {191fn deletion_of_remote_content_keeps_a_conflict_page_and_merges_later_work() {
190 let (source, space, outline, paragraphs, texts) = fixture();192 let (source, space, outline, paragraphs, texts) = fixture();
191 let mut page = page_of(&source, space);193 let mut page = page_of(&source, space);
192 insert_child(&mut page, paragraphs[0], "Descendant");194 insert_child(&mut page, paragraphs[0], "Descendant");
193 let source = PreparedEdit::page(&source, space, &page, "Author")195 let source = ops::saved(&source, space, &page)
194 .unwrap()196 .unwrap()
195 .as_bytes()197 .as_slice()
196 .to_vec();198 .to_vec();
197 let child = outline_of(&page_of(&source, space), outline).paragraphs[1]199 let child = outline_of(&page_of(&source, space), outline).paragraphs[1]
198 .text()200 .text()
...@@ -215,24 +217,20 @@ fn deletion_requires_review_of_remote_content_and_preserves_later_work() {...@@ -215,24 +217,20 @@ fn deletion_requires_review_of_remote_content_and_preserves_later_work() {
215 replace_text(page, texts[1], 0..0, "Local ");217 replace_text(page, texts[1], 0..0, "Local ");
216 };218 };
217 let id = save(&cache, texts[1], change).unwrap().unwrap();219 let id = save(&cache, texts[1], change).unwrap().unwrap();
218 let queue = cache.pending().unwrap();
219 let local = cache.snapshot().unwrap();
220 let mut server = Server::new(&remote_with(&source, space, remote));220 let mut server = Server::new(&remote_with(&source, space, remote));
221 assert!(conflicted(&cache, &mut server, space));
221 assert_eq!(222 assert_eq!(
222 cache.sync_once(&mut server).unwrap().edit,223 text_of(&page_of(&server.durable, space), texts[1]),
223 Some((id, EditStatus::Conflict(ConflictKind::ContentChanged)))224 "Local Sibling 1"
224 );225 );
225 assert_eq!(server.publications, 0);
226 assert_eq!(cache.pending().unwrap(), queue);
227 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));
228 drop(cache);226 drop(cache);
229 let cache = Replica::open(&path).unwrap();227 let cache = Replica::open(&path).unwrap();
230 assert_eq!(
231 cache.status(id).unwrap(),
232 Some(EditStatus::Conflict(ConflictKind::ContentChanged))
233 );
234 review(&cache, id, texts[1], change).unwrap();
235 published(&cache, &mut server, id);228 published(&cache, &mut server, id);
229 // The writer deletes the paragraph again, remote changes and all.
230 let again = save(&cache, texts[1], |page| delete_paragraph(page, texts[0]))
231 .unwrap()
232 .unwrap();
233 published(&cache, &mut server, again);
236 assert_eq!(234 assert_eq!(
237 order(&server.durable, space, outline),235 order(&server.durable, space, outline),
238 paragraphs[1..].to_vec()236 paragraphs[1..].to_vec()
...@@ -247,7 +245,7 @@ fn deletion_requires_review_of_remote_content_and_preserves_later_work() {...@@ -247,7 +245,7 @@ fn deletion_requires_review_of_remote_content_and_preserves_later_work() {
247#[test]245#[test]
248fn sibling_and_ancestry_changes_have_explicit_merge_or_conflict() {246fn sibling_and_ancestry_changes_have_explicit_merge_or_conflict() {
249 let (source, space, outline, paragraphs, texts) = fixture();247 let (source, space, outline, paragraphs, texts) = fixture();
250 // The published order of the fixture's paragraphs, or a conflict.248 // The published order of the fixture's paragraphs, or a conflict page.
251 let cases: [(Change<Page>, Option<&[usize]>); 5] = [249 let cases: [(Change<Page>, Option<&[usize]>); 5] = [
252 (250 (
253 Box::new(move |page| {251 Box::new(move |page| {
...@@ -279,14 +277,16 @@ fn sibling_and_ancestry_changes_have_explicit_merge_or_conflict() {...@@ -279,14 +277,16 @@ fn sibling_and_ancestry_changes_have_explicit_merge_or_conflict() {
279 .unwrap()277 .unwrap()
280 .unwrap();278 .unwrap();
281 let mut server = Server::new(&remote_with(&source, space, remote));279 let mut server = Server::new(&remote_with(&source, space, remote));
282 let result = cache.sync_once(&mut server).unwrap().edit.unwrap();280 let conflict = conflicted(&cache, &mut server, space);
283 assert_eq!(result.0, id);281 assert!(matches!(
282 cache.status(id).unwrap(),
283 Some(EditStatus::Published { .. })
284 ));
284 let Some(expected) = expected else {285 let Some(expected) = expected else {
285 assert_eq!(result.1, EditStatus::Conflict(ConflictKind::ContentChanged));286 assert!(conflict);
286 assert_eq!(server.publications, 0);
287 continue;287 continue;
288 };288 };
289 assert!(matches!(result.1, EditStatus::Published { .. }));289 assert!(!conflict);
290 let published: Vec<ExGuid> = order(&server.durable, space, outline)290 let published: Vec<ExGuid> = order(&server.durable, space, outline)
291 .into_iter()291 .into_iter()
292 .filter(|id| paragraphs.contains(id))292 .filter(|id| paragraphs.contains(id))
...@@ -352,7 +352,7 @@ fn cell_mut(page: &mut Page, id: ExGuid) -> &mut TableCell {...@@ -352,7 +352,7 @@ fn cell_mut(page: &mut Page, id: ExGuid) -> &mut TableCell {
352}352}
353353
354/// Moves a cell's sole paragraph to the end of another cell, or deletes it, leaving the354/// Moves a cell's sole paragraph to the end of another cell, or deletes it, leaving the
355/// cell an empty paragraph as the writers do.355/// cell a new empty paragraph, as an edit that empties a cell must.
356fn empty_cell(page: &mut Page, cell: ExGuid, destination: Option<ExGuid>) {356fn empty_cell(page: &mut Page, cell: ExGuid, destination: Option<ExGuid>) {
357 let emptied = cell_mut(page, cell);357 let emptied = cell_mut(page, cell);
358 let mut replacement = model_ops::fresh_paragraph(&emptied.paragraphs[0], "");358 let mut replacement = model_ops::fresh_paragraph(&emptied.paragraphs[0], "");
...@@ -419,7 +419,7 @@ fn emptied_cell_replacement_is_durable_and_cannot_be_silently_omitted_on_replay(...@@ -419,7 +419,7 @@ fn emptied_cell_replacement_is_durable_and_cannot_be_silently_omitted_on_replay(
419 empty_cell(page, cell, move_out.then_some(neighbour));419 empty_cell(page, cell, move_out.then_some(neighbour));
420 })420 })
421 .unwrap();421 .unwrap();
422 let local = cache.snapshot().unwrap();422 let local = snapshot(&cache);
423 let queue = cache.pending().unwrap();423 let queue = cache.pending().unwrap();
424 {424 {
425 let page = page_of(&local, space);425 let page = page_of(&local, space);
...@@ -431,7 +431,7 @@ fn emptied_cell_replacement_is_durable_and_cannot_be_silently_omitted_on_replay(...@@ -431,7 +431,7 @@ fn emptied_cell_replacement_is_durable_and_cannot_be_silently_omitted_on_replay(
431 drop(cache);431 drop(cache);
432 let cache = Replica::open(&path).unwrap();432 let cache = Replica::open(&path).unwrap();
433 assert_eq!(cache.pending().unwrap(), queue);433 assert_eq!(cache.pending().unwrap(), queue);
434 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));434 assert_eq!(pages(&snapshot(&cache)), pages(&local));
435 let mut server = Server::new(&if competing {435 let mut server = Server::new(&if competing {
436 remote_with(source, space, |page| {436 remote_with(source, space, |page| {
437 let target = cell_mut(page, cell);437 let target = cell_mut(page, cell);
...@@ -574,28 +574,20 @@ fn native_tree_and_layout_changes_reconcile_without_discarding_unreviewed_conten...@@ -574,28 +574,20 @@ fn native_tree_and_layout_changes_reconcile_without_discarding_unreviewed_conten
574 let path = directory.path().join("native.sqlite");574 let path = directory.path().join("native.sqlite");
575 let cache = Replica::create(&path, BEFORE).unwrap();575 let cache = Replica::create(&path, BEFORE).unwrap();
576 let id = save(&cache, dependent, save_both).unwrap().unwrap();576 let id = save(&cache, dependent, save_both).unwrap().unwrap();
577 let local = cache.snapshot().unwrap();577 let merged = !conflicted(&cache, &mut server, space);
578 let queue = cache.pending().unwrap();578 assert!(matches!(
579 let merged = match cache.sync_once(&mut server).unwrap().edit.unwrap() {579 cache.status(id).unwrap(),
580 (actual, EditStatus::Published { .. }) => {580 Some(EditStatus::Published { .. })
581 assert_eq!(actual, id, "{name}");581 ));
582 true582 if !merged {
583 }583 // The dependent text merged; the writer makes the change again.
584 result => {584 drop(cache);
585 assert_eq!(585 let cache = Replica::open(&path).unwrap();
586 result,586 // Unless the remote made the same change.
587 (id, EditStatus::Conflict(ConflictKind::ContentChanged)),587 if let Some(again) = save(&cache, dependent, &change).unwrap() {
588 "{name}"588 published(&cache, &mut server, again);
589 );
590 assert_eq!(cache.pending().unwrap(), queue);
591 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));
592 drop(cache);
593 let cache = Replica::open(&path).unwrap();
594 review(&cache, id, dependent, save_both).unwrap();
595 published(&cache, &mut server, id);
596 false
597 }589 }
598 };590 }
599 let durable = page_of(&server.durable, space);591 let durable = page_of(&server.durable, space);
600 assert!(592 assert!(
601 text_of(&durable, dependent).starts_with("Offline "),593 text_of(&durable, dependent).starts_with("Offline "),
crates/notebook/tests/sync_formatting.rs+42-49
...@@ -1,21 +1,23 @@...@@ -1,21 +1,23 @@
1//! Formatting and insertion as page-model differences: what reconciles with an independent1//! Formatting and insertion as page-model differences: what reconciles with an independent
2//! remote change, what conflicts, and what a reviewed conflict publishes.2//! remote change, what keeps a conflict page, and what the edit made again publishes.
33
4use notebook::{ConflictKind, EditStatus, Replica};4#[path = "../../onestore/tests/support/ops.rs"]
5mod ops;
6
7use notebook::{EditStatus, Replica};
5use onestore::{8use onestore::{
6 ExGuid, PreparedEdit, RevisionIndex, Store,9 ExGuid, RevisionIndex, Store,
7 document::{Document, Format},10 document::{Document, Format},
8 page::{Page, PageObject},11 page::{Page, PageObject},
9};12};
1013
11#[path = "support/server.rs"]14#[path = "support/server.rs"]
12mod server;15mod server;
13use server::{Fault, Server, pages};16use server::{Fault, Server, conflicted, pages, snapshot};
14#[path = "support/model_ops.rs"]17#[path = "support/model_ops.rs"]
15mod model_ops;18mod model_ops;
16use model_ops::{19use model_ops::{
17 AUTHOR, insert_after, insert_outline, page_of, paragraph_with, replace_text, restyle, review,20 AUTHOR, insert_after, insert_outline, page_of, paragraph_with, replace_text, restyle, save,
18 save,
19};21};
2022
21const OUTLINES: &[u8] =23const OUTLINES: &[u8] =
...@@ -80,9 +82,9 @@ fn remote_with(space: ExGuid, change: impl FnOnce(&mut Page)) -> Server {...@@ -80,9 +82,9 @@ fn remote_with(space: ExGuid, change: impl FnOnce(&mut Page)) -> Server {
80 let mut page = page_of(OUTLINES, space);82 let mut page = page_of(OUTLINES, space);
81 change(&mut page);83 change(&mut page);
82 Server::new(84 Server::new(
83 PreparedEdit::page(OUTLINES, space, &page, "Native author")85 ops::saved(OUTLINES, space, &page)
84 .unwrap()86 .unwrap()
85 .as_bytes(),87 .as_slice(),
86 )88 )
87}89}
8890
...@@ -112,7 +114,7 @@ fn offline_formatting_merges_with_a_remote_edit_to_another_paragraph() {...@@ -112,7 +114,7 @@ fn offline_formatting_merges_with_a_remote_edit_to_another_paragraph() {
112 .all(|format| format.bold == Some(true))114 .all(|format| format.bold == Some(true))
113 );115 );
114 assert_eq!(text_of(&published, ids[3]), "Remote Trailing sibling");116 assert_eq!(text_of(&published, ids[3]), "Remote Trailing sibling");
115 assert_eq!(cache.snapshot().unwrap(), server.durable);117 assert_eq!(snapshot(&cache), server.durable);
116}118}
117119
118#[test]120#[test]
...@@ -127,7 +129,7 @@ fn offline_formatting_conflicts_only_with_remote_changes_it_overlaps() {...@@ -127,7 +129,7 @@ fn offline_formatting_conflicts_only_with_remote_changes_it_overlaps() {
127 })129 })
128 .unwrap()130 .unwrap()
129 .unwrap();131 .unwrap();
130 let local = cache.snapshot().unwrap();132 let local = snapshot(&cache);
131 let mut server = remote_with(space, |page| {133 let mut server = remote_with(space, |page| {
132 if typed {134 if typed {
133 replace_text(page, ids[0], 0..0, "Remote ");135 replace_text(page, ids[0], 0..0, "Remote ");
...@@ -150,18 +152,18 @@ fn offline_formatting_conflicts_only_with_remote_changes_it_overlaps() {...@@ -150,18 +152,18 @@ fn offline_formatting_conflicts_only_with_remote_changes_it_overlaps() {
150 );152 );
151 }153 }
152 } else {154 } else {
153 assert_eq!(155 assert!(matches!(outcome, Some((_, EditStatus::Published { .. }))));
154 outcome,156 assert!(conflicted(&server.durable, space));
155 Some((id, EditStatus::Conflict(ConflictKind::ContentChanged)))157 let remote = page_of(&server.durable, space);
156 );158 assert!(formats(&remote, ids[0]).iter().all(|format| format.bold != Some(true)));
157 assert_eq!(server.publications, 0);159 assert_eq!(pages(&snapshot(&cache)), pages(&server.durable));
158 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));160 assert_ne!(pages(&local), pages(&server.durable));
159 }161 }
160 }162 }
161}163}
162164
163#[test]165#[test]
164fn competing_formatting_of_one_paragraph_conflicts_until_reviewed() {166fn competing_formatting_of_one_paragraph_keeps_a_conflict_page_until_made_again() {
165 let (space, page) = page_titled(OUTLINES, PAGE);167 let (space, page) = page_titled(OUTLINES, PAGE);
166 let ids = plain(&page);168 let ids = plain(&page);
167 let directory = tempfile::tempdir().unwrap();169 let directory = tempfile::tempdir().unwrap();
...@@ -172,24 +174,22 @@ fn competing_formatting_of_one_paragraph_conflicts_until_reviewed() {...@@ -172,24 +174,22 @@ fn competing_formatting_of_one_paragraph_conflicts_until_reviewed() {
172 })174 })
173 .unwrap()175 .unwrap()
174 .unwrap();176 .unwrap();
175 let pending = cache.pending().unwrap();
176 let mut server = remote_with(space, |page| {177 let mut server = remote_with(space, |page| {
177 restyle(page, ids[0], 0..3, |format| format.italic = Some(true));178 restyle(page, ids[0], 0..3, |format| format.italic = Some(true));
178 });179 });
179 for _ in 0..2 {180 assert!(matches!(
180 assert_eq!(181 cache.sync_once(&mut server).unwrap().edit,
181 cache.sync_once(&mut server).unwrap().edit,182 Some((_, EditStatus::Published { .. }))
182 Some((id, EditStatus::Conflict(ConflictKind::ContentChanged)))183 ));
183 );184 assert!(conflicted(&server.durable, space));
184 }185 assert_eq!(cache.sync_once(&mut server).unwrap().edit, None);
185 assert_eq!(cache.pending().unwrap(), pending);
186 drop(cache);186 drop(cache);
187 let cache = Replica::open(&path).unwrap();187 let cache = Replica::open(&path).unwrap();
188 assert_eq!(188 assert!(matches!(
189 cache.status(id).unwrap(),189 cache.status(id).unwrap(),
190 Some(EditStatus::Conflict(ConflictKind::ContentChanged))190 Some(EditStatus::Published { .. })
191 );191 ));
192 review(&cache, id, ids[0], |page| {192 save(&cache, ids[0], |page| {
193 restyle(page, ids[0], 0..6, |format| format.font_size = Some(14.0));193 restyle(page, ids[0], 0..6, |format| format.font_size = Some(14.0));
194 })194 })
195 .unwrap();195 .unwrap();
...@@ -287,7 +287,7 @@ fn seeded_formatting_reconciles_exactly_when_the_remote_left_the_paragraph_alone...@@ -287,7 +287,7 @@ fn seeded_formatting_reconciles_exactly_when_the_remote_left_the_paragraph_alone
287 })287 })
288 .unwrap()288 .unwrap()
289 .unwrap();289 .unwrap();
290 let local = cache.snapshot().unwrap();290 let local = snapshot(&cache);
291 let mut server = remote_with(space, |page| {291 let mut server = remote_with(space, |page| {
292 for (step, &at) in remote.iter().enumerate() {292 for (step, &at) in remote.iter().enumerate() {
293 if step % 2 == 0 {293 if step % 2 == 0 {
...@@ -313,17 +313,10 @@ fn seeded_formatting_reconciles_exactly_when_the_remote_left_the_paragraph_alone...@@ -313,17 +313,10 @@ fn seeded_formatting_reconciles_exactly_when_the_remote_left_the_paragraph_alone
313 }313 }
314 if underlined[target] && start == 0 {314 if underlined[target] && start == 0 {
315 conflicts += 1;315 conflicts += 1;
316 assert_eq!(316 assert!(matches!(outcome.1, EditStatus::Published { .. }), "seed {seed}");
317 outcome.1,317 assert!(conflicted(&server.durable, space), "seed {seed}");
318 EditStatus::Conflict(ConflictKind::ContentChanged),318 assert_eq!(page_of(&server.durable, space), remote_page, "seed {seed}");
319 "seed {seed}"319 assert_ne!(pages(&snapshot(&cache)), pages(&local), "seed {seed}");
320 );
321 assert_eq!(server.publications, 0, "seed {seed}");
322 assert_eq!(
323 pages(&cache.snapshot().unwrap()),
324 pages(&local),
325 "seed {seed}"
326 );
327 continue;320 continue;
328 }321 }
329 published += 1;322 published += 1;
...@@ -345,7 +338,7 @@ fn seeded_formatting_reconciles_exactly_when_the_remote_left_the_paragraph_alone...@@ -345,7 +338,7 @@ fn seeded_formatting_reconciles_exactly_when_the_remote_left_the_paragraph_alone
345 "seed {seed}"338 "seed {seed}"
346 );339 );
347 }340 }
348 assert_eq!(cache.snapshot().unwrap(), server.durable, "seed {seed}");341 assert_eq!(snapshot(&cache), server.durable, "seed {seed}");
349 }342 }
350 assert!(343 assert!(
351 published >= 16,344 published >= 16,
...@@ -366,11 +359,11 @@ fn offline_insertions_survive_reopen_and_carry_their_formatting() {...@@ -366,11 +359,11 @@ fn offline_insertions_survive_reopen_and_carry_their_formatting() {
366 })359 })
367 .unwrap()360 .unwrap()
368 .unwrap();361 .unwrap();
369 let local = cache.snapshot().unwrap();362 let local = snapshot(&cache);
370 let pending = cache.pending().unwrap();363 let pending = cache.pending().unwrap();
371 drop(cache);364 drop(cache);
372 cache = Replica::open(&path).unwrap();365 cache = Replica::open(&path).unwrap();
373 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));366 assert_eq!(pages(&snapshot(&cache)), pages(&local));
374 assert_eq!(cache.pending().unwrap(), pending);367 assert_eq!(cache.pending().unwrap(), pending);
375 let mut server = remote_with(space, |page| replace_text(page, ids[3], 0..0, "Remote "));368 let mut server = remote_with(space, |page| replace_text(page, ids[3], 0..0, "Remote "));
376 assert!(369 assert!(
...@@ -399,7 +392,7 @@ fn offline_insertions_survive_reopen_and_carry_their_formatting() {...@@ -399,7 +392,7 @@ fn offline_insertions_survive_reopen_and_carry_their_formatting() {
399 for (offset, format) in styles.iter().enumerate() {392 for (offset, format) in styles.iter().enumerate() {
400 assert_eq!(format.bold == Some(true), (8..17).contains(&offset));393 assert_eq!(format.bold == Some(true), (8..17).contains(&offset));
401 }394 }
402 assert_eq!(cache.snapshot().unwrap(), server.durable);395 assert_eq!(snapshot(&cache), server.durable);
403}396}
404397
405#[test]398#[test]
...@@ -415,7 +408,7 @@ fn uncertain_formatting_attempts_keep_the_original_attempt_and_never_replay() {...@@ -415,7 +408,7 @@ fn uncertain_formatting_attempts_keep_the_original_attempt_and_never_replay() {
415 })408 })
416 .unwrap()409 .unwrap()
417 .unwrap();410 .unwrap();
418 let local = cache.snapshot().unwrap();411 let local = snapshot(&cache);
419 let mut server = Server::new(OUTLINES);412 let mut server = Server::new(OUTLINES);
420 server.fault = fault;413 server.fault = fault;
421 assert!(cache.sync_once(&mut server).is_err());414 assert!(cache.sync_once(&mut server).is_err());
...@@ -433,7 +426,7 @@ fn uncertain_formatting_attempts_keep_the_original_attempt_and_never_replay() {...@@ -433,7 +426,7 @@ fn uncertain_formatting_attempts_keep_the_original_attempt_and_never_replay() {
433 }426 }
434 assert_eq!(server.publications, 1);427 assert_eq!(server.publications, 1);
435 assert_eq!(server.durable, OUTLINES);428 assert_eq!(server.durable, OUTLINES);
436 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));429 assert_eq!(pages(&snapshot(&cache)), pages(&local));
437 continue;430 continue;
438 }431 }
439 assert!(matches!(432 assert!(matches!(
...@@ -474,7 +467,7 @@ fn a_format_the_remote_already_carries_is_confirmed_without_republishing() {...@@ -474,7 +467,7 @@ fn a_format_the_remote_already_carries_is_confirmed_without_republishing() {
474 );467 );
475 assert_eq!((server.publications, server.confirmations), (0, 2));468 assert_eq!((server.publications, server.confirmations), (0, 2));
476 assert!(cache.pending().unwrap().is_empty());469 assert!(cache.pending().unwrap().is_empty());
477 let published = page_of(&cache.snapshot().unwrap(), space);470 let published = page_of(&snapshot(&cache), space);
478 assert_eq!(published, page_of(&server.durable, space));471 assert_eq!(published, page_of(&server.durable, space));
479 assert!(472 assert!(
480 formats(&published, ids[0])473 formats(&published, ids[0])
...@@ -552,7 +545,7 @@ fn paragraph_formatting_survives_reopen_rebase_and_a_lost_publication_reply() {...@@ -552,7 +545,7 @@ fn paragraph_formatting_survives_reopen_rebase_and_a_lost_publication_reply() {
552 let id = save(&cache, ids[0], |page| restyle(page, ids[0], 0..6, apply))545 let id = save(&cache, ids[0], |page| restyle(page, ids[0], 0..6, apply))
553 .unwrap()546 .unwrap()
554 .unwrap();547 .unwrap();
555 let expected = formats(&page_of(&cache.snapshot().unwrap(), space), ids[0]);548 let expected = formats(&page_of(&snapshot(&cache), space), ids[0]);
556 drop(cache);549 drop(cache);
557 let cache = Replica::open(directory.path().join("cache.sqlite")).unwrap();550 let cache = Replica::open(directory.path().join("cache.sqlite")).unwrap();
558 let mut server = remote_with(space, |page| replace_text(page, ids[3], 0..0, "Remote "));551 let mut server = remote_with(space, |page| replace_text(page, ids[3], 0..0, "Remote "));
crates/notebook/tests/sync_model.rs+70-66
...@@ -1,8 +1,11 @@...@@ -1,8 +1,11 @@
1//! Page-model saves: one intent per page until it is attempted, reconciliation against the1//! Page-model saves: one intent per page until it is attempted, reconciliation against the
2//! remote page, reviewed conflicts, and independence between pages.2//! remote page, reviewed conflicts, and independence between pages.
33
4use notebook::{ConflictKind, EditStatus, Replica};4#[path = "../../onestore/tests/support/ops.rs"]
5use onestore::{ExGuid, PreparedEdit, RevisionIndex, Store, document::Document};5mod ops;
6
7use notebook::{EditStatus, Replica};
8use onestore::{ExGuid, RevisionIndex, Store, document::Document};
69
7#[path = "support/server.rs"]10#[path = "support/server.rs"]
8mod server;11mod server;
...@@ -80,7 +83,7 @@ fn a_saved_page_model_publishes_and_reads_back() {...@@ -80,7 +83,7 @@ fn a_saved_page_model_publishes_and_reads_back() {
80 .unwrap()83 .unwrap()
81 .unwrap();84 .unwrap();
82 assert_eq!(cache.status(id).unwrap(), Some(EditStatus::Pending));85 assert_eq!(cache.status(id).unwrap(), Some(EditStatus::Pending));
83 assert_eq!(page_of(&cache.snapshot().unwrap(), space), after);86 assert_eq!(page_of(&snapshot(&cache), space), after);
84 let pending = cache.pending().unwrap();87 let pending = cache.pending().unwrap();
85 assert_eq!(pending.len(), 1);88 assert_eq!(pending.len(), 1);
86 assert_eq!(pending[0].author, "Model author");89 assert_eq!(pending[0].author, "Model author");
...@@ -113,7 +116,7 @@ fn saves_before_an_attempt_publish_together() {...@@ -113,7 +116,7 @@ fn saves_before_an_attempt_publish_together() {
113 let id = model_ops::save_as(&cache, space, &after, "Model author")116 let id = model_ops::save_as(&cache, space, &after, "Model author")
114 .unwrap()117 .unwrap()
115 .unwrap();118 .unwrap();
116 let mut later = page_of(&cache.snapshot().unwrap(), space);119 let mut later = page_of(&snapshot(&cache), space);
117 append(&mut later, " two");120 append(&mut later, " two");
118 let second = model_ops::save_as(&cache, space, &later, "Model author")121 let second = model_ops::save_as(&cache, space, &later, "Model author")
119 .unwrap()122 .unwrap()
...@@ -133,7 +136,7 @@ fn saves_before_an_attempt_publish_together() {...@@ -133,7 +136,7 @@ fn saves_before_an_attempt_publish_together() {
133 first_text(&page_of(&server.durable, space)),136 first_text(&page_of(&server.durable, space)),
134 first_text(&later)137 first_text(&later)
135 );138 );
136 let mut third = page_of(&cache.snapshot().unwrap(), space);139 let mut third = page_of(&snapshot(&cache), space);
137 append(&mut third, " three");140 append(&mut third, " three");
138 let next = model_ops::save_as(&cache, space, &third, "Model author")141 let next = model_ops::save_as(&cache, space, &third, "Model author")
139 .unwrap()142 .unwrap()
...@@ -157,7 +160,7 @@ fn an_attempted_save_is_never_rewritten() {...@@ -157,7 +160,7 @@ fn an_attempted_save_is_never_rewritten() {
157 cache.status(id).unwrap(),160 cache.status(id).unwrap(),
158 Some(EditStatus::AwaitingConfirmation { .. })161 Some(EditStatus::AwaitingConfirmation { .. })
159 ));162 ));
160 let mut more = page_of(&cache.snapshot().unwrap(), space);163 let mut more = page_of(&snapshot(&cache), space);
161 append(&mut more, " more");164 append(&mut more, " more");
162 let next = model_ops::save_as(&cache, space, &more, "Model author")165 let next = model_ops::save_as(&cache, space, &more, "Model author")
163 .unwrap()166 .unwrap()
...@@ -165,7 +168,7 @@ fn an_attempted_save_is_never_rewritten() {...@@ -165,7 +168,7 @@ fn an_attempted_save_is_never_rewritten() {
165 assert!(next > id);168 assert!(next > id);
166 let pending = cache.pending().unwrap();169 let pending = cache.pending().unwrap();
167 assert_eq!(pending.iter().map(|p| p.id).collect::<Vec<_>>(), [id, next]);170 assert_eq!(pending.iter().map(|p| p.id).collect::<Vec<_>>(), [id, next]);
168 assert_eq!(page_of(&cache.snapshot().unwrap(), space), more);171 assert_eq!(page_of(&snapshot(&cache), space), more);
169 assert!(matches!(172 assert!(matches!(
170 cache.status(id).unwrap(),173 cache.status(id).unwrap(),
171 Some(EditStatus::AwaitingConfirmation { .. })174 Some(EditStatus::AwaitingConfirmation { .. })
...@@ -173,8 +176,10 @@ fn an_attempted_save_is_never_rewritten() {...@@ -173,8 +176,10 @@ fn an_attempted_save_is_never_rewritten() {
173 assert_eq!(cache.status(next).unwrap(), Some(EditStatus::Pending));176 assert_eq!(cache.status(next).unwrap(), Some(EditStatus::Pending));
174}177}
175178
179/// A remote change to the saved text keeps the remote's page and the local one as its
180/// conflict page; the writer's change made again publishes over the remote's.
176#[test]181#[test]
177fn a_remote_change_to_the_saved_page_conflicts_until_reviewed() {182fn a_remote_change_to_the_saved_page_keeps_a_conflict_page() {
178 let (space, mut after) = page_titled(OUTLINES, "Move leaf down");183 let (space, mut after) = page_titled(OUTLINES, "Move leaf down");
179 let directory = tempfile::tempdir().unwrap();184 let directory = tempfile::tempdir().unwrap();
180 let cache = Replica::create(directory.path().join("cache.sqlite"), OUTLINES).unwrap();185 let cache = Replica::create(directory.path().join("cache.sqlite"), OUTLINES).unwrap();
...@@ -184,49 +189,37 @@ fn a_remote_change_to_the_saved_page_conflicts_until_reviewed() {...@@ -184,49 +189,37 @@ fn a_remote_change_to_the_saved_page_conflicts_until_reviewed() {
184 .unwrap();189 .unwrap();
185 let mut remote_page = page_of(OUTLINES, space);190 let mut remote_page = page_of(OUTLINES, space);
186 append(&mut remote_page, " remote");191 append(&mut remote_page, " remote");
187 let remote = PreparedEdit::page(OUTLINES, space, &remote_page, "Native author").unwrap();192 let remote = ops::saved(OUTLINES, space, &remote_page).unwrap();
188 let mut server = Server::new(remote.as_bytes());193 let mut server = Server::new(remote.as_slice());
189 assert_eq!(194 assert!(matches!(
190 cache.sync_once(&mut server).unwrap().edit,195 cache.sync_once(&mut server).unwrap().edit,
191 Some((id, EditStatus::Conflict(ConflictKind::ContentChanged)))196 Some((_, EditStatus::Published { .. }))
197 ));
198 assert!(matches!(
199 cache.status(id).unwrap(),
200 Some(EditStatus::Published { .. })
201 ));
202 assert_eq!(server.publications, 1);
203 let published = page_of(&server.durable, space);
204 assert_eq!(first_text(&published), first_text(&remote_page));
205 assert_eq!(
206 conflicts(&server.durable)[0].1[0].0,
207 "Model author",
208 "the conflict page names whose version it is"
192 );209 );
193 assert_eq!(server.publications, 0);210 let text = published
194 assert!(cache.pending().unwrap().len() == 1);211 .objects
195 let observed = cache.remote_snapshot().unwrap();212 .iter()
196 assert_eq!(observed, remote.as_bytes());213 .find_map(|object| match object {
197 let mut reviewed = page_of(&observed, space);214 PageObject::Outline(outline) => outline.paragraphs[0].text().map(|text| text.id),
198 append(&mut reviewed, " local");215 _ => None,
199 assert!(cache.resolve(id + 1, notebook::Resolution::Theirs).is_err());216 })
200 let text = model_ops::paragraph_with(&reviewed, {217 .unwrap();
201 let outline = reviewed218 let again = model_ops::save(&cache, text, |page| append(page, " local"))
202 .objects
203 .iter()
204 .find_map(|object| match object {
205 PageObject::Outline(outline) => Some(outline),
206 _ => None,
207 })
208 .unwrap();
209 outline.paragraphs[0].text().unwrap().id
210 })
211 .unwrap()
212 .text()
213 .unwrap()
214 .id;
215 let reviewed_id = model_ops::review(&cache, id, text, |page| append(page, " local"))
216 .unwrap()219 .unwrap()
217 .unwrap();220 .unwrap();
218 // The conflicted edit stays, emptied, and publishes with the reviewed one.
219 assert_eq!(cache.status(id).unwrap(), Some(EditStatus::Pending));
220 assert_eq!(
221 cache.status(reviewed_id).unwrap(),
222 Some(EditStatus::Pending)
223 );
224 assert!(221 assert!(
225 matches!(cache.sync_once(&mut server).unwrap().edit, Some((n, EditStatus::Published { .. })) if n == reviewed_id)222 matches!(cache.sync_once(&mut server).unwrap().edit, Some((n, EditStatus::Published { .. })) if n == again)
226 );
227 assert_eq!(
228 first_text(&page_of(&server.durable, space)),
229 first_text(&reviewed)
230 );223 );
231 assert!(first_text(&page_of(&server.durable, space)).ends_with(" remote local"));224 assert!(first_text(&page_of(&server.durable, space)).ends_with(" remote local"));
232}225}
...@@ -242,14 +235,14 @@ fn remote_changes_to_other_pages_do_not_conflict() {...@@ -242,14 +235,14 @@ fn remote_changes_to_other_pages_do_not_conflict() {
242 .unwrap()235 .unwrap()
243 .unwrap();236 .unwrap();
244 append(&mut other_page, " elsewhere");237 append(&mut other_page, " elsewhere");
245 let remote = PreparedEdit::page(OUTLINES, other, &other_page, "Native author").unwrap();238 let remote = ops::saved(OUTLINES, other, &other_page).unwrap();
246 let mut server = Server::new(remote.as_bytes());239 let mut server = Server::new(remote.as_slice());
247 assert!(240 assert!(
248 matches!(cache.sync_once(&mut server).unwrap().edit, Some((n, EditStatus::Published { .. })) if n == id)241 matches!(cache.sync_once(&mut server).unwrap().edit, Some((n, EditStatus::Published { .. })) if n == id)
249 );242 );
250 assert_eq!(page_of(&server.durable, space), after);243 assert_eq!(page_of(&server.durable, space), after);
251 assert_eq!(page_of(&server.durable, other), other_page);244 assert_eq!(page_of(&server.durable, other), other_page);
252 assert_eq!(page_of(&cache.snapshot().unwrap(), other), other_page);245 assert_eq!(page_of(&snapshot(&cache), other), other_page);
253}246}
254247
255#[test]248#[test]
...@@ -267,7 +260,7 @@ fn pending_saves_survive_reopening_the_cache() {...@@ -267,7 +260,7 @@ fn pending_saves_survive_reopening_the_cache() {
267 let reopened = Replica::open(&path).unwrap();260 let reopened = Replica::open(&path).unwrap();
268 assert_eq!(reopened.pending().unwrap(), pending);261 assert_eq!(reopened.pending().unwrap(), pending);
269 assert_eq!(reopened.status(id).unwrap(), Some(EditStatus::Pending));262 assert_eq!(reopened.status(id).unwrap(), Some(EditStatus::Pending));
270 assert_eq!(page_of(&reopened.snapshot().unwrap(), space), after);263 assert_eq!(page_of(&snapshot(&reopened), space), after);
271}264}
272265
273/// Edits paragraph `index` of the first body outline: replaces its text range with `replacement`.266/// Edits paragraph `index` of the first body outline: replaces its text range with `replacement`.
...@@ -309,11 +302,7 @@ fn texts(page: &Page) -> Vec<String> {...@@ -309,11 +302,7 @@ fn texts(page: &Page) -> Vec<String> {
309fn remote_with(space: ExGuid, change: impl FnOnce(&mut Page)) -> Server {302fn remote_with(space: ExGuid, change: impl FnOnce(&mut Page)) -> Server {
310 let mut page = page_of(OUTLINES, space);303 let mut page = page_of(OUTLINES, space);
311 change(&mut page);304 change(&mut page);
312 Server::new(305 Server::new(ops::saved(OUTLINES, space, &page).unwrap().as_slice())
313 PreparedEdit::page(OUTLINES, space, &page, "Native author")
314 .unwrap()
315 .as_bytes(),
316 )
317}306}
318307
319#[test]308#[test]
...@@ -366,11 +355,16 @@ fn concurrent_edits_to_one_paragraph_merge_unless_their_ranges_overlap() {...@@ -366,11 +355,16 @@ fn concurrent_edits_to_one_paragraph_merge_unless_their_ranges_overlap() {
366 .unwrap()355 .unwrap()
367 .unwrap();356 .unwrap();
368 let mut server = remote_with(space, |page| edit_paragraph(page, 0, 0..6, "Remote"));357 let mut server = remote_with(space, |page| edit_paragraph(page, 0, 0..6, "Remote"));
369 assert_eq!(358 assert!(matches!(
370 cache.sync_once(&mut server).unwrap().edit,359 cache.sync_once(&mut server).unwrap().edit,
371 Some((id, EditStatus::Conflict(ConflictKind::ContentChanged)))360 Some((_, EditStatus::Published { .. }))
372 );361 ));
373 assert_eq!(server.publications, 0);362 assert!(matches!(
363 cache.status(id).unwrap(),
364 Some(EditStatus::Published { .. })
365 ));
366 assert!(conflicted(&server.durable, space));
367 assert_eq!(server.publications, 1);
374}368}
375369
376#[test]370#[test]
...@@ -428,7 +422,7 @@ fn a_local_insertion_merges_with_a_remote_deletion_elsewhere() {...@@ -428,7 +422,7 @@ fn a_local_insertion_merges_with_a_remote_deletion_elsewhere() {
428}422}
429423
430#[test]424#[test]
431fn a_remote_deletion_of_the_edited_paragraph_conflicts() {425fn a_remote_deletion_of_the_edited_paragraph_keeps_a_conflict_page() {
432 let (space, mut after) = page_titled(OUTLINES, "Move leaf down");426 let (space, mut after) = page_titled(OUTLINES, "Move leaf down");
433 let directory = tempfile::tempdir().unwrap();427 let directory = tempfile::tempdir().unwrap();
434 let cache = Replica::create(directory.path().join("cache.sqlite"), OUTLINES).unwrap();428 let cache = Replica::create(directory.path().join("cache.sqlite"), OUTLINES).unwrap();
...@@ -447,10 +441,15 @@ fn a_remote_deletion_of_the_edited_paragraph_conflicts() {...@@ -447,10 +441,15 @@ fn a_remote_deletion_of_the_edited_paragraph_conflicts() {
447 .unwrap();441 .unwrap();
448 outline.paragraphs.pop();442 outline.paragraphs.pop();
449 });443 });
450 assert_eq!(444 assert!(matches!(
451 cache.sync_once(&mut server).unwrap().edit,445 cache.sync_once(&mut server).unwrap().edit,
452 Some((id, EditStatus::Conflict(ConflictKind::ContentChanged)))446 Some((_, EditStatus::Published { .. }))
453 );447 ));
448 assert!(matches!(
449 cache.status(id).unwrap(),
450 Some(EditStatus::Published { .. })
451 ));
452 assert!(conflicted(&server.durable, space));
454}453}
455454
456#[test]455#[test]
...@@ -515,10 +514,15 @@ fn outline_moves_merge_with_remote_text_edits_but_not_with_remote_moves() {...@@ -515,10 +514,15 @@ fn outline_moves_merge_with_remote_text_edits_but_not_with_remote_moves() {
515 };514 };
516 outline.layout.x = Some(50.0);515 outline.layout.x = Some(50.0);
517 });516 });
518 assert_eq!(517 assert!(matches!(
519 cache.sync_once(&mut server).unwrap().edit,518 cache.sync_once(&mut server).unwrap().edit,
520 Some((id, EditStatus::Conflict(ConflictKind::ContentChanged)))519 Some((_, EditStatus::Published { .. }))
521 );520 ));
521 assert!(matches!(
522 cache.status(id).unwrap(),
523 Some(EditStatus::Published { .. })
524 ));
525 assert!(conflicted(&server.durable, space));
522}526}
523527
524#[test]528#[test]
crates/notebook/tests/sync_outline.rs+82-73
...@@ -1,9 +1,12 @@...@@ -1,9 +1,12 @@
1//! Outline layout through page-model saves: reconciliation against the remote page,1//! Outline layout through page-model saves: reconciliation against the remote page,
2//! review of competing layout, twelve offline writers, and the native outline fixtures.2//! review of competing layout, twelve offline writers, and the native outline fixtures.
33
4use notebook::{ConflictKind, EditStatus, Replica};4#[path = "../../onestore/tests/support/ops.rs"]
5mod ops;
6
7use notebook::{EditStatus, Replica};
5use onestore::{8use onestore::{
6 ExGuid, PreparedEdit, RevisionIndex, Store,9 ExGuid, RevisionIndex, Store,
7 document::Document,10 document::Document,
8 page::{Outline, Page, PageObject, PageParagraph},11 page::{Outline, Page, PageObject, PageParagraph},
9};12};
...@@ -11,12 +14,12 @@ use serde_json::json;...@@ -11,12 +14,12 @@ use serde_json::json;
1114
12#[path = "support/server.rs"]15#[path = "support/server.rs"]
13mod server;16mod server;
14use server::{Fault, Server, pages};17use server::{Fault, Server, conflicts, pages, snapshot};
15#[path = "support/model_ops.rs"]18#[path = "support/model_ops.rs"]
16mod model_ops;19mod model_ops;
17use model_ops::{20use model_ops::{
18 delete_paragraph, insert_outline, outlines_mut, page_of, paragraph_with, replace_text, restyle,21 delete_paragraph, insert_outline, outlines_mut, page_of, paragraph_with, replace_text, restyle,
19 review, save,22 save,
20};23};
21#[path = "support/model_schedule.rs"]24#[path = "support/model_schedule.rs"]
22mod model_schedule;25mod model_schedule;
...@@ -99,12 +102,30 @@ pub(crate) fn text_of(page: &Page, text: ExGuid) -> String {...@@ -99,12 +102,30 @@ pub(crate) fn text_of(page: &Page, text: ExGuid) -> String {
99pub(crate) fn remote_with(source: &[u8], space: ExGuid, change: impl FnOnce(&mut Page)) -> Vec<u8> {102pub(crate) fn remote_with(source: &[u8], space: ExGuid, change: impl FnOnce(&mut Page)) -> Vec<u8> {
100 let mut page = page_of(source, space);103 let mut page = page_of(source, space);
101 change(&mut page);104 change(&mut page);
102 PreparedEdit::page(source, space, &page, "Native author")105 ops::saved(source, space, &page)
103 .unwrap()106 .unwrap()
104 .as_bytes()107 .as_slice()
105 .to_vec()108 .to_vec()
106}109}
107110
111/// Runs one synchronization step, which publishes; true when the remote's version of
112/// `space` stayed and the local one became a conflict page under it.
113pub(crate) fn conflicted(cache: &Replica, server: &mut Server, space: ExGuid) -> bool {
114 let count = |image: &[u8]| {
115 conflicts(image)
116 .into_iter()
117 .filter(|(page, _)| *page == space)
118 .map(|(_, pages)| pages.len())
119 .sum::<usize>()
120 };
121 let before = count(&server.durable);
122 assert!(matches!(
123 cache.sync_once(server).unwrap().edit,
124 Some((_, EditStatus::Published { .. }))
125 ));
126 count(&server.durable) > before
127}
128
108/// Publishes the batch holding `id`, unless an earlier step already did.129/// Publishes the batch holding `id`, unless an earlier step already did.
109pub(crate) fn published(cache: &Replica, server: &mut Server, id: u64) {130pub(crate) fn published(cache: &Replica, server: &mut Server, id: u64) {
110 if !matches!(131 if !matches!(
...@@ -149,13 +170,13 @@ fn layout_and_dependent_text_survive_reopen_and_a_remote_format_of_the_same_page...@@ -149,13 +170,13 @@ fn layout_and_dependent_text_survive_reopen_and_a_remote_format_of_the_same_page
149 let path = directory.path().join("cache.sqlite");170 let path = directory.path().join("cache.sqlite");
150 let cache = Replica::create(&path, &source).unwrap();171 let cache = Replica::create(&path, &source).unwrap();
151 let id = save(&cache, text_id, &change).unwrap().unwrap();172 let id = save(&cache, text_id, &change).unwrap().unwrap();
152 let expected = page_of(&cache.snapshot().unwrap(), space);173 let expected = page_of(&snapshot(&cache), space);
153 let pending = cache.pending().unwrap();174 let pending = cache.pending().unwrap();
154 let local = cache.snapshot().unwrap();175 let local = snapshot(&cache);
155 drop(cache);176 drop(cache);
156 let cache = Replica::open(&path).unwrap();177 let cache = Replica::open(&path).unwrap();
157 assert_eq!(cache.pending().unwrap(), pending);178 assert_eq!(cache.pending().unwrap(), pending);
158 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));179 assert_eq!(pages(&snapshot(&cache)), pages(&local));
159 let mut server = Server::new(&remote_with(&source, space, |page| {180 let mut server = Server::new(&remote_with(&source, space, |page| {
160 restyle(page, text_id, 0..8, |format| format.bold = Some(true));181 restyle(page, text_id, 0..8, |format| format.bold = Some(true));
161 }));182 }));
...@@ -208,9 +229,9 @@ fn an_uncertain_move_confirms_from_the_remote_revision_and_adopts_the_remote_tit...@@ -208,9 +229,9 @@ fn an_uncertain_move_confirms_from_the_remote_revision_and_adopts_the_remote_tit
208 let (source, space, outline, _) = fixture();229 let (source, space, outline, _) = fixture();
209 let mut page = page_of(&source, space);230 let mut page = page_of(&source, space);
210 insert_outline(&mut page, 144.0, 36.0, "Second");231 insert_outline(&mut page, 144.0, 36.0, "Second");
211 let source = PreparedEdit::page(&source, space, &page, "Author")232 let source = ops::saved(&source, space, &page)
212 .unwrap()233 .unwrap()
213 .as_bytes()234 .as_slice()
214 .to_vec();235 .to_vec();
215 let second = text_with(&page_of(&source, space), "Second");236 let second = text_with(&page_of(&source, space), "Second");
216 let directory = tempfile::tempdir().unwrap();237 let directory = tempfile::tempdir().unwrap();
...@@ -242,8 +263,9 @@ fn an_uncertain_move_confirms_from_the_remote_revision_and_adopts_the_remote_tit...@@ -242,8 +263,9 @@ fn an_uncertain_move_confirms_from_the_remote_revision_and_adopts_the_remote_tit
242 assert_eq!((server.publications, server.confirmations), (1, 1));263 assert_eq!((server.publications, server.confirmations), (1, 1));
243}264}
244265
266/// Competing layout keeps the remote's on the page and the local one on its conflict page.
245#[test]267#[test]
246fn competing_layout_conflicts_until_the_local_page_is_kept() {268fn competing_layout_keeps_the_local_one_on_a_conflict_page() {
247 let (source, space, outline, text_id) = fixture();269 let (source, space, outline, text_id) = fixture();
248 let cases: [(Change<Outline>, Change<Outline>); 2] = [270 let cases: [(Change<Outline>, Change<Outline>); 2] = [
249 (Box::new(moved(180.0, 216.0)), Box::new(moved(288.0, 216.0))),271 (Box::new(moved(180.0, 216.0)), Box::new(moved(288.0, 216.0))),
...@@ -256,46 +278,40 @@ fn competing_layout_conflicts_until_the_local_page_is_kept() {...@@ -256,46 +278,40 @@ fn competing_layout_conflicts_until_the_local_page_is_kept() {
256 let directory = tempfile::tempdir().unwrap();278 let directory = tempfile::tempdir().unwrap();
257 let path = directory.path().join("cache.sqlite");279 let path = directory.path().join("cache.sqlite");
258 let cache = Replica::create(&path, &source).unwrap();280 let cache = Replica::create(&path, &source).unwrap();
259 let id = save(&cache, text_id, |page| {281 save(&cache, text_id, |page| {
260 local_change(outline_mut(page, outline))282 local_change(outline_mut(page, outline))
261 })283 })
262 .unwrap()284 .unwrap()
263 .unwrap();285 .unwrap();
264 let local = cache.snapshot().unwrap();286 let local = page_of(&snapshot(&cache), space);
265 let mut server = Server::new(&remote_with(&source, space, |page| {287 let remote = remote_with(&source, space, |page| {
266 remote_change(outline_mut(page, outline));288 remote_change(outline_mut(page, outline));
267 }));289 });
290 let mut server = Server::new(&remote);
291 assert!(conflicted(&cache, &mut server, space));
292 assert_eq!(server.publications, 1);
293 let durable = page_of(&server.durable, space);
268 assert_eq!(294 assert_eq!(
269 cache.sync_once(&mut server).unwrap().edit,295 outline_of(&durable, outline).layout,
270 Some((id, EditStatus::Conflict(ConflictKind::ContentChanged)))296 outline_of(&page_of(&remote, space), outline).layout
271 );297 );
272 assert_eq!(server.publications, 0);298 let arena = onestore::Arena::default();
273 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));299 let mut section = onestore::Section::open(&arena, server.durable.clone()).unwrap();
300 let conflict = section.conflicts().unwrap()[0].1[0].space;
301 let kept = section.page(conflict).unwrap();
302 assert_eq!(body_outlines(&kept)[0].layout, outline_of(&local, outline).layout);
274 let dependent = save(&cache, text_id, |page| {303 let dependent = save(&cache, text_id, |page| {
275 replace_text(page, text_id, 0..0, "Local ")304 replace_text(page, text_id, 0..0, "Local ")
276 })305 })
277 .unwrap()306 .unwrap()
278 .unwrap();307 .unwrap();
279 let local = cache.snapshot().unwrap();
280 drop(cache);
281 let cache = Replica::open(&path).unwrap();
282 assert_eq!(
283 cache.status(id).unwrap(),
284 Some(EditStatus::Conflict(ConflictKind::ContentChanged))
285 );
286 cache.resolve(id, notebook::Resolution::Mine).unwrap();
287 assert_eq!(cache.status(id).unwrap(), Some(EditStatus::Pending));
288 drop(cache);308 drop(cache);
289 let cache = Replica::open(&path).unwrap();309 let cache = Replica::open(&path).unwrap();
290 for id in [id, dependent] {310 published(&cache, &mut server, dependent);
291 published(&cache, &mut server, id);
292 }
293 let durable = page_of(&server.durable, space);
294 assert_eq!(311 assert_eq!(
295 outline_of(&durable, outline).layout,312 text_of(&page_of(&server.durable, space), text_id),
296 outline_of(&page_of(&local, space), outline).layout313 "Local Original 🦀 é"
297 );314 );
298 assert_eq!(text_of(&durable, text_id), "Local Original 🦀 é");
299 }315 }
300}316}
301317
...@@ -346,15 +362,14 @@ fn twelve_offline_writers_preserve_all_layout_intents_through_review_and_restart...@@ -346,15 +362,14 @@ fn twelve_offline_writers_preserve_all_layout_intents_through_review_and_restart
346 paragraph_mut(page, text_id).collapsed = (actor + round) % 2 != 0;362 paragraph_mut(page, text_id).collapsed = (actor + round) % 2 != 0;
347 };363 };
348 let id = save(&cache, text_id, change).unwrap().unwrap();364 let id = save(&cache, text_id, change).unwrap().unwrap();
349 let expected = page_of(&cache.snapshot().unwrap(), space);365 let expected = page_of(&snapshot(&cache), space);
350 drop(cache);366 drop(cache);
351 let cache = Replica::open(&path).unwrap();367 let cache = Replica::open(&path).unwrap();
352 if cache.sync_once(&mut server).unwrap().edit368 if conflicted(&cache, &mut server, space) {
353 == Some((id, EditStatus::Conflict(ConflictKind::ContentChanged)))369 // The writer makes the change again over the remote's version.
354 {370 let again = save(&cache, text_id, change).unwrap().unwrap();
355 review(&cache, id, text_id, change).unwrap();
356 reviewed += 1;371 reviewed += 1;
357 published(&cache, &mut server, id);372 published(&cache, &mut server, again);
358 }373 }
359 assert!(matches!(374 assert!(matches!(
360 cache.status(id).unwrap(),375 cache.status(id).unwrap(),
...@@ -388,7 +403,7 @@ fn an_uncertain_layout_attempt_confirms_by_revision_or_by_an_equal_remote_page()...@@ -388,7 +403,7 @@ fn an_uncertain_layout_attempt_confirms_by_revision_or_by_an_equal_remote_page()
388 })403 })
389 .unwrap()404 .unwrap()
390 .unwrap();405 .unwrap();
391 let local = cache.snapshot().unwrap();406 let local = snapshot(&cache);
392 let mut server = Server::new(&source);407 let mut server = Server::new(&source);
393 server.fault = fault;408 server.fault = fault;
394 assert!(cache.sync_once(&mut server).is_err());409 assert!(cache.sync_once(&mut server).is_err());
...@@ -416,7 +431,7 @@ fn an_uncertain_layout_attempt_confirms_by_revision_or_by_an_equal_remote_page()...@@ -416,7 +431,7 @@ fn an_uncertain_layout_attempt_confirms_by_revision_or_by_an_equal_remote_page()
416 cache.sync_once(&mut server).unwrap().edit,431 cache.sync_once(&mut server).unwrap().edit,
417 Some((id, status.clone()))432 Some((id, status.clone()))
418 );433 );
419 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));434 assert_eq!(pages(&snapshot(&cache)), pages(&local));
420 server.visible = remote_with(&source, space, |page| {435 server.visible = remote_with(&source, space, |page| {
421 resized(144.0, true)(outline_mut(page, outline));436 resized(144.0, true)(outline_mut(page, outline));
422 });437 });
...@@ -483,7 +498,7 @@ pub(crate) fn native_pages(source: &[u8]) -> Vec<(String, ExGuid, ExGuid, ExGuid...@@ -483,7 +498,7 @@ pub(crate) fn native_pages(source: &[u8]) -> Vec<(String, ExGuid, ExGuid, ExGuid
483}498}
484499
485#[test]500#[test]
486fn native_moves_deletions_and_layout_changes_merge_or_require_review() {501fn native_moves_deletions_and_layout_changes_merge_or_keep_a_conflict_page() {
487 let mut server = Server::new(NATIVE);502 let mut server = Server::new(NATIVE);
488 let mut records = Vec::new();503 let mut records = Vec::new();
489 for (name, space, outline, target, dependent) in native_pages(BEFORE) {504 for (name, space, outline, target, dependent) in native_pages(BEFORE) {
...@@ -496,28 +511,20 @@ fn native_moves_deletions_and_layout_changes_merge_or_require_review() {...@@ -496,28 +511,20 @@ fn native_moves_deletions_and_layout_changes_merge_or_require_review() {
496 let path = directory.path().join("cache.sqlite");511 let path = directory.path().join("cache.sqlite");
497 let cache = Replica::create(&path, BEFORE).unwrap();512 let cache = Replica::create(&path, BEFORE).unwrap();
498 let id = save(&cache, dependent, save_both).unwrap().unwrap();513 let id = save(&cache, dependent, save_both).unwrap().unwrap();
499 let local = cache.snapshot().unwrap();514 let merged = !conflicted(&cache, &mut server, space);
500 let queue = cache.pending().unwrap();515 assert!(matches!(
501 let merged = match cache.sync_once(&mut server).unwrap().edit.unwrap() {516 cache.status(id).unwrap(),
502 (actual, EditStatus::Published { .. }) => {517 Some(EditStatus::Published { .. })
503 assert_eq!(actual, id, "{name}");518 ));
504 true519 if !merged {
505 }520 // The dependent text merged; the writer makes the change again.
506 result => {521 drop(cache);
507 assert_eq!(522 let cache = Replica::open(&path).unwrap();
508 result,523 // Unless the remote made the same change.
509 (id, EditStatus::Conflict(ConflictKind::ContentChanged)),524 if let Some(again) = save(&cache, dependent, &change).unwrap() {
510 "{name}"525 published(&cache, &mut server, again);
511 );
512 assert_eq!(cache.pending().unwrap(), queue);
513 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));
514 drop(cache);
515 let cache = Replica::open(&path).unwrap();
516 review(&cache, id, dependent, save_both).unwrap();
517 published(&cache, &mut server, id);
518 false
519 }526 }
520 };527 }
521 let durable = page_of(&server.durable, space);528 let durable = page_of(&server.durable, space);
522 assert!(529 assert!(
523 text_of(&durable, dependent).starts_with("Local "),530 text_of(&durable, dependent).starts_with("Local "),
...@@ -564,7 +571,7 @@ fn native_moves_deletions_and_layout_changes_merge_or_require_review() {...@@ -564,7 +571,7 @@ fn native_moves_deletions_and_layout_changes_merge_or_require_review() {
564}571}
565572
566#[test]573#[test]
567fn a_new_native_wrap_reservation_requires_review_before_width_replacement() {574fn a_new_native_wrap_reservation_keeps_the_local_width_on_a_conflict_page() {
568 let (_, space, outline, _, text_id) = native_pages(BEFORE)575 let (_, space, outline, _, text_id) = native_pages(BEFORE)
569 .into_iter()576 .into_iter()
570 .find(|(name, ..)| name == "Move outline")577 .find(|(name, ..)| name == "Move outline")
...@@ -584,15 +591,17 @@ fn a_new_native_wrap_reservation_requires_review_before_width_replacement() {...@@ -584,15 +591,17 @@ fn a_new_native_wrap_reservation_requires_review_before_width_replacement() {
584 .unwrap()591 .unwrap()
585 .unwrap();592 .unwrap();
586 let mut server = Server::new(NATIVE);593 let mut server = Server::new(NATIVE);
587 assert_eq!(594 assert!(conflicted(&cache, &mut server, space));
588 cache.sync_once(&mut server).unwrap().edit,595 assert!(matches!(
589 Some((id, EditStatus::Conflict(ConflictKind::ContentChanged)))596 cache.status(id).unwrap(),
590 );597 Some(EditStatus::Published { .. })
591 review(&cache, id, text_id, |page| {598 ));
599 let again = save(&cache, text_id, |page| {
592 resized(144.0, true)(outline_mut(page, outline))600 resized(144.0, true)(outline_mut(page, outline))
593 })601 })
602 .unwrap()
594 .unwrap();603 .unwrap();
595 published(&cache, &mut server, id);604 published(&cache, &mut server, again);
596 let layout = outline_of(&page_of(&server.durable, space), outline)605 let layout = outline_of(&page_of(&server.durable, space), outline)
597 .layout606 .layout
598 .clone();607 .clone();
crates/notebook/tests/sync_page.rs+85-92
...@@ -1,7 +1,10 @@...@@ -1,7 +1,10 @@
1//! Page creation reconciliation: dependent body saves, uncertain publication, anchor review,1//! Page creation reconciliation: dependent body saves, uncertain publication, lost anchors,
2//! and deterministic multi-actor schedules.2//! and deterministic multi-actor schedules.
33
4use notebook::{ConflictKind, EditStatus, Recovery, Replica, Resolution};4#[path = "../../onestore/tests/support/ops.rs"]
5mod ops;
6
7use notebook::{EditStatus, Recovery, Replica};
5use onestore::op::SectionOp;8use onestore::op::SectionOp;
6use onestore::{9use onestore::{
7 ExGuid, PageCreation, RevisionIndex, Store,10 ExGuid, PageCreation, RevisionIndex, Store,
...@@ -67,11 +70,11 @@ fn offline_page_creation_rebases_with_dependent_edits_and_duplicate_titles() {...@@ -67,11 +70,11 @@ fn offline_page_creation_rebases_with_dependent_edits_and_duplicate_titles() {
67 assert!(70 assert!(
68 matches!(&original[0].edit.ops[..], [onestore::op::Op::Section(SectionOp::Create(retained))] if retained == &page)71 matches!(&original[0].edit.ops[..], [onestore::op::Op::Section(SectionOp::Create(retained))] if retained == &page)
69 );72 );
70 let local = cache.snapshot().unwrap();73 let local = snapshot(&cache);
71 drop(cache);74 drop(cache);
72 let cache = Replica::open(&path).unwrap();75 let cache = Replica::open(&path).unwrap();
73 assert_eq!(cache.pending().unwrap(), original);76 assert_eq!(cache.pending().unwrap(), original);
74 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));77 assert_eq!(pages(&snapshot(&cache)), pages(&local));
75 assert!(matches!(78 assert!(matches!(
76 cache.sync_once(&mut server).unwrap().edit,79 cache.sync_once(&mut server).unwrap().edit,
77 Some((published, EditStatus::Published { .. })) if published == dependent80 Some((published, EditStatus::Published { .. })) if published == dependent
...@@ -120,7 +123,7 @@ fn uncertain_page_publication_retains_both_revisions_and_never_replays() {...@@ -120,7 +123,7 @@ fn uncertain_page_publication_retains_both_revisions_and_never_replays() {
120 let cache = Replica::create(&path, &source).unwrap();123 let cache = Replica::create(&path, &source).unwrap();
121 let page = PageCreation::new(None, Some("Created"), "Author").unwrap();124 let page = PageCreation::new(None, Some("Created"), "Author").unwrap();
122 let id = section_op(&cache, SectionOp::Create(page.clone()));125 let id = section_op(&cache, SectionOp::Create(page.clone()));
123 let local = cache.snapshot().unwrap();126 let local = snapshot(&cache);
124 let mut server = Server::new(&source);127 let mut server = Server::new(&source);
125 server.fault = fault;128 server.fault = fault;
126 let _ = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {129 let _ = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
...@@ -155,12 +158,12 @@ fn uncertain_page_publication_retains_both_revisions_and_never_replays() {...@@ -155,12 +158,12 @@ fn uncertain_page_publication_retains_both_revisions_and_never_replays() {
155 assert_eq!(server.confirmations, usize::from(visible));158 assert_eq!(server.confirmations, usize::from(visible));
156 if visible {159 if visible {
157 assert!(matches!(result.1, EditStatus::Published { .. }));160 assert!(matches!(result.1, EditStatus::Published { .. }));
158 assert!(cache.snapshot().unwrap() == observed);161 assert!(snapshot(&cache) == observed);
159 assert!(observed[..212] == server.durable[..212]);162 assert!(observed[..212] == server.durable[..212]);
160 assert!(observed[252..] == server.durable[252..]);163 assert!(observed[252..] == server.durable[252..]);
161 } else {164 } else {
162 assert_eq!(result, (id, attempted.clone()));165 assert_eq!(result, (id, attempted.clone()));
163 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));166 assert_eq!(pages(&snapshot(&cache)), pages(&local));
164 }167 }
165 }168 }
166}169}
...@@ -235,8 +238,10 @@ fn surviving_page_revision_alone_does_not_confirm_section_publication() {...@@ -235,8 +238,10 @@ fn surviving_page_revision_alone_does_not_confirm_section_publication() {
235 assert_eq!(server.publications, 1);238 assert_eq!(server.publications, 1);
236}239}
237240
241/// A page created before a page the remote moved goes before the next page the remote left
242/// in place, its content under the identities it was made with.
238#[test]243#[test]
239fn changed_page_anchor_conflicts_until_kept_without_regenerating_dependent_identities() {244fn a_moved_page_anchor_places_the_page_without_regenerating_dependent_identities() {
240 let source = include_bytes!("../../../corpus/page-lifecycle/03-renamed/notebook/Lifecycle.one");245 let source = include_bytes!("../../../corpus/page-lifecycle/03-renamed/notebook/Lifecycle.one");
241 let remote = include_bytes!("../../../corpus/page-lifecycle/04-nested/notebook/Lifecycle.one");246 let remote = include_bytes!("../../../corpus/page-lifecycle/04-nested/notebook/Lifecycle.one");
242 let store = Store::parse(source).unwrap();247 let store = Store::parse(source).unwrap();
...@@ -252,30 +257,29 @@ fn changed_page_anchor_conflicts_until_kept_without_regenerating_dependent_ident...@@ -252,30 +257,29 @@ fn changed_page_anchor_conflicts_until_kept_without_regenerating_dependent_ident
252 let save = model_ops::save_as(&cache, page.space(), &created, "Author")257 let save = model_ops::save_as(&cache, page.space(), &created, "Author")
253 .unwrap()258 .unwrap()
254 .unwrap();259 .unwrap();
255 let local = cache.snapshot().unwrap();260 let local = snapshot(&cache);
256 let pending = cache.pending().unwrap();
257 let mut server = Server::new(remote);261 let mut server = Server::new(remote);
258 assert_eq!(262 assert!(matches!(
259 cache.sync_once(&mut server).unwrap().edit,263 cache.sync_once(&mut server).unwrap().edit,
260 Some((save, EditStatus::Conflict(ConflictKind::StructureChanged)))264 Some((published, EditStatus::Published { .. })) if published == save
261 );265 ));
262 cache.resolve(id, Resolution::Mine).unwrap();
263 assert_eq!(cache.pending().unwrap()[1], pending[1]);
264 assert!(matches!(266 assert!(matches!(
265 &cache.pending().unwrap()[0].edit.ops[..],267 cache.status(id).unwrap(),
266 [onestore::op::Op::Section(SectionOp::Create(reviewed))] if *reviewed == page.reposition(None).unwrap()268 Some(EditStatus::Published { .. })
267 ));269 ));
270 assert_eq!(server.publications, 1);
268 assert_eq!(271 assert_eq!(
269 model_ops::page_of(&cache.snapshot().unwrap(), page.space()),272 model_ops::page_of(&server.durable, page.space()),
270 model_ops::page_of(&local, page.space())273 model_ops::page_of(&local, page.space())
271 );274 );
272 drop(cache);275 let arena = onestore::Arena::default();
273 let cache = Replica::open(&path).unwrap();276 let listed = onestore::Section::open(&arena, server.durable.clone())
274 assert!(matches!(277 .unwrap()
275 cache.sync_once(&mut server).unwrap().edit,278 .pages()
276 Some((_, EditStatus::Published { .. }))279 .unwrap();
277 ));280 // The remote made `child` and `grandchild` subpages; `Renamed` stayed.
278 assert_eq!(server.publications, 1);281 let at = listed.iter().position(|listed| listed.0 == pages[6].0).unwrap();
282 assert_eq!(listed[at - 1].0, page.space());
279 let store = Store::parse(&server.durable).unwrap();283 let store = Store::parse(&server.durable).unwrap();
280 let index = RevisionIndex::parse(&store).unwrap();284 let index = RevisionIndex::parse(&store).unwrap();
281 let document = Document::parse(&index).unwrap();285 let document = Document::parse(&index).unwrap();
...@@ -285,8 +289,9 @@ fn changed_page_anchor_conflicts_until_kept_without_regenerating_dependent_ident...@@ -285,8 +289,9 @@ fn changed_page_anchor_conflicts_until_kept_without_regenerating_dependent_ident
285 Kind::RichText { text, .. } if text == "Retained body"));289 Kind::RichText { text, .. } if text == "Retained body"));
286}290}
287291
292/// Edits of a page the remote removed put the local version back where it stood.
288#[test]293#[test]
289fn keeping_mine_puts_back_a_page_the_remote_removed_where_it_stood() {294fn a_page_the_remote_removed_comes_back_where_it_stood() {
290 let source = include_bytes!("../../../corpus/page-lifecycle/04-nested/notebook/Lifecycle.one");295 let source = include_bytes!("../../../corpus/page-lifecycle/04-nested/notebook/Lifecycle.one");
291 let order = |image: &[u8]| -> Vec<(ExGuid, String, u32)> {296 let order = |image: &[u8]| -> Vec<(ExGuid, String, u32)> {
292 let arena = onestore::Arena::default();297 let arena = onestore::Arena::default();
...@@ -319,73 +324,61 @@ fn keeping_mine_puts_back_a_page_the_remote_removed_where_it_stood() {...@@ -319,73 +324,61 @@ fn keeping_mine_puts_back_a_page_the_remote_removed_where_it_stood() {
319 _ => None,324 _ => None,
320 })325 })
321 .unwrap();326 .unwrap();
322 for keep in [Resolution::Mine, Resolution::Theirs] {327 let directory = tempfile::tempdir().unwrap();
323 let directory = tempfile::tempdir().unwrap();328 let cache = Replica::create(directory.path().join("pages.sqlite"), source).unwrap();
324 let cache = Replica::create(directory.path().join("pages.sqlite"), source).unwrap();329 model_ops::save(&cache, text, |page| {
325 let id = model_ops::save(&cache, text, |page| {330 model_ops::replace_text(page, text, 0..0, "Kept ")
326 model_ops::replace_text(page, text, 0..0, "Kept ")331 })
327 })332 .unwrap()
333 .unwrap();
334 let local = cache.page(space).unwrap();
335 let removed = ops::section_op(source, SectionOp::Delete([space].to_vec()))
328 .unwrap()336 .unwrap()
329 .unwrap();337 .as_bytes()
330 let local = cache.page(space).unwrap();338 .to_vec();
331 let removed = onestore::PreparedEdit::delete_pages_permanently(source, &[space])339 let mut server = Server::new(&removed);
332 .unwrap()340 assert!(matches!(
333 .as_bytes()341 cache.sync_once(&mut server).unwrap().edit,
334 .to_vec();342 Some((_, EditStatus::Published { .. }))
335 let mut server = Server::new(&removed);343 ));
336 assert_eq!(344 assert_eq!(
337 cache.sync_once(&mut server).unwrap().edit,345 server.publications, 1,
338 Some((id, EditStatus::Conflict(ConflictKind::TargetUnavailable)))346 "a removed page is not edited in place"
339 );347 );
340 assert_eq!(348 let published = order(&server.durable);
341 server.publications, 0,349 assert!(published.iter().all(|(page, ..)| *page != space));
342 "a removed page is not edited in place"350 assert_eq!(published.len(), pages.len());
343 );351 let (restored, title, restored_level) = published[at].clone();
344 cache.resolve(id, keep).unwrap();352 assert_eq!(
345 assert!(matches!(353 (title.as_str(), restored_level),
346 cache.sync_once(&mut server).unwrap().edit,354 (pages[at].1.as_str(), level)
347 Some((_, EditStatus::Published { .. }))355 );
348 ));356 let page = model_ops::page_of(&server.durable, restored);
349 let published = order(&server.durable);357 let texts = |page: &onestore::page::Page| -> Vec<String> {
350 assert!(published.iter().all(|(page, ..)| *page != space));358 page.objects
351 match keep {359 .iter()
352 Resolution::Mine => {360 .filter_map(|object| match object {
353 assert_eq!(published.len(), pages.len());361 onestore::page::PageObject::Outline(outline) => Some(outline),
354 let (restored, title, restored_level) = published[at].clone();362 _ => None,
355 assert_eq!(363 })
356 (title.as_str(), restored_level),364 .flat_map(|outline| &outline.paragraphs)
357 (pages[at].1.as_str(), level)365 .filter_map(|p| p.text().map(|t| t.text.text().to_owned()))
358 );366 .collect()
359 let page = model_ops::page_of(&server.durable, restored);367 };
360 let texts = |page: &onestore::page::Page| -> Vec<String> {368 assert_eq!(texts(&page), texts(&local));
361 page.objects369 assert!(texts(&page)[0].starts_with("Kept "));
362 .iter()370 let others: Vec<_> = published
363 .filter_map(|object| match object {371 .iter()
364 onestore::page::PageObject::Outline(outline) => Some(outline),372 .map(|(page, ..)| *page)
365 _ => None,373 .filter(|page| *page != restored)
366 })374 .collect();
367 .flat_map(|outline| &outline.paragraphs)375 let before: Vec<_> = pages
368 .filter_map(|p| p.text().map(|t| t.text.text().to_owned()))376 .iter()
369 .collect()377 .map(|(page, ..)| *page)
370 };378 .filter(|page| *page != space)
371 assert_eq!(texts(&page), texts(&local));379 .collect();
372 assert!(texts(&page)[0].starts_with("Kept "));380 assert_eq!(others, before);
373 let others: Vec<_> = published381 assert_eq!(pages_of(&cache), published);
374 .iter()
375 .map(|(page, ..)| *page)
376 .filter(|page| *page != restored)
377 .collect();
378 let before: Vec<_> = pages
379 .iter()
380 .map(|(page, ..)| *page)
381 .filter(|page| *page != space)
382 .collect();
383 assert_eq!(others, before);
384 }
385 Resolution::Theirs => assert_eq!(published, order(&removed)),
386 }
387 assert_eq!(pages_of(&cache), published);
388 }
389}382}
390383
391fn pages_of(cache: &Replica) -> Vec<(ExGuid, String, u32)> {384fn pages_of(cache: &Replica) -> Vec<(ExGuid, String, u32)> {
crates/notebook/tests/sync_pages.rs+165-116
...@@ -1,10 +1,14 @@...@@ -1,10 +1,14 @@
1//! Page batch reconciliation: atomic batches with dependent body saves, competing moves and1//! Page batch reconciliation: atomic batches with dependent body saves, competing moves and
2//! indentation, uncertain batches, and native page movement fixtures.2//! indentation (a page the remote moved keeps the remote's placement), uncertain batches, and
3//! native page movement fixtures.
34
4use notebook::{ConflictKind, EditStatus, Recovery, Replica, Resolution};5#[path = "../../onestore/tests/support/ops.rs"]
6mod ops;
7
8use notebook::{EditStatus, Recovery, Replica};
5use onestore::op::{Op, SectionOp};9use onestore::op::{Op, SectionOp};
6use onestore::{10use onestore::{
7 ExGuid, PageEdit, PagePosition, PreparedEdit, RevisionIndex, Store,11 ExGuid, PageEdit, PagePosition, RevisionIndex, Store,
8 document::{Document, Format, Kind, Layout},12 document::{Document, Format, Kind, Layout},
9 page::{13 page::{
10 Outline, Page, PageObject, PageParagraph, Paragraph, ParagraphContent, TextObject,14 Outline, Page, PageObject, PageParagraph, Paragraph, ParagraphContent, TextObject,
...@@ -135,12 +139,12 @@ fn atomic_page_batches_survive_reopen_with_dependent_text() {...@@ -135,12 +139,12 @@ fn atomic_page_batches_survive_reopen_with_dependent_text() {
135 assert!(139 assert!(
136 matches!(&queue[0].edit.ops[..], [Op::Section(SectionOp::Pages(batch))] if *batch == edits)140 matches!(&queue[0].edit.ops[..], [Op::Section(SectionOp::Pages(batch))] if *batch == edits)
137 );141 );
138 let local = cache.snapshot().unwrap();142 let local = snapshot(&cache);
139 drop(cache);143 drop(cache);
140 let cache = Replica::open(&path).unwrap();144 let cache = Replica::open(&path).unwrap();
141 assert_eq!(cache.pending().unwrap(), queue);145 assert_eq!(cache.pending().unwrap(), queue);
142 assert_eq!(146 assert_eq!(
143 server::pages(&cache.snapshot().unwrap()),147 server::pages(&snapshot(&cache)),
144 server::pages(&local)148 server::pages(&local)
145 );149 );
146 let mut server = Server::new(SOURCE);150 let mut server = Server::new(SOURCE);
...@@ -164,7 +168,7 @@ fn atomic_page_batches_survive_reopen_with_dependent_text() {...@@ -164,7 +168,7 @@ fn atomic_page_batches_survive_reopen_with_dependent_text() {
164}168}
165169
166#[test]170#[test]
167fn indentation_only_edits_preserve_remote_page_movement() {171fn indenting_a_page_the_remote_moved_keeps_the_remote_placement() {
168 let directory = tempfile::tempdir().unwrap();172 let directory = tempfile::tempdir().unwrap();
169 let cache = Replica::create(directory.path().join("pages.sqlite"), SOURCE).unwrap();173 let cache = Replica::create(directory.path().join("pages.sqlite"), SOURCE).unwrap();
170 let before = order(SOURCE);174 let before = order(SOURCE);
...@@ -174,86 +178,56 @@ fn indentation_only_edits_preserve_remote_page_movement() {...@@ -174,86 +178,56 @@ fn indentation_only_edits_preserve_remote_page_movement() {
174 SectionOp::Pages([PageEdit::set_level(sid, 3).unwrap()].to_vec()),178 SectionOp::Pages([PageEdit::set_level(sid, 3).unwrap()].to_vec()),
175 );179 );
176 let mut server = Server::new(SOURCE);180 let mut server = Server::new(SOURCE);
177 PreparedEdit::pages(181 ops::section_op(SOURCE, SectionOp::Pages([PageEdit::move_to(sid, Some(before[7].0), 2).unwrap()].to_vec()))
178 SOURCE,
179 &[PageEdit::move_to(sid, Some(before[7].0), 2).unwrap()],
180 )
181 .unwrap()182 .unwrap()
182 .commit(&mut server)183 .commit(&mut server)
183 .unwrap();184 .unwrap();
184 let mut expected = order(&server.durable);185 let expected = order(&server.durable);
185 expected.iter_mut().find(|p| p.0 == sid).unwrap().1 = 3;
186 assert!(matches!(cache.sync_once(&mut server).unwrap().edit,186 assert!(matches!(cache.sync_once(&mut server).unwrap().edit,
187 Some((published, EditStatus::Published { .. })) if published == id));187 Some((published, EditStatus::Published { .. })) if published == id));
188 assert_eq!(order(&server.durable), expected);188 assert_eq!(order(&server.durable), expected);
189}189}
190190
191/// A page both sides moved stays where the remote put it, as OneNote 2010 keeps it
192/// (corpus/conflict-page/native-pages).
191#[test]193#[test]
192fn competing_page_moves_conflict_until_mine_is_kept_with_its_identities() {194fn competing_page_moves_keep_the_remote_placement() {
193 let directory = tempfile::tempdir().unwrap();195 let directory = tempfile::tempdir().unwrap();
194 let cache = Replica::create(directory.path().join("pages.sqlite"), SOURCE).unwrap();196 let cache = Replica::create(directory.path().join("pages.sqlite"), SOURCE).unwrap();
195 let pages = order(SOURCE);197 let pages = order(SOURCE);
196 let sid = pages[6].0;198 let sid = pages[6].0;
197 let original = PageEdit::move_to(sid, None, 1).unwrap();199 let id = section_op(&cache, SectionOp::Pages(vec![PageEdit::move_to(sid, None, 1).unwrap()]));
198 let id = section_op(&cache, SectionOp::Pages(vec![original.clone()]));
199 let mut server = Server::new(SOURCE);200 let mut server = Server::new(SOURCE);
200 PreparedEdit::pages(201 ops::section_op(SOURCE, SectionOp::Pages([PageEdit::move_to(sid, Some(pages[0].0), 1).unwrap()].to_vec()))
201 SOURCE,
202 &[PageEdit::move_to(sid, Some(pages[0].0), 1).unwrap()],
203 )
204 .unwrap()202 .unwrap()
205 .commit(&mut server)203 .commit(&mut server)
206 .unwrap();204 .unwrap();
207 let remote = server.visible.clone();205 let expected = order(&server.visible);
208 let local = cache.snapshot().unwrap();
209 assert_eq!(
210 cache.sync_once(&mut server).unwrap().edit,
211 Some((id, EditStatus::Conflict(ConflictKind::StructureChanged)))
212 );
213 assert_eq!(server.publications, 0);
214 assert_eq!(
215 server::pages(&cache.snapshot().unwrap()),
216 server::pages(&local)
217 );
218 // Keeping mine moves the page last again, from where the remote put it.
219 cache.resolve(id, Resolution::Mine).unwrap();
220 assert!(matches!(&cache.pending().unwrap()[0].edit.ops[..],
221 [Op::Section(SectionOp::Pages(batch))] if *batch == [original.clone()]));
222 let expected = PreparedEdit::pages(&remote, &[original]).unwrap();
223 assert!(matches!(cache.sync_once(&mut server).unwrap().edit,206 assert!(matches!(cache.sync_once(&mut server).unwrap().edit,
224 Some((published, EditStatus::Published { .. })) if published == id));207 Some((published, EditStatus::Published { .. })) if published == id));
225 assert_eq!(order(&server.durable), order(expected.as_bytes()));208 assert_eq!(order(&server.durable), expected);
226}209}
227210
228#[test]211#[test]
229fn one_competing_level_prevents_publication_of_the_whole_batch() {212fn a_competing_level_keeps_the_remote_level_and_the_rest_of_the_batch() {
230 let directory = tempfile::tempdir().unwrap();213 let directory = tempfile::tempdir().unwrap();
231 let cache = Replica::create(directory.path().join("pages.sqlite"), SOURCE).unwrap();214 let cache = Replica::create(directory.path().join("pages.sqlite"), SOURCE).unwrap();
232 let pages = order(SOURCE);215 let pages = order(SOURCE);
233 let id = section_op(216 let edits = vec![
234 &cache,217 PageEdit::set_level(pages[4].0, 3).unwrap(),
235 SectionOp::Pages(vec![218 PageEdit::set_level(pages[5].0, 2).unwrap(),
236 PageEdit::set_level(pages[4].0, 3).unwrap(),219 ];
237 PageEdit::set_level(pages[5].0, 2).unwrap(),220 let id = section_op(&cache, SectionOp::Pages(edits.clone()));
238 ]),
239 );
240 let local = cache.snapshot().unwrap();
241 let mut server = Server::new(SOURCE);221 let mut server = Server::new(SOURCE);
242 PreparedEdit::pages(SOURCE, &[PageEdit::set_level(pages[5].0, 1).unwrap()])222 ops::section_op(SOURCE, SectionOp::Pages([PageEdit::set_level(pages[5].0, 1).unwrap()].to_vec()))
243 .unwrap()223 .unwrap()
244 .commit(&mut server)224 .commit(&mut server)
245 .unwrap();225 .unwrap();
246 let remote = server.durable.clone();226 let expected =
247 assert_eq!(227 ops::section_op(&server.durable.clone(), SectionOp::Pages(edits[..1].to_vec())).unwrap();
248 cache.sync_once(&mut server).unwrap().edit,228 assert!(matches!(cache.sync_once(&mut server).unwrap().edit,
249 Some((id, EditStatus::Conflict(ConflictKind::StructureChanged)))229 Some((published, EditStatus::Published { .. })) if published == id));
250 );230 assert_eq!(order(&server.durable), order(expected.as_bytes()));
251 assert_eq!(server.publications, 0);
252 assert_eq!(server.durable, remote);
253 assert_eq!(
254 server::pages(&cache.snapshot().unwrap()),
255 server::pages(&local)
256 );
257}231}
258232
259#[test]233#[test]
...@@ -273,7 +247,7 @@ fn uncertain_page_batches_survive_recovery_and_do_not_replay() {...@@ -273,7 +247,7 @@ fn uncertain_page_batches_survive_recovery_and_do_not_replay() {
273 PageEdit::set_level(pages[5].0, 2).unwrap(),247 PageEdit::set_level(pages[5].0, 2).unwrap(),
274 ];248 ];
275 let id = section_op(&cache, SectionOp::Pages(edits.to_vec()));249 let id = section_op(&cache, SectionOp::Pages(edits.to_vec()));
276 let local = cache.snapshot().unwrap();250 let local = snapshot(&cache);
277 let queue = cache.pending().unwrap();251 let queue = cache.pending().unwrap();
278 let mut server = Server::new(SOURCE);252 let mut server = Server::new(SOURCE);
279 server.fault = fault;253 server.fault = fault;
...@@ -293,7 +267,6 @@ fn uncertain_page_batches_survive_recovery_and_do_not_replay() {...@@ -293,7 +267,6 @@ fn uncertain_page_batches_survive_recovery_and_do_not_replay() {
293 server::pages(&recovery.snapshot().unwrap()),267 server::pages(&recovery.snapshot().unwrap()),
294 server::pages(&local)268 server::pages(&local)
295 );269 );
296 assert!(cache.resolve(id, Resolution::Mine).is_err());
297 let result = cache.sync_once(&mut server).unwrap().edit.unwrap();270 let result = cache.sync_once(&mut server).unwrap().edit.unwrap();
298 assert_eq!(server.publications, 1);271 assert_eq!(server.publications, 1);
299 if matches!(fault, Fault::UnknownAfter | Fault::PanicAfter) {272 if matches!(fault, Fault::UnknownAfter | Fault::PanicAfter) {
...@@ -319,9 +292,23 @@ fn an_unchanged_section_revision_cannot_confirm_a_changed_page() {...@@ -319,9 +292,23 @@ fn an_unchanged_section_revision_cannot_confirm_a_changed_page() {
319 let copy = *view.nodes.iter().find(|(_, node)|292 let copy = *view.nodes.iter().find(|(_, node)|
320 matches!(&node.kind, Kind::Metadata { title: Some(title), .. } if title == "child"))293 matches!(&node.kind, Kind::Metadata { title: Some(title), .. } if title == "child"))
321 .unwrap().0;294 .unwrap().0;
322 let source =295 // The section's copy of that page's metadata says level 3, stored where it lies.
323 onestore::replace_property_bytes(SOURCE, root, copy, 0x14001dff, &3_u32.to_le_bytes())296 let revision = index.resolve_active(root).unwrap();
324 .unwrap();297 let onestore::ObjectData::Properties(data) = revision.objects[&copy].data else {
298 panic!()
299 };
300 let properties = onestore::PropertySets::parse(data).unwrap();
301 let onestore::Value::Bytes(level) = properties.sets[0]
302 .iter()
303 .find(|property| property.id == 0x14001dff)
304 .unwrap()
305 .value
306 else {
307 panic!()
308 };
309 let at = level.as_ptr().addr() - SOURCE.as_ptr().addr();
310 let mut source = SOURCE.to_vec();
311 source[at..at + 4].copy_from_slice(&3_u32.to_le_bytes());
325 let directory = tempfile::tempdir().unwrap();312 let directory = tempfile::tempdir().unwrap();
326 let path = directory.path().join("pages.sqlite");313 let path = directory.path().join("pages.sqlite");
327 let cache = Replica::create(&path, &source).unwrap();314 let cache = Replica::create(&path, &source).unwrap();
...@@ -368,7 +355,7 @@ fn an_unchanged_section_revision_cannot_confirm_a_changed_page() {...@@ -368,7 +355,7 @@ fn an_unchanged_section_revision_cannot_confirm_a_changed_page() {
368}355}
369356
370#[test]357#[test]
371fn an_explicit_anchor_is_retained_even_when_originally_in_place() {358fn an_explicit_anchor_yields_to_a_remote_move() {
372 let directory = tempfile::tempdir().unwrap();359 let directory = tempfile::tempdir().unwrap();
373 let cache = Replica::create(directory.path().join("pages.sqlite"), SOURCE).unwrap();360 let cache = Replica::create(directory.path().join("pages.sqlite"), SOURCE).unwrap();
374 let pages = order(SOURCE);361 let pages = order(SOURCE);
...@@ -380,18 +367,18 @@ fn an_explicit_anchor_is_retained_even_when_originally_in_place() {...@@ -380,18 +367,18 @@ fn an_explicit_anchor_is_retained_even_when_originally_in_place() {
380 ]),367 ]),
381 );368 );
382 let mut server = Server::new(SOURCE);369 let mut server = Server::new(SOURCE);
383 PreparedEdit::pages(370 ops::section_op(SOURCE, SectionOp::Pages([PageEdit::move_to(pages[6].0, Some(pages[0].0), 1).unwrap()].to_vec()))
384 SOURCE,
385 &[PageEdit::move_to(pages[6].0, Some(pages[0].0), 1).unwrap()],
386 )
387 .unwrap()371 .unwrap()
388 .commit(&mut server)372 .commit(&mut server)
389 .unwrap();373 .unwrap();
390 assert_eq!(374 let expected = ops::section_op(
391 cache.sync_once(&mut server).unwrap().edit,375 &server.durable.clone(),
392 Some((id, EditStatus::Conflict(ConflictKind::StructureChanged)))376 SectionOp::Pages(vec![PageEdit::set_level(pages[4].0, 3).unwrap()]),
393 );377 )
394 assert_eq!(server.publications, 0);378 .unwrap();
379 assert!(matches!(cache.sync_once(&mut server).unwrap().edit,
380 Some((published, EditStatus::Published { .. })) if published == id));
381 assert_eq!(order(&server.durable), order(expected.as_bytes()));
395}382}
396383
397#[test]384#[test]
...@@ -404,7 +391,7 @@ fn convergent_page_indentation_requires_confirmation_without_republishing() {...@@ -404,7 +391,7 @@ fn convergent_page_indentation_requires_confirmation_without_republishing() {
404 SectionOp::Pages([PageEdit::set_level(sid, 1).unwrap()].to_vec()),391 SectionOp::Pages([PageEdit::set_level(sid, 1).unwrap()].to_vec()),
405 );392 );
406 let mut server = Server::new(SOURCE);393 let mut server = Server::new(SOURCE);
407 PreparedEdit::pages(SOURCE, &[PageEdit::set_level(sid, 1).unwrap()])394 ops::section_op(SOURCE, SectionOp::Pages([PageEdit::set_level(sid, 1).unwrap()].to_vec()))
408 .unwrap()395 .unwrap()
409 .commit(&mut server)396 .commit(&mut server)
410 .unwrap();397 .unwrap();
...@@ -418,7 +405,7 @@ fn convergent_page_indentation_requires_confirmation_without_republishing() {...@@ -418,7 +405,7 @@ fn convergent_page_indentation_requires_confirmation_without_republishing() {
418 assert_eq!(server.confirmations, 2);405 assert_eq!(server.confirmations, 2);
419}406}
420#[test]407#[test]
421fn native_page_changes_reconcile_with_atomic_offline_batches_and_review() {408fn native_page_changes_reconcile_with_atomic_offline_batches() {
422 let root = std::path::Path::new(env!("CARGO_MANIFEST_DIR"))409 let root = std::path::Path::new(env!("CARGO_MANIFEST_DIR"))
423 .join("../../corpus/page-lifecycle/movement");410 .join("../../corpus/page-lifecycle/movement");
424 let provenance: serde_json::Value =411 let provenance: serde_json::Value =
...@@ -455,48 +442,36 @@ fn native_page_changes_reconcile_with_atomic_offline_batches_and_review() {...@@ -455,48 +442,36 @@ fn native_page_changes_reconcile_with_atomic_offline_batches_and_review() {
455 .unwrap();442 .unwrap();
456 let native = std::fs::read(root.join(phase).join("notebook/Lifecycle.one")).unwrap();443 let native = std::fs::read(root.join(phase).join("notebook/Lifecycle.one")).unwrap();
457 let mut server = Server::new(&native);444 let mut server = Server::new(&native);
458 let local = cache.snapshot().unwrap();445 // The page list takes the edits of the pages OneNote left in their series, at
459 let expected_conflict = if mode == "indent" {446 // their level and after the same page.
460 matches!(447 let both: Vec<ExGuid> = order(&native)
461 phase.as_str(),448 .into_iter()
462 "04-subpage-move" | "07-promoted-root" | "08-collapsed-group-move"449 .map(|(sid, _)| sid)
463 )450 .filter(|sid| original.iter().any(|(s, _)| s == sid))
464 } else {451 .collect();
465 !matches!(452 let placement = |image: &[u8]| -> BTreeMap<ExGuid, (ExGuid, Option<ExGuid>, u32)> {
466 phase.as_str(),453 let arena = onestore::Arena::default();
467 "02-promoted-parent" | "03-selected-group-move" | "06-promoted-level-two"454 let series = onestore::Section::open(&arena, image.to_vec()).unwrap().series().unwrap();
468 )455 let shared: Vec<_> = order(image).into_iter().filter(|(sid, _)| both.contains(sid)).collect();
469 };456 (0..shared.len())
470 let first = if expected_conflict {457 .map(|at| {
471 cache.sync_once(&mut server).unwrap().edit.unwrap()458 let (sid, level) = shared[at];
472 } else {459 (sid, (series[&sid], at.checked_sub(1).map(|b| shared[b].0), level))
473 (text_id, EditStatus::Pending)460 })
461 .collect()
474 };462 };
475 let mut reviewed = edits.clone();463 let (before, after) = (placement(SOURCE), placement(&native));
476 if expected_conflict {464 let reviewed: Vec<PageEdit> = edits
477 assert_eq!(465 .iter()
478 first.1,466 .filter(|edit| after.get(&edit.space()) == before.get(&edit.space()))
479 EditStatus::Conflict(ConflictKind::StructureChanged),467 .cloned()
480 "{mode}:{phase}"468 .collect();
481 );469 let trimmed = reviewed != edits;
482 assert_eq!(server.publications, 0);
483 assert_eq!(
484 server::pages(&cache.snapshot().unwrap()),
485 server::pages(&local)
486 );
487 // Keeping mine drops the moves that no longer apply.
488 cache.resolve(id, Resolution::Mine).unwrap();
489 reviewed = match &cache.pending().unwrap()[0].edit.ops[..] {
490 [Op::Section(SectionOp::Pages(kept))] => kept.clone(),
491 [] => Vec::new(),
492 other => panic!("{other:?}"),
493 };
494 }
495 assert!(470 assert!(
496 matches!(cache.sync_once(&mut server).unwrap().edit, Some((published, EditStatus::Published { .. })) if published == text_id),471 matches!(cache.sync_once(&mut server).unwrap().edit, Some((published, EditStatus::Published { .. })) if published == text_id),
497 "{mode}:{phase}"472 "{mode}:{phase}"
498 );473 );
499 counts[usize::from(expected_conflict)] += 1;474 counts[usize::from(trimmed)] += 1;
500 drop(cache);475 drop(cache);
501 let cache = Replica::open(&path).unwrap();476 let cache = Replica::open(&path).unwrap();
502 assert!(cache.pending().unwrap().is_empty());477 assert!(cache.pending().unwrap().is_empty());
...@@ -525,7 +500,7 @@ fn native_page_changes_reconcile_with_atomic_offline_batches_and_review() {...@@ -525,7 +500,7 @@ fn native_page_changes_reconcile_with_atomic_offline_batches_and_review() {
525 path.join("manifest.json"),500 path.join("manifest.json"),
526 serde_json::to_vec_pretty(501 serde_json::to_vec_pretty(
527 &serde_json::json!({"mode": mode, "phase": phase, "original": edits,502 &serde_json::json!({"mode": mode, "phase": phase, "original": edits,
528 "reviewed": reviewed, "conflict": expected_conflict,503 "reviewed": reviewed, "conflict": trimmed,
529 "publications": server.publications, "confirmations": server.confirmations,504 "publications": server.publications, "confirmations": server.confirmations,
530 "page_receipt": format!("{:?}", cache.status(id).unwrap()),505 "page_receipt": format!("{:?}", cache.status(id).unwrap()),
531 "text_receipt": format!("{:?}", cache.status(text_id).unwrap())}),506 "text_receipt": format!("{:?}", cache.status(text_id).unwrap())}),
...@@ -536,7 +511,7 @@ fn native_page_changes_reconcile_with_atomic_offline_batches_and_review() {...@@ -536,7 +511,7 @@ fn native_page_changes_reconcile_with_atomic_offline_batches_and_review() {
536 }511 }
537 }512 }
538 }513 }
539 assert_eq!(counts, [9, 9]);514 assert_eq!(counts.iter().sum::<usize>(), 18);
540}515}
541516
542#[test]517#[test]
...@@ -545,7 +520,7 @@ fn twelve_disjoint_page_batches_merge_with_dependent_bodies_without_review() {...@@ -545,7 +520,7 @@ fn twelve_disjoint_page_batches_merge_with_dependent_bodies_without_review() {
545 let mut created = Vec::new();520 let mut created = Vec::new();
546 for _ in 0..36 {521 for _ in 0..36 {
547 let page = onestore::PageCreation::new(None, Some("Same title"), "Author").unwrap();522 let page = onestore::PageCreation::new(None, Some("Same title"), "Author").unwrap();
548 source = PreparedEdit::create_page(&source, &page)523 source = ops::section_op(&source, SectionOp::Create(page.clone()))
549 .unwrap()524 .unwrap()
550 .as_bytes()525 .as_bytes()
551 .to_vec();526 .to_vec();
...@@ -564,7 +539,7 @@ fn twelve_disjoint_page_batches_merge_with_dependent_bodies_without_review() {...@@ -564,7 +539,7 @@ fn twelve_disjoint_page_batches_merge_with_dependent_bodies_without_review() {
564 ];539 ];
565 section_op(&cache, SectionOp::Pages(edits.to_vec()));540 section_op(&cache, SectionOp::Pages(edits.to_vec()));
566 let body = format!("Actor {actor} 🦀 e\u{301}");541 let body = format!("Actor {actor} 🦀 e\u{301}");
567 let local = cache.snapshot().unwrap();542 let local = snapshot(&cache);
568 let mut model = model_ops::page_of(&local, group[1].space());543 let mut model = model_ops::page_of(&local, group[1].space());
569 let text = body_outline(&mut model, &body);544 let text = body_outline(&mut model, &body);
570 model_ops::save_as(&cache, group[1].space(), &model, "Author")545 model_ops::save_as(&cache, group[1].space(), &model, "Author")
...@@ -594,7 +569,7 @@ fn twelve_disjoint_page_batches_merge_with_dependent_bodies_without_review() {...@@ -594,7 +569,7 @@ fn twelve_disjoint_page_batches_merge_with_dependent_bodies_without_review() {
594 for (path, queue) in &queues {569 for (path, queue) in &queues {
595 let cache = Replica::open(path).unwrap();570 let cache = Replica::open(path).unwrap();
596 assert_eq!(cache.sync_once(&mut server).unwrap().edit, None);571 assert_eq!(cache.sync_once(&mut server).unwrap().edit, None);
597 assert_eq!(order(&cache.snapshot().unwrap()), expected);572 assert_eq!(order(&snapshot(&cache)), expected);
598 for edit in queue {573 for edit in queue {
599 assert!(matches!(574 assert!(matches!(
600 cache.status(edit.id).unwrap(),575 cache.status(edit.id).unwrap(),
...@@ -627,3 +602,77 @@ fn twelve_disjoint_page_batches_merge_with_dependent_bodies_without_review() {...@@ -627,3 +602,77 @@ fn twelve_disjoint_page_batches_merge_with_dependent_bodies_without_review() {
627 .unwrap();602 .unwrap();
628 }603 }
629}604}
605
606/// Two OneNote 2010 clients' page merges (`corpus/conflict-page/native-*`), replayed: client
607/// A, offline, `moves` pages (each last, or before another) and edits `Target`'s body while
608/// client B publishes `b-published`. The pages the merge lists after the first, and
609/// `Target`'s space and texts.
610fn native_page_merge(capture: &str, moves: &[(&str, Option<&str>)]) -> (Vec<String>, ExGuid, Vec<String>) {
611 let root = std::path::Path::new(env!("CARGO_MANIFEST_DIR"))
612 .join("../../corpus/conflict-page")
613 .join(capture);
614 let read = |phase: &str| std::fs::read(root.join(phase).join("notebook/synthetic.one")).unwrap();
615 let pages = |image: &[u8]| {
616 let arena = onestore::Arena::default();
617 onestore::Section::open(&arena, image.to_vec()).unwrap().pages().unwrap()
618 };
619 let initial = read("initial");
620 let listed = pages(&initial);
621 let space = |title: &str| listed.iter().find(|page| page.1 == title).unwrap().0;
622 let directory = tempfile::tempdir().unwrap();
623 let cache = Replica::create(directory.path().join("pages.sqlite"), &initial).unwrap();
624 for (page, before) in moves {
625 section_op(
626 &cache,
627 SectionOp::Pages(vec![PageEdit::move_to(space(page), before.map(space), 1).unwrap()]),
628 );
629 }
630 let body = texts(&initial)
631 .into_iter()
632 .find(|(_, text)| text == "Body Target.")
633 .unwrap()
634 .0
635 .1;
636 model_ops::save(&cache, body, |page| {
637 model_ops::replace_text(page, body, 0..12, "Body Target edited offline.")
638 })
639 .unwrap()
640 .unwrap();
641 let mut server = Server::new(&read("b-published"));
642 for _ in 0..moves.len() + 2 {
643 cache.sync_once(&mut server).unwrap();
644 }
645 assert!(cache.pending().unwrap().is_empty(), "{capture}");
646 let merged = pages(&server.durable);
647 let target = merged.iter().find(|page| page.1 == "Target").unwrap().0;
648 let texts = texts(&server.durable)
649 .into_iter()
650 .filter(|((page, _), _)| *page == target)
651 .map(|(_, text)| text)
652 .collect();
653 (merged[1..].iter().map(|page| page.1.clone()).collect(), target, texts)
654}
655
656/// Each side keeps the pages it gave a series of their own, the remote's where both did;
657/// a page the remote deleted comes back as a new page where the queue had it. OneNote's
658/// COM moves re-series the pages a page jumps over, so its moves replay as moves of those.
659#[test]
660fn page_moves_and_a_removed_page_merge_as_onenote_merges_them() {
661 for (capture, moves) in [
662 ("native-pages", &[("One", Some("Target")), ("Three", Some("Target")), ("Two", None)][..]),
663 ("native-restore", &[("One", None), ("Two", None), ("Target", None), ("Three", None)][..]),
664 ("native-restore", &[("Four", Some("One"))][..]),
665 ] {
666 let root = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../../corpus/conflict-page");
667 let native: serde_json::Value =
668 serde_json::from_slice(&std::fs::read(root.join(capture).join("merged.json")).unwrap()).unwrap();
669 let (order, target, texts) = native_page_merge(capture, moves);
670 let native: Vec<String> = serde_json::from_value(native["order"].clone()).unwrap();
671 assert_eq!(order, native[1..], "{capture}");
672 assert!(texts.contains(&"Body Target edited offline.".to_owned()), "{capture}");
673 let arena = onestore::Arena::default();
674 let initial = std::fs::read(root.join(capture).join("initial/notebook/synthetic.one")).unwrap();
675 let listed = onestore::Section::open(&arena, initial).unwrap().pages().unwrap();
676 assert!(listed.iter().all(|page| page.0 != target), "{capture}: a new page");
677 }
678}
crates/notebook/tests/sync_paragraph.rs+89-69
...@@ -1,9 +1,12 @@...@@ -1,9 +1,12 @@
1//! Paragraph splits and joins expressed as page-model differences: reconciliation against1//! Paragraph splits and joins expressed as page-model differences: reconciliation against
2//! native OneNote captures, dependent edits, uncertain attempts and competing clients.2//! native OneNote captures, dependent edits, uncertain attempts and competing clients.
33
4use notebook::{ConflictKind, EditStatus, Replica};4#[path = "../../onestore/tests/support/ops.rs"]
5mod ops;
6
7use notebook::{EditStatus, Replica};
5use onestore::{8use onestore::{
6 ExGuid, PreparedEdit, RevisionIndex, Store,9 ExGuid, RevisionIndex, Store,
7 document::Document,10 document::Document,
8 page::{Page, PageObject, PageParagraph, ParagraphContent, TextObject, text::new_id},11 page::{Page, PageObject, PageParagraph, ParagraphContent, TextObject, text::new_id},
9};12};
...@@ -11,7 +14,7 @@ use std::path::Path;...@@ -11,7 +14,7 @@ use std::path::Path;
1114
12#[path = "support/server.rs"]15#[path = "support/server.rs"]
13mod server;16mod server;
14use server::{Fault, Server, pages};17use server::{Fault, Server, conflicts, pages, snapshot, typed};
15#[path = "support/model_ops.rs"]18#[path = "support/model_ops.rs"]
16mod model_ops;19mod model_ops;
17use model_ops::{AUTHOR, outlines_mut, page_of, replace_text};20use model_ops::{AUTHOR, outlines_mut, page_of, replace_text};
...@@ -175,18 +178,27 @@ fn save_page(...@@ -175,18 +178,27 @@ fn save_page(
175 text: ExGuid,178 text: ExGuid,
176 edit: impl FnOnce(&mut Page),179 edit: impl FnOnce(&mut Page),
177) -> Result<Option<u64>, notebook::Error> {180) -> Result<Option<u64>, notebook::Error> {
178 let source = cache.snapshot()?;181 let source = snapshot(cache);
179 let (space, mut page) = holder(&source, text);182 let (space, mut page) = holder(&source, text);
180 edit(&mut page);183 edit(&mut page);
181 model_ops::save_as(cache, space, &page, AUTHOR)184 model_ops::save_as(cache, space, &page, AUTHOR)
182}185}
183186
184/// Resolves a conflict by taking the remote page, then saves `edit` of it.187/// Makes `edit` again on the page as the remote's version left it, as a writer does after
185fn review_page(cache: &Replica, id: u64, text: ExGuid, edit: impl FnOnce(&mut Page)) {188/// a conflict page kept the first try.
186 cache.resolve(id, notebook::Resolution::Theirs).unwrap();189fn redo_page(cache: &Replica, text: ExGuid, edit: impl FnOnce(&mut Page)) {
187 save_page(cache, text, edit).unwrap().unwrap();190 save_page(cache, text, edit).unwrap().unwrap();
188}191}
189192
193/// The conflict pages `image` lists under page `space`.
194fn conflict_pages(image: &[u8], space: ExGuid) -> usize {
195 conflicts(image)
196 .into_iter()
197 .filter(|(page, _)| *page == space)
198 .map(|(_, pages)| pages.len())
199 .sum()
200}
201
190/// The styled text of an object anywhere on a page.202/// The styled text of an object anywhere on a page.
191fn content(page: &Page, text: ExGuid) -> onestore::page::Paragraph {203fn content(page: &Page, text: ExGuid) -> onestore::page::Paragraph {
192 locate_in(&mut page.clone(), text, |list, index| {204 locate_in(&mut page.clone(), text, |list, index| {
...@@ -250,12 +262,15 @@ fn reconcile_controls() -> (Vec<u8>, Vec<serde_json::Value>) {...@@ -250,12 +262,15 @@ fn reconcile_controls() -> (Vec<u8>, Vec<serde_json::Value>) {
250 drop(cache);262 drop(cache);
251 cache = Replica::open(&path).unwrap();263 cache = Replica::open(&path).unwrap();
252 assert_eq!(cache.pending().unwrap(), pending);264 assert_eq!(cache.pending().unwrap(), pending);
253 let (actual, status) = cache.sync_once(&mut server).unwrap().edit.unwrap();265 let kept = conflict_pages(&server.visible, space);
254 assert_eq!(actual, id);266 let (_, status) = cache.sync_once(&mut server).unwrap().edit.unwrap();
255 let reviewed = matches!(status, EditStatus::Conflict(_));267 assert!(matches!(
268 cache.status(id).unwrap(),
269 Some(EditStatus::Published { .. })
270 ));
271 let reviewed = conflict_pages(&server.visible, space) > kept;
256 let receipt = if reviewed {272 let receipt = if reviewed {
257 assert_eq!(cache.pending().unwrap(), pending);273 redo_page(&cache, left, apply);
258 review_page(&cache, id, left, apply);
259 cache.sync_once(&mut server).unwrap().edit.unwrap().1274 cache.sync_once(&mut server).unwrap().edit.unwrap().1
260 } else {275 } else {
261 automatic += 1;276 automatic += 1;
...@@ -301,7 +316,7 @@ fn reconcile_controls() -> (Vec<u8>, Vec<serde_json::Value>) {...@@ -301,7 +316,7 @@ fn reconcile_controls() -> (Vec<u8>, Vec<serde_json::Value>) {
301 panic!("{name} dependent: {status:?}")316 panic!("{name} dependent: {status:?}")
302 };317 };
303 assert!(cache.pending().unwrap().is_empty());318 assert!(cache.pending().unwrap().is_empty());
304 assert_eq!(cache.snapshot().unwrap(), server.durable);319 assert_eq!(snapshot(&cache), server.durable);
305 recorded.push(320 recorded.push(
306 serde_json::json!({"case": name, "space": space, "intent": intent,321 serde_json::json!({"case": name, "space": space, "intent": intent,
307 "outcome": if reviewed { "reviewed" } else { "automatic" },322 "outcome": if reviewed { "reviewed" } else { "automatic" },
...@@ -309,12 +324,13 @@ fn reconcile_controls() -> (Vec<u8>, Vec<serde_json::Value>) {...@@ -309,12 +324,13 @@ fn reconcile_controls() -> (Vec<u8>, Vec<serde_json::Value>) {
309 );324 );
310 }325 }
311 assert_eq!((recorded.len(), automatic), (16, 8));326 assert_eq!((recorded.len(), automatic), (16, 8));
312 assert_eq!(server.publications, 32);327 // Each edit and its dependent, and each conflict page before its edit made again.
328 assert_eq!(server.publications, 32 + 16 - automatic);
313 (server.durable, recorded)329 (server.durable, recorded)
314}330}
315331
316#[test]332#[test]
317fn native_keyboard_edits_merge_with_offline_splits_and_joins_or_survive_review() {333fn native_keyboard_edits_merge_with_offline_splits_and_joins_or_keep_a_conflict_page() {
318 let (durable, cases) = reconcile_controls();334 let (durable, cases) = reconcile_controls();
319 let expected = [335 let expected = [
320 ("Split remote prefix", "automatic", vec!["Xab", "🦀Cd"]),336 ("Split remote prefix", "automatic", vec!["Xab", "🦀Cd"]),
...@@ -349,8 +365,9 @@ fn native_keyboard_edits_merge_with_offline_splits_and_joins_or_survive_review()...@@ -349,8 +365,9 @@ fn native_keyboard_edits_merge_with_offline_splits_and_joins_or_survive_review()
349}365}
350366
351#[test]367#[test]
352fn a_native_text_object_replacement_conflicts_with_every_offline_split_and_join() {368fn a_native_text_object_replacement_keeps_a_conflict_page_for_every_offline_split_and_join() {
353 let mut server = Server::new(REPLACED);369 let mut server = Server::new(REPLACED);
370 let mut count = 0;
354 for (space, name, split) in controls() {371 for (space, name, split) in controls() {
355 let original = page_of(BEFORE, space);372 let original = page_of(BEFORE, space);
356 let left = body(&original)[0].text().unwrap().id;373 let left = body(&original)[0].text().unwrap().id;
...@@ -366,24 +383,23 @@ fn a_native_text_object_replacement_conflicts_with_every_offline_split_and_join(...@@ -366,24 +383,23 @@ fn a_native_text_object_replacement_conflicts_with_every_offline_split_and_join(
366 let id = model_ops::save_as(&cache, space, &after, AUTHOR)383 let id = model_ops::save_as(&cache, space, &after, AUTHOR)
367 .unwrap()384 .unwrap()
368 .unwrap();385 .unwrap();
369 let local = cache.snapshot().unwrap();386 let remote = page_of(&server.visible, space);
370 let pending = cache.pending().unwrap();387 assert!(
371 assert_eq!(388 matches!(
372 cache.sync_once(&mut server).unwrap().edit,389 cache.sync_once(&mut server).unwrap().edit,
373 Some((id, EditStatus::Conflict(ConflictKind::ContentChanged))),390 Some((_, EditStatus::Published { .. }))
391 ),
374 "{name}"392 "{name}"
375 );393 );
376 drop(cache);394 assert!(
377 let cache = Replica::open(&path).unwrap();395 matches!(cache.status(id).unwrap(), Some(EditStatus::Published { .. })),
378 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));396 "{name}"
379 assert_eq!(cache.pending().unwrap(), pending);
380 assert_eq!(
381 cache.status(id).unwrap(),
382 Some(EditStatus::Conflict(ConflictKind::ContentChanged))
383 );397 );
398 assert_eq!(page_of(&server.visible, space), remote, "{name}");
399 assert_eq!(conflict_pages(&server.visible, space), 1, "{name}");
400 count += 1;
384 }401 }
385 assert_eq!(server.publications, 0);402 assert_eq!(server.publications, count);
386 assert_eq!(server.visible, REPLACED);
387}403}
388404
389/// Replays a recorded native split or join intent as a page-model difference.405/// Replays a recorded native split or join intent as a page-model difference.
...@@ -459,7 +475,7 @@ fn offline_paragraphs(output: Option<&Path>) {...@@ -459,7 +475,7 @@ fn offline_paragraphs(output: Option<&Path>) {
459 {475 {
460 let intent = &case["intent"];476 let intent = &case["intent"];
461 let left = origin(intent, split);477 let left = origin(intent, split);
462 let (space, page) = holder(&cache.snapshot().unwrap(), left);478 let (space, page) = holder(&snapshot(&cache), left);
463 let offset = if split {479 let offset = if split {
464 intent["offset"].as_u64().unwrap()480 intent["offset"].as_u64().unwrap()
465 } else {481 } else {
...@@ -468,7 +484,7 @@ fn offline_paragraphs(output: Option<&Path>) {...@@ -468,7 +484,7 @@ fn offline_paragraphs(output: Option<&Path>) {
468 let prefixed = offset > 0 && !content(&page, left).text().is_empty();484 let prefixed = offset > 0 && !content(&page, left).text().is_empty();
469 if prefixed {485 if prefixed {
470 server.visible =486 server.visible =
471 onestore::replace_text(&server.visible, space, left, 0..0, "\u{2602}").unwrap();487 typed(&server.visible, space, left, 0..0, "\u{2602}");
472 server.durable.clone_from(&server.visible);488 server.durable.clone_from(&server.visible);
473 }489 }
474 let Ok(queued) = save_page(&cache, left, |page| replay(page, intent, split)) else {490 let Ok(queued) = save_page(&cache, left, |page| replay(page, intent, split)) else {
...@@ -484,6 +500,8 @@ fn offline_paragraphs(output: Option<&Path>) {...@@ -484,6 +500,8 @@ fn offline_paragraphs(output: Option<&Path>) {
484 let case = entry["case"].as_str().unwrap().to_owned();500 let case = entry["case"].as_str().unwrap().to_owned();
485 let intent = entry["intent"].clone();501 let intent = entry["intent"].clone();
486 let mut uncertain = id % 2 == 0;502 let mut uncertain = id % 2 == 0;
503 let mut kept = conflict_pages(&server.visible, space);
504 let mut id = id;
487 let revision = loop {505 let revision = loop {
488 if uncertain {506 if uncertain {
489 server.fault = Fault::UnknownAfter;507 server.fault = Fault::UnknownAfter;
...@@ -501,7 +519,7 @@ fn offline_paragraphs(output: Option<&Path>) {...@@ -501,7 +519,7 @@ fn offline_paragraphs(output: Option<&Path>) {
501 Ok(notebook::Synced {519 Ok(notebook::Synced {
502 edit: Some((_, EditStatus::Published { revision })),520 edit: Some((_, EditStatus::Published { revision })),
503 ..521 ..
504 }) => {522 }) if conflict_pages(&server.visible, space) == kept => {
505 assert!(matches!(523 assert!(matches!(
506 cache.status(id).unwrap(),524 cache.status(id).unwrap(),
507 Some(EditStatus::Published { .. })525 Some(EditStatus::Published { .. })
...@@ -509,20 +527,21 @@ fn offline_paragraphs(output: Option<&Path>) {...@@ -509,20 +527,21 @@ fn offline_paragraphs(output: Option<&Path>) {
509 break revision;527 break revision;
510 }528 }
511 Ok(notebook::Synced {529 Ok(notebook::Synced {
512 edit: Some((actual, EditStatus::Conflict(_))),530 edit: Some((_, EditStatus::Published { .. })),
513 ..531 ..
514 }) => {532 }) => {
515 assert_eq!(actual, id);533 kept += 1;
516 server.fault = Fault::None;534 server.fault = Fault::None;
517 reviewed.push(case.clone());535 reviewed.push(case.clone());
518 // The reviewed split sits after the remote's one-unit prefix.536 // The split made again sits after the remote's one-unit prefix.
519 let mut placed = intent.clone();537 let mut placed = intent.clone();
520 if split && entry["remote_prefix"] == true {538 if split && entry["remote_prefix"] == true {
521 placed["offset"] = (intent["offset"].as_u64().unwrap() + 1).into();539 placed["offset"] = (intent["offset"].as_u64().unwrap() + 1).into();
522 }540 }
523 review_page(&cache, id, origin(&intent, split), |page| {541 redo_page(&cache, origin(&intent, split), |page| {
524 replay(page, &placed, split)542 replay(page, &placed, split)
525 });543 });
544 id = cache.pending().unwrap().last().unwrap().id;
526 }545 }
527 other => panic!("{name} {case}: {other:?}"),546 other => panic!("{name} {case}: {other:?}"),
528 }547 }
...@@ -543,7 +562,7 @@ fn offline_paragraphs(output: Option<&Path>) {...@@ -543,7 +562,7 @@ fn offline_paragraphs(output: Option<&Path>) {
543 );562 );
544 assert_eq!(server.publications, entries.len());563 assert_eq!(server.publications, entries.len());
545 assert_eq!(cache.sync_once(&mut server).unwrap().edit, None);564 assert_eq!(cache.sync_once(&mut server).unwrap().edit, None);
546 assert_eq!(cache.snapshot().unwrap(), server.durable);565 assert_eq!(snapshot(&cache), server.durable);
547 if output.is_some() {566 if output.is_some() {
548 cache567 cache
549 .export_recovery(folder.join("recovery.sqlite"))568 .export_recovery(folder.join("recovery.sqlite"))
...@@ -573,9 +592,9 @@ fn remote_with(space: ExGuid, change: impl FnOnce(&mut Page)) -> Server {...@@ -573,9 +592,9 @@ fn remote_with(space: ExGuid, change: impl FnOnce(&mut Page)) -> Server {
573 let mut page = page_of(BEFORE, space);592 let mut page = page_of(BEFORE, space);
574 change(&mut page);593 change(&mut page);
575 Server::new(594 Server::new(
576 PreparedEdit::page(BEFORE, space, &page, "Native author")595 ops::saved(BEFORE, space, &page)
577 .unwrap()596 .unwrap()
578 .as_bytes(),597 .as_slice(),
579 )598 )
580}599}
581600
...@@ -611,7 +630,7 @@ fn a_split_publishes_two_paragraphs_and_a_join_puts_them_back() {...@@ -611,7 +630,7 @@ fn a_split_publishes_two_paragraphs_and_a_join_puts_them_back() {
611 ["ab\u{1f980}cd", "Right", "Preserved sibling"]630 ["ab\u{1f980}cd", "Right", "Preserved sibling"]
612 );631 );
613 assert_eq!(server.publications, 2);632 assert_eq!(server.publications, 2);
614 assert_eq!(cache.snapshot().unwrap(), server.durable);633 assert_eq!(snapshot(&cache), server.durable);
615}634}
616635
617#[test]636#[test]
...@@ -637,7 +656,7 @@ fn a_split_merges_with_a_remote_edit_to_another_paragraph() {...@@ -637,7 +656,7 @@ fn a_split_merges_with_a_remote_edit_to_another_paragraph() {
637}656}
638657
639#[test]658#[test]
640fn a_join_conflicts_when_the_remote_changed_the_paragraph_it_removes() {659fn a_join_keeps_a_conflict_page_when_the_remote_changed_the_paragraph_it_removes() {
641 let (space, page) = control("Join remote prefix");660 let (space, page) = control("Join remote prefix");
642 let (left, right) = (661 let (left, right) = (
643 body(&page)[0].text().unwrap().id,662 body(&page)[0].text().unwrap().id,
...@@ -649,12 +668,16 @@ fn a_join_conflicts_when_the_remote_changed_the_paragraph_it_removes() {...@@ -649,12 +668,16 @@ fn a_join_conflicts_when_the_remote_changed_the_paragraph_it_removes() {
649 .unwrap()668 .unwrap()
650 .unwrap();669 .unwrap();
651 let mut server = remote_with(space, |page| replace_text(page, right, 0..0, "Remote "));670 let mut server = remote_with(space, |page| replace_text(page, right, 0..0, "Remote "));
652 assert_eq!(671 assert!(matches!(
653 cache.sync_once(&mut server).unwrap().edit,672 cache.sync_once(&mut server).unwrap().edit,
654 Some((id, EditStatus::Conflict(ConflictKind::ContentChanged)))673 Some((_, EditStatus::Published { .. }))
655 );674 ));
656 assert_eq!(server.publications, 0);675 assert!(matches!(
657 review_page(&cache, id, left, |page| join_paragraphs(page, left, right));676 cache.status(id).unwrap(),
677 Some(EditStatus::Published { .. })
678 ));
679 assert_eq!(conflict_pages(&server.durable, space), 1);
680 redo_page(&cache, left, |page| join_paragraphs(page, left, right));
658 assert!(matches!(681 assert!(matches!(
659 cache.sync_once(&mut server).unwrap().edit,682 cache.sync_once(&mut server).unwrap().edit,
660 Some((_, EditStatus::Published { .. }))683 Some((_, EditStatus::Published { .. }))
...@@ -680,7 +703,7 @@ fn edits_dependent_on_a_published_split_survive_reopen() {...@@ -680,7 +703,7 @@ fn edits_dependent_on_a_published_split_survive_reopen() {
680 .unwrap()703 .unwrap()
681 .unwrap();704 .unwrap();
682 cache.sync_once(&mut server).unwrap().edit.unwrap();705 cache.sync_once(&mut server).unwrap().edit.unwrap();
683 let tail = body(&page_of(&cache.snapshot().unwrap(), space))[1]706 let tail = body(&page_of(&snapshot(&cache), space))[1]
684 .text()707 .text()
685 .unwrap()708 .unwrap()
686 .id;709 .id;
...@@ -689,17 +712,17 @@ fn edits_dependent_on_a_published_split_survive_reopen() {...@@ -689,17 +712,17 @@ fn edits_dependent_on_a_published_split_survive_reopen() {
689 })712 })
690 .unwrap()713 .unwrap()
691 .unwrap();714 .unwrap();
692 let local = cache.snapshot().unwrap();715 let local = snapshot(&cache);
693 let pending = cache.pending().unwrap();716 let pending = cache.pending().unwrap();
694 drop(cache);717 drop(cache);
695 cache = Replica::open(&path).unwrap();718 cache = Replica::open(&path).unwrap();
696 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));719 assert_eq!(pages(&snapshot(&cache)), pages(&local));
697 assert_eq!(cache.pending().unwrap(), pending);720 assert_eq!(cache.pending().unwrap(), pending);
698 let mut remote = page_of(&server.durable, space);721 let mut remote = page_of(&server.durable, space);
699 replace_text(&mut remote, other, 0..0, "Remote ");722 replace_text(&mut remote, other, 0..0, "Remote ");
700 server.visible = PreparedEdit::page(&server.durable, space, &remote, "Native author")723 server.visible = ops::saved(&server.durable, space, &remote)
701 .unwrap()724 .unwrap()
702 .as_bytes()725 .as_slice()
703 .to_vec();726 .to_vec();
704 server.durable.clone_from(&server.visible);727 server.durable.clone_from(&server.visible);
705 assert!(728 assert!(
...@@ -742,7 +765,7 @@ fn uncertain_splits_and_joins_keep_the_original_attempt_across_reopen() {...@@ -742,7 +765,7 @@ fn uncertain_splits_and_joins_keep_the_original_attempt_across_reopen() {
742 })765 })
743 .unwrap()766 .unwrap()
744 .unwrap();767 .unwrap();
745 let local = cache.snapshot().unwrap();768 let local = snapshot(&cache);
746 let pending = cache.pending().unwrap();769 let pending = cache.pending().unwrap();
747 let mut server = Server::new(BEFORE);770 let mut server = Server::new(BEFORE);
748 server.fault = fault;771 server.fault = fault;
...@@ -764,7 +787,7 @@ fn uncertain_splits_and_joins_keep_the_original_attempt_across_reopen() {...@@ -764,7 +787,7 @@ fn uncertain_splits_and_joins_keep_the_original_attempt_across_reopen() {
764 Fault::UnknownBefore => {787 Fault::UnknownBefore => {
765 assert!(matches!(state, EditStatus::AwaitingConfirmation { .. }));788 assert!(matches!(state, EditStatus::AwaitingConfirmation { .. }));
766 assert_eq!(cache.pending().unwrap(), pending);789 assert_eq!(cache.pending().unwrap(), pending);
767 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));790 assert_eq!(pages(&snapshot(&cache)), pages(&local));
768 for _ in 0..3 {791 for _ in 0..3 {
769 assert_eq!(792 assert_eq!(
770 cache.sync_once(&mut server).unwrap().edit,793 cache.sync_once(&mut server).unwrap().edit,
...@@ -777,7 +800,7 @@ fn uncertain_splits_and_joins_keep_the_original_attempt_across_reopen() {...@@ -777,7 +800,7 @@ fn uncertain_splits_and_joins_keep_the_original_attempt_across_reopen() {
777 }800 }
778 Fault::UnknownAfter => {801 Fault::UnknownAfter => {
779 assert!(matches!(state, EditStatus::AwaitingConfirmation { .. }));802 assert!(matches!(state, EditStatus::AwaitingConfirmation { .. }));
780 assert_eq!(pages(&cache.snapshot().unwrap()), pages(&local));803 assert_eq!(pages(&snapshot(&cache)), pages(&local));
781 assert!(matches!(804 assert!(matches!(
782 cache.sync_once(&mut server).unwrap().edit.unwrap().1,805 cache.sync_once(&mut server).unwrap().edit.unwrap().1,
783 EditStatus::Published { .. }806 EditStatus::Published { .. }
...@@ -797,13 +820,13 @@ fn uncertain_splits_and_joins_keep_the_original_attempt_across_reopen() {...@@ -797,13 +820,13 @@ fn uncertain_splits_and_joins_keep_the_original_attempt_across_reopen() {
797 };820 };
798 assert_eq!(texts(&page_of(&server.durable, space)), expected);821 assert_eq!(texts(&page_of(&server.durable, space)), expected);
799 assert_eq!(cache.sync_once(&mut server).unwrap().edit, None);822 assert_eq!(cache.sync_once(&mut server).unwrap().edit, None);
800 assert_eq!(cache.snapshot().unwrap(), server.durable);823 assert_eq!(snapshot(&cache), server.durable);
801 }824 }
802 }825 }
803}826}
804827
805#[test]828#[test]
806fn a_competing_join_leaves_the_other_client_a_reviewable_split() {829fn a_competing_join_leaves_the_other_client_its_split_on_a_conflict_page() {
807 let (space, page) = control("Join remote prefix");830 let (space, page) = control("Join remote prefix");
808 let (left, right) = (831 let (left, right) = (
809 body(&page)[0].text().unwrap().id,832 body(&page)[0].text().unwrap().id,
...@@ -819,27 +842,24 @@ fn a_competing_join_leaves_the_other_client_a_reviewable_split() {...@@ -819,27 +842,24 @@ fn a_competing_join_leaves_the_other_client_a_reviewable_split() {
819 let id = save_page(&second, right, |page| split_paragraph(page, right, 2))842 let id = save_page(&second, right, |page| split_paragraph(page, right, 2))
820 .unwrap()843 .unwrap()
821 .unwrap();844 .unwrap();
822 let local = second.snapshot().unwrap();
823 let pending = second.pending().unwrap();
824 let mut server = Server::new(BEFORE);845 let mut server = Server::new(BEFORE);
825 assert!(matches!(846 assert!(matches!(
826 first.sync_once(&mut server).unwrap().edit.unwrap().1,847 first.sync_once(&mut server).unwrap().edit.unwrap().1,
827 EditStatus::Published { .. }848 EditStatus::Published { .. }
828 ));849 ));
829 assert_eq!(850 assert!(matches!(
830 second.sync_once(&mut server).unwrap().edit,851 second.sync_once(&mut server).unwrap().edit,
831 Some((id, EditStatus::Conflict(ConflictKind::ContentChanged)))852 Some((_, EditStatus::Published { .. }))
832 );853 ));
833 assert_eq!(server.publications, 1);854 assert_eq!(server.publications, 2);
855 assert_eq!(conflict_pages(&server.durable, space), 1);
834 drop(second);856 drop(second);
835 let second = Replica::open(&path).unwrap();857 let second = Replica::open(&path).unwrap();
836 assert_eq!(pages(&second.snapshot().unwrap()), pages(&local));858 assert!(matches!(
837 assert_eq!(second.pending().unwrap(), pending);
838 assert_eq!(
839 second.status(id).unwrap(),859 second.status(id).unwrap(),
840 Some(EditStatus::Conflict(ConflictKind::ContentChanged))860 Some(EditStatus::Published { .. })
841 );861 ));
842 review_page(&second, id, left, |page| {862 redo_page(&second, left, |page| {
843 split_paragraph(page, left, 2);863 split_paragraph(page, left, 2);
844 });864 });
845 assert!(matches!(865 assert!(matches!(
crates/notebook/tests/sync_worker.rs+116-115
...@@ -1,6 +1,7 @@...@@ -1,6 +1,7 @@
1//! The background worker: waking, backoff, ownership and error propagation.1//! The background worker: waking, backoff, ownership and error propagation.
22
3use notebook::{ConflictKind, EditStatus, Error, Remote, Replica};3use notebook::{EditStatus, Error, Remote, Replica};
4use onestore::Stamp;
4use onestore::{5use onestore::{
5 CommitError, CommitState, ExGuid, RevisionIndex, Store, Transaction, document::Document,6 CommitError, CommitState, ExGuid, RevisionIndex, Store, Transaction, document::Document,
6};7};
...@@ -25,14 +26,14 @@ impl Remote for Shared {...@@ -25,14 +26,14 @@ impl Remote for Shared {
25 fn read(&mut self) -> io::Result<Vec<u8>> {26 fn read(&mut self) -> io::Result<Vec<u8>> {
26 self.0.lock().unwrap().read()27 self.0.lock().unwrap().read()
27 }28 }
28 fn stamp(&mut self) -> io::Result<Option<onestore::Stamp>> {29 fn stamp(&mut self) -> io::Result<Stamp> {
29 self.0.lock().unwrap().stamp()30 self.0.lock().unwrap().stamp()
30 }31 }
31 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {32 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
32 self.0.lock().unwrap().publish(transaction)33 self.0.lock().unwrap().publish(transaction)
33 }34 }
34 fn confirm(&mut self, snapshot: &[u8]) -> Result<(), CommitError> {35 fn confirm(&mut self, base: &Stamp) -> Result<(), CommitError> {
35 self.0.lock().unwrap().confirm(snapshot)36 self.0.lock().unwrap().confirm(base)
36 }37 }
37}38}
3839
...@@ -75,11 +76,17 @@ fn reconnects_after_connect_read_and_uncertain_publish_without_replaying() {...@@ -75,11 +76,17 @@ fn reconnects_after_connect_read_and_uncertain_publish_without_replaying() {
75 }76 }
76 self.shared.read()77 self.shared.read()
77 }78 }
79 fn stamp(&mut self) -> io::Result<Stamp> {
80 if self.fail_read {
81 return Err(io::ErrorKind::ConnectionReset.into());
82 }
83 self.shared.stamp()
84 }
78 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {85 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
79 self.shared.publish(transaction)86 self.shared.publish(transaction)
80 }87 }
81 fn confirm(&mut self, snapshot: &[u8]) -> Result<(), CommitError> {88 fn confirm(&mut self, base: &Stamp) -> Result<(), CommitError> {
82 self.shared.confirm(snapshot)89 self.shared.confirm(base)
83 }90 }
84 }91 }
85 let dir = tempfile::tempdir().unwrap();92 let dir = tempfile::tempdir().unwrap();
...@@ -142,7 +149,7 @@ fn reconnects_after_connect_read_and_uncertain_publish_without_replaying() {...@@ -142,7 +149,7 @@ fn reconnects_after_connect_read_and_uncertain_publish_without_replaying() {
142 drop(cache);149 drop(cache);
143 let cache = Replica::open(&path).unwrap();150 let cache = Replica::open(&path).unwrap();
144 assert_eq!(cache.status(id).unwrap(), Some(published));151 assert_eq!(cache.status(id).unwrap(), Some(published));
145 let cached = cache.snapshot().unwrap();152 let cached = snapshot(&cache);
146 // Confirmation changes reader-notification fields without changing the committed graph.153 // Confirmation changes reader-notification fields without changing the committed graph.
147 assert_eq!(cached[..212], server.durable[..212]);154 assert_eq!(cached[..212], server.durable[..212]);
148 assert_eq!(cached[252..], server.durable[252..]);155 assert_eq!(cached[252..], server.durable[252..]);
...@@ -159,7 +166,7 @@ fn reconnects_after_connect_read_and_uncertain_publish_without_replaying() {...@@ -159,7 +166,7 @@ fn reconnects_after_connect_read_and_uncertain_publish_without_replaying() {
159fn local_saves_wake_an_idle_worker_and_publish_every_writer_marker() {166fn local_saves_wake_an_idle_worker_and_publish_every_writer_marker() {
160 let dir = tempfile::tempdir().unwrap();167 let dir = tempfile::tempdir().unwrap();
161 let source = onestore::create_section("worker.one", "abc", "Fixture").unwrap();168 let source = onestore::create_section("worker.one", "abc", "Fixture").unwrap();
162 let (_, oid, _) = text(&source);169 let (sid, oid, _) = text(&source);
163 let cache = Arc::new(Replica::create(dir.path().join("cache.sqlite"), &source).unwrap());170 let cache = Arc::new(Replica::create(dir.path().join("cache.sqlite"), &source).unwrap());
164 let server = Arc::new(Mutex::new(Server::new(&source)));171 let server = Arc::new(Mutex::new(Server::new(&source)));
165 let shared = Shared(Arc::clone(&server));172 let shared = Shared(Arc::clone(&server));
...@@ -190,13 +197,19 @@ fn local_saves_wake_an_idle_worker_and_publish_every_writer_marker() {...@@ -190,13 +197,19 @@ fn local_saves_wake_an_idle_worker_and_publish_every_writer_marker() {
190 (0..12)197 (0..12)
191 .map(|writer| {198 .map(|writer| {
192 let cache = &cache;199 let cache = &cache;
200 // Typed as an op: a page read before another writer's edit would lower stale.
193 scope.spawn(move || {201 scope.spawn(move || {
194 let marker = format!("[{writer}] ");202 let op = onestore::op::PageOp::Text {
195 model_ops::save(cache, oid, |page| {203 text: oid,
196 model_ops::replace_text(page, oid, 0..0, &marker)204 range: 0..0,
197 })205 with: format!("[{writer}] "),
198 .unwrap()206 };
199 .unwrap()207 let ops = vec![onestore::op::Op::Page { space: sid, op }];
208 let edit = onestore::op::Edit {
209 at: model_ops::now(),
210 ops,
211 };
212 cache.apply(model_ops::AUTHOR, edit).unwrap()
200 })213 })
201 })214 })
202 .collect::<Vec<_>>()215 .collect::<Vec<_>>()
...@@ -221,7 +234,7 @@ fn local_saves_wake_an_idle_worker_and_publish_every_writer_marker() {...@@ -221,7 +234,7 @@ fn local_saves_wake_an_idle_worker_and_publish_every_writer_marker() {
221 Some(EditStatus::Published { .. })234 Some(EditStatus::Published { .. })
222 ));235 ));
223 }236 }
224 let content = text(&cache.snapshot().unwrap()).2;237 let content = text(&snapshot(&cache)).2;
225 for writer in 0..12 {238 for writer in 0..12 {
226 assert_eq!(content.matches(&format!("[{writer}] ")).count(), 1);239 assert_eq!(content.matches(&format!("[{writer}] ")).count(), 1);
227 }240 }
...@@ -230,7 +243,7 @@ fn local_saves_wake_an_idle_worker_and_publish_every_writer_marker() {...@@ -230,7 +243,7 @@ fn local_saves_wake_an_idle_worker_and_publish_every_writer_marker() {
230 assert_eq!(text(&server.durable).2, content);243 assert_eq!(text(&server.durable).2, content);
231 // Edits queued while the worker was busy publish together.244 // Edits queued while the worker was busy publish together.
232 assert_eq!(server.publications, 1);245 assert_eq!(server.publications, 1);
233 assert_eq!(cache.snapshot().unwrap(), server.durable);246 assert_eq!(snapshot(&cache), server.durable);
234}247}
235248
236#[test]249#[test]
...@@ -244,13 +257,16 @@ fn dropping_during_publication_is_nonblocking_and_retains_ownership_until_recove...@@ -244,13 +257,16 @@ fn dropping_during_publication_is_nonblocking_and_retains_ownership_until_recove
244 fn read(&mut self) -> io::Result<Vec<u8>> {257 fn read(&mut self) -> io::Result<Vec<u8>> {
245 self.shared.read()258 self.shared.read()
246 }259 }
260 fn stamp(&mut self) -> io::Result<Stamp> {
261 self.shared.stamp()
262 }
247 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {263 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
248 self.entered.send(()).unwrap();264 self.entered.send(()).unwrap();
249 self.resume.recv_timeout(Duration::from_secs(5)).unwrap();265 self.resume.recv_timeout(Duration::from_secs(5)).unwrap();
250 self.shared.publish(transaction)266 self.shared.publish(transaction)
251 }267 }
252 fn confirm(&mut self, snapshot: &[u8]) -> Result<(), CommitError> {268 fn confirm(&mut self, base: &Stamp) -> Result<(), CommitError> {
253 self.shared.confirm(snapshot)269 self.shared.confirm(base)
254 }270 }
255 }271 }
256 let dir = tempfile::tempdir().unwrap();272 let dir = tempfile::tempdir().unwrap();
...@@ -358,81 +374,65 @@ fn cache_failures_stop_retries_and_return_the_error_without_remote_publication()...@@ -358,81 +374,65 @@ fn cache_failures_stop_retries_and_return_the_error_without_remote_publication()
358 assert!(rx.try_iter().next().is_none());374 assert!(rx.try_iter().next().is_none());
359 assert_eq!(server.lock().unwrap().publications, 0);375 assert_eq!(server.lock().unwrap().publications, 0);
360 assert_eq!(cache.status(id).unwrap(), Some(EditStatus::Pending));376 assert_eq!(cache.status(id).unwrap(), Some(EditStatus::Pending));
361 assert_eq!(text(&cache.snapshot().unwrap()).2, "L abc");377 assert_eq!(text(&snapshot(&cache)).2, "L abc");
362}378}
363379
364#[test]380#[test]
365fn polling_reports_a_blocked_queue_once_per_remote_change_without_replay() {381fn polling_reports_an_uncertain_attempt_once_per_remote_change_without_replay() {
366 for uncertain in [false, true] {382 let dir = tempfile::tempdir().unwrap();
367 let dir = tempfile::tempdir().unwrap();383 let source = onestore::create_section("worker.one", "abc", "Fixture").unwrap();
368 let source = onestore::create_section("worker.one", "abc", "Fixture").unwrap();384 let (sid, oid, _) = text(&source);
369 let (sid, oid, _) = text(&source);385 let cache = Arc::new(Replica::create(dir.path().join("cache.sqlite"), &source).unwrap());
370 let cache = Arc::new(Replica::create(dir.path().join("cache.sqlite"), &source).unwrap());386 let id = save(&cache, oid, 1..2, "L").unwrap();
371 let id = save(&cache, oid, 1..2, "L").unwrap();387 let local = text(&snapshot(&cache)).2;
372 let local = text(&cache.snapshot().unwrap()).2;388 let mut server = Server::new(&source);
373 let mut server = if uncertain {389 server.fault = Fault::UnknownBefore;
374 Server::new(&source)390 let server = Arc::new(Mutex::new(server));
375 } else {391 let shared = Shared(Arc::clone(&server));
376 Server::new(&onestore::replace_text(&source, sid, oid, 1..2, "R").unwrap())392 let (tx, rx) = mpsc::channel();
377 };393 let worker = cache
378 if uncertain {394 .start_sync(
379 server.fault = Fault::UnknownBefore;395 Duration::from_millis(10),
380 }396 move || Ok(shared.clone()),
381 let server = Arc::new(Mutex::new(server));397 move |result| {
382 let shared = Shared(Arc::clone(&server));398 tx.send(
383 let (tx, rx) = mpsc::channel();399 result
384 let worker = cache400 .as_ref()
385 .start_sync(401 .map(|synced| synced.edit.clone())
386 Duration::from_millis(10),402 .map_err(|_| ()),
387 move || Ok(shared.clone()),403 )
388 move |result| {
389 tx.send(
390 result
391 .as_ref()
392 .map(|synced| synced.edit.clone())
393 .map_err(|_| ()),
394 )
395 .unwrap();
396 },
397 )
398 .unwrap();
399 if uncertain {
400 assert!(rx.recv_timeout(Duration::from_secs(5)).unwrap().is_err());
401 }
402 let expected = |status: &EditStatus| {
403 if uncertain {
404 matches!(status, EditStatus::AwaitingConfirmation { .. })
405 } else {
406 *status == EditStatus::Conflict(ConflictKind::ContentChanged)
407 }
408 };
409 for round in 0..3 {
410 let (actual, status) = rx
411 .recv_timeout(Duration::from_secs(5))
412 .unwrap()
413 .unwrap()
414 .unwrap();404 .unwrap();
415 assert_eq!(actual, id);405 },
416 assert!(expected(&status), "{status:?}");406 )
417 // An unchanged remote is not read or reported again.407 .unwrap();
418 assert!(rx.recv_timeout(Duration::from_millis(100)).is_err());408 assert!(rx.recv_timeout(Duration::from_secs(5)).unwrap().is_err());
419 // Another writer's change elsewhere in the text is read and decided again.409 for round in 0..3 {
420 let mut server = server.lock().unwrap();410 let (actual, status) = rx
421 let changed =411 .recv_timeout(Duration::from_secs(5))
422 onestore::replace_text(&server.visible, sid, oid, 0..0, &round.to_string())412 .unwrap()
423 .unwrap();413 .unwrap()
424 server.visible = changed.clone();414 .unwrap();
425 server.durable = changed;415 assert_eq!(actual, id);
426 drop(server);416 assert!(
427 worker.wake();417 matches!(status, EditStatus::AwaitingConfirmation { .. }),
428 }418 "{status:?}"
429 worker.stop().unwrap();419 );
430 assert_eq!(text(&cache.snapshot().unwrap()).2, local);420 // An unchanged remote is not read or reported again.
431 assert_eq!(cache.pending().unwrap().len(), 1);421 assert!(rx.recv_timeout(Duration::from_millis(100)).is_err());
432 let server = server.lock().unwrap();422 // Another writer's change elsewhere in the text is read and decided again.
433 assert_eq!(server.publications, usize::from(uncertain));423 let mut server = server.lock().unwrap();
434 assert_eq!(server.confirmations, 0);424 let changed = typed(&server.visible, sid, oid, 0..0, &round.to_string());
425 server.visible = changed.clone();
426 server.durable = changed;
427 drop(server);
428 worker.wake();
435 }429 }
430 worker.stop().unwrap();
431 assert_eq!(text(&snapshot(&cache)).2, local);
432 assert_eq!(cache.pending().unwrap().len(), 1);
433 let server = server.lock().unwrap();
434 assert_eq!(server.publications, 1);
435 assert_eq!(server.confirmations, 0);
436}436}
437437
438#[test]438#[test]
...@@ -533,51 +533,46 @@ fn invalid_intervals_and_callback_panics_leave_worker_ownership_recoverable() {...@@ -533,51 +533,46 @@ fn invalid_intervals_and_callback_panics_leave_worker_ownership_recoverable() {
533 .unwrap();533 .unwrap();
534 assert_eq!(rx.recv_timeout(Duration::from_secs(5)).unwrap(), None);534 assert_eq!(rx.recv_timeout(Duration::from_secs(5)).unwrap(), None);
535 worker.stop().unwrap();535 worker.stop().unwrap();
536 assert_eq!(cache.snapshot().unwrap(), source);536 assert_eq!(snapshot(&cache), source);
537}537}
538538
539/// A conflict never blocks the worker: one step publishes the remote's version with the
540/// local one as its conflict page.
539#[test]541#[test]
540fn a_resolved_conflict_wakes_the_worker_and_publishes_the_original_edit_once() {542fn a_conflict_publishes_its_conflict_page_in_one_worker_step() {
541 let dir = tempfile::tempdir().unwrap();543 let dir = tempfile::tempdir().unwrap();
542 let source = onestore::create_section("resolve.one", "abc", "Fixture").unwrap();544 let source = onestore::create_section("resolve.one", "abc", "Fixture").unwrap();
543 let (sid, oid, _) = text(&source);545 let (sid, oid, _) = text(&source);
544 let cache = Arc::new(Replica::create(dir.path().join("cache.sqlite"), &source).unwrap());546 let cache = Arc::new(Replica::create(dir.path().join("cache.sqlite"), &source).unwrap());
545 let id = save(&cache, oid, 1..2, "L").unwrap();547 let id = save(&cache, oid, 1..2, "L").unwrap();
546 let remote = onestore::replace_text(&source, sid, oid, 1..2, "R").unwrap();548 let remote = typed(&source, sid, oid, 1..2, "R");
547 let server = Arc::new(Mutex::new(Server::new(&remote)));549 let server = Arc::new(Mutex::new(Server::new(&remote)));
548 let shared = Shared(Arc::clone(&server));550 let shared = Shared(Arc::clone(&server));
549 let (tx, rx) = mpsc::channel();551 let (tx, rx) = mpsc::channel();
550 let (resume_tx, resume_rx) = mpsc::channel();
551 let mut first = true;
552 let worker = cache552 let worker = cache
553 .start_sync(553 .start_sync(
554 Duration::from_secs(3600),554 Duration::from_secs(3600),
555 move || Ok(shared.clone()),555 move || Ok(shared.clone()),
556 move |result| {556 move |result| {
557 tx.send(result.as_ref().unwrap().edit.clone()).unwrap();557 tx.send(result.as_ref().unwrap().edit.clone()).unwrap();
558 if first {
559 first = false;
560 resume_rx.recv_timeout(Duration::from_secs(5)).unwrap();
561 }
562 },558 },
563 )559 )
564 .unwrap();560 .unwrap();
565 assert_eq!(
566 rx.recv_timeout(Duration::from_secs(5)).unwrap(),
567 Some((id, EditStatus::Conflict(ConflictKind::ContentChanged)))
568 );
569 cache.resolve(id, notebook::Resolution::Mine).unwrap();
570 assert_eq!(cache.status(id).unwrap(), Some(EditStatus::Pending));
571 resume_tx.send(()).unwrap();
572 assert!(561 assert!(
573 matches!(rx.recv_timeout(Duration::from_secs(5)).unwrap(), Some((actual, EditStatus::Published { .. })) if actual == id)562 matches!(rx.recv_timeout(Duration::from_secs(5)).unwrap(), Some((actual, EditStatus::Published { .. })) if actual > id)
574 );563 );
575 worker.stop().unwrap();564 worker.stop().unwrap();
576 assert!(cache.pending().unwrap().is_empty());565 assert!(cache.pending().unwrap().is_empty());
577 let server = server.lock().unwrap();566 let server = server.lock().unwrap();
578 assert_eq!(server.publications, 1);567 assert_eq!(server.publications, 1);
579 assert_eq!(text(&server.durable).2, "aLc");568 assert_eq!(
580 assert_eq!(cache.snapshot().unwrap(), server.durable);569 conflicts(&server.durable),
570 [(
571 sid,
572 vec![(model_ops::AUTHOR.to_owned(), vec!["aLc".to_owned()])]
573 )]
574 );
575 assert_eq!(snapshot(&cache), server.durable);
581}576}
582577
583#[test]578#[test]
...@@ -589,11 +584,14 @@ fn ordinary_read_and_unpublished_write_contention_reuse_the_connection() {...@@ -589,11 +584,14 @@ fn ordinary_read_and_unpublished_write_contention_reuse_the_connection() {
589 }584 }
590 impl Remote for Busy {585 impl Remote for Busy {
591 fn read(&mut self) -> io::Result<Vec<u8>> {586 fn read(&mut self) -> io::Result<Vec<u8>> {
587 self.server.read()
588 }
589 fn stamp(&mut self) -> io::Result<Stamp> {
592 self.reads += 1;590 self.reads += 1;
593 match self.reads {591 match self.reads {
594 1 => Err(io::ErrorKind::WouldBlock.into()),592 1 => Err(io::ErrorKind::WouldBlock.into()),
595 2 => Err(io::ErrorKind::ResourceBusy.into()),593 2 => Err(io::ErrorKind::ResourceBusy.into()),
596 _ => self.server.read(),594 _ => self.server.stamp(),
597 }595 }
598 }596 }
599 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {597 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
...@@ -611,8 +609,8 @@ fn ordinary_read_and_unpublished_write_contention_reuse_the_connection() {...@@ -611,8 +609,8 @@ fn ordinary_read_and_unpublished_write_contention_reuse_the_connection() {
611 _ => self.server.publish(transaction),609 _ => self.server.publish(transaction),
612 }610 }
613 }611 }
614 fn confirm(&mut self, source: &[u8]) -> Result<(), CommitError> {612 fn confirm(&mut self, base: &Stamp) -> Result<(), CommitError> {
615 self.server.confirm(source)613 self.server.confirm(base)
616 }614 }
617 }615 }
618 let dir = tempfile::tempdir().unwrap();616 let dir = tempfile::tempdir().unwrap();
...@@ -648,7 +646,7 @@ fn ordinary_read_and_unpublished_write_contention_reuse_the_connection() {...@@ -648,7 +646,7 @@ fn ordinary_read_and_unpublished_write_contention_reuse_the_connection() {
648 matches!(rx.recv_timeout(Duration::from_secs(5)).unwrap().unwrap(), Some((actual, EditStatus::Published { .. })) if actual == id)646 matches!(rx.recv_timeout(Duration::from_secs(5)).unwrap().unwrap(), Some((actual, EditStatus::Published { .. })) if actual == id)
649 );647 );
650 worker.stop().unwrap();648 worker.stop().unwrap();
651 assert_eq!(text(&cache.snapshot().unwrap()).2, "L abc");649 assert_eq!(text(&snapshot(&cache)).2, "L abc");
652 assert!(cache.pending().unwrap().is_empty());650 assert!(cache.pending().unwrap().is_empty());
653}651}
654652
...@@ -659,6 +657,9 @@ fn publication_backoff_drains_local_wakes_without_waiting_for_the_idle_poll() {...@@ -659,6 +657,9 @@ fn publication_backoff_drains_local_wakes_without_waiting_for_the_idle_poll() {
659 fn read(&mut self) -> io::Result<Vec<u8>> {657 fn read(&mut self) -> io::Result<Vec<u8>> {
660 self.0.read()658 self.0.read()
661 }659 }
660 fn stamp(&mut self) -> io::Result<Stamp> {
661 self.0.stamp()
662 }
662 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {663 fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> {
663 let server = &mut self.0;664 let server = &mut self.0;
664 if server.publications == 0 {665 if server.publications == 0 {
...@@ -670,8 +671,8 @@ fn publication_backoff_drains_local_wakes_without_waiting_for_the_idle_poll() {...@@ -670,8 +671,8 @@ fn publication_backoff_drains_local_wakes_without_waiting_for_the_idle_poll() {
670 }671 }
671 server.publish(transaction)672 server.publish(transaction)
672 }673 }
673 fn confirm(&mut self, source: &[u8]) -> Result<(), CommitError> {674 fn confirm(&mut self, base: &Stamp) -> Result<(), CommitError> {
674 self.0.confirm(source)675 self.0.confirm(base)
675 }676 }
676 }677 }
677 let dir = tempfile::tempdir().unwrap();678 let dir = tempfile::tempdir().unwrap();
...@@ -714,6 +715,6 @@ fn publication_backoff_drains_local_wakes_without_waiting_for_the_idle_poll() {...@@ -714,6 +715,6 @@ fn publication_backoff_drains_local_wakes_without_waiting_for_the_idle_poll() {
714 }715 }
715 worker.stop().unwrap();716 worker.stop().unwrap();
716 assert!(cache.pending().unwrap().is_empty());717 assert!(cache.pending().unwrap().is_empty());
717 assert_eq!(content(&cache.snapshot().unwrap(), oid), "L abc");718 assert_eq!(content(&snapshot(&cache), oid), "L abc");
718 assert_eq!(pages(&cache.snapshot().unwrap()), 2);719 assert_eq!(pages(&snapshot(&cache)), 2);
719}720}
crates/onestore/README.md+48-52
...@@ -49,32 +49,27 @@ harness also accepts `--client-profile release`....@@ -49,32 +49,27 @@ harness also accepts `--client-profile release`.
49| `page::Page`, `page::Paragraph`, `page::Ink`, `page::Math` | Build an editable page model (title, outlines, paragraphs with coalesced text spans, tables, images, attachments, ink drawings and handwriting decoded to stroke polylines in page points) with stored identities; equations parse from their linear text and run data into a tree that renders the MathML OneNote exports; content outside the model is retained as `Unsupported` |49| `page::Page`, `page::Paragraph`, `page::Ink`, `page::Math` | Build an editable page model (title, outlines, paragraphs with coalesced text spans, tables, images, attachments, ink drawings and handwriting decoded to stroke polylines in page points) with stored identities; equations parse from their linear text and run data into a tree that renders the MathML OneNote exports; content outside the model is retained as `Unsupported` |
50| `protected::UnlockedSection` (optional feature) | Own decoded buffers for explicit known-password inspection; clear those buffers on drop; derived document strings/exports remain caller-owned |50| `protected::UnlockedSection` (optional feature) | Own decoded buffers for explicit known-password inspection; clear those buffers on drop; derived document strings/exports remain caller-owned |
51| `create_section` | Create one page containing one plain-text paragraph and an author, including Unicode |51| `create_section` | Create one page containing one plain-text paragraph and an author, including Unicode |
52| `PageCreation`, `PreparedEdit::create_page` | Add an empty top-level page and its section entry atomically, retaining page identities across retries |52| `PageCreation`, `SectionOp::Create` | Add an empty top-level page and its section entry atomically, under identities the intent retains across retries |
53| `PageEdit`, `PreparedEdit::pages` | Publish explicitly selected page moves and indentation changes together, preserving page content and historical revisions |53| `PageEdit`, `SectionOp::Pages` | Publish explicitly selected page moves and indentation changes together, preserving page content and historical revisions |
54| `PreparedEdit::delete_pages_permanently` | Remove explicit pages and their section references atomically while retaining stored revisions |54| `SectionOp::Delete` | Remove explicit pages and their section references atomically while retaining stored revisions |
55| `create_table_of_contents` | Create ordered section entries from filenames and file identities |55| `create_table_of_contents` | Create ordered section entries from filenames and file identities |
56| `TocEdit`, `PreparedEdit::table_of_contents` | Add, rename, colour, order and remove a table of contents' section and group entries as one revision |56| `TocEdit`, `edit_table_of_contents` | Add, rename, order and remove a table of contents' section and group entries, and colour the notebook, as one revision's `Transaction`; a section's colour is `SectionOp::Color` in its own metadata |
57| `place`, `place_file` | Name a file for its notebook as OneNote does on adoption (parent TOC identity and name CRC in the header), so OneNote keeps its identity, on any `CommitIo` or under the filesystem adapter |57| `place`, `place_file` | Name a file for its notebook as OneNote does on adoption (parent TOC identity and name CRC in the header), so OneNote keeps its identity, on any `CommitIo` or under the filesystem adapter |
58| `replace_property_bytes` | Append one scalar-property revision; preserve prior revisions and unrelated property values and references |58| `TextAttribute`, `PageOp::Format` | Change character formatting over a UTF-16 range while sharing immutable styles; preserve unselected runs |
59| `replace_text`, `commit_text`, `commit_file_text` | Replace a UTF-16 range across ordinary text runs; publish text, run boundaries and modification time together |59| `OutlineEdit`, `PageOp::Outline` | Change ordinary outline position/width or a paragraph's saved expansion default, preserving identities and content |
60| `Insertion`, `PreparedEdit::insert` | Insert paragraphs into editable containers or positioned outlines into a page, retaining intent identities across rebases |60| `Transaction`, `Stamp` | The bytes a commit writes (appended data, in-place list-tail and log patches, header) and the header and length it requires unchanged; `commit` under caller-held exclusion, `commit_file` under the conservative filesystem adapter, `apply` to the base image in memory; serializable for queues |
61| `ParagraphSplit`, `PreparedEdit::split` | Split ordinary text at a UTF-16 scalar boundary, retaining the original left identities and moving children to the right |
62| `ParagraphJoin`, `PreparedEdit::join` | Join adjacent ordinary text while preserving inherited character styles and native text-identity rules |
63| `TextAttribute`, `PreparedEdit::format` | Change character formatting over a UTF-16 range while sharing immutable styles; preserve unselected runs |
64| `OutlineEdit`, `PreparedEdit::outline` | Change ordinary outline position/width or a paragraph's saved expansion default, preserving identities and content |
65| `PreparedEdit::page` | Publish an edited `page::Page` as one revision per changed space: text, character and paragraph formatting, hyperlinks (external and, via `page::link`, to pages, paragraphs and sections), bullets and numbering, note tags, table rows, columns, cell shading and indents, nested tables, inserted pictures (in paragraphs or on the page) and attachments, picture position, size and description, a stored attachment's shown name, source path and icon size, paragraph styles on new paragraphs, ink drawings and handwriting (strokes added and erased), equations (built from `page::Math` trees), paragraph insertion/split/join/move/deletion, outline insertion/deletion/position/width and saved collapse state, lowered onto the typed writers with the model's identities |
66| `TreeEdit`, `PreparedEdit::tree` | Move or delete a subtree on one page, normalize surviving containers, and replace an emptied table cell's paragraph atomically |
67| `Transaction`, `Stamp`, `PreparedEdit::transaction` | The bytes a commit writes (appended data, in-place list-tail and log patches, header) and the header and length it requires unchanged; `commit` under caller-held exclusion, `commit_file` under the conservative filesystem adapter, `apply` to the base image in memory; serializable for queues |
68| `Arena`, `Section` | Keep a section parsed across edits: `open` validates an image once; `seal` appends one revision per changed space as a `Transaction` on `stamp`, checking only what it appends; `replay` applies a queued transaction; `image` and `page` read the result |61| `Arena`, `Section` | Keep a section parsed across edits: `open` validates an image once; `seal` appends one revision per changed space as a `Transaction` on `stamp`, checking only what it appends; `replay` applies a queued transaction; `image` and `page` read the result |
69| `op::{Edit, Op, PageOp, TableEdit, SectionOp}`, `Section::apply` | Object-level edits (text, formatting, links, equations, paragraph insertion/split/join/move/deletion/levels, outlines, lists, tags, styles, paragraph formatting, tables, pictures, attachments, ink, page creation/import/moves/removal) applied whole or not to a kept-open section with emitter-chosen identities, UTF-16 ranges and the edit's time; refusals name the target, identity or structure at fault |62| `op::{Edit, Op, PageOp, TableEdit, SectionOp}`, `Section::apply` | Object-level edits (text, formatting, links, equations, paragraph insertion/split/join/move/deletion/levels, outlines, lists, tags, styles, paragraph formatting, tables, pictures, attachments, ink, page creation/import/moves/removal, conflict pages) applied whole or not to a kept-open section with emitter-chosen identities, UTF-16 ranges and the edit's time; refusals name the target, identity or structure at fault |
70| `op::lower`, `op::lower_page`, `Section::apply_page` | The ops turning a range of paragraphs, or a whole page model, into another, computed from the models alone |63| `SectionOp::Conflict`, `Section::conflicts`, `ConflictPage` | Keep the version of a page a merge could not take as OneNote 2010 does: a read-only conflict page (`jcidConflictPageMetaData`, `IsConflictPage`, conflict objects marked) under the page's manifest, which says it has conflict pages; list each page's, newest first, and delete one (`SectionOp::Delete`) as OneNote's Delete Conflict Page does |
64| `op::lower`, `op::lower_page` | The ops turning a range of paragraphs, or a whole page model, into another, computed from the models alone, for editors and imports |
65| `op::predict` | The page an op leaves, as the section stores it and reading it back shows it |
71| `read_file` | Read a snapshot under whole-file exclusion |66| `read_file` | Read a snapshot under whole-file exclusion |
72| `read_snapshot` | Read a validated snapshot through fresh positioned I/O while the caller excludes maintenance |67| `read_snapshot` | Read a validated snapshot through fresh positioned I/O while the caller excludes maintenance |
73| `commit_file_property` | Lock, check the source snapshot's stamp, append and flush, then publish the revision |68| `CommitIo`, `confirm`, `confirm_file` | Supply another storage backend with equivalent exclusion and ordered durability; check that a stamp still holds |
74| `CommitIo`, `commit_property_bytes` | Supply another storage backend with equivalent exclusion and ordered durability |
7569
76Scalar edits accept encoded values and require the caller to maintain MS-ONE70Edits enter a kept-open `Section` as ops and leave as one appended revision per
77semantics. Text edits maintain run boundaries, inherit the insertion run's formatting,71changed space; only section and page creation, imports and opening files handle
72whole images. Text edits maintain run boundaries, inherit the insertion run's formatting,
78and promote legacy text to Unicode when needed. Explicit and body-derived73and promote legacy text to Unicode when needed. Explicit and body-derived
79navigation titles update in the same transaction; unsupported fields,74navigation titles update in the same transaction; unsupported fields,
80protected objects and split surrogate pairs are75protected objects and split surrogate pairs are
...@@ -85,19 +80,19 @@ sections retain their encrypted structure and payloads. With the optional...@@ -85,19 +80,19 @@ sections retain their encrypted structure and payloads. With the optional
85`protected` feature, `protected::UnlockedSection` opens native OneNote 201080`protected` feature, `protected::UnlockedSection` opens native OneNote 2010
86AES-128/CBC, SHA-1 password wrappers into a borrowed document view. Incorrect81AES-128/CBC, SHA-1 password wrappers into a borrowed document view. Incorrect
87passwords, unsupported protection profiles and work-limit failures remain distinct.82passwords, unsupported protection profiles and work-limit failures remain distinct.
88The source stays encrypted. `PreparedEdit::page_protected` publishes a page-model83The source stays encrypted. `UnlockedSection::apply` applies page ops and seals them
89edit stored under the section's key (fresh IV per object, payloads under the file84under the section's key (fresh IV per object, payloads under the file IV); `Section`
90IV); the other writers reject protected sections.85rejects protected sections.
91`PageCreation::new` appends, or inserts before the first page space of an existing86`PageCreation::new` appends, or inserts before the first page space of an existing
92series. `Some("")` creates an empty title field; `None` omits the title node.87series. `Some("")` creates an empty title field; `None` omits the title node.
93The page has no body outlines, generated date/time text or applied template.88The page has no body outlines, generated date/time text or applied template.
94Retain the intent to preserve its page, title and space identities; existing89Retain the intent to preserve its page, title and space identities; existing
95identities require reconciliation before retry. Body insertion and title edits90identities require reconciliation before retry. Body insertion and title edits
96use those identities through the existing APIs. [Native page-creation fixtures](../../corpus/page-lifecycle/creation/README.md)91use those identities through page ops. [Native page-creation fixtures](../../corpus/page-lifecycle/creation/README.md)
97cover duplicate Unicode titles, native edits and Rust follow-up edits.92cover duplicate Unicode titles, native edits and Rust follow-up edits.
98`PageEdit::set_level` changes one page's indentation in place. `PageEdit::move_to`93`PageEdit::set_level` changes one page's indentation in place. `PageEdit::move_to`
99moves before an existing page space, or appends for `None`, and sets its level.94moves before an existing page space, or appends for `None`, and sets its level.
100`PreparedEdit::pages` applies moves in slice order and publishes the final order,95`SectionOp::Pages` applies moves in slice order and publishes the final order,
101series membership and metadata levels in one transaction. Each page occurs once;96series membership and metadata levels in one transaction. Each page occurs once;
102levels are 1–3 and the final first page must have level 1. A following deeper-level97levels are 1–3 and the final first page must have level 1. A following deeper-level
103page remains in place unless explicitly selected. To move a group, supply all its98page remains in place unless explicitly selected. To move a group, supply all its
...@@ -105,40 +100,39 @@ pages in order. Retain the intents across retries so newly formed series keep...@@ -105,40 +100,39 @@ pages in order. Retain the intents across retries so newly formed series keep
105their identities; `reposition(PagePosition, level)` revises their placement while100their identities; `reposition(PagePosition, level)` revises their placement while
106preserving those identities. [Native page-edit fixtures](../../corpus/page-lifecycle/page-edits/README.md)101preserving those identities. [Native page-edit fixtures](../../corpus/page-lifecycle/page-edits/README.md)
107cover individual tabs, selected and collapsed groups, nesting and promotion.102cover individual tabs, selected and collapsed groups, nesting and promotion.
108`PreparedEdit::delete_pages_permanently` removes exactly the supplied page spaces,103`SectionOp::Delete` removes exactly the supplied page spaces,
109including subpages only when selected explicitly. The first remaining page becomes104including subpages only when selected explicitly. The first remaining page becomes
110top-level; other page levels and surviving content are retained. The operation105top-level; other page levels and surviving content are retained. The operation
111creates no recycle-bin copies and preserves prior revisions, so it is not secure106creates no recycle-bin copies and preserves prior revisions, so it is not secure
112erasure. Duplicate, missing or non-page identities reject the entire batch;107erasure. Duplicate, missing or non-page identities reject the entire batch;
113an empty selection leaves the file unchanged. Commit uses the exact source108an empty selection leaves the file unchanged.
114snapshot and rejects a stale prepared removal before writing.109`PageOp::Insert` and `PageOp::Add` update child references, reference counts,
115Insertions update child references, reference counts, modification times and automatic110modification times and automatic titles atomically. Paragraphs can be nested or
116titles atomically. Paragraphs can be nested or inserted into table cells; outline111inserted into table cells; outline coordinates use points. Inserted text keeps its
117coordinates use points. `Insertion::with_formatting` includes nonoverlapping UTF-16112spans' formats; `PageOp::Format` over an empty text's `0..0` sets its insertion style.
118spans in that same publication; gaps keep the default insertion style. Empty text113New objects carry the emitter's identities, so a queued edit replays onto another
119accepts one `0..0` span for subsequent typing. Retain the `Insertion` value for rebasing: creating another114image unchanged; an identity already on the page refuses the edit. Formatting accepts explicit attributes, preserves inherited values,
120value creates different object identities. Duplicate insertion identities require
121reconciliation. Formatting accepts explicit attributes, preserves inherited values,
122and gives retired immutable styles zero current references while retaining history.115and gives retired immutable styles zero current references while retaining history.
123Outline layout edits use points. Width is at least 36 points; an explicit user width116Outline layout edits use points. Width is at least 36 points; an explicit user width
124and an automatic maximum-width hint remain distinct. Native layout generates the117and an automatic maximum-width hint remain distinct. Native layout generates the
125rendered height. Saved paragraph collapse defaults can be overridden by the native118rendered height. Saved paragraph collapse defaults can be overridden by the native
126client's cached view. [Native layout captures](../../corpus/outline-edit/README.md)119client's cached view. [Native layout captures](../../corpus/outline-edit/README.md)
127verify these edits through a fresh OneNote cache, including fields and nested content.120verify these edits through a fresh OneNote cache, including fields and nested content.
128`TreeEdit::move_to` takes an existing parent and an optional direct sibling to insert121`PageOp::Move` takes an existing parent and an optional direct sibling to insert
129before; `None` appends. Paragraphs retain their descendants and explicit list styles.122before; `None` appends. Paragraphs retain their descendants and explicit list styles.
130Outlines remain page children, so reordering changes their stacking order while123Outlines remain page children, so reordering changes their stacking order while
131retaining coordinates. `TreeEdit::delete` removes the selected subtree from the active124retaining coordinates. `PageOp::Delete` removes the selected subtree from the active
132graph and preserves historical objects. Empty outlines/groups are removed; surviving125graph and preserves historical objects. Empty outlines/groups are removed; surviving
133group indentation is normalized without shifting other paragraphs. Empty table cells126group indentation is normalized without shifting other paragraphs. An edit that
134receive new empty paragraph/text objects with identities retained in the intent.127leaves a table cell without a paragraph is refused; insert its replacement in the
128same edit.
135Title/protected content, ambiguous ancestry, cycles, and incompatible destinations129Title/protected content, ambiguous ancestry, cycles, and incompatible destinations
136reject before publication. Modification times, move attribution and automatic titles130reject before publication. Modification times, move attribution and automatic titles
137publish with the tree change. These explicit destinations differ from keyboard list131publish with the tree change. These explicit destinations differ from keyboard list
138indentation, which can also substitute list markers.132indentation, which can also substitute list markers.
139Paragraph splits preserve character formatting, retain tags on the left, and clone133Paragraph splits preserve character formatting, retain tags on the left, and clone
140mutable list objects without restarting numbering. The new right paragraph/text134mutable list objects without restarting numbering. `PageOp::Split` names
141identities belong to the retained `ParagraphSplit` intent. Its publication includes135the new right paragraph/text identities. Its publication includes
142the complete child graph and title metadata; repeating an existing identity requires136the complete child graph and title metadata; repeating an existing identity requires
143reconciliation. Title containers, generated fields, recording-linked text and137reconciliation. Title containers, generated fields, recording-linked text and
144associated run metadata are rejected before I/O. Native split controls and subsequent138associated run metadata are rejected before I/O. Native split controls and subsequent
...@@ -206,9 +200,8 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -206,9 +200,8 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
206```200```
207201
208`Store::parse` exposes checksum mismatches for diagnostic readers; the writer202`Store::parse` exposes checksum mismatches for diagnostic readers; the writer
209rejects them. The resolved graph borrows the snapshot. Select the object-space ID,203rejects them. The resolved graph borrows the snapshot. Open it as a `Section`,
210object ID, and property from that graph, then pass those IDs, the same snapshot,204apply ops naming objects from that graph, seal, and commit the `Transaction`. The
211and the replacement's encoded bytes to `commit_file_property`. The
212`edit_property` example demonstrates selection by JCID/property/expected bytes,205`edit_property` example demonstrates selection by JCID/property/expected bytes,
213with optional explicit IDs when conflict copies contain identical text.206with optional explicit IDs when conflict copies contain identical text.
214207
...@@ -228,7 +221,7 @@ the header (MS-ONESTORE 2.3.1), so the body is never compared. Live readers must...@@ -228,7 +221,7 @@ the header (MS-ONESTORE 2.3.1), so the body is never compared. Live readers must
228equivalent exclusion.221equivalent exclusion.
229Native conflict creation can still expose cross-space references before their222Native conflict creation can still expose cross-space references before their
230targets are saved; such snapshots must be rejected and reread while synchronization223targets are saved; such snapshots must be rejected and reread while synchronization
231proceeds. `read_file` and `commit_file_property` serialize within the process because224proceeds. `read_file` and `Transaction::commit_file` serialize within the process because
232macOS SMB locks can be reentrant. The lock is nonblocking across processes;225macOS SMB locks can be reentrant. The lock is nonblocking across processes;
233contention requires a fresh read and a later retry.226contention requires a fresh read and a later retry.
234227
...@@ -245,13 +238,16 @@ native acceptance captures. A no-op still flushes; a failed flush has an unknown...@@ -245,13 +238,16 @@ native acceptance captures. A no-op still flushes; a failed flush has an unknown
245durability outcome.238durability outcome.
246239
247The filesystem adapter uses whole-file locking. On macOS it acquires the lock240The filesystem adapter uses whole-file locking. On macOS it acquires the lock
248as part of opening the file (`O_EXLOCK | O_NONBLOCK`): separate open and `flock`241as part of opening the file (`O_EXLOCK` to commit, `O_SHLOCK` to read, with
249calls allowed overlapping exclusive holders and stranded server locks under242`O_NONBLOCK`): separate open and `flock` calls allowed overlapping exclusive holders
250multi-process SMB contention. `tools/smb_lock_race.py` reproduces that failure243and stranded server locks under multi-process SMB contention.
251without notebook parsing or writing. On the tested macOS SMB mount,244`tools/smb_lock_race.py` reproduces that failure without notebook parsing or writing,
252POSIX byte-range locks returned `ENOTSUP`; whole-file locks excluded native245and with `--shared-reads` checks that shared readers never overlap a writer. On the
253OneNote's lock ranges. This adapter serializes readers and writers during each246tested macOS SMB mount, POSIX byte-range locks returned `ENOTSUP`, and `flock` of
254operation; it does not reproduce native reader/writer concurrency. The247either kind became an exclusive lock over the whole file, which fails OneNote's reads.
248smbfs sends `O_SHLOCK` and `O_EXLOCK` as share modes: a read excludes writers,
249OneNote's included, but not OneNote's readers; a commit excludes everyone. It does not
250reproduce native reader/writer concurrency. The
255[locking audit](../../evidence/LOCKING.md) records native coordination bytes, write-open share251[locking audit](../../evidence/LOCKING.md) records native coordination bytes, write-open share
256modes, and a working macOS SMB-specific byte-range lock probe. `sync_all` falls252modes, and a working macOS SMB-specific byte-range lock probe. `sync_all` falls
257back to `fsync` on macOS only when `F_FULLFSYNC` is unsupported. Successful SMB FLUSH replies were observed on the253back to `fsync` on macOS only when `F_FULLFSYNC` is unsupported. Successful SMB FLUSH replies were observed on the
crates/onestore/examples/concurrent_client.rs+11-1
...@@ -2,6 +2,8 @@...@@ -2,6 +2,8 @@
2mod concurrent;2mod concurrent;
3#[path = "../src/flush.rs"]3#[path = "../src/flush.rs"]
4mod flush;4mod flush;
5#[path = "support/typing.rs"]
6mod typing;
57
6use std::{env, fs, io::Write};8use std::{env, fs, io::Write};
79
...@@ -22,7 +24,15 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -22,7 +24,15 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
22 &args,24 &args,
23 |path| onestore::read_file(path),25 |path| onestore::read_file(path),
24 |path, source, space, object, range, replacement| {26 |path, source, space, object, range, replacement| {
25 onestore::commit_file_text(path, source, space, object, range, replacement)27 let edit = typing::text(space, object, range, replacement);
28 match typing::sealed(source, "Concurrent client", &edit) {
29 Ok(Some(transaction)) => transaction.commit_file(path),
30 Ok(None) => Ok(()),
31 Err(error) => Err(onestore::CommitError {
32 state: onestore::CommitState::NotCommitted,
33 error: std::io::Error::new(std::io::ErrorKind::InvalidData, error),
34 }),
35 }
26 },36 },
27 )37 )
28}38}
crates/onestore/examples/edit_property.rs+63-6
...@@ -1,9 +1,15 @@...@@ -1,9 +1,15 @@
1//! Changes one stored property value: a text object's text (`6000e 1c001c22`, UTF-16LE,
2//! a trailing NUL dropped) as a text op, the notebook's colour on the table of contents root
3//! (`20001 14001cbe`) as a TOC edit, each appending a revision; any other value of the same
4//! length only in a copy, patched where it is stored, as a fixture that is no edit.
5
1#[path = "../src/flush.rs"]6#[path = "../src/flush.rs"]
2mod flush;7mod flush;
8#[path = "support/typing.rs"]
9mod typing;
310
4use onestore::{11use onestore::{
5 ExGuid, ObjectData, PropertySets, RevisionIndex, Store, Value, commit_file_property,12 ExGuid, ObjectData, PropertySets, RevisionIndex, Store, TocEdit, Transaction, Value,
6 replace_property_bytes,
7};13};
8use std::{env, fs, io::Write};14use std::{env, fs, io::Write};
915
...@@ -59,16 +65,67 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -59,16 +65,67 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
59 }65 }
60 }66 }
61 let (space, object) = selected.ok_or("No matching object")?;67 let (space, object) = selected.ok_or("No matching object")?;
68 let transaction: Option<Transaction> = match (jcid, id) {
69 (0x6000e, 0x1c001c22) => {
70 let units = |bytes: &[u8]| -> Vec<u16> {
71 let mut units: Vec<u16> = bytes
72 .chunks_exact(2)
73 .map(|pair| u16::from_le_bytes([pair[0], pair[1]]))
74 .collect();
75 if units.last() == Some(&0) {
76 units.pop();
77 }
78 units
79 };
80 let length = u32::try_from(units(&expected).len())?;
81 let text = String::from_utf16(&units(&value))?;
82 let edit = typing::text(space, object, 0..length, &text);
83 typing::sealed(&bytes, "Property editor", &edit)?
84 }
85 (0x20001, 0x14001cbe) => {
86 if index.resolve_active(space)?.roots.get(&1) != Some(&object) {
87 return Err("Only the notebook's colour, on the TOC root, changes".into());
88 }
89 let color = u32::from_le_bytes(value.as_slice().try_into()?);
90 onestore::edit_table_of_contents(&bytes, &[TocEdit::Color(color)])?
91 }
92 _ if args[1] != "--in-place" && value.len() == expected.len() => {
93 let revision = index.resolve_active(space)?;
94 let ObjectData::Properties(data) = revision.objects[&object].data else {
95 return Err("The object stores no properties".into());
96 };
97 let properties = PropertySets::parse(data)?;
98 let Some(Value::Bytes(stored)) = properties.sets[0]
99 .iter()
100 .find(|property| property.id == id)
101 .map(|property| &property.value)
102 else {
103 return Err("The property stores no bytes".into());
104 };
105 let at = stored.as_ptr().addr() - bytes.as_ptr().addr();
106 let mut patched = bytes.clone();
107 patched[at..at + value.len()].copy_from_slice(&value);
108 write(&args[1], &patched)?;
109 return Ok(());
110 }
111 _ => return Err("Only text and TOC colours change as edits; other values are patched into a copy of equal length".into()),
112 };
113 let transaction = transaction.ok_or("The value is already stored")?;
62 if args[1] == "--in-place" {114 if args[1] == "--in-place" {
63 commit_file_property(&args[0], &bytes, space, object, id, &value)?;115 transaction.commit_file(&args[0])?;
64 return Ok(());116 return Ok(());
65 }117 }
66 let written = replace_property_bytes(&bytes, space, object, id, &value)?;118 let mut written = bytes.clone();
119 transaction.apply(&mut written)?;
120 write(&args[1], &written)
121}
122
123fn write(path: &str, written: &[u8]) -> Result<(), Box<dyn std::error::Error>> {
67 let mut output = fs::OpenOptions::new()124 let mut output = fs::OpenOptions::new()
68 .write(true)125 .write(true)
69 .create_new(true)126 .create_new(true)
70 .open(&args[1])?;127 .open(path)?;
71 output.write_all(&written)?;128 output.write_all(written)?;
72 flush::flush(&output)?;129 flush::flush(&output)?;
73 Ok(())130 Ok(())
74}131}
crates/onestore/examples/header_faults.rs+19-15
...@@ -1,24 +1,28 @@...@@ -1,24 +1,28 @@
1#[path = "support/typing.rs"]
2mod typing;
3
1use onestore::*;4use onestore::*;
2fn main() -> Result<(), Box<dyn std::error::Error>> {5fn main() -> Result<(), Box<dyn std::error::Error>> {
3 let source =6 let source =
4 std::fs::read("corpus/native/20260905-05/snapshots/02-text/notebook/synthetic.one")?;7 std::fs::read("corpus/native/20260905-05/snapshots/02-text/notebook/synthetic.one")?;
5 let store = Store::parse(&source)?;8 let store = Store::parse(&source)?;
6 let index = RevisionIndex::parse(&store)?;9 let index = RevisionIndex::parse(&store)?;
7 let mut target = None;10 let document = document::Document::parse(&index)?;
8 for (osid, space) in &index.spaces {11 let (osid, oid) = document
9 let revision = index.resolve(*osid, space.labels[&(ExGuid::default(), 1)])?;12 .spaces
10 for (oid, object) in revision.objects {13 .iter()
11 if let ObjectData::Properties(data) = object.data {14 .find_map(|(sid, space)| {
12 for p in &PropertySets::parse(data)?.sets[0] {15 space.active()?.nodes.iter().find_map(|(oid, node)| {
13 if p.value == Value::Bytes(b"Fictitious plain text.") {16 matches!(&node.kind, document::Kind::RichText { text, .. } if text == "Fictitious plain text.")
14 target = Some((*osid, oid, p.id));17 .then_some((*sid, *oid))
15 }18 })
16 }19 })
17 }20 .ok_or("Missing native fixture text")?;
18 }21 let end = "Fictitious plain text.".encode_utf16().count() as u32;
19 }22 let edit = typing::text(osid, oid, 0..end, "A durable edit.");
20 let (osid, oid, property) = target.unwrap();23 let transaction = typing::sealed(&source, "Header faults", &edit)?.ok_or("No change")?;
21 let output = replace_property_bytes(&source, osid, oid, property, b"A durable edit.")?;24 let mut output = source.clone();
25 transaction.apply(&mut output)?;
22 let mut invalid = Vec::new();26 let mut invalid = Vec::new();
23 for prefix in 0..=1024 {27 for prefix in 0..=1024 {
24 let mut torn = output.clone();28 let mut torn = output.clone();
crates/onestore/examples/insert.rs+85-24
...@@ -1,37 +1,96 @@...@@ -1,37 +1,96 @@
1#[path = "../src/flush.rs"]1#[path = "../src/flush.rs"]
2mod flush;2mod flush;
3#[path = "support/typing.rs"]
4mod typing;
35
4use onestore::{Insertion, PreparedEdit};6use onestore::{
7 document::{Format, Layout},
8 op::{Edit, Op, PageOp},
9 page::{Outline, PageObject, PageParagraph, Paragraph, ParagraphContent, TextObject, text::new_id},
10};
5use std::{fs, io::Write};11use std::{fs, io::Write};
612
13/// A paragraph holding `text` in OneNote's default text style.
14fn paragraph(text: &str) -> Result<PageParagraph, Box<dyn std::error::Error>> {
15 let format = Format {
16 font: Some("Calibri".into()),
17 font_size: Some(11.0),
18 language: Some(0x409),
19 ..Format::default()
20 };
21 Ok(PageParagraph {
22 id: new_id()?,
23 parent: None,
24 level: 1,
25 style: None,
26 format: Format::default(),
27 content: ParagraphContent::Text(TextObject {
28 id: new_id()?,
29 date_field: None,
30 text: Paragraph::new(text.into(), format),
31 tags: Vec::new(),
32 }),
33 lists: Vec::new(),
34 tags: Vec::new(),
35 media: Default::default(),
36 collapsed: false,
37 })
38}
39
7fn main() -> Result<(), Box<dyn std::error::Error>> {40fn main() -> Result<(), Box<dyn std::error::Error>> {
8 let args: Vec<_> = std::env::args().skip(1).collect();41 let args: Vec<_> = std::env::args().skip(1).collect();
9 let usage = "Usage: insert paragraph INPUT OUTPUT|--in-place SPACE PARENT BEFORE|- TEXT AUTHOR\n insert outline INPUT OUTPUT|--in-place SPACE PAGE X Y TEXT AUTHOR";42 let usage = "Usage: insert paragraph INPUT OUTPUT|--in-place SPACE CONTAINER BEFORE|- TEXT AUTHOR\n insert outline INPUT OUTPUT|--in-place SPACE X Y TEXT AUTHOR";
10 let insertion = match args.first().map(String::as_str) {43 let (op, object, text, author) = match args.first().map(String::as_str) {
11 Some("paragraph") if args.len() == 8 => Insertion::paragraph(44 Some("paragraph") if args.len() == 8 => {
12 args[4].parse()?,45 let paragraph = paragraph(&args[6])?;
13 if args[5] == "-" {46 let (object, text) = (paragraph.id, paragraph.text().unwrap().id);
14 None47 let op = PageOp::Insert {
15 } else {48 container: args[4].parse()?,
16 Some(args[5].parse()?)49 before: if args[5] == "-" {
17 },50 None
18 &args[6],51 } else {
19 &args[7],52 Some(args[5].parse()?)
20 )?,53 },
21 Some("outline") if args.len() == 9 => Insertion::outline(54 paragraphs: vec![paragraph],
22 args[4].parse()?,55 };
23 args[5].parse()?,56 (op, object, text, &args[7])
24 args[6].parse()?,57 }
25 &args[7],58 Some("outline") if args.len() == 8 => {
26 &args[8],59 let paragraph = paragraph(&args[6])?;
27 )?,60 let text = paragraph.text().unwrap().id;
61 let object = new_id()?;
62 let op = PageOp::Add {
63 object: PageObject::Outline(Outline {
64 id: object,
65 title: false,
66 min_width: None,
67 layout: Layout {
68 x: Some(args[4].parse()?),
69 y: Some(args[5].parse()?),
70 ..Layout::default()
71 },
72 indents: Vec::new(),
73 paragraphs: vec![paragraph],
74 unsupported: Vec::new(),
75 }),
76 before: None,
77 };
78 (op, object, text, &args[7])
79 }
28 _ => return Err(usage.into()),80 _ => return Err(usage.into()),
29 };81 };
30 let source = onestore::read_file(&args[1])?;82 let source = onestore::read_file(&args[1])?;
31 let prepared = PreparedEdit::insert(&source, args[3].parse()?, &insertion)?;83 let edit = Edit {
32 let mut record = serde_json::json!({"intent": insertion, "object": insertion.object(), "text_object": insertion.text_object()});84 at: typing::now(),
85 ops: vec![Op::Page {
86 space: args[3].parse()?,
87 op,
88 }],
89 };
90 let transaction = typing::sealed(&source, author, &edit)?.ok_or("The insertion stores nothing")?;
91 let mut record = serde_json::json!({"edit": edit, "object": object, "text_object": text});
33 if args[2] == "--in-place" {92 if args[2] == "--in-place" {
34 if let Err(error) = prepared.commit_file(&args[1]) {93 if let Err(error) = transaction.commit_file(&args[1]) {
35 record["state"] = format!("{:?}", error.state).into();94 record["state"] = format!("{:?}", error.state).into();
36 record["error"] = error.error.to_string().into();95 record["error"] = error.error.to_string().into();
37 println!("{record}");96 println!("{record}");
...@@ -39,11 +98,13 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -39,11 +98,13 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
39 }98 }
40 record["state"] = "Committed".into();99 record["state"] = "Committed".into();
41 } else {100 } else {
101 let mut written = source;
102 transaction.apply(&mut written)?;
42 let mut file = fs::OpenOptions::new()103 let mut file = fs::OpenOptions::new()
43 .write(true)104 .write(true)
44 .create_new(true)105 .create_new(true)
45 .open(&args[2])?;106 .open(&args[2])?;
46 file.write_all(prepared.as_bytes())?;107 file.write_all(&written)?;
47 flush::flush(&file)?;108 flush::flush(&file)?;
48 record["state"] = "Created".into();109 record["state"] = "Created".into();
49 }110 }
crates/onestore/examples/maintenance_fixture.rs+7-1
...@@ -1,3 +1,6 @@...@@ -1,3 +1,6 @@
1#[path = "support/typing.rs"]
2mod typing;
3
1use onestore::{4use onestore::{
2 ExGuid, RevisionIndex, Store,5 ExGuid, RevisionIndex, Store,
3 document::{Document, Kind},6 document::{Document, Kind},
...@@ -46,7 +49,10 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -46,7 +49,10 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
46 } else {49 } else {
47 format!("Revision {revision}: {}", "x".repeat(65536))50 format!("Revision {revision}: {}", "x".repeat(65536))
48 };51 };
49 bytes = onestore::replace_text(&bytes, sid, oid, 0..end, &text)?;52 let edit = typing::text(sid, oid, 0..end, &text);
53 if let Some(transaction) = typing::sealed(&bytes, "Fixture", &edit)? {
54 transaction.apply(&mut bytes)?;
55 }
50 }56 }
51 let store = Store::parse(&bytes)?;57 let store = Store::parse(&bytes)?;
52 let toc = onestore::create_table_of_contents(58 let toc = onestore::create_table_of_contents(
crates/onestore/examples/power_loss.rs+49-36
...@@ -8,6 +8,8 @@ use trace::{Event, Trace};...@@ -8,6 +8,8 @@ use trace::{Event, Trace};
88
9#[path = "../tests/support/checkpoint.rs"]9#[path = "../tests/support/checkpoint.rs"]
10mod checkpoint;10mod checkpoint;
11#[path = "support/typing.rs"]
12mod typing;
1113
12use onestore::{14use onestore::{
13 ExGuid, RevisionIndex, Store,15 ExGuid, RevisionIndex, Store,
...@@ -15,6 +17,16 @@ use onestore::{...@@ -15,6 +17,16 @@ use onestore::{
15};17};
16use std::{collections::BTreeMap, fs, path::PathBuf};18use std::{collections::BTreeMap, fs, path::PathBuf};
1719
20/// `source`, a table of contents, with the notebook recoloured to `color`.
21fn recolored(source: &[u8], color: u32) -> Result<Vec<u8>, onestore::Error> {
22 let mut image = source.to_vec();
23 let edit = onestore::TocEdit::Color(color);
24 if let Some(transaction) = onestore::edit_table_of_contents(source, &[edit])? {
25 transaction.apply(&mut image)?;
26 }
27 Ok(image)
28}
29
18fn main() -> Result<(), Box<dyn std::error::Error>> {30fn main() -> Result<(), Box<dyn std::error::Error>> {
19 let mut args = std::env::args_os().skip(1);31 let mut args = std::env::args_os().skip(1);
20 let destination = PathBuf::from(args.next().ok_or("Provide a new evidence directory")?);32 let destination = PathBuf::from(args.next().ok_or("Provide a new evidence directory")?);
...@@ -69,12 +81,11 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -69,12 +81,11 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
69 let index = RevisionIndex::parse(&store)?;81 let index = RevisionIndex::parse(&store)?;
70 let document = Document::parse(&index)?;82 let document = Document::parse(&index)?;
71 let toc = store.header.file_type == onestore::FileType::TableOfContents;83 let toc = store.header.file_type == onestore::FileType::TableOfContents;
84 // A table of contents changes the notebook's colour.
72 let (sid, oid) = if toc {85 let (sid, oid) = if toc {
73 let root = &document.spaces[&document.root];86 let root = &document.spaces[&document.root];
74 (87 let root = &root.revisions[&root.contexts[&ExGuid::default()]];
75 document.root,88 (document.root, root.roots[&1])
76 root.revisions[&root.contexts[&ExGuid::default()]].roots[&1],
77 )
78 } else {89 } else {
79 document90 document
80 .spaces91 .spaces
...@@ -95,37 +106,43 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -95,37 +106,43 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
95 })106 })
96 .ok_or("Missing native fixture text")?107 .ok_or("Missing native fixture text")?
97 };108 };
98 if name.ends_with("checkpoint") {109 match name {
99 source =110 "checkpoint" => source = checkpoint::pending(&source, sid, oid),
100 checkpoint::pending(&source, sid, oid, if toc { 0x14001cbe } else { 0x14001d7a });111 "toc-checkpoint" => {
101 }112 for value in 0.. {
102 if name == "toc-rollover" {113 let store = Store::parse(&source)?;
103 loop {114 let index = RevisionIndex::parse(&store)?;
115 let rid = index.spaces[&sid].labels[&(ExGuid::default(), 1)];
116 let depth = std::iter::successors(Some(rid), |id| {
117 index.spaces[&sid].revisions[id].dependency
118 })
119 .count();
120 if depth == 512 {
121 break;
122 }
123 source = recolored(&source, value)?;
124 }
125 }
126 "toc-rollover" => loop {
104 let count = Store::parse(&source)?.header.transaction_count;127 let count = Store::parse(&source)?.header.transaction_count;
105 if count == 255 {128 if count == 255 {
106 break;129 break;
107 }130 }
108 assert!(count < 255);131 assert!(count < 255);
109 source = onestore::replace_property_bytes(132 source = recolored(&source, count)?;
110 &source,133 },
111 sid,134 _ => {}
112 oid,
113 0x14001cbe,
114 &count.to_le_bytes(),
115 )?;
116 }
117 }135 }
118 let apply = |io: &mut Trace,136 let apply = |io: &mut Trace,
119 bytes: &[u8],137 bytes: &[u8],
120 recovering: bool|138 recovering: bool|
121 -> Result<(), Box<dyn std::error::Error>> {139 -> Result<(), Box<dyn std::error::Error>> {
122 if toc {140 if toc {
123 let value = if recovering {141 let color = if recovering { 0x665544 } else { 0x996633 };
124 [0x44, 0x55, 0x66, 0]142 let edit = onestore::TocEdit::Color(color);
125 } else {143 let transaction =
126 [0x33, 0x66, 0x99, 0]144 onestore::edit_table_of_contents(bytes, &[edit])?.ok_or("No change")?;
127 };145 transaction.commit(io)?;
128 onestore::commit_property_bytes(io, bytes, sid, oid, 0x14001cbe, &value)?;
129 } else {146 } else {
130 let store = Store::parse(bytes)?;147 let store = Store::parse(bytes)?;
131 let index = RevisionIndex::parse(&store)?;148 let index = RevisionIndex::parse(&store)?;
...@@ -136,18 +153,14 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -136,18 +153,14 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
136 unreachable!()153 unreachable!()
137 };154 };
138 let end = u32::try_from(text.encode_utf16().count())?;155 let end = u32::try_from(text.encode_utf16().count())?;
139 onestore::commit_text(156 let with = if recovering {
140 io,157 " [recovered]"
141 bytes,158 } else {
142 sid,159 " [durable café 🦀]"
143 oid,160 };
144 end..end,161 let edit = typing::text(sid, oid, end..end, with);
145 if recovering {162 let transaction = typing::sealed(bytes, "Power loss", &edit)?.ok_or("No change")?;
146 " [recovered]"163 transaction.commit(io)?;
147 } else {
148 " [durable café 🦀]"
149 },
150 )?;
151 }164 }
152 Ok(())165 Ok(())
153 };166 };
crates/onestore/examples/random_edit.rs+12-3
...@@ -1,5 +1,7 @@...@@ -1,5 +1,7 @@
1#[path = "../src/flush.rs"]1#[path = "../src/flush.rs"]
2mod flush;2mod flush;
3#[path = "support/typing.rs"]
4mod typing;
35
4use onestore::{6use onestore::{
5 ExGuid, RevisionIndex, Store,7 ExGuid, RevisionIndex, Store,
...@@ -111,13 +113,20 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -111,13 +113,20 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
111 end113 end
112 };114 };
113 let range = offset + boundaries[start]..offset + boundaries[end];115 let range = offset + boundaries[start]..offset + boundaries[end];
114 let written = match onestore::replace_text(&source, sid, oid, range.clone(), replacement) {116 let edit = typing::text(sid, oid, range.clone(), replacement);
115 Ok(written) => written,117 let transaction = match typing::sealed(&source, "Random editor", &edit) {
118 Ok(Some(transaction)) => transaction,
119 Ok(None) => {
120 rejected.push("The edit stores nothing".into());
121 continue;
122 }
116 Err(error) => {123 Err(error) => {
117 rejected.push(error.to_string());124 rejected.push(error.to_string());
118 continue;125 continue;
119 }126 }
120 };127 };
128 let mut written = source.clone();
129 transaction.apply(&mut written)?;
121 let space = &document.spaces[&sid];130 let space = &document.spaces[&sid];
122 let revision = &space.revisions[&space.contexts[&ExGuid::default()]];131 let revision = &space.revisions[&space.contexts[&ExGuid::default()]];
123 let Kind::RichText { runs, .. } = &revision.nodes[&oid].kind else {132 let Kind::RichText { runs, .. } = &revision.nodes[&oid].kind else {
...@@ -133,7 +142,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -133,7 +142,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
133 "run_before": actual[selected].text, "run_start": runs[selected].start, "source_md5": format!("{:x}", md5::compute(&source)),142 "run_before": actual[selected].text, "run_start": runs[selected].start, "source_md5": format!("{:x}", md5::compute(&source)),
134 "started_ms": started, "rejected_candidates": rejected});143 "started_ms": started, "rejected_candidates": rejected});
135 let failed = if args[1] == "--in-place" {144 let failed = if args[1] == "--in-place" {
136 match onestore::commit_file_text(&args[0], &source, sid, oid, range, replacement) {145 match transaction.commit_file(&args[0]) {
137 Ok(()) => {146 Ok(()) => {
138 record["state"] = "Committed".into();147 record["state"] = "Committed".into();
139 false148 false
crates/onestore/examples/scratch_dump.rs created+29
...@@ -0,0 +1,29 @@
1use onestore::{ObjectData, PropertySets, RevisionIndex, Store, Value, document::Document};
2fn main() -> Result<(), Box<dyn std::error::Error>> {
3 let args: Vec<String> = std::env::args().skip(1).collect();
4 let bytes = std::fs::read(&args[0])?;
5 let store = Store::parse(&bytes)?;
6 let index = RevisionIndex::parse(&store)?;
7 let pages = Document::parse(&index).and_then(|document| document.pages()).unwrap_or_default();
8 let space = if args[1] == "root" { index.root } else { pages[args[1].parse::<usize>()?].0 };
9 println!("space {space}; pages {:?}", pages.iter().map(|p| p.0).collect::<Vec<_>>());
10 let raw = index.resolve_active(space)?;
11 for (id, object) in &raw.objects {
12 println!("{id} jcid {:#x}", object.jcid);
13 if let ObjectData::Properties(data) = object.data {
14 let sets = PropertySets::parse(data)?;
15 for (n, set) in sets.sets.iter().enumerate() {
16 for p in set {
17 let v = match &p.value {
18 Value::NoData => "-".to_string(),
19 Value::Bytes(b) => b.iter().map(|x| format!("{x:02x}")).collect(),
20 Value::References { stream, compact_ids } => format!("{stream:?} {}", compact_ids.iter().map(|x| format!("{x:02x}")).collect::<String>()),
21 Value::Sets(r) => format!("sets {r:?}"),
22 };
23 println!(" [{n}] {:#010x} {v}", p.id);
24 }
25 }
26 }
27 }
28 Ok(())
29}
crates/onestore/examples/support/concurrent.rs+110-32
...@@ -318,7 +318,46 @@ pub fn run(...@@ -318,7 +318,46 @@ pub fn run(
318#[cfg(test)]318#[cfg(test)]
319mod tests {319mod tests {
320 use super::*;320 use super::*;
321 use onestore::{Insertion, ParagraphJoin, ParagraphSplit, PreparedEdit, TextAttribute};321 use onestore::{
322 TextAttribute,
323 document::{Format, Layout},
324 op::{Edit, Op, PageOp},
325 page::{Outline, PageObject, PageParagraph, Paragraph, ParagraphContent, TextObject, text::new_id},
326 };
327
328 fn paragraph(text: &str, level: u32) -> PageParagraph {
329 PageParagraph {
330 id: new_id().unwrap(),
331 parent: None,
332 level,
333 style: None,
334 format: Format::default(),
335 content: ParagraphContent::Text(TextObject {
336 id: new_id().unwrap(),
337 date_field: None,
338 text: Paragraph::new(text.into(), Format {
339 font: Some("Calibri".into()),
340 font_size: Some(11.0),
341 language: Some(0x409),
342 ..Format::default()
343 }),
344 tags: Vec::new(),
345 }),
346 lists: Vec::new(),
347 tags: Vec::new(),
348 media: Default::default(),
349 collapsed: false,
350 }
351 }
352
353 fn edited(image: &[u8], space: ExGuid, ops: Vec<PageOp>) -> Vec<u8> {
354 let arena = onestore::Arena::default();
355 let mut section = onestore::Section::open(&arena, image.to_vec()).unwrap();
356 let ops = ops.into_iter().map(|op| Op::Page { space, op }).collect();
357 section.apply("Author", &Edit { at: 134_000_000_000_000_000, ops }).unwrap();
358 section.seal().unwrap();
359 section.image()
360 }
322361
323 #[test]362 #[test]
324 fn observation_follows_split_suffixes_and_moved_children_through_active_ancestry() {363 fn observation_follows_split_suffixes_and_moved_children_through_active_ancestry() {
...@@ -328,57 +367,96 @@ mod tests {...@@ -328,57 +367,96 @@ mod tests {
328 let index = RevisionIndex::parse(&store).unwrap();367 let index = RevisionIndex::parse(&store).unwrap();
329 let document = Document::parse(&index).unwrap();368 let document = Document::parse(&index).unwrap();
330 let (space, page) = document.pages().unwrap()[0];369 let (space, page) = document.pages().unwrap()[0];
331 let insertion = Insertion::outline(page, 144.0, 216.0, "Document w0:0 🦀", "Author")370 let first = paragraph("Document w0:0 🦀", 1);
332 .unwrap()371 let (paragraph_id, text) = (first.id, first.text().unwrap().id);
333 .with_formatting(0..16, &[TextAttribute::Bold(true)])372 let outline_id = new_id().unwrap();
334 .unwrap();373 let inserted = edited(
335 let inserted = PreparedEdit::insert(&source, space, &insertion).unwrap();374 &source,
336 let before = document_view(inserted.as_bytes()).unwrap();375 space,
376 vec![
377 PageOp::Add {
378 object: PageObject::Outline(Outline {
379 id: outline_id,
380 title: false,
381 min_width: None,
382 layout: Layout {
383 x: Some(144.0),
384 y: Some(216.0),
385 ..Default::default()
386 },
387 indents: Vec::new(),
388 paragraphs: vec![first],
389 unsupported: Vec::new(),
390 }),
391 before: None,
392 },
393 PageOp::Format {
394 text,
395 range: 0..16,
396 set: vec![TextAttribute::Bold(true)],
397 clear: Vec::new(),
398 },
399 ],
400 );
401 let before = document_view(&inserted).unwrap();
337 assert_eq!(before.texts.len(), 1);402 assert_eq!(before.texts.len(), 1);
338 assert_eq!(before.graph.len(), 2);403 assert_eq!(before.graph.len(), 2);
339 let outline = insertion.object().to_string();404 let outline = outline_id.to_string();
340 let paragraph = before.graph[&outline]["children"][0]405 let paragraph_name = paragraph_id.to_string();
341 .as_str()406 assert_eq!(before.graph[&outline]["children"], json!([paragraph_name]));
342 .unwrap()407 let child = paragraph("Unmarked child", 2);
343 .to_owned();408 let child_id = child.id;
344 let child =409 let with_child = edited(
345 Insertion::paragraph(paragraph.parse().unwrap(), None, "Unmarked child", "Author")410 &inserted,
346 .unwrap();411 space,
347 let with_child = PreparedEdit::insert(inserted.as_bytes(), space, &child).unwrap();412 vec![PageOp::Insert {
348 let before = document_view(with_child.as_bytes()).unwrap();413 container: paragraph_id,
414 before: None,
415 paragraphs: vec![child],
416 }],
417 );
418 let before = document_view(&with_child).unwrap();
349 assert_eq!(before.texts.len(), 2);419 assert_eq!(before.texts.len(), 2);
350 assert_eq!(before.graph.len(), 3);420 assert_eq!(before.graph.len(), 3);
351 assert_eq!(421 assert_eq!(
352 before.graph[&outline],422 before.graph[&outline],
353 json!({"parent":page.to_string(), "children":[paragraph], "content":[], "child_level":1, "position":{"x":144.0,"y":216.0}})423 json!({"parent":page.to_string(), "children":[paragraph_name], "content":[], "child_level":1, "position":{"x":144.0,"y":216.0}})
354 );424 );
355 for offset in [14, 16] {425 for offset in [14, 16] {
356 let split = ParagraphSplit::new(insertion.text_object(), offset, "Author").unwrap();426 let (split, right) = (new_id().unwrap(), new_id().unwrap());
357 let split_edit = PreparedEdit::split(with_child.as_bytes(), space, &split).unwrap();427 let split_edit = edited(
358 let observed = document_view(split_edit.as_bytes()).unwrap();428 &with_child,
429 space,
430 vec![PageOp::Split {
431 text,
432 at: offset,
433 paragraph: split,
434 right,
435 lists: Vec::new(),
436 }],
437 );
438 let observed = document_view(&split_edit).unwrap();
359 assert_eq!(observed.texts.len(), 3);439 assert_eq!(observed.texts.len(), 3);
360 assert_eq!(observed.graph.len(), 4);440 assert_eq!(observed.graph.len(), 4);
361 assert_eq!(441 assert_eq!(
362 observed.texts[&split.text_object().to_string()]["text"],442 observed.texts[&right.to_string()]["text"],
363 if offset == 14 { "🦀" } else { "" }443 if offset == 14 { "🦀" } else { "" }
364 );444 );
365 assert_eq!(445 assert_eq!(
366 observed.graph[&outline]["children"],446 observed.graph[&outline]["children"],
367 json!([paragraph, split.object().to_string()])447 json!([paragraph_name, split.to_string()])
368 );448 );
369 assert_eq!(observed.graph[&paragraph]["children"], json!([]));449 assert_eq!(observed.graph[&paragraph_name]["children"], json!([]));
370 assert_eq!(450 assert_eq!(
371 observed.graph[&split.object().to_string()]["children"],451 observed.graph[&split.to_string()]["children"],
372 json!([child.object().to_string()])452 json!([child_id.to_string()])
373 );453 );
374 assert_eq!(454 assert_eq!(
375 observed.graph[&child.object().to_string()]["parent"],455 observed.graph[&child_id.to_string()]["parent"],
376 split.object().to_string()456 split.to_string()
377 );457 );
378 let join =458 let joined = edited(&split_edit, space, vec![PageOp::Join { left: text, right }]);
379 ParagraphJoin::new(insertion.text_object(), split.text_object(), "Author").unwrap();459 let joined = document_view(&joined).unwrap();
380 let joined = PreparedEdit::join(split_edit.as_bytes(), space, &join).unwrap();
381 let joined = document_view(joined.as_bytes()).unwrap();
382 assert_eq!(joined.graph, before.graph);460 assert_eq!(joined.graph, before.graph);
383 let characters = |view: &DocumentView| {461 let characters = |view: &DocumentView| {
384 view.texts462 view.texts
crates/onestore/examples/support/typing.rs created+41
...@@ -0,0 +1,41 @@
1//! Text typed as ops through a `Section`, for the examples that edit a section file.
2#![allow(dead_code)]
3
4use onestore::{
5 Arena, ExGuid, Section, Transaction,
6 op::{Edit, Op, OpError, PageOp},
7};
8use std::{
9 ops::Range,
10 time::{SystemTime, UNIX_EPOCH},
11};
12
13/// FILETIME now, as an edit's `at`.
14pub fn now() -> u64 {
15 let unix = SystemTime::now()
16 .duration_since(UNIX_EPOCH)
17 .unwrap_or_default();
18 (unix.as_secs() + 11_644_473_600) * 10_000_000 + u64::from(unix.subsec_nanos() / 100)
19}
20
21/// The op replacing `range` of the text object `text` in `space` with `with`.
22pub fn text(space: ExGuid, text: ExGuid, range: Range<u32>, with: &str) -> Edit {
23 let op = PageOp::Text {
24 text,
25 range,
26 with: with.to_owned(),
27 };
28 Edit {
29 at: now(),
30 ops: vec![Op::Page { space, op }],
31 }
32}
33
34/// The transaction `edit` seals into on `image`, as `author` made it; none when it stores
35/// nothing.
36pub fn sealed(image: &[u8], author: &str, edit: &Edit) -> Result<Option<Transaction>, OpError> {
37 let arena = Arena::default();
38 let mut section = Section::open(&arena, image.to_vec()).map_err(OpError::Failed)?;
39 section.apply(author, edit)?;
40 section.seal().map_err(OpError::Failed)
41}
crates/onestore/src/active.rs+40-22
...@@ -1,7 +1,13 @@...@@ -1,7 +1,13 @@
1use crate::{1use crate::{
2 Error, ExGuid, FileType, Object, RevisionIndex, Store,2 Error, ExGuid, FileType, Object,
3 document::{Document, Element, Kind, Revision, Space},3 document::{Element, Kind, Revision},
4 write::{Commit, LiveRevision, PropertyObject, chain_depth, declared, differing},4 write::{Commit, LiveRevision, PropertyObject, declared, differing},
5};
6#[cfg(test)]
7use crate::{
8 RevisionIndex, Store,
9 document::{Document, Space},
10 write::chain_depth,
5};11};
6use bumpalo::Bump;12use bumpalo::Bump;
7use std::{13use std::{
...@@ -35,6 +41,8 @@ pub(crate) struct ActivePage<'a> {...@@ -35,6 +41,8 @@ pub(crate) struct ActivePage<'a> {
35}41}
3642
37impl<'a> ActivePage<'a> {43impl<'a> ActivePage<'a> {
44 /// The active page of `space`, for tests comparing writers.
45 #[cfg(test)]
38 pub(crate) fn parse(index: &'a RevisionIndex<'a>, space: ExGuid) -> Result<Self> {46 pub(crate) fn parse(index: &'a RevisionIndex<'a>, space: ExGuid) -> Result<Self> {
39 index.validate_current()?;47 index.validate_current()?;
40 let mut document = Document::parse(index)?;48 let mut document = Document::parse(index)?;
...@@ -131,6 +139,7 @@ impl<'a> ActivePage<'a> {...@@ -131,6 +139,7 @@ impl<'a> ActivePage<'a> {
131139
132 /// Stores a writer's objects and payloads as the next revision; false when it stores140 /// Stores a writer's objects and payloads as the next revision; false when it stores
133 /// nothing, as appending it would leave the image unchanged.141 /// nothing, as appending it would leave the image unchanged.
142 #[cfg(test)]
134 pub(crate) fn write(143 pub(crate) fn write(
135 &mut self,144 &mut self,
136 arena: &'a Bump,145 arena: &'a Bump,
...@@ -328,26 +337,19 @@ fn is_title(node: &Element<'_>) -> bool {...@@ -328,26 +337,19 @@ fn is_title(node: &Element<'_>) -> bool {
328 .is_some_and(|fields| fields.iter().any(|field| field.id == 0x88001cb4))337 .is_some_and(|fields| fields.iter().any(|field| field.id == 0x88001cb4))
329}338}
330339
331/// Writes one revision of a page space: `changes` computes its objects from the active page.
332pub(crate) fn write(
333 source: &[u8],
334 space: ExGuid,
335 changes: impl FnOnce(&ActivePage<'_>) -> Result<Changes>,
336) -> Result<Vec<u8>> {
337 let store = Store::parse(source)?;
338 let index = RevisionIndex::parse(&store)?;
339 let changes = changes(&ActivePage::parse(&index, space)?)?;
340 if changes.is_empty() {
341 return Ok(source.to_vec());
342 }
343 crate::write::write_revision_on(&index, space, |_| Ok(changes))
344}
345
346#[cfg(test)]340#[cfg(test)]
347pub(crate) mod tests {341pub(crate) mod tests {
348 use super::*;342 use super::*;
349 use crate::{Insertion, ParagraphSplit, TextAttribute, TreeEdit, document::Kind};343 use crate::{Insertion, ParagraphSplit, TextAttribute, TreeEdit, document::Kind};
350344
345 /// An identity from `fresh_guid`, which seeded tests draw deterministically per thread.
346 fn new_id() -> Result<ExGuid> {
347 Ok(ExGuid {
348 guid: crate::write::fresh_guid()?,
349 n: 1,
350 })
351 }
352
351 /// A seeded source of the typed writers' changes: insertions, moves, deletions, splits353 /// A seeded source of the typed writers' changes: insertions, moves, deletions, splits
352 /// and formatting of what a page holds.354 /// and formatting of what a page holds.
353 pub(crate) struct Writes(pub u64);355 pub(crate) struct Writes(pub u64);
...@@ -392,15 +394,31 @@ pub(crate) mod tests {...@@ -392,15 +394,31 @@ pub(crate) mod tests {
392 let text = *texts.get(self.pick(texts.len())).ok_or(unavailable)?;394 let text = *texts.get(self.pick(texts.len())).ok_or(unavailable)?;
393 let words = ["", "a", "Two words", "東京 🦀", "longer text here"];395 let words = ["", "a", "Two words", "東京 🦀", "longer text here"];
394 let word = words[self.pick(words.len())];396 let word = words[self.pick(words.len())];
395 match self.pick(8) {397 // `Section::apply` refuses an edit ending with a table cell emptied.
396 0..=3 => Insertion::paragraph(container, anchor, word, "Author")398 let fills_a_cell = active.parents[&paragraph].iter().any(|parent| {
397 .and_then(|insertion| insertion.changes(active)),399 matches!(nodes[parent].kind, Kind::Cell { .. })
400 && nodes[parent].children == [paragraph]
401 });
402 let choice = match self.pick(8) {
403 4 | 5 if fills_a_cell => 0,
404 choice => choice,
405 };
406 match choice {
407 0..=3 => Insertion::paragraph(container, anchor, word, "Author").and_then(
408 |insertion| {
409 let paragraph = new_id()?;
410 insertion.changes_as(active, paragraph, paragraph, new_id()?)
411 },
412 ),
398 4 => TreeEdit::move_to(paragraph, container, anchor, "Author")413 4 => TreeEdit::move_to(paragraph, container, anchor, "Author")
399 .and_then(|edit| edit.changes(active)),414 .and_then(|edit| edit.changes(active)),
400 5 => TreeEdit::delete(paragraph, "Author").and_then(|edit| edit.changes(active)),415 5 => TreeEdit::delete(paragraph, "Author").and_then(|edit| edit.changes(active)),
401 6 => {416 6 => {
402 let at = self.pick(3) as u32;417 let at = self.pick(3) as u32;
403 ParagraphSplit::new(text, at, "Author").and_then(|split| split.changes(active))418 let lists = (0..8).map(|_| new_id()).collect::<Result<Vec<_>>>()?;
419 ParagraphSplit::new(text, at, "Author").and_then(|split| {
420 split.changes_as(active, new_id()?, new_id()?, &lists)
421 })
404 }422 }
405 _ => {423 _ => {
406 let end = self.pick(2) as u32;424 let end = self.pick(2) as u32;
crates/onestore/src/commit.rs+20-313
...@@ -1,4 +1,3 @@...@@ -1,4 +1,3 @@
1use crate::{ExGuid, replace_property_bytes};
2use std::io::{self, ErrorKind};1use std::io::{self, ErrorKind};
32
4#[cfg(any(unix, windows))]3#[cfg(any(unix, windows))]
...@@ -33,8 +32,14 @@ impl FileIo {...@@ -33,8 +32,14 @@ impl FileIo {
33 #[cfg(target_os = "macos")]32 #[cfg(target_os = "macos")]
34 {33 {
35 use std::os::unix::fs::OpenOptionsExt;34 use std::os::unix::fs::OpenOptionsExt;
36 // SMB can lose exclusion when separate opens race with flock.35 // SMB can lose exclusion when separate opens race with flock. On smbfs these are
37 options.custom_flags(nix::libc::O_EXLOCK | nix::libc::O_NONBLOCK);36 // share modes: a shared reader denies only writers, so OneNote's readers proceed.
37 let lock = if write {
38 nix::libc::O_EXLOCK
39 } else {
40 nix::libc::O_SHLOCK
41 };
42 options.custom_flags(lock | nix::libc::O_NONBLOCK);
38 }43 }
39 let file = options.open(path)?;44 let file = options.open(path)?;
40 #[cfg(not(target_os = "macos"))]45 #[cfg(not(target_os = "macos"))]
...@@ -100,14 +105,14 @@ pub fn place_file(path: impl AsRef<Path>, ancestor: [u8; 16], name: &str) -> io:...@@ -100,14 +105,14 @@ pub fn place_file(path: impl AsRef<Path>, ancestor: [u8; 16], name: &str) -> io:
100 released105 released
101}106}
102107
103/// Reads a snapshot under the same whole-file exclusion used for commits.108/// Reads a snapshot excluding writers, as commits exclude everyone (macOS shares it with
104/// Native writers can expose incomplete graphs to unlocked filesystem reads.109/// other readers). Native writers can expose incomplete graphs to unlocked filesystem reads.
105#[cfg(any(unix, windows))]110#[cfg(any(unix, windows))]
106pub fn read_file(path: impl AsRef<Path>) -> io::Result<Vec<u8>> {111pub fn read_file(path: impl AsRef<Path>) -> io::Result<Vec<u8>> {
107 read_file_limited(path, usize::MAX)112 read_file_limited(path, usize::MAX)
108}113}
109114
110/// Reads under whole-file exclusion, rejecting a snapshot larger than the byte limit.115/// `read_file`, rejecting a snapshot larger than the byte limit.
111/// A size failure returns `FileTooLarge` without a partial snapshot.116/// A size failure returns `FileTooLarge` without a partial snapshot.
112#[cfg(any(unix, windows))]117#[cfg(any(unix, windows))]
113pub fn read_file_limited(path: impl AsRef<Path>, limit: usize) -> io::Result<Vec<u8>> {118pub fn read_file_limited(path: impl AsRef<Path>, limit: usize) -> io::Result<Vec<u8>> {
...@@ -147,278 +152,25 @@ impl CommitIo for FileIo {...@@ -147,278 +152,25 @@ impl CommitIo for FileIo {
147 }152 }
148}153}
149154
150/// Commits under an exclusive whole-file lock; contention returns WouldBlock.155/// `confirm` under the same whole-file exclusion `commit_file` uses.
151/// Calls are serialized within this process because macOS SMB locks are reentrant.
152/// Durability relies on the filesystem and server honoring their flush contract.
153/// A changed snapshot returns ResourceBusy without publishing the edit.
154#[cfg(any(unix, windows))]156#[cfg(any(unix, windows))]
155pub fn commit_file_property(157pub fn confirm_file(path: impl AsRef<Path>, base: &Stamp) -> Result<(), CommitError> {
156 path: impl AsRef<Path>,
157 source: &[u8],
158 space: ExGuid,
159 object: ExGuid,
160 property: u32,
161 value: &[u8],
162) -> Result<(), CommitError> {
163 let mut io = FileIo::open(path, true).map_err(|error| CommitError {158 let mut io = FileIo::open(path, true).map_err(|error| CommitError {
164 state: CommitState::NotCommitted,159 state: CommitState::NotCommitted,
165 error,160 error,
166 })?;161 })?;
167 let result = commit_property_bytes(&mut io, source, space, object, property, value);162 let result = confirm(&mut io, base);
168 io.finish(result)163 io.finish(result)
169}164}
170165
171/// Commits a text edit with the same exclusion and snapshot check as scalar edits.166/// Checks that the file still has `base`'s stamp and flushes it, then refreshes its header
172#[cfg(any(unix, windows))]167/// version metadata. No revision is added; reread before committing on `base` again.
173pub fn commit_file_text(
174 path: impl AsRef<Path>,
175 source: &[u8],
176 space: ExGuid,
177 object: ExGuid,
178 range: std::ops::Range<u32>,
179 replacement: &str,
180) -> Result<(), CommitError> {
181 let mut io = FileIo::open(path, true).map_err(|error| CommitError {
182 state: CommitState::NotCommitted,
183 error,
184 })?;
185 let result = commit_text(&mut io, source, space, object, range, replacement);
186 io.finish(result)
187}
188
189/// Atomically publishes text and its dependent run boundaries under caller-held exclusion.
190pub fn commit_text(
191 io: &mut impl CommitIo,
192 source: &[u8],
193 space: ExGuid,
194 object: ExGuid,
195 range: std::ops::Range<u32>,
196 replacement: &str,
197) -> Result<(), CommitError> {
198 PreparedEdit::text(source, space, object, range, replacement)
199 .map_err(|error| CommitError {
200 state: CommitState::NotCommitted,
201 error: io::Error::new(ErrorKind::InvalidData, error),
202 })?
203 .commit(io)
204}
205
206/// An immutable writer-generated transition tied to its original snapshot.
207/// Persist intended revision identities from `as_bytes` before publishing an offline edit.
208/// Missing identities after native maintenance do not prove an edit was never published.
209pub struct PreparedEdit<'a> {
210 source: &'a [u8],
211 written: Vec<u8>,
212}
213
214impl<'a> PreparedEdit<'a> {
215 /// Applies page moves in slice order and publishes final order and indentation atomically.
216 /// Each page occurs once; all pages to move must be explicit, including selected subpages.
217 pub fn pages(source: &'a [u8], edits: &[crate::PageEdit]) -> Result<Self, crate::Error> {
218 Ok(Self {
219 source,
220 written: crate::pages::edit_pages(source, edits, &[])?,
221 })
222 }
223
224 /// Applies table-of-contents edits (sections and section groups: add, rename, colour,
225 /// order, remove) as one revision of a `.onetoc2` file.
226 pub fn table_of_contents(
227 source: &'a [u8],
228 edits: &[crate::TocEdit],
229 ) -> Result<Self, crate::Error> {
230 Ok(Self {
231 source,
232 written: crate::toc::edit_table_of_contents(source, edits)?,
233 })
234 }
235
236 /// Permanently removes explicitly selected pages from the section in one transaction.
237 /// Subpages must be selected explicitly; a surviving first subpage becomes top-level.
238 /// Creates no recycle-bin copies. Prior revisions remain stored; this is not secure erasure.
239 pub fn delete_pages_permanently(
240 source: &'a [u8],
241 pages: &[crate::ExGuid],
242 ) -> Result<Self, crate::Error> {
243 Ok(Self {
244 source,
245 written: crate::pages::edit_pages(source, &[], pages)?,
246 })
247 }
248
249 /// Publishes an edited page model as one revision per changed space.
250 /// The model must come from this snapshot; new paragraphs and outlines carry the identities
251 /// the model assigned, and content outside the model is untouched.
252 pub fn page(
253 source: &'a [u8],
254 space: ExGuid,
255 page: &crate::page::Page,
256 author: &str,
257 ) -> Result<Self, crate::Error> {
258 Ok(Self {
259 source,
260 written: crate::page::write::write_page(source, space, page, author)?,
261 })
262 }
263
264 /// `page` for a password-protected section: the revision is stored under the section's
265 /// key. The model must come from this snapshot unlocked with `password`.
266 #[cfg(feature = "protected")]
267 pub fn page_protected(
268 source: &'a [u8],
269 password: &str,
270 space: ExGuid,
271 page: &crate::page::Page,
272 author: &str,
273 ) -> Result<Self, crate::protected::Error> {
274 Ok(Self {
275 source,
276 written: crate::protected::write_page(source, password, space, page, author)?,
277 })
278 }
279
280 /// Creates a page and its section entry in one transaction, retaining the intent's identities.
281 pub fn create_page(source: &'a [u8], page: &crate::PageCreation) -> Result<Self, crate::Error> {
282 Ok(Self {
283 source,
284 written: page.apply(source)?,
285 })
286 }
287
288 /// Moves or removes a subtree and normalizes its containers in one revision.
289 pub fn tree(
290 source: &'a [u8],
291 space: ExGuid,
292 edit: &crate::TreeEdit,
293 ) -> Result<Self, crate::Error> {
294 Ok(Self {
295 source,
296 written: edit.apply(source, space)?,
297 })
298 }
299
300 /// Changes outline geometry or a paragraph's saved expansion state, preserving content.
301 pub fn outline(
302 source: &'a [u8],
303 space: ExGuid,
304 object: ExGuid,
305 edit: crate::OutlineEdit,
306 ) -> Result<Self, crate::Error> {
307 Ok(Self {
308 source,
309 written: edit.apply(source, space, object)?,
310 })
311 }
312
313 /// Joins adjacent ordinary paragraphs with native left-tag and text-identity semantics.
314 pub fn join(
315 source: &'a [u8],
316 space: ExGuid,
317 join: &crate::ParagraphJoin,
318 ) -> Result<Self, crate::Error> {
319 Ok(Self {
320 source,
321 written: join.apply(source, space)?,
322 })
323 }
324
325 /// Splits a paragraph and updates its children, lists, tags and title metadata atomically.
326 /// Fields and associated run metadata are rejected before I/O.
327 pub fn split(
328 source: &'a [u8],
329 space: ExGuid,
330 split: &crate::ParagraphSplit,
331 ) -> Result<Self, crate::Error> {
332 Ok(Self {
333 source,
334 written: split.apply(source, space)?,
335 })
336 }
337
338 /// Prepares an insertion and its dependent metadata in one revision, without I/O.
339 pub fn insert(
340 source: &'a [u8],
341 space: ExGuid,
342 insertion: &crate::Insertion,
343 ) -> Result<Self, crate::Error> {
344 Ok(Self {
345 source,
346 written: insertion.apply(source, space)?,
347 })
348 }
349
350 /// Validates and prepares a text edit without I/O, with `replace_text` semantics.
351 pub fn text(
352 source: &'a [u8],
353 space: ExGuid,
354 object: ExGuid,
355 range: std::ops::Range<u32>,
356 replacement: &str,
357 ) -> Result<Self, crate::Error> {
358 Ok(Self {
359 source,
360 written: crate::replace_text(source, space, object, range, replacement)?,
361 })
362 }
363
364 /// Changes character formatting over a UTF-16 range, preserving unselected runs and styles.
365 /// A zero-length range sets the insertion style only when the paragraph is empty.
366 /// Fields, associated run objects and boundaries splitting preserved run data are rejected.
367 pub fn format(
368 source: &'a [u8],
369 space: ExGuid,
370 object: ExGuid,
371 range: std::ops::Range<u32>,
372 attributes: &[crate::TextAttribute],
373 ) -> Result<Self, crate::Error> {
374 Ok(Self {
375 source,
376 written: crate::formatting::format_text(source, space, object, range, attributes)?,
377 })
378 }
379
380 /// The exact complete image this edit will publish; identities do not regenerate on commit.
381 /// Do not overwrite a live notebook with this image; use `commit` under exclusion.
382 pub fn as_bytes(&self) -> &[u8] {
383 &self.written
384 }
385
386 /// The bytes publishing this edit writes, against its snapshot's stamp.
387 pub fn transaction(&self) -> Transaction {
388 Transaction::between(self.source, &self.written)
389 }
390
391 /// `Transaction::commit` of this edit.
392 pub fn commit(&self, io: &mut impl CommitIo) -> Result<(), CommitError> {
393 self.transaction().commit(io)
394 }
395
396 /// `Transaction::commit_file` of this edit.
397 #[cfg(any(unix, windows))]
398 pub fn commit_file(&self, path: impl AsRef<Path>) -> Result<(), CommitError> {
399 self.transaction().commit_file(path)
400 }
401}
402
403/// `confirm_snapshot` under the same whole-file exclusion `commit_file` uses.
404#[cfg(any(unix, windows))]
405pub fn confirm_file_snapshot(path: impl AsRef<Path>, source: &[u8]) -> Result<(), CommitError> {
406 let mut io = FileIo::open(path, true).map_err(|error| CommitError {
407 state: CommitState::NotCommitted,
408 error,
409 })?;
410 let result = confirm_snapshot(&mut io, source);
411 io.finish(result)
412}
413
414/// Checks and flushes a snapshot's stamp, then refreshes its header version metadata.
415/// No revision is added; reread before using the snapshot for another physical commit.
416/// The caller must hold OneNote-compatible exclusion and independently establish which168/// The caller must hold OneNote-compatible exclusion and independently establish which
417/// intents the snapshot contains. A successful read alone is not a durable acknowledgement.169/// intents the image contains. A successful read alone is not a durable acknowledgement.
418pub fn confirm_snapshot(io: &mut impl CommitIo, source: &[u8]) -> Result<(), CommitError> {170pub fn confirm(io: &mut impl CommitIo, base: &Stamp) -> Result<(), CommitError> {
419 let mut state = CommitState::NotCommitted;171 let mut state = CommitState::NotCommitted;
420 let result = (|| -> io::Result<()> {172 let result = (|| -> io::Result<()> {
421 let header = crate::Header::parse(source).map_err(io::Error::other)?;173 let header = crate::Header::parse(&base.header).map_err(io::Error::other)?;
422 let generation = header174 let generation = header
423 .generation175 .generation
424 .checked_add(1)176 .checked_add(1)
...@@ -427,7 +179,7 @@ pub fn confirm_snapshot(io: &mut impl CommitIo, source: &[u8]) -> Result<(), Com...@@ -427,7 +179,7 @@ pub fn confirm_snapshot(io: &mut impl CommitIo, source: &[u8]) -> Result<(), Com
427 version[..16].copy_from_slice(&crate::write::fresh_guid().map_err(io::Error::other)?);179 version[..16].copy_from_slice(&crate::write::fresh_guid().map_err(io::Error::other)?);
428 version[16..24].copy_from_slice(&generation.to_le_bytes());180 version[16..24].copy_from_slice(&generation.to_le_bytes());
429 version[24..].copy_from_slice(&crate::write::fresh_guid().map_err(io::Error::other)?);181 version[24..].copy_from_slice(&crate::write::fresh_guid().map_err(io::Error::other)?);
430 Stamp::of(source).map_err(io::Error::other)?.check(io)?;182 base.check(io)?;
431 state = CommitState::Unknown;183 state = CommitState::Unknown;
432 io.flush()?;184 io.flush()?;
433 write_all(io, 212, &version)?;185 write_all(io, 212, &version)?;
...@@ -493,26 +245,6 @@ fn write_all(io: &mut impl CommitIo, mut offset: u64, mut bytes: &[u8]) -> io::R...@@ -493,26 +245,6 @@ fn write_all(io: &mut impl CommitIo, mut offset: u64, mut bytes: &[u8]) -> io::R
493 Ok(())245 Ok(())
494}246}
495247
496/// Publishes a scalar revision after checking the locked file against its snapshot.
497/// Unknown outcomes require rereading; Committed errors affect lock release.
498pub fn commit_property_bytes(
499 io: &mut impl CommitIo,
500 source: &[u8],
501 space: ExGuid,
502 object: ExGuid,
503 property: u32,
504 value: &[u8],
505) -> Result<(), CommitError> {
506 let written =
507 replace_property_bytes(source, space, object, property, value).map_err(|error| {
508 CommitError {
509 state: CommitState::NotCommitted,
510 error: io::Error::new(ErrorKind::InvalidData, error),
511 }
512 })?;
513 Transaction::between(source, &written).commit(io)
514}
515
516/// What a commit requires unchanged since its snapshot: the header, which every committed248/// What a commit requires unchanged since its snapshot: the header, which every committed
517/// transaction and every placement rewrites (MS-ONESTORE 2.3.1 `guidFileVersion`), and the249/// transaction and every placement rewrites (MS-ONESTORE 2.3.1 `guidFileVersion`), and the
518/// length appends start from. Equal stamps name the same committed image.250/// length appends start from. Equal stamps name the same committed image.
...@@ -619,31 +351,6 @@ impl TryFrom<Wire> for Transaction {...@@ -619,31 +351,6 @@ impl TryFrom<Wire> for Transaction {
619}351}
620352
621impl Transaction {353impl Transaction {
622 /// The transaction turning `source`, a parsed image, into `written`, which extends it.
623 pub(crate) fn between(source: &[u8], written: &[u8]) -> Self {
624 let differs = |at: &usize| source[*at] != written[*at];
625 let mut patches = Vec::new();
626 let mut offset = 1024;
627 while let Some(start) = (offset..source.len()).find(differs) {
628 let mut end = start + 1;
629 // Equal gaps shorter than a write request's overhead join the patch.
630 while let Some(next) = (end..source.len().min(end + 64)).find(differs) {
631 end = next + 1;
632 }
633 patches.push((start as u64, written[start..end].to_vec()));
634 offset = end;
635 }
636 Self {
637 base: Stamp {
638 header: *source.first_chunk().unwrap(),
639 length: source.len() as u64,
640 },
641 append: written[source.len()..].to_vec(),
642 patches,
643 header: *written.first_chunk().unwrap(),
644 }
645 }
646
647 /// The image this transaction applies to.354 /// The image this transaction applies to.
648 pub fn base(&self) -> &Stamp {355 pub fn base(&self) -> &Stamp {
649 &self.base356 &self.base
crates/onestore/src/create.rs+55-1
...@@ -17,6 +17,21 @@ pub(crate) fn string(value: &str) -> Vec<u8> {...@@ -17,6 +17,21 @@ pub(crate) fn string(value: &str) -> Vec<u8> {
17 .collect()17 .collect()
18}18}
1919
20/// An author's initials as OneNote 2010 stores them beside the name (`0x1c001df8`, with
21/// no MS-ONE entry): Office's default, the first letter of each word of the name.
22pub(crate) fn initials(author: &str) -> String {
23 author
24 .split_whitespace()
25 .filter_map(|word| word.chars().next())
26 .flat_map(char::to_uppercase)
27 .collect()
28}
29
30/// An author object's properties as OneNote 2010 writes them: initials, then the name.
31pub(crate) fn author_properties(name: &str) -> Vec<(u32, Vec<u8>)> {
32 vec![(0x1c001df8, string(&initials(name))), (0x1c001d75, string(name))]
33}
34
20pub(crate) fn default_text_style() -> Vec<(u32, Vec<u8>)> {35pub(crate) fn default_text_style() -> Vec<(u32, Vec<u8>)> {
21 vec![36 vec![
22 (0x14001c3b, 0x409_u32.to_le_bytes().to_vec()),37 (0x14001c3b, 0x409_u32.to_le_bytes().to_vec()),
...@@ -249,6 +264,7 @@ pub fn create_section(file_name: &str, text: &str, author: &str) -> Result<Vec<u...@@ -249,6 +264,7 @@ pub fn create_section(file_name: &str, text: &str, author: &str) -> Result<Vec<u
249 last_modified(),264 last_modified(),
250 (0x24001c20, id(23)),265 (0x24001c20, id(23)),
251 (0x1c001d75, string(author)),266 (0x1c001d75, string(author)),
267 (0x1c001df8, string(&initials(author))),
252 (0x1c001d3c, string(crate::edit::automatic_title(text))),268 (0x1c001d3c, string(crate::edit::automatic_title(text))),
253 ],269 ],
254 page_margins(),270 page_margins(),
...@@ -294,7 +310,7 @@ pub fn create_section(file_name: &str, text: &str, author: &str) -> Result<Vec<u...@@ -294,7 +310,7 @@ pub fn create_section(file_name: &str, text: &str, author: &str) -> Result<Vec<u
294 NewObject {310 NewObject {
295 id: 26,311 id: 26,
296 jcid: 0x120001,312 jcid: 0x120001,
297 properties: vec![(0x1c001d75, string(author))],313 properties: author_properties(author),
298 },314 },
299 NewObject {315 NewObject {
300 id: 27,316 id: 27,
...@@ -307,6 +323,44 @@ pub fn create_section(file_name: &str, text: &str, author: &str) -> Result<Vec<u...@@ -307,6 +323,44 @@ pub fn create_section(file_name: &str, text: &str, author: &str) -> Result<Vec<u
307 create(file_name, FileType::Section, spaces)323 create(file_name, FileType::Section, spaces)
308}324}
309325
326/// Creates a section without pages, coloured `color` (COLORREF, `None` for no colour), as
327/// OneNote 2010 leaves one whose pages are all deleted: its section node lists no pages.
328/// Pages come as `op::SectionOp::Create` edits.
329pub fn create_empty_section(file_name: &str, color: Option<u32>) -> Result<Vec<u8>> {
330 if file_name.contains(['/', '\\', '\0']) || !file_name.to_ascii_lowercase().ends_with(".one") {
331 return Err(Error {
332 offset: 0,
333 message: "Invalid section name",
334 });
335 }
336 let timestamp = current_timestamps()?.1.to_le_bytes().to_vec();
337 let id = |value: u32| value.to_le_bytes().to_vec();
338 let spaces = vec![NewSpace {
339 id: 1,
340 roots: vec![(1, 10), (2, 11)],
341 objects: vec![
342 NewObject {
343 id: 10,
344 jcid: 0x60007,
345 properties: vec![
346 (0x1c001c30, fresh_guid()?.to_vec()),
347 (0x18001c65, timestamp),
348 ],
349 },
350 NewObject {
351 id: 11,
352 jcid: 0x20031,
353 properties: vec![
354 (0x14001d82, id(40)),
355 (0x1400348b, id(40)),
356 (0x14001cbe, id(color.unwrap_or(0xffff_ffff))),
357 ],
358 },
359 ],
360 }];
361 create(file_name, FileType::Section, spaces)
362}
363
310/// Creates a notebook table of contents from section filenames and file identities.364/// Creates a notebook table of contents from section filenames and file identities.
311pub fn create_table_of_contents(file_name: &str, sections: &[(&str, [u8; 16])]) -> Result<Vec<u8>> {365pub fn create_table_of_contents(file_name: &str, sections: &[(&str, [u8; 16])]) -> Result<Vec<u8>> {
312 if file_name.contains(['/', '\\', '\0'])366 if file_name.contains(['/', '\\', '\0'])
crates/onestore/src/edit.rs+4-22
...@@ -28,26 +28,6 @@ pub(crate) fn automatic_title(text: &str) -> &str {...@@ -28,26 +28,6 @@ pub(crate) fn automatic_title(text: &str) -> &str {
28/// Insertion at a run boundary uses the following run, except at the end of text.28/// Insertion at a run boundary uses the following run, except at the end of text.
29/// The final run retains its insertion style even when emptied.29/// The final run retains its insertion style even when emptied.
30/// Replacing a range with identical text leaves its existing formatting unchanged.30/// Replacing a range with identical text leaves its existing formatting unchanged.
31/// Returns a complete file image without I/O; use `commit_file_text` to update an existing file.
32pub fn replace_text(
33 source: &[u8],
34 space: ExGuid,
35 object: ExGuid,
36 range: Range<u32>,
37 replacement: &str,
38) -> Result<Vec<u8>, Error> {
39 if range.start > range.end || replacement.contains(['\0', '\n', '\r', '\u{fffc}']) {
40 return Err(Error {
41 offset: 0,
42 message: "Use a valid text range and ordinary paragraph text",
43 });
44 }
45 crate::active::write(source, space, |active| {
46 text_changes(active, object, range, replacement)
47 })
48}
49
50/// `replace_text` on an active page.
51pub(crate) fn text_changes(31pub(crate) fn text_changes(
52 active: &ActivePage<'_>,32 active: &ActivePage<'_>,
53 object: ExGuid,33 object: ExGuid,
...@@ -312,7 +292,9 @@ fn rewrite(...@@ -312,7 +292,9 @@ fn rewrite(
312 .roots292 .roots
313 .get(&2)293 .get(&2)
314 .ok_or_else(|| invalid("Page title metadata is unavailable"))?;294 .ok_or_else(|| invalid("Page title metadata is unavailable"))?;
315 let Kind::Metadata { title, .. } = &revision.nodes[metadata].kind else {295 let (Kind::Metadata { title, .. } | Kind::ConflictMetadata { title, .. }) =
296 &revision.nodes[metadata].kind
297 else {
316 return Err(invalid("Page title metadata is unavailable"));298 return Err(invalid("Page title metadata is unavailable"));
317 };299 };
318 let cached: Vec<_> = title_text300 let cached: Vec<_> = title_text
...@@ -544,7 +526,7 @@ pub(crate) fn update_title(...@@ -544,7 +526,7 @@ pub(crate) fn update_title(
544 .roots526 .roots
545 .get(&2)527 .get(&2)
546 .ok_or_else(|| invalid("Page title metadata is unavailable"))?;528 .ok_or_else(|| invalid("Page title metadata is unavailable"))?;
547 if raw.objects[metadata].jcid != 0x20030 {529 if ![0x20030, 0x20038].contains(&raw.objects[metadata].jcid) {
548 return Err(invalid("Page title metadata is unavailable"));530 return Err(invalid("Page title metadata is unavailable"));
549 }531 }
550 let title = string(&title);532 let title = string(&title);
crates/onestore/src/formatting.rs+2-14
...@@ -100,20 +100,8 @@ impl TextAttribute {...@@ -100,20 +100,8 @@ impl TextAttribute {
100 }100 }
101}101}
102102
103pub(crate) fn format_text(103/// Character formatting set over a UTF-16 range, also removing the `cleared` properties
104 source: &[u8],104/// from the range's run styles so the text inherits them.
105 space: ExGuid,
106 object: ExGuid,
107 range: Range<u32>,
108 attributes: &[TextAttribute],
109) -> Result<Vec<u8>, Error> {
110 crate::active::write(source, space, |active| {
111 format_changes(active, object, range, attributes, &[])
112 })
113}
114
115/// `format_text` on an active page, also removing the `cleared` properties from the range's
116/// run styles so the text inherits them.
117pub(crate) fn format_changes(105pub(crate) fn format_changes(
118 active: &ActivePage<'_>,106 active: &ActivePage<'_>,
119 object: ExGuid,107 object: ExGuid,
crates/onestore/src/insertion.rs+11-191
...@@ -6,10 +6,8 @@ use crate::{...@@ -6,10 +6,8 @@ use crate::{
6 op::content::{NATIVE_INDENTS, measurement_bytes},6 op::content::{NATIVE_INDENTS, measurement_bytes},
7 write::{PropertyObject, fresh_guid},7 write::{PropertyObject, fresh_guid},
8};8};
9use serde::{Deserialize, Serialize};
10use std::{9use std::{
11 collections::{BTreeMap, BTreeSet},10 collections::{BTreeMap, BTreeSet},
12 ops::Range,
13 sync::Arc,11 sync::Arc,
14};12};
1513
...@@ -17,59 +15,29 @@ fn invalid(message: &'static str) -> Error {...@@ -17,59 +15,29 @@ fn invalid(message: &'static str) -> Error {
17 Error { offset: 0, message }15 Error { offset: 0, message }
18}16}
1917
20#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]18#[derive(Debug, Clone, PartialEq)]
21#[serde(deny_unknown_fields)]
22enum Placement {19enum Placement {
23 Paragraph { before: Option<ExGuid> },20 Paragraph { before: Option<ExGuid> },
24 Outline { x: f32, y: f32 },21 Outline { x: f32, y: f32 },
25}22}
2623
27#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]24/// A paragraph or outline insertion with its creation time; `changes_as` names the new
28#[serde(deny_unknown_fields)]25/// objects.
29struct FormatSpan {26#[derive(Debug, Clone, PartialEq)]
30 guid: [u8; 16],27pub(crate) struct Insertion {
31 range: Range<u32>,
32 attributes: Vec<crate::TextAttribute>,
33}
34
35/// A paragraph or outline insertion with stable object identities and creation time.
36/// Retain this intent across rebases; constructing another intent allocates different identities.
37#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
38#[serde(deny_unknown_fields)]
39pub struct Insertion {
40 guid: [u8; 16],28 guid: [u8; 16],
41 parent: ExGuid,29 parent: ExGuid,
42 placement: Placement,30 placement: Placement,
43 text: String,31 text: String,
44 author: String,32 author: String,
45 created: u32,33 created: u32,
46 #[serde(default, skip_serializing_if = "Vec::is_empty")]
47 formats: Vec<FormatSpan>,
48}34}
4935
50impl Insertion {36impl Insertion {
51 /// Adds a nonoverlapping UTF-16 formatting span, retaining all existing identities.
52 /// Gaps use the ordinary insertion style; only empty text accepts a zero-length span.
53 pub fn with_formatting(
54 &self,
55 range: Range<u32>,
56 attributes: &[crate::TextAttribute],
57 ) -> Result<Self, Error> {
58 let mut intent = self.clone();
59 intent.formats.push(FormatSpan {
60 guid: fresh_guid()?,
61 range,
62 attributes: attributes.to_vec(),
63 });
64 intent.formats.sort_by_key(|span| span.range.start);
65 intent.validate()?;
66 Ok(intent)
67 }
68
69 /// Inserts before a direct child, or appends when `before` is None.37 /// Inserts before a direct child, or appends when `before` is None.
70 /// The parent must be an editable outline, paragraph, outline group or table cell.38 /// The parent must be an editable outline, paragraph, outline group or table cell.
71 /// Carriage returns represent soft line breaks; line feeds and embedded-field markers are rejected.39 /// Carriage returns represent soft line breaks; line feeds and embedded-field markers are rejected.
72 pub fn paragraph(40 pub(crate) fn paragraph(
73 parent: ExGuid,41 parent: ExGuid,
74 before: Option<ExGuid>,42 before: Option<ExGuid>,
75 text: &str,43 text: &str,
...@@ -79,38 +47,10 @@ impl Insertion {...@@ -79,38 +47,10 @@ impl Insertion {
79 }47 }
8048
81 /// Adds an outline to an editable page at coordinates measured in points.49 /// Adds an outline to an editable page at coordinates measured in points.
82 pub fn outline(page: ExGuid, x: f32, y: f32, text: &str, author: &str) -> Result<Self, Error> {50 pub(crate) fn outline(page: ExGuid, x: f32, y: f32, text: &str, author: &str) -> Result<Self, Error> {
83 Self::new(page, Placement::Outline { x, y }, text, author)51 Self::new(page, Placement::Outline { x, y }, text, author)
84 }52 }
8553
86 /// Changes a paragraph intent's placement while retaining its identities, text and author.
87 pub fn reposition_paragraph(
88 &self,
89 parent: ExGuid,
90 before: Option<ExGuid>,
91 ) -> Result<Self, Error> {
92 if !matches!(self.placement, Placement::Paragraph { .. }) {
93 return Err(invalid("An outline intent cannot become a paragraph"));
94 }
95 let mut intent = self.clone();
96 intent.parent = parent;
97 intent.placement = Placement::Paragraph { before };
98 intent.validate()?;
99 Ok(intent)
100 }
101
102 /// Changes an outline intent's placement while retaining its identities, text and author.
103 pub fn reposition_outline(&self, page: ExGuid, x: f32, y: f32) -> Result<Self, Error> {
104 if !matches!(self.placement, Placement::Outline { .. }) {
105 return Err(invalid("A paragraph intent cannot become an outline"));
106 }
107 let mut intent = self.clone();
108 intent.parent = page;
109 intent.placement = Placement::Outline { x, y };
110 intent.validate()?;
111 Ok(intent)
112 }
113
114 fn new(parent: ExGuid, placement: Placement, text: &str, author: &str) -> Result<Self, Error> {54 fn new(parent: ExGuid, placement: Placement, text: &str, author: &str) -> Result<Self, Error> {
115 let intent = Self {55 let intent = Self {
116 guid: fresh_guid()?,56 guid: fresh_guid()?,
...@@ -119,28 +59,11 @@ impl Insertion {...@@ -119,28 +59,11 @@ impl Insertion {
119 text: text.to_owned(),59 text: text.to_owned(),
120 author: author.to_owned(),60 author: author.to_owned(),
121 created: current_timestamps()?.0,61 created: current_timestamps()?.0,
122 formats: Vec::new(),
123 };62 };
124 intent.validate()?;63 intent.validate()?;
125 Ok(intent)64 Ok(intent)
126 }65 }
12766
128 /// Identity of the new paragraph, or the new outline for an outline insertion.
129 pub fn object(&self) -> ExGuid {
130 ExGuid {
131 guid: self.guid,
132 n: 1,
133 }
134 }
135
136 /// Identity of the insertion's ordinary rich-text object.
137 pub fn text_object(&self) -> ExGuid {
138 ExGuid {
139 guid: self.guid,
140 n: 2,
141 }
142 }
143
144 fn validate(&self) -> Result<(), Error> {67 fn validate(&self) -> Result<(), Error> {
145 if self.guid == [0; 16]68 if self.guid == [0; 16]
146 || self.parent.guid == [0; 16]69 || self.parent.guid == [0; 16]
...@@ -156,69 +79,11 @@ impl Insertion {...@@ -156,69 +79,11 @@ impl Insertion {
156 {79 {
157 return Err(invalid("Outline coordinates must be finite"));80 return Err(invalid("Outline coordinates must be finite"));
158 }81 }
159 if !self.formats.is_empty() {
160 let units: Vec<_> = self.text.encode_utf16().collect();
161 let length = u32::try_from(units.len())
162 .map_err(|_| invalid("Text exceeds UTF-16 offset range"))?;
163 let mut identities = BTreeSet::from([self.guid]);
164 let mut end = 0;
165 for span in &self.formats {
166 if span.guid == [0; 16] || !identities.insert(span.guid) {
167 return Err(invalid("Formatting identities must be distinct"));
168 }
169 if span.range.start > span.range.end
170 || span.range.end > length
171 || span.range.start < end
172 || (span.range.is_empty() && (!units.is_empty() || self.formats.len() != 1))
173 {
174 return Err(invalid(
175 "Formatting spans must be nonoverlapping ranges within the new text",
176 ));
177 }
178 for boundary in [span.range.start, span.range.end] {
179 if boundary > 0
180 && boundary < length
181 && (0xd800..=0xdbff).contains(&units[boundary as usize - 1])
182 {
183 return Err(invalid("Formatting boundary splits a surrogate pair"));
184 }
185 }
186 crate::formatting::attribute_values(&span.attributes)?;
187 end = span.range.end;
188 }
189 }
190 Ok(())82 Ok(())
191 }83 }
19284
193 pub(crate) fn apply(&self, source: &[u8], space: ExGuid) -> Result<Vec<u8>, Error> {85 /// The objects creating the outline or paragraph `object`, the paragraph `paragraph` and
194 self.validate()?;86 /// its text `text` under those identities.
195 crate::active::write(source, space, |active| self.changes(active))
196 }
197
198 pub(crate) fn changes(&self, active: &ActivePage<'_>) -> Result<Changes, Error> {
199 let paragraph = ExGuid {
200 guid: self.guid,
201 n: if matches!(self.placement, Placement::Outline { .. }) {
202 3
203 } else {
204 1
205 },
206 };
207 let changes = self.changes_as(active, self.object(), paragraph, self.text_object())?;
208 let raw = &active.live.revision;
209 if changes
210 .keys()
211 .any(|id| id.guid == self.guid && raw.objects.contains_key(id))
212 {
213 return Err(invalid(
214 "An insertion identity is already present; reconcile the existing edit",
215 ));
216 }
217 Ok(changes)
218 }
219
220 /// `changes` creating the outline or paragraph `object`, the paragraph `paragraph` and its
221 /// text `text` under those identities.
222 pub(crate) fn changes_as(87 pub(crate) fn changes_as(
223 &self,88 &self,
224 active: &ActivePage<'_>,89 active: &ActivePage<'_>,
...@@ -321,7 +186,7 @@ impl Insertion {...@@ -321,7 +186,7 @@ impl Insertion {
321 (0x10001cfe, 0x409_u16.to_le_bytes().to_vec()),186 (0x10001cfe, 0x409_u16.to_le_bytes().to_vec()),
322 ],187 ],
323 ),188 ),
324 (author, 0x120001, vec![(0x1c001d75, string(&self.author))]),189 (author, 0x120001, crate::create::author_properties(&self.author)),
325 (default, 0x12004d, default_text_style()),190 (default, 0x12004d, default_text_style()),
326 ] {191 ] {
327 new.insert(192 new.insert(
...@@ -333,51 +198,6 @@ impl Insertion {...@@ -333,51 +198,6 @@ impl Insertion {
333 },198 },
334 );199 );
335 }200 }
336 if !self.formats.is_empty() {
337 let mut segments = Vec::new();
338 let mut end = 0;
339 for span in &self.formats {
340 if end < span.range.start {
341 segments.push((span.range.start, default));
342 }
343 let id = ExGuid {
344 guid: span.guid,
345 n: 1,
346 };
347 let mut style = PropertyObject {
348 jcid: 0x12004d,
349 bytes: properties(&default_text_style())?,
350 global_ids: Arc::new(BTreeMap::from([(0, span.guid)])),
351 };
352 let values = crate::formatting::attribute_values(&span.attributes)?;
353 style.set(
354 &values
355 .iter()
356 .map(|(id, value)| (*id, value.as_slice()))
357 .collect::<Vec<_>>(),
358 )?;
359 new.insert(id, style);
360 segments.push((span.range.end, id));
361 end = span.range.end;
362 }
363 let length = u32::try_from(self.text.encode_utf16().count())
364 .map_err(|_| invalid("Text exceeds UTF-16 offset range"))?;
365 if end < length {
366 segments.push((length, default));
367 }
368 if !segments.iter().any(|(_, id)| *id == default) {
369 new.remove(&default);
370 }
371 let target = new.get_mut(&text).unwrap();
372 let mut references = Vec::new();
373 let mut ends = Vec::new();
374 for (end, id) in segments {
375 references.extend_from_slice(&target.reference(id)?);
376 ends.extend_from_slice(&end.to_le_bytes());
377 }
378 ends.truncate(ends.len() - 4);
379 target.set(&[(0x24001e13, &references), (0x1c001e12, &ends)])?;
380 }
381 if let Placement::Outline { x, y } = self.placement {201 if let Placement::Outline { x, y } = self.placement {
382 let children = reference(paragraph)?;202 let children = reference(paragraph)?;
383 new.insert(203 new.insert(
...@@ -449,7 +269,7 @@ impl Insertion {...@@ -449,7 +269,7 @@ impl Insertion {
449 .roots269 .roots
450 .get(&2)270 .get(&2)
451 .ok_or_else(|| invalid("Page title metadata is unavailable"))?;271 .ok_or_else(|| invalid("Page title metadata is unavailable"))?;
452 if raw.objects[metadata].jcid != 0x20030 {272 if ![0x20030, 0x20038].contains(&raw.objects[metadata].jcid) {
453 return Err(invalid("Page title metadata is unavailable"));273 return Err(invalid("Page title metadata is unavailable"));
454 }274 }
455 let title = string(&title);275 let title = string(&title);
crates/onestore/src/lib.rs+8-16
...@@ -28,29 +28,21 @@ mod toc;...@@ -28,29 +28,21 @@ mod toc;
28mod tree;28mod tree;
29mod write;29mod write;
3030
31pub use commit::{31pub use commit::{CommitError, CommitIo, CommitState, Stamp, Transaction, confirm, place};
32 CommitError, CommitIo, CommitState, PreparedEdit, Stamp, Transaction, commit_property_bytes,
33 commit_text, confirm_snapshot, place,
34};
35#[cfg(any(unix, windows))]32#[cfg(any(unix, windows))]
36pub use commit::{33pub use commit::{confirm_file, place_file, read_file, read_file_limited};
37 commit_file_property, commit_file_text, confirm_file_snapshot, place_file, read_file,34pub use create::{create_empty_section, create_section, create_table_of_contents};
38 read_file_limited,
39};
40pub use create::{create_section, create_table_of_contents};
41pub use edit::replace_text;
42pub use files::FileDataReference;35pub use files::FileDataReference;
43pub use formatting::{FONT_SIZES, TextAttribute};36pub use formatting::{FONT_SIZES, TextAttribute};
44pub use insertion::Insertion;37pub(crate) use insertion::Insertion;
45pub use objects::{Object, ObjectData, ObjectReferences, ResolvedRevision};38pub use objects::{Object, ObjectData, ObjectReferences, ResolvedRevision};
46pub use outline::OutlineEdit;39pub use outline::OutlineEdit;
47pub use pages::{PageCreation, PageEdit, PagePosition};40pub use pages::{ConflictPage, PageCreation, PageEdit, PagePosition};
48pub use paragraph::{ParagraphJoin, ParagraphSplit};41pub(crate) use paragraph::{ParagraphJoin, ParagraphSplit};
49pub use properties::{IdStream, Property, PropertySets, Value};42pub use properties::{IdStream, Property, PropertySets, Value};
50pub use revisions::{ExGuid, ObjectSpace, Revision, RevisionIndex};43pub use revisions::{ExGuid, ObjectSpace, Revision, RevisionIndex};
51pub use section::{Arena, Section};44pub use section::{Arena, Section};
52pub use snapshot::{read_snapshot, read_storage_snapshot};45pub use snapshot::{read_snapshot, read_storage_snapshot};
53pub use store::{Chunk, Error, FileType, Header, Node, NodeList, Reference, Store};46pub use store::{Chunk, Error, FileType, Header, Node, NodeList, Reference, Store};
54pub use toc::TocEdit;47pub use toc::{TocEdit, edit_table_of_contents};
55pub use tree::TreeEdit;48pub(crate) use tree::TreeEdit;
56pub use write::replace_property_bytes;
crates/onestore/src/op/apply.rs+278-35
...@@ -49,6 +49,28 @@ impl<'a> Section<'a> {...@@ -49,6 +49,28 @@ impl<'a> Section<'a> {
49 Op::Section(op) => section.apply_section(author, op)?,49 Op::Section(op) => section.apply_section(author, op)?,
50 }50 }
51 }51 }
52 let spaces: BTreeSet<_> = edit
53 .ops
54 .iter()
55 .filter_map(|op| match op {
56 Op::Page { space, .. } => Some(*space),
57 Op::Section(_) => None,
58 })
59 .collect();
60 for space in spaces {
61 // A page the edit went on to delete has no cells left to check.
62 let Ok(page) = section.active(space) else {
63 continue;
64 };
65 if page.view.nodes.values().any(|node| {
66 matches!(node.kind, Kind::Cell { .. }) && node.children.is_empty()
67 }) {
68 return Err(OpError::Unsupported(
69 "A table cell keeps a paragraph; insert its replacement first",
70 )
71 .into());
72 }
73 }
52 Ok(())74 Ok(())
53 })75 })
54 })76 })
...@@ -99,28 +121,26 @@ impl<'a> Section<'a> {...@@ -99,28 +121,26 @@ impl<'a> Section<'a> {
99 }121 }
100 SectionOp::Import { creation, page } => {122 SectionOp::Import { creation, page } => {
101 self.apply_section(author, &SectionOp::Create(creation.clone()))?;123 self.apply_section(author, &SectionOp::Create(creation.clone()))?;
124 self.fill(author, creation.space(), page)?;
125 }
126 SectionOp::Conflict {
127 of,
128 creation,
129 page,
130 objects,
131 } => {
132 let level = self
133 .pages()?
134 .into_iter()
135 .find(|(space, ..)| space == of)
136 .map(|(.., level)| level)
137 .ok_or(OpError::TargetUnavailable(*of))?;
102 let space = creation.space();138 let space = creation.space();
103 let before = self.page(space)?;139 let (roots, created, metadata) = creation.conflict(self, &page.title, level)?;
104 let mut after = page.clone();140 self.create_space(space, roots, created)?;
105 // The created page's title stays; the imported page brings the rest.141 self.fill(author, space, page)?;
106 after.objects.retain(|object| !matches!(object, PageObject::Title(_)));142 self.mark_conflict(space, creation, page, objects)?;
107 for object in before.objects.iter().rev() {143 self.link_conflict(*of, space, metadata)?;
108 if let PageObject::Title(_) = object {
109 after.objects.insert(0, object.clone());
110 }
111 }
112 after.title = before.title.clone();
113 after.identity = before.identity;
114 after.created = before.created;
115 after.margin_origin = before.margin_origin;
116 for op in super::lower_page(&before, &after)? {
117 Writer {
118 section: self,
119 space,
120 author,
121 }
122 .apply(&op)?;
123 }
124 }144 }
125 SectionOp::Pages(edits) => {145 SectionOp::Pages(edits) => {
126 for (space, changes) in crate::pages::section_changes(self, edits, &[])? {146 for (space, changes) in crate::pages::section_changes(self, edits, &[])? {
...@@ -128,13 +148,235 @@ impl<'a> Section<'a> {...@@ -128,13 +148,235 @@ impl<'a> Section<'a> {
128 }148 }
129 }149 }
130 SectionOp::Delete(pages) => {150 SectionOp::Delete(pages) => {
131 for (space, changes) in crate::pages::section_changes(self, &[], pages)? {151 let mut listed = Vec::new();
132 self.apply_changes(space, &[], changes)?;152 let conflicts = self.conflicts()?;
153 for space in pages {
154 let of = conflicts.iter().find_map(|(of, conflicts)| {
155 conflicts.iter().any(|page| page.space == *space).then_some(*of)
156 });
157 match of {
158 Some(of) => self.unlink_conflict(of, *space)?,
159 None => listed.push(*space),
160 }
161 }
162 if !listed.is_empty() {
163 for (space, changes) in crate::pages::section_changes(self, &[], &listed)? {
164 self.apply_changes(space, &[], changes)?;
165 }
133 }166 }
134 }167 }
168 SectionOp::Color(color) => {
169 let root = self.root();
170 let page = self.active(root)?;
171 let metadata = *page
172 .view
173 .roots
174 .get(&2)
175 .filter(|id| page.live.revision.objects.contains_key(id))
176 .ok_or_else(|| invalid("The section has no metadata to colour"))?;
177 let mut object = crate::write::PropertyObject::from_object(
178 &page.live.revision.objects[&metadata],
179 )?;
180 object.set(&[(0x14001cbe, &color.unwrap_or(0xffff_ffff).to_le_bytes())])?;
181 self.apply_changes(root, &[], BTreeMap::from([(metadata, object)]))?;
182 }
135 }183 }
136 Ok(())184 Ok(())
137 }185 }
186
187 /// Writes `page`'s content onto the page just created in `space`, whose title stays.
188 fn fill(&mut self, author: &str, space: ExGuid, page: &crate::page::Page) -> Result<(), Failure> {
189 let before = self.page(space)?;
190 let mut after = page.clone();
191 after.objects.retain(|object| !matches!(object, PageObject::Title(_)));
192 for object in &mut after.objects {
193 // Stroke points are page coordinates; new ink takes its position from them.
194 if let PageObject::Ink(ink) = object {
195 ink.layout = Default::default();
196 }
197 }
198 for object in before.objects.iter().rev() {
199 if let PageObject::Title(_) = object {
200 after.objects.insert(0, object.clone());
201 }
202 }
203 after.title = before.title.clone();
204 after.identity = before.identity;
205 after.created = before.created;
206 after.margin_origin = before.margin_origin;
207 for op in super::lower_page(&before, &after)? {
208 Writer {
209 section: self,
210 space,
211 author,
212 }
213 .apply(&op)?;
214 }
215 Ok(())
216 }
217
218 /// Marks a filled conflict page as OneNote 2010 does: the page read-only, deletable and
219 /// a conflict page (MS-ONE 2.1.2), and each of `objects` a conflict object (2.1.1). The
220 /// created title stands for `page`'s, so objects of that title mark its counterparts.
221 fn mark_conflict(
222 &mut self,
223 space: ExGuid,
224 creation: &crate::PageCreation,
225 page: &crate::page::Page,
226 objects: &[ExGuid],
227 ) -> Result<(), Failure> {
228 let created = |n| ExGuid {
229 guid: creation.space().guid,
230 n,
231 };
232 let mut titles = BTreeMap::new();
233 for object in &page.objects {
234 let PageObject::Title(title) = object else {
235 continue;
236 };
237 titles.insert(title.id, created(13));
238 if let Some(outline) = title.outlines.first() {
239 titles.insert(outline.id, created(14));
240 if let Some(paragraph) = outline.paragraphs.first() {
241 titles.insert(paragraph.id, created(15));
242 if let Some(text) = paragraph.text() {
243 titles.insert(text.id, created(16));
244 }
245 }
246 }
247 }
248 let active = self.active(space)?;
249 let mut changes = BTreeMap::new();
250 let mut page_node = crate::write::PropertyObject::from_object(
251 &active.live.revision.objects[&creation.object()],
252 )?;
253 page_node.set(&[(0x88001cde, &[]), (0x88001d0c, &[]), (0x88001d7c, &[])])?;
254 changes.insert(creation.object(), page_node);
255 for id in objects {
256 let id = titles.get(id).copied().unwrap_or(*id);
257 if !active.live.is_reachable(id) {
258 return Err(OpError::TargetUnavailable(id).into());
259 }
260 let mut object =
261 crate::write::PropertyObject::from_object(&active.live.revision.objects[&id])?;
262 // A table is rendered as conflicting but is not selectable as such.
263 if object.jcid == 0x60022 {
264 object.set(&[(0x88001d96, &[])])?;
265 } else {
266 object.set(&[(0x88001ddb, &[]), (0x88001d96, &[])])?;
267 }
268 changes.insert(id, object);
269 }
270 self.apply_changes(space, &[], changes)?;
271 Ok(())
272 }
273
274 /// Lists conflict page `space` under page `of` as OneNote 2010 does: the page's manifest
275 /// references it and keeps a copy of its metadata, and the page's metadata and the
276 /// section's copy of it say the page has conflict pages (MS-ONE 2.2.34, 2.3.71).
277 fn link_conflict(
278 &mut self,
279 of: ExGuid,
280 space: ExGuid,
281 mut metadata: crate::write::PropertyObject,
282 ) -> Result<(), Failure> {
283 use crate::{pages::set_references, write::PropertyObject};
284 let page = self.active(of)?;
285 let manifest_id = page.view.roots[&1];
286 let manifest_node = &page.view.nodes[&manifest_id];
287 let mut spaces = manifest_node.spaces.clone();
288 spaces.push(space);
289 let mut copies = objects(manifest_node, 0x24003442);
290 let copy = crate::pages::metadata_id(space);
291 copies.push(copy);
292 metadata.reference(copy)?;
293 let mut manifest = PropertyObject::from_object(&page.live.revision.objects[&manifest_id])?;
294 set_references(&mut manifest, 0x2c001d63, &spaces)?;
295 set_references(&mut manifest, 0x24003442, &copies)?;
296 self.apply_changes(
297 of,
298 &[],
299 BTreeMap::from([(manifest_id, manifest), (copy, metadata)]),
300 )?;
301 self.has_conflicts(of, true)
302 }
303
304 /// Deletes conflict page `space` of page `of` as OneNote 2010 does: the manifest stops
305 /// referencing it (its metadata copy stays) and the last one's removal clears
306 /// `HasConflictPages`; the conflict page's own space is left as it is.
307 fn unlink_conflict(&mut self, of: ExGuid, space: ExGuid) -> Result<(), Failure> {
308 use crate::{pages::set_references, write::PropertyObject};
309 let page = self.active(of)?;
310 let manifest_id = page.view.roots[&1];
311 let spaces: Vec<ExGuid> = page.view.nodes[&manifest_id]
312 .spaces
313 .iter()
314 .copied()
315 .filter(|listed| *listed != space)
316 .collect();
317 let mut manifest = PropertyObject::from_object(&page.live.revision.objects[&manifest_id])?;
318 if spaces.is_empty() {
319 manifest.remove(&[0x2c001d63])?;
320 } else {
321 set_references(&mut manifest, 0x2c001d63, &spaces)?;
322 }
323 self.apply_changes(of, &[], BTreeMap::from([(manifest_id, manifest)]))?;
324 if spaces.is_empty() {
325 self.has_conflicts(of, false)?;
326 }
327 Ok(())
328 }
329
330 /// Sets or clears `HasConflictPages` on page `of`'s metadata and on the section's copy
331 /// of it, where its series keeps one.
332 fn has_conflicts(&mut self, of: ExGuid, value: bool) -> Result<(), Failure> {
333 use crate::write::PropertyObject;
334 let flag = |object: &crate::Object<'_>| -> Result<PropertyObject, Error> {
335 let mut object = PropertyObject::from_object(object)?;
336 if value {
337 object.set(&[(0x88001d97, &[])])?;
338 } else {
339 object.remove(&[0x88001d97])?;
340 }
341 Ok(object)
342 };
343 let page = self.active(of)?;
344 let stored_id = page.view.roots[&2];
345 let identity = crate::pages::metadata_guid(&page.view.nodes[&stored_id])?;
346 let stored = flag(&page.live.revision.objects[&stored_id])?;
347 self.apply_changes(of, &[], BTreeMap::from([(stored_id, stored)]))?;
348 let root = self.root();
349 let section = self.active(root)?;
350 let copy = section
351 .view
352 .nodes
353 .values()
354 .find(|node| node.spaces.contains(&of))
355 .map(|series| objects(series, 0x24003442))
356 .unwrap_or_default()
357 .into_iter()
358 .find(|id| {
359 section.view.nodes.get(id).is_some_and(|node| {
360 crate::pages::metadata_guid(node).is_ok_and(|guid| guid == identity)
361 })
362 });
363 if let Some(id) = copy {
364 let object = flag(&section.live.revision.objects[&id])?;
365 self.apply_changes(root, &[], BTreeMap::from([(id, object)]))?;
366 }
367 Ok(())
368 }
369}
370
371/// The objects an element's reference property names.
372fn objects(node: &crate::document::Element<'_>, property: u32) -> Vec<ExGuid> {
373 node.extra[0]
374 .iter()
375 .find_map(|field| match &field.value {
376 crate::document::FieldValue::Objects(ids) if field.id == property => Some(ids.clone()),
377 _ => None,
378 })
379 .unwrap_or_default()
138}380}
139381
140/// Whether an op may store more than one revision of a page, so its failure part way needs382/// Whether an op may store more than one revision of a page, so its failure part way needs
...@@ -348,6 +590,19 @@ impl<'a> Writer<'_, 'a> {...@@ -348,6 +590,19 @@ impl<'a> Writer<'_, 'a> {
348 Ok(BTreeMap::from([(*metadata, object)]))590 Ok(BTreeMap::from([(*metadata, object)]))
349 })591 })
350 }592 }
593 PageOp::Color(color) => self.write(|page| {
594 let [node] = crate::active::manifest_pages(&page.view)[..] else {
595 return Err(invalid("The page has no page node to colour"));
596 };
597 let mut object =
598 crate::write::PropertyObject::from_object(&page.live.revision.objects[&node])?;
599 // OneNote's "No color" removes the property.
600 match color {
601 Some(color) => object.set(&[(crate::page::PAGE_COLOR, &color.to_le_bytes())])?,
602 None => object.remove(&[crate::page::PAGE_COLOR])?,
603 }
604 Ok(BTreeMap::from([(node, object)]))
605 }),
351 PageOp::Insert {606 PageOp::Insert {
352 container,607 container,
353 before,608 before,
...@@ -386,18 +641,6 @@ impl<'a> Writer<'_, 'a> {...@@ -386,18 +641,6 @@ impl<'a> Writer<'_, 'a> {
386 }641 }
387 PageOp::Delete { object } => {642 PageOp::Delete { object } => {
388 self.target(*object)?;643 self.target(*object)?;
389 let page = self.page()?;
390 if matches!(page.view.nodes[object].kind, Kind::Paragraph { .. }) {
391 let container = self.container(*object)?;
392 if matches!(self.page()?.view.nodes[&container].kind, Kind::Cell { .. })
393 && levels::flat_children(&self.page()?.view, container) == [*object]
394 {
395 return Err(OpError::Unsupported(
396 "A table cell keeps a paragraph; insert its replacement first",
397 )
398 .into());
399 }
400 }
401 let edit = TreeEdit::delete(*object, self.author)?;644 let edit = TreeEdit::delete(*object, self.author)?;
402 self.write(|page| edit.changes(page))645 self.write(|page| edit.changes(page))
403 }646 }
crates/onestore/src/op/levels.rs-32
...@@ -14,21 +14,6 @@ fn invalid(message: &'static str) -> Error {...@@ -14,21 +14,6 @@ fn invalid(message: &'static str) -> Error {
14 Error { offset: 0, message }14 Error { offset: 0, message }
15}15}
1616
17/// The children of `container` with its outline groups lifted into it, in order.
18pub(crate) fn flat_children(view: &Revision<'_>, container: ExGuid) -> Vec<ExGuid> {
19 fn flat(view: &Revision<'_>, id: ExGuid, out: &mut Vec<ExGuid>) {
20 for child in &view.nodes[&id].children {
21 match view.nodes[child].kind {
22 Kind::OutlineGroup => flat(view, *child, out),
23 _ => out.push(*child),
24 }
25 }
26 }
27 let mut children = Vec::new();
28 flat(view, container, &mut children);
29 children
30}
31
32/// How much deeper than `container` each of its children lies, groups lifted.17/// How much deeper than `container` each of its children lies, groups lifted.
33pub(crate) fn stored_depths(view: &Revision<'_>, container: ExGuid) -> Vec<(ExGuid, u32)> {18pub(crate) fn stored_depths(view: &Revision<'_>, container: ExGuid) -> Vec<(ExGuid, u32)> {
34 fn walk(view: &Revision<'_>, id: ExGuid, base: u32, out: &mut Vec<(ExGuid, u32)>) {19 fn walk(view: &Revision<'_>, id: ExGuid, base: u32, out: &mut Vec<(ExGuid, u32)>) {
...@@ -46,23 +31,6 @@ pub(crate) fn stored_depths(view: &Revision<'_>, container: ExGuid) -> Vec<(ExGu...@@ -46,23 +31,6 @@ pub(crate) fn stored_depths(view: &Revision<'_>, container: ExGuid) -> Vec<(ExGu
46 out31 out
47}32}
4833
49/// Lifts the children of `container`'s outline groups into it.
50pub(crate) fn ungroup_changes(active: &ActivePage<'_>, container: ExGuid) -> Result<Changes, Error> {
51 let children = flat_children(&active.view, container);
52 let parents = active.editable_parents(container)?;
53 let modified = crate::create::current_timestamps()?.0.to_le_bytes();
54 let raw = &active.live.revision;
55 let mut holder = PropertyObject::from_object(&raw.objects[&container])?;
56 let mut references = Vec::new();
57 for child in children {
58 references.extend(holder.reference(child)?);
59 }
60 holder.set(&[(0x24001c20, &references), (0x14001d7a, &modified)])?;
61 let mut changed = BTreeMap::from([(container, holder)]);
62 crate::formatting::touch_ancestors(raw, parents, container, &modified, &mut changed)?;
63 Ok(changed)
64}
65
66/// Children in runs: one child at the container's child level, or a run sharing a deeper34/// Children in runs: one child at the container's child level, or a run sharing a deeper
67/// depth.35/// depth.
68pub(crate) type Runs = Vec<(u32, Vec<ExGuid>)>;36pub(crate) type Runs = Vec<(u32, Vec<ExGuid>)>;
crates/onestore/src/op/lower.rs+4-15
...@@ -44,6 +44,7 @@ pub fn lower(...@@ -44,6 +44,7 @@ pub fn lower(
44 identity: None,44 identity: None,
45 created: None,45 created: None,
46 margin_origin: [0.0; 2],46 margin_origin: [0.0; 2],
47 color: None,
47 objects: vec![PageObject::Outline(Outline {48 objects: vec![PageObject::Outline(Outline {
48 id: container,49 id: container,
49 title: false,50 title: false,
...@@ -163,16 +164,6 @@ impl<'a> View<'a> {...@@ -163,16 +164,6 @@ impl<'a> View<'a> {
163 self.paragraphs.get(&paragraph).and_then(|p| p.text())164 self.paragraphs.get(&paragraph).and_then(|p| p.text())
164 }165 }
165166
166 /// How much deeper than `container` each of its children lies: their outline levels
167 /// less the paragraph's, or less nothing in an outline or cell.
168 pub(crate) fn depths(&self, container: ExGuid, children: &[ExGuid]) -> Option<Vec<u32>> {
169 let base = self.paragraphs.get(&container).map_or(0, |p| p.level);
170 children
171 .iter()
172 .map(|id| self.paragraphs.get(id)?.level.checked_sub(base))
173 .collect()
174 }
175
176 /// Each outline's paragraphs in document order, a table's cells following its paragraph.167 /// Each outline's paragraphs in document order, a table's cells following its paragraph.
177 pub(crate) fn documents(&self) -> impl Iterator<Item = Vec<&'a PageParagraph>> + '_ {168 pub(crate) fn documents(&self) -> impl Iterator<Item = Vec<&'a PageParagraph>> + '_ {
178 fn walk<'a>(list: &'a [PageParagraph], out: &mut Vec<&'a PageParagraph>) {169 fn walk<'a>(list: &'a [PageParagraph], out: &mut Vec<&'a PageParagraph>) {
...@@ -677,11 +668,6 @@ pub(crate) fn paragraph_fields(...@@ -677,11 +668,6 @@ pub(crate) fn paragraph_fields(
677 })668 })
678}669}
679670
680/// `paragraph_fields` encoded, as the page writer stores them.
681pub(crate) fn paragraph_change(stored: &Paragraph, target: &Paragraph) -> Result<Values, Error> {
682 paragraph_fields(stored, target)?.values()
683}
684
685/// Whether two tag lists store the same tags.671/// Whether two tag lists store the same tags.
686pub(crate) fn same_tags(a: &[Tag], b: &[Tag]) -> bool {672pub(crate) fn same_tags(a: &[Tag], b: &[Tag]) -> bool {
687 let key = |t: &Tag| {673 let key = |t: &Tag| {
...@@ -1555,6 +1541,9 @@ impl Lowering {...@@ -1555,6 +1541,9 @@ impl Lowering {
1555 let shown = fields.iter().all(|(id, shown)| {1541 let shown = fields.iter().all(|(id, shown)| {
1556 model::paragraph_text(&self.current, *id).is_some_and(|text| text.text.text() == shown)1542 model::paragraph_text(&self.current, *id).is_some_and(|text| text.text.text() == shown)
1557 });1543 });
1544 if before.color != after.color {
1545 self.emit(PageOp::Color(after.color))?;
1546 }
1558 if before.created == after.created && shown {1547 if before.created == after.created && shown {
1559 return Ok(());1548 return Ok(());
1560 }1549 }
crates/onestore/src/op/mod.rs+19-4
...@@ -19,7 +19,7 @@ pub(crate) mod model;...@@ -19,7 +19,7 @@ pub(crate) mod model;
19pub(crate) mod properties;19pub(crate) mod properties;
20pub(crate) mod table;20pub(crate) mod table;
21#[cfg(test)]21#[cfg(test)]
22mod tests;22pub(crate) mod tests;
2323
24pub use lower::{lower, lower_page};24pub use lower::{lower, lower_page};
25/// `op` applied to `page`, the model of what the section holds: the page `Section::apply`25/// `op` applied to `page`, the model of what the section holds: the page `Section::apply`
...@@ -29,7 +29,7 @@ pub use model::apply as predict;...@@ -29,7 +29,7 @@ pub use model::apply as predict;
29/// Property identifiers with their encoded values.29/// Property identifiers with their encoded values.
30pub(crate) type Values = Vec<(u32, Vec<u8>)>;30pub(crate) type Values = Vec<(u32, Vec<u8>)>;
3131
32/// One user action, applied whole or not at all.32/// One user action, applied whole or not at all; it leaves every table cell a paragraph.
33#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]33#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
34#[serde(deny_unknown_fields)]34#[serde(deny_unknown_fields)]
35pub struct Edit {35pub struct Edit {
...@@ -80,6 +80,8 @@ pub enum PageOp {...@@ -80,6 +80,8 @@ pub enum PageOp {
80 created: u64,80 created: u64,
81 fields: Vec<(ExGuid, String)>,81 fields: Vec<(ExGuid, String)>,
82 },82 },
83 /// The page's colour (View, Page Color), COLORREF; `None` is OneNote's "No color".
84 Color(Option<u32>),
83 /// Inserts paragraphs before a direct child of `container`, or last. Children follow85 /// Inserts paragraphs before a direct child of `container`, or last. Children follow
84 /// their parents in `paragraphs`; levels are absolute. Text keeps its spans' formats;86 /// their parents in `paragraphs`; levels are absolute. Text keeps its spans' formats;
85 /// lists, tags, styles and collapse state are set by their own ops.87 /// lists, tags, styles and collapse state are set by their own ops.
...@@ -108,7 +110,7 @@ pub enum PageOp {...@@ -108,7 +110,7 @@ pub enum PageOp {
108 parent: Option<ExGuid>,110 parent: Option<ExGuid>,
109 before: Option<ExGuid>,111 before: Option<ExGuid>,
110 },112 },
111 /// Removes a subtree; an emptied table cell receives an empty paragraph.113 /// Removes a subtree.
112 Delete { object: ExGuid },114 Delete { object: ExGuid },
113 /// Sets a paragraph's absolute outline level, regrouping its container.115 /// Sets a paragraph's absolute outline level, regrouping its container.
114 Level { paragraph: ExGuid, level: u32 },116 Level { paragraph: ExGuid, level: u32 },
...@@ -204,10 +206,23 @@ pub enum SectionOp {...@@ -204,10 +206,23 @@ pub enum SectionOp {
204 /// A page created with `creation` holding `page`'s content under the identities it206 /// A page created with `creation` holding `page`'s content under the identities it
205 /// carries (`Page::copy` gives fresh ones); the created title stays.207 /// carries (`Page::copy` gives fresh ones); the created title stays.
206 Import { creation: PageCreation, page: Page },208 Import { creation: PageCreation, page: Page },
209 /// A read-only conflict page of the listed page `of`, as OneNote 2010 keeps the version of
210 /// a page a merge could not take: created with `creation` (not placed, its author the user
211 /// whose version it is, a title only where `page` has one) and holding `page` as `Import`
212 /// does, with `objects`, identities on `page`, marked as the conflicting ones.
213 Conflict {
214 of: ExGuid,
215 creation: PageCreation,
216 page: Page,
217 objects: Vec<ExGuid>,
218 },
207 /// Page moves and indentation, in order.219 /// Page moves and indentation, in order.
208 Pages(Vec<PageEdit>),220 Pages(Vec<PageEdit>),
209 /// Removes pages permanently; their revisions stay stored.221 /// Removes pages permanently, listed or conflict pages; their revisions stay stored.
210 Delete(Vec<ExGuid>),222 Delete(Vec<ExGuid>),
223 /// The section's colour as COLORREF, kept in its own metadata; `None` is OneNote's
224 /// "no colour".
225 Color(Option<u32>),
211}226}
212227
213/// A character property that `PageOp::Format` can clear so the text inherits it.228/// A character property that `PageOp::Format` can clear so the text inherits it.
crates/onestore/src/op/model.rs+2-7
...@@ -603,6 +603,7 @@ fn interpret(page: &mut Page, op: &PageOp) -> Result<(), Error> {...@@ -603,6 +603,7 @@ fn interpret(page: &mut Page, op: &PageOp) -> Result<(), Error> {
603 }603 }
604 page.created = Some(*created);604 page.created = Some(*created);
605 }605 }
606 PageOp::Color(color) => page.color = *color,
606 PageOp::Text { text, range, with } => {607 PageOp::Text { text, range, with } => {
607 let object = text_mut(page, *text)?;608 let object = text_mut(page, *text)?;
608 object.text = replace_text(&object.text, range.clone(), with)?;609 object.text = replace_text(&object.text, range.clone(), with)?;
...@@ -861,13 +862,7 @@ fn interpret(page: &mut Page, op: &PageOp) -> Result<(), Error> {...@@ -861,13 +862,7 @@ fn interpret(page: &mut Page, op: &PageOp) -> Result<(), Error> {
861 let (path, index) =862 let (path, index) =
862 find(page, *object).ok_or_else(|| invalid("The object is not on the page"))?;863 find(page, *object).ok_or_else(|| invalid("The object is not on the page"))?;
863 let list = list_at(page, &path);864 let list = list_at(page, &path);
864 let range = subtree(list, index);865 list.drain(subtree(list, index));
865 if !path.cells.is_empty() && range.len() == list.len() {
866 return Err(invalid(
867 "A table cell keeps a paragraph; insert its replacement first",
868 ));
869 }
870 list.drain(range);
871 }866 }
872 drop_emptied(page);867 drop_emptied(page);
873 retitle(page, None);868 retitle(page, None);
crates/onestore/src/op/table.rs-1
...@@ -35,7 +35,6 @@ pub(crate) fn validate_table(table: &Table) -> Result<(), Error> {...@@ -35,7 +35,6 @@ pub(crate) fn validate_table(table: &Table) -> Result<(), Error> {
35 return Err(invalid("Every table row has one cell per column"));35 return Err(invalid("Every table row has one cell per column"));
36 }36 }
37 for cell in &row.cells {37 for cell in &row.cells {
38 // An emptied cell keeps a replacement paragraph, as the tree writer provides.
39 for paragraph in &cell.paragraphs {38 for paragraph in &cell.paragraphs {
40 if let crate::page::ParagraphContent::Table(nested) = &paragraph.content {39 if let crate::page::ParagraphContent::Table(nested) = &paragraph.content {
41 validate_table(nested)?;40 validate_table(nested)?;
crates/onestore/src/op/tests.rs+282-219
...@@ -1,5 +1,6 @@...@@ -1,5 +1,6 @@
1//! Ops against the page-model writer: for random edits of corpus pages, lowering the edit1//! Ops against the model: for random edits of corpus pages, lowering the edit and applying
2//! and applying the ops through a `Section` stores what `PreparedEdit::page` stores.2//! the ops through a `Section` stores the edited page, as the model oracle predicts and as
3//! the sealed image reads back.
34
4use super::*;5use super::*;
5use crate::{6use crate::{
...@@ -11,7 +12,7 @@ use crate::{...@@ -11,7 +12,7 @@ use crate::{
11 },12 },
12 write::GUIDS,13 write::GUIDS,
13};14};
14use std::collections::BTreeMap;15use std::collections::{BTreeMap, BTreeSet};
1516
16const SOURCES: &[(&str, &[u8])] = &[17const SOURCES: &[(&str, &[u8])] = &[
17 ("outline-edit", include_bytes!("../../../../corpus/outline-edit/before/notebook/synthetic.one")),18 ("outline-edit", include_bytes!("../../../../corpus/outline-edit/before/notebook/synthetic.one")),
...@@ -85,20 +86,25 @@ fn read(image: &[u8], space: ExGuid) -> Page {...@@ -85,20 +86,25 @@ fn read(image: &[u8], space: ExGuid) -> Page {
85}86}
8687
87/// `page` with writer-chosen identities replaced in document order: list nodes, which a88/// `page` with writer-chosen identities replaced in document order: list nodes, which a
88/// copy for a second owner gets, and tag field indices.89/// copy for a second owner gets (so equal definitions become one), tag field indices, and ink
90/// coordinates at the resolution they are stored in.
89pub(super) fn normalize(page: &Page) -> Page {91pub(super) fn normalize(page: &Page) -> Page {
90 let mut page = page.clone();92 let mut page = page.clone();
91 let mut lists = BTreeMap::new();93 let mut lists = BTreeMap::new();
94 let mut canonical = BTreeMap::new();
92 let mut paths: Vec<_> = model::lists(&page).into_iter().map(|(path, _, _)| path).collect();95 let mut paths: Vec<_> = model::lists(&page).into_iter().map(|(path, _, _)| path).collect();
93 paths.sort_by_key(|path| format!("{path:?}"));96 paths.sort_by_key(|path| format!("{path:?}"));
97 let definitions = page.definitions.clone();
94 for path in &paths {98 for path in &paths {
95 for paragraph in model::list_at(&mut page, path).iter_mut() {99 for paragraph in model::list_at(&mut page, path).iter_mut() {
96 for list in &mut paragraph.lists {100 for list in &mut paragraph.lists {
97 let next = ExGuid {101 let next = ExGuid {
98 guid: [0xee; 16],102 guid: [0xee; 16],
99 n: 1 + lists.len() as u32,103 n: 1 + canonical.len() as u32,
100 };104 };
101 *list = *lists.entry(*list).or_insert(next);105 let content = definitions.get(list).map_or(format!("{list:?}"), |d| format!("{d:?}"));
106 let id = *canonical.entry(content).or_insert(next);
107 *list = *lists.entry(*list).or_insert(id);
102 }108 }
103 for tag in paragraph.tags.iter_mut() {109 for tag in paragraph.tags.iter_mut() {
104 tag.extra_set = 0;110 tag.extra_set = 0;
...@@ -136,9 +142,113 @@ pub(super) fn normalize(page: &Page) -> Page {...@@ -136,9 +142,113 @@ pub(super) fn normalize(page: &Page) -> Page {
136 _ => Some((id, definition)),142 _ => Some((id, definition)),
137 })143 })
138 .collect();144 .collect();
145 fn quantize(ink: &mut Ink) {
146 for stroke in &mut ink.strokes {
147 for point in &mut stroke.points {
148 *point = point.map(|value| (value * 10.0).round() / 10.0);
149 }
150 }
151 ink.groups.iter_mut().for_each(quantize);
152 }
153 for object in &mut page.objects {
154 if let PageObject::Ink(ink) = object {
155 quantize(ink);
156 }
157 }
158 for path in &paths {
159 for paragraph in model::list_at(&mut page, path).iter_mut() {
160 if let ParagraphContent::Ink(ink) = &mut paragraph.content {
161 quantize(ink);
162 }
163 }
164 }
139 page165 page
140}166}
141167
168/// `after` with what the writers derive rather than store as given: the title from the
169/// title text, the values they give new objects where the model leaves them unset, and only
170/// the definitions something uses.
171fn as_written(after: &Page, before: &Page, stored: &Page) -> Page {
172 use serde_json::Value;
173 fn ids(value: &Value, out: &mut BTreeSet<String>) {
174 match value {
175 Value::Object(map) => {
176 if let Some(Value::String(id)) = map.get("id") {
177 out.insert(id.clone());
178 }
179 map.values().for_each(|value| ids(value, out));
180 }
181 Value::Array(values) => values.iter().for_each(|value| ids(value, out)),
182 _ => {}
183 }
184 }
185 fn adopt(after: &mut Value, stored: &Value, old: &BTreeSet<String>, new: bool) {
186 match (after, stored) {
187 (Value::Object(after), Value::Object(stored)) => {
188 let new = new
189 || matches!(after.get("id"), Some(Value::String(id)) if !old.contains(id));
190 for (key, value) in after.iter_mut() {
191 let Some(written) = stored.get(key) else {
192 continue;
193 };
194 // Ink coordinates are stored at a coarser resolution.
195 let near = |a: &Value, b: &Value| {
196 a.as_f64().zip(b.as_f64()).is_some_and(|(a, b)| (a - b).abs() < 0.05)
197 };
198 let close_points = key == "points"
199 && value.as_array().zip(written.as_array()).is_some_and(|(a, b)| {
200 a.len() == b.len()
201 && a.iter().zip(b).all(|(a, b)| {
202 a.as_array().zip(b.as_array()).is_some_and(|(a, b)| {
203 a.len() == b.len() && a.iter().zip(b).all(|(a, b)| near(a, b))
204 })
205 })
206 });
207 let unset = new && (value.is_null() || value.as_array().is_some_and(Vec::is_empty));
208 if close_points || unset {
209 *value = written.clone();
210 } else {
211 adopt(value, written, old, new);
212 }
213 }
214 }
215 (Value::Array(after), Value::Array(stored)) if after.len() == stored.len() => {
216 for (value, written) in after.iter_mut().zip(stored) {
217 adopt(value, written, old, new);
218 }
219 }
220 _ => {}
221 }
222 }
223 let mut old = BTreeSet::new();
224 ids(&serde_json::to_value(before).unwrap(), &mut old);
225 let mut value = serde_json::to_value(after).unwrap();
226 adopt(&mut value, &serde_json::to_value(stored).unwrap(), &old, false);
227 let mut page: Page = serde_json::from_value(value).unwrap();
228 page.title = stored.title.clone();
229 unused_dropped(&mut page);
230 page
231}
232
233/// Drops the definitions nothing on `page` uses, as reading a page leaves them out.
234fn unused_dropped(page: &mut Page) {
235 let mut used = BTreeSet::new();
236 for (_, _, list) in model::lists(page) {
237 for paragraph in list {
238 used.extend(paragraph.style);
239 used.extend(paragraph.lists.iter().copied());
240 used.extend(paragraph.tags.iter().filter_map(|tag| tag.definition));
241 if let ParagraphContent::Table(table) = &paragraph.content {
242 used.extend(table.tags.iter().filter_map(|tag| tag.definition));
243 }
244 if let Some(text) = paragraph.text() {
245 used.extend(text.tags.iter().filter_map(|tag| tag.definition));
246 }
247 }
248 }
249 page.definitions.retain(|id, _| used.contains(id));
250}
251
142#[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord)]252#[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord)]
143pub(super) enum Family {253pub(super) enum Family {
144 Text,254 Text,
...@@ -224,7 +334,17 @@ fn id() -> ExGuid {...@@ -224,7 +334,17 @@ fn id() -> ExGuid {
224 new_id().unwrap()334 new_id().unwrap()
225}335}
226336
227fn text_paragraph(text: &str, format: Format) -> PageParagraph {337/// `ops` on page `space` of `source` as one edit, sealed: the image they leave.
338pub(crate) fn edited(source: &[u8], space: ExGuid, ops: Vec<PageOp>) -> Result<Vec<u8>, OpError> {
339 let arena = Arena::default();
340 let mut section = Section::open(&arena, source.to_vec()).map_err(OpError::Failed)?;
341 let ops = ops.into_iter().map(|op| Op::Page { space, op }).collect();
342 section.apply("Author", &Edit { at: AT, ops })?;
343 section.seal().map_err(OpError::Failed)?;
344 Ok(section.image())
345}
346
347pub(crate) fn text_paragraph(text: &str, format: Format) -> PageParagraph {
228 PageParagraph {348 PageParagraph {
229 id: id(),349 id: id(),
230 parent: None,350 parent: None,
...@@ -473,8 +593,10 @@ pub(super) fn mutate(page: &mut Page, rng: &mut Rng, family: Family) -> bool {...@@ -473,8 +593,10 @@ pub(super) fn mutate(page: &mut Page, rng: &mut Rng, family: Family) -> bool {
473 }593 }
474 inserted.push(paragraph);594 inserted.push(paragraph);
475 }595 }
476 // Children follow their parent; keep them after the first inserted paragraph.596 // Children follow their parent, the first inserted paragraph, directly.
477 inserted.sort_by_key(|p| p.parent.is_some());597 let (children, mut inserted): (Vec<_>, Vec<_>) =
598 inserted.into_iter().partition(|p| p.parent.is_some());
599 inserted.splice(1..1, children);
478 let cell_level = list.first().map_or(1, |p| p.level);600 let cell_level = list.first().map_or(1, |p| p.level);
479 for paragraph in &mut inserted {601 for paragraph in &mut inserted {
480 if paragraph.parent.is_none() {602 if paragraph.parent.is_none() {
...@@ -993,125 +1115,70 @@ fn stroke(rng: &mut Rng) -> InkStroke {...@@ -993,125 +1115,70 @@ fn stroke(rng: &mut Rng) -> InkStroke {
993 }1115 }
994}1116}
9951117
996/// How one differential case came out.1118/// How one lowered edit came out.
997#[derive(Debug, PartialEq, Eq, PartialOrd, Ord, Clone)]1119#[derive(Debug, PartialEq, Eq, PartialOrd, Ord, Clone)]
998pub(super) enum Outcome {1120pub(super) enum Outcome {
999 /// The edit made no change the writers store.1121 /// The edit made no change the writers store.
1000 Unchanged,1122 Unchanged,
1001 /// Both refused the edit.1123 /// Lowering or the section refused the edit; the section is as it was.
1002 Refused,1124 Refused,
1003 /// The page writer refused; the ops were written.1125 /// The sealed image reads back as the edited page.
1004 OnlyOps,1126 Stored,
1005 /// Identical bytes.
1006 Bytes,
1007 /// Equal pages, both images valid, the op revision no larger.
1008 Model,
1009 /// The op revision declares more than the page writer's.
1010 Larger,
1011 Mismatch(String),1127 Mismatch(String),
1012}1128}
10131129
1014/// Objects the space's active revision declares in `image`.1130/// Lowers `after` over page `space` of `source`, applies and seals the ops, and compares
1015pub(super) fn declared(image: &[u8], space: ExGuid) -> usize {1131/// the page the section holds with the edited page, with the model oracle's prediction and
1016 let store = Store::parse(image).unwrap();1132/// with the page the sealed image reads back as.
1017 let index = RevisionIndex::parse(&store).unwrap();1133pub(super) fn stored(source: &[u8], space: ExGuid, after: &Page, seed: u64) -> Outcome {
1018 let rid = index.active(space).unwrap();
1019 index.spaces[&space].revisions[&rid]
1020 .nodes
1021 .iter()
1022 .filter_map(|node| match node.reference {
1023 Some(crate::Reference::NodeList(chunk)) if node.id == 0xb0 => Some(chunk),
1024 _ => None,
1025 })
1026 .map(|chunk| {
1027 store
1028 .lists
1029 .values()
1030 .find(|list| list.fragments.first().is_some_and(|f| f.offset == chunk.offset))
1031 .map_or(0, |list| {
1032 list.nodes
1033 .iter()
1034 .filter(|n| {
1035 matches!(n.id, 0x2d | 0x2e | 0x41 | 0x42 | 0xa4 | 0xa5 | 0xc4 | 0xc5 | 0x72 | 0x73)
1036 })
1037 .count()
1038 })
1039 })
1040 .sum()
1041}
1042
1043/// Writes `after` over page `space` of `source` both ways and compares the images.
1044pub(super) fn differential(source: &[u8], space: ExGuid, after: &Page, seed: u64) -> Outcome {
1045 let before = read(source, space);1134 let before = read(source, space);
1046 let oracle = seeded(seed, || {
1047 crate::create::at(AT, || crate::PreparedEdit::page(source, space, after, "Author"))
1048 });
1049 let arena = Arena::default();1135 let arena = Arena::default();
1050 let mut section = Section::open(&arena, source.to_vec()).unwrap();1136 let mut section = Section::open(&arena, source.to_vec()).unwrap();
1051 let ops = lower_page(&before, after);1137 let Ok(ops) = lower_page(&before, after) else {
1052 let written = ops.as_ref().ok().map(|ops| {1138 return Outcome::Refused;
1053 seeded(seed, || {1139 };
1054 let edit = Edit {1140 let edit = Edit {
1055 at: AT,1141 at: AT,
1056 ops: ops1142 ops: ops.iter().map(|op| Op::Page { space, op: op.clone() }).collect(),
1057 .iter()1143 };
1058 .map(|op| Op::Page {1144 if seeded(seed, || section.apply("Author", &edit)).is_err() {
1059 space,1145 if section.page(space).unwrap() != before {
1060 op: op.clone(),1146 return Outcome::Mismatch("a refused edit changed the page".into());
1061 })
1062 .collect(),
1063 };
1064 section.apply("Author", &edit).map(|()| {
1065 let predicted = {
1066 let mut page = before.clone();
1067 for op in ops {
1068 model::apply(&mut page, op).unwrap();
1069 }
1070 page
1071 };
1072 (section.seal().unwrap(), predicted)
1073 })
1074 })
1075 });
1076 let image = section.image();
1077 match (oracle, written) {
1078 (Err(_), None | Some(Err(_))) => Outcome::Refused,
1079 (Err(_), Some(Ok(_))) => Outcome::OnlyOps,
1080 (Ok(_), None) => Outcome::Mismatch(format!("lowering refused: {:?}", ops.unwrap_err())),
1081 (Ok(_), Some(Err(error))) => Outcome::Mismatch(format!("apply refused: {error:?} for {:?}", ops.unwrap())),
1082 (Ok(edit), Some(Ok((transaction, predicted)))) => {
1083 let stored = normalize(&read(&image, space));
1084 if normalize(&predicted) != stored {
1085 return Outcome::Mismatch(format!(
1086 "the model predicts otherwise: {}",
1087 first_difference(&stored, &normalize(&predicted))
1088 ));
1089 }
1090 let expected = edit.as_bytes();
1091 if expected == source && transaction.is_none() {
1092 return Outcome::Unchanged;
1093 }
1094 if expected == image.as_slice() {
1095 return Outcome::Bytes;
1096 }
1097 let store = Store::parse(&image).unwrap();
1098 let index = RevisionIndex::parse(&store).unwrap();
1099 if let Err(error) = index.validate_current() {
1100 return Outcome::Mismatch(format!("invalid image: {error:?}"));
1101 }
1102 let (ours, theirs) = (normalize(&read(&image, space)), normalize(&read(expected, space)));
1103 if ours != theirs {
1104 return Outcome::Mismatch(first_difference(&theirs, &ours));
1105 }
1106 if expected == source {
1107 return Outcome::Model;
1108 }
1109 if declared(&image, space) > declared(expected, space) {
1110 return Outcome::Larger;
1111 }
1112 Outcome::Model
1113 }1147 }
1148 return Outcome::Refused;
1149 }
1150 let mut predicted = before.clone();
1151 for op in &ops {
1152 model::apply(&mut predicted, op).unwrap();
1153 }
1154 let live = section.page(space).unwrap();
1155 if normalize(&predicted) != normalize(&live) {
1156 return Outcome::Mismatch(format!(
1157 "the model predicts otherwise: {}",
1158 first_difference(&normalize(&live), &normalize(&predicted))
1159 ));
1160 }
1161 let Some(_) = seeded(seed, || section.seal().unwrap()) else {
1162 return if live == before { Outcome::Unchanged } else {
1163 Outcome::Mismatch("the page changed and the seal stored nothing".into())
1164 };
1165 };
1166 let image = section.image();
1167 let store = Store::parse(&image).unwrap();
1168 let index = RevisionIndex::parse(&store).unwrap();
1169 if let Err(error) = index.validate_current() {
1170 return Outcome::Mismatch(format!("invalid image: {error:?}"));
1114 }1171 }
1172 let reread = read(&image, space);
1173 if reread != live {
1174 return Outcome::Mismatch(format!("reads back otherwise: {}", first_difference(&live, &reread)));
1175 }
1176 let written = as_written(after, &before, &reread);
1177 let (stored, wanted) = (normalize(&reread), normalize(&written));
1178 if stored != wanted {
1179 return Outcome::Mismatch(format!("stores another page: {}", first_difference(&wanted, &stored)));
1180 }
1181 Outcome::Stored
1115}1182}
11161183
1117fn spans(text: &Paragraph) -> String {1184fn spans(text: &Paragraph) -> String {
...@@ -1155,7 +1222,7 @@ pub(super) fn first_difference(expected: &Page, actual: &Page) -> String {...@@ -1155,7 +1222,7 @@ pub(super) fn first_difference(expected: &Page, actual: &Page) -> String {
11551222
1156/// Every family on every corpus page it applies to, several seeds each.1223/// Every family on every corpus page it applies to, several seeds each.
1157#[test]1224#[test]
1158fn ops_store_what_the_page_writer_stores() {1225fn lowered_edits_store_the_edited_page() {
1159 let mut tally: BTreeMap<(Family, String), usize> = BTreeMap::new();1226 let mut tally: BTreeMap<(Family, String), usize> = BTreeMap::new();
1160 let mut mismatches = Vec::new();1227 let mut mismatches = Vec::new();
1161 let seeds = std::env::var("OP_SEEDS").map_or(3, |s| s.parse().unwrap());1228 let seeds = std::env::var("OP_SEEDS").map_or(3, |s| s.parse().unwrap());
...@@ -1170,7 +1237,7 @@ fn ops_store_what_the_page_writer_stores() {...@@ -1170,7 +1237,7 @@ fn ops_store_what_the_page_writer_stores() {
1170 continue;1237 continue;
1171 }1238 }
1172 GUIDS.set(None);1239 GUIDS.set(None);
1173 let outcome = differential(source, space, &after, 1 << 40 | seed);1240 let outcome = stored(source, space, &after, 1 << 40 | seed);
1174 let key = match &outcome {1241 let key = match &outcome {
1175 Outcome::Mismatch(_) => "Mismatch".to_owned(),1242 Outcome::Mismatch(_) => "Mismatch".to_owned(),
1176 other => format!("{other:?}"),1243 other => format!("{other:?}"),
...@@ -1192,7 +1259,6 @@ fn ops_store_what_the_page_writer_stores() {...@@ -1192,7 +1259,6 @@ fn ops_store_what_the_page_writer_stores() {
1192 println!("{mismatch}\n");1259 println!("{mismatch}\n");
1193 }1260 }
1194 assert!(mismatches.is_empty(), "{} mismatches", mismatches.len());1261 assert!(mismatches.is_empty(), "{} mismatches", mismatches.len());
1195 assert!(!tally.keys().any(|(_, outcome)| outcome == "Larger"));
1196}1262}
11971263
1198#[test]1264#[test]
...@@ -1674,42 +1740,34 @@ fn random_ops_read_back_as_the_model_predicts() {...@@ -1674,42 +1740,34 @@ fn random_ops_read_back_as_the_model_predicts() {
1674 assert!(applied > refused);1740 assert!(applied > refused);
1675}1741}
16761742
1677/// Page creation, moves, indentation and removal through a section store what the image1743/// Page creation, moves, indentation and removal through a section leave the page list
1678/// writers store, and the section lists the pages the document does.1744/// they describe, and the sealed image lists the pages the section does.
1679#[test]1745#[test]
1680fn section_ops_store_what_the_image_writers_store() {1746fn section_ops_leave_the_page_list_they_describe() {
1681 let sources: Vec<Vec<u8>> = vec![1747 let sources: Vec<Vec<u8>> = vec![
1682 crate::create_section("pages.one", "First", "Author").unwrap(),1748 crate::create_section("pages.one", "First", "Author").unwrap(),
1683 include_bytes!("../../../../corpus/page-lifecycle/04-nested/notebook/Lifecycle.one").to_vec(),1749 include_bytes!("../../../../corpus/page-lifecycle/04-nested/notebook/Lifecycle.one").to_vec(),
1684 SOURCES[3].1.to_vec(),1750 SOURCES[3].1.to_vec(),
1685 ];1751 ];
1686 let (mut bytes, mut total) = (0, 0);1752 let mut applied = 0;
1687 for (n, source) in sources.iter().enumerate() {1753 for (n, source) in sources.iter().enumerate() {
1688 let arena = Arena::default();1754 let arena = Arena::default();
1689 let mut section = Section::open(&arena, source.to_vec()).unwrap();1755 let mut section = Section::open(&arena, source.to_vec()).unwrap();
1690 let mut image = source.clone();
1691 let mut rng = Rng(n as u64 + 17);1756 let mut rng = Rng(n as u64 + 17);
1692 for step in 0..12u64 {1757 for step in 0..12u64 {
1693 let spaces = pages(&image);1758 let listed = section.pages().unwrap();
1759 let spaces: Vec<ExGuid> = listed.iter().map(|(space, ..)| *space).collect();
1760 let anchor = spaces.get(rng.pick(spaces.len()).unwrap_or(0)).copied().filter(|_| rng.coin());
1694 let op = match step % 4 {1761 let op = match step % 4 {
1695 0 => SectionOp::Create(1762 0 => SectionOp::Create(crate::PageCreation::new(anchor, Some("Created"), "Author").unwrap()),
1696 seeded(1 << 50 | step << 20, || {
1697 crate::PageCreation::new(
1698 spaces.get(rng.pick(spaces.len()).unwrap_or(0)).copied().filter(|_| rng.coin()),
1699 Some("Created"),
1700 "Author",
1701 )
1702 })
1703 .unwrap(),
1704 ),
1705 1 => {1763 1 => {
1706 let Some(space) = spaces.get(rng.pick(spaces.len()).unwrap_or(0)).copied() else { continue };1764 let Some(space) = spaces.get(rng.pick(spaces.len()).unwrap_or(0)).copied() else { continue };
1707 let level = if spaces[0] == space { 1 } else { 1 + (rng.next() % 2) as u32 };1765 let level = if spaces[0] == space { 1 } else { 1 + (rng.next() % 2) as u32 };
1708 SectionOp::Pages(vec![seeded(1 << 51 | step << 20, || crate::PageEdit::set_level(space, level)).unwrap()])1766 SectionOp::Pages(vec![crate::PageEdit::set_level(space, level).unwrap()])
1709 }1767 }
1710 2 => {1768 2 => {
1711 let Some(space) = spaces.get(rng.pick(spaces.len()).unwrap_or(0)).copied() else { continue };1769 let Some(space) = spaces.get(rng.pick(spaces.len()).unwrap_or(0)).copied() else { continue };
1712 SectionOp::Pages(vec![seeded(1 << 52 | step << 20, || crate::PageEdit::move_to(space, None, 1)).unwrap()])1770 SectionOp::Pages(vec![crate::PageEdit::move_to(space, None, 1).unwrap()])
1713 }1771 }
1714 _ => {1772 _ => {
1715 if spaces.len() < 2 {1773 if spaces.len() < 2 {
...@@ -1718,63 +1776,71 @@ fn section_ops_store_what_the_image_writers_store() {...@@ -1718,63 +1776,71 @@ fn section_ops_store_what_the_image_writers_store() {
1718 SectionOp::Delete(vec![spaces[rng.pick(spaces.len()).unwrap()]])1776 SectionOp::Delete(vec![spaces[rng.pick(spaces.len()).unwrap()]])
1719 }1777 }
1720 };1778 };
1721 let expected = seeded(1 << 40 | step << 16, || {1779 let edit = Edit { at: AT, ops: vec![Op::Section(op.clone())] };
1722 crate::create::at(AT, || match &op {1780 if section.apply("Author", &edit).is_err() {
1723 SectionOp::Create(creation) => crate::PreparedEdit::create_page(&image, creation),1781 assert_eq!(section.pages().unwrap(), listed, "{n} step {step} {op:?}");
1724 SectionOp::Pages(edits) => crate::PreparedEdit::pages(&image, edits),1782 continue;
1725 SectionOp::Delete(spaces) => crate::PreparedEdit::delete_pages_permanently(&image, spaces),1783 }
1726 SectionOp::Import { .. } => unreachable!(),1784 section.seal().unwrap();
1727 })1785 applied += 1;
1728 .map(|edit| edit.as_bytes().to_vec())1786 let after = section.pages().unwrap();
1729 });1787 let order = |list: &[(ExGuid, String, u32)], without: ExGuid| -> Vec<ExGuid> {
1730 let applied = seeded(1 << 40 | step << 16, || {1788 list.iter().map(|(space, ..)| *space).filter(|space| *space != without).collect()
1731 section1789 };
1732 .apply("Author", &Edit { at: AT, ops: vec![Op::Section(op.clone())] })1790 match &op {
1733 .map(|()| section.seal().unwrap())1791 SectionOp::Create(creation) => {
1734 });1792 let at = after.iter().position(|(space, ..)| *space == creation.space()).unwrap();
1735 match (expected, applied) {1793 assert_eq!(after[at].1, "Created");
1736 (Err(_), Err(_)) => continue,1794 assert_eq!(order(&after, creation.space()), spaces, "{n} step {step}");
1737 (Ok(expected), Ok(_)) => {1795 match anchor {
1738 total += 1;1796 Some(before) => assert_eq!(after[at + 1].0, before),
1739 let written = section.image();1797 None => assert_eq!(at + 1, after.len()),
1740 if written == expected {1798 }
1741 bytes += 1;1799 }
1800 SectionOp::Pages(edits) => {
1801 let space = edits[0].space();
1802 let at = after.iter().position(|(listed, ..)| *listed == space).unwrap();
1803 assert_eq!(after[at].2, edits[0].level(), "{n} step {step} {op:?}");
1804 if edits[0].position() == crate::PagePosition::Keep {
1805 assert_eq!(order(&after, space), order(&listed, space));
1806 assert_eq!(after.iter().map(|(s, ..)| *s).collect::<Vec<_>>(), spaces);
1742 } else {1807 } else {
1743 let store = Store::parse(&written).unwrap();1808 assert_eq!(after.last().unwrap().0, space, "{n} step {step}");
1744 RevisionIndex::parse(&store).unwrap().validate_current().unwrap();
1745 let spaces = pages(&expected);
1746 assert_eq!(pages(&written), spaces, "{n} step {step} {op:?}");
1747 for space in spaces {
1748 assert_eq!(read(&written, space), read(&expected, space), "{n} step {step}");
1749 }
1750 }1809 }
1751 image = expected;
1752 let store = Store::parse(&image).unwrap();
1753 let index = RevisionIndex::parse(&store).unwrap();
1754 let document = Document::parse(&index).unwrap();
1755 let listed: Vec<(ExGuid, String, u32)> = document
1756 .pages()
1757 .unwrap()
1758 .into_iter()
1759 .map(|(space, page)| {
1760 let (title, level) = Page::heading(document.active(space).unwrap(), page);
1761 (space, title, level)
1762 })
1763 .collect();
1764 assert_eq!(section.pages().unwrap(), listed, "{n} step {step}");
1765 }1810 }
1766 (expected, applied) => panic!("{n} step {step} {op:?}: {:?} vs {:?}", expected.err(), applied.err()),1811 SectionOp::Delete(deleted) => {
1812 assert_eq!(after.iter().map(|(s, ..)| *s).collect::<Vec<_>>(), order(&listed, deleted[0]));
1813 }
1814 SectionOp::Import { .. } | SectionOp::Conflict { .. } | SectionOp::Color(_) => {
1815 unreachable!()
1816 }
1817 }
1818 let image = section.image();
1819 let store = Store::parse(&image).unwrap();
1820 let index = RevisionIndex::parse(&store).unwrap();
1821 index.validate_current().unwrap();
1822 let document = Document::parse(&index).unwrap();
1823 let reread: Vec<(ExGuid, String, u32)> = document
1824 .pages()
1825 .unwrap()
1826 .into_iter()
1827 .map(|(space, page)| {
1828 let (title, level) = Page::heading(document.active(space).unwrap(), page);
1829 (space, title, level)
1830 })
1831 .collect();
1832 assert_eq!(reread, after, "{n} step {step}");
1833 for (space, ..) in &after {
1834 assert_eq!(read(&image, *space), section.page(*space).unwrap(), "{n} step {step}");
1767 }1835 }
1768 }1836 }
1769 }1837 }
1770 println!("{bytes} of {total} section edits byte-identical");1838 assert!(applied > 12, "{applied} section edits applied");
1771 assert!(total > 12);
1772}1839}
17731840
1774/// Importing a page stores what creating it and writing its copy through the page writer1841/// An imported page reads back as its copy under the created page's title.
1775/// does.
1776#[test]1842#[test]
1777fn an_imported_page_reads_as_the_page_writer_leaves_it() {1843fn an_imported_page_reads_back_as_its_copy() {
1778 let target = crate::create_section("import.one", "First", "Author").unwrap();1844 let target = crate::create_section("import.one", "First", "Author").unwrap();
1779 let mut imported = 0;1845 let mut imported = 0;
1780 for (name, source) in SOURCES {1846 for (name, source) in SOURCES {
...@@ -1783,48 +1849,45 @@ fn an_imported_page_reads_as_the_page_writer_leaves_it() {...@@ -1783,48 +1849,45 @@ fn an_imported_page_reads_as_the_page_writer_leaves_it() {
1783 continue;1849 continue;
1784 };1850 };
1785 let creation = crate::PageCreation::new(None, Some(&copy.title), "Author").unwrap();1851 let creation = crate::PageCreation::new(None, Some(&copy.title), "Author").unwrap();
1786 let created = crate::PreparedEdit::create_page(&target, &creation).unwrap();1852 let arena = Arena::default();
1787 let created = created.as_bytes();1853 let mut section = Section::open(&arena, target.clone()).unwrap();
1788 let mut after = read(created, creation.space());1854 let import = |section: &mut Section<'_>, op: SectionOp| {
1789 after.objects.retain(|object| matches!(object, PageObject::Title(_)));1855 section.apply("Author", &Edit { at: AT, ops: vec![Op::Section(op)] })
1790 after.objects.extend(1856 };
1857 import(&mut section, SectionOp::Create(creation.clone())).unwrap();
1858 let created = section.page(creation.space()).unwrap();
1859 let mut expected = created.clone();
1860 let arena = Arena::default();
1861 let mut section = Section::open(&arena, target.clone()).unwrap();
1862 if import(&mut section, SectionOp::Import { creation: creation.clone(), page: copy.clone() }).is_err() {
1863 continue;
1864 }
1865 section.seal().unwrap();
1866 let image = section.image();
1867 let store = Store::parse(&image).unwrap();
1868 RevisionIndex::parse(&store).unwrap().validate_current().unwrap();
1869 expected.objects.retain(|object| matches!(object, PageObject::Title(_)));
1870 expected.objects.extend(
1791 copy.objects1871 copy.objects
1792 .iter()1872 .iter()
1793 .filter(|object| !matches!(object, PageObject::Title(_)))1873 .filter(|object| !matches!(object, PageObject::Title(_)))
1794 .cloned(),1874 .cloned(),
1795 );1875 );
1796 after.definitions = copy.definitions.clone();1876 expected.definitions.extend(copy.definitions.clone());
1797 let expected = crate::create::at(AT, || {1877 let stored = read(&image, creation.space());
1798 crate::PreparedEdit::page(created, creation.space(), &after, "Author")1878 // New page children stack as the writers add them.
1799 });1879 let order: Vec<ExGuid> = stored.objects.iter().map(PageObject::id).collect();
1800 let arena = Arena::default();1880 expected.objects.sort_by_key(|object| order.iter().position(|id| *id == object.id()));
1801 let mut section = Section::open(&arena, target.clone()).unwrap();1881 let mut expected = as_written(&expected, &created, &stored);
1802 let applied = section.apply(1882 // An outline's height follows its content; no op carries it.
1803 "Author",1883 for (object, stored) in expected.objects.iter_mut().zip(&stored.objects) {
1804 &Edit {1884 if let (PageObject::Outline(outline), PageObject::Outline(stored)) = (object, stored) {
1805 at: AT,1885 outline.layout.max_height = stored.layout.max_height;
1806 ops: vec![Op::Section(SectionOp::Import {
1807 creation: creation.clone(),
1808 page: copy.clone(),
1809 })],
1810 },
1811 );
1812 match (expected, applied) {
1813 (Err(_), Err(_)) => continue,
1814 (Ok(expected), Ok(())) => {
1815 section.seal().unwrap();
1816 let image = section.image();
1817 let store = Store::parse(&image).unwrap();
1818 RevisionIndex::parse(&store).unwrap().validate_current().unwrap();
1819 let (ours, theirs) = (
1820 normalize(&read(&image, creation.space())),
1821 normalize(&read(expected.as_bytes(), creation.space())),
1822 );
1823 assert!(ours == theirs, "{name}: {}", first_difference(&theirs, &ours));
1824 imported += 1;
1825 }1886 }
1826 (expected, applied) => panic!("{name}: {:?} vs {:?}", expected.err(), applied.err()),
1827 }1887 }
1888 let (ours, theirs) = (normalize(&stored), normalize(&expected));
1889 assert!(ours == theirs, "{name}: {}", first_difference(&theirs, &ours));
1890 imported += 1;
1828 }1891 }
1829 }1892 }
1830 assert!(imported > 10, "{imported} pages imported");1893 assert!(imported > 10, "{imported} pages imported");
crates/onestore/src/outline.rs+12-17
...@@ -25,15 +25,6 @@ pub enum OutlineEdit {...@@ -25,15 +25,6 @@ pub enum OutlineEdit {
25}25}
2626
27impl OutlineEdit {27impl OutlineEdit {
28 pub(crate) fn apply(
29 self,
30 source: &[u8],
31 space: ExGuid,
32 object: ExGuid,
33 ) -> Result<Vec<u8>, Error> {
34 crate::active::write(source, space, |active| self.changes(active, object))
35 }
36
37 pub(crate) fn changes(self, active: &ActivePage<'_>, object: ExGuid) -> Result<Changes, Error> {28 pub(crate) fn changes(self, active: &ActivePage<'_>, object: ExGuid) -> Result<Changes, Error> {
38 let invalid = |message| Error { offset: 0, message };29 let invalid = |message| Error { offset: 0, message };
39 let values = match self {30 let values = match self {
...@@ -146,8 +137,12 @@ impl OutlineEdit {...@@ -146,8 +137,12 @@ impl OutlineEdit {
146#[cfg(test)]137#[cfg(test)]
147mod tests {138mod tests {
148 use super::*;139 use super::*;
149 use crate::write::write_revision;140 use crate::{
150 use crate::{RevisionIndex, Store, document::Document};141 RevisionIndex, Store,
142 document::Document,
143 op::{PageOp, tests::edited},
144 write::write_revision,
145 };
151146
152 #[test]147 #[test]
153 fn protection_on_the_target_or_ancestor_prevents_layout_edits() {148 fn protection_on_the_target_or_ancestor_prevents_layout_edits() {
...@@ -184,7 +179,7 @@ mod tests {...@@ -184,7 +179,7 @@ mod tests {
184 (paragraph, OutlineEdit::Collapsed(true)),179 (paragraph, OutlineEdit::Collapsed(true)),
185 ] {180 ] {
186 assert_eq!(181 assert_eq!(
187 edit.apply(&bytes, sid, object).is_err(),182 edited(&bytes, sid, vec![PageOp::Outline { object, edit }]).is_err(),
188 enabled,183 enabled,
189 "{target} {property:x} {edit:?}"184 "{target} {property:x} {edit:?}"
190 );185 );
...@@ -199,10 +194,10 @@ mod tests {...@@ -199,10 +194,10 @@ mod tests {
199 Ok(BTreeMap::from([(outline, object)]))194 Ok(BTreeMap::from([(outline, object)]))
200 })195 })
201 .unwrap();196 .unwrap();
202 assert!(197 let collapse = PageOp::Outline {
203 OutlineEdit::Collapsed(true)198 object: paragraph,
204 .apply(&duplicated, sid, paragraph)199 edit: OutlineEdit::Collapsed(true),
205 .is_err()200 };
206 );201 assert!(edited(&duplicated, sid, vec![collapse]).is_err());
207 }202 }
208}203}
crates/onestore/src/page/mod.rs+49-4
...@@ -17,7 +17,6 @@ pub mod text;...@@ -17,7 +17,6 @@ pub mod text;
1717
18pub use ink::{Ink, InkStroke};18pub use ink::{Ink, InkStroke};
19pub use math::Math;19pub use math::Math;
20pub(crate) mod write;
2120
22/// Picture and attachment payloads travel with the model (queued intents replay them),21/// Picture and attachment payloads travel with the model (queued intents replay them),
23/// as base64 text.22/// as base64 text.
...@@ -52,6 +51,10 @@ mod payload {...@@ -52,6 +51,10 @@ mod payload {
52}51}
53pub use text::Paragraph;52pub use text::Paragraph;
5453
54/// The page node's colour (View, Page Color) as COLORREF, absent for "No color"; not in
55/// MS-ONE, observed in OneNote 2010's pages (`evidence/notebook-management`).
56pub(crate) const PAGE_COLOR: u32 = 0x14001d2a;
57
55/// The role of a title-outline paragraph that displays the page's creation date or time.58/// The role of a title-outline paragraph that displays the page's creation date or time.
56#[derive(Clone, Copy, Debug, PartialEq, serde::Serialize, serde::Deserialize)]59#[derive(Clone, Copy, Debug, PartialEq, serde::Serialize, serde::Deserialize)]
57pub enum DateField {60pub enum DateField {
...@@ -67,6 +70,9 @@ pub struct Page {...@@ -67,6 +70,9 @@ pub struct Page {
67 /// FILETIME ticks from the page's TopologyCreationTimeStamp.70 /// FILETIME ticks from the page's TopologyCreationTimeStamp.
68 pub created: Option<u64>,71 pub created: Option<u64>,
69 pub margin_origin: [f32; 2],72 pub margin_origin: [f32; 2],
73 /// The page's colour (View, Page Color), COLORREF; `None` is OneNote's "No color".
74 #[serde(default)]
75 pub color: Option<u32>,
70 pub objects: Vec<PageObject>,76 pub objects: Vec<PageObject>,
71 pub definitions: BTreeMap<ExGuid, Definition>,77 pub definitions: BTreeMap<ExGuid, Definition>,
72}78}
...@@ -394,9 +400,14 @@ impl Page {...@@ -394,9 +400,14 @@ impl Page {
394 /// links keep naming the original. Content outside the model has no copy, so an400 /// links keep naming the original. Content outside the model has no copy, so an
395 /// `Unsupported` object is an error.401 /// `Unsupported` object is an error.
396 pub fn copy(&self) -> Result<Self, Error> {402 pub fn copy(&self) -> Result<Self, Error> {
403 self.copy_with(&mut [])
404 }
405
406 /// `copy`, renaming `ids`, identities on this page, to theirs on the copy.
407 pub fn copy_with(&self, ids: &mut [ExGuid]) -> Result<Self, Error> {
397 let invalid = |message| Error { offset: 0, message };408 let invalid = |message| Error { offset: 0, message };
398 let mut value =409 let mut value = serde_json::to_value((self, &*ids))
399 serde_json::to_value(self).map_err(|_| invalid("Page is not serializable"))?;410 .map_err(|_| invalid("Page is not serializable"))?;
400 let mut fresh: BTreeMap<String, String> = BTreeMap::new();411 let mut fresh: BTreeMap<String, String> = BTreeMap::new();
401 fn walk(412 fn walk(
402 value: &mut serde_json::Value,413 value: &mut serde_json::Value,
...@@ -444,8 +455,9 @@ impl Page {...@@ -444,8 +455,9 @@ impl Page {
444 Ok(())455 Ok(())
445 }456 }
446 walk(&mut value, &mut fresh)?;457 walk(&mut value, &mut fresh)?;
447 let mut copy: Self =458 let (mut copy, renamed): (Self, Vec<ExGuid>) =
448 serde_json::from_value(value).map_err(|_| invalid("Page copy does not deserialize"))?;459 serde_json::from_value(value).map_err(|_| invalid("Page copy does not deserialize"))?;
460 ids.copy_from_slice(&renamed);
449 copy.identity = Some(crate::write::fresh_guid()?);461 copy.identity = Some(crate::write::fresh_guid()?);
450 // A paragraph's indent level follows its parent chain; a level beyond that comes462 // A paragraph's indent level follows its parent chain; a level beyond that comes
451 // from an outline group, which the model does not hold.463 // from an outline group, which the model does not hold.
...@@ -477,6 +489,28 @@ impl Page {...@@ -477,6 +489,28 @@ impl Page {
477 Ok(copy)489 Ok(copy)
478 }490 }
479491
492 /// The date and time the title's date fields show, when it has both.
493 pub fn date_text(&self) -> Option<[String; 2]> {
494 let title = self.objects.iter().find_map(|object| match object {
495 PageObject::Title(title) => Some(title),
496 _ => None,
497 })?;
498 let outline = title.outlines.iter().find(|outline| Some(outline.id) == title.date)?;
499 let [first, second] = &outline.paragraphs[..] else {
500 return None;
501 };
502 let [first, second] = [first.text()?, second.text()?];
503 let text = |object: &TextObject| object.text.text().to_owned();
504 // OneNote writes the date first; a field marked as the other role says otherwise.
505 Some(
506 if first.date_field == Some(DateField::Time) || second.date_field == Some(DateField::Date) {
507 [text(second), text(first)]
508 } else {
509 [text(first), text(second)]
510 },
511 )
512 }
513
480 /// The notebook-management identity of the page in an active page revision, which514 /// The notebook-management identity of the page in an active page revision, which
481 /// internal links name.515 /// internal links name.
482 pub fn identity_of(revision: &Revision<'_>) -> Option<[u8; 16]> {516 pub fn identity_of(revision: &Revision<'_>) -> Option<[u8; 16]> {
...@@ -562,6 +596,17 @@ impl Page {...@@ -562,6 +596,17 @@ impl Page {
562 margin_origin_x.unwrap_or(1.0_f32 * 36.0),596 margin_origin_x.unwrap_or(1.0_f32 * 36.0),
563 margin_origin_y.unwrap_or(0.4_f32 * 36.0),597 margin_origin_y.unwrap_or(0.4_f32 * 36.0),
564 ],598 ],
599 color: root
600 .extra
601 .first()
602 .and_then(|fields| fields.iter().find(|field| field.id == PAGE_COLOR))
603 .map(|field| match field.value {
604 FieldValue::Bytes(&[red, green, blue, alpha]) => {
605 Ok(u32::from_le_bytes([red, green, blue, alpha]))
606 }
607 _ => Err(invalid("The page colour is damaged")),
608 })
609 .transpose()?,
565 objects: Vec::new(),610 objects: Vec::new(),
566 definitions: BTreeMap::new(),611 definitions: BTreeMap::new(),
567 };612 };
crates/onestore/src/page/write.rs deleted-1646
...@@ -1,1646 +0,0 @@
1//! Publishes an edited page model by lowering the difference from the stored page onto
2//! the typed writers, whose revisions accumulate in memory, then squashing them into one
3//! revision per space. `PreparedEdit::page` keeps it until the op path replaces it.
4
5use super::{Image, Ink, Outline, Page, PageObject, PageParagraph, ParagraphContent};
6use crate::{
7 Error, ExGuid, Insertion, ObjectData, OutlineEdit, ParagraphJoin, ParagraphSplit, PropertySets,
8 RevisionIndex, Store, TreeEdit, Value,
9 active::{ActivePage, Changes},
10 document::Kind,
11 op::{
12 content::{self, AttachmentIds},
13 levels,
14 lower::{View, collect_containers, kept_set, text_edit, validate},
15 properties,
16 table::{self, Structure},
17 },
18 write::{PropertyObject, RevisionEdit},
19};
20use bumpalo::Bump;
21use std::collections::{BTreeMap, BTreeSet};
22
23fn invalid(message: &'static str) -> Error {
24 Error { offset: 0, message }
25}
26
27fn page_images(page: &Page) -> impl Iterator<Item = (ExGuid, &Image)> {
28 page.objects.iter().filter_map(|object| match object {
29 PageObject::Image(image) => Some((image.id, image)),
30 _ => None,
31 })
32}
33
34fn page_ink(page: &Page) -> impl Iterator<Item = (ExGuid, &Ink)> {
35 page.objects.iter().filter_map(|object| match object {
36 PageObject::Ink(ink) => Some((ink.id, ink)),
37 _ => None,
38 })
39}
40
41pub(crate) fn write_page(
42 source: &[u8],
43 space: ExGuid,
44 after: &Page,
45 author: &str,
46) -> Result<Vec<u8>, Error> {
47 if author.contains('\0') {
48 return Err(invalid("Choose an author name without NUL"));
49 }
50 let store = Store::parse(source)?;
51 let index = RevisionIndex::parse(&store)?;
52 let arena = Bump::new();
53 let active = ActivePage::parse(&index, space)?;
54 let [page] = active.pages[..] else {
55 return Err(invalid("Choose an object space containing one active page"));
56 };
57 let before = Page::from_revision(&active.view, page)?;
58 let mut lowering = Lowering {
59 active,
60 arena: &arena,
61 current: Some(before.clone()),
62 edited: false,
63 page,
64 author,
65 alias: BTreeMap::new(),
66 built: BTreeSet::new(),
67 };
68 lowering.run(&before, after)?;
69 if !lowering.edited {
70 return Ok(source.to_vec());
71 }
72 let active = &lowering.active;
73 squash(
74 &index,
75 &[(space, &active.live.revision)],
76 &active.payloads,
77 &lowering.alias,
78 None,
79 )
80}
81
82struct Lowering<'a> {
83 active: ActivePage<'a>,
84 /// Holds the objects the typed writers store for as long as `active` reads them.
85 arena: &'a Bump,
86 /// The page as `active` stores it, until a writer changes it.
87 current: Option<Page>,
88 /// Whether a writer stored anything.
89 edited: bool,
90 page: ExGuid,
91 author: &'a str,
92 /// Model identities of new objects mapped to the identities the typed writers allocated.
93 alias: BTreeMap<ExGuid, ExGuid>,
94 /// Tables whose structure this run has written.
95 built: BTreeSet<ExGuid>,
96}
97
98impl<'a> Lowering<'a> {
99 fn id(&self, model: ExGuid) -> ExGuid {
100 self.alias.get(&model).copied().unwrap_or(model)
101 }
102
103 /// A writer identity for a new object; squash renames it to the model's.
104 fn allocate(&mut self, model: ExGuid) -> Result<ExGuid, Error> {
105 let written = ExGuid {
106 guid: crate::write::fresh_guid()?,
107 n: 1,
108 };
109 self.alias.insert(model, written);
110 Ok(written)
111 }
112
113 /// Stores the next revision a typed writer computes from the page as stored so far.
114 fn write(
115 &mut self,
116 changes: impl FnOnce(&ActivePage<'a>) -> Result<Changes, Error>,
117 ) -> Result<(), Error> {
118 self.write_with(&[], changes)
119 }
120
121 fn write_with(
122 &mut self,
123 payloads: &[([u8; 16], &[u8])],
124 changes: impl FnOnce(&ActivePage<'a>) -> Result<Changes, Error>,
125 ) -> Result<(), Error> {
126 let changes = changes(&self.active)?;
127 if self.active.write(self.arena, payloads, changes)? {
128 self.current = None;
129 self.edited = true;
130 }
131 Ok(())
132 }
133
134 fn current(&mut self) -> Result<Page, Error> {
135 if self.current.is_none() {
136 self.current = Some(Page::from_revision(&self.active.view, self.page)?);
137 }
138 Ok(self.current.clone().unwrap())
139 }
140
141 fn run(&mut self, before: &Page, after: &Page) -> Result<(), Error> {
142 let old = View::new(before, false)?;
143 let new = View::new(after, false)?;
144 validate(before, after, &old, &new)?;
145 if after.created != before.created {
146 return Err(invalid("Page creation time cannot be edited"));
147 }
148 for id in new.outlines.keys().chain(new.paragraphs.keys()) {
149 if !old.outlines.contains_key(id)
150 && !old.paragraphs.contains_key(id)
151 && self.active.live.revision.objects.contains_key(id)
152 {
153 return Err(invalid(
154 "A new model identity already exists in the section",
155 ));
156 }
157 }
158 let mut placed: BTreeMap<ExGuid, Vec<ExGuid>> = BTreeMap::new();
159 for (container, children) in &old.children {
160 placed.insert(*container, children.clone());
161 }
162 let mut page_order: Vec<ExGuid> = old
163 .page_children
164 .iter()
165 .copied()
166 .filter(|id| !old.outlines.contains_key(id) || new.outlines.contains_key(id))
167 .collect();
168 self.insert_outlines(&old, &new, &mut placed, &mut page_order)?;
169 let mut consumed = BTreeSet::new();
170 self.ungroup(&old, &new)?;
171 self.split_and_join(&old, &new, &mut placed, &mut consumed)?;
172 self.place(&old, &new, &placed, &page_order)?;
173 self.delete(&old, &new, &consumed)?;
174 self.edit_levels(&new)?;
175 self.edit_text_identities(&new)?;
176 self.edit_equations(&new)?;
177 self.edit_text(&new)?;
178 self.edit_paragraph_styles(&old, &new, after)?;
179 self.edit_lists(after, &new)?;
180 self.edit_tags(after, &new)?;
181 self.edit_paragraph_formatting(&new)?;
182 self.edit_formatting(&new)?;
183 self.edit_layout(&old, &new)?;
184 Ok(())
185 }
186
187 fn insert_outlines(
188 &mut self,
189 old: &View<'_>,
190 new: &View<'_>,
191 placed: &mut BTreeMap<ExGuid, Vec<ExGuid>>,
192 page_order: &mut Vec<ExGuid>,
193 ) -> Result<(), Error> {
194 for id in &new.page_children {
195 let Some(outline) = new.outlines.get(id) else {
196 continue;
197 };
198 if old.outlines.contains_key(id) {
199 continue;
200 }
201 if new.title_outlines.contains(id) {
202 return Err(invalid("Title outlines cannot be added"));
203 }
204 let (Some(x), Some(y)) = (outline.layout.x, outline.layout.y) else {
205 return Err(invalid("A new outline needs a position"));
206 };
207 let Some(first) = new.children[id].first().copied() else {
208 return Err(invalid("A new outline needs a paragraph"));
209 };
210 // A new outline starts as one text paragraph; other content replaces it in
211 // its own pass, as for a paragraph inserted into an existing outline.
212 let (text, text_id) = match &new.paragraphs[&first].content {
213 ParagraphContent::Text(text) if text.date_field.is_none() => {
214 (text.text.text(), Some(text.id))
215 }
216 ParagraphContent::Table(_)
217 | ParagraphContent::Image(_)
218 | ParagraphContent::Attachment(_)
219 | ParagraphContent::Ink(_) => ("", None),
220 _ => {
221 return Err(invalid(
222 "A new outline starts with a paragraph the writer builds",
223 ));
224 }
225 };
226 let insertion = Insertion::outline(self.page, x, y, text, self.author)?;
227 self.write(|active| insertion.changes(active))?;
228 let outline_id = insertion.object();
229 self.alias.insert(*id, outline_id);
230 self.alias.insert(
231 first,
232 ExGuid {
233 guid: outline_id.guid,
234 n: 3,
235 },
236 );
237 if let Some(text_id) = text_id {
238 self.alias.insert(text_id, insertion.text_object());
239 }
240 placed.insert(*id, vec![first]);
241 placed.insert(first, Vec::new());
242 page_order.push(*id);
243 }
244 Ok(())
245 }
246
247 /// Splits and joins stored paragraphs as OneNote's Enter, Backspace and Delete do. A new
248 /// styled paragraph that is no exact split starts as Enter at the end of the nearest stored
249 /// paragraph that allows one, taking its properties as OneNote's new paragraphs do; later
250 /// passes place it and give it its text.
251 fn split_and_join(
252 &mut self,
253 old: &View<'_>,
254 new: &View<'_>,
255 placed: &mut BTreeMap<ExGuid, Vec<ExGuid>>,
256 consumed: &mut BTreeSet<ExGuid>,
257 ) -> Result<(), Error> {
258 // A new paragraph may be split from the stored paragraph nearest before it, whatever
259 // placement or pasted paragraphs the same save adds between them.
260 let mut splits = Vec::new();
261 for document in new.documents() {
262 let mut left = None;
263 for paragraph in document {
264 if old.paragraphs.contains_key(&paragraph.id) {
265 left = paragraph.text().is_some().then_some(paragraph.id);
266 } else if let Some(left) = left {
267 let container = new.container[&paragraph.id];
268 let at = new.children[&container]
269 .iter()
270 .position(|id| *id == paragraph.id);
271 splits.push(((container, at), left, paragraph.id));
272 }
273 }
274 }
275 splits.sort();
276 let mut split = BTreeSet::new();
277 for (_, left, right) in splits {
278 let (Some(previous), Some(after), Some(next)) =
279 (old.text(left), new.text(left), new.text(right))
280 else {
281 continue;
282 };
283 if split.contains(&left)
284 || after.id != previous.id
285 || new.paragraphs[&right].style != old.paragraphs[&left].style
286 {
287 continue;
288 }
289 let Ok(offset) = after.text.utf16_offset(after.text.text().len()) else {
290 continue;
291 };
292 // Splitting at the end only differs from appending a paragraph by the copied
293 // paragraph style; without one, an appended empty paragraph is an insertion.
294 if offset == previous.text.utf16_offset(previous.text.text().len())?
295 && new.paragraphs[&right].style.is_none()
296 {
297 continue;
298 }
299 let Ok(head) = previous.text.slice(0..offset) else {
300 continue;
301 };
302 let Ok(tail) = previous.text.slice(
303 offset
304 ..previous
305 .text
306 .utf16_offset(previous.text.text().len())
307 .unwrap_or(0),
308 ) else {
309 continue;
310 };
311 // An emptied side takes the writer's insertion style, so only its text must agree.
312 let same = |expected: &super::Paragraph, actual: &super::Paragraph| {
313 expected.text() == actual.text()
314 && (expected.text().is_empty() || expected == actual)
315 };
316 if !same(&head, &after.text) || !same(&tail, &next.text) {
317 continue;
318 }
319 let edit = ParagraphSplit::new(previous.id, offset, self.author)?;
320 if self.split(&edit, left, right, old, new, placed)? {
321 split.insert(left);
322 }
323 }
324 for document in new.documents() {
325 for (at, paragraph) in document.iter().enumerate() {
326 if old.paragraphs.contains_key(&paragraph.id)
327 || self.alias.contains_key(&paragraph.id)
328 || paragraph.style.is_none()
329 || paragraph.text().is_none()
330 {
331 continue;
332 }
333 let stored = |candidate: &&&PageParagraph| {
334 old.paragraphs.contains_key(&candidate.id) && candidate.text().is_some()
335 };
336 let before = document[..at].iter().rev().filter(stored);
337 for template in before.chain(document[at + 1..].iter().filter(stored)) {
338 let text = self.active.view.nodes[&template.id].content[0];
339 let Kind::RichText { text: content, .. } = &self.active.view.nodes[&text].kind
340 else {
341 continue;
342 };
343 let end = u32::try_from(content.encode_utf16().count())
344 .map_err(|_| invalid("Paragraph exceeds the UTF-16 offset range"))?;
345 let edit = ParagraphSplit::new(text, end, self.author)?;
346 if self.split(&edit, template.id, paragraph.id, old, new, placed)? {
347 break;
348 }
349 }
350 }
351 }
352 for (container, children) in &old.children {
353 if !new.children.contains_key(container) {
354 continue;
355 }
356 for pair in children.windows(2) {
357 let [left, right] = [pair[0], pair[1]];
358 let (Some(previous), Some(after), Some(removed)) =
359 (old.text(left), new.text(left), old.text(right))
360 else {
361 continue;
362 };
363 if new.paragraphs.contains_key(&right)
364 || !old.children[&left].is_empty()
365 || !old.children[&right].is_empty()
366 || removed.text.text().is_empty()
367 {
368 continue;
369 }
370 let mut joined = previous.text.clone();
371 if joined.append(removed.text.clone()).is_err() || joined != after.text {
372 continue;
373 }
374 let adopts_right = previous.text.text().is_empty();
375 if after.id
376 != if adopts_right {
377 removed.id
378 } else {
379 previous.id
380 }
381 {
382 continue;
383 }
384 let join = ParagraphJoin::new(previous.id, removed.id, self.author)?;
385 let Ok(changes) = join.changes(&self.active) else {
386 continue;
387 };
388 self.write(|_| Ok(changes))?;
389 consumed.insert(right);
390 placed.get_mut(container).unwrap().retain(|id| *id != right);
391 placed.remove(&right);
392 }
393 }
394 Ok(())
395 }
396
397 /// Stores `edit` of model paragraph `left` as model paragraph `right`, unless the split
398 /// cannot be made there.
399 fn split(
400 &mut self,
401 edit: &ParagraphSplit,
402 left: ExGuid,
403 right: ExGuid,
404 old: &View<'_>,
405 new: &View<'_>,
406 placed: &mut BTreeMap<ExGuid, Vec<ExGuid>>,
407 ) -> Result<bool, Error> {
408 let Ok(changes) = edit.changes(&self.active) else {
409 return Ok(false);
410 };
411 self.write(|_| Ok(changes))?;
412 self.alias.insert(right, edit.object());
413 // A text object a join moved into the paragraph is taken over after placement.
414 if let Some(text) = new.text(right)
415 && !self.active.live.revision.objects.contains_key(&text.id)
416 {
417 self.alias.insert(text.id, edit.text_object());
418 }
419 // The split copies the left paragraph's list nodes in order; a list the model shares
420 // with a stored paragraph stays that paragraph's.
421 if new.paragraphs[&right].lists.len() == old.paragraphs[&left].lists.len() {
422 for (n, list) in (5..).zip(&new.paragraphs[&right].lists) {
423 if !self.active.live.revision.objects.contains_key(list) {
424 let guid = edit.object().guid;
425 self.alias.insert(*list, ExGuid { guid, n });
426 }
427 }
428 }
429 // An earlier split may have carried the left paragraph to its tail.
430 let list = placed
431 .values_mut()
432 .find(|list| list.contains(&left))
433 .unwrap();
434 let at = list.iter().position(|id| *id == left).unwrap();
435 list.insert(at + 1, right);
436 placed.insert(right, placed[&left].clone());
437 placed.insert(left, Vec::new());
438 Ok(true)
439 }
440
441 fn place(
442 &mut self,
443 old: &View<'_>,
444 new: &View<'_>,
445 placed: &BTreeMap<ExGuid, Vec<ExGuid>>,
446 page_order: &[ExGuid],
447 ) -> Result<(), Error> {
448 let title = |page: &Page, id: ExGuid| {
449 page.objects
450 .iter()
451 .any(|object| matches!(object, PageObject::Title(title) if title.id == id))
452 };
453 self.edit_page_images(old, new)?;
454 self.edit_page_ink(old, new)?;
455 let survivors: Vec<ExGuid> = page_order
456 .iter()
457 .copied()
458 .filter(|id| {
459 (new.outlines.contains_key(id) || !old.outlines.contains_key(id))
460 && !title(old.page, *id)
461 })
462 .collect();
463 let after: Vec<ExGuid> = new
464 .page_children
465 .iter()
466 .copied()
467 .filter(|id| !title(new.page, *id))
468 .collect();
469 let kept = kept_set(&survivors, &after, |id| {
470 new.outlines.contains_key(&id) && !new.title_outlines.contains(&id)
471 })?;
472 let mut next = None;
473 for id in after.iter().rev() {
474 if !kept.contains(id) {
475 let edit = TreeEdit::move_to(
476 self.id(*id),
477 self.page,
478 next.map(|n| self.id(n)),
479 self.author,
480 )?;
481 self.write(|active| edit.changes(active))?;
482 }
483 next = Some(*id);
484 }
485 let mut containers: Vec<ExGuid> = Vec::new();
486 for object in &new.page.objects {
487 let outlines: Vec<&Outline> = match object {
488 PageObject::Outline(outline) => vec![outline],
489 PageObject::Title(title) => title.outlines.iter().collect(),
490 PageObject::Image(_) | PageObject::Ink(_) | PageObject::Unsupported(_) => {
491 Vec::new()
492 }
493 };
494 for outline in outlines {
495 containers.push(outline.id);
496 collect_containers(&outline.paragraphs, &mut containers);
497 }
498 }
499 // Cells that do not exist yet, and containers inside them, receive their
500 // paragraphs once the table structure does.
501 let new_cell = |id: &ExGuid| {
502 !(old.children.contains_key(id)
503 || new.paragraphs.contains_key(id)
504 || new.outlines.contains_key(id))
505 };
506 let (existing, deferred): (Vec<ExGuid>, Vec<ExGuid>) =
507 containers.into_iter().partition(|container| {
508 let mut at = *container;
509 loop {
510 if new_cell(&at) {
511 return false;
512 }
513 match new.container.get(&at) {
514 Some(parent) => at = *parent,
515 None => return true,
516 }
517 }
518 });
519 self.place_containers(old, new, placed, &existing)?;
520 // Each round builds the tables whose holders exist and fills their cells, which
521 // may hold further new tables.
522 let mut pending = deferred;
523 loop {
524 self.edit_table_structure(old, new)?;
525 if pending.is_empty() {
526 break;
527 }
528 let (ready, waiting): (Vec<ExGuid>, Vec<ExGuid>) =
529 pending.into_iter().partition(|container| {
530 self.alias.contains_key(container) || old.children.contains_key(container)
531 });
532 if ready.is_empty() {
533 return Err(invalid("Table content has no table to hold it"));
534 }
535 self.place_containers(old, new, placed, &ready)?;
536 pending = waiting;
537 }
538 self.edit_cells(old, new)?;
539 self.edit_images(old, new)?;
540 self.edit_ink_paragraphs(old, new)?;
541 self.edit_attachments(old, new)?;
542 Ok(())
543 }
544
545 /// Gives each new attachment paragraph what OneNote stores for an inserted file: the
546 /// payload embedded in the file-data store, an embedded-file container declaring it,
547 /// and an attachment object naming the file that the paragraph holds as content.
548 fn edit_attachments(&mut self, old: &View<'_>, new: &View<'_>) -> Result<(), Error> {
549 for (paragraph_id, paragraph) in &new.paragraphs {
550 let ParagraphContent::Attachment(attachment) = &paragraph.content else {
551 continue;
552 };
553 if let Some(previous) = old.paragraphs.get(paragraph_id) {
554 let ParagraphContent::Attachment(stored) = &previous.content else {
555 return Err(invalid("Paragraph content type cannot change"));
556 };
557 if stored != attachment {
558 self.write(|active| content::attachment_edit_changes(active, stored, attachment))?;
559 }
560 continue;
561 }
562 if attachment.recording.is_some() {
563 return Err(invalid("Recordings are captured by OneNote, not inserted"));
564 }
565 let Some(bytes) = &attachment.bytes else {
566 return Err(invalid("A new attachment needs its payload"));
567 };
568 let ids = AttachmentIds {
569 object: self.allocate(attachment.id)?,
570 file: ExGuid {
571 guid: crate::write::fresh_guid()?,
572 n: 1,
573 },
574 payload: crate::write::fresh_guid()?,
575 preview: match &attachment.preview {
576 Some(_) => {
577 let payload = crate::write::fresh_guid()?;
578 let icon = ExGuid {
579 guid: crate::write::fresh_guid()?,
580 n: 1,
581 };
582 Some((payload, icon))
583 }
584 None => None,
585 },
586 };
587 let mut payloads: Vec<([u8; 16], &[u8])> = vec![(ids.payload, bytes)];
588 if let (Some((payload, _)), Some(icon)) = (ids.preview, &attachment.preview) {
589 payloads.push((payload, icon));
590 }
591 let holder = self.id(*paragraph_id);
592 self.write_with(&payloads, |active| {
593 content::attachment_changes(active, attachment, &ids, holder)
594 })?;
595 }
596 Ok(())
597 }
598
599 /// Gives each new picture paragraph what OneNote stores for an inserted picture: the
600 /// payload embedded in the section's file-data store, a file-data object declaring it
601 /// by identity and extension, and a picture object the paragraph holds as content.
602 fn edit_images(&mut self, old: &View<'_>, new: &View<'_>) -> Result<(), Error> {
603 for (paragraph_id, paragraph) in &new.paragraphs {
604 let ParagraphContent::Image(image) = &paragraph.content else {
605 continue;
606 };
607 if let Some(previous) = old.paragraphs.get(paragraph_id) {
608 let ParagraphContent::Image(stored) = &previous.content else {
609 return Err(invalid("Paragraph content type cannot change"));
610 };
611 if stored != image {
612 self.edit_image(stored, image)?;
613 }
614 continue;
615 }
616 if image.layout.x.is_some() || image.layout.y.is_some() {
617 return Err(invalid("A paragraph picture has no position of its own"));
618 }
619 let holder = self.id(*paragraph_id);
620 self.insert_image(image, Some(holder))?;
621 }
622 Ok(())
623 }
624
625 /// Page-level pictures are direct page children, as OneNote stores a picture placed
626 /// outside any outline: new ones are appended for the placement pass to order, changed
627 /// ones are moved, resized or described, and removed ones are deleted in the delete pass.
628 fn edit_page_images(&mut self, old: &View<'_>, new: &View<'_>) -> Result<(), Error> {
629 let stored: BTreeMap<ExGuid, &Image> = page_images(old.page).collect();
630 for (id, image) in page_images(new.page) {
631 match stored.get(&id) {
632 Some(previous) => {
633 if *previous != image {
634 self.edit_image(previous, image)?;
635 }
636 }
637 None => {
638 if image.layout.x.is_none() || image.layout.y.is_none() {
639 return Err(invalid("A new page-level picture needs a position"));
640 }
641 self.insert_image(image, None)?;
642 }
643 }
644 }
645 Ok(())
646 }
647
648 /// Ink placed on the page: new drawings are appended for the placement pass to order,
649 /// changed ones have their stroke list rewritten, removed ones go in the delete pass.
650 fn edit_page_ink(&mut self, old: &View<'_>, new: &View<'_>) -> Result<(), Error> {
651 let stored: BTreeMap<ExGuid, &Ink> = page_ink(old.page).collect();
652 for (id, ink) in page_ink(new.page) {
653 match stored.get(&id) {
654 Some(previous) => {
655 if *previous != ink {
656 self.edit_ink(previous, ink)?;
657 }
658 }
659 None => self.insert_ink(ink, None)?,
660 }
661 }
662 Ok(())
663 }
664
665 fn edit_ink_paragraphs(&mut self, old: &View<'_>, new: &View<'_>) -> Result<(), Error> {
666 for (paragraph_id, paragraph) in &new.paragraphs {
667 let ParagraphContent::Ink(ink) = &paragraph.content else {
668 continue;
669 };
670 match old.paragraphs.get(paragraph_id) {
671 Some(previous) => {
672 let ParagraphContent::Ink(stored) = &previous.content else {
673 return Err(invalid("Paragraph content type cannot change"));
674 };
675 if stored != ink {
676 self.edit_ink(stored, ink)?;
677 }
678 }
679 None => {
680 let holder = self.id(*paragraph_id);
681 self.insert_ink(ink, Some(holder))?;
682 }
683 }
684 }
685 Ok(())
686 }
687
688 /// Writes new ink the way OneNote 2010 stores a drawing: a container the page lists as a
689 /// child (or a paragraph holds as content), its data node listing stroke objects, each
690 /// stroke's packet and half-inch origin, and one drawing-attribute object per distinct pen.
691 fn insert_ink(&mut self, ink: &Ink, holder: Option<ExGuid>) -> Result<(), Error> {
692 let container = self.allocate(ink.id)?;
693 let data = ExGuid {
694 guid: crate::write::fresh_guid()?,
695 n: 1,
696 };
697 let mut strokes = Vec::new();
698 for stroke in &ink.strokes {
699 strokes.push((self.allocate(stroke.id)?, stroke));
700 }
701 self.write(|active| content::ink_changes(active, ink, container, data, &strokes, holder))
702 }
703
704 /// Rewrites a drawing's stroke list: stored strokes stay as they are (OneNote erases whole
705 /// strokes rather than editing them), removed ones leave the list, new ones are created.
706 fn edit_ink(&mut self, stored: &Ink, ink: &Ink) -> Result<(), Error> {
707 if stored.id != ink.id {
708 return Err(invalid("Ink identity cannot change"));
709 }
710 if stored.layout != ink.layout || stored.groups != ink.groups {
711 return Err(invalid("Ink position and groups stay as stored"));
712 }
713 let mut kept = Vec::new();
714 let mut added = Vec::new();
715 for stroke in &ink.strokes {
716 match stored.strokes.iter().find(|s| s.id == stroke.id) {
717 Some(previous) if previous == stroke => kept.push(self.id(stroke.id)),
718 Some(_) => return Err(invalid("A stored stroke keeps its path and pen")),
719 None => added.push((self.allocate(stroke.id)?, stroke)),
720 }
721 }
722 let container = self.id(ink.id);
723 self.write(|active| content::strokes_changes(active, container, &kept, &added))
724 }
725
726 /// Gives a new picture what OneNote stores for an inserted one: the payload embedded
727 /// in the section's file-data store, a file-data object declaring it by identity and
728 /// extension, and a picture object that a paragraph holds as content or the page
729 /// lists as a child.
730 fn insert_image(&mut self, image: &Image, holder: Option<ExGuid>) -> Result<(), Error> {
731 let Some(bytes) = &image.bytes else {
732 return Err(invalid("A new picture needs its payload"));
733 };
734 let image_id = self.allocate(image.id)?;
735 let file_id = ExGuid {
736 guid: crate::write::fresh_guid()?,
737 n: 1,
738 };
739 let payload_guid = crate::write::fresh_guid()?;
740 let payload: &[u8] = bytes;
741 self.write_with(&[(payload_guid, payload)], |active| {
742 content::picture_changes(active, image, image_id, file_id, payload_guid, holder)
743 })
744 }
745
746 fn edit_image(&mut self, stored: &Image, image: &Image) -> Result<(), Error> {
747 content::picture_fixed_fields(stored, image)?;
748 let object = self.id(image.id);
749 self.write(|active| {
750 content::picture_edit_changes(
751 active,
752 object,
753 (&stored.layout, &stored.alt),
754 &image.layout,
755 &image.alt,
756 )
757 })
758 }
759
760 fn place_containers(
761 &mut self,
762 old: &View<'_>,
763 new: &View<'_>,
764 placed: &BTreeMap<ExGuid, Vec<ExGuid>>,
765 containers: &[ExGuid],
766 ) -> Result<(), Error> {
767 for container in containers {
768 let after = &new.children[container];
769 let current: Vec<ExGuid> = placed
770 .get(container)
771 .map(|list| {
772 list.iter()
773 .copied()
774 .filter(|id| new.paragraphs.contains_key(id))
775 .collect()
776 })
777 .unwrap_or_default();
778 let kept = kept_set(&current, after, |_| true)?;
779 let mut next = None;
780 for id in after.iter().rev() {
781 let anchor = next.map(|n| self.id(n));
782 if !old.paragraphs.contains_key(id) && !self.alias.contains_key(id) {
783 let paragraph = new.paragraphs[id];
784 // A styled paragraph appended to a stored outline is a split whose
785 // precondition failed under a merge; it stays for review. A new outline
786 // takes styled paragraphs as built (a copied page).
787 let mut root = *container;
788 while let Some(parent) = new.container.get(&root) {
789 root = *parent;
790 }
791 if paragraph.style.is_some() && old.outlines.contains_key(&root) {
792 return Err(invalid(
793 "New paragraphs in a stored outline contain plain text without styles",
794 ));
795 }
796 if paragraph.media != Default::default() {
797 return Err(invalid(
798 "Recording annotations are made by OneNote while it records",
799 ));
800 }
801 // A new table starts as an empty text paragraph whose content the
802 // structure pass replaces with the table.
803 let (text, text_id) = match &paragraph.content {
804 ParagraphContent::Text(text) if text.date_field.is_none() => {
805 (text.text.text(), Some(text.id))
806 }
807 ParagraphContent::Table(_)
808 | ParagraphContent::Image(_)
809 | ParagraphContent::Attachment(_)
810 | ParagraphContent::Ink(_) => ("", None),
811 _ => {
812 return Err(invalid(
813 "New paragraphs contain plain text without fields or styles",
814 ));
815 }
816 };
817 let insertion =
818 Insertion::paragraph(self.id(*container), anchor, text, self.author)?;
819 self.write(|active| insertion.changes(active))?;
820 self.alias.insert(*id, insertion.object());
821 if let Some(text_id) =
822 text_id.filter(|id| !self.active.live.revision.objects.contains_key(id))
823 {
824 self.alias.insert(text_id, insertion.text_object());
825 }
826 } else if !kept.contains(id) {
827 let edit =
828 TreeEdit::move_to(self.id(*id), self.id(*container), anchor, self.author)?;
829 self.write(|active| edit.changes(active))?;
830 }
831 next = Some(*id);
832 }
833 }
834 Ok(())
835 }
836
837 /// Changes the shading and indents of stored cells in place.
838 fn edit_cells(&mut self, old: &View<'_>, new: &View<'_>) -> Result<(), Error> {
839 let stored: BTreeMap<ExGuid, &super::TableCell> = old
840 .paragraphs
841 .values()
842 .filter_map(|paragraph| match &paragraph.content {
843 ParagraphContent::Table(table) => Some(table),
844 _ => None,
845 })
846 .flat_map(|table| table.rows.iter().flat_map(|row| &row.cells))
847 .map(|cell| (cell.id, cell))
848 .collect();
849 for paragraph in new.paragraphs.values() {
850 let ParagraphContent::Table(table) = &paragraph.content else {
851 continue;
852 };
853 for cell in table.rows.iter().flat_map(|row| &row.cells) {
854 let Some(previous) = stored.get(&cell.id) else {
855 continue;
856 };
857 if (cell.shading, &cell.indents) == (previous.shading, &previous.indents) {
858 continue;
859 }
860 let object = self.id(cell.id);
861 self.write(|active| {
862 table::cell_changes(
863 active,
864 object,
865 (previous.shading, &previous.indents),
866 cell.shading,
867 &cell.indents,
868 )
869 })?;
870 }
871 }
872 Ok(())
873 }
874
875 /// Creates tables, rows and cells the model added, rebuilds every table's row and cell
876 /// order to the model's, and writes the column widths, locks and border flag.
877 fn edit_table_structure(&mut self, old: &View<'_>, new: &View<'_>) -> Result<(), Error> {
878 let old_tables: BTreeMap<ExGuid, &super::Table> = old
879 .paragraphs
880 .values()
881 .filter_map(|p| match &p.content {
882 ParagraphContent::Table(table) => Some((table.id, table)),
883 _ => None,
884 })
885 .collect();
886 for (paragraph_id, paragraph) in &new.paragraphs {
887 let ParagraphContent::Table(table) = &paragraph.content else {
888 continue;
889 };
890 let previous = old_tables.get(&table.id).copied();
891 // A table is built once its holder is placed, and only once.
892 if !(old.paragraphs.contains_key(paragraph_id) || self.alias.contains_key(paragraph_id))
893 || self.built.contains(&table.id)
894 {
895 continue;
896 }
897 self.built.insert(table.id);
898 let unchanged = previous.is_some_and(|previous| {
899 previous.columns == table.columns
900 && previous.borders == table.borders
901 && previous.rows.len() == table.rows.len()
902 && previous.rows.iter().zip(&table.rows).all(|(a, b)| {
903 a.id == b.id
904 && a.cells.len() == b.cells.len()
905 && a.cells.iter().zip(&b.cells).all(|(x, y)| x.id == y.id)
906 })
907 });
908 if unchanged {
909 continue;
910 }
911 let table_id = if previous.is_some() {
912 self.id(table.id)
913 } else {
914 self.allocate(table.id)?
915 };
916 let existing_rows: BTreeSet<ExGuid> = previous
917 .map(|p| p.rows.iter().map(|r| r.id).collect())
918 .unwrap_or_default();
919 let existing_cells: BTreeSet<ExGuid> = previous
920 .map(|p| {
921 p.rows
922 .iter()
923 .flat_map(|r| r.cells.iter().map(|c| c.id))
924 .collect()
925 })
926 .unwrap_or_default();
927 let template = previous
928 .and_then(|p| p.rows.first())
929 .and_then(|r| r.cells.first())
930 .map(|cell| cell.indents.as_slice());
931 let mut structure = Structure {
932 rows: Vec::new(),
933 new_rows: BTreeSet::new(),
934 new_cells: BTreeMap::new(),
935 columns: &table.columns,
936 borders: table.borders,
937 };
938 for row in &table.rows {
939 let row_id = if existing_rows.contains(&row.id) {
940 self.id(row.id)
941 } else {
942 let written = self.allocate(row.id)?;
943 structure.new_rows.insert(written);
944 written
945 };
946 let mut cells = Vec::new();
947 for cell in &row.cells {
948 let cell_id = if existing_cells.contains(&cell.id) {
949 self.id(cell.id)
950 } else {
951 let written = self.allocate(cell.id)?;
952 structure.new_cells.insert(
953 written,
954 (table::cell_indents(&cell.indents, template), cell.shading),
955 );
956 written
957 };
958 cells.push(cell_id);
959 }
960 structure.rows.push((row_id, cells));
961 }
962 let holder = previous.is_none().then(|| self.id(*paragraph_id));
963 self.write(|active| table::table_changes(active, table_id, holder, &structure))?;
964 }
965 Ok(())
966 }
967
968 fn delete(
969 &mut self,
970 old: &View<'_>,
971 new: &View<'_>,
972 consumed: &BTreeSet<ExGuid>,
973 ) -> Result<(), Error> {
974 let removed_outline =
975 |id: ExGuid| old.outlines.contains_key(&id) && !new.outlines.contains_key(&id);
976 for id in old.paragraphs.keys() {
977 if new.paragraphs.contains_key(id) || consumed.contains(id) {
978 continue;
979 }
980 let mut ancestor = old.container[id];
981 let mut covered = false;
982 // A surviving ancestor is placed where the model says, carrying this paragraph along.
983 while !new.paragraphs.contains_key(&ancestor) {
984 if removed_outline(ancestor)
985 || (old.paragraphs.contains_key(&ancestor) && !consumed.contains(&ancestor))
986 || (old.children.contains_key(&ancestor)
987 && !old.paragraphs.contains_key(&ancestor)
988 && !old.outlines.contains_key(&ancestor)
989 && !new.children.contains_key(&ancestor))
990 {
991 covered = true;
992 break;
993 }
994 match old.container.get(&ancestor) {
995 Some(parent) => ancestor = *parent,
996 None => break,
997 }
998 }
999 if covered {
1000 continue;
1001 }
1002 let edit = TreeEdit::delete(*id, self.author)?;
1003 self.write(|active| edit.changes(active))?;
1004 }
1005 let kept: BTreeSet<ExGuid> = page_images(new.page)
1006 .map(|(id, _)| id)
1007 .chain(page_ink(new.page).map(|(id, _)| id))
1008 .collect();
1009 for id in page_images(old.page)
1010 .map(|(id, _)| id)
1011 .chain(page_ink(old.page).map(|(id, _)| id))
1012 {
1013 if !kept.contains(&id) {
1014 let edit = TreeEdit::delete(id, self.author)?;
1015 self.write(|active| edit.changes(active))?;
1016 }
1017 }
1018 for id in old.outlines.keys() {
1019 if removed_outline(*id) {
1020 if old.title_outlines.contains(id) {
1021 return Err(invalid("Title outlines cannot be removed"));
1022 }
1023 let edit = TreeEdit::delete(*id, self.author)?;
1024 self.write(|active| edit.changes(active))?;
1025 }
1026 }
1027 Ok(())
1028 }
1029
1030 /// Rewrites every equation paragraph whose stored text object differs from the model.
1031 fn edit_equations(&mut self, new: &View<'_>) -> Result<(), Error> {
1032 let current = self.current()?;
1033 let current = View::new(&current, false)?;
1034 for (id, paragraph) in &new.paragraphs {
1035 let Some(text) = paragraph.text() else {
1036 continue;
1037 };
1038 let stored = current.text(self.id(*id));
1039 // Deleting an equation leaves no math in the model, but its stored runs are still math.
1040 if !super::Math::is_equation(&text.text)
1041 && !stored.is_some_and(|stored| super::Math::is_equation(&stored.text))
1042 {
1043 continue;
1044 }
1045 let stored = stored
1046 .ok_or_else(|| invalid("An equation paragraph is missing after placement"))?;
1047 if stored.text == text.text {
1048 continue;
1049 }
1050 let object = stored.id;
1051 self.write(|active| content::equation_changes(active, object, &text.text))?;
1052 }
1053 Ok(())
1054 }
1055
1056 /// Lifts the paragraphs of outline groups into their container wherever this save changes
1057 /// the container's children or their depths, for placement to see them; `edit_levels`
1058 /// groups them again.
1059 fn ungroup(&mut self, old: &View<'_>, new: &View<'_>) -> Result<(), Error> {
1060 for (container, children) in &old.children {
1061 let Some(after) = new.children.get(container) else {
1062 continue;
1063 };
1064 let grouped = self.active.view.nodes[container]
1065 .children
1066 .iter()
1067 .any(|id| matches!(self.active.view.nodes[id].kind, Kind::OutlineGroup));
1068 if !grouped
1069 || (after == children
1070 && old.depths(*container, children) == new.depths(*container, after))
1071 {
1072 continue;
1073 }
1074 let object = *container;
1075 self.write(|active| levels::ungroup_changes(active, object))?;
1076 }
1077 Ok(())
1078 }
1079
1080 /// Indents each container's children to the depths the model gives them.
1081 fn edit_levels(&mut self, new: &View<'_>) -> Result<(), Error> {
1082 let current = self.current()?;
1083 let stored = View::new(&current, false)?;
1084 for (container, children) in &new.children {
1085 let depths = new
1086 .depths(*container, children)
1087 .ok_or_else(|| invalid("A paragraph lies no deeper than its parent"))?;
1088 let object = self.id(*container);
1089 let written: Vec<ExGuid> = children.iter().map(|id| self.id(*id)).collect();
1090 if children.is_empty() || stored.depths(object, &written) == Some(depths.clone()) {
1091 continue;
1092 }
1093 let is_cell =
1094 !new.paragraphs.contains_key(container) && !new.outlines.contains_key(container);
1095 let (level, runs) = levels::runs(&written, &depths, is_cell)?;
1096 self.write(|active| levels::regroup_changes(active, object, level, &runs))?;
1097 }
1098 Ok(())
1099 }
1100
1101 /// Gives a paragraph the text object its model names when a split and a join in one save
1102 /// moved text between paragraphs: a stored object is taken over, as OneNote moves a lower
1103 /// paragraph's text into an emptied upper one; a new one starts as a copy of the text
1104 /// the paragraph holds.
1105 fn edit_text_identities(&mut self, new: &View<'_>) -> Result<(), Error> {
1106 for (id, paragraph) in &new.paragraphs {
1107 let Some(text) = paragraph.text() else {
1108 continue;
1109 };
1110 let object = self.id(*id);
1111 let stored = self.active.view.nodes[&object].content[0];
1112 if stored == self.id(text.id) {
1113 continue;
1114 }
1115 let target = if self.active.live.revision.objects.contains_key(&text.id) {
1116 text.id
1117 } else {
1118 self.allocate(text.id)?
1119 };
1120 let unstyled = paragraph.style.is_none();
1121 self.write(|active| {
1122 let parents = active.editable_parents(object)?;
1123 let modified = crate::create::current_timestamps()?.0.to_le_bytes();
1124 let raw = &active.live.revision;
1125 let mut changed = BTreeMap::new();
1126 if !raw.objects.contains_key(&target) {
1127 let mut copy = PropertyObject::from_object(&raw.objects[&stored])?;
1128 copy.remove(&[0x1c001c98, 0x14001c99])?;
1129 if unstyled {
1130 copy.remove(&[0x2000342c])?;
1131 }
1132 copy.reference(target)?;
1133 changed.insert(target, copy);
1134 }
1135 let mut holder = PropertyObject::from_object(&raw.objects[&object])?;
1136 let reference = holder.reference(target)?;
1137 holder.set(&[(0x24001c1f, &reference), (0x14001d7a, &modified)])?;
1138 changed.insert(object, holder);
1139 crate::formatting::touch_ancestors(raw, parents, object, &modified, &mut changed)?;
1140 Ok(changed)
1141 })?;
1142 }
1143 Ok(())
1144 }
1145
1146 fn edit_text(&mut self, new: &View<'_>) -> Result<(), Error> {
1147 let current = self.current()?;
1148 let current = View::new(&current, false)?;
1149 for (id, paragraph) in &new.paragraphs {
1150 let Some(text) = paragraph.text() else {
1151 continue;
1152 };
1153 let image_id = self.id(*id);
1154 let Some(stored) = current.text(image_id) else {
1155 return Err(invalid("A paragraph is missing after structural edits"));
1156 };
1157 if stored.id != self.id(text.id) {
1158 return Err(invalid("Text object identities cannot change"));
1159 }
1160 if stored.text.text() == text.text.text() {
1161 continue;
1162 }
1163 let (range, replacement) = text_edit(stored.text.text(), text.text.text())?;
1164 let object = stored.id;
1165 self.write(|active| crate::edit::text_changes(active, object, range, &replacement))?;
1166 }
1167 Ok(())
1168 }
1169
1170 /// Gives each paragraph the list nodes its model references: a definition new to the
1171 /// section becomes a list node carrying the model's identity, a changed one is rewritten
1172 /// in place, a definition another paragraph already owns is copied because native list
1173 /// nodes belong to one paragraph, and a dropped reference leaves the node unreferenced.
1174 fn edit_lists(&mut self, after: &Page, new: &View<'_>) -> Result<(), Error> {
1175 let current = self.current()?;
1176 let stored = View::new(&current, false)?;
1177 let mut owners: BTreeMap<ExGuid, ExGuid> = BTreeMap::new();
1178 for (id, paragraph) in &stored.paragraphs {
1179 for list in &paragraph.lists {
1180 owners.insert(*list, *id);
1181 }
1182 }
1183 for (id, paragraph) in &new.paragraphs {
1184 let image_id = self.id(*id);
1185 let previous = stored
1186 .paragraphs
1187 .get(&image_id)
1188 .ok_or_else(|| invalid("A paragraph is missing after text edits"))?;
1189 let same = paragraph.lists.len() == previous.lists.len()
1190 && paragraph
1191 .lists
1192 .iter()
1193 .zip(&previous.lists)
1194 .all(|(model, node)| {
1195 self.id(*model) == *node
1196 && after.definitions.get(model) == current.definitions.get(node)
1197 });
1198 if same {
1199 continue;
1200 }
1201 let mut nodes = Vec::new();
1202 for list in &paragraph.lists {
1203 // Writer-allocated identities keep a nonzero sequence number so the node's
1204 // compact identity is never the null identity; squash renames them to the
1205 // model's. A copy for a second owner keeps its allocated identity.
1206 let node_id = match owners.get(&self.id(*list)) {
1207 Some(owner) if *owner == image_id => self.id(*list),
1208 Some(_) => ExGuid {
1209 guid: crate::write::fresh_guid()?,
1210 n: 1,
1211 },
1212 // A node its paragraph let go of in this save serves the model again.
1213 None if self.active.live.revision.objects.contains_key(list) => *list,
1214 None => {
1215 let written = ExGuid {
1216 guid: crate::write::fresh_guid()?,
1217 n: 1,
1218 };
1219 self.alias.insert(*list, written);
1220 written
1221 }
1222 };
1223 let definition = after
1224 .definitions
1225 .get(list)
1226 .ok_or_else(|| invalid("A paragraph references a missing list definition"))?;
1227 nodes.push((node_id, properties::list_values(definition)?));
1228 }
1229 let object = self.id(*id);
1230 self.write(|active| properties::list_changes(active, object, &nodes))?;
1231 }
1232 Ok(())
1233 }
1234
1235 /// Rewrites the note tags of paragraphs and text objects whose model tags differ from
1236 /// the stored ones. A tag definition new to the section becomes a definition object
1237 /// carrying the model's identity after squash.
1238 fn edit_tags(&mut self, after: &Page, new: &View<'_>) -> Result<(), Error> {
1239 let current = self.current()?;
1240 let stored = View::new(&current, false)?;
1241 for (id, paragraph) in &new.paragraphs {
1242 let image_id = self.id(*id);
1243 let previous = stored
1244 .paragraphs
1245 .get(&image_id)
1246 .ok_or_else(|| invalid("A paragraph is missing after text edits"))?;
1247 let mut targets = vec![(image_id, &paragraph.tags, &previous.tags)];
1248 if let (Some(text), Some(before)) = (paragraph.text(), previous.text()) {
1249 targets.push((self.id(text.id), &text.tags, &before.tags));
1250 }
1251 for (object, tags, stored_tags) in targets {
1252 if crate::op::lower::same_tags(tags, stored_tags) {
1253 continue;
1254 }
1255 let mut entries = Vec::new();
1256 for tag in tags {
1257 let definition = tag
1258 .definition
1259 .ok_or_else(|| invalid("A note tag names its definition"))?;
1260 let known = self.active.live.revision.objects.contains_key(&definition)
1261 || self.alias.contains_key(&definition);
1262 let written = if known {
1263 self.id(definition)
1264 } else {
1265 let allocated = ExGuid {
1266 guid: crate::write::fresh_guid()?,
1267 n: 1,
1268 };
1269 self.alias.insert(definition, allocated);
1270 allocated
1271 };
1272 entries.push((written, tag, after.definitions.get(&definition)));
1273 }
1274 self.write(|active| properties::tag_changes(active, object, &entries))?;
1275 }
1276 }
1277 Ok(())
1278 }
1279
1280 /// A new paragraph, or one whose style changed, references its paragraph style from
1281 /// its text object; a style the page has not stored yet is created from its
1282 /// definition, as OneNote keeps quick styles.
1283 fn edit_paragraph_styles(
1284 &mut self,
1285 old: &View<'_>,
1286 new: &View<'_>,
1287 after: &Page,
1288 ) -> Result<(), Error> {
1289 for (id, paragraph) in &new.paragraphs {
1290 let Some(definition) = paragraph.style else {
1291 continue;
1292 };
1293 if old
1294 .paragraphs
1295 .get(id)
1296 .is_some_and(|previous| previous.style == paragraph.style)
1297 {
1298 continue;
1299 }
1300 let Some(text) = paragraph.text() else {
1301 return Err(invalid("Only text paragraphs take a paragraph style"));
1302 };
1303 let known = self.active.live.revision.objects.contains_key(&definition)
1304 || self.alias.contains_key(&definition);
1305 let style_id = if known {
1306 self.id(definition)
1307 } else {
1308 let allocated = ExGuid {
1309 guid: crate::write::fresh_guid()?,
1310 n: 1,
1311 };
1312 self.alias.insert(definition, allocated);
1313 allocated
1314 };
1315 let object = self.id(text.id);
1316 let definition = after.definitions.get(&definition);
1317 self.write(|active| properties::style_changes(active, object, style_id, definition))?;
1318 }
1319 Ok(())
1320 }
1321
1322 fn edit_paragraph_formatting(&mut self, new: &View<'_>) -> Result<(), Error> {
1323 let current = self.current()?;
1324 let current = View::new(&current, false)?;
1325 for (id, paragraph) in &new.paragraphs {
1326 let Some(text) = paragraph.text() else {
1327 continue;
1328 };
1329 let stored = current
1330 .text(self.id(*id))
1331 .ok_or_else(|| invalid("A paragraph is missing after text edits"))?;
1332 let values = crate::op::lower::paragraph_change(&stored.text, &text.text)?;
1333 if values.is_empty() {
1334 continue;
1335 }
1336 if text.date_field.is_some() {
1337 return Err(invalid(
1338 "Generated title fields cannot be formatted as ordinary text",
1339 ));
1340 }
1341 let object = stored.id;
1342 self.write(|active| properties::paragraph_format_changes(active, object, &values))?;
1343 }
1344 Ok(())
1345 }
1346
1347 fn edit_formatting(&mut self, new: &View<'_>) -> Result<(), Error> {
1348 let current = self.current()?;
1349 let current = View::new(&current, false)?;
1350 for (id, paragraph) in &new.paragraphs {
1351 let Some(text) = paragraph.text() else {
1352 continue;
1353 };
1354 let stored = current
1355 .text(self.id(*id))
1356 .ok_or_else(|| invalid("A paragraph is missing after text edits"))?;
1357 if stored.text.text() != text.text.text() {
1358 return Err(invalid("Text edits did not converge on the model"));
1359 }
1360 let fresh = self.alias.contains_key(&text.id);
1361 for (range, set, clear) in crate::op::lower::format_edits(&stored.text, &text.text, fresh)? {
1362 let object = stored.id;
1363 self.write(|active| {
1364 crate::formatting::format_changes(active, object, range, &set, &clear)
1365 })?;
1366 }
1367 }
1368 Ok(())
1369 }
1370
1371 fn edit_layout(&mut self, old: &View<'_>, new: &View<'_>) -> Result<(), Error> {
1372 let current = self.current()?;
1373 let current = View::new(&current, false)?;
1374 for (id, paragraph) in &new.paragraphs {
1375 let stored = current
1376 .paragraphs
1377 .get(&self.id(*id))
1378 .ok_or_else(|| invalid("A paragraph is missing after text edits"))?;
1379 if stored.collapsed != paragraph.collapsed {
1380 let object = self.id(*id);
1381 let edit = OutlineEdit::Collapsed(paragraph.collapsed);
1382 self.write(|active| edit.changes(active, object))?;
1383 }
1384 }
1385 for (id, outline) in &new.outlines {
1386 if new.title_outlines.contains(id) {
1387 continue;
1388 }
1389 let stored = current
1390 .outlines
1391 .get(&self.id(*id))
1392 .ok_or_else(|| invalid("An outline is missing after text edits"))?;
1393 let object = self.id(*id);
1394 // A new outline takes the model's indentation table (a copied outline keeps
1395 // its levels' offsets); a stored table stays as it is.
1396 if !old.outlines.contains_key(id)
1397 && !outline.indents.is_empty()
1398 && outline.indents != stored.indents
1399 {
1400 let indents = content::measurement_bytes(&outline.indents, 4)?;
1401 self.write(|active| {
1402 let raw = &active.live.revision;
1403 let mut node = PropertyObject::from_object(&raw.objects[&object])?;
1404 node.set(&[(0x1c001c12, &indents)])?;
1405 Ok(BTreeMap::from([(object, node)]))
1406 })?;
1407 }
1408 if (outline.layout.x, outline.layout.y) != (stored.layout.x, stored.layout.y) {
1409 let (Some(x), Some(y)) = (outline.layout.x, outline.layout.y) else {
1410 return Err(invalid("An outline position needs both coordinates"));
1411 };
1412 let edit = OutlineEdit::Position { x, y };
1413 self.write(|active| edit.changes(active, object))?;
1414 }
1415 if (outline.layout.max_width, outline.layout.width_set_by_user)
1416 != (stored.layout.max_width, stored.layout.width_set_by_user)
1417 && (old.outlines.contains_key(id) || outline.layout.max_width.is_some())
1418 {
1419 let Some(points) = outline.layout.max_width else {
1420 return Err(invalid("An outline width cannot be removed"));
1421 };
1422 let edit = OutlineEdit::Width {
1423 points,
1424 user_set: outline.layout.width_set_by_user == Some(true),
1425 };
1426 self.write(|active| edit.changes(active, object))?;
1427 }
1428 }
1429 Ok(())
1430 }
1431}
1432
1433/// Payload identities the file-data store of `store` declares, in order.
1434pub(crate) fn declared_payloads(store: &Store<'_>) -> Vec<[u8; 16]> {
1435 store
1436 .lists
1437 .values()
1438 .flat_map(|list| &list.nodes)
1439 .filter(|node| node.id == 0x94)
1440 .filter_map(|node| node.payload.get(..16).and_then(|g| g.try_into().ok()))
1441 .collect()
1442}
1443
1444/// Writes the spaces the typed writers `edited` as one transaction on the validated
1445/// `source`, renaming writer-allocated identities to the model's and embedding the
1446/// `payloads` it lacks. A protected `source` takes the revisions its plaintext twin gained.
1447pub(crate) fn squash(
1448 source: &RevisionIndex<'_>,
1449 edited: &[(ExGuid, &crate::ResolvedRevision<'_>)],
1450 payloads: &[([u8; 16], &[u8])],
1451 alias: &BTreeMap<ExGuid, ExGuid>,
1452 protection: Option<&dyn crate::write::Protection>,
1453) -> Result<Vec<u8>, Error> {
1454 let rename: BTreeMap<ExGuid, ExGuid> = alias
1455 .iter()
1456 .map(|(model, image)| (*image, *model))
1457 .collect();
1458 let existing = declared_payloads(source.store);
1459 let payloads: Vec<_> = payloads
1460 .iter()
1461 .filter(|(guid, _)| !existing.contains(guid))
1462 .copied()
1463 .collect();
1464 let edit = |index: &RevisionIndex<'_>| {
1465 let mut changes = BTreeMap::new();
1466 for (sid, after) in edited {
1467 let before = match protection {
1468 Some(protection) => protection.resolve(*sid, index.active(*sid)?)?,
1469 None => index.resolve_active(*sid)?,
1470 };
1471 if before.roots != after.roots {
1472 return Err(invalid("Page edits cannot change revision roots"));
1473 }
1474 // Retired styles and deleted content stay in history; only live objects are written.
1475 let live = after.reachable()?;
1476 let mut changed = BTreeMap::new();
1477 for (id, object) in &after.objects {
1478 if !live.contains(id) {
1479 continue;
1480 }
1481 // Stored tables keep the entries an object names, so those decide.
1482 if let Some(previous) = before.objects.get(id)
1483 && previous.jcid == object.jcid
1484 && previous.data == object.data
1485 {
1486 let entries = match object.data {
1487 ObjectData::Properties(bytes) => crate::write::table_entries(bytes)?,
1488 _ => Vec::new(),
1489 };
1490 if entries
1491 .iter()
1492 .all(|entry| previous.global_ids.get(entry) == object.global_ids.get(entry))
1493 {
1494 continue;
1495 }
1496 }
1497 let id = rename.get(id).copied().unwrap_or(*id);
1498 let mut replacement = match object.data {
1499 ObjectData::Properties(_) => {
1500 let mut replacement = PropertyObject::from_object(object)?;
1501 remap(&mut replacement, &rename)?;
1502 replacement
1503 }
1504 ObjectData::File {
1505 reference,
1506 extension,
1507 } => {
1508 let text = |bytes: &[u8]| {
1509 String::from_utf16(
1510 &bytes
1511 .chunks_exact(2)
1512 .map(|pair| u16::from_le_bytes([pair[0], pair[1]]))
1513 .collect::<Vec<_>>(),
1514 )
1515 .map_err(|_| invalid("Invalid UTF-16 file-data declaration"))
1516 };
1517 let mut replacement =
1518 PropertyObject::file(id, &text(reference)?, &text(extension)?)?;
1519 replacement.jcid = object.jcid;
1520 replacement
1521 }
1522 ObjectData::Encrypted(_) => {
1523 return Err(invalid("Page edits only produce property objects"));
1524 }
1525 };
1526 replacement.reference(id)?;
1527 if before.objects.contains_key(&id) && rename.values().any(|model| *model == id) {
1528 return Err(invalid(
1529 "A new model identity already exists in the section",
1530 ));
1531 }
1532 changed.insert(id, replacement);
1533 }
1534 changes.insert(*sid, RevisionEdit::Update(changed));
1535 }
1536 Ok(changes)
1537 };
1538 let output = crate::write::build_on(source, &payloads, protection, edit)?;
1539 crate::write::check(&output, protection.is_none())?;
1540 Ok(output)
1541}
1542
1543fn remap(object: &mut PropertyObject, rename: &BTreeMap<ExGuid, ExGuid>) -> Result<(), Error> {
1544 let mut remapped = Vec::new();
1545 for property in PropertySets::parse(&object.bytes)?.sets.iter().flatten() {
1546 if let Value::References {
1547 stream: crate::IdStream::Objects,
1548 compact_ids,
1549 } = property.value
1550 {
1551 for bytes in compact_ids.chunks_exact(4) {
1552 let offset = bytes.as_ptr().addr() - object.bytes.as_ptr().addr();
1553 let id = crate::bytes::Cursor { bytes, offset }.compact(&object.global_ids)?;
1554 if let Some(model) = rename.get(&id) {
1555 remapped.push((offset, *model));
1556 }
1557 }
1558 }
1559 }
1560 for (offset, id) in remapped {
1561 let reference = object.reference(id)?;
1562 object.bytes[offset..offset + 4].copy_from_slice(&reference);
1563 }
1564 Ok(())
1565}
1566
1567#[cfg(test)]
1568mod tests {
1569 use super::*;
1570 use crate::{
1571 document::Format,
1572 page::{PageParagraph, Paragraph, TextObject, text::new_id},
1573 };
1574
1575 /// A new page in a new section and the page with `count` paragraphs added in an outline.
1576 fn generated(count: usize) -> (Vec<u8>, ExGuid, Page) {
1577 let section = crate::create_section("big.one", "First", "Author").unwrap();
1578 let creation = crate::PageCreation::new(None, Some("Big"), "Author").unwrap();
1579 let source = crate::PreparedEdit::create_page(&section, &creation)
1580 .unwrap()
1581 .as_bytes()
1582 .to_vec();
1583 let space = creation.space();
1584 let store = Store::parse(&source).unwrap();
1585 let index = RevisionIndex::parse(&store).unwrap();
1586 let mut page =
1587 Page::from_space(&crate::document::Document::parse(&index).unwrap(), space).unwrap();
1588 let paragraph = |i: usize| PageParagraph {
1589 id: new_id().unwrap(),
1590 parent: None,
1591 level: 1,
1592 style: None,
1593 format: Format::default(),
1594 content: ParagraphContent::Text(TextObject {
1595 id: new_id().unwrap(),
1596 date_field: None,
1597 text: Paragraph::new(format!("Paragraph {i}"), Format::default()),
1598 tags: Vec::new(),
1599 }),
1600 lists: Vec::new(),
1601 tags: Vec::new(),
1602 media: Default::default(),
1603 collapsed: false,
1604 };
1605 page.objects.push(PageObject::Outline(Outline {
1606 id: new_id().unwrap(),
1607 title: false,
1608 min_width: None,
1609 layout: crate::document::Layout {
1610 x: Some(36.0),
1611 y: Some(86.4),
1612 ..Default::default()
1613 },
1614 indents: Vec::new(),
1615 paragraphs: (0..count).map(paragraph).collect(),
1616 unsupported: Vec::new(),
1617 }));
1618 (source, space, page)
1619 }
1620
1621 #[test]
1622 fn a_page_write_builds_one_revision_however_many_paragraphs_it_adds() {
1623 for count in [10, 40] {
1624 let (source, space, after) = generated(count);
1625 let builds = crate::write::BUILDS.with(std::cell::Cell::get);
1626 let written = write_page(&source, space, &after, "Author").unwrap();
1627 assert_eq!(crate::write::BUILDS.with(std::cell::Cell::get) - builds, 1);
1628 let store = Store::parse(&written).unwrap();
1629 let index = RevisionIndex::parse(&store).unwrap();
1630 let read = Page::from_space(&crate::document::Document::parse(&index).unwrap(), space)
1631 .unwrap();
1632 let texts = |page: &Page| -> Vec<String> {
1633 page.objects
1634 .iter()
1635 .filter_map(|object| match object {
1636 PageObject::Outline(outline) => Some(outline),
1637 _ => None,
1638 })
1639 .flat_map(|outline| &outline.paragraphs)
1640 .filter_map(|paragraph| Some(paragraph.text()?.text.text().to_owned()))
1641 .collect()
1642 };
1643 assert_eq!(texts(&read), texts(&after));
1644 }
1645 }
1646}
crates/onestore/src/pages.rs+251-76
...@@ -1,9 +1,9 @@...@@ -1,9 +1,9 @@
1use crate::{1use crate::{
2 Error, ExGuid,2 Error, ExGuid,
3 create::{current_timestamps, properties, string},3 create::{current_timestamps, properties, string},
4 document::{Document, FieldValue, Kind},4 document::{FieldValue, Kind},
5 op::content::{NATIVE_INDENTS, measurement_bytes},5 op::content::{NATIVE_INDENTS, measurement_bytes},
6 write::{PropertyObject, RevisionEdit, fresh_guid, write_revisions},6 write::{PropertyObject, fresh_guid},
7};7};
8use serde::{Deserialize, Serialize};8use serde::{Deserialize, Serialize};
9use std::{9use std::{
...@@ -15,7 +15,7 @@ fn invalid(message: &'static str) -> Error {...@@ -15,7 +15,7 @@ fn invalid(message: &'static str) -> Error {
15 Error { offset: 0, message }15 Error { offset: 0, message }
16}16}
1717
18fn metadata_id(space: ExGuid) -> ExGuid {18pub(crate) fn metadata_id(space: ExGuid) -> ExGuid {
19 let mut id = ExGuid {19 let mut id = ExGuid {
20 guid: space.guid,20 guid: space.guid,
21 n: 1,21 n: 1,
...@@ -28,7 +28,7 @@ fn metadata_id(space: ExGuid) -> ExGuid {...@@ -28,7 +28,7 @@ fn metadata_id(space: ExGuid) -> ExGuid {
28 id28 id
29}29}
3030
31fn metadata_guid(metadata: &crate::document::Element<'_>) -> Result<[u8; 16], Error> {31pub(crate) fn metadata_guid(metadata: &crate::document::Element<'_>) -> Result<[u8; 16], Error> {
32 metadata.extra[0]32 metadata.extra[0]
33 .iter()33 .iter()
34 .find_map(|field| match field.value {34 .find_map(|field| match field.value {
...@@ -38,7 +38,7 @@ fn metadata_guid(metadata: &crate::document::Element<'_>) -> Result<[u8; 16], Er...@@ -38,7 +38,7 @@ fn metadata_guid(metadata: &crate::document::Element<'_>) -> Result<[u8; 16], Er
38 .ok_or_else(|| invalid("Page metadata needs its page identifier"))38 .ok_or_else(|| invalid("Page metadata needs its page identifier"))
39}39}
4040
41fn set_references(object: &mut PropertyObject, property: u32, ids: &[ExGuid]) -> Result<(), Error> {41pub(crate) fn set_references(object: &mut PropertyObject, property: u32, ids: &[ExGuid]) -> Result<(), Error> {
42 let mut references = Vec::new();42 let mut references = Vec::new();
43 for id in ids {43 for id in ids {
44 references.extend_from_slice(&object.reference(*id)?);44 references.extend_from_slice(&object.reference(*id)?);
...@@ -51,6 +51,20 @@ fn set_references(object: &mut PropertyObject, property: u32, ids: &[ExGuid]) ->...@@ -51,6 +51,20 @@ fn set_references(object: &mut PropertyObject, property: u32, ids: &[ExGuid]) ->
51 object.copy_property(&source, property)51 object.copy_property(&source, property)
52}52}
5353
54/// A conflict page (MS-ONE 2.1.2): the version of a page a merge could not take, kept
55/// read-only under the page it conflicts with.
56#[derive(Debug, Clone, PartialEq, Eq)]
57pub struct ConflictPage {
58 pub space: ExGuid,
59 pub title: String,
60 /// Whose version it is (`ConflictingUserName`).
61 pub user: String,
62 /// FILETIME of its `TopologyCreationTimeStamp`, when the merge made it.
63 pub created: Option<u64>,
64 /// The conflict objects on it (`IsConflictObjectForRender`), which OneNote highlights.
65 pub objects: Vec<ExGuid>,
66}
67
54/// Whether an edit preserves position or moves before a page (None appends).68/// Whether an edit preserves position or moves before a page (None appends).
55#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]69#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
56#[serde(deny_unknown_fields)]70#[serde(deny_unknown_fields)]
...@@ -138,42 +152,7 @@ pub(crate) struct PageParts<'a> {...@@ -138,42 +152,7 @@ pub(crate) struct PageParts<'a> {
138 pub stored_metadata: (ExGuid, PropertyObject),152 pub stored_metadata: (ExGuid, PropertyObject),
139}153}
140154
141pub(crate) fn edit_pages(155/// Page moves and removals on a section kept open: the objects each changed space stores.
142 source: &[u8],
143 edits: &[PageEdit],
144 removals: &[ExGuid],
145) -> Result<Vec<u8>, Error> {
146 write_revisions(source, |index| {
147 let document = Document::parse(index)?;
148 let pages: Vec<ExGuid> = document.pages()?.into_iter().map(|(sid, _)| sid).collect();
149 let root = document.active(document.root)?;
150 let raw = index.resolve_active(document.root)?;
151 let parts = |sid: ExGuid| -> Result<PageParts<'_>, Error> {
152 let page = document.active(sid)?;
153 let revision = index.resolve_active(sid)?;
154 let metadata = page
155 .roots
156 .get(&2)
157 .and_then(|id| page.nodes.get(id))
158 .ok_or_else(|| invalid("Page metadata is unavailable"))?;
159 let (manifest, stored) = (revision.roots[&1], revision.roots[&2]);
160 Ok(PageParts {
161 metadata: metadata.clone(),
162 manifest: (manifest, PropertyObject::from_object(&revision.objects[&manifest])?),
163 stored_metadata: (stored, PropertyObject::from_object(&revision.objects[&stored])?),
164 })
165 };
166 let Some(changes) = page_changes(document.root, root, &raw, &pages, parts, edits, removals)? else {
167 return Ok(BTreeMap::new());
168 };
169 Ok(changes
170 .into_iter()
171 .map(|(space, changes)| (space, RevisionEdit::Update(changes)))
172 .collect())
173 })
174}
175
176/// `edit_pages` on a section kept open: the objects each changed space stores.
177pub(crate) fn section_changes(156pub(crate) fn section_changes(
178 section: &mut crate::Section<'_>,157 section: &mut crate::Section<'_>,
179 edits: &[PageEdit],158 edits: &[PageEdit],
...@@ -197,12 +176,18 @@ pub(crate) fn section_changes(...@@ -197,12 +176,18 @@ pub(crate) fn section_changes(
197 let parts = |sid: ExGuid| -> Result<PageParts<'_>, Error> {176 let parts = |sid: ExGuid| -> Result<PageParts<'_>, Error> {
198 let revision = section.revision(sid)?;177 let revision = section.revision(sid)?;
199 let (manifest, stored) = (revision.roots[&1], revision.roots[&2]);178 let (manifest, stored) = (revision.roots[&1], revision.roots[&2]);
200 Ok(PageParts {179 let element = |id| {
201 metadata: crate::document::Element::parse_with(180 crate::document::Element::parse_with(
202 &revision.objects[&stored],181 &revision.objects[id],
203 crate::FileType::Section,182 crate::FileType::Section,
204 &mut |_| Err(invalid("Page metadata holds no payload")),183 &mut |_| Err(invalid("Page metadata holds no payload")),
205 )?,184 )
185 };
186 if element(&manifest)?.content.len() != 1 {
187 return Err(invalid("Each page space must contain one page with a valid level"));
188 }
189 Ok(PageParts {
190 metadata: element(&stored)?,
206 manifest: (manifest, PropertyObject::from_object(&revision.objects[&manifest])?),191 manifest: (manifest, PropertyObject::from_object(&revision.objects[&manifest])?),
207 stored_metadata: (stored, PropertyObject::from_object(&revision.objects[&stored])?),192 stored_metadata: (stored, PropertyObject::from_object(&revision.objects[&stored])?),
208 })193 })
...@@ -491,6 +476,12 @@ pub struct PageCreation {...@@ -491,6 +476,12 @@ pub struct PageCreation {
491 title: Option<String>,476 title: Option<String>,
492 author: String,477 author: String,
493 created: u32,478 created: u32,
479 /// The title's date and time fields' text, for a titled page.
480 #[serde(default, skip_serializing_if = "Option::is_none")]
481 date: Option<[String; 2]>,
482 /// The page identity and creation time (FILETIME) a page keeps from elsewhere.
483 #[serde(default, skip_serializing_if = "Option::is_none")]
484 kept: Option<([u8; 16], u64)>,
494}485}
495486
496impl PageCreation {487impl PageCreation {
...@@ -505,11 +496,57 @@ impl PageCreation {...@@ -505,11 +496,57 @@ impl PageCreation {
505 title: title.map(str::to_owned),496 title: title.map(str::to_owned),
506 author: author.to_owned(),497 author: author.to_owned(),
507 created: current_timestamps()?.0,498 created: current_timestamps()?.0,
499 date: None,
500 kept: None,
508 };501 };
509 page.validate()?;502 page.validate()?;
510 Ok(page)503 Ok(page)
511 }504 }
512505
506 /// Gives a titled page's title OneNote 2010's date and time fields, showing `date` and
507 /// `time`: the creation time (`created`) as the user's long date and short time.
508 pub fn dated(mut self, date: &str, time: &str) -> Result<Self, Error> {
509 if self.title.is_none() {
510 return Err(invalid("Only a titled page shows its date"));
511 }
512 self.date = Some([date.to_owned(), time.to_owned()]);
513 self.validate()?;
514 Ok(self)
515 }
516
517 /// Keeps `identity`, the page identity internal links name, and `created`, as OneNote
518 /// 2010 keeps both for a page it moves to the notebook's recycle bin.
519 pub fn keeping(mut self, identity: [u8; 16], created: u64) -> Result<Self, Error> {
520 if identity == [0; 16] {
521 return Err(invalid("Keep an existing page identity"));
522 }
523 let seconds = (created / 10_000_000)
524 .checked_sub(11_644_473_600 + 315_532_800)
525 .and_then(|seconds| u32::try_from(seconds).ok())
526 .ok_or_else(|| invalid("The kept creation time is outside OneNote's range"))?;
527 self.created = seconds;
528 self.kept = Some((identity, created));
529 Ok(self)
530 }
531
532 /// Who creates the page.
533 pub fn author(&self) -> &str {
534 &self.author
535 }
536
537 /// When the page was created, FILETIME, which its date and time fields show.
538 pub fn created(&self) -> u64 {
539 match self.kept {
540 Some((_, created)) => created,
541 None => (u64::from(self.created) + 315_532_800 + 11_644_473_600) * 10_000_000,
542 }
543 }
544
545 /// The page identity internal links name as `page-id`.
546 fn identity(&self) -> [u8; 16] {
547 self.kept.map_or(self.guid, |(identity, _)| identity)
548 }
549
513 pub fn space(&self) -> ExGuid {550 pub fn space(&self) -> ExGuid {
514 ExGuid {551 ExGuid {
515 guid: self.guid,552 guid: self.guid,
...@@ -531,6 +568,11 @@ impl PageCreation {...@@ -531,6 +568,11 @@ impl PageCreation {
531 })568 })
532 }569 }
533570
571 /// The page the new page goes before; `None` appends it.
572 pub fn before(&self) -> Option<ExGuid> {
573 self.before
574 }
575
534 /// Changes the insertion anchor while retaining all identities and creation metadata.576 /// Changes the insertion anchor while retaining all identities and creation metadata.
535 pub fn reposition(&self, before: Option<ExGuid>) -> Result<Self, Error> {577 pub fn reposition(&self, before: Option<ExGuid>) -> Result<Self, Error> {
536 let mut page = self.clone();578 let mut page = self.clone();
...@@ -547,8 +589,9 @@ impl PageCreation {...@@ -547,8 +589,9 @@ impl PageCreation {
547 || self.before.is_some_and(|id| id.guid == [0; 16])589 || self.before.is_some_and(|id| id.guid == [0; 16])
548 || self590 || self
549 .title591 .title
550 .as_ref()592 .iter()
551 .is_some_and(|title| title.contains(['\0', '\r', '\n', '\u{fffc}', '\u{fddf}']))593 .chain(self.date.iter().flatten())
594 .any(|text| text.contains(['\0', '\r', '\n', '\u{fffc}', '\u{fddf}']))
552 {595 {
553 return Err(invalid(596 return Err(invalid(
554 "Use a new page identity, an existing page anchor, and ordinary single-line title text",597 "Use a new page identity, an existing page anchor, and ordinary single-line title text",
...@@ -557,28 +600,7 @@ impl PageCreation {...@@ -557,28 +600,7 @@ impl PageCreation {
557 Ok(())600 Ok(())
558 }601 }
559602
560 pub(crate) fn apply(&self, source: &[u8]) -> Result<Vec<u8>, Error> {603 /// The root space's changes creating this page, and the new space's roots and objects.
561 self.validate()?;
562 write_revisions(source, |index| {
563 if index.spaces.contains_key(&self.space()) {
564 return Err(invalid(
565 "This page identity already exists; reconcile the original creation",
566 ));
567 }
568 let document = Document::parse(index)?;
569 document.pages()?;
570 let view = document.active(document.root)?;
571 let raw = index.resolve_active(document.root)?;
572 let (root, roots, objects) = self.creation(view, &raw)?;
573 Ok(BTreeMap::from([
574 (document.root, RevisionEdit::Update(root)),
575 (self.space(), RevisionEdit::Create { roots, objects }),
576 ]))
577 })
578 }
579
580 /// `apply` on a section kept open: the root space's changes, and the new space's roots
581 /// and objects.
582 pub(crate) fn changes(604 pub(crate) fn changes(
583 &self,605 &self,
584 section: &mut crate::Section<'_>,606 section: &mut crate::Section<'_>,
...@@ -594,6 +616,133 @@ impl PageCreation {...@@ -594,6 +616,133 @@ impl PageCreation {
594 self.creation(&page.view, &page.live.revision)616 self.creation(&page.view, &page.live.revision)
595 }617 }
596618
619 /// The object space of a conflict page this creation makes, as OneNote 2010 stores one:
620 /// the page's space with `jcidConflictPageMetaData` (MS-ONE 2.2.35) as its metadata,
621 /// naming this creation's author as the conflicting user and `title` as the page's.
622 /// Returns the roots, the objects and the metadata, which the conflicting page's
623 /// manifest also keeps a copy of.
624 pub(crate) fn conflict(
625 &self,
626 section: &mut crate::Section<'_>,
627 title: &str,
628 level: u32,
629 ) -> Result<(BTreeMap<u32, ExGuid>, crate::active::Changes, PropertyObject), Error> {
630 if self.before.is_some() {
631 return Err(invalid("A conflict page is not placed in the page list"));
632 }
633 let (_, roots, mut objects) = self.changes(section)?;
634 let timestamp = self.created();
635 let initials = crate::create::initials(&self.author);
636 let metadata = PropertyObject {
637 jcid: 0x20038,
638 bytes: properties(&[
639 (0x1c001cf3, string(title)),
640 (0x1c001c30, self.identity().to_vec()),
641 (0x14001dff, level.to_le_bytes().to_vec()),
642 (0x14001d82, 40_u32.to_le_bytes().to_vec()),
643 (0x1400348b, 40_u32.to_le_bytes().to_vec()),
644 (0x18001c65, timestamp.to_le_bytes().to_vec()),
645 (0x1c001d9e, string(&self.author)),
646 (0x1c001d9f, string(&initials)),
647 ])?,
648 global_ids: Arc::new(BTreeMap::from([(0, self.guid)])),
649 };
650 objects.insert(roots[&2], metadata.clone());
651 Ok((roots, objects, metadata))
652 }
653
654 /// The title's second outline as OneNote 2010 writes it for a new page (object numbers,
655 /// jcids and properties): a read-only outline of two paragraphs, the date then the
656 /// time, in the `PageDateTime` style.
657 fn date_fields(&self, date: &str, time: &str) -> Vec<(u32, u32, Vec<(u32, Vec<u8>)>)> {
658 let reference = |n: u32| n.to_le_bytes().to_vec();
659 let modified = || (0x14001d7a, self.created.to_le_bytes().to_vec());
660 let yes = |id: u32| (id | 1 << 31, Vec::new());
661 let paragraph = |text: u32| {
662 vec![
663 modified(),
664 (0x20001d79, reference(17)),
665 (0x20001d78, reference(17)),
666 (0x14001d09, self.created.to_le_bytes().to_vec()),
667 (0x0c001c03, vec![1]),
668 (0x24001c1f, reference(text)),
669 yes(0x08001cb2),
670 yes(0x08001d0c),
671 (0x08001c34, Vec::new()),
672 ]
673 };
674 let text = |value: &str, role: u32| {
675 // OneNote keeps Latin-1 text as TextExtendedAscii, other text as Unicode.
676 let content = match value.chars().map(|c| u8::try_from(c).ok()).collect::<Option<Vec<u8>>>() {
677 Some(latin) => (0x1c003498, latin),
678 None => (0x1c001c22, string(value)),
679 };
680 vec![
681 modified(),
682 (0x2000342c, reference(23)),
683 yes(0x08001cde),
684 yes(0x08001c00),
685 yes(0x08001c88),
686 yes(0x08001d0c),
687 yes(role),
688 (0x10001cfe, 0x409_u16.to_le_bytes().to_vec()),
689 (0x1c001cc8, vec![0, 0, 0, 0, 3]),
690 content,
691 (0x24001e13, reference(19)),
692 yes(0x080034dd),
693 ]
694 };
695 let no = |id: u32| (id, Vec::new());
696 vec![
697 (
698 20,
699 0x6000c,
700 vec![
701 modified(),
702 (0x24001c20, [reference(21), reference(24)].concat()),
703 (
704 0x1c001c12,
705 measurement_bytes(&NATIVE_INDENTS, 4).expect("Native indents encode"),
706 ),
707 (0x0c001c13, vec![0]),
708 (0x0c001c03, vec![1]),
709 (0x14001c1c, 0.6_f32.to_le_bytes().to_vec()),
710 yes(0x08001cb5),
711 yes(0x08001c00),
712 yes(0x08001cb2),
713 yes(0x08001cde),
714 yes(0x08001d0c),
715 (0x14001c3e, reference(0)),
716 (0x14001c84, reference(0xc)),
717 ],
718 ),
719 (21, 0x6000d, paragraph(22)),
720 (22, 0x6000e, text(date, 0x08001cb5)),
721 (
722 23,
723 0x12004d,
724 vec![
725 (0x1c00345a, string("PageDateTime")),
726 no(0x08001c04),
727 no(0x08001c05),
728 no(0x08001c06),
729 (0x14001c0c, 0x0080_8080_u32.to_le_bytes().to_vec()),
730 (0x1c001c0a, string("Calibri")),
731 (0x10001c0b, 20_u16.to_le_bytes().to_vec()),
732 (0x1400342e, 0_f32.to_le_bytes().to_vec()),
733 (0x1400342f, 0_f32.to_le_bytes().to_vec()),
734 (0x14003430, 0_f32.to_le_bytes().to_vec()),
735 no(0x08001c07),
736 (0x14001c0d, 0xff00_0000_u32.to_le_bytes().to_vec()),
737 no(0x08001c09),
738 no(0x08001c08),
739 ],
740 ),
741 (24, 0x6000d, paragraph(25)),
742 (25, 0x6000e, text(time, 0x08001c87)),
743 ]
744 }
745
597 fn creation(746 fn creation(
598 &self,747 &self,
599 view: &crate::document::Revision<'_>,748 view: &crate::document::Revision<'_>,
...@@ -621,9 +770,7 @@ impl PageCreation {...@@ -621,9 +770,7 @@ impl PageCreation {
621 };770 };
622 let id = |n| ExGuid { guid: self.guid, n };771 let id = |n| ExGuid { guid: self.guid, n };
623 let reference = |n: u32| n.to_le_bytes().to_vec();772 let reference = |n: u32| n.to_le_bytes().to_vec();
624 let timestamp = ((u64::from(self.created) + 315532800 + 11644473600) * 10000000)773 let timestamp = self.created().to_le_bytes().to_vec();
625 .to_le_bytes()
626 .to_vec();
627 let modified = || (0x14001d7a, self.created.to_le_bytes().to_vec());774 let modified = || (0x14001d7a, self.created.to_le_bytes().to_vec());
628 let mut table = BTreeMap::from([(0, self.guid)]);775 let mut table = BTreeMap::from([(0, self.guid)]);
629 let metadata_id = metadata_id(self.space());776 let metadata_id = metadata_id(self.space());
...@@ -638,7 +785,7 @@ impl PageCreation {...@@ -638,7 +785,7 @@ impl PageCreation {
638 };785 };
639 let title = self.title.as_deref().unwrap_or_default().trim_start();786 let title = self.title.as_deref().unwrap_or_default().trim_start();
640 let metadata = vec![787 let metadata = vec![
641 (0x1c001c30, self.guid.to_vec()),788 (0x1c001c30, self.identity().to_vec()),
642 (0x1c001cf3, string(title)),789 (0x1c001cf3, string(title)),
643 (0x14001d82, reference(40)),790 (0x14001d82, reference(40)),
644 (0x1400348b, reference(40)),791 (0x1400348b, reference(40)),
...@@ -649,6 +796,7 @@ impl PageCreation {...@@ -649,6 +796,7 @@ impl PageCreation {
649 vec![796 vec![
650 modified(),797 modified(),
651 (0x1c001d75, string(&self.author)),798 (0x1c001d75, string(&self.author)),
799 (0x1c001df8, string(&crate::create::initials(&self.author))),
652 (0x1c001d3c, string("")),800 (0x1c001d3c, string("")),
653 ],801 ],
654 crate::create::page_margins(),802 crate::create::page_margins(),
...@@ -669,7 +817,11 @@ impl PageCreation {...@@ -669,7 +817,11 @@ impl PageCreation {
669 0x6002c,817 0x6002c,
670 vec![818 vec![
671 modified(),819 modified(),
672 (0x24001c20, reference(14)),820 (
821 0x24001c20,
822 [reference(14), self.date.iter().flat_map(|_| reference(20)).collect()]
823 .concat(),
824 ),
673 (0x14001c14, 0_f32.to_le_bytes().to_vec()),825 (0x14001c14, 0_f32.to_le_bytes().to_vec()),
674 (0x14001c15, 0_f32.to_le_bytes().to_vec()),826 (0x14001c15, 0_f32.to_le_bytes().to_vec()),
675 (0x14001c3e, reference(0x0009_000c)),827 (0x14001c3e, reference(0x0009_000c)),
...@@ -723,7 +875,7 @@ impl PageCreation {...@@ -723,7 +875,7 @@ impl PageCreation {
723 (0x88001cb4, vec![]),875 (0x88001cb4, vec![]),
724 ],876 ],
725 ),877 ),
726 (17, 0x120001, vec![(0x1c001d75, string(&self.author))]),878 (17, 0x120001, crate::create::author_properties(&self.author)),
727 (879 (
728 18,880 18,
729 0x12004d,881 0x12004d,
...@@ -737,6 +889,11 @@ impl PageCreation {...@@ -737,6 +889,11 @@ impl PageCreation {
737 ] {889 ] {
738 objects.insert(id(n), object(jcid, values)?);890 objects.insert(id(n), object(jcid, values)?);
739 }891 }
892 if let Some([date, time]) = &self.date {
893 for (n, jcid, values) in self.date_fields(date, time) {
894 objects.insert(id(n), object(jcid, values)?);
895 }
896 }
740 }897 }
741 let series = object(898 let series = object(
742 0x60008,899 0x60008,
...@@ -774,7 +931,25 @@ impl PageCreation {...@@ -774,7 +931,25 @@ impl PageCreation {
774#[cfg(test)]931#[cfg(test)]
775mod tests {932mod tests {
776 use super::*;933 use super::*;
777 use crate::{RevisionIndex, Store};934 use crate::{
935 RevisionIndex, Store,
936 document::Document,
937 op::{Edit, Op, OpError, SectionOp},
938 write::{RevisionEdit, write_revisions},
939 };
940
941 /// Page moves and removals through a section, sealed: the image they leave.
942 fn edit_pages(source: &[u8], edits: &[PageEdit], removals: &[ExGuid]) -> Result<Vec<u8>, OpError> {
943 let arena = crate::Arena::default();
944 let mut section = crate::Section::open(&arena, source.to_vec()).map_err(OpError::Failed)?;
945 let mut ops = vec![Op::Section(SectionOp::Pages(edits.to_vec()))];
946 if !removals.is_empty() {
947 ops.push(Op::Section(SectionOp::Delete(removals.to_vec())));
948 }
949 section.apply("Author", &Edit { at: 133_700_000_000_000_000, ops })?;
950 section.seal().map_err(OpError::Failed)?;
951 Ok(section.image())
952 }
778953
779 #[test]954 #[test]
780 fn page_movement_restores_an_optional_metadata_cache_from_history() {955 fn page_movement_restores_an_optional_metadata_cache_from_history() {
crates/onestore/src/paragraph.rs+10-89
...@@ -6,7 +6,6 @@ use crate::{...@@ -6,7 +6,6 @@ use crate::{
6 edit::update_title,6 edit::update_title,
7 write::{PropertyObject, fresh_guid},7 write::{PropertyObject, fresh_guid},
8};8};
9use serde::{Deserialize, Serialize};
10use std::{9use std::{
11 collections::{BTreeMap, BTreeSet},10 collections::{BTreeMap, BTreeSet},
12 ops::Range,11 ops::Range,
...@@ -17,11 +16,10 @@ fn invalid(message: &'static str) -> Error {...@@ -17,11 +16,10 @@ fn invalid(message: &'static str) -> Error {
17 Error { offset: 0, message }16 Error { offset: 0, message }
18}17}
1918
20/// Splits ordinary paragraph text while retaining the new objects' identities across retries.19/// Splits ordinary paragraph text. The original paragraph/text remain on the left; nested
21/// The original paragraph/text remain on the left; nested children move to the right.20/// children move to the right.
22#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]21#[derive(Debug, Clone, PartialEq)]
23#[serde(deny_unknown_fields)]22pub(crate) struct ParagraphSplit {
24pub struct ParagraphSplit {
25 guid: [u8; 16],23 guid: [u8; 16],
26 text: ExGuid,24 text: ExGuid,
27 offset: u32,25 offset: u32,
...@@ -31,7 +29,7 @@ pub struct ParagraphSplit {...@@ -31,7 +29,7 @@ pub struct ParagraphSplit {
3129
32impl ParagraphSplit {30impl ParagraphSplit {
33 /// The offset is measured in UTF-16 code units and must lie between Unicode scalars.31 /// The offset is measured in UTF-16 code units and must lie between Unicode scalars.
34 pub fn new(text: ExGuid, offset: u32, author: &str) -> Result<Self, Error> {32 pub(crate) fn new(text: ExGuid, offset: u32, author: &str) -> Result<Self, Error> {
35 if text.guid == [0; 16] || author.contains('\0') {33 if text.guid == [0; 16] || author.contains('\0') {
36 return Err(invalid(34 return Err(invalid(
37 "Choose paragraph text and an author name without NUL",35 "Choose paragraph text and an author name without NUL",
...@@ -46,64 +44,6 @@ impl ParagraphSplit {...@@ -46,64 +44,6 @@ impl ParagraphSplit {
46 })44 })
47 }45 }
4846
49 /// Identity of the new right paragraph.
50 pub fn object(&self) -> ExGuid {
51 ExGuid {
52 guid: self.guid,
53 n: 1,
54 }
55 }
56
57 /// Identity of the new right paragraph's text.
58 pub fn text_object(&self) -> ExGuid {
59 ExGuid {
60 guid: self.guid,
61 n: 2,
62 }
63 }
64
65 /// Original text identity and UTF-16 split boundary.
66 pub fn position(&self) -> (ExGuid, u32) {
67 (self.text, self.offset)
68 }
69
70 /// Changes the split boundary while retaining allocated identities and creation time.
71 /// Preparing the edit validates the offset against the supplied text.
72 pub fn reposition(&self, offset: u32) -> Self {
73 let mut split = self.clone();
74 split.offset = offset;
75 split
76 }
77
78 pub(crate) fn apply(&self, source: &[u8], space: ExGuid) -> Result<Vec<u8>, Error> {
79 if self.guid == [0; 16] || self.text.guid == [0; 16] || self.author.contains('\0') {
80 return Err(invalid(
81 "Choose paragraph text and an author name without NUL",
82 ));
83 }
84 crate::active::write(source, space, |active| self.changes(active))
85 }
86
87 pub(crate) fn changes(&self, active: &ActivePage<'_>) -> Result<Changes, Error> {
88 let lists = (5..256)
89 .map(|n| ExGuid {
90 guid: self.guid,
91 n,
92 })
93 .collect::<Vec<_>>();
94 let changes = self.changes_as(active, self.object(), self.text_object(), &lists)?;
95 let raw = &active.live.revision;
96 if changes
97 .keys()
98 .any(|id| id.guid == self.guid && raw.objects.contains_key(id))
99 {
100 return Err(invalid(
101 "A split identity already exists; reconcile the existing edit",
102 ));
103 }
104 Ok(changes)
105 }
106
107 /// `changes` creating the right paragraph `paragraph` with text `right`, and a copy of47 /// `changes` creating the right paragraph `paragraph` with text `right`, and a copy of
108 /// the left paragraph's list nodes under `lists`, in order.48 /// the left paragraph's list nodes under `lists`, in order.
109 pub(crate) fn changes_as(49 pub(crate) fn changes_as(
...@@ -244,7 +184,7 @@ impl ParagraphSplit {...@@ -244,7 +184,7 @@ impl ParagraphSplit {
244 author_id,184 author_id,
245 PropertyObject {185 PropertyObject {
246 jcid: 0x120001,186 jcid: 0x120001,
247 bytes: properties(&[(0x1c001d75, string(&self.author))])?,187 bytes: properties(&crate::create::author_properties(&self.author))?,
248 global_ids: Arc::new(BTreeMap::from([(0, self.guid)])),188 global_ids: Arc::new(BTreeMap::from([(0, self.guid)])),
249 },189 },
250 );190 );
...@@ -411,22 +351,16 @@ fn ordinary_text<'a>(...@@ -411,22 +351,16 @@ fn ordinary_text<'a>(
411/// The left paragraph survives. Empty left text adopts the right text identity;351/// The left paragraph survives. Empty left text adopts the right text identity;
412/// otherwise the left text survives. The left paragraph's tags are retained.352/// otherwise the left text survives. The left paragraph's tags are retained.
413/// Right-side tags are removed from active text, including when the left text is empty.353/// Right-side tags are removed from active text, including when the left text is empty.
414#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]354#[derive(Debug, Clone, PartialEq)]
415#[serde(deny_unknown_fields)]355pub(crate) struct ParagraphJoin {
416pub struct ParagraphJoin {
417 left: ExGuid,356 left: ExGuid,
418 right: ExGuid,357 right: ExGuid,
419 author: String,358 author: String,
420}359}
421360
422impl ParagraphJoin {361impl ParagraphJoin {
423 /// Ordered left and right text identities.
424 pub fn texts(&self) -> [ExGuid; 2] {
425 [self.left, self.right]
426 }
427
428 /// Select the preceding leaf paragraph's text and the following paragraph's text.362 /// Select the preceding leaf paragraph's text and the following paragraph's text.
429 pub fn new(left: ExGuid, right: ExGuid, author: &str) -> Result<Self, Error> {363 pub(crate) fn new(left: ExGuid, right: ExGuid, author: &str) -> Result<Self, Error> {
430 if left == right || left.guid == [0; 16] || right.guid == [0; 16] || author.contains('\0') {364 if left == right || left.guid == [0; 16] || right.guid == [0; 16] || author.contains('\0') {
431 return Err(invalid(365 return Err(invalid(
432 "Choose two distinct text objects and an author name without NUL",366 "Choose two distinct text objects and an author name without NUL",
...@@ -439,19 +373,6 @@ impl ParagraphJoin {...@@ -439,19 +373,6 @@ impl ParagraphJoin {
439 })373 })
440 }374 }
441375
442 pub(crate) fn apply(&self, source: &[u8], space: ExGuid) -> Result<Vec<u8>, Error> {
443 if self.left == self.right
444 || self.left.guid == [0; 16]
445 || self.right.guid == [0; 16]
446 || self.author.contains('\0')
447 {
448 return Err(invalid(
449 "Choose two distinct text objects and an author name without NUL",
450 ));
451 }
452 crate::active::write(source, space, |active| self.changes(active))
453 }
454
455 pub(crate) fn changes(&self, active: &ActivePage<'_>) -> Result<Changes, Error> {376 pub(crate) fn changes(&self, active: &ActivePage<'_>) -> Result<Changes, Error> {
456 let pages = &active.pages;377 let pages = &active.pages;
457 if pages.len() != 1 {378 if pages.len() != 1 {
...@@ -672,7 +593,7 @@ impl ParagraphJoin {...@@ -672,7 +593,7 @@ impl ParagraphJoin {
672 author,593 author,
673 PropertyObject {594 PropertyObject {
674 jcid: 0x120001,595 jcid: 0x120001,
675 bytes: properties(&[(0x1c001d75, string(&self.author))])?,596 bytes: properties(&crate::create::author_properties(&self.author))?,
676 global_ids: Arc::new(BTreeMap::from([(0, author.guid)])),597 global_ids: Arc::new(BTreeMap::from([(0, author.guid)])),
677 },598 },
678 );599 );
crates/onestore/src/protected/crypto.rs+1
...@@ -8,6 +8,7 @@ use zeroize::{Zeroize, Zeroizing};...@@ -8,6 +8,7 @@ use zeroize::{Zeroize, Zeroizing};
8const NS: &str = "http://schemas.microsoft.com/office/2006/encryption";8const NS: &str = "http://schemas.microsoft.com/office/2006/encryption";
9const PASSWORD_NS: &str = "http://schemas.microsoft.com/office/2006/keyEncryptor/password";9const PASSWORD_NS: &str = "http://schemas.microsoft.com/office/2006/keyEncryptor/password";
1010
11#[derive(Clone)]
11pub(super) struct Key {12pub(super) struct Key {
12 value: Zeroizing<[u8; 16]>,13 value: Zeroizing<[u8; 16]>,
13 file_iv: [u8; 16],14 file_iv: [u8; 16],
crates/onestore/src/protected/mod.rs+76-56
...@@ -7,7 +7,11 @@...@@ -7,7 +7,11 @@
77
8mod crypto;8mod crypto;
99
10use crate::{ExGuid, FileDataReference, ObjectData, Reference, RevisionIndex, document::Document};10use crate::{
11 ExGuid, FileDataReference, ObjectData, Reference, RevisionIndex, Transaction,
12 document::Document,
13 op::{Edit, Op, OpError},
14};
11use std::{15use std::{
12 collections::{BTreeMap, BTreeSet},16 collections::{BTreeMap, BTreeSet},
13 fmt,17 fmt,
...@@ -107,7 +111,18 @@ pub struct UnlockedSection<'a> {...@@ -107,7 +111,18 @@ pub struct UnlockedSection<'a> {
107impl<'a> UnlockedSection<'a> {111impl<'a> UnlockedSection<'a> {
108 /// Verifies the password and every labeled revision before exposing a view.112 /// Verifies the password and every labeled revision before exposing a view.
109 /// Metadata and password input are each bounded to 64 KiB.113 /// Metadata and password input are each bounded to 64 KiB.
110 pub fn open(index: &'a RevisionIndex<'a>, password: &str, mut limits: Limits) -> Result<Self> {114 pub fn open(index: &'a RevisionIndex<'a>, password: &str, limits: Limits) -> Result<Self> {
115 Self::unlock(index, limits, |metadata, rounds| {
116 crypto::Key::open(metadata, password, rounds)
117 })
118 }
119
120 /// `open` with each distinct encryption metadata's key from `key`.
121 fn unlock(
122 index: &'a RevisionIndex<'a>,
123 mut limits: Limits,
124 mut key: impl FnMut(&[u8], &mut u64) -> Result<crypto::Key>,
125 ) -> Result<Self> {
111 if index.store.header.file_type != crate::FileType::Section {126 if index.store.header.file_type != crate::FileType::Section {
112 return Err(Error::Unsupported);127 return Err(Error::Unsupported);
113 }128 }
...@@ -163,10 +178,7 @@ impl<'a> UnlockedSection<'a> {...@@ -163,10 +178,7 @@ impl<'a> UnlockedSection<'a> {
163 let metadata =178 let metadata =
164 metadata.ok_or_else(|| invalid("Protected object space has no revision"))?;179 metadata.ok_or_else(|| invalid("Protected object space has no revision"))?;
165 if !keys.contains_key(metadata) {180 if !keys.contains_key(metadata) {
166 keys.insert(181 keys.insert(metadata, key(metadata, &mut limits.kdf_rounds)?);
167 metadata,
168 crypto::Key::open(metadata, password, &mut limits.kdf_rounds)?,
169 );
170 }182 }
171 let key = &keys[metadata];183 let key = &keys[metadata];
172 for rid in space.labels.values().copied().collect::<BTreeSet<_>>() {184 for rid in space.labels.values().copied().collect::<BTreeSet<_>>() {
...@@ -294,8 +306,10 @@ impl UnlockedSection<'_> {...@@ -294,8 +306,10 @@ impl UnlockedSection<'_> {
294 use crate::write::{PropertyObject, RevisionEdit, append_revisions};306 use crate::write::{PropertyObject, RevisionEdit, append_revisions};
295 let copy = |space: ExGuid, revision: ExGuid| {307 let copy = |space: ExGuid, revision: ExGuid| {
296 let revision = self.resolve(space, revision)?;308 let revision = self.resolve(space, revision)?;
309 // A revision still declares objects its edits detached; a new one holds live ones.
310 let live = revision.reachable()?;
297 let mut objects = BTreeMap::new();311 let mut objects = BTreeMap::new();
298 for (id, object) in &revision.objects {312 for (id, object) in revision.objects.iter().filter(|(id, _)| live.contains(id)) {
299 let copy = match object.data {313 let copy = match object.data {
300 ObjectData::File {314 ObjectData::File {
301 reference,315 reference,
...@@ -390,60 +404,66 @@ impl UnlockedSection<'_> {...@@ -390,60 +404,66 @@ impl UnlockedSection<'_> {
390 }404 }
391}405}
392406
393/// An edited page of a protected section as one stored revision per changed space, the407impl UnlockedSection<'_> {
394/// protected counterpart of [`crate::PreparedEdit::page`].408 /// Applies `edit` to this section as `Section::apply` applies it to an ordinary one and
395pub(crate) fn write_page(409 /// seals the result as one revision per changed space, stored under the section's key.
396 source: &[u8],410 /// Page ops only: a protected section's page list is not edited.
397 password: &str,411 pub fn apply(&self, author: &str, edit: &Edit) -> std::result::Result<Transaction, OpError> {
398 space: ExGuid,412 let failed = |error: crate::Error| OpError::Failed(error);
399 page: &crate::page::Page,413 if self.keys.len() != 1 {
400 author: &str,414 return Err(OpError::Unsupported(
401) -> Result<Vec<u8>> {415 "Only a section under one key takes edits",
402 let store = crate::Store::parse(source)?;416 ));
403 let index = RevisionIndex::parse(&store)?;417 }
404 let unlocked = UnlockedSection::open(&index, password, Limits::default())?;418 if edit.ops.iter().any(|op| matches!(op, Op::Section(_))) {
405 if unlocked.keys.len() != 1 {419 return Err(OpError::Unsupported("A protected section's page list is not edited"));
406 return Err(Error::Unsupported);
407 }
408 let twin = unlocked.twin()?;
409 let applied = Zeroizing::new(crate::page::write::write_page(&twin, space, page, author)?);
410 let applied_store = crate::Store::parse(&applied)?;
411 let applied_index = RevisionIndex::parse(&applied_store)?;
412 let twin_store = crate::Store::parse(&twin)?;
413 if RevisionIndex::parse(&twin_store)?
414 .spaces
415 .keys()
416 .ne(applied_index.spaces.keys())
417 {
418 return Err(invalid("Page edits cannot create object spaces"));
419 }
420 // The twin's scaffold spaces are not the section's.
421 let mut edited = Vec::new();
422 for (sid, space) in &index.spaces {
423 let rid = applied_index.active(*sid)?;
424 if space.labels.get(&(ExGuid::default(), 1)) != Some(&rid) {
425 edited.push((*sid, applied_index.resolve(*sid, rid)?));
426 }420 }
421 let twin = self.twin().map_err(failed)?;
422 let arena = crate::Arena::default();
423 let mut section = crate::Section::open(&arena, twin.to_vec()).map_err(failed)?;
424 section.apply(author, edit)?;
425 section.seal().map_err(failed)?;
426 let applied = Zeroizing::new(section.image());
427 drop(section);
428 let applied_store = crate::Store::parse(&applied).map_err(failed)?;
429 let applied_index = RevisionIndex::parse(&applied_store).map_err(failed)?;
430 // The twin's scaffold spaces are not the section's.
431 let edited = self
432 .index
433 .spaces
434 .keys()
435 .map(|space| Ok((*space, applied_index.resolve_active(*space)?)))
436 .collect::<std::result::Result<Vec<_>, crate::Error>>()
437 .map_err(failed)?;
438 let edited: Vec<_> = edited.iter().map(|(space, after)| (*space, after)).collect();
439 let payloads = crate::write::declared_payloads(&applied_store)
440 .into_iter()
441 .map(|guid| Ok((guid, applied_store.file_data(guid)?)))
442 .collect::<std::result::Result<Vec<_>, crate::Error>>()
443 .map_err(failed)?;
444 let transaction = crate::write::squash(self.index, &edited, &payloads, Some(self))
445 .map_err(failed)?
446 .ok_or(OpError::Unsupported("The edit changes nothing"))?;
447 let written = crate::write::applied(self.index.store.data, Some(&transaction))
448 .map_err(failed)?;
449 let store = crate::Store::parse(&written).map_err(failed)?;
450 let index = RevisionIndex::parse(&store).map_err(failed)?;
451 UnlockedSection::unlock(&index, Limits::default(), |metadata, _| {
452 self.keys.get(metadata).cloned().ok_or(Error::PasswordMismatch)
453 })
454 .map_err(|error| match error {
455 Error::Invalid(error) => failed(error),
456 _ => failed(crate::Error {
457 offset: 0,
458 message: "The written section does not unlock under its key",
459 }),
460 })?;
461 Ok(transaction)
427 }462 }
428 let edited: Vec<_> = edited.iter().map(|(sid, after)| (*sid, after)).collect();
429 let payloads = crate::page::write::declared_payloads(&applied_store)
430 .into_iter()
431 .map(|guid| Ok((guid, applied_store.file_data(guid)?)))
432 .collect::<std::result::Result<Vec<_>, crate::Error>>()?;
433 let written = crate::page::write::squash(
434 &index,
435 &edited,
436 &payloads,
437 &BTreeMap::new(),
438 Some(&unlocked),
439 )?;
440 let store = crate::Store::parse(&written)?;
441 UnlockedSection::open(&RevisionIndex::parse(&store)?, password, Limits::default())?;
442 Ok(written)
443}463}
444464
445impl UnlockedSection<'_> {465impl UnlockedSection<'_> {
446 /// The section key; `write_page` admits sections with exactly one.466 /// The section key; `apply` admits sections with exactly one.
447 fn key(&self) -> &crypto::Key {467 fn key(&self) -> &crypto::Key {
448 self.keys.values().next().expect("one section key")468 self.keys.values().next().expect("one section key")
449 }469 }
crates/onestore/src/section.rs+175-130
...@@ -226,15 +226,108 @@ impl<'a> Section<'a> {...@@ -226,15 +226,108 @@ impl<'a> Section<'a> {
226 }226 }
227 }227 }
228228
229 /// The conflict pages of each page that has them, in section order, each page's in the
230 /// order OneNote 2010 lists them under it: the last stored first. Read from the pages'
231 /// manifests and the conflict pages' metadata alone.
232 pub fn conflicts(&mut self) -> Result<Vec<(ExGuid, Vec<crate::ConflictPage>)>> {
233 let listed: Vec<ExGuid> = self.pages()?.into_iter().map(|(space, ..)| space).collect();
234 let element = |object: &crate::Object<'a>| {
235 crate::document::Element::parse_with(object, FileType::Section, &mut |_| {
236 Err(Error {
237 offset: 0,
238 message: "Page metadata holds no payload",
239 })
240 })
241 };
242 let mut conflicts = Vec::new();
243 for page in listed {
244 let revision = self.revision(page)?;
245 let Some(manifest) = revision
246 .roots
247 .get(&1)
248 .and_then(|id| revision.objects.get(id))
249 else {
250 continue;
251 };
252 let mut pages = Vec::new();
253 for space in element(manifest)?.spaces.into_iter().rev() {
254 let Some(metadata) = self.revision(space).ok().and_then(|revision| {
255 revision
256 .roots
257 .get(&2)
258 .and_then(|id| revision.objects.get(id))
259 }) else {
260 continue;
261 };
262 let metadata = element(metadata)?;
263 let crate::document::Kind::ConflictMetadata { title, author, .. } = metadata.kind
264 else {
265 continue;
266 };
267 let created = metadata.extra[0]
268 .iter()
269 .find_map(|field| match field.value {
270 crate::document::FieldValue::Bytes(bytes) if field.id == 0x18001c65 => {
271 bytes.try_into().ok().map(u64::from_le_bytes)
272 }
273 _ => None,
274 });
275 let mut objects = Vec::new();
276 for (id, object) in &self.revision(space)?.objects {
277 let ObjectData::Properties(bytes) = object.data else {
278 continue;
279 };
280 if crate::PropertySets::parse(bytes)?.sets[0]
281 .iter()
282 .any(|property| property.id == 0x88001d96)
283 {
284 objects.push(*id);
285 }
286 }
287 pages.push(crate::ConflictPage {
288 space,
289 title: title.unwrap_or_default(),
290 user: author.unwrap_or_default(),
291 created,
292 objects,
293 });
294 }
295 if !pages.is_empty() {
296 conflicts.push((page, pages));
297 }
298 }
299 Ok(conflicts)
300 }
301
302 /// The page series holding each listed page. Moving a page gives it a series of its
303 /// own, so a page in another series than before was moved.
304 pub fn series(&mut self) -> Result<BTreeMap<ExGuid, ExGuid>> {
305 let view = &self.active(self.root)?.view;
306 let section = view.roots.get(&1).and_then(|id| view.nodes.get(id)).ok_or(Error {
307 offset: 0,
308 message: "Section root is unavailable",
309 })?;
310 Ok(section
311 .children
312 .iter()
313 .filter_map(|series| Some((*series, view.nodes.get(series)?)))
314 .flat_map(|(series, node)| node.spaces.iter().map(move |page| (*page, series)))
315 .collect())
316 }
317
229 /// Page spaces in section order with the title and outline level (1 at the top) the318 /// Page spaces in section order with the title and outline level (1 at the top) the
230 /// page list shows, read from each page's metadata alone.319 /// page list shows, read from each page's metadata alone.
231 pub fn pages(&mut self) -> Result<Vec<(ExGuid, String, u32)>> {320 pub fn pages(&mut self) -> Result<Vec<(ExGuid, String, u32)>> {
232 let root = self.root;321 let root = self.root;
233 let view = &self.active(root)?.view;322 let view = &self.active(root)?.view;
234 let section = view.roots.get(&1).and_then(|id| view.nodes.get(id)).ok_or(Error {323 let section = view
235 offset: 0,324 .roots
236 message: "Section root is unavailable",325 .get(&1)
237 })?;326 .and_then(|id| view.nodes.get(id))
327 .ok_or(Error {
328 offset: 0,
329 message: "Section root is unavailable",
330 })?;
238 let spaces: Vec<ExGuid> = section331 let spaces: Vec<ExGuid> = section
239 .children332 .children
240 .iter()333 .iter()
...@@ -253,7 +346,10 @@ impl<'a> Section<'a> {...@@ -253,7 +346,10 @@ impl<'a> Section<'a> {
253 for space in spaces {346 for space in spaces {
254 let revision = self.revision(space)?;347 let revision = self.revision(space)?;
255 let (mut title, mut level) = (None, 1);348 let (mut title, mut level) = (None, 1);
256 if let Some(metadata) = revision.roots.get(&2).and_then(|id| revision.objects.get(id))349 if let Some(metadata) = revision
350 .roots
351 .get(&2)
352 .and_then(|id| revision.objects.get(id))
257 && let crate::document::Kind::Metadata {353 && let crate::document::Kind::Metadata {
258 title: stored,354 title: stored,
259 level: stored_level,355 level: stored_level,
...@@ -263,7 +359,10 @@ impl<'a> Section<'a> {...@@ -263,7 +359,10 @@ impl<'a> Section<'a> {
263 level = stored_level.unwrap_or(1);359 level = stored_level.unwrap_or(1);
264 }360 }
265 if title.is_none() {361 if title.is_none() {
266 let manifest = revision.roots.get(&1).and_then(|id| revision.objects.get(id));362 let manifest = revision
363 .roots
364 .get(&1)
365 .and_then(|id| revision.objects.get(id));
267 if let Some(manifest) = manifest366 if let Some(manifest) = manifest
268 && let Some(page) = element(manifest)?367 && let Some(page) = element(manifest)?
269 .content368 .content
...@@ -695,6 +794,17 @@ mod tests {...@@ -695,6 +794,17 @@ mod tests {
695 result794 result
696 }795 }
697796
797 /// The image the image writer leaves: the `edited` revisions squashed onto the source
798 /// `index` parsed.
799 fn squashed(
800 index: &RevisionIndex<'_>,
801 edited: &[(ExGuid, &ResolvedRevision<'_>)],
802 payloads: &[([u8; 16], &[u8])],
803 ) -> Vec<u8> {
804 let transaction = crate::write::squash(index, edited, payloads, None).unwrap();
805 crate::write::applied(index.store.data, transaction.as_ref()).unwrap()
806 }
807
698 /// The page spaces of an image, fullest first.808 /// The page spaces of an image, fullest first.
699 fn page_spaces(image: &[u8]) -> Vec<ExGuid> {809 fn page_spaces(image: &[u8]) -> Vec<ExGuid> {
700 let store = Store::parse(image).unwrap();810 let store = Store::parse(image).unwrap();
...@@ -729,65 +839,62 @@ mod tests {...@@ -729,65 +839,62 @@ mod tests {
729 GUIDS.set(Some(1 << 62 | seed << 40));839 GUIDS.set(Some(1 << 62 | seed << 40));
730 let (mut sealed, mut checkpoints) = (0, 0);840 let (mut sealed, mut checkpoints) = (0, 0);
731 for batch in 0..batches {841 for batch in 0..batches {
732 let space = spaces[batch % spaces.len().min(pages)];842 // The writers stamp modification times; a fixed clock keeps the batches alike.
733 let store = Store::parse(&image).unwrap();843 let at = 134_000_000_000_000_000 + batch as u64 * 10_000_000;
734 let index = RevisionIndex::parse(&store).unwrap();844 crate::create::at(at, || {
735 let bump = Bump::new();845 let space = spaces[batch % spaces.len().min(pages)];
736 let mut legacy = ActivePage::parse(&index, space).unwrap();
737 for _ in 0..1 + (writes.0 as usize >> 40) % largest {
738 let Ok(changes) = writes.next(section.active(space).unwrap()) else {
739 continue;
740 };
741 let stored = legacy.write(&bump, &[], changes.clone()).unwrap();
742 assert_eq!(
743 section.apply_changes(space, &[], changes).unwrap(),
744 stored,
745 "batch {batch}"
746 );
747 }
748 let guids = (batch as u64 + 1) << 32;
749 let expected = seeded(guids, || {
750 crate::page::write::squash(
751 &index,
752 &[(space, &legacy.live.revision)],
753 &legacy.payloads,
754 &BTreeMap::new(),
755 None,
756 )
757 .unwrap()
758 });
759 let before = section.stamp().clone();
760 let chains = depths(&section);
761 let transaction = seeded(guids, || section.seal().unwrap());
762 checkpoints += depths(&section)
763 .iter()
764 .filter(|(id, depth)| chains.get(id).is_some_and(|before| before > depth))
765 .count();
766 let mut written = image.clone();
767 if let Some(transaction) = &transaction {
768 assert_eq!(transaction.base, before);
769 transaction.apply(&mut written).unwrap();
770 sealed += 1;
771 }
772 assert!(written == expected, "batch {batch}");
773 assert_eq!(Stamp::of(&written).unwrap(), *section.stamp());
774 drop(legacy);
775 drop(index);
776 drop(store);
777 image = expected;
778 if batch % 25 == 0 || batch + 1 == batches {
779 assert!(section.image() == image);
780 let store = Store::parse(&image).unwrap();846 let store = Store::parse(&image).unwrap();
781 let index = RevisionIndex::parse(&store).unwrap();847 let index = RevisionIndex::parse(&store).unwrap();
782 index.validate_current().unwrap();848 let bump = Bump::new();
783 let document = Document::parse(&index).unwrap();849 let mut legacy = ActivePage::parse(&index, space).unwrap();
784 for space in &spaces {850 for _ in 0..1 + (writes.0 as usize >> 40) % largest {
851 let Ok(changes) = writes.next(section.active(space).unwrap()) else {
852 continue;
853 };
854 let stored = legacy.write(&bump, &[], changes.clone()).unwrap();
785 assert_eq!(855 assert_eq!(
786 section.page(*space).unwrap(),856 section.apply_changes(space, &[], changes).unwrap(),
787 Page::from_space(&document, *space).unwrap()857 stored,
858 "batch {batch}"
788 );859 );
789 }860 }
790 }861 let guids = (batch as u64 + 1) << 32;
862 let expected = seeded(guids, || {
863 squashed(&index, &[(space, &legacy.live.revision)], &legacy.payloads)
864 });
865 let before = section.stamp().clone();
866 let chains = depths(&section);
867 let transaction = seeded(guids, || section.seal().unwrap());
868 checkpoints += depths(&section)
869 .iter()
870 .filter(|(id, depth)| chains.get(id).is_some_and(|before| before > depth))
871 .count();
872 let mut written = image.clone();
873 if let Some(transaction) = &transaction {
874 assert_eq!(transaction.base, before);
875 transaction.apply(&mut written).unwrap();
876 sealed += 1;
877 }
878 assert!(written == expected, "batch {batch}");
879 assert_eq!(Stamp::of(&written).unwrap(), *section.stamp());
880 drop(legacy);
881 drop(index);
882 drop(store);
883 image = expected;
884 if batch % 25 == 0 || batch + 1 == batches {
885 assert!(section.image() == image);
886 let store = Store::parse(&image).unwrap();
887 let index = RevisionIndex::parse(&store).unwrap();
888 index.validate_current().unwrap();
889 let document = Document::parse(&index).unwrap();
890 for space in &spaces {
891 assert_eq!(
892 section.page(*space).unwrap(),
893 Page::from_space(&document, *space).unwrap()
894 );
895 }
896 }
897 });
791 }898 }
792 GUIDS.set(None);899 GUIDS.set(None);
793 assert!(sealed * 3 > batches, "{sealed} of {batches} batches sealed");900 assert!(sealed * 3 > batches, "{sealed} of {batches} batches sealed");
...@@ -932,14 +1039,7 @@ mod tests {...@@ -932,14 +1039,7 @@ mod tests {
932 legacy.write(&bump, &[], changes).unwrap();1039 legacy.write(&bump, &[], changes).unwrap();
933 let expected = seeded(guids, || {1040 let expected = seeded(guids, || {
934 let revision = &legacy.live.revision;1041 let revision = &legacy.live.revision;
935 crate::page::write::squash(1042 squashed(&index, &[(space, revision)], &[])
936 &index,
937 &[(space, revision)],
938 &[],
939 &BTreeMap::new(),
940 None,
941 )
942 .unwrap()
943 });1043 });
944 drop(legacy);1044 drop(legacy);
945 drop(index);1045 drop(index);
...@@ -971,14 +1071,7 @@ mod tests {...@@ -971,14 +1071,7 @@ mod tests {
971 let legacy = ActivePage::parse(&index, space).unwrap();1071 let legacy = ActivePage::parse(&index, space).unwrap();
972 let expected = seeded(seed, || {1072 let expected = seeded(seed, || {
973 let revision = &legacy.live.revision;1073 let revision = &legacy.live.revision;
974 crate::page::write::squash(1074 squashed(&index, &[(space, revision)], &[payload])
975 &index,
976 &[(space, revision)],
977 &[payload],
978 &BTreeMap::new(),
979 None,
980 )
981 .unwrap()
982 });1075 });
983 let transaction = seeded(seed, || section.seal().unwrap());1076 let transaction = seeded(seed, || section.seal().unwrap());
984 assert_eq!(transaction.is_some(), expected != image);1077 assert_eq!(transaction.is_some(), expected != image);
...@@ -1036,8 +1129,12 @@ mod tests {...@@ -1036,8 +1129,12 @@ mod tests {
1036 && page.parents.contains_key(id)1129 && page.parents.contains_key(id)
1037 })1130 })
1038 .find_map(|(outline, _)| {1131 .find_map(|(outline, _)| {
1132 let paragraph = crate::page::text::new_id().ok()?;
1133 let text = crate::page::text::new_id().ok()?;
1039 Insertion::paragraph(*outline, None, "Tampered text", "Author")1134 Insertion::paragraph(*outline, None, "Tampered text", "Author")
1040 .and_then(|insertion| insertion.changes(page))1135 .and_then(|insertion| {
1136 insertion.changes_as(page, paragraph, paragraph, text)
1137 })
1041 .ok()1138 .ok()
1042 })1139 })
1043 .unwrap();1140 .unwrap();
...@@ -1127,8 +1224,7 @@ mod probe {...@@ -1127,8 +1224,7 @@ mod probe {
1127 (space, texts[texts.len() / 2], texts.len())1224 (space, texts[texts.len() / 2], texts.len())
1128 }1225 }
11291226
1130 /// One-character edits of a large page through the image writer and through a1227 /// One-character edits of a large page through a `Section`: `SECTION_PROBE=path cargo test --release -p onestore --lib probe -- --ignored --nocapture`.
1131 /// `Section`: `SECTION_PROBE=path cargo test --release -p onestore --lib probe -- --ignored --nocapture`.
1132 #[test]1228 #[test]
1133 #[ignore]1229 #[ignore]
1134 fn keystrokes() {1230 fn keystrokes() {
...@@ -1140,57 +1236,6 @@ mod probe {...@@ -1140,57 +1236,6 @@ mod probe {
1140 image.len()1236 image.len()
1141 );1237 );
11421238
1143 let mut samples = Vec::new();
1144 let mut written = 0;
1145 for _ in 0..9 {
1146 let start = Instant::now();
1147 let edit = crate::PreparedEdit::text(&image, space, text, 0..0, "x").unwrap();
1148 samples.push(start.elapsed());
1149 let transaction = edit.transaction();
1150 written = transaction.append.len()
1151 + transaction
1152 .patches
1153 .iter()
1154 .map(|(_, bytes)| bytes.len())
1155 .sum::<usize>();
1156 }
1157 println!(
1158 "image writer, one keystroke: {:?}, {written} bytes written",
1159 median(samples)
1160 );
1161
1162 let before = {
1163 let store = Store::parse(&image).unwrap();
1164 let index = RevisionIndex::parse(&store).unwrap();
1165 Page::from_space(&crate::document::Document::parse(&index).unwrap(), space).unwrap()
1166 };
1167 let mut after = before.clone();
1168 let edited = after
1169 .objects
1170 .iter_mut()
1171 .filter_map(|object| match object {
1172 crate::page::PageObject::Outline(outline) => Some(outline),
1173 _ => None,
1174 })
1175 .flat_map(|outline| &mut outline.paragraphs)
1176 .find_map(|paragraph| paragraph.text_mut().filter(|object| object.id == text))
1177 .unwrap();
1178 let format = edited.text.format_at(0).unwrap().clone();
1179 edited
1180 .text
1181 .apply(crate::page::text::Edit {
1182 range: 0..0,
1183 replacement: crate::page::Paragraph::new("x".into(), format),
1184 })
1185 .unwrap();
1186 let mut samples = Vec::new();
1187 for _ in 0..9 {
1188 let start = Instant::now();
1189 crate::PreparedEdit::page(&image, space, &after, "Probe").unwrap();
1190 samples.push(start.elapsed());
1191 }
1192 println!("page-model writer, one keystroke: {:?}", median(samples));
1193
1194 let arena = Arena::default();1239 let arena = Arena::default();
1195 let start = Instant::now();1240 let start = Instant::now();
1196 let mut section = Section::open(&arena, image.clone()).unwrap();1241 let mut section = Section::open(&arena, image.clone()).unwrap();
crates/onestore/src/toc.rs+32-37
...@@ -1,14 +1,15 @@...@@ -1,14 +1,15 @@
1//! Edits to a notebook's table of contents: the documented `jcidPersistablePropertyContainerForTOC`1//! Edits to a notebook's table of contents: the documented `jcidPersistablePropertyContainerForTOC`
2//! root lists section and section-group entries, each carrying a file identity, an ordering2//! root carries the notebook's colour and lists section and section-group entries, each
3//! number, a filename and (for sections) a colour.3//! carrying a file identity, an ordering number and a filename. A section's colour lives in
4//! its own metadata (`op::SectionOp::Color`); its entry keeps 0xffffffff, as OneNote writes.
45
5use crate::{6use crate::{
6 Error, ExGuid, PropertySets, Store, Value,7 Error, ExGuid, PropertySets, Store, Transaction, Value,
7 document::{Document, Kind},8 document::{Document, Kind},
8 revisions::RevisionIndex,9 revisions::RevisionIndex,
9 write::{PropertyObject, fresh_guid, write_revision},10 write::{PropertyObject, RevisionEdit, applied, build_on, check, fresh_guid},
10};11};
11use std::collections::BTreeMap;12use std::collections::{BTreeMap, BTreeSet};
1213
13fn invalid(message: &'static str) -> Error {14fn invalid(message: &'static str) -> Error {
14 Error { offset: 0, message }15 Error { offset: 0, message }
...@@ -27,11 +28,8 @@ pub enum TocEdit {...@@ -27,11 +28,8 @@ pub enum TocEdit {
27 identity: [u8; 16],28 identity: [u8; 16],
28 filename: String,29 filename: String,
29 },30 },
30 /// Section colour as COLORREF; `None` restores OneNote's "undefined" 0xffffffff.31 /// The notebook's colour as COLORREF.
31 Color {32 Color(u32),
32 identity: [u8; 16],
33 color: Option<u32>,
34 },
35 /// Every entry, in the wanted order; entries left out keep their relative order after these.33 /// Every entry, in the wanted order; entries left out keep their relative order after these.
36 Order(Vec<[u8; 16]>),34 Order(Vec<[u8; 16]>),
37 Remove {35 Remove {
...@@ -43,15 +41,19 @@ fn component(name: &str) -> bool {...@@ -43,15 +41,19 @@ fn component(name: &str) -> bool {
43 !name.is_empty() && !name.contains(['/', '\\', '\0']) && name != "." && name != ".."41 !name.is_empty() && !name.contains(['/', '\\', '\0']) && name != "." && name != ".."
44}42}
4543
46/// The table of contents with `edits` applied in order.44/// Applies table-of-contents edits (sections and section groups: add, rename, order,
47type PropertyChange = (u32, Option<Vec<u8>>);45/// remove; the notebook's colour) in order as one revision of a `.onetoc2` file; none when they change
4846/// nothing.
49pub(crate) fn edit_table_of_contents(source: &[u8], edits: &[TocEdit]) -> Result<Vec<u8>, Error> {47pub fn edit_table_of_contents(
48 source: &[u8],
49 edits: &[TocEdit],
50) -> Result<Option<Transaction>, Error> {
50 let store = Store::parse(source)?;51 let store = Store::parse(source)?;
51 if store.header.file_type != crate::FileType::TableOfContents {52 if store.header.file_type != crate::FileType::TableOfContents {
52 return Err(invalid("Choose a table-of-contents file"));53 return Err(invalid("Choose a table-of-contents file"));
53 }54 }
54 let index = RevisionIndex::parse(&store)?;55 let index = RevisionIndex::parse(&store)?;
56 index.validate_current()?;
55 let document = Document::parse(&index)?;57 let document = Document::parse(&index)?;
56 let space = document.root;58 let space = document.root;
57 let revision = document.active(space)?;59 let revision = document.active(space)?;
...@@ -76,8 +78,8 @@ pub(crate) fn edit_table_of_contents(source: &[u8], edits: &[TocEdit]) -> Result...@@ -76,8 +78,8 @@ pub(crate) fn edit_table_of_contents(source: &[u8], edits: &[TocEdit]) -> Result
76 listed.push((*id, *identity, filename.clone().unwrap_or_default()));78 listed.push((*id, *identity, filename.clone().unwrap_or_default()));
77 }79 }
78 let mut created: BTreeMap<ExGuid, PropertyObject> = BTreeMap::new();80 let mut created: BTreeMap<ExGuid, PropertyObject> = BTreeMap::new();
79 // Property values to set (or remove, `None`) on each existing entry.81 let mut renamed = BTreeSet::new();
80 let mut changes: BTreeMap<ExGuid, Vec<PropertyChange>> = BTreeMap::new();82 let mut color = None;
81 for edit in edits {83 for edit in edits {
82 let position = |identity: &[u8; 16]| {84 let position = |identity: &[u8; 16]| {
83 listed85 listed
...@@ -143,18 +145,9 @@ pub(crate) fn edit_table_of_contents(source: &[u8], edits: &[TocEdit]) -> Result...@@ -143,18 +145,9 @@ pub(crate) fn edit_table_of_contents(source: &[u8], edits: &[TocEdit]) -> Result
143 }145 }
144 let id = listed[at].0;146 let id = listed[at].0;
145 listed[at].2 = filename.clone();147 listed[at].2 = filename.clone();
146 changes148 renamed.insert(id);
147 .entry(id)
148 .or_default()
149 .push((0x1c001d6b, Some(crate::create::string(filename))));
150 }
151 TocEdit::Color { identity, color } => {
152 let id = listed[position(identity)?].0;
153 changes.entry(id).or_default().push((
154 0x14001cbe,
155 Some(color.unwrap_or(0xffff_ffff).to_le_bytes().to_vec()),
156 ));
157 }149 }
150 TocEdit::Color(value) => color = Some(*value),
158 TocEdit::Order(wanted) => {151 TocEdit::Order(wanted) => {
159 let mut ordered: Vec<(ExGuid, [u8; 16], String)> = Vec::new();152 let mut ordered: Vec<(ExGuid, [u8; 16], String)> = Vec::new();
160 for identity in wanted {153 for identity in wanted {
...@@ -175,17 +168,17 @@ pub(crate) fn edit_table_of_contents(source: &[u8], edits: &[TocEdit]) -> Result...@@ -175,17 +168,17 @@ pub(crate) fn edit_table_of_contents(source: &[u8], edits: &[TocEdit]) -> Result
175 TocEdit::Remove { identity } => {168 TocEdit::Remove { identity } => {
176 let at = position(identity)?;169 let at = position(identity)?;
177 let (id, _, _) = listed.remove(at);170 let (id, _, _) = listed.remove(at);
178 changes.remove(&id);
179 created.remove(&id);171 created.remove(&id);
180 }172 }
181 }173 }
182 }174 }
183 let listed = listed;175 let listed = listed;
184 write_revision(source, space, |raw| {176 let transaction = build_on(&index, &[], None, |index| {
177 let raw = index.resolve_active(space)?;
185 let mut changed = BTreeMap::new();178 let mut changed = BTreeMap::new();
186 let mut root_object = PropertyObject::from_object(&raw.objects[&root])?;179 let mut root_object = PropertyObject::from_object(&raw.objects[&root])?;
187 let mut references = Vec::new();180 let mut references = Vec::new();
188 for (order, (id, _, _)) in listed.iter().enumerate() {181 for (order, (id, _, filename)) in listed.iter().enumerate() {
189 let mut object = match created.remove(id) {182 let mut object = match created.remove(id) {
190 Some(object) => object,183 Some(object) => object,
191 None => PropertyObject::from_object(&raw.objects[id])?,184 None => PropertyObject::from_object(&raw.objects[id])?,
...@@ -201,11 +194,8 @@ pub(crate) fn edit_table_of_contents(source: &[u8], edits: &[TocEdit]) -> Result...@@ -201,11 +194,8 @@ pub(crate) fn edit_table_of_contents(source: &[u8], edits: &[TocEdit]) -> Result
201 if stored_order != Some(order as u32 + 1) {194 if stored_order != Some(order as u32 + 1) {
202 updates.push((0x14001cb9, (order as u32 + 1).to_le_bytes().to_vec()));195 updates.push((0x14001cb9, (order as u32 + 1).to_le_bytes().to_vec()));
203 }196 }
204 for (property, value) in changes.get(id).into_iter().flatten() {197 if renamed.contains(id) {
205 match value {198 updates.push((0x1c001d6b, crate::create::string(filename)));
206 Some(bytes) => updates.push((*property, bytes.clone())),
207 None => object.remove(&[*property])?,
208 }
209 }199 }
210 let updates: Vec<(u32, &[u8])> = updates200 let updates: Vec<(u32, &[u8])> = updates
211 .iter()201 .iter()
...@@ -220,7 +210,12 @@ pub(crate) fn edit_table_of_contents(source: &[u8], edits: &[TocEdit]) -> Result...@@ -220,7 +210,12 @@ pub(crate) fn edit_table_of_contents(source: &[u8], edits: &[TocEdit]) -> Result
220 }210 }
221 }211 }
222 root_object.set(&[(0x24001cf6, &references)])?;212 root_object.set(&[(0x24001cf6, &references)])?;
213 if let Some(color) = color {
214 root_object.set(&[(0x14001cbe, &color.to_le_bytes())])?;
215 }
223 changed.insert(root, root_object);216 changed.insert(root, root_object);
224 Ok(changed)217 Ok(BTreeMap::from([(space, RevisionEdit::Update(changed))]))
225 })218 })?;
219 check(&applied(source, transaction.as_ref())?, true)?;
220 Ok(transaction)
226}221}
crates/onestore/src/tree.rs+66-167
...@@ -1,12 +1,11 @@...@@ -1,12 +1,11 @@
1use crate::{1use crate::{
2 Error, ExGuid,2 Error, ExGuid,
3 active::{ActivePage, Changes},3 active::{ActivePage, Changes},
4 create::{current_timestamps, properties, string},4 create::{current_timestamps, properties},
5 document::{Kind, Revision},5 document::{Kind, Revision},
6 edit::{editable_parents, update_title},6 edit::{editable_parents, update_title},
7 write::{PropertyObject, fresh_guid},7 write::{PropertyObject, fresh_guid},
8};8};
9use serde::{Deserialize, Serialize};
10use std::{9use std::{
11 collections::{BTreeMap, BTreeSet},10 collections::{BTreeMap, BTreeSet},
12 sync::Arc,11 sync::Arc,
...@@ -16,8 +15,7 @@ fn invalid(message: &'static str) -> Error {...@@ -16,8 +15,7 @@ fn invalid(message: &'static str) -> Error {
16 Error { offset: 0, message }15 Error { offset: 0, message }
17}16}
1817
19#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]18#[derive(Debug, Clone, PartialEq)]
20#[serde(deny_unknown_fields)]
21enum Placement {19enum Placement {
22 Delete,20 Delete,
23 Move {21 Move {
...@@ -27,40 +25,25 @@ enum Placement {...@@ -27,40 +25,25 @@ enum Placement {
27}25}
2826
29/// An atomic subtree move or deletion on one active page.27/// An atomic subtree move or deletion on one active page.
30/// Retain the intent across retries so an emptied cell's replacement keeps its identity.28#[derive(Debug, Clone, PartialEq)]
31#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]29pub(crate) struct TreeEdit {
32#[serde(deny_unknown_fields)]
33pub struct TreeEdit {
34 guid: [u8; 16],30 guid: [u8; 16],
35 object: ExGuid,31 object: ExGuid,
36 placement: Placement,32 placement: Placement,
37 author: String,33 author: String,
38 created: u32,
39}34}
4035
41impl TreeEdit {36impl TreeEdit {
42 pub fn object(&self) -> ExGuid {
43 self.object
44 }
45
46 /// Move destination and direct sibling anchor; deletion has no destination.
47 pub fn destination(&self) -> Option<(ExGuid, Option<ExGuid>)> {
48 match self.placement {
49 Placement::Delete => None,
50 Placement::Move { parent, before } => Some((parent, before)),
51 }
52 }
53
54 /// Removes a paragraph or ordinary outline and its descendants from the active tree.37 /// Removes a paragraph or ordinary outline and its descendants from the active tree.
55 /// Historical objects remain available; an emptied table cell receives an empty paragraph.38 /// Historical objects remain available.
56 pub fn delete(object: ExGuid, author: &str) -> Result<Self, Error> {39 pub(crate) fn delete(object: ExGuid, author: &str) -> Result<Self, Error> {
57 Self::new(object, Placement::Delete, author)40 Self::new(object, Placement::Delete, author)
58 }41 }
5942
60 /// Moves before a direct child, or appends when `before` is None.43 /// Moves before a direct child, or appends when `before` is None.
61 /// Paragraph destinations are outlines, groups, paragraphs or cells on the same page.44 /// Paragraph destinations are outlines, groups, paragraphs or cells on the same page.
62 /// Outlines remain direct page children. Content and explicit list formatting are preserved.45 /// Outlines remain direct page children. Content and explicit list formatting are preserved.
63 pub fn move_to(46 pub(crate) fn move_to(
64 object: ExGuid,47 object: ExGuid,
65 parent: ExGuid,48 parent: ExGuid,
66 before: Option<ExGuid>,49 before: Option<ExGuid>,
...@@ -75,7 +58,6 @@ impl TreeEdit {...@@ -75,7 +58,6 @@ impl TreeEdit {
75 object,58 object,
76 placement,59 placement,
77 author: author.to_owned(),60 author: author.to_owned(),
78 created: current_timestamps()?.0,
79 };61 };
80 intent.validate()?;62 intent.validate()?;
81 Ok(intent)63 Ok(intent)
...@@ -95,11 +77,6 @@ impl TreeEdit {...@@ -95,11 +77,6 @@ impl TreeEdit {
95 Ok(())77 Ok(())
96 }78 }
9779
98 pub(crate) fn apply(&self, source: &[u8], space: ExGuid) -> Result<Vec<u8>, Error> {
99 self.validate()?;
100 crate::active::write(source, space, |active| self.changes(active))
101 }
102
103 pub(crate) fn changes(&self, active: &ActivePage<'_>) -> Result<Changes, Error> {80 pub(crate) fn changes(&self, active: &ActivePage<'_>) -> Result<Changes, Error> {
104 let pages = &active.pages;81 let pages = &active.pages;
105 let [page] = pages.as_slice() else {82 let [page] = pages.as_slice() else {
...@@ -164,7 +141,6 @@ impl TreeEdit {...@@ -164,7 +141,6 @@ impl TreeEdit {
164 .retain(|id| *id != self.object);141 .retain(|id| *id != self.object);
165 }142 }
166143
167 let mut empty_cell = None;
168 let mut at = source_parent;144 let mut at = source_parent;
169 loop {145 loop {
170 let node = &view.nodes[&at];146 let node = &view.nodes[&at];
...@@ -174,8 +150,8 @@ impl TreeEdit {...@@ -174,8 +150,8 @@ impl TreeEdit {
174 break;150 break;
175 }151 }
176 if node.children.is_empty() {152 if node.children.is_empty() {
153 // `Section::apply` refuses an edit that ends with the cell still empty.
177 if matches!(node.kind, Kind::Cell { .. }) {154 if matches!(node.kind, Kind::Cell { .. }) {
178 empty_cell = Some(at);
179 break;155 break;
180 }156 }
181 if !matches!(node.kind, Kind::Outline { .. } | Kind::OutlineGroup) {157 if !matches!(node.kind, Kind::Outline { .. } | Kind::OutlineGroup) {
...@@ -278,53 +254,14 @@ impl TreeEdit {...@@ -278,53 +254,14 @@ impl TreeEdit {
278 guid: self.guid,254 guid: self.guid,
279 n: 3,255 n: 3,
280 };256 };
281 if let Some(cell) = empty_cell {
282 let paragraph = ExGuid {
283 guid: self.guid,
284 n: 1,
285 };
286 let text = ExGuid {
287 guid: self.guid,
288 n: 2,
289 };
290 view.nodes.get_mut(&cell).unwrap().children.push(paragraph);
291 for (id, jcid, values) in [
292 (
293 paragraph,
294 0x6000d,
295 vec![
296 (0x24001c1f, 2_u32.to_le_bytes().to_vec()),
297 (0x0c001c03, vec![1]),
298 (0x14001d09, self.created.to_le_bytes().to_vec()),
299 (0x14001d7a, modified.to_vec()),
300 (0x20001d78, 3_u32.to_le_bytes().to_vec()),
301 (0x20001d79, 3_u32.to_le_bytes().to_vec()),
302 ],
303 ),
304 (
305 text,
306 0x6000e,
307 vec![(0x1c001c22, Vec::new()), (0x14001d7a, modified.to_vec())],
308 ),
309 ] {
310 changed.insert(
311 id,
312 PropertyObject {
313 jcid,
314 bytes: properties(&values)?,
315 global_ids: Arc::new(BTreeMap::from([(0, self.guid)])),
316 },
317 );
318 }
319 }
320 let moved_paragraph = matches!(self.placement, Placement::Move { .. })257 let moved_paragraph = matches!(self.placement, Placement::Move { .. })
321 && matches!(view.nodes[&self.object].kind, Kind::Paragraph { .. });258 && matches!(view.nodes[&self.object].kind, Kind::Paragraph { .. });
322 if empty_cell.is_some() || moved_paragraph {259 if moved_paragraph {
323 changed.insert(260 changed.insert(
324 author_id,261 author_id,
325 PropertyObject {262 PropertyObject {
326 jcid: 0x120001,263 jcid: 0x120001,
327 bytes: properties(&[(0x1c001d75, string(&self.author))])?,264 bytes: properties(&crate::create::author_properties(&self.author))?,
328 global_ids: Arc::new(BTreeMap::from([(0, self.guid)])),265 global_ids: Arc::new(BTreeMap::from([(0, self.guid)])),
329 },266 },
330 );267 );
...@@ -413,47 +350,15 @@ fn checked_path(...@@ -413,47 +350,15 @@ fn checked_path(
413#[cfg(test)]350#[cfg(test)]
414mod tests {351mod tests {
415 use super::*;352 use super::*;
416 use crate::write::write_revision;353 use crate::{
417 use crate::{Insertion, PreparedEdit, RevisionIndex, Store, document::Document};354 RevisionIndex, Store,
418355 document::{Document, Format},
419 #[test]356 op::{
420 fn a_retained_move_intent_can_reuse_its_immutable_author_after_another_move() {357 PageOp,
421 let source = crate::create_section("move.one", "First", "Author").unwrap();358 tests::{edited, text_paragraph},
422 let store = Store::parse(&source).unwrap();359 },
423 let index = RevisionIndex::parse(&store).unwrap();360 write::write_revision,
424 let document = Document::parse(&index).unwrap();361 };
425 let (sid, page) = document.pages().unwrap()[0];
426 let view = document.active(sid).unwrap();
427 let outline = *view.nodes[&page]
428 .children
429 .iter()
430 .find(|id| matches!(view.nodes[id].kind, Kind::Outline { .. }))
431 .unwrap();
432 let first = view.nodes[&outline].children[0];
433 let second = Insertion::paragraph(outline, None, "Second", "Author").unwrap();
434 let source = PreparedEdit::insert(&source, sid, &second)
435 .unwrap()
436 .as_bytes()
437 .to_vec();
438 let retained = TreeEdit::move_to(first, outline, None, "Tree author").unwrap();
439 let moved = PreparedEdit::tree(&source, sid, &retained).unwrap();
440 let reversed =
441 TreeEdit::move_to(first, outline, Some(second.object()), "Tree author").unwrap();
442 let reversed = PreparedEdit::tree(moved.as_bytes(), sid, &reversed).unwrap();
443 let repeated = PreparedEdit::tree(reversed.as_bytes(), sid, &retained).unwrap();
444 let store = Store::parse(repeated.as_bytes()).unwrap();
445 let index = RevisionIndex::parse(&store).unwrap();
446 let document = Document::parse(&index).unwrap();
447 let view = document.active(sid).unwrap();
448 assert_eq!(view.nodes[&outline].children, [second.object(), first]);
449 assert_eq!(
450 view.nodes[&first].latest_author,
451 Some(ExGuid {
452 guid: retained.guid,
453 n: 3
454 })
455 );
456 }
457362
458 #[test]363 #[test]
459 fn group_normalization_preserves_unequal_indentation_and_overlapping_moves() {364 fn group_normalization_preserves_unequal_indentation_and_overlapping_moves() {
...@@ -469,22 +374,23 @@ mod tests {...@@ -469,22 +374,23 @@ mod tests {
469 .find(|id| matches!(view.nodes[id].kind, Kind::Outline { .. }))374 .find(|id| matches!(view.nodes[id].kind, Kind::Outline { .. }))
470 .unwrap();375 .unwrap();
471 let first = view.nodes[&outline].children[0];376 let first = view.nodes[&outline].children[0];
472 let second = Insertion::paragraph(outline, None, "Second", "Author").unwrap();377 let paragraphs = vec![
473 let third = Insertion::paragraph(outline, None, "Third", "Author").unwrap();378 text_paragraph("Second", Format::default()),
474 let source = PreparedEdit::insert(&source, sid, &second)379 text_paragraph("Third", Format::default()),
475 .unwrap()380 ];
476 .as_bytes()381 let (second, third) = (paragraphs[0].id, paragraphs[1].id);
477 .to_vec();382 let insert = PageOp::Insert {
478 let source = PreparedEdit::insert(&source, sid, &third)383 container: outline,
479 .unwrap()384 before: None,
480 .as_bytes()385 paragraphs,
481 .to_vec();386 };
387 let source = edited(&source, sid, vec![insert]).unwrap();
482 let guid = fresh_guid().unwrap();388 let guid = fresh_guid().unwrap();
483 let a = ExGuid { guid, n: 1 };389 let a = ExGuid { guid, n: 1 };
484 let b = ExGuid { guid, n: 2 };390 let b = ExGuid { guid, n: 2 };
485 let source = write_revision(&source, sid, |raw| {391 let source = write_revision(&source, sid, |raw| {
486 let mut changed = BTreeMap::new();392 let mut changed = BTreeMap::new();
487 for (id, child, level) in [(a, first, 2), (b, second.object(), 1)] {393 for (id, child, level) in [(a, first, 2), (b, second, 1)] {
488 let mut group = PropertyObject {394 let mut group = PropertyObject {
489 jcid: 0x60019,395 jcid: 0x60019,
490 bytes: properties(&[(0x0c001c03, vec![level])])?,396 bytes: properties(&[(0x0c001c03, vec![level])])?,
...@@ -496,7 +402,7 @@ mod tests {...@@ -496,7 +402,7 @@ mod tests {
496 }402 }
497 let mut object = PropertyObject::from_object(&raw.objects[&outline])?;403 let mut object = PropertyObject::from_object(&raw.objects[&outline])?;
498 let mut references = Vec::new();404 let mut references = Vec::new();
499 for id in [a, b, third.object()] {405 for id in [a, b, third] {
500 references.extend_from_slice(&object.reference(id)?);406 references.extend_from_slice(&object.reference(id)?);
501 }407 }
502 object.set(&[(0x24001c20, &references)])?;408 object.set(&[(0x24001c20, &references)])?;
...@@ -504,38 +410,37 @@ mod tests {...@@ -504,38 +410,37 @@ mod tests {
504 Ok(changed)410 Ok(changed)
505 })411 })
506 .unwrap();412 .unwrap();
507 for moving in [false, true] {413 let page = |image: &[u8]| {
508 let intent = if moving {414 let store = Store::parse(image).unwrap();
509 TreeEdit::move_to(third.object(), a, Some(first), "Author")
510 } else {
511 TreeEdit::delete(third.object(), "Author")
512 }
513 .unwrap();
514 let edit = PreparedEdit::tree(&source, sid, &intent).unwrap();
515 let store = Store::parse(edit.as_bytes()).unwrap();
516 let index = RevisionIndex::parse(&store).unwrap();415 let index = RevisionIndex::parse(&store).unwrap();
517 index.validate_current().unwrap();416 index.validate_current().unwrap();
518 let document = Document::parse(&index).unwrap();417 crate::page::Page::from_space(&Document::parse(&index).unwrap(), sid).unwrap()
519 let view = document.active(sid).unwrap();418 };
520 assert_eq!(view.nodes[&outline].children, [a, second.object()]);419 let written = edited(&source, sid, vec![PageOp::Delete { object: third }]).unwrap();
521 assert_eq!(view.nodes[&outline].child_level, Some(2));420 let store = Store::parse(&written).unwrap();
522 assert_eq!(view.nodes[&a].child_level, Some(1));421 let index = RevisionIndex::parse(&store).unwrap();
523 assert_eq!(422 let document = Document::parse(&index).unwrap();
524 view.nodes[&a].children,423 let view = document.active(sid).unwrap();
525 if moving {424 assert_eq!(view.nodes[&outline].children, [a, second]);
526 vec![third.object(), first]425 assert_eq!(view.nodes[&outline].child_level, Some(2));
527 } else {426 assert_eq!(view.nodes[&a].child_level, Some(1));
528 vec![first]427 assert_eq!(view.nodes[&a].children, [first]);
529 }428 if let Some(output) = std::env::var_os("ONESTORE_TREE_OUTPUT") {
530 );429 let output = std::path::PathBuf::from(output).join("unequal-groups");
531 if !moving && let Some(output) = std::env::var_os("ONESTORE_TREE_OUTPUT") {430 assert!(output.is_absolute());
532 let output = std::path::PathBuf::from(output).join("unequal-groups");431 std::fs::create_dir_all(output.parent().unwrap()).unwrap();
533 assert!(output.is_absolute());432 std::fs::create_dir(&output).unwrap();
534 std::fs::create_dir_all(output.parent().unwrap()).unwrap();433 std::fs::write(output.join("groups.one"), &written).unwrap();
535 std::fs::create_dir(&output).unwrap();
536 std::fs::write(output.join("groups.one"), edit.as_bytes()).unwrap();
537 }
538 }434 }
435 // A move keeps the paragraph's level; the writers regroup the outline around it.
436 let moved = PageOp::Move {
437 object: third,
438 parent: Some(outline),
439 before: Some(first),
440 };
441 let mut predicted = page(&source);
442 crate::op::predict(&mut predicted, &moved).unwrap();
443 assert_eq!(page(&edited(&source, sid, vec![moved]).unwrap()), predicted);
539 }444 }
540445
541 #[test]446 #[test]
...@@ -553,7 +458,9 @@ mod tests {...@@ -553,7 +458,9 @@ mod tests {
553 .unwrap();458 .unwrap();
554 let paragraph = view.nodes[&outline].children[0];459 let paragraph = view.nodes[&outline].children[0];
555 let text = view.nodes[&paragraph].content[0];460 let text = view.nodes[&paragraph].content[0];
556 let deletion = TreeEdit::delete(paragraph, "Author").unwrap();461 let deleted = |bytes: &[u8]| {
462 edited(bytes, sid, vec![PageOp::Delete { object: paragraph }]).is_ok()
463 };
557 for target in [page, outline, paragraph, text] {464 for target in [page, outline, paragraph, text] {
558 for property in [0x08001cde, 0x08001cb4, 0x08001cf9, 0x08001cb2] {465 for property in [0x08001cde, 0x08001cb4, 0x08001cf9, 0x08001cb2] {
559 for enabled in [false, true] {466 for enabled in [false, true] {
...@@ -563,11 +470,7 @@ mod tests {...@@ -563,11 +470,7 @@ mod tests {
563 Ok(BTreeMap::from([(target, object)]))470 Ok(BTreeMap::from([(target, object)]))
564 })471 })
565 .unwrap();472 .unwrap();
566 assert_eq!(473 assert_eq!(deleted(&bytes), !enabled, "{target}: {property:#x}");
567 PreparedEdit::tree(&bytes, sid, &deletion).is_err(),
568 enabled,
569 "{target}: {property:#x}"
570 );
571 }474 }
572 }475 }
573 }476 }
...@@ -578,7 +481,7 @@ mod tests {...@@ -578,7 +481,7 @@ mod tests {
578 Ok(BTreeMap::from([(protected, object)]))481 Ok(BTreeMap::from([(protected, object)]))
579 })482 })
580 .unwrap();483 .unwrap();
581 assert!(PreparedEdit::tree(&bytes, sid, &deletion).is_err());484 assert!(!deleted(&bytes));
582 }485 }
583 let bytes = write_revision(&source, sid, |raw| {486 let bytes = write_revision(&source, sid, |raw| {
584 let mut object = PropertyObject::from_object(&raw.objects[&outline])?;487 let mut object = PropertyObject::from_object(&raw.objects[&outline])?;
...@@ -587,10 +490,6 @@ mod tests {...@@ -587,10 +490,6 @@ mod tests {
587 Ok(BTreeMap::from([(outline, object)]))490 Ok(BTreeMap::from([(outline, object)]))
588 })491 })
589 .unwrap();492 .unwrap();
590 assert!(PreparedEdit::tree(&bytes, sid, &deletion).is_err());493 assert!(!deleted(&bytes));
591 let mut value = serde_json::to_value(&deletion).unwrap();
592 value["guid"] = serde_json::to_value([0_u8; 16]).unwrap();
593 let restored = serde_json::from_value(value).unwrap();
594 assert!(PreparedEdit::tree(&source, sid, &restored).is_err());
595 }494 }
596}495}
crates/onestore/src/write.rs+108-73
...@@ -323,45 +323,6 @@ pub(crate) struct ObjectEdit<'a> {...@@ -323,45 +323,6 @@ pub(crate) struct ObjectEdit<'a> {
323 pub inserts: &'a [(u32, &'a [u8])],323 pub inserts: &'a [(u32, &'a [u8])],
324}324}
325325
326/// Replaces a root-level scalar byte property in the default active revision.
327/// The caller supplies its encoded value and is responsible for MS-ONE semantics.
328/// Returns a complete file image without I/O; use `commit_file_property` to update an existing file.
329pub fn replace_property_bytes(
330 source: &[u8],
331 space: ExGuid,
332 object_id: ExGuid,
333 property: u32,
334 value: &[u8],
335) -> Result<Vec<u8>> {
336 if !(3..=7).contains(&((property >> 26) & 31)) {
337 return Err(Error {
338 offset: 0,
339 message: "Property does not contain scalar bytes",
340 });
341 }
342 let store = Store::parse(source)?;
343 let index = RevisionIndex::parse(&store)?;
344 index.validate_current()?;
345 replace_objects(
346 &index,
347 space,
348 &[ObjectEdit {
349 object: object_id,
350 updates: &[(property, value)],
351 inserts: &[],
352 }],
353 )
354}
355
356/// Patches objects of a source the caller has parsed and validated.
357pub(crate) fn replace_objects(
358 index: &RevisionIndex<'_>,
359 space: ExGuid,
360 edits: &[ObjectEdit<'_>],
361) -> Result<Vec<u8>> {
362 write_revision_on(index, space, |revision| patched(revision, edits))
363}
364
365/// Objects of `revision` with `edits` applied.326/// Objects of `revision` with `edits` applied.
366pub(crate) fn patched(327pub(crate) fn patched(
367 revision: &crate::ResolvedRevision<'_>,328 revision: &crate::ResolvedRevision<'_>,
...@@ -511,6 +472,7 @@ pub(crate) struct PropertyObject {...@@ -511,6 +472,7 @@ pub(crate) struct PropertyObject {
511472
512pub(crate) enum RevisionEdit {473pub(crate) enum RevisionEdit {
513 Update(BTreeMap<ExGuid, PropertyObject>),474 Update(BTreeMap<ExGuid, PropertyObject>),
475 #[cfg(any(test, feature = "protected"))]
514 Create {476 Create {
515 roots: BTreeMap<u32, ExGuid>,477 roots: BTreeMap<u32, ExGuid>,
516 objects: BTreeMap<ExGuid, PropertyObject>,478 objects: BTreeMap<ExGuid, PropertyObject>,
...@@ -1269,24 +1231,18 @@ pub(crate) fn chain_depth(index: &RevisionIndex<'_>, space: ExGuid, rid: ExGuid)...@@ -1269,24 +1231,18 @@ pub(crate) fn chain_depth(index: &RevisionIndex<'_>, space: ExGuid, rid: ExGuid)
1269 .count()1231 .count()
1270}1232}
12711233
1234/// A test fixture: `source` with one revision of `space` storing `edit`'s objects.
1235#[cfg(test)]
1272pub(crate) fn write_revision(1236pub(crate) fn write_revision(
1273 source: &[u8],1237 source: &[u8],
1274 space: ExGuid,1238 space: ExGuid,
1275 edit: impl FnOnce(&crate::ResolvedRevision<'_>) -> Result<BTreeMap<ExGuid, PropertyObject>>,1239 edit: impl FnOnce(&crate::ResolvedRevision<'_>) -> Result<BTreeMap<ExGuid, PropertyObject>>,
1276) -> Result<Vec<u8>> {1240) -> Result<Vec<u8>> {
1277 write_revision_with_payloads(source, space, &[], edit)1241 write_revisions(source, update(space, edit))
1278}
1279
1280pub(crate) fn write_revision_with_payloads(
1281 source: &[u8],
1282 space: ExGuid,
1283 payloads: &[([u8; 16], &[u8])],
1284 edit: impl FnOnce(&crate::ResolvedRevision<'_>) -> Result<BTreeMap<ExGuid, PropertyObject>>,
1285) -> Result<Vec<u8>> {
1286 write_revisions_with_payloads(source, payloads, update(space, edit))
1287}1242}
12881243
1289/// One space's active revision edited into its update.1244/// One space's active revision edited into its update.
1245#[cfg(test)]
1290fn update(1246fn update(
1291 space: ExGuid,1247 space: ExGuid,
1292 edit: impl FnOnce(&crate::ResolvedRevision<'_>) -> Result<BTreeMap<ExGuid, PropertyObject>>,1248 edit: impl FnOnce(&crate::ResolvedRevision<'_>) -> Result<BTreeMap<ExGuid, PropertyObject>>,
...@@ -1300,26 +1256,30 @@ fn update(...@@ -1300,26 +1256,30 @@ fn update(
1300 }1256 }
1301}1257}
13021258
1303pub(crate) fn write_revisions(1259/// A test fixture: the transaction appending `edit`'s revisions to `source`.
1260#[cfg(test)]
1261pub(crate) fn revisions(
1304 source: &[u8],1262 source: &[u8],
1305 edit: impl FnOnce(&RevisionIndex<'_>) -> Result<BTreeMap<ExGuid, RevisionEdit>>,1263 edit: impl FnOnce(&RevisionIndex<'_>) -> Result<BTreeMap<ExGuid, RevisionEdit>>,
1306) -> Result<Vec<u8>> {1264) -> Result<Option<crate::Transaction>> {
1307 write_revisions_with_payloads(source, &[], edit)1265 let store = Store::parse(source)?;
1266 let index = RevisionIndex::parse(&store)?;
1267 index.validate_current()?;
1268 build_on(&index, &[], None, edit)
1308}1269}
13091270
1310/// `write_revisions` that also stores embedded payloads: each becomes a file-data store1271/// A test fixture: `source` with `edit`'s revisions appended.
1311/// object referenced from the root file node list under its identity, as OneNote embeds1272#[cfg(test)]
1312/// pictures and attachments.1273pub(crate) fn write_revisions(
1313pub(crate) fn write_revisions_with_payloads(
1314 source: &[u8],1274 source: &[u8],
1315 payloads: &[([u8; 16], &[u8])],
1316 edit: impl FnOnce(&RevisionIndex<'_>) -> Result<BTreeMap<ExGuid, RevisionEdit>>,1275 edit: impl FnOnce(&RevisionIndex<'_>) -> Result<BTreeMap<ExGuid, RevisionEdit>>,
1317) -> Result<Vec<u8>> {1276) -> Result<Vec<u8>> {
1318 publish(source, payloads, None, true, edit)1277 publish(source, &[], None, true, edit)
1319}1278}
13201279
1321/// `write_revisions_with_payloads` without validating that current revisions are1280/// `source` with `edit`'s revisions and the `payloads` appended, without validating that
1322/// complete: a protected section is validated by unlocking it, through `protection`.1281/// current revisions are complete: a protected section is validated by unlocking it,
1282/// through `protection`.
1323#[cfg(feature = "protected")]1283#[cfg(feature = "protected")]
1324pub(crate) fn append_revisions(1284pub(crate) fn append_revisions(
1325 source: &[u8],1285 source: &[u8],
...@@ -1330,6 +1290,7 @@ pub(crate) fn append_revisions(...@@ -1330,6 +1290,7 @@ pub(crate) fn append_revisions(
1330 publish(source, payloads, protection, false, edit)1290 publish(source, payloads, protection, false, edit)
1331}1291}
13321292
1293#[cfg(any(test, feature = "protected"))]
1333fn publish(1294fn publish(
1334 source: &[u8],1295 source: &[u8],
1335 payloads: &[([u8; 16], &[u8])],1296 payloads: &[([u8; 16], &[u8])],
...@@ -1355,7 +1316,8 @@ pub(crate) fn check(output: &[u8], validate: bool) -> Result<()> {...@@ -1355,7 +1316,8 @@ pub(crate) fn check(output: &[u8], validate: bool) -> Result<()> {
13551316
1356/// The written image, unchecked: `check` follows once the caller has released whatever1317/// The written image, unchecked: `check` follows once the caller has released whatever
1357/// `edit` borrowed.1318/// `edit` borrowed.
1358pub(crate) fn build(1319#[cfg(any(test, feature = "protected"))]
1320fn build(
1359 source: &[u8],1321 source: &[u8],
1360 payloads: &[([u8; 16], &[u8])],1322 payloads: &[([u8; 16], &[u8])],
1361 protection: Option<&dyn Protection>,1323 protection: Option<&dyn Protection>,
...@@ -1367,16 +1329,20 @@ pub(crate) fn build(...@@ -1367,16 +1329,20 @@ pub(crate) fn build(
1367 if validate {1329 if validate {
1368 index.validate_current()?;1330 index.validate_current()?;
1369 }1331 }
1370 build_on(&index, payloads, protection, edit)1332 applied(source, build_on(&index, payloads, protection, edit)?.as_ref())
1371}1333}
13721334
1373/// `write_revision` on a source the caller has parsed and validated.1335/// `write_revision` on a source the caller has parsed and validated.
1336#[cfg(test)]
1374pub(crate) fn write_revision_on(1337pub(crate) fn write_revision_on(
1375 index: &RevisionIndex<'_>,1338 index: &RevisionIndex<'_>,
1376 space: ExGuid,1339 space: ExGuid,
1377 edit: impl FnOnce(&crate::ResolvedRevision<'_>) -> Result<BTreeMap<ExGuid, PropertyObject>>,1340 edit: impl FnOnce(&crate::ResolvedRevision<'_>) -> Result<BTreeMap<ExGuid, PropertyObject>>,
1378) -> Result<Vec<u8>> {1341) -> Result<Vec<u8>> {
1379 let output = build_on(index, &[], None, update(space, edit))?;1342 let output = applied(
1343 index.store.data,
1344 build_on(index, &[], None, update(space, edit))?.as_ref(),
1345 )?;
1380 check(&output, true)?;1346 check(&output, true)?;
1381 Ok(output)1347 Ok(output)
1382}1348}
...@@ -1387,12 +1353,14 @@ thread_local! {...@@ -1387,12 +1353,14 @@ thread_local! {
1387 pub(crate) static BUILDS: std::cell::Cell<usize> = const { std::cell::Cell::new(0) };1353 pub(crate) static BUILDS: std::cell::Cell<usize> = const { std::cell::Cell::new(0) };
1388}1354}
13891355
1356/// The transaction appending `edit`'s revisions and the `payloads` to the parsed store;
1357/// none when nothing changes.
1390pub(crate) fn build_on(1358pub(crate) fn build_on(
1391 index: &RevisionIndex<'_>,1359 index: &RevisionIndex<'_>,
1392 payloads: &[([u8; 16], &[u8])],1360 payloads: &[([u8; 16], &[u8])],
1393 protection: Option<&dyn Protection>,1361 protection: Option<&dyn Protection>,
1394 edit: impl FnOnce(&RevisionIndex<'_>) -> Result<BTreeMap<ExGuid, RevisionEdit>>,1362 edit: impl FnOnce(&RevisionIndex<'_>) -> Result<BTreeMap<ExGuid, RevisionEdit>>,
1395) -> Result<Vec<u8>> {1363) -> Result<Option<crate::Transaction>> {
1396 #[cfg(test)]1364 #[cfg(test)]
1397 BUILDS.with(|builds| builds.set(builds.get() + 1));1365 BUILDS.with(|builds| builds.set(builds.get() + 1));
1398 let store = index.store;1366 let store = index.store;
...@@ -1411,13 +1379,13 @@ pub(crate) fn build_on(...@@ -1411,13 +1379,13 @@ pub(crate) fn build_on(
1411 let changes = edit(index)?;1379 let changes = edit(index)?;
1412 let mut appending = Appending::new(store.state()?);1380 let mut appending = Appending::new(store.state()?);
1413 for (space, change) in changes {1381 for (space, change) in changes {
1414 let new_space = matches!(change, RevisionEdit::Create { .. });
1415 let current = (ExGuid::default(), 1_u32);1382 let current = (ExGuid::default(), 1_u32);
1416 let (rid, label, revision, replacements) = match change {1383 let (rid, label, revision, replacements, new_space) = match change {
1417 RevisionEdit::Update(objects) => {1384 RevisionEdit::Update(objects) => {
1418 let rid = index.active(space)?;1385 let rid = index.active(space)?;
1419 (Some(rid), current, resolve(space, rid)?, objects)1386 (Some(rid), current, resolve(space, rid)?, objects, false)
1420 }1387 }
1388 #[cfg(any(test, feature = "protected"))]
1421 RevisionEdit::Create { roots, objects } => {1389 RevisionEdit::Create { roots, objects } => {
1422 if !is_section || space.guid == [0; 16] || index.spaces.contains_key(&space) {1390 if !is_section || space.guid == [0; 16] || index.spaces.contains_key(&space) {
1423 return Err(Error {1391 return Err(Error {
...@@ -1439,6 +1407,7 @@ pub(crate) fn build_on(...@@ -1439,6 +1407,7 @@ pub(crate) fn build_on(
1439 objects: BTreeMap::new(),1407 objects: BTreeMap::new(),
1440 },1408 },
1441 objects,1409 objects,
1410 true,
1442 )1411 )
1443 }1412 }
1444 #[cfg(feature = "protected")]1413 #[cfg(feature = "protected")]
...@@ -1462,6 +1431,7 @@ pub(crate) fn build_on(...@@ -1462,6 +1431,7 @@ pub(crate) fn build_on(
1462 objects: BTreeMap::new(),1431 objects: BTreeMap::new(),
1463 },1432 },
1464 objects,1433 objects,
1434 false,
1465 )1435 )
1466 }1436 }
1467 };1437 };
...@@ -1525,13 +1495,78 @@ pub(crate) fn build_on(...@@ -1525,13 +1495,78 @@ pub(crate) fn build_on(
1525 )?;1495 )?;
1526 }1496 }
1527 appending.payloads(payloads, protection)?;1497 appending.payloads(payloads, protection)?;
1528 Ok(match appending.finish()? {1498 Ok(appending.finish()?.map(|(transaction, _)| transaction))
1529 Some((transaction, _)) => {1499}
1530 let mut output = source.to_vec();1500
1531 transaction.apply(&mut output)?;1501/// `source` with `transaction` applied.
1532 output1502pub(crate) fn applied(source: &[u8], transaction: Option<&crate::Transaction>) -> Result<Vec<u8>> {
1503 let mut output = source.to_vec();
1504 if let Some(transaction) = transaction {
1505 transaction.apply(&mut output)?;
1506 }
1507 Ok(output)
1508}
1509
1510/// Payload identities the file-data store of `store` declares, in order.
1511#[cfg(any(test, feature = "protected"))]
1512pub(crate) fn declared_payloads(store: &Store<'_>) -> Vec<[u8; 16]> {
1513 store
1514 .lists
1515 .values()
1516 .flat_map(|list| &list.nodes)
1517 .filter(|node| node.id == 0x94)
1518 .filter_map(|node| node.payload.get(..16).and_then(|g| g.try_into().ok()))
1519 .collect()
1520}
1521
1522/// Writes the live objects of each `edited` revision that differ from its space's active
1523/// revision in the validated `source` as one revision per space, embedding the `payloads`
1524/// it lacks. A protected `source` takes the revisions its plaintext twin gained.
1525#[cfg(any(test, feature = "protected"))]
1526pub(crate) fn squash(
1527 source: &RevisionIndex<'_>,
1528 edited: &[(ExGuid, &crate::ResolvedRevision<'_>)],
1529 payloads: &[([u8; 16], &[u8])],
1530 protection: Option<&dyn Protection>,
1531) -> Result<Option<crate::Transaction>> {
1532 let existing = declared_payloads(source.store);
1533 let payloads: Vec<_> = payloads
1534 .iter()
1535 .filter(|(guid, _)| !existing.contains(guid))
1536 .copied()
1537 .collect();
1538 build_on(source, &payloads, protection, |index| {
1539 let mut changes = BTreeMap::new();
1540 for (space, after) in edited {
1541 let before = match protection {
1542 Some(protection) => protection.resolve(*space, index.active(*space)?)?,
1543 None => index.resolve_active(*space)?,
1544 };
1545 if before.roots != after.roots {
1546 return Err(Error {
1547 offset: 0,
1548 message: "Page edits cannot change revision roots",
1549 });
1550 }
1551 // Retired styles and deleted content stay in history; only live objects are written.
1552 let live = after.reachable()?;
1553 let mut changed = BTreeMap::new();
1554 for (id, object) in &after.objects {
1555 if !live.contains(id)
1556 || before
1557 .objects
1558 .get(id)
1559 .map(|previous| unchanged(previous, object))
1560 .transpose()?
1561 == Some(true)
1562 {
1563 continue;
1564 }
1565 changed.insert(*id, replacement(*id, object)?);
1566 }
1567 changes.insert(*space, RevisionEdit::Update(changed));
1533 }1568 }
1534 None => source.to_vec(),1569 Ok(changes)
1535 })1570 })
1536}1571}
15371572
crates/onestore/src/write/tests.rs+95-55
...@@ -72,7 +72,15 @@ fn copying_reference_arrays_remaps_every_entry_in_each_stream() {...@@ -72,7 +72,15 @@ fn copying_reference_arrays_remaps_every_entry_in_each_stream() {
72#[test]72#[test]
73fn document_insertions_and_formatting_respect_readonly_ancestors() {73fn document_insertions_and_formatting_respect_readonly_ancestors() {
74 use super::{PropertyObject, write_revision};74 use super::{PropertyObject, write_revision};
75 use crate::{ExGuid, Insertion, PreparedEdit, RevisionIndex, Store};75 use crate::{
76 ExGuid, RevisionIndex, Store, TextAttribute,
77 document::{Format, Layout},
78 op::{
79 PageOp,
80 tests::{edited, text_paragraph},
81 },
82 page::{Outline, PageObject, text::new_id},
83 };
76 use std::collections::BTreeMap;84 use std::collections::BTreeMap;
77 let source = crate::create_section("readonly.one", "Original", "Author").unwrap();85 let source = crate::create_section("readonly.one", "Original", "Author").unwrap();
78 let store = Store::parse(&source).unwrap();86 let store = Store::parse(&source).unwrap();
...@@ -97,9 +105,23 @@ fn document_insertions_and_formatting_respect_readonly_ancestors() {...@@ -97,9 +105,23 @@ fn document_insertions_and_formatting_respect_readonly_ancestors() {
97 ))105 ))
98 })106 })
99 .unwrap();107 .unwrap();
100 let right = Insertion::paragraph(outline, None, "Right", "Author").unwrap();108 let right = text_paragraph("Right", Format::default());
101 let expanded = PreparedEdit::insert(&source, sid, &right).unwrap();109 let right_text = right.text().unwrap().id;
102 let source = expanded.as_bytes();110 let insert = |container, paragraph| PageOp::Insert {
111 container,
112 before: None,
113 paragraphs: vec![paragraph],
114 };
115 let expanded = edited(&source, sid, vec![insert(outline, right.clone())]).unwrap();
116 let source = expanded.as_slice();
117 let raw = index
118 .resolve(sid, index.spaces[&sid].labels[&(ExGuid::default(), 1)])
119 .unwrap();
120 let text = raw
121 .objects
122 .iter()
123 .find_map(|(id, object)| (object.jcid == 0x6000e).then_some(*id))
124 .unwrap();
103 for blocked in [page, outline, paragraph] {125 for blocked in [page, outline, paragraph] {
104 let protected = write_revision(source, sid, |raw| {126 let protected = write_revision(source, sid, |raw| {
105 let mut object = PropertyObject::from_object(&raw.objects[&blocked])?;127 let mut object = PropertyObject::from_object(&raw.objects[&blocked])?;
...@@ -107,47 +129,55 @@ fn document_insertions_and_formatting_respect_readonly_ancestors() {...@@ -107,47 +129,55 @@ fn document_insertions_and_formatting_respect_readonly_ancestors() {
107 Ok(BTreeMap::from([(blocked, object)]))129 Ok(BTreeMap::from([(blocked, object)]))
108 })130 })
109 .unwrap();131 .unwrap();
110 let child = Insertion::paragraph(paragraph, None, "Nested", "Author").unwrap();132 let refused = |op| edited(&protected, sid, vec![op]).is_err();
111 assert!(PreparedEdit::insert(&protected, sid, &child).is_err());133 assert!(refused(insert(paragraph, text_paragraph("Nested", Format::default()))));
112 let raw = index134 assert!(refused(PageOp::Split {
113 .resolve(sid, index.spaces[&sid].labels[&(ExGuid::default(), 1)])135 text,
114 .unwrap();136 at: 1,
115 let text = raw137 paragraph: new_id().unwrap(),
116 .objects138 right: new_id().unwrap(),
117 .iter()139 lists: Vec::new(),
118 .find_map(|(id, object)| (object.jcid == 0x6000e).then_some(*id))140 }));
119 .unwrap();141 assert!(refused(PageOp::Join {
120 let split = crate::ParagraphSplit::new(text, 1, "Author").unwrap();142 left: text,
121 assert!(PreparedEdit::split(&protected, sid, &split).is_err());143 right: right_text,
122 let join = crate::ParagraphJoin::new(text, right.text_object(), "Author").unwrap();144 }));
123 assert!(PreparedEdit::join(&protected, sid, &join).is_err());145 assert!(refused(PageOp::Format {
124 assert!(146 text,
125 PreparedEdit::format(147 range: 1..3,
126 &protected,148 set: vec![TextAttribute::Bold(true)],
127 sid,149 clear: Vec::new(),
128 text,150 }));
129 1..3,
130 &[crate::TextAttribute::Bold(true)]
131 )
132 .is_err()
133 );
134 if blocked == page {151 if blocked == page {
135 let outline = Insertion::outline(page, 36.0, 36.0, "Outline", "Author").unwrap();152 assert!(refused(PageOp::Add {
136 assert!(PreparedEdit::insert(&protected, sid, &outline).is_err());153 object: PageObject::Outline(Outline {
154 id: new_id().unwrap(),
155 title: false,
156 min_width: None,
157 layout: Layout {
158 x: Some(36.0),
159 y: Some(36.0),
160 ..Default::default()
161 },
162 indents: Vec::new(),
163 paragraphs: vec![text_paragraph("Outline", Format::default())],
164 unsupported: Vec::new(),
165 }),
166 before: None,
167 }));
137 }168 }
138 }169 }
139 let protected = write_revision(source, sid, |raw| {170 let protected = write_revision(source, sid, |raw| {
140 let mut object = PropertyObject::from_object(&raw.objects[&right.object()])?;171 let mut object = PropertyObject::from_object(&raw.objects[&right.id])?;
141 object.set(&[(0x88001cde, &[])])?;172 object.set(&[(0x88001cde, &[])])?;
142 Ok(BTreeMap::from([(right.object(), object)]))173 Ok(BTreeMap::from([(right.id, object)]))
143 })174 })
144 .unwrap();175 .unwrap();
145 let document = crate::document::Document::parse(&index).unwrap();176 let join = PageOp::Join {
146 let space = &document.spaces[&sid];177 left: text,
147 let view = space.active().unwrap();178 right: right_text,
148 let text = view.nodes[&paragraph].content[0];179 };
149 let join = crate::ParagraphJoin::new(text, right.text_object(), "Author").unwrap();180 assert!(edited(&protected, sid, vec![join]).is_err());
150 assert!(PreparedEdit::join(&protected, sid, &join).is_err());
151}181}
152182
153fn add_paragraph(source: &[u8], number: u32) -> Vec<u8> {183fn add_paragraph(source: &[u8], number: u32) -> Vec<u8> {
...@@ -774,8 +804,9 @@ fn invalid_property_splices_are_rejected() {...@@ -774,8 +804,9 @@ fn invalid_property_splices_are_rejected() {
774fn character_formatting_preserves_inheritance_and_associated_data() {804fn character_formatting_preserves_inheritance_and_associated_data() {
775 use super::{PropertyObject, write_revision};805 use super::{PropertyObject, write_revision};
776 use crate::{806 use crate::{
777 ExGuid, PreparedEdit, RevisionIndex, Store, TextAttribute,807 ExGuid, RevisionIndex, Store, TextAttribute,
778 document::{Document, Kind},808 document::{Document, Kind},
809 op::{PageOp, tests::edited},
779 };810 };
780 use std::collections::BTreeMap;811 use std::collections::BTreeMap;
781 let source = crate::create_section("inherited.one", "abcdef", "Author").unwrap();812 let source = crate::create_section("inherited.one", "abcdef", "Author").unwrap();
...@@ -858,13 +889,29 @@ fn character_formatting_preserves_inheritance_and_associated_data() {...@@ -858,13 +889,29 @@ fn character_formatting_preserves_inheritance_and_associated_data() {
858 Ok(BTreeMap::from([(text, target)]))889 Ok(BTreeMap::from([(text, target)]))
859 })890 })
860 .unwrap();891 .unwrap();
892 let text_edit = |range, with: &str| {
893 let with = with.to_owned();
894 edited(&fixture, sid, vec![PageOp::Text { text, range, with }])
895 };
896 let bold = |range| {
897 edited(
898 &fixture,
899 sid,
900 vec![PageOp::Format {
901 text,
902 range,
903 set: vec![TextAttribute::Bold(true)],
904 clear: Vec::new(),
905 }],
906 )
907 };
861 if variant == 9 {908 if variant == 9 {
862 assert!(PreparedEdit::text(&fixture, sid, text, 0..0, "x").is_err());909 assert!(text_edit(0..0, "x").is_err());
863 continue;910 continue;
864 }911 }
865 if variant >= 7 {912 if variant >= 7 {
866 let edit = PreparedEdit::text(&fixture, sid, text, 1..3, "longer").unwrap();913 let edit = text_edit(1..3, "longer").unwrap();
867 let [before, after] = [&fixture, edit.as_bytes()].map(|bytes| {914 let [before, after] = [&fixture, &edit].map(|bytes| {
868 let store = Store::parse(bytes).unwrap();915 let store = Store::parse(bytes).unwrap();
869 let index = RevisionIndex::parse(&store).unwrap();916 let index = RevisionIndex::parse(&store).unwrap();
870 let document = Document::parse(&index).unwrap();917 let document = Document::parse(&index).unwrap();
...@@ -877,24 +924,20 @@ fn character_formatting_preserves_inheritance_and_associated_data() {...@@ -877,24 +924,20 @@ fn character_formatting_preserves_inheritance_and_associated_data() {
877 format!("{:?}", view.nodes[&text].extra)924 format!("{:?}", view.nodes[&text].extra)
878 });925 });
879 assert_eq!(before, after);926 assert_eq!(before, after);
880 assert!(PreparedEdit::text(&fixture, sid, text, 0..4, "x").is_err());927 assert!(text_edit(0..4, "x").is_err());
881 assert_eq!(928 assert_eq!(text_edit(3..5, "x").is_ok(), variant == 7);
882 PreparedEdit::text(&fixture, sid, text, 3..5, "x").is_ok(),
883 variant == 7
884 );
885 continue;929 continue;
886 }930 }
887 // Hidden and hyperlink runs are ordinary text with flags; associated data,931 // Hidden and hyperlink runs are ordinary text with flags; associated data,
888 // equations and embedded objects are not.932 // equations and embedded objects are not.
889 let text_edit = PreparedEdit::text(&fixture, sid, text, 1..5, "x");933 assert_eq!(text_edit(1..5, "x").is_ok(), matches!(variant, 0 | 1 | 4));
890 assert_eq!(text_edit.is_ok(), matches!(variant, 0 | 1 | 4));934 let edit = bold(0..6);
891 let edit = PreparedEdit::format(&fixture, sid, text, 0..6, &[TextAttribute::Bold(true)]);
892 if matches!(variant, 2 | 5 | 6) {935 if matches!(variant, 2 | 5 | 6) {
893 assert!(edit.is_err());936 assert!(edit.is_err());
894 continue;937 continue;
895 }938 }
896 let edit = edit.unwrap();939 let edit = edit.unwrap();
897 let store = Store::parse(edit.as_bytes()).unwrap();940 let store = Store::parse(&edit).unwrap();
898 let index = RevisionIndex::parse(&store).unwrap();941 let index = RevisionIndex::parse(&store).unwrap();
899 let doc = Document::parse(&index).unwrap();942 let doc = Document::parse(&index).unwrap();
900 let s = &doc.spaces[&sid];943 let s = &doc.spaces[&sid];
...@@ -911,10 +954,7 @@ fn character_formatting_preserves_inheritance_and_associated_data() {...@@ -911,10 +954,7 @@ fn character_formatting_preserves_inheritance_and_associated_data() {
911 let data = &view.nodes[&text].extra[runs[0].extra_set.unwrap()];954 let data = &view.nodes[&text].extra[runs[0].extra_set.unwrap()];
912 assert_eq!(data.len(), 1);955 assert_eq!(data.len(), 1);
913 assert_eq!(data[0].id, 0x14001234);956 assert_eq!(data[0].id, 0x14001234);
914 assert!(957 assert!(bold(1..3).is_err());
915 PreparedEdit::format(&fixture, sid, text, 1..3, &[TextAttribute::Bold(true)])
916 .is_err()
917 );
918 let previous = Store::parse(&fixture).unwrap();958 let previous = Store::parse(&fixture).unwrap();
919 let previous = RevisionIndex::parse(&previous).unwrap();959 let previous = RevisionIndex::parse(&previous).unwrap();
920 let previous_doc = Document::parse(&previous).unwrap();960 let previous_doc = Document::parse(&previous).unwrap();
crates/onestore/src/write/tests/multi.rs+15-19
...@@ -1,7 +1,8 @@...@@ -1,7 +1,8 @@
1use crate::{1use crate::{
2 ExGuid, RevisionIndex, Store,2 ExGuid, RevisionIndex, Store,
3 document::{Document, Kind},3 document::{Document, Kind},
4 write::{PropertyObject, RevisionEdit, write_revisions},4 Transaction,
5 write::{PropertyObject, RevisionEdit, applied, revisions, write_revisions},
5};6};
6use std::collections::BTreeMap;7use std::collections::BTreeMap;
78
...@@ -18,8 +19,8 @@ mod current {...@@ -18,8 +19,8 @@ mod current {
18const SOURCE: &[u8] =19const SOURCE: &[u8] =
19 include_bytes!("../../../../../corpus/page-lifecycle/03-renamed/notebook/Lifecycle.one");20 include_bytes!("../../../../../corpus/page-lifecycle/03-renamed/notebook/Lifecycle.one");
2021
21fn nest(source: &[u8]) -> Vec<u8> {22fn nest(source: &[u8]) -> Option<Transaction> {
22 write_revisions(source, |index| {23 revisions(source, |index| {
23 let document = Document::parse(index)?;24 let document = Document::parse(index)?;
24 let pages = document.pages()?;25 let pages = document.pages()?;
25 assert_eq!(pages.len(), 9);26 assert_eq!(pages.len(), 9);
...@@ -100,7 +101,7 @@ fn nest(source: &[u8]) -> Vec<u8> {...@@ -100,7 +101,7 @@ fn nest(source: &[u8]) -> Vec<u8> {
100101
101#[test]102#[test]
102fn nesting_publishes_section_order_and_page_levels_in_one_transaction() {103fn nesting_publishes_section_order_and_page_levels_in_one_transaction() {
103 let written = nest(SOURCE);104 let written = applied(SOURCE, nest(SOURCE).as_ref()).unwrap();
104 let before = Store::parse(SOURCE).unwrap();105 let before = Store::parse(SOURCE).unwrap();
105 let after = Store::parse(&written).unwrap();106 let after = Store::parse(&written).unwrap();
106 assert_eq!(107 assert_eq!(
...@@ -160,7 +161,7 @@ fn nesting_publishes_section_order_and_page_levels_in_one_transaction() {...@@ -160,7 +161,7 @@ fn nesting_publishes_section_order_and_page_levels_in_one_transaction() {
160 .collect::<Vec<_>>(),161 .collect::<Vec<_>>(),
161 [1, 1, 1, 3, 1, 1, 1]162 [1, 1, 1, 3, 1, 1, 1]
162 );163 );
163 assert_eq!(nest(&written), written);164 assert_eq!(nest(&written), None);
164 if let Some(output) = std::env::var_os("ONESTORE_PAGE_BATCH_OUTPUT") {165 if let Some(output) = std::env::var_os("ONESTORE_PAGE_BATCH_OUTPUT") {
165 std::fs::create_dir(&output).unwrap();166 std::fs::create_dir(&output).unwrap();
166 std::fs::write(167 std::fs::write(
...@@ -173,7 +174,8 @@ fn nesting_publishes_section_order_and_page_levels_in_one_transaction() {...@@ -173,7 +174,8 @@ fn nesting_publishes_section_order_and_page_levels_in_one_transaction() {
173174
174#[test]175#[test]
175fn interrupted_multi_space_publication_never_exposes_a_partial_nesting() {176fn interrupted_multi_space_publication_never_exposes_a_partial_nesting() {
176 let written = nest(SOURCE);177 let transaction = nest(SOURCE).unwrap();
178 let written = applied(SOURCE, Some(&transaction)).unwrap();
177 let old = current::current(SOURCE);179 let old = current::current(SOURCE);
178 let new = current::current(&written);180 let new = current::current(&written);
179 assert_ne!(old, new);181 assert_ne!(old, new);
...@@ -186,9 +188,7 @@ fn interrupted_multi_space_publication_never_exposes_a_partial_nesting() {...@@ -186,9 +188,7 @@ fn interrupted_multi_space_publication_never_exposes_a_partial_nesting() {
186 write_limit,188 write_limit,
187 random: 1951,189 random: 1951,
188 };190 };
189 crate::commit::Transaction::between(SOURCE, &written)191 transaction.commit(&mut complete).unwrap();
190 .commit(&mut complete)
191 .unwrap();
192 assert_eq!(complete.durable, written);192 assert_eq!(complete.durable, written);
193 for fail_at in 1..=complete.operation {193 for fail_at in 1..=complete.operation {
194 let mut interrupted = disk::Disk {194 let mut interrupted = disk::Disk {
...@@ -199,9 +199,7 @@ fn interrupted_multi_space_publication_never_exposes_a_partial_nesting() {...@@ -199,9 +199,7 @@ fn interrupted_multi_space_publication_never_exposes_a_partial_nesting() {
199 write_limit,199 write_limit,
200 random: 1951 + fail_at as u64,200 random: 1951 + fail_at as u64,
201 };201 };
202 crate::commit::Transaction::between(SOURCE, &written)202 transaction.commit(&mut interrupted).unwrap_err();
203 .commit(&mut interrupted)
204 .unwrap_err();
205 let observed = current::current(&interrupted.durable);203 let observed = current::current(&interrupted.durable);
206 assert!(204 assert!(
207 observed == old || observed == new,205 observed == old || observed == new,
...@@ -234,7 +232,7 @@ fn repeated_multi_space_edits_cross_counter_carries_and_checkpoint_each_space()...@@ -234,7 +232,7 @@ fn repeated_multi_space_edits_cross_counter_carries_and_checkpoint_each_space()
234 let initial_index = RevisionIndex::parse(&initial_store).unwrap();232 let initial_index = RevisionIndex::parse(&initial_store).unwrap();
235 let mut checkpoints = BTreeMap::new();233 let mut checkpoints = BTreeMap::new();
236 for step in 1_u32..=514 {234 for step in 1_u32..=514 {
237 let written = write_revisions(&source, |index| {235 let transaction = revisions(&source, |index| {
238 let mut changes = BTreeMap::new();236 let mut changes = BTreeMap::new();
239 for (sid, space) in &index.spaces {237 for (sid, space) in &index.spaces {
240 let raw = index.resolve(*sid, space.labels[&(ExGuid::default(), 1)])?;238 let raw = index.resolve(*sid, space.labels[&(ExGuid::default(), 1)])?;
...@@ -259,7 +257,9 @@ fn repeated_multi_space_edits_cross_counter_carries_and_checkpoint_each_space()...@@ -259,7 +257,9 @@ fn repeated_multi_space_edits_cross_counter_carries_and_checkpoint_each_space()
259 .map(|(sid, objects)| (sid, RevisionEdit::Update(objects)))257 .map(|(sid, objects)| (sid, RevisionEdit::Update(objects)))
260 .collect())258 .collect())
261 })259 })
260 .unwrap()
262 .unwrap();261 .unwrap();
262 let written = applied(&source, Some(&transaction)).unwrap();
263 let store = Store::parse(&written).unwrap();263 let store = Store::parse(&written).unwrap();
264 assert_eq!(264 assert_eq!(
265 store.header.transaction_count,265 store.header.transaction_count,
...@@ -290,9 +290,7 @@ fn repeated_multi_space_edits_cross_counter_carries_and_checkpoint_each_space()...@@ -290,9 +290,7 @@ fn repeated_multi_space_edits_cross_counter_carries_and_checkpoint_each_space()
290 write_limit: 4096,290 write_limit: 4096,
291 random: 1952,291 random: 1952,
292 };292 };
293 crate::commit::Transaction::between(&source, &written)293 transaction.commit(&mut complete).unwrap();
294 .commit(&mut complete)
295 .unwrap();
296 for fail_at in 1..=complete.operation {294 for fail_at in 1..=complete.operation {
297 let mut interrupted = disk::Disk {295 let mut interrupted = disk::Disk {
298 visible: source.clone(),296 visible: source.clone(),
...@@ -302,9 +300,7 @@ fn repeated_multi_space_edits_cross_counter_carries_and_checkpoint_each_space()...@@ -302,9 +300,7 @@ fn repeated_multi_space_edits_cross_counter_carries_and_checkpoint_each_space()
302 write_limit: 4096,300 write_limit: 4096,
303 random: 1952 + fail_at as u64,301 random: 1952 + fail_at as u64,
304 };302 };
305 crate::commit::Transaction::between(&source, &written)303 transaction.commit(&mut interrupted).unwrap_err();
306 .commit(&mut interrupted)
307 .unwrap_err();
308 let observed = current::current(&interrupted.durable);304 let observed = current::current(&interrupted.durable);
309 assert!(observed == old || observed == new, "{step}:{fail_at}");305 assert!(observed == old || observed == new, "{step}:{fail_at}");
310 }306 }
crates/onestore/tests/commit.rs+87-137
...@@ -2,104 +2,94 @@...@@ -2,104 +2,94 @@
2mod disk;2mod disk;
3use disk::Disk;3use disk::Disk;
4use onestore::{4use onestore::{
5 ExGuid, ObjectData, PropertySets, RevisionIndex, Store, Value, replace_property_bytes,5 Arena, ExGuid, RevisionIndex, Section, Store, Transaction,
6 document::{Document, Kind},
7 op::{Edit, Op, PageOp},
6};8};
7use std::fs;9use std::fs;
810
9#[test]11#[test]
10fn incomplete_native_save_is_rejected_before_storage_io() {12fn incomplete_native_save_is_rejected_before_an_edit() {
11 let source = fs::read("../../corpus/malformed/native-inflight.one").unwrap();13 let source = fs::read("../../corpus/malformed/native-inflight.one").unwrap();
12 let store = Store::parse(&source).unwrap();14 let store = Store::parse(&source).unwrap();
15 let graph_error = RevisionIndex::parse(&store)
16 .unwrap()
17 .validate_current()
18 .unwrap_err();
19 let arena = Arena::default();
20 let error = Section::open(&arena, source.clone()).err().unwrap();
21 assert_eq!(error.to_string(), graph_error.to_string());
22}
23
24/// The text object holding `text` and its page space.
25fn target(source: &[u8], text: &str) -> (ExGuid, ExGuid) {
26 let store = Store::parse(source).unwrap();
13 let index = RevisionIndex::parse(&store).unwrap();27 let index = RevisionIndex::parse(&store).unwrap();
14 let graph_error = index.validate_current().unwrap_err();28 let document = Document::parse(&index).unwrap();
15 let mut checked = 0;29 document
16 for (osid, space) in &index.spaces {30 .spaces
17 let revision = index31 .iter()
18 .resolve(*osid, space.labels[&(ExGuid::default(), 1)])32 .find_map(|(space, view)| {
19 .unwrap();33 view.active()?.nodes.iter().find_map(|(id, node)| {
20 for oid in revision.reachable().unwrap() {34 matches!(&node.kind, Kind::RichText { text: stored, .. } if stored == text)
21 let object = &revision.objects[&oid];35 .then_some((*space, *id))
22 if object.jcid != 0x6000e {36 })
23 continue;37 })
24 }38 .unwrap()
25 let ObjectData::Properties(data) = object.data else {39}
26 continue;40
27 };41/// The text `object` holds in `image`, which must be valid.
28 for property in &PropertySets::parse(data).unwrap().sets[0] {42fn text_of(image: &[u8], space: ExGuid, object: ExGuid) -> String {
29 if property.value != Value::Bytes(b"Rust same paragraph.") {43 let store = Store::parse(image).unwrap();
30 continue;44 assert!(store.checksum_mismatches.is_empty());
31 }45 let index = RevisionIndex::parse(&store).unwrap();
32 let mut disk = Disk {46 index.validate_current().unwrap();
33 visible: source.clone(),47 let document = Document::parse(&index).unwrap();
34 durable: source.clone(),48 let Kind::RichText { text, .. } = &document.spaces[&space].active().unwrap().nodes[&object].kind
35 operation: 0,49 else {
36 fail_at: None,50 panic!()
37 write_limit: 1024,51 };
38 random: 1,52 text.clone()
39 };
40 let error = onestore::commit_property_bytes(
41 &mut disk,
42 &source,
43 *osid,
44 oid,
45 property.id,
46 b"Rust same paragraph.",
47 )
48 .unwrap_err();
49 assert_eq!(error.state, onestore::CommitState::NotCommitted);
50 assert_eq!(error.error.to_string(), graph_error.to_string());
51 assert_eq!(disk.operation, 0);
52 checked += 1;
53 }
54 }
55 }
56 assert_eq!(checked, 1);
57}53}
5854
59#[test]55#[test]
60fn repeated_appends_cross_log_fragments_and_expose_counter_tears() {56fn repeated_appends_cross_log_fragments_and_expose_counter_tears() {
61 let mut source =57 let initial =
62 fs::read("../../corpus/native/20260905-05/snapshots/02-text/notebook/synthetic.one")58 fs::read("../../corpus/native/20260905-05/snapshots/02-text/notebook/synthetic.one")
63 .unwrap();59 .unwrap();
64 let store = Store::parse(&source).unwrap();60 let store = Store::parse(&initial).unwrap();
65 let index = RevisionIndex::parse(&store).unwrap();61 let (space, object) = target(&initial, "Fictitious plain text.");
66 let mut target = None;
67 for (osid, space) in &index.spaces {
68 let revision = index
69 .resolve(*osid, space.labels[&(ExGuid::default(), 1)])
70 .unwrap();
71 for (oid, object) in revision.objects {
72 if let ObjectData::Properties(data) = object.data {
73 for property in &PropertySets::parse(data).unwrap().sets[0] {
74 if property.value == Value::Bytes(b"Fictitious plain text.") {
75 target = Some((*osid, oid, property.id));
76 }
77 }
78 }
79 }
80 }
81 let (osid, oid, property) = target.unwrap();
82 let root = store.header.root;62 let root = store.header.root;
83 let log = store.header.transaction_log;63 let log = store.header.transaction_log;
84 let initial_count = store.header.transaction_count;64 let initial_count = store.header.transaction_count;
65 let arena = Arena::default();
66 let mut section = Section::open(&arena, initial.clone()).unwrap();
67 let mut source = initial.clone();
68 let mut length = "Fictitious plain text.".encode_utf16().count() as u32;
85 let mut saw_counter_tear = false;69 let mut saw_counter_tear = false;
86 for transaction in initial_count + 1..=258 {70 for count in initial_count + 1..=258 {
87 let value = format!("Transaction {transaction}");71 let value = format!("Transaction {count}");
88 let written =72 let op = PageOp::Text {
89 replace_property_bytes(&source, osid, oid, property, value.as_bytes()).unwrap();73 text: object,
74 range: 0..length,
75 with: value.clone(),
76 };
77 length = value.encode_utf16().count() as u32;
78 section
79 .apply("Author", &Edit { at: 133_700_000_000_000_000 + u64::from(count), ops: vec![Op::Page { space, op }] })
80 .unwrap();
81 let transaction = section.seal().unwrap().unwrap();
82 let mut written = source.clone();
83 transaction.apply(&mut written).unwrap();
90 let store = Store::parse(&written).unwrap();84 let store = Store::parse(&written).unwrap();
91 assert_eq!(store.header.root, root);85 assert_eq!(store.header.root, root);
92 assert_eq!(store.header.transaction_log, log);86 assert_eq!(store.header.transaction_log, log);
93 assert_eq!(store.header.transaction_count, transaction);87 assert_eq!(store.header.transaction_count, count);
94 assert!(store.checksum_mismatches.is_empty());88 assert_eq!(text_of(&written, space, object), value);
95 RevisionIndex::parse(&store)
96 .unwrap()
97 .validate_current()
98 .unwrap();
99 let mut previous = written.clone();89 let mut previous = written.clone();
100 previous[..1024].copy_from_slice(&source[..1024]);90 previous[..1024].copy_from_slice(&source[..1024]);
101 let old = Store::parse(&previous).unwrap();91 let old = Store::parse(&previous).unwrap();
102 assert_eq!(old.header.transaction_count, transaction - 1);92 assert_eq!(old.header.transaction_count, count - 1);
103 assert!(old.checksum_mismatches.is_empty());93 assert!(old.checksum_mismatches.is_empty());
104 RevisionIndex::parse(&old)94 RevisionIndex::parse(&old)
105 .unwrap()95 .unwrap()
...@@ -110,17 +100,17 @@ fn repeated_appends_cross_log_fragments_and_expose_counter_tears() {...@@ -110,17 +100,17 @@ fn repeated_appends_cross_log_fragments_and_expose_counter_tears() {
110 torn[prefix..1024].copy_from_slice(&source[prefix..1024]);100 torn[prefix..1024].copy_from_slice(&source[prefix..1024]);
111 let result = Store::parse(&torn)101 let result = Store::parse(&torn)
112 .and_then(|store| RevisionIndex::parse(&store)?.validate_current());102 .and_then(|store| RevisionIndex::parse(&store)?.validate_current());
113 if transaction == 256 && prefix == 97 {103 if count == 256 && prefix == 97 {
114 assert!(result.is_err());104 assert!(result.is_err());
115 saw_counter_tear = true;105 saw_counter_tear = true;
116 } else {106 } else {
117 result.unwrap();107 result.unwrap();
118 }108 }
119 }109 }
120 if transaction == initial_count + 1 || transaction == 256 {110 if count == initial_count + 1 || count == 256 {
121 check_crashes(&source, osid, oid, property, value.as_bytes());111 check_crashes(&source, &transaction, space, object, &value);
122 }112 }
123 if transaction == initial_count + 1 {113 if count == initial_count + 1 {
124 let path = std::env::temp_dir().join(format!(114 let path = std::env::temp_dir().join(format!(
125 "onestore-commit-{}-{}.one",115 "onestore-commit-{}-{}.one",
126 std::process::id(),116 std::process::id(),
...@@ -130,61 +120,29 @@ fn repeated_appends_cross_log_fragments_and_expose_counter_tears() {...@@ -130,61 +120,29 @@ fn repeated_appends_cross_log_fragments_and_expose_counter_tears() {
130 .as_nanos()120 .as_nanos()
131 ));121 ));
132 fs::write(&path, &source).unwrap();122 fs::write(&path, &source).unwrap();
133 onestore::commit_file_property(&path, &source, osid, oid, property, value.as_bytes())123 transaction.commit_file(&path).unwrap();
134 .unwrap();
135 let committed = fs::read(&path).unwrap();124 let committed = fs::read(&path).unwrap();
136 let parsed = Store::parse(&committed).unwrap();125 assert_eq!(committed, written);
137 assert_eq!(parsed.header.transaction_count, transaction);126 // The file moved past the transaction's base: nothing is written again.
138 RevisionIndex::parse(&parsed)127 let stale = transaction.commit_file(&path).unwrap_err();
139 .unwrap()
140 .validate_current()
141 .unwrap();
142 let stale =
143 onestore::commit_file_property(&path, &source, osid, oid, property, b"stale")
144 .unwrap_err();
145 assert_eq!(stale.state, onestore::CommitState::NotCommitted);128 assert_eq!(stale.state, onestore::CommitState::NotCommitted);
146 assert_eq!(fs::read(&path).unwrap(), committed);129 assert_eq!(fs::read(&path).unwrap(), committed);
147 onestore::commit_file_property(
148 &path,
149 &committed,
150 osid,
151 oid,
152 property,
153 value.as_bytes(),
154 )
155 .unwrap();
156 assert_eq!(fs::read(&path).unwrap(), committed);
157 fs::remove_file(path).unwrap();130 fs::remove_file(path).unwrap();
158 }131 }
159 source = written;132 source = written;
160 }133 }
161 assert!(saw_counter_tear);134 assert!(saw_counter_tear);
135 assert!(section.image() == source);
162}136}
163137
164fn check_crashes(source: &[u8], osid: ExGuid, oid: ExGuid, property: u32, value: &[u8]) {138fn check_crashes(
165 let current_value = |data: &[u8]| {139 source: &[u8],
166 let store = Store::parse(data).unwrap();140 transaction: &Transaction,
167 assert!(store.checksum_mismatches.is_empty());141 space: ExGuid,
168 let index = RevisionIndex::parse(&store).unwrap();142 object: ExGuid,
169 index.validate_current().unwrap();143 value: &str,
170 let revision = index144) {
171 .resolve(osid, index.spaces[&osid].labels[&(ExGuid::default(), 1)])145 let before = text_of(source, space, object);
172 .unwrap();
173 let ObjectData::Properties(blob) = revision.objects[&oid].data else {
174 panic!()
175 };
176 let properties = PropertySets::parse(blob).unwrap();
177 let Value::Bytes(value) = properties.sets[0]
178 .iter()
179 .find(|p| p.id == property)
180 .unwrap()
181 .value
182 else {
183 panic!()
184 };
185 value.to_vec()
186 };
187 let before = current_value(source);
188 for limit in [17, 1024] {146 for limit in [17, 1024] {
189 let disk = |fail_at, random| Disk {147 let disk = |fail_at, random| Disk {
190 visible: source.to_vec(),148 visible: source.to_vec(),
...@@ -195,22 +153,13 @@ fn check_crashes(source: &[u8], osid: ExGuid, oid: ExGuid, property: u32, value:...@@ -195,22 +153,13 @@ fn check_crashes(source: &[u8], osid: ExGuid, oid: ExGuid, property: u32, value:
195 random,153 random,
196 };154 };
197 let mut successful = disk(None, 1);155 let mut successful = disk(None, 1);
198 onestore::commit_property_bytes(&mut successful, source, osid, oid, property, value)156 transaction.commit(&mut successful).unwrap();
199 .unwrap();157 assert_eq!(text_of(&successful.durable, space, object), value);
200 assert_eq!(current_value(&successful.durable), value);
201 for at in 1..=successful.operation {158 for at in 1..=successful.operation {
202 for seed in [0, 1, 42, u64::MAX] {159 for seed in [0, 1, 42, u64::MAX] {
203 let mut interrupted = disk(Some(at), seed);160 let mut interrupted = disk(Some(at), seed);
204 let failure = onestore::commit_property_bytes(161 let failure = transaction.commit(&mut interrupted).unwrap_err();
205 &mut interrupted,162 let persisted = text_of(&interrupted.durable, space, object);
206 source,
207 osid,
208 oid,
209 property,
210 value,
211 )
212 .unwrap_err();
213 let persisted = current_value(&interrupted.durable);
214 match failure.state {163 match failure.state {
215 onestore::CommitState::NotCommitted => assert_eq!(persisted, before),164 onestore::CommitState::NotCommitted => assert_eq!(persisted, before),
216 onestore::CommitState::Committed => assert_eq!(persisted, value),165 onestore::CommitState::Committed => assert_eq!(persisted, value),
...@@ -222,6 +171,7 @@ fn check_crashes(source: &[u8], osid: ExGuid, oid: ExGuid, property: u32, value:...@@ -222,6 +171,7 @@ fn check_crashes(source: &[u8], osid: ExGuid, oid: ExGuid, property: u32, value:
222 }171 }
223 }172 }
224}173}
174
225#[test]175#[test]
226#[cfg(any(unix, windows))]176#[cfg(any(unix, windows))]
227fn bounded_file_reads_reject_partial_images_and_release_the_owner() {177fn bounded_file_reads_reject_partial_images_and_release_the_owner() {
crates/onestore/tests/conflict.rs created+357
...@@ -0,0 +1,357 @@
1use onestore::{
2 Arena, ConflictPage, ExGuid, ObjectData, PageCreation, PropertySets, RevisionIndex, Section,
3 Store, Value,
4 op::{Edit, Op, PageOp, SectionOp},
5 page::{Page, PageObject},
6};
7use std::collections::BTreeMap;
8
9const AT: u64 = 134_030_000_000_000_000;
10
11fn read(path: &str) -> Vec<u8> {
12 std::fs::read(format!("{}/../../{path}", env!("CARGO_MANIFEST_DIR"))).unwrap()
13}
14
15/// Each object's jcid and root-set property identities, by space and object.
16fn objects(image: &[u8]) -> BTreeMap<ExGuid, BTreeMap<ExGuid, (u32, Vec<u32>)>> {
17 let store = Store::parse(image).unwrap();
18 let index = RevisionIndex::parse(&store).unwrap();
19 index
20 .spaces
21 .keys()
22 .map(|space| {
23 let revision = index.resolve_active(*space).unwrap();
24 let objects = revision
25 .objects
26 .iter()
27 .map(|(id, object)| {
28 let ids = match object.data {
29 ObjectData::Properties(bytes) => PropertySets::parse(bytes).unwrap().sets
30 [0]
31 .iter()
32 .map(|property| property.id)
33 .collect(),
34 _ => Vec::new(),
35 };
36 (*id, (object.jcid, ids))
37 })
38 .collect();
39 (*space, objects)
40 })
41 .collect()
42}
43
44/// The object-space references a property holds.
45fn spaces(image: &[u8], space: ExGuid, object: ExGuid, property: u32) -> usize {
46 let store = Store::parse(image).unwrap();
47 let index = RevisionIndex::parse(&store).unwrap();
48 let revision = index.resolve_active(space).unwrap();
49 let ObjectData::Properties(bytes) = revision.objects[&object].data else {
50 panic!()
51 };
52 PropertySets::parse(bytes).unwrap().sets[0]
53 .iter()
54 .find(|field| field.id == property)
55 .map_or(0, |field| match &field.value {
56 Value::References { compact_ids, .. } => compact_ids.len() / 4,
57 _ => panic!(),
58 })
59}
60
61fn texts(page: &Page) -> Vec<(ExGuid, String)> {
62 page.objects
63 .iter()
64 .flat_map(|object| match object {
65 PageObject::Outline(outline) => outline.paragraphs.clone(),
66 PageObject::Title(title) => title
67 .outlines
68 .iter()
69 .flat_map(|outline| outline.paragraphs.clone())
70 .collect(),
71 _ => Vec::new(),
72 })
73 .filter_map(|paragraph| {
74 paragraph
75 .text()
76 .map(|text| (text.id, text.text.text().to_owned()))
77 })
78 .collect()
79}
80
81#[test]
82fn native_conflict_pages_are_listed_under_their_page_last_first() {
83 let arena = Arena::default();
84 let mut section = Section::open(
85 &arena,
86 read("corpus/collaboration/round-01/offline/notebook/synthetic.one"),
87 )
88 .unwrap();
89 let pages = section.pages().unwrap();
90 let conflicts = section.conflicts().unwrap();
91 assert_eq!(conflicts.len(), 1);
92 let (page, listed) = &conflicts[0];
93 assert_eq!(*page, pages[0].0);
94 let [ConflictPage {
95 space,
96 title,
97 user,
98 created,
99 objects,
100 }] = listed.as_slice()
101 else {
102 panic!("{listed:?}")
103 };
104 assert_eq!(
105 (title.as_str(), user.as_str(), *created),
106 (
107 "Native edited while disconnected.",
108 "snow",
109 Some(0x01dd3d10d7afd9e0)
110 )
111 );
112 let conflict = section.page(*space).unwrap();
113 let marked: Vec<_> = texts(&conflict)
114 .into_iter()
115 .filter(|(id, _)| objects.contains(id))
116 .map(|(_, text)| text)
117 .collect();
118 assert_eq!(marked, ["Native edited while disconnected."]);
119
120 // Three clients' conflict pages: OneNote lists the one stored last first.
121 let arena = Arena::default();
122 let mut section = Section::open(
123 &arena,
124 read("corpus/conflict-page/native-three/notebook/synthetic.one"),
125 )
126 .unwrap();
127 let users: Vec<_> = section.conflicts().unwrap()[0]
128 .1
129 .iter()
130 .map(|page| page.user.clone())
131 .collect();
132 assert_eq!(
133 users,
134 ["ONE-M6-053BD357", "ONE-M6-9451B6C6", "ONE-M6-DDE10E0D"]
135 );
136}
137
138/// The conflict page a merge makes of `local`, the version of `space`'s page it could not
139/// take, marking the text object `conflicting`.
140fn conflict(space: ExGuid, local: &Page, conflicting: ExGuid, user: &str) -> Op {
141 let titled = local
142 .objects
143 .iter()
144 .any(|object| matches!(object, PageObject::Title(_)));
145 let mut objects = [conflicting];
146 let page = local.copy_with(&mut objects).unwrap();
147 Op::Section(SectionOp::Conflict {
148 of: space,
149 creation: PageCreation::new(None, titled.then_some(local.title.as_str()), user).unwrap(),
150 page,
151 objects: objects.to_vec(),
152 })
153}
154
155/// A conflict page written through ops stores what OneNote 2010 writes for one
156/// (`corpus/collaboration/round-01/offline`): its own space under the page's manifest, with
157/// conflict metadata, a read-only deletable conflict page and the conflicting text marked.
158/// `ONESTORE_CONFLICT_EXPORT` names a directory to write the notebook to, for a cold read.
159#[test]
160fn a_conflict_page_is_stored_as_onenote_stores_one() {
161 let notebook = "corpus/native-ink/cold-ui-ink/notebook";
162 let source = read(&format!("{notebook}/synthetic.one"));
163 let arena = Arena::default();
164 let mut section = Section::open(&arena, source.clone()).unwrap();
165 let space = section.pages().unwrap()[0].0;
166 let base = section.page(space).unwrap();
167 let (text, _) = texts(&base)
168 .into_iter()
169 .find(|(_, text)| text.starts_with("Fictitious: café"))
170 .unwrap();
171 // Both machines renamed the paragraph's first word, within its Latin run.
172 let replace = |with: &str| Edit {
173 at: AT,
174 ops: vec![Op::Page {
175 space,
176 op: PageOp::Text {
177 text,
178 range: 0..10,
179 with: with.into(),
180 },
181 }],
182 };
183 section.apply("Remote", &replace("Remote")).unwrap();
184 let scratch = Arena::default();
185 let mut local = Section::open(&scratch, source).unwrap();
186 local.apply("Clover Snow", &replace("Snowbound")).unwrap();
187 let local = local.page(space).unwrap();
188 let op = conflict(space, &local, text, "Clover Snow");
189 let Op::Section(SectionOp::Conflict { creation, .. }) = &op else {
190 unreachable!()
191 };
192 let created = creation.space();
193 section
194 .apply(
195 "Clover Snow",
196 &Edit {
197 at: AT + 10_000_000,
198 ops: vec![op.clone()],
199 },
200 )
201 .unwrap();
202 section.seal().unwrap();
203 let image = section.image();
204
205 let arena = Arena::default();
206 let mut reopened = Section::open(&arena, image.clone()).unwrap();
207 assert_eq!(reopened.pages().unwrap().len(), 1);
208 let listed = reopened.conflicts().unwrap();
209 assert_eq!(listed.len(), 1);
210 assert_eq!(listed[0].0, space);
211 assert_eq!(
212 listed[0].1,
213 [ConflictPage {
214 space: created,
215 title: "Snowbound: café, 東京, مرحبا".into(),
216 user: "Clover Snow".into(),
217 created: listed[0].1[0].created,
218 objects: listed[0].1[0].objects.clone(),
219 }]
220 );
221 let kept = reopened.page(created).unwrap();
222 let marked: Vec<_> = texts(&kept)
223 .into_iter()
224 .filter(|(id, _)| listed[0].1[0].objects.contains(id))
225 .map(|(_, text)| text)
226 .collect();
227 assert_eq!(marked, ["Snowbound: café, 東京, مرحبا"]);
228 assert!(
229 texts(&reopened.page(space).unwrap())
230 .iter()
231 .any(|(_, text)| text == "Remote: café, 東京, مرحبا")
232 );
233
234 // The stored form, property by property, as OneNote writes it.
235 let stored = objects(&image);
236 let conflict_space = &stored[&created];
237 let store = Store::parse(&image).unwrap();
238 let index = RevisionIndex::parse(&store).unwrap();
239 let roots = index.resolve_active(created).unwrap().roots;
240 let (jcid, metadata) = &conflict_space[&roots[&2]];
241 assert_eq!(*jcid, 0x20038);
242 assert_eq!(
243 metadata,
244 &[
245 0x1c001cf3, 0x1c001c30, 0x14001dff, 0x14001d82, 0x1400348b, 0x18001c65, 0x1c001d9e,
246 0x1c001d9f
247 ]
248 );
249 let page_node = conflict_space
250 .values()
251 .find(|(jcid, _)| *jcid == 0x6000b)
252 .unwrap();
253 for flag in [0x88001cde, 0x88001d0c, 0x88001d7c] {
254 assert!(page_node.1.contains(&flag), "{flag:#x}");
255 }
256 let marked: Vec<_> = conflict_space
257 .iter()
258 .filter(|(_, (_, ids))| ids.contains(&0x88001d96))
259 .map(|(_, (jcid, ids))| (*jcid, ids.contains(&0x88001ddb)))
260 .collect();
261 assert_eq!(marked, [(0x6000e, true)]);
262 let main = &stored[&space];
263 let main_roots = index.resolve_active(space).unwrap().roots;
264 assert_eq!(spaces(&image, space, main_roots[&1], 0x2c001d63), 1);
265 assert!(main[&main_roots[&2]].1.contains(&0x88001d97));
266 assert_eq!(
267 main.values()
268 .filter(|(jcid, ids)| *jcid == 0x20038 && ids.contains(&0x1c001d9e))
269 .count(),
270 1
271 );
272 assert_eq!(
273 stored[&index.root]
274 .values()
275 .filter(|(jcid, ids)| *jcid == 0x20030 && ids.contains(&0x88001d97))
276 .count(),
277 1
278 );
279
280 if let Some(output) = std::env::var_os("ONESTORE_CONFLICT_EXPORT") {
281 let output = std::path::Path::new(&output);
282 std::fs::create_dir_all(output).unwrap();
283 std::fs::write(output.join("synthetic.one"), &image).unwrap();
284 std::fs::copy(
285 format!("{}/../../{notebook}/Open Notebook.onetoc2", env!("CARGO_MANIFEST_DIR")),
286 output.join("Open Notebook.onetoc2"),
287 )
288 .unwrap();
289 }
290}
291
292/// Deleting a conflict page stores what OneNote 2010's Delete Conflict Page does
293/// (`corpus/conflict-page/native-delete`): the page's manifest stops listing the conflict
294/// page, keeping its metadata copy, and the page and the section's copy of its metadata no
295/// longer say it has conflict pages; the conflict page's space is left alone.
296/// `ONESTORE_CONFLICT_DELETE_EXPORT` names a directory to write the notebook to.
297#[test]
298fn deleting_a_conflict_page_stores_what_onenote_stores() {
299 let notebook = "corpus/conflict-page/native/conflict/notebook";
300 let arena = Arena::default();
301 let mut section = Section::open(&arena, read(&format!("{notebook}/synthetic.one"))).unwrap();
302 let (page, listed) = section.conflicts().unwrap().remove(0);
303 let conflict = listed[0].space;
304 section
305 .apply(
306 "Clover Snow",
307 &Edit {
308 at: AT,
309 ops: vec![Op::Section(SectionOp::Delete(vec![conflict]))],
310 },
311 )
312 .unwrap();
313 section.seal().unwrap();
314 let image = section.image();
315 let arena = Arena::default();
316 let mut reopened = Section::open(&arena, image.clone()).unwrap();
317 assert!(reopened.conflicts().unwrap().is_empty());
318 assert_eq!(reopened.pages().unwrap().len(), 1);
319
320 // Each space's objects and their property identities, as OneNote left them.
321 let native = read("corpus/conflict-page/native-delete/notebook/synthetic.one");
322 let shapes = |image: &[u8]| {
323 let store = Store::parse(image).unwrap();
324 let index = RevisionIndex::parse(&store).unwrap();
325 let root = index.root;
326 let roots = index.resolve_active(page).unwrap().roots;
327 let objects = objects(image);
328 let manifest = objects[&page][&roots[&1]].clone();
329 let metadata = objects[&page][&roots[&2]].clone();
330 let flagged = objects[&root]
331 .values()
332 .filter(|(_, ids)| ids.contains(&0x88001d97))
333 .count();
334 (manifest, metadata, flagged, objects[&conflict].clone())
335 };
336 let (manifest, metadata, flagged, space) = shapes(&image);
337 let (native_manifest, native_metadata, native_flagged, native_space) = shapes(&native);
338 assert_eq!(manifest.1.iter().filter(|id| **id != 0x3400347b).collect::<Vec<_>>(), [&0x24001c1f, &0x24003442]);
339 assert_eq!(
340 native_manifest.1.iter().filter(|id| **id != 0x3400347b).collect::<Vec<_>>(),
341 [&0x24001c1f, &0x24003442]
342 );
343 assert!(!metadata.1.contains(&0x88001d97) && !native_metadata.1.contains(&0x88001d97));
344 assert_eq!((flagged, native_flagged), (0, 0));
345 assert_eq!(space, native_space);
346
347 if let Some(output) = std::env::var_os("ONESTORE_CONFLICT_DELETE_EXPORT") {
348 let output = std::path::Path::new(&output);
349 std::fs::create_dir_all(output).unwrap();
350 std::fs::write(output.join("synthetic.one"), &image).unwrap();
351 std::fs::copy(
352 format!("{}/../../{notebook}/Open Notebook.onetoc2", env!("CARGO_MANIFEST_DIR")),
353 output.join("Open Notebook.onetoc2"),
354 )
355 .unwrap();
356 }
357}
crates/onestore/tests/document.rs+21-16
...@@ -482,10 +482,6 @@ fn rust_unicode_and_native_latin1_use_utf16_run_offsets() {...@@ -482,10 +482,6 @@ fn rust_unicode_and_native_latin1_use_utf16_run_offsets() {
482 };482 };
483 assert_eq!(actual, text);483 assert_eq!(actual, text);
484 assert_eq!(runs[0].end, 7);484 assert_eq!(runs[0].end, 7);
485 let bytes = fs::read("../../corpus/m6/ansi/notebook/synthetic.one").unwrap();
486 let store = Store::parse(&bytes).unwrap();
487 let index = RevisionIndex::parse(&store).unwrap();
488 let document = Document::parse(&index).unwrap();
489 let expected = format!(485 let expected = format!(
490 "Extended bytes: {}",486 "Extended bytes: {}",
491 (128u8..=255)487 (128u8..=255)
...@@ -493,14 +489,23 @@ fn rust_unicode_and_native_latin1_use_utf16_run_offsets() {...@@ -493,14 +489,23 @@ fn rust_unicode_and_native_latin1_use_utf16_run_offsets() {
493 .collect::<Vec<_>>()489 .collect::<Vec<_>>()
494 .join(" ")490 .join(" ")
495 );491 );
496 assert!(492 // OneNote 2010 reads TextExtendedAscii as Latin-1 whatever the run's language
497 document493 // (corpus/extended-ascii/{en,ru,ja}/cold).
498 .spaces494 for language in ["en", "ru", "ja"] {
499 .values()495 let path = format!("../../corpus/extended-ascii/{language}/candidate/synthetic.one");
500 .flat_map(|s| s.revisions.values())496 let bytes = fs::read(path).unwrap();
501 .flat_map(|r| r.nodes.values())497 let store = Store::parse(&bytes).unwrap();
502 .any(|n| matches!(&n.kind, Kind::RichText { text, .. } if text == &expected))498 let index = RevisionIndex::parse(&store).unwrap();
503 );499 let document = Document::parse(&index).unwrap();
500 assert!(
501 document
502 .spaces
503 .values()
504 .flat_map(|s| s.revisions.values())
505 .flat_map(|r| r.nodes.values())
506 .any(|n| matches!(&n.kind, Kind::RichText { text, .. } if text == &expected))
507 );
508 }
504}509}
505510
506#[test]511#[test]
...@@ -717,11 +722,11 @@ fn malformed_run_boundaries_fail_after_valid_storage_decoding() {...@@ -717,11 +722,11 @@ fn malformed_run_boundaries_fail_after_valid_storage_decoding() {
717 else {722 else {
718 unreachable!()723 unreachable!()
719 };724 };
725 // Damage the stored boundaries where they lie, as no writer would store them.
726 let at = boundaries.as_ptr().addr() - bytes.as_ptr().addr();
720 for invalid in [5u32, u32::MAX] {727 for invalid in [5u32, u32::MAX] {
721 let mut changed = boundaries.to_vec();728 let mut mutated = bytes.to_vec();
722 changed[..4].copy_from_slice(&invalid.to_le_bytes());729 mutated[at..at + 4].copy_from_slice(&invalid.to_le_bytes());
723 let mutated =
724 onestore::replace_property_bytes(&bytes, sid, oid, 0x1c001e12, &changed).unwrap();
725 let store = Store::parse(&mutated).unwrap();730 let store = Store::parse(&mutated).unwrap();
726 let index = RevisionIndex::parse(&store).unwrap();731 let index = RevisionIndex::parse(&store).unwrap();
727 index.validate_current().unwrap();732 index.validate_current().unwrap();
crates/onestore/tests/edit.rs+118-58
...@@ -2,14 +2,18 @@...@@ -2,14 +2,18 @@
2mod checkpoint;2mod checkpoint;
3#[path = "support/disk.rs"]3#[path = "support/disk.rs"]
4mod disk;4mod disk;
5#[path = "support/ops.rs"]
6mod ops;
5#[path = "support/trace.rs"]7#[path = "support/trace.rs"]
6mod trace;8mod trace;
79
8use disk::Disk;10use disk::Disk;
9use onestore::{11use onestore::{
10 CommitState, ExGuid, RevisionIndex, Store,12 CommitError, CommitIo, CommitState, ExGuid, RevisionIndex, Store, Transaction,
11 document::{Document, Kind},13 document::{Document, Kind},
14 op::{Op, OpError, PageOp},
12};15};
16use std::ops::Range;
1317
14const SOURCE: &[u8] = include_bytes!(18const SOURCE: &[u8] = include_bytes!(
15 "../../../corpus/native/20260905-05/snapshots/03-format-unicode/notebook/synthetic.one"19 "../../../corpus/native/20260905-05/snapshots/03-format-unicode/notebook/synthetic.one"
...@@ -31,6 +35,55 @@ fn target(source: &[u8]) -> (ExGuid, ExGuid) {...@@ -31,6 +35,55 @@ fn target(source: &[u8]) -> (ExGuid, ExGuid) {
31 panic!("Missing native text fixture")35 panic!("Missing native text fixture")
32}36}
3337
38fn typed(space: ExGuid, text: ExGuid, range: Range<u32>, with: &str) -> Vec<Op> {
39 let op = PageOp::Text {
40 text,
41 range,
42 with: with.into(),
43 };
44 vec![Op::Page { space, op }]
45}
46
47/// `source` with `range` of the text object replaced.
48fn replace_text(
49 source: &[u8],
50 space: ExGuid,
51 text: ExGuid,
52 range: Range<u32>,
53 with: &str,
54) -> Result<Vec<u8>, OpError> {
55 ops::edited(source, typed(space, text, range, with))
56}
57
58/// The transaction replacing `range` of the text object in `source`.
59fn text_transaction(
60 source: &[u8],
61 space: ExGuid,
62 text: ExGuid,
63 range: Range<u32>,
64 with: &str,
65) -> Result<Transaction, CommitError> {
66 let refused = |error: String| CommitError {
67 state: CommitState::NotCommitted,
68 error: std::io::Error::new(std::io::ErrorKind::InvalidData, error),
69 };
70 ops::transaction(source, "Author", typed(space, text, range, with))
71 .map_err(|error| refused(error.to_string()))?
72 .ok_or_else(|| refused("The edit changes nothing".into()))
73}
74
75/// `text_transaction` committed under caller-held exclusion.
76fn commit_text(
77 io: &mut impl CommitIo,
78 source: &[u8],
79 space: ExGuid,
80 text: ExGuid,
81 range: Range<u32>,
82 with: &str,
83) -> Result<(), CommitError> {
84 text_transaction(source, space, text, range, with)?.commit(io)
85}
86
34fn text_runs(source: &[u8], sid: ExGuid, oid: ExGuid) -> serde_json::Value {87fn text_runs(source: &[u8], sid: ExGuid, oid: ExGuid) -> serde_json::Value {
35 let store = Store::parse(source).unwrap();88 let store = Store::parse(source).unwrap();
36 assert!(store.checksum_mismatches.is_empty());89 assert!(store.checksum_mismatches.is_empty());
...@@ -61,9 +114,11 @@ fn prepared_publication_preserves_its_identity_through_every_io_failure() {...@@ -61,9 +114,11 @@ fn prepared_publication_preserves_its_identity_through_every_io_failure() {
61 let source =114 let source =
62 onestore::create_section("prepared.one", "Fictitious: café 🦀", "Fixture").unwrap();115 onestore::create_section("prepared.one", "Fictitious: café 🦀", "Fixture").unwrap();
63 let (sid, oid) = target(&source);116 let (sid, oid) = target(&source);
64 let source = checkpoint::pending(&source, sid, oid, 0x14001d7a);117 let source = checkpoint::pending(&source, sid, oid);
65 let edit = onestore::PreparedEdit::text(&source, sid, oid, 0..0, "Prepared 🐈 ").unwrap();118 let edit = text_transaction(&source, sid, oid, 0..0, "Prepared 🐈 ").unwrap();
66 let store = Store::parse(edit.as_bytes()).unwrap();119 let mut written = source.clone();
120 edit.apply(&mut written).unwrap();
121 let store = Store::parse(&written).unwrap();
67 let index = RevisionIndex::parse(&store).unwrap();122 let index = RevisionIndex::parse(&store).unwrap();
68 let planned = index.spaces[&sid].labels[&(ExGuid::default(), 1)];123 let planned = index.spaces[&sid].labels[&(ExGuid::default(), 1)];
69 let persisted: ExGuid =124 let persisted: ExGuid =
...@@ -72,7 +127,7 @@ fn prepared_publication_preserves_its_identity_through_every_io_failure() {...@@ -72,7 +127,7 @@ fn prepared_publication_preserves_its_identity_through_every_io_failure() {
72 let before_index = RevisionIndex::parse(&before_store).unwrap();127 let before_index = RevisionIndex::parse(&before_store).unwrap();
73 assert!(!before_index.spaces[&sid].revisions.contains_key(&persisted));128 assert!(!before_index.spaces[&sid].revisions.contains_key(&persisted));
74 let before = text_runs(&source, sid, oid);129 let before = text_runs(&source, sid, oid);
75 let after = text_runs(edit.as_bytes(), sid, oid);130 let after = text_runs(&written, sid, oid);
76 for write_limit in [17, 4096] {131 for write_limit in [17, 4096] {
77 let disk = |fail_at| Disk {132 let disk = |fail_at| Disk {
78 visible: source.clone(),133 visible: source.clone(),
...@@ -84,12 +139,12 @@ fn prepared_publication_preserves_its_identity_through_every_io_failure() {...@@ -84,12 +139,12 @@ fn prepared_publication_preserves_its_identity_through_every_io_failure() {
84 };139 };
85 let mut success = disk(None);140 let mut success = disk(None);
86 edit.commit(&mut success).unwrap();141 edit.commit(&mut success).unwrap();
87 assert_eq!(success.durable, edit.as_bytes());142 assert_eq!(success.durable, written);
88 let operations = success.operation;143 let operations = success.operation;
89 let error = edit.commit(&mut success).unwrap_err();144 let error = edit.commit(&mut success).unwrap_err();
90 assert_eq!(error.state, CommitState::NotCommitted);145 assert_eq!(error.state, CommitState::NotCommitted);
91 assert_eq!(error.error.kind(), std::io::ErrorKind::ResourceBusy);146 assert_eq!(error.error.kind(), std::io::ErrorKind::ResourceBusy);
92 assert_eq!(success.durable, edit.as_bytes());147 assert_eq!(success.durable, written);
93 for at in 1..=operations {148 for at in 1..=operations {
94 let mut interrupted = disk(Some(at));149 let mut interrupted = disk(Some(at));
95 let error = edit.commit(&mut interrupted).unwrap_err();150 let error = edit.commit(&mut interrupted).unwrap_err();
...@@ -108,7 +163,8 @@ fn prepared_publication_preserves_its_identity_through_every_io_failure() {...@@ -108,7 +163,8 @@ fn prepared_publication_preserves_its_identity_through_every_io_failure() {
108 interrupted.visible.clone_from(&snapshot);163 interrupted.visible.clone_from(&snapshot);
109 interrupted.fail_at = None;164 interrupted.fail_at = None;
110 interrupted.write_limit = 17;165 interrupted.write_limit = 17;
111 onestore::confirm_snapshot(&mut interrupted, &snapshot).unwrap();166 onestore::confirm(&mut interrupted, &onestore::Stamp::of(&snapshot).unwrap())
167 .unwrap();
112 assert_refreshed(&snapshot, &interrupted.durable);168 assert_refreshed(&snapshot, &interrupted.durable);
113 }169 }
114 }170 }
...@@ -126,9 +182,9 @@ fn snapshot_confirmation_needs_no_surviving_edit_target() {...@@ -126,9 +182,9 @@ fn snapshot_confirmation_needs_no_surviving_edit_target() {
126 random: 911,182 random: 911,
127 };183 };
128 assert!(184 assert!(
129 onestore::replace_text(SOURCE, ExGuid::default(), ExGuid::default(), 0..0, "").is_err()185 replace_text(SOURCE, ExGuid::default(), ExGuid::default(), 0..0, "").is_err()
130 );186 );
131 onestore::confirm_snapshot(&mut disk, SOURCE).unwrap();187 onestore::confirm(&mut disk, &onestore::Stamp::of(SOURCE).unwrap()).unwrap();
132 assert_refreshed(SOURCE, &disk.durable);188 assert_refreshed(SOURCE, &disk.durable);
133 let flush = disk.operation;189 let flush = disk.operation;
134 for (failure, state) in [190 for (failure, state) in [
...@@ -140,7 +196,7 @@ fn snapshot_confirmation_needs_no_surviving_edit_target() {...@@ -140,7 +196,7 @@ fn snapshot_confirmation_needs_no_surviving_edit_target() {
140 disk.operation = 0;196 disk.operation = 0;
141 disk.fail_at = Some(failure);197 disk.fail_at = Some(failure);
142 assert_eq!(198 assert_eq!(
143 onestore::confirm_snapshot(&mut disk, SOURCE)199 onestore::confirm(&mut disk, &onestore::Stamp::of(SOURCE).unwrap())
144 .unwrap_err()200 .unwrap_err()
145 .state,201 .state,
146 state202 state
...@@ -153,7 +209,7 @@ fn confirmation_notifies_cached_readers_after_interrupted_version_publication()...@@ -153,7 +209,7 @@ fn confirmation_notifies_cached_readers_after_interrupted_version_publication()
153 let source =209 let source =
154 onestore::create_section("confirmation.one", "Fictitious: before", "Fixture").unwrap();210 onestore::create_section("confirmation.one", "Fictitious: before", "Fixture").unwrap();
155 let (sid, oid) = target(&source);211 let (sid, oid) = target(&source);
156 let mut snapshot = onestore::replace_text(&source, sid, oid, 0..0, "Recovered ").unwrap();212 let mut snapshot = replace_text(&source, sid, oid, 0..0, "Recovered ").unwrap();
157 snapshot[212..252].copy_from_slice(&source[212..252]);213 snapshot[212..252].copy_from_slice(&source[212..252]);
158 let expected = text_runs(&snapshot, sid, oid);214 let expected = text_runs(&snapshot, sid, oid);
159 assert_ne!(expected, text_runs(&source, sid, oid));215 assert_ne!(expected, text_runs(&source, sid, oid));
...@@ -167,11 +223,13 @@ fn confirmation_notifies_cached_readers_after_interrupted_version_publication()...@@ -167,11 +223,13 @@ fn confirmation_notifies_cached_readers_after_interrupted_version_publication()
167 random: 911,223 random: 911,
168 };224 };
169 let mut success = disk(None);225 let mut success = disk(None);
170 onestore::confirm_snapshot(&mut success, &snapshot).unwrap();226 onestore::confirm(&mut success, &onestore::Stamp::of(&snapshot).unwrap()).unwrap();
171 assert_refreshed(&snapshot, &success.durable);227 assert_refreshed(&snapshot, &success.durable);
172 for failure in 1..=success.operation {228 for failure in 1..=success.operation {
173 let mut interrupted = disk(Some(failure));229 let mut interrupted = disk(Some(failure));
174 let error = onestore::confirm_snapshot(&mut interrupted, &snapshot).unwrap_err();230 let error =
231 onestore::confirm(&mut interrupted, &onestore::Stamp::of(&snapshot).unwrap())
232 .unwrap_err();
175 assert_ne!(error.state, CommitState::Committed);233 assert_ne!(error.state, CommitState::Committed);
176 assert_eq!(text_runs(&interrupted.durable, sid, oid), expected);234 assert_eq!(text_runs(&interrupted.durable, sid, oid), expected);
177 assert_eq!(&interrupted.durable[..212], &snapshot[..212]);235 assert_eq!(&interrupted.durable[..212], &snapshot[..212]);
...@@ -183,51 +241,53 @@ fn confirmation_notifies_cached_readers_after_interrupted_version_publication()...@@ -183,51 +241,53 @@ fn confirmation_notifies_cached_readers_after_interrupted_version_publication()
183#[test]241#[test]
184fn confirming_visible_text_requires_flush_without_another_revision() {242fn confirming_visible_text_requires_flush_without_another_revision() {
185 let (sid, oid) = target(SOURCE);243 let (sid, oid) = target(SOURCE);
186 let visible = onestore::replace_text(SOURCE, sid, oid, 0..0, "Recovered ").unwrap();244 let visible = replace_text(SOURCE, sid, oid, 0..0, "Recovered ").unwrap();
245 let base = onestore::Stamp::of(&visible).unwrap();
187 let mut disk = Disk {246 let mut disk = Disk {
188 visible: visible.clone(),247 visible: visible.clone(),
189 durable: SOURCE.to_vec(),248 durable: SOURCE.to_vec(),
190 operation: 0,249 operation: 0,
191 fail_at: None,250 fail_at: None,
192 write_limit: 0,251 write_limit: 64,
193 random: 1,252 random: 1,
194 };253 };
195 onestore::commit_text(&mut disk, &visible, sid, oid, 0..0, "").unwrap();254 onestore::confirm(&mut disk, &base).unwrap();
196 assert_eq!(disk.visible, visible);255 assert_refreshed(&visible, &disk.durable);
197 assert_eq!(disk.durable, visible);
198 let flush = disk.operation;256 let flush = disk.operation;
199 for (failure, state) in [257 for (failure, state) in [
200 (1, CommitState::NotCommitted),258 (1, CommitState::NotCommitted),
201 (flush, CommitState::Unknown),259 (flush, CommitState::Unknown),
202 ] {260 ] {
261 disk.visible.clone_from(&visible);
203 disk.durable = SOURCE.to_vec();262 disk.durable = SOURCE.to_vec();
204 disk.operation = 0;263 disk.operation = 0;
205 disk.fail_at = Some(failure);264 disk.fail_at = Some(failure);
206 let error = onestore::commit_text(&mut disk, &visible, sid, oid, 0..0, "").unwrap_err();265 let error = onestore::confirm(&mut disk, &base).unwrap_err();
207 assert_eq!(error.state, state);266 assert_eq!(error.state, state);
208 assert_eq!(disk.visible, visible);267 assert_eq!(&disk.visible[..212], &visible[..212]);
268 assert_eq!(&disk.visible[252..], &visible[252..]);
209 }269 }
210}270}
211271
212#[test]272#[test]
213fn confirmation_rejects_a_changed_header_or_length_before_any_write() {273fn confirmation_rejects_a_changed_header_or_length_before_any_write() {
214 let (sid, oid) = target(SOURCE);
215 let mut source = SOURCE.to_vec();274 let mut source = SOURCE.to_vec();
216 source.resize(3 * 1024 * 1024 + 131, 0);275 source.resize(3 * 1024 * 1024 + 131, 0);
276 let base = onestore::Stamp::of(&source).unwrap();
217 let mut disk = trace::Trace {277 let mut disk = trace::Trace {
218 bytes: source.clone(),278 bytes: source.clone(),
219 events: Vec::new(),279 events: Vec::new(),
220 };280 };
221 onestore::commit_text(&mut disk, &source, sid, oid, 0..0, "").unwrap();281 let confirmed = |events: &[trace::Event]| {
222 assert_eq!(disk.events.len(), 1);282 matches!(events, [trace::Event::Flush, trace::Event::Write(212, version), trace::Event::Flush] if version.len() == 40)
223 if let trace::Event::Write(offset, bytes) = &disk.events[0] {283 };
224 panic!("Confirmation wrote {} bytes at {offset}", bytes.len());284 onestore::confirm(&mut disk, &base).unwrap();
225 }285 assert!(confirmed(&disk.events));
226 for changed in [96, 128, 212, 1023] {286 for changed in [96, 128, 212, 1023] {
227 disk.bytes.clone_from(&source);287 disk.bytes.clone_from(&source);
228 disk.bytes[changed] ^= 1;288 disk.bytes[changed] ^= 1;
229 disk.events.clear();289 disk.events.clear();
230 let error = onestore::commit_text(&mut disk, &source, sid, oid, 0..0, "").unwrap_err();290 let error = onestore::confirm(&mut disk, &base).unwrap_err();
231 assert_eq!(error.state, CommitState::NotCommitted);291 assert_eq!(error.state, CommitState::NotCommitted);
232 assert_eq!(error.error.kind(), std::io::ErrorKind::ResourceBusy);292 assert_eq!(error.error.kind(), std::io::ErrorKind::ResourceBusy);
233 assert!(disk.events.is_empty());293 assert!(disk.events.is_empty());
...@@ -237,14 +297,14 @@ fn confirmation_rejects_a_changed_header_or_length_before_any_write() {...@@ -237,14 +297,14 @@ fn confirmation_rejects_a_changed_header_or_length_before_any_write() {
237 disk.bytes.clone_from(&source);297 disk.bytes.clone_from(&source);
238 disk.bytes[changed] ^= 1;298 disk.bytes[changed] ^= 1;
239 disk.events.clear();299 disk.events.clear();
240 onestore::commit_text(&mut disk, &source, sid, oid, 0..0, "").unwrap();300 onestore::confirm(&mut disk, &base).unwrap();
241 assert!(matches!(disk.events[..], [trace::Event::Flush]));301 assert!(confirmed(&disk.events));
242 }302 }
243 for length in [source.len() - 1, source.len() + 1] {303 for length in [source.len() - 1, source.len() + 1] {
244 disk.bytes.clone_from(&source);304 disk.bytes.clone_from(&source);
245 disk.bytes.resize(length, 0);305 disk.bytes.resize(length, 0);
246 disk.events.clear();306 disk.events.clear();
247 let error = onestore::commit_text(&mut disk, &source, sid, oid, 0..0, "").unwrap_err();307 let error = onestore::confirm(&mut disk, &base).unwrap_err();
248 assert_eq!(error.state, CommitState::NotCommitted);308 assert_eq!(error.state, CommitState::NotCommitted);
249 assert!(disk.events.is_empty());309 assert!(disk.events.is_empty());
250 }310 }
...@@ -332,7 +392,7 @@ fn text_splices_preserve_formats_objects_and_history_in_one_transaction() {...@@ -332,7 +392,7 @@ fn text_splices_preserve_formats_objects_and_history_in_one_transaction() {
332 let old_index = RevisionIndex::parse(&old).unwrap();392 let old_index = RevisionIndex::parse(&old).unwrap();
333 let before = text_runs(SOURCE, sid, oid);393 let before = text_runs(SOURCE, sid, oid);
334 for replacement in ["", "🦀", "A longer replacement with 日本語 and e\u{301}"] {394 for replacement in ["", "🦀", "A longer replacement with 日本語 and e\u{301}"] {
335 let edited = onestore::replace_text(SOURCE, sid, oid, 0..10, replacement).unwrap();395 let edited = replace_text(SOURCE, sid, oid, 0..10, replacement).unwrap();
336 let store = Store::parse(&edited).unwrap();396 let store = Store::parse(&edited).unwrap();
337 assert_eq!(397 assert_eq!(
338 store.header.transaction_count,398 store.header.transaction_count,
...@@ -379,7 +439,7 @@ fn text_splices_preserve_formats_objects_and_history_in_one_transaction() {...@@ -379,7 +439,7 @@ fn text_splices_preserve_formats_objects_and_history_in_one_transaction() {
379#[test]439#[test]
380fn invalid_text_edits_never_touch_storage() {440fn invalid_text_edits_never_touch_storage() {
381 let (sid, oid) = target(SOURCE);441 let (sid, oid) = target(SOURCE);
382 let source = onestore::replace_text(SOURCE, sid, oid, 0..0, "🦀").unwrap();442 let source = replace_text(SOURCE, sid, oid, 0..0, "🦀").unwrap();
383 for (range, replacement) in [443 for (range, replacement) in [
384 (1..1, "x"),444 (1..1, "x"),
385 (0..999, "x"),445 (0..999, "x"),
...@@ -396,7 +456,7 @@ fn invalid_text_edits_never_touch_storage() {...@@ -396,7 +456,7 @@ fn invalid_text_edits_never_touch_storage() {
396 random: 1,456 random: 1,
397 };457 };
398 let error =458 let error =
399 onestore::commit_text(&mut disk, &source, sid, oid, range, replacement).unwrap_err();459 commit_text(&mut disk, &source, sid, oid, range, replacement).unwrap_err();
400 assert_eq!(error.state, CommitState::NotCommitted);460 assert_eq!(error.state, CommitState::NotCommitted);
401 assert_eq!(disk.operation, 0);461 assert_eq!(disk.operation, 0);
402 assert_eq!(disk.durable, source);462 assert_eq!(disk.durable, source);
...@@ -437,7 +497,7 @@ fn cross_run_splices_match_a_character_model_and_preserve_history() {...@@ -437,7 +497,7 @@ fn cross_run_splices_match_a_character_model_and_preserve_history() {
437 let mut expected = before.clone();497 let mut expected = before.clone();
438 expected.splice(a..b, replacement.chars().map(|c| (c, before[a].1.clone())));498 expected.splice(a..b, replacement.chars().map(|c| (c, before[a].1.clone())));
439 let edited =499 let edited =
440 onestore::replace_text(SOURCE, sid, oid, offsets[a]..offsets[b], replacement)500 replace_text(SOURCE, sid, oid, offsets[a]..offsets[b], replacement)
441 .unwrap();501 .unwrap();
442 assert_eq!(characters(&edited), expected, "characters {a}..{b}");502 assert_eq!(characters(&edited), expected, "characters {a}..{b}");
443 let current_store = Store::parse(&edited).unwrap();503 let current_store = Store::parse(&edited).unwrap();
...@@ -460,8 +520,8 @@ fn cross_run_splices_match_a_character_model_and_preserve_history() {...@@ -460,8 +520,8 @@ fn cross_run_splices_match_a_character_model_and_preserve_history() {
460 }520 }
461 }521 }
462 let cleared =522 let cleared =
463 onestore::replace_text(SOURCE, sid, oid, 0..*offsets.last().unwrap(), "").unwrap();523 replace_text(SOURCE, sid, oid, 0..*offsets.last().unwrap(), "").unwrap();
464 let typed = onestore::replace_text(&cleared, sid, oid, 0..0, "new").unwrap();524 let typed = replace_text(&cleared, sid, oid, 0..0, "new").unwrap();
465 let format = &before.last().unwrap().1;525 let format = &before.last().unwrap().1;
466 assert_eq!(526 assert_eq!(
467 characters(&typed),527 characters(&typed),
...@@ -472,7 +532,7 @@ fn cross_run_splices_match_a_character_model_and_preserve_history() {...@@ -472,7 +532,7 @@ fn cross_run_splices_match_a_character_model_and_preserve_history() {
472 );532 );
473 let text: String = before[..before.len() - 1].iter().map(|(c, _)| *c).collect();533 let text: String = before[..before.len() - 1].iter().map(|(c, _)| *c).collect();
474 assert_eq!(534 assert_eq!(
475 onestore::replace_text(SOURCE, sid, oid, 0..*offsets.last().unwrap(), &text).unwrap(),535 replace_text(SOURCE, sid, oid, 0..*offsets.last().unwrap(), &text).unwrap(),
476 SOURCE536 SOURCE
477 );537 );
478}538}
...@@ -520,9 +580,9 @@ fn export_native_cross_run_edits() {...@@ -520,9 +580,9 @@ fn export_native_cross_run_edits() {
520 .unwrap();580 .unwrap();
521 let range = range.start..range.end.min(length);581 let range = range.start..range.end.min(length);
522 let mut edited =582 let mut edited =
523 onestore::replace_text(&source, sid, oid, range.clone(), replacement).unwrap();583 replace_text(&source, sid, oid, range.clone(), replacement).unwrap();
524 if retype {584 if retype {
525 edited = onestore::replace_text(&edited, sid, oid, 0..0, "Retyped 🦀").unwrap();585 edited = replace_text(&edited, sid, oid, 0..0, "Retyped 🦀").unwrap();
526 }586 }
527 let candidate = output.join(name).join("candidate");587 let candidate = output.join(name).join("candidate");
528 fs::create_dir_all(&candidate).unwrap();588 fs::create_dir_all(&candidate).unwrap();
...@@ -540,7 +600,7 @@ fn export_native_cross_run_edits() {...@@ -540,7 +600,7 @@ fn export_native_cross_run_edits() {
540fn insertion_at_a_style_boundary_uses_the_following_style() {600fn insertion_at_a_style_boundary_uses_the_following_style() {
541 let (sid, oid) = target(SOURCE);601 let (sid, oid) = target(SOURCE);
542 let before = text_runs(SOURCE, sid, oid);602 let before = text_runs(SOURCE, sid, oid);
543 let edited = onestore::replace_text(SOURCE, sid, oid, 18..18, "🦀").unwrap();603 let edited = replace_text(SOURCE, sid, oid, 18..18, "🦀").unwrap();
544 let mut expected = before.clone();604 let mut expected = before.clone();
545 expected[1]["text"] = format!("🦀{}", before[1]["text"].as_str().unwrap()).into();605 expected[1]["text"] = format!("🦀{}", before[1]["text"].as_str().unwrap()).into();
546 assert_eq!(text_runs(&edited, sid, oid), expected);606 assert_eq!(text_runs(&edited, sid, oid), expected);
...@@ -588,10 +648,10 @@ fn automatic_titles_follow_native_line_and_utf16_limits() {...@@ -588,10 +648,10 @@ fn automatic_titles_follow_native_line_and_utf16_limits() {
588 };648 };
589 let (sid, oid) = check(&source, &expected);649 let (sid, oid) = check(&source, &expected);
590 let cleared =650 let cleared =
591 onestore::replace_text(&source, sid, oid, 0..text.encode_utf16().count() as u32, "")651 replace_text(&source, sid, oid, 0..text.encode_utf16().count() as u32, "")
592 .unwrap();652 .unwrap();
593 check(&cleared, "");653 check(&cleared, "");
594 let written = onestore::replace_text(&cleared, sid, oid, 0..0, " new 🦀 name ").unwrap();654 let written = replace_text(&cleared, sid, oid, 0..0, " new 🦀 name ").unwrap();
595 check(&written, "new 🦀 name");655 check(&written, "new 🦀 name");
596 }656 }
597}657}
...@@ -641,7 +701,7 @@ fn automatic_titles_follow_native_rtl_and_attachment_order() {...@@ -641,7 +701,7 @@ fn automatic_titles_follow_native_rtl_and_attachment_order() {
641 if node.extra[0].iter().any(|field| field.id == 0x88001cb4) {701 if node.extra[0].iter().any(|field| field.id == 0x88001cb4) {
642 continue;702 continue;
643 }703 }
644 let changed = onestore::replace_text(&source, sid, *oid, 0..0, "Edited ").unwrap();704 let changed = replace_text(&source, sid, *oid, 0..0, "Edited ").unwrap();
645 let store = Store::parse(&changed).unwrap();705 let store = Store::parse(&changed).unwrap();
646 let current = RevisionIndex::parse(&store).unwrap();706 let current = RevisionIndex::parse(&store).unwrap();
647 let after = Document::parse(&current).unwrap();707 let after = Document::parse(&current).unwrap();
...@@ -675,8 +735,8 @@ fn automatic_titles_follow_native_rtl_and_attachment_order() {...@@ -675,8 +735,8 @@ fn automatic_titles_follow_native_rtl_and_attachment_order() {
675fn an_emptied_final_run_retains_its_insertion_style() {735fn an_emptied_final_run_retains_its_insertion_style() {
676 let (sid, oid) = target(SOURCE);736 let (sid, oid) = target(SOURCE);
677 let before = text_runs(SOURCE, sid, oid);737 let before = text_runs(SOURCE, sid, oid);
678 let erased = onestore::replace_text(SOURCE, sid, oid, 22..27, "").unwrap();738 let erased = replace_text(SOURCE, sid, oid, 22..27, "").unwrap();
679 let edited = onestore::replace_text(&erased, sid, oid, 22..22, "a").unwrap();739 let edited = replace_text(&erased, sid, oid, 22..22, "a").unwrap();
680 let mut expected = before.clone();740 let mut expected = before.clone();
681 expected[3]["text"] = "a".into();741 expected[3]["text"] = "a".into();
682 assert_eq!(text_runs(&edited, sid, oid), expected);742 assert_eq!(text_runs(&edited, sid, oid), expected);
...@@ -739,12 +799,12 @@ fn title_text_and_navigation_caches_publish_together() {...@@ -739,12 +799,12 @@ fn title_text_and_navigation_caches_publish_together() {
739 .collect::<std::collections::BTreeMap<_, _>>(),799 .collect::<std::collections::BTreeMap<_, _>>(),
740 ])800 ])
741 };801 };
742 let checkpoint = checkpoint::pending(&source, sid, *oid, 0x14001d7a);802 let checkpoint = checkpoint::pending(&source, sid, *oid);
743 for (source, write_limit) in [(source.as_slice(), 17), (checkpoint.as_slice(), 257)] {803 for (source, write_limit) in [(source.as_slice(), 17), (checkpoint.as_slice(), 257)] {
744 let before = state(source);804 let before = state(source);
745 for replacement in ["Renamed 🦀 日本語", ""] {805 for replacement in ["Renamed 🦀 日本語", ""] {
746 let edit =806 let edit =
747 onestore::PreparedEdit::text(source, sid, *oid, 0..end, replacement).unwrap();807 text_transaction(source, sid, *oid, 0..end, replacement).unwrap();
748 let disk = |fail_at| Disk {808 let disk = |fail_at| Disk {
749 visible: source.to_vec(),809 visible: source.to_vec(),
750 durable: source.to_vec(),810 durable: source.to_vec(),
...@@ -790,7 +850,7 @@ fn title_text_and_navigation_caches_publish_together() {...@@ -790,7 +850,7 @@ fn title_text_and_navigation_caches_publish_together() {
790 );850 );
791 }851 }
792 assert_other_objects_preserved(&old, &current, *oid);852 assert_other_objects_preserved(&old, &current, *oid);
793 for at in source.len().div_ceil(193)..=success.operation {853 for at in 1..=success.operation {
794 let mut interrupted = disk(Some(at));854 let mut interrupted = disk(Some(at));
795 let error = edit.commit(&mut interrupted).unwrap_err();855 let error = edit.commit(&mut interrupted).unwrap_err();
796 let observed = state(&interrupted.durable);856 let observed = state(&interrupted.durable);
...@@ -827,7 +887,7 @@ fn native_conflict_pages_are_readable_but_not_random_edit_targets() {...@@ -827,7 +887,7 @@ fn native_conflict_pages_are_readable_but_not_random_edit_targets() {
827 if !matches!(node.kind, Kind::RichText { .. }) {887 if !matches!(node.kind, Kind::RichText { .. }) {
828 continue;888 continue;
829 }889 }
830 assert!(onestore::replace_text(&source, *sid, *oid, 0..0, "edit").is_err());890 assert!(replace_text(&source, *sid, *oid, 0..0, "edit").is_err());
831 conflicts += 1;891 conflicts += 1;
832 }892 }
833 }893 }
...@@ -837,7 +897,7 @@ fn native_conflict_pages_are_readable_but_not_random_edit_targets() {...@@ -837,7 +897,7 @@ fn native_conflict_pages_are_readable_but_not_random_edit_targets() {
837#[test]897#[test]
838fn interrupted_text_commits_never_publish_mismatched_run_boundaries() {898fn interrupted_text_commits_never_publish_mismatched_run_boundaries() {
839 let (sid, oid) = target(SOURCE);899 let (sid, oid) = target(SOURCE);
840 let checkpoint = checkpoint::pending(SOURCE, sid, oid, 0x14001d7a);900 let checkpoint = checkpoint::pending(SOURCE, sid, oid);
841 for source in [SOURCE, checkpoint.as_slice()] {901 for source in [SOURCE, checkpoint.as_slice()] {
842 let make_disk = |fail_at, random| Disk {902 let make_disk = |fail_at, random| Disk {
843 visible: source.to_vec(),903 visible: source.to_vec(),
...@@ -849,14 +909,14 @@ fn interrupted_text_commits_never_publish_mismatched_run_boundaries() {...@@ -849,14 +909,14 @@ fn interrupted_text_commits_never_publish_mismatched_run_boundaries() {
849 };909 };
850 let before = text_runs(source, sid, oid);910 let before = text_runs(source, sid, oid);
851 let mut success = make_disk(None, 1);911 let mut success = make_disk(None, 1);
852 onestore::commit_text(&mut success, source, sid, oid, 0..23, "🐈 mixed edit").unwrap();912 commit_text(&mut success, source, sid, oid, 0..23, "🐈 mixed edit").unwrap();
853 let after = text_runs(&success.durable, sid, oid);913 let after = text_runs(&success.durable, sid, oid);
854 assert_ne!(before, after);914 assert_ne!(before, after);
855 for at in source.len().div_ceil(193)..=success.operation {915 for at in 1..=success.operation {
856 for seed in [1, 42] {916 for seed in [1, 42] {
857 let mut disk = make_disk(Some(at), seed);917 let mut disk = make_disk(Some(at), seed);
858 let error =918 let error =
859 onestore::commit_text(&mut disk, source, sid, oid, 0..23, "🐈 mixed edit")919 commit_text(&mut disk, source, sid, oid, 0..23, "🐈 mixed edit")
860 .unwrap_err();920 .unwrap_err();
861 let observed = text_runs(&disk.durable, sid, oid);921 let observed = text_runs(&disk.durable, sid, oid);
862 match error.state {922 match error.state {
...@@ -906,7 +966,7 @@ fn empty_and_legacy_native_text_gain_unicode_without_losing_existing_properties(...@@ -906,7 +966,7 @@ fn empty_and_legacy_native_text_gain_unicode_without_losing_existing_properties(
906 let properties = PropertySets::parse(blob).unwrap();966 let properties = PropertySets::parse(blob).unwrap();
907 assert!(!properties.sets[0].iter().any(|p| p.id == 0x1c001c22));967 assert!(!properties.sets[0].iter().any(|p| p.id == 0x1c001c22));
908 let expected = format!("🦀 café {original}");968 let expected = format!("🦀 café {original}");
909 let written = onestore::replace_text(&source, sid, oid, 0..0, "🦀 café ").unwrap();969 let written = replace_text(&source, sid, oid, 0..0, "🦀 café ").unwrap();
910 let after_store = Store::parse(&written).unwrap();970 let after_store = Store::parse(&written).unwrap();
911 assert!(after_store.checksum_mismatches.is_empty());971 assert!(after_store.checksum_mismatches.is_empty());
912 let after = RevisionIndex::parse(&after_store).unwrap();972 let after = RevisionIndex::parse(&after_store).unwrap();
...@@ -953,12 +1013,12 @@ fn empty_and_legacy_native_text_gain_unicode_without_losing_existing_properties(...@@ -953,12 +1013,12 @@ fn empty_and_legacy_native_text_gain_unicode_without_losing_existing_properties(
953 random,1013 random,
954 };1014 };
955 let mut success = disk(None, 1);1015 let mut success = disk(None, 1);
956 onestore::commit_text(&mut success, &source, sid, oid, 0..0, "🦀 café ").unwrap();1016 commit_text(&mut success, &source, sid, oid, 0..0, "🦀 café ").unwrap();
957 for at in source.len().div_ceil(193)..=success.operation {1017 for at in 1..=success.operation {
958 for seed in [1, 42] {1018 for seed in [1, 42] {
959 let mut interrupted = disk(Some(at), seed);1019 let mut interrupted = disk(Some(at), seed);
960 let error =1020 let error =
961 onestore::commit_text(&mut interrupted, &source, sid, oid, 0..0, "🦀 café ")1021 commit_text(&mut interrupted, &source, sid, oid, 0..0, "🦀 café ")
962 .unwrap_err();1022 .unwrap_err();
963 let actual = text_runs(&interrupted.durable, sid, oid);1023 let actual = text_runs(&interrupted.durable, sid, oid);
964 match error.state {1024 match error.state {
crates/onestore/tests/formatting.rs+18-63
...@@ -1,10 +1,13 @@...@@ -1,10 +1,13 @@
1#[path = "support/ops.rs"]
2mod ops;
1#[path = "support/current.rs"]3#[path = "support/current.rs"]
2mod current;4mod current;
3#[path = "support/disk.rs"]5#[path = "support/disk.rs"]
4mod disk;6mod disk;
5use onestore::{7use onestore::{
6 ExGuid, PreparedEdit, RevisionIndex, Store, TextAttribute as A,8 ExGuid, RevisionIndex, Store, TextAttribute as A,
7 document::{Document, Kind},9 document::{Document, Kind},
10 op::PageOp,
8};11};
9use serde_json::{Value, json};12use serde_json::{Value, json};
10fn target(source: &[u8]) -> (ExGuid, ExGuid) {13fn target(source: &[u8]) -> (ExGuid, ExGuid) {
...@@ -85,13 +88,7 @@ fn overlapping_unicode_format_edits_match_an_independent_character_model() {...@@ -85,13 +88,7 @@ fn overlapping_unicode_format_edits_match_an_independent_character_model() {
85 json!(0x00ff00_u32),88 json!(0x00ff00_u32),
86 ),89 ),
87 };90 };
88 let edit = PreparedEdit::format(91 let edit = ops::page_op(&source, sid, PageOp::Format { text: id, range: offsets[start]..offsets[end], set: std::slice::from_ref(&attribute).to_vec(), clear: Vec::new() })
89 &source,
90 sid,
91 id,
92 offsets[start]..offsets[end],
93 std::slice::from_ref(&attribute),
94 )
95 .unwrap();92 .unwrap();
96 for (_, style) in &mut expected[start..end] {93 for (_, style) in &mut expected[start..end] {
97 style[key] = value.clone();94 style[key] = value.clone();
...@@ -102,13 +99,7 @@ fn overlapping_unicode_format_edits_match_an_independent_character_model() {...@@ -102,13 +99,7 @@ fn overlapping_unicode_format_edits_match_an_independent_character_model() {
102 "seed {seed}, step {step}"99 "seed {seed}, step {step}"
103 );100 );
104 assert_eq!(101 assert_eq!(
105 PreparedEdit::format(102 ops::page_op(edit.as_bytes(), sid, PageOp::Format { text: id, range: offsets[start]..offsets[end], set: vec![attribute], clear: Vec::new() })
106 edit.as_bytes(),
107 sid,
108 id,
109 offsets[start]..offsets[end],
110 &[attribute]
111 )
112 .unwrap()103 .unwrap()
113 .as_bytes(),104 .as_bytes(),
114 edit.as_bytes()105 edit.as_bytes()
...@@ -130,21 +121,9 @@ fn overlapping_unicode_format_edits_match_an_independent_character_model() {...@@ -130,21 +121,9 @@ fn overlapping_unicode_format_edits_match_an_independent_character_model() {
130fn script_positions_are_exclusive_and_explicit_false_overrides_true() {121fn script_positions_are_exclusive_and_explicit_false_overrides_true() {
131 let source = onestore::create_section("script.one", "abc", "Author").unwrap();122 let source = onestore::create_section("script.one", "abc", "Author").unwrap();
132 let (sid, id) = target(&source);123 let (sid, id) = target(&source);
133 let superscript = PreparedEdit::format(124 let superscript = ops::page_op(&source, sid, PageOp::Format { text: id, range: 0..3, set: vec![A::Superscript(true), A::Bold(true)], clear: Vec::new() })
134 &source,
135 sid,
136 id,
137 0..3,
138 &[A::Superscript(true), A::Bold(true)],
139 )
140 .unwrap();125 .unwrap();
141 let subscript = PreparedEdit::format(126 let subscript = ops::page_op(superscript.as_bytes(), sid, PageOp::Format { text: id, range: 1..2, set: vec![A::Subscript(true), A::Bold(false)], clear: Vec::new() })
142 superscript.as_bytes(),
143 sid,
144 id,
145 1..2,
146 &[A::Subscript(true), A::Bold(false)],
147 )
148 .unwrap();127 .unwrap();
149 let chars = characters(subscript.as_bytes(), sid, id);128 let chars = characters(subscript.as_bytes(), sid, id);
150 assert_eq!(chars[0].1["superscript"], true);129 assert_eq!(chars[0].1["superscript"], true);
...@@ -159,15 +138,9 @@ fn script_positions_are_exclusive_and_explicit_false_overrides_true() {...@@ -159,15 +138,9 @@ fn script_positions_are_exclusive_and_explicit_false_overrides_true() {
159fn empty_paragraph_style_is_used_by_later_text_edits() {138fn empty_paragraph_style_is_used_by_later_text_edits() {
160 let source = onestore::create_section("empty.one", "", "Author").unwrap();139 let source = onestore::create_section("empty.one", "", "Author").unwrap();
161 let (sid, id) = target(&source);140 let (sid, id) = target(&source);
162 let formatted = PreparedEdit::format(141 let formatted = ops::page_op(&source, sid, PageOp::Format { text: id, range: 0..0, set: vec![A::Italic(true), A::FontSize(18.0)], clear: Vec::new() })
163 &source,
164 sid,
165 id,
166 0..0,
167 &[A::Italic(true), A::FontSize(18.0)],
168 )
169 .unwrap();142 .unwrap();
170 let filled = PreparedEdit::text(formatted.as_bytes(), sid, id, 0..0, "Added 🦀").unwrap();143 let filled = ops::page_op(formatted.as_bytes(), sid, PageOp::Text { text: id, range: 0..0, with: "Added 🦀".into() }).unwrap();
171 for (_, format) in characters(filled.as_bytes(), sid, id) {144 for (_, format) in characters(filled.as_bytes(), sid, id) {
172 assert_eq!(format["italic"], true);145 assert_eq!(format["italic"], true);
173 assert_eq!(format["font_size"], 18.0);146 assert_eq!(format["font_size"], 18.0);
...@@ -179,7 +152,7 @@ fn invalid_ranges_attributes_and_fields_are_rejected() {...@@ -179,7 +152,7 @@ fn invalid_ranges_attributes_and_fields_are_rejected() {
179 let (sid, id) = target(&source);152 let (sid, id) = target(&source);
180 for (start, end) in [(2, 3), (1, 2), (0, 8), (2, 2), (3, 1), (1, 1)] {153 for (start, end) in [(2, 3), (1, 2), (0, 8), (2, 2), (3, 1), (1, 1)] {
181 let range = start..end;154 let range = start..end;
182 assert!(PreparedEdit::format(&source, sid, id, range, &[A::Bold(true)]).is_err());155 assert!(ops::page_op(&source, sid, PageOp::Format { text: id, range, set: vec![A::Bold(true)], clear: Vec::new() }).is_err());
183 }156 }
184 for attributes in [157 for attributes in [
185 vec![],158 vec![],
...@@ -188,24 +161,18 @@ fn invalid_ranges_attributes_and_fields_are_rejected() {...@@ -188,24 +161,18 @@ fn invalid_ranges_attributes_and_fields_are_rejected() {
188 vec![A::Font("".into())],161 vec![A::Font("".into())],
189 vec![A::Font("a\0b".into())],162 vec![A::Font("a\0b".into())],
190 ] {163 ] {
191 assert!(PreparedEdit::format(&source, sid, id, 0..4, &attributes).is_err());164 assert!(ops::page_op(&source, sid, PageOp::Format { text: id, range: 0..4, set: attributes.to_vec(), clear: Vec::new() }).is_err());
192 }165 }
193 for size in [f32::NAN, f32::INFINITY, 0.0, 5.5, 130.5, 144.0, 144.5, 12.1] {166 for size in [f32::NAN, f32::INFINITY, 0.0, 5.5, 130.5, 144.0, 144.5, 12.1] {
194 assert!(PreparedEdit::format(&source, sid, id, 0..4, &[A::FontSize(size)]).is_err());167 assert!(ops::page_op(&source, sid, PageOp::Format { text: id, range: 0..4, set: vec![A::FontSize(size)], clear: Vec::new() }).is_err());
195 }168 }
196 assert!(PreparedEdit::format(&source, sid, ExGuid::default(), 0..4, &[A::Bold(true)]).is_err());169 assert!(ops::page_op(&source, sid, PageOp::Format { text: ExGuid::default(), range: 0..4, set: vec![A::Bold(true)], clear: Vec::new() }).is_err());
197}170}
198#[test]171#[test]
199fn formatting_publication_faults_preserve_complete_old_or_new_styles() {172fn formatting_publication_faults_preserve_complete_old_or_new_styles() {
200 let source = onestore::create_section("atomic.one", "Before 🦀 after", "Author").unwrap();173 let source = onestore::create_section("atomic.one", "Before 🦀 after", "Author").unwrap();
201 let (sid, id) = target(&source);174 let (sid, id) = target(&source);
202 let edit = PreparedEdit::format(175 let edit = ops::page_op(&source, sid, PageOp::Format { text: id, range: 2..10, set: vec![A::Bold(true), A::Color(Some([8, 64, 128]))], clear: Vec::new() })
203 &source,
204 sid,
205 id,
206 2..10,
207 &[A::Bold(true), A::Color(Some([8, 64, 128]))],
208 )
209 .unwrap();176 .unwrap();
210 let before = current::current(&source);177 let before = current::current(&source);
211 let after = current::current(edit.as_bytes());178 let after = current::current(edit.as_bytes());
...@@ -251,7 +218,7 @@ fn export_native_formatting_candidates() {...@@ -251,7 +218,7 @@ fn export_native_formatting_candidates() {
251 let mut save =218 let mut save =
252 |name: &str, source: &[u8], sid, id, range: std::ops::Range<u32>, attributes: &[A]| {219 |name: &str, source: &[u8], sid, id, range: std::ops::Range<u32>, attributes: &[A]| {
253 let prepared =220 let prepared =
254 PreparedEdit::format(source, sid, id, range.clone(), attributes).unwrap();221 ops::page_op(source, sid, PageOp::Format { text: id, range: range.clone(), set: attributes.to_vec(), clear: Vec::new() }).unwrap();
255 fs::write(output.join(format!("{name}.one")), prepared.as_bytes()).unwrap();222 fs::write(output.join(format!("{name}.one")), prepared.as_bytes()).unwrap();
256 manifest.push(223 manifest.push(
257 json!({"name":name,"space":sid,"object":id,"range":range,"attributes":attributes}),224 json!({"name":name,"space":sid,"object":id,"range":range,"attributes":attributes}),
...@@ -329,21 +296,9 @@ fn export_native_formatting_candidates() {...@@ -329,21 +296,9 @@ fn export_native_formatting_candidates() {
329 );296 );
330 let empty = onestore::create_section("empty.one", "", "Author").unwrap();297 let empty = onestore::create_section("empty.one", "", "Author").unwrap();
331 let (empty_sid, empty_id) = target(&empty);298 let (empty_sid, empty_id) = target(&empty);
332 let formatted = PreparedEdit::format(299 let formatted = ops::page_op(&empty, empty_sid, PageOp::Format { text: empty_id, range: 0..0, set: vec![A::FontSize(18.0), A::Italic(true)], clear: Vec::new() })
333 &empty,
334 empty_sid,
335 empty_id,
336 0..0,
337 &[A::FontSize(18.0), A::Italic(true)],
338 )
339 .unwrap();300 .unwrap();
340 let typed = PreparedEdit::text(301 let typed = ops::page_op(formatted.as_bytes(), empty_sid, PageOp::Text { text: empty_id, range: 0..0, with: "Typed café 🦀".into() })
341 formatted.as_bytes(),
342 empty_sid,
343 empty_id,
344 0..0,
345 "Typed café 🦀",
346 )
347 .unwrap();302 .unwrap();
348 fs::write(output.join("empty-then-type.one"), typed.as_bytes()).unwrap();303 fs::write(output.join("empty-then-type.one"), typed.as_bytes()).unwrap();
349 let native = include_bytes!("../../../corpus/native-ink/20260905-ui/notebook/synthetic.one");304 let native = include_bytes!("../../../corpus/native-ink/20260905-ui/notebook/synthetic.one");
crates/onestore/tests/insertion.rs+231-309
...@@ -4,11 +4,65 @@ mod checkpoint;...@@ -4,11 +4,65 @@ mod checkpoint;
4mod current;4mod current;
5#[path = "support/disk.rs"]5#[path = "support/disk.rs"]
6mod disk;6mod disk;
7#[path = "support/ops.rs"]
8mod ops;
79
8use onestore::{10use onestore::{
9 ExGuid, Insertion, PreparedEdit, RevisionIndex, Store, TextAttribute as A,11 ExGuid, RevisionIndex, Store, TextAttribute as A,
10 document::{Document, Kind},12 document::{Document, Kind},
13 op::{Edit, Op, PageOp},
11};14};
15use std::ops::Range;
16
17/// Where an insertion goes: a paragraph in a container before a child of it, or last; or a
18/// new outline at a point on the page.
19#[derive(Clone, Copy)]
20enum Place {
21 In(ExGuid, Option<ExGuid>),
22 Outline(f32, f32),
23}
24
25/// The ops inserting `text` at `place` with `formats` set over UTF-16 ranges of it, and the
26/// identities of the new paragraph (or outline) and text.
27fn insertion(
28 space: ExGuid,
29 place: Place,
30 text: &str,
31 formats: &[(Range<u32>, Vec<A>)],
32) -> (Vec<Op>, ExGuid, ExGuid) {
33 let paragraph = ops::paragraph(text);
34 let text = paragraph.text().unwrap().id;
35 let (op, object) = match place {
36 Place::In(container, before) => {
37 let id = paragraph.id;
38 let paragraphs = vec![paragraph];
39 (
40 PageOp::Insert {
41 container,
42 before,
43 paragraphs,
44 },
45 id,
46 )
47 }
48 Place::Outline(x, y) => {
49 let object = ops::outline(x, y, vec![paragraph]);
50 let id = object.id();
51 (PageOp::Add { object, before: None }, id)
52 }
53 };
54 let mut result = vec![Op::Page { space, op }];
55 for (range, set) in formats {
56 let op = PageOp::Format {
57 text,
58 range: range.clone(),
59 set: set.clone(),
60 clear: Vec::new(),
61 };
62 result.push(Op::Page { space, op });
63 }
64 (result, object, text)
65}
1266
13fn targets(source: &[u8]) -> (ExGuid, ExGuid, ExGuid, ExGuid, ExGuid) {67fn targets(source: &[u8]) -> (ExGuid, ExGuid, ExGuid, ExGuid, ExGuid) {
14 let store = Store::parse(source).unwrap();68 let store = Store::parse(source).unwrap();
...@@ -34,23 +88,15 @@ fn atomic_formatted_insertions_match_a_character_model_and_preserve_history() {...@@ -34,23 +88,15 @@ fn atomic_formatted_insertions_match_a_character_model_and_preserve_history() {
34 let source = onestore::create_section("rich.one", "Original", "Author").unwrap();88 let source = onestore::create_section("rich.one", "Original", "Author").unwrap();
35 let (sid, page, outline, _, _) = targets(&source);89 let (sid, page, outline, _, _) = targets(&source);
36 let text = "ab🦀 e\u{301} 東京\rEnd\t!";90 let text = "ab🦀 e\u{301} 東京\rEnd\t!";
37 let plain = Insertion::paragraph(outline, None, text, "Author").unwrap();91 let (plain, _, plain_text) = insertion(sid, Place::In(outline, None), text, &[]);
38 assert!(92 let base = ops::edited(&source, plain).unwrap();
39 serde_json::to_value(&plain)93 let base_store = Store::parse(&base).unwrap();
40 .unwrap()
41 .get("formats")
42 .is_none()
43 );
44 let restored: Insertion = serde_json::from_slice(&serde_json::to_vec(&plain).unwrap()).unwrap();
45 assert_eq!(plain, restored);
46 let base = PreparedEdit::insert(&source, sid, &plain).unwrap();
47 let base_store = Store::parse(base.as_bytes()).unwrap();
48 let base_index = RevisionIndex::parse(&base_store).unwrap();94 let base_index = RevisionIndex::parse(&base_store).unwrap();
49 let base_doc = Document::parse(&base_index).unwrap();95 let base_doc = Document::parse(&base_index).unwrap();
50 let base_space = &base_doc.spaces[&sid];96 let base_space = &base_doc.spaces[&sid];
51 let base_view = &base_space.revisions[&base_space.contexts[&ExGuid::default()]];97 let base_view = &base_space.revisions[&base_space.contexts[&ExGuid::default()]];
52 let default =98 let default =
53 serde_json::to_value(&base_view.text_runs(plain.text_object()).unwrap()[0].format).unwrap();99 serde_json::to_value(&base_view.text_runs(plain_text).unwrap()[0].format).unwrap();
54 let offsets: Vec<u32> = std::iter::once(0)100 let offsets: Vec<u32> = std::iter::once(0)
55 .chain(text.chars().scan(0, |n, c| {101 .chain(text.chars().scan(0, |n, c| {
56 *n += c.len_utf16() as u32;102 *n += c.len_utf16() as u32;
...@@ -62,12 +108,8 @@ fn atomic_formatted_insertions_match_a_character_model_and_preserve_history() {...@@ -62,12 +108,8 @@ fn atomic_formatted_insertions_match_a_character_model_and_preserve_history() {
62 for seed in 1..=48_u64 {108 for seed in 1..=48_u64 {
63 let mut random = seed;109 let mut random = seed;
64 let mut expected: Vec<_> = text.chars().map(|c| (c, default.clone())).collect();110 let mut expected: Vec<_> = text.chars().map(|c| (c, default.clone())).collect();
65 let mut intent = if seed % 2 == 0 {111 let mut formats = Vec::new();
66 Insertion::outline(page, 72.0, 144.0, text, "Author").unwrap()112 // Descending construction also exercises the order formats apply in.
67 } else {
68 plain.clone()
69 };
70 // Descending construction also exercises persisted ordering independently of call order.
71 for i in (0..expected.len()).rev() {113 for i in (0..expected.len()).rev() {
72 random ^= random << 13;114 random ^= random << 13;
73 random ^= random >> 7;115 random ^= random >> 7;
...@@ -106,18 +148,22 @@ fn atomic_formatted_insertions_match_a_character_model_and_preserve_history() {...@@ -106,18 +148,22 @@ fn atomic_formatted_insertions_match_a_character_model_and_preserve_history() {
106 json!({"bold":false,"color":0xff000000_u32,"highlight":0xff000000_u32}),148 json!({"bold":false,"color":0xff000000_u32,"highlight":0xff000000_u32}),
107 ),149 ),
108 };150 };
109 intent = intent151 formats.push((offsets[i]..offsets[i + 1], attributes));
110 .with_formatting(offsets[i]..offsets[i + 1], &attributes)
111 .unwrap();
112 for (key, value) in values.as_object().unwrap() {152 for (key, value) in values.as_object().unwrap() {
113 expected[i].1[key] = value.clone();153 expected[i].1[key] = value.clone();
114 }154 }
115 }155 }
116 let restored: Insertion =156 let place = if seed % 2 == 0 {
117 serde_json::from_slice(&serde_json::to_vec(&intent).unwrap()).unwrap();157 Place::Outline(72.0, 144.0)
118 assert_eq!(restored, intent);158 } else {
119 let edited = PreparedEdit::insert(&source, sid, &restored).unwrap();159 Place::In(outline, None)
120 let store = Store::parse(edited.as_bytes()).unwrap();160 };
161 let (ops, _, inserted) = insertion(sid, place, text, &formats);
162 let edit = Edit { at: ops::now(), ops };
163 let restored: Edit = serde_json::from_slice(&serde_json::to_vec(&edit).unwrap()).unwrap();
164 assert_eq!(restored, edit);
165 let edited = ops::edited(&source, restored.ops).unwrap();
166 let store = Store::parse(&edited).unwrap();
121 assert_eq!(167 assert_eq!(
122 store.header.transaction_count,168 store.header.transaction_count,
123 old_store.header.transaction_count + 1169 old_store.header.transaction_count + 1
...@@ -139,8 +185,11 @@ fn atomic_formatted_insertions_match_a_character_model_and_preserve_history() {...@@ -139,8 +185,11 @@ fn atomic_formatted_insertions_match_a_character_model_and_preserve_history() {
139 let document = Document::parse(&index).unwrap();185 let document = Document::parse(&index).unwrap();
140 let space = &document.spaces[&sid];186 let space = &document.spaces[&sid];
141 let view = &space.revisions[&space.contexts[&ExGuid::default()]];187 let view = &space.revisions[&space.contexts[&ExGuid::default()]];
188 if let Place::Outline(..) = place {
189 assert!(view.nodes[&page].children.len() > 1);
190 }
142 let actual: Vec<_> = view191 let actual: Vec<_> = view
143 .text_runs(intent.text_object())192 .text_runs(inserted)
144 .unwrap()193 .unwrap()
145 .into_iter()194 .into_iter()
146 .flat_map(|run| {195 .flat_map(|run| {
...@@ -152,88 +201,34 @@ fn atomic_formatted_insertions_match_a_character_model_and_preserve_history() {...@@ -152,88 +201,34 @@ fn atomic_formatted_insertions_match_a_character_model_and_preserve_history() {
152 }201 }
153}202}
154203
155#[test]
156fn invalid_or_forged_insertion_formats_fail_before_source_parsing() {
157 use serde_json::json;
158 let source = onestore::create_section("invalid-rich.one", "Original", "Author").unwrap();
159 let (sid, _, outline, _, _) = targets(&source);
160 let plain = Insertion::paragraph(outline, None, "a🦀e\u{301}b", "Author").unwrap();
161 for (start, end) in [(0, 0), (2, 3), (1, 2), (4, 3), (0, 7)] {
162 let range = start..end;
163 assert!(plain.with_formatting(range, &[A::Bold(true)]).is_err());
164 }
165 for attributes in [
166 vec![],
167 vec![A::Bold(true), A::Bold(false)],
168 vec![A::FontSize(f32::NAN)],
169 vec![A::FontSize(5.5)],
170 vec![A::Font("a\0b".into())],
171 vec![A::Superscript(true), A::Subscript(true)],
172 ] {
173 assert!(plain.with_formatting(1..3, &attributes).is_err());
174 }
175 let formatted = plain.with_formatting(1..3, &[A::Bold(true)]).unwrap();
176 for range in [0..3, 1..3, 1..4] {
177 assert!(
178 formatted
179 .with_formatting(range, &[A::Italic(true)])
180 .is_err()
181 );
182 }
183 let encoded = serde_json::to_value(&formatted).unwrap();
184 for mutate in 0..6 {
185 let mut value = encoded.clone();
186 match mutate {
187 0 => value["formats"][0]["guid"] = serde_json::to_value([0; 16]).unwrap(),
188 1 => value["formats"][0]["guid"] = value["guid"].clone(),
189 2 => {
190 let duplicate = value["formats"][0].clone();
191 value["formats"].as_array_mut().unwrap().push(duplicate);
192 }
193 3 => value["formats"][0]["range"] = json!({"start":2,"end":3}),
194 4 => value["formats"][0]["attributes"] = json!([]),
195 _ => value["formats"][0]["range"] = json!({"start":0,"end":u32::MAX}),
196 }
197 let forged: Insertion = serde_json::from_value(value).unwrap();
198 let error = PreparedEdit::insert(&[], sid, &forged).err().unwrap();
199 assert_eq!(
200 error,
201 PreparedEdit::insert(&source, sid, &forged).err().unwrap()
202 );
203 }
204}
205
206#[test]204#[test]
207fn formatted_empty_insertions_preserve_the_insertion_style_for_later_typing() {205fn formatted_empty_insertions_preserve_the_insertion_style_for_later_typing() {
208 let source = onestore::create_section("empty-rich.one", "Original", "Author").unwrap();206 let source = onestore::create_section("empty-rich.one", "Original", "Author").unwrap();
209 let (sid, page, outline, _, _) = targets(&source);207 let (sid, _, outline, _, _) = targets(&source);
210 for plain in [208 for place in [Place::Outline(72.0, 144.0), Place::In(outline, None)] {
211 Insertion::outline(page, 72.0, 144.0, "", "Author").unwrap(),209 let style = [(0..0, vec![A::Italic(true), A::FontSize(18.0)])];
212 Insertion::paragraph(outline, None, "", "Author").unwrap(),210 let (ops, _, text) = insertion(sid, place, "", &style);
213 ] {211 let inserted = ops::edited(&source, ops).unwrap();
214 let intent = plain212 let typed = ops::page_edited(
215 .with_formatting(0..0, &[A::Italic(true), A::FontSize(18.0)])213 &inserted,
216 .unwrap();
217 assert!(intent.with_formatting(0..0, &[A::Bold(true)]).is_err());
218 let inserted = PreparedEdit::insert(&source, sid, &intent).unwrap();
219 let typed = PreparedEdit::text(
220 inserted.as_bytes(),
221 sid,214 sid,
222 intent.text_object(),215 vec![PageOp::Text {
223 0..0,216 text,
224 "Typed 🦀",217 range: 0..0,
218 with: "Typed 🦀".into(),
219 }],
225 )220 )
226 .unwrap();221 .unwrap();
227 for (bytes, text) in [(inserted.as_bytes(), ""), (typed.as_bytes(), "Typed 🦀")] {222 for (bytes, expected) in [(&inserted, ""), (&typed, "Typed 🦀")] {
228 let store = Store::parse(bytes).unwrap();223 let store = Store::parse(bytes).unwrap();
229 let index = RevisionIndex::parse(&store).unwrap();224 let index = RevisionIndex::parse(&store).unwrap();
230 index.validate_current().unwrap();225 index.validate_current().unwrap();
231 let document = Document::parse(&index).unwrap();226 let document = Document::parse(&index).unwrap();
232 let space = &document.spaces[&sid];227 let space = &document.spaces[&sid];
233 let view = &space.revisions[&space.contexts[&ExGuid::default()]];228 let view = &space.revisions[&space.contexts[&ExGuid::default()]];
234 let runs = view.text_runs(intent.text_object()).unwrap();229 let runs = view.text_runs(text).unwrap();
235 assert_eq!(runs.len(), 1);230 assert_eq!(runs.len(), 1);
236 assert_eq!(runs[0].text, text);231 assert_eq!(runs[0].text, expected);
237 assert_eq!(runs[0].format.italic, Some(true));232 assert_eq!(runs[0].format.italic, Some(true));
238 assert_eq!(runs[0].format.font_size, Some(18.0));233 assert_eq!(runs[0].format.font_size, Some(18.0));
239 }234 }
...@@ -263,11 +258,9 @@ fn export_native_formatted_insertions() {...@@ -263,11 +258,9 @@ fn export_native_formatted_insertions() {
263 ("empty", generated.as_slice(), true),258 ("empty", generated.as_slice(), true),
264 ("empty-typed", generated.as_slice(), true),259 ("empty-typed", generated.as_slice(), true),
265 ] {260 ] {
266 let (sid, page, outline, paragraph, _) = targets(source);261 let (sid, _, outline, paragraph, _) = targets(source);
267 let insertion = match name {262 let place = match name {
268 "native-paragraph" => {263 "native-paragraph" => Place::In(outline, Some(paragraph)),
269 Insertion::paragraph(outline, Some(paragraph), text, "Rich author")
270 }
271 "native-cell" => {264 "native-cell" => {
272 let store = Store::parse(source).unwrap();265 let store = Store::parse(source).unwrap();
273 let index = RevisionIndex::parse(&store).unwrap();266 let index = RevisionIndex::parse(&store).unwrap();
...@@ -280,24 +273,14 @@ fn export_native_formatted_insertions() {...@@ -280,24 +273,14 @@ fn export_native_formatted_insertions() {
280 .find(|(_, node)| matches!(node.kind, Kind::Cell { .. }))273 .find(|(_, node)| matches!(node.kind, Kind::Cell { .. }))
281 .unwrap()274 .unwrap()
282 .0;275 .0;
283 Insertion::paragraph(cell, None, text, "Rich author")276 Place::In(cell, None)
284 }277 }
285 _ => Insertion::outline(278 _ => Place::Outline(72.0, 144.0),
286 page,279 };
287 72.0,280 let formats = if empty {
288 144.0,281 vec![(0..0, vec![A::Italic(true), A::FontSize(18.0)])]
289 if empty { "" } else { text },
290 "Rich author",
291 ),
292 }
293 .unwrap();
294 let mut intent = insertion;
295 if empty {
296 intent = intent
297 .with_formatting(0..0, &[A::Italic(true), A::FontSize(18.0)])
298 .unwrap();
299 } else {282 } else {
300 for (label, attributes) in [283 [
301 ("Bold", vec![A::Bold(true), A::Underline(true)]),284 ("Bold", vec![A::Bold(true), A::Underline(true)]),
302 ("🦀", vec![A::Font("Arial".into()), A::FontSize(13.5)]),285 ("🦀", vec![A::Font("Arial".into()), A::FontSize(13.5)]),
303 ("italic", vec![A::Italic(true), A::Strike(true)]),286 ("italic", vec![A::Italic(true), A::Strike(true)]),
...@@ -319,36 +302,33 @@ fn export_native_formatted_insertions() {...@@ -319,36 +302,33 @@ fn export_native_formatted_insertions() {
319 A::Highlight(None),302 A::Highlight(None),
320 ],303 ],
321 ),304 ),
322 ] {305 ]
306 .into_iter()
307 .map(|(label, attributes)| {
323 let start = text[..text.find(label).unwrap()].encode_utf16().count() as u32;308 let start = text[..text.find(label).unwrap()].encode_utf16().count() as u32;
324 intent = intent309 (start..start + label.encode_utf16().count() as u32, attributes)
325 .with_formatting(310 })
326 start..start + label.encode_utf16().count() as u32,311 .collect()
327 &attributes,
328 )
329 .unwrap();
330 }
331 }
332 let inserted = PreparedEdit::insert(source, sid, &intent).unwrap();
333 let bytes = if name == "empty-typed" {
334 PreparedEdit::text(
335 inserted.as_bytes(),
336 sid,
337 intent.text_object(),
338 0..0,
339 "Typed café 🦀",
340 )
341 .unwrap()
342 .as_bytes()
343 .to_vec()
344 } else {
345 inserted.as_bytes().to_vec()
346 };312 };
313 let (mut ops, object, inserted) =
314 insertion(sid, place, if empty { "" } else { text }, &formats);
315 if name == "empty-typed" {
316 let op = PageOp::Text {
317 text: inserted,
318 range: 0..0,
319 with: "Typed café 🦀".into(),
320 };
321 ops.push(Op::Page { space: sid, op });
322 }
323 let edit = Edit { at: ops::now(), ops };
324 let bytes = ops::edited(source, edit.ops.clone()).unwrap();
347 current::current(&bytes);325 current::current(&bytes);
348 let notebook = output.join(name);326 let notebook = output.join(name);
349 fs::create_dir(&notebook).unwrap();327 fs::create_dir(&notebook).unwrap();
350 fs::write(notebook.join("synthetic.one"), &bytes).unwrap();328 fs::write(notebook.join("synthetic.one"), &bytes).unwrap();
351 manifest.push(serde_json::json!({"name":name,"space":sid,"intent":intent}));329 manifest.push(serde_json::json!({
330 "name": name, "space": sid, "object": object, "text": inserted, "edit": edit
331 }));
352 }332 }
353 fs::write(333 fs::write(
354 output.join("manifest.json"),334 output.join("manifest.json"),
...@@ -361,23 +341,15 @@ fn export_native_formatted_insertions() {...@@ -361,23 +341,15 @@ fn export_native_formatted_insertions() {
361fn paragraph_and_outline_insertions_publish_metadata_and_references_together() {341fn paragraph_and_outline_insertions_publish_metadata_and_references_together() {
362 let source = onestore::create_section("insertion.one", "Original", "Original author").unwrap();342 let source = onestore::create_section("insertion.one", "Original", "Original author").unwrap();
363 let (sid, page, outline, paragraph, original_text) = targets(&source);343 let (sid, page, outline, paragraph, original_text) = targets(&source);
364 for (intent, expected_parent, expected_x, expected_y) in [344 for (place, expected_parent, expected_x, expected_y) in [
365 (345 (Place::In(outline, Some(paragraph)), outline, None, None),
366 Insertion::paragraph(outline, Some(paragraph), "First 🦀\rSecond", "New author")346 (Place::Outline(144.0, 18.0), page, Some(144.0), Some(18.0)),
367 .unwrap(),
368 outline,
369 None,
370 None,
371 ),
372 (
373 Insertion::outline(page, 144.0, 18.0, "First 🦀\rSecond", "New author").unwrap(),
374 page,
375 Some(144.0),
376 Some(18.0),
377 ),
378 ] {347 ] {
379 let prepared = PreparedEdit::insert(&source, sid, &intent).unwrap();348 let (ops, object, text) = insertion(sid, place, "First 🦀\rSecond", &[]);
380 let store = Store::parse(prepared.as_bytes()).unwrap();349 let transaction = ops::transaction(&source, "New author", ops).unwrap().unwrap();
350 let mut written = source.clone();
351 transaction.apply(&mut written).unwrap();
352 let store = Store::parse(&written).unwrap();
381 assert_eq!(353 assert_eq!(
382 store.header.transaction_count,354 store.header.transaction_count,
383 Store::parse(&source).unwrap().header.transaction_count + 1355 Store::parse(&source).unwrap().header.transaction_count + 1
...@@ -387,18 +359,14 @@ fn paragraph_and_outline_insertions_publish_metadata_and_references_together() {...@@ -387,18 +359,14 @@ fn paragraph_and_outline_insertions_publish_metadata_and_references_together() {
387 let document = Document::parse(&index).unwrap();359 let document = Document::parse(&index).unwrap();
388 let space = &document.spaces[&sid];360 let space = &document.spaces[&sid];
389 let view = &space.revisions[&space.contexts[&ExGuid::default()]];361 let view = &space.revisions[&space.contexts[&ExGuid::default()]];
390 assert!(362 assert!(view.nodes[&expected_parent].children.contains(&object));
391 view.nodes[&expected_parent]363 assert_eq!(view.nodes[&object].layout.x, expected_x);
392 .children364 assert_eq!(view.nodes[&object].layout.y, expected_y);
393 .contains(&intent.object())
394 );
395 assert_eq!(view.nodes[&intent.object()].layout.x, expected_x);
396 assert_eq!(view.nodes[&intent.object()].layout.y, expected_y);
397 assert!(365 assert!(
398 matches!(&view.nodes[&original_text].kind, Kind::RichText { text, .. } if text == "Original")366 matches!(&view.nodes[&original_text].kind, Kind::RichText { text, .. } if text == "Original")
399 );367 );
400 assert!(368 assert!(
401 matches!(&view.nodes[&intent.text_object()].kind, Kind::RichText { text, .. } if text == "First 🦀\rSecond")369 matches!(&view.nodes[&text].kind, Kind::RichText { text, .. } if text == "First 🦀\rSecond")
402 );370 );
403 assert!(371 assert!(
404 matches!(&view.nodes[&page].kind, Kind::Page { alternate_title, .. } if alternate_title.as_deref() == Some("First 🦀"))372 matches!(&view.nodes[&page].kind, Kind::Page { alternate_title, .. } if alternate_title.as_deref() == Some("First 🦀"))
...@@ -411,7 +379,7 @@ fn paragraph_and_outline_insertions_publish_metadata_and_references_together() {...@@ -411,7 +379,7 @@ fn paragraph_and_outline_insertions_publish_metadata_and_references_together() {
411 .values()379 .values()
412 .any(|node| matches!(&node.kind, Kind::Author { name } if name.as_deref() == Some("New author")))380 .any(|node| matches!(&node.kind, Kind::Author { name } if name.as_deref() == Some("New author")))
413 );381 );
414 let runs = view.text_runs(intent.text_object()).unwrap();382 let runs = view.text_runs(text).unwrap();
415 assert_eq!(runs.len(), 1);383 assert_eq!(runs.len(), 1);
416 assert_eq!(runs[0].format.font.as_deref(), Some("Calibri"));384 assert_eq!(runs[0].format.font.as_deref(), Some("Calibri"));
417 assert_eq!(runs[0].format.font_size, Some(11.0));385 assert_eq!(runs[0].format.font_size, Some(11.0));
...@@ -434,26 +402,20 @@ fn repeated_insertions_and_formatting_share_immutable_objects() {...@@ -434,26 +402,20 @@ fn repeated_insertions_and_formatting_share_immutable_objects() {
434 let (sid, _, outline, _, _) = targets(&source);402 let (sid, _, outline, _, _) = targets(&source);
435 let mut texts = Vec::new();403 let mut texts = Vec::new();
436 for _ in 0..12 {404 for _ in 0..12 {
437 let insertion =405 let (ops, _, text) = insertion(sid, Place::In(outline, None), "Repeated paragraph", &[]);
438 Insertion::paragraph(outline, None, "Repeated paragraph", "Same author").unwrap();406 let transaction = ops::transaction(&source, "Same author", ops).unwrap().unwrap();
439 source = PreparedEdit::insert(&source, sid, &insertion)407 transaction.apply(&mut source).unwrap();
408 let bold = PageOp::Format {
409 text,
410 range: 1..8,
411 set: vec![A::Bold(true), A::FontSize(20.0)],
412 clear: Vec::new(),
413 };
414 let transaction = ops::transaction(&source, "Same author", vec![Op::Page { space: sid, op: bold }])
440 .unwrap()415 .unwrap()
441 .as_bytes()416 .unwrap();
442 .to_vec();417 transaction.apply(&mut source).unwrap();
443 source = PreparedEdit::format(418 texts.push(text);
444 &source,
445 sid,
446 insertion.text_object(),
447 1..8,
448 &[
449 onestore::TextAttribute::Bold(true),
450 onestore::TextAttribute::FontSize(20.0),
451 ],
452 )
453 .unwrap()
454 .as_bytes()
455 .to_vec();
456 texts.push(insertion.text_object());
457 }419 }
458 let store = Store::parse(&source).unwrap();420 let store = Store::parse(&source).unwrap();
459 let index = RevisionIndex::parse(&store).unwrap();421 let index = RevisionIndex::parse(&store).unwrap();
...@@ -494,103 +456,54 @@ fn repeated_insertions_and_formatting_share_immutable_objects() {...@@ -494,103 +456,54 @@ fn repeated_insertions_and_formatting_share_immutable_objects() {
494fn serialized_insertions_rebase_with_the_same_objects_and_preserve_remote_edits() {456fn serialized_insertions_rebase_with_the_same_objects_and_preserve_remote_edits() {
495 let source = onestore::create_section("rebase.one", "Original", "Author").unwrap();457 let source = onestore::create_section("rebase.one", "Original", "Author").unwrap();
496 let (sid, _, outline, paragraph, text) = targets(&source);458 let (sid, _, outline, paragraph, text) = targets(&source);
497 let intent = Insertion::paragraph(outline, Some(paragraph), "Inserted", "Author")459 let formats = [(0..3, vec![A::Bold(true), A::Color(Some([12, 34, 56]))])];
498 .unwrap()460 let (ops, object, inserted) =
499 .with_formatting(0..3, &[A::Bold(true), A::Color(Some([12, 34, 56]))])461 insertion(sid, Place::In(outline, Some(paragraph)), "Inserted", &formats);
500 .unwrap();462 let encoded = serde_json::to_vec(&ops).unwrap();
501 let encoded = serde_json::to_vec(&intent).unwrap();463 let restored: Vec<Op> = serde_json::from_slice(&encoded).unwrap();
502 let restored: Insertion = serde_json::from_slice(&encoded).unwrap();464 assert_eq!(ops, restored);
503 assert_eq!(intent.object(), restored.object());465 let original = ops::edited(&source, ops).unwrap();
504 assert_eq!(intent.text_object(), restored.text_object());466 let remote = ops::page_edited(
505 assert_eq!(intent, restored);467 &source,
506 let original_preparation = PreparedEdit::insert(&source, sid, &intent).unwrap();468 sid,
507 let remote = PreparedEdit::text(&source, sid, text, 0..0, "Remote ").unwrap();469 vec![PageOp::Text {
508 let updated = PreparedEdit::insert(remote.as_bytes(), sid, &restored).unwrap();470 text,
509 let store = Store::parse(updated.as_bytes()).unwrap();471 range: 0..0,
472 with: "Remote ".into(),
473 }],
474 )
475 .unwrap();
476 let updated = ops::edited(&remote, restored.clone()).unwrap();
477 let store = Store::parse(&updated).unwrap();
510 let index = RevisionIndex::parse(&store).unwrap();478 let index = RevisionIndex::parse(&store).unwrap();
511 let document = Document::parse(&index).unwrap();479 let document = Document::parse(&index).unwrap();
512 let space = &document.spaces[&sid];480 let space = &document.spaces[&sid];
513 let view = &space.revisions[&space.contexts[&ExGuid::default()]];481 let view = &space.revisions[&space.contexts[&ExGuid::default()]];
514 assert_eq!(482 assert_eq!(view.nodes[&outline].children, [object, paragraph]);
515 view.nodes[&outline].children,
516 [restored.object(), paragraph]
517 );
518 assert!(483 assert!(
519 matches!(&view.nodes[&text].kind, Kind::RichText { text, .. } if text == "Remote Original")484 matches!(&view.nodes[&text].kind, Kind::RichText { text, .. } if text == "Remote Original")
520 );485 );
521 assert!(486 assert!(
522 matches!(&view.nodes[&restored.text_object()].kind, Kind::RichText { text, .. } if text == "Inserted")487 matches!(&view.nodes[&inserted].kind, Kind::RichText { text, .. } if text == "Inserted")
523 );488 );
524 let runs = view.text_runs(restored.text_object()).unwrap();489 let runs = view.text_runs(inserted).unwrap();
525 assert_eq!(runs[0].text, "Ins");490 assert_eq!(runs[0].text, "Ins");
526 assert_eq!(runs[0].format.bold, Some(true));491 assert_eq!(runs[0].format.bold, Some(true));
527 assert_eq!(runs[0].format.color, Some(0x38220c));492 assert_eq!(runs[0].format.color, Some(0x38220c));
528 assert_eq!(runs[1].text, "erted");493 assert_eq!(runs[1].text, "erted");
529 assert_ne!(runs[1].format.bold, Some(true));494 assert_ne!(runs[1].format.bold, Some(true));
530 assert!(PreparedEdit::insert(original_preparation.as_bytes(), sid, &restored).is_err());495 assert!(ops::edited(&original, restored.clone()).is_err());
531 assert!(PreparedEdit::insert(updated.as_bytes(), sid, &restored).is_err());496 assert!(ops::edited(&updated, restored).is_err());
532}
533
534#[test]
535fn repositioning_preserves_intent_identity_and_kind() {
536 let source = onestore::create_section("placement.one", "Original", "Author").unwrap();
537 let (sid, page, outline, paragraph, _) = targets(&source);
538 let p = Insertion::paragraph(outline, Some(paragraph), "Inserted", "Author")
539 .unwrap()
540 .with_formatting(1..4, &[A::Italic(true)])
541 .unwrap();
542 let o = Insertion::outline(page, 144.0, 144.0, "Inserted", "Author")
543 .unwrap()
544 .with_formatting(0..8, &[A::FontSize(18.0)])
545 .unwrap();
546 assert!(p.reposition_outline(page, 72.0, 72.0).is_err());
547 assert!(o.reposition_paragraph(outline, None).is_err());
548 assert!(p.reposition_paragraph(ExGuid::default(), None).is_err());
549 assert!(o.reposition_outline(page, f32::NAN, 72.0).is_err());
550 for (original, moved) in [
551 (&p, p.reposition_paragraph(outline, None).unwrap()),
552 (&o, o.reposition_outline(page, 288.0, 360.0).unwrap()),
553 ] {
554 let mut before = serde_json::to_value(original).unwrap();
555 let mut after = serde_json::to_value(&moved).unwrap();
556 for name in ["parent", "placement"] {
557 before.as_object_mut().unwrap().remove(name);
558 after.as_object_mut().unwrap().remove(name);
559 }
560 assert_eq!(before, after);
561 assert_eq!(original.object(), moved.object());
562 assert_eq!(original.text_object(), moved.text_object());
563 let restored: Insertion =
564 serde_json::from_value(serde_json::to_value(&moved).unwrap()).unwrap();
565 let prepared = PreparedEdit::insert(&source, sid, &restored).unwrap();
566 let store = Store::parse(prepared.as_bytes()).unwrap();
567 let index = RevisionIndex::parse(&store).unwrap();
568 index.validate_current().unwrap();
569 let doc = Document::parse(&index).unwrap();
570 let space = &doc.spaces[&sid];
571 let view = &space.revisions[&space.contexts[&ExGuid::default()]];
572 if original == &p {
573 assert_eq!(view.nodes[&outline].children, [paragraph, moved.object()]);
574 } else {
575 assert_eq!(
576 (
577 view.nodes[&moved.object()].layout.x,
578 view.nodes[&moved.object()].layout.y
579 ),
580 (Some(288.0), Some(360.0))
581 );
582 }
583 }
584}497}
585498
586#[test]499#[test]
587#[cfg(any(unix, windows))]500#[cfg(any(unix, windows))]
588fn filesystem_insertion_uses_the_prepared_identity_and_rejects_stale_replay() {501fn filesystem_insertion_commits_once_and_rejects_stale_replay() {
589 use std::{fs, io::Write};502 use std::{fs, io::Write};
590 let source = onestore::create_section("file.one", "Original", "Author").unwrap();503 let source = onestore::create_section("file.one", "Original", "Author").unwrap();
591 let (sid, _, outline, _, _) = targets(&source);504 let (sid, _, outline, _, _) = targets(&source);
592 let intent = Insertion::paragraph(outline, None, "File insertion", "Author").unwrap();505 let (ops, object, _) = insertion(sid, Place::In(outline, None), "File insertion", &[]);
593 let path = std::env::temp_dir().join(format!("onestore-insertion-{}.one", intent.object()));506 let path = std::env::temp_dir().join(format!("onestore-insertion-{object}.one"));
594 let mut file = fs::OpenOptions::new()507 let mut file = fs::OpenOptions::new()
595 .write(true)508 .write(true)
596 .create_new(true)509 .create_new(true)
...@@ -599,60 +512,71 @@ fn filesystem_insertion_uses_the_prepared_identity_and_rejects_stale_replay() {...@@ -599,60 +512,71 @@ fn filesystem_insertion_uses_the_prepared_identity_and_rejects_stale_replay() {
599 file.write_all(&source).unwrap();512 file.write_all(&source).unwrap();
600 file.sync_all().unwrap();513 file.sync_all().unwrap();
601 drop(file);514 drop(file);
602 let edit = PreparedEdit::insert(&source, sid, &intent).unwrap();515 let transaction = ops::transaction(&source, "Author", ops).unwrap().unwrap();
603 let result = edit.commit_file(&path);516 let result = transaction.commit_file(&path);
604 let written = onestore::read_file(&path).unwrap();517 let written = onestore::read_file(&path).unwrap();
605 let repeated = edit.commit_file(&path).unwrap_err();518 let repeated = transaction.commit_file(&path).unwrap_err();
606 fs::remove_file(path).unwrap();519 fs::remove_file(path).unwrap();
607 result.unwrap();520 result.unwrap();
608 assert_eq!(written, edit.as_bytes());521 let mut expected = source.clone();
522 transaction.apply(&mut expected).unwrap();
523 assert_eq!(written, expected);
609 assert_eq!(repeated.state, onestore::CommitState::NotCommitted);524 assert_eq!(repeated.state, onestore::CommitState::NotCommitted);
610 assert_eq!(repeated.error.kind(), std::io::ErrorKind::ResourceBusy);525 assert_eq!(repeated.error.kind(), std::io::ErrorKind::ResourceBusy);
611}526}
612527
613#[test]528#[test]
614fn anchors_targets_serialized_identities_and_text_are_validated_before_publication() {529fn anchors_targets_identities_and_text_are_validated_before_publication() {
615 let source = onestore::create_section("invalid.one", "Original", "Author").unwrap();530 let source = onestore::create_section("invalid.one", "Original", "Author").unwrap();
616 let (sid, page, outline, paragraph, text) = targets(&source);531 let (sid, page, outline, _, text) = targets(&source);
617 assert!(Insertion::outline(page, f32::NAN, 0.0, "Text", "Author").is_err());532 let refused = |source: &[u8], place, content: &str| {
618 assert!(Insertion::outline(page, 0.0, f32::INFINITY, "Text", "Author").is_err());533 ops::edited(source, insertion(sid, place, content, &[]).0).is_err()
534 };
535 assert!(refused(&source, Place::Outline(f32::NAN, 0.0), "Text"));
536 assert!(refused(&source, Place::Outline(0.0, f32::INFINITY), "Text"));
619 for content in ["a\0b", "a\nb", "a\u{fffc}b"] {537 for content in ["a\0b", "a\nb", "a\u{fffc}b"] {
620 assert!(Insertion::paragraph(outline, None, content, "Author").is_err());538 assert!(refused(&source, Place::In(outline, None), content));
621 }539 }
622 assert!(Insertion::paragraph(outline, None, "Text", "a\0b").is_err());540 assert!(
623 for intent in [541 ops::transaction(&source, "a\0b", insertion(sid, Place::In(outline, None), "Text", &[]).0)
624 Insertion::paragraph(outline, Some(text), "Text", "Author").unwrap(),542 .is_err()
625 Insertion::paragraph(text, None, "Text", "Author").unwrap(),543 );
626 Insertion::paragraph(page, None, "Text", "Author").unwrap(),544 for place in [
627 Insertion::outline(paragraph, 0.0, 0.0, "Text", "Author").unwrap(),545 Place::In(outline, Some(text)),
546 Place::In(text, None),
547 Place::In(page, None),
628 ] {548 ] {
629 assert!(PreparedEdit::insert(&source, sid, &intent).is_err());549 assert!(refused(&source, place, "Text"));
630 }550 }
631 let intent = Insertion::paragraph(outline, None, "Text", "Author").unwrap();551 let (ops, object, _) = insertion(sid, Place::In(outline, None), "Text", &[]);
632 let created = PreparedEdit::insert(&source, sid, &intent).unwrap();552 let created = ops::edited(&source, ops.clone()).unwrap();
633 let before_missing =553 let before_created = Place::In(outline, Some(object));
634 Insertion::paragraph(outline, Some(intent.object()), "Anchored", "Author").unwrap();554 assert!(refused(&source, before_created, "Anchored"));
635 assert!(PreparedEdit::insert(&source, sid, &before_missing).is_err());555 assert!(!refused(&created, before_created, "Anchored"));
636 assert!(PreparedEdit::insert(created.as_bytes(), sid, &before_missing).is_ok());556 // A new object's identity may not already be on the page.
637 let mut encoded = serde_json::to_value(&intent).unwrap();557 assert!(ops::edited(&created, ops).is_err());
638 encoded["parent"] = serde_json::to_value(intent.object()).unwrap();
639 let collision: Insertion = serde_json::from_value(encoded).unwrap();
640 assert!(PreparedEdit::insert(created.as_bytes(), sid, &collision).is_err());
641}558}
642559
643#[test]560#[test]
644fn insertions_into_nested_paragraphs_and_native_table_cells_preserve_structure() {561fn insertions_into_nested_paragraphs_and_native_table_cells_preserve_structure() {
645 let source = onestore::create_section("nested.one", "Original", "Author").unwrap();562 let source = onestore::create_section("nested.one", "Original", "Author").unwrap();
646 let (sid, _, outline, paragraph, _) = targets(&source);563 let (sid, _, outline, paragraph, _) = targets(&source);
647 let child = Insertion::paragraph(paragraph, None, "Nested", "Author").unwrap();564 let mut nested = ops::paragraph("Nested");
648 let changed = PreparedEdit::insert(&source, sid, &child).unwrap();565 nested.level = 2;
649 let store = Store::parse(changed.as_bytes()).unwrap();566 let child = nested.id;
567 let insert = PageOp::Insert {
568 container: paragraph,
569 before: None,
570 paragraphs: vec![nested],
571 };
572 let changed = ops::page_edited(&source, sid, vec![insert]).unwrap();
573 let store = Store::parse(&changed).unwrap();
650 let index = RevisionIndex::parse(&store).unwrap();574 let index = RevisionIndex::parse(&store).unwrap();
651 let document = Document::parse(&index).unwrap();575 let document = Document::parse(&index).unwrap();
652 let space = &document.spaces[&sid];576 let space = &document.spaces[&sid];
653 let view = &space.revisions[&space.contexts[&ExGuid::default()]];577 let view = &space.revisions[&space.contexts[&ExGuid::default()]];
654 assert_eq!(view.nodes[&outline].children, [paragraph]);578 assert_eq!(view.nodes[&outline].children, [paragraph]);
655 assert_eq!(view.nodes[&paragraph].children, [child.object()]);579 assert_eq!(view.nodes[&paragraph].children, [child]);
656 let source = include_bytes!(580 let source = include_bytes!(
657 "../../../corpus/native/20260905-05/snapshots/07-table/notebook/synthetic.one"581 "../../../corpus/native/20260905-05/snapshots/07-table/notebook/synthetic.one"
658 );582 );
...@@ -669,34 +593,32 @@ fn insertions_into_nested_paragraphs_and_native_table_cells_preserve_structure()...@@ -669,34 +593,32 @@ fn insertions_into_nested_paragraphs_and_native_table_cells_preserve_structure()
669 })593 })
670 })594 })
671 .unwrap();595 .unwrap();
672 let insertion = Insertion::paragraph(cell, None, "Added to cell", "Author").unwrap();596 let (ops, _, _) = insertion(sid, Place::In(cell, None), "Added to cell", &[]);
673 let changed = PreparedEdit::insert(source, sid, &insertion).unwrap();597 current::current(&ops::edited(source, ops).unwrap());
674 current::current(changed.as_bytes());
675}598}
676599
677#[test]600#[test]
678fn insertion_publication_faults_expose_only_complete_graphs_and_title_caches() {601fn insertion_publication_faults_expose_only_complete_graphs_and_title_caches() {
679 let original = onestore::create_section("atomic.one", "Original", "Author").unwrap();602 let original = onestore::create_section("atomic.one", "Original", "Author").unwrap();
680 let (sid, _, _, _, text) = targets(&original);603 let (sid, _, _, _, text) = targets(&original);
681 let checkpoint = checkpoint::pending(&original, sid, text, 0x14001d7a);604 let checkpoint = checkpoint::pending(&original, sid, text);
682 for source in [605 for source in [
683 original.as_slice(),606 original.as_slice(),
684 include_bytes!("../../../corpus/append/round-01/tx-255/notebook/synthetic.one"),607 include_bytes!("../../../corpus/append/round-01/tx-255/notebook/synthetic.one"),
685 &checkpoint,608 &checkpoint,
686 ] {609 ] {
687 let (sid, page, outline, paragraph, _) = targets(source);610 let (sid, _, outline, paragraph, _) = targets(source);
688 for intent in [611 for place in [Place::In(outline, Some(paragraph)), Place::Outline(0.0, 0.0)] {
689 Insertion::paragraph(outline, Some(paragraph), "First 🦀", "New author").unwrap(),612 let formats = [
690 Insertion::outline(page, 0.0, 0.0, "First 🦀", "New author").unwrap(),613 (0..3, vec![A::Bold(true), A::FontSize(18.0)]),
691 ] {614 (6..8, vec![A::Italic(true), A::Highlight(Some([255, 255, 0]))]),
692 let intent = intent615 ];
693 .with_formatting(0..3, &[A::Bold(true), A::FontSize(18.0)])616 let (ops, _, _) = insertion(sid, place, "First 🦀", &formats);
694 .unwrap()617 let edit = ops::transaction(source, "New author", ops).unwrap().unwrap();
695 .with_formatting(6..8, &[A::Italic(true), A::Highlight(Some([255, 255, 0]))])618 let mut written = source.to_vec();
696 .unwrap();619 edit.apply(&mut written).unwrap();
697 let edit = PreparedEdit::insert(source, sid, &intent).unwrap();
698 let before = current::current(source);620 let before = current::current(source);
699 let after = current::current(edit.as_bytes());621 let after = current::current(&written);
700 for write_limit in [17, 4096] {622 for write_limit in [17, 4096] {
701 let disk = |fail_at| disk::Disk {623 let disk = |fail_at| disk::Disk {
702 visible: source.to_vec(),624 visible: source.to_vec(),
...@@ -708,7 +630,7 @@ fn insertion_publication_faults_expose_only_complete_graphs_and_title_caches() {...@@ -708,7 +630,7 @@ fn insertion_publication_faults_expose_only_complete_graphs_and_title_caches() {
708 };630 };
709 let mut successful = disk(None);631 let mut successful = disk(None);
710 edit.commit(&mut successful).unwrap();632 edit.commit(&mut successful).unwrap();
711 assert_eq!(successful.durable, edit.as_bytes());633 assert_eq!(successful.durable, written);
712 for at in 1..=successful.operation {634 for at in 1..=successful.operation {
713 let mut interrupted = disk(Some(at));635 let mut interrupted = disk(Some(at));
714 let failure = edit.commit(&mut interrupted).unwrap_err();636 let failure = edit.commit(&mut interrupted).unwrap_err();
crates/onestore/tests/outline.rs+29-51
...@@ -1,6 +1,7 @@...@@ -1,6 +1,7 @@
1use onestore::{1use onestore::{
2 ExGuid, OutlineEdit as Edit, PreparedEdit, RevisionIndex, Store,2 ExGuid, OutlineEdit as Edit, RevisionIndex, Store,
3 document::{Document, Kind},3 document::{Document, Kind},
4 op::PageOp,
4};5};
5use std::collections::BTreeSet;6use std::collections::BTreeSet;
67
...@@ -8,6 +9,8 @@ use std::collections::BTreeSet;...@@ -8,6 +9,8 @@ use std::collections::BTreeSet;
8mod current;9mod current;
9#[path = "support/disk.rs"]10#[path = "support/disk.rs"]
10mod disk;11mod disk;
12#[path = "support/ops.rs"]
13mod ops;
1114
12const SOURCE: &[u8] = include_bytes!("../../../corpus/outline-edit/before/notebook/synthetic.one");15const SOURCE: &[u8] = include_bytes!("../../../corpus/outline-edit/before/notebook/synthetic.one");
1316
...@@ -62,7 +65,7 @@ fn geometry_and_expansion_preserve_unrelated_properties_objects_and_history() {...@@ -62,7 +65,7 @@ fn geometry_and_expansion_preserve_unrelated_properties_objects_and_history() {
62 for (name, sid, object, edit) in cases(&document) {65 for (name, sid, object, edit) in cases(&document) {
63 let restored = serde_json::from_value(serde_json::to_value(edit).unwrap()).unwrap();66 let restored = serde_json::from_value(serde_json::to_value(edit).unwrap()).unwrap();
64 assert_eq!(edit, restored);67 assert_eq!(edit, restored);
65 let prepared = PreparedEdit::outline(SOURCE, sid, object, restored).unwrap();68 let prepared = ops::page_op(SOURCE, sid, PageOp::Outline { object, edit: restored }).unwrap();
66 let after_store = Store::parse(prepared.as_bytes()).unwrap();69 let after_store = Store::parse(prepared.as_bytes()).unwrap();
67 assert!(after_store.checksum_mismatches.is_empty());70 assert!(after_store.checksum_mismatches.is_empty());
68 let after_index = RevisionIndex::parse(&after_store).unwrap();71 let after_index = RevisionIndex::parse(&after_store).unwrap();
...@@ -139,7 +142,7 @@ fn geometry_and_expansion_preserve_unrelated_properties_objects_and_history() {...@@ -139,7 +142,7 @@ fn geometry_and_expansion_preserve_unrelated_properties_objects_and_history() {
139 assert_eq!(old_space.labels, after_index.spaces[space_id].labels);142 assert_eq!(old_space.labels, after_index.spaces[space_id].labels);
140 }143 }
141 }144 }
142 let again = PreparedEdit::outline(prepared.as_bytes(), sid, object, edit).unwrap();145 let again = ops::page_op(prepared.as_bytes(), sid, PageOp::Outline { object, edit }).unwrap();
143 assert_eq!(again.as_bytes(), prepared.as_bytes());146 assert_eq!(again.as_bytes(), prepared.as_bytes());
144 }147 }
145}148}
...@@ -159,20 +162,15 @@ fn invalid_geometry_and_non_outline_targets_are_rejected() {...@@ -159,20 +162,15 @@ fn invalid_geometry_and_non_outline_targets_are_rejected() {
159 user_set: true,162 user_set: true,
160 },163 },
161 ] {164 ] {
162 assert!(PreparedEdit::outline(SOURCE, sid, object, edit).is_err());165 assert!(ops::page_op(SOURCE, sid, PageOp::Outline { object, edit }).is_err());
163 }166 }
164 }167 }
165 for points in [-1.0, 0.0, 35.999] {168 for points in [-1.0, 0.0, 35.999] {
166 assert!(169 assert!(
167 PreparedEdit::outline(170 ops::page_op(SOURCE, sid, PageOp::Outline { object, edit: Edit::Width {
168 SOURCE,
169 sid,
170 object,
171 Edit::Width {
172 points,171 points,
173 user_set: false172 user_set: false
174 }173 } })
175 )
176 .is_err()174 .is_err()
177 );175 );
178 }176 }
...@@ -181,7 +179,7 @@ fn invalid_geometry_and_non_outline_targets_are_rejected() {...@@ -181,7 +179,7 @@ fn invalid_geometry_and_non_outline_targets_are_rejected() {
181 for (id, node) in &view.nodes {179 for (id, node) in &view.nodes {
182 if !matches!(node.kind, Kind::Outline { .. }) {180 if !matches!(node.kind, Kind::Outline { .. }) {
183 assert!(181 assert!(
184 PreparedEdit::outline(SOURCE, sid, *id, Edit::Position { x: 72.0, y: 72.0 })182 ops::page_op(SOURCE, sid, PageOp::Outline { object: *id, edit: Edit::Position { x: 72.0, y: 72.0 } })
185 .is_err()183 .is_err()
186 );184 );
187 }185 }
...@@ -197,20 +195,20 @@ fn invalid_geometry_and_non_outline_targets_are_rejected() {...@@ -197,20 +195,20 @@ fn invalid_geometry_and_non_outline_targets_are_rejected() {
197 },195 },
198 Edit::Collapsed(true),196 Edit::Collapsed(true),
199 ] {197 ] {
200 assert!(PreparedEdit::outline(SOURCE, sid, child, edit).is_err());198 assert!(ops::page_op(SOURCE, sid, PageOp::Outline { object: child, edit }).is_err());
201 }199 }
202 }200 }
203 }201 }
204 if !matches!(node.kind, Kind::Paragraph { .. }) {202 if !matches!(node.kind, Kind::Paragraph { .. }) {
205 assert!(PreparedEdit::outline(SOURCE, sid, *id, Edit::Collapsed(true)).is_err());203 assert!(ops::page_op(SOURCE, sid, PageOp::Outline { object: *id, edit: Edit::Collapsed(true) }).is_err());
206 }204 }
207 }205 }
208 assert!(206 assert!(
209 PreparedEdit::outline(SOURCE, ExGuid::default(), object, Edit::Collapsed(true))207 ops::page_op(SOURCE, ExGuid::default(), PageOp::Outline { object, edit: Edit::Collapsed(true) })
210 .is_err()208 .is_err()
211 );209 );
212 assert!(210 assert!(
213 PreparedEdit::outline(SOURCE, sid, ExGuid::default(), Edit::Collapsed(true)).is_err()211 ops::page_op(SOURCE, sid, PageOp::Outline { object: ExGuid::default(), edit: Edit::Collapsed(true) }).is_err()
214 );212 );
215 }213 }
216}214}
...@@ -228,15 +226,10 @@ fn resizing_a_native_reserved_width_preserves_content() {...@@ -228,15 +226,10 @@ fn resizing_a_native_reserved_width_preserves_content() {
228 let space = &document.spaces[&sid];226 let space = &document.spaces[&sid];
229 let before = &space.revisions[&space.contexts[&ExGuid::default()]];227 let before = &space.revisions[&space.contexts[&ExGuid::default()]];
230 assert!(before.nodes[&object].layout.reserved_width.is_some());228 assert!(before.nodes[&object].layout.reserved_width.is_some());
231 let edit = PreparedEdit::outline(229 let edit = ops::page_op(source, sid, PageOp::Outline { object, edit: Edit::Width {
232 source,
233 sid,
234 object,
235 Edit::Width {
236 points: 144.0,230 points: 144.0,
237 user_set: true,231 user_set: true,
238 },232 } })
239 )
240 .unwrap();233 .unwrap();
241 let after_store = Store::parse(edit.as_bytes()).unwrap();234 let after_store = Store::parse(edit.as_bytes()).unwrap();
242 let after_index = RevisionIndex::parse(&after_store).unwrap();235 let after_index = RevisionIndex::parse(&after_store).unwrap();
...@@ -270,12 +263,8 @@ fn outline_movement_updates_both_automatic_title_fields_without_changing_content...@@ -270,12 +263,8 @@ fn outline_movement_updates_both_automatic_title_fields_without_changing_content
270 let index = RevisionIndex::parse(&store).unwrap();263 let index = RevisionIndex::parse(&store).unwrap();
271 let document = Document::parse(&index).unwrap();264 let document = Document::parse(&index).unwrap();
272 let (sid, page) = document.pages().unwrap()[0];265 let (sid, page) = document.pages().unwrap()[0];
273 let insertion =266 let (add, insertion, _) = ops::new_outline(144.0, 144.0, "Second 🦋 é");
274 onestore::Insertion::outline(page, 144.0, 144.0, "Second 🦋 é", "Author").unwrap();267 let source = ops::page_op(&source, sid, add).unwrap().image;
275 let source = PreparedEdit::insert(&source, sid, &insertion)
276 .unwrap()
277 .as_bytes()
278 .to_vec();
279 let store = Store::parse(&source).unwrap();268 let store = Store::parse(&source).unwrap();
280 let index = RevisionIndex::parse(&store).unwrap();269 let index = RevisionIndex::parse(&store).unwrap();
281 let document = Document::parse(&index).unwrap();270 let document = Document::parse(&index).unwrap();
...@@ -288,7 +277,7 @@ fn outline_movement_updates_both_automatic_title_fields_without_changing_content...@@ -288,7 +277,7 @@ fn outline_movement_updates_both_automatic_title_fields_without_changing_content
288 (144.0, 36.0, "First"),277 (144.0, 36.0, "First"),
289 (0.0, 144.0, "First"),278 (0.0, 144.0, "First"),
290 ] {279 ] {
291 let edit = PreparedEdit::outline(&source, sid, insertion.object(), Edit::Position { x, y })280 let edit = ops::page_op(&source, sid, PageOp::Outline { object: insertion, edit: Edit::Position { x, y } })
292 .unwrap();281 .unwrap();
293 let saved = Store::parse(edit.as_bytes()).unwrap();282 let saved = Store::parse(edit.as_bytes()).unwrap();
294 let saved_index = RevisionIndex::parse(&saved).unwrap();283 let saved_index = RevisionIndex::parse(&saved).unwrap();
...@@ -308,7 +297,7 @@ fn outline_movement_updates_both_automatic_title_fields_without_changing_content...@@ -308,7 +297,7 @@ fn outline_movement_updates_both_automatic_title_fields_without_changing_content
308 after.nodes.keys().collect::<Vec<_>>()297 after.nodes.keys().collect::<Vec<_>>()
309 );298 );
310 for (oid, node) in &before.nodes {299 for (oid, node) in &before.nodes {
311 if ![page, metadata, insertion.object()].contains(oid) {300 if ![page, metadata, insertion].contains(oid) {
312 assert_eq!(301 assert_eq!(
313 serde_json::to_value(node).unwrap(),302 serde_json::to_value(node).unwrap(),
314 serde_json::to_value(&after.nodes[oid]).unwrap()303 serde_json::to_value(&after.nodes[oid]).unwrap()
...@@ -320,12 +309,7 @@ fn outline_movement_updates_both_automatic_title_fields_without_changing_content...@@ -320,12 +309,7 @@ fn outline_movement_updates_both_automatic_title_fields_without_changing_content
320 format!("{:?}", saved_index.resolve(sid, rid).unwrap())309 format!("{:?}", saved_index.resolve(sid, rid).unwrap())
321 );310 );
322 assert_eq!(311 assert_eq!(
323 PreparedEdit::outline(312 ops::page_op(edit.as_bytes(), sid, PageOp::Outline { object: insertion, edit: Edit::Position { x, y } })
324 edit.as_bytes(),
325 sid,
326 insertion.object(),
327 Edit::Position { x, y }
328 )
329 .unwrap()313 .unwrap()
330 .as_bytes(),314 .as_bytes(),
331 edit.as_bytes()315 edit.as_bytes()
...@@ -366,7 +350,7 @@ fn native_automatic_and_explicit_titles_follow_outline_movement() {...@@ -366,7 +350,7 @@ fn native_automatic_and_explicit_titles_follow_outline_movement() {
366 72.0350 72.0
367 },351 },
368 };352 };
369 candidate = PreparedEdit::outline(&candidate, sid, outline, change)353 candidate = ops::page_op(&candidate, sid, PageOp::Outline { object: outline, edit: change })
370 .unwrap()354 .unwrap()
371 .as_bytes()355 .as_bytes()
372 .to_vec();356 .to_vec();
...@@ -418,11 +402,8 @@ fn outline_publication_interruptions_reopen_as_complete_old_or_new_layout() {...@@ -418,11 +402,8 @@ fn outline_publication_interruptions_reopen_as_complete_old_or_new_layout() {
418 .find(|id| matches!(view.nodes[id].kind, Kind::Outline { .. }))402 .find(|id| matches!(view.nodes[id].kind, Kind::Outline { .. }))
419 .unwrap();403 .unwrap();
420 let paragraph = view.nodes[&outline].children[0];404 let paragraph = view.nodes[&outline].children[0];
421 let second = onestore::Insertion::outline(page, 144.0, 36.0, "Second", "Author").unwrap();405 let (add, _, _) = ops::new_outline(144.0, 36.0, "Second");
422 let source = PreparedEdit::insert(&source, sid, &second)406 let source = ops::page_op(&source, sid, add).unwrap().image;
423 .unwrap()
424 .as_bytes()
425 .to_vec();
426 for (object, operation) in [407 for (object, operation) in [
427 (outline, Edit::Position { x: 216.0, y: 72.0 }),408 (outline, Edit::Position { x: 216.0, y: 72.0 }),
428 (409 (
...@@ -434,7 +415,7 @@ fn outline_publication_interruptions_reopen_as_complete_old_or_new_layout() {...@@ -434,7 +415,7 @@ fn outline_publication_interruptions_reopen_as_complete_old_or_new_layout() {
434 ),415 ),
435 (paragraph, Edit::Collapsed(true)),416 (paragraph, Edit::Collapsed(true)),
436 ] {417 ] {
437 let edit = PreparedEdit::outline(&source, sid, object, operation).unwrap();418 let edit = ops::page_op(&source, sid, PageOp::Outline { object, edit: operation }).unwrap();
438 let before = current::current(&source);419 let before = current::current(&source);
439 let after = current::current(edit.as_bytes());420 let after = current::current(edit.as_bytes());
440 for write_limit in [17, 4096] {421 for write_limit in [17, 4096] {
...@@ -481,11 +462,8 @@ fn repeated_geometry_changes_and_expansion_match_an_independent_model() {...@@ -481,11 +462,8 @@ fn repeated_geometry_changes_and_expansion_match_an_independent_model() {
481 .unwrap();462 .unwrap();
482 let paragraph = view.nodes[&outline].children[0];463 let paragraph = view.nodes[&outline].children[0];
483 let text = view.nodes[&paragraph].content[0];464 let text = view.nodes[&paragraph].content[0];
484 let second = onestore::Insertion::outline(page, 144.0, 36.0, "Second", "Author").unwrap();465 let (add, _, _) = ops::new_outline(144.0, 36.0, "Second");
485 let original = PreparedEdit::insert(&original, sid, &second)466 let original = ops::page_op(&original, sid, add).unwrap().image;
486 .unwrap()
487 .as_bytes()
488 .to_vec();
489 for seed in 1..=16_u64 {467 for seed in 1..=16_u64 {
490 let mut rng = seed;468 let mut rng = seed;
491 let mut source = original.clone();469 let mut source = original.clone();
...@@ -520,7 +498,7 @@ fn repeated_geometry_changes_and_expansion_match_an_independent_model() {...@@ -520,7 +498,7 @@ fn repeated_geometry_changes_and_expansion_match_an_independent_model() {
520 (paragraph, Edit::Collapsed(rng & 1 != 0))498 (paragraph, Edit::Collapsed(rng & 1 != 0))
521 }499 }
522 };500 };
523 source = PreparedEdit::outline(&source, sid, object, edit)501 source = ops::page_op(&source, sid, PageOp::Outline { object, edit })
524 .unwrap()502 .unwrap()
525 .as_bytes()503 .as_bytes()
526 .to_vec();504 .to_vec();
...@@ -574,7 +552,7 @@ fn export_native_outline_candidates() {...@@ -574,7 +552,7 @@ fn export_native_outline_candidates() {
574 let document = Document::parse(&index).unwrap();552 let document = Document::parse(&index).unwrap();
575 let mut bytes = SOURCE.to_vec();553 let mut bytes = SOURCE.to_vec();
576 for (_, sid, object, edit) in cases(&document) {554 for (_, sid, object, edit) in cases(&document) {
577 bytes = PreparedEdit::outline(&bytes, sid, object, edit)555 bytes = ops::page_op(&bytes, sid, PageOp::Outline { object, edit })
578 .unwrap()556 .unwrap()
579 .as_bytes()557 .as_bytes()
580 .to_vec();558 .to_vec();
crates/onestore/tests/page.rs+94-24
...@@ -1,6 +1,7 @@...@@ -1,6 +1,7 @@
1use onestore::{1use onestore::{
2 ExGuid, Insertion, PageCreation, PreparedEdit, RevisionIndex, Store,2 ExGuid, PageCreation, RevisionIndex, Store,
3 document::{Document, Kind},3 document::{Document, Kind},
4 op::{PageOp, SectionOp},
4 page::{Page, PageObject},5 page::{Page, PageObject},
5};6};
67
...@@ -8,6 +9,8 @@ use onestore::{...@@ -8,6 +9,8 @@ use onestore::{
8mod current;9mod current;
9#[path = "support/disk.rs"]10#[path = "support/disk.rs"]
10mod disk;11mod disk;
12#[path = "support/ops.rs"]
13mod ops;
1114
12const SOURCE: &[u8] =15const SOURCE: &[u8] =
13 include_bytes!("../../../corpus/page-lifecycle/03-renamed/notebook/Lifecycle.one");16 include_bytes!("../../../corpus/page-lifecycle/03-renamed/notebook/Lifecycle.one");
...@@ -28,7 +31,7 @@ fn native_section_accepts_empty_and_titled_pages_with_preserved_history() {...@@ -28,7 +31,7 @@ fn native_section_accepts_empty_and_titled_pages_with_preserved_history() {
28 let serialized = serde_json::to_vec(&intent).unwrap();31 let serialized = serde_json::to_vec(&intent).unwrap();
29 let retained = serde_json::from_slice(&serialized).unwrap();32 let retained = serde_json::from_slice(&serialized).unwrap();
30 assert_eq!(intent, retained);33 assert_eq!(intent, retained);
31 let prepared = PreparedEdit::create_page(&source, &retained).unwrap();34 let prepared = ops::section_op(&source, SectionOp::Create(retained.clone())).unwrap();
32 let store = Store::parse(prepared.as_bytes()).unwrap();35 let store = Store::parse(prepared.as_bytes()).unwrap();
33 assert_eq!(36 assert_eq!(
34 store.header.transaction_count,37 store.header.transaction_count,
...@@ -78,7 +81,7 @@ fn native_section_accepts_empty_and_titled_pages_with_preserved_history() {...@@ -78,7 +81,7 @@ fn native_section_accepts_empty_and_titled_pages_with_preserved_history() {
78 assert_eq!(space.labels, index.spaces[sid].labels);81 assert_eq!(space.labels, index.spaces[sid].labels);
79 }82 }
80 }83 }
81 assert!(PreparedEdit::create_page(prepared.as_bytes(), &intent).is_err());84 assert!(ops::section_op(prepared.as_bytes(), SectionOp::Create(intent.clone())).is_err());
82 source = prepared.as_bytes().to_vec();85 source = prepared.as_bytes().to_vec();
83 }86 }
84 if let Some(output) = std::env::var_os("ONESTORE_PAGE_CREATION_OUTPUT") {87 if let Some(output) = std::env::var_os("ONESTORE_PAGE_CREATION_OUTPUT") {
...@@ -92,13 +95,12 @@ fn created_pages_support_title_edits_and_body_insertion() {...@@ -92,13 +95,12 @@ fn created_pages_support_title_edits_and_body_insertion() {
92 let mut source = onestore::create_section("pages.one", "Original", "Author").unwrap();95 let mut source = onestore::create_section("pages.one", "Original", "Author").unwrap();
93 for title in [None, Some(""), Some("Explicit 🦋 é")] {96 for title in [None, Some(""), Some("Explicit 🦋 é")] {
94 let intent = PageCreation::new(None, title, "Author").unwrap();97 let intent = PageCreation::new(None, title, "Author").unwrap();
95 source = PreparedEdit::create_page(&source, &intent)98 source = ops::section_op(&source, SectionOp::Create(intent.clone()))
96 .unwrap()99 .unwrap()
97 .as_bytes()100 .as_bytes()
98 .to_vec();101 .to_vec();
99 let insertion =102 let (add, ..) = ops::new_outline(36.0, 36.0, "Body 🦀 é");
100 Insertion::outline(intent.object(), 36.0, 36.0, "Body 🦀 é", "Author").unwrap();103 source = ops::page_op(&source, intent.space(), add)
101 source = PreparedEdit::insert(&source, intent.space(), &insertion)
102 .unwrap()104 .unwrap()
103 .as_bytes()105 .as_bytes()
104 .to_vec();106 .to_vec();
...@@ -115,13 +117,7 @@ fn created_pages_support_title_edits_and_body_insertion() {...@@ -115,13 +117,7 @@ fn created_pages_support_title_edits_and_body_insertion() {
115 Some(title.filter(|s| !s.is_empty()).unwrap_or("Body 🦀 é"))117 Some(title.filter(|s| !s.is_empty()).unwrap_or("Body 🦀 é"))
116 );118 );
117 if let Some(text) = intent.title_object() {119 if let Some(text) = intent.title_object() {
118 source = PreparedEdit::text(120 source = ops::page_op(&source, intent.space(), PageOp::Text { text, range: 0..title.unwrap().encode_utf16().count() as u32, with: "Renamed".into() })
119 &source,
120 intent.space(),
121 text,
122 0..title.unwrap().encode_utf16().count() as u32,
123 "Renamed",
124 )
125 .unwrap()121 .unwrap()
126 .as_bytes()122 .as_bytes()
127 .to_vec();123 .to_vec();
...@@ -149,13 +145,13 @@ fn created_body_outlines_carry_the_indentation_table_onenote_writes() {...@@ -149,13 +145,13 @@ fn created_body_outlines_carry_the_indentation_table_onenote_writes() {
149 let [first] = document.active(space).unwrap().nodes[&page].children[..] else {145 let [first] = document.active(space).unwrap().nodes[&page].children[..] else {
150 panic!()146 panic!()
151 };147 };
152 let insertion = Insertion::outline(page, 72.0, 144.0, "Second", "Author").unwrap();148 let (add, insertion, _) = ops::new_outline(72.0, 144.0, "Second");
153 let written = PreparedEdit::insert(&source, space, &insertion).unwrap();149 let written = ops::page_op(&source, space, add).unwrap();
154 let store = Store::parse(written.as_bytes()).unwrap();150 let store = Store::parse(written.as_bytes()).unwrap();
155 let index = RevisionIndex::parse(&store).unwrap();151 let index = RevisionIndex::parse(&store).unwrap();
156 let document = Document::parse(&index).unwrap();152 let document = Document::parse(&index).unwrap();
157 let view = document.active(space).unwrap();153 let view = document.active(space).unwrap();
158 for outline in [first, insertion.object()] {154 for outline in [first, insertion] {
159 let Kind::Outline { indents } = &view.nodes[&outline].kind else {155 let Kind::Outline { indents } = &view.nodes[&outline].kind else {
160 panic!()156 panic!()
161 };157 };
...@@ -167,7 +163,7 @@ fn created_body_outlines_carry_the_indentation_table_onenote_writes() {...@@ -167,7 +163,7 @@ fn created_body_outlines_carry_the_indentation_table_onenote_writes() {
167fn a_created_title_outline_is_stored_and_read_as_onenote_stores_it() {163fn a_created_title_outline_is_stored_and_read_as_onenote_stores_it() {
168 let source = onestore::create_section("pages.one", "Original", "Author").unwrap();164 let source = onestore::create_section("pages.one", "Original", "Author").unwrap();
169 let intent = PageCreation::new(None, Some("Title"), "Author").unwrap();165 let intent = PageCreation::new(None, Some("Title"), "Author").unwrap();
170 let prepared = PreparedEdit::create_page(&source, &intent).unwrap();166 let prepared = ops::section_op(&source, SectionOp::Create(intent.clone())).unwrap();
171 let store = Store::parse(prepared.as_bytes()).unwrap();167 let store = Store::parse(prepared.as_bytes()).unwrap();
172 let index = RevisionIndex::parse(&store).unwrap();168 let index = RevisionIndex::parse(&store).unwrap();
173 let document = Document::parse(&index).unwrap();169 let document = Document::parse(&index).unwrap();
...@@ -222,11 +218,11 @@ fn invalid_page_intents_and_nonleading_anchors_reject_before_publication() {...@@ -222,11 +218,11 @@ fn invalid_page_intents_and_nonleading_anchors_reject_before_publication() {
222 let index = RevisionIndex::parse(&store).unwrap();218 let index = RevisionIndex::parse(&store).unwrap();
223 let pages = Document::parse(&index).unwrap().pages().unwrap();219 let pages = Document::parse(&index).unwrap().pages().unwrap();
224 let intent = PageCreation::new(Some(pages[4].0), Some("Child"), "Author").unwrap();220 let intent = PageCreation::new(Some(pages[4].0), Some("Child"), "Author").unwrap();
225 assert!(PreparedEdit::create_page(source, &intent).is_err());221 assert!(ops::section_op(source, SectionOp::Create(intent.clone())).is_err());
226 let mut value = serde_json::to_value(PageCreation::new(None, None, "Author").unwrap()).unwrap();222 let mut value = serde_json::to_value(PageCreation::new(None, None, "Author").unwrap()).unwrap();
227 value["guid"] = serde_json::to_value([0_u8; 16]).unwrap();223 value["guid"] = serde_json::to_value([0_u8; 16]).unwrap();
228 let invalid: PageCreation = serde_json::from_value(value.clone()).unwrap();224 let invalid: PageCreation = serde_json::from_value(value.clone()).unwrap();
229 assert!(PreparedEdit::create_page(source, &invalid).is_err());225 assert!(ops::section_op(source, SectionOp::Create(invalid.clone())).is_err());
230 value["unknown"] = true.into();226 value["unknown"] = true.into();
231 assert!(serde_json::from_value::<PageCreation>(value).is_err());227 assert!(serde_json::from_value::<PageCreation>(value).is_err());
232}228}
...@@ -235,7 +231,7 @@ fn invalid_page_intents_and_nonleading_anchors_reject_before_publication() {...@@ -235,7 +231,7 @@ fn invalid_page_intents_and_nonleading_anchors_reject_before_publication() {
235fn new_space_and_section_entry_publish_as_one_complete_action() {231fn new_space_and_section_entry_publish_as_one_complete_action() {
236 let source = onestore::create_section("pages.one", "Original", "Author").unwrap();232 let source = onestore::create_section("pages.one", "Original", "Author").unwrap();
237 let intent = PageCreation::new(None, Some("New 🦋 é"), "Author").unwrap();233 let intent = PageCreation::new(None, Some("New 🦋 é"), "Author").unwrap();
238 let prepared = PreparedEdit::create_page(&source, &intent).unwrap();234 let prepared = ops::section_op(&source, SectionOp::Create(intent.clone())).unwrap();
239 let old = current::current(&source);235 let old = current::current(&source);
240 let new = current::current(prepared.as_bytes());236 let new = current::current(prepared.as_bytes());
241 assert_eq!(new.len(), old.len() + 1);237 assert_eq!(new.len(), old.len() + 1);
...@@ -285,7 +281,7 @@ fn repeated_page_creation_crosses_root_fragments_counters_and_section_checkpoint...@@ -285,7 +281,7 @@ fn repeated_page_creation_crosses_root_fragments_counters_and_section_checkpoint
285 "Stress author",281 "Stress author",
286 )282 )
287 .unwrap();283 .unwrap();
288 let prepared = PreparedEdit::create_page(&source, &intent).unwrap();284 let prepared = ops::section_op(&source, SectionOp::Create(intent.clone())).unwrap();
289 let mut unpublished = prepared.as_bytes().to_vec();285 let mut unpublished = prepared.as_bytes().to_vec();
290 unpublished[96..100].copy_from_slice(&source[96..100]);286 unpublished[96..100].copy_from_slice(&source[96..100]);
291 let old_store = Store::parse(&unpublished).unwrap();287 let old_store = Store::parse(&unpublished).unwrap();
...@@ -363,7 +359,7 @@ fn native_changes_on_created_pages_accept_rust_followups() {...@@ -363,7 +359,7 @@ fn native_changes_on_created_pages_accept_rust_followups() {
363 let mut source = initial.to_vec();359 let mut source = initial.to_vec();
364 for (sid, title, body) in targets {360 for (sid, title, body) in targets {
365 for (object, prefix) in [(body, "Rust + "), (title, "Reviewed ")] {361 for (object, prefix) in [(body, "Rust + "), (title, "Reviewed ")] {
366 source = PreparedEdit::text(&source, sid, object, 0..0, prefix)362 source = ops::page_op(&source, sid, PageOp::Text { text: object, range: 0..0, with: prefix.into() })
367 .unwrap()363 .unwrap()
368 .as_bytes()364 .as_bytes()
369 .to_vec();365 .to_vec();
...@@ -400,7 +396,7 @@ fn page_creation_preserves_native_features_and_file_data() {...@@ -400,7 +396,7 @@ fn page_creation_preserves_native_features_and_file_data() {
400 let old = RevisionIndex::parse(&before).unwrap();396 let old = RevisionIndex::parse(&before).unwrap();
401 let original = Document::parse(&old).unwrap();397 let original = Document::parse(&old).unwrap();
402 let intent = PageCreation::new(None, Some("Preserved features"), "Author").unwrap();398 let intent = PageCreation::new(None, Some("Preserved features"), "Author").unwrap();
403 let prepared = PreparedEdit::create_page(source, &intent).unwrap();399 let prepared = ops::section_op(source, SectionOp::Create(intent.clone())).unwrap();
404 let after = Store::parse(prepared.as_bytes()).unwrap();400 let after = Store::parse(prepared.as_bytes()).unwrap();
405 let new = RevisionIndex::parse(&after).unwrap();401 let new = RevisionIndex::parse(&after).unwrap();
406 let document = Document::parse(&new).unwrap();402 let document = Document::parse(&new).unwrap();
...@@ -427,3 +423,77 @@ fn page_creation_preserves_native_features_and_file_data() {...@@ -427,3 +423,77 @@ fn page_creation_preserves_native_features_and_file_data() {
427 }423 }
428 }424 }
429}425}
426
427/// A page created into an empty section as OneNote 2010 creates one (its title's date and
428/// time fields, then a colour) reads back with both; a page kept from elsewhere keeps its
429/// identity and creation time.
430#[test]
431fn dated_pages_in_an_empty_section_read_back_with_their_date_and_colour() {
432 let source = onestore::create_empty_section("New Section 1.one", Some(0x00e4a88a)).unwrap();
433 let store = Store::parse(&source).unwrap();
434 let index = RevisionIndex::parse(&store).unwrap();
435 index.validate_current().unwrap();
436 assert!(Document::parse(&index).unwrap().pages().unwrap().is_empty());
437 let creation = PageCreation::new(None, Some(""), "Rust Author")
438 .unwrap()
439 .dated("Sunday, September 27, 2026", "2:23 AM")
440 .unwrap();
441 let retained: PageCreation =
442 serde_json::from_slice(&serde_json::to_vec(&creation).unwrap()).unwrap();
443 assert_eq!(retained, creation);
444 let space = creation.space();
445 let identity = [7; 16];
446 let kept = PageCreation::new(None, Some("Kept"), "Rust Author")
447 .unwrap()
448 .keeping(identity, 133_700_000_001_234_567)
449 .unwrap();
450 let image = ops::edited(
451 &source,
452 vec![
453 onestore::op::Op::Section(SectionOp::Create(creation.clone())),
454 onestore::op::Op::Page {
455 space,
456 op: PageOp::Color(Some(0x00f2f9d4)),
457 },
458 onestore::op::Op::Section(SectionOp::Create(kept.clone())),
459 ],
460 )
461 .unwrap();
462 let store = Store::parse(&image).unwrap();
463 let index = RevisionIndex::parse(&store).unwrap();
464 index.validate_current().unwrap();
465 let document = Document::parse(&index).unwrap();
466 let page = Page::from_space(&document, space).unwrap();
467 assert_eq!(page.color, Some(0x00f2f9d4));
468 assert_eq!(page.created, Some(creation.created()));
469 let [PageObject::Title(title)] = &page.objects[..] else {
470 panic!("{:?}", page.objects)
471 };
472 let shown: Vec<_> = title.outlines[1]
473 .paragraphs
474 .iter()
475 .map(|paragraph| {
476 let text = paragraph.text().unwrap();
477 (text.date_field, text.text.text().to_owned())
478 })
479 .collect();
480 assert_eq!(title.date, Some(title.outlines[1].id));
481 assert_eq!(
482 shown,
483 [
484 // OneNote's own new pages read the same: the fields' order names their roles.
485 (None, "Sunday, September 27, 2026".to_owned()),
486 (None, "2:23 AM".to_owned()),
487 ]
488 );
489 let kept_page = Page::from_space(&document, kept.space()).unwrap();
490 assert_eq!(kept_page.identity, Some(identity));
491 assert_eq!(kept_page.created, Some(133_700_000_001_234_567));
492 assert_eq!(kept_page.color, None);
493 // Clearing the colour stores OneNote's "No color".
494 let cleared = ops::page_edited(&image, space, vec![PageOp::Color(None)]).unwrap();
495 let store = Store::parse(&cleared).unwrap();
496 let index = RevisionIndex::parse(&store).unwrap();
497 let document = Document::parse(&index).unwrap();
498 assert_eq!(Page::from_space(&document, space).unwrap().color, None);
499}
crates/onestore/tests/page_attachments.rs+16-15
...@@ -1,5 +1,7 @@...@@ -1,5 +1,7 @@
1#[path = "support/ops.rs"]
2mod ops;
1use onestore::{3use onestore::{
2 ExGuid, PreparedEdit, RevisionIndex, Store,4 ExGuid, RevisionIndex, Store,
3 document::{Document, Format},5 document::{Document, Format},
4 page::{6 page::{
5 Attachment, Page, PageObject, PageParagraph, ParagraphContent, TextObject, text::new_id,7 Attachment, Page, PageObject, PageParagraph, ParagraphContent, TextObject, text::new_id,
...@@ -9,7 +11,6 @@ use std::sync::Arc;...@@ -9,7 +11,6 @@ use std::sync::Arc;
911
10const NATIVE: &[u8] =12const NATIVE: &[u8] =
11 include_bytes!("../../../corpus/native/cold-05-06-attachment/notebook/synthetic.one");13 include_bytes!("../../../corpus/native/cold-05-06-attachment/notebook/synthetic.one");
12const AUTHOR: &str = "Attachment author";
13const PAYLOAD: &[u8] = b"Rust wrote this attachment.\n";14const PAYLOAD: &[u8] = b"Rust wrote this attachment.\n";
1415
15fn first_page(bytes: &[u8]) -> (ExGuid, Page) {16fn first_page(bytes: &[u8]) -> (ExGuid, Page) {
...@@ -117,8 +118,8 @@ fn insert_attachment(section: &str, preview: Option<Arc<[u8]>>) -> Vec<u8> {...@@ -117,8 +118,8 @@ fn insert_attachment(section: &str, preview: Option<Arc<[u8]>>) -> Vec<u8> {
117 });118 });
118 body_paragraphs(&mut after).push(holder.clone());119 body_paragraphs(&mut after).push(holder.clone());
119 body_paragraphs(&mut after).push(plain_paragraph(&template, "After the file"));120 body_paragraphs(&mut after).push(plain_paragraph(&template, "After the file"));
120 let written = PreparedEdit::page(&source, space, &after, AUTHOR).unwrap();121 let written = ops::saved(&source, space, &after).unwrap();
121 let stored = page_in(written.as_bytes(), space);122 let stored = page_in(written.as_slice(), space);
122 let mut expected = after.clone();123 let mut expected = after.clone();
123 expected.title = stored.title.clone();124 expected.title = stored.title.clone();
124 assert_eq!(stored, expected);125 assert_eq!(stored, expected);
...@@ -129,16 +130,16 @@ fn insert_attachment(section: &str, preview: Option<Arc<[u8]>>) -> Vec<u8> {...@@ -129,16 +130,16 @@ fn insert_attachment(section: &str, preview: Option<Arc<[u8]>>) -> Vec<u8> {
129 assert_eq!(stored_attachment.bytes.as_deref(), Some(PAYLOAD));130 assert_eq!(stored_attachment.bytes.as_deref(), Some(PAYLOAD));
130 assert_eq!(stored_attachment.preview, preview);131 assert_eq!(stored_attachment.preview, preview);
131 assert_eq!(132 assert_eq!(
132 PreparedEdit::page(written.as_bytes(), space, &stored, AUTHOR)133 ops::saved(written.as_slice(), space, &stored)
133 .unwrap()134 .unwrap()
134 .as_bytes(),135 .as_slice(),
135 written.as_bytes()136 written.as_slice()
136 );137 );
137 let mut removed = stored.clone();138 let mut removed = stored.clone();
138 body_paragraphs(&mut removed).retain(|p| p.id != holder.id);139 body_paragraphs(&mut removed).retain(|p| p.id != holder.id);
139 let again = PreparedEdit::page(written.as_bytes(), space, &removed, AUTHOR).unwrap();140 let again = ops::saved(written.as_slice(), space, &removed).unwrap();
140 assert!(attachments(&page_in(again.as_bytes(), space)).is_empty());141 assert!(attachments(&page_in(again.as_slice(), space)).is_empty());
141 written.as_bytes().to_vec()142 written.as_slice().to_vec()
142}143}
143144
144/// `ONESTORE_ATTACHMENT_EXPORT` names a directory receiving both candidates for a cold reopen:145/// `ONESTORE_ATTACHMENT_EXPORT` names a directory receiving both candidates for a cold reopen:
...@@ -186,7 +187,7 @@ fn attachments_need_a_file_name_and_stored_ones_are_renamed_in_place() {...@@ -186,7 +187,7 @@ fn attachments_need_a_file_name_and_stored_ones_are_renamed_in_place() {
186 recording: None,187 recording: None,
187 });188 });
188 body_paragraphs(&mut after).push(holder);189 body_paragraphs(&mut after).push(holder);
189 assert!(PreparedEdit::page(&source, space, &after, AUTHOR).is_err());190 assert!(ops::saved(&source, space, &after).is_err());
190 let (space, native) = first_page(NATIVE);191 let (space, native) = first_page(NATIVE);
191 let mut renamed = native.clone();192 let mut renamed = native.clone();
192 let mut found = 0;193 let mut found = 0;
...@@ -202,8 +203,8 @@ fn attachments_need_a_file_name_and_stored_ones_are_renamed_in_place() {...@@ -202,8 +203,8 @@ fn attachments_need_a_file_name_and_stored_ones_are_renamed_in_place() {
202 }203 }
203 }204 }
204 assert_eq!(found, 1);205 assert_eq!(found, 1);
205 let written = PreparedEdit::page(NATIVE, space, &renamed, AUTHOR).unwrap();206 let written = ops::saved(NATIVE, space, &renamed).unwrap();
206 let stored = page_in(written.as_bytes(), space);207 let stored = page_in(written.as_slice(), space);
207 let (before, after) = (attachments(&native)[0], attachments(&stored)[0]);208 let (before, after) = (attachments(&native)[0], attachments(&stored)[0]);
208 assert_eq!(after.filename, "renamed \u{1f980}.txt");209 assert_eq!(after.filename, "renamed \u{1f980}.txt");
209 assert_eq!(210 assert_eq!(
...@@ -216,8 +217,8 @@ fn attachments_need_a_file_name_and_stored_ones_are_renamed_in_place() {...@@ -216,8 +217,8 @@ fn attachments_need_a_file_name_and_stored_ones_are_renamed_in_place() {
216 if let Some(directory) = std::env::var_os("ONESTORE_ATTACHMENT_RENAME_EXPORT") {217 if let Some(directory) = std::env::var_os("ONESTORE_ATTACHMENT_RENAME_EXPORT") {
217 let directory = std::path::PathBuf::from(directory);218 let directory = std::path::PathBuf::from(directory);
218 std::fs::create_dir_all(&directory).unwrap();219 std::fs::create_dir_all(&directory).unwrap();
219 std::fs::write(directory.join("synthetic.one"), written.as_bytes()).unwrap();220 std::fs::write(directory.join("synthetic.one"), written.as_slice()).unwrap();
220 let file_id = Store::parse(written.as_bytes()).unwrap().header.file_id;221 let file_id = Store::parse(written.as_slice()).unwrap().header.file_id;
221 std::fs::write(222 std::fs::write(
222 directory.join("Open Notebook.onetoc2"),223 directory.join("Open Notebook.onetoc2"),
223 onestore::create_table_of_contents(224 onestore::create_table_of_contents(
crates/onestore/tests/page_deletion.rs+23-13
...@@ -1,6 +1,7 @@...@@ -1,6 +1,7 @@
1use onestore::{1use onestore::{
2 ExGuid, FileDataReference, ObjectData, PreparedEdit, PropertySets, RevisionIndex, Store,2 ExGuid, FileDataReference, ObjectData, PropertySets, RevisionIndex, Store,
3 document::{Document, Kind},3 document::{Document, Kind},
4 op::{PageOp, SectionOp},
4};5};
5use std::collections::BTreeSet;6use std::collections::BTreeSet;
67
...@@ -130,7 +131,7 @@ fn removal_matches_all_native_cases_and_retains_every_old_revision() {...@@ -130,7 +131,7 @@ fn removal_matches_all_native_cases_and_retains_every_old_revision() {
130 .into_iter()131 .into_iter()
131 .map(|ordinal| pages[ordinal].0)132 .map(|ordinal| pages[ordinal].0)
132 .collect();133 .collect();
133 let prepared = PreparedEdit::delete_pages_permanently(&original, &removed).unwrap();134 let prepared = ops::section_op(&original, SectionOp::Delete(removed.to_vec())).unwrap();
134 if let Some(output) = std::env::var_os("ONESTORE_PAGE_REMOVAL_OUTPUT") {135 if let Some(output) = std::env::var_os("ONESTORE_PAGE_REMOVAL_OUTPUT") {
135 let output = std::path::Path::new(&output);136 let output = std::path::Path::new(&output);
136 assert!(output.is_absolute());137 assert!(output.is_absolute());
...@@ -140,7 +141,7 @@ fn removal_matches_all_native_cases_and_retains_every_old_revision() {...@@ -140,7 +141,7 @@ fn removal_matches_all_native_cases_and_retains_every_old_revision() {
140 std::fs::write(case.join("Lifecycle.one"), prepared.as_bytes()).unwrap();141 std::fs::write(case.join("Lifecycle.one"), prepared.as_bytes()).unwrap();
141 }142 }
142 verify(&original, prepared.as_bytes(), &native, &removed);143 verify(&original, prepared.as_bytes(), &native, &removed);
143 assert!(PreparedEdit::delete_pages_permanently(prepared.as_bytes(), &removed).is_err());144 assert!(ops::section_op(prepared.as_bytes(), SectionOp::Delete(removed.to_vec())).is_err());
144 }145 }
145}146}
146147
...@@ -160,22 +161,22 @@ fn removal_rejects_duplicate_missing_and_non_page_spaces_without_publication() {...@@ -160,22 +161,22 @@ fn removal_rejects_duplicate_missing_and_non_page_spaces_without_publication() {
160 }],161 }],
161 vec![pages[0].0, index.root],162 vec![pages[0].0, index.root],
162 ] {163 ] {
163 assert!(PreparedEdit::delete_pages_permanently(SOURCE, &selected).is_err());164 assert!(ops::section_op(SOURCE, SectionOp::Delete(selected.to_vec())).is_err());
164 }165 }
165 assert_eq!(166 assert_eq!(
166 PreparedEdit::delete_pages_permanently(SOURCE, &[])167 ops::section_op(SOURCE, SectionOp::Delete([].to_vec()))
167 .unwrap()168 .unwrap()
168 .as_bytes(),169 .as_bytes(),
169 SOURCE170 SOURCE
170 );171 );
171 let written = PreparedEdit::delete_pages_permanently(SOURCE, &[pages[0].0]).unwrap();172 let written = ops::section_op(SOURCE, SectionOp::Delete([pages[0].0].to_vec())).unwrap();
172 assert!(173 assert!(
173 PreparedEdit::delete_pages_permanently(written.as_bytes(), &[pages[1].0, pages[0].0])174 ops::section_op(written.as_bytes(), SectionOp::Delete([pages[1].0, pages[0].0].to_vec()))
174 .is_err()175 .is_err()
175 );176 );
176}177}
177178
178use page_schedule::{current, disk};179use page_schedule::{current, disk, ops};
179180
180/// OneNote created a page in each section after Rust permanently removed pages from it;181/// OneNote created a page in each section after Rust permanently removed pages from it;
181/// the surviving pages keep their revisions through the native save and a cold reopen.182/// the surviving pages keep their revisions through the native save and a cold reopen.
...@@ -267,7 +268,7 @@ fn tombstones_and_first_page_promotion_survive_each_storage_interruption() {...@@ -267,7 +268,7 @@ fn tombstones_and_first_page_promotion_survive_each_storage_interruption() {
267 .take(if all { pages.len() } else { 1 })268 .take(if all { pages.len() } else { 1 })
268 .map(|page| page.0)269 .map(|page| page.0)
269 .collect();270 .collect();
270 let prepared = PreparedEdit::delete_pages_permanently(source, &removed).unwrap();271 let prepared = ops::section_op(source, SectionOp::Delete(removed.to_vec())).unwrap();
271 let old = current::current(source);272 let old = current::current(source);
272 let new = current::current(prepared.as_bytes());273 let new = current::current(prepared.as_bytes());
273 assert_ne!(old, new);274 assert_ne!(old, new);
...@@ -331,8 +332,17 @@ fn stale_removal_cannot_delete_a_newer_page_edit() {...@@ -331,8 +332,17 @@ fn stale_removal_cannot_delete_a_newer_page_edit() {
331 _ => None,332 _ => None,
332 })333 })
333 .unwrap();334 .unwrap();
334 let removal = PreparedEdit::delete_pages_permanently(SOURCE, &[sid]).unwrap();335 let removal = ops::section_op(SOURCE, SectionOp::Delete([sid].to_vec())).unwrap();
335 let remote = onestore::replace_text(SOURCE, sid, text, 0..0, "New remote content ").unwrap();336 let remote = ops::page_edited(
337 SOURCE,
338 sid,
339 vec![PageOp::Text {
340 text,
341 range: 0..0,
342 with: "New remote content ".into(),
343 }],
344 )
345 .unwrap();
336 let mut disk = disk::Disk {346 let mut disk = disk::Disk {
337 visible: remote.clone(),347 visible: remote.clone(),
338 durable: remote.clone(),348 durable: remote.clone(),
...@@ -380,7 +390,7 @@ fn external_payloads_and_their_historical_references_survive_file_removal() {...@@ -380,7 +390,7 @@ fn external_payloads_and_their_historical_references_survive_file_removal() {
380 let pages = Document::parse(&index).unwrap().pages().unwrap();390 let pages = Document::parse(&index).unwrap().pages().unwrap();
381 assert_eq!(pages.len(), 3);391 assert_eq!(pages.len(), 3);
382 let selected: Vec<_> = pages.iter().map(|page| page.0).collect();392 let selected: Vec<_> = pages.iter().map(|page| page.0).collect();
383 let prepared = PreparedEdit::delete_pages_permanently(&original, &selected).unwrap();393 let prepared = ops::section_op(&original, SectionOp::Delete(selected.to_vec())).unwrap();
384 let output = std::env::var_os("ONESTORE_PAGE_REMOVAL_EXTERNAL_OUTPUT");394 let output = std::env::var_os("ONESTORE_PAGE_REMOVAL_EXTERNAL_OUTPUT");
385 let root = output395 let root = output
386 .as_ref()396 .as_ref()
...@@ -409,7 +419,7 @@ fn external_payloads_and_their_historical_references_survive_file_removal() {...@@ -409,7 +419,7 @@ fn external_payloads_and_their_historical_references_survive_file_removal() {
409 assert_eq!(payloads.len(), 3);419 assert_eq!(payloads.len(), 3);
410 let path = root.join("synthetic.one");420 let path = root.join("synthetic.one");
411 std::fs::write(&path, &original).unwrap();421 std::fs::write(&path, &original).unwrap();
412 prepared.commit_file(&path).unwrap();422 prepared.transaction.as_ref().unwrap().commit_file(&path).unwrap();
413 let written = onestore::read_file(&path).unwrap();423 let written = onestore::read_file(&path).unwrap();
414 assert_eq!(written, prepared.as_bytes());424 assert_eq!(written, prepared.as_bytes());
415 let current_store = Store::parse(&written).unwrap();425 let current_store = Store::parse(&written).unwrap();
crates/onestore/tests/page_images.rs+39-38
...@@ -1,5 +1,7 @@...@@ -1,5 +1,7 @@
1#[path = "support/ops.rs"]
2mod ops;
1use onestore::{3use onestore::{
2 ExGuid, PreparedEdit, RevisionIndex, Store,4 ExGuid, RevisionIndex, Store,
3 document::{Document, Format, Layout},5 document::{Document, Format, Layout},
4 page::{Image, Page, PageObject, PageParagraph, ParagraphContent, TextObject, text::new_id},6 page::{Image, Page, PageObject, PageParagraph, ParagraphContent, TextObject, text::new_id},
5};7};
...@@ -11,7 +13,6 @@ const NATIVE_RESIZE: &[u8] =...@@ -11,7 +13,6 @@ const NATIVE_RESIZE: &[u8] =
11 include_bytes!("../../../corpus/picture-edit/native-resize/notebook/pictures.one");13 include_bytes!("../../../corpus/picture-edit/native-resize/notebook/pictures.one");
12const NATIVE_PAGE_LEVEL: &[u8] =14const NATIVE_PAGE_LEVEL: &[u8] =
13 include_bytes!("../../../corpus/picture-edit/native-page-level/notebook/pictures.one");15 include_bytes!("../../../corpus/picture-edit/native-page-level/notebook/pictures.one");
14const AUTHOR: &str = "Picture author";
15/// A one-pixel PNG.16/// A one-pixel PNG.
16const PNG: &[u8] = &[17const PNG: &[u8] = &[
17 0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a, 0x00, 0x00, 0x00, 0x0d, 0x49, 0x48, 0x44, 0x52,18 0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a, 0x00, 0x00, 0x00, 0x0d, 0x49, 0x48, 0x44, 0x52,
...@@ -120,8 +121,8 @@ fn a_picture_inserted_on_a_fresh_page_reads_back_and_can_be_removed() {...@@ -120,8 +121,8 @@ fn a_picture_inserted_on_a_fresh_page_reads_back_and_can_be_removed() {
120 holder.content = ParagraphContent::Image(image.clone());121 holder.content = ParagraphContent::Image(image.clone());
121 body_paragraphs(&mut after).push(holder.clone());122 body_paragraphs(&mut after).push(holder.clone());
122 body_paragraphs(&mut after).push(plain_paragraph(&template, "After the picture"));123 body_paragraphs(&mut after).push(plain_paragraph(&template, "After the picture"));
123 let written = PreparedEdit::page(&source, space, &after, AUTHOR).unwrap();124 let written = ops::saved(&source, space, &after).unwrap();
124 let stored = page_in(written.as_bytes(), space);125 let stored = page_in(written.as_slice(), space);
125 let mut expected = after.clone();126 let mut expected = after.clone();
126 expected.title = stored.title.clone();127 expected.title = stored.title.clone();
127 assert_eq!(stored, expected);128 assert_eq!(stored, expected);
...@@ -131,16 +132,16 @@ fn a_picture_inserted_on_a_fresh_page_reads_back_and_can_be_removed() {...@@ -131,16 +132,16 @@ fn a_picture_inserted_on_a_fresh_page_reads_back_and_can_be_removed() {
131 };132 };
132 assert_eq!(picture.bytes.as_deref(), Some(PNG));133 assert_eq!(picture.bytes.as_deref(), Some(PNG));
133 assert_eq!(134 assert_eq!(
134 PreparedEdit::page(written.as_bytes(), space, &stored, AUTHOR)135 ops::saved(written.as_slice(), space, &stored)
135 .unwrap()136 .unwrap()
136 .as_bytes(),137 .as_slice(),
137 written.as_bytes()138 written.as_slice()
138 );139 );
139 if let Some(directory) = std::env::var_os("ONESTORE_IMAGE_EXPORT") {140 if let Some(directory) = std::env::var_os("ONESTORE_IMAGE_EXPORT") {
140 let directory = std::path::PathBuf::from(directory);141 let directory = std::path::PathBuf::from(directory);
141 std::fs::create_dir(&directory).unwrap();142 std::fs::create_dir(&directory).unwrap();
142 std::fs::write(directory.join("pictures.one"), written.as_bytes()).unwrap();143 std::fs::write(directory.join("pictures.one"), written.as_slice()).unwrap();
143 let written_store = Store::parse(written.as_bytes()).unwrap();144 let written_store = Store::parse(written.as_slice()).unwrap();
144 std::fs::write(145 std::fs::write(
145 directory.join("Open Notebook.onetoc2"),146 directory.join("Open Notebook.onetoc2"),
146 onestore::create_table_of_contents(147 onestore::create_table_of_contents(
...@@ -153,8 +154,8 @@ fn a_picture_inserted_on_a_fresh_page_reads_back_and_can_be_removed() {...@@ -153,8 +154,8 @@ fn a_picture_inserted_on_a_fresh_page_reads_back_and_can_be_removed() {
153 }154 }
154 let mut removed = stored.clone();155 let mut removed = stored.clone();
155 body_paragraphs(&mut removed).retain(|p| p.id != holder.id);156 body_paragraphs(&mut removed).retain(|p| p.id != holder.id);
156 let again = PreparedEdit::page(written.as_bytes(), space, &removed, AUTHOR).unwrap();157 let again = ops::saved(written.as_slice(), space, &removed).unwrap();
157 let stored = page_in(again.as_bytes(), space);158 let stored = page_in(again.as_slice(), space);
158 assert!(pictures(&stored).is_empty());159 assert!(pictures(&stored).is_empty());
159 assert_eq!(body_paragraphs(&mut stored.clone()).len(), 2);160 assert_eq!(body_paragraphs(&mut stored.clone()).len(), 2);
160}161}
...@@ -187,20 +188,20 @@ fn a_native_picture_is_resized_and_described_then_reset() {...@@ -187,20 +188,20 @@ fn a_native_picture_is_resized_and_described_then_reset() {
187 ..Default::default()188 ..Default::default()
188 };189 };
189 resize(&mut resized, layout, Some("Resized in Rust"));190 resize(&mut resized, layout, Some("Resized in Rust"));
190 let written = PreparedEdit::page(NATIVE, space, &resized, AUTHOR).unwrap();191 let written = ops::saved(NATIVE, space, &resized).unwrap();
191 let stored = page_in(written.as_bytes(), space);192 let stored = page_in(written.as_slice(), space);
192 assert_eq!(stored, resized);193 assert_eq!(stored, resized);
193 assert_eq!(194 assert_eq!(
194 PreparedEdit::page(written.as_bytes(), space, &stored, AUTHOR)195 ops::saved(written.as_slice(), space, &stored)
195 .unwrap()196 .unwrap()
196 .as_bytes(),197 .as_slice(),
197 written.as_bytes()198 written.as_slice()
198 );199 );
199 if let Some(directory) = std::env::var_os("ONESTORE_IMAGE_RESIZE_EXPORT") {200 if let Some(directory) = std::env::var_os("ONESTORE_IMAGE_RESIZE_EXPORT") {
200 let directory = std::path::PathBuf::from(directory);201 let directory = std::path::PathBuf::from(directory);
201 std::fs::create_dir(&directory).unwrap();202 std::fs::create_dir(&directory).unwrap();
202 std::fs::write(directory.join("synthetic.one"), written.as_bytes()).unwrap();203 std::fs::write(directory.join("synthetic.one"), written.as_slice()).unwrap();
203 let written_store = Store::parse(written.as_bytes()).unwrap();204 let written_store = Store::parse(written.as_slice()).unwrap();
204 std::fs::write(205 std::fs::write(
205 directory.join("Open Notebook.onetoc2"),206 directory.join("Open Notebook.onetoc2"),
206 onestore::create_table_of_contents(207 onestore::create_table_of_contents(
...@@ -213,8 +214,8 @@ fn a_native_picture_is_resized_and_described_then_reset() {...@@ -213,8 +214,8 @@ fn a_native_picture_is_resized_and_described_then_reset() {
213 }214 }
214 let mut reset = stored.clone();215 let mut reset = stored.clone();
215 resize(&mut reset, Layout::default(), None);216 resize(&mut reset, Layout::default(), None);
216 let again = PreparedEdit::page(written.as_bytes(), space, &reset, AUTHOR).unwrap();217 let again = ops::saved(written.as_slice(), space, &reset).unwrap();
217 assert_eq!(page_in(again.as_bytes(), space), native);218 assert_eq!(page_in(again.as_slice(), space), native);
218}219}
219220
220#[test]221#[test]
...@@ -234,8 +235,8 @@ fn a_rust_picture_resized_natively_reads_back_with_its_size_and_description() {...@@ -234,8 +235,8 @@ fn a_rust_picture_resized_natively_reads_back_with_its_size_and_description() {
234 let mut layout = picture.layout.clone();235 let mut layout = picture.layout.clone();
235 layout.max_width = Some(200.0);236 layout.max_width = Some(200.0);
236 resize(&mut wider, layout, Some("Wider in Rust"));237 resize(&mut wider, layout, Some("Wider in Rust"));
237 let written = PreparedEdit::page(NATIVE_RESIZE, space, &wider, AUTHOR).unwrap();238 let written = ops::saved(NATIVE_RESIZE, space, &wider).unwrap();
238 assert_eq!(page_in(written.as_bytes(), space), wider);239 assert_eq!(page_in(written.as_slice(), space), wider);
239}240}
240241
241fn page_pictures(page: &Page) -> Vec<&Image> {242fn page_pictures(page: &Page) -> Vec<&Image> {
...@@ -270,8 +271,8 @@ fn a_page_level_picture_placed_natively_reads_as_a_page_object() {...@@ -270,8 +271,8 @@ fn a_page_level_picture_placed_natively_reads_as_a_page_object() {
270 image.layout.y = Some(400.0);271 image.layout.y = Some(400.0);
271 }272 }
272 }273 }
273 let written = PreparedEdit::page(NATIVE_PAGE_LEVEL, space, &moved, AUTHOR).unwrap();274 let written = ops::saved(NATIVE_PAGE_LEVEL, space, &moved).unwrap();
274 assert_eq!(page_in(written.as_bytes(), space), moved);275 assert_eq!(page_in(written.as_slice(), space), moved);
275}276}
276277
277/// `ONESTORE_PAGE_IMAGE_EXPORT` names a new directory receiving the candidate for a cold reopen.278/// `ONESTORE_PAGE_IMAGE_EXPORT` names a new directory receiving the candidate for a cold reopen.
...@@ -296,23 +297,23 @@ fn a_page_level_picture_is_inserted_moved_and_removed() {...@@ -296,23 +297,23 @@ fn a_page_level_picture_is_inserted_moved_and_removed() {
296 background: false,297 background: false,
297 };298 };
298 after.objects.push(PageObject::Image(image.clone()));299 after.objects.push(PageObject::Image(image.clone()));
299 let written = PreparedEdit::page(&source, space, &after, AUTHOR).unwrap();300 let written = ops::saved(&source, space, &after).unwrap();
300 let stored = page_in(written.as_bytes(), space);301 let stored = page_in(written.as_slice(), space);
301 let mut expected = after.clone();302 let mut expected = after.clone();
302 expected.title = stored.title.clone();303 expected.title = stored.title.clone();
303 assert_eq!(stored, expected);304 assert_eq!(stored, expected);
304 assert_eq!(page_pictures(&stored)[0].bytes.as_deref(), Some(PNG));305 assert_eq!(page_pictures(&stored)[0].bytes.as_deref(), Some(PNG));
305 assert_eq!(306 assert_eq!(
306 PreparedEdit::page(written.as_bytes(), space, &stored, AUTHOR)307 ops::saved(written.as_slice(), space, &stored)
307 .unwrap()308 .unwrap()
308 .as_bytes(),309 .as_slice(),
309 written.as_bytes()310 written.as_slice()
310 );311 );
311 if let Some(directory) = std::env::var_os("ONESTORE_PAGE_IMAGE_EXPORT") {312 if let Some(directory) = std::env::var_os("ONESTORE_PAGE_IMAGE_EXPORT") {
312 let directory = std::path::PathBuf::from(directory);313 let directory = std::path::PathBuf::from(directory);
313 std::fs::create_dir(&directory).unwrap();314 std::fs::create_dir(&directory).unwrap();
314 std::fs::write(directory.join("pictures.one"), written.as_bytes()).unwrap();315 std::fs::write(directory.join("pictures.one"), written.as_slice()).unwrap();
315 let written_store = Store::parse(written.as_bytes()).unwrap();316 let written_store = Store::parse(written.as_slice()).unwrap();
316 std::fs::write(317 std::fs::write(
317 directory.join("Open Notebook.onetoc2"),318 directory.join("Open Notebook.onetoc2"),
318 onestore::create_table_of_contents(319 onestore::create_table_of_contents(
...@@ -330,19 +331,19 @@ fn a_page_level_picture_is_inserted_moved_and_removed() {...@@ -330,19 +331,19 @@ fn a_page_level_picture_is_inserted_moved_and_removed() {
330 image.alt = None;331 image.alt = None;
331 }332 }
332 }333 }
333 let again = PreparedEdit::page(written.as_bytes(), space, &moved, AUTHOR).unwrap();334 let again = ops::saved(written.as_slice(), space, &moved).unwrap();
334 assert_eq!(page_in(again.as_bytes(), space), moved);335 assert_eq!(page_in(again.as_slice(), space), moved);
335 let mut removed = moved.clone();336 let mut removed = moved.clone();
336 removed.objects.retain(|object| object.id() != image.id);337 removed.objects.retain(|object| object.id() != image.id);
337 let last = PreparedEdit::page(again.as_bytes(), space, &removed, AUTHOR).unwrap();338 let last = ops::saved(again.as_slice(), space, &removed).unwrap();
338 let stored = page_in(last.as_bytes(), space);339 let stored = page_in(last.as_slice(), space);
339 assert!(page_pictures(&stored).is_empty());340 assert!(page_pictures(&stored).is_empty());
340 assert_eq!(stored, removed);341 assert_eq!(stored, removed);
341 let mut unplaced = stored.clone();342 let mut unplaced = stored.clone();
342 let mut nowhere = image.clone();343 let mut nowhere = image.clone();
343 nowhere.layout = Layout::default();344 nowhere.layout = Layout::default();
344 unplaced.objects.push(PageObject::Image(nowhere));345 unplaced.objects.push(PageObject::Image(nowhere));
345 assert!(PreparedEdit::page(last.as_bytes(), space, &unplaced, AUTHOR).is_err());346 assert!(ops::saved(last.as_slice(), space, &unplaced).is_err());
346}347}
347348
348#[test]349#[test]
...@@ -361,7 +362,7 @@ fn pictures_need_a_recognised_payload_and_stored_ones_stay_fixed() {...@@ -361,7 +362,7 @@ fn pictures_need_a_recognised_payload_and_stored_ones_stay_fixed() {
361 background: false,362 background: false,
362 });363 });
363 body_paragraphs(&mut after).push(holder);364 body_paragraphs(&mut after).push(holder);
364 assert!(PreparedEdit::page(&source, space, &after, AUTHOR).is_err());365 assert!(ops::saved(&source, space, &after).is_err());
365 let (space, native) = first_page(NATIVE);366 let (space, native) = first_page(NATIVE);
366 let mut resized = native.clone();367 let mut resized = native.clone();
367 let mut found = 0;368 let mut found = 0;
...@@ -376,5 +377,5 @@ fn pictures_need_a_recognised_payload_and_stored_ones_stay_fixed() {...@@ -376,5 +377,5 @@ fn pictures_need_a_recognised_payload_and_stored_ones_stay_fixed() {
376 }377 }
377 }378 }
378 assert_eq!(found, 1);379 assert_eq!(found, 1);
379 assert!(PreparedEdit::page(NATIVE, space, &resized, AUTHOR).is_err());380 assert!(ops::saved(NATIVE, space, &resized).is_err());
380}381}
crates/onestore/tests/page_ink.rs+17-16
...@@ -1,10 +1,11 @@...@@ -1,10 +1,11 @@
1#[path = "support/ops.rs"]
2mod ops;
1use onestore::{3use onestore::{
2 ExGuid, PreparedEdit, RevisionIndex, Store,4 ExGuid, RevisionIndex, Store,
3 document::Document,5 document::Document,
4 page::{Ink, InkStroke, Page, PageObject, text::new_id},6 page::{Ink, InkStroke, Page, PageObject, text::new_id},
5};7};
68
7const AUTHOR: &str = "Ink author";
89
9/// Two drawings made with the mouse in OneNote 2010 (`tools/native/ink.ahk`): a diamond and a10/// Two drawings made with the mouse in OneNote 2010 (`tools/native/ink.ahk`): a diamond and a
10/// diagonal line, whose native read reports their positions and sizes.11/// diagonal line, whose native read reports their positions and sizes.
...@@ -142,8 +143,8 @@ fn an_ink_drawing_is_written_erased_stroke_by_stroke_and_removed() {...@@ -142,8 +143,8 @@ fn an_ink_drawing_is_written_erased_stroke_by_stroke_and_removed() {
142 let mut after = before.clone();143 let mut after = before.clone();
143 let ink = drawing();144 let ink = drawing();
144 after.objects.push(PageObject::Ink(ink.clone()));145 after.objects.push(PageObject::Ink(ink.clone()));
145 let written = PreparedEdit::page(&source, space, &after, AUTHOR).unwrap();146 let written = ops::saved(&source, space, &after).unwrap();
146 let stored = page_in(written.as_bytes(), space);147 let stored = page_in(written.as_slice(), space);
147 let mut expected = after.clone();148 let mut expected = after.clone();
148 expected.title = stored.title.clone();149 expected.title = stored.title.clone();
149 assert_eq!(stored, expected);150 assert_eq!(stored, expected);
...@@ -157,12 +158,12 @@ fn an_ink_drawing_is_written_erased_stroke_by_stroke_and_removed() {...@@ -157,12 +158,12 @@ fn an_ink_drawing_is_written_erased_stroke_by_stroke_and_removed() {
157 ]158 ]
158 ));159 ));
159 assert_eq!(160 assert_eq!(
160 PreparedEdit::page(written.as_bytes(), space, &stored, AUTHOR)161 ops::saved(written.as_slice(), space, &stored)
161 .unwrap()162 .unwrap()
162 .as_bytes(),163 .as_slice(),
163 written.as_bytes()164 written.as_slice()
164 );165 );
165 export("ONESTORE_INK_EXPORT", "ink.one", written.as_bytes());166 export("ONESTORE_INK_EXPORT", "ink.one", written.as_slice());
166 let mut erased = stored.clone();167 let mut erased = stored.clone();
167 for object in &mut erased.objects {168 for object in &mut erased.objects {
168 if let PageObject::Ink(ink) = object {169 if let PageObject::Ink(ink) = object {
...@@ -171,13 +172,13 @@ fn an_ink_drawing_is_written_erased_stroke_by_stroke_and_removed() {...@@ -171,13 +172,13 @@ fn an_ink_drawing_is_written_erased_stroke_by_stroke_and_removed() {
171 .push(stroke(&[[330.0, 100.0], [330.0, 200.0]], Some(0xff0000)));172 .push(stroke(&[[330.0, 100.0], [330.0, 200.0]], Some(0xff0000)));
172 }173 }
173 }174 }
174 let again = PreparedEdit::page(written.as_bytes(), space, &erased, AUTHOR).unwrap();175 let again = ops::saved(written.as_slice(), space, &erased).unwrap();
175 let stored = page_in(again.as_bytes(), space);176 let stored = page_in(again.as_slice(), space);
176 assert_eq!(stored, erased);177 assert_eq!(stored, erased);
177 let mut removed = stored.clone();178 let mut removed = stored.clone();
178 removed.objects.retain(|object| object.id() != ink.id);179 removed.objects.retain(|object| object.id() != ink.id);
179 let last = PreparedEdit::page(again.as_bytes(), space, &removed, AUTHOR).unwrap();180 let last = ops::saved(again.as_slice(), space, &removed).unwrap();
180 let stored = page_in(last.as_bytes(), space);181 let stored = page_in(last.as_slice(), space);
181 assert!(drawings(&stored).is_empty());182 assert!(drawings(&stored).is_empty());
182 assert_eq!(stored, removed);183 assert_eq!(stored, removed);
183}184}
...@@ -209,15 +210,15 @@ fn handwriting_is_written_as_paragraph_content() {...@@ -209,15 +210,15 @@ fn handwriting_is_written_as_paragraph_content() {
209 paragraph.format = Default::default();210 paragraph.format = Default::default();
210 paragraph.content = ParagraphContent::Ink(drawing());211 paragraph.content = ParagraphContent::Ink(drawing());
211 outline.paragraphs.push(paragraph);212 outline.paragraphs.push(paragraph);
212 let written = PreparedEdit::page(&source, space, &after, AUTHOR).unwrap();213 let written = ops::saved(&source, space, &after).unwrap();
213 let stored = page_in(written.as_bytes(), space);214 let stored = page_in(written.as_slice(), space);
214 let mut expected = after.clone();215 let mut expected = after.clone();
215 expected.title = stored.title.clone();216 expected.title = stored.title.clone();
216 assert_eq!(stored, expected);217 assert_eq!(stored, expected);
217 export(218 export(
218 "ONESTORE_INK_PARAGRAPH_EXPORT",219 "ONESTORE_INK_PARAGRAPH_EXPORT",
219 "ink.one",220 "ink.one",
220 written.as_bytes(),221 written.as_slice(),
221 );222 );
222}223}
223224
...@@ -230,5 +231,5 @@ fn stored_strokes_keep_their_paths() {...@@ -230,5 +231,5 @@ fn stored_strokes_keep_their_paths() {
230 ink.strokes[0].points[0][0] += 1.0;231 ink.strokes[0].points[0][0] += 1.0;
231 }232 }
232 }233 }
233 assert!(PreparedEdit::page(NATIVE, space, &moved, AUTHOR).is_err());234 assert!(ops::saved(NATIVE, space, &moved).is_err());
234}235}
crates/onestore/tests/page_links.rs+17-15
...@@ -1,5 +1,7 @@...@@ -1,5 +1,7 @@
1#[path = "support/ops.rs"]
2mod ops;
1use onestore::{3use onestore::{
2 ExGuid, PreparedEdit, RevisionIndex, Store,4 ExGuid, RevisionIndex, Store,
3 document::Document,5 document::Document,
4 page::{6 page::{
5 Page, PageObject, PageParagraph, Paragraph,7 Page, PageObject, PageParagraph, Paragraph,
...@@ -13,7 +15,6 @@ const OUTLINES: &[u8] =...@@ -13,7 +15,6 @@ const OUTLINES: &[u8] =
13const NATIVE_LINKS: &[u8] =15const NATIVE_LINKS: &[u8] =
14 include_bytes!("../../../corpus/link-edit/native-links/notebook/links.one");16 include_bytes!("../../../corpus/link-edit/native-links/notebook/links.one");
15const NATIVE_BASE_PATH: &str = r"C:\one-tests\runs\capture\notebook\links.one";17const NATIVE_BASE_PATH: &str = r"C:\one-tests\runs\capture\notebook\links.one";
16const AUTHOR: &str = "Link author";
17const CODE: &str = "\u{fddf}HYPERLINK \"https://example.invalid/rust\"";18const CODE: &str = "\u{fddf}HYPERLINK \"https://example.invalid/rust\"";
1819
19fn page_by_title(bytes: &[u8], title: &str) -> (ExGuid, Page) {20fn page_by_title(bytes: &[u8], title: &str) -> (ExGuid, Page) {
...@@ -133,8 +134,8 @@ fn a_native_link_is_removed_leaving_plain_label_text() {...@@ -133,8 +134,8 @@ fn a_native_link_is_removed_leaving_plain_label_text() {
133 replacement: Paragraph::new(content[end..].to_owned(), plain),134 replacement: Paragraph::new(content[end..].to_owned(), plain),
134 })135 })
135 .unwrap();136 .unwrap();
136 let written = PreparedEdit::page(OUTLINES, space, &after, AUTHOR).unwrap();137 let written = ops::saved(OUTLINES, space, &after).unwrap();
137 let stored = assert_same(written.as_bytes(), space, &after);138 let stored = assert_same(written.as_slice(), space, &after);
138 let mut stored = stored;139 let mut stored = stored;
139 let text = &body_paragraphs(&mut stored)[at].text().unwrap().text;140 let text = &body_paragraphs(&mut stored)[at].text().unwrap().text;
140 assert!(text.text().ends_with(" Link label"));141 assert!(text.text().ends_with(" Link label"));
...@@ -159,8 +160,8 @@ fn a_link_is_added_to_a_fresh_page_and_reads_back() {...@@ -159,8 +160,8 @@ fn a_link_is_added_to_a_fresh_page_and_reads_back() {
159 &mut body_paragraphs(&mut after)[0].text_mut().unwrap().text,160 &mut body_paragraphs(&mut after)[0].text_mut().unwrap().text,
160 "the Rust site",161 "the Rust site",
161 );162 );
162 let written = PreparedEdit::page(&source, space, &after, AUTHOR).unwrap();163 let written = ops::saved(&source, space, &after).unwrap();
163 let stored = assert_same(written.as_bytes(), space, &after);164 let stored = assert_same(written.as_slice(), space, &after);
164 let mut stored = stored;165 let mut stored = stored;
165 let text = &body_paragraphs(&mut stored)[0].text().unwrap().text;166 let text = &body_paragraphs(&mut stored)[0].text().unwrap().text;
166 let flags: Vec<(Option<bool>, Option<bool>, Option<bool>)> = text167 let flags: Vec<(Option<bool>, Option<bool>, Option<bool>)> = text
...@@ -185,8 +186,8 @@ fn a_link_is_added_to_a_fresh_page_and_reads_back() {...@@ -185,8 +186,8 @@ fn a_link_is_added_to_a_fresh_page_and_reads_back() {
185 if let Some(directory) = std::env::var_os("ONESTORE_LINK_EXPORT") {186 if let Some(directory) = std::env::var_os("ONESTORE_LINK_EXPORT") {
186 let directory = std::path::PathBuf::from(directory);187 let directory = std::path::PathBuf::from(directory);
187 std::fs::create_dir(&directory).unwrap();188 std::fs::create_dir(&directory).unwrap();
188 std::fs::write(directory.join("links.one"), written.as_bytes()).unwrap();189 std::fs::write(directory.join("links.one"), written.as_slice()).unwrap();
189 let written_store = Store::parse(written.as_bytes()).unwrap();190 let written_store = Store::parse(written.as_slice()).unwrap();
190 std::fs::write(191 std::fs::write(
191 directory.join("Open Notebook.onetoc2"),192 directory.join("Open Notebook.onetoc2"),
192 onestore::create_table_of_contents(193 onestore::create_table_of_contents(
...@@ -292,10 +293,11 @@ fn internal_links_match_what_onenote_stores() {...@@ -292,10 +293,11 @@ fn internal_links_match_what_onenote_stores() {
292fn a_page_links_to_another_page_and_its_paragraph() {293fn a_page_links_to_another_page_and_its_paragraph() {
293 let source = onestore::create_section("links.one", "Linking page", "Author").unwrap();294 let source = onestore::create_section("links.one", "Linking page", "Author").unwrap();
294 let creation = onestore::PageCreation::new(None, Some("Link target"), "Author").unwrap();295 let creation = onestore::PageCreation::new(None, Some("Link target"), "Author").unwrap();
295 let with_target = onestore::PreparedEdit::create_page(&source, &creation)296 let with_target = ops::edited(
296 .unwrap()297 &source,
297 .as_bytes()298 vec![onestore::op::Op::Section(onestore::op::SectionOp::Create(creation))],
298 .to_vec();299 )
300 .unwrap();
299 let section = Store::parse(&with_target).unwrap().header.file_id;301 let section = Store::parse(&with_target).unwrap().header.file_id;
300 let (_, target) = page_by_title(&with_target, "Link target");302 let (_, target) = page_by_title(&with_target, "Link target");
301 let (space, before) = page_by_title(&with_target, "Linking page");303 let (space, before) = page_by_title(&with_target, "Linking page");
...@@ -351,12 +353,12 @@ fn a_page_links_to_another_page_and_its_paragraph() {...@@ -351,12 +353,12 @@ fn a_page_links_to_another_page_and_its_paragraph() {
351 replacement: Paragraph::from_runs(runs),353 replacement: Paragraph::from_runs(runs),
352 })354 })
353 .unwrap();355 .unwrap();
354 let written = PreparedEdit::page(&with_target, space, &after, AUTHOR).unwrap();356 let written = ops::saved(&with_target, space, &after).unwrap();
355 assert_same(written.as_bytes(), space, &after);357 assert_same(written.as_slice(), space, &after);
356 if let Some(directory) = std::env::var_os("ONESTORE_INTERNAL_LINK_EXPORT") {358 if let Some(directory) = std::env::var_os("ONESTORE_INTERNAL_LINK_EXPORT") {
357 let directory = std::path::PathBuf::from(directory);359 let directory = std::path::PathBuf::from(directory);
358 std::fs::create_dir(&directory).unwrap();360 std::fs::create_dir(&directory).unwrap();
359 std::fs::write(directory.join("links.one"), written.as_bytes()).unwrap();361 std::fs::write(directory.join("links.one"), written.as_slice()).unwrap();
360 std::fs::write(362 std::fs::write(
361 directory.join("Open Notebook.onetoc2"),363 directory.join("Open Notebook.onetoc2"),
362 onestore::create_table_of_contents("Open Notebook.onetoc2", &[("links.one", section)])364 onestore::create_table_of_contents("Open Notebook.onetoc2", &[("links.one", section)])
crates/onestore/tests/page_lists.rs+25-24
...@@ -1,5 +1,7 @@...@@ -1,5 +1,7 @@
1#[path = "support/ops.rs"]
2mod ops;
1use onestore::{3use onestore::{
2 ExGuid, PreparedEdit, RevisionIndex, Store,4 ExGuid, RevisionIndex, Store,
3 document::{Document, Format, Kind},5 document::{Document, Format, Kind},
4 page::{6 page::{
5 Definition, Page, PageObject, PageParagraph, ParagraphContent, TextObject, text::new_id,7 Definition, Page, PageObject, PageParagraph, ParagraphContent, TextObject, text::new_id,
...@@ -9,7 +11,6 @@ use std::collections::BTreeMap;...@@ -9,7 +11,6 @@ use std::collections::BTreeMap;
911
10const TREES: &[u8] =12const TREES: &[u8] =
11 include_bytes!("../../../corpus/outline-edit/tree/before/notebook/synthetic.one");13 include_bytes!("../../../corpus/outline-edit/tree/before/notebook/synthetic.one");
12const AUTHOR: &str = "List author";
1314
14fn page_by_title(bytes: &[u8], title: &str) -> (ExGuid, Page) {15fn page_by_title(bytes: &[u8], title: &str) -> (ExGuid, Page) {
15 let store = Store::parse(bytes).unwrap();16 let store = Store::parse(bytes).unwrap();
...@@ -114,8 +115,8 @@ fn a_numbered_paragraph_becomes_plain_while_its_neighbours_keep_their_nodes() {...@@ -114,8 +115,8 @@ fn a_numbered_paragraph_becomes_plain_while_its_neighbours_keep_their_nodes() {
114 assert!(numbered.len() >= 2, "{lists:?}");115 assert!(numbered.len() >= 2, "{lists:?}");
115 let mut after = before.clone();116 let mut after = before.clone();
116 body_paragraphs(&mut after)[numbered[0]].lists.clear();117 body_paragraphs(&mut after)[numbered[0]].lists.clear();
117 let written = PreparedEdit::page(TREES, space, &after, AUTHOR).unwrap();118 let written = ops::saved(TREES, space, &after).unwrap();
118 let stored = page_in(written.as_bytes(), space);119 let stored = page_in(written.as_slice(), space);
119 let mut expected = after.clone();120 let mut expected = after.clone();
120 expected.title = stored.title.clone();121 expected.title = stored.title.clone();
121 expected.definitions.remove(&lists[numbered[0]][0]);122 expected.definitions.remove(&lists[numbered[0]][0]);
...@@ -123,16 +124,16 @@ fn a_numbered_paragraph_becomes_plain_while_its_neighbours_keep_their_nodes() {...@@ -123,16 +124,16 @@ fn a_numbered_paragraph_becomes_plain_while_its_neighbours_keep_their_nodes() {
123 for other in &numbered[1..] {124 for other in &numbered[1..] {
124 for node in &lists[*other] {125 for node in &lists[*other] {
125 assert_eq!(126 assert_eq!(
126 raw_object(written.as_bytes(), space, *node),127 raw_object(written.as_slice(), space, *node),
127 raw_object(TREES, space, *node)128 raw_object(TREES, space, *node)
128 );129 );
129 }130 }
130 }131 }
131 assert_eq!(132 assert_eq!(
132 PreparedEdit::page(written.as_bytes(), space, &stored, AUTHOR)133 ops::saved(written.as_slice(), space, &stored)
133 .unwrap()134 .unwrap()
134 .as_bytes(),135 .as_slice(),
135 written.as_bytes()136 written.as_slice()
136 );137 );
137}138}
138139
...@@ -157,16 +158,16 @@ fn bullets_and_numbering_write_from_definitions_and_read_back() {...@@ -157,16 +158,16 @@ fn bullets_and_numbering_write_from_definitions_and_read_back() {
157 let mut fresh = plain_paragraph(&template, "Numbered from three");158 let mut fresh = plain_paragraph(&template, "Numbered from three");
158 fresh.lists = vec![number];159 fresh.lists = vec![number];
159 body_paragraphs(&mut after).push(fresh);160 body_paragraphs(&mut after).push(fresh);
160 let written = PreparedEdit::page(TREES, space, &after, AUTHOR).unwrap();161 let written = ops::saved(TREES, space, &after).unwrap();
161 let stored = page_in(written.as_bytes(), space);162 let stored = page_in(written.as_slice(), space);
162 let mut expected = after.clone();163 let mut expected = after.clone();
163 expected.title = stored.title.clone();164 expected.title = stored.title.clone();
164 assert_eq!(stored, expected);165 assert_eq!(stored, expected);
165 assert_eq!(166 assert_eq!(
166 PreparedEdit::page(written.as_bytes(), space, &stored, AUTHOR)167 ops::saved(written.as_slice(), space, &stored)
167 .unwrap()168 .unwrap()
168 .as_bytes(),169 .as_slice(),
169 written.as_bytes()170 written.as_slice()
170 );171 );
171}172}
172173
...@@ -183,8 +184,8 @@ fn a_list_definition_changes_in_place() {...@@ -183,8 +184,8 @@ fn a_list_definition_changes_in_place() {
183 };184 };
184 *restart = Some(7);185 *restart = Some(7);
185 definition.format.bold = Some(true);186 definition.format.bold = Some(true);
186 let written = PreparedEdit::page(TREES, space, &after, AUTHOR).unwrap();187 let written = ops::saved(TREES, space, &after).unwrap();
187 let stored = page_in(written.as_bytes(), space);188 let stored = page_in(written.as_slice(), space);
188 let mut expected = after.clone();189 let mut expected = after.clone();
189 expected.title = stored.title.clone();190 expected.title = stored.title.clone();
190 assert_eq!(stored, expected);191 assert_eq!(stored, expected);
...@@ -199,8 +200,8 @@ fn a_definition_shared_between_paragraphs_becomes_a_node_per_paragraph() {...@@ -199,8 +200,8 @@ fn a_definition_shared_between_paragraphs_becomes_a_node_per_paragraph() {
199 let plain = (0..lists.len()).find(|i| lists[*i].is_empty()).unwrap();200 let plain = (0..lists.len()).find(|i| lists[*i].is_empty()).unwrap();
200 let mut shared = before.clone();201 let mut shared = before.clone();
201 body_paragraphs(&mut shared)[plain].lists = lists[numbered].clone();202 body_paragraphs(&mut shared)[plain].lists = lists[numbered].clone();
202 let written = PreparedEdit::page(TREES, space, &shared, AUTHOR).unwrap();203 let written = ops::saved(TREES, space, &shared).unwrap();
203 let stored = page_in(written.as_bytes(), space);204 let stored = page_in(written.as_slice(), space);
204 let after = list_ids(&stored);205 let after = list_ids(&stored);
205 assert_eq!(after[numbered], lists[numbered]);206 assert_eq!(after[numbered], lists[numbered]);
206 assert_eq!(after[plain].len(), 1);207 assert_eq!(after[plain].len(), 1);
...@@ -218,12 +219,12 @@ fn missing_and_foreign_list_definitions_are_refused() {...@@ -218,12 +219,12 @@ fn missing_and_foreign_list_definitions_are_refused() {
218 let plain = (0..lists.len()).find(|i| lists[*i].is_empty()).unwrap();219 let plain = (0..lists.len()).find(|i| lists[*i].is_empty()).unwrap();
219 let mut missing = before.clone();220 let mut missing = before.clone();
220 body_paragraphs(&mut missing)[plain].lists = vec![new_id().unwrap()];221 body_paragraphs(&mut missing)[plain].lists = vec![new_id().unwrap()];
221 assert!(PreparedEdit::page(TREES, space, &missing, AUTHOR).is_err());222 assert!(ops::saved(TREES, space, &missing).is_err());
222 let mut foreign = before.clone();223 let mut foreign = before.clone();
223 let paragraph = body_paragraphs(&mut foreign)[plain].id;224 let paragraph = body_paragraphs(&mut foreign)[plain].id;
224 foreign.definitions.insert(paragraph, bullet_definition());225 foreign.definitions.insert(paragraph, bullet_definition());
225 body_paragraphs(&mut foreign)[plain].lists = vec![paragraph];226 body_paragraphs(&mut foreign)[plain].lists = vec![paragraph];
226 assert!(PreparedEdit::page(TREES, space, &foreign, AUTHOR).is_err());227 assert!(ops::saved(TREES, space, &foreign).is_err());
227 let mut style: BTreeMap<ExGuid, Definition> = before.definitions.clone();228 let mut style: BTreeMap<ExGuid, Definition> = before.definitions.clone();
228 if let Some((id, definition)) = style229 if let Some((id, definition)) = style
229 .iter_mut()230 .iter_mut()
...@@ -232,7 +233,7 @@ fn missing_and_foreign_list_definitions_are_refused() {...@@ -232,7 +233,7 @@ fn missing_and_foreign_list_definitions_are_refused() {
232 definition.format.bold = Some(true);233 definition.format.bold = Some(true);
233 let mut edited = before.clone();234 let mut edited = before.clone();
234 edited.definitions.insert(*id, definition.clone());235 edited.definitions.insert(*id, definition.clone());
235 assert!(PreparedEdit::page(TREES, space, &edited, AUTHOR).is_err());236 assert!(ops::saved(TREES, space, &edited).is_err());
236 }237 }
237}238}
238239
...@@ -299,16 +300,16 @@ fn bullets_numbering_nesting_and_restarts_publish_on_a_fresh_page() {...@@ -299,16 +300,16 @@ fn bullets_numbering_nesting_and_restarts_publish_on_a_fresh_page() {
299 body_paragraphs(&mut after).push(paragraph);300 body_paragraphs(&mut after).push(paragraph);
300 }301 }
301 body_paragraphs(&mut after).push(plain_paragraph(&template, "Plain again"));302 body_paragraphs(&mut after).push(plain_paragraph(&template, "Plain again"));
302 let written = PreparedEdit::page(&source, space, &after, AUTHOR).unwrap();303 let written = ops::saved(&source, space, &after).unwrap();
303 let stored = page_in(written.as_bytes(), space);304 let stored = page_in(written.as_slice(), space);
304 let mut expected = after.clone();305 let mut expected = after.clone();
305 expected.title = stored.title.clone();306 expected.title = stored.title.clone();
306 assert_eq!(stored, expected);307 assert_eq!(stored, expected);
307 if let Some(directory) = std::env::var_os("ONESTORE_LIST_EXPORT") {308 if let Some(directory) = std::env::var_os("ONESTORE_LIST_EXPORT") {
308 let directory = std::path::PathBuf::from(directory);309 let directory = std::path::PathBuf::from(directory);
309 std::fs::create_dir(&directory).unwrap();310 std::fs::create_dir(&directory).unwrap();
310 std::fs::write(directory.join("lists.one"), written.as_bytes()).unwrap();311 std::fs::write(directory.join("lists.one"), written.as_slice()).unwrap();
311 let written_store = Store::parse(written.as_bytes()).unwrap();312 let written_store = Store::parse(written.as_slice()).unwrap();
312 std::fs::write(313 std::fs::write(
313 directory.join("Open Notebook.onetoc2"),314 directory.join("Open Notebook.onetoc2"),
314 onestore::create_table_of_contents(315 onestore::create_table_of_contents(
crates/onestore/tests/page_math.rs+8-6
...@@ -1,3 +1,5 @@...@@ -1,3 +1,5 @@
1#[path = "support/ops.rs"]
2mod ops;
1use onestore::{3use onestore::{
2 RevisionIndex, Store,4 RevisionIndex, Store,
3 document::Document,5 document::Document,
...@@ -272,8 +274,8 @@ fn equations_are_written_and_read_back() {...@@ -272,8 +274,8 @@ fn equations_are_written_and_read_back() {
272 }274 }
273 outline.paragraphs.push(paragraph);275 outline.paragraphs.push(paragraph);
274 }276 }
275 let written = onestore::PreparedEdit::page(&source, space, &after, "Math author").unwrap();277 let written = ops::saved(&source, space, &after).unwrap();
276 let store = Store::parse(written.as_bytes()).unwrap();278 let store = Store::parse(written.as_slice()).unwrap();
277 let index = RevisionIndex::parse(&store).unwrap();279 let index = RevisionIndex::parse(&store).unwrap();
278 let document = Document::parse(&index).unwrap();280 let document = Document::parse(&index).unwrap();
279 let stored = Page::from_space(&document, space).unwrap();281 let stored = Page::from_space(&document, space).unwrap();
...@@ -305,13 +307,13 @@ fn equations_are_written_and_read_back() {...@@ -305,13 +307,13 @@ fn equations_are_written_and_read_back() {
305 }307 }
306 assert_eq!(stored, expected);308 assert_eq!(stored, expected);
307 assert_eq!(309 assert_eq!(
308 rendered_equations(written.as_bytes(), &stored.title),310 rendered_equations(written.as_slice(), &stored.title),
309 expected_mathml311 expected_mathml
310 );312 );
311 if let Some(directory) = std::env::var_os("ONESTORE_MATH_EXPORT") {313 if let Some(directory) = std::env::var_os("ONESTORE_MATH_EXPORT") {
312 let directory = std::path::PathBuf::from(directory);314 let directory = std::path::PathBuf::from(directory);
313 std::fs::create_dir(&directory).unwrap();315 std::fs::create_dir(&directory).unwrap();
314 std::fs::write(directory.join("math.one"), written.as_bytes()).unwrap();316 std::fs::write(directory.join("math.one"), written.as_slice()).unwrap();
315 std::fs::write(317 std::fs::write(
316 directory.join("Open Notebook.onetoc2"),318 directory.join("Open Notebook.onetoc2"),
317 onestore::create_table_of_contents(319 onestore::create_table_of_contents(
...@@ -361,8 +363,8 @@ fn a_paragraph_emptied_of_its_inline_math_saves() {...@@ -361,8 +363,8 @@ fn a_paragraph_emptied_of_its_inline_math_saves() {
361 let plain = text.text.spans()[0].format.clone();363 let plain = text.text.spans()[0].format.clone();
362 assert_ne!(plain.math, Some(true));364 assert_ne!(plain.math, Some(true));
363 text.text = onestore::page::Paragraph::new(String::new(), plain);365 text.text = onestore::page::Paragraph::new(String::new(), plain);
364 let written = onestore::PreparedEdit::page(NATIVE, space, &after, "Author").unwrap();366 let written = ops::saved(NATIVE, space, &after).unwrap();
365 let store = Store::parse(written.as_bytes()).unwrap();367 let store = Store::parse(written.as_slice()).unwrap();
366 let index = RevisionIndex::parse(&store).unwrap();368 let index = RevisionIndex::parse(&store).unwrap();
367 let document = Document::parse(&index).unwrap();369 let document = Document::parse(&index).unwrap();
368 assert_eq!(Page::from_space(&document, space).unwrap(), after);370 assert_eq!(Page::from_space(&document, space).unwrap(), after);
crates/onestore/tests/page_media.rs+10-8
...@@ -1,5 +1,7 @@...@@ -1,5 +1,7 @@
1#[path = "support/ops.rs"]
2mod ops;
1use onestore::{3use onestore::{
2 ExGuid, PreparedEdit, RevisionIndex, Store,4 ExGuid, RevisionIndex, Store,
3 document::Document,5 document::Document,
4 page::{MediaIndex, Page, PageObject, ParagraphContent, Recording},6 page::{MediaIndex, Page, PageObject, ParagraphContent, Recording},
5};7};
...@@ -96,8 +98,8 @@ fn editing_a_recorded_page_keeps_the_recording_and_its_annotation() {...@@ -96,8 +98,8 @@ fn editing_a_recorded_page_keeps_the_recording_and_its_annotation() {
96 tags: Vec::new(),98 tags: Vec::new(),
97 });99 });
98 outline.paragraphs.push(added.clone());100 outline.paragraphs.push(added.clone());
99 let written = PreparedEdit::page(NATIVE, space, &after, "Media author").unwrap();101 let written = ops::saved(NATIVE, space, &after).unwrap();
100 let (_, stored) = page(written.as_bytes());102 let (_, stored) = page(written.as_slice());
101 assert_eq!(recording(&stored), recording(&before));103 assert_eq!(recording(&stored), recording(&before));
102 assert_eq!(stored.objects.len(), before.objects.len());104 assert_eq!(stored.objects.len(), before.objects.len());
103105
...@@ -112,7 +114,7 @@ fn editing_a_recorded_page_keeps_the_recording_and_its_annotation() {...@@ -112,7 +114,7 @@ fn editing_a_recorded_page_keeps_the_recording_and_its_annotation() {
112 .unwrap();114 .unwrap();
113 let last = outline.paragraphs.last_mut().unwrap();115 let last = outline.paragraphs.last_mut().unwrap();
114 last.media = recording(&before).1;116 last.media = recording(&before).1;
115 assert!(PreparedEdit::page(written.as_bytes(), space, &annotated, "x").is_err());117 assert!(ops::saved(written.as_slice(), space, &annotated).is_err());
116 let mut recaptured = after.clone();118 let mut recaptured = after.clone();
117 for object in &mut recaptured.objects {119 for object in &mut recaptured.objects {
118 if let PageObject::Outline(outline) = object {120 if let PageObject::Outline(outline) = object {
...@@ -123,13 +125,13 @@ fn editing_a_recorded_page_keeps_the_recording_and_its_annotation() {...@@ -123,13 +125,13 @@ fn editing_a_recorded_page_keeps_the_recording_and_its_annotation() {
123 }125 }
124 }126 }
125 }127 }
126 assert!(PreparedEdit::page(written.as_bytes(), space, &recaptured, "x").is_err());128 assert!(ops::saved(written.as_slice(), space, &recaptured).is_err());
127129
128 if let Some(directory) = std::env::var_os("ONESTORE_MEDIA_EXPORT") {130 if let Some(directory) = std::env::var_os("ONESTORE_MEDIA_EXPORT") {
129 let directory = std::path::PathBuf::from(directory);131 let directory = std::path::PathBuf::from(directory);
130 std::fs::create_dir_all(&directory).unwrap();132 std::fs::create_dir_all(&directory).unwrap();
131 std::fs::write(directory.join("Features.one"), written.as_bytes()).unwrap();133 std::fs::write(directory.join("Features.one"), written.as_slice()).unwrap();
132 let file_id = Store::parse(written.as_bytes()).unwrap().header.file_id;134 let file_id = Store::parse(written.as_slice()).unwrap().header.file_id;
133 std::fs::write(135 std::fs::write(
134 directory.join("Open Notebook.onetoc2"),136 directory.join("Open Notebook.onetoc2"),
135 onestore::create_table_of_contents(137 onestore::create_table_of_contents(
...@@ -139,7 +141,7 @@ fn editing_a_recorded_page_keeps_the_recording_and_its_annotation() {...@@ -139,7 +141,7 @@ fn editing_a_recorded_page_keeps_the_recording_and_its_annotation() {
139 .unwrap(),141 .unwrap(),
140 )142 )
141 .unwrap();143 .unwrap();
142 let store = Store::parse(written.as_bytes()).unwrap();144 let store = Store::parse(written.as_slice()).unwrap();
143 let index = RevisionIndex::parse(&store).unwrap();145 let index = RevisionIndex::parse(&store).unwrap();
144 let count = Document::parse(&index).unwrap().pages().unwrap().len();146 let count = Document::parse(&index).unwrap().pages().unwrap().len();
145 std::fs::write(directory.join("expected-count.txt"), count.to_string()).unwrap();147 std::fs::write(directory.join("expected-count.txt"), count.to_string()).unwrap();
crates/onestore/tests/page_movement.rs+25-24
...@@ -1,6 +1,10 @@...@@ -1,6 +1,10 @@
1#[path = "support/ops.rs"]
2mod ops;
3
1use onestore::{4use onestore::{
2 ExGuid, PageEdit, PreparedEdit, RevisionIndex, Store,5 ExGuid, PageEdit, RevisionIndex, Store,
3 document::{Document, Kind},6 document::{Document, Kind},
7 op::{PageOp, SectionOp},
4};8};
59
6#[path = "support/current.rs"]10#[path = "support/current.rs"]
...@@ -175,10 +179,10 @@ fn explicit_page_edits_match_native_selection_and_indentation() {...@@ -175,10 +179,10 @@ fn explicit_page_edits_match_native_selection_and_indentation() {
175 let serialized = serde_json::to_vec(&edits).unwrap();179 let serialized = serde_json::to_vec(&edits).unwrap();
176 let retained: Vec<PageEdit> = serde_json::from_slice(&serialized).unwrap();180 let retained: Vec<PageEdit> = serde_json::from_slice(&serialized).unwrap();
177 assert_eq!(edits, retained);181 assert_eq!(edits, retained);
178 let prepared = PreparedEdit::pages(&source, &retained).unwrap();182 let prepared = ops::section_op(&source, SectionOp::Pages(retained.to_vec())).unwrap();
179 verify(&source, prepared.as_bytes(), native);183 verify(&source, prepared.as_bytes(), native);
180 assert_eq!(184 assert_eq!(
181 PreparedEdit::pages(prepared.as_bytes(), &retained)185 ops::section_op(prepared.as_bytes(), SectionOp::Pages(retained.to_vec()))
182 .unwrap()186 .unwrap()
183 .as_bytes(),187 .as_bytes(),
184 prepared.as_bytes()188 prepared.as_bytes()
...@@ -187,7 +191,7 @@ fn explicit_page_edits_match_native_selection_and_indentation() {...@@ -187,7 +191,7 @@ fn explicit_page_edits_match_native_selection_and_indentation() {
187 source = prepared.as_bytes().to_vec();191 source = prepared.as_bytes().to_vec();
188 }192 }
189 let edit = PageEdit::move_to(ids[3], None, 1).unwrap();193 let edit = PageEdit::move_to(ids[3], None, 1).unwrap();
190 let single = PreparedEdit::pages(SOURCE, &[edit]).unwrap();194 let single = ops::section_op(SOURCE, SectionOp::Pages([edit].to_vec())).unwrap();
191 verify(195 verify(
192 SOURCE,196 SOURCE,
193 single.as_bytes(),197 single.as_bytes(),
...@@ -206,32 +210,29 @@ fn invalid_and_empty_page_edits_do_not_publish() {...@@ -206,32 +210,29 @@ fn invalid_and_empty_page_edits_do_not_publish() {
206 }210 }
207 assert!(PageEdit::move_to(ids[0], Some(ids[0]), 1).is_err());211 assert!(PageEdit::move_to(ids[0], Some(ids[0]), 1).is_err());
208 assert!(PageEdit::set_level(ExGuid::default(), 1).is_err());212 assert!(PageEdit::set_level(ExGuid::default(), 1).is_err());
209 assert_eq!(PreparedEdit::pages(SOURCE, &[]).unwrap().as_bytes(), SOURCE);213 assert_eq!(ops::section_op(SOURCE, SectionOp::Pages([].to_vec())).unwrap().as_bytes(), SOURCE);
210 assert_eq!(214 assert_eq!(
211 PreparedEdit::pages(SOURCE, &[PageEdit::set_level(ids[0], 1).unwrap()])215 ops::section_op(SOURCE, SectionOp::Pages([PageEdit::set_level(ids[0], 1).unwrap()].to_vec()))
212 .unwrap()216 .unwrap()
213 .as_bytes(),217 .as_bytes(),
214 SOURCE218 SOURCE
215 );219 );
216 assert!(PreparedEdit::pages(SOURCE, &[PageEdit::set_level(ids[0], 2).unwrap()]).is_err());220 assert!(ops::section_op(SOURCE, SectionOp::Pages([PageEdit::set_level(ids[0], 2).unwrap()].to_vec())).is_err());
217 let duplicate = PageEdit::set_level(ids[3], 2).unwrap();221 let duplicate = PageEdit::set_level(ids[3], 2).unwrap();
218 assert!(PreparedEdit::pages(SOURCE, &[duplicate.clone(), duplicate]).is_err());222 assert!(ops::section_op(SOURCE, SectionOp::Pages([duplicate.clone(), duplicate].to_vec())).is_err());
219 let missing = ExGuid {223 let missing = ExGuid {
220 guid: [0x77; 16],224 guid: [0x77; 16],
221 n: 1,225 n: 1,
222 };226 };
223 assert!(PreparedEdit::pages(SOURCE, &[PageEdit::set_level(missing, 1).unwrap()]).is_err());227 assert!(ops::section_op(SOURCE, SectionOp::Pages([PageEdit::set_level(missing, 1).unwrap()].to_vec())).is_err());
224 assert!(228 assert!(
225 PreparedEdit::pages(229 ops::section_op(SOURCE, SectionOp::Pages([PageEdit::move_to(ids[3], Some(missing), 1).unwrap()].to_vec()))
226 SOURCE,
227 &[PageEdit::move_to(ids[3], Some(missing), 1).unwrap()]
228 )
229 .is_err()230 .is_err()
230 );231 );
231 let mut value = serde_json::to_value(PageEdit::set_level(ids[3], 2).unwrap()).unwrap();232 let mut value = serde_json::to_value(PageEdit::set_level(ids[3], 2).unwrap()).unwrap();
232 value["level"] = 4.into();233 value["level"] = 4.into();
233 let invalid: PageEdit = serde_json::from_value(value.clone()).unwrap();234 let invalid: PageEdit = serde_json::from_value(value.clone()).unwrap();
234 assert!(PreparedEdit::pages(SOURCE, &[invalid]).is_err());235 assert!(ops::section_op(SOURCE, SectionOp::Pages([invalid].to_vec())).is_err());
235 value["unknown"] = true.into();236 value["unknown"] = true.into();
236 assert!(serde_json::from_value::<PageEdit>(value).is_err());237 assert!(serde_json::from_value::<PageEdit>(value).is_err());
237}238}
...@@ -246,12 +247,12 @@ fn moving_pages_preserves_ink_file_data_and_native_feature_objects() {...@@ -246,12 +247,12 @@ fn moving_pages_preserves_ink_file_data_and_native_feature_objects() {
246 let first = pages(source)[0];247 let first = pages(source)[0];
247 let created =248 let created =
248 onestore::PageCreation::new(Some(first), Some("Movement anchor"), "Author").unwrap();249 onestore::PageCreation::new(Some(first), Some("Movement anchor"), "Author").unwrap();
249 let source = PreparedEdit::create_page(source, &created)250 let source = ops::section_op(source, SectionOp::Create(created.clone()))
250 .unwrap()251 .unwrap()
251 .as_bytes()252 .as_bytes()
252 .to_vec();253 .to_vec();
253 let moved =254 let moved =
254 PreparedEdit::pages(&source, &[PageEdit::move_to(first, None, 1).unwrap()]).unwrap();255 ops::section_op(&source, SectionOp::Pages([PageEdit::move_to(first, None, 1).unwrap()].to_vec())).unwrap();
255 assert_eq!(pages(moved.as_bytes()).last(), Some(&first));256 assert_eq!(pages(moved.as_bytes()).last(), Some(&first));
256 let stores = [&source, moved.as_bytes()].map(|bytes| Store::parse(bytes).unwrap());257 let stores = [&source, moved.as_bytes()].map(|bytes| Store::parse(bytes).unwrap());
257 let indexes = stores258 let indexes = stores
...@@ -295,7 +296,7 @@ fn section_reordering_refreshes_stale_native_metadata_levels() {...@@ -295,7 +296,7 @@ fn section_reordering_refreshes_stale_native_metadata_levels() {
295 .enumerate()296 .enumerate()
296 .map(|(i, sid)| PageEdit::move_to(*sid, None, u32::try_from(i + 1).unwrap()).unwrap())297 .map(|(i, sid)| PageEdit::move_to(*sid, None, u32::try_from(i + 1).unwrap()).unwrap())
297 .collect();298 .collect();
298 let prepared = PreparedEdit::pages(source, &edits).unwrap();299 let prepared = ops::section_op(source, SectionOp::Pages(edits.to_vec())).unwrap();
299 let stores = [source.as_slice(), prepared.as_bytes()].map(|b| Store::parse(b).unwrap());300 let stores = [source.as_slice(), prepared.as_bytes()].map(|b| Store::parse(b).unwrap());
300 let indexes = stores.each_ref().map(|s| RevisionIndex::parse(s).unwrap());301 let indexes = stores.each_ref().map(|s| RevisionIndex::parse(s).unwrap());
301 let documents = indexes.each_ref().map(|i| Document::parse(i).unwrap());302 let documents = indexes.each_ref().map(|i| Document::parse(i).unwrap());
...@@ -335,7 +336,7 @@ fn metadata_copy_order_does_not_change_page_identity() {...@@ -335,7 +336,7 @@ fn metadata_copy_order_does_not_change_page_identity() {
335 "../../../corpus/page-lifecycle/page-edits/optional-cache/source-cold/notebook/Lifecycle.one"336 "../../../corpus/page-lifecycle/page-edits/optional-cache/source-cold/notebook/Lifecycle.one"
336 );337 );
337 let sid = pages(source)[3];338 let sid = pages(source)[3];
338 let edited = PreparedEdit::pages(source, &[PageEdit::set_level(sid, 2).unwrap()]).unwrap();339 let edited = ops::section_op(source, SectionOp::Pages([PageEdit::set_level(sid, 2).unwrap()].to_vec())).unwrap();
339 let expected = include_bytes!(340 let expected = include_bytes!(
340 "../../../corpus/page-lifecycle/page-edits/optional-cache/candidate/Lifecycle.one"341 "../../../corpus/page-lifecycle/page-edits/optional-cache/candidate/Lifecycle.one"
341 );342 );
...@@ -371,8 +372,8 @@ fn native_edits_on_moved_pages_support_more_rust_changes() {...@@ -371,8 +372,8 @@ fn native_edits_on_moved_pages_support_more_rust_changes() {
371 let bodies: Vec<_> = view.nodes.iter().filter_map(|(id, node)|372 let bodies: Vec<_> = view.nodes.iter().filter_map(|(id, node)|
372 matches!(&node.kind, Kind::RichText { text, .. } if text == "Native body after a Rust page move.").then_some(*id)).collect();373 matches!(&node.kind, Kind::RichText { text, .. } if text == "Native body after a Rust page move.").then_some(*id)).collect();
373 let [body] = bodies.as_slice() else { panic!() };374 let [body] = bodies.as_slice() else { panic!() };
374 let moved = PreparedEdit::pages(source, &[PageEdit::set_level(sid, 2).unwrap()]).unwrap();375 let moved = ops::section_op(source, SectionOp::Pages([PageEdit::set_level(sid, 2).unwrap()].to_vec())).unwrap();
375 let edited = PreparedEdit::text(moved.as_bytes(), sid, *body, 0..0, "Rust + ").unwrap();376 let edited = ops::page_op(moved.as_bytes(), sid, PageOp::Text { text: *body, range: 0..0, with: "Rust + ".into() }).unwrap();
376 let after_store = Store::parse(edited.as_bytes()).unwrap();377 let after_store = Store::parse(edited.as_bytes()).unwrap();
377 let after_index = RevisionIndex::parse(&after_store).unwrap();378 let after_index = RevisionIndex::parse(&after_store).unwrap();
378 let after = Document::parse(&after_index).unwrap();379 let after = Document::parse(&after_index).unwrap();
...@@ -401,7 +402,7 @@ fn grouped_page_levels_and_membership_survive_each_storage_interruption() {...@@ -401,7 +402,7 @@ fn grouped_page_levels_and_membership_survive_each_storage_interruption() {
401 let mut source = onestore::create_section("movement.one", "Original", "Author").unwrap();402 let mut source = onestore::create_section("movement.one", "Original", "Author").unwrap();
402 for _ in 0..2 {403 for _ in 0..2 {
403 let page = onestore::PageCreation::new(None, Some("Same title"), "Author").unwrap();404 let page = onestore::PageCreation::new(None, Some("Same title"), "Author").unwrap();
404 source = PreparedEdit::create_page(&source, &page)405 source = ops::section_op(&source, SectionOp::Create(page.clone()))
405 .unwrap()406 .unwrap()
406 .as_bytes()407 .as_bytes()
407 .to_vec();408 .to_vec();
...@@ -421,7 +422,7 @@ fn grouped_page_levels_and_membership_survive_each_storage_interruption() {...@@ -421,7 +422,7 @@ fn grouped_page_levels_and_membership_survive_each_storage_interruption() {
421 PageEdit::set_level(ids[first], 2).unwrap(),422 PageEdit::set_level(ids[first], 2).unwrap(),
422 PageEdit::set_level(ids[first + 1], 3).unwrap(),423 PageEdit::set_level(ids[first + 1], 3).unwrap(),
423 ];424 ];
424 let prepared = PreparedEdit::pages(source, &edits).unwrap();425 let prepared = ops::section_op(source, SectionOp::Pages(edits.to_vec())).unwrap();
425 let old = current::current(source);426 let old = current::current(source);
426 let new = current::current(prepared.as_bytes());427 let new = current::current(prepared.as_bytes());
427 assert_eq!(new.len(), old.len());428 assert_eq!(new.len(), old.len());
...@@ -460,7 +461,7 @@ fn grouped_page_levels_and_membership_survive_each_storage_interruption() {...@@ -460,7 +461,7 @@ fn grouped_page_levels_and_membership_survive_each_storage_interruption() {
460fn repeated_nesting_retains_series_history_across_revision_checkpoints() {461fn repeated_nesting_retains_series_history_across_revision_checkpoints() {
461 let mut source = onestore::create_section("movement.one", "Original", "Author").unwrap();462 let mut source = onestore::create_section("movement.one", "Original", "Author").unwrap();
462 let page = onestore::PageCreation::new(None, Some("Child"), "Author").unwrap();463 let page = onestore::PageCreation::new(None, Some("Child"), "Author").unwrap();
463 source = PreparedEdit::create_page(&source, &page)464 source = ops::section_op(&source, SectionOp::Create(page.clone()))
464 .unwrap()465 .unwrap()
465 .as_bytes()466 .as_bytes()
466 .to_vec();467 .to_vec();
...@@ -468,7 +469,7 @@ fn repeated_nesting_retains_series_history_across_revision_checkpoints() {...@@ -468,7 +469,7 @@ fn repeated_nesting_retains_series_history_across_revision_checkpoints() {
468 for step in 0..520 {469 for step in 0..520 {
469 let level = if step % 2 == 0 { 2 } else { 1 };470 let level = if step % 2 == 0 { 2 } else { 1 };
470 let edit = PageEdit::set_level(page.space(), level).unwrap();471 let edit = PageEdit::set_level(page.space(), level).unwrap();
471 let written = PreparedEdit::pages(&source, &[edit])472 let written = ops::section_op(&source, SectionOp::Pages([edit].to_vec()))
472 .unwrap()473 .unwrap()
473 .as_bytes()474 .as_bytes()
474 .to_vec();475 .to_vec();
crates/onestore/tests/page_removal.rs+12-1
...@@ -1,3 +1,5 @@...@@ -1,3 +1,5 @@
1#[path = "support/ops.rs"]
2mod ops;
1use onestore::{3use onestore::{
2 ExGuid, FileDataReference, IdStream, ObjectData, PropertySets, RevisionIndex, Store, Value,4 ExGuid, FileDataReference, IdStream, ObjectData, PropertySets, RevisionIndex, Store, Value,
3 document::Document,5 document::Document,
...@@ -525,7 +527,16 @@ fn copy_comparison_rejects_text_and_payload_changes() {...@@ -525,7 +527,16 @@ fn copy_comparison_rejects_text_and_payload_changes() {
525 .then_some(*oid)527 .then_some(*oid)
526 })528 })
527 .unwrap();529 .unwrap();
528 let changed = onestore::replace_text(copied, copy_sid, text, 0..0, "Changed ").unwrap();530 let changed = ops::page_edited(
531 copied,
532 copy_sid,
533 vec![onestore::op::PageOp::Text {
534 text,
535 range: 0..0,
536 with: "Changed ".into(),
537 }],
538 )
539 .unwrap();
529 let changed_store = Store::parse(&changed).unwrap();540 let changed_store = Store::parse(&changed).unwrap();
530 RevisionIndex::parse(&changed_store)541 RevisionIndex::parse(&changed_store)
531 .unwrap()542 .unwrap()
crates/onestore/tests/page_tables.rs+34-36
...@@ -1,5 +1,7 @@...@@ -1,5 +1,7 @@
1#[path = "support/ops.rs"]
2mod ops;
1use onestore::{3use onestore::{
2 ExGuid, PreparedEdit, RevisionIndex, Store,4 ExGuid, RevisionIndex, Store,
3 document::{Document, Format},5 document::{Document, Format},
4 page::{6 page::{
5 Page, PageObject, PageParagraph, ParagraphContent, Table, TableCell, TableColumn, TableRow,7 Page, PageObject, PageParagraph, ParagraphContent, Table, TableCell, TableColumn, TableRow,
...@@ -9,7 +11,6 @@ use onestore::{...@@ -9,7 +11,6 @@ use onestore::{
911
10const TREES: &[u8] =12const TREES: &[u8] =
11 include_bytes!("../../../corpus/outline-edit/tree/before/notebook/synthetic.one");13 include_bytes!("../../../corpus/outline-edit/tree/before/notebook/synthetic.one");
12const AUTHOR: &str = "Table author";
1314
14fn page_by_title(bytes: &[u8], title: &str) -> (ExGuid, Page) {15fn page_by_title(bytes: &[u8], title: &str) -> (ExGuid, Page) {
15 let store = Store::parse(bytes).unwrap();16 let store = Store::parse(bytes).unwrap();
...@@ -174,19 +175,19 @@ fn a_row_and_a_column_are_added_to_a_native_table_and_removed_again() {...@@ -174,19 +175,19 @@ fn a_row_and_a_column_are_added_to_a_native_table_and_removed_again() {
174 new_cell(&template, "Second row, third column"),175 new_cell(&template, "Second row, third column"),
175 ],176 ],
176 });177 });
177 let written = PreparedEdit::page(TREES, space, &after, AUTHOR).unwrap();178 let written = ops::saved(TREES, space, &after).unwrap();
178 let stored = assert_same(written.as_bytes(), space, &after);179 let stored = assert_same(written.as_slice(), space, &after);
179 assert_eq!(180 assert_eq!(
180 PreparedEdit::page(written.as_bytes(), space, &stored, AUTHOR)181 ops::saved(written.as_slice(), space, &stored)
181 .unwrap()182 .unwrap()
182 .as_bytes(),183 .as_slice(),
183 written.as_bytes()184 written.as_slice()
184 );185 );
185 if let Some(directory) = std::env::var_os("ONESTORE_TABLE_EDIT_EXPORT") {186 if let Some(directory) = std::env::var_os("ONESTORE_TABLE_EDIT_EXPORT") {
186 let directory = std::path::PathBuf::from(directory);187 let directory = std::path::PathBuf::from(directory);
187 std::fs::create_dir(&directory).unwrap();188 std::fs::create_dir(&directory).unwrap();
188 std::fs::write(directory.join("synthetic.one"), written.as_bytes()).unwrap();189 std::fs::write(directory.join("synthetic.one"), written.as_slice()).unwrap();
189 let written_store = Store::parse(written.as_bytes()).unwrap();190 let written_store = Store::parse(written.as_slice()).unwrap();
190 std::fs::write(191 std::fs::write(
191 directory.join("Open Notebook.onetoc2"),192 directory.join("Open Notebook.onetoc2"),
192 onestore::create_table_of_contents(193 onestore::create_table_of_contents(
...@@ -204,8 +205,8 @@ fn a_row_and_a_column_are_added_to_a_native_table_and_removed_again() {...@@ -204,8 +205,8 @@ fn a_row_and_a_column_are_added_to_a_native_table_and_removed_again() {
204 row.cells.remove(1);205 row.cells.remove(1);
205 }206 }
206 table.rows.remove(0);207 table.rows.remove(0);
207 let again = PreparedEdit::page(written.as_bytes(), space, &smaller, AUTHOR).unwrap();208 let again = ops::saved(written.as_slice(), space, &smaller).unwrap();
208 let mut stored = assert_same(again.as_bytes(), space, &smaller);209 let mut stored = assert_same(again.as_slice(), space, &smaller);
209 let table = table_mut(&mut stored);210 let table = table_mut(&mut stored);
210 assert_eq!((table.rows.len(), table.columns.len()), (1, 2));211 assert_eq!((table.rows.len(), table.columns.len()), (1, 2));
211}212}
...@@ -218,8 +219,8 @@ fn column_widths_locks_and_borders_change_in_place() {...@@ -218,8 +219,8 @@ fn column_widths_locks_and_borders_change_in_place() {
218 table.columns[0].width = 300.0;219 table.columns[0].width = 300.0;
219 table.columns[1].locked = false;220 table.columns[1].locked = false;
220 table.borders = Some(false);221 table.borders = Some(false);
221 let written = PreparedEdit::page(TREES, space, &after, AUTHOR).unwrap();222 let written = ops::saved(TREES, space, &after).unwrap();
222 assert_same(written.as_bytes(), space, &after);223 assert_same(written.as_slice(), space, &after);
223}224}
224225
225#[test]226#[test]
...@@ -232,17 +233,17 @@ fn cell_shading_and_indents_change_in_place() {...@@ -232,17 +233,17 @@ fn cell_shading_and_indents_change_in_place() {
232 cell.shading = Some(0x0000ffff);233 cell.shading = Some(0x0000ffff);
233 let original_indents = cell.indents.clone();234 let original_indents = cell.indents.clone();
234 cell.indents.clear();235 cell.indents.clear();
235 assert!(PreparedEdit::page(TREES, space, &after, AUTHOR).is_err());236 assert!(ops::saved(TREES, space, &after).is_err());
236 let cell = &mut table_mut(&mut after).rows[0].cells[0];237 let cell = &mut table_mut(&mut after).rows[0].cells[0];
237 cell.indents = vec![18.0, 0.0, 27.0, 27.0];238 cell.indents = vec![18.0, 0.0, 27.0, 27.0];
238 let written = PreparedEdit::page(TREES, space, &after, AUTHOR).unwrap();239 let written = ops::saved(TREES, space, &after).unwrap();
239 let stored = assert_same(written.as_bytes(), space, &after);240 let stored = assert_same(written.as_slice(), space, &after);
240 let mut reset = stored.clone();241 let mut reset = stored.clone();
241 let cell = &mut table_mut(&mut reset).rows[0].cells[0];242 let cell = &mut table_mut(&mut reset).rows[0].cells[0];
242 cell.shading = None;243 cell.shading = None;
243 cell.indents = original_indents;244 cell.indents = original_indents;
244 let again = PreparedEdit::page(written.as_bytes(), space, &reset, AUTHOR).unwrap();245 let again = ops::saved(written.as_slice(), space, &reset).unwrap();
245 assert_eq!(assert_same(again.as_bytes(), space, &reset), before);246 assert_eq!(assert_same(again.as_slice(), space, &reset), before);
246}247}
247248
248/// `ONESTORE_NESTED_TABLE_EXPORT` names a new directory receiving the candidate for a cold reopen.249/// `ONESTORE_NESTED_TABLE_EXPORT` names a new directory receiving the candidate for a cold reopen.
...@@ -286,7 +287,7 @@ fn a_table_nests_inside_a_cell_and_is_removed_again() {...@@ -286,7 +287,7 @@ fn a_table_nests_inside_a_cell_and_is_removed_again() {
286 table_mut(&mut after).rows[0].cells[1]287 table_mut(&mut after).rows[0].cells[1]
287 .paragraphs288 .paragraphs
288 .push(holder);289 .push(holder);
289 let written = PreparedEdit::page(TREES, space, &after, AUTHOR).unwrap();290 let written = ops::saved(TREES, space, &after).unwrap();
290 let ParagraphContent::Table(nested) = &mut table_mut(&mut after).rows[0].cells[1]291 let ParagraphContent::Table(nested) = &mut table_mut(&mut after).rows[0].cells[1]
291 .paragraphs292 .paragraphs
292 .last_mut()293 .last_mut()
...@@ -296,12 +297,12 @@ fn a_table_nests_inside_a_cell_and_is_removed_again() {...@@ -296,12 +297,12 @@ fn a_table_nests_inside_a_cell_and_is_removed_again() {
296 panic!()297 panic!()
297 };298 };
298 with_native_indents(nested);299 with_native_indents(nested);
299 let stored = assert_same(written.as_bytes(), space, &after);300 let stored = assert_same(written.as_slice(), space, &after);
300 if let Some(directory) = std::env::var_os("ONESTORE_NESTED_TABLE_EXPORT") {301 if let Some(directory) = std::env::var_os("ONESTORE_NESTED_TABLE_EXPORT") {
301 let directory = std::path::PathBuf::from(directory);302 let directory = std::path::PathBuf::from(directory);
302 std::fs::create_dir(&directory).unwrap();303 std::fs::create_dir(&directory).unwrap();
303 std::fs::write(directory.join("synthetic.one"), written.as_bytes()).unwrap();304 std::fs::write(directory.join("synthetic.one"), written.as_slice()).unwrap();
304 let written_store = Store::parse(written.as_bytes()).unwrap();305 let written_store = Store::parse(written.as_slice()).unwrap();
305 std::fs::write(306 std::fs::write(
306 directory.join("Open Notebook.onetoc2"),307 directory.join("Open Notebook.onetoc2"),
307 onestore::create_table_of_contents(308 onestore::create_table_of_contents(
...@@ -316,8 +317,8 @@ fn a_table_nests_inside_a_cell_and_is_removed_again() {...@@ -316,8 +317,8 @@ fn a_table_nests_inside_a_cell_and_is_removed_again() {
316 table_mut(&mut removed).rows[0].cells[1]317 table_mut(&mut removed).rows[0].cells[1]
317 .paragraphs318 .paragraphs
318 .retain(|paragraph| paragraph.id != holder_id);319 .retain(|paragraph| paragraph.id != holder_id);
319 let again = PreparedEdit::page(written.as_bytes(), space, &removed, AUTHOR).unwrap();320 let again = ops::saved(written.as_slice(), space, &removed).unwrap();
320 assert_eq!(assert_same(again.as_bytes(), space, &removed), before);321 assert_eq!(assert_same(again.as_slice(), space, &removed), before);
321}322}
322323
323#[test]324#[test]
...@@ -328,20 +329,17 @@ fn inconsistent_tables_are_refused() {...@@ -328,20 +329,17 @@ fn inconsistent_tables_are_refused() {
328 table_mut(&mut ragged).rows[0]329 table_mut(&mut ragged).rows[0]
329 .cells330 .cells
330 .push(new_cell(&template, "Extra"));331 .push(new_cell(&template, "Extra"));
331 assert!(PreparedEdit::page(TREES, space, &ragged, AUTHOR).is_err());332 assert!(ops::saved(TREES, space, &ragged).is_err());
333 // A cell keeps a paragraph.
332 let mut empty = before.clone();334 let mut empty = before.clone();
333 table_mut(&mut empty).rows[0].cells[0].paragraphs.clear();335 table_mut(&mut empty).rows[0].cells[0].paragraphs.clear();
334 let written = PreparedEdit::page(TREES, space, &empty, AUTHOR).unwrap();336 assert!(ops::saved(TREES, space, &empty).is_err());
335 let mut stored = page_in(written.as_bytes(), space);
336 let cell = &table_mut(&mut stored).rows[0].cells[0];
337 assert_eq!(cell.paragraphs.len(), 1);
338 assert_eq!(cell.paragraphs[0].text().unwrap().text.text(), "");
339 let mut narrow = before.clone();337 let mut narrow = before.clone();
340 table_mut(&mut narrow).columns[0].width = 10.0;338 table_mut(&mut narrow).columns[0].width = 10.0;
341 assert!(PreparedEdit::page(TREES, space, &narrow, AUTHOR).is_err());339 assert!(ops::saved(TREES, space, &narrow).is_err());
342 let mut rowless = before.clone();340 let mut rowless = before.clone();
343 table_mut(&mut rowless).rows.clear();341 table_mut(&mut rowless).rows.clear();
344 assert!(PreparedEdit::page(TREES, space, &rowless, AUTHOR).is_err());342 assert!(ops::saved(TREES, space, &rowless).is_err());
345}343}
346344
347/// `ONESTORE_TABLE_EXPORT` names a new directory receiving the candidate for a cold reopen.345/// `ONESTORE_TABLE_EXPORT` names a new directory receiving the candidate for a cold reopen.
...@@ -394,14 +392,14 @@ fn a_new_table_is_created_on_a_fresh_page() {...@@ -394,14 +392,14 @@ fn a_new_table_is_created_on_a_fresh_page() {
394 holder.format = Format::default();392 holder.format = Format::default();
395 body_paragraphs(&mut after).push(holder);393 body_paragraphs(&mut after).push(holder);
396 body_paragraphs(&mut after).push(cell_paragraph(&template, "After the table"));394 body_paragraphs(&mut after).push(cell_paragraph(&template, "After the table"));
397 let written = PreparedEdit::page(&source, space, &after, AUTHOR).unwrap();395 let written = ops::saved(&source, space, &after).unwrap();
398 with_native_indents(table_mut(&mut after));396 with_native_indents(table_mut(&mut after));
399 assert_same(written.as_bytes(), space, &after);397 assert_same(written.as_slice(), space, &after);
400 if let Some(directory) = std::env::var_os("ONESTORE_TABLE_EXPORT") {398 if let Some(directory) = std::env::var_os("ONESTORE_TABLE_EXPORT") {
401 let directory = std::path::PathBuf::from(directory);399 let directory = std::path::PathBuf::from(directory);
402 std::fs::create_dir(&directory).unwrap();400 std::fs::create_dir(&directory).unwrap();
403 std::fs::write(directory.join("tables.one"), written.as_bytes()).unwrap();401 std::fs::write(directory.join("tables.one"), written.as_slice()).unwrap();
404 let written_store = Store::parse(written.as_bytes()).unwrap();402 let written_store = Store::parse(written.as_slice()).unwrap();
405 std::fs::write(403 std::fs::write(
406 directory.join("Open Notebook.onetoc2"),404 directory.join("Open Notebook.onetoc2"),
407 onestore::create_table_of_contents(405 onestore::create_table_of_contents(
crates/onestore/tests/page_tags.rs+20-26
...@@ -1,5 +1,7 @@...@@ -1,5 +1,7 @@
1#[path = "support/ops.rs"]
2mod ops;
1use onestore::{3use onestore::{
2 ExGuid, PreparedEdit, RevisionIndex, Store,4 ExGuid, RevisionIndex, Store,
3 document::{Document, Format, Kind, Tag},5 document::{Document, Format, Kind, Tag},
4 page::{6 page::{
5 Definition, Page, PageObject, PageParagraph, ParagraphContent, TextObject, text::new_id,7 Definition, Page, PageObject, PageParagraph, ParagraphContent, TextObject, text::new_id,
...@@ -8,7 +10,6 @@ use onestore::{...@@ -8,7 +10,6 @@ use onestore::{
810
9const TREES: &[u8] =11const TREES: &[u8] =
10 include_bytes!("../../../corpus/outline-edit/tree/before/notebook/synthetic.one");12 include_bytes!("../../../corpus/outline-edit/tree/before/notebook/synthetic.one");
11const AUTHOR: &str = "Tag author";
1213
13fn page_by_title(bytes: &[u8], title: &str) -> (ExGuid, Page) {14fn page_by_title(bytes: &[u8], title: &str) -> (ExGuid, Page) {
14 let store = Store::parse(bytes).unwrap();15 let store = Store::parse(bytes).unwrap();
...@@ -161,13 +162,13 @@ fn removing_a_tag_leaves_its_definition_and_the_other_paragraphs_untouched() {...@@ -161,13 +162,13 @@ fn removing_a_tag_leaves_its_definition_and_the_other_paragraphs_untouched() {
161 .unwrap()162 .unwrap()
162 .tags163 .tags
163 .clear();164 .clear();
164 let written = PreparedEdit::page(TREES, space, &after, AUTHOR).unwrap();165 let written = ops::saved(TREES, space, &after).unwrap();
165 let mut expected = after.clone();166 let mut expected = after.clone();
166 expected.definitions.remove(&definition);167 expected.definitions.remove(&definition);
167 let stored = assert_same(written.as_bytes(), space, &expected);168 let stored = assert_same(written.as_slice(), space, &expected);
168 assert!(!stored.definitions.contains_key(&definition));169 assert!(!stored.definitions.contains_key(&definition));
169 assert_eq!(170 assert_eq!(
170 raw_object(written.as_bytes(), space, definition),171 raw_object(written.as_slice(), space, definition),
171 raw_object(TREES, space, definition)172 raw_object(TREES, space, definition)
172 );173 );
173 let count = body_paragraphs(&mut after).len();174 let count = body_paragraphs(&mut after).len();
...@@ -176,7 +177,7 @@ fn removing_a_tag_leaves_its_definition_and_the_other_paragraphs_untouched() {...@@ -176,7 +177,7 @@ fn removing_a_tag_leaves_its_definition_and_the_other_paragraphs_untouched() {
176 .unwrap()177 .unwrap()
177 .id;178 .id;
178 assert_eq!(179 assert_eq!(
179 raw_object(written.as_bytes(), space, other),180 raw_object(written.as_slice(), space, other),
180 raw_object(TREES, space, other)181 raw_object(TREES, space, other)
181 );182 );
182}183}
...@@ -195,24 +196,17 @@ fn an_existing_definition_tags_another_paragraph_and_a_task_completes_in_place()...@@ -195,24 +196,17 @@ fn an_existing_definition_tags_another_paragraph_and_a_task_completes_in_place()
195 .unwrap()196 .unwrap()
196 .tags197 .tags
197 .push(tag(definition, false));198 .push(tag(definition, false));
198 let written = PreparedEdit::page(TREES, space, &after, AUTHOR).unwrap();199 let written = ops::saved(TREES, space, &after).unwrap();
199 let stored = assert_same(written.as_bytes(), space, &after);200 let stored = assert_same(written.as_slice(), space, &after);
200 let mut completed = stored.clone();201 let mut completed = stored.clone();
201 let text = body_paragraphs(&mut completed)[other].text_mut().unwrap();202 let text = body_paragraphs(&mut completed)[other].text_mut().unwrap();
202 text.tags[0].status = 1;203 text.tags[0].status = 1;
203 text.tags[0].completed = Some(1_262_405_106);204 text.tags[0].completed = Some(1_262_405_106);
204 let again = PreparedEdit::page(written.as_bytes(), space, &completed, AUTHOR).unwrap();205 let again = ops::saved(written.as_slice(), space, &completed).unwrap();
205 assert_same(again.as_bytes(), space, &completed);206 assert_same(again.as_slice(), space, &completed);
206 assert_eq!(207 assert_eq!(
207 PreparedEdit::page(208 ops::saved(&again, space, &page_in(&again, space)).unwrap(),
208 again.as_bytes(),209 again
209 space,
210 &page_in(again.as_bytes(), space),
211 AUTHOR
212 )
213 .unwrap()
214 .as_bytes(),
215 again.as_bytes()
216 );210 );
217}211}
218212
...@@ -225,7 +219,7 @@ fn tags_without_a_known_definition_are_refused() {...@@ -225,7 +219,7 @@ fn tags_without_a_known_definition_are_refused() {
225 .unwrap()219 .unwrap()
226 .tags220 .tags
227 .push(tag(new_id().unwrap(), false));221 .push(tag(new_id().unwrap(), false));
228 assert!(PreparedEdit::page(TREES, space, &missing, AUTHOR).is_err());222 assert!(ops::saved(TREES, space, &missing).is_err());
229 let mut repeated = before.clone();223 let mut repeated = before.clone();
230 let at = tagged(&repeated);224 let at = tagged(&repeated);
231 let existing = body_paragraphs(&mut repeated)[at].text().unwrap().tags[0].clone();225 let existing = body_paragraphs(&mut repeated)[at].text().unwrap().tags[0].clone();
...@@ -234,7 +228,7 @@ fn tags_without_a_known_definition_are_refused() {...@@ -234,7 +228,7 @@ fn tags_without_a_known_definition_are_refused() {
234 .unwrap()228 .unwrap()
235 .tags229 .tags
236 .push(tag(existing.definition.unwrap(), false));230 .push(tag(existing.definition.unwrap(), false));
237 assert!(PreparedEdit::page(TREES, space, &repeated, AUTHOR).is_err());231 assert!(ops::saved(TREES, space, &repeated).is_err());
238 let mut wrong = before.clone();232 let mut wrong = before.clone();
239 let list = wrong233 let list = wrong
240 .definitions234 .definitions
...@@ -247,7 +241,7 @@ fn tags_without_a_known_definition_are_refused() {...@@ -247,7 +241,7 @@ fn tags_without_a_known_definition_are_refused() {
247 .unwrap()241 .unwrap()
248 .tags242 .tags
249 .push(tag(list, false));243 .push(tag(list, false));
250 assert!(PreparedEdit::page(TREES, space, &wrong, AUTHOR).is_err());244 assert!(ops::saved(TREES, space, &wrong).is_err());
251 }245 }
252}246}
253247
...@@ -293,13 +287,13 @@ fn new_definitions_and_tags_publish_on_a_fresh_page() {...@@ -293,13 +287,13 @@ fn new_definitions_and_tags_publish_on_a_fresh_page() {
293 both.text_mut().unwrap().tags.push(tag(task, false));287 both.text_mut().unwrap().tags.push(tag(task, false));
294 body_paragraphs(&mut after).push(both);288 body_paragraphs(&mut after).push(both);
295 body_paragraphs(&mut after).push(plain_paragraph(&template, "Untagged"));289 body_paragraphs(&mut after).push(plain_paragraph(&template, "Untagged"));
296 let written = PreparedEdit::page(&source, space, &after, AUTHOR).unwrap();290 let written = ops::saved(&source, space, &after).unwrap();
297 assert_same(written.as_bytes(), space, &after);291 assert_same(written.as_slice(), space, &after);
298 if let Some(directory) = std::env::var_os("ONESTORE_TAG_EXPORT") {292 if let Some(directory) = std::env::var_os("ONESTORE_TAG_EXPORT") {
299 let directory = std::path::PathBuf::from(directory);293 let directory = std::path::PathBuf::from(directory);
300 std::fs::create_dir(&directory).unwrap();294 std::fs::create_dir(&directory).unwrap();
301 std::fs::write(directory.join("tags.one"), written.as_bytes()).unwrap();295 std::fs::write(directory.join("tags.one"), written.as_slice()).unwrap();
302 let written_store = Store::parse(written.as_bytes()).unwrap();296 let written_store = Store::parse(written.as_slice()).unwrap();
303 std::fs::write(297 std::fs::write(
304 directory.join("Open Notebook.onetoc2"),298 directory.join("Open Notebook.onetoc2"),
305 onestore::create_table_of_contents(299 onestore::create_table_of_contents(
crates/onestore/tests/page_writer.rs+155-107
...@@ -1,15 +1,34 @@...@@ -1,15 +1,34 @@
1//! The page writer must publish the same graph the typed writers publish for the same edit,1//! A page edit lowered whole (`op::lower_page`) must publish the same graph its ops applied
2//! in one transaction, and must round-trip the model it was given.2//! one by one publish, in one transaction, and must round-trip the model it was given.
3
4#[path = "support/ops.rs"]
5mod ops;
36
4use onestore::{7use onestore::{
5 ExGuid, Insertion, OutlineEdit, ParagraphJoin, ParagraphSplit, PreparedEdit, RevisionIndex,8 ExGuid, OutlineEdit, RevisionIndex, Store, TextAttribute,
6 Store, TextAttribute, TreeEdit,
7 document::{Document, Element, Kind, Revision},9 document::{Document, Element, Kind, Revision},
10 op::PageOp,
8 page::{Page, PageObject, ParagraphContent, text::new_id},11 page::{Page, PageObject, ParagraphContent, text::new_id},
9};12};
10use std::collections::BTreeSet;13use std::collections::BTreeSet;
1114
12const AUTHOR: &str = "Page author";15/// `source` after `ops` on the page in `space`.
16fn edited(source: &[u8], space: ExGuid, ops: Vec<PageOp>) -> Vec<u8> {
17 ops::page_edited(source, space, ops).unwrap()
18}
19
20/// A plain paragraph of `text` inserted in `container` before `before`, at `level`.
21fn insert(container: ExGuid, before: Option<ExGuid>, text: &str, level: u32) -> PageOp {
22 let mut paragraph = ops::paragraph(text);
23 paragraph.level = level;
24 PageOp::Insert {
25 container,
26 before,
27 paragraphs: vec![paragraph],
28 }
29}
30
31const AUTHOR: &str = "Author";
13const PARAGRAPHS: &[u8] =32const PARAGRAPHS: &[u8] =
14 include_bytes!("../../../corpus/paragraph-edit/before/notebook/synthetic.one");33 include_bytes!("../../../corpus/paragraph-edit/before/notebook/synthetic.one");
15const OUTLINES: &[u8] =34const OUTLINES: &[u8] =
...@@ -144,7 +163,7 @@ fn same_element(revision_a: &Revision<'_>, revision_b: &Revision<'_>, id: ExGuid...@@ -144,7 +163,7 @@ fn same_element(revision_a: &Revision<'_>, revision_b: &Revision<'_>, id: ExGuid
144 }163 }
145}164}
146165
147/// Every active revision the typed writers produced is reproduced by the page writer,166/// Every active revision the ops produced one by one is reproduced by the page lowered whole,
148/// ignoring writer-allocated immutable styles and author records. Modification times are167/// ignoring writer-allocated immutable styles and author records. Modification times are
149/// not compared: a reorder can be attributed to either sibling, and the model cannot say which.168/// not compared: a reorder can be attributed to either sibling, and the model cannot say which.
150fn equivalent(reference: &[u8], written: &[u8]) {169fn equivalent(reference: &[u8], written: &[u8]) {
...@@ -182,11 +201,11 @@ fn equivalent(reference: &[u8], written: &[u8]) {...@@ -182,11 +201,11 @@ fn equivalent(reference: &[u8], written: &[u8]) {
182 }201 }
183}202}
184203
185/// Runs the page writer on the model the typed writers produced and checks both contracts.204/// Lowers the page the ops produced whole and checks both contracts.
186fn oracle(source: &[u8], space: ExGuid, reference: &[u8]) -> Vec<u8> {205fn oracle(source: &[u8], space: ExGuid, reference: &[u8]) -> Vec<u8> {
187 let after = page_in(reference, space);206 let after = page_in(reference, space);
188 let prepared = PreparedEdit::page(source, space, &after, AUTHOR).unwrap();207 let prepared = ops::saved(source, space, &after).unwrap();
189 let written = prepared.as_bytes().to_vec();208 let written = prepared.as_slice().to_vec();
190 assert_eq!(page_in(&written, space), after, "model round trip");209 assert_eq!(page_in(&written, space), after, "model round trip");
191 equivalent(reference, &written);210 equivalent(reference, &written);
192 let transactions = |bytes: &[u8]| Store::parse(bytes).unwrap().header.transaction_count;211 let transactions = |bytes: &[u8]| Store::parse(bytes).unwrap().header.transaction_count;
...@@ -209,9 +228,12 @@ fn text_replacement_matches_replace_text() {...@@ -209,9 +228,12 @@ fn text_replacement_matches_replace_text() {
209 (end..end, " end"),228 (end..end, " end"),
210 (0..end, ""),229 (0..end, ""),
211 ] {230 ] {
212 let reference =231 let op = PageOp::Text {
213 onestore::replace_text(PARAGRAPHS, space, text, range, replacement).unwrap();232 text,
214 oracle(PARAGRAPHS, space, &reference);233 range,
234 with: replacement.into(),
235 };
236 oracle(PARAGRAPHS, space, &edited(PARAGRAPHS, space, vec![op]));
215 }237 }
216}238}
217239
...@@ -220,28 +242,16 @@ fn formatting_matches_format_text() {...@@ -220,28 +242,16 @@ fn formatting_matches_format_text() {
220 let (space, page) = page_by_title(PARAGRAPHS, "Split style boundary");242 let (space, page) = page_by_title(PARAGRAPHS, "Split style boundary");
221 let (_, text, content) = text_ids(body(&page)[0])[0].clone();243 let (_, text, content) = text_ids(body(&page)[0])[0].clone();
222 let end = u32::try_from(content.encode_utf16().count()).unwrap();244 let end = u32::try_from(content.encode_utf16().count()).unwrap();
223 let reference = PreparedEdit::format(245 let reference = ops::page_op(PARAGRAPHS, space, PageOp::Format { text, range: 1..end.min(4), set: vec![
224 PARAGRAPHS,
225 space,
226 text,
227 1..end.min(4),
228 &[
229 TextAttribute::Bold(true),246 TextAttribute::Bold(true),
230 TextAttribute::Color(Some([255, 0, 0])),247 TextAttribute::Color(Some([255, 0, 0])),
231 TextAttribute::FontSize(14.0),248 TextAttribute::FontSize(14.0),
232 ],249 ], clear: Vec::new() })
233 )
234 .unwrap();250 .unwrap();
235 let reference = PreparedEdit::format(251 let reference = ops::page_op(reference.as_bytes(), space, PageOp::Format { text, range: 0..1, set: vec![
236 reference.as_bytes(),
237 space,
238 text,
239 0..1,
240 &[
241 TextAttribute::Italic(true),252 TextAttribute::Italic(true),
242 TextAttribute::Font("Consolas".into()),253 TextAttribute::Font("Consolas".into()),
243 ],254 ], clear: Vec::new() })
244 )
245 .unwrap();255 .unwrap();
246 oracle(PARAGRAPHS, space, reference.as_bytes());256 oracle(PARAGRAPHS, space, reference.as_bytes());
247}257}
...@@ -257,14 +267,13 @@ fn paragraph_insertions_match_insertion() {...@@ -257,14 +267,13 @@ fn paragraph_insertions_match_insertion() {
257 .find(|p| p.parent.is_none())267 .find(|p| p.parent.is_none())
258 .unwrap()268 .unwrap()
259 .id;269 .id;
260 for intent in [270 for op in [
261 Insertion::paragraph(outline.id, None, "Appended 🦋 é", AUTHOR).unwrap(),271 insert(outline.id, None, "Appended 🦋 é", 1),
262 Insertion::paragraph(outline.id, Some(paragraphs[0].0), "First", AUTHOR).unwrap(),272 insert(outline.id, Some(paragraphs[0].0), "First", 1),
263 Insertion::paragraph(parent, None, "Nested child", AUTHOR).unwrap(),273 insert(parent, None, "Nested child", 2),
264 Insertion::paragraph(outline.id, None, "", AUTHOR).unwrap(),274 insert(outline.id, None, "", 1),
265 ] {275 ] {
266 let reference = PreparedEdit::insert(OUTLINES, space, &intent).unwrap();276 oracle(OUTLINES, space, &edited(OUTLINES, space, vec![op]));
267 oracle(OUTLINES, space, reference.as_bytes());
268 }277 }
269}278}
270279
...@@ -281,27 +290,23 @@ fn cell_insertion_and_deletion_match() {...@@ -281,27 +290,23 @@ fn cell_insertion_and_deletion_match() {
281 })290 })
282 .unwrap();291 .unwrap();
283 let cell = &table.rows[0].cells[0];292 let cell = &table.rows[0].cells[0];
284 let intent = Insertion::paragraph(cell.id, None, "Cell text", AUTHOR).unwrap();293 let reference = edited(TREES, space, vec![insert(cell.id, None, "Cell text", 1)]);
285 let reference = PreparedEdit::insert(TREES, space, &intent).unwrap();294 oracle(TREES, space, &reference);
286 oracle(TREES, space, reference.as_bytes());
287 let victim = cell295 let victim = cell
288 .paragraphs296 .paragraphs
289 .iter()297 .iter()
290 .find(|p| p.parent.is_none())298 .find(|p| p.parent.is_none())
291 .unwrap()299 .unwrap()
292 .id;300 .id;
293 let reference =301 let reference = edited(&reference, space, vec![PageOp::Delete { object: victim }]);
294 PreparedEdit::tree(TREES, space, &TreeEdit::delete(victim, AUTHOR).unwrap()).unwrap();302 oracle(TREES, space, &reference);
295 oracle(TREES, space, reference.as_bytes());
296}303}
297304
298#[test]305#[test]
299fn outline_insertion_matches_insertion() {306fn outline_insertion_matches_insertion() {
300 let (space, _) = page_by_title(OUTLINES, "Move leaf down");307 let (space, _) = page_by_title(OUTLINES, "Move leaf down");
301 let page = page_id(OUTLINES, space);308 let (add, ..) = ops::new_outline(144.0, 200.0, "New outline 東京");
302 let intent = Insertion::outline(page, 144.0, 200.0, "New outline 東京", AUTHOR).unwrap();309 oracle(OUTLINES, space, &edited(OUTLINES, space, vec![add]));
303 let reference = PreparedEdit::insert(OUTLINES, space, &intent).unwrap();
304 oracle(OUTLINES, space, reference.as_bytes());
305}310}
306311
307#[test]312#[test]
...@@ -325,16 +330,24 @@ fn splits_and_joins_match_their_writers() {...@@ -325,16 +330,24 @@ fn splits_and_joins_match_their_writers() {
325 .is_none_or(|u| !(0xd800..=0xdbff).contains(&u))330 .is_none_or(|u| !(0xd800..=0xdbff).contains(&u))
326 })331 })
327 .unwrap();332 .unwrap();
328 let split = ParagraphSplit::new(text, offset, AUTHOR).unwrap();333 let lists = body(&page)[0].paragraphs[0].lists.len();
329 let reference = PreparedEdit::split(PARAGRAPHS, space, &split).unwrap();334 let split = PageOp::Split {
330 oracle(PARAGRAPHS, space, reference.as_bytes());335 text,
336 at: offset,
337 paragraph: new_id().unwrap(),
338 right: new_id().unwrap(),
339 lists: (0..lists).map(|_| new_id().unwrap()).collect(),
340 };
341 oracle(PARAGRAPHS, space, &edited(PARAGRAPHS, space, vec![split]));
331 }342 }
332 for title in ["Split middle", "Split empty"] {343 for title in ["Split middle", "Split empty"] {
333 let (space, page) = page_by_title(PARAGRAPHS, title);344 let (space, page) = page_by_title(PARAGRAPHS, title);
334 let paragraphs = text_ids(body(&page)[0]);345 let paragraphs = text_ids(body(&page)[0]);
335 let join = ParagraphJoin::new(paragraphs[0].1, paragraphs[1].1, AUTHOR).unwrap();346 let join = PageOp::Join {
336 let reference = PreparedEdit::join(PARAGRAPHS, space, &join).unwrap();347 left: paragraphs[0].1,
337 oracle(PARAGRAPHS, space, reference.as_bytes());348 right: paragraphs[1].1,
349 };
350 oracle(PARAGRAPHS, space, &edited(PARAGRAPHS, space, vec![join]));
338 }351 }
339}352}
340353
...@@ -357,18 +370,28 @@ fn subtree_moves_and_deletions_match_tree_edits() {...@@ -357,18 +370,28 @@ fn subtree_moves_and_deletions_match_tree_edits() {
357 .filter(|p| p.parent.is_none())370 .filter(|p| p.parent.is_none())
358 .map(|p| p.id)371 .map(|p| p.id)
359 .collect();372 .collect();
360 let edit = match build {373 let op = match build {
361 0 | 1 => TreeEdit::move_to(top[0], outline.id, top.get(2).copied(), AUTHOR).unwrap(),374 0 | 1 => PageOp::Move {
362 2 | 3 => TreeEdit::delete(top[0], AUTHOR).unwrap(),375 object: top[0],
363 4 => TreeEdit::move_to(outline.id, page_id(OUTLINES, space), None, AUTHOR).unwrap(),376 parent: Some(outline.id),
364 _ => TreeEdit::delete(outline.id, AUTHOR).unwrap(),377 before: top.get(2).copied(),
378 },
379 2 | 3 => PageOp::Delete { object: top[0] },
380 4 => PageOp::Move {
381 object: outline.id,
382 parent: None,
383 before: None,
384 },
385 _ => PageOp::Delete { object: outline.id },
365 };386 };
366 let reference = PreparedEdit::tree(OUTLINES, space, &edit).unwrap();387 oracle(OUTLINES, space, &edited(OUTLINES, space, vec![op]));
367 oracle(OUTLINES, space, reference.as_bytes());
368 if outlines.len() > 1 && build == 0 {388 if outlines.len() > 1 && build == 0 {
369 let across = TreeEdit::move_to(top[1], outlines[1].id, None, AUTHOR).unwrap();389 let across = PageOp::Move {
370 let reference = PreparedEdit::tree(OUTLINES, space, &across).unwrap();390 object: top[1],
371 oracle(OUTLINES, space, reference.as_bytes());391 parent: Some(outlines[1].id),
392 before: None,
393 };
394 oracle(OUTLINES, space, &edited(OUTLINES, space, vec![across]));
372 }395 }
373 }396 }
374}397}
...@@ -401,7 +424,7 @@ fn outline_layout_and_collapse_match_outline_edits() {...@@ -401,7 +424,7 @@ fn outline_layout_and_collapse_match_outline_edits() {
401 ),424 ),
402 (parent, OutlineEdit::Collapsed(true)),425 (parent, OutlineEdit::Collapsed(true)),
403 ] {426 ] {
404 let reference = PreparedEdit::outline(OUTLINES, space, object, edit).unwrap();427 let reference = ops::page_op(OUTLINES, space, PageOp::Outline { object, edit }).unwrap();
405 oracle(OUTLINES, space, reference.as_bytes());428 oracle(OUTLINES, space, reference.as_bytes());
406 }429 }
407}430}
...@@ -411,38 +434,41 @@ fn composed_edits_publish_one_transaction() {...@@ -411,38 +434,41 @@ fn composed_edits_publish_one_transaction() {
411 let (space, page) = page_by_title(OUTLINES, "Move subtree up");434 let (space, page) = page_by_title(OUTLINES, "Move subtree up");
412 let outline = body(&page)[0];435 let outline = body(&page)[0];
413 let paragraphs = text_ids(outline);436 let paragraphs = text_ids(outline);
414 let mut reference =437 let mut reference = edited(
415 onestore::replace_text(OUTLINES, space, paragraphs[0].1, 0..0, "Lead ").unwrap();438 OUTLINES,
416 let insertion =439 space,
417 Insertion::paragraph(outline.id, Some(paragraphs[1].0), "Inserted", AUTHOR).unwrap();440 vec![PageOp::Text {
418 reference = PreparedEdit::insert(&reference, space, &insertion)441 text: paragraphs[0].1,
419 .unwrap()442 range: 0..0,
420 .as_bytes()443 with: "Lead ".into(),
421 .to_vec();444 }],
422 reference = PreparedEdit::format(445 );
446 let insertion = insert(outline.id, Some(paragraphs[1].0), "Inserted", 1);
447 let PageOp::Insert { paragraphs: inserted, .. } = &insertion else {
448 unreachable!()
449 };
450 let inserted = inserted[0].text().unwrap().id;
451 reference = edited(&reference, space, vec![insertion]);
452 reference = edited(
423 &reference,453 &reference,
424 space,454 space,
425 insertion.text_object(),455 vec![PageOp::Format {
426 0..3,456 text: inserted,
427 &[TextAttribute::Bold(true)],457 range: 0..3,
428 )458 set: vec![TextAttribute::Bold(true)],
429 .unwrap()459 clear: Vec::new(),
430 .as_bytes()460 }],
431 .to_vec();461 );
432 let last = paragraphs.last().unwrap().0;462 let last = paragraphs.last().unwrap().0;
433 reference = PreparedEdit::tree(&reference, space, &TreeEdit::delete(last, AUTHOR).unwrap())463 reference = edited(&reference, space, vec![PageOp::Delete { object: last }]);
434 .unwrap()464 reference = edited(
435 .as_bytes()
436 .to_vec();
437 reference = PreparedEdit::outline(
438 &reference,465 &reference,
439 space,466 space,
440 outline.id,467 vec![PageOp::Outline {
441 OutlineEdit::Position { x: 60.0, y: 60.0 },468 object: outline.id,
442 )469 edit: OutlineEdit::Position { x: 60.0, y: 60.0 },
443 .unwrap()470 }],
444 .as_bytes()471 );
445 .to_vec();
446 let written = oracle(OUTLINES, space, &reference);472 let written = oracle(OUTLINES, space, &reference);
447 assert_eq!(473 assert_eq!(
448 Store::parse(&reference).unwrap().header.transaction_count,474 Store::parse(&reference).unwrap().header.transaction_count,
...@@ -535,8 +561,8 @@ fn model_edits_round_trip_and_leave_other_objects_untouched() {...@@ -535,8 +561,8 @@ fn model_edits_round_trip_and_leave_other_objects_untouched() {
535 );561 );
536 outline.paragraphs.push(paragraph);562 outline.paragraphs.push(paragraph);
537 after.objects.push(PageObject::Outline(outline));563 after.objects.push(PageObject::Outline(outline));
538 let written = PreparedEdit::page(TREES, space, &after, AUTHOR).unwrap();564 let written = ops::saved(TREES, space, &after).unwrap();
539 let stored = page_in(written.as_bytes(), space);565 let stored = page_in(written.as_slice(), space);
540 let normalize = |page: &Page| -> Vec<(ExGuid, Vec<(ExGuid, String)>)> {566 let normalize = |page: &Page| -> Vec<(ExGuid, Vec<(ExGuid, String)>)> {
541 body(page)567 body(page)
542 .iter()568 .iter()
...@@ -565,7 +591,7 @@ fn model_edits_round_trip_and_leave_other_objects_untouched() {...@@ -565,7 +591,7 @@ fn model_edits_round_trip_and_leave_other_objects_untouched() {
565 let store = Store::parse(TREES).unwrap();591 let store = Store::parse(TREES).unwrap();
566 let index = RevisionIndex::parse(&store).unwrap();592 let index = RevisionIndex::parse(&store).unwrap();
567 let raw_before = index.resolve_active(space).unwrap();593 let raw_before = index.resolve_active(space).unwrap();
568 let store = Store::parse(written.as_bytes()).unwrap();594 let store = Store::parse(written.as_slice()).unwrap();
569 let index = RevisionIndex::parse(&store).unwrap();595 let index = RevisionIndex::parse(&store).unwrap();
570 let raw_after = index.resolve_active(space).unwrap();596 let raw_after = index.resolve_active(space).unwrap();
571 let touched: BTreeSet<ExGuid> = body(&after)597 let touched: BTreeSet<ExGuid> = body(&after)
...@@ -604,7 +630,7 @@ fn unsupported_model_edits_are_rejected_before_writing() {...@@ -604,7 +630,7 @@ fn unsupported_model_edits_are_rejected_before_writing() {
604 }630 }
605 }631 }
606 }632 }
607 assert!(PreparedEdit::page(TREES, space, &widened, AUTHOR).is_ok());633 assert!(ops::saved(TREES, space, &widened).is_ok());
608 let mut relisted = page_in(TREES, space);634 let mut relisted = page_in(TREES, space);
609 let outline = body(&page)[0];635 let outline = body(&page)[0];
610 for object in &mut relisted.objects {636 for object in &mut relisted.objects {
...@@ -614,11 +640,33 @@ fn unsupported_model_edits_are_rejected_before_writing() {...@@ -614,11 +640,33 @@ fn unsupported_model_edits_are_rejected_before_writing() {
614 o.paragraphs[0].lists.push(new_id().unwrap());640 o.paragraphs[0].lists.push(new_id().unwrap());
615 }641 }
616 }642 }
617 assert!(PreparedEdit::page(TREES, space, &relisted, AUTHOR).is_err());643 assert!(ops::saved(TREES, space, &relisted).is_err());
618 let mut renamed = page_in(TREES, space);644 // A page's date is an op of its own now.
619 renamed.created = Some(1);645 let mut redated = page_in(TREES, space);
620 assert!(PreparedEdit::page(TREES, space, &renamed, AUTHOR).is_err());646 redated.created = Some(1);
621 assert!(PreparedEdit::page(TREES, space, &page, "a\0b").is_err());647 let written = ops::saved(TREES, space, &redated).unwrap();
648 assert_eq!(page_in(&written, space).created, Some(1));
649 let text = page
650 .objects
651 .iter()
652 .find_map(|object| match object {
653 PageObject::Title(title) => title
654 .outlines
655 .iter()
656 .flat_map(|outline| &outline.paragraphs)
657 .find_map(|paragraph| paragraph.text().map(|text| text.id)),
658 _ => None,
659 })
660 .unwrap();
661 let op = onestore::op::Op::Page {
662 space,
663 op: PageOp::Text {
664 text,
665 range: 0..0,
666 with: "x".into(),
667 },
668 };
669 assert!(ops::transaction(TREES, "a\0b", vec![op]).is_err());
622}670}
623671
624/// Unset run values stay unset; only the language, which MS-ONE requires on every run, comes672/// Unset run values stay unset; only the language, which MS-ONE requires on every run, comes
...@@ -655,8 +703,8 @@ fn a_new_paragraph_with_unspecified_formatting_rereads_with_only_the_required_la...@@ -655,8 +703,8 @@ fn a_new_paragraph_with_unspecified_formatting_rereads_with_only_the_required_la
655 unsupported: Vec::new(),703 unsupported: Vec::new(),
656 };704 };
657 page.objects.insert(0, PageObject::Outline(outline));705 page.objects.insert(0, PageObject::Outline(outline));
658 let written = PreparedEdit::page(OUTLINES, space, &page, AUTHOR).unwrap();706 let written = ops::saved(OUTLINES, space, &page).unwrap();
659 let stored = page_in(written.as_bytes(), space);707 let stored = page_in(written.as_slice(), space);
660 let read_back = body(&stored)708 let read_back = body(&stored)
661 .iter()709 .iter()
662 .flat_map(|outline| outline.paragraphs.iter())710 .flat_map(|outline| outline.paragraphs.iter())
...@@ -673,18 +721,18 @@ fn a_new_paragraph_with_unspecified_formatting_rereads_with_only_the_required_la...@@ -673,18 +721,18 @@ fn a_new_paragraph_with_unspecified_formatting_rereads_with_only_the_required_la
673 }721 }
674 );722 );
675 assert_eq!(723 assert_eq!(
676 PreparedEdit::page(written.as_bytes(), space, &stored, AUTHOR)724 ops::saved(written.as_slice(), space, &stored)
677 .unwrap()725 .unwrap()
678 .as_bytes(),726 .as_slice(),
679 written.as_bytes()727 written.as_slice()
680 );728 );
681}729}
682730
683#[test]731#[test]
684fn an_unchanged_model_publishes_nothing() {732fn an_unchanged_model_publishes_nothing() {
685 let (space, page) = page_by_title(OUTLINES, "Resize outline");733 let (space, page) = page_by_title(OUTLINES, "Resize outline");
686 let prepared = PreparedEdit::page(OUTLINES, space, &page, AUTHOR).unwrap();734 let prepared = ops::saved(OUTLINES, space, &page).unwrap();
687 assert_eq!(prepared.as_bytes(), OUTLINES);735 assert_eq!(prepared.as_slice(), OUTLINES);
688}736}
689737
690const JOIN_PAGE: &str = "Bold 🦀 italic e\u{301} color 東京";738const JOIN_PAGE: &str = "Bold 🦀 italic e\u{301} color 東京";
...@@ -1001,8 +1049,8 @@ fn export_native_page_model_candidates() {...@@ -1001,8 +1049,8 @@ fn export_native_page_model_candidates() {
1001 });1049 });
1002 }1050 }
1003 }1051 }
1004 let edit = PreparedEdit::page(&bytes, space, &after, "Rust page writer").unwrap();1052 let edit = ops::saved(&bytes, space, &after).unwrap();
1005 bytes = edit.as_bytes().to_vec();1053 bytes = edit.as_slice().to_vec();
1006 let stored = page_in(&bytes, space);1054 let stored = page_in(&bytes, space);
1007 let index = {1055 let index = {
1008 let store = Store::parse(&bytes).unwrap();1056 let store = Store::parse(&bytes).unwrap();
crates/onestore/tests/paragraph.rs+164-104
...@@ -1,6 +1,8 @@...@@ -1,6 +1,8 @@
1use onestore::{1use onestore::{
2 ExGuid, ParagraphJoin, ParagraphSplit, PreparedEdit, RevisionIndex, Store,2 ExGuid, RevisionIndex, Store,
3 document::{Document, Kind, Revision},3 document::{Document, Kind, Revision},
4 op::{Op, OpError, PageOp},
5 page::text::new_id,
4};6};
5use serde_json::Value;7use serde_json::Value;
68
...@@ -10,6 +12,40 @@ mod checkpoint;...@@ -10,6 +12,40 @@ mod checkpoint;
10mod current;12mod current;
11#[path = "support/disk.rs"]13#[path = "support/disk.rs"]
12mod disk;14mod disk;
15#[path = "support/ops.rs"]
16mod ops;
17
18/// Enter at `at` in `text`, with fresh identities for the new paragraph, its text and a
19/// copy of each list node of the paragraph: the op, the new paragraph and its text.
20fn split(source: &[u8], space: ExGuid, text: ExGuid, at: u32) -> (Op, ExGuid, ExGuid) {
21 let store = Store::parse(source).unwrap();
22 let index = RevisionIndex::parse(&store).unwrap();
23 let document = Document::parse(&index).unwrap();
24 let lists = document.spaces[&space]
25 .active()
26 .unwrap()
27 .nodes
28 .values()
29 .find_map(|node| match &node.kind {
30 Kind::Paragraph { lists, .. } if node.content == [text] => Some(lists.len()),
31 _ => None,
32 })
33 .unwrap_or(0);
34 let (paragraph, right) = (new_id().unwrap(), new_id().unwrap());
35 let op = PageOp::Split {
36 text,
37 at,
38 paragraph,
39 right,
40 lists: (0..lists).map(|_| new_id().unwrap()).collect(),
41 };
42 (Op::Page { space, op }, paragraph, right)
43}
44
45/// `source` after joining the paragraph of text `right` to the one of text `left`.
46fn join(source: &[u8], space: ExGuid, left: ExGuid, right: ExGuid) -> Result<Vec<u8>, OpError> {
47 ops::edited(source, vec![Op::Page { space, op: PageOp::Join { left, right } }])
48}
1349
14const SOURCE: &[u8] =50const SOURCE: &[u8] =
15 include_bytes!("../../../corpus/paragraph-edit/before/notebook/synthetic.one");51 include_bytes!("../../../corpus/paragraph-edit/before/notebook/synthetic.one");
...@@ -114,12 +150,9 @@ fn joins_match_native_graphs_tags_and_inherited_character_styles() {...@@ -114,12 +150,9 @@ fn joins_match_native_graphs_tags_and_inherited_character_styles() {
114 .contains_key(&left)150 .contains_key(&left)
115 })151 })
116 .unwrap();152 .unwrap();
117 let intent = ParagraphJoin::new(left, right, "Join author").unwrap();153 let edited =
118 let restored = serde_json::from_value(serde_json::to_value(&intent).unwrap()).unwrap();154 join(source, *sid, left, right).unwrap_or_else(|error| panic!("{name}: {error}"));
119 assert_eq!(intent, restored);155 let current_store = Store::parse(&edited).unwrap();
120 let edited = PreparedEdit::join(source, *sid, &restored)
121 .unwrap_or_else(|error| panic!("{name}: {error}"));
122 let current_store = Store::parse(edited.as_bytes()).unwrap();
123 assert_eq!(156 assert_eq!(
124 current_store.header.transaction_count,157 current_store.header.transaction_count,
125 store.header.transaction_count + 1158 store.header.transaction_count + 1
...@@ -276,7 +309,7 @@ fn joins_match_native_graphs_tags_and_inherited_character_styles() {...@@ -276,7 +309,7 @@ fn joins_match_native_graphs_tags_and_inherited_character_styles() {
276 }309 }
277 }310 }
278 }311 }
279 assert!(PreparedEdit::join(edited.as_bytes(), *sid, &intent).is_err());312 assert!(join(&edited, *sid, left, right).is_err());
280 assert_eq!(313 assert_eq!(
281 space.contexts.len(),314 space.contexts.len(),
282 current_document.spaces[sid].contexts.len()315 current_document.spaces[sid].contexts.len()
...@@ -321,10 +354,8 @@ fn splits_partition_native_paragraphs_at_every_scalar_boundary() {...@@ -321,10 +354,8 @@ fn splits_partition_native_paragraphs_at_every_scalar_boundary() {
321 }))354 }))
322 .collect();355 .collect();
323 for (position, offset) in offsets.iter().enumerate() {356 for (position, offset) in offsets.iter().enumerate() {
324 let intent = ParagraphSplit::new(text, *offset, "Split author").unwrap();357 let (op, new_paragraph, new_text) = split(SOURCE, *sid, text, *offset);
325 let restored = serde_json::from_value(serde_json::to_value(&intent).unwrap()).unwrap();358 let edited = ops::edited(SOURCE, vec![op.clone()]);
326 assert_eq!(intent, restored);
327 let edited = PreparedEdit::split(SOURCE, *sid, &restored);
328 // OneNote 2010 splits before or after a hyperlink and ignores Enter inside one.359 // OneNote 2010 splits before or after a hyperlink and ignores Enter inside one.
329 let link = |at: usize| {360 let link = |at: usize| {
330 expected361 expected
...@@ -341,7 +372,7 @@ fn splits_partition_native_paragraphs_at_every_scalar_boundary() {...@@ -341,7 +372,7 @@ fn splits_partition_native_paragraphs_at_every_scalar_boundary() {
341 }372 }
342 let edited =373 let edited =
343 edited.unwrap_or_else(|error| panic!("{} at {offset}: {error}", case["case"]));374 edited.unwrap_or_else(|error| panic!("{} at {offset}: {error}", case["case"]));
344 let current_store = Store::parse(edited.as_bytes()).unwrap();375 let current_store = Store::parse(&edited).unwrap();
345 assert_eq!(376 assert_eq!(
346 current_store.header.transaction_count,377 current_store.header.transaction_count,
347 store.header.transaction_count + 1378 store.header.transaction_count + 1
...@@ -384,7 +415,7 @@ fn splits_partition_native_paragraphs_at_every_scalar_boundary() {...@@ -384,7 +415,7 @@ fn splits_partition_native_paragraphs_at_every_scalar_boundary() {
384 case["case"]415 case["case"]
385 );416 );
386 assert_eq!(417 assert_eq!(
387 characters(after, intent.text_object()),418 characters(after, new_text),
388 expected[position..],419 expected[position..],
389 "{} suffix at {offset}",420 "{} suffix at {offset}",
390 case["case"]421 case["case"]
...@@ -392,7 +423,7 @@ fn splits_partition_native_paragraphs_at_every_scalar_boundary() {...@@ -392,7 +423,7 @@ fn splits_partition_native_paragraphs_at_every_scalar_boundary() {
392 if case["offset_utf16"] == *offset {423 if case["offset_utf16"] == *offset {
393 let native = &native_document.spaces[sid];424 let native = &native_document.spaces[sid];
394 let native = &native.revisions[&native.contexts[&ExGuid::default()]];425 let native = &native.revisions[&native.contexts[&ExGuid::default()]];
395 let new_text = serde_json::from_value(case["new_text"].clone()).unwrap();426 let native_text = serde_json::from_value(case["new_text"].clone()).unwrap();
396 assert_eq!(427 assert_eq!(
397 characters(after, text),428 characters(after, text),
398 characters(native, text),429 characters(native, text),
...@@ -400,8 +431,8 @@ fn splits_partition_native_paragraphs_at_every_scalar_boundary() {...@@ -400,8 +431,8 @@ fn splits_partition_native_paragraphs_at_every_scalar_boundary() {
400 case["case"]431 case["case"]
401 );432 );
402 assert_eq!(433 assert_eq!(
403 characters(after, intent.text_object()),434 characters(after, new_text),
404 characters(native, new_text),435 characters(native, native_text),
405 "{}",436 "{}",
406 case["case"]437 case["case"]
407 );438 );
...@@ -409,31 +440,31 @@ fn splits_partition_native_paragraphs_at_every_scalar_boundary() {...@@ -409,31 +440,31 @@ fn splits_partition_native_paragraphs_at_every_scalar_boundary() {
409 let mut children = parent_node.children.clone();440 let mut children = parent_node.children.clone();
410 children.insert(441 children.insert(
411 children.iter().position(|id| *id == paragraph).unwrap() + 1,442 children.iter().position(|id| *id == paragraph).unwrap() + 1,
412 intent.object(),443 new_paragraph,
413 );444 );
414 assert_eq!(after.nodes[parent].children, children);445 assert_eq!(after.nodes[parent].children, children);
415 assert_eq!(after.nodes[&paragraph].content, [text]);446 assert_eq!(after.nodes[&paragraph].content, [text]);
416 assert!(after.nodes[&paragraph].children.is_empty());447 assert!(after.nodes[&paragraph].children.is_empty());
417 assert_eq!(448 assert_eq!(
418 after.nodes[&intent.object()].content,449 after.nodes[&new_paragraph].content,
419 [intent.text_object()]450 [new_text]
420 );451 );
421 assert_eq!(452 assert_eq!(
422 after.nodes[&intent.object()].children,453 after.nodes[&new_paragraph].children,
423 before.nodes[&paragraph].children454 before.nodes[&paragraph].children
424 );455 );
425 assert_eq!(456 assert_eq!(
426 serde_json::to_value(&after.nodes[&text].tags).unwrap(),457 serde_json::to_value(&after.nodes[&text].tags).unwrap(),
427 serde_json::to_value(&before.nodes[&text].tags).unwrap()458 serde_json::to_value(&before.nodes[&text].tags).unwrap()
428 );459 );
429 assert!(after.nodes[&intent.text_object()].tags.is_empty());460 assert!(after.nodes[&new_text].tags.is_empty());
430 let Kind::Paragraph {461 let Kind::Paragraph {
431 lists: old_lists, ..462 lists: old_lists, ..
432 } = &before.nodes[&paragraph].kind463 } = &before.nodes[&paragraph].kind
433 else {464 else {
434 panic!()465 panic!()
435 };466 };
436 let Kind::Paragraph { lists, .. } = &after.nodes[&intent.object()].kind else {467 let Kind::Paragraph { lists, .. } = &after.nodes[&new_paragraph].kind else {
437 panic!()468 panic!()
438 };469 };
439 assert_eq!(old_lists.len(), lists.len());470 assert_eq!(old_lists.len(), lists.len());
...@@ -453,26 +484,23 @@ fn splits_partition_native_paragraphs_at_every_scalar_boundary() {...@@ -453,26 +484,23 @@ fn splits_partition_native_paragraphs_at_every_scalar_boundary() {
453 }484 }
454 }485 }
455 }486 }
456 assert!(PreparedEdit::split(edited.as_bytes(), *sid, &intent).is_err());487 assert!(ops::edited(&edited, vec![op]).is_err());
457 }488 }
458 for offset in 0..=*offsets.last().unwrap() + 1 {489 for offset in 0..=*offsets.last().unwrap() + 1 {
459 if offsets.contains(&offset) {490 if offsets.contains(&offset) {
460 continue;491 continue;
461 }492 }
462 let intent = ParagraphSplit::new(text, offset, "Author").unwrap();493 assert!(ops::edited(SOURCE, vec![split(SOURCE, *sid, text, offset).0]).is_err());
463 assert!(PreparedEdit::split(SOURCE, *sid, &intent).is_err());
464 }494 }
465 }495 }
466}496}
467497
468#[test]498#[test]
469fn invalid_split_identities_and_title_targets_are_rejected() {499fn invalid_split_identities_and_title_targets_are_rejected() {
470 assert!(ParagraphSplit::new(ExGuid::default(), 0, "Author").is_err());
471 let manifest: Value =500 let manifest: Value =
472 serde_json::from_str(include_str!("../../../corpus/paragraph-edit/manifest.json")).unwrap();501 serde_json::from_str(include_str!("../../../corpus/paragraph-edit/manifest.json")).unwrap();
473 let text: ExGuid =502 let text: ExGuid =
474 serde_json::from_value(manifest["cases"][0]["original_text"].clone()).unwrap();503 serde_json::from_value(manifest["cases"][0]["original_text"].clone()).unwrap();
475 assert!(ParagraphSplit::new(text, 0, "a\0b").is_err());
476 let store = Store::parse(SOURCE).unwrap();504 let store = Store::parse(SOURCE).unwrap();
477 let index = RevisionIndex::parse(&store).unwrap();505 let index = RevisionIndex::parse(&store).unwrap();
478 let document = Document::parse(&index).unwrap();506 let document = Document::parse(&index).unwrap();
...@@ -486,18 +514,19 @@ fn invalid_split_identities_and_title_targets_are_rejected() {...@@ -486,18 +514,19 @@ fn invalid_split_identities_and_title_targets_are_rejected() {
486 })514 })
487 .unwrap()515 .unwrap()
488 .0;516 .0;
489 let intent = ParagraphSplit::new(text, 1, "Author").unwrap();517 let (op, ..) = split(SOURCE, sid, text, 1);
490 for (field, value) in [518 assert!(ops::transaction(SOURCE, "a\0b", vec![op.clone()]).is_err());
491 ("guid", serde_json::to_value([0_u8; 16]).unwrap()),519 let Op::Page { op: PageOp::Split { paragraph, right, lists, .. }, .. } = op else {
492 ("guid", serde_json::to_value(text.guid).unwrap()),520 unreachable!()
493 ("text", serde_json::to_value(ExGuid::default()).unwrap()),521 };
494 ("author", serde_json::json!("a\0b")),522 for (text, at, paragraph) in [
495 ("offset", serde_json::json!(u32::MAX)),523 (ExGuid::default(), 1, paragraph),
524 (text, u32::MAX, paragraph),
525 (text, 1, ExGuid::default()),
526 (text, 1, text),
496 ] {527 ] {
497 let mut encoded = serde_json::to_value(&intent).unwrap();528 let op = PageOp::Split { text, at, paragraph, right, lists: lists.clone() };
498 encoded[field] = value;529 assert!(ops::page_edited(SOURCE, sid, vec![op]).is_err());
499 let forged = serde_json::from_value(encoded).unwrap();
500 assert!(PreparedEdit::split(SOURCE, sid, &forged).is_err());
501 }530 }
502 let mut titles = 0;531 let mut titles = 0;
503 for (sid, _) in document.pages().unwrap() {532 for (sid, _) in document.pages().unwrap() {
...@@ -516,8 +545,7 @@ fn invalid_split_identities_and_title_targets_are_rejected() {...@@ -516,8 +545,7 @@ fn invalid_split_identities_and_title_targets_are_rejected() {
516 let node = &view.nodes[&id];545 let node = &view.nodes[&id];
517 pending.extend(node.children.iter().chain(&node.content).copied());546 pending.extend(node.children.iter().chain(&node.content).copied());
518 if matches!(node.kind, Kind::RichText { .. }) {547 if matches!(node.kind, Kind::RichText { .. }) {
519 let intent = ParagraphSplit::new(id, 0, "Author").unwrap();548 assert!(ops::edited(SOURCE, vec![split(SOURCE, sid, id, 0).0]).is_err());
520 assert!(PreparedEdit::split(SOURCE, sid, &intent).is_err());
521 titles += 1;549 titles += 1;
522 }550 }
523 }551 }
...@@ -551,16 +579,14 @@ fn export_native_paragraph_splits() {...@@ -551,16 +579,14 @@ fn export_native_paragraph_splits() {
551 })579 })
552 .unwrap()580 .unwrap()
553 .0;581 .0;
554 let intent = ParagraphSplit::new(582 let at = serde_json::from_value(case["offset_utf16"].clone()).unwrap();
555 text,583 let (op, new_paragraph, new_text) = split(&source, sid, text, at);
556 serde_json::from_value(case["offset_utf16"].clone()).unwrap(),584 let transaction = ops::transaction(&source, "Rust split author", vec![op.clone()])
557 "Rust split author",585 .unwrap()
558 )586 .unwrap();
559 .unwrap();587 written.push(serde_json::json!({"case": case["case"], "op": op,
560 let edited = PreparedEdit::split(&source, sid, &intent).unwrap();588 "new_paragraph": new_paragraph, "new_text": new_text}));
561 written.push(serde_json::json!({"case": case["case"], "intent": intent,589 transaction.apply(&mut source).unwrap();
562 "new_paragraph": intent.object(), "new_text": intent.text_object()}));
563 source = edited.as_bytes().to_vec();
564 }590 }
565 let candidate = output.join("candidate");591 let candidate = output.join("candidate");
566 fs::create_dir(&candidate).unwrap();592 fs::create_dir(&candidate).unwrap();
...@@ -587,18 +613,36 @@ fn interrupted_splits_publish_a_complete_graph_or_retain_the_original() {...@@ -587,18 +613,36 @@ fn interrupted_splits_publish_a_complete_graph_or_retain_the_original() {
587 .find(|id| matches!(view.nodes[id].kind, Kind::Outline { .. }))613 .find(|id| matches!(view.nodes[id].kind, Kind::Outline { .. }))
588 .unwrap();614 .unwrap();
589 let paragraph = view.nodes[outline].children[0];615 let paragraph = view.nodes[outline].children[0];
590 let insertion = onestore::Insertion::paragraph(paragraph, None, "a🦀b", "Author")616 let mut child = ops::paragraph("a🦀b");
591 .unwrap()617 child.level = 2;
592 .with_formatting(1..3, &[onestore::TextAttribute::Bold(true)])618 let text = child.text().unwrap().id;
593 .unwrap();619 let inserted = ops::page_edited(
594 let inserted = PreparedEdit::insert(&original, sid, &insertion).unwrap();620 &original,
595 let source = inserted.as_bytes();621 sid,
596 let checkpoint = checkpoint::pending(source, sid, insertion.text_object(), 0x14001d7a);622 vec![
623 PageOp::Insert {
624 container: paragraph,
625 before: None,
626 paragraphs: vec![child],
627 },
628 PageOp::Format {
629 text,
630 range: 1..3,
631 set: vec![onestore::TextAttribute::Bold(true)],
632 clear: Vec::new(),
633 },
634 ],
635 )
636 .unwrap();
637 let source = inserted.as_slice();
638 let checkpoint = checkpoint::pending(source, sid, text);
597 for source in [source, &checkpoint] {639 for source in [source, &checkpoint] {
598 let intent = ParagraphSplit::new(insertion.text_object(), 1, "Author").unwrap();640 let (op, ..) = split(source, sid, text, 1);
599 let edit = PreparedEdit::split(source, sid, &intent).unwrap();641 let edit = ops::transaction(source, "Author", vec![op]).unwrap().unwrap();
642 let mut written = source.to_vec();
643 edit.apply(&mut written).unwrap();
600 let before = current::current(source);644 let before = current::current(source);
601 let after = current::current(edit.as_bytes());645 let after = current::current(&written);
602 for write_limit in [17, 4096] {646 for write_limit in [17, 4096] {
603 let disk = |fail_at| disk::Disk {647 let disk = |fail_at| disk::Disk {
604 visible: source.to_vec(),648 visible: source.to_vec(),
...@@ -610,7 +654,7 @@ fn interrupted_splits_publish_a_complete_graph_or_retain_the_original() {...@@ -610,7 +654,7 @@ fn interrupted_splits_publish_a_complete_graph_or_retain_the_original() {
610 };654 };
611 let mut successful = disk(None);655 let mut successful = disk(None);
612 edit.commit(&mut successful).unwrap();656 edit.commit(&mut successful).unwrap();
613 assert_eq!(successful.durable, edit.as_bytes());657 assert_eq!(successful.durable, written);
614 for at in 1..=successful.operation {658 for at in 1..=successful.operation {
615 let mut interrupted = disk(Some(at));659 let mut interrupted = disk(Some(at));
616 let error = edit.commit(&mut interrupted).unwrap_err();660 let error = edit.commit(&mut interrupted).unwrap_err();
...@@ -654,10 +698,12 @@ fn export_native_paragraph_joins() {...@@ -654,10 +698,12 @@ fn export_native_paragraph_joins() {
654 })698 })
655 .unwrap()699 .unwrap()
656 .0;700 .0;
657 let intent = ParagraphJoin::new(left, right, "Rust join author").unwrap();701 let op = Op::Page { space: sid, op: PageOp::Join { left, right } };
658 let edit = PreparedEdit::join(&source, sid, &intent).unwrap();702 let transaction = ops::transaction(&source, "Rust join author", vec![op.clone()])
659 source = edit.as_bytes().to_vec();703 .unwrap()
660 manifest.push(serde_json::json!({"case":name,"space":sid,"intent":intent}));704 .unwrap();
705 transaction.apply(&mut source).unwrap();
706 manifest.push(serde_json::json!({"case":name,"space":sid,"op":op}));
661 }707 }
662 let folder = output.join(["split", "inheritance", "tags"][number]);708 let folder = output.join(["split", "inheritance", "tags"][number]);
663 fs::create_dir_all(folder.join("candidate")).unwrap();709 fs::create_dir_all(folder.join("candidate")).unwrap();
...@@ -685,22 +731,48 @@ fn interrupted_joins_preserve_complete_graphs_and_empty_text_adoption() {...@@ -685,22 +731,48 @@ fn interrupted_joins_preserve_complete_graphs_and_empty_text_adoption() {
685 .find(|id| matches!(view.nodes[id].kind, Kind::Outline { .. }))731 .find(|id| matches!(view.nodes[id].kind, Kind::Outline { .. }))
686 .unwrap();732 .unwrap();
687 let left = view.nodes[&view.nodes[&outline].children[0]].content[0];733 let left = view.nodes[&view.nodes[&outline].children[0]].content[0];
688 let right = onestore::Insertion::paragraph(outline, None, "Right 🦀", "Author")734 let right = ops::paragraph("Right 🦀");
689 .unwrap()735 let right_text = right.text().unwrap().id;
690 .with_formatting(0..5, &[onestore::TextAttribute::Bold(true)])736 let mut child = ops::paragraph("Retained child");
691 .unwrap();737 child.parent = Some(right.id);
692 let inserted = PreparedEdit::insert(&source, sid, &right).unwrap();738 child.level = 2;
693 let child =739 let original = ops::page_edited(
694 onestore::Insertion::paragraph(right.object(), None, "Retained child", "Author").unwrap();740 &source,
695 let original = PreparedEdit::insert(inserted.as_bytes(), sid, &child).unwrap();741 sid,
696 let original = original.as_bytes();742 vec![
697 let empty = onestore::replace_text(original, sid, left, 0..4, "").unwrap();743 PageOp::Insert {
698 let checkpoint = checkpoint::pending(original, sid, left, 0x14001d7a);744 container: outline,
745 before: None,
746 paragraphs: vec![right, child],
747 },
748 PageOp::Format {
749 text: right_text,
750 range: 0..5,
751 set: vec![onestore::TextAttribute::Bold(true)],
752 clear: Vec::new(),
753 },
754 ],
755 )
756 .unwrap();
757 let original = original.as_slice();
758 let empty = ops::page_edited(
759 original,
760 sid,
761 vec![PageOp::Text {
762 text: left,
763 range: 0..4,
764 with: String::new(),
765 }],
766 )
767 .unwrap();
768 let checkpoint = checkpoint::pending(original, sid, left);
699 for source in [original, &empty, &checkpoint] {769 for source in [original, &empty, &checkpoint] {
700 let intent = ParagraphJoin::new(left, right.text_object(), "Join author").unwrap();770 let op = Op::Page { space: sid, op: PageOp::Join { left, right: right_text } };
701 let edit = PreparedEdit::join(source, sid, &intent).unwrap();771 let edit = ops::transaction(source, "Join author", vec![op]).unwrap().unwrap();
772 let mut written = source.to_vec();
773 edit.apply(&mut written).unwrap();
702 let before = current::current(source);774 let before = current::current(source);
703 let after = current::current(edit.as_bytes());775 let after = current::current(&written);
704 for write_limit in [17, 4096] {776 for write_limit in [17, 4096] {
705 let disk = |fail_at| disk::Disk {777 let disk = |fail_at| disk::Disk {
706 visible: source.to_vec(),778 visible: source.to_vec(),
...@@ -712,8 +784,8 @@ fn interrupted_joins_preserve_complete_graphs_and_empty_text_adoption() {...@@ -712,8 +784,8 @@ fn interrupted_joins_preserve_complete_graphs_and_empty_text_adoption() {
712 };784 };
713 let mut successful = disk(None);785 let mut successful = disk(None);
714 edit.commit(&mut successful).unwrap();786 edit.commit(&mut successful).unwrap();
715 assert_eq!(successful.durable, edit.as_bytes());787 assert_eq!(successful.durable, written);
716 for at in std::iter::once(1).chain(source.len().div_ceil(193)..=successful.operation) {788 for at in 1..=successful.operation {
717 let mut interrupted = disk(Some(at));789 let mut interrupted = disk(Some(at));
718 let error = edit.commit(&mut interrupted).unwrap_err();790 let error = edit.commit(&mut interrupted).unwrap_err();
719 let recovered = current::current(&interrupted.durable);791 let recovered = current::current(&interrupted.durable);
...@@ -748,26 +820,16 @@ fn joins_reject_invalid_identities_wrong_order_and_unrelated_pages() {...@@ -748,26 +820,16 @@ fn joins_reject_invalid_identities_wrong_order_and_unrelated_pages() {
748 .contains_key(left)820 .contains_key(left)
749 })821 })
750 .unwrap();822 .unwrap();
751 assert!(ParagraphJoin::new(*left, *left, "Author").is_err());823 for (a, b) in [
752 assert!(ParagraphJoin::new(ExGuid::default(), *right, "Author").is_err());824 (*left, *left),
753 assert!(ParagraphJoin::new(*left, *right, "a\0b").is_err());825 (ExGuid::default(), *right),
754 let intent = ParagraphJoin::new(*left, *right, "Author").unwrap();826 (*right, *left),
755 for (field, value) in [827 (*left, cases[1].2),
756 ("left", serde_json::to_value(ExGuid::default()).unwrap()),
757 ("right", serde_json::to_value(left).unwrap()),
758 ("author", serde_json::json!("a\0b")),
759 ] {
760 let mut encoded = serde_json::to_value(&intent).unwrap();
761 encoded[field] = value;
762 let invalid = serde_json::from_value(encoded).unwrap();
763 assert!(PreparedEdit::join(source, *sid, &invalid).is_err());
764 }
765 for intent in [
766 ParagraphJoin::new(*right, *left, "Author").unwrap(),
767 ParagraphJoin::new(*left, cases[1].2, "Author").unwrap(),
768 ] {828 ] {
769 assert!(PreparedEdit::join(source, *sid, &intent).is_err());829 assert!(join(source, *sid, a, b).is_err());
770 }830 }
831 let op = Op::Page { space: *sid, op: PageOp::Join { left: *left, right: *right } };
832 assert!(ops::transaction(source, "a\0b", vec![op]).is_err());
771 let view = &space.revisions[&space.contexts[&ExGuid::default()]];833 let view = &space.revisions[&space.contexts[&ExGuid::default()]];
772 let paragraph = *view834 let paragraph = *view
773 .nodes835 .nodes
...@@ -781,8 +843,7 @@ fn joins_reject_invalid_identities_wrong_order_and_unrelated_pages() {...@@ -781,8 +843,7 @@ fn joins_reject_invalid_identities_wrong_order_and_unrelated_pages() {
781 .find(|node| node.children.contains(&paragraph))843 .find(|node| node.children.contains(&paragraph))
782 .unwrap();844 .unwrap();
783 let last = view.nodes[parent.children.last().unwrap()].content[0];845 let last = view.nodes[parent.children.last().unwrap()].content[0];
784 let nonadjacent = ParagraphJoin::new(*left, last, "Author").unwrap();846 assert!(join(source, *sid, *left, last).is_err());
785 assert!(PreparedEdit::join(source, *sid, &nonadjacent).is_err());
786 let title = view847 let title = view
787 .nodes848 .nodes
788 .iter()849 .iter()
...@@ -794,8 +855,7 @@ fn joins_reject_invalid_identities_wrong_order_and_unrelated_pages() {...@@ -794,8 +855,7 @@ fn joins_reject_invalid_identities_wrong_order_and_unrelated_pages() {
794 let node = &view.nodes[&id];855 let node = &view.nodes[&id];
795 pending.extend(node.children.iter().chain(&node.content).copied());856 pending.extend(node.children.iter().chain(&node.content).copied());
796 if matches!(node.kind, Kind::RichText { .. }) {857 if matches!(node.kind, Kind::RichText { .. }) {
797 let intent = ParagraphJoin::new(id, *right, "Author").unwrap();858 assert!(join(source, *sid, id, *right).is_err());
798 assert!(PreparedEdit::join(source, *sid, &intent).is_err());
799 rejected += 1;859 rejected += 1;
800 }860 }
801 }861 }
crates/onestore/tests/paragraph_formatting.rs+9-7
...@@ -1,5 +1,7 @@...@@ -1,5 +1,7 @@
1#[path = "support/ops.rs"]
2mod ops;
1use onestore::{3use onestore::{
2 ExGuid, PreparedEdit, RevisionIndex, Store,4 ExGuid, RevisionIndex, Store,
3 document::{Document, Kind},5 document::{Document, Kind},
4 page::{Page, PageObject, Paragraph},6 page::{Page, PageObject, Paragraph},
5};7};
...@@ -48,12 +50,12 @@ fn paragraph_properties_preserve_native_mixed_runs_and_shared_styles() {...@@ -48,12 +50,12 @@ fn paragraph_properties_preserve_native_mixed_runs_and_shared_styles() {
48 }50 }
49 }51 }
50 assert_eq!(count, 6);52 assert_eq!(count, 6);
51 let prepared = PreparedEdit::page(SOURCE, space, &after, "Format author").unwrap();53 let prepared = ops::saved(SOURCE, space, &after).unwrap();
52 assert_eq!(page(prepared.as_bytes()).1, after);54 assert_eq!(page(prepared.as_slice()).1, after);
53 let before_store = Store::parse(SOURCE).unwrap();55 let before_store = Store::parse(SOURCE).unwrap();
54 let before_index = RevisionIndex::parse(&before_store).unwrap();56 let before_index = RevisionIndex::parse(&before_store).unwrap();
55 let before_doc = Document::parse(&before_index).unwrap();57 let before_doc = Document::parse(&before_index).unwrap();
56 let after_store = Store::parse(prepared.as_bytes()).unwrap();58 let after_store = Store::parse(prepared.as_slice()).unwrap();
57 let after_index = RevisionIndex::parse(&after_store).unwrap();59 let after_index = RevisionIndex::parse(&after_store).unwrap();
58 let old = before_index.resolve_active(space).unwrap();60 let old = before_index.resolve_active(space).unwrap();
59 let new = after_index.resolve_active(space).unwrap();61 let new = after_index.resolve_active(space).unwrap();
...@@ -65,7 +67,7 @@ fn paragraph_properties_preserve_native_mixed_runs_and_shared_styles() {...@@ -65,7 +67,7 @@ fn paragraph_properties_preserve_native_mixed_runs_and_shared_styles() {
65 if let Some(directory) = std::env::var_os("ONESTORE_PARAGRAPH_FORMAT_EXPORT") {67 if let Some(directory) = std::env::var_os("ONESTORE_PARAGRAPH_FORMAT_EXPORT") {
66 let directory = std::path::PathBuf::from(directory);68 let directory = std::path::PathBuf::from(directory);
67 std::fs::create_dir(&directory).unwrap();69 std::fs::create_dir(&directory).unwrap();
68 std::fs::write(directory.join("synthetic.one"), prepared.as_bytes()).unwrap();70 std::fs::write(directory.join("synthetic.one"), prepared.as_slice()).unwrap();
69 std::fs::write(71 std::fs::write(
70 directory.join("Open Notebook.onetoc2"),72 directory.join("Open Notebook.onetoc2"),
71 onestore::create_table_of_contents(73 onestore::create_table_of_contents(
...@@ -103,7 +105,7 @@ fn paragraph_formatting_rejects_invalid_values_and_conflicting_run_settings() {...@@ -103,7 +105,7 @@ fn paragraph_formatting_rejects_invalid_values_and_conflicting_run_settings() {
103 let mut format = text.text.format_at(0).unwrap().clone();105 let mut format = text.text.format_at(0).unwrap().clone();
104 change(&mut format);106 change(&mut format);
105 text.text = Paragraph::new(text.text.text().to_owned(), format);107 text.text = Paragraph::new(text.text.text().to_owned(), format);
106 assert!(PreparedEdit::page(SOURCE, space, &after, "Author").is_err());108 assert!(ops::saved(SOURCE, space, &after).is_err());
107 }109 }
108 let mut after = before.clone();110 let mut after = before.clone();
109 let text = after111 let text = after
...@@ -121,5 +123,5 @@ fn paragraph_formatting_rejects_invalid_values_and_conflicting_run_settings() {...@@ -121,5 +123,5 @@ fn paragraph_formatting_rejects_invalid_values_and_conflicting_run_settings() {
121 left.alignment = Some(1);123 left.alignment = Some(1);
122 right.alignment = Some(2);124 right.alignment = Some(2);
123 text.text = Paragraph::from_runs([("Left".into(), left), ("Right".into(), right)]);125 text.text = Paragraph::from_runs([("Left".into(), left), ("Right".into(), right)]);
124 assert!(PreparedEdit::page(SOURCE, space, &after, "Author").is_err());126 assert!(ops::saved(SOURCE, space, &after).is_err());
125}127}
crates/onestore/tests/protected.rs+27-9
...@@ -3,6 +3,7 @@...@@ -3,6 +3,7 @@
3use onestore::{3use onestore::{
4 RevisionIndex, Store,4 RevisionIndex, Store,
5 document::{Document, Kind},5 document::{Document, Kind},
6 op::{Edit, Op, PageOp},
6 protected::{Error, Limits, UnlockedSection},7 protected::{Error, Limits, UnlockedSection},
7};8};
8use std::{fs, path::Path};9use std::{fs, path::Path};
...@@ -33,10 +34,19 @@ fn known_passwords_open_independent_native_fixtures() {...@@ -33,10 +34,19 @@ fn known_passwords_open_independent_native_fixtures() {
33 .iter()34 .iter()
34 .find(|(_, node)| matches!(node.kind, Kind::RichText { .. }))35 .find(|(_, node)| matches!(node.kind, Kind::RichText { .. }))
35 .unwrap();36 .unwrap();
36 assert!(37 // The ordinary section writer refuses a protected section.
37 onestore::PreparedEdit::text(&bytes, space, *object, 0..0, "Must remain protected")38 let arena = onestore::Arena::default();
38 .is_err()39 assert!(onestore::Section::open(&arena, bytes.clone()).is_err());
39 );40 let op = PageOp::Text {
41 text: *object,
42 range: 0..0,
43 with: "Must remain protected".into(),
44 };
45 let edit = Edit {
46 at: 134_000_000_000_000_000,
47 ops: vec![Op::Page { space, op }],
48 };
49 assert!(unlocked.apply("Rust", &edit).is_ok());
40 assert!(50 assert!(
41 document51 document
42 .spaces52 .spaces
...@@ -139,6 +149,7 @@ fn a_protected_page_edit_is_stored_under_the_section_key() {...@@ -139,6 +149,7 @@ fn a_protected_page_edit_is_stored_under_the_section_key() {
139 let mut expected = pages(&bytes);149 let mut expected = pages(&bytes);
140 let mut edited = 0;150 let mut edited = 0;
141 for (space, page) in &mut expected {151 for (space, page) in &mut expected {
152 let before = page.clone();
142 let paragraphs = page.objects.iter_mut().find_map(|object| match object {153 let paragraphs = page.objects.iter_mut().find_map(|object| match object {
143 PageObject::Outline(outline)154 PageObject::Outline(outline)
144 if outline.paragraphs.iter().any(|p| p.text().is_some()) =>155 if outline.paragraphs.iter().any(|p| p.text().is_some()) =>
...@@ -174,14 +185,21 @@ fn a_protected_page_edit_is_stored_under_the_section_key() {...@@ -174,14 +185,21 @@ fn a_protected_page_edit_is_stored_under_the_section_key() {
174 text.text185 text.text
175 .append(Paragraph::new(" still protected".to_owned(), format))186 .append(Paragraph::new(" still protected".to_owned(), format))
176 .unwrap();187 .unwrap();
177 let edit =188 let ops = onestore::op::lower_page(&before, page).unwrap();
178 onestore::PreparedEdit::page_protected(&bytes, password, *space, page, "Rust")189 let edit = Edit {
179 .unwrap();190 at: 134_000_000_000_000_000,
191 ops: ops.into_iter().map(|op| Op::Page { space: *space, op }).collect(),
192 };
193 let store = Store::parse(&bytes).unwrap();
194 let index = RevisionIndex::parse(&store).unwrap();
180 assert!(matches!(195 assert!(matches!(
181 onestore::PreparedEdit::page_protected(&bytes, "wrong", *space, page, "Rust"),196 UnlockedSection::open(&index, "wrong", Limits::default()),
182 Err(Error::PasswordMismatch)197 Err(Error::PasswordMismatch)
183 ));198 ));
184 let written = edit.as_bytes().to_vec();199 let unlocked = UnlockedSection::open(&index, password, Limits::default()).unwrap();
200 let transaction = unlocked.apply("Rust", &edit).unwrap();
201 let mut written = bytes.clone();
202 transaction.apply(&mut written).unwrap();
185 for clear in [&b" still protected"[..], b"stays sealed"] {203 for clear in [&b" still protected"[..], b"stays sealed"] {
186 assert!(!written.windows(clear.len()).any(|w| w == clear));204 assert!(!written.windows(clear.len()).any(|w| w == clear));
187 }205 }
crates/onestore/tests/shared_snapshot.rs+27-5
...@@ -4,6 +4,8 @@ use current::current;...@@ -4,6 +4,8 @@ use current::current;
44
5#[path = "support/checkpoint.rs"]5#[path = "support/checkpoint.rs"]
6mod checkpoint;6mod checkpoint;
7#[path = "support/ops.rs"]
8mod ops;
7#[path = "support/trace.rs"]9#[path = "support/trace.rs"]
8mod trace;10mod trace;
911
...@@ -15,6 +17,26 @@ use onestore::{...@@ -15,6 +17,26 @@ use onestore::{
15use std::{fs, io};17use std::{fs, io};
16use trace::{Event, Trace};18use trace::{Event, Trace};
1719
20/// Types `with` over `range` of the text object in `source` and commits it to `io`.
21fn commit_text(
22 io: &mut impl onestore::CommitIo,
23 source: &[u8],
24 space: ExGuid,
25 text: ExGuid,
26 range: std::ops::Range<u32>,
27 with: &str,
28) -> Result<(), onestore::CommitError> {
29 let op = onestore::op::PageOp::Text {
30 text,
31 range,
32 with: with.into(),
33 };
34 ops::transaction(source, "Author", vec![onestore::op::Op::Page { space, op }])
35 .unwrap()
36 .unwrap()
37 .commit(io)
38}
39
18#[test]40#[test]
19fn storage_inspection_preserves_opaque_images_without_claiming_edit_readiness() {41fn storage_inspection_preserves_opaque_images_without_claiming_edit_readiness() {
20 for path in [42 for path in [
...@@ -111,7 +133,7 @@ fn version_cached_readers_cannot_miss_a_completed_publication() {...@@ -111,7 +133,7 @@ fn version_cached_readers_cannot_miss_a_completed_publication() {
111 bytes: source.clone(),133 bytes: source.clone(),
112 events: Vec::new(),134 events: Vec::new(),
113 };135 };
114 onestore::commit_text(&mut trace, &source, sid, oid, end..end, " [cached reader]").unwrap();136 commit_text(&mut trace, &source, sid, oid, end..end, " [cached reader]").unwrap();
115 let final_content = current(&trace.bytes);137 let final_content = current(&trace.bytes);
116 let mut visible = source;138 let mut visible = source;
117 for event in &trace.events {139 for event in &trace.events {
...@@ -168,7 +190,7 @@ fn published_snapshots_survive_interleaved_commit_io() {...@@ -168,7 +190,7 @@ fn published_snapshots_survive_interleaved_commit_io() {
168 let mut source = fs::read(format!("../../corpus/{path}")).unwrap();190 let mut source = fs::read(format!("../../corpus/{path}")).unwrap();
169 let (sid, oid, _) = target(&source);191 let (sid, oid, _) = target(&source);
170 if name == "checkpoint" {192 if name == "checkpoint" {
171 source = checkpoint::pending(&source, sid, oid, 0x14001d7a);193 source = checkpoint::pending(&source, sid, oid);
172 }194 }
173 let (_, _, end) = target(&source);195 let (_, _, end) = target(&source);
174 let before = current(&source);196 let before = current(&source);
...@@ -176,7 +198,7 @@ fn published_snapshots_survive_interleaved_commit_io() {...@@ -176,7 +198,7 @@ fn published_snapshots_survive_interleaved_commit_io() {
176 bytes: source.clone(),198 bytes: source.clone(),
177 events: Vec::new(),199 events: Vec::new(),
178 };200 };
179 onestore::commit_text(&mut trace, &source, sid, oid, end..end, " [reader café 🦀]")201 commit_text(&mut trace, &source, sid, oid, end..end, " [reader café 🦀]")
180 .unwrap();202 .unwrap();
181 let after = current(&trace.bytes);203 let after = current(&trace.bytes);
182 assert_ne!(before, after);204 assert_ne!(before, after);
...@@ -407,7 +429,7 @@ fn unpublished_append_remains_available_for_retry() {...@@ -407,7 +429,7 @@ fn unpublished_append_remains_available_for_retry() {
407 bytes: source.clone(),429 bytes: source.clone(),
408 events: Vec::new(),430 events: Vec::new(),
409 };431 };
410 onestore::commit_text(&mut trace, &source, sid, oid, end..end, " abandoned").unwrap();432 commit_text(&mut trace, &source, sid, oid, end..end, " abandoned").unwrap();
411 let Event::Write(offset, append) = &trace.events[0] else {433 let Event::Write(offset, append) = &trace.events[0] else {
412 panic!()434 panic!()
413 };435 };
...@@ -432,7 +454,7 @@ fn unpublished_append_remains_available_for_retry() {...@@ -432,7 +454,7 @@ fn unpublished_append_remains_available_for_retry() {
432 bytes: persisted,454 bytes: persisted,
433 events: Vec::new(),455 events: Vec::new(),
434 };456 };
435 onestore::commit_text(&mut retry, &snapshot, sid, oid, end..end, " retry").unwrap();457 commit_text(&mut retry, &snapshot, sid, oid, end..end, " retry").unwrap();
436 assert_ne!(current(&retry.bytes), current(&source));458 assert_ne!(current(&retry.bytes), current(&source));
437 }459 }
438}460}
crates/onestore/tests/storage.rs+1-1
...@@ -23,7 +23,7 @@ fn native_file_data_alignment_is_relative_to_the_object() {...@@ -23,7 +23,7 @@ fn native_file_data_alignment_is_relative_to_the_object() {
23 assert_eq!(payload.len(), 10056);23 assert_eq!(payload.len(), 10056);
24 assert_eq!(24 assert_eq!(
25 format!("{:x}", md5::compute(payload)),25 format!("{:x}", md5::compute(payload)),
26 "e1b57a8851177dd25dc05b50b904656a"26 "fec8f53bd6db9305eabac4cd4b71fb08"
27 );27 );
28 let blob = store.chunk_data(chunk).unwrap();28 let blob = store.chunk_data(chunk).unwrap();
29 for alignment in 0..8 {29 for alignment in 0..8 {
crates/onestore/tests/support/checkpoint.rs+34-17
...@@ -1,21 +1,38 @@...@@ -1,21 +1,38 @@
1use onestore::{ExGuid, RevisionIndex, Store};1use onestore::{
2 Arena, ExGuid, RevisionIndex, Section, Store,
3 op::{Edit, Op, PageOp},
4};
25
3pub fn pending(source: &[u8], sid: ExGuid, oid: ExGuid, property: u32) -> Vec<u8> {6/// `source` with revisions of `space` appended until its chain is 512 deep, so its next
4 let mut source = source.to_vec();7/// revision is a checkpoint. Each retouches the text object `text`, typing a character and
5 for value in 1_000_000_u32.. {8/// removing it again.
6 let store = Store::parse(&source).unwrap();9pub fn pending(source: &[u8], space: ExGuid, text: ExGuid) -> Vec<u8> {
10 let depth = |image: &[u8]| {
11 let store = Store::parse(image).unwrap();
7 let index = RevisionIndex::parse(&store).unwrap();12 let index = RevisionIndex::parse(&store).unwrap();
8 let rid = index.spaces[&sid].labels[&(ExGuid::default(), 1)];13 let rid = index.spaces[&space].labels[&(ExGuid::default(), 1)];
9 let depth =14 std::iter::successors(Some(rid), |id| {
10 std::iter::successors(Some(rid), |id| index.spaces[&sid].revisions[id].dependency)15 index.spaces[&space].revisions[id].dependency
11 .count();16 })
12 assert!(depth <= 512);17 .count()
13 if depth == 512 {18 };
14 return source;19 let arena = Arena::default();
15 }20 let mut section = Section::open(&arena, source.to_vec()).unwrap();
16 source =21 let retouch = |range, with: &str| Op::Page {
17 onestore::replace_property_bytes(&source, sid, oid, property, &value.to_le_bytes())22 space,
18 .unwrap();23 op: PageOp::Text {
24 text,
25 range,
26 with: with.into(),
27 },
28 };
29 for at in 0..512 - depth(source) as u64 {
30 let ops = vec![retouch(0..0, "x"), retouch(0..1, "")];
31 let at = 134_000_000_000_000_000 + at * 10_000_000;
32 section.apply("Author", &Edit { at, ops }).unwrap();
33 section.seal().unwrap().unwrap();
19 }34 }
20 unreachable!()35 let image = section.image();
36 assert_eq!(depth(&image), 512);
37 image
21}38}
crates/onestore/tests/support/ops.rs created+187
...@@ -0,0 +1,187 @@
1//! Edits as ops through a `Section`, sealed: the image an edit leaves, for tests that read
2//! it back.
3#![allow(dead_code)]
4
5use onestore::{
6 Arena, ExGuid, Section, Transaction,
7 op::{Edit, Op, OpError, PageOp},
8 page::Page,
9};
10
11/// FILETIME now, as an edit's `at`.
12pub fn now() -> u64 {
13 let unix = std::time::SystemTime::now()
14 .duration_since(std::time::UNIX_EPOCH)
15 .unwrap();
16 (unix.as_secs() + 11_644_473_600) * 10_000_000 + u64::from(unix.subsec_nanos() / 100)
17}
18
19/// The transaction `ops` seal into on `source`, as one edit by `author`; none when they
20/// store nothing.
21pub fn transaction(
22 source: &[u8],
23 author: &str,
24 ops: Vec<Op>,
25) -> Result<Option<Transaction>, OpError> {
26 let arena = Arena::default();
27 let mut section = Section::open(&arena, source.to_vec()).map_err(OpError::Failed)?;
28 section.apply(author, &Edit { at: now(), ops })?;
29 section.seal().map_err(OpError::Failed)
30}
31
32/// `source` after `ops`, applied as one edit and sealed.
33pub fn edited(source: &[u8], ops: Vec<Op>) -> Result<Vec<u8>, OpError> {
34 let mut image = source.to_vec();
35 if let Some(transaction) = transaction(source, "Author", ops)? {
36 transaction.apply(&mut image).map_err(OpError::Failed)?;
37 }
38 Ok(image)
39}
40
41/// `source` after `ops` on the page in `space`.
42pub fn page_edited(source: &[u8], space: ExGuid, ops: Vec<PageOp>) -> Result<Vec<u8>, OpError> {
43 edited(
44 source,
45 ops.into_iter().map(|op| Op::Page { space, op }).collect(),
46 )
47}
48
49/// `source` with the page in `space` taken to `after` by the ops `op::lower_page` finds,
50/// as a test states an edit by the page it leaves.
51pub fn saved(source: &[u8], space: ExGuid, after: &Page) -> Result<Vec<u8>, String> {
52 let arena = Arena::default();
53 let section = Section::open(&arena, source.to_vec()).map_err(|error| error.to_string())?;
54 let before = section.page(space).map_err(|error| error.to_string())?;
55 let ops = onestore::op::lower_page(&before, after).map_err(|error| error.to_string())?;
56 page_edited(source, space, ops).map_err(|error| error.to_string())
57}
58
59/// A plain paragraph holding `text` in OneNote's default text style, with fresh identities.
60pub fn paragraph(text: &str) -> onestore::page::PageParagraph {
61 use onestore::page::{PageParagraph, Paragraph, ParagraphContent, TextObject, text::new_id};
62 let format = onestore::document::Format {
63 font: Some("Calibri".into()),
64 font_size: Some(11.0),
65 language: Some(0x409),
66 ..Default::default()
67 };
68 PageParagraph {
69 id: new_id().unwrap(),
70 parent: None,
71 level: 1,
72 style: None,
73 format: Default::default(),
74 content: ParagraphContent::Text(TextObject {
75 id: new_id().unwrap(),
76 date_field: None,
77 text: Paragraph::new(text.into(), format),
78 tags: Vec::new(),
79 }),
80 lists: Vec::new(),
81 tags: Vec::new(),
82 media: Default::default(),
83 collapsed: false,
84 }
85}
86
87/// A new outline at `(x, y)` in points holding `paragraphs`.
88pub fn outline(
89 x: f32,
90 y: f32,
91 paragraphs: Vec<onestore::page::PageParagraph>,
92) -> onestore::page::PageObject {
93 use onestore::page::{Outline, PageObject, text::new_id};
94 PageObject::Outline(Outline {
95 id: new_id().unwrap(),
96 title: false,
97 min_width: None,
98 layout: onestore::document::Layout {
99 x: Some(x),
100 y: Some(y),
101 ..Default::default()
102 },
103 indents: Vec::new(),
104 paragraphs,
105 unsupported: Vec::new(),
106 })
107}
108
109/// An edit applied through a `Section`: the image it leaves and the transaction publishing
110/// it, none when it stores nothing.
111pub struct Applied {
112 pub image: Vec<u8>,
113 pub transaction: Option<Transaction>,
114}
115
116impl Applied {
117 pub fn as_bytes(&self) -> &[u8] {
118 &self.image
119 }
120
121 /// Publishes the transaction under the filesystem adapter's exclusion.
122 #[cfg(any(unix, windows))]
123 pub fn commit_file(
124 &self,
125 path: impl AsRef<std::path::Path>,
126 ) -> Result<(), onestore::CommitError> {
127 self.transaction
128 .as_ref()
129 .expect("the edit stores a revision")
130 .commit_file(path)
131 }
132
133 /// Publishes the transaction under caller-held exclusion.
134 pub fn commit(&self, io: &mut impl onestore::CommitIo) -> Result<(), onestore::CommitError> {
135 self.transaction
136 .as_ref()
137 .expect("the edit stores a revision")
138 .commit(io)
139 }
140}
141
142/// `ops` applied to `source` as one edit by `author`.
143pub fn apply(source: &[u8], author: &str, ops: Vec<Op>) -> Result<Applied, OpError> {
144 let transaction = transaction(source, author, ops)?;
145 let mut image = source.to_vec();
146 if let Some(transaction) = &transaction {
147 transaction.apply(&mut image).map_err(OpError::Failed)?;
148 }
149 Ok(Applied { image, transaction })
150}
151
152/// `saved` as an `Applied`, to publish it.
153pub fn save(source: &[u8], space: ExGuid, after: &Page) -> Result<Applied, String> {
154 let arena = Arena::default();
155 let section = Section::open(&arena, source.to_vec()).map_err(|error| error.to_string())?;
156 let before = section.page(space).map_err(|error| error.to_string())?;
157 let ops = onestore::op::lower_page(&before, after).map_err(|error| error.to_string())?;
158 let ops = ops.into_iter().map(|op| Op::Page { space, op }).collect();
159 apply(source, "Author", ops).map_err(|error| error.to_string())
160}
161
162/// `op` on the page in `space`, applied to `source`.
163pub fn page_op(source: &[u8], space: ExGuid, op: PageOp) -> Result<Applied, OpError> {
164 apply(source, "Author", vec![Op::Page { space, op }])
165}
166
167/// `op` on the section, applied to `source`.
168pub fn section_op(source: &[u8], op: onestore::op::SectionOp) -> Result<Applied, OpError> {
169 apply(source, "Author", vec![Op::Section(op)])
170}
171
172/// Adds an outline at `(x, y)` holding one paragraph of `text`: the op and the identities
173/// of the outline and its text.
174pub fn new_outline(x: f32, y: f32, text: &str) -> (PageOp, ExGuid, ExGuid) {
175 let paragraph = paragraph(text);
176 let text = paragraph.text().unwrap().id;
177 let object = outline(x, y, vec![paragraph]);
178 let id = object.id();
179 (
180 PageOp::Add {
181 object,
182 before: None,
183 },
184 id,
185 text,
186 )
187}
crates/onestore/tests/support/page_edits.rs+29-7
...@@ -1,8 +1,11 @@...@@ -1,8 +1,11 @@
1//! Drives random page-model mutations through the page writer and checks its contract:1//! Drives random page-model mutations through `op::lower_page` and a `Section` and checks
2//! a published model reads back as written, and objects outside the edit keep their bytes.2//! the contract: the page the section holds is what the model oracle predicts from the ops,
3//! the sealed image reads back as the edited model, and objects outside the edit keep their
4//! bytes.
35
4use onestore::{6use onestore::{
5 ExGuid, PreparedEdit, RevisionIndex, Store,7 Arena, ExGuid, RevisionIndex, Section, Store,
8 op::{self, Op},
6 document::{Document, Format, Layout},9 document::{Document, Format, Layout},
7 page::{10 page::{
8 Outline, Page, PageObject, PageParagraph, Paragraph, ParagraphContent,11 Outline, Page, PageObject, PageParagraph, Paragraph, ParagraphContent,
...@@ -564,13 +567,31 @@ pub fn run(input: &[u8]) {...@@ -564,13 +567,31 @@ pub fn run(input: &[u8]) {
564 let Some(mut after) = model(source, space) else {567 let Some(mut after) = model(source, space) else {
565 return;568 return;
566 };569 };
570 let before = after.clone();
567 mutate(&mut after, &mut bytes);571 mutate(&mut after, &mut bytes);
568 let Ok(edit) = PreparedEdit::page(source, space, &after, "Fuzz author") else {572 let Ok(ops) = onestore::op::lower_page(&before, &after) else {
569 return;573 return;
570 };574 };
571 let written = edit.as_bytes();575 let arena = Arena::default();
572 if written == source.as_slice() {576 let mut section = Section::open(&arena, source.clone()).unwrap();
573 let before = model(source, space).unwrap();577 let edit = op::Edit {
578 at: 134_000_000_000_000_000,
579 ops: ops.iter().map(|op| Op::Page { space, op: op.clone() }).collect(),
580 };
581 if section.apply("Fuzz author", &edit).is_err() {
582 assert_eq!(section.page(space).unwrap(), before, "a refused edit changes nothing");
583 return;
584 }
585 let mut predicted = before.clone();
586 for op in &ops {
587 onestore::op::predict(&mut predicted, op).unwrap();
588 }
589 assert_eq!(
590 projection(&section.page(space).unwrap()),
591 projection(&predicted),
592 "the model oracle"
593 );
594 if section.seal().unwrap().is_none() {
574 assert_eq!(595 assert_eq!(
575 projection(&before),596 projection(&before),
576 projection(&after),597 projection(&after),
...@@ -578,6 +599,7 @@ pub fn run(input: &[u8]) {...@@ -578,6 +599,7 @@ pub fn run(input: &[u8]) {
578 );599 );
579 return;600 return;
580 }601 }
602 let written = &section.image();
581 current::current(written);603 current::current(written);
582 let stored = model(written, space).expect("the written page reads back");604 let stored = model(written, space).expect("the written page reads back");
583 let (stored, expected) = (projection(&stored), projection(&after));605 let (stored, expected) = (projection(&stored), projection(&after));
crates/onestore/tests/support/page_schedule.rs+19-19
...@@ -1,6 +1,7 @@...@@ -1,6 +1,7 @@
1use onestore::{1use onestore::{
2 CommitState, ExGuid, Insertion, PageCreation, PageEdit, PreparedEdit, RevisionIndex, Store,2 CommitState, ExGuid, PageCreation, PageEdit, RevisionIndex, Store,
3 document::{Document, FieldValue, Kind},3 document::{Document, FieldValue, Kind},
4 op::{PageOp, SectionOp},
4};5};
5use std::{collections::BTreeMap, sync::LazyLock};6use std::{collections::BTreeMap, sync::LazyLock};
67
...@@ -8,6 +9,8 @@ use std::{collections::BTreeMap, sync::LazyLock};...@@ -8,6 +9,8 @@ use std::{collections::BTreeMap, sync::LazyLock};
8pub(crate) mod current;9pub(crate) mod current;
9#[path = "disk.rs"]10#[path = "disk.rs"]
10pub(crate) mod disk;11pub(crate) mod disk;
12#[path = "ops.rs"]
13pub(crate) mod ops;
1114
12static SOURCE: LazyLock<Vec<u8>> = LazyLock::new(|| {15static SOURCE: LazyLock<Vec<u8>> = LazyLock::new(|| {
13 onestore::create_section("pages.one", "Original 🦀 é 東京", "Author").unwrap()16 onestore::create_section("pages.one", "Original 🦀 é 東京", "Author").unwrap()
...@@ -24,17 +27,17 @@ pub fn run(input: &[u8]) {...@@ -24,17 +27,17 @@ pub fn run(input: &[u8]) {
24 _ => SOURCE.as_slice(),27 _ => SOURCE.as_slice(),
25 };28 };
26 if let Ok(intent) = serde_json::from_slice::<PageCreation>(input)29 if let Ok(intent) = serde_json::from_slice::<PageCreation>(input)
27 && let Ok(prepared) = PreparedEdit::create_page(source, &intent)30 && let Ok(prepared) = ops::section_op(source, SectionOp::Create(intent.clone()))
28 {31 {
29 current::current(prepared.as_bytes());32 current::current(prepared.as_bytes());
30 }33 }
31 if let Ok(edits) = serde_json::from_slice::<Vec<PageEdit>>(input)34 if let Ok(edits) = serde_json::from_slice::<Vec<PageEdit>>(input)
32 && let Ok(prepared) = PreparedEdit::pages(source, &edits)35 && let Ok(prepared) = ops::section_op(source, SectionOp::Pages(edits.to_vec()))
33 {36 {
34 current::current(prepared.as_bytes());37 current::current(prepared.as_bytes());
35 }38 }
36 if let Ok(pages) = serde_json::from_slice::<Vec<ExGuid>>(input)39 if let Ok(pages) = serde_json::from_slice::<Vec<ExGuid>>(input)
37 && let Ok(prepared) = PreparedEdit::delete_pages_permanently(source, &pages)40 && let Ok(prepared) = ops::section_op(source, SectionOp::Delete(pages.to_vec()))
38 {41 {
39 current::current(prepared.as_bytes());42 current::current(prepared.as_bytes());
40 }43 }
...@@ -65,7 +68,7 @@ pub fn run(input: &[u8]) {...@@ -65,7 +68,7 @@ pub fn run(input: &[u8]) {
65 .get(usize::from(step[2]) % pages.len().max(1))68 .get(usize::from(step[2]) % pages.len().max(1))
66 .copied();69 .copied();
67 let text = ["", "Same title", "é 🦋 東京", " spaces "][usize::from(step[3]) % 4];70 let text = ["", "Same title", "é 🦋 東京", " spaces "][usize::from(step[3]) % 4];
68 let existing = if let Some((sid, page)) = selected {71 let existing = if let Some((sid, _)) = selected {
69 let space = &document.spaces[&sid];72 let space = &document.spaces[&sid];
70 let view = &space.revisions[&space.contexts[&ExGuid::default()]];73 let view = &space.revisions[&space.contexts[&ExGuid::default()]];
71 let titles: Vec<_> = view74 let titles: Vec<_> = view
...@@ -91,7 +94,7 @@ pub fn run(input: &[u8]) {...@@ -91,7 +94,7 @@ pub fn run(input: &[u8]) {
91 let removed: Vec<_> = (0..count)94 let removed: Vec<_> = (0..count)
92 .map(|i| pages[(usize::from(step[2]) + i) % pages.len()].0)95 .map(|i| pages[(usize::from(step[2]) + i) % pages.len()].0)
93 .collect();96 .collect();
94 let prepared = PreparedEdit::delete_pages_permanently(source, &removed).unwrap();97 let prepared = ops::section_op(source, SectionOp::Delete(removed.to_vec())).unwrap();
95 pages.retain(|page| !removed.contains(&page.0));98 pages.retain(|page| !removed.contains(&page.0));
96 for sid in removed {99 for sid in removed {
97 levels.remove(&sid);100 levels.remove(&sid);
...@@ -128,7 +131,7 @@ pub fn run(input: &[u8]) {...@@ -128,7 +131,7 @@ pub fn run(input: &[u8]) {
128 let restored: Vec<PageEdit> =131 let restored: Vec<PageEdit> =
129 serde_json::from_value(serde_json::to_value(&edits).unwrap()).unwrap();132 serde_json::from_value(serde_json::to_value(&edits).unwrap()).unwrap();
130 assert_eq!(edits, restored);133 assert_eq!(edits, restored);
131 let prepared = PreparedEdit::pages(source, &restored);134 let prepared = ops::section_op(source, SectionOp::Pages(restored.to_vec()));
132 if levels[&pages[0].0] != 1 {135 if levels[&pages[0].0] != 1 {
133 assert!(prepared.is_err());136 assert!(prepared.is_err());
134 continue;137 continue;
...@@ -140,21 +143,15 @@ pub fn run(input: &[u8]) {...@@ -140,21 +143,15 @@ pub fn run(input: &[u8]) {
140 unreachable!()143 unreachable!()
141 };144 };
142 Some((145 Some((
143 PreparedEdit::text(146 ops::page_op(source, sid, PageOp::Text { text: object, range: 0..u32::try_from(original.encode_utf16().count()).unwrap(), with: text.into() })
144 source,
145 sid,
146 object,
147 0..u32::try_from(original.encode_utf16().count()).unwrap(),
148 text,
149 )
150 .unwrap(),147 .unwrap(),
151 Some((sid, object, text)),148 Some((sid, object, text)),
152 ))149 ))
153 } else if step[1] & 16 != 0 {150 } else if step[1] & 16 != 0 {
154 let intent = Insertion::outline(page, 36.0, 36.0, text, "Page fuzz").unwrap();151 let (add, _, inserted) = ops::new_outline(36.0, 36.0, text);
155 Some((152 Some((
156 PreparedEdit::insert(source, sid, &intent).unwrap(),153 ops::page_op(source, sid, add).unwrap(),
157 Some((sid, intent.text_object(), text)),154 Some((sid, inserted, text)),
158 ))155 ))
159 } else {156 } else {
160 None157 None
...@@ -182,7 +179,7 @@ pub fn run(input: &[u8]) {...@@ -182,7 +179,7 @@ pub fn run(input: &[u8]) {
182 let intent = PageCreation::new(before, title, "Page fuzz").unwrap();179 let intent = PageCreation::new(before, title, "Page fuzz").unwrap();
183 let restored = serde_json::from_value(serde_json::to_value(&intent).unwrap()).unwrap();180 let restored = serde_json::from_value(serde_json::to_value(&intent).unwrap()).unwrap();
184 assert_eq!(intent, restored);181 assert_eq!(intent, restored);
185 let edit = PreparedEdit::create_page(source, &restored).unwrap();182 let edit = ops::section_op(source, SectionOp::Create(restored.clone())).unwrap();
186 let position = before.map_or(pages.len(), |sid| {183 let position = before.map_or(pages.len(), |sid| {
187 pages.iter().position(|p| p.0 == sid).unwrap()184 pages.iter().position(|p| p.0 == sid).unwrap()
188 });185 });
...@@ -257,7 +254,10 @@ pub fn run(input: &[u8]) {...@@ -257,7 +254,10 @@ pub fn run(input: &[u8]) {
257 write_limit: if step[6] & 1 == 0 { 17 } else { 4096 },254 write_limit: if step[6] & 1 == 0 { 17 } else { 4096 },
258 random: u64::from(step[7]) + 1,255 random: u64::from(step[7]) + 1,
259 };256 };
260 let result = edit.commit(&mut disk);257 let Some(transaction) = &edit.transaction else {
258 continue;
259 };
260 let result = transaction.commit(&mut disk);
261 let observed = current::current(&disk.durable);261 let observed = current::current(&disk.durable);
262 match result {262 match result {
263 Ok(()) => assert_eq!(observed, after),263 Ok(()) => assert_eq!(observed, after),
crates/onestore/tests/support/tree_model.rs+69-208
...@@ -1,16 +1,18 @@...@@ -1,16 +1,18 @@
1use onestore::{1use onestore::{
2 CommitState, ExGuid, Insertion, PreparedEdit, RevisionIndex, Store, TreeEdit,2 CommitState, ExGuid, RevisionIndex, Store,
3 document::{Document, Kind, Revision},3 document::{Document, Kind, Revision},
4 op::{Op, PageOp},
5 page::Page,
4};6};
5use std::{7use std::{
6 collections::{BTreeMap, BTreeSet},8 collections::BTreeMap,
7 sync::LazyLock,9 sync::LazyLock,
8};10};
911
10use crate::{current, disk};12use crate::{current, disk, ops};
1113
12static SOURCE: LazyLock<Vec<u8>> = LazyLock::new(|| {14static SOURCE: LazyLock<Vec<u8>> = LazyLock::new(|| {
13 let mut source = onestore::create_section("tree.one", "Original", "Author").unwrap();15 let source = onestore::create_section("tree.one", "Original", "Author").unwrap();
14 let store = Store::parse(&source).unwrap();16 let store = Store::parse(&source).unwrap();
15 let index = RevisionIndex::parse(&store).unwrap();17 let index = RevisionIndex::parse(&store).unwrap();
16 let document = Document::parse(&index).unwrap();18 let document = Document::parse(&index).unwrap();
...@@ -22,26 +24,24 @@ static SOURCE: LazyLock<Vec<u8>> = LazyLock::new(|| {...@@ -22,26 +24,24 @@ static SOURCE: LazyLock<Vec<u8>> = LazyLock::new(|| {
22 .iter()24 .iter()
23 .find(|id| matches!(view.nodes[id].kind, Kind::Outline { .. }))25 .find(|id| matches!(view.nodes[id].kind, Kind::Outline { .. }))
24 .unwrap();26 .unwrap();
25 let other = Insertion::outline(page, 360.0, 72.0, "Other", "Author").unwrap();27 let (add, other, _) = ops::new_outline(360.0, 72.0, "Other");
26 source = PreparedEdit::insert(&source, sid, &other)28 let mut edits = vec![add];
27 .unwrap()29 for container in [outline, other] {
28 .as_bytes()30 let mut paragraphs = Vec::new();
29 .to_vec();31 for text in ["First 🦀 é", "Second 東京", "Third"] {
30 for parent in [outline, other.object()] {32 let paragraph = ops::paragraph(text);
31 for text in ["First 🦀 é", "Second 東京", "Third"] {33 let mut child = ops::paragraph("Child");
32 let insertion = Insertion::paragraph(parent, None, text, "Author").unwrap();34 child.parent = Some(paragraph.id);
33 source = PreparedEdit::insert(&source, sid, &insertion)35 child.level = 2;
34 .unwrap()36 paragraphs.extend([paragraph, child]);
35 .as_bytes()
36 .to_vec();
37 let child = Insertion::paragraph(insertion.object(), None, "Child", "Author").unwrap();
38 source = PreparedEdit::insert(&source, sid, &child)
39 .unwrap()
40 .as_bytes()
41 .to_vec();
42 }37 }
38 edits.push(PageOp::Insert {
39 container,
40 before: None,
41 paragraphs,
42 });
43 }43 }
44 source44 ops::page_edited(&source, sid, edits).unwrap()
45});45});
4646
47fn forest(view: &Revision<'_>, root: ExGuid) -> BTreeMap<ExGuid, (ExGuid, u32, usize)> {47fn forest(view: &Revision<'_>, root: ExGuid) -> BTreeMap<ExGuid, (ExGuid, u32, usize)> {
...@@ -69,6 +69,9 @@ fn forest(view: &Revision<'_>, root: ExGuid) -> BTreeMap<ExGuid, (ExGuid, u32, u...@@ -69,6 +69,9 @@ fn forest(view: &Revision<'_>, root: ExGuid) -> BTreeMap<ExGuid, (ExGuid, u32, u
69 positions69 positions
70}70}
7171
72/// Random subtree moves and deletions by twelve clients, each on its own possibly stale
73/// image, committed under injected storage faults: an edit stores the page the model
74/// predicts, and the file always holds a complete old or new graph.
72pub fn run(input: &[u8]) {75pub fn run(input: &[u8]) {
73 let Some((&selector, input)) = input.split_first() else {76 let Some((&selector, input)) = input.split_first() else {
74 return;77 return;
...@@ -110,194 +113,74 @@ pub fn run(input: &[u8]) {...@@ -110,194 +113,74 @@ pub fn run(input: &[u8]) {
110 }113 }
111 let target = targets[usize::from(step[3]) % targets.len()];114 let target = targets[usize::from(step[3]) % targets.len()];
112 let subtree = forest(view, target);115 let subtree = forest(view, target);
113 let parents: BTreeMap<_, _> = before116 let outline = matches!(view.nodes[&target].kind, Kind::Outline { .. });
114 .keys()
115 .flat_map(|id| view.nodes[id].children.iter().map(|child| (*child, *id)))
116 .collect();
117 let parent = parents[&target];
118 let deleting = step[1] & 6 == 0;117 let deleting = step[1] & 6 == 0;
119 let mut expected: BTreeMap<_, _> = before118 let (op, cycle) = if deleting {
120 .iter()119 (PageOp::Delete { object: target }, false)
121 .filter_map(|(id, pos)| {
122 matches!(view.nodes[id].kind, Kind::Paragraph { .. })
123 .then_some((*id, (pos.0, pos.1)))
124 })
125 .collect();
126 let mut ordered: Vec<_> = expected.keys().copied().collect();
127 ordered.sort_by_key(|id| before[id].2);
128 let mut expected_order = BTreeMap::<_, Vec<_>>::new();
129 for id in ordered {
130 expected_order.entry(before[&id].0).or_default().push(id);
131 }
132 let intent;
133 let mut reversal = None;
134 let edit = if deleting {
135 expected.retain(|id, _| !subtree.contains_key(id));
136 for ids in expected_order.values_mut() {
137 ids.retain(|id| !subtree.contains_key(id));
138 }
139 intent = TreeEdit::delete(target, "Tree schedule").unwrap();
140 PreparedEdit::tree(source, sid, &intent)
141 } else {120 } else {
142 let mut destinations: Vec<_> = before121 let mut destinations: Vec<_> = before
143 .keys()122 .keys()
144 .copied()123 .copied()
145 .filter(|id| {124 .filter(|id| {
146 if matches!(view.nodes[&target].kind, Kind::Outline { .. }) {125 if outline {
147 *id == page126 *id == page
148 } else {127 } else {
149 matches!(128 matches!(
150 view.nodes[id].kind,129 view.nodes[id].kind,
151 Kind::Outline { .. }130 Kind::Outline { .. } | Kind::Paragraph { .. } | Kind::Cell { .. }
152 | Kind::OutlineGroup
153 | Kind::Paragraph { .. }
154 | Kind::Cell { .. }
155 )131 )
156 }132 }
157 })133 })
158 .collect();134 .collect();
159 destinations.sort_by_key(|id| before[id].2);135 destinations.sort_by_key(|id| before[id].2);
160 if destinations.is_empty() {
161 continue;
162 }
163 let destination = destinations[usize::from(step[4]) % destinations.len()];136 let destination = destinations[usize::from(step[4]) % destinations.len()];
164 let children = &view.nodes[&destination].children;137 // Outline groups are the writers' business: anchors are the paragraphs in them.
165 let anchor = children138 let mut children = Vec::new();
166 .get(usize::from(step[5]) % (children.len() + 1))139 let mut pending: Vec<ExGuid> = view.nodes[&destination].children.iter().rev().copied().collect();
167 .copied();140 while let Some(id) = pending.pop() {
168 intent = TreeEdit::move_to(target, destination, anchor, "Tree schedule").unwrap();141 match view.nodes[&id].kind {
169 let edit = PreparedEdit::tree(source, sid, &intent);142 Kind::OutlineGroup => pending.extend(view.nodes[&id].children.iter().rev()),
170 if subtree.contains_key(&destination) {143 _ if id == target => {}
171 assert!(edit.is_err());144 _ => children.push(id),
172 continue;
173 }
174 if destination == parent && step[9] & 3 == 0 {
175 let original = &view.nodes[&parent].children;
176 let next = original.iter().position(|id| *id == target).unwrap() + 1;
177 reversal = Some(
178 TreeEdit::move_to(target, parent, original.get(next).copied(), "Tree schedule")
179 .unwrap(),
180 );
181 }
182 if matches!(view.nodes[&target].kind, Kind::Paragraph { .. }) {
183 let (scope, base, _) = before[&destination];
184 let level = base + u32::from(view.nodes[&destination].child_level.unwrap_or(1));
185 let (old_scope, old_level, _) = before[&target];
186 if anchor != Some(target) {
187 let moved: Vec<_> = expected_order[&old_scope]
188 .iter()
189 .copied()
190 .filter(|id| subtree.contains_key(id))
191 .collect();
192 expected_order
193 .get_mut(&old_scope)
194 .unwrap()
195 .retain(|id| !subtree.contains_key(id));
196 let boundary = anchor.map(|id| before[&id].2).unwrap_or_else(|| {
197 forest(view, destination)
198 .keys()
199 .map(|id| before[id].2)
200 .max()
201 .unwrap()
202 + 1
203 });
204 let sequence = expected_order.entry(scope).or_default();
205 let position = sequence
206 .iter()
207 .position(|id| before[id].2 >= boundary)
208 .unwrap_or(sequence.len());
209 sequence.splice(position..position, moved);
210 }
211 for (id, pos) in &mut expected {
212 if subtree.contains_key(id) && pos.0 == old_scope {
213 *pos = (scope, level + pos.1 - old_level);
214 }
215 }145 }
216 }146 }
217 edit147 let before = children
148 .get(usize::from(step[5]) % (children.len() + 1))
149 .copied();
150 let op = PageOp::Move {
151 object: target,
152 parent: (!outline).then_some(destination),
153 before,
154 };
155 (op, subtree.contains_key(&destination))
218 };156 };
219 let restored = serde_json::from_value(serde_json::to_value(&intent).unwrap()).unwrap();157 let model = Page::from_space(&document, sid).unwrap();
220 assert_eq!(intent, restored);158 let ops = vec![Op::Page { space: sid, op: op.clone() }];
221 let edit = match edit {159 let Ok(edit) = ops::apply(source, "Tree schedule", ops) else {
222 Ok(edit) => edit,160 continue;
223 Err(error)
224 if error.message
225 == "Removing this group would exceed 31 child indentation levels" =>
226 {
227 continue;
228 }
229 Err(error) => panic!("{intent:?}: {error:?}"),
230 };161 };
162 assert!(!cycle, "{op:?}");
231 let after_store = Store::parse(edit.as_bytes()).unwrap();163 let after_store = Store::parse(edit.as_bytes()).unwrap();
232 let after_index = RevisionIndex::parse(&after_store).unwrap();164 let after_index = RevisionIndex::parse(&after_store).unwrap();
233 after_index.validate_current().unwrap();165 after_index.validate_current().unwrap();
234 let after_document = Document::parse(&after_index).unwrap();166 let after_document = Document::parse(&after_index).unwrap();
167 let mut predicted = model.clone();
168 onestore::op::predict(&mut predicted, &op).unwrap();
169 let stored = Page::from_space(&after_document, sid).unwrap();
170 if stored != predicted {
171 let (a, b) = (format!("{stored:?}"), format!("{predicted:?}"));
172 let at = a.bytes().zip(b.bytes()).take_while(|(x, y)| x == y).count();
173 panic!("{op:?}\n stored …{}\n predicted …{}", &a[a.floor_char_boundary(at.saturating_sub(300))..a.floor_char_boundary((at + 200).min(a.len()))], &b[b.floor_char_boundary(at.saturating_sub(300))..b.floor_char_boundary((at + 200).min(b.len()))]);
174 }
235 let space = &after_document.spaces[&sid];175 let space = &after_document.spaces[&sid];
236 let after = &space.revisions[&space.contexts[&ExGuid::default()]];176 let after = &space.revisions[&space.contexts[&ExGuid::default()]];
237 let actual = forest(after, page);177 for (id, _) in forest(after, page) {
238 let mut added = BTreeSet::new();178 let node = &after.nodes[&id];
239 for (id, pos) in &actual {179 if before.contains_key(&id) {
240 let node = &after.nodes[id];180 assert_eq!(node.content, view.nodes[&id].content);
241 if let Kind::Paragraph { .. } = node.kind {
242 if let Some(wanted) = expected.remove(id) {
243 assert_eq!((pos.0, pos.1), wanted, "{intent:?}: {id}");
244 } else {
245 assert!(!before.contains_key(id));
246 assert!(matches!(view.nodes[&parent].kind, Kind::Cell { .. }));
247 assert_eq!(view.nodes[&parent].children, [target]);
248 assert_eq!(node.children.len(), 0);
249 assert_eq!(node.content.len(), 1);
250 assert!(node.tags.is_empty());
251 assert!(
252 matches!(&after.nodes[&node.content[0]].kind,Kind::RichText{text,..} if text.is_empty())
253 );
254 added.insert(*id);
255 }
256 }
257 if before.contains_key(id) {
258 assert_eq!(node.content, view.nodes[id].content);
259 let mut expected_kind = serde_json::to_value(&view.nodes[id].kind).unwrap();
260 if *id == page && expected_kind["alternate_title"].is_string() {
261 let mut roots = after.nodes[&page].children.clone();
262 roots.sort_by(|a, b| {
263 let a = &after.nodes[a].layout;
264 let b = &after.nodes[b].layout;
265 a.y.unwrap_or(0.0)
266 .total_cmp(&b.y.unwrap_or(0.0))
267 .then_with(|| a.x.unwrap_or(0.0).total_cmp(&b.x.unwrap_or(0.0)))
268 });
269 let mut title = String::new();
270 for root in roots {
271 let mut nodes: Vec<_> = forest(after, root).into_iter().collect();
272 nodes.sort_by_key(|(_, pos)| pos.2);
273 for (id, _) in nodes {
274 if let Kind::RichText { text, .. } = &after.nodes[&id].kind {
275 title = text
276 .trim_start()
277 .split('\r')
278 .next()
279 .unwrap()
280 .trim_end()
281 .to_owned();
282 if !title.is_empty() {
283 break;
284 }
285 }
286 }
287 if !title.is_empty() {
288 break;
289 }
290 }
291 expected_kind["alternate_title"] = title.into();
292 let Kind::Metadata { title, .. } = &after.nodes[&after.roots[&2]].kind else {
293 panic!()
294 };
295 assert_eq!(title.as_deref(), expected_kind["alternate_title"].as_str());
296 }
297 assert_eq!(serde_json::to_value(&node.kind).unwrap(), expected_kind);
298 assert_eq!(181 assert_eq!(
299 serde_json::to_value(&node.tags).unwrap(),182 serde_json::to_value(&node.tags).unwrap(),
300 serde_json::to_value(&view.nodes[id].tags).unwrap()183 serde_json::to_value(&view.nodes[&id].tags).unwrap()
301 );184 );
302 }185 }
303 if matches!(186 if matches!(
...@@ -311,31 +194,6 @@ pub fn run(input: &[u8]) {...@@ -311,31 +194,6 @@ pub fn run(input: &[u8]) {
311 );194 );
312 }195 }
313 }196 }
314 assert!(expected.is_empty());
315 assert!(added.len() <= 1);
316 let mut ordered: Vec<_> = actual
317 .iter()
318 .filter(|(id, _)| {
319 matches!(after.nodes[id].kind, Kind::Paragraph { .. }) && !added.contains(id)
320 })
321 .collect();
322 ordered.sort_by_key(|(_, pos)| pos.2);
323 let mut actual_order = BTreeMap::<_, Vec<_>>::new();
324 for (id, pos) in ordered {
325 actual_order.entry(pos.0).or_default().push(*id);
326 }
327 expected_order.retain(|_, ids| !ids.is_empty());
328 assert_eq!(actual_order, expected_order, "{intent:?}");
329 if let Some(reversal) = reversal {
330 let reversed = PreparedEdit::tree(edit.as_bytes(), sid, &reversal).unwrap();
331 let replayed = PreparedEdit::tree(reversed.as_bytes(), sid, &intent).unwrap();
332 let store = Store::parse(replayed.as_bytes()).unwrap();
333 let index = RevisionIndex::parse(&store).unwrap();
334 let document = Document::parse(&index).unwrap();
335 let space = &document.spaces[&sid];
336 let replayed = &space.revisions[&space.contexts[&ExGuid::default()]];
337 assert_eq!(forest(replayed, page), actual);
338 }
339 let before = current::current(&persisted);197 let before = current::current(&persisted);
340 let after = current::current(edit.as_bytes());198 let after = current::current(edit.as_bytes());
341 let mut disk = disk::Disk {199 let mut disk = disk::Disk {
...@@ -346,7 +204,10 @@ pub fn run(input: &[u8]) {...@@ -346,7 +204,10 @@ pub fn run(input: &[u8]) {
346 write_limit: if step[8] & 1 == 0 { 17 } else { 4096 },204 write_limit: if step[8] & 1 == 0 { 17 } else { 4096 },
347 random: u64::from(step[9]) + 1,205 random: u64::from(step[9]) + 1,
348 };206 };
349 let result = edit.commit(&mut disk);207 let Some(transaction) = &edit.transaction else {
208 continue;
209 };
210 let result = transaction.commit(&mut disk);
350 let actual = current::current(&disk.durable);211 let actual = current::current(&disk.durable);
351 match result {212 match result {
352 Ok(()) => assert_eq!(actual, after),213 Ok(()) => assert_eq!(actual, after),
crates/onestore/tests/table_growth.rs+8-5
...@@ -1,8 +1,11 @@...@@ -1,8 +1,11 @@
1//! A rewritten object stores the global id table entries it names, not the table of the1//! A rewritten object stores the global id table entries it names, not the table of the
2//! revision it was read from.2//! revision it was read from.
33
4#[path = "support/ops.rs"]
5mod ops;
6
4use onestore::{7use onestore::{
5 PreparedEdit, RevisionIndex, Store,8 RevisionIndex, Store,
6 document::Document,9 document::Document,
7 page::{Page, PageObject, Paragraph},10 page::{Page, PageObject, Paragraph},
8};11};
...@@ -64,17 +67,17 @@ fn a_text_edit_stores_the_table_entries_it_names() {...@@ -64,17 +67,17 @@ fn a_text_edit_stores_the_table_entries_it_names() {
64 text.text67 text.text
65 .append(Paragraph::new(" and a few more words".to_owned(), format))68 .append(Paragraph::new(" and a few more words".to_owned(), format))
66 .unwrap();69 .unwrap();
67 let written = PreparedEdit::page(&bytes, space, &page, "Rust")70 let written = ops::saved(&bytes, space, &page)
68 .unwrap()71 .unwrap()
69 .as_bytes()72 .as_slice()
70 .to_vec();73 .to_vec();
71 for (entries, span) in tables(&written, bytes.len()) {74 for (entries, span) in tables(&written, bytes.len()) {
72 sparse |= (entries as u32) < span;75 sparse |= (entries as u32) < span;
73 }76 }
74 assert_eq!(77 assert_eq!(
75 PreparedEdit::page(&written, space, &page, "Rust")78 ops::saved(&written, space, &page)
76 .unwrap()79 .unwrap()
77 .as_bytes(),80 .as_slice(),
78 written81 written
79 );82 );
80 bytes = written;83 bytes = written;
crates/onestore/tests/toc.rs+34-18
...@@ -1,4 +1,25 @@...@@ -1,4 +1,25 @@
1use onestore::{PreparedEdit, RevisionIndex, Store, TocEdit, document::Document};1use onestore::{RevisionIndex, Store, TocEdit, document::Document};
2
3/// `source` with `edits` applied.
4fn edited(source: &[u8], edits: &[TocEdit]) -> Result<Vec<u8>, onestore::Error> {
5 let mut image = source.to_vec();
6 if let Some(transaction) = onestore::edit_table_of_contents(source, edits)? {
7 transaction.apply(&mut image)?;
8 }
9 Ok(image)
10}
11
12/// The notebook's colour.
13fn color(bytes: &[u8]) -> Option<u32> {
14 let store = Store::parse(bytes).unwrap();
15 let index = RevisionIndex::parse(&store).unwrap();
16 let document = Document::parse(&index).unwrap();
17 let revision = document.active(document.root).unwrap();
18 match &revision.nodes[&revision.roots[&1]].kind {
19 onestore::document::Kind::Toc { color, .. } => *color,
20 _ => panic!(),
21 }
22}
223
3fn entries(bytes: &[u8]) -> Vec<(String, [u8; 16], u32, Option<u32>)> {24fn entries(bytes: &[u8]) -> Vec<(String, [u8; 16], u32, Option<u32>)> {
4 let store = Store::parse(bytes).unwrap();25 let store = Store::parse(bytes).unwrap();
...@@ -31,7 +52,7 @@ fn entries(bytes: &[u8]) -> Vec<(String, [u8; 16], u32, Option<u32>)> {...@@ -31,7 +52,7 @@ fn entries(bytes: &[u8]) -> Vec<(String, [u8; 16], u32, Option<u32>)> {
31}52}
3253
33#[test]54#[test]
34fn sections_and_groups_are_added_renamed_coloured_ordered_and_removed() {55fn sections_and_groups_are_added_renamed_ordered_and_removed_and_the_notebook_coloured() {
35 let a = [1; 16];56 let a = [1; 16];
36 let b = [2; 16];57 let b = [2; 16];
37 let toc = onestore::create_table_of_contents(58 let toc = onestore::create_table_of_contents(
...@@ -41,7 +62,7 @@ fn sections_and_groups_are_added_renamed_coloured_ordered_and_removed() {...@@ -41,7 +62,7 @@ fn sections_and_groups_are_added_renamed_coloured_ordered_and_removed() {
41 .unwrap();62 .unwrap();
42 let c = [3; 16];63 let c = [3; 16];
43 let group = [4; 16];64 let group = [4; 16];
44 let written = PreparedEdit::table_of_contents(65 let written = edited(
45 &toc,66 &toc,
46 &[67 &[
47 TocEdit::Add {68 TocEdit::Add {
...@@ -58,7 +79,7 @@ fn sections_and_groups_are_added_renamed_coloured_ordered_and_removed() {...@@ -58,7 +79,7 @@ fn sections_and_groups_are_added_renamed_coloured_ordered_and_removed() {
58 )79 )
59 .unwrap();80 .unwrap();
60 assert_eq!(81 assert_eq!(
61 entries(written.as_bytes()),82 entries(&written),
62 [83 [
63 ("Alpha.one".to_owned(), a, 1, Some(0xffff_ffff)),84 ("Alpha.one".to_owned(), a, 1, Some(0xffff_ffff)),
64 ("Beta.one".to_owned(), b, 2, Some(0xffff_ffff)),85 ("Beta.one".to_owned(), b, 2, Some(0xffff_ffff)),
...@@ -66,33 +87,31 @@ fn sections_and_groups_are_added_renamed_coloured_ordered_and_removed() {...@@ -66,33 +87,31 @@ fn sections_and_groups_are_added_renamed_coloured_ordered_and_removed() {
66 ("Archive".to_owned(), group, 4, None),87 ("Archive".to_owned(), group, 4, None),
67 ]88 ]
68 );89 );
69 let again = PreparedEdit::table_of_contents(90 let again = edited(
70 written.as_bytes(),91 &written,
71 &[92 &[
72 TocEdit::Rename {93 TocEdit::Rename {
73 identity: b,94 identity: b,
74 filename: "Renamed.one".into(),95 filename: "Renamed.one".into(),
75 },96 },
76 TocEdit::Color {97 TocEdit::Color(0x00d7ff),
77 identity: c,
78 color: Some(0x00d7ff),
79 },
80 TocEdit::Order(vec![group, c]),98 TocEdit::Order(vec![group, c]),
81 TocEdit::Remove { identity: a },99 TocEdit::Remove { identity: a },
82 ],100 ],
83 )101 )
84 .unwrap();102 .unwrap();
85 assert_eq!(103 assert_eq!(
86 entries(again.as_bytes()),104 entries(&again),
87 [105 [
88 ("Archive".to_owned(), group, 1, None),106 ("Archive".to_owned(), group, 1, None),
89 ("Gamma.one".to_owned(), c, 2, Some(0x00d7ff)),107 ("Gamma.one".to_owned(), c, 2, Some(0xffff_ffff)),
90 ("Renamed.one".to_owned(), b, 3, Some(0xffff_ffff)),108 ("Renamed.one".to_owned(), b, 3, Some(0xffff_ffff)),
91 ]109 ]
92 );110 );
111 assert_eq!(color(&again), Some(0x00d7ff));
93 assert!(112 assert!(
94 PreparedEdit::table_of_contents(113 edited(
95 again.as_bytes(),114 &again,
96 &[TocEdit::Add {115 &[TocEdit::Add {
97 filename: "renamed.one".into(),116 filename: "renamed.one".into(),
98 identity: [9; 16],117 identity: [9; 16],
...@@ -101,8 +120,5 @@ fn sections_and_groups_are_added_renamed_coloured_ordered_and_removed() {...@@ -101,8 +120,5 @@ fn sections_and_groups_are_added_renamed_coloured_ordered_and_removed() {
101 )120 )
102 .is_err()121 .is_err()
103 );122 );
104 assert!(123 assert!(edited(&again, &[TocEdit::Remove { identity: a }]).is_err());
105 PreparedEdit::table_of_contents(again.as_bytes(), &[TocEdit::Remove { identity: a }])
106 .is_err()
107 );
108}124}
crates/onestore/tests/transaction.rs+27-9
...@@ -1,6 +1,10 @@...@@ -1,6 +1,10 @@
1#[path = "support/ops.rs"]
2mod ops;
3
1use onestore::{4use onestore::{
2 CommitIo, CommitState, ExGuid, PreparedEdit, RevisionIndex, Stamp, Store,5 CommitIo, CommitState, ExGuid, RevisionIndex, Stamp, Store, Transaction,
3 document::{Document, Kind},6 document::{Document, Kind},
7 op::{Op, PageOp},
4};8};
5use std::io;9use std::io;
610
...@@ -23,6 +27,21 @@ fn text(source: &[u8]) -> (ExGuid, ExGuid) {...@@ -23,6 +27,21 @@ fn text(source: &[u8]) -> (ExGuid, ExGuid) {
23 panic!("Missing text fixture")27 panic!("Missing text fixture")
24}28}
2529
30/// The transaction typing `with` at the start of the text object, and the image it leaves.
31fn typed(space: ExGuid, text: ExGuid, with: &str) -> (Transaction, Vec<u8>) {
32 let op = PageOp::Text {
33 text,
34 range: 0..0,
35 with: with.into(),
36 };
37 let transaction = ops::transaction(SOURCE, "Author", vec![Op::Page { space, op }])
38 .unwrap()
39 .unwrap();
40 let mut image = SOURCE.to_vec();
41 transaction.apply(&mut image).unwrap();
42 (transaction, image)
43}
44
26/// A file that records how many bytes a commit reads and writes.45/// A file that records how many bytes a commit reads and writes.
27struct Counted {46struct Counted {
28 bytes: Vec<u8>,47 bytes: Vec<u8>,
...@@ -54,20 +73,19 @@ impl CommitIo for Counted {...@@ -54,20 +73,19 @@ impl CommitIo for Counted {
54#[test]73#[test]
55fn a_transaction_reads_the_header_and_writes_only_its_edit() {74fn a_transaction_reads_the_header_and_writes_only_its_edit() {
56 let (sid, oid) = text(SOURCE);75 let (sid, oid) = text(SOURCE);
57 let edit = PreparedEdit::text(SOURCE, sid, oid, 0..0, "é").unwrap();76 let (transaction, written) = typed(sid, oid, "é");
58 let transaction = edit.transaction();
59 let mut image = SOURCE.to_vec();77 let mut image = SOURCE.to_vec();
60 transaction.apply(&mut image).unwrap();78 transaction.apply(&mut image).unwrap();
61 assert_eq!(image, edit.as_bytes());79 assert_eq!(image, written);
62 let mut file = Counted {80 let mut file = Counted {
63 bytes: SOURCE.to_vec(),81 bytes: SOURCE.to_vec(),
64 read: 0,82 read: 0,
65 written: 0,83 written: 0,
66 };84 };
67 transaction.commit(&mut file).unwrap();85 transaction.commit(&mut file).unwrap();
68 assert_eq!(file.bytes, edit.as_bytes());86 assert_eq!(file.bytes, written);
69 assert!(file.read <= 1024 + 2, "read {} bytes", file.read);87 assert!(file.read <= 1024 + 2, "read {} bytes", file.read);
70 let appended = edit.as_bytes().len() - SOURCE.len();88 let appended = written.len() - SOURCE.len();
71 assert!(89 assert!(
72 file.written < appended + 2048,90 file.written < appended + 2048,
73 "wrote {} bytes for {appended} appended",91 "wrote {} bytes for {appended} appended",
...@@ -86,9 +104,9 @@ fn a_transaction_reads_the_header_and_writes_only_its_edit() {...@@ -86,9 +104,9 @@ fn a_transaction_reads_the_header_and_writes_only_its_edit() {
86#[test]104#[test]
87fn stamps_name_the_committed_image() {105fn stamps_name_the_committed_image() {
88 let (sid, oid) = text(SOURCE);106 let (sid, oid) = text(SOURCE);
89 let edit = PreparedEdit::text(SOURCE, sid, oid, 0..0, "x").unwrap();107 let (transaction, written) = typed(sid, oid, "x");
90 let before = Stamp::of(SOURCE).unwrap();108 let before = Stamp::of(SOURCE).unwrap();
91 let after = Stamp::of(edit.as_bytes()).unwrap();109 let after = Stamp::of(&written).unwrap();
92 assert_ne!(before, after);110 assert_ne!(before, after);
93 assert_eq!(before.length, SOURCE.len() as u64);111 assert_eq!(before.length, SOURCE.len() as u64);
94 // An unpublished tail changes the length, so a commit built without it is refused.112 // An unpublished tail changes the length, so a commit built without it is refused.
...@@ -100,7 +118,7 @@ fn stamps_name_the_committed_image() {...@@ -100,7 +118,7 @@ fn stamps_name_the_committed_image() {
100 read: 0,118 read: 0,
101 written: 0,119 written: 0,
102 };120 };
103 let error = edit.commit(&mut file).unwrap_err();121 let error = transaction.commit(&mut file).unwrap_err();
104 assert_eq!(error.error.kind(), io::ErrorKind::ResourceBusy);122 assert_eq!(error.error.kind(), io::ErrorKind::ResourceBusy);
105 assert_eq!(file.written, 0);123 assert_eq!(file.written, 0);
106 assert!(Stamp::of(&SOURCE[..1023]).is_err());124 assert!(Stamp::of(&SOURCE[..1023]).is_err());
crates/onestore/tests/tree.rs+121-55
...@@ -1,6 +1,7 @@...@@ -1,6 +1,7 @@
1use onestore::{1use onestore::{
2 ExGuid, PreparedEdit, RevisionIndex, Store, TreeEdit,2 ExGuid, RevisionIndex, Store,
3 document::{Document, Kind, Revision},3 document::{Document, Kind, Revision},
4 op::{Op, PageOp},
4};5};
5use std::collections::{BTreeMap, BTreeSet};6use std::collections::{BTreeMap, BTreeSet};
67
...@@ -8,6 +9,8 @@ use std::collections::{BTreeMap, BTreeSet};...@@ -8,6 +9,8 @@ use std::collections::{BTreeMap, BTreeSet};
8mod current;9mod current;
9#[path = "support/disk.rs"]10#[path = "support/disk.rs"]
10mod disk;11mod disk;
12#[path = "support/ops.rs"]
13mod ops;
11#[path = "support/tree_model.rs"]14#[path = "support/tree_model.rs"]
12mod tree_model;15mod tree_model;
1316
...@@ -55,7 +58,35 @@ fn active(view: &Revision<'_>, page: ExGuid) -> BTreeSet<ExGuid> {...@@ -55,7 +58,35 @@ fn active(view: &Revision<'_>, page: ExGuid) -> BTreeSet<ExGuid> {
55 ids58 ids
56}59}
5760
58fn operation(view: &Revision<'_>, page: ExGuid) -> Option<(String, ExGuid, TreeEdit)> {61/// The container `id` lies in, outline groups passed over.
62fn container(view: &Revision<'_>, ids: &BTreeSet<ExGuid>, id: ExGuid) -> ExGuid {
63 let mut at = *ids
64 .iter()
65 .find(|parent| view.nodes[parent].children.contains(&id))
66 .unwrap();
67 while matches!(view.nodes[&at].kind, Kind::OutlineGroup) {
68 at = container(view, ids, at);
69 }
70 at
71}
72
73/// The outline level of each paragraph in `outline`.
74fn levels(view: &Revision<'_>, outline: ExGuid) -> BTreeMap<ExGuid, u32> {
75 let mut levels = BTreeMap::new();
76 let mut pending = vec![(outline, 0)];
77 while let Some((id, level)) = pending.pop() {
78 let node = &view.nodes[&id];
79 if matches!(node.kind, Kind::Paragraph { .. }) {
80 levels.insert(id, level);
81 }
82 let child = level + u32::from(node.child_level.unwrap_or(1));
83 pending.extend(node.children.iter().map(|id| (*id, child)));
84 }
85 levels
86}
87
88/// The named edit of a fixture page as ops, with the object it acts on.
89fn operation(view: &Revision<'_>, page: ExGuid) -> Option<(String, ExGuid, Vec<PageOp>)> {
59 let Kind::Metadata {90 let Kind::Metadata {
60 title: Some(name), ..91 title: Some(name), ..
61 } = &view.nodes[&view.roots[&2]].kind92 } = &view.nodes[&view.roots[&2]].kind
...@@ -85,32 +116,46 @@ fn operation(view: &Revision<'_>, page: ExGuid) -> Option<(String, ExGuid, TreeE...@@ -85,32 +116,46 @@ fn operation(view: &Revision<'_>, page: ExGuid) -> Option<(String, ExGuid, TreeE
85 .iter()116 .iter()
86 .find(|id| matches!(view.nodes[id].kind, Kind::Outline { .. }))117 .find(|id| matches!(view.nodes[id].kind, Kind::Outline { .. }))
87 .unwrap();118 .unwrap();
88 let parent = *ids119 let parent = container(view, &ids, target);
89 .iter()120 let move_to = |parent, before| PageOp::Move {
90 .find(|id| view.nodes[id].children.contains(&target))121 object: target,
91 .unwrap();122 parent: Some(parent),
123 before,
124 };
92 let edit = match name.as_str() {125 let edit = match name.as_str() {
93 "Delete outline" => TreeEdit::delete(outline, "Tree author"),126 "Delete outline" => vec![PageOp::Delete { object: outline }],
94 name if name.starts_with("Delete") => TreeEdit::delete(target, "Tree author"),127 "Delete sole cell paragraph" => vec![
128 PageOp::Insert {
129 container: parent,
130 before: None,
131 paragraphs: vec![ops::paragraph("")],
132 },
133 PageOp::Delete { object: target },
134 ],
135 name if name.starts_with("Delete") => vec![PageOp::Delete { object: target }],
95 "Move leaf down"136 "Move leaf down"
96 | "Move subtree down"137 | "Move subtree down"
97 | "Move numbered subtree down"138 | "Move numbered subtree down"
98 | "Move cell subtree down" => TreeEdit::move_to(target, parent, None, "Tree author"),139 | "Move cell subtree down" => vec![move_to(parent, None)],
99 "Move subtree up" => {140 "Move subtree up" => vec![move_to(parent, Some(paragraphs["Anchor"]))],
100 TreeEdit::move_to(target, parent, Some(paragraphs["Anchor"]), "Tree author")141 "Indent subtree" | "Indent bullet subtree" => vec![move_to(paragraphs["Anchor"], None)],
101 }142 "Outdent subtree" | "Outdent first group" => {
102 "Indent subtree" | "Indent bullet subtree" => {143 // Levels are absolute: the subtree moves up to the outline's first level.
103 TreeEdit::move_to(target, paragraphs["Anchor"], None, "Tree author")144 let levels = levels(view, outline);
145 let shift = levels[&target] - 1;
146 let mut ops = vec![move_to(outline, Some(paragraphs["Trailing sibling"]))];
147 let mut pending = vec![target];
148 while let Some(paragraph) = pending.pop() {
149 ops.push(PageOp::Level {
150 paragraph,
151 level: levels[&paragraph] - shift,
152 });
153 pending.extend(view.nodes[&paragraph].children.iter().rev());
154 }
155 ops
104 }156 }
105 "Outdent subtree" | "Outdent first group" => TreeEdit::move_to(
106 target,
107 outline,
108 Some(paragraphs["Trailing sibling"]),
109 "Tree author",
110 ),
111 _ => return None,157 _ => return None,
112 }158 };
113 .unwrap();
114 Some((159 Some((
115 name.clone(),160 name.clone(),
116 if name == "Delete outline" {161 if name == "Delete outline" {
...@@ -139,9 +184,9 @@ fn native_subtree_controls_match_with_preserved_fields_and_history() {...@@ -139,9 +184,9 @@ fn native_subtree_controls_match_with_preserved_fields_and_history() {
139 let Some((name, object, edit)) = operation(before, page) else {184 let Some((name, object, edit)) = operation(before, page) else {
140 continue;185 continue;
141 };186 };
142 let restored = serde_json::from_value(serde_json::to_value(&edit).unwrap()).unwrap();187 let edit: Vec<Op> = edit.into_iter().map(|op| Op::Page { space: sid, op }).collect();
143 assert_eq!(edit, restored);188 let prepared = ops::apply(source, "Tree author", edit.clone())
144 let prepared = PreparedEdit::tree(source, sid, &restored).unwrap();189 .unwrap_or_else(|error| panic!("{name}: {error}"));
145 let after_store = Store::parse(prepared.as_bytes()).unwrap();190 let after_store = Store::parse(prepared.as_bytes()).unwrap();
146 assert!(after_store.checksum_mismatches.is_empty());191 assert!(after_store.checksum_mismatches.is_empty());
147 let after_index = RevisionIndex::parse(&after_store).unwrap();192 let after_index = RevisionIndex::parse(&after_store).unwrap();
...@@ -267,11 +312,19 @@ fn native_subtree_controls_match_with_preserved_fields_and_history() {...@@ -267,11 +312,19 @@ fn native_subtree_controls_match_with_preserved_fields_and_history() {
267 serde_json::to_value(&old.tags).unwrap(),312 serde_json::to_value(&old.tags).unwrap(),
268 "{name}: tags {id}"313 "{name}: tags {id}"
269 );314 );
270 } else if matches!(node.kind, Kind::RichText { .. }) {315 } else if let (
271 assert_eq!(316 Kind::RichText {
272 serde_json::to_value(&node.kind).unwrap(),317 text, boilerplate, ..
273 serde_json::to_value(&wanted.kind).unwrap()318 },
274 );319 Kind::RichText {
320 text: native,
321 boilerplate: native_boilerplate,
322 ..
323 },
324 ) = (&node.kind, &wanted.kind)
325 {
326 // A new run carries the language MS-ONE requires, which OneNote omits.
327 assert_eq!((text, boilerplate), (native, native_boilerplate));
275 }328 }
276 }329 }
277 for (sid, space) in &index.spaces {330 for (sid, space) in &index.spaces {
...@@ -284,7 +337,7 @@ fn native_subtree_controls_match_with_preserved_fields_and_history() {...@@ -284,7 +337,7 @@ fn native_subtree_controls_match_with_preserved_fields_and_history() {
284 }337 }
285 }338 }
286 }339 }
287 let another = PreparedEdit::tree(source, sid, &edit).unwrap();340 let another = ops::apply(source, "Tree author", edit.clone()).unwrap();
288 let second_store = Store::parse(another.as_bytes()).unwrap();341 let second_store = Store::parse(another.as_bytes()).unwrap();
289 let second_index = RevisionIndex::parse(&second_store).unwrap();342 let second_index = RevisionIndex::parse(&second_store).unwrap();
290 let second_doc = Document::parse(&second_index).unwrap();343 let second_doc = Document::parse(&second_index).unwrap();
...@@ -293,10 +346,7 @@ fn native_subtree_controls_match_with_preserved_fields_and_history() {...@@ -293,10 +346,7 @@ fn native_subtree_controls_match_with_preserved_fields_and_history() {
293 actual,346 actual,
294 active(&space.revisions[&space.contexts[&ExGuid::default()]], page)347 active(&space.revisions[&space.contexts[&ExGuid::default()]], page)
295 );348 );
296 candidate = PreparedEdit::tree(&candidate, sid, &edit)349 candidate = ops::apply(&candidate, "Tree author", edit).unwrap().image;
297 .unwrap()
298 .as_bytes()
299 .to_vec();
300 count += 1;350 count += 1;
301 }351 }
302 if let Some(output) = std::env::var_os("ONESTORE_TREE_OUTPUT") {352 if let Some(output) = std::env::var_os("ONESTORE_TREE_OUTPUT") {
...@@ -329,22 +379,24 @@ fn moves_reject_cycles_wrong_parents_and_stale_siblings() {...@@ -329,22 +379,24 @@ fn moves_reject_cycles_wrong_parents_and_stale_siblings() {
329 .unwrap();379 .unwrap();
330 let paragraph = view.nodes[&outline].children[0];380 let paragraph = view.nodes[&outline].children[0];
331 let text = view.nodes[&paragraph].content[0];381 let text = view.nodes[&paragraph].content[0];
332 for intent in [382 let move_to = |object, parent, before| PageOp::Move {
333 TreeEdit::delete(page, "Author"),383 object,
334 TreeEdit::delete(text, "Author"),384 parent: Some(parent),
335 TreeEdit::move_to(outline, paragraph, None, "Author"),385 before,
336 TreeEdit::move_to(paragraph, paragraph, None, "Author"),386 };
337 TreeEdit::move_to(paragraph, text, None, "Author"),387 for op in [
338 TreeEdit::move_to(paragraph, outline, Some(page), "Author"),388 PageOp::Delete { object: page },
389 PageOp::Delete { object: text },
390 move_to(outline, paragraph, None),
391 move_to(paragraph, paragraph, None),
392 move_to(paragraph, text, None),
393 move_to(paragraph, outline, Some(page)),
339 ] {394 ] {
340 assert!(PreparedEdit::tree(&source, sid, &intent.unwrap()).is_err());395 assert!(ops::page_op(&source, sid, op).is_err());
341 }396 }
342 for before in [None, Some(paragraph)] {397 for before in [None, Some(paragraph)] {
343 let edit = TreeEdit::move_to(paragraph, outline, before, "Author").unwrap();398 let op = move_to(paragraph, outline, before);
344 assert_eq!(399 assert_eq!(ops::page_op(&source, sid, op).unwrap().as_bytes(), source);
345 PreparedEdit::tree(&source, sid, &edit).unwrap().as_bytes(),
346 source
347 );
348 }400 }
349}401}
350402
...@@ -370,7 +422,8 @@ fn publication_interruptions_expose_complete_old_or_new_trees() {...@@ -370,7 +422,8 @@ fn publication_interruptions_expose_complete_old_or_new_trees() {
370 ) {422 ) {
371 continue;423 continue;
372 }424 }
373 let edit = PreparedEdit::tree(source, sid, &intent).unwrap();425 let intent = intent.into_iter().map(|op| Op::Page { space: sid, op }).collect();
426 let edit = ops::apply(source, "Tree author", intent).unwrap();
374 let after = current::current(edit.as_bytes());427 let after = current::current(edit.as_bytes());
375 for write_limit in [17, 4096] {428 for write_limit in [17, 4096] {
376 let disk = |fail_at| disk::Disk {429 let disk = |fail_at| disk::Disk {
...@@ -453,8 +506,24 @@ fn cross_container_moves_keep_tables_and_replace_emptied_cells() {...@@ -453,8 +506,24 @@ fn cross_container_moves_keep_tables_and_replace_emptied_cells() {
453 "Move cell subtree down" => (target, outlines[1]),506 "Move cell subtree down" => (target, outlines[1]),
454 _ => (view.nodes[&outlines[0]].children[0], outlines[1]),507 _ => (view.nodes[&outlines[0]].children[0], outlines[1]),
455 };508 };
456 let intent = TreeEdit::move_to(object, destination, None, "Tree author").unwrap();509 let mut intent = vec![Op::Page {
457 let edit = PreparedEdit::tree(source, sid, &intent).unwrap();510 space: sid,
511 op: PageOp::Move {
512 object,
513 parent: Some(destination),
514 before: None,
515 },
516 }];
517 // A cell keeps a paragraph: its replacement goes in first.
518 if name == "Delete sole cell paragraph" {
519 let op = PageOp::Insert {
520 container: parents[&target],
521 before: None,
522 paragraphs: vec![ops::paragraph("")],
523 };
524 intent.insert(0, Op::Page { space: sid, op });
525 }
526 let edit = ops::apply(source, "Tree author", intent.clone()).unwrap();
458 let store = Store::parse(edit.as_bytes()).unwrap();527 let store = Store::parse(edit.as_bytes()).unwrap();
459 let index = RevisionIndex::parse(&store).unwrap();528 let index = RevisionIndex::parse(&store).unwrap();
460 let document = Document::parse(&index).unwrap();529 let document = Document::parse(&index).unwrap();
...@@ -503,10 +572,7 @@ fn cross_container_moves_keep_tables_and_replace_emptied_cells() {...@@ -503,10 +572,7 @@ fn cross_container_moves_keep_tables_and_replace_emptied_cells() {
503 "Delete cell subtree" => assert!(!actual.contains(&outlines[0])),572 "Delete cell subtree" => assert!(!actual.contains(&outlines[0])),
504 _ => assert_eq!(after.nodes[&parents[&target]].children.len(), 1),573 _ => assert_eq!(after.nodes[&parents[&target]].children.len(), 1),
505 }574 }
506 candidate = PreparedEdit::tree(&candidate, sid, &intent)575 candidate = ops::apply(&candidate, "Tree author", intent).unwrap().image;
507 .unwrap()
508 .as_bytes()
509 .to_vec();
510 count += 1;576 count += 1;
511 }577 }
512 assert_eq!(count, 4);578 assert_eq!(count, 4);
crates/onestore/tests/writer.rs+112-83
...@@ -1,39 +1,63 @@...@@ -1,39 +1,63 @@
1#[path = "support/ops.rs"]
2mod ops;
3
1use onestore::{4use onestore::{
2 ExGuid, ObjectData, PropertySets, RevisionIndex, Store, Value, replace_property_bytes,5 ExGuid, ObjectData, PropertySets, RevisionIndex, Store, TocEdit, Value,
6 document::{Document, Kind},
7 op::{Edit, Op, PageOp},
3};8};
4use std::fs;9use std::fs;
510
11/// The text object holding `text` in `source` and its page space.
12fn text_object(source: &[u8], text: &str) -> (ExGuid, ExGuid) {
13 let store = Store::parse(source).unwrap();
14 let index = RevisionIndex::parse(&store).unwrap();
15 let document = Document::parse(&index).unwrap();
16 let mut found = document.spaces.iter().flat_map(|(space, view)| {
17 view.active()
18 .unwrap()
19 .nodes
20 .iter()
21 .filter(|(_, node)| matches!(&node.kind, Kind::RichText { text: stored, .. } if stored == text))
22 .map(|(id, _)| (*space, *id))
23 .collect::<Vec<_>>()
24 });
25 let target = found.next().unwrap();
26 assert!(found.next().is_none());
27 target
28}
29
30/// `source`, a table of contents, with the notebook recoloured.
31fn recolored(source: &[u8], color: u32) -> Vec<u8> {
32 let mut image = source.to_vec();
33 let edit = TocEdit::Color(color);
34 if let Some(transaction) = onestore::edit_table_of_contents(source, &[edit]).unwrap() {
35 transaction.apply(&mut image).unwrap();
36 }
37 image
38}
39
6#[test]40#[test]
7fn scalar_edit_appends_a_revision_and_preserves_every_prior_object() {41fn text_edit_appends_a_revision_and_preserves_every_prior_object() {
8 let source =42 let source =
9 fs::read("../../corpus/native/20260905-05/snapshots/02-text/notebook/synthetic.one")43 fs::read("../../corpus/native/20260905-05/snapshots/02-text/notebook/synthetic.one")
10 .unwrap();44 .unwrap();
11 let store = Store::parse(&source).unwrap();45 let store = Store::parse(&source).unwrap();
12 let index = RevisionIndex::parse(&store).unwrap();46 let index = RevisionIndex::parse(&store).unwrap();
13 let mut selected = None;47 let (osid, oid) = text_object(&source, "Fictitious plain text.");
14 for (osid, space) in &index.spaces {48 let whole = 0.."Fictitious plain text.".encode_utf16().count() as u32;
15 let revision = index49 let typed = |with: &str| {
16 .resolve(*osid, space.labels[&(ExGuid::default(), 1)])50 let op = PageOp::Text {
17 .unwrap();51 text: oid,
18 for oid in revision.reachable().unwrap() {52 range: whole.clone(),
19 let object = &revision.objects[&oid];53 with: with.into(),
20 if let ObjectData::Properties(bytes) = object.data {54 };
21 for property in &PropertySets::parse(bytes).unwrap().sets[0] {55 ops::page_edited(&source, osid, vec![op]).unwrap()
22 if property.value == Value::Bytes(b"Fictitious plain text.") {56 };
23 assert!(selected.replace((*osid, oid, property.id)).is_none());57 assert_eq!(typed("Fictitious plain text."), source);
24 }
25 }
26 }
27 }
28 }
29 let (osid, oid, property) = selected.unwrap();
30 assert_eq!(
31 replace_property_bytes(&source, osid, oid, property, b"Fictitious plain text.").unwrap(),
32 source
33 );
34 for length in 0..=40 {58 for length in 0..=40 {
35 let value = vec![b'x'; length];59 let value = "x".repeat(length);
36 let written = replace_property_bytes(&source, osid, oid, property, &value).unwrap();60 let written = typed(&value);
37 assert_preserved_committed_bytes(&source, &written);61 assert_preserved_committed_bytes(&source, &written);
38 let parsed = Store::parse(&written).unwrap();62 let parsed = Store::parse(&written).unwrap();
39 assert!(parsed.checksum_mismatches.is_empty());63 assert!(parsed.checksum_mismatches.is_empty());
...@@ -64,22 +88,11 @@ fn scalar_edit_appends_a_revision_and_preserves_every_prior_object() {...@@ -64,22 +88,11 @@ fn scalar_edit_appends_a_revision_and_preserves_every_prior_object() {
64 }88 }
65 }89 }
66 }90 }
67 let space = &after.spaces[&osid];91 let document = Document::parse(&after).unwrap();
68 let active = after92 let Kind::RichText { text, .. } = &document.active(osid).unwrap().nodes[&oid].kind else {
69 .resolve(osid, space.labels[&(ExGuid::default(), 1)])
70 .unwrap();
71 let ObjectData::Properties(data) = active.objects[&oid].data else {
72 panic!()93 panic!()
73 };94 };
74 let properties = PropertySets::parse(data).unwrap();95 assert_eq!(*text, value);
75 assert_eq!(
76 properties.sets[0]
77 .iter()
78 .find(|candidate| candidate.id == property)
79 .unwrap()
80 .value,
81 Value::Bytes(&value)
82 );
83 }96 }
84}97}
8598
...@@ -93,26 +106,33 @@ fn toc_color_edit_uses_its_native_revision_encoding_and_crc() {...@@ -93,26 +106,33 @@ fn toc_color_edit_uses_its_native_revision_encoding_and_crc() {
93 let index = RevisionIndex::parse(&store).unwrap();106 let index = RevisionIndex::parse(&store).unwrap();
94 let rid = index.spaces[&index.root].labels[&(ExGuid::default(), 1)];107 let rid = index.spaces[&index.root].labels[&(ExGuid::default(), 1)];
95 let revision = index.resolve(index.root, rid).unwrap();108 let revision = index.resolve(index.root, rid).unwrap();
96 let oid = revision.roots[&1];109 let written = recolored(&source, 0x996633);
97 let written =
98 replace_property_bytes(&source, index.root, oid, 0x14001cbe, &[0x33, 0x66, 0x99, 0])
99 .unwrap();
100 let after = Store::parse(&written).unwrap();110 let after = Store::parse(&written).unwrap();
101 assert!(after.checksum_mismatches.is_empty());111 assert!(after.checksum_mismatches.is_empty());
102 assert_preserved_committed_bytes(&source, &written);112 assert_preserved_committed_bytes(&source, &written);
103 let current = RevisionIndex::parse(&after).unwrap();113 let current = RevisionIndex::parse(&after).unwrap();
104 current.validate_current().unwrap();114 current.validate_current().unwrap();
105 assert_eq!(
106 current.resolve(index.root, rid).unwrap().objects[&oid].data,
107 revision.objects[&oid].data
108 );
109 let active = current115 let active = current
110 .resolve(116 .resolve(
111 index.root,117 index.root,
112 current.spaces[&index.root].labels[&(ExGuid::default(), 1)],118 current.spaces[&index.root].labels[&(ExGuid::default(), 1)],
113 )119 )
114 .unwrap();120 .unwrap();
115 let ObjectData::Properties(blob) = active.objects[&oid].data else {121 let recoloured: Vec<_> = active
122 .objects
123 .iter()
124 .filter(|(id, object)| {
125 revision
126 .objects
127 .get(id)
128 .is_none_or(|old| old.data != object.data)
129 })
130 .map(|(_, object)| object)
131 .collect();
132 let [entry] = recoloured[..] else {
133 panic!("{} objects changed", recoloured.len())
134 };
135 let ObjectData::Properties(blob) = entry.data else {
116 panic!()136 panic!()
117 };137 };
118 assert!(138 assert!(
...@@ -302,13 +322,8 @@ fn toc_writes_cross_fragment_and_counter_boundaries_without_crc_drift() {...@@ -302,13 +322,8 @@ fn toc_writes_cross_fragment_and_counter_boundaries_without_crc_drift() {
302 let store = Store::parse(&bytes).unwrap();322 let store = Store::parse(&bytes).unwrap();
303 assert!(store.checksum_mismatches.is_empty(), "edit {i}");323 assert!(store.checksum_mismatches.is_empty(), "edit {i}");
304 let index = RevisionIndex::parse(&store).unwrap();324 let index = RevisionIndex::parse(&store).unwrap();
305 let root = index.root;325 index.validate_current().unwrap();
306 let revision = index326 let written = recolored(&bytes, i);
307 .resolve(root, index.spaces[&root].labels[&(ExGuid::default(), 1)])
308 .unwrap();
309 let oid = revision.roots[&1];
310 let written =
311 replace_property_bytes(&bytes, root, oid, 0x14001cbe, &i.to_le_bytes()).unwrap();
312 assert_preserved_committed_bytes(&bytes, &written);327 assert_preserved_committed_bytes(&bytes, &written);
313 bytes = written;328 bytes = written;
314 }329 }
...@@ -332,31 +347,31 @@ fn checkpoints_bound_dependencies_and_preserve_native_objects_and_history() {...@@ -332,31 +347,31 @@ fn checkpoints_bound_dependencies_and_preserve_native_objects_and_history() {
332 let mut bytes = fs::read(path).unwrap();347 let mut bytes = fs::read(path).unwrap();
333 let initial = Store::parse(&bytes).unwrap();348 let initial = Store::parse(&bytes).unwrap();
334 let index = RevisionIndex::parse(&initial).unwrap();349 let index = RevisionIndex::parse(&initial).unwrap();
350 // A section retypes text objects of one page; a table of contents recolours the notebook.
351 let document = Document::parse(&index).unwrap();
335 let mut candidates = Vec::new();352 let mut candidates = Vec::new();
336 for (sid, space) in &index.spaces {353 for (sid, view) in &document.spaces {
337 let revision = index354 for (oid, node) in &view.active().unwrap().nodes {
338 .resolve(*sid, space.labels[&(ExGuid::default(), 1)])355 match &node.kind {
339 .unwrap();356 Kind::RichText {
340 for oid in revision.reachable().unwrap() {357 text,
341 let object = &revision.objects[&oid];358 boilerplate: false,
342 if object.jcid != if section { 0x6000e } else { 0x20001 } {359 ..
343 continue;360 } if section && !text.is_empty() => {
344 }361 candidates.push((*sid, *oid));
345 let ObjectData::Properties(blob) = object.data else {362 }
346 panic!()363 Kind::Toc { entries, .. } if !section && !entries.is_empty() => {
347 };364 candidates.push((*sid, *oid));
348 let property = if section { 0x14001d7a } else { 0x14001cbe };365 }
349 if PropertySets::parse(blob).unwrap().sets[0]366 _ => {}
350 .iter()
351 .any(|p| p.id == property)
352 {
353 candidates.push((*sid, oid, property));
354 }367 }
355 }368 }
356 }369 }
357 assert!(!candidates.is_empty());370 assert!(!candidates.is_empty());
358 let sid = candidates[0].0;371 let sid = candidates[0].0;
359 candidates.retain(|candidate| candidate.0 == sid);372 candidates.retain(|candidate| candidate.0 == sid);
373 // What each edit rewrites: modification times, or the notebook's colour.
374 let property = if section { 0x14001d7a } else { 0x14001cbe };
360 if section {375 if section {
361 let revision = index376 let revision = index
362 .resolve(sid, index.spaces[&sid].labels[&(ExGuid::default(), 1)])377 .resolve(sid, index.spaces[&sid].labels[&(ExGuid::default(), 1)])
...@@ -371,12 +386,30 @@ fn checkpoints_bound_dependencies_and_preserve_native_objects_and_history() {...@@ -371,12 +386,30 @@ fn checkpoints_bound_dependencies_and_preserve_native_objects_and_history() {
371 }386 }
372 let mut checkpoints = 0;387 let mut checkpoints = 0;
373 for operation in 0_u32..1025 {388 for operation in 0_u32..1025 {
374 let (_, oid, property) = candidates[operation as usize % candidates.len()];389 let (_, oid) = candidates[operation as usize % candidates.len()];
375 let store = Store::parse(&bytes).unwrap();390 let store = Store::parse(&bytes).unwrap();
376 let before = RevisionIndex::parse(&store).unwrap();391 let before = RevisionIndex::parse(&store).unwrap();
377 let rid = before.spaces[&sid].labels[&(ExGuid::default(), 1)];392 let rid = before.spaces[&sid].labels[&(ExGuid::default(), 1)];
378 let value = (1_000_000 + operation).to_le_bytes();393 let written = if section {
379 let written = replace_property_bytes(&bytes, sid, oid, property, &value).unwrap();394 let typed = |range, with: &str| Op::Page {
395 space: sid,
396 op: PageOp::Text {
397 text: oid,
398 range,
399 with: with.into(),
400 },
401 };
402 // Modification times count seconds: each edit is a second later.
403 let at = 134_000_000_000_000_000 + u64::from(operation) * 10_000_000;
404 let ops = vec![typed(0..0, "x"), typed(0..1, "")];
405 let arena = onestore::Arena::default();
406 let mut section = onestore::Section::open(&arena, bytes.clone()).unwrap();
407 section.apply("Author", &Edit { at, ops }).unwrap();
408 section.seal().unwrap().unwrap();
409 section.image()
410 } else {
411 recolored(&bytes, 1_000_000 + operation)
412 };
380 let updated = Store::parse(&written).unwrap();413 let updated = Store::parse(&written).unwrap();
381 assert!(updated.checksum_mismatches.is_empty());414 assert!(updated.checksum_mismatches.is_empty());
382 assert_eq!(415 assert_eq!(
...@@ -412,9 +445,6 @@ fn checkpoints_bound_dependencies_and_preserve_native_objects_and_history() {...@@ -412,9 +445,6 @@ fn checkpoints_bound_dependencies_and_preserve_native_objects_and_history() {
412 .iter()445 .iter()
413 .all(|(entry, guid)| object.global_ids.get(entry) == Some(guid))446 .all(|(entry, guid)| object.global_ids.get(entry) == Some(guid))
414 );447 );
415 if *id != oid {
416 assert_eq!(object.data, same.data);
417 }
418 }448 }
419 if let (ObjectData::Properties(a), ObjectData::Properties(b)) =449 if let (ObjectData::Properties(a), ObjectData::Properties(b)) =
420 (object.data, same.data)450 (object.data, same.data)
...@@ -426,10 +456,9 @@ fn checkpoints_bound_dependencies_and_preserve_native_objects_and_history() {...@@ -426,10 +456,9 @@ fn checkpoints_bound_dependencies_and_preserve_native_objects_and_history() {
426 assert_eq!(a.len(), b.len());456 assert_eq!(a.len(), b.len());
427 for (a, b) in a.iter().zip(b) {457 for (a, b) in a.iter().zip(b) {
428 assert_eq!(a.id, b.id);458 assert_eq!(a.id, b.id);
429 if *id == oid && a.id == property {459 if a.id != property && !matches!(a.value, Value::References { .. })
430 assert_eq!(b.value, Value::Bytes(&value));460 {
431 } else if !matches!(a.value, Value::References { .. }) {461 assert_eq!(a.value, b.value, "{id} {:#x}", a.id);
432 assert_eq!(a.value, b.value);
433 }462 }
434 }463 }
435 }464 }
tools/canvas/README.md+4-2
...@@ -82,9 +82,11 @@ Superscript and subscript runs draw at two thirds of their size, a third of the...@@ -82,9 +82,11 @@ Superscript and subscript runs draw at two thirds of their size, a third of the
8282
83A paragraph stored as an equation draws in two dimensions from the tree `onestore::page::Math` parses: sub- and superscripts at 70%, fractions over a rule on the math axis, radicals drawn with the pen, n-ary operators at display size with limits above and below (integrals take them as scripts), stretched fences, accents and bars above, boxes, matrices and equation arrays. Latin letters draw in mathematical italic. Cambria Math falls back to STIX Two Math on macOS, and atoms are placed by their glyph ink rather than the math font's tall line box; an equation line is at least as tall as a line of text. The linear text stays the editable source, laid out in the body font so the caret keeps a text height; editing equations in two dimensions is a follow-up.83A paragraph stored as an equation draws in two dimensions from the tree `onestore::page::Math` parses: sub- and superscripts at 70%, fractions over a rule on the math axis, radicals drawn with the pen, n-ary operators at display size with limits above and below (integrals take them as scripts), stretched fences, accents and bars above, boxes, matrices and equation arrays. Latin letters draw in mathematical italic. Cambria Math falls back to STIX Two Math on macOS, and atoms are placed by their glyph ink rather than the math font's tall line box; an equation line is at least as tall as a line of text. The linear text stays the editable source, laid out in the body font so the caret keeps a text height; editing equations in two dimensions is a follow-up.
8484
85## Optional font substitution85## Font substitution
8686
87The host and `layout-probe` accept repeated `--substitute-font FONT_FILE` options. Supply Arimo regular/italic variable fonts or Carlito regular/bold/italic/bold-italic files. Each file is registered only in this process, as Arial or Calibri respectively; this explicitly replaces that family even when a system copy exists. Stored `Format.font` values remain unchanged. No font files are installed or bundled by this option.87`TextEngine::default()` lays out Calibri, Arial, Times New Roman and Courier New in the bundled Carlito 1.104, Arimo 1.341, Tinos 1.340 and Cousine 1.241 (`crates/canvas/assets/fonts`, SIL Open Font License) wherever the system lacks them. The font box names such a family with its substitute, as "Carlito (Calibri)"; stored `Format.font` values keep the document's name.
88
89The host and `layout-probe` also accept repeated `--substitute-font FONT_FILE` options for another Arimo, Carlito, Tinos or Cousine file. Each file is registered only in this process, as its family's counterpart; this explicitly replaces that family even when a system copy exists. No font files are installed.
8890
89```sh91```sh
90"target/Snowbound.app/Contents/MacOS/Snowbound" TEXT_FILE 240 --substitute-font ARIMO_FILE92"target/Snowbound.app/Contents/MacOS/Snowbound" TEXT_FILE 240 --substitute-font ARIMO_FILE
tools/document_model.py+17-1
...@@ -1,8 +1,24 @@...@@ -1,8 +1,24 @@
1"""Traversal of the exported document's referenced revisions."""1"""Traversal of the exported document's referenced revisions."""
2import os2import os
3from pathlib import Path3from pathlib import Path
4import shutil
5import subprocess
46
5EXPORTER = Path(os.environ.get('ONESTORE_DOCUMENT', Path(__file__).resolve().parent.parent / 'target/debug/examples/document'))7
8def _exporter():
9 """The document exporter built with protected sections, copied aside because a workspace
10 build without the feature overwrites target/debug/examples/document."""
11 if 'ONESTORE_DOCUMENT' in os.environ:
12 return Path(os.environ['ONESTORE_DOCUMENT'])
13 root = Path(__file__).resolve().parent.parent
14 subprocess.run(['cargo', 'build', '--quiet', '-p', 'notebook', '--features', 'protected',
15 '--example', 'document'], cwd=root, check=True)
16 protected = root / 'target/debug/examples/document-protected'
17 shutil.copy2(root / 'target/debug/examples/document', protected)
18 return protected
19
20
21EXPORTER = _exporter()
6BRIDGE = Path(__file__).resolve().parent.parent / 'target/debug/onestore-diagnostic'22BRIDGE = Path(__file__).resolve().parent.parent / 'target/debug/onestore-diagnostic'
723
8DEFAULT_CONTEXT = '{00000000-0000-0000-0000-000000000000},0'24DEFAULT_CONTEXT = '{00000000-0000-0000-0000-000000000000},0'
tools/native/collaborate.ps1+40
...@@ -47,6 +47,46 @@ try {...@@ -47,6 +47,46 @@ try {
47 Invoke-Stress $app $section $command $output $shared47 Invoke-Stress $app $section $command $output $shared
48 }48 }
49 elseif ($command.action -eq 'sync') { $app.SyncHierarchy($notebook) }49 elseif ($command.action -eq 'sync') { $app.SyncHierarchy($notebook) }
50 elseif ($command.action -in @('pages', 'move', 'delete-page')) {
51 $namespace = 'http://schemas.microsoft.com/office/onenote/2010/onenote'
52 $hierarchy = ''
53 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
54 [xml]$tree = $hierarchy
55 $listed = @($tree.SelectNodes('//*[local-name()="Page"]'))
56 $named = { param($title) @($listed | Where-Object { $_.GetAttribute('name') -eq $title }) }
57 if ($command.action -eq 'pages') {
58 # New titled pages with one body paragraph each, appended in order.
59 foreach ($title in $command.titles) {
60 $page = ''
61 $app.CreateNewPage($section, [ref]$page, 2)
62 $body = [Security.SecurityElement]::Escape('Body ' + $title + '.')
63 $name = [Security.SecurityElement]::Escape($title)
64 $xml = '<one:Page xmlns:one="' + $namespace + '" ID="' + $page + '"><one:Title><one:OE><one:T>' + $name + '</one:T></one:OE></one:Title><one:Outline><one:OEChildren><one:OE><one:T>' + $body + '</one:T></one:OE></one:OEChildren></one:Outline></one:Page>'
65 $app.UpdatePageContent($xml, [DateTime]::MinValue, 1, $false)
66 }
67 } elseif ($command.action -eq 'move') {
68 # Moves the page titled `page` before the one titled `before`, or last.
69 $moved = @(& $named $command.page)
70 if ($moved.Count -ne 1) { throw 'Expected one page to move.' }
71 $order = [Collections.ArrayList]@($listed | Where-Object { $_ -ne $moved[0] })
72 $at = $order.Count
73 if ($command.before) {
74 $anchor = @(& $named $command.before)
75 if ($anchor.Count -ne 1) { throw 'Expected one anchor page.' }
76 $at = $order.IndexOf($anchor[0])
77 }
78 $order.Insert($at, $moved[0])
79 $xml = '<one:Section xmlns:one="' + $namespace + '" ID="' + $section + '">'
80 foreach ($node in $order) {
81 $xml += '<one:Page ID="' + $node.GetAttribute('ID') + '" pageLevel="' + $node.GetAttribute('pageLevel') + '"/>'
82 }
83 $app.UpdateHierarchy($xml + '</one:Section>', 1)
84 } else {
85 $doomed = @(& $named $command.page)
86 if ($doomed.Count -ne 1) { throw 'Expected one page to delete.' }
87 $app.DeleteHierarchy($doomed[0].GetAttribute('ID'), [DateTime]::MinValue, $false)
88 }
89 }
50 elseif ($command.action -eq 'kill-process') {90 elseif ($command.action -eq 'kill-process') {
51 Get-Process ONENOTE | Stop-Process -Force91 Get-Process ONENOTE | Stop-Process -Force
52 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)92 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
tools/native_collaboration.py+4-4
...@@ -18,7 +18,7 @@ import xml.etree.ElementTree as ET...@@ -18,7 +18,7 @@ import xml.etree.ElementTree as ET
1818
19from native_runner import ROOT, clone, command, windows, vm, collect_artifacts19from native_runner import ROOT, clone, command, windows, vm, collect_artifacts
20from native_xml import texts20from native_xml import texts
21from document_model import ordered_pages, view, walk21from document_model import EXPORTER, ordered_pages, view, walk
22import linux_vm22import linux_vm
2323
2424
...@@ -260,7 +260,7 @@ def replay(output, server, stress_clients=0, stress_operations=30, sync_every=1,...@@ -260,7 +260,7 @@ def replay(output, server, stress_clients=0, stress_operations=30, sync_every=1,
260 target = output / label / 'notebook'260 target = output / label / 'notebook'
261 target.mkdir(parents=True)261 target.mkdir(parents=True)
262 for name, data in snapshot.items(): (target / name).write_bytes(data)262 for name, data in snapshot.items(): (target / name).write_bytes(data)
263 result = subprocess.run([ROOT / 'target/debug/examples/document', target / 'synthetic.one', target.parent / 'model'], capture_output=True, text=True)263 result = subprocess.run([EXPORTER, target / 'synthetic.one', target.parent / 'model'], capture_output=True, text=True)
264 if result.returncode == 0:264 if result.returncode == 0:
265 (target.parent / 'locks.txt').write_text(ssh('sudo smbstatus --locks'))265 (target.parent / 'locks.txt').write_text(ssh('sudo smbstatus --locks'))
266 print('Captured collaboration state:', label, flush=True)266 print('Captured collaboration state:', label, flush=True)
...@@ -318,7 +318,7 @@ def replay(output, server, stress_clients=0, stress_operations=30, sync_every=1,...@@ -318,7 +318,7 @@ def replay(output, server, stress_clients=0, stress_operations=30, sync_every=1,
318 recovered.mkdir()318 recovered.mkdir()
319 with (recovered / 'synthetic.one').open('wb') as stream:319 with (recovered / 'synthetic.one').open('wb') as stream:
320 subprocess.run(linux_vm.ssh_argv(server, 'cat /srv/agent/m6-collaboration/synthetic.one'), stdout=stream, check=True, timeout=60)320 subprocess.run(linux_vm.ssh_argv(server, 'cat /srv/agent/m6-collaboration/synthetic.one'), stdout=stream, check=True, timeout=60)
321 subprocess.run([ROOT / 'target/debug/examples/document', recovered / 'synthetic.one', recovered / 'model'], check=True)321 subprocess.run([EXPORTER, recovered / 'synthetic.one', recovered / 'model'], check=True)
322 model = json.loads((recovered / 'model/document.json').read_text())322 model = json.loads((recovered / 'model/document.json').read_text())
323 text = active_text(model)323 text = active_text(model)
324 retained = verify_text('Concurrent edits:', text, logs)324 retained = verify_text('Concurrent edits:', text, logs)
...@@ -459,7 +459,7 @@ def replay(output, server, stress_clients=0, stress_operations=30, sync_every=1,...@@ -459,7 +459,7 @@ def replay(output, server, stress_clients=0, stress_operations=30, sync_every=1,
459 if temporary.exists(): shutil.rmtree(temporary)459 if temporary.exists(): shutil.rmtree(temporary)
460 snapshot = output / 'conflict.one'460 snapshot = output / 'conflict.one'
461 snapshot.write_bytes(snapshot_file('synthetic.one'))461 snapshot.write_bytes(snapshot_file('synthetic.one'))
462 subprocess.run([ROOT / 'target/debug/examples/document', snapshot, temporary], check=True)462 subprocess.run([EXPORTER, snapshot, temporary], check=True)
463 model = json.loads((temporary / 'document.json').read_text())463 model = json.loads((temporary / 'document.json').read_text())
464 values = [node['kind'].get('text') for space in model['spaces'].values() for revision in space['revisions'].values() for node in revision['nodes'].values()]464 values = [node['kind'].get('text') for space in model['spaces'].values() for revision in space['revisions'].values() for node in revision['nodes'].values()]
465 if 'Client A concurrent edit.' in values and 'Client B concurrent edit.' in values: break465 if 'Client A concurrent edit.' in values and 'Client B concurrent edit.' in values: break
tools/native_conflict.py created+362
...@@ -0,0 +1,362 @@
1#!/usr/bin/env python3
2"""OneNote 2010 conflict pages: how they arise, what they store and how OneNote shows them.
3
4`sync LINUX OUTPUT` runs two disposable OneNote clients on one Samba notebook: client A edits
5two paragraphs offline while client B edits one of them and a table cell online; A then
6reconnects, which makes OneNote write a conflict page. The server file is kept before and
7after, A and B are screenshotted with the information bar, the page list shown, the conflict
8page open and its menu open, and A then deletes the conflict page through that menu.
9
10`pages LINUX OUTPUT` runs the same two clients on page-list changes: client A, offline, moves
11a page that client B also moves, moves another page, and edits a page B deletes; A then
12reconnects. The server's page order and pages, with the recycle bin, are kept at each step.
13
14`restore LINUX OUTPUT` runs them on a page in the middle of the list: client A, offline, edits
15it and moves a page B leaves alone while B deletes it; A then reconnects.
16
17`cold NOTEBOOK OUTPUT [--delete]` cold-opens a notebook holding a conflict page in a fresh
18clone and screenshots the same views; `--delete` deletes the conflict page through the menu
19and captures the notebook OneNote leaves.
20"""
21import argparse
22from contextlib import ExitStack
23from concurrent.futures import ThreadPoolExecutor
24import base64
25import json
26from pathlib import Path
27import shutil
28import subprocess
29import sys
30import time
31import xml.etree.ElementTree as ET
32
33sys.path.insert(0, str(Path(__file__).resolve().parent))
34sys.path.insert(0, str(Path(__file__).resolve().parent / 'w7'))
35import native_runner as runner
36from native_runner import ROOT, clone, command, windows, vm
37import linux_vm
38
39FIXTURE = ROOT / 'corpus/native-ink/cold-ui-ink/notebook'
40SHARE = 'm6-conflict'
41ORIGINAL = 'Fictitious: café, 東京, مرحبا'
42OTHER_OUTLINE = 'Fictitious positioned outline.'
43CELL = 'Left cell'
44
45# Coordinates on OneNote 2010 maximized at 800x600, from the stage-5, m8 and sync-01 captures:
46# the information bar's text, and the first conflict page listed under its page.
47BAR = (332, 100)
48FIRST_CONFLICT = (725, 142)
49# Delete Conflict Page, first in the menu the bar opens on a conflict page.
50DELETE = (420, 113)
51
52
53def shot(name, output, label, script):
54 """Runs an AutoHotkey step on the clone, keeping its screenshot as `label.png`."""
55 (output / f'{label}.ahk').write_text(script)
56 result = windows.do_exec(script, target=name, timeout_ms=120000, shot_delay_ms=1500)
57 screenshot = result.pop('png_b64', None)
58 if screenshot:
59 (output / f'{label}.png').write_bytes(base64.b64decode(screenshot))
60 (output / f'{label}.json').write_text(json.dumps(result, indent=2))
61 if result.get('error') or result.get('exit') != 0:
62 raise RuntimeError(f'{label}: {result}')
63 return result
64
65
66def click(x, y):
67 return ('WinActivate("ahk_exe ONENOTE.EXE")\nCoordMode("Mouse", "Screen")\n'
68 f'Click({x}, {y})\nSleep(1200)\n')
69
70
71# Reports any dialog OneNote raised, so a confirmation shows in the step's record.
72DIALOG = ('if WinExist("ahk_class #32770 ahk_exe ONENOTE.EXE") {\n'
73 ' FileAppend("dialog: " WinGetTitle() " | " WinGetText() "`n", "*", "UTF-8")\n'
74 '}\n')
75
76
77def conflict_views(name, output, page, prefix, delete=False):
78 """Screenshots a page with a conflict page: the bar, the list shown, the conflict page and
79 its menu; `delete` then chooses Delete Conflict Page."""
80 shot(name, output, f'{prefix}-main', runner.navigation_script(page) + 'Sleep(2000)\n')
81 shot(name, output, f'{prefix}-shown', click(*BAR))
82 shot(name, output, f'{prefix}-conflict', click(*FIRST_CONFLICT))
83 shot(name, output, f'{prefix}-menu', click(*BAR))
84 if delete:
85 shot(name, output, f'{prefix}-delete', click(*DELETE) + DIALOG)
86 else:
87 shot(name, output, f'{prefix}-closed', 'WinActivate("ahk_exe ONENOTE.EXE")\nSend("{Esc}")\nSleep(800)\n')
88
89
90def cold(notebook, output, delete):
91 def interaction(name, output):
92 page = ET.parse(sorted((output / 'read').glob('page-*.xml'))[0]).getroot().attrib['ID']
93 conflict_views(name, output, page, 'cold', delete)
94 if delete:
95 # OneNote writes the deletion in the background; give it time before the re-read.
96 time.sleep(20)
97 runner.capture(notebook, output, collect_notebook=True, interaction=interaction)
98
99
100class Lab:
101 """Two OneNote clients on one Samba notebook: `action` runs a controller command,
102 `state` copies the share and exports its section, `settle` syncs until a check holds."""
103
104
105def page_list(model):
106 """The section's pages in order, each as its sorted texts."""
107 from document_model import ordered_pages, walk
108 return [sorted(node['kind']['text'] for _, node in walk(page, oid) if node['kind']['type'] == 'RichText')
109 for _, _, page, oid in ordered_pages(model)]
110
111
112def edits(lab):
113 from native_collaboration import reachable_page_text
114 a, b, output, action, settle = lab.a, lab.b, lab.output, lab.action, lab.settle
115 state = lambda label: reachable_page_text(lab.state(label))
116 texts = lambda state: {text for values in state.values() for text in values}
117 state('initial')
118 vm.qmp(a['name'], 'set_link', {'name': 'lab', 'up': False})
119 try:
120 action(a, 'edit', expected=ORIGINAL, text='Client A conflicting edit.')
121 action(a, 'edit', expected=OTHER_OUTLINE, text='Client A disjoint edit.')
122 action(a, 'snapshot')
123 action(b, 'edit', expected=ORIGINAL, text='Client B conflicting edit.')
124 action(b, 'edit', expected=CELL, text='Client B disjoint cell.')
125 settle(b, lambda label: {'Client B conflicting edit.', 'Client B disjoint cell.'} <= texts(state(label)), 'b-published')
126 finally:
127 vm.qmp(a['name'], 'set_link', {'name': 'lab', 'up': True})
128 settle(a, lambda label: len(found := state(label)) > 1 and 'Client A disjoint edit.' in texts(found), 'conflict')
129 (output / 'conflict.json').write_text(json.dumps(state('conflict'), indent=2, ensure_ascii=False))
130 page = action(a, 'snapshot')[0].attrib['ID']
131 conflict_views(a['name'], output / 'a', page, 'a')
132 action(b, 'sync')
133 time.sleep(5)
134 page_b = action(b, 'snapshot')[0].attrib['ID']
135 conflict_views(b['name'], output / 'b', page_b, 'b')
136 conflict_views(a['name'], output / 'a', page, 'a-again', delete=True)
137 settle(a, lambda label: len(state(label)) == 1, 'deleted')
138 action(b, 'sync')
139 time.sleep(5)
140 shot(b['name'], output / 'b', 'b-after-delete', runner.navigation_script(page_b) + 'Sleep(2000)\n')
141
142
143PAGES = ['One', 'Two', 'Three', 'Four', 'Target']
144
145
146def titled(model, titles):
147 """The section's pages in order, each named by the title among `titles` it holds."""
148 return [next((text for text in texts if text in titles), '') for texts in page_list(model)]
149
150
151def offline(lab, titles, a_actions, b_actions):
152 """Creates the titled pages; A runs its actions offline while B runs and publishes its
153 own; A then reconnects. Keeps the merged order and pages, and what each client lists."""
154 a, b, output, action = lab.a, lab.b, lab.output, lab.action
155 action(a, 'pages', titles=titles)
156 lab.settle(a, lambda label: 'Body Target.' in str(page_list(lab.state(label))), 'initial')
157 deadline = time.monotonic() + 300
158 while not any(node.get('name') == 'Target' for node in action(b, 'snapshot')):
159 action(b, 'sync')
160 if time.monotonic() > deadline: raise TimeoutError('B never listed the new pages')
161 vm.qmp(a['name'], 'set_link', {'name': 'lab', 'up': False})
162 try:
163 for kind, parameters in a_actions:
164 action(a, kind, **parameters)
165 action(a, 'snapshot')
166 for kind, parameters in b_actions:
167 action(b, kind, **parameters)
168 action(b, 'snapshot')
169 lab.settle(b, lambda label: 'Body Target.' not in str(page_list(lab.state(label))), 'b-published')
170 finally:
171 vm.qmp(a['name'], 'set_link', {'name': 'lab', 'up': True})
172 # A's changes reach the server when its sync writes; wait until the file stops changing.
173 previous, steady = None, 0
174 deadline = time.monotonic() + 300
175 while steady < 3 and time.monotonic() < deadline:
176 action(a, 'sync')
177 time.sleep(10)
178 current = page_list(lab.state('merged'))
179 steady = steady + 1 if current == previous else 0
180 previous = current
181 (output / 'merged.json').write_text(json.dumps({'order': titled(lab.state('merged'), titles), 'pages': previous}, indent=2, ensure_ascii=False))
182 for client, label in ((a, 'a'), (b, 'b')):
183 action(client, 'sync')
184 time.sleep(5)
185 listed = action(client, 'snapshot')
186 (output / label / 'listed.json').write_text(json.dumps([node.get('name') for node in listed], ensure_ascii=False))
187 if listed:
188 shot(client['name'], output / label, f'{label}-merged', runner.navigation_script(listed[0].get('ID')) + 'Sleep(2000)\n')
189
190
191def pages(lab):
192 """Competing page moves, and an offline edit of a page the other client deletes."""
193 offline(lab, PAGES, [
194 ('move', {'page': 'Four', 'before': 'One'}),
195 ('move', {'page': 'Two', 'before': None}),
196 ('edit', {'expected': 'Body Target.', 'text': 'Body Target edited offline.'}),
197 ], [
198 ('move', {'page': 'Four', 'before': None}),
199 ('move', {'page': 'Three', 'before': 'One'}),
200 ('delete-page', {'page': 'Target'}),
201 ])
202
203
204def restore(lab):
205 """An offline edit of a page in the middle of the list the other client deletes, with an
206 offline move of a page the other client leaves alone."""
207 offline(lab, ['One', 'Two', 'Target', 'Three', 'Four'], [
208 ('edit', {'expected': 'Body Target.', 'text': 'Body Target edited offline.'}),
209 ('move', {'page': 'Four', 'before': 'One'}),
210 ], [
211 ('delete-page', {'page': 'Target'}),
212 ])
213
214
215def sync(server, output, scenario):
216 from document_model import EXPORTER
217
218 if linux_vm.instance_path(server).exists():
219 raise ValueError('Choose a new Linux VM name; existing machines are not owned by this run.')
220 output = output.resolve()
221 output.mkdir(parents=True, exist_ok=False)
222 scripts = output / 'scripts'
223 scripts.mkdir()
224 for name in ('cold.ps1', 'collaborate.ps1', 'network.ps1', 'text.ps1', 'stress.ps1'):
225 shutil.copyfile(ROOT / 'tools/native' / name, scripts / name)
226 shutil.copyfile(Path(__file__), output / 'native_conflict.py')
227
228 def ssh(text, timeout=90):
229 result = linux_vm.run_ssh(server, text, timeout=timeout)
230 with (output / 'server.jsonl').open('a') as log:
231 log.write(json.dumps({'command': text, 'exit': result.returncode, 'stdout': result.stdout, 'stderr': result.stderr}) + '\n')
232 result.check_returncode()
233 return result.stdout
234
235 def server_state(label):
236 """Copies the server's notebook, recycle bin included, until its section parses,
237 keeping it as `label/notebook`; returns the section's exported model."""
238 deadline = time.monotonic() + 120
239 while True:
240 target = output / label
241 shutil.rmtree(target, ignore_errors=True)
242 (target / 'notebook').mkdir(parents=True)
243 archive = subprocess.run(linux_vm.ssh_argv(server, f"tar cf - -C '/srv/agent/{SHARE}' ."), capture_output=True, check=True, timeout=60).stdout
244 subprocess.run(['tar', 'xf', '-', '-C', target / 'notebook'], input=archive, check=True)
245 result = subprocess.run([EXPORTER, target / 'notebook/synthetic.one', target / 'model'], capture_output=True, text=True)
246 if result.returncode == 0:
247 return json.loads((target / 'model/document.json').read_text())
248 if time.monotonic() > deadline:
249 raise RuntimeError(f'{label}: {result.stderr}')
250 time.sleep(1)
251
252 linux_vm.create_instance(server)
253 try:
254 linux_vm.launch(server)
255 linux_vm.wait_instance(server, 600)
256 (output / 'linux.json').write_text(json.dumps(linux_vm.load_instance(server), indent=2))
257 ssh(f'mkdir /srv/agent/{SHARE}')
258 for name in ('synthetic.one', 'Open Notebook.onetoc2'):
259 with (FIXTURE / name).open('rb') as stream:
260 subprocess.run(linux_vm.ssh_argv(server, f"cat > '/srv/agent/{SHARE}/{name}'"), stdin=stream, check=True)
261 ssh(f'chmod u+w /srv/agent/{SHARE}/*')
262 with ExitStack() as stack:
263 labels = ['a', 'b']
264 for label in labels:
265 (output / label).mkdir()
266
267 def start(label):
268 manager = clone(output / label)
269 name = manager.__enter__()
270 return label, name, manager
271 with ThreadPoolExecutor(max_workers=2) as pool:
272 started = list(pool.map(start, labels))
273 for _, _, manager in started:
274 stack.push(manager)
275 clients = {}
276 for label, name, _ in started:
277 folder = output / label
278 for local in scripts.iterdir():
279 remote = 'cold-current.ps1' if local.name == 'cold.ps1' else local.name
280 result = windows.do_put(local, 'C:\\one-tests\\' + remote, name)
281 if result.get('error'): raise RuntimeError(result['error'])
282 command(name, 'mkdir C:\\one-tests\\runs\\capture', folder)
283 command(name, 'powershell -NoProfile -ExecutionPolicy Bypass -File C:\\one-tests\\network.ps1 -LabMac ' + vm.lab_mac(name), folder)
284 result = windows.do_spawn('cmd /c powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\collaborate.ps1 -Root C:\\one-tests\\runs\\capture -SharedPath \\\\192.168.77.1\\agent\\' + SHARE + ' -CloneHost ONE-' + name.upper() + ' > C:\\one-tests\\runs\\capture\\controller.log 2>&1', name)
285 if result.get('error'): raise RuntimeError(result['error'])
286 deadline = time.monotonic() + 180
287 while windows.do_cmd('if exist C:\\one-tests\\runs\\capture\\ready (echo ready)', target=name).get('stdout', '').find('ready') < 0:
288 if time.monotonic() > deadline: raise RuntimeError('The collaboration controller did not open the notebook')
289 time.sleep(1)
290 clients[label] = {'name': name, 'folder': folder, 'sequence': 0}
291 (output / 'clients.json').write_text(json.dumps({label: client['name'] for label, client in clients.items()}, indent=2))
292
293 def action(client, kind, **parameters):
294 client['sequence'] += 1
295 sequence = client['sequence']
296 local = client['folder'] / f'command-{sequence:04}.json'
297 local.write_text(json.dumps({'action': kind, **parameters}, ensure_ascii=False))
298 inbox = f'C:\\one-tests\\runs\\capture\\inbox\\{sequence:04}'
299 result = windows.do_put(local, inbox + '.tmp', client['name'])
300 if result.get('error'): raise RuntimeError(result['error'])
301 command(client['name'], f'move {inbox}.tmp {inbox}.json', client['folder'])
302 remote = f'C:\\one-tests\\runs\\capture\\outbox\\{sequence}'
303 deadline = time.monotonic() + 600
304 while True:
305 result = windows.do_cmd(f'if exist {remote}\\error (type {remote}\\error & exit /b 1) else (if exist {remote}\\done (echo complete))', target=client['name'])
306 if result.get('exit') != 0: raise RuntimeError(str(result))
307 if 'complete' in result.get('stdout', ''): break
308 if time.monotonic() > deadline: raise TimeoutError(f'{kind} did not complete')
309 time.sleep(.5)
310 if kind != 'snapshot':
311 return None
312 hierarchy = client['folder'] / f'hierarchy-{sequence:04}.xml'
313 windows.do_get(remote + '\\hierarchy.xml', hierarchy, client['name'])
314 pages = [node for node in ET.fromstring(hierarchy.read_text(encoding='utf-8-sig').strip()).iter() if node.tag.endswith('}Page')]
315 for i in range(len(pages)):
316 windows.do_get(remote + f'\\page-{i}.xml', client['folder'] / f'snapshot-{sequence:04}-{i}.xml', client['name'])
317 return pages
318
319 def settle(client, check, label):
320 deadline = time.monotonic() + 300
321 while True:
322 action(client, 'sync')
323 if check(label): return
324 if time.monotonic() > deadline: raise TimeoutError(label)
325 time.sleep(2)
326
327 lab = Lab()
328 lab.a, lab.b, lab.output = clients['a'], clients['b'], output
329 lab.action, lab.state, lab.settle = action, server_state, settle
330 scenario(lab)
331
332 for client in clients.values():
333 try:
334 action(client, 'close')
335 except Exception as error:
336 (client['folder'] / 'close-failure.txt').write_text(str(error))
337 server_state('final')
338 finally:
339 if linux_vm.running(server):
340 linux_vm.shutdown(server, 120)
341 linux_vm.delete_instance(server)
342 (output / 'linux-teardown.json').write_text(json.dumps({'absent': not linux_vm.instance_path(server).exists()}) + '\n')
343
344
345if __name__ == '__main__':
346 parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
347 commands = parser.add_subparsers(dest='mode', required=True)
348 scenarios = {'sync': edits, 'pages': pages, 'restore': restore}
349 for mode in scenarios:
350 lab = commands.add_parser(mode)
351 lab.add_argument('linux')
352 lab.add_argument('output', type=Path)
353 reopen = commands.add_parser('cold')
354 reopen.add_argument('notebook', type=Path)
355 reopen.add_argument('output', type=Path)
356 reopen.add_argument('--delete', action='store_true')
357 args = parser.parse_args()
358 if args.mode in scenarios:
359 sync(args.linux, args.output, scenarios[args.mode])
360 else:
361 cold(args.notebook, args.output, args.delete)
362 print('Captured', args.output)
tools/smb-proxy.py+28-4
...@@ -8,6 +8,18 @@ import struct...@@ -8,6 +8,18 @@ import struct
8import time8import time
99
1010
11def lease_state(frame, at, contexts):
12 """The lease state of a CREATE's `RqLs` context, if any (MS-SMB2 2.2.13.2.8, 2.2.14.2.10)."""
13 offset, length = struct.unpack_from('<II', frame, at + contexts)
14 cursor = at + offset if length else None
15 while cursor is not None:
16 following, name_offset, name_length, _, data_offset = struct.unpack_from('<IHHHH', frame, cursor)
17 if frame[cursor + name_offset:cursor + name_offset + name_length] == b'RqLs':
18 return struct.unpack_from('<I', frame, cursor + data_offset + 16)[0]
19 cursor = cursor + following if following else None
20 return None
21
22
11def header_fields(offset, data):23def header_fields(offset, data):
12 result = {}24 result = {}
13 for name, start, length in [('transactions', 96, 4), ('version', 212, 16),25 for name, start, length in [('transactions', 96, 4), ('version', 212, 16),
...@@ -77,7 +89,7 @@ async def main():...@@ -77,7 +89,7 @@ async def main():
77 at = 089 at = 0
78 while frame[at:at+4] == b'\xfeSMB':90 while frame[at:at+4] == b'\xfeSMB':
79 command = struct.unpack_from('<H', frame, at+12)[0]91 command = struct.unpack_from('<H', frame, at+12)[0]
80 entry = {'connection': connection_id, 'direction': direction, 'command': command,92 entry = {'connection': connection_id, 'direction': direction, 'command': command, 'frame': frames[direction],
81 'message': struct.unpack_from('<Q', frame, at+24)[0],93 'message': struct.unpack_from('<Q', frame, at+24)[0],
82 'credit_charge': struct.unpack_from('<H', frame, at+6)[0],94 'credit_charge': struct.unpack_from('<H', frame, at+6)[0],
83 'credits': struct.unpack_from('<H', frame, at+14)[0]}95 'credits': struct.unpack_from('<H', frame, at+14)[0]}
...@@ -110,12 +122,16 @@ async def main():...@@ -110,12 +122,16 @@ async def main():
110 elif command == 5:122 elif command == 5:
111 entry['oplock'] = frame[at+67]123 entry['oplock'] = frame[at+67]
112 entry['access'] = struct.unpack_from('<I', frame, at+88)[0]124 entry['access'] = struct.unpack_from('<I', frame, at+88)[0]
113 entry['share'] = struct.unpack_from('<I', frame, at+96)[0]125 entry['share'], entry['disposition'], entry['options'] = struct.unpack_from('<III', frame, at+96)
114 offset, length = struct.unpack_from('<HH', frame, at+108)126 offset, length = struct.unpack_from('<HH', frame, at+108)
115 entry['path'] = frame[at+offset:at+offset+length].decode('utf-16-le')127 entry['path'] = frame[at+offset:at+offset+length].decode('utf-16-le')
128 if entry['oplock'] == 0xff:
129 entry['lease'] = lease_state(frame, at, 112)
116 elif command == 5 and entry['status'] == '0x0':130 elif command == 5 and entry['status'] == '0x0':
117 entry['oplock'] = frame[at+66]131 entry['oplock'] = frame[at+66]
118 entry['file_id'] = frame[at+128:at+144].hex()132 entry['file_id'] = frame[at+128:at+144].hex()
133 if entry['oplock'] == 0xff:
134 entry['lease'] = lease_state(frame, at, 144)
119 elif command == 9 and entry['status'] == '0x0':135 elif command == 9 and entry['status'] == '0x0':
120 entry['written'] = struct.unpack_from('<I', frame, at+68)[0]136 entry['written'] = struct.unpack_from('<I', frame, at+68)[0]
121 if direction == 'request':137 if direction == 'request':
...@@ -145,12 +161,20 @@ async def main():...@@ -145,12 +161,20 @@ async def main():
145 if not next_command:161 if not next_command:
146 break162 break
147 at += next_command163 at += next_command
164 frames[direction] += 1
148 if frame[:4] == b'\xfdSMB':165 if frame[:4] == b'\xfdSMB':
149 record(encrypted=True, direction=direction)166 record(encrypted=True, direction=direction)
150 writer.write(prefix + frame)167 if state.get('delay_ms'):
151 await writer.drain()168 # Half the round trip each way, as on a wide-area link: every frame
169 # arrives that much later, in order, without queueing behind others.
170 loop = asyncio.get_running_loop()
171 loop.call_at(loop.time() + state['delay_ms'] / 2000, writer.write, prefix + frame)
172 else:
173 writer.write(prefix + frame)
174 await writer.drain()
152175
153 requests = {}176 requests = {}
177 frames = {'request': 0, 'response': 0}
154 tasks = [asyncio.create_task(forward(client, server_writer, 'request')),178 tasks = [asyncio.create_task(forward(client, server_writer, 'request')),
155 asyncio.create_task(forward(server, client_writer, 'response'))]179 asyncio.create_task(forward(server, client_writer, 'response'))]
156 done, pending = await asyncio.wait(tasks, return_when=asyncio.FIRST_COMPLETED)180 done, pending = await asyncio.wait(tasks, return_when=asyncio.FIRST_COMPLETED)
tools/smb_lock_race.py+21-11
...@@ -13,13 +13,17 @@ import sys...@@ -13,13 +13,17 @@ import sys
13import time13import time
1414
1515
16def worker(file, marker, mode, operations, atomic):16def worker(file, marker, mode, operations, atomic, shared):
17 counts = Counter()17 counts = Counter()
18 for i in range(operations):18 for i in range(operations):
19 access = ('r+b' if i % 2 else 'rb') if mode == 'mixed' else mode19 access = ('r+b' if i % 2 else 'rb') if mode == 'mixed' else mode
20 # With shared reads, readers open O_SHLOCK and may overlap each other, never a writer.
21 reading = shared and access == 'rb'
22 held = marker.with_name(f'{marker.name}-reader-{os.getpid()}') if reading else marker
20 stage = 'open'23 stage = 'open'
21 try:24 try:
22 with (os.fdopen(os.open(file, (os.O_RDWR if access == 'r+b' else os.O_RDONLY) | os.O_EXLOCK | os.O_NONBLOCK), access) if atomic else file.open(access)) as stream:25 lock = os.O_SHLOCK if reading else os.O_EXLOCK
26 with (os.fdopen(os.open(file, (os.O_RDWR if access == 'r+b' else os.O_RDONLY) | lock | os.O_NONBLOCK), access) if atomic else file.open(access)) as stream:
23 stage = 'lock'27 stage = 'lock'
24 if not atomic:28 if not atomic:
25 fcntl.flock(stream, fcntl.LOCK_EX | fcntl.LOCK_NB)29 fcntl.flock(stream, fcntl.LOCK_EX | fcntl.LOCK_NB)
...@@ -27,8 +31,13 @@ def worker(file, marker, mode, operations, atomic):...@@ -27,8 +31,13 @@ def worker(file, marker, mode, operations, atomic):
27 owned = False31 owned = False
28 try:32 try:
29 try:33 try:
30 marker.mkdir()34 held.mkdir()
31 owned = True35 owned = True
36 others = [p for p in marker.parent.glob(marker.name + '*') if p != held]
37 if others and not (reading and all('-reader-' in p.name for p in others)):
38 counts['overlap'] += 1
39 elif others:
40 counts['shared'] += 1
32 except FileExistsError:41 except FileExistsError:
33 counts['overlap'] += 142 counts['overlap'] += 1
34 time.sleep(.002)43 time.sleep(.002)
...@@ -38,7 +47,7 @@ def worker(file, marker, mode, operations, atomic):...@@ -38,7 +47,7 @@ def worker(file, marker, mode, operations, atomic):
38 counts[f'read:{error.errno}'] += 147 counts[f'read:{error.errno}'] += 1
39 finally:48 finally:
40 if owned:49 if owned:
41 marker.rmdir()50 held.rmdir()
42 try:51 try:
43 fcntl.flock(stream, fcntl.LOCK_UN)52 fcntl.flock(stream, fcntl.LOCK_UN)
44 except OSError as error:53 except OSError as error:
...@@ -50,10 +59,10 @@ def worker(file, marker, mode, operations, atomic):...@@ -50,10 +59,10 @@ def worker(file, marker, mode, operations, atomic):
50 print(json.dumps(counts), flush=True)59 print(json.dumps(counts), flush=True)
5160
5261
53def run(output, directory, operations, atomic, clients):62def run(output, directory, operations, atomic, clients, shared):
54 output.mkdir(parents=True, exist_ok=False)63 output.mkdir(parents=True, exist_ok=False)
55 directory.mkdir(exist_ok=False)64 directory.mkdir(exist_ok=False)
56 (output / 'run.json').write_text(json.dumps({'operations': operations, 'clients': clients, 'atomic': atomic,65 (output / 'run.json').write_text(json.dumps({'operations': operations, 'clients': clients, 'atomic': atomic, 'shared_reads': shared,
57 'platform': platform.platform(), 'script_sha256': hashlib.sha256(Path(__file__).read_bytes()).hexdigest()}, indent=2))66 'platform': platform.platform(), 'script_sha256': hashlib.sha256(Path(__file__).read_bytes()).hexdigest()}, indent=2))
58 results = {}67 results = {}
59 for mode in ('rb', 'r+b', 'mixed'):68 for mode in ('rb', 'r+b', 'mixed'):
...@@ -63,7 +72,7 @@ def run(output, directory, operations, atomic, clients):...@@ -63,7 +72,7 @@ def run(output, directory, operations, atomic, clients):
63 processes = []72 processes = []
64 try:73 try:
65 for i in range(clients):74 for i in range(clients):
66 processes.append(subprocess.Popen([sys.executable, __file__, '--worker', str(file), str(marker), mode, str(operations), str(int(atomic))],75 processes.append(subprocess.Popen([sys.executable, __file__, '--worker', str(file), str(marker), mode, str(operations), str(int(atomic)), str(int(shared))],
67 stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True))76 stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True))
68 results[mode] = []77 results[mode] = []
69 for process in processes:78 for process in processes:
...@@ -83,13 +92,13 @@ def run(output, directory, operations, atomic, clients):...@@ -83,13 +92,13 @@ def run(output, directory, operations, atomic, clients):
83 (output / 'result.json').write_text(json.dumps(results, indent=2))92 (output / 'result.json').write_text(json.dumps(results, indent=2))
84 print(mode, results[mode], flush=True)93 print(mode, results[mode], flush=True)
85 assert all(c.get('acquired', 0) and not c.get('overlap', 0) and94 assert all(c.get('acquired', 0) and not c.get('overlap', 0) and
86 all(k in ('acquired', 'open:35', 'lock:35') for k in c)95 all(k in ('acquired', 'shared', 'open:35', 'lock:35') for k in c)
87 for counters in results.values() for c in counters), 'Lock exclusion or progress failed'96 for counters in results.values() for c in counters), 'Lock exclusion or progress failed'
8897
8998
90if __name__ == '__main__':99if __name__ == '__main__':
91 if len(sys.argv) == 7 and sys.argv[1] == '--worker':100 if len(sys.argv) == 8 and sys.argv[1] == '--worker':
92 worker(Path(sys.argv[2]), Path(sys.argv[3]), sys.argv[4], int(sys.argv[5]), bool(int(sys.argv[6])))101 worker(Path(sys.argv[2]), Path(sys.argv[3]), sys.argv[4], int(sys.argv[5]), bool(int(sys.argv[6])), bool(int(sys.argv[7])))
93 else:102 else:
94 parser = argparse.ArgumentParser(description=__doc__)103 parser = argparse.ArgumentParser(description=__doc__)
95 parser.add_argument('output', type=Path)104 parser.add_argument('output', type=Path)
...@@ -97,5 +106,6 @@ if __name__ == '__main__':...@@ -97,5 +106,6 @@ if __name__ == '__main__':
97 parser.add_argument('--operations', type=int, default=500)106 parser.add_argument('--operations', type=int, default=500)
98 parser.add_argument('--atomic', action='store_true')107 parser.add_argument('--atomic', action='store_true')
99 parser.add_argument('--clients', type=int, default=4)108 parser.add_argument('--clients', type=int, default=4)
109 parser.add_argument('--shared-reads', action='store_true', help='With --atomic, readers open O_SHLOCK.')
100 args = parser.parse_args()110 args = parser.parse_args()
101 run(args.output.resolve(), args.directory.resolve(), args.operations, args.atomic, args.clients)111 run(args.output.resolve(), args.directory.resolve(), args.operations, args.atomic, args.clients, args.shared_reads)
tools/test_conflict_page.py created+166
...@@ -0,0 +1,166 @@
1"""Conflict pages (corpus/conflict-page): what OneNote 2010 stores and shows for one, and
2that it reads the ones Rust writes and deletes."""
3import json
4from pathlib import Path
5import runpy
6import shutil
7import subprocess
8from tempfile import TemporaryDirectory
9import unittest
10import hashlib
11
12from native_xml import pages, texts
13from document_model import EXPORTER, ordered_pages, view
14
15ROOT = Path(__file__).resolve().parent.parent
16FIXTURE = ROOT / 'corpus/conflict-page'
17compare = runpy.run_path(str(ROOT / 'tools/verify-document.py'))['compare']
18DEFAULT = '{00000000-0000-0000-0000-000000000000},0'
19RENDER, SELECT = 0x88001D96, 0x88001DDB
20
21
22def model(notebook, section='synthetic.one'):
23 with TemporaryDirectory() as temporary:
24 output = Path(temporary) / 'model'
25 subprocess.run([EXPORTER, notebook / section, output], check=True, capture_output=True)
26 return json.loads((output / 'document.json').read_text())
27
28
29def conflicts(document):
30 """Each listed page's conflict pages: whose version, its texts and the marked texts."""
31 listed = {}
32 for sid, _, page, _ in ordered_pages(document):
33 manifest = page['nodes'][page['roots']['1']]
34 versions = []
35 for space in manifest['spaces']:
36 _, revision = view(document, space)
37 metadata = revision['nodes'][revision['roots']['2']]['kind']
38 rich = [node for node in revision['nodes'].values() if node['kind']['type'] == 'RichText']
39 flagged = lambda node, flag: any(field['id'] == flag for fields in node['extra'] for field in fields)
40 versions.append({'user': metadata['author'], 'texts': sorted(node['kind']['text'] for node in rich),
41 'marked': sorted(node['kind']['text'] for node in rich if flagged(node, RENDER) and flagged(node, SELECT))})
42 listed[sid] = versions
43 return listed
44
45
46def page_texts(document, sid):
47 _, page = view(document, sid)
48 return sorted(node['kind']['text'] for node in page['nodes'].values() if node['kind']['type'] == 'RichText')
49
50
51TITLES = {'One', 'Two', 'Three', 'Four', 'Target'}
52
53
54def series(document):
55 """The section's page series in order, each as the titles of its pages."""
56 _, root = view(document, document['root'])
57 section = root['nodes'][root['roots']['1']]
58 return {oid: [next((text for text in page_texts(document, space) if text in TITLES), '')
59 for space in root['nodes'][oid]['spaces']] for oid in section['children']}
60
61
62def spaces(document):
63 """Each listed page's space by its title."""
64 return {next((text for text in page_texts(document, sid) if text in TITLES), ''): sid
65 for sid, *_ in ordered_pages(document)}
66
67
68class ConflictPageTest(unittest.TestCase):
69 def test_onenote_keeps_the_remote_version_and_the_local_one_as_a_conflict_page(self):
70 document = model(FIXTURE / 'native/conflict/notebook')
71 (sid, versions), = conflicts(document).items()
72 # Client A's disjoint edit merged; the paragraph both changed keeps client B's text.
73 self.assertIn('Client A disjoint edit.', page_texts(document, sid))
74 self.assertIn('Client B conflicting edit.', page_texts(document, sid))
75 self.assertEqual(versions, [{'user': 'snow',
76 'texts': ['', 'Client A conflicting edit.', 'Client A disjoint edit.', 'Left cell', 'Right cell'],
77 'marked': ['Client A conflicting edit.']}])
78 # COM lists the page alone; the conflict page shows only through its bar.
79 for client in 'ab':
80 self.assertTrue((FIXTURE / f'native/{client}/{client}-shown.png').is_file())
81
82 def test_onenote_deletes_a_conflict_page_from_its_page_only(self):
83 document = model(FIXTURE / 'native-delete/notebook')
84 self.assertEqual(conflicts(document), {sid: [] for sid in conflicts(document)})
85 self.assertEqual(len(pages(FIXTURE / 'native-delete/read')), 1)
86
87 def test_three_clients_conflict_pages_list_the_last_stored_first(self):
88 document = model(FIXTURE / 'native-three/notebook')
89 (_, versions), = conflicts(document).items()
90 self.assertEqual([version['user'] for version in versions],
91 ['ONE-M6-DDE10E0D', 'ONE-M6-9451B6C6', 'ONE-M6-053BD357'])
92
93 def test_onenote_reads_a_rust_conflict_page_shows_its_bar_and_deletes_it(self):
94 source = json.loads((FIXTURE / 'cold/source.json').read_text())
95 written = hashlib.sha256((FIXTURE / 'candidate/synthetic.one').read_bytes()).hexdigest()
96 self.assertIn(written, [item['sha256'] for item in source])
97 with TemporaryDirectory() as temporary:
98 read = Path(temporary) / 'read'
99 shutil.copytree(FIXTURE / 'cold/before-read', read)
100 compare(FIXTURE / 'candidate', read)
101 (page,) = pages(FIXTURE / 'cold/before-read')
102 self.assertIn('Remote: café, 東京, مرحبا', texts(page))
103 document = model(FIXTURE / 'candidate')
104 (_, versions), = conflicts(document).items()
105 self.assertEqual(versions[0]['user'], 'Clover Snow')
106 self.assertEqual(versions[0]['marked'], ['Snowbound: café, 東京, مرحبا'])
107 # The bar on the page opens the conflict page, as OneNote's own do, and its menu
108 # deletes it, leaving the page without a bar.
109 titles = {name: json.loads((FIXTURE / f'cold/cold-{name}.json').read_text())['win']['title']
110 for name in ('main', 'shown', 'delete')}
111 self.assertEqual(titles, {'main': 'Remote: café, 東京, مرحبا - Microsoft OneNote',
112 'shown': 'Snowbound: café, 東京, مرحبا - Microsoft OneNote',
113 'delete': 'Remote: café, 東京, مرحبا - Microsoft OneNote'})
114 deleted = model(FIXTURE / 'cold/notebook')
115 self.assertEqual(conflicts(deleted), {sid: [] for sid in conflicts(deleted)})
116
117 def test_onenote_reads_a_conflict_page_rust_deleted(self):
118 document = model(FIXTURE / 'candidate-delete')
119 self.assertEqual(conflicts(document), {sid: [] for sid in conflicts(document)})
120 with TemporaryDirectory() as temporary:
121 read = Path(temporary) / 'read'
122 shutil.copytree(FIXTURE / 'cold-delete/read', read)
123 compare(FIXTURE / 'candidate-delete', read)
124 (page,) = pages(FIXTURE / 'cold-delete/read')
125 self.assertIn('Client B conflicting edit.', texts(page))
126
127 def test_page_moves_merge_as_page_series_do(self):
128 """Each client gives a page it moves a series of its own; the merge keeps each page
129 in the series of the client that gave it one, the server's where both did, and
130 leaves the other client's series empty. A page the server deleted comes back in a
131 new series where the offline client had it."""
132 for capture, order, empty, server, kept, client in (
133 ('native-pages', ['One', 'Two', 'Four', 'Three', 'Target'], 2, ['One', 'Two', 'Four'], [], ['Three', 'Target']),
134 ('native-restore', ['Four', 'One', 'Two', 'Target', 'Three'], 0, [], ['Four'], ['One', 'Two', 'Three', 'Target'])):
135 root = FIXTURE / capture
136 initial, published, merged = (series(model(root / phase / 'notebook'))
137 for phase in ('initial', 'b-published', 'merged'))
138 self.assertEqual([titles for titles in merged.values() if titles][1:], [[title] for title in order])
139 self.assertEqual(sum(not titles for titles in merged.values()), empty, capture)
140 self.assertEqual(json.loads((root / 'merged.json').read_text())['order'][1:], order)
141 for listed in 'ab':
142 self.assertEqual(json.loads((root / listed / 'listed.json').read_text())[1:], order)
143 owner = lambda state, title: next((oid for oid, titles in state.items() if title in titles), None)
144 for title in server:
145 self.assertEqual(owner(merged, title), owner(published, title), title)
146 self.assertNotIn(owner(merged, title), initial, title)
147 for title in kept:
148 self.assertEqual(owner(merged, title), owner(initial, title), title)
149 for title in client:
150 self.assertNotIn(owner(merged, title), {**initial, **published}, title)
151
152 def test_a_page_edited_offline_and_deleted_online_comes_back_as_a_new_page(self):
153 for capture in ('native-pages', 'native-restore'):
154 root = FIXTURE / capture
155 before = spaces(model(root / 'initial/notebook'))
156 document = model(root / 'merged/notebook')
157 target = spaces(document)['Target']
158 self.assertNotEqual(target, before['Target'], capture)
159 self.assertIn('Body Target edited offline.', page_texts(document, target))
160 # The deleting client's version stays in the recycle bin.
161 deleted = model(root / 'merged/notebook/OneNote_RecycleBin', 'OneNote_DeletedPages.one')
162 self.assertIn('Body Target.', page_texts(deleted, spaces(deleted)['Target']), capture)
163
164
165if __name__ == '__main__':
166 unittest.main()
tools/test_extended_ascii.py created+31
...@@ -0,0 +1,31 @@
1from pathlib import Path
2import runpy
3import shutil
4from tempfile import TemporaryDirectory
5import unittest
6import xml.etree.ElementTree as ET
7
8from native_xml import ns
9
10ROOT = Path(__file__).resolve().parent.parent
11FIXTURE = ROOT / 'corpus/extended-ascii'
12compare = runpy.run_path(str(ROOT / 'tools/verify-document.py'))['compare']
13
14
15class ExtendedAsciiTest(unittest.TestCase):
16 def test_onenote_reads_extended_ascii_as_latin_1_whatever_the_language(self):
17 expected = 'Extended bytes: ' + ' '.join(chr(byte) for byte in range(128, 256))
18 for language in ('en', 'ru', 'ja'):
19 with self.subTest(language=language):
20 capture = FIXTURE / language / 'cold'
21 with TemporaryDirectory() as temporary:
22 read = Path(temporary) / 'read'
23 shutil.copytree(capture / 'read', read)
24 compare(FIXTURE / language / 'candidate', read)
25 page = ET.parse(capture / 'read/page-000.xml').getroot()
26 texts = [t.text for t in page.iterfind('.//one:Outline//one:T', ns)]
27 self.assertEqual(texts, [expected.replace('\xa0', '&nbsp;')])
28
29
30if __name__ == '__main__':
31 unittest.main()
tools/test_native_stress.py+2-2
...@@ -9,7 +9,7 @@ from unittest.mock import patch...@@ -9,7 +9,7 @@ from unittest.mock import patch
9import native_stress9import native_stress
10from native_stress import edit_history, native_history, verify_capture10from native_stress import edit_history, native_history, verify_capture
11from native_collaboration import reachable_page_text11from native_collaboration import reachable_page_text
12from document_model import DEFAULT_CONTEXT12from document_model import DEFAULT_CONTEXT, EXPORTER
1313
1414
15class NativeHistoryTests(unittest.TestCase):15class NativeHistoryTests(unittest.TestCase):
...@@ -17,7 +17,7 @@ class NativeHistoryTests(unittest.TestCase):...@@ -17,7 +17,7 @@ class NativeHistoryTests(unittest.TestCase):
17 root = Path(__file__).resolve().parent.parent17 root = Path(__file__).resolve().parent.parent
18 with tempfile.TemporaryDirectory() as folder:18 with tempfile.TemporaryDirectory() as folder:
19 output = Path(folder) / 'model'19 output = Path(folder) / 'model'
20 subprocess.run([root / 'target/debug/examples/document',20 subprocess.run([EXPORTER,
21 root / 'corpus/collaboration/round-01/offline/notebook/synthetic.one', output], check=True)21 root / 'corpus/collaboration/round-01/offline/notebook/synthetic.one', output], check=True)
22 model = json.loads((output / 'document.json').read_text())22 model = json.loads((output / 'document.json').read_text())
23 retained = reachable_page_text(model)23 retained = reachable_page_text(model)