authorgravatar for git@paperclover.netclover caruso <git@paperclover.net> 2026-09-08 00:02:13-07:00
committergravatar for git@paperclover.netclover caruso <git@paperclover.net> 2026-09-08 00:02:13-07:00
log93784cadb44ec833ca6d7f3063c5d87255c1d6af
treef302efb1f9d5fdea8f4672603c3c3ab3821651da
parent49bc9b2d95d93ceb9f25de3e238e89ce6fbfb21e
signature Signed by SSH key SHA256:52mNGHRsVFBDED9IAX5pe+LRWUefqTbxEReunq21QvU

feat: replace text across ordinary formatting runs

Preserve surviving style references and terminal typing style while replacing scalar-aligned UTF-16 ranges. Reject affected fields and preserved run data before publication; reuse existing offline text intents after rebasing onto remote styles. Validate exhaustive ranges, history preservation, interrupted commits and lost replies. Retain six independent native fixtures and extend the twelve-client native/Rust workload to insertion, formatting and cross-run replacement. Checks: 180 Rust tests, eight doctests, 126 Python tests, Clippy and formatting; 7,987 ASan executions; 1,280 mixed editing intents through an outage with independent cold OneNote verification. Assisted-by: gpt-6-astra

82 files changed, 1018 insertions(+), 131 deletions(-)

corpus/cross-run-edit/README.md created+20
......@@ -0,0 +1,20 @@
1# Cross-run text replacements
2
3Six edits of the public formatted-insertion notebooks, independently opened by
4OneNote 2010 in disposable cold-cache machines. Cases cover partial and complete
5replacement, a native table cell, deletion, typing after deletion, and insertion
6at a style boundary. Each directory is a separate notebook because the source
7section identities can be shared.
8
9The inserted characters inherit the style at the start of the replaced range;
10surviving characters retain their styles. Empty final runs retain their typing
11style. These are library edit semantics; the native captures check that OneNote
12reads the resulting content and formatting correctly.
13
14Generate candidates with the ignored `export_native_cross_run_edits` test and
15`ONESTORE_CROSS_RUN_OUTPUT` set to a new absolute directory. Capture each candidate
16with `tools/native_runner.py --expected-pages 1 --collect-notebook` and compare
17with `tools/verify-document.py`. The manifest records source paths, target IDs,
18UTF-16 ranges and file hashes. The attachment export links to its canonical copy.
19`tools/test_cross_run_edit.py` checks the retained
20native oracles without starting a VM.
corpus/cross-run-edit/boundary/candidate/synthetic.one created
Binary files /dev/null and b/corpus/cross-run-edit/boundary/candidate/synthetic.one differ
corpus/cross-run-edit/boundary/cold/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/cross-run-edit/boundary/cold/notebook/Open Notebook.onetoc2 differ
corpus/cross-run-edit/boundary/cold/notebook/synthetic.one created
Binary files /dev/null and b/corpus/cross-run-edit/boundary/cold/notebook/synthetic.one differ
corpus/cross-run-edit/boundary/cold/read/environment.json created+7
......@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-107F0FF0",
5 "cold": true,
6 "onenote": "14.0.4763.1000"
7}
corpus/cross-run-edit/boundary/cold/read/hierarchy.xml created+2
......@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{2055F8D1-7921-41C1-9C97-7A20D9C33AB3}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-08T07:07:16.000Z" color="#B49EDE"><one:Section name="synthetic" ID="{22E2CA5A-AB75-0C2F-27EE-1002E7557596}{1}{B0}" path="C:\one-tests\runs\capture\notebook\synthetic.one" lastModifiedTime="2026-09-08T07:07:16.000Z" color="#8AA8E4"><one:Page ID="{0EAB39DC-22BE-4F3E-A089-B8931B6D6989}{1}{B0}" name="BoldMiddleitalic é color 東京" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-08T07:07:16.000Z" pageLevel="1"/></one:Section></one:Notebook>
corpus/cross-run-edit/boundary/cold/read/page-000.xml created+12
......@@ -0,0 +1,12 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{0EAB39DC-22BE-4F3E-A089-B8931B6D6989}{1}{B0}" name="BoldMiddleitalic é color 東京" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-08T07:07:16.000Z" pageLevel="1" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="Rich author" lastModifiedBy="Rich author" lastModifiedTime="2026-09-08T07:02:08.000Z" objectID="{69679160-69CB-4C88-AF7B-3448E4F18CE8}{10}{B0}"><one:Position x="36.0" y="14.40000057220459" z="0"/><one:Size width="127.5136947631836" height="41.15543365478516"/><one:OEChildren><one:OE creationTime="2026-09-08T04:47:24.000Z" lastModifiedTime="2026-09-08T07:02:08.000Z" objectID="{6E023997-9AB6-0782-1F2F-F7422F9AFE1E}{1}{B0}" alignment="left" style="font-family:Calibri;font-size:11.0pt"><one:T><![CDATA[<span
3style='font-weight:bold;text-decoration:underline'>Bold</span>Middle<span
4style='font-style:italic;text-decoration:line-through'>italic</span> <span
5style='vertical-align:super'>é</span> <span style='color:#0C2238;background:
6yellow;mso-highlight:yellow'>color</span> <span style='vertical-align:sub'>東京</span><br>
7End]]></one:T></one:OE><one:OE author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" creationTime="2026-09-05T05:06:55.000Z" lastModifiedTime="2026-09-05T05:06:56.000Z" objectID="{69679160-69CB-4C88-AF7B-3448E4F18CE8}{11}{B0}" alignment="left" quickStyleIndex="0" style="font-size:11.0pt;color:#1F4E79"><one:T><![CDATA[<span
8style='font-weight:bold;font-family:Calibri' lang=en-US>Fictitious: café, </span><span
9style='font-weight:bold;font-family:SimSun' lang=en-US>東京</span><span
10style='font-weight:bold;font-family:Calibri' lang=en-US>, </span><span
11style='font-weight:bold;font-family:Arial;direction:rtl;unicode-bidi:embed'
12lang=ar-SA>مرحبا</span>]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/cross-run-edit/boundary/cold/read/payloads.json created+3
......@@ -0,0 +1,3 @@
1[
2
3]
\ No newline at end of file
corpus/cross-run-edit/boundary/cold/run.json created+18
......@@ -0,0 +1,18 @@
1{
2 "notebook": "/Users/clo/dev/one/evidence/m10/cross-run-native/boundary/candidate",
3 "expected_pages": 1,
4 "author": null,
5 "author_timeout_seconds": 600,
6 "inspect": false,
7 "collect_notebook": true,
8 "base": {
9 "file": "win7-office-base.qcow2",
10 "format": "qcow2",
11 "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346",
12 "virtual_size": 68719476736
13 },
14 "scripts": {
15 "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331",
16 "read.ps1": "04013bfcccee40a17e2a225f9b9e96f40a3a8daad9e350609654f8eda3ccbb41"
17 }
18}
corpus/cross-run-edit/boundary/cold/source.json created+8
......@@ -0,0 +1,8 @@
1[
2 {
3 "path": "synthetic.one",
4 "bytes": 20840,
5 "sha256": "a3fac402a10648e99d4bdb4b37b41c825e2b8b34f12d8b1c234e28bd2ec8ac0b",
6 "mtime_ns": 1788850928468116429
7 }
8]
corpus/cross-run-edit/boundary/cold/teardown.json created+1
......@@ -0,0 +1 @@
1{"absent": true}
corpus/cross-run-edit/clear/candidate/synthetic.one created
Binary files /dev/null and b/corpus/cross-run-edit/clear/candidate/synthetic.one differ
corpus/cross-run-edit/clear/cold/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/cross-run-edit/clear/cold/notebook/Open Notebook.onetoc2 differ
corpus/cross-run-edit/clear/cold/notebook/synthetic.one created
Binary files /dev/null and b/corpus/cross-run-edit/clear/cold/notebook/synthetic.one differ
corpus/cross-run-edit/clear/cold/read/environment.json created+7
......@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-8787CA35",
5 "cold": true,
6 "onenote": "14.0.4763.1000"
7}
corpus/cross-run-edit/clear/cold/read/hierarchy.xml created+2
......@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{DB8815C1-7E0F-4425-AF09-7274689810EC}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-08T07:07:14.000Z" color="#B49EDE"><one:Section name="synthetic" ID="{0858E4C2-01A9-068A-13A5-8233A106496B}{1}{B0}" path="C:\one-tests\runs\capture\notebook\synthetic.one" lastModifiedTime="2026-09-08T07:07:14.000Z" color="#8AA8E4"><one:Page ID="{24111744-8862-459B-94C2-2AA25D3E5574}{1}{B0}" name="Fictitious: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-08T07:07:14.000Z" pageLevel="1"/></one:Section></one:Notebook>
corpus/cross-run-edit/clear/cold/read/page-000.xml created+7
......@@ -0,0 +1,7 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{24111744-8862-459B-94C2-2AA25D3E5574}{1}{B0}" name="Fictitious: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-08T07:07:14.000Z" pageLevel="1" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="Rich author" lastModifiedBy="Rich author" lastModifiedTime="2026-09-08T07:02:08.000Z" objectID="{43DDBFF8-C317-462D-9B30-A679A2A2B015}{10}{B0}"><one:Position x="36.0" y="14.40000057220459" z="0"/><one:Size width="127.5136947631836" height="27.72771453857422"/><one:OEChildren><one:OE creationTime="2026-09-08T04:47:24.000Z" lastModifiedTime="2026-09-08T07:02:08.000Z" objectID="{44B8170F-306A-0D27-2B64-657369C9C2E3}{1}{B0}" alignment="left" style="font-family:Calibri;font-size:11.0pt"><one:T><![CDATA[]]></one:T></one:OE><one:OE author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" creationTime="2026-09-05T05:06:55.000Z" lastModifiedTime="2026-09-05T05:06:56.000Z" objectID="{43DDBFF8-C317-462D-9B30-A679A2A2B015}{11}{B0}" alignment="left" quickStyleIndex="0" style="font-size:11.0pt;color:#1F4E79"><one:T><![CDATA[<span
3style='font-weight:bold;font-family:Calibri' lang=en-US>Fictitious: café, </span><span
4style='font-weight:bold;font-family:SimSun' lang=en-US>東京</span><span
5style='font-weight:bold;font-family:Calibri' lang=en-US>, </span><span
6style='font-weight:bold;font-family:Arial;direction:rtl;unicode-bidi:embed'
7lang=ar-SA>مرحبا</span>]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/cross-run-edit/clear/cold/read/payloads.json created+3
......@@ -0,0 +1,3 @@
1[
2
3]
\ No newline at end of file
corpus/cross-run-edit/clear/cold/run.json created+18
......@@ -0,0 +1,18 @@
1{
2 "notebook": "/Users/clo/dev/one/evidence/m10/cross-run-native/clear/candidate",
3 "expected_pages": 1,
4 "author": null,
5 "author_timeout_seconds": 600,
6 "inspect": false,
7 "collect_notebook": true,
8 "base": {
9 "file": "win7-office-base.qcow2",
10 "format": "qcow2",
11 "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346",
12 "virtual_size": 68719476736
13 },
14 "scripts": {
15 "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331",
16 "read.ps1": "04013bfcccee40a17e2a225f9b9e96f40a3a8daad9e350609654f8eda3ccbb41"
17 }
18}
corpus/cross-run-edit/clear/cold/source.json created+8
......@@ -0,0 +1,8 @@
1[
2 {
3 "path": "synthetic.one",
4 "bytes": 21736,
5 "sha256": "d6d8c775b4b87e3b8379757773c313a0a720da52dd5e76b4799f2b8e16bb5320",
6 "mtime_ns": 1788850928454071185
7 }
8]
corpus/cross-run-edit/clear/cold/teardown.json created+1
......@@ -0,0 +1 @@
1{"absent": true}
corpus/cross-run-edit/manifest.json created+89
......@@ -0,0 +1,89 @@
1{
2 "generator": "crates/onestore/tests/edit.rs:export_native_cross_run_edits",
3 "cases": [
4 {
5 "name": "partial",
6 "object": "{04B13EBF-F611-4402-829D-C72E23221C8B},2",
7 "range": {
8 "end": 16,
9 "start": 2
10 },
11 "replacement": "\u4e2d\ud83e\udd80",
12 "retype": false,
13 "source": "formatted-insertion/native-paragraph/candidate/synthetic.one",
14 "space": "{64183EF4-4E19-0CBE-3D3B-88FF17D58B1C},1",
15 "candidate_sha256": "0a93fdb8c8378989a6522b5de803f073109b8972088e61afdf97639eeaea6a02",
16 "native_sha256": "85e90850f7f0582cd9e4e8f93bb8f77cd773d883cd993b24280b57bd70078a4f"
17 },
18 {
19 "name": "table",
20 "object": "{6928B2F5-7D36-44AF-BE3B-36CEBACE8657},2",
21 "range": {
22 "end": 24,
23 "start": 0
24 },
25 "replacement": "Across \ud83d\udc08 ",
26 "retype": false,
27 "source": "formatted-insertion/native-cell/candidate/synthetic.one",
28 "space": "{64183EF4-4E19-0CBE-3D3B-88FF17D58B1C},1",
29 "candidate_sha256": "89ce5a50f050c05ecbfb92c8c8eb0538bd041d0596290a5b974a9878e171e6e9",
30 "native_sha256": "561222f0ecb1ccc17592c9fd44f08f6f4e0b0bfc6120f3b099a902f46c3e8ed2"
31 },
32 {
33 "name": "clear",
34 "object": "{04B13EBF-F611-4402-829D-C72E23221C8B},2",
35 "range": {
36 "end": 30,
37 "start": 0
38 },
39 "replacement": "",
40 "retype": false,
41 "source": "formatted-insertion/native-paragraph/candidate/synthetic.one",
42 "space": "{64183EF4-4E19-0CBE-3D3B-88FF17D58B1C},1",
43 "candidate_sha256": "d6d8c775b4b87e3b8379757773c313a0a720da52dd5e76b4799f2b8e16bb5320",
44 "native_sha256": "30dfd26ad6912225f3e637f529f5135f67f45de0c1a0b705ad70bf97866a554e"
45 },
46 {
47 "name": "replace",
48 "object": "{04B13EBF-F611-4402-829D-C72E23221C8B},2",
49 "range": {
50 "end": 30,
51 "start": 0
52 },
53 "replacement": "Replacement e\u0301\ud83e\udd80",
54 "retype": false,
55 "source": "formatted-insertion/native-paragraph/candidate/synthetic.one",
56 "space": "{64183EF4-4E19-0CBE-3D3B-88FF17D58B1C},1",
57 "candidate_sha256": "4b0ae05431ade4777541592cd8040f7ff18dbfe4ada35a686cda963299988e81",
58 "native_sha256": "f0cca32ec56e15b7d19aa581dbe7ae3d59bd0cc498450a0576ae9d168365c377"
59 },
60 {
61 "name": "retype",
62 "object": "{04B13EBF-F611-4402-829D-C72E23221C8B},2",
63 "range": {
64 "end": 30,
65 "start": 0
66 },
67 "replacement": "",
68 "retype": true,
69 "source": "formatted-insertion/native-paragraph/candidate/synthetic.one",
70 "space": "{64183EF4-4E19-0CBE-3D3B-88FF17D58B1C},1",
71 "candidate_sha256": "0d37a5a762e9b08f40ea47b2fad48591fcb5f782d1a130f5363f5116b1175fb3",
72 "native_sha256": "7b70f0ebe27f09c31a623f27b37ffac4362f9513dd26b2d1a69d1e2c0a38c15a"
73 },
74 {
75 "name": "boundary",
76 "object": "{04B13EBF-F611-4402-829D-C72E23221C8B},2",
77 "range": {
78 "end": 8,
79 "start": 4
80 },
81 "replacement": "Middle",
82 "retype": false,
83 "source": "formatted-insertion/native-paragraph/candidate/synthetic.one",
84 "space": "{64183EF4-4E19-0CBE-3D3B-88FF17D58B1C},1",
85 "candidate_sha256": "a3fac402a10648e99d4bdb4b37b41c825e2b8b34f12d8b1c234e28bd2ec8ac0b",
86 "native_sha256": "e9852bed7545600f4fec537be8e614a1cdb4ec8d7ee40c7933a609bd51a7ce8b"
87 }
88 ]
89}
corpus/cross-run-edit/partial/candidate/synthetic.one created
Binary files /dev/null and b/corpus/cross-run-edit/partial/candidate/synthetic.one differ
corpus/cross-run-edit/partial/cold/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/cross-run-edit/partial/cold/notebook/Open Notebook.onetoc2 differ
corpus/cross-run-edit/partial/cold/notebook/synthetic.one created
Binary files /dev/null and b/corpus/cross-run-edit/partial/cold/notebook/synthetic.one differ
corpus/cross-run-edit/partial/cold/read/environment.json created+7
......@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-A348FDB7",
5 "cold": true,
6 "onenote": "14.0.4763.1000"
7}
corpus/cross-run-edit/partial/cold/read/hierarchy.xml created+2
......@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{93F1F719-446A-4165-8228-120BD9DCB466}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-08T07:04:03.000Z" color="#B49EDE"><one:Section name="synthetic" ID="{D1CA9D05-C385-0C6D-079D-1C98BA259D9C}{1}{B0}" path="C:\one-tests\runs\capture\notebook\synthetic.one" lastModifiedTime="2026-09-08T07:04:03.000Z" color="#8AA8E4"><one:Page ID="{FD836E83-4A4E-4F7C-80FA-B409461D8183}{1}{B0}" name="Bo中🦀́ color 東京" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-08T07:04:03.000Z" pageLevel="1"/></one:Section></one:Notebook>
corpus/cross-run-edit/partial/cold/read/page-000.xml created+11
......@@ -0,0 +1,11 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{FD836E83-4A4E-4F7C-80FA-B409461D8183}{1}{B0}" name="Bo中🦀́ color 東京" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-08T07:04:03.000Z" pageLevel="1" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="Rich author" lastModifiedBy="Rich author" lastModifiedTime="2026-09-08T07:02:08.000Z" objectID="{9A4FC63F-013B-4CCA-8F08-38D2B98164E2}{10}{B0}"><one:Position x="36.0" y="14.40000057220459" z="0"/><one:Size width="127.5136947631836" height="41.15543365478516"/><one:OEChildren><one:OE creationTime="2026-09-08T04:47:24.000Z" lastModifiedTime="2026-09-08T07:02:08.000Z" objectID="{9D2A6EC8-F246-07C0-3F5C-FBD872EA1614}{1}{B0}" alignment="left" style="font-family:Calibri;font-size:11.0pt"><one:T><![CDATA[<span
3style='font-weight:bold;text-decoration:underline'>Bo中&#129408;</span><span
4style='vertical-align:super'>́</span> <span style='color:#0C2238;background:
5yellow;mso-highlight:yellow'>color</span> <span style='vertical-align:sub'>東京</span><br>
6End]]></one:T></one:OE><one:OE author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" creationTime="2026-09-05T05:06:55.000Z" lastModifiedTime="2026-09-05T05:06:56.000Z" objectID="{9A4FC63F-013B-4CCA-8F08-38D2B98164E2}{11}{B0}" alignment="left" quickStyleIndex="0" style="font-size:11.0pt;color:#1F4E79"><one:T><![CDATA[<span
7style='font-weight:bold;font-family:Calibri' lang=en-US>Fictitious: café, </span><span
8style='font-weight:bold;font-family:SimSun' lang=en-US>東京</span><span
9style='font-weight:bold;font-family:Calibri' lang=en-US>, </span><span
10style='font-weight:bold;font-family:Arial;direction:rtl;unicode-bidi:embed'
11lang=ar-SA>مرحبا</span>]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/cross-run-edit/partial/cold/read/payloads.json created+3
......@@ -0,0 +1,3 @@
1[
2
3]
\ No newline at end of file
corpus/cross-run-edit/partial/cold/run.json created+18
......@@ -0,0 +1,18 @@
1{
2 "notebook": "/Users/clo/dev/one/evidence/m10/cross-run-native/partial/candidate",
3 "expected_pages": 1,
4 "author": null,
5 "author_timeout_seconds": 600,
6 "inspect": false,
7 "collect_notebook": true,
8 "base": {
9 "file": "win7-office-base.qcow2",
10 "format": "qcow2",
11 "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346",
12 "virtual_size": 68719476736
13 },
14 "scripts": {
15 "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331",
16 "read.ps1": "04013bfcccee40a17e2a225f9b9e96f40a3a8daad9e350609654f8eda3ccbb41"
17 }
18}
corpus/cross-run-edit/partial/cold/source.json created+8
......@@ -0,0 +1,8 @@
1[
2 {
3 "path": "synthetic.one",
4 "bytes": 20944,
5 "sha256": "0a93fdb8c8378989a6522b5de803f073109b8972088e61afdf97639eeaea6a02",
6 "mtime_ns": 1788850928443537242
7 }
8]
corpus/cross-run-edit/partial/cold/teardown.json created+1
......@@ -0,0 +1 @@
1{"absent": true}
corpus/cross-run-edit/replace/candidate/synthetic.one created
Binary files /dev/null and b/corpus/cross-run-edit/replace/candidate/synthetic.one differ
corpus/cross-run-edit/replace/cold/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/cross-run-edit/replace/cold/notebook/Open Notebook.onetoc2 differ
corpus/cross-run-edit/replace/cold/notebook/synthetic.one created
Binary files /dev/null and b/corpus/cross-run-edit/replace/cold/notebook/synthetic.one differ
corpus/cross-run-edit/replace/cold/read/environment.json created+7
......@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-105B87CC",
5 "cold": true,
6 "onenote": "14.0.4763.1000"
7}
corpus/cross-run-edit/replace/cold/read/hierarchy.xml created+2
......@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{0A502D6E-6E90-496E-B55C-03E1AC281CF0}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-08T07:04:07.000Z" color="#B49EDE"><one:Section name="synthetic" ID="{8FC8AEE6-30D3-0B67-02C3-29ADD51A2A5C}{1}{B0}" path="C:\one-tests\runs\capture\notebook\synthetic.one" lastModifiedTime="2026-09-08T07:04:07.000Z" color="#8AA8E4"><one:Page ID="{A3815D60-B918-4876-85A4-813C29223643}{1}{B0}" name="Replacement é🦀" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-08T07:04:07.000Z" pageLevel="1"/></one:Section></one:Notebook>
corpus/cross-run-edit/replace/cold/read/page-000.xml created+8
......@@ -0,0 +1,8 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{A3815D60-B918-4876-85A4-813C29223643}{1}{B0}" name="Replacement é🦀" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-08T07:04:07.000Z" pageLevel="1" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="Rich author" lastModifiedBy="Rich author" lastModifiedTime="2026-09-08T07:02:08.000Z" objectID="{C44DF5DC-F26D-4BC0-8A56-0DE7D6BED322}{10}{B0}"><one:Position x="36.0" y="14.40000057220459" z="0"/><one:Size width="127.5136947631836" height="27.72771453857422"/><one:OEChildren><one:OE creationTime="2026-09-08T04:47:24.000Z" lastModifiedTime="2026-09-08T07:02:08.000Z" objectID="{C3285D2B-0110-00CA-3A02-CEED1DD5A1D4}{1}{B0}" alignment="left" style="font-family:Calibri;font-size:11.0pt"><one:T><![CDATA[<span
3style='font-weight:bold;text-decoration:underline'>Replacement é&#129408;</span>]]></one:T></one:OE><one:OE author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" creationTime="2026-09-05T05:06:55.000Z" lastModifiedTime="2026-09-05T05:06:56.000Z" objectID="{C44DF5DC-F26D-4BC0-8A56-0DE7D6BED322}{11}{B0}" alignment="left" quickStyleIndex="0" style="font-size:11.0pt;color:#1F4E79"><one:T><![CDATA[<span
4style='font-weight:bold;font-family:Calibri' lang=en-US>Fictitious: café, </span><span
5style='font-weight:bold;font-family:SimSun' lang=en-US>東京</span><span
6style='font-weight:bold;font-family:Calibri' lang=en-US>, </span><span
7style='font-weight:bold;font-family:Arial;direction:rtl;unicode-bidi:embed'
8lang=ar-SA>مرحبا</span>]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/cross-run-edit/replace/cold/read/payloads.json created+3
......@@ -0,0 +1,3 @@
1[
2
3]
\ No newline at end of file
corpus/cross-run-edit/replace/cold/run.json created+18
......@@ -0,0 +1,18 @@
1{
2 "notebook": "/Users/clo/dev/one/evidence/m10/cross-run-native/replace/candidate",
3 "expected_pages": 1,
4 "author": null,
5 "author_timeout_seconds": 600,
6 "inspect": false,
7 "collect_notebook": true,
8 "base": {
9 "file": "win7-office-base.qcow2",
10 "format": "qcow2",
11 "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346",
12 "virtual_size": 68719476736
13 },
14 "scripts": {
15 "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331",
16 "read.ps1": "04013bfcccee40a17e2a225f9b9e96f40a3a8daad9e350609654f8eda3ccbb41"
17 }
18}
corpus/cross-run-edit/replace/cold/source.json created+8
......@@ -0,0 +1,8 @@
1[
2 {
3 "path": "synthetic.one",
4 "bytes": 21520,
5 "sha256": "4b0ae05431ade4777541592cd8040f7ff18dbfe4ada35a686cda963299988e81",
6 "mtime_ns": 1788850928458083445
7 }
8]
corpus/cross-run-edit/replace/cold/teardown.json created+1
......@@ -0,0 +1 @@
1{"absent": true}
corpus/cross-run-edit/retype/candidate/synthetic.one created
Binary files /dev/null and b/corpus/cross-run-edit/retype/candidate/synthetic.one differ
corpus/cross-run-edit/retype/cold/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/cross-run-edit/retype/cold/notebook/Open Notebook.onetoc2 differ
corpus/cross-run-edit/retype/cold/notebook/synthetic.one created
Binary files /dev/null and b/corpus/cross-run-edit/retype/cold/notebook/synthetic.one differ
corpus/cross-run-edit/retype/cold/read/environment.json created+7
......@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-71FBAE77",
5 "cold": true,
6 "onenote": "14.0.4763.1000"
7}
corpus/cross-run-edit/retype/cold/read/hierarchy.xml created+2
......@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{2C5C26FC-21E8-467D-BA08-524B57C107FD}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-08T07:07:16.000Z" color="#B49EDE"><one:Section name="synthetic" ID="{1F28C682-3F93-018E-3DDE-73E7F6610C3E}{1}{B0}" path="C:\one-tests\runs\capture\notebook\synthetic.one" lastModifiedTime="2026-09-08T07:07:16.000Z" color="#8AA8E4"><one:Page ID="{33613504-B658-429F-BAB9-DB760A591021}{1}{B0}" name="Retyped 🦀" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-08T07:07:16.000Z" pageLevel="1"/></one:Section></one:Notebook>
corpus/cross-run-edit/retype/cold/read/page-000.xml created+7
......@@ -0,0 +1,7 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{33613504-B658-429F-BAB9-DB760A591021}{1}{B0}" name="Retyped 🦀" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-08T07:07:16.000Z" pageLevel="1" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="Rich author" lastModifiedBy="Rich author" lastModifiedTime="2026-09-08T07:02:08.000Z" objectID="{54AD9DB8-FD2D-4129-B54B-57ADF5C5F540}{10}{B0}"><one:Position x="36.0" y="14.40000057220459" z="0"/><one:Size width="127.5136947631836" height="27.72771453857422"/><one:OEChildren><one:OE creationTime="2026-09-08T04:47:24.000Z" lastModifiedTime="2026-09-08T07:02:08.000Z" objectID="{53C8354F-0E50-0A23-051F-94A73EAE87B6}{1}{B0}" alignment="left" style="font-family:Calibri;font-size:11.0pt"><one:T><![CDATA[Retyped &#129408;]]></one:T></one:OE><one:OE author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" creationTime="2026-09-05T05:06:55.000Z" lastModifiedTime="2026-09-05T05:06:56.000Z" objectID="{54AD9DB8-FD2D-4129-B54B-57ADF5C5F540}{11}{B0}" alignment="left" quickStyleIndex="0" style="font-size:11.0pt;color:#1F4E79"><one:T><![CDATA[<span
3style='font-weight:bold;font-family:Calibri' lang=en-US>Fictitious: café, </span><span
4style='font-weight:bold;font-family:SimSun' lang=en-US>東京</span><span
5style='font-weight:bold;font-family:Calibri' lang=en-US>, </span><span
6style='font-weight:bold;font-family:Arial;direction:rtl;unicode-bidi:embed'
7lang=ar-SA>مرحبا</span>]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/cross-run-edit/retype/cold/read/payloads.json created+3
......@@ -0,0 +1,3 @@
1[
2
3]
\ No newline at end of file
corpus/cross-run-edit/retype/cold/run.json created+18
......@@ -0,0 +1,18 @@
1{
2 "notebook": "/Users/clo/dev/one/evidence/m10/cross-run-native/retype/candidate",
3 "expected_pages": 1,
4 "author": null,
5 "author_timeout_seconds": 600,
6 "inspect": false,
7 "collect_notebook": true,
8 "base": {
9 "file": "win7-office-base.qcow2",
10 "format": "qcow2",
11 "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346",
12 "virtual_size": 68719476736
13 },
14 "scripts": {
15 "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331",
16 "read.ps1": "04013bfcccee40a17e2a225f9b9e96f40a3a8daad9e350609654f8eda3ccbb41"
17 }
18}
corpus/cross-run-edit/retype/cold/source.json created+8
......@@ -0,0 +1,8 @@
1[
2 {
3 "path": "synthetic.one",
4 "bytes": 22832,
5 "sha256": "0d37a5a762e9b08f40ea47b2fad48591fcb5f782d1a130f5363f5116b1175fb3",
6 "mtime_ns": 1788850928464340419
7 }
8]
corpus/cross-run-edit/retype/cold/teardown.json created+1
......@@ -0,0 +1 @@
1{"absent": true}
corpus/cross-run-edit/table/candidate/synthetic.one created
Binary files /dev/null and b/corpus/cross-run-edit/table/candidate/synthetic.one differ
corpus/cross-run-edit/table/cold/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/cross-run-edit/table/cold/notebook/Open Notebook.onetoc2 differ
corpus/cross-run-edit/table/cold/notebook/synthetic.one created
Binary files /dev/null and b/corpus/cross-run-edit/table/cold/notebook/synthetic.one differ
corpus/cross-run-edit/table/cold/read/af1af4b8987109ce1e335e538910fb3860b7c57eb3b6110a06aa2114e06d69a7.attachment created+1
......@@ -0,0 +1 @@
1../../../../append/round-01/native/append-01-complex/read/af1af4b8987109ce1e335e538910fb3860b7c57eb3b6110a06aa2114e06d69a7.attachment
\ No newline at end of file
corpus/cross-run-edit/table/cold/read/environment.json created+7
......@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-46174C24",
5 "cold": true,
6 "onenote": "14.0.4763.1000"
7}
corpus/cross-run-edit/table/cold/read/hierarchy.xml created+2
......@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{9F6E2E81-C2FD-4BB7-A300-60EF3CF93913}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-08T07:04:07.000Z" color="#B49EDE"><one:Section name="synthetic" ID="{756534DD-2F0E-0901-12BA-F33A7AD1A925}{1}{B0}" path="C:\one-tests\runs\capture\notebook\synthetic.one" lastModifiedTime="2026-09-08T07:04:07.000Z" color="#8AA8E4"><one:Page ID="{592CC75B-A6C5-4A10-95DD-5BAB86E9B53A}{1}{B0}" name="Fictitious: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-08T07:04:07.000Z" pageLevel="1"/></one:Section></one:Notebook>
corpus/cross-run-edit/table/cold/read/page-000.xml created+11
......@@ -0,0 +1,11 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{592CC75B-A6C5-4A10-95DD-5BAB86E9B53A}{1}{B0}" name="Fictitious: café, 東京, مرحبا" dateTime="2026-09-05T05:06:54.000Z" lastModifiedTime="2026-09-08T07:04:07.000Z" pageLevel="1" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-05T05:06:56.000Z" objectID="{3EE06FE7-EDB0-49A6-9A2F-D7707975505B}{10}{B0}"><one:Position x="36.0" y="14.40000057220459" z="0"/><one:Size width="127.5136947631836" height="14.30000114440918"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:55.000Z" lastModifiedTime="2026-09-05T05:06:56.000Z" objectID="{3EE06FE7-EDB0-49A6-9A2F-D7707975505B}{11}{B0}" alignment="left" quickStyleIndex="0" style="font-size:11.0pt;color:#1F4E79"><one:T><![CDATA[<span
3style='font-weight:bold;font-family:Calibri' lang=en-US>Fictitious: café, </span><span
4style='font-weight:bold;font-family:SimSun' lang=en-US>東京</span><span
5style='font-weight:bold;font-family:Calibri' lang=en-US>, </span><span
6style='font-weight:bold;font-family:Arial;direction:rtl;unicode-bidi:embed'
7lang=ar-SA>مرحبا</span>]]></one:T></one:OE></one:OEChildren></one:Outline><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-05T05:06:57.000Z" objectID="{6850B3F2-5EE5-43FB-8099-55F8DD79B412}{10}{B0}"><one:Position x="144.0" y="96.0" z="1"/><one:Size width="167.0449523925781" height="13.42771339416504"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:57.000Z" lastModifiedTime="2026-09-05T05:06:57.000Z" objectID="{6850B3F2-5EE5-43FB-8099-55F8DD79B412}{11}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Fictitious positioned outline.]]></one:T></one:OE></one:OEChildren></one:Outline><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-05T05:06:58.000Z" objectID="{5579DB7D-B87F-4A55-B6E9-FE0C3114FC9C}{10}{B0}"><one:Position x="144.0" y="192.0" z="2"/><one:Size width="72.0" height="0.750005722045898"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:58.000Z" lastModifiedTime="2026-09-05T05:06:58.000Z" objectID="{5579DB7D-B87F-4A55-B6E9-FE0C3114FC9C}{11}{B0}" alignment="left"><one:Image format="png" originalPageNumber="0"><one:Data>iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAusB9Y9J1uoA
8AAAASUVORK5CYII=</one:Data></one:Image></one:OE></one:OEChildren></one:Outline><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-05T05:06:59.000Z" objectID="{D063C31D-7E07-4993-9344-99F205C533AF}{10}{B0}"><one:Position x="264.0" y="192.0" z="3"/><one:Size width="72.00001525878906" height="63.0"/><one:OEChildren><one:OE creationTime="2026-09-05T05:06:59.000Z" lastModifiedTime="2026-09-05T05:06:59.000Z" objectID="{D063C31D-7E07-4993-9344-99F205C533AF}{11}{B0}" alignment="left"><one:InsertedFile pathCache="C:\Users\clover\AppData\Local\Microsoft\OneNote\14.0\OneNoteOfflineCache_Files\458001d8-0f1e-451f-81cb-621828fc0b39.txt" pathSource="C:\one-tests\runs\20260905-05\assets\fictitious-attachment.txt" preferredName="fictitious-attachment.txt"/></one:OE></one:OEChildren></one:Outline><one:Outline author="snow" authorInitials="S" lastModifiedBy="snow" lastModifiedByInitials="S" lastModifiedTime="2026-09-08T07:02:08.000Z" objectID="{FEBCBA4D-1F97-4213-8C4C-7BA72D96C8E9}{10}{B0}"><one:Position x="144.0" y="312.0" z="4"/><one:Size width="125.6063766479492" height="48.80315780639648"/><one:OEChildren><one:OE creationTime="2026-09-05T05:07:00.000Z" lastModifiedTime="2026-09-08T07:02:08.000Z" objectID="{FEBCBA4D-1F97-4213-8C4C-7BA72D96C8E9}{11}{B0}" alignment="left"><one:Table bordersVisible="true" lastModifiedTime="2026-09-08T07:02:08.000Z" objectID="{FEBCBA4D-1F97-4213-8C4C-7BA72D96C8E9}{12}{B0}"><one:Columns><one:Column index="0" width="72.33071136474609"/><one:Column index="1" width="45.14567184448242"/></one:Columns><one:Row objectID="{FEBCBA4D-1F97-4213-8C4C-7BA72D96C8E9}{13}{B0}" lastModifiedTime="2026-09-08T07:02:08.000Z"><one:Cell lastModifiedTime="2026-09-08T07:02:08.000Z" objectID="{FEBCBA4D-1F97-4213-8C4C-7BA72D96C8E9}{14}{B0}" lastModifiedByInitials="S"><one:OEChildren><one:OE creationTime="2026-09-05T05:07:00.000Z" lastModifiedTime="2026-09-05T05:07:00.000Z" objectID="{FEBCBA4D-1F97-4213-8C4C-7BA72D96C8E9}{17}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Left cell]]></one:T></one:OE><one:OE author="Rich author" lastModifiedBy="Rich author" creationTime="2026-09-08T04:47:24.000Z" lastModifiedTime="2026-09-08T07:02:08.000Z" objectID="{541C4B5A-95EA-0201-16DD-E59A2BF2B871}{1}{B0}" alignment="left" style="font-family:Calibri;font-size:11.0pt"><one:T><![CDATA[<span
9style='font-weight:bold;text-decoration:underline'>Across &#128008; </span><span
10style='vertical-align:sub'>東京</span><br>
11End]]></one:T></one:OE></one:OEChildren></one:Cell><one:Cell lastModifiedTime="2026-09-05T05:07:00.000Z" objectID="{FEBCBA4D-1F97-4213-8C4C-7BA72D96C8E9}{22}{B0}" lastModifiedByInitials="S"><one:OEChildren><one:OE creationTime="2026-09-05T05:07:00.000Z" lastModifiedTime="2026-09-05T05:07:00.000Z" objectID="{FEBCBA4D-1F97-4213-8C4C-7BA72D96C8E9}{25}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Right cell]]></one:T></one:OE></one:OEChildren></one:Cell></one:Row></one:Table></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/cross-run-edit/table/cold/read/payloads.json created+10
......@@ -0,0 +1,10 @@
1[
2 {
3 "sha256": "af1af4b8987109ce1e335e538910fb3860b7c57eb3b6110a06aa2114e06d69a7",
4 "name": "fictitious-attachment.txt",
5 "object": "{D063C31D-7E07-4993-9344-99F205C533AF}{11}{B0}",
6 "kind": "InsertedFile",
7 "page": "{592CC75B-A6C5-4A10-95DD-5BAB86E9B53A}{1}{B0}",
8 "bytes": 43
9 }
10]
\ No newline at end of file
corpus/cross-run-edit/table/cold/run.json created+18
......@@ -0,0 +1,18 @@
1{
2 "notebook": "/Users/clo/dev/one/evidence/m10/cross-run-native/table/candidate",
3 "expected_pages": 1,
4 "author": null,
5 "author_timeout_seconds": 600,
6 "inspect": false,
7 "collect_notebook": true,
8 "base": {
9 "file": "win7-office-base.qcow2",
10 "format": "qcow2",
11 "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346",
12 "virtual_size": 68719476736
13 },
14 "scripts": {
15 "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331",
16 "read.ps1": "04013bfcccee40a17e2a225f9b9e96f40a3a8daad9e350609654f8eda3ccbb41"
17 }
18}
corpus/cross-run-edit/table/cold/source.json created+8
......@@ -0,0 +1,8 @@
1[
2 {
3 "path": "synthetic.one",
4 "bytes": 30864,
5 "sha256": "89ce5a50f050c05ecbfb92c8c8eb0538bd041d0596290a5b974a9878e171e6e9",
6 "mtime_ns": 1788850928450254092
7 }
8]
corpus/cross-run-edit/table/cold/teardown.json created+1
......@@ -0,0 +1 @@
1{"absent": true}
crates/onestore-offline/examples/smb_offline_client.rs+23-6
......@@ -23,6 +23,8 @@ mod support {
2323use concurrent::document_view;
2424use support::view::view;
2525
26const DOCUMENT_OPERATIONS: [&str; 3] = ["insert", "format", "text"];
27
2628fn now() -> u128 {
2729 SystemTime::now()
2830 .duration_since(UNIX_EPOCH)
......@@ -275,7 +277,7 @@ fn queue_document(
275277 operation: usize,
276278 parent: Option<ExGuid>,
277279 deadline: Instant,
278) -> Result<(ExGuid, [u64; 2]), Box<dyn std::error::Error>> {
280) -> Result<(ExGuid, [u64; DOCUMENT_OPERATIONS.len()]), Box<dyn std::error::Error>> {
279281 let source = cache.snapshot()?;
280282 let store = Store::parse(&source)?;
281283 let index = RevisionIndex::parse(&store)?;
......@@ -320,8 +322,15 @@ fn queue_document(
320322 TextAttribute::FontSize(18.0 + (operation % 9) as f32),
321323 TextAttribute::Color(Some([0x12, 0x34, 0x56])),
322324 ];
323 let mut ids = [0; 2];
325 let mut ids = [0; DOCUMENT_OPERATIONS.len()];
324326 for (step, id) in ids.iter_mut().enumerate() {
327 let kind = DOCUMENT_OPERATIONS[step];
328 let range = if step == 2 {
329 let end = u32::try_from(text.encode_utf16().count())?;
330 end - 3..end
331 } else {
332 range.clone()
333 };
325334 loop {
326335 if Instant::now() >= deadline {
327336 return Err("Document queue timed out; cache retained".into());
......@@ -330,7 +339,7 @@ fn queue_document(
330339 let started = now();
331340 let result = if step == 0 {
332341 cache.insert(&source, space, &insertion)
333 } else {
342 } else if step == 1 {
334343 cache.format(
335344 &source,
336345 space,
......@@ -338,13 +347,21 @@ fn queue_document(
338347 range.clone(),
339348 &attributes,
340349 )
350 } else {
351 cache.edit_text(
352 &source,
353 space,
354 insertion.text_object(),
355 range.clone(),
356 " e\u{301}🐈",
357 )
341358 };
342359 match result {
343360 Ok(Some(acknowledged)) => {
344361 *id = acknowledged;
345362 println!(
346363 "{}",
347 json!({"event":"local_document_commit","id":acknowledged,"operation":operation,"kind":if step==0 {"insert"} else {"format"},"space":space.to_string(),"object":insertion.text_object().to_string(),"text":text,"insertion":if step==0 {Some(&insertion)} else {None},"range":[range.start,range.end],"attributes":attributes,"started_us":started,"finished_us":now()})
364 json!({"event":"local_document_commit","id":acknowledged,"operation":operation,"kind":kind,"space":space.to_string(),"object":insertion.text_object().to_string(),"text":text,"insertion":if step==0 {Some(&insertion)} else {None},"range":[range.start,range.end],"attributes":attributes,"replacement":if step==2 {Some(" e\u{301}🐈")} else {None},"started_us":started,"finished_us":now()})
348365 );
349366 break;
350367 }
......@@ -400,7 +417,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
400417 let cache = Arc::new(Replica::create(&cache_path, &source)?);
401418 println!(
402419 "{}",
403 json!({"event":"ready", "pid":std::process::id(), "actor":args[2], "offline":true, "document_operations":documents})
420 json!({"event":"ready", "pid":std::process::id(), "actor":args[2], "offline":true, "document_operations":documents,"document_kinds":if documents {DOCUMENT_OPERATIONS.as_slice()} else {&[]}})
404421 );
405422 while !Path::new(&args[4]).exists() {
406423 if Instant::now() >= deadline {
......@@ -470,7 +487,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
470487 {
471488 1
472489 } else if documents && outage.is_some() {
473 24
490 8 * (1 + DOCUMENT_OPERATIONS.len())
474491 } else {
475492 8
476493 };
crates/onestore-offline/src/lib.rs+1
......@@ -180,6 +180,7 @@ impl Replica {
180180
181181 /// Atomically records an intent and its resulting local image; returns its durable ID.
182182 /// Unchanged text returns `None`. A stale image returns `Io(ResourceBusy)`.
183 /// On synchronization, replacement text inherits the remote style at the rebased start.
183184 /// After a database error, reopen and inspect the cache before retrying the edit.
184185 pub fn edit_text(
185186 &self,
crates/onestore-offline/tests/sync.rs+76
......@@ -1888,6 +1888,82 @@ fn uncertain_insertions_reconcile_the_original_revision_without_duplicate_object
18881888 }
18891889}
18901890
1891#[test]
1892fn cross_run_text_rebases_preserve_remote_styles_and_survive_lost_replies() {
1893 use onestore::TextAttribute as A;
1894 for fault in [Fault::None, Fault::UnknownAfter, Fault::PanicAfter] {
1895 let directory = tempfile::tempdir().unwrap();
1896 let path = directory.path().join("cross-run.sqlite");
1897 let plain = onestore::create_section("cross.one", "ab🦀cde\u{301}fg", "Author").unwrap();
1898 let (sid, oid, _) = text(&plain);
1899 let bold = PreparedEdit::format(&plain, sid, oid, 1..4, &[A::Bold(true)]).unwrap();
1900 let source = PreparedEdit::format(bold.as_bytes(), sid, oid, 4..7, &[A::Italic(true)])
1901 .unwrap()
1902 .as_bytes()
1903 .to_vec();
1904 let cache = Replica::create(&path, &source).unwrap();
1905 let first = cache
1906 .edit_text(&source, sid, oid, 2..7, "日本語")
1907 .unwrap()
1908 .unwrap();
1909 let local = cache.snapshot().unwrap();
1910 let second = cache
1911 .edit_text(&local, sid, oid, 3..4, "🐈")
1912 .unwrap()
1913 .unwrap();
1914 let pending = cache.pending().unwrap();
1915 drop(cache);
1916 let cache = Replica::open(&path).unwrap();
1917 assert_eq!(cache.pending().unwrap(), pending);
1918 let prefix = PreparedEdit::text(&source, sid, oid, 0..0, "Prefix ").unwrap();
1919 let remote =
1920 PreparedEdit::format(prefix.as_bytes(), sid, oid, 9..11, &[A::Underline(true)])
1921 .unwrap();
1922 let mut server = Server::new(remote.as_bytes());
1923 server.fault = fault;
1924 let attempted = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
1925 cache.sync_once(&mut server)
1926 }));
1927 if matches!(fault, Fault::None) {
1928 assert!(
1929 matches!(attempted.unwrap().unwrap(), Some((id, EditStatus::Published { .. })) if id == first)
1930 );
1931 } else {
1932 assert!(matches!(
1933 cache.status(first).unwrap(),
1934 Some(EditStatus::AwaitingConfirmation { .. })
1935 ));
1936 }
1937 drop(cache);
1938 let cache = Replica::open(&path).unwrap();
1939 while let Some((_, status)) = cache.sync_once(&mut server).unwrap() {
1940 assert!(matches!(status, EditStatus::Published { .. }));
1941 }
1942 assert_eq!(server.publications, 2);
1943 assert!(matches!(
1944 cache.status(second).unwrap(),
1945 Some(EditStatus::Published { .. })
1946 ));
1947 assert_eq!(text(&server.durable).2, "Prefix ab日🐈語\u{301}fg");
1948 assert_eq!(cache.snapshot().unwrap(), server.durable);
1949 let store = Store::parse(&server.durable).unwrap();
1950 let index = RevisionIndex::parse(&store).unwrap();
1951 let document = Document::parse(&index).unwrap();
1952 let space = &document.spaces[&sid];
1953 let revision = &space.revisions[&space.contexts[&ExGuid::default()]];
1954 let runs = revision.text_runs(oid).unwrap();
1955 let replacement = runs.iter().find(|run| run.text == "日🐈語").unwrap();
1956 assert_eq!(replacement.format.bold, Some(true));
1957 assert_eq!(replacement.format.underline, Some(true));
1958 assert_ne!(replacement.format.italic, Some(true));
1959 let suffix = runs.last().unwrap();
1960 assert_eq!(suffix.text, "\u{301}fg");
1961 assert_ne!(suffix.format.bold, Some(true));
1962 assert_ne!(suffix.format.underline, Some(true));
1963 assert_ne!(suffix.format.italic, Some(true));
1964 }
1965}
1966
18911967#[test]
18921968fn offline_formatting_rebases_text_and_merges_independent_attributes() {
18931969 use onestore::TextAttribute as A;
crates/onestore/README.md+1-1
......@@ -47,7 +47,7 @@ harness also accepts `--client-profile release`.
4747| `create_section` | Create one page containing one plain-text paragraph and an author, including Unicode |
4848| `create_table_of_contents` | Create ordered section entries from filenames and file identities |
4949| `replace_property_bytes` | Append one scalar-property revision; preserve prior revisions and unrelated property values and references |
50| `replace_text`, `commit_text`, `commit_file_text` | Replace a UTF-16 range within one ordinary text run; publish text, run boundaries and modification time together |
50| `replace_text`, `commit_text`, `commit_file_text` | Replace a UTF-16 range across ordinary text runs; publish text, run boundaries and modification time together |
5151| `Insertion`, `PreparedEdit::insert` | Insert paragraphs into editable containers or positioned outlines into a page, retaining intent identities across rebases |
5252| `TextAttribute`, `PreparedEdit::format` | Change character formatting over a UTF-16 range while sharing immutable styles; preserve unselected runs |
5353| `PreparedEdit::commit`, `PreparedEdit::commit_file` | Publish the exact prepared image under caller-held exclusion or the conservative filesystem adapter |
crates/onestore/src/edit.rs+64-29
......@@ -18,9 +18,11 @@ pub(crate) fn automatic_title(text: &str) -> &str {
1818 line
1919}
2020
21/// Replaces UTF-16 character positions within one ordinary text run.
22/// The inserted text inherits that run's formatting; other runs retain their identities.
21/// Replaces UTF-16 character positions across ordinary text runs.
22/// Inserted text inherits the run at the start; surviving text retains its formatting.
2323/// Insertion at a run boundary uses the following run, except at the end of text.
24/// The final run retains its insertion style even when emptied.
25/// Replacing a range with identical text leaves its existing formatting unchanged.
2426/// Returns a complete file image without I/O; use `commit_file_text` to update an existing file.
2527pub fn replace_text(
2628 source: &[u8],
......@@ -79,22 +81,25 @@ pub fn replace_text(
7981 }
8082 let selected = runs
8183 .iter()
82 .rposition(|run| run.start <= range.start && range.end <= run.end)
83 .ok_or_else(|| invalid("The edit must stay within one text run"))?;
84 .rposition(|run| run.start <= range.start && range.start <= run.end)
85 .ok_or_else(|| invalid("The edit range exceeds the text"))?;
8486 let resolved = revision.text_runs(object)?;
85 let format = &resolved[selected].format;
86 if [
87 format.hidden,
88 format.hyperlink,
89 format.math,
90 format.embedded_object,
91 ]
92 .contains(&Some(true))
93 || runs[selected]
94 .extra_set
95 .is_some_and(|set| !node.extra[set].is_empty())
96 {
97 return Err(invalid("This text run contains a field or embedded data"));
87 for (i, run) in runs.iter().enumerate() {
88 if i == selected || (run.start < range.end && range.start < run.end) {
89 let format = &resolved[i].format;
90 if [
91 format.hidden,
92 format.hyperlink,
93 format.math,
94 format.embedded_object,
95 ]
96 .contains(&Some(true))
97 || resolved[i].text.contains(['\u{fffc}', '\u{fddf}'])
98 || run.extra_set.is_some_and(|set| !node.extra[set].is_empty())
99 {
100 return Err(invalid("This text run contains a field or embedded data"));
101 }
102 }
98103 }
99104 let ObjectData::Properties(blob) = raw.objects[&object].data else {
100105 unreachable!()
......@@ -141,20 +146,47 @@ pub fn replace_text(
141146 .checked_sub(removed)
142147 .and_then(|n| n.checked_add(added))
143148 .ok_or_else(|| invalid("Edited text exceeds the UTF-16 offset range"))?;
144 let mut boundaries = Vec::new();
145 let mut previous = None;
146 for (i, run) in runs[..runs.len() - 1].iter().enumerate() {
147 let end = if i >= selected {
148 run.end - removed + added
149 } else {
150 run.end
151 };
152 if previous.is_some_and(|p| p >= end) {
153 return Err(invalid("The edit would collapse a formatting boundary"));
149 let mut segments = Vec::new();
150 let mut position = 0;
151 for (i, run) in runs.iter().enumerate() {
152 let length = run.end.min(range.start).saturating_sub(run.start)
153 + run.end.saturating_sub(run.start.max(range.end))
154 + if i == selected { added } else { 0 };
155 let untouched = i != selected && (run.end <= range.start || range.end <= run.start);
156 if length > 0 || i == runs.len() - 1 || untouched {
157 position += length;
158 segments.push((i, position));
154159 }
155 previous = Some(end);
156 boundaries.extend_from_slice(&end.to_le_bytes());
157160 }
161 if segments.len() != runs.len() && properties.sets[0].iter().any(|p| p.id == 0x40003499) {
162 return Err(invalid("Text edits cannot remove preserved run data"));
163 }
164 if segments[..segments.len() - 1]
165 .windows(2)
166 .any(|pair| pair[0].1 >= pair[1].1)
167 {
168 return Err(invalid("Text-run boundaries must be strictly increasing"));
169 }
170 let boundaries: Vec<_> = segments[..segments.len() - 1]
171 .iter()
172 .flat_map(|(_, end)| end.to_le_bytes())
173 .collect();
174 let formats = properties.sets[0]
175 .iter()
176 .find(|p| p.id == 0x24001e13)
177 .map(|p| {
178 let crate::Value::References { compact_ids, .. } = p.value else {
179 unreachable!()
180 };
181 if compact_ids.is_empty() {
182 Vec::new()
183 } else {
184 segments
185 .iter()
186 .flat_map(|(i, _)| compact_ids[i * 4..i * 4 + 4].iter().copied())
187 .collect()
188 }
189 });
158190 let mut changed = String::with_capacity(text.len() - (end - start) + replacement.len());
159191 changed.push_str(&text[..start]);
160192 changed.push_str(replacement);
......@@ -183,6 +215,9 @@ pub fn replace_text(
183215 if properties.sets[0].iter().any(|p| p.id == 0x1c001e12) {
184216 updates.push((0x1c001e12, &boundaries));
185217 }
218 if let Some(formats) = &formats {
219 updates.push((0x24001e13, formats));
220 }
186221 let modified = crate::create::current_timestamps()?.0.to_le_bytes();
187222 updates.push((0x14001d7a, &modified));
188223 let mut edits = vec![crate::write::ObjectEdit {
crates/onestore/src/write/tests.rs+50-7
......@@ -589,7 +589,7 @@ fn character_formatting_preserves_inheritance_and_associated_data() {
589589 .find_map(|(id, o)| (o.jcid == 0x6000e).then_some((*sid, *id)))
590590 })
591591 .unwrap();
592 for variant in 0..7 {
592 for variant in 0..10 {
593593 let fixture = write_revision(&source, sid, |raw| {
594594 let mut target = PropertyObject::from_object(&raw.objects[&text])?;
595595 target.bytes = properties(&[
......@@ -617,8 +617,17 @@ fn character_formatting_preserves_inheritance_and_associated_data() {
617617 let reference = target.reference(author)?;
618618 target.set(&[(0x24003458, &reference)])?;
619619 }
620 3 => {
621 // An unknown nested run property must survive a style-only edit byte for byte.
620 3 | 7 | 8 | 9 => {
621 if variant != 3 {
622 let ends = if variant == 9 { [2_u32, 2] } else { [2, 4] };
623 target.set(&[(
624 0x1c001e12,
625 &ends
626 .into_iter()
627 .flat_map(u32::to_le_bytes)
628 .collect::<Vec<_>>(),
629 )])?;
630 }
622631 let parsed = PropertySets::parse(&target.bytes)?;
623632 let at = parsed.root_ids.as_ptr().addr() - target.bytes.as_ptr().addr();
624633 let count = parsed.sets[0].len();
......@@ -626,11 +635,17 @@ fn character_formatting_preserves_inheritance_and_associated_data() {
626635 let mut bytes = target.bytes[..end].to_vec();
627636 bytes[at - 2..at].copy_from_slice(&((count + 1) as u16).to_le_bytes());
628637 bytes.splice(at + count * 4..at + count * 4, 0x40003499_u32.to_le_bytes());
629 bytes.extend_from_slice(&1_u32.to_le_bytes());
638 let runs: u32 = if variant == 3 { 1 } else { 3 };
639 bytes.extend_from_slice(&runs.to_le_bytes());
630640 bytes.extend_from_slice(&0x44001234_u32.to_le_bytes());
631 bytes.extend_from_slice(&1_u16.to_le_bytes());
632 bytes.extend_from_slice(&0x14001234_u32.to_le_bytes());
633 bytes.extend_from_slice(&0xdeadbeef_u32.to_le_bytes());
641 for run in 0..runs {
642 let populated = (variant == 3 || variant == 8) && run == runs - 1;
643 bytes.extend_from_slice(&u16::from(populated).to_le_bytes());
644 if populated {
645 bytes.extend_from_slice(&0x14001234_u32.to_le_bytes());
646 bytes.extend_from_slice(&0xdeadbeef_u32.to_le_bytes());
647 }
648 }
634649 bytes.resize(bytes.len().next_multiple_of(8), 0);
635650 target.bytes = bytes;
636651 }
......@@ -639,6 +654,34 @@ fn character_formatting_preserves_inheritance_and_associated_data() {
639654 Ok(BTreeMap::from([(text, target)]))
640655 })
641656 .unwrap();
657 if variant == 9 {
658 assert!(PreparedEdit::text(&fixture, sid, text, 0..0, "x").is_err());
659 continue;
660 }
661 if variant >= 7 {
662 let edit = PreparedEdit::text(&fixture, sid, text, 1..3, "longer").unwrap();
663 let [before, after] = [&fixture, edit.as_bytes()].map(|bytes| {
664 let store = Store::parse(bytes).unwrap();
665 let index = RevisionIndex::parse(&store).unwrap();
666 let document = Document::parse(&index).unwrap();
667 let space = &document.spaces[&sid];
668 let view = &space.revisions[&space.contexts[&ExGuid::default()]];
669 let Kind::RichText { runs, .. } = &view.nodes[&text].kind else {
670 panic!()
671 };
672 assert_eq!(runs.len(), 3);
673 format!("{:?}", view.nodes[&text].extra)
674 });
675 assert_eq!(before, after);
676 assert!(PreparedEdit::text(&fixture, sid, text, 0..4, "x").is_err());
677 assert_eq!(
678 PreparedEdit::text(&fixture, sid, text, 3..5, "x").is_ok(),
679 variant == 7
680 );
681 continue;
682 }
683 let text_edit = PreparedEdit::text(&fixture, sid, text, 1..5, "x");
684 assert_eq!(text_edit.is_ok(), variant == 0);
642685 let edit = PreparedEdit::format(&fixture, sid, text, 0..6, &[TextAttribute::Bold(true)]);
643686 if matches!(variant, 1 | 2 | 4 | 5 | 6) {
644687 assert!(edit.is_err());
crates/onestore/tests/edit.rs+177-3
......@@ -368,9 +368,10 @@ fn invalid_text_edits_never_touch_storage() {
368368 for (range, replacement) in [
369369 (1..1, "x"),
370370 (0..999, "x"),
371 (0..21, "x"),
372371 (0..0, "\n"),
373372 (0..0, "\0"),
373 (0..0, "\u{fffc}"),
374 (0..0, "\u{fddf}"),
374375 ] {
375376 let mut disk = Disk {
376377 visible: source.clone(),
......@@ -388,6 +389,179 @@ fn invalid_text_edits_never_touch_storage() {
388389 }
389390}
390391
392#[test]
393fn cross_run_edits_reject_native_fields_before_storage_io() {
394 let source = include_bytes!("../../../corpus/m6/native-probes-01/notebook/synthetic.one");
395 let store = Store::parse(source).unwrap();
396 let index = RevisionIndex::parse(&store).unwrap();
397 let document = Document::parse(&index).unwrap();
398 let mut fields = 0;
399 for (sid, space) in &document.spaces {
400 let revision = &space.revisions[&space.contexts[&ExGuid::default()]];
401 for (oid, node) in &revision.nodes {
402 let Kind::RichText { text, runs, .. } = &node.kind else {
403 continue;
404 };
405 let resolved = revision.text_runs(*oid).unwrap();
406 for (run, resolved) in runs.iter().zip(resolved) {
407 if resolved.format.hyperlink != Some(true) {
408 continue;
409 }
410 fields += 1;
411 for range in [0..run.end, run.start..text.encode_utf16().count() as u32] {
412 let mut disk = Disk {
413 visible: source.to_vec(),
414 durable: source.to_vec(),
415 operation: 0,
416 fail_at: None,
417 write_limit: 17,
418 random: 9,
419 };
420 let error = onestore::commit_text(&mut disk, source, *sid, *oid, range, "x")
421 .unwrap_err();
422 assert_eq!(error.state, CommitState::NotCommitted);
423 assert_eq!(disk.operation, 0);
424 assert_eq!(disk.durable, source);
425 }
426 }
427 }
428 }
429 assert!(fields > 0);
430}
431
432#[test]
433fn cross_run_splices_match_a_character_model_and_preserve_history() {
434 let (sid, oid) = target(SOURCE);
435 let characters = |source: &[u8]| {
436 let runs = text_runs(source, sid, oid);
437 let runs = runs.as_array().unwrap();
438 let mut result = Vec::new();
439 for run in runs {
440 result.extend(
441 run["text"]
442 .as_str()
443 .unwrap()
444 .chars()
445 .map(|c| (c, run["format"].clone())),
446 );
447 }
448 result.push(('\0', runs.last().unwrap()["format"].clone()));
449 result
450 };
451 let before = characters(SOURCE);
452 let offsets: Vec<_> = std::iter::once(0)
453 .chain(before[..before.len() - 1].iter().scan(0, |at, (c, _)| {
454 *at += c.len_utf16() as u32;
455 Some(*at)
456 }))
457 .collect();
458 let store = Store::parse(SOURCE).unwrap();
459 let index = RevisionIndex::parse(&store).unwrap();
460 for a in 0..before.len() {
461 for b in a..before.len() {
462 let replacement = ["", "🦀e\u{301}", "日本語"][b % 3];
463 let mut expected = before.clone();
464 expected.splice(a..b, replacement.chars().map(|c| (c, before[a].1.clone())));
465 let edited =
466 onestore::replace_text(SOURCE, sid, oid, offsets[a]..offsets[b], replacement)
467 .unwrap();
468 assert_eq!(characters(&edited), expected, "characters {a}..{b}");
469 let current_store = Store::parse(&edited).unwrap();
470 assert_eq!(
471 current_store.header.transaction_count,
472 store.header.transaction_count
473 + usize::from(a != b || !replacement.is_empty()) as u32
474 );
475 let current = RevisionIndex::parse(&current_store).unwrap();
476 for (space, history) in &index.spaces {
477 for revision in history.revisions.keys() {
478 let old = index.resolve(*space, *revision).unwrap();
479 let retained = current.resolve(*space, *revision).unwrap();
480 assert_eq!(old.roots, retained.roots);
481 for (id, object) in old.objects {
482 assert_eq!(object.data, retained.objects[&id].data);
483 }
484 }
485 }
486 }
487 }
488 let cleared =
489 onestore::replace_text(SOURCE, sid, oid, 0..*offsets.last().unwrap(), "").unwrap();
490 let typed = onestore::replace_text(&cleared, sid, oid, 0..0, "new").unwrap();
491 let format = &before.last().unwrap().1;
492 assert_eq!(
493 characters(&typed),
494 "new\0"
495 .chars()
496 .map(|c| (c, format.clone()))
497 .collect::<Vec<_>>()
498 );
499 let text: String = before[..before.len() - 1].iter().map(|(c, _)| *c).collect();
500 assert_eq!(
501 onestore::replace_text(SOURCE, sid, oid, 0..*offsets.last().unwrap(), &text).unwrap(),
502 SOURCE
503 );
504}
505
506#[test]
507#[ignore = "exports cross-run edits for independent native validation"]
508fn export_native_cross_run_edits() {
509 use std::{fs, path::PathBuf};
510 let output = PathBuf::from(std::env::var_os("ONESTORE_CROSS_RUN_OUTPUT").unwrap());
511 assert!(output.is_absolute());
512 fs::create_dir(&output).unwrap();
513 let mut manifest = Vec::new();
514 for (name, fixture, range, replacement, retype) in [
515 ("partial", "native-paragraph", 2..16, "中🦀", false),
516 ("table", "native-cell", 0..24, "Across 🐈 ", false),
517 ("clear", "native-paragraph", 0..u32::MAX, "", false),
518 (
519 "replace",
520 "native-paragraph",
521 0..u32::MAX,
522 "Replacement e\u{301}🦀",
523 false,
524 ),
525 ("retype", "native-paragraph", 0..u32::MAX, "", true),
526 ("boundary", "native-paragraph", 4..8, "Middle", false),
527 ] {
528 let input = format!("../../corpus/formatted-insertion/{fixture}/candidate/synthetic.one");
529 let source = fs::read(&input).unwrap();
530 let store = Store::parse(&source).unwrap();
531 let index = RevisionIndex::parse(&store).unwrap();
532 let document = Document::parse(&index).unwrap();
533 let (sid, oid, length) = document
534 .spaces
535 .iter()
536 .find_map(|(sid, space)| {
537 let view = &space.revisions[&space.contexts[&ExGuid::default()]];
538 view.nodes.iter().find_map(|(oid, node)| {
539 let Kind::RichText { text, .. } = &node.kind else {
540 return None;
541 };
542 text.starts_with("Bold ")
543 .then(|| (*sid, *oid, text.encode_utf16().count() as u32))
544 })
545 })
546 .unwrap();
547 let range = range.start..range.end.min(length);
548 let mut edited =
549 onestore::replace_text(&source, sid, oid, range.clone(), replacement).unwrap();
550 if retype {
551 edited = onestore::replace_text(&edited, sid, oid, 0..0, "Retyped 🦀").unwrap();
552 }
553 let candidate = output.join(name).join("candidate");
554 fs::create_dir_all(&candidate).unwrap();
555 fs::write(candidate.join("synthetic.one"), &edited).unwrap();
556 manifest.push(serde_json::json!({"name":name,"source":input,"space":sid,"object":oid,"range":range,"replacement":replacement,"retype":retype}));
557 }
558 fs::write(
559 output.join("manifest.json"),
560 serde_json::to_vec_pretty(&manifest).unwrap(),
561 )
562 .unwrap();
563}
564
391565#[test]
392566fn insertion_at_a_style_boundary_uses_the_following_style() {
393567 let (sid, oid) = target(SOURCE);
......@@ -701,14 +875,14 @@ fn interrupted_text_commits_never_publish_mismatched_run_boundaries() {
701875 };
702876 let before = text_runs(source, sid, oid);
703877 let mut success = make_disk(None, 1);
704 onestore::commit_text(&mut success, source, sid, oid, 0..10, "🐈 mixed edit").unwrap();
878 onestore::commit_text(&mut success, source, sid, oid, 0..23, "🐈 mixed edit").unwrap();
705879 let after = text_runs(&success.durable, sid, oid);
706880 assert_ne!(before, after);
707881 for at in source.len().div_ceil(193)..=success.operation {
708882 for seed in [1, 42] {
709883 let mut disk = make_disk(Some(at), seed);
710884 let error =
711 onestore::commit_text(&mut disk, source, sid, oid, 0..10, "🐈 mixed edit")
885 onestore::commit_text(&mut disk, source, sid, oid, 0..23, "🐈 mixed edit")
712886 .unwrap_err();
713887 let observed = text_runs(&disk.durable, sid, oid);
714888 match error.state {
fuzz/fuzz_targets/edit_text.rs+18-2
......@@ -129,7 +129,7 @@ fuzz_target!(|input: &[u8]| {
129129 let (source, sid, oid) = &CASES[usize::from(input.last().copied().unwrap_or(0)) % CASES.len()];
130130 let (sid, oid) = (*sid, *oid);
131131 let mut persisted_source = source.clone();
132 let mut caches = std::array::from_fn::<_, 6, _>(|_| source.clone());
132 let mut caches = std::array::from_fn::<_, 12, _>(|_| source.clone());
133133 for step in input.chunks_exact(8).take(16) {
134134 let actor = usize::from(step[6]) % caches.len();
135135 if step[7] % 3 == 0 {
......@@ -154,7 +154,14 @@ fuzz_target!(|input: &[u8]| {
154154 let replacement = ["", "a", "🦀e\u{301}", "日本語", "\n", "\0"][usize::from(step[2]) % 6];
155155 let mut expected = before.clone();
156156 let format = before[start].1.clone();
157 expected.splice(start..end, replacement.chars().map(|c| (c, format.clone())));
157 if before[start..end]
158 .iter()
159 .map(|(c, _)| *c)
160 .collect::<String>()
161 != replacement
162 {
163 expected.splice(start..end, replacement.chars().map(|c| (c, format.clone())));
164 }
158165 let mut storage = Disk {
159166 visible: persisted_source.clone(),
160167 durable: persisted_source.clone(),
......@@ -173,6 +180,15 @@ fuzz_target!(|input: &[u8]| {
173180 replacement,
174181 );
175182 let persisted = characters(&storage.durable, sid, oid);
183 if !replacement.contains(['\n', '\0']) {
184 assert!(
185 result
186 .as_ref()
187 .err()
188 .is_none_or(|error| error.error.kind() != std::io::ErrorKind::InvalidData),
189 "Valid ordinary text edit was rejected: {result:?}"
190 );
191 }
176192 match result {
177193 Ok(()) => assert_eq!(persisted, expected),
178194 Err(error) => match error.state {
tools/native_collaboration.py+8
......@@ -54,6 +54,14 @@ def replay(output, server, stress_clients=0, stress_operations=30, sync_every=1,
5454 raise ValueError('Choose a new Linux VM name; existing machines are not owned by this run.')
5555 output = output.resolve()
5656 output.mkdir(parents=True, exist_ok=False)
57 for profile in (('debug', 'release') if client_profile == 'release' else ('debug',)):
58 build = ['cargo', 'build', '--locked', '--workspace', '--all-features', '--examples']
59 if profile == 'release': build.append('--release')
60 with (output / f'build-{profile}.log').open('w') as log:
61 result = subprocess.run(build, cwd=ROOT, env={**os.environ, 'CARGO_TARGET_DIR': str(ROOT / 'target')},
62 stdout=log, stderr=subprocess.STDOUT)
63 (output / f'build-{profile}.json').write_text(json.dumps({'command': build, 'exit': result.returncode}, indent=2))
64 result.check_returncode()
5765 mount = output / 'mount'
5866 mount.mkdir()
5967 scripts = output / 'scripts'
tools/native_stress.py+5-5
......@@ -96,7 +96,7 @@ def verify_capture(output, capture):
9696 if config.get('document_operations'):
9797 from offline_document_history import document_history
9898 documents = document_history(logs, operations)
99 expected.extend(document['text'] for document in documents.values())
99 expected.extend(''.join(char for char, *_ in list(document['states'].values())[-1]['characters']) for document in documents.values())
100100 page_file, = capture.glob('page-*.xml')
101101 page = ET.parse(page_file).getroot()
102102 paragraphs = native_characters(page, page.findall('one:Outline', ns))
......@@ -111,8 +111,8 @@ def verify_capture(output, capture):
111111 checks += 1
112112 if documents:
113113 from offline_document_history import verify_native
114 checks += verify_native(paragraphs, documents)
115 return {'rust_intents': len(commits), 'document_intents': len(documents)*2, 'native_intents': sum(map(len, native)),
114 checks += verify_native(paragraphs, (list(document['states'].values())[-1]['characters'] for document in documents.values()))
115 return {'rust_intents': len(commits), 'document_intents': sum(len(document['states']) for document in documents.values()), 'native_intents': sum(map(len, native)),
116116 'exact_paragraphs': len(expected), 'native_intended_format_checks': checks}
117117
118118
......@@ -231,7 +231,7 @@ def exercise(output, shared, clients, action, wait_action, wait_text, checkpoint
231231 if config.get('document_operations'):
232232 from offline_document_history import document_history
233233 documents = document_history(rust, operations)
234 expected.extend(document['text'] for document in documents.values())
234 expected.extend(''.join(char for char, *_ in list(document['states'].values())[-1]['characters']) for document in documents.values())
235235 for client in clients: wait_text(client, expected)
236236 checkpoint('stress-final')
237237 model = json.loads((output / 'stress-final/model/document.json').read_text())
......@@ -262,7 +262,7 @@ def exercise(output, shared, clients, action, wait_action, wait_text, checkpoint
262262 earliest_end = (native['updated_ticks'] - 621355968000000000) // 10 - high
263263 overlap += sum(max(latest_start, rust['started_us']) < min(earliest_end, rust['finished_us']) for rust in commits)
264264 result = {'native_writers': len(clients), 'rust_writers': rust_writers, 'rust_readers': rust_readers,
265 'sync_every': sync_every, 'edit': edit, 'seed': seed, 'offline': config.get('offline', False), 'native_edits': operations * len(clients), 'rust_commits': len(commits), 'document_commits': len(documents)*2, 'rust_reads': len(reads),
265 'sync_every': sync_every, 'edit': edit, 'seed': seed, 'offline': config.get('offline', False), 'native_edits': operations * len(clients), 'rust_commits': len(commits), 'document_commits': sum(len(document['states']) for document in documents.values()), 'rust_reads': len(reads),
266266 'clock_bounded_native_rust_call_overlaps': overlap, 'converged_paragraphs': len(expected)}
267267 (output / 'result.json').write_text(json.dumps(result, indent=2))
268268 assert overlap, 'No native/Rust call overlap established within clock uncertainty'
tools/offline_document_history.py+42-20
......@@ -15,14 +15,21 @@ def characters(observed):
1515 return actual
1616
1717
18def operation_kinds(events):
19 kinds = tuple(events[0].get('document_kinds', ('insert', 'format')))
20 assert kinds in (('insert', 'format'), ('insert', 'format', 'text')), 'Unknown document workload'
21 return kinds
22
23
1824def document_history(logs, operations):
1925 documents = {}
2026 for actor, events in logs.items():
2127 if not actor.startswith('w'): continue
2228 assert events[0].get('document_operations') is True, 'Writer omitted document operations'
29 kinds = operation_kinds(events)
2330 edits = [row for row in events if row['event'] == 'local_document_commit']
2431 assert [(row['operation'], row['kind']) for row in edits] == [
25 (i, kind) for i in range(operations) for kind in ('insert', 'format')], 'Missing or duplicate document intent'
32 (i, kind) for i in range(operations) for kind in kinds], 'Missing or duplicate document intent'
2633 ids = [row['id'] for row in edits]
2734 assert ids == sorted(set(ids)), 'Document intent IDs are duplicated or unordered'
2835 assert not set(ids) & {row['id'] for row in events if row['event'] == 'local_commit'}, 'Text and document intents share an ID'
......@@ -64,7 +71,8 @@ def document_history(logs, operations):
6471 else:
6572 assert receipt['revision'] == attempt['revision']
6673 linked[intent['id']] = {**attempt, 'acknowledged_us': receipt['at_us'], 'receipt_revision': receipt['revision']}
67 for inserted, formatted in zip(edits[::2], edits[1::2], strict=True):
74 for at in range(0, len(edits), len(kinds)):
75 inserted, formatted, *replaced = edits[at:at + len(kinds)]
6876 number = inserted['operation']
6977 text = f'Document {actor}:{number} 🦀'
7078 insertion = inserted['insertion']
......@@ -76,7 +84,7 @@ def document_history(logs, operations):
7684 assert insertion['placement'] == {'Outline': {'x': 144 + int(actor[1:]) * 240, 'y': 144 + number * 72}}, 'Outline placement differs from intent'
7785 else:
7886 assert insertion['placement'] == {'Paragraph': {'before': None}}
79 assert insertion['parent'] == identity(edits[(number-1)*2]['insertion'], 1), 'Paragraph lost its outline parent'
87 assert insertion['parent'] == identity(edits[(number-1)*len(kinds)]['insertion'], 1), 'Paragraph lost its outline parent'
8088 assert formatted['range'] == [1, len(text.encode('utf-16-le')) // 2 - 2]
8189 assert formatted['attributes'] == [{'Bold': True}, {'FontSize': 18 + number % 9}, {'Color': [18, 52, 86]}]
8290 old = [(char, False, 11, 0xff000000) for char in text]
......@@ -91,8 +99,20 @@ def document_history(logs, operations):
9199 assert created['finished_us'] <= changed['started_us'], 'Formatting preceded its insertion'
92100 assert characters(created['documents'][target]) == old, 'Insertion publication differs from its local intent'
93101 assert characters(changed['documents'][target]) == new, 'Formatting publication differs from its local intent'
94 documents[target] = {'text': text, 'insertion': insertion, 'space': inserted['space'],
95 'old': old, 'new': new, 'insert': created, 'format': changed}
102 states = {'insert': {'characters': old, 'attempt': created},
103 'format': {'characters': new, 'attempt': changed}}
104 if replaced:
105 replacement, = replaced
106 end = len(text.encode('utf-16-le')) // 2
107 assert replacement['object'] == target and replacement['space'] == inserted['space'], 'Text edit addresses another object'
108 assert replacement['text'] == text and replacement['range'] == [end-3, end], 'Cross-run text range differs from workload'
109 assert replacement['replacement'] == ' e\u0301🐈', 'Cross-run replacement differs from workload'
110 final = new[:-2] + [(char, *new[-2][1:]) for char in replacement['replacement']]
111 attempt = linked[replacement['id']]
112 assert changed['finished_us'] <= attempt['started_us'], 'Text replacement preceded its formatting'
113 assert characters(attempt['documents'][target]) == final, 'Text publication differs from its local intent'
114 states['text'] = {'characters': final, 'attempt': attempt}
115 documents[target] = {'insertion': insertion, 'space': inserted['space'], 'states': states}
96116 assert documents, 'No document operations were recorded'
97117 for actor, events in logs.items():
98118 previous = {}
......@@ -103,19 +123,20 @@ def document_history(logs, operations):
103123 observed = read['documents']
104124 assert set(previous) <= set(observed) <= set(documents), 'Reader lost an object or observed an unrecorded insertion'
105125 for target, document in documents.items():
106 if read['started_us'] > document['insert']['acknowledged_us']:
126 states = list(document['states'].values())
127 if read['started_us'] > states[0]['attempt']['acknowledged_us']:
107128 assert target in observed, 'Reader missed an acknowledged insertion'
108129 if target not in observed: continue
109 assert read['finished_us'] >= document['insert']['started_us'], 'Reader observed a future insertion'
110130 actual = characters(observed[target])
111 assert actual in (document['old'], document['new']), 'Reader observed partial or invented formatting'
112 formatted = actual == document['new']
113 assert not previous.get(target, False) or formatted, 'Reader reverted acknowledged formatting'
114 if read['started_us'] > document['format']['acknowledged_us']:
115 assert formatted, 'Reader missed acknowledged formatting'
116 if formatted:
117 assert read['finished_us'] >= document['format']['started_us'], 'Reader observed future formatting'
118 previous[target] = formatted
131 matches = [i for i, state in enumerate(states) if actual == state['characters']]
132 assert len(matches) == 1, 'Reader observed partial or invented document content'
133 current, = matches
134 assert current >= previous.get(target, 0), 'Reader reverted document content'
135 assert read['finished_us'] >= states[current]['attempt']['started_us'], 'Reader observed future document content'
136 for i, state in enumerate(states):
137 if read['started_us'] > state['attempt']['acknowledged_us']:
138 assert current >= i, 'Reader missed acknowledged document content'
139 previous[target] = current
119140 return documents
120141
121142
......@@ -138,7 +159,8 @@ def verify_model(model, documents):
138159 found.add(target)
139160 expected = documents[target]
140161 insertion = expected['insertion']
141 assert sid == expected['space'] and node['kind']['text'] == expected['text']
162 final = list(expected['states'].values())[-1]['characters']
163 assert sid == expected['space'] and node['kind']['text'] == ''.join(char for char, *_ in final)
142164 object_id = identity(insertion, 1)
143165 assert object_id in nodes[insertion['parent']]['children'], 'Insertion lost its parent'
144166 paragraph = identity(insertion, 3) if 'Outline' in insertion['placement'] else object_id
......@@ -150,13 +172,13 @@ def verify_model(model, documents):
150172 assert found == set(documents), 'Final model omitted an inserted object'
151173
152174
153def verify_native(paragraphs, documents):
175def verify_native(paragraphs, expected):
154176 from PIL import ImageColor
155177 by_text = {''.join(char for char, _ in paragraph): paragraph for paragraph in paragraphs}
156178 checks = 0
157 for document in documents.values():
158 actual = by_text[document['text']]
159 for (char, style), (wanted, bold, size, color) in zip(actual, document['new'], strict=True):
179 for final in expected:
180 actual = by_text[''.join(char for char, *_ in final)]
181 for (char, style), (wanted, bold, size, color) in zip(actual, final, strict=True):
160182 assert char == wanted and bool(style.get('bold')) == bold, 'Native text or bold differs from intent'
161183 assert style.get('font_size', 11) == size, 'Native font size differs from intent'
162184 native_color = style.get('color', 'automatic')
tools/offline_outage.py+9-7
......@@ -8,6 +8,7 @@ import time
88from native_runner import windows
99import linux_vm
1010from verify_smb_overlap import verify
11from offline_document_history import operation_kinds
1112
1213
1314def interrupt(output, clients, sequences, processes):
......@@ -104,7 +105,7 @@ def interrupt(output, clients, sequences, processes):
104105 return
105106 wait_for(lambda: all((folder / f'offline-paused-{actor}').exists() for actor in writers)
106107 and (not config.get('document_operations') or all(
107 sum(row['event'] == 'local_document_commit' for row in logs()[actor]) == 2 for actor in writers))
108 sum(row['event'] == 'local_document_commit' for row in logs()[actor]) == len(operation_kinds(logs()[actor])) for actor in writers))
108109 and all(any(row['event'] == 'read' for row in logs()[actor]) for actor in readers),
109110 'Clients did not reach the pre-publication outage barrier')
110111 samples['native_before'] = native_counts('before')
......@@ -115,7 +116,7 @@ def interrupt(output, clients, sequences, processes):
115116 (folder / 'offline-outage-down').touch()
116117 wait_for(lambda: all(sum(row['event'] == 'local_commit' for row in events) == 8 for actor, events in logs().items() if actor in writers)
117118 and (not config.get('document_operations') or all(
118 sum(row['event'] == 'local_document_commit' for row in logs()[actor]) == 16 for actor in writers))
119 sum(row['event'] == 'local_document_commit' for row in logs()[actor]) == 8 * len(operation_kinds(logs()[actor])) for actor in writers))
119120 and all(sum(row['event'] == 'transport_read_error' for row in logs()[actor]) > samples['reader_errors_before'][actor] for actor in readers),
120121 'Local queues or disconnected readers failed to progress during the outage')
121122 time.sleep(3)
......@@ -185,16 +186,17 @@ def verify_outage(output):
185186 receipts = [row for row in events if row['event'] == 'remote_receipt']
186187 assert len(receipts) == config['stress_operations'] and all(row['at_us'] > up for row in receipts)
187188 if config.get('document_operations'):
189 kinds = operation_kinds(events)
188190 edits = [row for row in events if row['event'] == 'local_document_commit']
189191 assert [(row['operation'], row['kind']) for row in edits] == [
190 (operation, kind) for operation in range(config['stress_operations']) for kind in ('insert', 'format')]
191 assert all(row['finished_us'] < down for row in edits[:2]), 'Initial document edits missed the outage barrier'
192 (operation, kind) for operation in range(config['stress_operations']) for kind in kinds]
193 assert all(row['finished_us'] < down for row in edits[:len(kinds)]), 'Initial document edits missed the outage barrier'
192194 queued = [row for row in edits if down < row['started_us'] <= row['finished_us'] < up]
193195 assert [(row['operation'], row['kind']) for row in queued] == [
194 (operation, kind) for operation in range(1, 8) for kind in ('insert', 'format')], 'Document edits did not persist during the outage'
196 (operation, kind) for operation in range(1, 8) for kind in kinds], 'Document edits did not persist during the outage'
195197 queues[actor] += len(queued)
196198 receipts = [row for row in events if row['event'] == 'document_receipt']
197 assert len(receipts) == config['stress_operations'] * 2 and all(row['at_us'] > up for row in receipts)
199 assert len(receipts) == config['stress_operations'] * len(kinds) and all(row['at_us'] > up for row in receipts)
198200 trace = [json.loads(line) for line in (output / 'smb-trace.jsonl').read_text().splitlines()]
199201 controls = [row['control'] for row in trace if row.get('control', {}).get('phase', '').startswith('offline-')]
200202 assert controls == [{'phase': 'offline-down', 'mode': 'down'}, {'phase': 'offline-reconnected'}], 'Unexpected outage control sequence'
......@@ -233,7 +235,7 @@ def verify_lost_reply(output):
233235 receipts = [row for row in logs[actor] if row['event'] in ('remote_receipt', 'document_receipt') and row['revision'] == attempt['revision']]
234236 if not receipts and documents:
235237 target, = attempt['document_changes']
236 confirmed_revision = documents[target]['format']['receipt_revision']
238 confirmed_revision = documents[target]['states']['format']['attempt']['receipt_revision']
237239 receipts = [row for row in logs[actor] if row['event'] == 'document_receipt' and row['revision'] == confirmed_revision]
238240 assert len(receipts) == 1 and receipts[0]['at_us'] > sample['up_started_us']
239241 peer_progress = {}
tools/test_cross_run_edit.py created+25
......@@ -0,0 +1,25 @@
1import json
2from pathlib import Path
3import runpy
4import shutil
5from tempfile import TemporaryDirectory
6import unittest
7
8ROOT = Path(__file__).resolve().parent.parent
9FIXTURE = ROOT / 'corpus/cross-run-edit'
10compare = runpy.run_path(str(ROOT / 'tools/verify-document.py'))['compare']
11
12
13class CrossRunEditTest(unittest.TestCase):
14 def test_cold_native_text_styles_and_structure(self):
15 for case in json.loads((FIXTURE / 'manifest.json').read_text())['cases']:
16 with self.subTest(case=case['name']), TemporaryDirectory() as temporary:
17 fixture = FIXTURE / case['name']
18 native = Path(temporary) / 'read'
19 shutil.copytree(fixture / 'cold/read', native)
20 compare(fixture / 'candidate', native)
21 compare(fixture / 'cold/notebook', native)
22
23
24if __name__ == '__main__':
25 unittest.main()
tools/test_native_runner.py+21
......@@ -1,4 +1,5 @@
11import json
2import subprocess
23from pathlib import Path
34import tempfile
45import unittest
......@@ -8,6 +9,26 @@ import native_runner as runner
89
910
1011class NativeRunnerTest(unittest.TestCase):
12 def test_failed_client_build_cannot_start_a_lab_with_stale_binaries(self):
13 import native_collaboration as collaboration
14 with tempfile.TemporaryDirectory() as temporary:
15 root = Path(temporary)
16 for profile, failed in (('debug', 'debug'), ('release', 'debug'), ('release', 'release')):
17 output = root / (profile + '-' + failed)
18 results = [subprocess.CompletedProcess(['cargo'], 1)]
19 if failed == 'release': results.insert(0, subprocess.CompletedProcess(['cargo'], 0))
20 with patch.object(collaboration, 'linux_vm') as linux, \
21 patch.object(collaboration.subprocess, 'run', side_effect=results) as build:
22 linux.instance_path.return_value = root / 'absent'
23 with self.assertRaises(subprocess.CalledProcessError):
24 collaboration.replay(output, 'fixture', client_profile=profile)
25 self.assertEqual(build.call_count, len(results))
26 linux.create_instance.assert_not_called()
27 linux.launch.assert_not_called()
28 receipt = json.loads((output / f'build-{failed}.json').read_text())
29 self.assertEqual(receipt['exit'], 1)
30 self.assertEqual('--release' in receipt['command'], failed == 'release')
31
1132 def test_shutdown_timeout_still_removes_the_owned_clone(self):
1233 with tempfile.TemporaryDirectory() as temporary:
1334 output = Path(temporary)
tools/test_offline_confirmation.py+2-2
......@@ -62,14 +62,14 @@ class ConfirmationOracle(unittest.TestCase):
6262 markup = '<one:OE><one:T><![CDATA[<span style="font-weight:bold;font-size:18pt;color:#123456">x</span>]]></one:T></one:OE>'
6363 content = xml('Concurrent edits: [w0:0]', 'Native 0:').replace('</one:OEChildren>', markup + '</one:OEChildren>')
6464 page.write_text(content)
65 with patch('offline_document_history.document_history', return_value={'target': {'text': 'x', 'format': {'receipt_revision': 'revision'}}}) as documents:
65 with patch('offline_document_history.document_history', return_value={'target': {'states': {'format': {'attempt': {'receipt_revision': 'revision'}}}}}) as documents:
6666 result = verify(root, cold)
6767 self.assertEqual(result['validated_paragraphs'], 3)
6868 self.assertEqual(result['native_intended_format_checks'], 3)
6969 documents.assert_called_once_with({'w0': rows}, 1)
7070 rows[3]['revisions'] = {'space': ['current']}
7171 rows[3]['current_revisions'] = {'space': 'current'}
72 documents.return_value['target']['format']['receipt_revision'] = 'current'
72 documents.return_value['target']['states']['format']['attempt']['receipt_revision'] = 'current'
7373 (root / 'rust/w0.jsonl').write_text('\n'.join(map(json.dumps, rows)))
7474 self.assertEqual(verify(root, cold)['confirmed_revision'], 'current')
7575 rows[3]['current_revisions'] = {'space': 'unrelated'}
tools/test_offline_document_history.py+83-42
......@@ -6,52 +6,92 @@ from unittest.mock import patch
66from offline_document_history import document_history, identity, verify_model, verify_native
77
88
9def history(text_edits=False):
10 logs = {'w0': [{'event': 'ready', 'document_operations': True, 'document_kinds': ['insert', 'format', 'text'] if text_edits else ['insert', 'format']}]}
11 events = logs['w0']
12 observed = {}
13 previous = None
14 for operation in range(2):
15 insertion = {'guid': list(uuid.UUID(int=operation+1).bytes_le), 'text': f'Document w0:{operation} 🦀',
16 'parent': 'page' if operation == 0 else identity(previous, 1), 'author': 'Offline document writer',
17 'placement': {'Outline': {'x': 144, 'y': 144}} if operation == 0 else {'Paragraph': {'before': None}}}
18 previous = insertion
19 target = identity(insertion, 2)
20 for step, kind in enumerate(('insert', 'format', 'text') if text_edits else ('insert', 'format')):
21 timestamp = 10 + operation*30 + step*10
22 local_id = operation*3 + step + 2
23 event = {'event': 'local_document_commit', 'id': local_id, 'operation': operation, 'kind': kind,
24 'space': 'space', 'object': target, 'text': insertion['text'], 'insertion': insertion if step == 0 else None,
25 'range': [1, len(insertion['text'].encode('utf-16-le'))//2-2],
26 'attributes': [{'Bold': True}, {'FontSize': 18+operation}, {'Color': [18, 52, 86]}],
27 'started_us': timestamp-2, 'finished_us': timestamp-1}
28 if kind == 'text':
29 end = len(insertion['text'].encode('utf-16-le'))//2
30 event.update(range=[end-3, end], replacement=' e\u0301🐈')
31 events.append(event)
32 value = insertion['text'].replace(' 🦀', ' e\u0301🐈') if kind == 'text' else insertion['text']
33 runs = []
34 for index, char in enumerate(value):
35 selected = index > 0 and (kind == 'text' or (kind == 'format' and index < len(value)-1))
36 runs.append({'text': char, 'bold': selected, 'size': 18+operation if selected else 11,
37 'color': 0x563412 if selected else 0xff000000})
38 observed[target] = {'text': value, 'runs': runs}
39 revision = f'revision-{local_id}'
40 events.append({'event': 'remote_attempt', 'revision': revision, 'state': 'Committed',
41 'document_changes': {target: copy.deepcopy(observed[target])},
42 'documents': copy.deepcopy(observed), 'started_us': timestamp, 'finished_us': timestamp+1})
43 events.append({'event': 'document_receipt', 'id': local_id, 'revision': revision, 'at_us': timestamp+2})
44 if text_edits:
45 events.append({'event': 'read', 'started_us': timestamp+3, 'finished_us': timestamp+4, 'documents': copy.deepcopy(observed)})
46 events.extend({'event': 'reopened_document_receipt', 'id': row['id'], 'revision': row['revision']}
47 for row in list(events) if row['event'] == 'document_receipt')
48 read = {'event': 'read', 'started_us': 100, 'finished_us': 101, 'documents': observed}
49 events.extend([read, {'event': 'done'}])
50 logs['r0'] = [{'event': 'ready'}, copy.deepcopy(read), {'event': 'done'}]
51 return logs
52
53
954class DocumentHistoryTests(unittest.TestCase):
1055 def setUp(self):
11 self.logs = {'w0': [{'event': 'ready', 'document_operations': True}]}
12 events = self.logs['w0']
13 observed = {}
14 previous = None
15 for operation in range(2):
16 insertion = {'guid': list(uuid.UUID(int=operation+1).bytes_le), 'text': f'Document w0:{operation} 🦀',
17 'parent': 'page' if operation == 0 else identity(previous, 1), 'author': 'Offline document writer',
18 'placement': {'Outline': {'x': 144, 'y': 144}} if operation == 0 else {'Paragraph': {'before': None}}}
19 previous = insertion
20 target = identity(insertion, 2)
21 for step, kind in enumerate(('insert', 'format')):
22 timestamp = 10 + operation*30 + step*10
23 local_id = operation*3 + step + 2
24 event = {'event': 'local_document_commit', 'id': local_id, 'operation': operation, 'kind': kind,
25 'space': 'space', 'object': target, 'text': insertion['text'], 'insertion': insertion if step == 0 else None,
26 'range': [1, len(insertion['text'].encode('utf-16-le'))//2-2],
27 'attributes': [{'Bold': True}, {'FontSize': 18+operation}, {'Color': [18, 52, 86]}],
28 'started_us': timestamp-2, 'finished_us': timestamp-1}
29 events.append(event)
30 runs = []
31 for index, char in enumerate(insertion['text']):
32 selected = kind == 'format' and 0 < index < len(insertion['text'])-1
33 runs.append({'text': char, 'bold': selected, 'size': 18+operation if selected else 11,
34 'color': 0x563412 if selected else 0xff000000})
35 observed[target] = {'text': insertion['text'], 'runs': runs}
36 revision = f'revision-{local_id}'
37 events.append({'event': 'remote_attempt', 'revision': revision, 'state': 'Committed',
38 'document_changes': {target: copy.deepcopy(observed[target])},
39 'documents': copy.deepcopy(observed), 'started_us': timestamp, 'finished_us': timestamp+1})
40 events.append({'event': 'document_receipt', 'id': local_id, 'revision': revision, 'at_us': timestamp+2})
41 events.extend({'event': 'reopened_document_receipt', 'id': row['id'], 'revision': row['revision']}
42 for row in list(events) if row['event'] == 'document_receipt')
43 read = {'event': 'read', 'started_us': 100, 'finished_us': 101, 'documents': observed}
44 events.extend([read, {'event': 'done'}])
45 self.logs['r0'] = [{'event': 'ready'}, copy.deepcopy(read), {'event': 'done'}]
56 self.logs = history()
4657
4758 def test_document_receipts_and_reader_states_match_the_intents(self):
4859 documents = document_history(self.logs, 2)
4960 self.assertEqual(len(documents), 2)
61 expected = [list(row['states'].values())[-1]['characters'] for row in documents.values()]
5062 paragraphs = [[(char, {'bold': bold, 'font_size': size, 'color': 'automatic' if color == 0xff000000 else '#123456'})
51 for char, bold, size, color in row['new']] for row in documents.values()]
52 self.assertEqual(verify_native(paragraphs, documents), sum(len(row['new'])*3 for row in documents.values()))
63 for char, bold, size, color in row] for row in expected]
64 self.assertEqual(verify_native(paragraphs, expected), sum(len(row)*3 for row in expected))
5365 paragraphs[0][1][1]['font_size'] = 19
54 with self.assertRaisesRegex(AssertionError, 'font size'): verify_native(paragraphs, documents)
66 with self.assertRaisesRegex(AssertionError, 'font size'): verify_native(paragraphs, expected)
67
68 def test_cross_run_text_requires_complete_ordered_states_and_exact_receipts(self):
69 logs = history(True)
70 documents = document_history(logs, 2)
71 for document in documents.values():
72 self.assertEqual(list(document['states']), ['insert', 'format', 'text'])
73 final = document['states']['text']['characters']
74 self.assertEqual(''.join(char for char, *_ in final), document['insertion']['text'].replace(' 🦀', ' e\u0301🐈'))
75 for event, field, value in [('local_document_commit', 'replacement', 'wrong'),
76 ('local_document_commit', 'range', [0, 1]),
77 ('document_receipt', 'revision', 'wrong')]:
78 changed = copy.deepcopy(logs)
79 row = next(row for row in changed['w0'] if row['event'] == event and
80 (row.get('kind') == 'text' or row.get('id') == 4))
81 row[field] = value
82 with self.subTest(event=event, field=field), self.assertRaises(AssertionError):
83 document_history(changed, 2)
84 changed = copy.deepcopy(logs)
85 read = changed['r0'][1]
86 target = next(iter(read['documents']))
87 read['documents'][target]['runs'][-1]['bold'] = False
88 with self.assertRaisesRegex(AssertionError, 'partial or invented'):
89 document_history(changed, 2)
90 prior = next(row for row in logs['w0'] if row['event'] == 'read' and row['started_us'] == 23)
91 changed = copy.deepcopy(logs)
92 changed['r0'][1]['documents'][target] = copy.deepcopy(prior['documents'][target])
93 with self.assertRaisesRegex(AssertionError, 'missed acknowledged'):
94 document_history(changed, 2)
5595
5696 def test_missing_intents_receipts_or_reopen_records_are_rejected(self):
5797 for name in ('local_document_commit', 'document_receipt', 'reopened_document_receipt', 'remote_attempt'):
......@@ -75,10 +115,11 @@ class DocumentHistoryTests(unittest.TestCase):
75115 run['size'] = 13.5
76116 run['color'] = 0x665544
77117 documents = document_history(self.logs, 2)
118 expected = [list(row['states'].values())[-1]['characters'] for row in documents.values()]
78119 paragraphs = [[(char, {'bold': bold, 'font_size': size,
79120 'color': '#123456' if color == 0x563412 else '#445566'})
80 for char, bold, size, color in row['new']] for row in documents.values()]
81 self.assertEqual(verify_native(paragraphs, documents), sum(len(row['new'])*3 for row in documents.values()))
121 for char, bold, size, color in row] for row in expected]
122 self.assertEqual(verify_native(paragraphs, expected), sum(len(row)*3 for row in expected))
82123 baseline = copy.deepcopy(self.logs)
83124 for event, key in [('remote_attempt', 'document_changes'), ('read', 'documents')]:
84125 for field, value in [('size', 11), ('color', 0xff000000)]:
......@@ -102,7 +143,7 @@ class DocumentHistoryTests(unittest.TestCase):
102143 events[index:index] = [read, confirmation]
103144 result = document_history(self.logs, 2)
104145 target, = attempt['document_changes']
105 self.assertEqual(result[target]['format']['receipt_revision'], 'current-revision')
146 self.assertEqual(result[target]['states']['format']['attempt']['receipt_revision'], 'current-revision')
106147 for field, value in [('current_revisions', {'space': 'unrelated'}),
107148 ('revisions', {'space': ['revision-3', 'current-revision']}),
108149 ('state', 'NotCommitted')]:
......@@ -123,7 +164,7 @@ class DocumentHistoryTests(unittest.TestCase):
123164 nodes[outline].update(children=[paragraph, appended], layout={'x': 144, 'y': 144})
124165 for parent, (target, document) in zip([paragraph, appended], documents.items(), strict=True):
125166 nodes[parent]['content'] = [target]
126 nodes[target]['kind'] = {'text': document['text']}
167 nodes[target]['kind'] = {'text': ''.join(char for char, *_ in list(document['states'].values())[-1]['characters'])}
127168 revision = {'nodes': nodes}
128169 with patch('offline_document_history.ordered_pages', return_value=[('space', 'revision', revision, 'page')]):
129170 verify_model({}, documents)
tools/test_offline_outage.py+1-1
......@@ -85,7 +85,7 @@ class OutageOracle(unittest.TestCase):
8585 (self.root / 'rust' / f'{actor}.jsonl').write_text('\n'.join(map(json.dumps, rows)))
8686 with patch('offline_history.publication_links') as ledger, patch('offline_document_history.document_history') as documents, patch('offline_outage.verify', return_value={'guarded_pairs': 1}) as overlap:
8787 if self.config.get('offline_client_reply'):
88 documents.return_value = {'target': {'format': {'receipt_revision': 'effect-revision'}}}
88 documents.return_value = {'target': {'states': {'format': {'attempt': {'receipt_revision': 'effect-revision'}}}}}
8989 result = verify_lost_reply(self.root)
9090 ledger.assert_called_once_with(self.logs, self.config['stress_operations'])
9191 if self.config.get('document_operations'):
tools/verify_offline.py+2-2
......@@ -32,7 +32,7 @@ def verify(output, max_gap=120):
3232 edits = {event['id']: event for event in events if event['event'] in ('local_commit', 'local_document_commit')}
3333 receipts = {event['id']: event for event in events if event['event'] in ('remote_receipt', 'document_receipt')}
3434 attempts = {event['revision']: event for event in events if event['event'] == 'remote_attempt'}
35 publication_starts = {id: documents[edit['object']][edit['kind']]['started_us'] if edit['event'] == 'local_document_commit'
35 publication_starts = {id: documents[edit['object']]['states'][edit['kind']]['attempt']['started_us'] if edit['event'] == 'local_document_commit'
3636 else attempts[receipts[id]['revision']]['started_us'] for id, edit in edits.items()}
3737 queues[actor] = max(sum(edit['finished_us'] <= at < publication_starts[id] for id, edit in edits.items()) for at in [edit['finished_us'] for edit in edits.values()])
3838 assert queues[actor] >= 2, 'Writer did not establish a durable local queue before publication'
......@@ -57,7 +57,7 @@ def verify(output, max_gap=120):
5757 assert times == sorted(times)
5858 progress[f'n{i}'] = max((b-a for a, b in zip(times, times[1:])), default=0) / 10_000_000
5959 assert all(gap <= max_gap for gap in progress.values()), f'Client progress exceeded {max_gap}s: {progress}'
60 return {'remote_publications': len(commits), 'document_publications': len(documents)*2, 'remote_text_sha256': hashlib.sha256(text.encode()).hexdigest(), 'maximum_progress_gap_seconds': progress,
60 return {'remote_publications': len(commits), 'document_publications': sum(len(document['states']) for document in documents.values()), 'remote_text_sha256': hashlib.sha256(text.encode()).hexdigest(), 'maximum_progress_gap_seconds': progress,
6161 'queued_before_publication_lower_bound': queues, 'reviewed_placements': sum(event['event'] == 'reviewed_append' for events in logs.values() for event in events), 'caches': caches}
6262
6363
tools/verify_offline_confirmation.py+3-4
......@@ -68,9 +68,8 @@ def verify(output, cold, *, partial=False):
6868 paragraphs.remove(expected)
6969 if observed:
7070 from offline_document_history import characters, verify_native
71 expected_documents = {target: {'text': documents[target]['text'], 'new': characters(value)} for target, value in observed.items()}
72 format_checks += verify_native(formatted, expected_documents)
73 for document in expected_documents.values():
71 format_checks += verify_native(formatted, (characters(value) for value in observed.values()))
72 for document in observed.values():
7473 assert paragraphs.count(document['text']) == 1, 'Confirmation duplicated a document paragraph'
7574 paragraphs.remove(document['text'])
7675 for index, versions in enumerate(native):
......@@ -78,7 +77,7 @@ def verify(output, cold, *, partial=False):
7877 assert len(selected) == 1 and selected[0] in versions, 'Native confirmation image contains an unrecorded edit or loses a prefix'
7978 checks += len(native) + 1 + len(observed)
8079 assert json.loads((cold / 'teardown.json').read_text()) == {'absent': True}
81 confirmed_revision = documents[next(iter(attempt['document_changes']))]['format']['receipt_revision'] if documents else attempt['revision']
80 confirmed_revision = documents[next(iter(attempt['document_changes']))]['states']['format']['attempt']['receipt_revision'] if documents else attempt['revision']
8281 return {'complete_workload': not partial, 'attempted_revision': attempt['revision'], 'confirmed_revision': confirmed_revision, 'native_images': len(results), 'exact_rust_paragraphs': len(results),
8382 'validated_paragraphs': checks, 'native_intended_format_checks': format_checks,
8483 'maximum_native_export_seconds': max(row['seconds'] for row in results)}