authorgravatar for git@paperclover.netclover caruso <git@paperclover.net> 2026-09-27 17:36:32-07:00
committergravatar for git@paperclover.netclover caruso <git@paperclover.net> 2026-09-28 05:43:14-07:00
log71e4e308b60bf1a52ad839646150a0f9e324ecee
tree0afa25a027033419097876770d0258af46e37efc
parentb589fec9d51d404e16bcb68a176f226d5b5fa459
signature Signed by SSH key SHA256:52mNGHRsVFBDED9IAX5pe+LRWUefqTbxEReunq21QvU

feat(data): page versions, editable links and math, TOC and discovery repairs

Page versions follow OneNote 2010: old revisions kept alive by revision labels and a version list, with restore, delete and copy written as OneNote writes them. Links and equations are editable end to end (UnicodeMath linear form, OneNote's Enter rules at equation and link boundaries), and titles no longer keep a link's field code. Discovery tolerates unreadable or delete-pending folders, table-of-contents edits keep OneNote's order values on rename, and recycle-bin placement repairs stale entries. New corpus rows: page-versions, link-edit and math-edit captures, recycle-bin-repair. Assisted-by: claude-opus-5.5

393 files changed, 9412 insertions(+), 1125 deletions(-)

corpus/link-edit/README.md+39
...@@ -37,3 +37,42 @@ second page, whose first page links to that page and to its first paragraph...@@ -37,3 +37,42 @@ second page, whose first page links to that page and to its first paragraph
37with URLs built by `onestore::page::link::internal_link`. `internal/cold` is37with URLs built by `onestore::page::link::internal_link`. `internal/cold` is
38its cold read with both links. Regenerate with38its cold read with both links. Regenerate with
39`ONESTORE_INTERNAL_LINK_EXPORT` and cold-open with `--expected-pages 2`.39`ONESTORE_INTERNAL_LINK_EXPORT` and cold-open with `--expected-pages 2`.
40
41## Typed links
42
43`native-typed/` is OneNote 2010 typing on the Rust-authored page above, driven by
44`tools/native_links.py` (`links.ahk` the AutoHotkey session, `links.png` the
45desktop afterwards). Each URL typed and ended by a space or Enter became a link
46of its own text, a hyperlink run with no field code and no label flag: `http`,
47`https`, `ftp`, `file`, `mailto`, `news` and `onenote` schemes, `www.` and
48`\\server\share` paths. Trailing punctuation and an unmatched closing bracket
49stay outside; `me@example.com` and `example.com` stay text. The Link dialog
50(Ctrl+K) on the word at the caret, on a selection and on nothing stored the
51address as typed (`example.net/x`, which the COM read shows as
52`http://example.net/x`) in a hidden field code before a label flagged as one,
53the address itself when the text was empty; text typed after a label is plain.
54Remove Link (Shift+F10, R) left the URL as plain text, its runs' link flag
55false. An interactive session on a copy showed the rest: a click on a link opens
56it at once, typing straight after a link of its own text extends it and its
57address, and Copy Link to Page puts
58`onenote:///<section path>#<title>&section-id={…}&page-id={…}&end` on the
59clipboard (`&object-id={…}&n` in place of `&end` for Copy Link to Paragraph),
60which pastes as a labelled link. `crates/canvas/src/editor/link.rs` replays
61the session through the editor and compares every paragraph's link runs.
62
63## Links and equations from the editor
64
65`editor/candidate` is the canvas editor's output for
66`crates/canvas/tests/links_equations.rs` on the internal-link section above:
67URLs typed and ended by a space or Enter, the Link dialog on a selection, on the
68word at the caret and on nothing, links to the other page and to a paragraph,
69Remove Link, and equations typed after Alt+= (one switched to Linear, one after
70text in its paragraph), all saved as the ops the editor recorded.
71`editor/cold` is its cold OneNote 2010 read: `tools/test_link_edit.py`
72compares it with the candidate and checks the links and MathML in
73`editor/expected.json`. OneNote's read opens an address without a scheme over
74http and links URL text again when it opens a page; its own section does the
75same after Remove Link (`native-typed/cold`, its cold read, shows `Gone
76ftp://h.example/f` linked again). Regenerate with
77`CANVAS_LINKS_EQUATIONS_EXPORT` set to a new directory, move `expected.json`
78beside the candidate, and cold-open with `--expected-pages 2 --screenshots`.
corpus/link-edit/editor/candidate/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/link-edit/editor/candidate/Open Notebook.onetoc2 differ
corpus/link-edit/editor/candidate/links.one created
Binary files /dev/null and b/corpus/link-edit/editor/candidate/links.one differ
corpus/link-edit/editor/cold/commands.jsonl created+3
...@@ -0,0 +1,3 @@
1{"command": "powershell -NoProfile -Command \"Expand-Archive -LiteralPath C:\\one-tests\\transfer.zip -DestinationPath C:\\one-tests\\runs\\capture\\notebook\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-8C62D976 -ExpectedPages 2 -KeepOpen", "exit": 0, "stdout": "Read 1 sections and 2 pages.\r\n", "stderr": "", "error": null}
3{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
corpus/link-edit/editor/cold/machine.json created+1
...@@ -0,0 +1 @@
1{"name": "m6-8c62d976", "hostname": "ONE-M6-8C62D976"}
corpus/link-edit/editor/cold/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/link-edit/editor/cold/notebook/Open Notebook.onetoc2 differ
corpus/link-edit/editor/cold/notebook/links.one created
Binary files /dev/null and b/corpus/link-edit/editor/cold/notebook/links.one differ
corpus/link-edit/editor/cold/read/environment.json created+7
...@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-8C62D976",
5 "cold": true,
6 "onenote": "14.0.4763.1000"
7}
corpus/link-edit/editor/cold/read/hierarchy.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{73BFB197-D55A-45AA-B33D-13AE4D508815}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-28T07:41:49.000Z" color="#B49EDE"><one:Section name="links" ID="{E018864F-A1D8-099B-1541-A508D3F390DC}{1}{B0}" path="C:\one-tests\runs\capture\notebook\links.one" lastModifiedTime="2026-09-28T07:41:49.000Z" color="#8AA8E4"><one:Page ID="{2075D4FE-ABBA-0064-39D5-AE81ADA51B22}{1}{B0}" name="Linking page Link target" dateTime="2026-09-26T06:32:48.000Z" lastModifiedTime="2026-09-28T07:41:49.000Z" pageLevel="1"/><one:Page ID="{237B9A56-69AE-037B-0E57-A9B13FBCE71B}{1}{B0}" name="Link target" dateTime="2026-09-26T06:32:48.000Z" lastModifiedTime="2026-09-28T07:41:49.000Z" pageLevel="1"/></one:Section></one:Notebook>
corpus/link-edit/editor/cold/read/page-000.navigation.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Linking page Link target - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/link-edit/editor/cold/read/page-000.png created
Binary files /dev/null and b/corpus/link-edit/editor/cold/read/page-000.png differ
corpus/link-edit/editor/cold/read/page-000.xml created+16
...@@ -0,0 +1,16 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{2075D4FE-ABBA-0064-39D5-AE81ADA51B22}{1}{B0}" name="Linking page Link target" dateTime="2026-09-26T06:32:48.000Z" lastModifiedTime="2026-09-28T07:41:49.000Z" pageLevel="1" style="font-family:Calibri;font-size:11.0pt" lang="en-US"><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="Author" lastModifiedBy="Author" lastModifiedTime="2026-09-26T06:32:48.000Z" objectID="{E018864F-A1D8-099B-1541-A508D3F390DC}{23}{B0}"><one:Position x="36.0" y="36.0" z="0"/><one:Size width="468.0" height="387.6758728027344"/><one:Indents><one:Indent level="0" indent="0.0"/></one:Indents><one:OEChildren><one:OE creationTime="2026-09-26T06:32:48.000Z" lastModifiedTime="2026-09-26T06:32:48.000Z" objectID="{E018864F-A1D8-099B-1541-A508D3F390DC}{24}{B0}" alignment="left"><one:T><![CDATA[Linking page <a
3href="onenote:#Link%20target&amp;section-id={912D18F5-8E97-4C22-B11D-AA61E6E7D4B1}&amp;page-id={54104D3B-F876-47C6-8009-D90D3DEC5217}&amp;end&amp;base-path=C:\one-tests\runs\capture\notebook\links.one">Link target</a>]]></one:T></one:OE><one:OE author="Snowbound" authorInitials="S" lastModifiedBy="Snowbound" lastModifiedByInitials="S" creationTime="2022-06-18T04:26:40.000Z" lastModifiedTime="2022-06-18T04:26:40.000Z" objectID="{F3330A81-6E1B-07A5-343B-138D77A42F82}{7}{B0}" alignment="left"><one:T><![CDATA[Visit <a
4href="http://www.example.com">www.example.com</a> or (<a
5href="https://example.org/a?b=1">https://example.org/a?b=1</a>). ]]></one:T></one:OE><one:OE author="Snowbound" authorInitials="S" lastModifiedBy="Snowbound" lastModifiedByInitials="S" creationTime="2022-06-18T04:26:40.000Z" lastModifiedTime="2022-06-18T04:26:40.000Z" objectID="{F3330A81-6E1B-07A5-343B-138D77A42F82}{11}{B0}" alignment="left"><one:T><![CDATA[Ends with <a
6href="http://example.net/end">http://example.net/end</a>]]></one:T></one:OE><one:OE author="Snowbound" authorInitials="S" lastModifiedBy="Snowbound" lastModifiedByInitials="S" creationTime="2022-06-18T04:26:40.000Z" lastModifiedTime="2022-06-18T04:26:40.000Z" objectID="{F3330A81-6E1B-07A5-343B-138D77A42F82}{15}{B0}" alignment="left"><one:T><![CDATA[Read the <a
7href="https://doc.rust-lang.org/book/">Rust book</a> today]]></one:T></one:OE><one:OE author="Snowbound" authorInitials="S" lastModifiedBy="Snowbound" lastModifiedByInitials="S" creationTime="2022-06-18T04:26:40.000Z" lastModifiedTime="2022-06-18T04:26:40.000Z" objectID="{F3330A81-6E1B-07A5-343B-138D77A42F82}{19}{B0}" alignment="left"><one:T><![CDATA[A <a
8href="http://example.net/x">word</a> here]]></one:T></one:OE><one:OE author="Snowbound" authorInitials="S" lastModifiedBy="Snowbound" lastModifiedByInitials="S" creationTime="2022-06-18T04:26:40.000Z" lastModifiedTime="2022-06-18T04:26:40.000Z" objectID="{F3330A81-6E1B-07A5-343B-138D77A42F82}{23}{B0}" alignment="left"><one:T><![CDATA[<a
9href="https://example.com/only">https://example.com/only</a> tail]]></one:T></one:OE><one:OE author="Snowbound" authorInitials="S" lastModifiedBy="Snowbound" lastModifiedByInitials="S" creationTime="2022-06-18T04:26:40.000Z" lastModifiedTime="2022-06-18T04:26:40.000Z" objectID="{F3330A81-6E1B-07A5-343B-138D77A42F82}{25}{B0}" alignment="left"><one:T><![CDATA[See <a
10href="onenote:#Link%20target&amp;section-id={912D18F5-8E97-4C22-B11D-AA61E6E7D4B1}&amp;page-id={54104D3B-F876-47C6-8009-D90D3DEC5217}&amp;end&amp;base-path=C:\one-tests\runs\capture\notebook\links.one">the target page</a>]]></one:T></one:OE><one:OE author="Snowbound" authorInitials="S" lastModifiedBy="Snowbound" lastModifiedByInitials="S" creationTime="2022-06-18T04:26:40.000Z" lastModifiedTime="2022-06-18T04:26:40.000Z" objectID="{F3330A81-6E1B-07A5-343B-138D77A42F82}{27}{B0}" alignment="left"><one:T><![CDATA[<a
11href="onenote:#Linking%20page&amp;section-id={912D18F5-8E97-4C22-B11D-AA61E6E7D4B1}&amp;page-id={3225B89C-1ACE-4B05-A5C8-FF6E178A21D7}&amp;object-id={97735122-3000-4D26-9B1F-D5B4D1A325D0}&amp;24&amp;base-path=C:\one-tests\runs\capture\notebook\links.one">Back to the top</a>]]></one:T></one:OE><one:OE author="Snowbound" authorInitials="S" lastModifiedBy="Snowbound" lastModifiedByInitials="S" creationTime="2022-06-18T04:26:40.000Z" lastModifiedTime="2022-06-18T04:26:40.000Z" objectID="{F3330A81-6E1B-07A5-343B-138D77A42F82}{29}{B0}" alignment="left"><one:T><![CDATA[Gone <a
12href="ftp://h.example/f">ftp://h.example/f</a> z]]></one:T></one:OE><one:OE author="Snowbound" authorInitials="S" lastModifiedBy="Snowbound" lastModifiedByInitials="S" creationTime="2022-06-18T04:26:40.000Z" lastModifiedTime="2022-06-18T04:26:40.000Z" objectID="{F3330A81-6E1B-07A5-343B-138D77A42F82}{31}{B0}" alignment="left" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:msub><mml:mi>a</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo>+</mml:mo><mml:msub><mml:mi>b</mml:mi><mml:mn>2</mml:mn></mml:msub></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Snowbound" authorInitials="S" lastModifiedBy="Snowbound" lastModifiedByInitials="S" creationTime="2022-06-18T04:26:40.000Z" lastModifiedTime="2022-06-18T04:26:40.000Z" objectID="{F3330A81-6E1B-07A5-343B-138D77A42F82}{33}{B0}" alignment="left" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:msubsup><mml:mo stretchy="false">∫</mml:mo><mml:mn>0</mml:mn><mml:mn>1</mml:mn></mml:msubsup><mml:mrow><mml:mi>x</mml:mi><mml:mo>&nbsp;</mml:mo><mml:mi>d</mml:mi><mml:mi>x</mml:mi><mml:mo>&nbsp;</mml:mo></mml:mrow></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Snowbound" authorInitials="S" lastModifiedBy="Snowbound" lastModifiedByInitials="S" creationTime="2022-06-18T04:26:40.000Z" lastModifiedTime="2022-06-18T04:26:40.000Z" objectID="{F3330A81-6E1B-07A5-343B-138D77A42F82}{35}{B0}" alignment="left" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:mfrac><mml:mrow><mml:mi>a</mml:mi><mml:mo>+</mml:mo><mml:mi>b</mml:mi></mml:mrow><mml:mrow><mml:mi>c</mml:mi><mml:mo>+</mml:mo><mml:mi>d</mml:mi></mml:mrow></mml:mfrac></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Snowbound" authorInitials="S" lastModifiedBy="Snowbound" lastModifiedByInitials="S" creationTime="2022-06-18T04:26:40.000Z" lastModifiedTime="2022-06-18T04:26:40.000Z" objectID="{F3330A81-6E1B-07A5-343B-138D77A42F82}{37}{B0}" alignment="left" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:msqrt><mml:mi>x</mml:mi></mml:msqrt><mml:mo>+</mml:mo><mml:mn>1</mml:mn><mml:mo>&nbsp;</mml:mo></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Snowbound" authorInitials="S" lastModifiedBy="Snowbound" lastModifiedByInitials="S" creationTime="2022-06-18T04:26:40.000Z" lastModifiedTime="2022-06-18T04:26:40.000Z" objectID="{F3330A81-6E1B-07A5-343B-138D77A42F82}{39}{B0}" alignment="left" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:munderover><mml:mo stretchy="false">∑</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>n</mml:mi></mml:munderover><mml:mrow><mml:mi>i</mml:mi><mml:mo>&nbsp;</mml:mo></mml:mrow></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Snowbound" authorInitials="S" lastModifiedBy="Snowbound" lastModifiedByInitials="S" creationTime="2022-06-18T04:26:40.000Z" lastModifiedTime="2022-06-18T04:26:40.000Z" objectID="{F3330A81-6E1B-07A5-343B-138D77A42F82}{41}{B0}" alignment="left" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:mtable><mml:mtr><mml:mtd><mml:mn>1</mml:mn></mml:mtd><mml:mtd><mml:mn>2</mml:mn></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mn>3</mml:mn></mml:mtd><mml:mtd><mml:mn>4</mml:mn></mml:mtd></mml:mtr></mml:mtable></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Snowbound" authorInitials="S" lastModifiedBy="Snowbound" lastModifiedByInitials="S" creationTime="2022-06-18T04:26:40.000Z" lastModifiedTime="2022-06-18T04:26:40.000Z" objectID="{F3330A81-6E1B-07A5-343B-138D77A42F82}{43}{B0}" alignment="left" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:mi>e</mml:mi><mml:mo>^</mml:mo><mml:mo fence="false">(</mml:mo><mml:mi>x</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn><mml:mo fence="false">)</mml:mo></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Snowbound" authorInitials="S" lastModifiedBy="Snowbound" lastModifiedByInitials="S" creationTime="2022-06-18T04:26:40.000Z" lastModifiedTime="2022-06-18T04:26:40.000Z" objectID="{F3330A81-6E1B-07A5-343B-138D77A42F82}{45}{B0}" alignment="left" style="font-size:11.0pt"><one:T><![CDATA[<span
13style='font-family:Calibri' lang=en-US>Area </span><!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>&#120587;</mml:mi><mml:msup><mml:mi>r</mml:mi><mml:mn>2</mml:mn></mml:msup></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Snowbound" authorInitials="S" lastModifiedBy="Snowbound" lastModifiedByInitials="S" creationTime="2022-06-18T04:26:40.000Z" lastModifiedTime="2022-06-18T04:26:40.000Z" objectID="{F3330A81-6E1B-07A5-343B-138D77A42F82}{47}{B0}" alignment="left"><one:T><![CDATA[Split ]]></one:T></one:OE><one:OE author="Snowbound" authorInitials="S" lastModifiedBy="Snowbound" lastModifiedByInitials="S" creationTime="2022-06-18T04:26:40.000Z" lastModifiedTime="2022-06-18T04:26:40.000Z" objectID="{F3330A81-6E1B-07A5-343B-138D77A42F82}{49}{B0}" alignment="left" style="font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:msup><mml:mi>y</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mo>+</mml:mo></mml:math><![endif]--><span
14style='font-family:Calibri' lang=en-US><br>
15</span><!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:mn>1</mml:mn><mml:mo>&nbsp;</mml:mo></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Snowbound" authorInitials="S" lastModifiedBy="Snowbound" lastModifiedByInitials="S" creationTime="2022-06-18T04:26:40.000Z" lastModifiedTime="2022-06-18T04:26:40.000Z" objectID="{F3330A81-6E1B-07A5-343B-138D77A42F82}{51}{B0}" alignment="left" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:mfrac><mml:mrow><mml:mi>a</mml:mi><mml:mo>+</mml:mo><mml:mi>b</mml:mi></mml:mrow><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:maligngroup/><mml:mi>c</mml:mi></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:maligngroup/><mml:mo>+</mml:mo><mml:mi>d</mml:mi></mml:mtd></mml:mtr></mml:mtable></mml:mrow></mml:mfrac></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Snowbound" authorInitials="S" lastModifiedBy="Snowbound" lastModifiedByInitials="S" creationTime="2022-06-18T04:26:40.000Z" lastModifiedTime="2022-06-18T04:26:40.000Z" objectID="{F3330A81-6E1B-07A5-343B-138D77A42F82}{53}{B0}" alignment="left"><one:T><![CDATA[Moved ]]></one:T></one:OE><one:OE author="Snowbound" authorInitials="S" lastModifiedBy="Snowbound" lastModifiedByInitials="S" creationTime="2022-06-18T04:26:40.000Z" lastModifiedTime="2022-06-18T04:26:40.000Z" objectID="{F3330A81-6E1B-07A5-343B-138D77A42F82}{55}{B0}" alignment="left"><one:T><![CDATA[<a
16href="https://example.com/moved">down</a>]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/link-edit/editor/cold/read/page-001.navigation.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Link target - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/link-edit/editor/cold/read/page-001.png created
Binary files /dev/null and b/corpus/link-edit/editor/cold/read/page-001.png differ
corpus/link-edit/editor/cold/read/page-001.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{237B9A56-69AE-037B-0E57-A9B13FBCE71B}{1}{B0}" name="Link target" dateTime="2026-09-26T06:32:48.000Z" lastModifiedTime="2026-09-28T07:41:49.000Z" pageLevel="1"><one:QuickStyleDef index="0" name="PageTitle" font="Calibri" fontSize="17.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Title showDate="false" showTime="false" lang="en-US"><one:OE author="Author" lastModifiedBy="Author" creationTime="2026-09-26T06:32:48.000Z" lastModifiedTime="2026-09-26T06:32:48.000Z" objectID="{237B9A56-69AE-037B-0E57-A9B13FBCE71B}{15}{B0}" alignment="left" quickStyleIndex="0" style="font-family:Calibri;font-size:17.0pt"><one:T><![CDATA[Link target]]></one:T></one:OE></one:Title></one:Page>
corpus/link-edit/editor/cold/read/payloads.json created+3
...@@ -0,0 +1,3 @@
1[
2
3]
\ No newline at end of file
corpus/link-edit/editor/cold/run.json created+18
...@@ -0,0 +1,18 @@
1{
2 "notebook": "/private/tmp/cands/le3nb",
3 "expected_pages": 2,
4 "author": null,
5 "author_timeout_seconds": 600,
6 "inspect": false,
7 "collect_notebook": true,
8 "base": {
9 "file": "win7-office-base.qcow2",
10 "format": "qcow2",
11 "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346",
12 "virtual_size": 68719476736
13 },
14 "scripts": {
15 "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331",
16 "read.ps1": "04013bfcccee40a17e2a225f9b9e96f40a3a8daad9e350609654f8eda3ccbb41"
17 }
18}
corpus/link-edit/editor/cold/scripts/cold.ps1 created+27
...@@ -0,0 +1,27 @@
1param([Parameter(Mandatory=$true)][string]$Root, [string]$CloneHost = '')
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
5if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
6 throw 'Choose a run directly below C:\one-tests\runs.'
7}
8if (Get-Process ONENOTE -ErrorAction SilentlyContinue) { throw 'Close OneNote before resetting its test cache.' }
9$key = 'HKCU:\Software\Microsoft\Office\14.0\OneNote'
10if ($CloneHost) {
11 if ($CloneHost -notmatch '^ONE-[A-Z0-9-]+$' -or [Environment]::MachineName -ne $CloneHost) {
12 throw 'The disposable clone hostname does not match this machine.'
13 }
14 New-Item "$key\Options\Paths" -Force | Out-Null
15 New-ItemProperty "$key\Options\Paths" -Name UnfiledNotesSection -PropertyType ExpandString -Value 'C:\one-tests\Loose.one' -Force | Out-Null
16} elseif ((Get-ItemProperty "$key\Options\Paths").UnfiledNotesSection -ne 'C:\one-tests\Loose.one' -or
17 -not (Test-Path 'C:\one-tests\profile-original-cache')) {
18 throw 'Park the personal OneNote profile before resetting the test cache.'
19}
20$cache = Join-Path $env:LOCALAPPDATA 'Microsoft\OneNote\14.0'
21$parked = Join-Path 'C:\one-tests\caches' ([IO.Path]::GetFileName($root))
22if (Test-Path $parked) { throw 'Choose a new run; its parked cache already exists.' }
23New-Item -ItemType Directory -Path 'C:\one-tests\caches' -Force | Out-Null
24if (Test-Path $cache) { Move-Item -LiteralPath $cache -Destination $parked }
25if (Test-Path "$key\OpenNotebooks") { Remove-Item "$key\OpenNotebooks" -Recurse }
26New-Item "$key\OpenNotebooks" | Out-Null
27New-ItemProperty "$key\OpenNotebooks" -Name '1' -PropertyType String -Value "$root\notebook" | Out-Null
corpus/link-edit/editor/cold/scripts/read.ps1 created+142
...@@ -0,0 +1,142 @@
1param(
2 [Parameter(Mandatory=$true)][string]$Root,
3 [int]$ExpectedPages = -1,
4 [switch]$UseCurrentCache,
5 [switch]$Pdf,
6 [switch]$KeepOpen,
7 [string]$CloneHost = ''
8)
9Set-StrictMode -Version Latest
10$ErrorActionPreference = 'Stop'
11$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
12if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
13 throw 'Choose a run directly below C:\one-tests\runs.'
14}
15$notebook = Join-Path $root 'notebook'
16$output = Join-Path $root 'read'
17if (Test-Path $output) { throw 'Choose a new read destination.' }
18if ($UseCurrentCache) {
19 if ((Get-ItemProperty 'HKCU:\Software\Microsoft\Office\14.0\OneNote\Options\Paths').UnfiledNotesSection -ne 'C:\one-tests\Loose.one') {
20 throw 'Park the personal OneNote profile before reading test notebooks.'
21 }
22} else {
23 & "$PSScriptRoot\cold-current.ps1" -Root $root -CloneHost $CloneHost
24}
25New-Item -ItemType Directory -Path $output | Out-Null
26$app = New-Object -ComObject OneNote.Application
27$notebookId = ''
28$failure = $null
29try {
30 $app.OpenHierarchy($notebook, '', [ref]$notebookId, 0)
31 $process = Get-Process ONENOTE
32 @{ hostname = [Environment]::MachineName; onenote = $process.MainModule.FileVersionInfo.FileVersion;
33 powershell = $PSVersionTable.PSVersion.ToString(); schema = 'xs2010'; cold = (-not $UseCurrentCache.IsPresent) } |
34 ConvertTo-Json | Set-Content (Join-Path $output 'environment.json') -Encoding UTF8
35 $sections = @()
36 foreach ($file in @(Get-ChildItem $notebook -Recurse | Where-Object { $_.Extension -eq '.one' })) {
37 $id = ''
38 $app.OpenHierarchy($file.FullName, '', [ref]$id, 0)
39 $sections += $id
40 }
41 $deadline = [DateTime]::UtcNow.AddSeconds(300)
42 $previous = ''
43 $lastChange = ''
44 $stableSince = [DateTime]::UtcNow
45 $settled = $false
46 do {
47 $pages = @{}
48 foreach ($section in $sections) {
49 $hierarchy = ''
50 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
51 [xml]$xml = $hierarchy
52 foreach ($node in $xml.SelectNodes('//*[@path]')) {
53 if (-not $node.GetAttribute('path').StartsWith("$notebook\", [StringComparison]::OrdinalIgnoreCase)) {
54 throw 'OneNote opened a section outside the copied notebook.'
55 }
56 }
57 foreach ($node in $xml.SelectNodes('//*[local-name()="Page"]')) {
58 $id = $node.GetAttribute('ID')
59 $content = ''
60 $app.GetPageContent($id, [ref]$content, 1, 1)
61 $pages[$id] = $content
62 }
63 }
64 $signature = [String]::Join('|', @($pages.Keys | Sort-Object | ForEach-Object { $_ + $pages[$_] }))
65 if ($signature -ne $previous) {
66 $lastChange = $previous
67 $previous = $signature
68 $stableSince = [DateTime]::UtcNow
69 }
70 if ((($ExpectedPages -ge 0 -and $pages.Count -eq $ExpectedPages) -or
71 ($ExpectedPages -lt 0 -and $pages.Count -gt 0)) -and
72 ([DateTime]::UtcNow - $stableSince).TotalSeconds -ge 2) { $settled = $true; break }
73 Start-Sleep -Milliseconds 250
74 } while ([DateTime]::UtcNow -lt $deadline)
75 if (-not $settled -or ($ExpectedPages -ge 0 -and $pages.Count -ne $ExpectedPages) -or ($ExpectedPages -lt 0 -and $pages.Count -eq 0)) {
76 [IO.File]::WriteAllText((Join-Path $output 'previous-signature.txt'), $lastChange, [Text.Encoding]::UTF8)
77 $index = 0
78 foreach ($id in @($pages.Keys | Sort-Object)) {
79 [IO.File]::WriteAllText((Join-Path $output ('unsettled-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
80 $index++
81 }
82 $hierarchy = ''
83 $app.GetHierarchy($notebookId, 4, [ref]$hierarchy, 1)
84 [IO.File]::WriteAllText((Join-Path $output 'unsettled-hierarchy.xml'), $hierarchy, [Text.Encoding]::UTF8)
85 throw "Expected $ExpectedPages stable pages; OneNote returned $($pages.Count), settled=$settled."
86 }
87 $index = 0
88 $payloads = @()
89 foreach ($id in @($pages.Keys | Sort-Object)) {
90 [IO.File]::WriteAllText((Join-Path $output ('page-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
91 if ($Pdf) {
92 $pdfPath = Join-Path $output ('page-{0:d3}.pdf' -f $index)
93 $app.NavigateTo($id, '', $false)
94 $app.Publish($id, $pdfPath, 3, '')
95 if (-not (Test-Path $pdfPath) -or (Get-Item $pdfPath).Length -eq 0) { throw 'OneNote did not publish the page PDF.' }
96 }
97 [xml]$page = $pages[$id]
98 foreach ($file in $page.SelectNodes('//*[local-name()="InsertedFile" or local-name()="MediaFile"]')) {
99 $bytes = [IO.File]::ReadAllBytes($file.GetAttribute('pathCache'))
100 $hash = [BitConverter]::ToString([Security.Cryptography.SHA256]::Create().ComputeHash($bytes)).Replace('-', '').ToLowerInvariant()
101 [IO.File]::WriteAllBytes((Join-Path $output ($hash + '.attachment')), $bytes)
102 $payloads += @{ page = $id; object = $file.ParentNode.GetAttribute('objectID');
103 kind = $file.LocalName; name = $file.GetAttribute('preferredName');
104 sha256 = $hash; bytes = $bytes.Length }
105 }
106 $index++
107 }
108 [IO.File]::WriteAllText((Join-Path $output 'payloads.json'), (ConvertTo-Json -InputObject $payloads -Depth 4), [Text.Encoding]::UTF8)
109 $all = ''
110 $app.GetHierarchy($notebookId, 4, [ref]$all, 1)
111 [xml]$finalTree = $all
112 $finalIds = @($finalTree.SelectNodes('//*[local-name()="Page"]') | ForEach-Object { $_.GetAttribute('ID') } | Sort-Object -Unique)
113 if ($finalIds.Count -ne $pages.Count -or @($finalIds | Where-Object { -not $pages.ContainsKey($_) }).Count -ne 0) {
114 throw 'The notebook hierarchy changed while collecting page evidence; repeat the cold read.'
115 }
116 [IO.File]::WriteAllText((Join-Path $output 'hierarchy.xml'), $all, [Text.Encoding]::UTF8)
117 Write-Output "Read $($sections.Count) sections and $($pages.Count) pages."
118} catch {
119 $failure = $_
120 [IO.File]::WriteAllText((Join-Path $output 'failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
121 throw
122} finally {
123 try {
124 try {
125 if ($notebookId -and $CloneHost) { $app.SyncHierarchy($notebookId) }
126 if ($notebookId -and -not $KeepOpen -and (-not $UseCurrentCache -or $CloneHost)) {
127 $app.CloseNotebook($notebookId, $false)
128 }
129 } catch {
130 if ($null -eq $failure) { throw }
131 [IO.File]::WriteAllText((Join-Path $output 'cleanup-failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
132 }
133 } finally {
134 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
135 }
136 $app = $null
137 [GC]::Collect()
138 [GC]::WaitForPendingFinalizers()
139 if (-not $UseCurrentCache -and -not $CloneHost) {
140 Get-Process ONENOTE -ErrorAction SilentlyContinue | Wait-Process -Timeout 10
141 }
142}
corpus/link-edit/editor/cold/source.json created+14
...@@ -0,0 +1,14 @@
1[
2 {
3 "path": "Open Notebook.onetoc2",
4 "bytes": 1604,
5 "sha256": "7cf4bd0ba068eaccb8e2d3306472ac133a08966eb9dc42cc2b4f7c3b0f44b03f",
6 "mtime_ns": 1790581219409357669
7 },
8 {
9 "path": "links.one",
10 "bytes": 24856,
11 "sha256": "0111a366e9e1b11029856360d54a7fb6665d47184a6622321e5639b945051811",
12 "mtime_ns": 1790581219407781429
13 }
14]
corpus/link-edit/editor/cold/teardown.json created+1
...@@ -0,0 +1 @@
1{"absent": true}
corpus/link-edit/editor/expected.json created+28
...@@ -0,0 +1,28 @@
1{
2 "hrefs": [
3 "onenote:#Link%20target&section-id={912D18F5-8E97-4C22-B11D-AA61E6E7D4B1}&page-id={54104D3B-F876-47C6-8009-D90D3DEC5217}&end&base-path=C:\\one-tests\\runs\\capture\\notebook\\links.one",
4 "http://www.example.com",
5 "https://example.org/a?b=1",
6 "http://example.net/end",
7 "https://doc.rust-lang.org/book/",
8 "http://example.net/x",
9 "https://example.com/only",
10 "onenote:#Link%20target&section-id={912D18F5-8E97-4C22-B11D-AA61E6E7D4B1}&page-id={54104D3B-F876-47C6-8009-D90D3DEC5217}&end&base-path=C:\\one-tests\\runs\\capture\\notebook\\links.one",
11 "onenote:#Linking%20page&section-id={912D18F5-8E97-4C22-B11D-AA61E6E7D4B1}&page-id={3225B89C-1ACE-4B05-A5C8-FF6E178A21D7}&object-id={97735122-3000-4D26-9B1F-D5B4D1A325D0}&24&base-path=C:\\one-tests\\runs\\capture\\notebook\\links.one",
12 "ftp://h.example/f",
13 "https://example.com/moved"
14 ],
15 "mathml": [
16 "<mml:msub><mml:mi>a</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo>+</mml:mo><mml:msub><mml:mi>b</mml:mi><mml:mn>2</mml:mn></mml:msub>",
17 "<mml:msubsup><mml:mo stretchy=\"false\">∫</mml:mo><mml:mn>0</mml:mn><mml:mn>1</mml:mn></mml:msubsup><mml:mrow><mml:mi>x</mml:mi><mml:mo>&nbsp;</mml:mo><mml:mi>d</mml:mi><mml:mi>x</mml:mi><mml:mo>&nbsp;</mml:mo></mml:mrow>",
18 "<mml:mfrac><mml:mrow><mml:mi>a</mml:mi><mml:mo>+</mml:mo><mml:mi>b</mml:mi></mml:mrow><mml:mrow><mml:mi>c</mml:mi><mml:mo>+</mml:mo><mml:mi>d</mml:mi></mml:mrow></mml:mfrac>",
19 "<mml:msqrt><mml:mi>x</mml:mi></mml:msqrt><mml:mo>+</mml:mo><mml:mn>1</mml:mn><mml:mo>&nbsp;</mml:mo>",
20 "<mml:munderover><mml:mo stretchy=\"false\">∑</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>n</mml:mi></mml:munderover><mml:mrow><mml:mi>i</mml:mi><mml:mo>&nbsp;</mml:mo></mml:mrow>",
21 "<mml:mtable><mml:mtr><mml:mtd><mml:mn>1</mml:mn></mml:mtd><mml:mtd><mml:mn>2</mml:mn></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mn>3</mml:mn></mml:mtd><mml:mtd><mml:mn>4</mml:mn></mml:mtd></mml:mtr></mml:mtable>",
22 "<mml:mi>e</mml:mi><mml:mo>^</mml:mo><mml:mo fence=\"false\">(</mml:mo><mml:mi>x</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn><mml:mo fence=\"false\">)</mml:mo>",
23 "<mml:mi>𝜋</mml:mi><mml:msup><mml:mi>r</mml:mi><mml:mn>2</mml:mn></mml:msup>",
24 "<mml:msup><mml:mi>y</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mo>+</mml:mo>",
25 "<mml:mn>1</mml:mn><mml:mo>&nbsp;</mml:mo>",
26 "<mml:mfrac><mml:mrow><mml:mi>a</mml:mi><mml:mo>+</mml:mo><mml:mi>b</mml:mi></mml:mrow><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:maligngroup/><mml:mi>c</mml:mi></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:maligngroup/><mml:mo>+</mml:mo><mml:mi>d</mml:mi></mml:mtd></mml:mtr></mml:mtable></mml:mrow></mml:mfrac>"
27 ]
28}
\ No newline at end of file
corpus/link-edit/native-typed/before-read/environment.json created+7
...@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-AFF74949",
5 "cold": true,
6 "onenote": "14.0.4763.1000"
7}
corpus/link-edit/native-typed/before-read/hierarchy.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{41F2DADE-F5F0-433F-8ECC-52B84008DDE6}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-28T01:31:26.000Z" color="#B49EDE"><one:Section name="links" ID="{6A1E8E3D-937E-0251-09BA-580AD5F39CC6}{1}{B0}" path="C:\one-tests\runs\capture\notebook\links.one" lastModifiedTime="2026-09-28T01:31:26.000Z" color="#8AA8E4"><one:Page ID="{CB1348B4-FB69-0D6C-1F11-786A2D10153D}{1}{B0}" name="Read about Rust ﷟HYPERLINK &quot;https://example.invalid/rust&quot;the Rust site" dateTime="2026-09-12T09:49:16.000Z" lastModifiedTime="2026-09-28T01:31:15.000Z" pageLevel="1"/></one:Section></one:Notebook>
corpus/link-edit/native-typed/before-read/page-000.xml created+3
...@@ -0,0 +1,3 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{CB1348B4-FB69-0D6C-1F11-786A2D10153D}{1}{B0}" name="Read about Rust ﷟HYPERLINK &quot;https://example.invalid/rust&quot;the Rust site" dateTime="2026-09-12T09:49:16.000Z" lastModifiedTime="2026-09-28T01:31:15.000Z" pageLevel="1" style="font-family:Calibri;font-size:11.0pt" lang="en-US"><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="Author" lastModifiedBy="Author" lastModifiedTime="2026-09-12T09:49:16.000Z" objectID="{6A1E8E3D-937E-0251-09BA-580AD5F39CC6}{23}{B0}"><one:Position x="36.0" y="36.0" z="0"/><one:Size width="198.3904571533203" height="13.4277172088623"/><one:Indents><one:Indent level="0" indent="0.0"/></one:Indents><one:OEChildren><one:OE creationTime="2026-09-12T09:49:16.000Z" lastModifiedTime="2026-09-12T09:49:16.000Z" objectID="{6A1E8E3D-937E-0251-09BA-580AD5F39CC6}{24}{B0}" alignment="left"><one:T><![CDATA[Read about Rust <a
3href="https://example.invalid/rust">the Rust site</a>]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/link-edit/native-typed/before-read/payloads.json created+3
...@@ -0,0 +1,3 @@
1[
2
3]
\ No newline at end of file
corpus/link-edit/native-typed/cold/commands.jsonl created+3
...@@ -0,0 +1,3 @@
1{"command": "powershell -NoProfile -Command \"Expand-Archive -LiteralPath C:\\one-tests\\transfer.zip -DestinationPath C:\\one-tests\\runs\\capture\\notebook\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-367C90E8 -ExpectedPages 1 -KeepOpen", "exit": 0, "stdout": "Read 1 sections and 1 pages.\r\n", "stderr": "", "error": null}
3{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\read -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
corpus/link-edit/native-typed/cold/machine.json created+1
...@@ -0,0 +1 @@
1{"name": "m6-367c90e8", "hostname": "ONE-M6-367C90E8"}
corpus/link-edit/native-typed/cold/read/environment.json created+7
...@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-367C90E8",
5 "cold": true,
6 "onenote": "14.0.4763.1000"
7}
corpus/link-edit/native-typed/cold/read/hierarchy.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{0F4C9CD6-B03C-4483-B9D6-7B666F385EBA}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-28T05:41:37.000Z" color="#B49EDE"><one:Section name="links" ID="{183267C0-E406-07DC-0B96-9F1B5960351B}{1}{B0}" path="C:\one-tests\runs\capture\notebook\links.one" lastModifiedTime="2026-09-28T05:41:37.000Z" color="#8AA8E4"><one:Page ID="{B93FA149-8C11-08E1-1D3D-BF7BA183BCE0}{1}{B0}" name="Read about Rust the Rust site" dateTime="2026-09-12T09:49:16.000Z" lastModifiedTime="2026-09-28T01:32:56.000Z" pageLevel="1"/></one:Section></one:Notebook>
corpus/link-edit/native-typed/cold/read/page-000.navigation.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Read about Rust the Rust site - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/link-edit/native-typed/cold/read/page-000.png created
Binary files /dev/null and b/corpus/link-edit/native-typed/cold/read/page-000.png differ
corpus/link-edit/native-typed/cold/read/page-000.xml created+21
...@@ -0,0 +1,21 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{B93FA149-8C11-08E1-1D3D-BF7BA183BCE0}{1}{B0}" name="Read about Rust the Rust site" dateTime="2026-09-12T09:49:16.000Z" lastModifiedTime="2026-09-28T01:32:56.000Z" pageLevel="1" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="Author" lastModifiedBy="Author" lastModifiedTime="2026-09-28T01:32:56.000Z" objectID="{183267C0-E406-07DC-0B96-9F1B5960351B}{23}{B0}"><one:Position x="36.0" y="36.0" z="0"/><one:Size width="247.9598236083984" height="281.9819946289062"/><one:Indents><one:Indent level="0" indent="0.0"/></one:Indents><one:OEChildren><one:OE creationTime="2026-09-12T09:49:16.000Z" lastModifiedTime="2026-09-28T01:32:38.000Z" objectID="{183267C0-E406-07DC-0B96-9F1B5960351B}{24}{B0}" alignment="left" style="font-family:Calibri;font-size:11.0pt"><one:T><![CDATA[Read about Rust <a
3href="https://example.invalid/rust">the Rust site</a>]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:43.000Z" lastModifiedTime="2026-09-28T01:32:49.000Z" objectID="{F739B035-CE59-4E3D-A5F7-400AA3776C53}{12}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[T <a
4href="http://www.example.com">www.example.com</a> z]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:43.000Z" lastModifiedTime="2026-09-28T01:32:49.000Z" objectID="{F739B035-CE59-4E3D-A5F7-400AA3776C53}{17}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[T <a
5href="http://example.org/a?b=1">http://example.org/a?b=1</a> z]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:43.000Z" lastModifiedTime="2026-09-28T01:32:49.000Z" objectID="{F739B035-CE59-4E3D-A5F7-400AA3776C53}{19}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[T <a
6href="https://a.example/p">https://a.example/p</a>. Z]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:44.000Z" lastModifiedTime="2026-09-28T01:32:49.000Z" objectID="{F739B035-CE59-4E3D-A5F7-400AA3776C53}{21}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[T (<a
7href="http://b.example/q">http://b.example/q</a>) z]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:44.000Z" lastModifiedTime="2026-09-28T01:32:49.000Z" objectID="{F739B035-CE59-4E3D-A5F7-400AA3776C53}{23}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[T <a
8href="ftp://c.example/f">ftp://c.example/f</a> z]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:44.000Z" lastModifiedTime="2026-09-28T01:32:49.000Z" objectID="{F739B035-CE59-4E3D-A5F7-400AA3776C53}{25}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[T <a
9href="mailto:me@example.com">mailto:me@example.com</a> z]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:45.000Z" lastModifiedTime="2026-09-28T01:32:49.000Z" objectID="{F739B035-CE59-4E3D-A5F7-400AA3776C53}{27}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[T me@example.com z]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:45.000Z" lastModifiedTime="2026-09-28T01:32:49.000Z" objectID="{F739B035-CE59-4E3D-A5F7-400AA3776C53}{29}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[T example.com z]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:45.000Z" lastModifiedTime="2026-09-28T01:32:45.000Z" objectID="{F739B035-CE59-4E3D-A5F7-400AA3776C53}{31}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[T <a
10href="onenote:#Page&amp;section-id={x}">onenote:#Page&amp;section-id={x}</a> z]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:46.000Z" lastModifiedTime="2026-09-28T01:32:49.000Z" objectID="{F739B035-CE59-4E3D-A5F7-400AA3776C53}{33}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[T <a
11href="file:///c:\windows">file://c:/windows</a> z]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:46.000Z" lastModifiedTime="2026-09-28T01:32:49.000Z" objectID="{F739B035-CE59-4E3D-A5F7-400AA3776C53}{35}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[T <a
12href="file:///\\server\share\f">\\server\share\f</a> z]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:47.000Z" lastModifiedTime="2026-09-28T01:32:49.000Z" objectID="{F739B035-CE59-4E3D-A5F7-400AA3776C53}{37}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[T <a
13href="news:comp.lang">news:comp.lang</a> z]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:47.000Z" lastModifiedTime="2026-09-28T01:32:49.000Z" objectID="{F739B035-CE59-4E3D-A5F7-400AA3776C53}{39}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[T <a
14href="http://www.d.example">www.d.example</a>, z]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:47.000Z" lastModifiedTime="2026-09-28T01:32:49.000Z" objectID="{F739B035-CE59-4E3D-A5F7-400AA3776C53}{41}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[T <a
15href="HTTP://E.EXAMPLE/UP">HTTP://E.EXAMPLE/UP</a> z]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:48.000Z" lastModifiedTime="2026-09-28T01:32:49.000Z" objectID="{F739B035-CE59-4E3D-A5F7-400AA3776C53}{43}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[T <a
16href="https://f.example/a_b-c~d%20e#frag">https://f.example/a_b-c~d%20e#frag</a> z]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:48.000Z" lastModifiedTime="2026-09-28T01:32:49.000Z" objectID="{F739B035-CE59-4E3D-A5F7-400AA3776C53}{45}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[See <a
17href="http://g.example/end">http://g.example/end</a>]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:49.000Z" lastModifiedTime="2026-09-28T01:32:49.000Z" objectID="{F739B035-CE59-4E3D-A5F7-400AA3776C53}{47}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Zplain words <a
18href="http://example.net/x">here</a>Q]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:51.000Z" lastModifiedTime="2026-09-28T01:32:51.000Z" objectID="{F739B035-CE59-4E3D-A5F7-400AA3776C53}{51}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Select <a
19href="https://example.com/selected">me please</a>]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:54.000Z" lastModifiedTime="2026-09-28T01:32:55.000Z" objectID="{F739B035-CE59-4E3D-A5F7-400AA3776C53}{53}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[<a
20href="https://example.com/only">https://example.com/only</a> tail]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:56.000Z" lastModifiedTime="2026-09-28T01:32:56.000Z" objectID="{F739B035-CE59-4E3D-A5F7-400AA3776C53}{55}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Gone <a
21href="ftp://h.example/f">ftp://h.example/f</a> z]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/link-edit/native-typed/cold/read/payloads.json created+3
...@@ -0,0 +1,3 @@
1[
2
3]
\ No newline at end of file
corpus/link-edit/native-typed/cold/run.json created+18
...@@ -0,0 +1,18 @@
1{
2 "notebook": "/private/tmp/cands/native-typed-copy",
3 "expected_pages": 1,
4 "author": null,
5 "author_timeout_seconds": 600,
6 "inspect": false,
7 "collect_notebook": false,
8 "base": {
9 "file": "win7-office-base.qcow2",
10 "format": "qcow2",
11 "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346",
12 "virtual_size": 68719476736
13 },
14 "scripts": {
15 "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331",
16 "read.ps1": "04013bfcccee40a17e2a225f9b9e96f40a3a8daad9e350609654f8eda3ccbb41"
17 }
18}
corpus/link-edit/native-typed/cold/scripts/cold.ps1 created+27
...@@ -0,0 +1,27 @@
1param([Parameter(Mandatory=$true)][string]$Root, [string]$CloneHost = '')
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
5if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
6 throw 'Choose a run directly below C:\one-tests\runs.'
7}
8if (Get-Process ONENOTE -ErrorAction SilentlyContinue) { throw 'Close OneNote before resetting its test cache.' }
9$key = 'HKCU:\Software\Microsoft\Office\14.0\OneNote'
10if ($CloneHost) {
11 if ($CloneHost -notmatch '^ONE-[A-Z0-9-]+$' -or [Environment]::MachineName -ne $CloneHost) {
12 throw 'The disposable clone hostname does not match this machine.'
13 }
14 New-Item "$key\Options\Paths" -Force | Out-Null
15 New-ItemProperty "$key\Options\Paths" -Name UnfiledNotesSection -PropertyType ExpandString -Value 'C:\one-tests\Loose.one' -Force | Out-Null
16} elseif ((Get-ItemProperty "$key\Options\Paths").UnfiledNotesSection -ne 'C:\one-tests\Loose.one' -or
17 -not (Test-Path 'C:\one-tests\profile-original-cache')) {
18 throw 'Park the personal OneNote profile before resetting the test cache.'
19}
20$cache = Join-Path $env:LOCALAPPDATA 'Microsoft\OneNote\14.0'
21$parked = Join-Path 'C:\one-tests\caches' ([IO.Path]::GetFileName($root))
22if (Test-Path $parked) { throw 'Choose a new run; its parked cache already exists.' }
23New-Item -ItemType Directory -Path 'C:\one-tests\caches' -Force | Out-Null
24if (Test-Path $cache) { Move-Item -LiteralPath $cache -Destination $parked }
25if (Test-Path "$key\OpenNotebooks") { Remove-Item "$key\OpenNotebooks" -Recurse }
26New-Item "$key\OpenNotebooks" | Out-Null
27New-ItemProperty "$key\OpenNotebooks" -Name '1' -PropertyType String -Value "$root\notebook" | Out-Null
corpus/link-edit/native-typed/cold/scripts/read.ps1 created+142
...@@ -0,0 +1,142 @@
1param(
2 [Parameter(Mandatory=$true)][string]$Root,
3 [int]$ExpectedPages = -1,
4 [switch]$UseCurrentCache,
5 [switch]$Pdf,
6 [switch]$KeepOpen,
7 [string]$CloneHost = ''
8)
9Set-StrictMode -Version Latest
10$ErrorActionPreference = 'Stop'
11$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
12if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
13 throw 'Choose a run directly below C:\one-tests\runs.'
14}
15$notebook = Join-Path $root 'notebook'
16$output = Join-Path $root 'read'
17if (Test-Path $output) { throw 'Choose a new read destination.' }
18if ($UseCurrentCache) {
19 if ((Get-ItemProperty 'HKCU:\Software\Microsoft\Office\14.0\OneNote\Options\Paths').UnfiledNotesSection -ne 'C:\one-tests\Loose.one') {
20 throw 'Park the personal OneNote profile before reading test notebooks.'
21 }
22} else {
23 & "$PSScriptRoot\cold-current.ps1" -Root $root -CloneHost $CloneHost
24}
25New-Item -ItemType Directory -Path $output | Out-Null
26$app = New-Object -ComObject OneNote.Application
27$notebookId = ''
28$failure = $null
29try {
30 $app.OpenHierarchy($notebook, '', [ref]$notebookId, 0)
31 $process = Get-Process ONENOTE
32 @{ hostname = [Environment]::MachineName; onenote = $process.MainModule.FileVersionInfo.FileVersion;
33 powershell = $PSVersionTable.PSVersion.ToString(); schema = 'xs2010'; cold = (-not $UseCurrentCache.IsPresent) } |
34 ConvertTo-Json | Set-Content (Join-Path $output 'environment.json') -Encoding UTF8
35 $sections = @()
36 foreach ($file in @(Get-ChildItem $notebook -Recurse | Where-Object { $_.Extension -eq '.one' })) {
37 $id = ''
38 $app.OpenHierarchy($file.FullName, '', [ref]$id, 0)
39 $sections += $id
40 }
41 $deadline = [DateTime]::UtcNow.AddSeconds(300)
42 $previous = ''
43 $lastChange = ''
44 $stableSince = [DateTime]::UtcNow
45 $settled = $false
46 do {
47 $pages = @{}
48 foreach ($section in $sections) {
49 $hierarchy = ''
50 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
51 [xml]$xml = $hierarchy
52 foreach ($node in $xml.SelectNodes('//*[@path]')) {
53 if (-not $node.GetAttribute('path').StartsWith("$notebook\", [StringComparison]::OrdinalIgnoreCase)) {
54 throw 'OneNote opened a section outside the copied notebook.'
55 }
56 }
57 foreach ($node in $xml.SelectNodes('//*[local-name()="Page"]')) {
58 $id = $node.GetAttribute('ID')
59 $content = ''
60 $app.GetPageContent($id, [ref]$content, 1, 1)
61 $pages[$id] = $content
62 }
63 }
64 $signature = [String]::Join('|', @($pages.Keys | Sort-Object | ForEach-Object { $_ + $pages[$_] }))
65 if ($signature -ne $previous) {
66 $lastChange = $previous
67 $previous = $signature
68 $stableSince = [DateTime]::UtcNow
69 }
70 if ((($ExpectedPages -ge 0 -and $pages.Count -eq $ExpectedPages) -or
71 ($ExpectedPages -lt 0 -and $pages.Count -gt 0)) -and
72 ([DateTime]::UtcNow - $stableSince).TotalSeconds -ge 2) { $settled = $true; break }
73 Start-Sleep -Milliseconds 250
74 } while ([DateTime]::UtcNow -lt $deadline)
75 if (-not $settled -or ($ExpectedPages -ge 0 -and $pages.Count -ne $ExpectedPages) -or ($ExpectedPages -lt 0 -and $pages.Count -eq 0)) {
76 [IO.File]::WriteAllText((Join-Path $output 'previous-signature.txt'), $lastChange, [Text.Encoding]::UTF8)
77 $index = 0
78 foreach ($id in @($pages.Keys | Sort-Object)) {
79 [IO.File]::WriteAllText((Join-Path $output ('unsettled-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
80 $index++
81 }
82 $hierarchy = ''
83 $app.GetHierarchy($notebookId, 4, [ref]$hierarchy, 1)
84 [IO.File]::WriteAllText((Join-Path $output 'unsettled-hierarchy.xml'), $hierarchy, [Text.Encoding]::UTF8)
85 throw "Expected $ExpectedPages stable pages; OneNote returned $($pages.Count), settled=$settled."
86 }
87 $index = 0
88 $payloads = @()
89 foreach ($id in @($pages.Keys | Sort-Object)) {
90 [IO.File]::WriteAllText((Join-Path $output ('page-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
91 if ($Pdf) {
92 $pdfPath = Join-Path $output ('page-{0:d3}.pdf' -f $index)
93 $app.NavigateTo($id, '', $false)
94 $app.Publish($id, $pdfPath, 3, '')
95 if (-not (Test-Path $pdfPath) -or (Get-Item $pdfPath).Length -eq 0) { throw 'OneNote did not publish the page PDF.' }
96 }
97 [xml]$page = $pages[$id]
98 foreach ($file in $page.SelectNodes('//*[local-name()="InsertedFile" or local-name()="MediaFile"]')) {
99 $bytes = [IO.File]::ReadAllBytes($file.GetAttribute('pathCache'))
100 $hash = [BitConverter]::ToString([Security.Cryptography.SHA256]::Create().ComputeHash($bytes)).Replace('-', '').ToLowerInvariant()
101 [IO.File]::WriteAllBytes((Join-Path $output ($hash + '.attachment')), $bytes)
102 $payloads += @{ page = $id; object = $file.ParentNode.GetAttribute('objectID');
103 kind = $file.LocalName; name = $file.GetAttribute('preferredName');
104 sha256 = $hash; bytes = $bytes.Length }
105 }
106 $index++
107 }
108 [IO.File]::WriteAllText((Join-Path $output 'payloads.json'), (ConvertTo-Json -InputObject $payloads -Depth 4), [Text.Encoding]::UTF8)
109 $all = ''
110 $app.GetHierarchy($notebookId, 4, [ref]$all, 1)
111 [xml]$finalTree = $all
112 $finalIds = @($finalTree.SelectNodes('//*[local-name()="Page"]') | ForEach-Object { $_.GetAttribute('ID') } | Sort-Object -Unique)
113 if ($finalIds.Count -ne $pages.Count -or @($finalIds | Where-Object { -not $pages.ContainsKey($_) }).Count -ne 0) {
114 throw 'The notebook hierarchy changed while collecting page evidence; repeat the cold read.'
115 }
116 [IO.File]::WriteAllText((Join-Path $output 'hierarchy.xml'), $all, [Text.Encoding]::UTF8)
117 Write-Output "Read $($sections.Count) sections and $($pages.Count) pages."
118} catch {
119 $failure = $_
120 [IO.File]::WriteAllText((Join-Path $output 'failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
121 throw
122} finally {
123 try {
124 try {
125 if ($notebookId -and $CloneHost) { $app.SyncHierarchy($notebookId) }
126 if ($notebookId -and -not $KeepOpen -and (-not $UseCurrentCache -or $CloneHost)) {
127 $app.CloseNotebook($notebookId, $false)
128 }
129 } catch {
130 if ($null -eq $failure) { throw }
131 [IO.File]::WriteAllText((Join-Path $output 'cleanup-failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
132 }
133 } finally {
134 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
135 }
136 $app = $null
137 [GC]::Collect()
138 [GC]::WaitForPendingFinalizers()
139 if (-not $UseCurrentCache -and -not $CloneHost) {
140 Get-Process ONENOTE -ErrorAction SilentlyContinue | Wait-Process -Timeout 10
141 }
142}
corpus/link-edit/native-typed/cold/source.json created+14
...@@ -0,0 +1,14 @@
1[
2 {
3 "path": "Open Notebook.onetoc2",
4 "bytes": 3764,
5 "sha256": "71fdfa91de2db76fdb2ecd3c05a4c5688f9c97c10bfb077bc0945d5e01c8efa2",
6 "mtime_ns": 1790573993933440881
7 },
8 {
9 "path": "links.one",
10 "bytes": 18320,
11 "sha256": "d716882cf7be8938b6506293d1b1542b8f458f8cd7c4ee74b1545848bc8c28c4",
12 "mtime_ns": 1790573993934334422
13 }
14]
corpus/link-edit/native-typed/cold/teardown.json created+1
...@@ -0,0 +1 @@
1{"absent": true}
corpus/link-edit/native-typed/commands.jsonl created+6
...@@ -0,0 +1,6 @@
1{"command": "powershell -NoProfile -Command \"Expand-Archive -LiteralPath C:\\one-tests\\transfer.zip -DestinationPath C:\\one-tests\\runs\\capture\\notebook\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-AFF74949 -ExpectedPages 1 -KeepOpen", "exit": 0, "stdout": "Read 1 sections and 1 pages.\r\n", "stderr": "", "error": null}
3{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
4{"command": "powershell -NoProfile -Command \"Rename-Item C:\\one-tests\\runs\\capture\\read before-read\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
5{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-AFF74949 -UseCurrentCache -ExpectedPages -1", "exit": 0, "stdout": "Read 1 sections and 1 pages.\r\n", "stderr": "", "error": null}
6{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
corpus/link-edit/native-typed/links.ahk created+133
...@@ -0,0 +1,133 @@
1ComObject("OneNote.Application").NavigateTo("{CB1348B4-FB69-0D6C-1F11-786A2D10153D}{1}{B0}", "", false)
2WinWait("ahk_exe ONENOTE.EXE", , 30)
3WinActivate("ahk_exe ONENOTE.EXE")
4WinWaitActive("ahk_exe ONENOTE.EXE", , 30)
5WinMaximize("ahk_exe ONENOTE.EXE")
6Sleep(3000)
7CoordMode("Mouse", "Screen")
8Click(200, 140)
9Sleep(500)
10Send("{End}")
11Sleep(300)
12Send("{Enter}")
13Sleep(500)
14SendText("t www.example.com")
15Send(" ")
16SendText("z")
17Send("{Enter}")
18Sleep(300)
19SendText("t http://example.org/a?b=1")
20Send(" ")
21SendText("z")
22Send("{Enter}")
23Sleep(300)
24SendText("t https://a.example/p.")
25Send(" ")
26SendText("z")
27Send("{Enter}")
28Sleep(300)
29SendText("t (http://b.example/q)")
30Send(" ")
31SendText("z")
32Send("{Enter}")
33Sleep(300)
34SendText("t ftp://c.example/f")
35Send(" ")
36SendText("z")
37Send("{Enter}")
38Sleep(300)
39SendText("t mailto:me@example.com")
40Send(" ")
41SendText("z")
42Send("{Enter}")
43Sleep(300)
44SendText("t me@example.com")
45Send(" ")
46SendText("z")
47Send("{Enter}")
48Sleep(300)
49SendText("t example.com")
50Send(" ")
51SendText("z")
52Send("{Enter}")
53Sleep(300)
54SendText("t onenote:#Page&section-id={x}")
55Send(" ")
56SendText("z")
57Send("{Enter}")
58Sleep(300)
59SendText("t file://c:/windows")
60Send(" ")
61SendText("z")
62Send("{Enter}")
63Sleep(300)
64SendText("t \\server\share\f")
65Send(" ")
66SendText("z")
67Send("{Enter}")
68Sleep(300)
69SendText("t news:comp.lang")
70Send(" ")
71SendText("z")
72Send("{Enter}")
73Sleep(300)
74SendText("t www.d.example,")
75Send(" ")
76SendText("z")
77Send("{Enter}")
78Sleep(300)
79SendText("t HTTP://E.EXAMPLE/UP")
80Send(" ")
81SendText("z")
82Send("{Enter}")
83Sleep(300)
84SendText("t https://f.example/a_b-c~d%20e#frag")
85Send(" ")
86SendText("z")
87Send("{Enter}")
88Sleep(300)
89SendText("see http://g.example/end")
90Send("{Enter}")
91Send("{Up}{End}")
92SendText("Z")
93Send("{Down}{End}")
94Sleep(300)
95SendText("plain words here")
96Send("^{Left}")
97Send("^k")
98Sleep(1500)
99SendText("example.net/x")
100Send("{Enter}")
101Sleep(800)
102Send("{End}")
103SendText("Q")
104Send("{Enter}")
105Sleep(300)
106SendText("select me please")
107Send("+^{Left}+^{Left}")
108Send("^k")
109Sleep(1500)
110SendText("https://example.com/selected")
111Send("{Enter}")
112Sleep(800)
113Send("{End}{Enter}")
114Sleep(300)
115Send("^k")
116Sleep(1500)
117Send("!e")
118Sleep(300)
119SendText("https://example.com/only")
120Send("{Enter}")
121Sleep(800)
122SendText(" tail")
123Send("{Enter}")
124Sleep(300)
125SendText("gone ftp://h.example/f")
126Send(" ")
127SendText("z")
128Send("{Home}^{Right}^{Right}")
129Send("+{F10}")
130Sleep(900)
131Send("r")
132Sleep(800)
133Sleep(2000)
corpus/link-edit/native-typed/links.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Read about Rust the Rust site - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/link-edit/native-typed/links.png created
Binary files /dev/null and b/corpus/link-edit/native-typed/links.png differ
corpus/link-edit/native-typed/machine.json created+1
...@@ -0,0 +1 @@
1{"name": "m6-aff74949", "hostname": "ONE-M6-AFF74949"}
corpus/link-edit/native-typed/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/link-edit/native-typed/notebook/Open Notebook.onetoc2 differ
corpus/link-edit/native-typed/notebook/links.one created
Binary files /dev/null and b/corpus/link-edit/native-typed/notebook/links.one differ
corpus/link-edit/native-typed/read/environment.json created+7
...@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-AFF74949",
5 "cold": false,
6 "onenote": "14.0.4763.1000"
7}
corpus/link-edit/native-typed/read/hierarchy.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{41F2DADE-F5F0-433F-8ECC-52B84008DDE6}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-28T01:32:56.000Z" color="#B49EDE" isCurrentlyViewed="true"><one:Section name="links" ID="{6A1E8E3D-937E-0251-09BA-580AD5F39CC6}{1}{B0}" path="C:\one-tests\runs\capture\notebook\links.one" lastModifiedTime="2026-09-28T01:32:56.000Z" color="#8AA8E4" isCurrentlyViewed="true"><one:Page ID="{CB1348B4-FB69-0D6C-1F11-786A2D10153D}{1}{B0}" name="Read about Rust the Rust site" dateTime="2026-09-12T09:49:16.000Z" lastModifiedTime="2026-09-28T01:32:56.000Z" pageLevel="1" isCurrentlyViewed="true"/></one:Section></one:Notebook>
corpus/link-edit/native-typed/read/page-000.xml created+21
...@@ -0,0 +1,21 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{CB1348B4-FB69-0D6C-1F11-786A2D10153D}{1}{B0}" name="Read about Rust the Rust site" dateTime="2026-09-12T09:49:16.000Z" lastModifiedTime="2026-09-28T01:32:56.000Z" pageLevel="1" isCurrentlyViewed="true" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="Author" lastModifiedBy="Author" lastModifiedTime="2026-09-28T01:32:56.000Z" objectID="{6A1E8E3D-937E-0251-09BA-580AD5F39CC6}{23}{B0}"><one:Position x="36.0" y="36.0" z="0"/><one:Size width="247.9598236083984" height="281.9819946289062"/><one:Indents><one:Indent level="0" indent="0.0"/></one:Indents><one:OEChildren><one:OE creationTime="2026-09-12T09:49:16.000Z" lastModifiedTime="2026-09-28T01:32:38.000Z" objectID="{6A1E8E3D-937E-0251-09BA-580AD5F39CC6}{24}{B0}" alignment="left" style="font-family:Calibri;font-size:11.0pt"><one:T><![CDATA[Read about Rust <a
3href="https://example.invalid/rust">the Rust site</a>]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:43.000Z" lastModifiedTime="2026-09-28T01:32:49.000Z" objectID="{851559C8-B921-4BB0-A7DB-871B2FE4C58E}{12}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[T <a
4href="http://www.example.com">www.example.com</a> z]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:43.000Z" lastModifiedTime="2026-09-28T01:32:49.000Z" objectID="{851559C8-B921-4BB0-A7DB-871B2FE4C58E}{17}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[T <a
5href="http://example.org/a?b=1">http://example.org/a?b=1</a> z]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:43.000Z" lastModifiedTime="2026-09-28T01:32:49.000Z" objectID="{851559C8-B921-4BB0-A7DB-871B2FE4C58E}{19}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[T <a
6href="https://a.example/p">https://a.example/p</a>. Z]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:44.000Z" lastModifiedTime="2026-09-28T01:32:49.000Z" objectID="{851559C8-B921-4BB0-A7DB-871B2FE4C58E}{21}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[T (<a
7href="http://b.example/q">http://b.example/q</a>) z]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:44.000Z" lastModifiedTime="2026-09-28T01:32:49.000Z" objectID="{851559C8-B921-4BB0-A7DB-871B2FE4C58E}{23}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[T <a
8href="ftp://c.example/f">ftp://c.example/f</a> z]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:44.000Z" lastModifiedTime="2026-09-28T01:32:49.000Z" objectID="{851559C8-B921-4BB0-A7DB-871B2FE4C58E}{25}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[T <a
9href="mailto:me@example.com">mailto:me@example.com</a> z]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:45.000Z" lastModifiedTime="2026-09-28T01:32:49.000Z" objectID="{851559C8-B921-4BB0-A7DB-871B2FE4C58E}{27}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[T me@example.com z]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:45.000Z" lastModifiedTime="2026-09-28T01:32:49.000Z" objectID="{851559C8-B921-4BB0-A7DB-871B2FE4C58E}{29}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[T example.com z]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:45.000Z" lastModifiedTime="2026-09-28T01:32:45.000Z" objectID="{851559C8-B921-4BB0-A7DB-871B2FE4C58E}{31}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[T <a
10href="onenote:#Page&amp;section-id={x}">onenote:#Page&amp;section-id={x}</a> z]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:46.000Z" lastModifiedTime="2026-09-28T01:32:49.000Z" objectID="{851559C8-B921-4BB0-A7DB-871B2FE4C58E}{33}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[T <a
11href="file:///c:\windows">file://c:/windows</a> z]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:46.000Z" lastModifiedTime="2026-09-28T01:32:49.000Z" objectID="{851559C8-B921-4BB0-A7DB-871B2FE4C58E}{35}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[T <a
12href="file:///\\server\share\f">\\server\share\f</a> z]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:47.000Z" lastModifiedTime="2026-09-28T01:32:49.000Z" objectID="{851559C8-B921-4BB0-A7DB-871B2FE4C58E}{37}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[T <a
13href="news:comp.lang">news:comp.lang</a> z]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:47.000Z" lastModifiedTime="2026-09-28T01:32:49.000Z" objectID="{851559C8-B921-4BB0-A7DB-871B2FE4C58E}{39}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[T <a
14href="http://www.d.example">www.d.example</a>, z]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:47.000Z" lastModifiedTime="2026-09-28T01:32:49.000Z" objectID="{851559C8-B921-4BB0-A7DB-871B2FE4C58E}{41}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[T <a
15href="HTTP://E.EXAMPLE/UP">HTTP://E.EXAMPLE/UP</a> z]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:48.000Z" lastModifiedTime="2026-09-28T01:32:49.000Z" objectID="{851559C8-B921-4BB0-A7DB-871B2FE4C58E}{43}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[T <a
16href="https://f.example/a_b-c~d%20e#frag">https://f.example/a_b-c~d%20e#frag</a> z]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:48.000Z" lastModifiedTime="2026-09-28T01:32:49.000Z" objectID="{851559C8-B921-4BB0-A7DB-871B2FE4C58E}{45}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[See <a
17href="http://g.example/end">http://g.example/end</a>]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:49.000Z" lastModifiedTime="2026-09-28T01:32:49.000Z" objectID="{851559C8-B921-4BB0-A7DB-871B2FE4C58E}{47}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Zplain words <a
18href="http://example.net/x">here</a>Q]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:51.000Z" lastModifiedTime="2026-09-28T01:32:51.000Z" objectID="{851559C8-B921-4BB0-A7DB-871B2FE4C58E}{51}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Select <a
19href="https://example.com/selected">me please</a>]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:54.000Z" lastModifiedTime="2026-09-28T01:32:55.000Z" objectID="{851559C8-B921-4BB0-A7DB-871B2FE4C58E}{53}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[<a
20href="https://example.com/only">https://example.com/only</a> tail]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:32:56.000Z" lastModifiedTime="2026-09-28T01:32:56.000Z" objectID="{851559C8-B921-4BB0-A7DB-871B2FE4C58E}{55}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Gone <a
21href="ftp://h.example/f">ftp://h.example/f</a> z]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/link-edit/native-typed/read/payloads.json created+3
...@@ -0,0 +1,3 @@
1[
2
3]
\ No newline at end of file
corpus/link-edit/native-typed/run.json created+18
...@@ -0,0 +1,18 @@
1{
2 "notebook": "/Users/clo/dev/one/corpus/link-edit/candidate",
3 "expected_pages": 1,
4 "author": null,
5 "author_timeout_seconds": 600,
6 "inspect": false,
7 "collect_notebook": true,
8 "base": {
9 "file": "win7-office-base.qcow2",
10 "format": "qcow2",
11 "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346",
12 "virtual_size": 68719476736
13 },
14 "scripts": {
15 "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331",
16 "read.ps1": "04013bfcccee40a17e2a225f9b9e96f40a3a8daad9e350609654f8eda3ccbb41"
17 }
18}
corpus/link-edit/native-typed/scripts/cold.ps1 created+27
...@@ -0,0 +1,27 @@
1param([Parameter(Mandatory=$true)][string]$Root, [string]$CloneHost = '')
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
5if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
6 throw 'Choose a run directly below C:\one-tests\runs.'
7}
8if (Get-Process ONENOTE -ErrorAction SilentlyContinue) { throw 'Close OneNote before resetting its test cache.' }
9$key = 'HKCU:\Software\Microsoft\Office\14.0\OneNote'
10if ($CloneHost) {
11 if ($CloneHost -notmatch '^ONE-[A-Z0-9-]+$' -or [Environment]::MachineName -ne $CloneHost) {
12 throw 'The disposable clone hostname does not match this machine.'
13 }
14 New-Item "$key\Options\Paths" -Force | Out-Null
15 New-ItemProperty "$key\Options\Paths" -Name UnfiledNotesSection -PropertyType ExpandString -Value 'C:\one-tests\Loose.one' -Force | Out-Null
16} elseif ((Get-ItemProperty "$key\Options\Paths").UnfiledNotesSection -ne 'C:\one-tests\Loose.one' -or
17 -not (Test-Path 'C:\one-tests\profile-original-cache')) {
18 throw 'Park the personal OneNote profile before resetting the test cache.'
19}
20$cache = Join-Path $env:LOCALAPPDATA 'Microsoft\OneNote\14.0'
21$parked = Join-Path 'C:\one-tests\caches' ([IO.Path]::GetFileName($root))
22if (Test-Path $parked) { throw 'Choose a new run; its parked cache already exists.' }
23New-Item -ItemType Directory -Path 'C:\one-tests\caches' -Force | Out-Null
24if (Test-Path $cache) { Move-Item -LiteralPath $cache -Destination $parked }
25if (Test-Path "$key\OpenNotebooks") { Remove-Item "$key\OpenNotebooks" -Recurse }
26New-Item "$key\OpenNotebooks" | Out-Null
27New-ItemProperty "$key\OpenNotebooks" -Name '1' -PropertyType String -Value "$root\notebook" | Out-Null
corpus/link-edit/native-typed/scripts/read.ps1 created+142
...@@ -0,0 +1,142 @@
1param(
2 [Parameter(Mandatory=$true)][string]$Root,
3 [int]$ExpectedPages = -1,
4 [switch]$UseCurrentCache,
5 [switch]$Pdf,
6 [switch]$KeepOpen,
7 [string]$CloneHost = ''
8)
9Set-StrictMode -Version Latest
10$ErrorActionPreference = 'Stop'
11$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
12if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
13 throw 'Choose a run directly below C:\one-tests\runs.'
14}
15$notebook = Join-Path $root 'notebook'
16$output = Join-Path $root 'read'
17if (Test-Path $output) { throw 'Choose a new read destination.' }
18if ($UseCurrentCache) {
19 if ((Get-ItemProperty 'HKCU:\Software\Microsoft\Office\14.0\OneNote\Options\Paths').UnfiledNotesSection -ne 'C:\one-tests\Loose.one') {
20 throw 'Park the personal OneNote profile before reading test notebooks.'
21 }
22} else {
23 & "$PSScriptRoot\cold-current.ps1" -Root $root -CloneHost $CloneHost
24}
25New-Item -ItemType Directory -Path $output | Out-Null
26$app = New-Object -ComObject OneNote.Application
27$notebookId = ''
28$failure = $null
29try {
30 $app.OpenHierarchy($notebook, '', [ref]$notebookId, 0)
31 $process = Get-Process ONENOTE
32 @{ hostname = [Environment]::MachineName; onenote = $process.MainModule.FileVersionInfo.FileVersion;
33 powershell = $PSVersionTable.PSVersion.ToString(); schema = 'xs2010'; cold = (-not $UseCurrentCache.IsPresent) } |
34 ConvertTo-Json | Set-Content (Join-Path $output 'environment.json') -Encoding UTF8
35 $sections = @()
36 foreach ($file in @(Get-ChildItem $notebook -Recurse | Where-Object { $_.Extension -eq '.one' })) {
37 $id = ''
38 $app.OpenHierarchy($file.FullName, '', [ref]$id, 0)
39 $sections += $id
40 }
41 $deadline = [DateTime]::UtcNow.AddSeconds(300)
42 $previous = ''
43 $lastChange = ''
44 $stableSince = [DateTime]::UtcNow
45 $settled = $false
46 do {
47 $pages = @{}
48 foreach ($section in $sections) {
49 $hierarchy = ''
50 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
51 [xml]$xml = $hierarchy
52 foreach ($node in $xml.SelectNodes('//*[@path]')) {
53 if (-not $node.GetAttribute('path').StartsWith("$notebook\", [StringComparison]::OrdinalIgnoreCase)) {
54 throw 'OneNote opened a section outside the copied notebook.'
55 }
56 }
57 foreach ($node in $xml.SelectNodes('//*[local-name()="Page"]')) {
58 $id = $node.GetAttribute('ID')
59 $content = ''
60 $app.GetPageContent($id, [ref]$content, 1, 1)
61 $pages[$id] = $content
62 }
63 }
64 $signature = [String]::Join('|', @($pages.Keys | Sort-Object | ForEach-Object { $_ + $pages[$_] }))
65 if ($signature -ne $previous) {
66 $lastChange = $previous
67 $previous = $signature
68 $stableSince = [DateTime]::UtcNow
69 }
70 if ((($ExpectedPages -ge 0 -and $pages.Count -eq $ExpectedPages) -or
71 ($ExpectedPages -lt 0 -and $pages.Count -gt 0)) -and
72 ([DateTime]::UtcNow - $stableSince).TotalSeconds -ge 2) { $settled = $true; break }
73 Start-Sleep -Milliseconds 250
74 } while ([DateTime]::UtcNow -lt $deadline)
75 if (-not $settled -or ($ExpectedPages -ge 0 -and $pages.Count -ne $ExpectedPages) -or ($ExpectedPages -lt 0 -and $pages.Count -eq 0)) {
76 [IO.File]::WriteAllText((Join-Path $output 'previous-signature.txt'), $lastChange, [Text.Encoding]::UTF8)
77 $index = 0
78 foreach ($id in @($pages.Keys | Sort-Object)) {
79 [IO.File]::WriteAllText((Join-Path $output ('unsettled-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
80 $index++
81 }
82 $hierarchy = ''
83 $app.GetHierarchy($notebookId, 4, [ref]$hierarchy, 1)
84 [IO.File]::WriteAllText((Join-Path $output 'unsettled-hierarchy.xml'), $hierarchy, [Text.Encoding]::UTF8)
85 throw "Expected $ExpectedPages stable pages; OneNote returned $($pages.Count), settled=$settled."
86 }
87 $index = 0
88 $payloads = @()
89 foreach ($id in @($pages.Keys | Sort-Object)) {
90 [IO.File]::WriteAllText((Join-Path $output ('page-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
91 if ($Pdf) {
92 $pdfPath = Join-Path $output ('page-{0:d3}.pdf' -f $index)
93 $app.NavigateTo($id, '', $false)
94 $app.Publish($id, $pdfPath, 3, '')
95 if (-not (Test-Path $pdfPath) -or (Get-Item $pdfPath).Length -eq 0) { throw 'OneNote did not publish the page PDF.' }
96 }
97 [xml]$page = $pages[$id]
98 foreach ($file in $page.SelectNodes('//*[local-name()="InsertedFile" or local-name()="MediaFile"]')) {
99 $bytes = [IO.File]::ReadAllBytes($file.GetAttribute('pathCache'))
100 $hash = [BitConverter]::ToString([Security.Cryptography.SHA256]::Create().ComputeHash($bytes)).Replace('-', '').ToLowerInvariant()
101 [IO.File]::WriteAllBytes((Join-Path $output ($hash + '.attachment')), $bytes)
102 $payloads += @{ page = $id; object = $file.ParentNode.GetAttribute('objectID');
103 kind = $file.LocalName; name = $file.GetAttribute('preferredName');
104 sha256 = $hash; bytes = $bytes.Length }
105 }
106 $index++
107 }
108 [IO.File]::WriteAllText((Join-Path $output 'payloads.json'), (ConvertTo-Json -InputObject $payloads -Depth 4), [Text.Encoding]::UTF8)
109 $all = ''
110 $app.GetHierarchy($notebookId, 4, [ref]$all, 1)
111 [xml]$finalTree = $all
112 $finalIds = @($finalTree.SelectNodes('//*[local-name()="Page"]') | ForEach-Object { $_.GetAttribute('ID') } | Sort-Object -Unique)
113 if ($finalIds.Count -ne $pages.Count -or @($finalIds | Where-Object { -not $pages.ContainsKey($_) }).Count -ne 0) {
114 throw 'The notebook hierarchy changed while collecting page evidence; repeat the cold read.'
115 }
116 [IO.File]::WriteAllText((Join-Path $output 'hierarchy.xml'), $all, [Text.Encoding]::UTF8)
117 Write-Output "Read $($sections.Count) sections and $($pages.Count) pages."
118} catch {
119 $failure = $_
120 [IO.File]::WriteAllText((Join-Path $output 'failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
121 throw
122} finally {
123 try {
124 try {
125 if ($notebookId -and $CloneHost) { $app.SyncHierarchy($notebookId) }
126 if ($notebookId -and -not $KeepOpen -and (-not $UseCurrentCache -or $CloneHost)) {
127 $app.CloseNotebook($notebookId, $false)
128 }
129 } catch {
130 if ($null -eq $failure) { throw }
131 [IO.File]::WriteAllText((Join-Path $output 'cleanup-failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
132 }
133 } finally {
134 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
135 }
136 $app = $null
137 [GC]::Collect()
138 [GC]::WaitForPendingFinalizers()
139 if (-not $UseCurrentCache -and -not $CloneHost) {
140 Get-Process ONENOTE -ErrorAction SilentlyContinue | Wait-Process -Timeout 10
141 }
142}
corpus/link-edit/native-typed/source.json created+14
...@@ -0,0 +1,14 @@
1[
2 {
3 "path": "Open Notebook.onetoc2",
4 "bytes": 1604,
5 "sha256": "6731c4214437fe1cb6aa347b0004542b7f7e4260f47773bf9ec07c4e894de47c",
6 "mtime_ns": 1789206821144296891
7 },
8 {
9 "path": "links.one",
10 "bytes": 6128,
11 "sha256": "0cfb356ed374fe919f73fb73fdbe5d66e0baa484d4119f70074d9f869d6db78a",
12 "mtime_ns": 1789206821116200533
13 }
14]
corpus/link-edit/native-typed/teardown.json created+1
...@@ -0,0 +1 @@
1{"absent": true}
corpus/math-edit/README.md+38
...@@ -53,3 +53,41 @@ corpus/link-edit/candidate OUTPUT` and the written one with...@@ -53,3 +53,41 @@ corpus/link-edit/candidate OUTPUT` and the written one with
53`ONESTORE_MATH_EXPORT` set while running the test, then53`ONESTORE_MATH_EXPORT` set while running the test, then
54`tools/native_runner.py OUTPUT COLD --expected-pages 1 --collect-notebook54`tools/native_runner.py OUTPUT COLD --expected-pages 1 --collect-notebook
55--screenshots`.55--screenshots`.
56
57`native-editor-4/` is a fourth `tools/native_math.py` session repeating most
58of the forms above and adding `e^(x+1)`, `\alpha+\beta` (which stays linear
59text: one run between objects, Greek in mathematical italic),
60`(a+b)/(c+d)`, `x^2+y^2=z^2`, `lim_(n\to\infty) a_n` (a function-apply
61object, 17 with U+2061, around a limit; not built by the linear reading
62here) and `{a+b}`; `x_i^2` kept the editor's “Type equation here.”
63placeholder (11 with `⬚`), as the editor was still opening.
64
65`native-linear/` is the linear form OneNote 2010 shows and stores after
66Equation Tools, Design, Linear, applied to each equation of a session like the
67fourth one plus `x^2+1` and `(a+b)/(c+d)`. It was authored interactively in a
68lab clone through its desktop agent (clicking each equation, then Linear), as
69the key tips and the equation's context menu would not switch it from a
70script; `notebook/links.one` is the section OneNote saved on closing. Each
71equation becomes one run of math between objects: letters in mathematical
72italic, arguments of more than one factor in parentheses, an n-ary body after
73`▒` inside `〖〗`, `√(n&x)` for a root's degree and `∛` for a cube root,
74`■(1&2@3&4)` and `█(x&=1@y&=2)` for arrays, `𝑥┴2`, `𝑥┬2`, `□𝑥`, `▭𝑥`, `¯𝑥`,
75and an accent after a no-break space.
76
77`crates/onestore/tests/page_math.rs` builds every typed expression with
78`Math::from_linear` and compares it with what the editor stored, run by run,
79writes each tree back as the linear text OneNote showed, and rebuilds every
80native equation from its linear form; `crates/canvas/src/editor/equation.rs`
81types the same keys into the canvas editor, building up at each space.
82
83`native-enter/` is OneNote 2010 pressing Enter around equations and a link,
84driven by `tools/native_enter.py`: Enter at an equation's start moves the
85equation to a paragraph of its own; inside the equation's own row it stores a
86line break (`\r`, a plain run) in the paragraph, the equation's two sides
87becoming two equations (two MathML blocks in the read); inside a fraction's
88denominator it makes that argument an equation array (15, `█`) of the two
89parts; at a link's start the whole link, field code and label, moves down. An
90interactive session showed that Enter inside a link opens it instead.
91`crates/canvas/tests/links_equations.rs` presses the same keys in the editor and
92compares every paragraph's runs, and `page_math.rs` compares each equation's
93MathML with the read.
corpus/math-edit/native-editor-4/before-read/environment.json created+7
...@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-649AC1D2",
5 "cold": true,
6 "onenote": "14.0.4763.1000"
7}
corpus/math-edit/native-editor-4/before-read/hierarchy.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{7C806B9F-5A4D-4B11-825E-82AA1CE0C357}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-28T01:48:20.000Z" color="#B49EDE"><one:Section name="links" ID="{B9C7BD81-9DC3-0446-2D55-5B561FA3FB12}{1}{B0}" path="C:\one-tests\runs\capture\notebook\links.one" lastModifiedTime="2026-09-28T01:48:20.000Z" color="#8AA8E4"><one:Page ID="{18CA7B08-F5D4-0B7B-3BFE-7B36E74072E9}{1}{B0}" name="Read about Rust ﷟HYPERLINK &quot;https://example.invalid/rust&quot;the Rust site" dateTime="2026-09-12T09:49:16.000Z" lastModifiedTime="2026-09-28T01:48:11.000Z" pageLevel="1"/></one:Section></one:Notebook>
corpus/math-edit/native-editor-4/before-read/page-000.xml created+3
...@@ -0,0 +1,3 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{18CA7B08-F5D4-0B7B-3BFE-7B36E74072E9}{1}{B0}" name="Read about Rust ﷟HYPERLINK &quot;https://example.invalid/rust&quot;the Rust site" dateTime="2026-09-12T09:49:16.000Z" lastModifiedTime="2026-09-28T01:48:11.000Z" pageLevel="1" style="font-family:Calibri;font-size:11.0pt" lang="en-US"><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="Author" lastModifiedBy="Author" lastModifiedTime="2026-09-12T09:49:16.000Z" objectID="{B9C7BD81-9DC3-0446-2D55-5B561FA3FB12}{23}{B0}"><one:Position x="36.0" y="36.0" z="0"/><one:Size width="198.3904571533203" height="13.4277172088623"/><one:Indents><one:Indent level="0" indent="0.0"/></one:Indents><one:OEChildren><one:OE creationTime="2026-09-12T09:49:16.000Z" lastModifiedTime="2026-09-12T09:49:16.000Z" objectID="{B9C7BD81-9DC3-0446-2D55-5B561FA3FB12}{24}{B0}" alignment="left"><one:T><![CDATA[Read about Rust <a
3href="https://example.invalid/rust">the Rust site</a>]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/math-edit/native-editor-4/before-read/payloads.json created+3
...@@ -0,0 +1,3 @@
1[
2
3]
\ No newline at end of file
corpus/math-edit/native-editor-4/commands.jsonl created+6
...@@ -0,0 +1,6 @@
1{"command": "powershell -NoProfile -Command \"Expand-Archive -LiteralPath C:\\one-tests\\transfer.zip -DestinationPath C:\\one-tests\\runs\\capture\\notebook\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-649AC1D2 -ExpectedPages 1 -KeepOpen", "exit": 0, "stdout": "Read 1 sections and 1 pages.\r\n", "stderr": "", "error": null}
3{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
4{"command": "powershell -NoProfile -Command \"Rename-Item C:\\one-tests\\runs\\capture\\read before-read\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
5{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-649AC1D2 -UseCurrentCache -ExpectedPages -1", "exit": 0, "stdout": "Read 1 sections and 1 pages.\r\n", "stderr": "", "error": null}
6{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
corpus/math-edit/native-editor-4/equations.ahk created+248
...@@ -0,0 +1,248 @@
1ComObject("OneNote.Application").NavigateTo("{18CA7B08-F5D4-0B7B-3BFE-7B36E74072E9}{1}{B0}", "", false)
2WinWait("ahk_exe ONENOTE.EXE", , 30)
3WinActivate("ahk_exe ONENOTE.EXE")
4WinWaitActive("ahk_exe ONENOTE.EXE", , 30)
5WinMaximize("ahk_exe ONENOTE.EXE")
6Sleep(3000)
7CoordMode("Mouse", "Screen")
8Click(300, 140)
9Sleep(500)
10Send("{End}")
11Sleep(300)
12Send("{Enter}")
13Sleep(500)
14Send("!=")
15Sleep(600)
16SendText("a_1+b_2")
17Send(" ")
18Sleep(600)
19Send("{End}")
20Sleep(300)
21Send("{Enter}")
22Sleep(600)
23Send("!=")
24Sleep(600)
25SendText("x_i^2")
26Send(" ")
27Sleep(600)
28Send("{End}")
29Sleep(300)
30Send("{Enter}")
31Sleep(600)
32Send("!=")
33Sleep(600)
34SendText("\sum_(i=1)^n i")
35Send(" ")
36Sleep(600)
37Send("{End}")
38Sleep(300)
39Send("{Enter}")
40Sleep(600)
41Send("!=")
42Sleep(600)
43SendText("\int_0^1 x dx")
44Send(" ")
45Sleep(600)
46Send("{End}")
47Sleep(300)
48Send("{Enter}")
49Sleep(600)
50Send("!=")
51Sleep(600)
52SendText("\sqrt x+1")
53Send(" ")
54Sleep(600)
55Send("{End}")
56Sleep(300)
57Send("{Enter}")
58Sleep(600)
59Send("!=")
60Sleep(600)
61SendText("\cbrt(x)")
62Send(" ")
63Sleep(600)
64Send("{End}")
65Sleep(300)
66Send("{Enter}")
67Sleep(600)
68Send("!=")
69Sleep(600)
70SendText("\sqrt(n&x)")
71Send(" ")
72Sleep(600)
73Send("{End}")
74Sleep(300)
75Send("{Enter}")
76Sleep(600)
77Send("!=")
78Sleep(600)
79SendText("a/b")
80Send(" ")
81Sleep(600)
82Send("{End}")
83Sleep(300)
84Send("{Enter}")
85Sleep(600)
86Send("!=")
87Sleep(600)
88SendText("\prod_(k=1)^n k")
89Send(" ")
90Sleep(600)
91Send("{End}")
92Sleep(300)
93Send("{Enter}")
94Sleep(600)
95Send("!=")
96Sleep(600)
97SendText("[a+b]")
98Send(" ")
99Sleep(600)
100Send("{End}")
101Sleep(300)
102Send("{Enter}")
103Sleep(600)
104Send("!=")
105Sleep(600)
106SendText("\overline(x)")
107Send(" ")
108Sleep(600)
109Send("{End}")
110Sleep(300)
111Send("{Enter}")
112Sleep(600)
113Send("!=")
114Sleep(600)
115SendText("x\hat")
116Send(" ")
117Sleep(600)
118Send("{End}")
119Sleep(300)
120Send("{Enter}")
121Sleep(600)
122Send("!=")
123Sleep(600)
124SendText("\matrix(1&2@3&4)")
125Send(" ")
126Sleep(600)
127Send("{End}")
128Sleep(300)
129Send("{Enter}")
130Sleep(600)
131Send("!=")
132Sleep(600)
133SendText("\eqarray(x&=1@y&=2)")
134Send(" ")
135Sleep(600)
136Send("{End}")
137Sleep(300)
138Send("{Enter}")
139Sleep(600)
140Send("!=")
141Sleep(600)
142SendText("x\above 2")
143Send(" ")
144Sleep(600)
145Send("{End}")
146Sleep(300)
147Send("{Enter}")
148Sleep(600)
149Send("!=")
150Sleep(600)
151SendText("x\below 2")
152Send(" ")
153Sleep(600)
154Send("{End}")
155Sleep(300)
156Send("{Enter}")
157Sleep(600)
158Send("!=")
159Sleep(600)
160SendText("\box(x)")
161Send(" ")
162Sleep(600)
163Send("{End}")
164Sleep(300)
165Send("{Enter}")
166Sleep(600)
167Send("!=")
168Sleep(600)
169SendText("\rect(x)")
170Send(" ")
171Sleep(600)
172Send("{End}")
173Sleep(300)
174Send("{Enter}")
175Sleep(600)
176Send("!=")
177Sleep(600)
178SendText("f(x)/(x^2+1)")
179Send(" ")
180Sleep(600)
181Send("{End}")
182Sleep(300)
183Send("{Enter}")
184Sleep(600)
185Send("!=")
186Sleep(600)
187SendText("\sum^n x")
188Send(" ")
189Sleep(600)
190Send("{End}")
191Sleep(300)
192Send("{Enter}")
193Sleep(600)
194Send("!=")
195Sleep(600)
196SendText("e^(x+1)")
197Send(" ")
198Sleep(600)
199Send("{End}")
200Sleep(300)
201Send("{Enter}")
202Sleep(600)
203Send("!=")
204Sleep(600)
205SendText("\alpha+\beta")
206Send(" ")
207Sleep(600)
208Send("{End}")
209Sleep(300)
210Send("{Enter}")
211Sleep(600)
212Send("!=")
213Sleep(600)
214SendText("(a+b)/(c+d)")
215Send(" ")
216Sleep(600)
217Send("{End}")
218Sleep(300)
219Send("{Enter}")
220Sleep(600)
221Send("!=")
222Sleep(600)
223SendText("x^2+y^2=z^2")
224Send(" ")
225Sleep(600)
226Send("{End}")
227Sleep(300)
228Send("{Enter}")
229Sleep(600)
230Send("!=")
231Sleep(600)
232SendText("lim_(n\to\infty) a_n")
233Send(" ")
234Sleep(600)
235Send("{End}")
236Sleep(300)
237Send("{Enter}")
238Sleep(600)
239Send("!=")
240Sleep(600)
241SendText("{a+b}")
242Send(" ")
243Sleep(600)
244Send("{End}")
245Sleep(300)
246Send("{Enter}")
247Sleep(600)
248Sleep(2000)
corpus/math-edit/native-editor-4/equations.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Read about Rust the Rust site - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/math-edit/native-editor-4/equations.png created
Binary files /dev/null and b/corpus/math-edit/native-editor-4/equations.png differ
corpus/math-edit/native-editor-4/machine.json created+1
...@@ -0,0 +1 @@
1{"name": "m6-649ac1d2", "hostname": "ONE-M6-649AC1D2"}
corpus/math-edit/native-editor-4/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/math-edit/native-editor-4/notebook/Open Notebook.onetoc2 differ
corpus/math-edit/native-editor-4/notebook/links.one created
Binary files /dev/null and b/corpus/math-edit/native-editor-4/notebook/links.one differ
corpus/math-edit/native-editor-4/read/environment.json created+7
...@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-649AC1D2",
5 "cold": false,
6 "onenote": "14.0.4763.1000"
7}
corpus/math-edit/native-editor-4/read/hierarchy.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{7C806B9F-5A4D-4B11-825E-82AA1CE0C357}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-28T01:50:43.000Z" color="#B49EDE" isCurrentlyViewed="true"><one:Section name="links" ID="{B9C7BD81-9DC3-0446-2D55-5B561FA3FB12}{1}{B0}" path="C:\one-tests\runs\capture\notebook\links.one" lastModifiedTime="2026-09-28T01:50:43.000Z" color="#8AA8E4" isCurrentlyViewed="true"><one:Page ID="{18CA7B08-F5D4-0B7B-3BFE-7B36E74072E9}{1}{B0}" name="Read about Rust the Rust site" dateTime="2026-09-12T09:49:16.000Z" lastModifiedTime="2026-09-28T01:50:43.000Z" pageLevel="1" isCurrentlyViewed="true"/></one:Section></one:Notebook>
corpus/math-edit/native-editor-4/read/page-000.xml created+3
...@@ -0,0 +1,3 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{18CA7B08-F5D4-0B7B-3BFE-7B36E74072E9}{1}{B0}" name="Read about Rust the Rust site" dateTime="2026-09-12T09:49:16.000Z" lastModifiedTime="2026-09-28T01:50:44.000Z" pageLevel="1" isCurrentlyViewed="true" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="Author" lastModifiedBy="Author" lastModifiedTime="2026-09-28T01:50:44.000Z" objectID="{B9C7BD81-9DC3-0446-2D55-5B561FA3FB12}{23}{B0}"><one:Position x="36.0" y="36.0" z="0"/><one:Size width="198.3904571533203" height="518.656005859375"/><one:Indents><one:Indent level="0" indent="0.0"/></one:Indents><one:OEChildren><one:OE creationTime="2026-09-12T09:49:16.000Z" lastModifiedTime="2026-09-28T01:49:44.000Z" objectID="{B9C7BD81-9DC3-0446-2D55-5B561FA3FB12}{24}{B0}" alignment="left" style="font-family:Calibri;font-size:11.0pt"><one:T><![CDATA[Read about Rust <a
3href="https://example.invalid/rust">the Rust site</a>]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:49:49.000Z" lastModifiedTime="2026-09-28T01:49:50.000Z" objectID="{8C825D84-DE0B-42AA-96B4-41E992DE9554}{12}{B0}" alignment="left" quickStyleIndex="0" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:msub><mml:mi>a</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo>+</mml:mo><mml:msub><mml:mi>b</mml:mi><mml:mn>2</mml:mn></mml:msub></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:49:51.000Z" lastModifiedTime="2026-09-28T01:49:52.000Z" objectID="{8C825D84-DE0B-42AA-96B4-41E992DE9554}{18}{B0}" alignment="left" quickStyleIndex="0" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:mpadded><mml:mrow><mml:mtext>Type&nbsp;equation&nbsp;here.</mml:mtext></mml:mrow></mml:mpadded></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:49:54.000Z" lastModifiedTime="2026-09-28T01:49:55.000Z" objectID="{8C825D84-DE0B-42AA-96B4-41E992DE9554}{22}{B0}" alignment="left" quickStyleIndex="0" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:munderover><mml:mo stretchy="false">∑</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>n</mml:mi></mml:munderover><mml:mrow><mml:mi>i</mml:mi><mml:mo>&nbsp;</mml:mo></mml:mrow></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:49:56.000Z" lastModifiedTime="2026-09-28T01:49:57.000Z" objectID="{8C825D84-DE0B-42AA-96B4-41E992DE9554}{25}{B0}" alignment="left" quickStyleIndex="0" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:msubsup><mml:mo stretchy="false">∫</mml:mo><mml:mn>0</mml:mn><mml:mn>1</mml:mn></mml:msubsup><mml:mrow><mml:mi>x</mml:mi><mml:mo>&nbsp;</mml:mo><mml:mi>d</mml:mi><mml:mi>x</mml:mi><mml:mo>&nbsp;</mml:mo></mml:mrow></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:49:58.000Z" lastModifiedTime="2026-09-28T01:49:59.000Z" objectID="{8C825D84-DE0B-42AA-96B4-41E992DE9554}{28}{B0}" alignment="left" quickStyleIndex="0" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:msqrt><mml:mi>x</mml:mi></mml:msqrt><mml:mo>+</mml:mo><mml:mn>1</mml:mn><mml:mo>&nbsp;</mml:mo></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:50:00.000Z" lastModifiedTime="2026-09-28T01:50:01.000Z" objectID="{8C825D84-DE0B-42AA-96B4-41E992DE9554}{31}{B0}" alignment="left" quickStyleIndex="0" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:mroot><mml:mi>x</mml:mi><mml:mn>3</mml:mn></mml:mroot></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:50:02.000Z" lastModifiedTime="2026-09-28T01:50:03.000Z" objectID="{8C825D84-DE0B-42AA-96B4-41E992DE9554}{34}{B0}" alignment="left" quickStyleIndex="0" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:mroot><mml:mi>x</mml:mi><mml:mi>n</mml:mi></mml:mroot></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:50:04.000Z" lastModifiedTime="2026-09-28T01:50:05.000Z" objectID="{8C825D84-DE0B-42AA-96B4-41E992DE9554}{37}{B0}" alignment="left" quickStyleIndex="0" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:mfrac><mml:mi>a</mml:mi><mml:mi>b</mml:mi></mml:mfrac></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:50:06.000Z" lastModifiedTime="2026-09-28T01:50:07.000Z" objectID="{8C825D84-DE0B-42AA-96B4-41E992DE9554}{40}{B0}" alignment="left" quickStyleIndex="0" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:munderover><mml:mo stretchy="false">∏</mml:mo><mml:mrow><mml:mi>k</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>n</mml:mi></mml:munderover><mml:mrow><mml:mi>k</mml:mi><mml:mo>&nbsp;</mml:mo></mml:mrow></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:50:08.000Z" lastModifiedTime="2026-09-28T01:50:09.000Z" objectID="{8C825D84-DE0B-42AA-96B4-41E992DE9554}{43}{B0}" alignment="left" quickStyleIndex="0" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:mfenced open="[" close="]"><mml:mrow><mml:mi>a</mml:mi><mml:mo>+</mml:mo><mml:mi>b</mml:mi></mml:mrow></mml:mfenced></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:50:11.000Z" lastModifiedTime="2026-09-28T01:50:11.000Z" objectID="{8C825D84-DE0B-42AA-96B4-41E992DE9554}{46}{B0}" alignment="left" quickStyleIndex="0" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:mover accent="false"><mml:mi>x</mml:mi><mml:mo stretchy="true">¯</mml:mo></mml:mover></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:50:12.000Z" lastModifiedTime="2026-09-28T01:50:14.000Z" objectID="{8C825D84-DE0B-42AA-96B4-41E992DE9554}{49}{B0}" alignment="left" quickStyleIndex="0" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:mover accent="true"><mml:mi>x</mml:mi><mml:mo>^</mml:mo></mml:mover></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:50:15.000Z" lastModifiedTime="2026-09-28T01:50:16.000Z" objectID="{8C825D84-DE0B-42AA-96B4-41E992DE9554}{52}{B0}" alignment="left" quickStyleIndex="0" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:mtable><mml:mtr><mml:mtd><mml:mn>1</mml:mn></mml:mtd><mml:mtd><mml:mn>2</mml:mn></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mn>3</mml:mn></mml:mtd><mml:mtd><mml:mn>4</mml:mn></mml:mtd></mml:mtr></mml:mtable></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:50:17.000Z" lastModifiedTime="2026-09-28T01:50:18.000Z" objectID="{8C825D84-DE0B-42AA-96B4-41E992DE9554}{55}{B0}" alignment="left" quickStyleIndex="0" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:mtable><mml:mtr><mml:mtd><mml:maligngroup/><mml:mi>x</mml:mi><mml:malignmark/><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:maligngroup/><mml:mi>y</mml:mi><mml:malignmark/><mml:mo>=</mml:mo><mml:mn>2</mml:mn></mml:mtd></mml:mtr></mml:mtable></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:50:18.000Z" lastModifiedTime="2026-09-28T01:50:20.000Z" objectID="{8C825D84-DE0B-42AA-96B4-41E992DE9554}{58}{B0}" alignment="left" quickStyleIndex="0" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:mover><mml:mi>x</mml:mi><mml:mn>2</mml:mn></mml:mover></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:50:21.000Z" lastModifiedTime="2026-09-28T01:50:22.000Z" objectID="{8C825D84-DE0B-42AA-96B4-41E992DE9554}{61}{B0}" alignment="left" quickStyleIndex="0" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:munder><mml:mi>x</mml:mi><mml:mn>2</mml:mn></mml:munder></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:50:23.000Z" lastModifiedTime="2026-09-28T01:50:24.000Z" objectID="{8C825D84-DE0B-42AA-96B4-41E992DE9554}{64}{B0}" alignment="left" quickStyleIndex="0" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:mpadded><mml:mi>x</mml:mi></mml:mpadded></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:50:26.000Z" lastModifiedTime="2026-09-28T01:50:26.000Z" objectID="{8C825D84-DE0B-42AA-96B4-41E992DE9554}{67}{B0}" alignment="left" quickStyleIndex="0" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:menclose notation="box"><mml:mi>x</mml:mi></mml:menclose></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:50:27.000Z" lastModifiedTime="2026-09-28T01:50:29.000Z" objectID="{8C825D84-DE0B-42AA-96B4-41E992DE9554}{70}{B0}" alignment="left" quickStyleIndex="0" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:mfrac><mml:mrow><mml:mi>f</mml:mi><mml:mfenced><mml:mi>x</mml:mi></mml:mfenced></mml:mrow><mml:mrow><mml:msup><mml:mi>x</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:mfrac></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:50:30.000Z" lastModifiedTime="2026-09-28T01:50:31.000Z" objectID="{8C825D84-DE0B-42AA-96B4-41E992DE9554}{73}{B0}" alignment="left" quickStyleIndex="0" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:mover><mml:mo stretchy="false">∑</mml:mo><mml:mi>n</mml:mi></mml:mover><mml:mrow><mml:mi>x</mml:mi><mml:mo>&nbsp;</mml:mo></mml:mrow></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:50:31.000Z" lastModifiedTime="2026-09-28T01:50:33.000Z" objectID="{8C825D84-DE0B-42AA-96B4-41E992DE9554}{76}{B0}" alignment="left" quickStyleIndex="0" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mi>x</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:50:34.000Z" lastModifiedTime="2026-09-28T01:50:35.000Z" objectID="{8C825D84-DE0B-42AA-96B4-41E992DE9554}{79}{B0}" alignment="left" quickStyleIndex="0" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:mi>&#120572;</mml:mi><mml:mo>+</mml:mo><mml:mi>&#120573;</mml:mi></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:50:35.000Z" lastModifiedTime="2026-09-28T01:50:37.000Z" objectID="{8C825D84-DE0B-42AA-96B4-41E992DE9554}{82}{B0}" alignment="left" quickStyleIndex="0" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:mfrac><mml:mrow><mml:mi>a</mml:mi><mml:mo>+</mml:mo><mml:mi>b</mml:mi></mml:mrow><mml:mrow><mml:mi>c</mml:mi><mml:mo>+</mml:mo><mml:mi>d</mml:mi></mml:mrow></mml:mfrac></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:50:38.000Z" lastModifiedTime="2026-09-28T01:50:39.000Z" objectID="{8C825D84-DE0B-42AA-96B4-41E992DE9554}{85}{B0}" alignment="left" quickStyleIndex="0" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:msup><mml:mi>x</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mo>+</mml:mo><mml:msup><mml:mi>y</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mo>=</mml:mo><mml:msup><mml:mi>z</mml:mi><mml:mn>2</mml:mn></mml:msup></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:50:40.000Z" lastModifiedTime="2026-09-28T01:50:42.000Z" objectID="{8C825D84-DE0B-42AA-96B4-41E992DE9554}{88}{B0}" alignment="left" quickStyleIndex="0" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:mrow><mml:mrow><mml:munder><mml:mrow><mml:mi>lim</mml:mi></mml:mrow><mml:mrow><mml:mi>n</mml:mi><mml:mo>→</mml:mo><mml:mo>∞</mml:mo></mml:mrow></mml:munder></mml:mrow><mml:mo>⁡</mml:mo><mml:mrow><mml:msub><mml:mi>a</mml:mi><mml:mi>n</mml:mi></mml:msub></mml:mrow></mml:mrow></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:50:42.000Z" lastModifiedTime="2026-09-28T01:50:43.000Z" objectID="{8C825D84-DE0B-42AA-96B4-41E992DE9554}{92}{B0}" alignment="left" quickStyleIndex="0" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:mfenced open="{" close="}"><mml:mrow><mml:mi>a</mml:mi><mml:mo>+</mml:mo><mml:mi>b</mml:mi></mml:mrow></mml:mfenced></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T01:50:43.000Z" lastModifiedTime="2026-09-28T01:50:44.000Z" objectID="{8C825D84-DE0B-42AA-96B4-41E992DE9554}{95}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/math-edit/native-editor-4/read/payloads.json created+3
...@@ -0,0 +1,3 @@
1[
2
3]
\ No newline at end of file
corpus/math-edit/native-editor-4/run.json created+18
...@@ -0,0 +1,18 @@
1{
2 "notebook": "/Users/clo/dev/one/corpus/link-edit/candidate",
3 "expected_pages": 1,
4 "author": null,
5 "author_timeout_seconds": 600,
6 "inspect": false,
7 "collect_notebook": true,
8 "base": {
9 "file": "win7-office-base.qcow2",
10 "format": "qcow2",
11 "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346",
12 "virtual_size": 68719476736
13 },
14 "scripts": {
15 "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331",
16 "read.ps1": "04013bfcccee40a17e2a225f9b9e96f40a3a8daad9e350609654f8eda3ccbb41"
17 }
18}
corpus/math-edit/native-editor-4/scripts/cold.ps1 created+27
...@@ -0,0 +1,27 @@
1param([Parameter(Mandatory=$true)][string]$Root, [string]$CloneHost = '')
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
5if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
6 throw 'Choose a run directly below C:\one-tests\runs.'
7}
8if (Get-Process ONENOTE -ErrorAction SilentlyContinue) { throw 'Close OneNote before resetting its test cache.' }
9$key = 'HKCU:\Software\Microsoft\Office\14.0\OneNote'
10if ($CloneHost) {
11 if ($CloneHost -notmatch '^ONE-[A-Z0-9-]+$' -or [Environment]::MachineName -ne $CloneHost) {
12 throw 'The disposable clone hostname does not match this machine.'
13 }
14 New-Item "$key\Options\Paths" -Force | Out-Null
15 New-ItemProperty "$key\Options\Paths" -Name UnfiledNotesSection -PropertyType ExpandString -Value 'C:\one-tests\Loose.one' -Force | Out-Null
16} elseif ((Get-ItemProperty "$key\Options\Paths").UnfiledNotesSection -ne 'C:\one-tests\Loose.one' -or
17 -not (Test-Path 'C:\one-tests\profile-original-cache')) {
18 throw 'Park the personal OneNote profile before resetting the test cache.'
19}
20$cache = Join-Path $env:LOCALAPPDATA 'Microsoft\OneNote\14.0'
21$parked = Join-Path 'C:\one-tests\caches' ([IO.Path]::GetFileName($root))
22if (Test-Path $parked) { throw 'Choose a new run; its parked cache already exists.' }
23New-Item -ItemType Directory -Path 'C:\one-tests\caches' -Force | Out-Null
24if (Test-Path $cache) { Move-Item -LiteralPath $cache -Destination $parked }
25if (Test-Path "$key\OpenNotebooks") { Remove-Item "$key\OpenNotebooks" -Recurse }
26New-Item "$key\OpenNotebooks" | Out-Null
27New-ItemProperty "$key\OpenNotebooks" -Name '1' -PropertyType String -Value "$root\notebook" | Out-Null
corpus/math-edit/native-editor-4/scripts/read.ps1 created+142
...@@ -0,0 +1,142 @@
1param(
2 [Parameter(Mandatory=$true)][string]$Root,
3 [int]$ExpectedPages = -1,
4 [switch]$UseCurrentCache,
5 [switch]$Pdf,
6 [switch]$KeepOpen,
7 [string]$CloneHost = ''
8)
9Set-StrictMode -Version Latest
10$ErrorActionPreference = 'Stop'
11$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
12if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
13 throw 'Choose a run directly below C:\one-tests\runs.'
14}
15$notebook = Join-Path $root 'notebook'
16$output = Join-Path $root 'read'
17if (Test-Path $output) { throw 'Choose a new read destination.' }
18if ($UseCurrentCache) {
19 if ((Get-ItemProperty 'HKCU:\Software\Microsoft\Office\14.0\OneNote\Options\Paths').UnfiledNotesSection -ne 'C:\one-tests\Loose.one') {
20 throw 'Park the personal OneNote profile before reading test notebooks.'
21 }
22} else {
23 & "$PSScriptRoot\cold-current.ps1" -Root $root -CloneHost $CloneHost
24}
25New-Item -ItemType Directory -Path $output | Out-Null
26$app = New-Object -ComObject OneNote.Application
27$notebookId = ''
28$failure = $null
29try {
30 $app.OpenHierarchy($notebook, '', [ref]$notebookId, 0)
31 $process = Get-Process ONENOTE
32 @{ hostname = [Environment]::MachineName; onenote = $process.MainModule.FileVersionInfo.FileVersion;
33 powershell = $PSVersionTable.PSVersion.ToString(); schema = 'xs2010'; cold = (-not $UseCurrentCache.IsPresent) } |
34 ConvertTo-Json | Set-Content (Join-Path $output 'environment.json') -Encoding UTF8
35 $sections = @()
36 foreach ($file in @(Get-ChildItem $notebook -Recurse | Where-Object { $_.Extension -eq '.one' })) {
37 $id = ''
38 $app.OpenHierarchy($file.FullName, '', [ref]$id, 0)
39 $sections += $id
40 }
41 $deadline = [DateTime]::UtcNow.AddSeconds(300)
42 $previous = ''
43 $lastChange = ''
44 $stableSince = [DateTime]::UtcNow
45 $settled = $false
46 do {
47 $pages = @{}
48 foreach ($section in $sections) {
49 $hierarchy = ''
50 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
51 [xml]$xml = $hierarchy
52 foreach ($node in $xml.SelectNodes('//*[@path]')) {
53 if (-not $node.GetAttribute('path').StartsWith("$notebook\", [StringComparison]::OrdinalIgnoreCase)) {
54 throw 'OneNote opened a section outside the copied notebook.'
55 }
56 }
57 foreach ($node in $xml.SelectNodes('//*[local-name()="Page"]')) {
58 $id = $node.GetAttribute('ID')
59 $content = ''
60 $app.GetPageContent($id, [ref]$content, 1, 1)
61 $pages[$id] = $content
62 }
63 }
64 $signature = [String]::Join('|', @($pages.Keys | Sort-Object | ForEach-Object { $_ + $pages[$_] }))
65 if ($signature -ne $previous) {
66 $lastChange = $previous
67 $previous = $signature
68 $stableSince = [DateTime]::UtcNow
69 }
70 if ((($ExpectedPages -ge 0 -and $pages.Count -eq $ExpectedPages) -or
71 ($ExpectedPages -lt 0 -and $pages.Count -gt 0)) -and
72 ([DateTime]::UtcNow - $stableSince).TotalSeconds -ge 2) { $settled = $true; break }
73 Start-Sleep -Milliseconds 250
74 } while ([DateTime]::UtcNow -lt $deadline)
75 if (-not $settled -or ($ExpectedPages -ge 0 -and $pages.Count -ne $ExpectedPages) -or ($ExpectedPages -lt 0 -and $pages.Count -eq 0)) {
76 [IO.File]::WriteAllText((Join-Path $output 'previous-signature.txt'), $lastChange, [Text.Encoding]::UTF8)
77 $index = 0
78 foreach ($id in @($pages.Keys | Sort-Object)) {
79 [IO.File]::WriteAllText((Join-Path $output ('unsettled-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
80 $index++
81 }
82 $hierarchy = ''
83 $app.GetHierarchy($notebookId, 4, [ref]$hierarchy, 1)
84 [IO.File]::WriteAllText((Join-Path $output 'unsettled-hierarchy.xml'), $hierarchy, [Text.Encoding]::UTF8)
85 throw "Expected $ExpectedPages stable pages; OneNote returned $($pages.Count), settled=$settled."
86 }
87 $index = 0
88 $payloads = @()
89 foreach ($id in @($pages.Keys | Sort-Object)) {
90 [IO.File]::WriteAllText((Join-Path $output ('page-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
91 if ($Pdf) {
92 $pdfPath = Join-Path $output ('page-{0:d3}.pdf' -f $index)
93 $app.NavigateTo($id, '', $false)
94 $app.Publish($id, $pdfPath, 3, '')
95 if (-not (Test-Path $pdfPath) -or (Get-Item $pdfPath).Length -eq 0) { throw 'OneNote did not publish the page PDF.' }
96 }
97 [xml]$page = $pages[$id]
98 foreach ($file in $page.SelectNodes('//*[local-name()="InsertedFile" or local-name()="MediaFile"]')) {
99 $bytes = [IO.File]::ReadAllBytes($file.GetAttribute('pathCache'))
100 $hash = [BitConverter]::ToString([Security.Cryptography.SHA256]::Create().ComputeHash($bytes)).Replace('-', '').ToLowerInvariant()
101 [IO.File]::WriteAllBytes((Join-Path $output ($hash + '.attachment')), $bytes)
102 $payloads += @{ page = $id; object = $file.ParentNode.GetAttribute('objectID');
103 kind = $file.LocalName; name = $file.GetAttribute('preferredName');
104 sha256 = $hash; bytes = $bytes.Length }
105 }
106 $index++
107 }
108 [IO.File]::WriteAllText((Join-Path $output 'payloads.json'), (ConvertTo-Json -InputObject $payloads -Depth 4), [Text.Encoding]::UTF8)
109 $all = ''
110 $app.GetHierarchy($notebookId, 4, [ref]$all, 1)
111 [xml]$finalTree = $all
112 $finalIds = @($finalTree.SelectNodes('//*[local-name()="Page"]') | ForEach-Object { $_.GetAttribute('ID') } | Sort-Object -Unique)
113 if ($finalIds.Count -ne $pages.Count -or @($finalIds | Where-Object { -not $pages.ContainsKey($_) }).Count -ne 0) {
114 throw 'The notebook hierarchy changed while collecting page evidence; repeat the cold read.'
115 }
116 [IO.File]::WriteAllText((Join-Path $output 'hierarchy.xml'), $all, [Text.Encoding]::UTF8)
117 Write-Output "Read $($sections.Count) sections and $($pages.Count) pages."
118} catch {
119 $failure = $_
120 [IO.File]::WriteAllText((Join-Path $output 'failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
121 throw
122} finally {
123 try {
124 try {
125 if ($notebookId -and $CloneHost) { $app.SyncHierarchy($notebookId) }
126 if ($notebookId -and -not $KeepOpen -and (-not $UseCurrentCache -or $CloneHost)) {
127 $app.CloseNotebook($notebookId, $false)
128 }
129 } catch {
130 if ($null -eq $failure) { throw }
131 [IO.File]::WriteAllText((Join-Path $output 'cleanup-failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
132 }
133 } finally {
134 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
135 }
136 $app = $null
137 [GC]::Collect()
138 [GC]::WaitForPendingFinalizers()
139 if (-not $UseCurrentCache -and -not $CloneHost) {
140 Get-Process ONENOTE -ErrorAction SilentlyContinue | Wait-Process -Timeout 10
141 }
142}
corpus/math-edit/native-editor-4/source.json created+14
...@@ -0,0 +1,14 @@
1[
2 {
3 "path": "Open Notebook.onetoc2",
4 "bytes": 1604,
5 "sha256": "6731c4214437fe1cb6aa347b0004542b7f7e4260f47773bf9ec07c4e894de47c",
6 "mtime_ns": 1789206821144296891
7 },
8 {
9 "path": "links.one",
10 "bytes": 6128,
11 "sha256": "0cfb356ed374fe919f73fb73fdbe5d66e0baa484d4119f70074d9f869d6db78a",
12 "mtime_ns": 1789206821116200533
13 }
14]
corpus/math-edit/native-editor-4/teardown.json created+1
...@@ -0,0 +1 @@
1{"absent": true}
corpus/math-edit/native-enter/before-read/environment.json created+7
...@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-37FE88DC",
5 "cold": true,
6 "onenote": "14.0.4763.1000"
7}
corpus/math-edit/native-enter/before-read/hierarchy.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{D317B2FA-1D06-4CE7-AFA4-6641021828C4}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-28T06:48:36.000Z" color="#B49EDE"><one:Section name="links" ID="{4B703ABE-5CB3-09D4-04ED-D01FC2641497}{1}{B0}" path="C:\one-tests\runs\capture\notebook\links.one" lastModifiedTime="2026-09-28T06:48:36.000Z" color="#8AA8E4"><one:Page ID="{EA7DFC37-34A4-06E9-1246-F07F3A879D6C}{1}{B0}" name="Read about Rust ﷟HYPERLINK &quot;https://example.invalid/rust&quot;the Rust site" dateTime="2026-09-12T09:49:16.000Z" lastModifiedTime="2026-09-28T06:48:36.000Z" pageLevel="1"/></one:Section></one:Notebook>
corpus/math-edit/native-enter/before-read/page-000.xml created+3
...@@ -0,0 +1,3 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{EA7DFC37-34A4-06E9-1246-F07F3A879D6C}{1}{B0}" name="Read about Rust ﷟HYPERLINK &quot;https://example.invalid/rust&quot;the Rust site" dateTime="2026-09-12T09:49:16.000Z" lastModifiedTime="2026-09-28T06:48:36.000Z" pageLevel="1" style="font-family:Calibri;font-size:11.0pt" lang="en-US"><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="Author" lastModifiedBy="Author" lastModifiedTime="2026-09-12T09:49:16.000Z" objectID="{4B703ABE-5CB3-09D4-04ED-D01FC2641497}{23}{B0}"><one:Position x="36.0" y="36.0" z="0"/><one:Size width="198.3904571533203" height="13.4277172088623"/><one:Indents><one:Indent level="0" indent="0.0"/></one:Indents><one:OEChildren><one:OE creationTime="2026-09-12T09:49:16.000Z" lastModifiedTime="2026-09-12T09:49:16.000Z" objectID="{4B703ABE-5CB3-09D4-04ED-D01FC2641497}{24}{B0}" alignment="left"><one:T><![CDATA[Read about Rust <a
3href="https://example.invalid/rust">the Rust site</a>]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/math-edit/native-enter/before-read/payloads.json created+3
...@@ -0,0 +1,3 @@
1[
2
3]
\ No newline at end of file
corpus/math-edit/native-enter/commands.jsonl created+6
...@@ -0,0 +1,6 @@
1{"command": "powershell -NoProfile -Command \"Expand-Archive -LiteralPath C:\\one-tests\\transfer.zip -DestinationPath C:\\one-tests\\runs\\capture\\notebook\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-37FE88DC -ExpectedPages 1 -KeepOpen", "exit": 0, "stdout": "Read 1 sections and 1 pages.\r\n", "stderr": "", "error": null}
3{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
4{"command": "powershell -NoProfile -Command \"Rename-Item C:\\one-tests\\runs\\capture\\read before-read\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
5{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-37FE88DC -UseCurrentCache -ExpectedPages -1", "exit": 0, "stdout": "Read 1 sections and 1 pages.\r\n", "stderr": "", "error": null}
6{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
corpus/math-edit/native-enter/enter.ahk created+53
...@@ -0,0 +1,53 @@
1ComObject("OneNote.Application").NavigateTo("{EA7DFC37-34A4-06E9-1246-F07F3A879D6C}{1}{B0}", "", false)
2WinWait("ahk_exe ONENOTE.EXE", , 30)
3WinActivate("ahk_exe ONENOTE.EXE")
4WinWaitActive("ahk_exe ONENOTE.EXE", , 30)
5WinMaximize("ahk_exe ONENOTE.EXE")
6Sleep(3000)
7CoordMode("Mouse", "Screen")
8Click(200, 140)
9Sleep(500)
10Send("{End}")
11Sleep(300)
12Send("{Enter}")
13Sleep(500)
14SendText("Before ")
15Send("!=")
16Sleep(600)
17SendText("y^2+1")
18Send(" ")
19Sleep(700)
20Send("{Home}")
21Send("{Right 7}")
22Sleep(300)
23Send("{Enter}")
24Sleep(600)
25Send("{End}")
26Send("{Left 2}")
27Sleep(300)
28Send("{Enter}")
29Sleep(600)
30Send("{End}{Enter}")
31Send("!=")
32Sleep(600)
33SendText("(a+b)/(c+d)")
34Send(" ")
35Sleep(700)
36Send("{Left 3}")
37Sleep(300)
38Send("{Enter}")
39Sleep(600)
40Send("^{End}{Enter}")
41SendText("Select me please")
42Send("+^{Left}+^{Left}")
43Send("^k")
44Sleep(1500)
45SendText("https://example.com/selected")
46Send("{Enter}")
47Sleep(800)
48Send("{End}")
49Send("{Left 9}")
50Sleep(300)
51Send("{Enter}")
52Sleep(600)
53Sleep(2000)
corpus/math-edit/native-enter/enter.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Read about Rust the Rust site - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/math-edit/native-enter/enter.png created
Binary files /dev/null and b/corpus/math-edit/native-enter/enter.png differ
corpus/math-edit/native-enter/machine.json created+1
...@@ -0,0 +1 @@
1{"name": "m6-37fe88dc", "hostname": "ONE-M6-37FE88DC"}
corpus/math-edit/native-enter/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/math-edit/native-enter/notebook/Open Notebook.onetoc2 differ
corpus/math-edit/native-enter/notebook/links.one created
Binary files /dev/null and b/corpus/math-edit/native-enter/notebook/links.one differ
corpus/math-edit/native-enter/read/environment.json created+7
...@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-37FE88DC",
5 "cold": false,
6 "onenote": "14.0.4763.1000"
7}
corpus/math-edit/native-enter/read/hierarchy.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{D317B2FA-1D06-4CE7-AFA4-6641021828C4}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-28T06:50:24.000Z" color="#B49EDE" isCurrentlyViewed="true"><one:Section name="links" ID="{4B703ABE-5CB3-09D4-04ED-D01FC2641497}{1}{B0}" path="C:\one-tests\runs\capture\notebook\links.one" lastModifiedTime="2026-09-28T06:50:24.000Z" color="#8AA8E4" isCurrentlyViewed="true"><one:Page ID="{EA7DFC37-34A4-06E9-1246-F07F3A879D6C}{1}{B0}" name="Read about Rust the Rust site" dateTime="2026-09-12T09:49:16.000Z" lastModifiedTime="2026-09-28T06:50:24.000Z" pageLevel="1" isCurrentlyViewed="true"/></one:Section></one:Notebook>
corpus/math-edit/native-enter/read/page-000.xml created+6
...@@ -0,0 +1,6 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{EA7DFC37-34A4-06E9-1246-F07F3A879D6C}{1}{B0}" name="Read about Rust the Rust site" dateTime="2026-09-12T09:49:16.000Z" lastModifiedTime="2026-09-28T06:50:24.000Z" pageLevel="1" isCurrentlyViewed="true" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="Author" lastModifiedBy="Author" lastModifiedTime="2026-09-28T06:50:24.000Z" objectID="{4B703ABE-5CB3-09D4-04ED-D01FC2641497}{23}{B0}"><one:Position x="36.0" y="36.0" z="0"/><one:Size width="352.7999572753906" height="114.4321136474609"/><one:Indents><one:Indent level="0" indent="0.0"/></one:Indents><one:OEChildren><one:OE creationTime="2026-09-12T09:49:16.000Z" lastModifiedTime="2026-09-28T06:50:13.000Z" objectID="{4B703ABE-5CB3-09D4-04ED-D01FC2641497}{24}{B0}" alignment="left" style="font-family:Calibri;font-size:11.0pt"><one:T><![CDATA[Read about Rust <a
3href="https://example.invalid/rust">the Rust site</a>]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T06:50:17.000Z" lastModifiedTime="2026-09-28T06:50:18.000Z" objectID="{0C653CD9-749D-4D90-8840-58225BF9B002}{12}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Before ]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T06:50:18.000Z" lastModifiedTime="2026-09-28T06:50:19.000Z" objectID="{0C653CD9-749D-4D90-8840-58225BF9B002}{18}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:msup><mml:mi>y</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mo>+</mml:mo></mml:math><![endif]--><span
4style='font-family:Calibri' lang=en-US><br>
5</span><!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:mn>1</mml:mn><mml:mo>&nbsp;</mml:mo></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T06:50:19.000Z" lastModifiedTime="2026-09-28T06:50:22.000Z" objectID="{0C653CD9-749D-4D90-8840-58225BF9B002}{20}{B0}" alignment="left" quickStyleIndex="0" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:mfrac><mml:mrow><mml:mi>a</mml:mi><mml:mo>+</mml:mo><mml:mi>b</mml:mi></mml:mrow><mml:mrow><mml:mtable><mml:mtr><mml:mtd><mml:maligngroup/><mml:mi>c</mml:mi></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:maligngroup/><mml:mo>+</mml:mo><mml:mi>d</mml:mi></mml:mtd></mml:mtr></mml:mtable></mml:mrow></mml:mfrac></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T06:50:23.000Z" lastModifiedTime="2026-09-28T06:50:24.000Z" objectID="{0C653CD9-749D-4D90-8840-58225BF9B002}{23}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[Select ]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T06:50:24.000Z" lastModifiedTime="2026-09-28T06:50:24.000Z" objectID="{0C653CD9-749D-4D90-8840-58225BF9B002}{27}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[<a
6href="https://example.com/selected">me please</a>]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/math-edit/native-enter/read/payloads.json created+3
...@@ -0,0 +1,3 @@
1[
2
3]
\ No newline at end of file
corpus/math-edit/native-enter/run.json created+18
...@@ -0,0 +1,18 @@
1{
2 "notebook": "/Users/clo/dev/one/corpus/link-edit/candidate",
3 "expected_pages": 1,
4 "author": null,
5 "author_timeout_seconds": 600,
6 "inspect": false,
7 "collect_notebook": true,
8 "base": {
9 "file": "win7-office-base.qcow2",
10 "format": "qcow2",
11 "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346",
12 "virtual_size": 68719476736
13 },
14 "scripts": {
15 "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331",
16 "read.ps1": "04013bfcccee40a17e2a225f9b9e96f40a3a8daad9e350609654f8eda3ccbb41"
17 }
18}
corpus/math-edit/native-enter/scripts/cold.ps1 created+27
...@@ -0,0 +1,27 @@
1param([Parameter(Mandatory=$true)][string]$Root, [string]$CloneHost = '')
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
5if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
6 throw 'Choose a run directly below C:\one-tests\runs.'
7}
8if (Get-Process ONENOTE -ErrorAction SilentlyContinue) { throw 'Close OneNote before resetting its test cache.' }
9$key = 'HKCU:\Software\Microsoft\Office\14.0\OneNote'
10if ($CloneHost) {
11 if ($CloneHost -notmatch '^ONE-[A-Z0-9-]+$' -or [Environment]::MachineName -ne $CloneHost) {
12 throw 'The disposable clone hostname does not match this machine.'
13 }
14 New-Item "$key\Options\Paths" -Force | Out-Null
15 New-ItemProperty "$key\Options\Paths" -Name UnfiledNotesSection -PropertyType ExpandString -Value 'C:\one-tests\Loose.one' -Force | Out-Null
16} elseif ((Get-ItemProperty "$key\Options\Paths").UnfiledNotesSection -ne 'C:\one-tests\Loose.one' -or
17 -not (Test-Path 'C:\one-tests\profile-original-cache')) {
18 throw 'Park the personal OneNote profile before resetting the test cache.'
19}
20$cache = Join-Path $env:LOCALAPPDATA 'Microsoft\OneNote\14.0'
21$parked = Join-Path 'C:\one-tests\caches' ([IO.Path]::GetFileName($root))
22if (Test-Path $parked) { throw 'Choose a new run; its parked cache already exists.' }
23New-Item -ItemType Directory -Path 'C:\one-tests\caches' -Force | Out-Null
24if (Test-Path $cache) { Move-Item -LiteralPath $cache -Destination $parked }
25if (Test-Path "$key\OpenNotebooks") { Remove-Item "$key\OpenNotebooks" -Recurse }
26New-Item "$key\OpenNotebooks" | Out-Null
27New-ItemProperty "$key\OpenNotebooks" -Name '1' -PropertyType String -Value "$root\notebook" | Out-Null
corpus/math-edit/native-enter/scripts/read.ps1 created+142
...@@ -0,0 +1,142 @@
1param(
2 [Parameter(Mandatory=$true)][string]$Root,
3 [int]$ExpectedPages = -1,
4 [switch]$UseCurrentCache,
5 [switch]$Pdf,
6 [switch]$KeepOpen,
7 [string]$CloneHost = ''
8)
9Set-StrictMode -Version Latest
10$ErrorActionPreference = 'Stop'
11$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
12if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
13 throw 'Choose a run directly below C:\one-tests\runs.'
14}
15$notebook = Join-Path $root 'notebook'
16$output = Join-Path $root 'read'
17if (Test-Path $output) { throw 'Choose a new read destination.' }
18if ($UseCurrentCache) {
19 if ((Get-ItemProperty 'HKCU:\Software\Microsoft\Office\14.0\OneNote\Options\Paths').UnfiledNotesSection -ne 'C:\one-tests\Loose.one') {
20 throw 'Park the personal OneNote profile before reading test notebooks.'
21 }
22} else {
23 & "$PSScriptRoot\cold-current.ps1" -Root $root -CloneHost $CloneHost
24}
25New-Item -ItemType Directory -Path $output | Out-Null
26$app = New-Object -ComObject OneNote.Application
27$notebookId = ''
28$failure = $null
29try {
30 $app.OpenHierarchy($notebook, '', [ref]$notebookId, 0)
31 $process = Get-Process ONENOTE
32 @{ hostname = [Environment]::MachineName; onenote = $process.MainModule.FileVersionInfo.FileVersion;
33 powershell = $PSVersionTable.PSVersion.ToString(); schema = 'xs2010'; cold = (-not $UseCurrentCache.IsPresent) } |
34 ConvertTo-Json | Set-Content (Join-Path $output 'environment.json') -Encoding UTF8
35 $sections = @()
36 foreach ($file in @(Get-ChildItem $notebook -Recurse | Where-Object { $_.Extension -eq '.one' })) {
37 $id = ''
38 $app.OpenHierarchy($file.FullName, '', [ref]$id, 0)
39 $sections += $id
40 }
41 $deadline = [DateTime]::UtcNow.AddSeconds(300)
42 $previous = ''
43 $lastChange = ''
44 $stableSince = [DateTime]::UtcNow
45 $settled = $false
46 do {
47 $pages = @{}
48 foreach ($section in $sections) {
49 $hierarchy = ''
50 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
51 [xml]$xml = $hierarchy
52 foreach ($node in $xml.SelectNodes('//*[@path]')) {
53 if (-not $node.GetAttribute('path').StartsWith("$notebook\", [StringComparison]::OrdinalIgnoreCase)) {
54 throw 'OneNote opened a section outside the copied notebook.'
55 }
56 }
57 foreach ($node in $xml.SelectNodes('//*[local-name()="Page"]')) {
58 $id = $node.GetAttribute('ID')
59 $content = ''
60 $app.GetPageContent($id, [ref]$content, 1, 1)
61 $pages[$id] = $content
62 }
63 }
64 $signature = [String]::Join('|', @($pages.Keys | Sort-Object | ForEach-Object { $_ + $pages[$_] }))
65 if ($signature -ne $previous) {
66 $lastChange = $previous
67 $previous = $signature
68 $stableSince = [DateTime]::UtcNow
69 }
70 if ((($ExpectedPages -ge 0 -and $pages.Count -eq $ExpectedPages) -or
71 ($ExpectedPages -lt 0 -and $pages.Count -gt 0)) -and
72 ([DateTime]::UtcNow - $stableSince).TotalSeconds -ge 2) { $settled = $true; break }
73 Start-Sleep -Milliseconds 250
74 } while ([DateTime]::UtcNow -lt $deadline)
75 if (-not $settled -or ($ExpectedPages -ge 0 -and $pages.Count -ne $ExpectedPages) -or ($ExpectedPages -lt 0 -and $pages.Count -eq 0)) {
76 [IO.File]::WriteAllText((Join-Path $output 'previous-signature.txt'), $lastChange, [Text.Encoding]::UTF8)
77 $index = 0
78 foreach ($id in @($pages.Keys | Sort-Object)) {
79 [IO.File]::WriteAllText((Join-Path $output ('unsettled-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
80 $index++
81 }
82 $hierarchy = ''
83 $app.GetHierarchy($notebookId, 4, [ref]$hierarchy, 1)
84 [IO.File]::WriteAllText((Join-Path $output 'unsettled-hierarchy.xml'), $hierarchy, [Text.Encoding]::UTF8)
85 throw "Expected $ExpectedPages stable pages; OneNote returned $($pages.Count), settled=$settled."
86 }
87 $index = 0
88 $payloads = @()
89 foreach ($id in @($pages.Keys | Sort-Object)) {
90 [IO.File]::WriteAllText((Join-Path $output ('page-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
91 if ($Pdf) {
92 $pdfPath = Join-Path $output ('page-{0:d3}.pdf' -f $index)
93 $app.NavigateTo($id, '', $false)
94 $app.Publish($id, $pdfPath, 3, '')
95 if (-not (Test-Path $pdfPath) -or (Get-Item $pdfPath).Length -eq 0) { throw 'OneNote did not publish the page PDF.' }
96 }
97 [xml]$page = $pages[$id]
98 foreach ($file in $page.SelectNodes('//*[local-name()="InsertedFile" or local-name()="MediaFile"]')) {
99 $bytes = [IO.File]::ReadAllBytes($file.GetAttribute('pathCache'))
100 $hash = [BitConverter]::ToString([Security.Cryptography.SHA256]::Create().ComputeHash($bytes)).Replace('-', '').ToLowerInvariant()
101 [IO.File]::WriteAllBytes((Join-Path $output ($hash + '.attachment')), $bytes)
102 $payloads += @{ page = $id; object = $file.ParentNode.GetAttribute('objectID');
103 kind = $file.LocalName; name = $file.GetAttribute('preferredName');
104 sha256 = $hash; bytes = $bytes.Length }
105 }
106 $index++
107 }
108 [IO.File]::WriteAllText((Join-Path $output 'payloads.json'), (ConvertTo-Json -InputObject $payloads -Depth 4), [Text.Encoding]::UTF8)
109 $all = ''
110 $app.GetHierarchy($notebookId, 4, [ref]$all, 1)
111 [xml]$finalTree = $all
112 $finalIds = @($finalTree.SelectNodes('//*[local-name()="Page"]') | ForEach-Object { $_.GetAttribute('ID') } | Sort-Object -Unique)
113 if ($finalIds.Count -ne $pages.Count -or @($finalIds | Where-Object { -not $pages.ContainsKey($_) }).Count -ne 0) {
114 throw 'The notebook hierarchy changed while collecting page evidence; repeat the cold read.'
115 }
116 [IO.File]::WriteAllText((Join-Path $output 'hierarchy.xml'), $all, [Text.Encoding]::UTF8)
117 Write-Output "Read $($sections.Count) sections and $($pages.Count) pages."
118} catch {
119 $failure = $_
120 [IO.File]::WriteAllText((Join-Path $output 'failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
121 throw
122} finally {
123 try {
124 try {
125 if ($notebookId -and $CloneHost) { $app.SyncHierarchy($notebookId) }
126 if ($notebookId -and -not $KeepOpen -and (-not $UseCurrentCache -or $CloneHost)) {
127 $app.CloseNotebook($notebookId, $false)
128 }
129 } catch {
130 if ($null -eq $failure) { throw }
131 [IO.File]::WriteAllText((Join-Path $output 'cleanup-failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
132 }
133 } finally {
134 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
135 }
136 $app = $null
137 [GC]::Collect()
138 [GC]::WaitForPendingFinalizers()
139 if (-not $UseCurrentCache -and -not $CloneHost) {
140 Get-Process ONENOTE -ErrorAction SilentlyContinue | Wait-Process -Timeout 10
141 }
142}
corpus/math-edit/native-enter/source.json created+14
...@@ -0,0 +1,14 @@
1[
2 {
3 "path": "Open Notebook.onetoc2",
4 "bytes": 1604,
5 "sha256": "6731c4214437fe1cb6aa347b0004542b7f7e4260f47773bf9ec07c4e894de47c",
6 "mtime_ns": 1789206821144296891
7 },
8 {
9 "path": "links.one",
10 "bytes": 6128,
11 "sha256": "0cfb356ed374fe919f73fb73fdbe5d66e0baa484d4119f70074d9f869d6db78a",
12 "mtime_ns": 1789206821116200533
13 }
14]
corpus/math-edit/native-enter/teardown.json created+1
...@@ -0,0 +1 @@
1{"absent": true}
corpus/math-edit/native-linear/notebook/links.one created
Binary files /dev/null and b/corpus/math-edit/native-linear/notebook/links.one differ
corpus/page-versions/README.md created+65
...@@ -0,0 +1,65 @@
1# Page versions
2
3What OneNote 2010 stores for a page's earlier versions (Share, Page Versions), what its
4Restore Version, Delete Version, Copy Page To and Delete All Versions write, and that it
5reads and deletes the versions Rust writes. `tools/test_page_versions.py` checks the rows
6without a VM; `crates/onestore/tests/versions.rs` compares Rust's writes with OneNote's.
7
8## Storage
9
10A version is an earlier revision of the page's own object space, kept current under a
11context of its own (MS-ONE 2.1.17, 2.1.18); nothing is copied. The page's *version history*
12is another revision of the same space, current under context
13`{7111497F-1B6B-4209-9491-C98B04CF4C5A},1`: root `jcidVersionHistoryContent` (`0x6003c`)
14whose `ElementChildNodesOfVersionHistory` (`0x24001c20`) lists a `jcidVersionProxy`
15(`0x6003d`) per version, newest first. A proxy holds `VersionContextNodes` (`0x3400347b`,
16the version's context), `LastModifiedTimeStamp` and `AuthorMostRecent` of the version (from
17its revision metadata, root role 4) and a `CreationTimeStamp` of when the version was made.
18The page manifest names the history context in `0x3400347b` from the page's creation, and
19the page's metadata and the section's copy of it carry `HasVersionPages` (`0x88003462`)
20while any version is listed. A version's context is its revision identity XOR
21`9D52FE7C-8FBC-4495-9300-129B5797A9EA`, its proxy's identity the context XOR
22`88440BCA-118C-0E81-32BF-83DB8CB8B6B5`.
23
24- **Making one.** When a different author edits the page, OneNote labels the page's last
25 revision as a version (a `0x5d` RevisionRoleAndContextDeclaration in the space's list)
26 and appends a history revision listing its proxy. One author's edits make none.
27- **Restore Version** (`step-03` to `step-04`): the page's next revision depends on the
28 version's revision (the page's chain forks there), its revision metadata naming who
29 restored it; the page as it stood is labelled as the newest version and the history lists
30 it first. The version restored stays listed.
31- **Delete Version** (`step-04` to `step-05`): only the history changes, dropping the proxy.
32 The version's label and revisions stay in the file until Optimize drops them. Deleting
33 the last one (`step-07` to `step-08`) also clears `HasVersionPages`.
34- **Delete All Versions in Section** (`step-05` to `step-06`) asks first ("Do you want to
35 delete all page versions in the section "History"?") and writes the same as deleting each.
36- **Copy Page To** (`step-02` to `step-03`) copies the version as a new page without
37 versions; the copy's manifest still names the history context, which no revision
38 carries. Readers must accept that.
39- **Optimize** (`step-10`, two years on) rewrites each kept version and the history as
40 checkpoints; versions are not expired by age here.
41
42## Interface
43
44A version is listed under its page as "M/D/YYYY author" (`shots/06-shown-again.png`) once
45the page's Show Page Versions (page menu, `shots/22-page-menu.png`) or Share, Page
46Versions is chosen. An open version is read-only under a yellow bar (RGB 255, 238, 194):
47"This is an earlier version of the page. It will be deleted over time. Click here to restore
48or delete this version." What the version changed since the one before it is banded green
49(RGB 214, 255, 214). The bar's menu (`shots/02-version-menu.png`): Restore Version, Delete
50Version, Copy Page To…; Delete All Versions in Section, Section Group, Notebook; Disable
51History for This Notebook; Hide Page Versions. Share also has Hide Authors, Recent Edits and
52Find by Author (`shots/14`–`16`).
53
54## Rows
55
56- `native`: one OneNote client driven by hand, authors switched through the Office user
57 name. `step-NN/notebook` is the section after each step above, `shots/` the screens.
58- `native-scripted`: the same steps, one to nine, by `tools/native_versions.py observe`.
59- `candidate-restore`, `candidate-delete-all`: `restoring_a_version_stores_what_onenote_stores`
60 (Restore Version of `step-03`'s version) and `deleting_versions_stores_what_onenote_stores`
61 (Delete Version of `step-05`'s last one), `ONESTORE_VERSIONS_EXPORT`.
62- `cold-restore`: OneNote's fresh read of `candidate-restore` (`native_versions.py cold
63 --delete`): both versions listed, the newest one Rust labelled opened read-only under the
64 bar and deleted through its menu (`notebook` is what OneNote left). `cold-delete-all`: its
65 fresh read of `candidate-delete-all`, the page without versions.
corpus/page-versions/candidate-delete-all/History.one created
Binary files /dev/null and b/corpus/page-versions/candidate-delete-all/History.one differ
corpus/page-versions/candidate-delete-all/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/page-versions/candidate-delete-all/Open Notebook.onetoc2 differ
corpus/page-versions/candidate-restore/History.one created
Binary files /dev/null and b/corpus/page-versions/candidate-restore/History.one differ
corpus/page-versions/candidate-restore/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/page-versions/candidate-restore/Open Notebook.onetoc2 differ
corpus/page-versions/cold-delete-all/before-read/environment.json created+7
...@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-9A6697E1",
5 "cold": true,
6 "onenote": "14.0.4763.1000"
7}
corpus/page-versions/cold-delete-all/before-read/hierarchy.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{8E516E85-6F39-4FC7-9D9E-14BE1F339DC1}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-28T03:50:41.000Z" color="#EE9597"><one:Section name="History" ID="{CC235E69-ECFE-02B0-0E8C-F3802399792B}{1}{B0}" path="C:\one-tests\runs\capture\notebook\History.one" lastModifiedTime="2026-09-28T03:50:41.000Z" color="#B49EDE"><one:Page ID="{9193CD4B-2F8C-055F-3BDB-4EFF291B5152}{1}{B0}" name="Versioned" dateTime="2026-09-28T00:42:43.000Z" lastModifiedTime="2026-09-28T00:47:43.000Z" pageLevel="1"/><one:Page ID="{D9B5FD1A-04B4-0CF5-0EA8-B062ED89939C}{1}{B0}" name="Versioned" dateTime="2026-09-28T00:42:43.000Z" lastModifiedTime="2026-09-28T00:43:34.000Z" pageLevel="1"/></one:Section></one:Notebook>
corpus/page-versions/cold-delete-all/before-read/page-000.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{9193CD4B-2F8C-055F-3BDB-4EFF291B5152}{1}{B0}" name="Versioned" dateTime="2026-09-28T00:42:43.000Z" lastModifiedTime="2026-09-28T00:47:43.000Z" pageLevel="1" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Title lang="en-US"><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T00:42:44.000Z" lastModifiedTime="2026-09-28T00:43:03.000Z" objectID="{41C4BEEF-F582-06A0-154D-9894867A7B00}{33}{B0}" alignment="left" quickStyleIndex="0" style="font-family:Calibri;font-size:17.0pt"><one:T><![CDATA[Versioned]]></one:T></one:OE></one:Title><one:Outline author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" lastModifiedTime="2026-09-28T00:43:34.000Z" objectID="{41C4BEEF-F582-06A0-154D-9894867A7B00}{39}{B0}"><one:Position x="36.0" y="86.0" z="0"/><one:Size width="163.7793731689453" height="13.42771339416504"/><one:OEChildren><one:OE creationTime="2026-09-28T00:43:34.000Z" lastModifiedTime="2026-09-28T00:43:34.000Z" objectID="{41C4BEEF-F582-06A0-154D-9894867A7B00}{40}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[First state. Typed by virtual.]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/page-versions/cold-delete-all/before-read/page-001.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{D9B5FD1A-04B4-0CF5-0EA8-B062ED89939C}{1}{B0}" name="Versioned" dateTime="2026-09-28T00:42:43.000Z" lastModifiedTime="2026-09-28T00:43:34.000Z" pageLevel="1" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Title lang="en-US"><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T00:42:44.000Z" lastModifiedTime="2026-09-28T00:43:03.000Z" objectID="{29C73CCC-F3F6-0810-2004-F2FC1C59B929}{22}{B0}" alignment="left" quickStyleIndex="0" style="font-family:Calibri;font-size:17.0pt"><one:T><![CDATA[Versioned]]></one:T></one:OE></one:Title><one:Outline author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" lastModifiedTime="2026-09-28T00:47:16.000Z" objectID="{29C73CCC-F3F6-0810-2004-F2FC1C59B929}{15}{B0}"><one:Position x="36.0" y="86.0" z="0"/><one:Size width="163.7793731689453" height="13.42771339416504"/><one:OEChildren><one:OE creationTime="2026-09-28T00:43:34.000Z" lastModifiedTime="2026-09-28T00:43:34.000Z" objectID="{29C73CCC-F3F6-0810-2004-F2FC1C59B929}{16}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[First state. Typed by virtual.]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/page-versions/cold-delete-all/before-read/payloads.json created+3
...@@ -0,0 +1,3 @@
1[
2
3]
\ No newline at end of file
corpus/page-versions/cold-delete-all/commands.jsonl created+6
...@@ -0,0 +1,6 @@
1{"command": "powershell -NoProfile -Command \"Expand-Archive -LiteralPath C:\\one-tests\\transfer.zip -DestinationPath C:\\one-tests\\runs\\capture\\notebook\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-9A6697E1 -ExpectedPages -1 -KeepOpen", "exit": 0, "stdout": "Read 1 sections and 2 pages.\r\n", "stderr": "", "error": null}
3{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
4{"command": "powershell -NoProfile -Command \"Rename-Item C:\\one-tests\\runs\\capture\\read before-read\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
5{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-9A6697E1 -UseCurrentCache -ExpectedPages -1", "exit": 0, "stdout": "Read 1 sections and 2 pages.\r\n", "stderr": "", "error": null}
6{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
corpus/page-versions/cold-delete-all/machine.json created+1
...@@ -0,0 +1 @@
1{"name": "m6-9a6697e1", "hostname": "ONE-M6-9A6697E1"}
corpus/page-versions/cold-delete-all/notebook/History.one created
Binary files /dev/null and b/corpus/page-versions/cold-delete-all/notebook/History.one differ
corpus/page-versions/cold-delete-all/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/page-versions/cold-delete-all/notebook/Open Notebook.onetoc2 differ
corpus/page-versions/cold-delete-all/read/environment.json created+7
...@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-9A6697E1",
5 "cold": false,
6 "onenote": "14.0.4763.1000"
7}
corpus/page-versions/cold-delete-all/read/hierarchy.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{8E516E85-6F39-4FC7-9D9E-14BE1F339DC1}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-28T03:50:41.000Z" color="#EE9597" isCurrentlyViewed="true"><one:Section name="History" ID="{CC235E69-ECFE-02B0-0E8C-F3802399792B}{1}{B0}" path="C:\one-tests\runs\capture\notebook\History.one" lastModifiedTime="2026-09-28T03:50:41.000Z" color="#B49EDE" isCurrentlyViewed="true"><one:Page ID="{9193CD4B-2F8C-055F-3BDB-4EFF291B5152}{1}{B0}" name="Versioned" dateTime="2026-09-28T00:42:43.000Z" lastModifiedTime="2026-09-28T00:47:43.000Z" pageLevel="1"/><one:Page ID="{D9B5FD1A-04B4-0CF5-0EA8-B062ED89939C}{1}{B0}" name="Versioned" dateTime="2026-09-28T00:42:43.000Z" lastModifiedTime="2026-09-28T00:43:34.000Z" pageLevel="1" isCurrentlyViewed="true"/></one:Section></one:Notebook>
corpus/page-versions/cold-delete-all/read/page-000.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{9193CD4B-2F8C-055F-3BDB-4EFF291B5152}{1}{B0}" name="Versioned" dateTime="2026-09-28T00:42:43.000Z" lastModifiedTime="2026-09-28T03:52:19.000Z" pageLevel="1" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Title lang="en-US"><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T00:42:44.000Z" lastModifiedTime="2026-09-28T00:43:03.000Z" objectID="{41C4BEEF-F582-06A0-154D-9894867A7B00}{33}{B0}" alignment="left" quickStyleIndex="0" style="font-family:Calibri;font-size:17.0pt"><one:T><![CDATA[Versioned]]></one:T></one:OE></one:Title><one:Outline author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" lastModifiedTime="2026-09-28T00:43:34.000Z" objectID="{41C4BEEF-F582-06A0-154D-9894867A7B00}{39}{B0}"><one:Position x="36.0" y="86.0" z="0"/><one:Size width="163.7793731689453" height="13.42771339416504"/><one:OEChildren><one:OE creationTime="2026-09-28T00:43:34.000Z" lastModifiedTime="2026-09-28T00:43:34.000Z" objectID="{41C4BEEF-F582-06A0-154D-9894867A7B00}{40}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[First state. Typed by virtual.]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/page-versions/cold-delete-all/read/page-001.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{D9B5FD1A-04B4-0CF5-0EA8-B062ED89939C}{1}{B0}" name="Versioned" dateTime="2026-09-28T00:42:43.000Z" lastModifiedTime="2026-09-28T03:52:31.000Z" pageLevel="1" isCurrentlyViewed="true" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Title lang="en-US"><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T00:42:44.000Z" lastModifiedTime="2026-09-28T00:43:03.000Z" objectID="{29C73CCC-F3F6-0810-2004-F2FC1C59B929}{22}{B0}" alignment="left" quickStyleIndex="0" style="font-family:Calibri;font-size:17.0pt"><one:T><![CDATA[Versioned]]></one:T></one:OE></one:Title><one:Outline author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" lastModifiedTime="2026-09-28T00:47:16.000Z" objectID="{29C73CCC-F3F6-0810-2004-F2FC1C59B929}{15}{B0}"><one:Position x="36.0" y="86.0" z="0"/><one:Size width="163.7793731689453" height="13.42771339416504"/><one:OEChildren><one:OE creationTime="2026-09-28T00:43:34.000Z" lastModifiedTime="2026-09-28T00:43:34.000Z" objectID="{29C73CCC-F3F6-0810-2004-F2FC1C59B929}{16}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[First state. Typed by virtual.]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/page-versions/cold-delete-all/read/payloads.json created+3
...@@ -0,0 +1,3 @@
1[
2
3]
\ No newline at end of file
corpus/page-versions/cold-delete-all/run.json created+18
...@@ -0,0 +1,18 @@
1{
2 "notebook": "/Users/clo/dev/one/corpus/page-versions/candidate-delete-all",
3 "expected_pages": -1,
4 "author": null,
5 "author_timeout_seconds": 600,
6 "inspect": false,
7 "collect_notebook": true,
8 "base": {
9 "file": "win7-office-base.qcow2",
10 "format": "qcow2",
11 "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346",
12 "virtual_size": 68719476736
13 },
14 "scripts": {
15 "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331",
16 "read.ps1": "04013bfcccee40a17e2a225f9b9e96f40a3a8daad9e350609654f8eda3ccbb41"
17 }
18}
corpus/page-versions/cold-delete-all/scripts/cold.ps1 created+27
...@@ -0,0 +1,27 @@
1param([Parameter(Mandatory=$true)][string]$Root, [string]$CloneHost = '')
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
5if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
6 throw 'Choose a run directly below C:\one-tests\runs.'
7}
8if (Get-Process ONENOTE -ErrorAction SilentlyContinue) { throw 'Close OneNote before resetting its test cache.' }
9$key = 'HKCU:\Software\Microsoft\Office\14.0\OneNote'
10if ($CloneHost) {
11 if ($CloneHost -notmatch '^ONE-[A-Z0-9-]+$' -or [Environment]::MachineName -ne $CloneHost) {
12 throw 'The disposable clone hostname does not match this machine.'
13 }
14 New-Item "$key\Options\Paths" -Force | Out-Null
15 New-ItemProperty "$key\Options\Paths" -Name UnfiledNotesSection -PropertyType ExpandString -Value 'C:\one-tests\Loose.one' -Force | Out-Null
16} elseif ((Get-ItemProperty "$key\Options\Paths").UnfiledNotesSection -ne 'C:\one-tests\Loose.one' -or
17 -not (Test-Path 'C:\one-tests\profile-original-cache')) {
18 throw 'Park the personal OneNote profile before resetting the test cache.'
19}
20$cache = Join-Path $env:LOCALAPPDATA 'Microsoft\OneNote\14.0'
21$parked = Join-Path 'C:\one-tests\caches' ([IO.Path]::GetFileName($root))
22if (Test-Path $parked) { throw 'Choose a new run; its parked cache already exists.' }
23New-Item -ItemType Directory -Path 'C:\one-tests\caches' -Force | Out-Null
24if (Test-Path $cache) { Move-Item -LiteralPath $cache -Destination $parked }
25if (Test-Path "$key\OpenNotebooks") { Remove-Item "$key\OpenNotebooks" -Recurse }
26New-Item "$key\OpenNotebooks" | Out-Null
27New-ItemProperty "$key\OpenNotebooks" -Name '1' -PropertyType String -Value "$root\notebook" | Out-Null
corpus/page-versions/cold-delete-all/scripts/read.ps1 created+142
...@@ -0,0 +1,142 @@
1param(
2 [Parameter(Mandatory=$true)][string]$Root,
3 [int]$ExpectedPages = -1,
4 [switch]$UseCurrentCache,
5 [switch]$Pdf,
6 [switch]$KeepOpen,
7 [string]$CloneHost = ''
8)
9Set-StrictMode -Version Latest
10$ErrorActionPreference = 'Stop'
11$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
12if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
13 throw 'Choose a run directly below C:\one-tests\runs.'
14}
15$notebook = Join-Path $root 'notebook'
16$output = Join-Path $root 'read'
17if (Test-Path $output) { throw 'Choose a new read destination.' }
18if ($UseCurrentCache) {
19 if ((Get-ItemProperty 'HKCU:\Software\Microsoft\Office\14.0\OneNote\Options\Paths').UnfiledNotesSection -ne 'C:\one-tests\Loose.one') {
20 throw 'Park the personal OneNote profile before reading test notebooks.'
21 }
22} else {
23 & "$PSScriptRoot\cold-current.ps1" -Root $root -CloneHost $CloneHost
24}
25New-Item -ItemType Directory -Path $output | Out-Null
26$app = New-Object -ComObject OneNote.Application
27$notebookId = ''
28$failure = $null
29try {
30 $app.OpenHierarchy($notebook, '', [ref]$notebookId, 0)
31 $process = Get-Process ONENOTE
32 @{ hostname = [Environment]::MachineName; onenote = $process.MainModule.FileVersionInfo.FileVersion;
33 powershell = $PSVersionTable.PSVersion.ToString(); schema = 'xs2010'; cold = (-not $UseCurrentCache.IsPresent) } |
34 ConvertTo-Json | Set-Content (Join-Path $output 'environment.json') -Encoding UTF8
35 $sections = @()
36 foreach ($file in @(Get-ChildItem $notebook -Recurse | Where-Object { $_.Extension -eq '.one' })) {
37 $id = ''
38 $app.OpenHierarchy($file.FullName, '', [ref]$id, 0)
39 $sections += $id
40 }
41 $deadline = [DateTime]::UtcNow.AddSeconds(300)
42 $previous = ''
43 $lastChange = ''
44 $stableSince = [DateTime]::UtcNow
45 $settled = $false
46 do {
47 $pages = @{}
48 foreach ($section in $sections) {
49 $hierarchy = ''
50 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
51 [xml]$xml = $hierarchy
52 foreach ($node in $xml.SelectNodes('//*[@path]')) {
53 if (-not $node.GetAttribute('path').StartsWith("$notebook\", [StringComparison]::OrdinalIgnoreCase)) {
54 throw 'OneNote opened a section outside the copied notebook.'
55 }
56 }
57 foreach ($node in $xml.SelectNodes('//*[local-name()="Page"]')) {
58 $id = $node.GetAttribute('ID')
59 $content = ''
60 $app.GetPageContent($id, [ref]$content, 1, 1)
61 $pages[$id] = $content
62 }
63 }
64 $signature = [String]::Join('|', @($pages.Keys | Sort-Object | ForEach-Object { $_ + $pages[$_] }))
65 if ($signature -ne $previous) {
66 $lastChange = $previous
67 $previous = $signature
68 $stableSince = [DateTime]::UtcNow
69 }
70 if ((($ExpectedPages -ge 0 -and $pages.Count -eq $ExpectedPages) -or
71 ($ExpectedPages -lt 0 -and $pages.Count -gt 0)) -and
72 ([DateTime]::UtcNow - $stableSince).TotalSeconds -ge 2) { $settled = $true; break }
73 Start-Sleep -Milliseconds 250
74 } while ([DateTime]::UtcNow -lt $deadline)
75 if (-not $settled -or ($ExpectedPages -ge 0 -and $pages.Count -ne $ExpectedPages) -or ($ExpectedPages -lt 0 -and $pages.Count -eq 0)) {
76 [IO.File]::WriteAllText((Join-Path $output 'previous-signature.txt'), $lastChange, [Text.Encoding]::UTF8)
77 $index = 0
78 foreach ($id in @($pages.Keys | Sort-Object)) {
79 [IO.File]::WriteAllText((Join-Path $output ('unsettled-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
80 $index++
81 }
82 $hierarchy = ''
83 $app.GetHierarchy($notebookId, 4, [ref]$hierarchy, 1)
84 [IO.File]::WriteAllText((Join-Path $output 'unsettled-hierarchy.xml'), $hierarchy, [Text.Encoding]::UTF8)
85 throw "Expected $ExpectedPages stable pages; OneNote returned $($pages.Count), settled=$settled."
86 }
87 $index = 0
88 $payloads = @()
89 foreach ($id in @($pages.Keys | Sort-Object)) {
90 [IO.File]::WriteAllText((Join-Path $output ('page-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
91 if ($Pdf) {
92 $pdfPath = Join-Path $output ('page-{0:d3}.pdf' -f $index)
93 $app.NavigateTo($id, '', $false)
94 $app.Publish($id, $pdfPath, 3, '')
95 if (-not (Test-Path $pdfPath) -or (Get-Item $pdfPath).Length -eq 0) { throw 'OneNote did not publish the page PDF.' }
96 }
97 [xml]$page = $pages[$id]
98 foreach ($file in $page.SelectNodes('//*[local-name()="InsertedFile" or local-name()="MediaFile"]')) {
99 $bytes = [IO.File]::ReadAllBytes($file.GetAttribute('pathCache'))
100 $hash = [BitConverter]::ToString([Security.Cryptography.SHA256]::Create().ComputeHash($bytes)).Replace('-', '').ToLowerInvariant()
101 [IO.File]::WriteAllBytes((Join-Path $output ($hash + '.attachment')), $bytes)
102 $payloads += @{ page = $id; object = $file.ParentNode.GetAttribute('objectID');
103 kind = $file.LocalName; name = $file.GetAttribute('preferredName');
104 sha256 = $hash; bytes = $bytes.Length }
105 }
106 $index++
107 }
108 [IO.File]::WriteAllText((Join-Path $output 'payloads.json'), (ConvertTo-Json -InputObject $payloads -Depth 4), [Text.Encoding]::UTF8)
109 $all = ''
110 $app.GetHierarchy($notebookId, 4, [ref]$all, 1)
111 [xml]$finalTree = $all
112 $finalIds = @($finalTree.SelectNodes('//*[local-name()="Page"]') | ForEach-Object { $_.GetAttribute('ID') } | Sort-Object -Unique)
113 if ($finalIds.Count -ne $pages.Count -or @($finalIds | Where-Object { -not $pages.ContainsKey($_) }).Count -ne 0) {
114 throw 'The notebook hierarchy changed while collecting page evidence; repeat the cold read.'
115 }
116 [IO.File]::WriteAllText((Join-Path $output 'hierarchy.xml'), $all, [Text.Encoding]::UTF8)
117 Write-Output "Read $($sections.Count) sections and $($pages.Count) pages."
118} catch {
119 $failure = $_
120 [IO.File]::WriteAllText((Join-Path $output 'failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
121 throw
122} finally {
123 try {
124 try {
125 if ($notebookId -and $CloneHost) { $app.SyncHierarchy($notebookId) }
126 if ($notebookId -and -not $KeepOpen -and (-not $UseCurrentCache -or $CloneHost)) {
127 $app.CloseNotebook($notebookId, $false)
128 }
129 } catch {
130 if ($null -eq $failure) { throw }
131 [IO.File]::WriteAllText((Join-Path $output 'cleanup-failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
132 }
133 } finally {
134 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
135 }
136 $app = $null
137 [GC]::Collect()
138 [GC]::WaitForPendingFinalizers()
139 if (-not $UseCurrentCache -and -not $CloneHost) {
140 Get-Process ONENOTE -ErrorAction SilentlyContinue | Wait-Process -Timeout 10
141 }
142}
corpus/page-versions/cold-delete-all/shots/cold-closed.ahk created+4
...@@ -0,0 +1,4 @@
1WinActivate("ahk_exe ONENOTE.EXE")
2CoordMode("Mouse", "Screen")
3Send("{Esc}")
4Sleep(800)
corpus/page-versions/cold-delete-all/shots/cold-closed.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "",
10 "class": "Net UI Tool Window",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/page-versions/cold-delete-all/shots/cold-closed.png created
Binary files /dev/null and b/corpus/page-versions/cold-delete-all/shots/cold-closed.png differ
corpus/page-versions/cold-delete-all/shots/cold-main.ahk created+14
...@@ -0,0 +1,14 @@
1OnError((exception, mode) => (FileAppend(exception.Message, "**"), ExitApp(1)))
2DetectHiddenWindows false
3app := ComObject("OneNote.Application")
4app.NavigateTo("{9193CD4B-2F8C-055F-3BDB-4EFF291B5152}{1}{B0}", "", false)
5hwnd := WinWait("ahk_class Framework::CFrame ahk_exe ONENOTE.EXE",, 10)
6if !hwnd
7 throw Error("OneNote window did not appear")
8WinMaximize(hwnd)
9WinActivate(hwnd)
10if !WinWaitActive(hwnd,, 10)
11 throw Error("OneNote window did not become active")
12app.NavigateTo("{9193CD4B-2F8C-055F-3BDB-4EFF291B5152}{1}{B0}", "", false)
13Sleep 300
14Sleep(2000)
corpus/page-versions/cold-delete-all/shots/cold-main.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Versioned - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/page-versions/cold-delete-all/shots/cold-main.png created
Binary files /dev/null and b/corpus/page-versions/cold-delete-all/shots/cold-main.png differ
corpus/page-versions/cold-delete-all/shots/cold-menu.ahk created+4
...@@ -0,0 +1,4 @@
1WinActivate("ahk_exe ONENOTE.EXE")
2CoordMode("Mouse", "Screen")
3Click(320, 93)
4Sleep(1000)
corpus/page-versions/cold-delete-all/shots/cold-menu.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Versioned - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/page-versions/cold-delete-all/shots/cold-menu.png created
Binary files /dev/null and b/corpus/page-versions/cold-delete-all/shots/cold-menu.png differ
corpus/page-versions/cold-delete-all/shots/cold-shown.ahk created+6
...@@ -0,0 +1,6 @@
1WinActivate("ahk_exe ONENOTE.EXE")
2CoordMode("Mouse", "Screen")
3Click(202, 36)
4Sleep(600)
5Click(443, 80)
6Sleep(1500)
corpus/page-versions/cold-delete-all/shots/cold-shown.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "",
10 "class": "Net UI Tool Window",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/page-versions/cold-delete-all/shots/cold-shown.png created
Binary files /dev/null and b/corpus/page-versions/cold-delete-all/shots/cold-shown.png differ
corpus/page-versions/cold-delete-all/shots/cold-version.ahk created+4
...@@ -0,0 +1,4 @@
1WinActivate("ahk_exe ONENOTE.EXE")
2CoordMode("Mouse", "Screen")
3Click(710, 143)
4Sleep(1500)
corpus/page-versions/cold-delete-all/shots/cold-version.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "",
10 "class": "Net UI Tool Window",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/page-versions/cold-delete-all/shots/cold-version.png created
Binary files /dev/null and b/corpus/page-versions/cold-delete-all/shots/cold-version.png differ
corpus/page-versions/cold-delete-all/source.json created+14
...@@ -0,0 +1,14 @@
1[
2 {
3 "path": "History.one",
4 "bytes": 23048,
5 "sha256": "ae4b6cc0de2cc2384c8bd67f55a2d468ce100698c5c7fc0585d30008d77257cb",
6 "mtime_ns": 1790567288571022815
7 },
8 {
9 "path": "Open Notebook.onetoc2",
10 "bytes": 4808,
11 "sha256": "21f33fee6a5f2946e7d9394036a37cc1bf062856449eed5a0eceaf682dde76fc",
12 "mtime_ns": 1790567288570444733
13 }
14]
corpus/page-versions/cold-delete-all/teardown.json created+1
...@@ -0,0 +1 @@
1{"absent": true}
corpus/page-versions/cold-restore/before-read/environment.json created+7
...@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-7319BB2D",
5 "cold": true,
6 "onenote": "14.0.4763.1000"
7}
corpus/page-versions/cold-restore/before-read/hierarchy.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{5B778416-7313-4D11-87CC-FA8573783DF1}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-28T03:50:30.000Z" color="#EE9597"><one:Section name="History" ID="{E9212367-F870-0A56-22F4-5CE45E73E5B8}{1}{B0}" path="C:\one-tests\runs\capture\notebook\History.one" lastModifiedTime="2026-09-28T03:50:30.000Z" color="#B49EDE"><one:Page ID="{B491B045-3B02-0DB9-17A3-E19B54F1CDC1}{1}{B0}" name="Versioned" dateTime="2026-09-28T00:42:43.000Z" lastModifiedTime="2025-10-03T21:20:00.000Z" pageLevel="1"/><one:Page ID="{FCB78014-103A-0413-22D0-1F0690630F0F}{1}{B0}" name="Versioned" dateTime="2026-09-28T00:42:43.000Z" lastModifiedTime="2026-09-28T00:43:34.000Z" pageLevel="1"/></one:Section></one:Notebook>
corpus/page-versions/cold-restore/before-read/page-000.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{B491B045-3B02-0DB9-17A3-E19B54F1CDC1}{1}{B0}" name="Versioned" dateTime="2026-09-28T00:42:43.000Z" lastModifiedTime="2025-10-03T21:20:00.000Z" pageLevel="1" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Title lang="en-US"><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T00:42:44.000Z" lastModifiedTime="2026-09-28T00:43:03.000Z" objectID="{64C6C3E1-E10C-0E46-3935-37F0FB90E793}{33}{B0}" alignment="left" quickStyleIndex="0" style="font-family:Calibri;font-size:17.0pt"><one:T><![CDATA[Versioned]]></one:T></one:OE></one:Title><one:Outline author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" lastModifiedTime="2026-09-28T00:43:34.000Z" objectID="{64C6C3E1-E10C-0E46-3935-37F0FB90E793}{39}{B0}"><one:Position x="36.0" y="86.0" z="0"/><one:Size width="163.7793731689453" height="13.42771339416504"/><one:OEChildren><one:OE creationTime="2026-09-28T00:43:34.000Z" lastModifiedTime="2026-09-28T00:43:34.000Z" objectID="{64C6C3E1-E10C-0E46-3935-37F0FB90E793}{40}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[First state. Typed by virtual.]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/page-versions/cold-restore/before-read/page-001.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{FCB78014-103A-0413-22D0-1F0690630F0F}{1}{B0}" name="Versioned" dateTime="2026-09-28T00:42:43.000Z" lastModifiedTime="2026-09-28T00:43:34.000Z" pageLevel="1" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Title lang="en-US"><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T00:42:44.000Z" lastModifiedTime="2026-09-28T00:43:03.000Z" objectID="{0CC541C2-E778-00F6-0C7C-5D9861B325BA}{22}{B0}" alignment="left" quickStyleIndex="0" style="font-family:Calibri;font-size:17.0pt"><one:T><![CDATA[Versioned]]></one:T></one:OE></one:Title><one:Outline author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" lastModifiedTime="2026-09-28T00:47:16.000Z" objectID="{0CC541C2-E778-00F6-0C7C-5D9861B325BA}{15}{B0}"><one:Position x="36.0" y="86.0" z="0"/><one:Size width="163.7793731689453" height="13.42771339416504"/><one:OEChildren><one:OE creationTime="2026-09-28T00:43:34.000Z" lastModifiedTime="2026-09-28T00:43:34.000Z" objectID="{0CC541C2-E778-00F6-0C7C-5D9861B325BA}{16}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[First state. Typed by virtual.]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/page-versions/cold-restore/before-read/payloads.json created+3
...@@ -0,0 +1,3 @@
1[
2
3]
\ No newline at end of file
corpus/page-versions/cold-restore/commands.jsonl created+6
...@@ -0,0 +1,6 @@
1{"command": "powershell -NoProfile -Command \"Expand-Archive -LiteralPath C:\\one-tests\\transfer.zip -DestinationPath C:\\one-tests\\runs\\capture\\notebook\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-7319BB2D -ExpectedPages -1 -KeepOpen", "exit": 0, "stdout": "Read 1 sections and 2 pages.\r\n", "stderr": "", "error": null}
3{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
4{"command": "powershell -NoProfile -Command \"Rename-Item C:\\one-tests\\runs\\capture\\read before-read\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
5{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-7319BB2D -UseCurrentCache -ExpectedPages -1", "exit": 0, "stdout": "Read 1 sections and 2 pages.\r\n", "stderr": "", "error": null}
6{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
corpus/page-versions/cold-restore/machine.json created+1
...@@ -0,0 +1 @@
1{"name": "m6-7319bb2d", "hostname": "ONE-M6-7319BB2D"}
corpus/page-versions/cold-restore/notebook/History.one created
Binary files /dev/null and b/corpus/page-versions/cold-restore/notebook/History.one differ
corpus/page-versions/cold-restore/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/page-versions/cold-restore/notebook/Open Notebook.onetoc2 differ
corpus/page-versions/cold-restore/read/environment.json created+7
...@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-7319BB2D",
5 "cold": false,
6 "onenote": "14.0.4763.1000"
7}
corpus/page-versions/cold-restore/read/hierarchy.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{5B778416-7313-4D11-87CC-FA8573783DF1}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-28T03:50:30.000Z" color="#EE9597" isCurrentlyViewed="true"><one:Section name="History" ID="{E9212367-F870-0A56-22F4-5CE45E73E5B8}{1}{B0}" path="C:\one-tests\runs\capture\notebook\History.one" lastModifiedTime="2026-09-28T03:50:30.000Z" color="#B49EDE" isCurrentlyViewed="true"><one:Page ID="{B491B045-3B02-0DB9-17A3-E19B54F1CDC1}{1}{B0}" name="Versioned" dateTime="2026-09-28T00:42:43.000Z" lastModifiedTime="2025-10-03T21:20:00.000Z" pageLevel="1" isCurrentlyViewed="true"/><one:Page ID="{FCB78014-103A-0413-22D0-1F0690630F0F}{1}{B0}" name="Versioned" dateTime="2026-09-28T00:42:43.000Z" lastModifiedTime="2026-09-28T00:43:34.000Z" pageLevel="1"/></one:Section></one:Notebook>
corpus/page-versions/cold-restore/read/page-000.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{B491B045-3B02-0DB9-17A3-E19B54F1CDC1}{1}{B0}" name="Versioned" dateTime="2026-09-28T00:42:43.000Z" lastModifiedTime="2026-09-28T03:51:38.000Z" pageLevel="1" isCurrentlyViewed="true" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Title lang="en-US"><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T00:42:44.000Z" lastModifiedTime="2026-09-28T00:43:03.000Z" objectID="{64C6C3E1-E10C-0E46-3935-37F0FB90E793}{33}{B0}" alignment="left" quickStyleIndex="0" style="font-family:Calibri;font-size:17.0pt"><one:T><![CDATA[Versioned]]></one:T></one:OE></one:Title><one:Outline author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" lastModifiedTime="2026-09-28T00:43:34.000Z" objectID="{64C6C3E1-E10C-0E46-3935-37F0FB90E793}{39}{B0}"><one:Position x="36.0" y="86.0" z="0"/><one:Size width="163.7793731689453" height="13.42771339416504"/><one:OEChildren><one:OE creationTime="2026-09-28T00:43:34.000Z" lastModifiedTime="2026-09-28T00:43:34.000Z" objectID="{64C6C3E1-E10C-0E46-3935-37F0FB90E793}{40}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[First state. Typed by virtual.]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/page-versions/cold-restore/read/page-001.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{FCB78014-103A-0413-22D0-1F0690630F0F}{1}{B0}" name="Versioned" dateTime="2026-09-28T00:42:43.000Z" lastModifiedTime="2026-09-28T00:43:34.000Z" pageLevel="1" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Title lang="en-US"><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-28T00:42:44.000Z" lastModifiedTime="2026-09-28T00:43:03.000Z" objectID="{0CC541C2-E778-00F6-0C7C-5D9861B325BA}{22}{B0}" alignment="left" quickStyleIndex="0" style="font-family:Calibri;font-size:17.0pt"><one:T><![CDATA[Versioned]]></one:T></one:OE></one:Title><one:Outline author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" lastModifiedTime="2026-09-28T00:47:16.000Z" objectID="{0CC541C2-E778-00F6-0C7C-5D9861B325BA}{15}{B0}"><one:Position x="36.0" y="86.0" z="0"/><one:Size width="163.7793731689453" height="13.42771339416504"/><one:OEChildren><one:OE creationTime="2026-09-28T00:43:34.000Z" lastModifiedTime="2026-09-28T00:43:34.000Z" objectID="{0CC541C2-E778-00F6-0C7C-5D9861B325BA}{16}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[First state. Typed by virtual.]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/page-versions/cold-restore/read/payloads.json created+3
...@@ -0,0 +1,3 @@
1[
2
3]
\ No newline at end of file
corpus/page-versions/cold-restore/run.json created+18
...@@ -0,0 +1,18 @@
1{
2 "notebook": "/Users/clo/dev/one/corpus/page-versions/candidate-restore",
3 "expected_pages": -1,
4 "author": null,
5 "author_timeout_seconds": 600,
6 "inspect": false,
7 "collect_notebook": true,
8 "base": {
9 "file": "win7-office-base.qcow2",
10 "format": "qcow2",
11 "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346",
12 "virtual_size": 68719476736
13 },
14 "scripts": {
15 "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331",
16 "read.ps1": "04013bfcccee40a17e2a225f9b9e96f40a3a8daad9e350609654f8eda3ccbb41"
17 }
18}
corpus/page-versions/cold-restore/scripts/cold.ps1 created+27
...@@ -0,0 +1,27 @@
1param([Parameter(Mandatory=$true)][string]$Root, [string]$CloneHost = '')
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
5if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
6 throw 'Choose a run directly below C:\one-tests\runs.'
7}
8if (Get-Process ONENOTE -ErrorAction SilentlyContinue) { throw 'Close OneNote before resetting its test cache.' }
9$key = 'HKCU:\Software\Microsoft\Office\14.0\OneNote'
10if ($CloneHost) {
11 if ($CloneHost -notmatch '^ONE-[A-Z0-9-]+$' -or [Environment]::MachineName -ne $CloneHost) {
12 throw 'The disposable clone hostname does not match this machine.'
13 }
14 New-Item "$key\Options\Paths" -Force | Out-Null
15 New-ItemProperty "$key\Options\Paths" -Name UnfiledNotesSection -PropertyType ExpandString -Value 'C:\one-tests\Loose.one' -Force | Out-Null
16} elseif ((Get-ItemProperty "$key\Options\Paths").UnfiledNotesSection -ne 'C:\one-tests\Loose.one' -or
17 -not (Test-Path 'C:\one-tests\profile-original-cache')) {
18 throw 'Park the personal OneNote profile before resetting the test cache.'
19}
20$cache = Join-Path $env:LOCALAPPDATA 'Microsoft\OneNote\14.0'
21$parked = Join-Path 'C:\one-tests\caches' ([IO.Path]::GetFileName($root))
22if (Test-Path $parked) { throw 'Choose a new run; its parked cache already exists.' }
23New-Item -ItemType Directory -Path 'C:\one-tests\caches' -Force | Out-Null
24if (Test-Path $cache) { Move-Item -LiteralPath $cache -Destination $parked }
25if (Test-Path "$key\OpenNotebooks") { Remove-Item "$key\OpenNotebooks" -Recurse }
26New-Item "$key\OpenNotebooks" | Out-Null
27New-ItemProperty "$key\OpenNotebooks" -Name '1' -PropertyType String -Value "$root\notebook" | Out-Null
corpus/page-versions/cold-restore/scripts/read.ps1 created+142
...@@ -0,0 +1,142 @@
1param(
2 [Parameter(Mandatory=$true)][string]$Root,
3 [int]$ExpectedPages = -1,
4 [switch]$UseCurrentCache,
5 [switch]$Pdf,
6 [switch]$KeepOpen,
7 [string]$CloneHost = ''
8)
9Set-StrictMode -Version Latest
10$ErrorActionPreference = 'Stop'
11$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
12if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
13 throw 'Choose a run directly below C:\one-tests\runs.'
14}
15$notebook = Join-Path $root 'notebook'
16$output = Join-Path $root 'read'
17if (Test-Path $output) { throw 'Choose a new read destination.' }
18if ($UseCurrentCache) {
19 if ((Get-ItemProperty 'HKCU:\Software\Microsoft\Office\14.0\OneNote\Options\Paths').UnfiledNotesSection -ne 'C:\one-tests\Loose.one') {
20 throw 'Park the personal OneNote profile before reading test notebooks.'
21 }
22} else {
23 & "$PSScriptRoot\cold-current.ps1" -Root $root -CloneHost $CloneHost
24}
25New-Item -ItemType Directory -Path $output | Out-Null
26$app = New-Object -ComObject OneNote.Application
27$notebookId = ''
28$failure = $null
29try {
30 $app.OpenHierarchy($notebook, '', [ref]$notebookId, 0)
31 $process = Get-Process ONENOTE
32 @{ hostname = [Environment]::MachineName; onenote = $process.MainModule.FileVersionInfo.FileVersion;
33 powershell = $PSVersionTable.PSVersion.ToString(); schema = 'xs2010'; cold = (-not $UseCurrentCache.IsPresent) } |
34 ConvertTo-Json | Set-Content (Join-Path $output 'environment.json') -Encoding UTF8
35 $sections = @()
36 foreach ($file in @(Get-ChildItem $notebook -Recurse | Where-Object { $_.Extension -eq '.one' })) {
37 $id = ''
38 $app.OpenHierarchy($file.FullName, '', [ref]$id, 0)
39 $sections += $id
40 }
41 $deadline = [DateTime]::UtcNow.AddSeconds(300)
42 $previous = ''
43 $lastChange = ''
44 $stableSince = [DateTime]::UtcNow
45 $settled = $false
46 do {
47 $pages = @{}
48 foreach ($section in $sections) {
49 $hierarchy = ''
50 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
51 [xml]$xml = $hierarchy
52 foreach ($node in $xml.SelectNodes('//*[@path]')) {
53 if (-not $node.GetAttribute('path').StartsWith("$notebook\", [StringComparison]::OrdinalIgnoreCase)) {
54 throw 'OneNote opened a section outside the copied notebook.'
55 }
56 }
57 foreach ($node in $xml.SelectNodes('//*[local-name()="Page"]')) {
58 $id = $node.GetAttribute('ID')
59 $content = ''
60 $app.GetPageContent($id, [ref]$content, 1, 1)
61 $pages[$id] = $content
62 }
63 }
64 $signature = [String]::Join('|', @($pages.Keys | Sort-Object | ForEach-Object { $_ + $pages[$_] }))
65 if ($signature -ne $previous) {
66 $lastChange = $previous
67 $previous = $signature
68 $stableSince = [DateTime]::UtcNow
69 }
70 if ((($ExpectedPages -ge 0 -and $pages.Count -eq $ExpectedPages) -or
71 ($ExpectedPages -lt 0 -and $pages.Count -gt 0)) -and
72 ([DateTime]::UtcNow - $stableSince).TotalSeconds -ge 2) { $settled = $true; break }
73 Start-Sleep -Milliseconds 250
74 } while ([DateTime]::UtcNow -lt $deadline)
75 if (-not $settled -or ($ExpectedPages -ge 0 -and $pages.Count -ne $ExpectedPages) -or ($ExpectedPages -lt 0 -and $pages.Count -eq 0)) {
76 [IO.File]::WriteAllText((Join-Path $output 'previous-signature.txt'), $lastChange, [Text.Encoding]::UTF8)
77 $index = 0
78 foreach ($id in @($pages.Keys | Sort-Object)) {
79 [IO.File]::WriteAllText((Join-Path $output ('unsettled-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
80 $index++
81 }
82 $hierarchy = ''
83 $app.GetHierarchy($notebookId, 4, [ref]$hierarchy, 1)
84 [IO.File]::WriteAllText((Join-Path $output 'unsettled-hierarchy.xml'), $hierarchy, [Text.Encoding]::UTF8)
85 throw "Expected $ExpectedPages stable pages; OneNote returned $($pages.Count), settled=$settled."
86 }
87 $index = 0
88 $payloads = @()
89 foreach ($id in @($pages.Keys | Sort-Object)) {
90 [IO.File]::WriteAllText((Join-Path $output ('page-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
91 if ($Pdf) {
92 $pdfPath = Join-Path $output ('page-{0:d3}.pdf' -f $index)
93 $app.NavigateTo($id, '', $false)
94 $app.Publish($id, $pdfPath, 3, '')
95 if (-not (Test-Path $pdfPath) -or (Get-Item $pdfPath).Length -eq 0) { throw 'OneNote did not publish the page PDF.' }
96 }
97 [xml]$page = $pages[$id]
98 foreach ($file in $page.SelectNodes('//*[local-name()="InsertedFile" or local-name()="MediaFile"]')) {
99 $bytes = [IO.File]::ReadAllBytes($file.GetAttribute('pathCache'))
100 $hash = [BitConverter]::ToString([Security.Cryptography.SHA256]::Create().ComputeHash($bytes)).Replace('-', '').ToLowerInvariant()
101 [IO.File]::WriteAllBytes((Join-Path $output ($hash + '.attachment')), $bytes)
102 $payloads += @{ page = $id; object = $file.ParentNode.GetAttribute('objectID');
103 kind = $file.LocalName; name = $file.GetAttribute('preferredName');
104 sha256 = $hash; bytes = $bytes.Length }
105 }
106 $index++
107 }
108 [IO.File]::WriteAllText((Join-Path $output 'payloads.json'), (ConvertTo-Json -InputObject $payloads -Depth 4), [Text.Encoding]::UTF8)
109 $all = ''
110 $app.GetHierarchy($notebookId, 4, [ref]$all, 1)
111 [xml]$finalTree = $all
112 $finalIds = @($finalTree.SelectNodes('//*[local-name()="Page"]') | ForEach-Object { $_.GetAttribute('ID') } | Sort-Object -Unique)
113 if ($finalIds.Count -ne $pages.Count -or @($finalIds | Where-Object { -not $pages.ContainsKey($_) }).Count -ne 0) {
114 throw 'The notebook hierarchy changed while collecting page evidence; repeat the cold read.'
115 }
116 [IO.File]::WriteAllText((Join-Path $output 'hierarchy.xml'), $all, [Text.Encoding]::UTF8)
117 Write-Output "Read $($sections.Count) sections and $($pages.Count) pages."
118} catch {
119 $failure = $_
120 [IO.File]::WriteAllText((Join-Path $output 'failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
121 throw
122} finally {
123 try {
124 try {
125 if ($notebookId -and $CloneHost) { $app.SyncHierarchy($notebookId) }
126 if ($notebookId -and -not $KeepOpen -and (-not $UseCurrentCache -or $CloneHost)) {
127 $app.CloseNotebook($notebookId, $false)
128 }
129 } catch {
130 if ($null -eq $failure) { throw }
131 [IO.File]::WriteAllText((Join-Path $output 'cleanup-failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
132 }
133 } finally {
134 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
135 }
136 $app = $null
137 [GC]::Collect()
138 [GC]::WaitForPendingFinalizers()
139 if (-not $UseCurrentCache -and -not $CloneHost) {
140 Get-Process ONENOTE -ErrorAction SilentlyContinue | Wait-Process -Timeout 10
141 }
142}
corpus/page-versions/cold-restore/shots/cold-delete.ahk created+7
...@@ -0,0 +1,7 @@
1WinActivate("ahk_exe ONENOTE.EXE")
2CoordMode("Mouse", "Screen")
3Click(393, 128)
4Sleep(2500)
5if WinExist("ahk_class #32770 ahk_exe ONENOTE.EXE") {
6 FileAppend("dialog: " WinGetTitle() " | " WinGetText() "`n", "*", "UTF-8")
7}
corpus/page-versions/cold-restore/shots/cold-delete.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Versioned - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/page-versions/cold-restore/shots/cold-delete.png created
Binary files /dev/null and b/corpus/page-versions/cold-restore/shots/cold-delete.png differ
corpus/page-versions/cold-restore/shots/cold-main.ahk created+14
...@@ -0,0 +1,14 @@
1OnError((exception, mode) => (FileAppend(exception.Message, "**"), ExitApp(1)))
2DetectHiddenWindows false
3app := ComObject("OneNote.Application")
4app.NavigateTo("{B491B045-3B02-0DB9-17A3-E19B54F1CDC1}{1}{B0}", "", false)
5hwnd := WinWait("ahk_class Framework::CFrame ahk_exe ONENOTE.EXE",, 10)
6if !hwnd
7 throw Error("OneNote window did not appear")
8WinMaximize(hwnd)
9WinActivate(hwnd)
10if !WinWaitActive(hwnd,, 10)
11 throw Error("OneNote window did not become active")
12app.NavigateTo("{B491B045-3B02-0DB9-17A3-E19B54F1CDC1}{1}{B0}", "", false)
13Sleep 300
14Sleep(2000)
corpus/page-versions/cold-restore/shots/cold-main.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Versioned - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/page-versions/cold-restore/shots/cold-main.png created
Binary files /dev/null and b/corpus/page-versions/cold-restore/shots/cold-main.png differ
corpus/page-versions/cold-restore/shots/cold-menu.ahk created+4
...@@ -0,0 +1,4 @@
1WinActivate("ahk_exe ONENOTE.EXE")
2CoordMode("Mouse", "Screen")
3Click(320, 93)
4Sleep(1000)
corpus/page-versions/cold-restore/shots/cold-menu.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "",
10 "class": "Net UI Tool Window",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/page-versions/cold-restore/shots/cold-menu.png created
Binary files /dev/null and b/corpus/page-versions/cold-restore/shots/cold-menu.png differ
corpus/page-versions/cold-restore/shots/cold-shown.ahk created+6
...@@ -0,0 +1,6 @@
1WinActivate("ahk_exe ONENOTE.EXE")
2CoordMode("Mouse", "Screen")
3Click(202, 36)
4Sleep(600)
5Click(443, 80)
6Sleep(1500)
corpus/page-versions/cold-restore/shots/cold-shown.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "",
10 "class": "Net UI Tool Window",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/page-versions/cold-restore/shots/cold-shown.png created
Binary files /dev/null and b/corpus/page-versions/cold-restore/shots/cold-shown.png differ
corpus/page-versions/cold-restore/shots/cold-version.ahk created+4
...@@ -0,0 +1,4 @@
1WinActivate("ahk_exe ONENOTE.EXE")
2CoordMode("Mouse", "Screen")
3Click(710, 143)
4Sleep(1500)
corpus/page-versions/cold-restore/shots/cold-version.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Versioned - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/page-versions/cold-restore/shots/cold-version.png created
Binary files /dev/null and b/corpus/page-versions/cold-restore/shots/cold-version.png differ
corpus/page-versions/cold-restore/source.json created+14
...@@ -0,0 +1,14 @@
1[
2 {
3 "path": "History.one",
4 "bytes": 21128,
5 "sha256": "51694895069efb861d51e420ff463fbe9ce1207091a27aa7e98de0ebd6e469f7",
6 "mtime_ns": 1790567288549426487
7 },
8 {
9 "path": "Open Notebook.onetoc2",
10 "bytes": 4808,
11 "sha256": "21f33fee6a5f2946e7d9394036a37cc1bf062856449eed5a0eceaf682dde76fc",
12 "mtime_ns": 1790567288548592364
13 }
14]
corpus/page-versions/cold-restore/teardown.json created+1
...@@ -0,0 +1 @@
1{"absent": true}
corpus/page-versions/native-scripted/commands.jsonl created+12
...@@ -0,0 +1,12 @@
1{"command": "mkdir C:\\one-tests\\versions", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\versions\\setup.ps1", "exit": 0, "stdout": "", "stderr": "", "error": null}
3{"command": "powershell -NoProfile -Command \"$f = 'C:\\one-tests\\versions/Versions/History.one'; $last = ''; $still = 0; for ($i = 0; $i -lt 120 -and $still -lt 10; $i++) { Start-Sleep -Milliseconds 500; $item = Get-Item $f; $now = '' + $item.Length + $item.LastWriteTimeUtc.Ticks; if ($now -eq $last) { $still++ } else { $still = 0; $last = $now } }\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
4{"command": "powershell -NoProfile -Command \"$f = 'C:\\one-tests\\versions/Versions/History.one'; $last = ''; $still = 0; for ($i = 0; $i -lt 120 -and $still -lt 10; $i++) { Start-Sleep -Milliseconds 500; $item = Get-Item $f; $now = '' + $item.Length + $item.LastWriteTimeUtc.Ticks; if ($now -eq $last) { $still++ } else { $still = 0; $last = $now } }\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
5{"command": "powershell -NoProfile -Command \"$f = 'C:\\one-tests\\versions/Versions/History.one'; $last = ''; $still = 0; for ($i = 0; $i -lt 120 -and $still -lt 10; $i++) { Start-Sleep -Milliseconds 500; $item = Get-Item $f; $now = '' + $item.Length + $item.LastWriteTimeUtc.Ticks; if ($now -eq $last) { $still++ } else { $still = 0; $last = $now } }\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
6{"command": "powershell -NoProfile -Command \"$f = 'C:\\one-tests\\versions/Versions/History.one'; $last = ''; $still = 0; for ($i = 0; $i -lt 120 -and $still -lt 10; $i++) { Start-Sleep -Milliseconds 500; $item = Get-Item $f; $now = '' + $item.Length + $item.LastWriteTimeUtc.Ticks; if ($now -eq $last) { $still++ } else { $still = 0; $last = $now } }\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
7{"command": "powershell -NoProfile -Command \"$f = 'C:\\one-tests\\versions/Versions/History.one'; $last = ''; $still = 0; for ($i = 0; $i -lt 120 -and $still -lt 10; $i++) { Start-Sleep -Milliseconds 500; $item = Get-Item $f; $now = '' + $item.Length + $item.LastWriteTimeUtc.Ticks; if ($now -eq $last) { $still++ } else { $still = 0; $last = $now } }\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
8{"command": "powershell -NoProfile -Command \"$f = 'C:\\one-tests\\versions/Versions/History.one'; $last = ''; $still = 0; for ($i = 0; $i -lt 120 -and $still -lt 10; $i++) { Start-Sleep -Milliseconds 500; $item = Get-Item $f; $now = '' + $item.Length + $item.LastWriteTimeUtc.Ticks; if ($now -eq $last) { $still++ } else { $still = 0; $last = $now } }\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
9{"command": "powershell -NoProfile -Command \"$f = 'C:\\one-tests\\versions/Versions/History.one'; $last = ''; $still = 0; for ($i = 0; $i -lt 120 -and $still -lt 10; $i++) { Start-Sleep -Milliseconds 500; $item = Get-Item $f; $now = '' + $item.Length + $item.LastWriteTimeUtc.Ticks; if ($now -eq $last) { $still++ } else { $still = 0; $last = $now } }\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
10{"command": "powershell -NoProfile -Command \"$f = 'C:\\one-tests\\versions/Versions/History.one'; $last = ''; $still = 0; for ($i = 0; $i -lt 120 -and $still -lt 10; $i++) { Start-Sleep -Milliseconds 500; $item = Get-Item $f; $now = '' + $item.Length + $item.LastWriteTimeUtc.Ticks; if ($now -eq $last) { $still++ } else { $still = 0; $last = $now } }\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
11{"command": "powershell -NoProfile -Command \"$f = 'C:\\one-tests\\versions/Versions/History.one'; $last = ''; $still = 0; for ($i = 0; $i -lt 120 -and $still -lt 10; $i++) { Start-Sleep -Milliseconds 500; $item = Get-Item $f; $now = '' + $item.Length + $item.LastWriteTimeUtc.Ticks; if ($now -eq $last) { $still++ } else { $still = 0; $last = $now } }\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
12{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 1, "stdout": "", "stderr": "Compress-Archive : The path 'C:\\one-tests\\runs\\capture\\*' either does not \r\nexist or is not a valid file system path.\r\nAt line:1 char:1\r\n+ Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -Destinatio ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : InvalidArgument: (C:\\one-tests\\runs\\capture\\*:St \r\n ring) [Compress-Archive], InvalidOperationException\r\n + FullyQualifiedErrorId : ArchiveCmdletPathNotFound,Compress-Archive\r\n \r\n", "error": null}
corpus/page-versions/native-scripted/machine.json created+1
...@@ -0,0 +1 @@
1{"name": "m6-5ffe803a", "hostname": "ONE-M6-5FFE803A"}
corpus/page-versions/native-scripted/setup.ps1 created+11
...@@ -0,0 +1,11 @@
1$ErrorActionPreference = 'Stop'
2$app = New-Object -ComObject OneNote.Application
3$notebook = ''
4$app.OpenHierarchy('C:\one-tests\versions\Versions', '', [ref]$notebook, 1)
5$section = ''
6$app.OpenHierarchy('History.one', $notebook, [ref]$section, 3)
7$page = ''
8$app.CreateNewPage($section, [ref]$page, 0)
9$namespace = 'http://schemas.microsoft.com/office/onenote/2010/onenote'
10$app.UpdatePageContent('<one:Page xmlns:one="' + $namespace + '" ID="' + $page + '"><one:Title><one:OE><one:T>Versioned</one:T></one:OE></one:Title><one:Outline><one:Position x="36" y="86"/><one:OEChildren><one:OE><one:T>First state.</one:T></one:OE></one:OEChildren></one:Outline></one:Page>')
11$page | Out-File 'C:\one-tests\versions\page.txt' -Encoding UTF8
corpus/page-versions/native-scripted/shots/01-typed.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Versioned - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/page-versions/native-scripted/shots/01-typed.png created
Binary files /dev/null and b/corpus/page-versions/native-scripted/shots/01-typed.png differ
corpus/page-versions/native-scripted/shots/02-menu.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "",
10 "class": "Net UI Tool Window",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/page-versions/native-scripted/shots/02-menu.png created
Binary files /dev/null and b/corpus/page-versions/native-scripted/shots/02-menu.png differ
corpus/page-versions/native-scripted/shots/02-second-author.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Versioned - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/page-versions/native-scripted/shots/02-second-author.png created
Binary files /dev/null and b/corpus/page-versions/native-scripted/shots/02-second-author.png differ
corpus/page-versions/native-scripted/shots/02-shown.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "",
10 "class": "Net UI Tool Window",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/page-versions/native-scripted/shots/02-shown.png created
Binary files /dev/null and b/corpus/page-versions/native-scripted/shots/02-shown.png differ
corpus/page-versions/native-scripted/shots/02-version-menu.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "",
10 "class": "Net UI Tool Window",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/page-versions/native-scripted/shots/02-version-menu.png created
Binary files /dev/null and b/corpus/page-versions/native-scripted/shots/02-version-menu.png differ
corpus/page-versions/native-scripted/shots/02-version.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Versioned - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/page-versions/native-scripted/shots/02-version.png created
Binary files /dev/null and b/corpus/page-versions/native-scripted/shots/02-version.png differ
corpus/page-versions/native-scripted/shots/03-copied.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Versioned - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/page-versions/native-scripted/shots/03-copied.png created
Binary files /dev/null and b/corpus/page-versions/native-scripted/shots/03-copied.png differ
corpus/page-versions/native-scripted/shots/03-copy-dialog.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Move or Copy Pages",
10 "class": "NUIDialog",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/page-versions/native-scripted/shots/03-copy-dialog.png created
Binary files /dev/null and b/corpus/page-versions/native-scripted/shots/03-copy-dialog.png differ
corpus/page-versions/native-scripted/shots/04-restored.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Versioned - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/page-versions/native-scripted/shots/04-restored.png created
Binary files /dev/null and b/corpus/page-versions/native-scripted/shots/04-restored.png differ
corpus/page-versions/native-scripted/shots/04-shown.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Versioned - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/page-versions/native-scripted/shots/04-shown.png created
Binary files /dev/null and b/corpus/page-versions/native-scripted/shots/04-shown.png differ
corpus/page-versions/native-scripted/shots/05-deleted.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Versioned - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/page-versions/native-scripted/shots/05-deleted.png created
Binary files /dev/null and b/corpus/page-versions/native-scripted/shots/05-deleted.png differ
corpus/page-versions/native-scripted/shots/05-newest.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Versioned - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/page-versions/native-scripted/shots/05-newest.png created
Binary files /dev/null and b/corpus/page-versions/native-scripted/shots/05-newest.png differ
corpus/page-versions/native-scripted/shots/06-confirm.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "dialog: Microsoft OneNote | &Yes\r\n&No\r\nDo you want to delete all page versions in the section \"History\"?\r\n\n",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Microsoft OneNote",
10 "class": "#32770",
11 "dialog": true
12 }
13}
\ No newline at end of file
corpus/page-versions/native-scripted/shots/06-confirm.png created
Binary files /dev/null and b/corpus/page-versions/native-scripted/shots/06-confirm.png differ
corpus/page-versions/native-scripted/shots/06-deleted-all.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Versioned - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/page-versions/native-scripted/shots/06-deleted-all.png created
Binary files /dev/null and b/corpus/page-versions/native-scripted/shots/06-deleted-all.png differ
corpus/page-versions/native-scripted/shots/07-third-author-edit.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Versioned - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/page-versions/native-scripted/shots/07-third-author-edit.png created
Binary files /dev/null and b/corpus/page-versions/native-scripted/shots/07-third-author-edit.png differ
corpus/page-versions/native-scripted/shots/08-deleted-only.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Versioned - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/page-versions/native-scripted/shots/08-deleted-only.png created
Binary files /dev/null and b/corpus/page-versions/native-scripted/shots/08-deleted-only.png differ
corpus/page-versions/native-scripted/shots/09-authors.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Versioned - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/page-versions/native-scripted/shots/09-authors.png created
Binary files /dev/null and b/corpus/page-versions/native-scripted/shots/09-authors.png differ
corpus/page-versions/native-scripted/shots/09-find-by-author.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Versioned - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/page-versions/native-scripted/shots/09-find-by-author.png created
Binary files /dev/null and b/corpus/page-versions/native-scripted/shots/09-find-by-author.png differ
corpus/page-versions/native-scripted/shots/09-recent-edits.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "",
10 "class": "Net UI Tool Window",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/page-versions/native-scripted/shots/09-recent-edits.png created
Binary files /dev/null and b/corpus/page-versions/native-scripted/shots/09-recent-edits.png differ
corpus/page-versions/native-scripted/step-01/notebook/History.one created
Binary files /dev/null and b/corpus/page-versions/native-scripted/step-01/notebook/History.one differ
corpus/page-versions/native-scripted/step-01/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/page-versions/native-scripted/step-01/notebook/Open Notebook.onetoc2 differ
corpus/page-versions/native-scripted/step-02/notebook/History.one created
Binary files /dev/null and b/corpus/page-versions/native-scripted/step-02/notebook/History.one differ
corpus/page-versions/native-scripted/step-03/notebook/History.one created
Binary files /dev/null and b/corpus/page-versions/native-scripted/step-03/notebook/History.one differ
corpus/page-versions/native-scripted/step-04/notebook/History.one created
Binary files /dev/null and b/corpus/page-versions/native-scripted/step-04/notebook/History.one differ
corpus/page-versions/native-scripted/step-05/notebook/History.one created
Binary files /dev/null and b/corpus/page-versions/native-scripted/step-05/notebook/History.one differ
corpus/page-versions/native-scripted/step-06/notebook/History.one created
Binary files /dev/null and b/corpus/page-versions/native-scripted/step-06/notebook/History.one differ
corpus/page-versions/native-scripted/step-07/notebook/History.one created
Binary files /dev/null and b/corpus/page-versions/native-scripted/step-07/notebook/History.one differ
corpus/page-versions/native-scripted/step-08/notebook/History.one created
Binary files /dev/null and b/corpus/page-versions/native-scripted/step-08/notebook/History.one differ
corpus/page-versions/native-scripted/step-09/notebook/History.one created
Binary files /dev/null and b/corpus/page-versions/native-scripted/step-09/notebook/History.one differ
corpus/page-versions/native/shots/01-version-open.png created
Binary files /dev/null and b/corpus/page-versions/native/shots/01-version-open.png differ
corpus/page-versions/native/shots/02-version-menu.png created
Binary files /dev/null and b/corpus/page-versions/native/shots/02-version-menu.png differ
corpus/page-versions/native/shots/03-closed.png created
Binary files /dev/null and b/corpus/page-versions/native/shots/03-closed.png differ
corpus/page-versions/native/shots/04-copy-dialog.png created
Binary files /dev/null and b/corpus/page-versions/native/shots/04-copy-dialog.png differ
corpus/page-versions/native/shots/05-copied.png created
Binary files /dev/null and b/corpus/page-versions/native/shots/05-copied.png differ
corpus/page-versions/native/shots/06-shown-again.png created
Binary files /dev/null and b/corpus/page-versions/native/shots/06-shown-again.png differ
corpus/page-versions/native/shots/07-restored.png created
Binary files /dev/null and b/corpus/page-versions/native/shots/07-restored.png differ
corpus/page-versions/native/shots/08-other-version.png created
Binary files /dev/null and b/corpus/page-versions/native/shots/08-other-version.png differ
corpus/page-versions/native/shots/09-deleted.png created
Binary files /dev/null and b/corpus/page-versions/native/shots/09-deleted.png differ
corpus/page-versions/native/shots/10-delete-all-dialog.png created
Binary files /dev/null and b/corpus/page-versions/native/shots/10-delete-all-dialog.png differ
corpus/page-versions/native/shots/11-deleted-all.png created
Binary files /dev/null and b/corpus/page-versions/native/shots/11-deleted-all.png differ
corpus/page-versions/native/shots/14-authors-shown.png created
Binary files /dev/null and b/corpus/page-versions/native/shots/14-authors-shown.png differ
corpus/page-versions/native/shots/15-recent-edits.png created
Binary files /dev/null and b/corpus/page-versions/native/shots/15-recent-edits.png differ
corpus/page-versions/native/shots/16-find-by-author.png created
Binary files /dev/null and b/corpus/page-versions/native/shots/16-find-by-author.png differ
corpus/page-versions/native/shots/17-future.png created
Binary files /dev/null and b/corpus/page-versions/native/shots/17-future.png differ
corpus/page-versions/native/shots/18-options.png created
Binary files /dev/null and b/corpus/page-versions/native/shots/18-options.png differ
corpus/page-versions/native/shots/19-save-backup.png created
Binary files /dev/null and b/corpus/page-versions/native/shots/19-save-backup.png differ
corpus/page-versions/native/shots/20-optimized.png created
Binary files /dev/null and b/corpus/page-versions/native/shots/20-optimized.png differ
corpus/page-versions/native/shots/21-after-optimize.png created
Binary files /dev/null and b/corpus/page-versions/native/shots/21-after-optimize.png differ
corpus/page-versions/native/shots/22-page-menu.png created
Binary files /dev/null and b/corpus/page-versions/native/shots/22-page-menu.png differ
corpus/page-versions/native/shots/23-page-menu-shown.png created
Binary files /dev/null and b/corpus/page-versions/native/shots/23-page-menu-shown.png differ
corpus/page-versions/native/shots/24-tooltip.png created
Binary files /dev/null and b/corpus/page-versions/native/shots/24-tooltip.png differ
corpus/page-versions/native/step-01/notebook/History.one created
Binary files /dev/null and b/corpus/page-versions/native/step-01/notebook/History.one differ
corpus/page-versions/native/step-01/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/page-versions/native/step-01/notebook/Open Notebook.onetoc2 differ
corpus/page-versions/native/step-02/notebook/History.one created
Binary files /dev/null and b/corpus/page-versions/native/step-02/notebook/History.one differ
corpus/page-versions/native/step-02/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/page-versions/native/step-02/notebook/Open Notebook.onetoc2 differ
corpus/page-versions/native/step-03/notebook/History.one created
Binary files /dev/null and b/corpus/page-versions/native/step-03/notebook/History.one differ
corpus/page-versions/native/step-03/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/page-versions/native/step-03/notebook/Open Notebook.onetoc2 differ
corpus/page-versions/native/step-04/notebook/History.one created
Binary files /dev/null and b/corpus/page-versions/native/step-04/notebook/History.one differ
corpus/page-versions/native/step-04/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/page-versions/native/step-04/notebook/Open Notebook.onetoc2 differ
corpus/page-versions/native/step-05/notebook/History.one created
Binary files /dev/null and b/corpus/page-versions/native/step-05/notebook/History.one differ
corpus/page-versions/native/step-05/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/page-versions/native/step-05/notebook/Open Notebook.onetoc2 differ
corpus/page-versions/native/step-06/notebook/History.one created
Binary files /dev/null and b/corpus/page-versions/native/step-06/notebook/History.one differ
corpus/page-versions/native/step-06/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/page-versions/native/step-06/notebook/Open Notebook.onetoc2 differ
corpus/page-versions/native/step-07/notebook/History.one created
Binary files /dev/null and b/corpus/page-versions/native/step-07/notebook/History.one differ
corpus/page-versions/native/step-07/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/page-versions/native/step-07/notebook/Open Notebook.onetoc2 differ
corpus/page-versions/native/step-08/notebook/History.one created
Binary files /dev/null and b/corpus/page-versions/native/step-08/notebook/History.one differ
corpus/page-versions/native/step-08/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/page-versions/native/step-08/notebook/Open Notebook.onetoc2 differ
corpus/page-versions/native/step-09/notebook/History.one created
Binary files /dev/null and b/corpus/page-versions/native/step-09/notebook/History.one differ
corpus/page-versions/native/step-09/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/page-versions/native/step-09/notebook/Open Notebook.onetoc2 differ
corpus/page-versions/native/step-10/notebook/History.one created
Binary files /dev/null and b/corpus/page-versions/native/step-10/notebook/History.one differ
corpus/page-versions/native/step-10/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/page-versions/native/step-10/notebook/Open Notebook.onetoc2 differ
corpus/recycle-bin-repair/README.md created+19
...@@ -0,0 +1,19 @@
1# Recycle bin repair
2
3`candidate` is the notebook
4`a_page_delete_lists_the_recycle_bin_it_finds_or_makes_and_leaves_an_unreadable_one` in
5`crates/notebook/tests/structure.rs` exports through `NOTEBOOK_RECYCLE_EXPORT`: a page
6deleted into a new recycle bin whose name the root TOC still listed for a bin gone from the
7folder (the stale entry goes, the new bin is listed), then deleted again into a bin whose
8TOC OneNote might name `OneNote Table Of Contents.onetoc2` and which did not list
9`OneNote_DeletedPages.one` (the bin's TOC is kept, the deleted pages placed under it and
10listed).
11
12`cold` is its fresh OneNote 2010 read: the recycle bin (`isRecycleBin`) holds "Deleted
13Pages" (`isDeletedPages`) with both deleted pages, and OneNote left every file byte for
14byte as written, so it repaired nothing. `tools/test_recycle_bin_repair.py` checks this
15without a VM.
16
17Regenerate with `NOTEBOOK_RECYCLE_EXPORT` set to a new directory while running the test,
18then `tools/native_runner.py OUTPUT COLD --expected-pages -1 --collect-notebook
19--screenshots`.
corpus/recycle-bin-repair/candidate/New Section 1.one created
Binary files /dev/null and b/corpus/recycle-bin-repair/candidate/New Section 1.one differ
corpus/recycle-bin-repair/candidate/OneNote_RecycleBin/OneNote Table Of Contents.onetoc2 created
Binary files /dev/null and b/corpus/recycle-bin-repair/candidate/OneNote_RecycleBin/OneNote Table Of Contents.onetoc2 differ
corpus/recycle-bin-repair/candidate/OneNote_RecycleBin/OneNote_DeletedPages.one created
Binary files /dev/null and b/corpus/recycle-bin-repair/candidate/OneNote_RecycleBin/OneNote_DeletedPages.one differ
corpus/recycle-bin-repair/candidate/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/recycle-bin-repair/candidate/Open Notebook.onetoc2 differ
corpus/recycle-bin-repair/cold/commands.jsonl created+3
...@@ -0,0 +1,3 @@
1{"command": "powershell -NoProfile -Command \"Expand-Archive -LiteralPath C:\\one-tests\\transfer.zip -DestinationPath C:\\one-tests\\runs\\capture\\notebook\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
2{"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-18ABF18A -ExpectedPages -1 -KeepOpen", "exit": 0, "stdout": "Read 2 sections and 3 pages.\r\n", "stderr": "", "error": null}
3{"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null}
corpus/recycle-bin-repair/cold/machine.json created+1
...@@ -0,0 +1 @@
1{"name": "m6-18abf18a", "hostname": "ONE-M6-18ABF18A"}
corpus/recycle-bin-repair/cold/notebook/New Section 1.one created
Binary files /dev/null and b/corpus/recycle-bin-repair/cold/notebook/New Section 1.one differ
corpus/recycle-bin-repair/cold/notebook/OneNote_RecycleBin/OneNote Table Of Contents.onetoc2 created
Binary files /dev/null and b/corpus/recycle-bin-repair/cold/notebook/OneNote_RecycleBin/OneNote Table Of Contents.onetoc2 differ
corpus/recycle-bin-repair/cold/notebook/OneNote_RecycleBin/OneNote_DeletedPages.one created
Binary files /dev/null and b/corpus/recycle-bin-repair/cold/notebook/OneNote_RecycleBin/OneNote_DeletedPages.one differ
corpus/recycle-bin-repair/cold/notebook/Open Notebook.onetoc2 created
Binary files /dev/null and b/corpus/recycle-bin-repair/cold/notebook/Open Notebook.onetoc2 differ
corpus/recycle-bin-repair/cold/read/environment.json created+7
...@@ -0,0 +1,7 @@
1{
2 "powershell": "5.1.14409.1005",
3 "schema": "xs2010",
4 "hostname": "ONE-M6-18ABF18A",
5 "cold": true,
6 "onenote": "14.0.4763.1000"
7}
corpus/recycle-bin-repair/cold/read/hierarchy.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{4A7F185F-1A48-43E7-BD06-5CC5B53BDBAF}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-28T08:06:34.000Z" color="#FFD700"><one:Section name="New Section 1" ID="{73ACD765-279A-0D4E-0BCC-965A2F853DE6}{1}{B0}" path="C:\one-tests\runs\capture\notebook\New Section 1.one" lastModifiedTime="2026-09-28T08:06:34.000Z" color="#8AA8E4"><one:Page ID="{1906471E-0D0A-0E51-0A33-6158092860A4}{1}{B0}" name="Kept" dateTime="2026-09-28T08:04:50.000Z" lastModifiedTime="2026-09-28T08:06:34.000Z" pageLevel="1"/></one:Section><one:SectionGroup name="OneNote_RecycleBin" ID="{E3457349-0965-4F8B-A86B-E5ED5EFA53F0}{1}{B0}" path="C:\one-tests\runs\capture\notebook\OneNote_RecycleBin" lastModifiedTime="2026-09-28T08:06:34.000Z" isRecycleBin="true"><one:Section name="Deleted Pages" ID="{CA06A3F3-B5E8-03CC-20A3-828848B29E61}{1}{B0}" path="C:\one-tests\runs\capture\notebook\OneNote_RecycleBin\OneNote_DeletedPages.one" lastModifiedTime="2026-09-28T08:06:34.000Z" color="#E1E1E1" isInRecycleBin="true" isDeletedPages="true"><one:Page ID="{AFFE36A9-90C2-0C90-25EC-A4F7606E5EB1}{1}{B0}" name="Kept" dateTime="2026-09-28T08:04:50.000Z" lastModifiedTime="2026-09-28T08:06:34.000Z" pageLevel="1" isInRecycleBin="true"/><one:Page ID="{C9906D9C-6FE2-085A-2DB6-0405237AB185}{1}{B0}" name="Kept" dateTime="2026-09-28T08:04:50.000Z" lastModifiedTime="2026-09-28T08:06:34.000Z" pageLevel="1" isInRecycleBin="true"/></one:Section></one:SectionGroup></one:Notebook>
corpus/recycle-bin-repair/cold/read/page-000.navigation.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Kept - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/recycle-bin-repair/cold/read/page-000.png created
Binary files /dev/null and b/corpus/recycle-bin-repair/cold/read/page-000.png differ
corpus/recycle-bin-repair/cold/read/page-000.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{1906471E-0D0A-0E51-0A33-6158092860A4}{1}{B0}" name="Kept" dateTime="2026-09-28T08:04:50.000Z" lastModifiedTime="2026-09-28T08:06:34.000Z" pageLevel="1"><one:QuickStyleDef index="0" name="PageTitle" font="Calibri" fontSize="17.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Title showDate="false" showTime="false" lang="en-US"><one:OE author="Author" authorInitials="A" lastModifiedBy="Author" lastModifiedByInitials="A" creationTime="2026-09-28T08:04:50.000Z" lastModifiedTime="2026-09-28T08:04:50.000Z" objectID="{1906471E-0D0A-0E51-0A33-6158092860A4}{15}{B0}" alignment="left" quickStyleIndex="0" style="font-family:Calibri;font-size:17.0pt"><one:T><![CDATA[Kept]]></one:T></one:OE></one:Title></one:Page>
corpus/recycle-bin-repair/cold/read/page-001.navigation.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Kept (Read-Only - Recycle Bin) - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/recycle-bin-repair/cold/read/page-001.png created
Binary files /dev/null and b/corpus/recycle-bin-repair/cold/read/page-001.png differ
corpus/recycle-bin-repair/cold/read/page-001.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{AFFE36A9-90C2-0C90-25EC-A4F7606E5EB1}{1}{B0}" name="Kept" dateTime="2026-09-28T08:04:50.000Z" lastModifiedTime="2026-09-28T08:06:34.000Z" pageLevel="1" isInRecycleBin="true"><one:QuickStyleDef index="0" name="PageTitle" font="Calibri" fontSize="17.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Title showDate="false" showTime="false" lang="en-US"><one:OE author="Author" authorInitials="A" lastModifiedBy="Author" lastModifiedByInitials="A" creationTime="2026-09-28T08:04:50.000Z" lastModifiedTime="2026-09-28T08:04:50.000Z" objectID="{AFFE36A9-90C2-0C90-25EC-A4F7606E5EB1}{15}{B0}" alignment="left" quickStyleIndex="0" style="font-family:Calibri;font-size:17.0pt"><one:T><![CDATA[Kept]]></one:T></one:OE></one:Title></one:Page>
corpus/recycle-bin-repair/cold/read/page-002.navigation.json created+13
...@@ -0,0 +1,13 @@
1{
2 "exit": 0,
3 "stdout": "",
4 "stderr": "",
5 "w": 800,
6 "h": 600,
7 "error": null,
8 "win": {
9 "title": "Kept (Read-Only - Recycle Bin) - Microsoft OneNote",
10 "class": "Framework::CFrame",
11 "dialog": false
12 }
13}
\ No newline at end of file
corpus/recycle-bin-repair/cold/read/page-002.png created
Binary files /dev/null and b/corpus/recycle-bin-repair/cold/read/page-002.png differ
corpus/recycle-bin-repair/cold/read/page-002.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{C9906D9C-6FE2-085A-2DB6-0405237AB185}{1}{B0}" name="Kept" dateTime="2026-09-28T08:04:50.000Z" lastModifiedTime="2026-09-28T08:06:34.000Z" pageLevel="1" isInRecycleBin="true"><one:QuickStyleDef index="0" name="PageTitle" font="Calibri" fontSize="17.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Title showDate="false" showTime="false" lang="en-US"><one:OE author="Author" authorInitials="A" lastModifiedBy="Author" lastModifiedByInitials="A" creationTime="2026-09-28T08:04:50.000Z" lastModifiedTime="2026-09-28T08:04:50.000Z" objectID="{C9906D9C-6FE2-085A-2DB6-0405237AB185}{15}{B0}" alignment="left" quickStyleIndex="0" style="font-family:Calibri;font-size:17.0pt"><one:T><![CDATA[Kept]]></one:T></one:OE></one:Title></one:Page>
corpus/recycle-bin-repair/cold/read/payloads.json created+3
...@@ -0,0 +1,3 @@
1[
2
3]
\ No newline at end of file
corpus/recycle-bin-repair/cold/run.json created+18
...@@ -0,0 +1,18 @@
1{
2 "notebook": "/private/tmp/recycle-gate/candidate",
3 "expected_pages": -1,
4 "author": null,
5 "author_timeout_seconds": 600,
6 "inspect": false,
7 "collect_notebook": true,
8 "base": {
9 "file": "win7-office-base.qcow2",
10 "format": "qcow2",
11 "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346",
12 "virtual_size": 68719476736
13 },
14 "scripts": {
15 "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331",
16 "read.ps1": "04013bfcccee40a17e2a225f9b9e96f40a3a8daad9e350609654f8eda3ccbb41"
17 }
18}
corpus/recycle-bin-repair/cold/scripts/cold.ps1 created+27
...@@ -0,0 +1,27 @@
1param([Parameter(Mandatory=$true)][string]$Root, [string]$CloneHost = '')
2Set-StrictMode -Version Latest
3$ErrorActionPreference = 'Stop'
4$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
5if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
6 throw 'Choose a run directly below C:\one-tests\runs.'
7}
8if (Get-Process ONENOTE -ErrorAction SilentlyContinue) { throw 'Close OneNote before resetting its test cache.' }
9$key = 'HKCU:\Software\Microsoft\Office\14.0\OneNote'
10if ($CloneHost) {
11 if ($CloneHost -notmatch '^ONE-[A-Z0-9-]+$' -or [Environment]::MachineName -ne $CloneHost) {
12 throw 'The disposable clone hostname does not match this machine.'
13 }
14 New-Item "$key\Options\Paths" -Force | Out-Null
15 New-ItemProperty "$key\Options\Paths" -Name UnfiledNotesSection -PropertyType ExpandString -Value 'C:\one-tests\Loose.one' -Force | Out-Null
16} elseif ((Get-ItemProperty "$key\Options\Paths").UnfiledNotesSection -ne 'C:\one-tests\Loose.one' -or
17 -not (Test-Path 'C:\one-tests\profile-original-cache')) {
18 throw 'Park the personal OneNote profile before resetting the test cache.'
19}
20$cache = Join-Path $env:LOCALAPPDATA 'Microsoft\OneNote\14.0'
21$parked = Join-Path 'C:\one-tests\caches' ([IO.Path]::GetFileName($root))
22if (Test-Path $parked) { throw 'Choose a new run; its parked cache already exists.' }
23New-Item -ItemType Directory -Path 'C:\one-tests\caches' -Force | Out-Null
24if (Test-Path $cache) { Move-Item -LiteralPath $cache -Destination $parked }
25if (Test-Path "$key\OpenNotebooks") { Remove-Item "$key\OpenNotebooks" -Recurse }
26New-Item "$key\OpenNotebooks" | Out-Null
27New-ItemProperty "$key\OpenNotebooks" -Name '1' -PropertyType String -Value "$root\notebook" | Out-Null
corpus/recycle-bin-repair/cold/scripts/read.ps1 created+142
...@@ -0,0 +1,142 @@
1param(
2 [Parameter(Mandatory=$true)][string]$Root,
3 [int]$ExpectedPages = -1,
4 [switch]$UseCurrentCache,
5 [switch]$Pdf,
6 [switch]$KeepOpen,
7 [string]$CloneHost = ''
8)
9Set-StrictMode -Version Latest
10$ErrorActionPreference = 'Stop'
11$root = [IO.Path]::GetFullPath($Root).TrimEnd('\')
12if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') {
13 throw 'Choose a run directly below C:\one-tests\runs.'
14}
15$notebook = Join-Path $root 'notebook'
16$output = Join-Path $root 'read'
17if (Test-Path $output) { throw 'Choose a new read destination.' }
18if ($UseCurrentCache) {
19 if ((Get-ItemProperty 'HKCU:\Software\Microsoft\Office\14.0\OneNote\Options\Paths').UnfiledNotesSection -ne 'C:\one-tests\Loose.one') {
20 throw 'Park the personal OneNote profile before reading test notebooks.'
21 }
22} else {
23 & "$PSScriptRoot\cold-current.ps1" -Root $root -CloneHost $CloneHost
24}
25New-Item -ItemType Directory -Path $output | Out-Null
26$app = New-Object -ComObject OneNote.Application
27$notebookId = ''
28$failure = $null
29try {
30 $app.OpenHierarchy($notebook, '', [ref]$notebookId, 0)
31 $process = Get-Process ONENOTE
32 @{ hostname = [Environment]::MachineName; onenote = $process.MainModule.FileVersionInfo.FileVersion;
33 powershell = $PSVersionTable.PSVersion.ToString(); schema = 'xs2010'; cold = (-not $UseCurrentCache.IsPresent) } |
34 ConvertTo-Json | Set-Content (Join-Path $output 'environment.json') -Encoding UTF8
35 $sections = @()
36 foreach ($file in @(Get-ChildItem $notebook -Recurse | Where-Object { $_.Extension -eq '.one' })) {
37 $id = ''
38 $app.OpenHierarchy($file.FullName, '', [ref]$id, 0)
39 $sections += $id
40 }
41 $deadline = [DateTime]::UtcNow.AddSeconds(300)
42 $previous = ''
43 $lastChange = ''
44 $stableSince = [DateTime]::UtcNow
45 $settled = $false
46 do {
47 $pages = @{}
48 foreach ($section in $sections) {
49 $hierarchy = ''
50 $app.GetHierarchy($section, 4, [ref]$hierarchy, 1)
51 [xml]$xml = $hierarchy
52 foreach ($node in $xml.SelectNodes('//*[@path]')) {
53 if (-not $node.GetAttribute('path').StartsWith("$notebook\", [StringComparison]::OrdinalIgnoreCase)) {
54 throw 'OneNote opened a section outside the copied notebook.'
55 }
56 }
57 foreach ($node in $xml.SelectNodes('//*[local-name()="Page"]')) {
58 $id = $node.GetAttribute('ID')
59 $content = ''
60 $app.GetPageContent($id, [ref]$content, 1, 1)
61 $pages[$id] = $content
62 }
63 }
64 $signature = [String]::Join('|', @($pages.Keys | Sort-Object | ForEach-Object { $_ + $pages[$_] }))
65 if ($signature -ne $previous) {
66 $lastChange = $previous
67 $previous = $signature
68 $stableSince = [DateTime]::UtcNow
69 }
70 if ((($ExpectedPages -ge 0 -and $pages.Count -eq $ExpectedPages) -or
71 ($ExpectedPages -lt 0 -and $pages.Count -gt 0)) -and
72 ([DateTime]::UtcNow - $stableSince).TotalSeconds -ge 2) { $settled = $true; break }
73 Start-Sleep -Milliseconds 250
74 } while ([DateTime]::UtcNow -lt $deadline)
75 if (-not $settled -or ($ExpectedPages -ge 0 -and $pages.Count -ne $ExpectedPages) -or ($ExpectedPages -lt 0 -and $pages.Count -eq 0)) {
76 [IO.File]::WriteAllText((Join-Path $output 'previous-signature.txt'), $lastChange, [Text.Encoding]::UTF8)
77 $index = 0
78 foreach ($id in @($pages.Keys | Sort-Object)) {
79 [IO.File]::WriteAllText((Join-Path $output ('unsettled-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
80 $index++
81 }
82 $hierarchy = ''
83 $app.GetHierarchy($notebookId, 4, [ref]$hierarchy, 1)
84 [IO.File]::WriteAllText((Join-Path $output 'unsettled-hierarchy.xml'), $hierarchy, [Text.Encoding]::UTF8)
85 throw "Expected $ExpectedPages stable pages; OneNote returned $($pages.Count), settled=$settled."
86 }
87 $index = 0
88 $payloads = @()
89 foreach ($id in @($pages.Keys | Sort-Object)) {
90 [IO.File]::WriteAllText((Join-Path $output ('page-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8)
91 if ($Pdf) {
92 $pdfPath = Join-Path $output ('page-{0:d3}.pdf' -f $index)
93 $app.NavigateTo($id, '', $false)
94 $app.Publish($id, $pdfPath, 3, '')
95 if (-not (Test-Path $pdfPath) -or (Get-Item $pdfPath).Length -eq 0) { throw 'OneNote did not publish the page PDF.' }
96 }
97 [xml]$page = $pages[$id]
98 foreach ($file in $page.SelectNodes('//*[local-name()="InsertedFile" or local-name()="MediaFile"]')) {
99 $bytes = [IO.File]::ReadAllBytes($file.GetAttribute('pathCache'))
100 $hash = [BitConverter]::ToString([Security.Cryptography.SHA256]::Create().ComputeHash($bytes)).Replace('-', '').ToLowerInvariant()
101 [IO.File]::WriteAllBytes((Join-Path $output ($hash + '.attachment')), $bytes)
102 $payloads += @{ page = $id; object = $file.ParentNode.GetAttribute('objectID');
103 kind = $file.LocalName; name = $file.GetAttribute('preferredName');
104 sha256 = $hash; bytes = $bytes.Length }
105 }
106 $index++
107 }
108 [IO.File]::WriteAllText((Join-Path $output 'payloads.json'), (ConvertTo-Json -InputObject $payloads -Depth 4), [Text.Encoding]::UTF8)
109 $all = ''
110 $app.GetHierarchy($notebookId, 4, [ref]$all, 1)
111 [xml]$finalTree = $all
112 $finalIds = @($finalTree.SelectNodes('//*[local-name()="Page"]') | ForEach-Object { $_.GetAttribute('ID') } | Sort-Object -Unique)
113 if ($finalIds.Count -ne $pages.Count -or @($finalIds | Where-Object { -not $pages.ContainsKey($_) }).Count -ne 0) {
114 throw 'The notebook hierarchy changed while collecting page evidence; repeat the cold read.'
115 }
116 [IO.File]::WriteAllText((Join-Path $output 'hierarchy.xml'), $all, [Text.Encoding]::UTF8)
117 Write-Output "Read $($sections.Count) sections and $($pages.Count) pages."
118} catch {
119 $failure = $_
120 [IO.File]::WriteAllText((Join-Path $output 'failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
121 throw
122} finally {
123 try {
124 try {
125 if ($notebookId -and $CloneHost) { $app.SyncHierarchy($notebookId) }
126 if ($notebookId -and -not $KeepOpen -and (-not $UseCurrentCache -or $CloneHost)) {
127 $app.CloseNotebook($notebookId, $false)
128 }
129 } catch {
130 if ($null -eq $failure) { throw }
131 [IO.File]::WriteAllText((Join-Path $output 'cleanup-failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8)
132 }
133 } finally {
134 [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app)
135 }
136 $app = $null
137 [GC]::Collect()
138 [GC]::WaitForPendingFinalizers()
139 if (-not $UseCurrentCache -and -not $CloneHost) {
140 Get-Process ONENOTE -ErrorAction SilentlyContinue | Wait-Process -Timeout 10
141 }
142}
corpus/recycle-bin-repair/cold/source.json created+26
...@@ -0,0 +1,26 @@
1[
2 {
3 "path": "New Section 1.one",
4 "bytes": 6024,
5 "sha256": "b1444e361cef2e5f0d716ae021ad42b051129ec9ac32c36b54f4d4c7ddfc3f84",
6 "mtime_ns": 1790582690747351251
7 },
8 {
9 "path": "OneNote_RecycleBin/OneNote Table Of Contents.onetoc2",
10 "bytes": 3848,
11 "sha256": "b0e0f0928773e1fe41fac3e0377162201083a86e789a193ba0977a03cf043808",
12 "mtime_ns": 1790582691261467141
13 },
14 {
15 "path": "OneNote_RecycleBin/OneNote_DeletedPages.one",
16 "bytes": 8352,
17 "sha256": "05b75f0780567db72cc70dc8277fe8a947d6edc5f6bb51bea76f488669d6cdd0",
18 "mtime_ns": 1790582691300920689
19 },
20 {
21 "path": "Open Notebook.onetoc2",
22 "bytes": 5456,
23 "sha256": "f282959ea3de40d2460fe941ade4b7ab14f22e569530056bf7690f041b0118c6",
24 "mtime_ns": 1790582691234269352
25 }
26]
corpus/recycle-bin-repair/cold/teardown.json created+1
...@@ -0,0 +1 @@
1{"absent": true}
crates/notebook/README.md+10-2
...@@ -226,7 +226,10 @@ Reserved `_onefiles` directories are excluded from section-group traversal;...@@ -226,7 +226,10 @@ Reserved `_onefiles` directories are excluded from section-group traversal;
226OneNote's `OneNote_RecycleBin` is listed as the section group OneNote shows.226OneNote's `OneNote_RecycleBin` is listed as the section group OneNote shows.
227Encrypted sections and valid storage with an unreadable document graph retain227Encrypted sections and valid storage with an unreadable document graph retain
228their identity as `Locked` or `Unreadable`, without being presented as empty pages.228their identity as `Locked` or `Unreadable`, without being presented as empty pages.
229Malformed storage, failed reads and ambiguous identities reject the discovery.229A child folder or section file that is denied or gone while listing (a folder
230another client holds delete-pending reads as denied) is kept as `unavailable` and
231tried again on the next discovery; the root itself, malformed storage, other failed
232reads and ambiguous identities reject the discovery.
230233
231Each file read must be a consistent, bounded snapshot. The result is an234Each file read must be a consistent, bounded snapshot. The result is an
232observation across multiple files, not an atomic notebook transaction or235observation across multiple files, not an atomic notebook transaction or
...@@ -252,7 +255,12 @@ section moves into `OneNote_RecycleBin`, a group with its own TOC, and a deleted...@@ -252,7 +255,12 @@ section moves into `OneNote_RecycleBin`, a group with its own TOC, and a deleted
252sections move there too before its folders go. `recycle_pages` keeps copies of pages a255sections move there too before its folders go. `recycle_pages` keeps copies of pages a
253section is about to delete in `OneNote_RecycleBin/OneNote_DeletedPages.one`, with their256section is about to delete in `OneNote_RecycleBin/OneNote_DeletedPages.one`, with their
254identities, titles, dates and creation times, as OneNote does257identities, titles, dates and creation times, as OneNote does
255(`corpus/notebook-management`). Every created, renamed or moved file is placed with258(`corpus/notebook-management`). A bin or deleted-pages file its TOC does not list is
259placed and listed again, a bin whose TOC OneNote named otherwise keeps it, and an
260unavailable bin refuses the delete (`corpus/recycle-bin-repair`). An entry a TOC still
261lists for a file gone from its folder gives way to the file an edit gives its name.
262Edits keep entries in the order their ordering numbers give; a rename or colour keeps
263the numbers. Every created, renamed or moved file is placed with
256`onestore::place_file`, which sets the header's ancestor to the parent TOC's264`onestore::place_file`, which sets the header's ancestor to the parent TOC's
257identity and the name CRC OneNote checks on open; a file without them is265identity and the name CRC OneNote checks on open; a file without them is
258re-identified and listed anew. They run over `Storage`: a mounted directory266re-identified and listed anew. They run over `Storage`: a mounted directory
crates/notebook/examples/migrate_probe.rs+1-2
...@@ -95,8 +95,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -95,8 +95,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
95 // The remote the queue was made on, as a recovery archive records it.95 // The remote the queue was made on, as a recovery archive records it.
96 let archive = copy.with_extension("probe-recovery");96 let archive = copy.with_extension("probe-recovery");
97 replica.export_recovery(&archive)?;97 replica.export_recovery(&archive)?;
98 let mut remote =98 let mut remote = Memory(notebook::Recovery::open(&archive)?.remote_snapshot()?);
99 Memory(notebook::Recovery::open(&archive)?.remote_snapshot()?);
100 std::fs::remove_file(&archive)?;99 std::fs::remove_file(&archive)?;
101 let synced = replica.sync_once(&mut remote)?;100 let synced = replica.sync_once(&mut remote)?;
102 let published = {101 let published = {
crates/notebook/examples/queue_scale.rs+12-2
...@@ -36,9 +36,19 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -36,9 +36,19 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
36 let arena = Arena::default();36 let arena = Arena::default();
37 let mut section = Section::open(&arena, source)?;37 let mut section = Section::open(&arena, source)?;
38 let ops = vec![Op::Section(SectionOp::Create(creation))];38 let ops = vec![Op::Section(SectionOp::Create(creation))];
39 section.apply("Fixture", &Edit { at: 133_000_000_000_000_000, ops })?;39 section.apply(
40 "Fixture",
41 &Edit {
42 at: 133_000_000_000_000_000,
43 ops,
44 },
45 )?;
40 section.seal()?;46 section.seal()?;
41 let pages: Vec<ExGuid> = section.pages()?.into_iter().map(|(space, ..)| space).collect();47 let pages: Vec<ExGuid> = section
48 .pages()?
49 .into_iter()
50 .map(|(space, ..)| space)
51 .collect();
42 (section.image(), pages)52 (section.image(), pages)
43 };53 };
44 assert_eq!(pages.len(), 2);54 assert_eq!(pages.len(), 2);
crates/notebook/examples/smb_offline_client.rs+4-5
...@@ -404,10 +404,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -404,10 +404,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
404 let at = u32::try_from(text.encode_utf16().count())?;404 let at = u32::try_from(text.encode_utf16().count())?;
405 let token = format!(" [{}:{}]", args[2], generated);405 let token = format!(" [{}:{}]", args[2], generated);
406 let started = now();406 let started = now();
407 match cache.apply(407 match cache.apply("Offline document writer", append(space, object, at, &token)) {
408 "Offline document writer",
409 append(space, object, at, &token),
410 ) {
411 Ok(id) => {408 Ok(id) => {
412 println!(409 println!(
413 "{}",410 "{}",
...@@ -432,7 +429,9 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -432,7 +429,9 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
432 .collect();429 .collect();
433 let mut at = u32::try_from(text.encode_utf16().count())?;430 let mut at = u32::try_from(text.encode_utf16().count())?;
434 for token in &missing {431 for token in &missing {
435 ids.push(cache.apply("Offline document writer", append(space, object, at, token))?);432 ids.push(
433 cache.apply("Offline document writer", append(space, object, at, token))?,
434 );
436 at += u32::try_from(token.encode_utf16().count())?;435 at += u32::try_from(token.encode_utf16().count())?;
437 }436 }
438 if !missing.is_empty() {437 if !missing.is_empty() {
crates/notebook/examples/support/view.rs+11-2
...@@ -76,7 +76,14 @@ fn in_page(page: &onestore::page::Page) -> Option<(ExGuid, String)> {...@@ -76,7 +76,14 @@ fn in_page(page: &onestore::page::Page) -> Option<(ExGuid, String)> {
76 return Some((text.id, text.text.text().to_owned()));76 return Some((text.id, text.text.text().to_owned()));
77 }77 }
78 if let onestore::page::ParagraphContent::Table(table) = &paragraph.content {78 if let onestore::page::ParagraphContent::Table(table) = &paragraph.content {
79 pending.extend(table.rows.iter().flat_map(|row| &row.cells).flat_map(|cell| &cell.paragraphs).rev());79 pending.extend(
80 table
81 .rows
82 .iter()
83 .flat_map(|row| &row.cells)
84 .flat_map(|cell| &cell.paragraphs)
85 .rev(),
86 );
80 }87 }
81 }88 }
82 None89 None
...@@ -84,7 +91,9 @@ fn in_page(page: &onestore::page::Page) -> Option<(ExGuid, String)> {...@@ -84,7 +91,9 @@ fn in_page(page: &onestore::page::Page) -> Option<(ExGuid, String)> {
8491
85/// The concurrent-edit paragraph as the queued edits leave it: its page, text object and92/// The concurrent-edit paragraph as the queued edits leave it: its page, text object and
86/// text.93/// text.
87pub fn cached(cache: &notebook::Replica) -> Result<(ExGuid, ExGuid, String), Box<dyn std::error::Error>> {94pub fn cached(
95 cache: &notebook::Replica,
96) -> Result<(ExGuid, ExGuid, String), Box<dyn std::error::Error>> {
88 let mut found = Vec::new();97 let mut found = Vec::new();
89 for (space, ..) in cache.pages()? {98 for (space, ..) in cache.pages()? {
90 if let Some((object, text)) = in_page(&cache.page(space)?) {99 if let Some((object, text)) = in_page(&cache.page(space)?) {
crates/notebook/src/discover.rs+49-14
...@@ -41,6 +41,16 @@ pub struct Folder {...@@ -41,6 +41,16 @@ pub struct Folder {
41 pub toc: Option<Toc>,41 pub toc: Option<Toc>,
42 pub sections: Vec<Section>,42 pub sections: Vec<Section>,
43 pub groups: Vec<Folder>,43 pub groups: Vec<Folder>,
44 /// Child section files and groups that could not be read.
45 pub unavailable: Vec<Unavailable>,
46}
47
48/// A section file or group folder denied or gone while listing; each discovery tries it again.
49#[derive(Debug, Serialize)]
50pub struct Unavailable {
51 pub path: String,
52 pub group: bool,
53 pub error: String,
44}54}
4555
46#[derive(Debug, Serialize)]56#[derive(Debug, Serialize)]
...@@ -196,6 +206,7 @@ fn scan(...@@ -196,6 +206,7 @@ fn scan(
196 toc: None,206 toc: None,
197 sections: Vec::new(),207 sections: Vec::new(),
198 groups: Vec::new(),208 groups: Vec::new(),
209 unavailable: Vec::new(),
199 };210 };
200 for entry in &listing {211 for entry in &listing {
201 let child = join(path, &entry.name);212 let child = join(path, &entry.name);
...@@ -203,14 +214,18 @@ fn scan(...@@ -203,14 +214,18 @@ fn scan(
203 if entry.name.to_ascii_lowercase().ends_with("_onefiles") {214 if entry.name.to_ascii_lowercase().ends_with("_onefiles") {
204 continue;215 continue;
205 }216 }
206 result.groups.push(scan(217 // A group whose own listing or TOC cannot be read is unavailable, not fatal.
207 source,218 match scan(source, &child, limits, depth + 1, remaining, identities) {
208 &child,219 Ok(group) => result.groups.push(group),
209 limits,220 Err(error @ Error::Io { .. }) if unavailable(&error) => {
210 depth + 1,221 result.unavailable.push(Unavailable {
211 remaining,222 path: child,
212 identities,223 group: true,
213 )?);224 error: error.to_string(),
225 });
226 }
227 Err(error) => return Err(error),
228 }
214 continue;229 continue;
215 }230 }
216 let lower = entry.name.to_ascii_lowercase();231 let lower = entry.name.to_ascii_lowercase();
...@@ -226,12 +241,24 @@ fn scan(...@@ -226,12 +241,24 @@ fn scan(
226 {241 {
227 return Err(Error::Entry { path: child });242 return Err(Error::Entry { path: child });
228 }243 }
229 let bytes = source244 let bytes = match source.read(&child, limits.bytes_per_file) {
230 .read(&child, limits.bytes_per_file)245 Ok(bytes) => bytes,
231 .map_err(|error| Error::Io {246 Err(error) => {
232 path: child.clone(),247 let error = Error::Io {
233 error,248 path: child.clone(),
234 })?;249 error,
250 };
251 if expected == FileType::Section && unavailable(&error) {
252 result.unavailable.push(Unavailable {
253 path: child,
254 group: false,
255 error: error.to_string(),
256 });
257 continue;
258 }
259 return Err(error);
260 }
261 };
235 if bytes.len() > limits.bytes_per_file {262 if bytes.len() > limits.bytes_per_file {
236 return Err(Error::Limit { path: child });263 return Err(Error::Limit { path: child });
237 }264 }
...@@ -404,6 +431,14 @@ fn scan(...@@ -404,6 +431,14 @@ fn scan(
404 Ok(result)431 Ok(result)
405}432}
406433
434/// Access denied or a file gone mid-listing; a lost connection stays fatal.
435fn unavailable(error: &Error) -> bool {
436 matches!(error, Error::Io { error, .. } if matches!(
437 error.kind(),
438 io::ErrorKind::PermissionDenied | io::ErrorKind::NotFound
439 ))
440}
441
407fn component(name: &str) -> bool {442fn component(name: &str) -> bool {
408 !name.is_empty() && name != "." && name != ".." && !name.contains(['/', '\\', '\0'])443 !name.is_empty() && name != "." && name != ".." && !name.contains(['/', '\\', '\0'])
409}444}
crates/notebook/src/lib.rs+14
...@@ -243,6 +243,20 @@ impl Replica {...@@ -243,6 +243,20 @@ impl Replica {
243 self.ask(|reply| working::Request::Conflicts { reply })243 self.ask(|reply| working::Request::Conflicts { reply })
244 }244 }
245245
246 /// The versions of each page that has them (`onestore::Section::versions`).
247 pub fn versions(&self) -> Result<Vec<(ExGuid, Vec<onestore::PageVersion>)>> {
248 self.ask(|reply| working::Request::Versions { reply })
249 }
250
251 /// A page as one of its versions holds it; O(section).
252 pub fn version(&self, space: ExGuid, version: ExGuid) -> Result<Page> {
253 self.ask(|reply| working::Request::Version {
254 space,
255 version,
256 reply,
257 })
258 }
259
246 /// The section image the queued edits leave, the unsealed ones sealed as one more260 /// The section image the queued edits leave, the unsealed ones sealed as one more
247 /// revision whose identities differ per call: O(section).261 /// revision whose identities differ per call: O(section).
248 pub(crate) fn snapshot(&self) -> Result<Vec<u8>> {262 pub(crate) fn snapshot(&self) -> Result<Vec<u8>> {
crates/notebook/src/merge.rs+54-12
...@@ -156,7 +156,9 @@ fn named(op: &PageOp) -> Vec<ExGuid> {...@@ -156,7 +156,9 @@ fn named(op: &PageOp) -> Vec<ExGuid> {
156 PageOp::Color(_) => Vec::new(),156 PageOp::Color(_) => Vec::new(),
157 PageOp::Join { left, right } => vec![*left, *right],157 PageOp::Join { left, right } => vec![*left, *right],
158 PageOp::Insert { container, .. } => vec![*container],158 PageOp::Insert { container, .. } => vec![*container],
159 PageOp::Move { object, .. } | PageOp::Delete { object } | PageOp::Outline { object, .. } => {159 PageOp::Move { object, .. }
160 | PageOp::Delete { object }
161 | PageOp::Outline { object, .. } => {
160 vec![*object]162 vec![*object]
161 }163 }
162 PageOp::Level { paragraph, .. }164 PageOp::Level { paragraph, .. }
...@@ -165,7 +167,9 @@ fn named(op: &PageOp) -> Vec<ExGuid> {...@@ -165,7 +167,9 @@ fn named(op: &PageOp) -> Vec<ExGuid> {
165 | PageOp::List { paragraph, .. } => vec![*paragraph],167 | PageOp::List { paragraph, .. } => vec![*paragraph],
166 PageOp::Tags { target, .. } => vec![*target],168 PageOp::Tags { target, .. } => vec![*target],
167 PageOp::Add { object, .. } => vec![object.id()],169 PageOp::Add { object, .. } => vec![object.id()],
168 PageOp::Picture { picture: object, .. }170 PageOp::Picture {
171 picture: object, ..
172 }
169 | PageOp::Attachment {173 | PageOp::Attachment {
170 attachment: object, ..174 attachment: object, ..
171 }175 }
...@@ -196,7 +200,8 @@ pub(crate) fn conflict_page(...@@ -196,7 +200,8 @@ pub(crate) fn conflict_page(
196 at: u64,200 at: u64,
197) -> Result<Edit> {201) -> Result<Edit> {
198 let mut page = page.clone();202 let mut page = page.clone();
199 page.objects.retain(|object| !matches!(object, PageObject::Unsupported(_)));203 page.objects
204 .retain(|object| !matches!(object, PageObject::Unsupported(_)));
200 let listed = order(new)?;205 let listed = order(new)?;
201 let ops = if listed.iter().any(|(listed, _)| *listed == space) {206 let ops = if listed.iter().any(|(listed, _)| *listed == space) {
202 let index = Index::of(&page);207 let index = Index::of(&page);
...@@ -224,9 +229,7 @@ pub(crate) fn conflict_page(...@@ -224,9 +229,7 @@ pub(crate) fn conflict_page(
224 .map_or(&[][..], |at| &queued[at + 1..])229 .map_or(&[][..], |at| &queued[at + 1..])
225 .iter()230 .iter()
226 .map(|(next, _)| *next)231 .map(|(next, _)| *next)
227 .find(|next| {232 .find(|next| !moved.contains(next) && listed.iter().any(|(listed, _)| listed == next));
228 !moved.contains(next) && listed.iter().any(|(listed, _)| listed == next)
229 });
230 let creation = PageCreation::new(before, Some(&page.title), author)?;233 let creation = PageCreation::new(before, Some(&page.title), author)?;
231 let mut ops = vec![Op::Section(SectionOp::Import {234 let mut ops = vec![Op::Section(SectionOp::Import {
232 creation: creation.clone(),235 creation: creation.clone(),
...@@ -260,7 +263,8 @@ fn moved(old: &mut Section<'_>, new: &mut Section<'_>) -> Result<BTreeSet<ExGuid...@@ -260,7 +263,8 @@ fn moved(old: &mut Section<'_>, new: &mut Section<'_>) -> Result<BTreeSet<ExGuid
260 .iter()263 .iter()
261 .filter_map(|&(space, level)| {264 .filter_map(|&(space, level)| {
262 let (at, before) = base.get(&space)?;265 let (at, before) = base.get(&space)?;
263 (*before == level && series.0.get(&space) == series.1.get(&space)).then_some((space, *at))266 (*before == level && series.0.get(&space) == series.1.get(&space))
267 .then_some((space, *at))
264 })268 })
265 .collect();269 .collect();
266 // A longest subsequence of `kept` in base order (patience sorting).270 // A longest subsequence of `kept` in base order (patience sorting).
...@@ -313,7 +317,10 @@ fn advance(local: &mut Order, op: &SectionOp) {...@@ -313,7 +317,10 @@ fn advance(local: &mut Order, op: &SectionOp) {
313 }317 }
314 }318 }
315 SectionOp::Delete(spaces) => local.retain(|(space, _)| !spaces.contains(space)),319 SectionOp::Delete(spaces) => local.retain(|(space, _)| !spaces.contains(space)),
316 SectionOp::Conflict { .. } | SectionOp::Color(_) => {}320 SectionOp::Conflict { .. }
321 | SectionOp::Color(_)
322 | SectionOp::RestoreVersion { .. }
323 | SectionOp::DeleteVersions { .. } => {}
317 }324 }
318}325}
319326
...@@ -411,10 +418,18 @@ impl Replay {...@@ -411,10 +418,18 @@ impl Replay {
411 /// from `before` on, in the queue's order, that the remote lists where the base had it,418 /// from `before` on, in the queue's order, that the remote lists where the base had it,
412 /// or last, as OneNote 2010 places a page series only one side changed.419 /// or last, as OneNote 2010 places a page series only one side changed.
413 fn anchor(&self, listed: &Order, page: ExGuid, before: Option<ExGuid>) -> Option<ExGuid> {420 fn anchor(&self, listed: &Order, page: ExGuid, before: Option<ExGuid>) -> Option<ExGuid> {
414 let from = self.local.iter().position(|(space, _)| Some(*space) == before)?;421 let from = self
415 self.local[from..].iter().map(|(space, _)| *space).find(|space| {422 .local
416 *space != page && !self.moved.contains(space) && listed.iter().any(|(listed, _)| listed == space)423 .iter()
417 })424 .position(|(space, _)| Some(*space) == before)?;
425 self.local[from..]
426 .iter()
427 .map(|(space, _)| *space)
428 .find(|space| {
429 *space != page
430 && !self.moved.contains(space)
431 && listed.iter().any(|(listed, _)| listed == space)
432 })
418 }433 }
419434
420 /// A section op as it replays on the remote section, or `None`: a page placed by435 /// A section op as it replays on the remote section, or `None`: a page placed by
...@@ -467,6 +482,23 @@ impl Replay {...@@ -467,6 +482,23 @@ impl Replay {
467 }482 }
468 (!kept.is_empty()).then_some(SectionOp::Pages(kept))483 (!kept.is_empty()).then_some(SectionOp::Pages(kept))
469 }484 }
485 // A version the remote deleted is gone; restoring one keeps the remote's page as
486 // the newest version.
487 SectionOp::RestoreVersion { page, version, .. } => (self.listed(*page)
488 && listed_versions(new, *page)?.contains(version))
489 .then(|| op.clone()),
490 SectionOp::DeleteVersions { page, versions } => {
491 let listed = listed_versions(new, *page)?;
492 let versions: Vec<ExGuid> = versions
493 .iter()
494 .filter(|version| listed.contains(version))
495 .copied()
496 .collect();
497 (!versions.is_empty()).then_some(SectionOp::DeleteVersions {
498 page: *page,
499 versions,
500 })
501 }
470 SectionOp::Delete(deleted) => {502 SectionOp::Delete(deleted) => {
471 let deleted: Vec<ExGuid> = deleted503 let deleted: Vec<ExGuid> = deleted
472 .iter()504 .iter()
...@@ -482,6 +514,16 @@ impl Replay {...@@ -482,6 +514,16 @@ impl Replay {
482 }514 }
483}515}
484516
517/// The versions `section` lists for `page`.
518fn listed_versions(section: &mut Section<'_>, page: ExGuid) -> Result<Vec<ExGuid>> {
519 Ok(section
520 .versions()?
521 .into_iter()
522 .filter(|(listed, _)| *listed == page)
523 .flat_map(|(_, versions)| versions.into_iter().map(|version| version.context))
524 .collect())
525}
526
485/// One page as the base stored it and as the remote does, with the remote's text changes527/// One page as the base stored it and as the remote does, with the remote's text changes
486/// in the coordinates of the replayed edits.528/// in the coordinates of the replayed edits.
487struct Diff {529struct Diff {
crates/notebook/src/migrate.rs+11-4
...@@ -116,9 +116,12 @@ fn conflict(space: ExGuid, remote: &Page, local: &Page, author: &str) -> Result<...@@ -116,9 +116,12 @@ fn conflict(space: ExGuid, remote: &Page, local: &Page, author: &str) -> Result<
116 for object in &page.objects {116 for object in &page.objects {
117 match object {117 match object {
118 onestore::page::PageObject::Outline(outline) => pending.extend(&outline.paragraphs),118 onestore::page::PageObject::Outline(outline) => pending.extend(&outline.paragraphs),
119 onestore::page::PageObject::Title(title) => {119 onestore::page::PageObject::Title(title) => pending.extend(
120 pending.extend(title.outlines.iter().flat_map(|outline| &outline.paragraphs))120 title
121 }121 .outlines
122 .iter()
123 .flat_map(|outline| &outline.paragraphs),
124 ),
122 _ => {}125 _ => {}
123 }126 }
124 }127 }
...@@ -128,7 +131,11 @@ fn conflict(space: ExGuid, remote: &Page, local: &Page, author: &str) -> Result<...@@ -128,7 +131,11 @@ fn conflict(space: ExGuid, remote: &Page, local: &Page, author: &str) -> Result<
128 texts.insert(text.id, format!("{:?}", text.text));131 texts.insert(text.id, format!("{:?}", text.text));
129 }132 }
130 onestore::page::ParagraphContent::Table(table) => pending.extend(133 onestore::page::ParagraphContent::Table(table) => pending.extend(
131 table.rows.iter().flat_map(|row| &row.cells).flat_map(|cell| &cell.paragraphs),134 table
135 .rows
136 .iter()
137 .flat_map(|row| &row.cells)
138 .flat_map(|cell| &cell.paragraphs),
132 ),139 ),
133 _ => {}140 _ => {}
134 }141 }
crates/notebook/src/queue.rs+2
...@@ -240,6 +240,8 @@ pub(crate) fn spaces(edit: &Edit, root: onestore::ExGuid) -> Vec<onestore::ExGui...@@ -240,6 +240,8 @@ pub(crate) fn spaces(edit: &Edit, root: onestore::ExGuid) -> Vec<onestore::ExGui
240 }240 }
241 SectionOp::Pages(edits) => spaces.extend(edits.iter().map(|edit| edit.space())),241 SectionOp::Pages(edits) => spaces.extend(edits.iter().map(|edit| edit.space())),
242 SectionOp::Delete(pages) => spaces.extend(pages),242 SectionOp::Delete(pages) => spaces.extend(pages),
243 SectionOp::RestoreVersion { page, .. }
244 | SectionOp::DeleteVersions { page, .. } => spaces.push(*page),
243 SectionOp::Color(_) => {}245 SectionOp::Color(_) => {}
244 }246 }
245 }247 }
crates/notebook/src/session.rs+165-25
...@@ -2,8 +2,7 @@...@@ -2,8 +2,7 @@
2//! publishes their edits to the section file in the background.2//! publishes their edits to the section file in the background.
33
4use crate::{4use crate::{
5 EditStatus, Error, PendingEdit, Remote, Replica, Resolution, Result,5 EditStatus, Error, PendingEdit, Remote, Replica, Resolution, Result, SyncWorker, discover,
6 SyncWorker, discover,
7};6};
8use onestore::{7use onestore::{
9 CommitError, ExGuid, PageCreation, RevisionIndex, Stamp, Store, Transaction,8 CommitError, ExGuid, PageCreation, RevisionIndex, Stamp, Store, Transaction,
...@@ -388,9 +387,37 @@ impl Notebook {...@@ -388,9 +387,37 @@ impl Notebook {
388 self.commit_toc(&toc, edits)387 self.commit_toc(&toc, edits)
389 }388 }
390389
390 /// Commits `edits` to the TOC at `toc`. A name it still lists for a file gone from its
391 /// folder passes to the file an edit gives that name; the stale entry goes.
391 fn commit_toc(&self, toc: &str, edits: &[onestore::TocEdit]) -> Result<()> {392 fn commit_toc(&self, toc: &str, edits: &[onestore::TocEdit]) -> Result<()> {
393 let unresolved = self
394 .folder(split(toc).0)
395 .ok()
396 .and_then(|folder| folder.toc.as_ref())
397 .map_or(&[][..], |toc| &toc.unresolved[..]);
398 let mut superseded = Vec::new();
399 for edit in edits {
400 if let onestore::TocEdit::Add { filename, .. }
401 | onestore::TocEdit::Rename { filename, .. } = edit
402 {
403 superseded.extend(
404 unresolved
405 .iter()
406 .filter(|entry| {
407 entry
408 .filename
409 .as_deref()
410 .is_some_and(|name| name.eq_ignore_ascii_case(filename))
411 })
412 .map(|entry| onestore::TocEdit::Remove {
413 identity: entry.file,
414 }),
415 );
416 }
417 superseded.push(edit.clone());
418 }
392 let source = self.storage.read(toc)?;419 let source = self.storage.read(toc)?;
393 match onestore::edit_table_of_contents(&source, edits)? {420 match onestore::edit_table_of_contents(&source, &superseded)? {
394 Some(transaction) => self.storage.commit(toc, &transaction),421 Some(transaction) => self.storage.commit(toc, &transaction),
395 None => Ok(()),422 None => Ok(()),
396 }423 }
...@@ -399,7 +426,12 @@ impl Notebook {...@@ -399,7 +426,12 @@ impl Notebook {
399 /// Creates `name.one` in `folder` holding the page `page` creates, in the next of426 /// Creates `name.one` in `folder` holding the page `page` creates, in the next of
400 /// OneNote's section colours, and lists it last in the folder's TOC, as OneNote creates427 /// OneNote's section colours, and lists it last in the folder's TOC, as OneNote creates
401 /// a section. Returns the new catalog path.428 /// a section. Returns the new catalog path.
402 pub fn create_section(&mut self, folder: &str, name: &str, page: &PageCreation) -> Result<String> {429 pub fn create_section(
430 &mut self,
431 folder: &str,
432 name: &str,
433 page: &PageCreation,
434 ) -> Result<String> {
403 let filename = format!("{name}.one");435 let filename = format!("{name}.one");
404 let color = next_color(self.folder(folder)?);436 let color = next_color(self.folder(folder)?);
405 let (_, ancestor) = self.toc(folder)?;437 let (_, ancestor) = self.toc(folder)?;
...@@ -541,13 +573,24 @@ impl Notebook {...@@ -541,13 +573,24 @@ impl Notebook {
541 let mut folders = Vec::new();573 let mut folders = Vec::new();
542 let mut pending = vec![self.folder(path)?];574 let mut pending = vec![self.folder(path)?];
543 while let Some(group) = pending.pop() {575 while let Some(group) = pending.pop() {
576 // Its folder could not be emptied, so nothing moves.
577 if let Some(entry) = group.unavailable.first() {
578 return Err(io::Error::new(
579 io::ErrorKind::PermissionDenied,
580 entry.error.clone(),
581 )
582 .into());
583 }
544 sections.extend(584 sections.extend(
545 group585 group
546 .sections586 .sections
547 .iter()587 .iter()
548 .map(|section| (section.path.clone(), section.file_id)),588 .map(|section| (section.path.clone(), section.file_id)),
549 );589 );
550 folders.push((group.path.clone(), group.toc.as_ref().map(|toc| toc.filename.clone())));590 folders.push((
591 group.path.clone(),
592 group.toc.as_ref().map(|toc| toc.filename.clone()),
593 ));
551 pending.extend(&group.groups);594 pending.extend(&group.groups);
552 }595 }
553 for (section, identity) in sections {596 for (section, identity) in sections {
...@@ -566,28 +609,50 @@ impl Notebook {...@@ -566,28 +609,50 @@ impl Notebook {
566 }609 }
567610
568 /// The recycle bin's TOC path and file identity, creating the bin as OneNote does: a611 /// The recycle bin's TOC path and file identity, creating the bin as OneNote does: a
569 /// section group the root TOC lists.612 /// section group the root TOC lists. A bin the root TOC misses is listed again.
570 fn bin(&self) -> Result<(String, [u8; 16])> {613 fn bin(&self) -> Result<(String, [u8; 16])> {
571 let bin_toc = catalog_path(RECYCLE_BIN, TOC);614 if let Some(entry) = self
572 if !self.storage.exists(&bin_toc) {615 .catalog
573 if !self.storage.exists(RECYCLE_BIN) {616 .unavailable
574 self.storage.create_directory(RECYCLE_BIN)?;617 .iter()
618 .find(|entry| entry.path == RECYCLE_BIN)
619 {
620 return Err(
621 io::Error::new(io::ErrorKind::PermissionDenied, entry.error.clone()).into(),
622 );
623 }
624 // A bin OneNote made may name its TOC otherwise; a second TOC would hide the folder.
625 let bin_toc = match self.folder(RECYCLE_BIN) {
626 Ok(discover::Folder { toc: Some(toc), .. }) => catalog_path(RECYCLE_BIN, &toc.filename),
627 _ => {
628 let bin_toc = catalog_path(RECYCLE_BIN, TOC);
629 if !self.storage.exists(&bin_toc) {
630 if !self.storage.exists(RECYCLE_BIN) {
631 self.storage.create_directory(RECYCLE_BIN)?;
632 }
633 let bytes = onestore::create_table_of_contents(TOC, &[])?;
634 self.storage.create(&bin_toc, &bytes)?;
635 }
636 bin_toc
575 }637 }
576 let bytes = onestore::create_table_of_contents(TOC, &[])?;638 };
577 self.storage.create(&bin_toc, &bytes)?;639 let identity = onestore::Store::parse(&self.storage.read(&bin_toc)?)?
578 self.storage.place(&bin_toc, self.toc("")?.1, RECYCLE_BIN)?;640 .header
579 let bin_identity = onestore::Store::parse(&bytes)?.header.file_id;641 .file_id;
580 self.edit_toc(642 let (root, root_identity) = self.toc("")?;
581 "",643 // OneNote takes a TOC placed under another parent for a new one.
644 if !lists(&self.storage.read(&root)?, identity)? {
645 self.storage.place(&bin_toc, root_identity, RECYCLE_BIN)?;
646 self.commit_toc(
647 &root,
582 &[onestore::TocEdit::Add {648 &[onestore::TocEdit::Add {
583 filename: RECYCLE_BIN.into(),649 filename: RECYCLE_BIN.into(),
584 identity: bin_identity,650 identity,
585 group: true,651 group: true,
586 }],652 }],
587 )?;653 )?;
588 }654 }
589 let source = self.storage.read(&bin_toc)?;655 Ok((bin_toc, identity))
590 Ok((bin_toc, onestore::Store::parse(&source)?.header.file_id))
591 }656 }
592657
593 /// Moves the section file at `path` into the recycle bin, under a name no binned658 /// Moves the section file at `path` into the recycle bin, under a name no binned
...@@ -626,23 +691,35 @@ impl Notebook {...@@ -626,23 +691,35 @@ impl Notebook {
626 // OneNote's "Deleted Pages" section is grey.691 // OneNote's "Deleted Pages" section is grey.
627 let bytes = onestore::create_empty_section(DELETED, Some(0x00e1e1e1))?;692 let bytes = onestore::create_empty_section(DELETED, Some(0x00e1e1e1))?;
628 self.storage.create(&path, &bytes)?;693 self.storage.create(&path, &bytes)?;
694 }
695 let mut bytes = self.storage.read(&path)?;
696 // Listed and placed too when an earlier delete made the file but not its entry.
697 let identity = onestore::Store::parse(&bytes)?.header.file_id;
698 if !lists(&self.storage.read(&bin_toc)?, identity)? {
629 self.storage.place(&path, bin_identity, DELETED)?;699 self.storage.place(&path, bin_identity, DELETED)?;
630 self.commit_toc(700 self.commit_toc(
631 &bin_toc,701 &bin_toc,
632 &[onestore::TocEdit::Add {702 &[onestore::TocEdit::Add {
633 filename: DELETED.into(),703 filename: DELETED.into(),
634 identity: onestore::Store::parse(&bytes)?.header.file_id,704 identity,
635 group: false,705 group: false,
636 }],706 }],
637 )?;707 )?;
708 bytes = self.storage.read(&path)?;
638 }709 }
639 let arena = onestore::Arena::default();710 let arena = onestore::Arena::default();
640 let mut section = onestore::Section::open(&arena, self.storage.read(&path)?)?;711 let mut section = onestore::Section::open(&arena, bytes)?;
641 let mut ops = Vec::new();712 let mut ops = Vec::new();
642 for page in pages {713 for page in pages {
643 ops.push(moved(page, author)?);714 ops.push(moved(page, author)?);
644 }715 }
645 section.apply(author, &Edit { at: crate::now(), ops })?;716 section.apply(
717 author,
718 &Edit {
719 at: crate::now(),
720 ops,
721 },
722 )?;
646 if let Some(transaction) = section.seal()? {723 if let Some(transaction) = section.seal()? {
647 self.storage.commit(&path, &transaction)?;724 self.storage.commit(&path, &transaction)?;
648 }725 }
...@@ -866,9 +943,8 @@ pub fn stored_pages(image: &[u8]) -> Result<Vec<StoredPage>> {...@@ -866,9 +943,8 @@ pub fn stored_pages(image: &[u8]) -> Result<Vec<StoredPage>> {
866/// order it gives them (`corpus/notebook-management/native/section-colors`: sixteen sections943/// order it gives them (`corpus/notebook-management/native/section-colors`: sixteen sections
867/// made with New Section in a new notebook).944/// made with New Section in a new notebook).
868const SECTION_COLORS: [u32; 16] = [945const SECTION_COLORS: [u32; 16] = [
869 0x00e4a88a, 0x0078b0f6, 0x00bba4d5, 0x00d2bb9b, 0x00b79cab, 0x0099d1e8, 0x006ff9f5,946 0x00e4a88a, 0x0078b0f6, 0x00bba4d5, 0x00d2bb9b, 0x00b79cab, 0x0099d1e8, 0x006ff9f5, 0x0092e7ad,
870 0x0092e7ad, 0x00cabc4d, 0x007575ba, 0x00aa9595, 0x00e4a88a, 0x0069d8ff, 0x0097c9b7,947 0x00cabc4d, 0x007575ba, 0x00aa9595, 0x00e4a88a, 0x0069d8ff, 0x0097c9b7, 0x009795ee, 0x00de9eb4,
871 0x009795ee, 0x00de9eb4,
872];948];
873949
874/// The colour OneNote gives a new section in `folder`, by how many sections it holds.950/// The colour OneNote gives a new section in `folder`, by how many sections it holds.
...@@ -897,6 +973,28 @@ fn component(name: &str) -> bool {...@@ -897,6 +973,28 @@ fn component(name: &str) -> bool {
897 !name.is_empty() && !name.contains(['/', '\\', '\0']) && name != "." && name != ".."973 !name.is_empty() && !name.contains(['/', '\\', '\0']) && name != "." && name != ".."
898}974}
899975
976/// Whether the TOC `image` holds has an entry for the file identity `file`.
977fn lists(image: &[u8], file: [u8; 16]) -> Result<bool> {
978 let store = Store::parse(image)?;
979 let index = RevisionIndex::parse(&store)?;
980 let document = Document::parse(&index)?;
981 let revision = document.active(document.root)?;
982 let entries = revision
983 .roots
984 .get(&1)
985 .and_then(|id| revision.nodes.get(id))
986 .map_or(&[][..], |node| match &node.kind {
987 onestore::document::Kind::Toc { entries, .. } => &entries[..],
988 _ => &[],
989 });
990 Ok(entries.iter().any(|id| {
991 matches!(
992 revision.nodes.get(id).map(|node| &node.kind),
993 Some(onestore::document::Kind::Toc { identity: Some(identity), .. }) if *identity == file
994 )
995 }))
996}
997
900fn split(path: &str) -> (&str, &str) {998fn split(path: &str) -> (&str, &str) {
901 match path.rsplit_once('/') {999 match path.rsplit_once('/') {
902 Some((folder, name)) => (folder, name),1000 Some((folder, name)) => (folder, name),
...@@ -1167,6 +1265,48 @@ impl Section {...@@ -1167,6 +1265,48 @@ impl Section {
1167 self.replica.conflicts()1265 self.replica.conflicts()
1168 }1266 }
11691267
1268 /// The versions of each page that has them, newest first, as the local edits leave them
1269 /// (`onestore::Section::versions`); `version` reads one.
1270 pub fn versions(&self) -> Result<Vec<(ExGuid, Vec<onestore::PageVersion>)>> {
1271 self.replica.versions()
1272 }
1273
1274 /// Page `space` as its version `version` holds it; O(section).
1275 pub fn version(&self, space: ExGuid, version: ExGuid) -> Result<Page> {
1276 self.replica.version(space, version)
1277 }
1278
1279 /// Makes a page's version its current state, the page as it stood becoming the newest
1280 /// version, queued like the user's own edits.
1281 pub fn restore_version(&self, space: ExGuid, version: ExGuid, author: &str) -> Result<u64> {
1282 self.replica.apply(
1283 author,
1284 Edit {
1285 at: crate::now(),
1286 ops: vec![Op::Section(SectionOp::restore(space, version)?)],
1287 },
1288 )
1289 }
1290
1291 /// Deletes versions of pages, queued like the user's own edits.
1292 pub fn delete_versions(&self, versions: &[(ExGuid, Vec<ExGuid>)]) -> Result<u64> {
1293 self.replica.apply(
1294 "",
1295 Edit {
1296 at: crate::now(),
1297 ops: versions
1298 .iter()
1299 .map(|(page, versions)| {
1300 Op::Section(SectionOp::DeleteVersions {
1301 page: *page,
1302 versions: versions.clone(),
1303 })
1304 })
1305 .collect(),
1306 },
1307 )
1308 }
1309
1170 /// Retires an uncertain attempt after review, exporting the queue to `archive` first:1310 /// Retires an uncertain attempt after review, exporting the queue to `archive` first:
1171 /// `Mine` publishes the local edits again, `Theirs` abandons them. Neither claims the1311 /// `Mine` publishes the local edits again, `Theirs` abandons them. Neither claims the
1172 /// attempt was acknowledged.1312 /// attempt was acknowledged.
crates/notebook/src/smb/mod.rs+2-1
...@@ -206,7 +206,8 @@ impl Client {...@@ -206,7 +206,8 @@ impl Client {
206 0xc0000043 | 0xc0000054 | 0xc0000055 => io::ErrorKind::WouldBlock,206 0xc0000043 | 0xc0000054 | 0xc0000055 => io::ErrorKind::WouldBlock,
207 0xc0000011 => io::ErrorKind::UnexpectedEof,207 0xc0000011 => io::ErrorKind::UnexpectedEof,
208 0xc0000034 | 0xc000003a => io::ErrorKind::NotFound,208 0xc0000034 | 0xc000003a => io::ErrorKind::NotFound,
209 0xc0000022 => io::ErrorKind::PermissionDenied,209 // Windows reports a delete-pending file as access denied too.
210 0xc0000022 | 0xc0000056 => io::ErrorKind::PermissionDenied,
210 0xc0000103 => io::ErrorKind::NotADirectory,211 0xc0000103 => io::ErrorKind::NotADirectory,
211 _ => io::ErrorKind::Other,212 _ => io::ErrorKind::Other,
212 };213 };
crates/notebook/src/smb/tests.rs+12-2
...@@ -609,13 +609,23 @@ fn live_structure() {...@@ -609,13 +609,23 @@ fn live_structure() {
609 let mut notebook =609 let mut notebook =
610 Notebook::open_smb(std::sync::Arc::clone(&client), &root, cache.path()).unwrap();610 Notebook::open_smb(std::sync::Arc::clone(&client), &root, cache.path()).unwrap();
611 assert_eq!(611 assert_eq!(
612 notebook.create_section("", "Second", &onestore::PageCreation::new(None, Some(""), "Author").unwrap()).unwrap(),612 notebook
613 .create_section(
614 "",
615 "Second",
616 &onestore::PageCreation::new(None, Some(""), "Author").unwrap()
617 )
618 .unwrap(),
613 "Second.one"619 "Second.one"
614 );620 );
615 assert_eq!(notebook.create_group("", "Archive").unwrap(), "Archive");621 assert_eq!(notebook.create_group("", "Archive").unwrap(), "Archive");
616 assert_eq!(622 assert_eq!(
617 notebook623 notebook
618 .create_section("Archive", "Inner", &onestore::PageCreation::new(None, Some(""), "Author").unwrap())624 .create_section(
625 "Archive",
626 "Inner",
627 &onestore::PageCreation::new(None, Some(""), "Author").unwrap()
628 )
619 .unwrap(),629 .unwrap(),
620 "Archive/Inner.one"630 "Archive/Inner.one"
621 );631 );
crates/notebook/src/sync.rs+11-2
...@@ -341,10 +341,19 @@ impl Replica {...@@ -341,10 +341,19 @@ impl Replica {
341 if let Some(sealed) = sealed {341 if let Some(sealed) = sealed {
342 local.replay(sealed)?;342 local.replay(sealed)?;
343 }343 }
344 let remote = onestore::Section::open(&remote, image.to_vec())?;344 let mut remote = onestore::Section::open(&remote, image.to_vec())?;
345 // A page's versions change without its page changing.
346 let versions = |section: &mut onestore::Section<'_>| {
347 section.versions().map(|versions| {
348 versions
349 .into_iter()
350 .filter(|(page, _)| revisions.contains_key(page))
351 .collect::<Vec<_>>()
352 })
353 };
345 Ok(revisions.keys().all(354 Ok(revisions.keys().all(
346 |space| matches!((local.page(*space), remote.page(*space)), (Ok(a), Ok(b)) if a == b),355 |space| matches!((local.page(*space), remote.page(*space)), (Ok(a), Ok(b)) if a == b),
347 ))356 ) && versions(&mut local)? == versions(&mut remote)?)
348 }357 }
349358
350 /// Asks the section thread to replay the queue on `image`, returning the pages the359 /// Asks the section thread to replay the queue on `image`, returning the pages the
crates/notebook/src/working.rs+41-6
...@@ -3,9 +3,7 @@...@@ -3,9 +3,7 @@
3//! here and are written in one SQLite transaction per burst; the sync thread asks it to seal3//! here and are written in one SQLite transaction per burst; the sync thread asks it to seal
4//! and, when the remote changed, to rebase the queue.4//! and, when the remote changed, to rebase the queue.
55
6use crate::{6use crate::{Result, base, lock, merge, queue, worker::Signal};
7 Result, base, lock, merge, queue, worker::Signal,
8};
9use onestore::{7use onestore::{
10 Arena, ExGuid, Section, Transaction,8 Arena, ExGuid, Section, Transaction,
11 op::{Edit, OpError},9 op::{Edit, OpError},
...@@ -37,6 +35,14 @@ pub(crate) enum Request {...@@ -37,6 +35,14 @@ pub(crate) enum Request {
37 Conflicts {35 Conflicts {
38 reply: Reply<Vec<(ExGuid, Vec<onestore::ConflictPage>)>>,36 reply: Reply<Vec<(ExGuid, Vec<onestore::ConflictPage>)>>,
39 },37 },
38 Versions {
39 reply: Reply<Vec<(ExGuid, Vec<onestore::PageVersion>)>>,
40 },
41 Version {
42 space: ExGuid,
43 version: ExGuid,
44 reply: Reply<Page>,
45 },
40 /// Answers once the edits before it are written.46 /// Answers once the edits before it are written.
41 Flush {47 Flush {
42 reply: Reply<()>,48 reply: Reply<()>,
...@@ -79,6 +85,8 @@ impl Request {...@@ -79,6 +85,8 @@ impl Request {
79 Self::Page { reply, .. } => reply(Err(error())),85 Self::Page { reply, .. } => reply(Err(error())),
80 Self::Pages { reply } => reply(Err(error())),86 Self::Pages { reply } => reply(Err(error())),
81 Self::Conflicts { reply } => reply(Err(error())),87 Self::Conflicts { reply } => reply(Err(error())),
88 Self::Versions { reply } => reply(Err(error())),
89 Self::Version { reply, .. } => reply(Err(error())),
82 Self::Flush { reply } => reply(Err(error())),90 Self::Flush { reply } => reply(Err(error())),
83 Self::Seal { reply } => reply(Err(error())),91 Self::Seal { reply } => reply(Err(error())),
84 Self::Rebase { reply, .. } => reply(Err(error())),92 Self::Rebase { reply, .. } => reply(Err(error())),
...@@ -321,6 +329,14 @@ impl<'a> Working<'a> {...@@ -321,6 +329,14 @@ impl<'a> Working<'a> {
321 Request::Conflicts { reply } => {329 Request::Conflicts { reply } => {
322 reply(self.section.conflicts().map_err(Into::into))330 reply(self.section.conflicts().map_err(Into::into))
323 }331 }
332 Request::Versions { reply } => {
333 reply(self.section.versions().map_err(Into::into))
334 }
335 Request::Version {
336 space,
337 version,
338 reply,
339 } => reply(self.section.version(space, version).map_err(Into::into)),
324 Request::Handover(to) => {340 Request::Handover(to) => {
325 let mut waiting = held.take().unwrap_or_default();341 let mut waiting = held.take().unwrap_or_default();
326 waiting.extend(burst.drain(..).chain(later.drain(..)));342 waiting.extend(burst.drain(..).chain(later.drain(..)));
...@@ -355,6 +371,18 @@ impl<'a> Working<'a> {...@@ -355,6 +371,18 @@ impl<'a> Working<'a> {
355 reply(self.section.conflicts().map_err(Into::into));371 reply(self.section.conflicts().map_err(Into::into));
356 false372 false
357 }373 }
374 Request::Versions { reply } => {
375 reply(self.section.versions().map_err(Into::into));
376 false
377 }
378 Request::Version {
379 space,
380 version,
381 reply,
382 } => {
383 reply(self.section.version(space, version).map_err(Into::into));
384 false
385 }
358 request @ (Request::Flush { .. } | Request::Seal { .. })386 request @ (Request::Flush { .. } | Request::Seal { .. })
359 if !burst.is_empty() =>387 if !burst.is_empty() =>
360 {388 {
...@@ -390,7 +418,9 @@ impl<'a> Working<'a> {...@@ -390,7 +418,9 @@ impl<'a> Working<'a> {
390 .then(VecDeque::new);418 .then(VecDeque::new);
391 false419 false
392 } else {420 } else {
393 reply(Err(io::Error::other("The queue could not be written").into()));421 reply(Err(
422 io::Error::other("The queue could not be written").into()
423 ));
394 true424 true
395 }425 }
396 }426 }
...@@ -546,7 +576,7 @@ impl<'a> Working<'a> {...@@ -546,7 +576,7 @@ impl<'a> Working<'a> {
546 let root = self.section.root();576 let root = self.section.root();
547 let revisions: BTreeMap<ExGuid, ExGuid> = self577 let revisions: BTreeMap<ExGuid, ExGuid> = self
548 .section578 .section
549 .revisions()579 .newest()
550 .filter(|(space, _)| {580 .filter(|(space, _)| {
551 self.touched.contains(space) || (self.touched.is_empty() && *space == root)581 self.touched.contains(space) || (self.touched.is_empty() && *space == root)
552 })582 })
...@@ -677,7 +707,12 @@ fn rebase(connection: &Mutex<Connection>, image: Option<Vec<u8>>) -> Result<Vec<...@@ -677,7 +707,12 @@ fn rebase(connection: &Mutex<Connection>, image: Option<Vec<u8>>) -> Result<Vec<
677 .iter()707 .iter()
678 .filter(|(space, rid)| after.get(space) != Some(rid))708 .filter(|(space, rid)| after.get(space) != Some(rid))
679 .map(|(space, _)| *space)709 .map(|(space, _)| *space)
680 .chain(after.keys().filter(|space| !before.contains_key(space)).copied())710 .chain(
711 after
712 .keys()
713 .filter(|space| !before.contains_key(space))
714 .copied(),
715 )
681 .collect();716 .collect();
682 changed.extend(converged);717 changed.extend(converged);
683 Ok(changed.into_iter().collect())718 Ok(changed.into_iter().collect())
crates/notebook/tests/assets.rs+1-4
...@@ -118,10 +118,7 @@ fn downloaded_media_survives_reopen_and_recovery_with_the_queue_intact() {...@@ -118,10 +118,7 @@ fn downloaded_media_survives_reopen_and_recovery_with_the_queue_intact() {
118 (summary.cached_assets, summary.cached_asset_bytes),118 (summary.cached_assets, summary.cached_asset_bytes),
119 (2, 1024)119 (2, 1024)
120 );120 );
121 assert_eq!(121 assert_eq!(server::pages(&snapshot(&cache)), server::pages(&working));
122 server::pages(&snapshot(&cache)),
123 server::pages(&working)
124 );
125 assert_eq!(remote_snapshot(&cache), remote);122 assert_eq!(remote_snapshot(&cache), remote);
126 assert_eq!(cache.pending().unwrap(), pending);123 assert_eq!(cache.pending().unwrap(), pending);
127 assert_eq!(cache.status(1).unwrap(), uncertain);124 assert_eq!(cache.status(1).unwrap(), uncertain);
crates/notebook/tests/cache.rs+1-4
...@@ -110,10 +110,7 @@ fn cache_reopen_preserves_the_base_and_queued_edits() {...@@ -110,10 +110,7 @@ fn cache_reopen_preserves_the_base_and_queued_edits() {
110 .apply(model_ops::AUTHOR, typed(sid, oid, "Recovered "))110 .apply(model_ops::AUTHOR, typed(sid, oid, "Recovered "))
111 .unwrap();111 .unwrap();
112 assert!(second > first);112 assert!(second > first);
113 assert_eq!(113 assert_eq!(target(&snapshot(&replica)).2, "Recovered café 🐈 日本語");
114 target(&snapshot(&replica)).2,
115 "Recovered café 🐈 日本語"
116 );
117 assert_eq!(replica.pending().unwrap().len(), 2);114 assert_eq!(replica.pending().unwrap().len(), 2);
118}115}
119116
crates/notebook/tests/changes.rs+14-2
...@@ -24,9 +24,21 @@ fn external_structure_changes_are_reported_by_identity() {...@@ -24,9 +24,21 @@ fn external_structure_changes_are_reported_by_identity() {
24 let mut other = Notebook::open(&root, temporary.path().join("other")).unwrap();24 let mut other = Notebook::open(&root, temporary.path().join("other")).unwrap();
25 assert_eq!(watcher.refresh().unwrap(), []);25 assert_eq!(watcher.refresh().unwrap(), []);
2626
27 other.create_section("", "Second", &onestore::PageCreation::new(None, Some(""), "Author").unwrap()).unwrap();27 other
28 .create_section(
29 "",
30 "Second",
31 &onestore::PageCreation::new(None, Some(""), "Author").unwrap(),
32 )
33 .unwrap();
28 other.create_group("", "Group").unwrap();34 other.create_group("", "Group").unwrap();
29 other.create_section("Group", "Inner", &onestore::PageCreation::new(None, Some(""), "Author").unwrap()).unwrap();35 other
36 .create_section(
37 "Group",
38 "Inner",
39 &onestore::PageCreation::new(None, Some(""), "Author").unwrap(),
40 )
41 .unwrap();
30 assert_eq!(42 assert_eq!(
31 watcher.refresh().unwrap(),43 watcher.refresh().unwrap(),
32 [44 [
crates/notebook/tests/discovery.rs+79
...@@ -278,3 +278,82 @@ fn local_access_stays_inside_the_selected_root() {...@@ -278,3 +278,82 @@ fn local_access_stays_inside_the_selected_root() {
278 );278 );
279 }279 }
280}280}
281
282/// Restores permissions a test took away, so the temporary directory can be removed.
283struct Restore(Vec<(std::path::PathBuf, u32)>);
284
285impl Drop for Restore {
286 fn drop(&mut self) {
287 use std::os::unix::fs::PermissionsExt;
288 for (path, mode) in &self.0 {
289 let _ = fs::set_permissions(path, fs::Permissions::from_mode(*mode));
290 }
291 }
292}
293
294#[test]
295fn unreadable_children_are_listed_as_unavailable_and_retried() {
296 use std::os::unix::fs::PermissionsExt;
297 // Permission bits do not bind the superuser.
298 if unsafe { libc::geteuid() } == 0 {
299 return;
300 }
301 let directory = tempfile::tempdir().unwrap();
302 let root = directory.path().join("notebook");
303 fs::create_dir(&root).unwrap();
304 fixture(&root);
305 fs::create_dir(root.join("Art")).unwrap();
306 fs::create_dir(root.join("Group")).unwrap();
307 for name in ["kept.one", "denied.one"] {
308 fs::write(
309 root.join("Group").join(name),
310 onestore::create_section(name, "Text", "Fixture").unwrap(),
311 )
312 .unwrap();
313 }
314 fs::write(
315 root.join("Group/Open Notebook.onetoc2"),
316 onestore::create_table_of_contents("Open Notebook.onetoc2", &[]).unwrap(),
317 )
318 .unwrap();
319 let _restore = Restore(vec![
320 (root.join("Art"), 0o755),
321 (root.join("Group/denied.one"), 0o644),
322 ]);
323 for path in [root.join("Art"), root.join("Group/denied.one")] {
324 fs::set_permissions(path, fs::Permissions::from_mode(0o000)).unwrap();
325 }
326
327 let mut notebook =
328 notebook::session::Notebook::open(&root, directory.path().join("cache")).unwrap();
329 let catalog = notebook.catalog();
330 assert_eq!(catalog.sections.len(), 1);
331 assert_eq!(catalog.groups.len(), 1);
332 assert_eq!(catalog.unavailable.len(), 1);
333 assert_eq!(catalog.unavailable[0].path, "Art");
334 assert!(catalog.unavailable[0].group);
335 assert!(catalog.unavailable[0].error.contains("Permission denied"));
336 let group = &catalog.groups[0];
337 assert_eq!(group.sections.len(), 1);
338 assert_eq!(group.sections[0].path, "Group/kept.one");
339 assert_eq!(group.unavailable[0].path, "Group/denied.one");
340 assert!(!group.unavailable[0].group);
341
342 // An unavailable entry is not the catalog's to change, nor a group holding one.
343 assert!(notebook.delete("Art").is_err());
344 assert!(notebook.delete("Group").is_err());
345 assert!(root.join("Group/kept.one").exists());
346 assert!(notebook.rename("Art", "Other").is_err());
347 assert!(notebook.create_group("", "Art").is_err());
348
349 fs::set_permissions(root.join("Art"), fs::Permissions::from_mode(0o755)).unwrap();
350 notebook.refresh().unwrap();
351 assert!(notebook.catalog().unavailable.is_empty());
352 assert_eq!(notebook.catalog().groups.len(), 2);
353
354 // The notebook's own folder is still the notebook.
355 fs::set_permissions(&root, fs::Permissions::from_mode(0o000)).unwrap();
356 let denied = notebook.refresh();
357 fs::set_permissions(&root, fs::Permissions::from_mode(0o755)).unwrap();
358 assert!(denied.is_err());
359}
crates/notebook/tests/migrate.rs+13-9
...@@ -191,9 +191,7 @@ fn a_schema_14_queue_converts_to_ops_that_reach_its_working_pages_and_publish()...@@ -191,9 +191,7 @@ fn a_schema_14_queue_converts_to_ops_that_reach_its_working_pages_and_publish()
191 Queued::Page(191 Queued::Page(
192 a,192 a,
193 appended(193 appended(
194 ops::saved(SOURCE, a, &first)194 ops::saved(SOURCE, a, &first).unwrap().as_slice(),
195 .unwrap()
196 .as_slice(),
197 a,195 a,
198 a_text,196 a_text,
199 " two",197 " two",
...@@ -306,10 +304,7 @@ fn an_uncertain_attempt_keeps_its_state_and_a_conflict_becomes_a_conflict_page()...@@ -306,10 +304,7 @@ fn an_uncertain_attempt_keeps_its_state_and_a_conflict_becomes_a_conflict_page()
306 .as_slice()304 .as_slice()
307 .to_vec();305 .to_vec();
308 let local = appended(SOURCE, a, a_text, " local");306 let local = appended(SOURCE, a, a_text, " local");
309 let working = ops::saved(SOURCE, a, &local)307 let working = ops::saved(SOURCE, a, &local).unwrap().as_slice().to_vec();
310 .unwrap()
311 .as_slice()
312 .to_vec();
313 {308 {
314 // Schema 14 stored the remote as the base and kept the local working image.309 // Schema 14 stored the remote as the base and kept the local working image.
315 v14(&path, &remote, &[], Some(&working));310 v14(&path, &remote, &[], Some(&working));
...@@ -346,7 +341,10 @@ fn an_uncertain_attempt_keeps_its_state_and_a_conflict_becomes_a_conflict_page()...@@ -346,7 +341,10 @@ fn an_uncertain_attempt_keeps_its_state_and_a_conflict_becomes_a_conflict_page()
346 conflicts(&server.durable),341 conflicts(&server.durable),
347 [(342 [(
348 a,343 a,
349 vec![("Author".to_owned(), page_texts(&model_ops::page_of(&working, a)))]344 vec![(
345 "Author".to_owned(),
346 page_texts(&model_ops::page_of(&working, a))
347 )]
350 )]348 )]
351 );349 );
352}350}
...@@ -447,7 +445,13 @@ fn a_schema_15_cache_upgrades_in_place_and_its_conflict_becomes_a_conflict_page(...@@ -447,7 +445,13 @@ fn a_schema_15_cache_upgrades_in_place_and_its_conflict_becomes_a_conflict_page(
447 assert_eq!(version, 16);445 assert_eq!(version, 16);
448 assert_eq!(columns, ["id", "sealed", "revisions", "attempted"]);446 assert_eq!(columns, ["id", "sealed", "revisions", "attempted"]);
449 drop(connection);447 drop(connection);
450 assert_eq!(notebook::Recovery::open(&archive).unwrap().pending().unwrap(), pending);448 assert_eq!(
449 notebook::Recovery::open(&archive)
450 .unwrap()
451 .pending()
452 .unwrap(),
453 pending
454 );
451 let cache = Replica::open(&path).unwrap();455 let cache = Replica::open(&path).unwrap();
452 let remote = typed(SOURCE, space, text, 0..0, "Remote ");456 let remote = typed(SOURCE, space, text, 0..0, "Remote ");
453 let mut server = Server::new(&remote);457 let mut server = Server::new(&remote);
crates/notebook/tests/session.rs+77-6
...@@ -43,7 +43,13 @@ fn edited(page: &Page, text: &str) -> Page {...@@ -43,7 +43,13 @@ fn edited(page: &Page, text: &str) -> Page {
4343
44/// Replaces `range` of the page's first body text as the editor does: the edit's id once44/// Replaces `range` of the page's first body text as the editor does: the edit's id once
45/// it is durable.45/// it is durable.
46fn typed(section: &Section, space: ExGuid, page: &Page, range: std::ops::Range<u32>, text: &str) -> u64 {46fn typed(
47 section: &Section,
48 space: ExGuid,
49 page: &Page,
50 range: std::ops::Range<u32>,
51 text: &str,
52) -> u64 {
47 let op = PageOp::Text {53 let op = PageOp::Text {
48 text: first_text(page),54 text: first_text(page),
49 range,55 range,
...@@ -240,7 +246,10 @@ fn an_external_change_reloads_the_page_and_later_edits_apply_to_it() {...@@ -240,7 +246,10 @@ fn an_external_change_reloads_the_page_and_later_edits_apply_to_it() {
240 }246 }
241 }247 }
242 section.wake();248 section.wake();
243 wait(&section, |event| matches!(event, Event::Changed(spaces) if spaces.contains(&space)));249 wait(
250 &section,
251 |event| matches!(event, Event::Changed(spaces) if spaces.contains(&space)),
252 );
244 assert_same(section.page(space).unwrap(), &native);253 assert_same(section.page(space).unwrap(), &native);
245 // An edit made on the page shown before the change applies to the page as it is now.254 // An edit made on the page shown before the change applies to the page as it is now.
246 let id = typed(&section, space, &before, 0..0, "Local ");255 let id = typed(&section, space, &before, 0..0, "Local ");
...@@ -312,7 +321,10 @@ fn edits_apply_in_order_without_waiting_and_finish_before_close() {...@@ -312,7 +321,10 @@ fn edits_apply_in_order_without_waiting_and_finish_before_close() {
312 }321 }
313 }322 }
314 section.wake();323 section.wake();
315 wait(&section, |event| matches!(event, Event::Changed(spaces) if spaces.contains(&space)));324 wait(
325 &section,
326 |event| matches!(event, Event::Changed(spaces) if spaces.contains(&space)),
327 );
316 apply("Local ");328 apply("Local ");
317 section.close().unwrap();329 section.close().unwrap();
318 let (section, _) = open(&file, &cache);330 let (section, _) = open(&file, &cache);
...@@ -715,9 +727,10 @@ fn a_conflicting_save_keeps_the_native_page_and_a_conflict_page_the_session_dele...@@ -715,9 +727,10 @@ fn a_conflicting_save_keeps_the_native_page_and_a_conflict_page_the_session_dele
715 }727 }
716 }728 }
717 section.wake();729 section.wake();
718 wait(&section, |event| {730 wait(
719 matches!(event, Event::Attempt { id: n, status: EditStatus::Published { .. } } if *n > id)731 &section,
720 });732 |event| matches!(event, Event::Attempt { id: n, status: EditStatus::Published { .. } } if *n > id),
733 );
721 assert!(matches!(734 assert!(matches!(
722 section.status(id).unwrap(),735 section.status(id).unwrap(),
723 Some(EditStatus::Published { .. })736 Some(EditStatus::Published { .. })
...@@ -846,3 +859,61 @@ fn space_of(source: &[u8]) -> ExGuid {...@@ -846,3 +859,61 @@ fn space_of(source: &[u8]) -> ExGuid {
846 .unwrap()[0]859 .unwrap()[0]
847 .0860 .0
848}861}
862
863/// Restoring and deleting page versions go through the queue and reach the file as OneNote
864/// 2010 stores them (`corpus/page-versions`); a publication that changes only a page's
865/// versions confirms by the history revision it wrote.
866#[test]
867fn page_versions_restore_and_delete_through_the_session() {
868 let directory = tempfile::tempdir().unwrap();
869 let file = directory.path().join("History.one");
870 let cache = directory.path().join("cache");
871 std::fs::copy(
872 Path::new(env!("CARGO_MANIFEST_DIR"))
873 .join("../../corpus/page-versions/native/step-02/notebook/History.one"),
874 &file,
875 )
876 .unwrap();
877 let (section, _) = open(&file, &cache);
878 let listed = section.versions().unwrap();
879 let (space, versions) = listed[0].clone();
880 assert_eq!(versions.len(), 1);
881 let standing = section.page(space).unwrap();
882 let old = section.version(space, versions[0].context).unwrap();
883 let restored = section
884 .restore_version(space, versions[0].context, "Editor")
885 .unwrap();
886 published(&section, restored);
887 assert_eq!(section.page(space).unwrap().objects, old.objects);
888 let stored = |file: &Path| {
889 let arena = onestore::Arena::default();
890 let mut stored =
891 onestore::Section::open(&arena, onestore::read_file(file).unwrap()).unwrap();
892 let versions = stored.versions().unwrap();
893 (stored.page(space).unwrap(), versions)
894 };
895 let (page, after) = stored(&file);
896 assert_eq!(page.objects, old.objects);
897 assert_eq!(after[0].1.len(), 2);
898 assert_eq!(after[0].1[0].author.as_deref(), Some("Other Person"));
899 assert_eq!(
900 section.version(space, after[0].1[0].context).unwrap(),
901 standing
902 );
903 section.close().unwrap();
904
905 // After a relaunch, deleting every version publishes the history alone.
906 let (section, _) = open(&file, &cache);
907 let all: Vec<ExGuid> = section.versions().unwrap()[0]
908 .1
909 .iter()
910 .map(|version| version.context)
911 .collect();
912 let deleted = section.delete_versions(&[(space, all)]).unwrap();
913 published(&section, deleted);
914 assert!(section.versions().unwrap().is_empty());
915 let (page, after) = stored(&file);
916 assert!(after.is_empty());
917 assert_eq!(page.objects, old.objects);
918 section.close().unwrap();
919}
crates/notebook/tests/smb_queue.rs+2-8
...@@ -142,10 +142,7 @@ fn live_keystrokes_publish_reading_only_the_header_and_a_native_change_merges()...@@ -142,10 +142,7 @@ fn live_keystrokes_publish_reading_only_the_header_and_a_native_change_merges()
142 }142 }
143 assert_eq!(remote.reads, 0, "publications read only the header");143 assert_eq!(remote.reads, 0, "publications read only the header");
144 assert_eq!(remote.stamps, 20);144 assert_eq!(remote.stamps, 20);
145 assert_eq!(145 assert_eq!(lab.read(&path, 1 << 24).unwrap(), snapshot(&replica));
146 lab.read(&path, 1 << 24).unwrap(),
147 snapshot(&replica)
148 );
149 for _ in 0..5 {146 for _ in 0..5 {
150 assert_eq!(replica.sync_once(&mut remote).unwrap().edit, None);147 assert_eq!(replica.sync_once(&mut remote).unwrap().edit, None);
151 }148 }
...@@ -175,10 +172,7 @@ fn live_keystrokes_publish_reading_only_the_header_and_a_native_change_merges()...@@ -175,10 +172,7 @@ fn live_keystrokes_publish_reading_only_the_header_and_a_native_change_merges()
175 let merged = text_of(&replica, space, text);172 let merged = text_of(&replica, space, text);
176 assert!(merged.starts_with("Native Body"), "{merged}");173 assert!(merged.starts_with("Native Body"), "{merged}");
177 assert!(merged.ends_with(" local"), "{merged}");174 assert!(merged.ends_with(" local"), "{merged}");
178 assert_eq!(175 assert_eq!(lab.read(&path, 1 << 24).unwrap(), snapshot(&replica));
179 lab.read(&path, 1 << 24).unwrap(),
180 snapshot(&replica)
181 );
182 lab.delete(&path).unwrap();176 lab.delete(&path).unwrap();
183}177}
184178
crates/notebook/tests/structure.rs+165-2
...@@ -53,13 +53,23 @@ fn sections_and_groups_are_created_renamed_coloured_ordered_and_deleted() {...@@ -53,13 +53,23 @@ fn sections_and_groups_are_created_renamed_coloured_ordered_and_deleted() {
53 onestore::place_file(root.join("First.one"), toc_id, "First.one").unwrap();53 onestore::place_file(root.join("First.one"), toc_id, "First.one").unwrap();
54 let mut notebook = Notebook::open(&root, temporary.path().join("cache")).unwrap();54 let mut notebook = Notebook::open(&root, temporary.path().join("cache")).unwrap();
55 assert_eq!(55 assert_eq!(
56 notebook.create_section("", "Second", &onestore::PageCreation::new(None, Some(""), "Author").unwrap()).unwrap(),56 notebook
57 .create_section(
58 "",
59 "Second",
60 &onestore::PageCreation::new(None, Some(""), "Author").unwrap()
61 )
62 .unwrap(),
57 "Second.one"63 "Second.one"
58 );64 );
59 assert_eq!(notebook.create_group("", "Archive").unwrap(), "Archive");65 assert_eq!(notebook.create_group("", "Archive").unwrap(), "Archive");
60 assert_eq!(66 assert_eq!(
61 notebook67 notebook
62 .create_section("Archive", "Inner", &onestore::PageCreation::new(None, Some(""), "Author").unwrap())68 .create_section(
69 "Archive",
70 "Inner",
71 &onestore::PageCreation::new(None, Some(""), "Author").unwrap()
72 )
63 .unwrap(),73 .unwrap(),
64 "Archive/Inner.one"74 "Archive/Inner.one"
65 );75 );
...@@ -171,3 +181,156 @@ fn copy_dir(from: &std::path::Path, to: &std::path::Path) {...@@ -171,3 +181,156 @@ fn copy_dir(from: &std::path::Path, to: &std::path::Path) {
171 }181 }
172 }182 }
173}183}
184
185#[test]
186fn a_rename_keeps_its_place_in_the_notebook_and_in_a_group() {
187 let temporary = tempfile::tempdir().unwrap();
188 let root = temporary.path().join("notebook");
189 let page = onestore::PageCreation::new(None, Some(""), "Author").unwrap();
190 let mut notebook =
191 Notebook::create(&root, temporary.path().join("cache"), 0x00d7ff, &page).unwrap();
192 notebook.create_group("", "Group").unwrap();
193 for folder in ["", "Group"] {
194 for name in ["Video", "Song", "Album"] {
195 notebook.create_section(folder, name, &page).unwrap();
196 }
197 }
198 let path = |folder: &str, name: &str| {
199 if folder.is_empty() {
200 format!("{name}.one")
201 } else {
202 format!("{folder}/{name}.one")
203 }
204 };
205 notebook
206 .reorder(
207 "",
208 &["Album.one", "Video.one", "New Section 1.one", "Song.one"],
209 )
210 .unwrap();
211 notebook
212 .reorder("Group", &["Group/Album.one", "Group/Video.one"])
213 .unwrap();
214 notebook.rename("Video.one", "Kitchen").unwrap();
215 notebook.rename("Group/Video.one", "Kitchen").unwrap();
216 notebook.rename("Group", "Renamed").unwrap();
217 assert_eq!(
218 names(notebook.catalog()),
219 (
220 ["Album", "Kitchen", "New Section 1", "Song"]
221 .map(|name| path("", name))
222 .to_vec(),
223 vec!["Renamed".to_owned()]
224 )
225 );
226 assert_eq!(
227 names(&notebook.catalog().groups[0]).0,
228 ["Album", "Kitchen", "Song"].map(|name| path("Renamed", name))
229 );
230}
231
232/// Each entry a TOC file lists: its filename and file identity.
233fn listed(path: &std::path::Path) -> Vec<(String, [u8; 16])> {
234 let image = onestore::read_file(path).unwrap();
235 let store = onestore::Store::parse(&image).unwrap();
236 let index = onestore::RevisionIndex::parse(&store).unwrap();
237 let document = onestore::document::Document::parse(&index).unwrap();
238 let revision = document.active(document.root).unwrap();
239 let onestore::document::Kind::Toc { entries, .. } = &revision.nodes[&revision.roots[&1]].kind
240 else {
241 panic!()
242 };
243 entries
244 .iter()
245 .map(|id| match &revision.nodes[id].kind {
246 onestore::document::Kind::Toc {
247 filename, identity, ..
248 } => (filename.clone().unwrap_or_default(), identity.unwrap()),
249 _ => panic!(),
250 })
251 .collect()
252}
253
254fn file_id(path: &std::path::Path) -> [u8; 16] {
255 onestore::Store::parse(&onestore::read_file(path).unwrap())
256 .unwrap()
257 .header
258 .file_id
259}
260
261#[test]
262fn a_page_delete_lists_the_recycle_bin_it_finds_or_makes_and_leaves_an_unreadable_one() {
263 use std::os::unix::fs::PermissionsExt;
264 let temporary = tempfile::tempdir().unwrap();
265 let root = temporary.path().join("notebook");
266 let bin = root.join("OneNote_RecycleBin");
267 let page = onestore::PageCreation::new(None, Some("Kept"), "Author").unwrap();
268 let mut notebook =
269 Notebook::create(&root, temporary.path().join("cache"), 0x00d7ff, &page).unwrap();
270 let recycle = |notebook: &mut Notebook| {
271 let image = notebook.read_section("New Section 1.one").unwrap();
272 let pages = notebook::session::stored_pages(&image).unwrap();
273 notebook.recycle_pages(&[pages[0].page.clone()], "Author")
274 };
275 let bins = |root: &std::path::Path| -> Vec<[u8; 16]> {
276 listed(&root.join("Open Notebook.onetoc2"))
277 .into_iter()
278 .filter(|(name, _)| name == "OneNote_RecycleBin")
279 .map(|(_, identity)| identity)
280 .collect()
281 };
282 recycle(&mut notebook).unwrap();
283
284 // Gone from the folder, the bin's entry is stale; the next bin takes its name.
285 std::fs::remove_dir_all(&bin).unwrap();
286 notebook.refresh().unwrap();
287 recycle(&mut notebook).unwrap();
288 let toc = bin.join("Open Notebook.onetoc2");
289 assert_eq!(bins(&root), [file_id(&toc)]);
290 let deleted = file_id(&bin.join("OneNote_DeletedPages.one"));
291 assert_eq!(listed(&toc), [("OneNote_DeletedPages.one".into(), deleted)]);
292
293 // A bin whose TOC has another name and misses the deleted pages is kept and completed.
294 std::fs::remove_file(&toc).unwrap();
295 let other = bin.join("OneNote Table Of Contents.onetoc2");
296 std::fs::write(
297 &other,
298 onestore::create_table_of_contents("OneNote Table Of Contents.onetoc2", &[]).unwrap(),
299 )
300 .unwrap();
301 let parent = file_id(&root.join("Open Notebook.onetoc2"));
302 onestore::place_file(&other, parent, "OneNote_RecycleBin").unwrap();
303 notebook.refresh().unwrap();
304 recycle(&mut notebook).unwrap();
305 assert!(!toc.exists());
306 assert_eq!(bins(&root), [file_id(&other)]);
307 assert_eq!(
308 listed(&other),
309 [("OneNote_DeletedPages.one".into(), deleted)]
310 );
311 let ancestor =
312 onestore::Store::parse(&onestore::read_file(bin.join("OneNote_DeletedPages.one")).unwrap())
313 .unwrap()
314 .header
315 .ancestor;
316 assert_eq!(ancestor, file_id(&other));
317 // `NOTEBOOK_RECYCLE_EXPORT` names a new directory receiving it for a cold reopen.
318 if let Some(directory) = std::env::var_os("NOTEBOOK_RECYCLE_EXPORT") {
319 copy_dir(&root, std::path::Path::new(&directory));
320 }
321 // Permission bits do not bind the superuser.
322 if unsafe { libc::geteuid() } == 0 {
323 return;
324 }
325
326 // A bin that cannot be read is neither replaced nor added to.
327 std::fs::set_permissions(&bin, std::fs::Permissions::from_mode(0o000)).unwrap();
328 let denied = notebook.refresh().and_then(|_| recycle(&mut notebook));
329 std::fs::set_permissions(&bin, std::fs::Permissions::from_mode(0o755)).unwrap();
330 assert!(denied.is_err());
331 let names: Vec<_> = std::fs::read_dir(&bin)
332 .unwrap()
333 .map(|entry| entry.unwrap().file_name().into_string().unwrap())
334 .collect();
335 assert_eq!(names.len(), 2, "{names:?}");
336}
crates/notebook/tests/support/model_schedule.rs+13-19
...@@ -3,6 +3,7 @@...@@ -3,6 +3,7 @@
3//! pages the merges leave.3//! pages the merges leave.
44
5use crate::model_ops::{self, AUTHOR};5use crate::model_ops::{self, AUTHOR};
6use crate::ops;
6use crate::server::{Fault, Server, remote_snapshot, snapshot};7use crate::server::{Fault, Server, remote_snapshot, snapshot};
7use notebook::{EditStatus, Remote, Replica};8use notebook::{EditStatus, Remote, Replica};
8use onestore::{9use onestore::{
...@@ -12,9 +13,6 @@ use onestore::{...@@ -12,9 +13,6 @@ use onestore::{
12};13};
13use std::{io, sync::LazyLock};14use std::{io, sync::LazyLock};
1415
15#[path = "../../../onestore/tests/support/ops.rs"]
16mod ops;
17
18static SOURCE: LazyLock<(Vec<u8>, ExGuid)> = LazyLock::new(|| {16static SOURCE: LazyLock<(Vec<u8>, ExGuid)> = LazyLock::new(|| {
19 let source =17 let source =
20 onestore::create_section("model-schedule.one", "Original 🦀 é 0", "Author").unwrap();18 onestore::create_section("model-schedule.one", "Original 🦀 é 0", "Author").unwrap();
...@@ -282,13 +280,16 @@ pub fn run(input: &[u8]) {...@@ -282,13 +280,16 @@ pub fn run(input: &[u8]) {
282 let next = cache.pending().unwrap();280 let next = cache.pending().unwrap();
283 // Edits leave the queue oldest first, each with a durable receipt; a rebase281 // Edits leave the queue oldest first, each with a durable receipt; a rebase
284 // queues the conflict pages it makes after them.282 // queues the conflict pages it makes after them.
285 let queued = |edit: &notebook::PendingEdit| pending.iter().any(|kept| kept.id == edit.id);283 let queued =
284 |edit: &notebook::PendingEdit| pending.iter().any(|kept| kept.id == edit.id);
286 let left = pending.len() - next.iter().filter(|edit| queued(edit)).count();285 let left = pending.len() - next.iter().filter(|edit| queued(edit)).count();
287 assert!(next.iter().all(|edit| {286 assert!(next.iter().all(|edit| {
288 pending[left..].iter().any(|kept| kept.id == edit.id)287 pending[left..].iter().any(|kept| kept.id == edit.id)
289 || matches!(288 || matches!(
290 &edit.edit.ops[..],289 &edit.edit.ops[..],
291 [onestore::op::Op::Section(onestore::op::SectionOp::Conflict { .. })]290 [onestore::op::Op::Section(
291 onestore::op::SectionOp::Conflict { .. }
292 )]
292 )293 )
293 }));294 }));
294 for edit in &pending[..left] {295 for edit in &pending[..left] {
...@@ -301,24 +302,18 @@ pub fn run(input: &[u8]) {...@@ -301,24 +302,18 @@ pub fn run(input: &[u8]) {
301 if next.is_empty() && result.is_ok() {302 if next.is_empty() && result.is_ok() {
302 assert_eq!(303 assert_eq!(
303 local(cache),304 local(cache),
304 shape(&model_ops::page_of(305 shape(&model_ops::page_of(&remote_snapshot(cache), *space))
305 &remote_snapshot(cache),
306 *space
307 ))
308 );306 );
309 }307 }
310 }308 }
311 5 => {309 5 => {
312 let remote = rows(&server.visible);310 let remote = rows(&server.visible);
313 let row = remote[usize::from(step[2]) % remote.len()].clone();311 let row = remote[usize::from(step[2]) % remote.len()].clone();
314 if !row.2.is_empty() {312 let letter = char::from(b'a' + step[3] % 26);
313 // Retyping the letter already there changes nothing, so stores no revision.
314 if row.2.chars().next().is_some_and(|first| first != letter) {
315 let mut page = model_ops::page_of(&server.visible, *space);315 let mut page = model_ops::page_of(&server.visible, *space);
316 model_ops::replace_text(316 model_ops::replace_text(&mut page, row.1, 0..1, &letter.to_string());
317 &mut page,
318 row.1,
319 0..1,
320 &char::from(b'a' + step[3] % 26).to_string(),
321 );
322 let visible = server.visible.clone();317 let visible = server.visible.clone();
323 ops::save(&visible, *space, &page)318 ops::save(&visible, *space, &page)
324 .unwrap()319 .unwrap()
...@@ -334,9 +329,8 @@ pub fn run(input: &[u8]) {...@@ -334,9 +329,8 @@ pub fn run(input: &[u8]) {
334 .find(|(page, _)| page == space)329 .find(|(page, _)| page == space)
335 .map(|(_, pages)| pages[usize::from(step[2]) % pages.len()].space)330 .map(|(_, pages)| pages[usize::from(step[2]) % pages.len()].space)
336 {331 {
337 let delete = onestore::op::Op::Section(onestore::op::SectionOp::Delete(vec![332 let delete =
338 conflict,333 onestore::op::Op::Section(onestore::op::SectionOp::Delete(vec![conflict]));
339 ]));
340 cache334 cache
341 .apply(335 .apply(
342 AUTHOR,336 AUTHOR,
crates/notebook/tests/support/page_schedule.rs+4-8
...@@ -1,5 +1,5 @@...@@ -1,5 +1,5 @@
1use crate::{1use crate::{
2 disk, model_ops,2 disk, model_ops, ops,
3 server::{remote_snapshot, snapshot},3 server::{remote_snapshot, snapshot},
4};4};
5use notebook::{Remote, Replica};5use notebook::{Remote, Replica};
...@@ -14,9 +14,6 @@ use onestore::{...@@ -14,9 +14,6 @@ use onestore::{
14};14};
15use std::{io, sync::LazyLock};15use std::{io, sync::LazyLock};
1616
17#[path = "../../../onestore/tests/support/ops.rs"]
18mod ops;
19
20const BODY: &str = "Body 🦋 é";17const BODY: &str = "Body 🦋 é";
2118
22#[path = "../../../onestore/tests/support/current.rs"]19#[path = "../../../onestore/tests/support/current.rs"]
...@@ -242,10 +239,9 @@ pub fn run(input: &[u8]) {...@@ -242,10 +239,9 @@ pub fn run(input: &[u8]) {
242 .collect();239 .collect();
243 let expected = ops::section_op(&image, SectionOp::Pages(edits.to_vec()));240 let expected = ops::section_op(&image, SectionOp::Pages(edits.to_vec()));
244 match section_op(cache, onestore::op::SectionOp::Pages(edits)) {241 match section_op(cache, onestore::op::SectionOp::Pages(edits)) {
245 Ok(_) => assert_eq!(242 Ok(_) => {
246 pages(&snapshot(cache)),243 assert_eq!(pages(&snapshot(cache)), pages(expected.unwrap().as_bytes()))
247 pages(expected.unwrap().as_bytes())244 }
248 ),
249 Err(_) => {245 Err(_) => {
250 assert!(expected.is_err());246 assert!(expected.is_err());
251 assert_eq!(pages(&snapshot(cache)), listed);247 assert_eq!(pages(&snapshot(cache)), listed);
crates/notebook/tests/sync.rs+23-7
...@@ -57,8 +57,7 @@ fn an_unchanged_stamp_publishes_and_settles_without_reading_the_remote() {...@@ -57,8 +57,7 @@ fn an_unchanged_stamp_publishes_and_settles_without_reading_the_remote() {
57 assert_eq!(cache.sync_once(&mut remote).unwrap().edit, None);57 assert_eq!(cache.sync_once(&mut remote).unwrap().edit, None);
58 assert_eq!(remote.reads, 0);58 assert_eq!(remote.reads, 0);
59 // Another writer's commit moves the header, so the next step reads the file.59 // Another writer's commit moves the header, so the next step reads the file.
60 let native =60 let native = typed(&remote.server.visible, sid, object, 0..0, "Native ");
61 typed(&remote.server.visible, sid, object, 0..0, "Native ");
62 remote.server.visible.clone_from(&native);61 remote.server.visible.clone_from(&native);
63 assert_eq!(cache.sync_once(&mut remote).unwrap().edit, None);62 assert_eq!(cache.sync_once(&mut remote).unwrap().edit, None);
64 assert_eq!(remote.reads, 1);63 assert_eq!(remote.reads, 1);
...@@ -252,7 +251,10 @@ fn overlapping_changes_keep_both_versions_and_survive_restart() {...@@ -252,7 +251,10 @@ fn overlapping_changes_keep_both_versions_and_survive_restart() {
252 assert_eq!(content(&server.durable, oid), "aRc");251 assert_eq!(content(&server.durable, oid), "aRc");
253 assert_eq!(252 assert_eq!(
254 conflicts(&server.durable),253 conflicts(&server.durable),
255 [(sid, vec![(model_ops::AUTHOR.to_owned(), vec!["aLc".to_owned()])])]254 [(
255 sid,
256 vec![(model_ops::AUTHOR.to_owned(), vec!["aLc".to_owned()])]
257 )]
256 );258 );
257 assert_eq!(snapshot(&cache), server.durable);259 assert_eq!(snapshot(&cache), server.durable);
258 assert!(cache.pending().unwrap().is_empty());260 assert!(cache.pending().unwrap().is_empty());
...@@ -635,7 +637,10 @@ fn edits_after_a_conflict_apply_to_the_remote_version_and_survive_reopen() {...@@ -635,7 +637,10 @@ fn edits_after_a_conflict_apply_to_the_remote_version_and_survive_reopen() {
635 assert_eq!(content(&server.durable, oid), expected);637 assert_eq!(content(&server.durable, oid), expected);
636 assert_eq!(638 assert_eq!(
637 conflicts(&server.durable),639 conflicts(&server.durable),
638 [(sid, vec![(model_ops::AUTHOR.to_owned(), vec!["aLc".to_owned()])])]640 [(
641 sid,
642 vec![(model_ops::AUTHOR.to_owned(), vec!["aLc".to_owned()])]
643 )]
639 );644 );
640 assert!(cache.pending().unwrap().is_empty());645 assert!(cache.pending().unwrap().is_empty());
641}646}
...@@ -714,7 +719,11 @@ fn seeded_conflicts_keep_the_remote_text_and_later_edits_publish_once() {...@@ -714,7 +719,11 @@ fn seeded_conflicts_keep_the_remote_text_and_later_edits_publish_once() {
714 ),719 ),
715 "case {case}, seed {seed}"720 "case {case}, seed {seed}"
716 );721 );
717 assert_eq!(content(&server.durable, oid), remote_text, "case {case}, seed {seed}");722 assert_eq!(
723 content(&server.durable, oid),
724 remote_text,
725 "case {case}, seed {seed}"
726 );
718 assert_eq!(727 assert_eq!(
719 conflicts(&server.durable),728 conflicts(&server.durable),
720 [(sid, vec![(model_ops::AUTHOR.to_owned(), vec![local])])],729 [(sid, vec![(model_ops::AUTHOR.to_owned(), vec![local])])],
...@@ -729,7 +738,11 @@ fn seeded_conflicts_keep_the_remote_text_and_later_edits_publish_once() {...@@ -729,7 +738,11 @@ fn seeded_conflicts_keep_the_remote_text_and_later_edits_publish_once() {
729 let mut expected = original.clone();738 let mut expected = original.clone();
730 expected.splice(at..at + 1, "λ🐕μ".chars());739 expected.splice(at..at + 1, "λ🐕μ".chars());
731 let expected = prefix + &expected.iter().collect::<String>() + &suffix;740 let expected = prefix + &expected.iter().collect::<String>() + &suffix;
732 assert_eq!(content(&server.durable, oid), expected, "case {case}, seed {seed}");741 assert_eq!(
742 content(&server.durable, oid),
743 expected,
744 "case {case}, seed {seed}"
745 );
733 assert_eq!(server.publications, 2);746 assert_eq!(server.publications, 2);
734 assert!(cache.pending().unwrap().is_empty());747 assert!(cache.pending().unwrap().is_empty());
735 }748 }
...@@ -919,7 +932,10 @@ fn a_remote_restore_keeps_local_work_on_a_conflict_page() {...@@ -919,7 +932,10 @@ fn a_remote_restore_keeps_local_work_on_a_conflict_page() {
919 conflicts(&server.durable),932 conflicts(&server.durable),
920 [(933 [(
921 space,934 space,
922 vec![(model_ops::AUTHOR.to_owned(), vec!["Revised Original".to_owned()])]935 vec![(
936 model_ops::AUTHOR.to_owned(),
937 vec!["Revised Original".to_owned()]
938 )]
923 )]939 )]
924 );940 );
925 drop(cache);941 drop(cache);
crates/notebook/tests/sync_formatting.rs+14-8
...@@ -83,11 +83,7 @@ fn formats(page: &Page, text: ExGuid) -> Vec<Format> {...@@ -83,11 +83,7 @@ fn formats(page: &Page, text: ExGuid) -> Vec<Format> {
83fn remote_with(space: ExGuid, change: impl FnOnce(&mut Page)) -> Server {83fn remote_with(space: ExGuid, change: impl FnOnce(&mut Page)) -> Server {
84 let mut page = page_of(OUTLINES, space);84 let mut page = page_of(OUTLINES, space);
85 change(&mut page);85 change(&mut page);
86 Server::new(86 Server::new(ops::saved(OUTLINES, space, &page).unwrap().as_slice())
87 ops::saved(OUTLINES, space, &page)
88 .unwrap()
89 .as_slice(),
90 )
91}87}
9288
93fn cache(directory: &tempfile::TempDir) -> Replica {89fn cache(directory: &tempfile::TempDir) -> Replica {
...@@ -157,7 +153,11 @@ fn offline_formatting_conflicts_only_with_remote_changes_it_overlaps() {...@@ -157,7 +153,11 @@ fn offline_formatting_conflicts_only_with_remote_changes_it_overlaps() {
157 assert!(matches!(outcome, Some((_, EditStatus::Published { .. }))));153 assert!(matches!(outcome, Some((_, EditStatus::Published { .. }))));
158 assert!(conflicted(&server.durable, space));154 assert!(conflicted(&server.durable, space));
159 let remote = page_of(&server.durable, space);155 let remote = page_of(&server.durable, space);
160 assert!(formats(&remote, ids[0]).iter().all(|format| format.bold != Some(true)));156 assert!(
157 formats(&remote, ids[0])
158 .iter()
159 .all(|format| format.bold != Some(true))
160 );
161 assert_eq!(pages(&snapshot(&cache)), pages(&server.durable));161 assert_eq!(pages(&snapshot(&cache)), pages(&server.durable));
162 assert_ne!(pages(&local), pages(&server.durable));162 assert_ne!(pages(&local), pages(&server.durable));
163 }163 }
...@@ -302,7 +302,6 @@ fn seeded_formatting_reconciles_exactly_when_the_remote_left_the_paragraph_alone...@@ -302,7 +302,6 @@ fn seeded_formatting_reconciles_exactly_when_the_remote_left_the_paragraph_alone
302 });302 });
303 let remote_page = page_of(&server.durable, space);303 let remote_page = page_of(&server.durable, space);
304 let outcome = cache.sync_once(&mut server).unwrap().edit.unwrap();304 let outcome = cache.sync_once(&mut server).unwrap().edit.unwrap();
305 assert_eq!(outcome.0, id, "seed {seed}");
306 // Bullets the remote typed before the target's text, and whether it underlined a305 // Bullets the remote typed before the target's text, and whether it underlined a
307 // character the target already had.306 // character the target already had.
308 let mut bullets = vec![0_u32; ids.len()];307 let mut bullets = vec![0_u32; ids.len()];
...@@ -316,13 +315,20 @@ fn seeded_formatting_reconciles_exactly_when_the_remote_left_the_paragraph_alone...@@ -316,13 +315,20 @@ fn seeded_formatting_reconciles_exactly_when_the_remote_left_the_paragraph_alone
316 }315 }
317 if underlined[target] && start == 0 {316 if underlined[target] && start == 0 {
318 conflicts += 1;317 conflicts += 1;
319 assert!(matches!(outcome.1, EditStatus::Published { .. }), "seed {seed}");318 // The step names its batch by the newest edit: the conflict page the rebase queued.
319 assert!(outcome.0 > id, "seed {seed}");
320 assert!(
321 matches!(outcome.1, EditStatus::Published { .. }),
322 "seed {seed}"
323 );
324 assert_eq!(cache.status(id).unwrap(), Some(outcome.1), "seed {seed}");
320 assert!(conflicted(&server.durable, space), "seed {seed}");325 assert!(conflicted(&server.durable, space), "seed {seed}");
321 assert_eq!(page_of(&server.durable, space), remote_page, "seed {seed}");326 assert_eq!(page_of(&server.durable, space), remote_page, "seed {seed}");
322 assert_ne!(pages(&snapshot(&cache)), pages(&local), "seed {seed}");327 assert_ne!(pages(&snapshot(&cache)), pages(&local), "seed {seed}");
323 continue;328 continue;
324 }329 }
325 published += 1;330 published += 1;
331 assert_eq!(outcome.0, id, "seed {seed}");
326 assert!(332 assert!(
327 matches!(outcome.1, EditStatus::Published { .. }),333 matches!(outcome.1, EditStatus::Published { .. }),
328 "seed {seed}: {outcome:?}"334 "seed {seed}: {outcome:?}"
crates/notebook/tests/sync_outline.rs+4-1
...@@ -301,7 +301,10 @@ fn competing_layout_keeps_the_local_one_on_a_conflict_page() {...@@ -301,7 +301,10 @@ fn competing_layout_keeps_the_local_one_on_a_conflict_page() {
301 let mut section = onestore::Section::open(&arena, server.durable.clone()).unwrap();301 let mut section = onestore::Section::open(&arena, server.durable.clone()).unwrap();
302 let conflict = section.conflicts().unwrap()[0].1[0].space;302 let conflict = section.conflicts().unwrap()[0].1[0].space;
303 let kept = section.page(conflict).unwrap();303 let kept = section.page(conflict).unwrap();
304 assert_eq!(body_outlines(&kept)[0].layout, outline_of(&local, outline).layout);304 assert_eq!(
305 body_outlines(&kept)[0].layout,
306 outline_of(&local, outline).layout
307 );
305 let dependent = save(&cache, text_id, |page| {308 let dependent = save(&cache, text_id, |page| {
306 replace_text(page, text_id, 0..0, "Local ")309 replace_text(page, text_id, 0..0, "Local ")
307 })310 })
crates/notebook/tests/sync_page.rs+4-1
...@@ -280,7 +280,10 @@ fn a_moved_page_anchor_places_the_page_without_regenerating_dependent_identities...@@ -280,7 +280,10 @@ fn a_moved_page_anchor_places_the_page_without_regenerating_dependent_identities
280 .pages()280 .pages()
281 .unwrap();281 .unwrap();
282 // The remote made `child` and `grandchild` subpages; `Renamed` stayed.282 // The remote made `child` and `grandchild` subpages; `Renamed` stayed.
283 let at = listed.iter().position(|listed| listed.0 == pages[6].0).unwrap();283 let at = listed
284 .iter()
285 .position(|listed| listed.0 == pages[6].0)
286 .unwrap();
284 assert_eq!(listed[at - 1].0, page.space());287 assert_eq!(listed[at - 1].0, page.space());
285 let store = Store::parse(&server.durable).unwrap();288 let store = Store::parse(&server.durable).unwrap();
286 let index = RevisionIndex::parse(&store).unwrap();289 let index = RevisionIndex::parse(&store).unwrap();
crates/notebook/tests/sync_pages.rs+101-34
...@@ -143,10 +143,7 @@ fn atomic_page_batches_survive_reopen_with_dependent_text() {...@@ -143,10 +143,7 @@ fn atomic_page_batches_survive_reopen_with_dependent_text() {
143 drop(cache);143 drop(cache);
144 let cache = Replica::open(&path).unwrap();144 let cache = Replica::open(&path).unwrap();
145 assert_eq!(cache.pending().unwrap(), queue);145 assert_eq!(cache.pending().unwrap(), queue);
146 assert_eq!(146 assert_eq!(server::pages(&snapshot(&cache)), server::pages(&local));
147 server::pages(&snapshot(&cache)),
148 server::pages(&local)
149 );
150 let mut server = Server::new(SOURCE);147 let mut server = Server::new(SOURCE);
151 assert!(matches!(148 assert!(matches!(
152 cache.sync_once(&mut server).unwrap().edit,149 cache.sync_once(&mut server).unwrap().edit,
...@@ -178,7 +175,10 @@ fn indenting_a_page_the_remote_moved_keeps_the_remote_placement() {...@@ -178,7 +175,10 @@ fn indenting_a_page_the_remote_moved_keeps_the_remote_placement() {
178 SectionOp::Pages([PageEdit::set_level(sid, 3).unwrap()].to_vec()),175 SectionOp::Pages([PageEdit::set_level(sid, 3).unwrap()].to_vec()),
179 );176 );
180 let mut server = Server::new(SOURCE);177 let mut server = Server::new(SOURCE);
181 ops::section_op(SOURCE, SectionOp::Pages([PageEdit::move_to(sid, Some(before[7].0), 2).unwrap()].to_vec()))178 ops::section_op(
179 SOURCE,
180 SectionOp::Pages([PageEdit::move_to(sid, Some(before[7].0), 2).unwrap()].to_vec()),
181 )
182 .unwrap()182 .unwrap()
183 .commit(&mut server)183 .commit(&mut server)
184 .unwrap();184 .unwrap();
...@@ -196,9 +196,15 @@ fn competing_page_moves_keep_the_remote_placement() {...@@ -196,9 +196,15 @@ fn competing_page_moves_keep_the_remote_placement() {
196 let cache = Replica::create(directory.path().join("pages.sqlite"), SOURCE).unwrap();196 let cache = Replica::create(directory.path().join("pages.sqlite"), SOURCE).unwrap();
197 let pages = order(SOURCE);197 let pages = order(SOURCE);
198 let sid = pages[6].0;198 let sid = pages[6].0;
199 let id = section_op(&cache, SectionOp::Pages(vec![PageEdit::move_to(sid, None, 1).unwrap()]));199 let id = section_op(
200 &cache,
201 SectionOp::Pages(vec![PageEdit::move_to(sid, None, 1).unwrap()]),
202 );
200 let mut server = Server::new(SOURCE);203 let mut server = Server::new(SOURCE);
201 ops::section_op(SOURCE, SectionOp::Pages([PageEdit::move_to(sid, Some(pages[0].0), 1).unwrap()].to_vec()))204 ops::section_op(
205 SOURCE,
206 SectionOp::Pages([PageEdit::move_to(sid, Some(pages[0].0), 1).unwrap()].to_vec()),
207 )
202 .unwrap()208 .unwrap()
203 .commit(&mut server)209 .commit(&mut server)
204 .unwrap();210 .unwrap();
...@@ -219,12 +225,18 @@ fn a_competing_level_keeps_the_remote_level_and_the_rest_of_the_batch() {...@@ -219,12 +225,18 @@ fn a_competing_level_keeps_the_remote_level_and_the_rest_of_the_batch() {
219 ];225 ];
220 let id = section_op(&cache, SectionOp::Pages(edits.clone()));226 let id = section_op(&cache, SectionOp::Pages(edits.clone()));
221 let mut server = Server::new(SOURCE);227 let mut server = Server::new(SOURCE);
222 ops::section_op(SOURCE, SectionOp::Pages([PageEdit::set_level(pages[5].0, 1).unwrap()].to_vec()))228 ops::section_op(
223 .unwrap()229 SOURCE,
224 .commit(&mut server)230 SectionOp::Pages([PageEdit::set_level(pages[5].0, 1).unwrap()].to_vec()),
225 .unwrap();231 )
226 let expected =232 .unwrap()
227 ops::section_op(&server.durable.clone(), SectionOp::Pages(edits[..1].to_vec())).unwrap();233 .commit(&mut server)
234 .unwrap();
235 let expected = ops::section_op(
236 &server.durable.clone(),
237 SectionOp::Pages(edits[..1].to_vec()),
238 )
239 .unwrap();
228 assert!(matches!(cache.sync_once(&mut server).unwrap().edit,240 assert!(matches!(cache.sync_once(&mut server).unwrap().edit,
229 Some((published, EditStatus::Published { .. })) if published == id));241 Some((published, EditStatus::Published { .. })) if published == id));
230 assert_eq!(order(&server.durable), order(expected.as_bytes()));242 assert_eq!(order(&server.durable), order(expected.as_bytes()));
...@@ -367,7 +379,10 @@ fn an_explicit_anchor_yields_to_a_remote_move() {...@@ -367,7 +379,10 @@ fn an_explicit_anchor_yields_to_a_remote_move() {
367 ]),379 ]),
368 );380 );
369 let mut server = Server::new(SOURCE);381 let mut server = Server::new(SOURCE);
370 ops::section_op(SOURCE, SectionOp::Pages([PageEdit::move_to(pages[6].0, Some(pages[0].0), 1).unwrap()].to_vec()))382 ops::section_op(
383 SOURCE,
384 SectionOp::Pages([PageEdit::move_to(pages[6].0, Some(pages[0].0), 1).unwrap()].to_vec()),
385 )
371 .unwrap()386 .unwrap()
372 .commit(&mut server)387 .commit(&mut server)
373 .unwrap();388 .unwrap();
...@@ -391,10 +406,13 @@ fn convergent_page_indentation_requires_confirmation_without_republishing() {...@@ -391,10 +406,13 @@ fn convergent_page_indentation_requires_confirmation_without_republishing() {
391 SectionOp::Pages([PageEdit::set_level(sid, 1).unwrap()].to_vec()),406 SectionOp::Pages([PageEdit::set_level(sid, 1).unwrap()].to_vec()),
392 );407 );
393 let mut server = Server::new(SOURCE);408 let mut server = Server::new(SOURCE);
394 ops::section_op(SOURCE, SectionOp::Pages([PageEdit::set_level(sid, 1).unwrap()].to_vec()))409 ops::section_op(
395 .unwrap()410 SOURCE,
396 .commit(&mut server)411 SectionOp::Pages([PageEdit::set_level(sid, 1).unwrap()].to_vec()),
397 .unwrap();412 )
413 .unwrap()
414 .commit(&mut server)
415 .unwrap();
398 server.fault = Fault::Confirm;416 server.fault = Fault::Confirm;
399 assert!(cache.sync_once(&mut server).is_err());417 assert!(cache.sync_once(&mut server).is_err());
400 assert_eq!(cache.status(id).unwrap(), Some(EditStatus::Pending));418 assert_eq!(cache.status(id).unwrap(), Some(EditStatus::Pending));
...@@ -451,12 +469,21 @@ fn native_page_changes_reconcile_with_atomic_offline_batches() {...@@ -451,12 +469,21 @@ fn native_page_changes_reconcile_with_atomic_offline_batches() {
451 .collect();469 .collect();
452 let placement = |image: &[u8]| -> BTreeMap<ExGuid, (ExGuid, Option<ExGuid>, u32)> {470 let placement = |image: &[u8]| -> BTreeMap<ExGuid, (ExGuid, Option<ExGuid>, u32)> {
453 let arena = onestore::Arena::default();471 let arena = onestore::Arena::default();
454 let series = onestore::Section::open(&arena, image.to_vec()).unwrap().series().unwrap();472 let series = onestore::Section::open(&arena, image.to_vec())
455 let shared: Vec<_> = order(image).into_iter().filter(|(sid, _)| both.contains(sid)).collect();473 .unwrap()
474 .series()
475 .unwrap();
476 let shared: Vec<_> = order(image)
477 .into_iter()
478 .filter(|(sid, _)| both.contains(sid))
479 .collect();
456 (0..shared.len())480 (0..shared.len())
457 .map(|at| {481 .map(|at| {
458 let (sid, level) = shared[at];482 let (sid, level) = shared[at];
459 (sid, (series[&sid], at.checked_sub(1).map(|b| shared[b].0), level))483 (
484 sid,
485 (series[&sid], at.checked_sub(1).map(|b| shared[b].0), level),
486 )
460 })487 })
461 .collect()488 .collect()
462 };489 };
...@@ -607,14 +634,21 @@ fn twelve_disjoint_page_batches_merge_with_dependent_bodies_without_review() {...@@ -607,14 +634,21 @@ fn twelve_disjoint_page_batches_merge_with_dependent_bodies_without_review() {
607/// A, offline, `moves` pages (each last, or before another) and edits `Target`'s body while634/// A, offline, `moves` pages (each last, or before another) and edits `Target`'s body while
608/// client B publishes `b-published`. The pages the merge lists after the first, and635/// client B publishes `b-published`. The pages the merge lists after the first, and
609/// `Target`'s space and texts.636/// `Target`'s space and texts.
610fn native_page_merge(capture: &str, moves: &[(&str, Option<&str>)]) -> (Vec<String>, ExGuid, Vec<String>) {637fn native_page_merge(
638 capture: &str,
639 moves: &[(&str, Option<&str>)],
640) -> (Vec<String>, ExGuid, Vec<String>) {
611 let root = std::path::Path::new(env!("CARGO_MANIFEST_DIR"))641 let root = std::path::Path::new(env!("CARGO_MANIFEST_DIR"))
612 .join("../../corpus/conflict-page")642 .join("../../corpus/conflict-page")
613 .join(capture);643 .join(capture);
614 let read = |phase: &str| std::fs::read(root.join(phase).join("notebook/synthetic.one")).unwrap();644 let read =
645 |phase: &str| std::fs::read(root.join(phase).join("notebook/synthetic.one")).unwrap();
615 let pages = |image: &[u8]| {646 let pages = |image: &[u8]| {
616 let arena = onestore::Arena::default();647 let arena = onestore::Arena::default();
617 onestore::Section::open(&arena, image.to_vec()).unwrap().pages().unwrap()648 onestore::Section::open(&arena, image.to_vec())
649 .unwrap()
650 .pages()
651 .unwrap()
618 };652 };
619 let initial = read("initial");653 let initial = read("initial");
620 let listed = pages(&initial);654 let listed = pages(&initial);
...@@ -624,7 +658,9 @@ fn native_page_merge(capture: &str, moves: &[(&str, Option<&str>)]) -> (Vec<Stri...@@ -624,7 +658,9 @@ fn native_page_merge(capture: &str, moves: &[(&str, Option<&str>)]) -> (Vec<Stri
624 for (page, before) in moves {658 for (page, before) in moves {
625 section_op(659 section_op(
626 &cache,660 &cache,
627 SectionOp::Pages(vec![PageEdit::move_to(space(page), before.map(space), 1).unwrap()]),661 SectionOp::Pages(vec![
662 PageEdit::move_to(space(page), before.map(space), 1).unwrap(),
663 ]),
628 );664 );
629 }665 }
630 let body = texts(&initial)666 let body = texts(&initial)
...@@ -650,7 +686,11 @@ fn native_page_merge(capture: &str, moves: &[(&str, Option<&str>)]) -> (Vec<Stri...@@ -650,7 +686,11 @@ fn native_page_merge(capture: &str, moves: &[(&str, Option<&str>)]) -> (Vec<Stri
650 .filter(|((page, _), _)| *page == target)686 .filter(|((page, _), _)| *page == target)
651 .map(|(_, text)| text)687 .map(|(_, text)| text)
652 .collect();688 .collect();
653 (merged[1..].iter().map(|page| page.1.clone()).collect(), target, texts)689 (
690 merged[1..].iter().map(|page| page.1.clone()).collect(),
691 target,
692 texts,
693 )
654}694}
655695
656/// Each side keeps the pages it gave a series of their own, the remote's where both did;696/// Each side keeps the pages it gave a series of their own, the remote's where both did;
...@@ -659,20 +699,47 @@ fn native_page_merge(capture: &str, moves: &[(&str, Option<&str>)]) -> (Vec<Stri...@@ -659,20 +699,47 @@ fn native_page_merge(capture: &str, moves: &[(&str, Option<&str>)]) -> (Vec<Stri
659#[test]699#[test]
660fn page_moves_and_a_removed_page_merge_as_onenote_merges_them() {700fn page_moves_and_a_removed_page_merge_as_onenote_merges_them() {
661 for (capture, moves) in [701 for (capture, moves) in [
662 ("native-pages", &[("One", Some("Target")), ("Three", Some("Target")), ("Two", None)][..]),702 (
663 ("native-restore", &[("One", None), ("Two", None), ("Target", None), ("Three", None)][..]),703 "native-pages",
704 &[
705 ("One", Some("Target")),
706 ("Three", Some("Target")),
707 ("Two", None),
708 ][..],
709 ),
710 (
711 "native-restore",
712 &[
713 ("One", None),
714 ("Two", None),
715 ("Target", None),
716 ("Three", None),
717 ][..],
718 ),
664 ("native-restore", &[("Four", Some("One"))][..]),719 ("native-restore", &[("Four", Some("One"))][..]),
665 ] {720 ] {
666 let root = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../../corpus/conflict-page");721 let root =
722 std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../../corpus/conflict-page");
667 let native: serde_json::Value =723 let native: serde_json::Value =
668 serde_json::from_slice(&std::fs::read(root.join(capture).join("merged.json")).unwrap()).unwrap();724 serde_json::from_slice(&std::fs::read(root.join(capture).join("merged.json")).unwrap())
725 .unwrap();
669 let (order, target, texts) = native_page_merge(capture, moves);726 let (order, target, texts) = native_page_merge(capture, moves);
670 let native: Vec<String> = serde_json::from_value(native["order"].clone()).unwrap();727 let native: Vec<String> = serde_json::from_value(native["order"].clone()).unwrap();
671 assert_eq!(order, native[1..], "{capture}");728 assert_eq!(order, native[1..], "{capture}");
672 assert!(texts.contains(&"Body Target edited offline.".to_owned()), "{capture}");729 assert!(
730 texts.contains(&"Body Target edited offline.".to_owned()),
731 "{capture}"
732 );
673 let arena = onestore::Arena::default();733 let arena = onestore::Arena::default();
674 let initial = std::fs::read(root.join(capture).join("initial/notebook/synthetic.one")).unwrap();734 let initial =
675 let listed = onestore::Section::open(&arena, initial).unwrap().pages().unwrap();735 std::fs::read(root.join(capture).join("initial/notebook/synthetic.one")).unwrap();
676 assert!(listed.iter().all(|page| page.0 != target), "{capture}: a new page");736 let listed = onestore::Section::open(&arena, initial)
737 .unwrap()
738 .pages()
739 .unwrap();
740 assert!(
741 listed.iter().all(|page| page.0 != target),
742 "{capture}: a new page"
743 );
677 }744 }
678}745}
crates/notebook/tests/sync_paragraph.rs+6-8
...@@ -392,7 +392,10 @@ fn a_native_text_object_replacement_keeps_a_conflict_page_for_every_offline_spli...@@ -392,7 +392,10 @@ fn a_native_text_object_replacement_keeps_a_conflict_page_for_every_offline_spli
392 "{name}"392 "{name}"
393 );393 );
394 assert!(394 assert!(
395 matches!(cache.status(id).unwrap(), Some(EditStatus::Published { .. })),395 matches!(
396 cache.status(id).unwrap(),
397 Some(EditStatus::Published { .. })
398 ),
396 "{name}"399 "{name}"
397 );400 );
398 assert_eq!(page_of(&server.visible, space), remote, "{name}");401 assert_eq!(page_of(&server.visible, space), remote, "{name}");
...@@ -483,8 +486,7 @@ fn offline_paragraphs(output: Option<&Path>) {...@@ -483,8 +486,7 @@ fn offline_paragraphs(output: Option<&Path>) {
483 };486 };
484 let prefixed = offset > 0 && !content(&page, left).text().is_empty();487 let prefixed = offset > 0 && !content(&page, left).text().is_empty();
485 if prefixed {488 if prefixed {
486 server.visible =489 server.visible = typed(&server.visible, space, left, 0..0, "\u{2602}");
487 typed(&server.visible, space, left, 0..0, "\u{2602}");
488 server.durable.clone_from(&server.visible);490 server.durable.clone_from(&server.visible);
489 }491 }
490 let Ok(queued) = save_page(&cache, left, |page| replay(page, intent, split)) else {492 let Ok(queued) = save_page(&cache, left, |page| replay(page, intent, split)) else {
...@@ -591,11 +593,7 @@ fn control(title: &str) -> (ExGuid, Page) {...@@ -591,11 +593,7 @@ fn control(title: &str) -> (ExGuid, Page) {
591fn remote_with(space: ExGuid, change: impl FnOnce(&mut Page)) -> Server {593fn remote_with(space: ExGuid, change: impl FnOnce(&mut Page)) -> Server {
592 let mut page = page_of(BEFORE, space);594 let mut page = page_of(BEFORE, space);
593 change(&mut page);595 change(&mut page);
594 Server::new(596 Server::new(ops::saved(BEFORE, space, &page).unwrap().as_slice())
595 ops::saved(BEFORE, space, &page)
596 .unwrap()
597 .as_slice(),
598 )
599}597}
600598
601#[test]599#[test]
crates/onestore/README.md+1
...@@ -61,6 +61,7 @@ harness also accepts `--client-profile release`....@@ -61,6 +61,7 @@ harness also accepts `--client-profile release`.
61| `Arena`, `Section` | Keep a section parsed across edits: `open` validates an image once; `seal` appends one revision per changed space as a `Transaction` on `stamp`, checking only what it appends; `replay` applies a queued transaction; `image` and `page` read the result |61| `Arena`, `Section` | Keep a section parsed across edits: `open` validates an image once; `seal` appends one revision per changed space as a `Transaction` on `stamp`, checking only what it appends; `replay` applies a queued transaction; `image` and `page` read the result |
62| `op::{Edit, Op, PageOp, TableEdit, SectionOp}`, `Section::apply` | Object-level edits (text, formatting, links, equations, paragraph insertion/split/join/move/deletion/levels, outlines, lists, tags, styles, paragraph formatting, tables, pictures, attachments, ink, page creation/import/moves/removal, conflict pages) applied whole or not to a kept-open section with emitter-chosen identities, UTF-16 ranges and the edit's time; refusals name the target, identity or structure at fault |62| `op::{Edit, Op, PageOp, TableEdit, SectionOp}`, `Section::apply` | Object-level edits (text, formatting, links, equations, paragraph insertion/split/join/move/deletion/levels, outlines, lists, tags, styles, paragraph formatting, tables, pictures, attachments, ink, page creation/import/moves/removal, conflict pages) applied whole or not to a kept-open section with emitter-chosen identities, UTF-16 ranges and the edit's time; refusals name the target, identity or structure at fault |
63| `SectionOp::Conflict`, `Section::conflicts`, `ConflictPage` | Keep the version of a page a merge could not take as OneNote 2010 does: a read-only conflict page (`jcidConflictPageMetaData`, `IsConflictPage`, conflict objects marked) under the page's manifest, which says it has conflict pages; list each page's, newest first, and delete one (`SectionOp::Delete`) as OneNote's Delete Conflict Page does |63| `SectionOp::Conflict`, `Section::conflicts`, `ConflictPage` | Keep the version of a page a merge could not take as OneNote 2010 does: a read-only conflict page (`jcidConflictPageMetaData`, `IsConflictPage`, conflict objects marked) under the page's manifest, which says it has conflict pages; list each page's, newest first, and delete one (`SectionOp::Delete`) as OneNote's Delete Conflict Page does |
64| `Section::versions`, `Section::version`, `PageVersion`, `SectionOp::RestoreVersion`, `SectionOp::DeleteVersions` | A page's versions as OneNote 2010 keeps them: earlier revisions of the page's own space, each current under its own context and listed by the page's version history, newest first with its author and time; read one as a page (O(section)); restore one as Restore Version does (the page's next revision builds on it and the page as it stood becomes the newest version) or unlist some as Delete Version does, their revisions staying stored ([corpus](../../corpus/page-versions/README.md)) |
64| `op::lower`, `op::lower_page` | The ops turning a range of paragraphs, or a whole page model, into another, computed from the models alone, for editors and imports |65| `op::lower`, `op::lower_page` | The ops turning a range of paragraphs, or a whole page model, into another, computed from the models alone, for editors and imports |
65| `op::predict` | The page an op leaves, as the section stores it and reading it back shows it |66| `op::predict` | The page an op leaves, as the section stores it and reading it back shows it |
66| `read_file` | Read a snapshot under whole-file exclusion |67| `read_file` | Read a snapshot under whole-file exclusion |
crates/onestore/examples/edit_property.rs+2-2
...@@ -84,7 +84,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -84,7 +84,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
84 }84 }
85 (0x20001, 0x14001cbe) => {85 (0x20001, 0x14001cbe) => {
86 if index.resolve_active(space)?.roots.get(&1) != Some(&object) {86 if index.resolve_active(space)?.roots.get(&1) != Some(&object) {
87 return Err("Only the notebook's colour, on the TOC root, changes".into());87 return Err("Only the notebook's color, on the TOC root, changes".into());
88 }88 }
89 let color = u32::from_le_bytes(value.as_slice().try_into()?);89 let color = u32::from_le_bytes(value.as_slice().try_into()?);
90 onestore::edit_table_of_contents(&bytes, &[TocEdit::Color(color)])?90 onestore::edit_table_of_contents(&bytes, &[TocEdit::Color(color)])?
...@@ -108,7 +108,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -108,7 +108,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
108 write(&args[1], &patched)?;108 write(&args[1], &patched)?;
109 return Ok(());109 return Ok(());
110 }110 }
111 _ => return Err("Only text and TOC colours change as edits; other values are patched into a copy of equal length".into()),111 _ => return Err("Only text and TOC colors change as edits; other values are patched into a copy of equal length".into()),
112 };112 };
113 let transaction = transaction.ok_or("The value is already stored")?;113 let transaction = transaction.ok_or("The value is already stored")?;
114 if args[1] == "--in-place" {114 if args[1] == "--in-place" {
crates/onestore/examples/insert.rs+5-2
...@@ -6,7 +6,9 @@ mod typing;...@@ -6,7 +6,9 @@ mod typing;
6use onestore::{6use onestore::{
7 document::{Format, Layout},7 document::{Format, Layout},
8 op::{Edit, Op, PageOp},8 op::{Edit, Op, PageOp},
9 page::{Outline, PageObject, PageParagraph, Paragraph, ParagraphContent, TextObject, text::new_id},9 page::{
10 Outline, PageObject, PageParagraph, Paragraph, ParagraphContent, TextObject, text::new_id,
11 },
10};12};
11use std::{fs, io::Write};13use std::{fs, io::Write};
1214
...@@ -87,7 +89,8 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {...@@ -87,7 +89,8 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
87 op,89 op,
88 }],90 }],
89 };91 };
90 let transaction = typing::sealed(&source, author, &edit)?.ok_or("The insertion stores nothing")?;92 let transaction =
93 typing::sealed(&source, author, &edit)?.ok_or("The insertion stores nothing")?;
91 let mut record = serde_json::json!({"edit": edit, "object": object, "text_object": text});94 let mut record = serde_json::json!({"edit": edit, "object": object, "text_object": text});
92 if args[2] == "--in-place" {95 if args[2] == "--in-place" {
93 if let Err(error) = transaction.commit_file(&args[1]) {96 if let Err(error) = transaction.commit_file(&args[1]) {
crates/onestore/examples/support/concurrent.rs+22-8
...@@ -322,7 +322,10 @@ mod tests {...@@ -322,7 +322,10 @@ mod tests {
322 TextAttribute,322 TextAttribute,
323 document::{Format, Layout},323 document::{Format, Layout},
324 op::{Edit, Op, PageOp},324 op::{Edit, Op, PageOp},
325 page::{Outline, PageObject, PageParagraph, Paragraph, ParagraphContent, TextObject, text::new_id},325 page::{
326 Outline, PageObject, PageParagraph, Paragraph, ParagraphContent, TextObject,
327 text::new_id,
328 },
326 };329 };
327330
328 fn paragraph(text: &str, level: u32) -> PageParagraph {331 fn paragraph(text: &str, level: u32) -> PageParagraph {
...@@ -335,12 +338,15 @@ mod tests {...@@ -335,12 +338,15 @@ mod tests {
335 content: ParagraphContent::Text(TextObject {338 content: ParagraphContent::Text(TextObject {
336 id: new_id().unwrap(),339 id: new_id().unwrap(),
337 date_field: None,340 date_field: None,
338 text: Paragraph::new(text.into(), Format {341 text: Paragraph::new(
339 font: Some("Calibri".into()),342 text.into(),
340 font_size: Some(11.0),343 Format {
341 language: Some(0x409),344 font: Some("Calibri".into()),
342 ..Format::default()345 font_size: Some(11.0),
343 }),346 language: Some(0x409),
347 ..Format::default()
348 },
349 ),
344 tags: Vec::new(),350 tags: Vec::new(),
345 }),351 }),
346 lists: Vec::new(),352 lists: Vec::new(),
...@@ -354,7 +360,15 @@ mod tests {...@@ -354,7 +360,15 @@ mod tests {
354 let arena = onestore::Arena::default();360 let arena = onestore::Arena::default();
355 let mut section = onestore::Section::open(&arena, image.to_vec()).unwrap();361 let mut section = onestore::Section::open(&arena, image.to_vec()).unwrap();
356 let ops = ops.into_iter().map(|op| Op::Page { space, op }).collect();362 let ops = ops.into_iter().map(|op| Op::Page { space, op }).collect();
357 section.apply("Author", &Edit { at: 134_000_000_000_000_000, ops }).unwrap();363 section
364 .apply(
365 "Author",
366 &Edit {
367 at: 134_000_000_000_000_000,
368 ops,
369 },
370 )
371 .unwrap();
358 section.seal().unwrap();372 section.seal().unwrap();
359 section.image()373 section.image()
360 }374 }
crates/onestore/src/active.rs+12-8
...@@ -58,7 +58,12 @@ impl<'a> ActivePage<'a> {...@@ -58,7 +58,12 @@ impl<'a> ActivePage<'a> {
58 let rid = index.active(space)?;58 let rid = index.active(space)?;
59 let live = LiveRevision::new(index.resolve(space, rid)?, chain_depth(index, space, rid))?;59 let live = LiveRevision::new(index.resolve(space, rid)?, chain_depth(index, space, rid))?;
60 let store: &'a Store<'a> = index.store;60 let store: &'a Store<'a> = index.store;
61 Self::viewed(std::rc::Rc::new(|guid| store.file_data(guid)), pages, live, view)61 Self::viewed(
62 std::rc::Rc::new(|guid| store.file_data(guid)),
63 pages,
64 live,
65 view,
66 )
62 }67 }
6368
64 /// The page space `live` holds, whose stored payloads `files` reads.69 /// The page space `live` holds, whose stored payloads `files` reads.
...@@ -404,21 +409,20 @@ pub(crate) mod tests {...@@ -404,21 +409,20 @@ pub(crate) mod tests {
404 choice => choice,409 choice => choice,
405 };410 };
406 match choice {411 match choice {
407 0..=3 => Insertion::paragraph(container, anchor, word, "Author").and_then(412 0..=3 => {
408 |insertion| {413 Insertion::paragraph(container, anchor, word, "Author").and_then(|insertion| {
409 let paragraph = new_id()?;414 let paragraph = new_id()?;
410 insertion.changes_as(active, paragraph, paragraph, new_id()?)415 insertion.changes_as(active, paragraph, paragraph, new_id()?)
411 },416 })
412 ),417 }
413 4 => TreeEdit::move_to(paragraph, container, anchor, "Author")418 4 => TreeEdit::move_to(paragraph, container, anchor, "Author")
414 .and_then(|edit| edit.changes(active)),419 .and_then(|edit| edit.changes(active)),
415 5 => TreeEdit::delete(paragraph, "Author").and_then(|edit| edit.changes(active)),420 5 => TreeEdit::delete(paragraph, "Author").and_then(|edit| edit.changes(active)),
416 6 => {421 6 => {
417 let at = self.pick(3) as u32;422 let at = self.pick(3) as u32;
418 let lists = (0..8).map(|_| new_id()).collect::<Result<Vec<_>>>()?;423 let lists = (0..8).map(|_| new_id()).collect::<Result<Vec<_>>>()?;
419 ParagraphSplit::new(text, at, "Author").and_then(|split| {424 ParagraphSplit::new(text, at, "Author")
420 split.changes_as(active, new_id()?, new_id()?, &lists)425 .and_then(|split| split.changes_as(active, new_id()?, new_id()?, &lists))
421 })
422 }426 }
423 _ => {427 _ => {
424 let end = self.pick(2) as u32;428 let end = self.pick(2) as u32;
crates/onestore/src/commit.rs+5-1
...@@ -360,7 +360,11 @@ impl Transaction {...@@ -360,7 +360,11 @@ impl Transaction {
360 /// last, covers bytes 0..1024.360 /// last, covers bytes 0..1024.
361 pub fn writes(&self) -> impl Iterator<Item = (u64, &[u8])> {361 pub fn writes(&self) -> impl Iterator<Item = (u64, &[u8])> {
362 std::iter::once((self.base.length, self.append.as_slice()))362 std::iter::once((self.base.length, self.append.as_slice()))
363 .chain(self.patches.iter().map(|(offset, bytes)| (*offset, bytes.as_slice())))363 .chain(
364 self.patches
365 .iter()
366 .map(|(offset, bytes)| (*offset, bytes.as_slice())),
367 )
364 .chain(std::iter::once((0, self.header.as_slice())))368 .chain(std::iter::once((0, self.header.as_slice())))
365 }369 }
366370
crates/onestore/src/create.rs+12-7
...@@ -29,7 +29,10 @@ pub(crate) fn initials(author: &str) -> String {...@@ -29,7 +29,10 @@ pub(crate) fn initials(author: &str) -> String {
2929
30/// An author object's properties as OneNote 2010 writes them: initials, then the name.30/// An author object's properties as OneNote 2010 writes them: initials, then the name.
31pub(crate) fn author_properties(name: &str) -> Vec<(u32, Vec<u8>)> {31pub(crate) fn author_properties(name: &str) -> Vec<(u32, Vec<u8>)> {
32 vec![(0x1c001df8, string(&initials(name))), (0x1c001d75, string(name))]32 vec![
33 (0x1c001df8, string(&initials(name))),
34 (0x1c001d75, string(name)),
35 ]
33}36}
3437
35pub(crate) fn default_text_style() -> Vec<(u32, Vec<u8>)> {38pub(crate) fn default_text_style() -> Vec<(u32, Vec<u8>)> {
...@@ -160,10 +163,12 @@ pub(crate) fn at<T>(filetime: u64, f: impl FnOnce() -> T) -> T {...@@ -160,10 +163,12 @@ pub(crate) fn at<T>(filetime: u64, f: impl FnOnce() -> T) -> T {
160/// Seconds since 1980 as OneNote's Time32, and FILETIME, of the writers' clock.163/// Seconds since 1980 as OneNote's Time32, and FILETIME, of the writers' clock.
161pub(crate) fn current_timestamps() -> Result<(u32, u64)> {164pub(crate) fn current_timestamps() -> Result<(u32, u64)> {
162 let now = match CLOCK.get() {165 let now = match CLOCK.get() {
163 Some(filetime) => (filetime / 10_000_000).checked_sub(11644473600).ok_or(Error {166 Some(filetime) => (filetime / 10_000_000)
164 offset: 0,167 .checked_sub(11644473600)
165 message: "An edit time precedes the Unix epoch",168 .ok_or(Error {
166 })?,169 offset: 0,
170 message: "An edit time precedes the Unix epoch",
171 })?,
167 None => SystemTime::now()172 None => SystemTime::now()
168 .duration_since(UNIX_EPOCH)173 .duration_since(UNIX_EPOCH)
169 .map_err(|_| Error {174 .map_err(|_| Error {
...@@ -200,7 +205,7 @@ pub fn create_section(file_name: &str, text: &str, author: &str) -> Result<Vec<u...@@ -200,7 +205,7 @@ pub fn create_section(file_name: &str, text: &str, author: &str) -> Result<Vec<u
200 let page_guid = fresh_guid()?;205 let page_guid = fresh_guid()?;
201 let metadata = vec![206 let metadata = vec![
202 (0x1c001c30, page_guid.to_vec()),207 (0x1c001c30, page_guid.to_vec()),
203 (0x1c001cf3, string(crate::edit::automatic_title(text))),208 (0x1c001cf3, string(&crate::edit::automatic_title(text))),
204 (0x14001d82, 40_u32.to_le_bytes().to_vec()),209 (0x14001d82, 40_u32.to_le_bytes().to_vec()),
205 (0x1400348b, 40_u32.to_le_bytes().to_vec()),210 (0x1400348b, 40_u32.to_le_bytes().to_vec()),
206 (0x14001dff, 1_u32.to_le_bytes().to_vec()),211 (0x14001dff, 1_u32.to_le_bytes().to_vec()),
...@@ -265,7 +270,7 @@ pub fn create_section(file_name: &str, text: &str, author: &str) -> Result<Vec<u...@@ -265,7 +270,7 @@ pub fn create_section(file_name: &str, text: &str, author: &str) -> Result<Vec<u
265 (0x24001c20, id(23)),270 (0x24001c20, id(23)),
266 (0x1c001d75, string(author)),271 (0x1c001d75, string(author)),
267 (0x1c001df8, string(&initials(author))),272 (0x1c001df8, string(&initials(author))),
268 (0x1c001d3c, string(crate::edit::automatic_title(text))),273 (0x1c001d3c, string(&crate::edit::automatic_title(text))),
269 ],274 ],
270 page_margins(),275 page_margins(),
271 ]276 ]
crates/onestore/src/document.rs+13-6
...@@ -897,11 +897,16 @@ impl<'a> Document<'a> {...@@ -897,11 +897,16 @@ impl<'a> Document<'a> {
897 if space.contexts.contains_key(&context) {897 if space.contexts.contains_key(&context) {
898 continue;898 continue;
899 }899 }
900 let rid = *index900 let rid = index
901 .spaces901 .spaces
902 .get(&id)902 .get(&id)
903 .and_then(|s| s.labels.get(&(context, 1)))903 .and_then(|s| s.labels.get(&(context, 1)));
904 .ok_or_else(|| invalid("Document context has no revision"))?;904 let rid = match rid {
905 Some(rid) => *rid,
906 // OneNote's copy of a page version names a history it never wrote.
907 None if context == crate::section::HISTORY => continue,
908 None => return Err(invalid("Document context has no revision")),
909 };
905 space.contexts.insert(context, rid);910 space.contexts.insert(context, rid);
906 if space.revisions.contains_key(&rid) {911 if space.revisions.contains_key(&rid) {
907 continue;912 continue;
...@@ -924,8 +929,10 @@ impl<'a> Document<'a> {...@@ -924,8 +929,10 @@ impl<'a> Document<'a> {
924 Kind::VersionProxy { context, .. } => (*context, false),929 Kind::VersionProxy { context, .. } => (*context, false),
925 _ => continue,930 _ => continue,
926 };931 };
927 let rid = space.contexts[&context];932 let Some(rid) = space.contexts.get(&context) else {
928 let revision = &space.revisions[&rid];933 continue;
934 };
935 let revision = &space.revisions[rid];
929 let root = revision.roots.get(&1).and_then(|id| revision.nodes.get(id));936 let root = revision.roots.get(&1).and_then(|id| revision.nodes.get(id));
930 if history {937 if history {
931 if !matches!(root.map(|n| &n.kind), Some(Kind::VersionHistory)) {938 if !matches!(root.map(|n| &n.kind), Some(Kind::VersionHistory)) {
...@@ -934,7 +941,7 @@ impl<'a> Document<'a> {...@@ -934,7 +941,7 @@ impl<'a> Document<'a> {
934 ));941 ));
935 }942 }
936 } else if !matches!(root.map(|n| &n.kind), Some(Kind::Manifest { .. }))943 } else if !matches!(root.map(|n| &n.kind), Some(Kind::Manifest { .. }))
937 || space.contexts.get(&ExGuid::default()) == Some(&rid)944 || space.contexts.get(&ExGuid::default()) == Some(rid)
938 {945 {
939 return Err(invalid(946 return Err(invalid(
940 "Version context does not resolve to a historical page",947 "Version context does not resolve to a historical page",
crates/onestore/src/edit.rs+29-6
...@@ -11,16 +11,39 @@ fn title_line(text: &str) -> &str {...@@ -11,16 +11,39 @@ fn title_line(text: &str) -> &str {
11 text.trim_start().split('\r').next().unwrap()11 text.trim_start().split('\r').next().unwrap()
12}12}
1313
14pub(crate) fn automatic_title(text: &str) -> &str {14/// `text` as it shows: hyperlink field instructions, which the text holds hidden, left out.
15 let line = title_line(text).trim_end();15pub(crate) fn without_fields(text: &str) -> String {
16 let mut line = String::new();
17 let mut rest = text;
18 while let Some(start) = rest.find('\u{fddf}') {
19 line.push_str(&rest[..start]);
20 rest = &rest[start..];
21 let Some(label) = rest
22 .strip_prefix("\u{fddf}HYPERLINK \"")
23 .and_then(|target| target.split_once('"'))
24 .map(|(_, label)| label)
25 else {
26 break;
27 };
28 rest = label;
29 }
30 line.push_str(rest);
31 line
32}
33
34/// The first line of `text` as OneNote titles a page after it: without hyperlink field
35/// instructions, which the text holds hidden, and at most 255 UTF-16 units long.
36pub(crate) fn automatic_title(text: &str) -> String {
37 let line = without_fields(title_line(text));
38 let line = line.trim();
16 let mut units = 0;39 let mut units = 0;
17 for (byte, character) in line.char_indices() {40 for (byte, character) in line.char_indices() {
18 if units >= 255 {41 if units >= 255 {
19 return line[..byte].trim_end();42 return line[..byte].trim_end().to_owned();
20 }43 }
21 units += character.len_utf16();44 units += character.len_utf16();
22 }45 }
23 line46 line.to_owned()
24}47}
2548
26/// Replaces UTF-16 character positions across ordinary text runs.49/// Replaces UTF-16 character positions across ordinary text runs.
...@@ -446,7 +469,7 @@ pub(crate) fn title_of<'n>(...@@ -446,7 +469,7 @@ pub(crate) fn title_of<'n>(
446 let Kind::Page { rtl, .. } = &node(*page).unwrap().kind else {469 let Kind::Page { rtl, .. } = &node(*page).unwrap().kind else {
447 unreachable!()470 unreachable!()
448 };471 };
449 let mut title_text = title_line(title_text);472 let mut title_text = without_fields(title_line(title_text));
450 if automatic {473 if automatic {
451 let mut roots = node(*page).unwrap().children.clone();474 let mut roots = node(*page).unwrap().children.clone();
452 roots.sort_by(|a, b| {475 roots.sort_by(|a, b| {
...@@ -495,7 +518,7 @@ pub(crate) fn title_of<'n>(...@@ -495,7 +518,7 @@ pub(crate) fn title_of<'n>(
495 pending.extend(element.structure.iter().rev().copied());518 pending.extend(element.structure.iter().rev().copied());
496 }519 }
497 }520 }
498 Ok(Some((*page, automatic, title_text.to_owned())))521 Ok(Some((*page, automatic, title_text)))
499}522}
500523
501/// Updates the page title for `changed` objects made on `view`, a copy of the page's view.524/// Updates the page title for `changed` objects made on `view`, a copy of the page's view.
crates/onestore/src/insertion.rs+12-2
...@@ -47,7 +47,13 @@ impl Insertion {...@@ -47,7 +47,13 @@ impl Insertion {
47 }47 }
4848
49 /// Adds an outline to an editable page at coordinates measured in points.49 /// Adds an outline to an editable page at coordinates measured in points.
50 pub(crate) fn outline(page: ExGuid, x: f32, y: f32, text: &str, author: &str) -> Result<Self, Error> {50 pub(crate) fn outline(
51 page: ExGuid,
52 x: f32,
53 y: f32,
54 text: &str,
55 author: &str,
56 ) -> Result<Self, Error> {
51 Self::new(page, Placement::Outline { x, y }, text, author)57 Self::new(page, Placement::Outline { x, y }, text, author)
52 }58 }
5359
...@@ -186,7 +192,11 @@ impl Insertion {...@@ -186,7 +192,11 @@ impl Insertion {
186 (0x10001cfe, 0x409_u16.to_le_bytes().to_vec()),192 (0x10001cfe, 0x409_u16.to_le_bytes().to_vec()),
187 ],193 ],
188 ),194 ),
189 (author, 0x120001, crate::create::author_properties(&self.author)),195 (
196 author,
197 0x120001,
198 crate::create::author_properties(&self.author),
199 ),
190 (default, 0x12004d, default_text_style()),200 (default, 0x12004d, default_text_style()),
191 ] {201 ] {
192 new.insert(202 new.insert(
crates/onestore/src/lib.rs+1-1
...@@ -44,7 +44,7 @@ pub use pages::{ConflictPage, PageCreation, PageEdit, PagePosition};...@@ -44,7 +44,7 @@ pub use pages::{ConflictPage, PageCreation, PageEdit, PagePosition};
44pub(crate) use paragraph::{ParagraphJoin, ParagraphSplit};44pub(crate) use paragraph::{ParagraphJoin, ParagraphSplit};
45pub use properties::{IdStream, Property, PropertySets, Value};45pub use properties::{IdStream, Property, PropertySets, Value};
46pub use revisions::{ExGuid, ObjectSpace, Revision, RevisionIndex};46pub use revisions::{ExGuid, ObjectSpace, Revision, RevisionIndex};
47pub use section::{Arena, Section};47pub use section::{Arena, PageVersion, Section};
48pub use snapshot::{read_snapshot, read_storage_snapshot};48pub use snapshot::{read_snapshot, read_storage_snapshot};
49pub use store::{Chunk, Error, FileType, Header, Node, NodeList, Reference, Store};49pub use store::{Chunk, Error, FileType, Header, Node, NodeList, Reference, Store};
50pub use toc::{TocEdit, edit_table_of_contents};50pub use toc::{TocEdit, edit_table_of_contents};
crates/onestore/src/objects.rs+4-1
...@@ -209,7 +209,10 @@ impl<'a> RevisionIndex<'a> {...@@ -209,7 +209,10 @@ impl<'a> RevisionIndex<'a> {
209 targets.insert((target, rid));209 targets.insert((target, rid));
210 }210 }
211 for context in references.contexts {211 for context in references.contexts {
212 if !space.labels.contains_key(&(context, 1)) {212 // OneNote's copy of a page version names a history it never wrote.
213 if !space.labels.contains_key(&(context, 1))
214 && context != crate::section::HISTORY
215 {
213 return Err(Error {216 return Err(Error {
214 offset: 0,217 offset: 0,
215 message: "Context reference has no current revision",218 message: "Context reference has no current revision",
crates/onestore/src/op/apply.rs+151-63
...@@ -15,8 +15,8 @@ use crate::{...@@ -15,8 +15,8 @@ use crate::{
15 active::{ActivePage, Changes},15 active::{ActivePage, Changes},
16 document::Kind,16 document::Kind,
17 page::{17 page::{
18 Attachment, Image, Ink, PageObject, PageParagraph, Paragraph, ParagraphContent,18 Attachment, Image, Ink, PageObject, PageParagraph, Paragraph, ParagraphContent, TableCell,
19 TableCell, TableColumn,19 TableColumn,
20 },20 },
21};21};
22use std::{22use std::{
...@@ -24,6 +24,10 @@ use std::{...@@ -24,6 +24,10 @@ use std::{
24 ops::Range,24 ops::Range,
25};25};
2626
27/// `HasConflictPages` and `HasVersionPages` (MS-ONE 2.3.71, 2.3.84), true when present.
28const HAS_CONFLICT_PAGES: u32 = 0x88001d97;
29const HAS_VERSION_PAGES: u32 = 0x88003462;
30
27fn invalid(message: &'static str) -> Error {31fn invalid(message: &'static str) -> Error {
28 Error { offset: 0, message }32 Error { offset: 0, message }
29}33}
...@@ -87,7 +91,9 @@ impl<'a> Section<'a> {...@@ -87,7 +91,9 @@ impl<'a> Section<'a> {
87 space: ExGuid,91 space: ExGuid,
88 after: &crate::page::Page,92 after: &crate::page::Page,
89 ) -> Result<Vec<PageOp>, OpError> {93 ) -> Result<Vec<PageOp>, OpError> {
90 let before = self.page(space).map_err(|error| self.classify(Failure::Rejected(error)))?;94 let before = self
95 .page(space)
96 .map_err(|error| self.classify(Failure::Rejected(error)))?;
91 let ops = super::lower_page(&before, after)97 let ops = super::lower_page(&before, after)
92 .map_err(|error| OpError::Unsupported(error.message))?;98 .map_err(|error| OpError::Unsupported(error.message))?;
93 let edit = Edit {99 let edit = Edit {
...@@ -152,7 +158,10 @@ impl<'a> Section<'a> {...@@ -152,7 +158,10 @@ impl<'a> Section<'a> {
152 let conflicts = self.conflicts()?;158 let conflicts = self.conflicts()?;
153 for space in pages {159 for space in pages {
154 let of = conflicts.iter().find_map(|(of, conflicts)| {160 let of = conflicts.iter().find_map(|(of, conflicts)| {
155 conflicts.iter().any(|page| page.space == *space).then_some(*of)161 conflicts
162 .iter()
163 .any(|page| page.space == *space)
164 .then_some(*of)
156 });165 });
157 match of {166 match of {
158 Some(of) => self.unlink_conflict(of, *space)?,167 Some(of) => self.unlink_conflict(of, *space)?,
...@@ -165,6 +174,19 @@ impl<'a> Section<'a> {...@@ -165,6 +174,19 @@ impl<'a> Section<'a> {
165 }174 }
166 }175 }
167 }176 }
177 SectionOp::RestoreVersion {
178 page,
179 version,
180 guid,
181 } => {
182 self.restore(author, *page, *version, *guid)?;
183 self.flag_metadata(*page, HAS_VERSION_PAGES, true)?;
184 }
185 SectionOp::DeleteVersions { page, versions } => {
186 if self.unlist(*page, versions)? {
187 self.flag_metadata(*page, HAS_VERSION_PAGES, false)?;
188 }
189 }
168 SectionOp::Color(color) => {190 SectionOp::Color(color) => {
169 let root = self.root();191 let root = self.root();
170 let page = self.active(root)?;192 let page = self.active(root)?;
...@@ -173,7 +195,7 @@ impl<'a> Section<'a> {...@@ -173,7 +195,7 @@ impl<'a> Section<'a> {
173 .roots195 .roots
174 .get(&2)196 .get(&2)
175 .filter(|id| page.live.revision.objects.contains_key(id))197 .filter(|id| page.live.revision.objects.contains_key(id))
176 .ok_or_else(|| invalid("The section has no metadata to colour"))?;198 .ok_or_else(|| invalid("The section has no metadata to color"))?;
177 let mut object = crate::write::PropertyObject::from_object(199 let mut object = crate::write::PropertyObject::from_object(
178 &page.live.revision.objects[&metadata],200 &page.live.revision.objects[&metadata],
179 )?;201 )?;
...@@ -185,10 +207,17 @@ impl<'a> Section<'a> {...@@ -185,10 +207,17 @@ impl<'a> Section<'a> {
185 }207 }
186208
187 /// Writes `page`'s content onto the page just created in `space`, whose title stays.209 /// Writes `page`'s content onto the page just created in `space`, whose title stays.
188 fn fill(&mut self, author: &str, space: ExGuid, page: &crate::page::Page) -> Result<(), Failure> {210 fn fill(
211 &mut self,
212 author: &str,
213 space: ExGuid,
214 page: &crate::page::Page,
215 ) -> Result<(), Failure> {
189 let before = self.page(space)?;216 let before = self.page(space)?;
190 let mut after = page.clone();217 let mut after = page.clone();
191 after.objects.retain(|object| !matches!(object, PageObject::Title(_)));218 after
219 .objects
220 .retain(|object| !matches!(object, PageObject::Title(_)));
192 for object in &mut after.objects {221 for object in &mut after.objects {
193 // Stroke points are page coordinates; new ink takes its position from them.222 // Stroke points are page coordinates; new ink takes its position from them.
194 if let PageObject::Ink(ink) = object {223 if let PageObject::Ink(ink) = object {
...@@ -298,7 +327,7 @@ impl<'a> Section<'a> {...@@ -298,7 +327,7 @@ impl<'a> Section<'a> {
298 &[],327 &[],
299 BTreeMap::from([(manifest_id, manifest), (copy, metadata)]),328 BTreeMap::from([(manifest_id, manifest), (copy, metadata)]),
300 )?;329 )?;
301 self.has_conflicts(of, true)330 self.flag_metadata(of, HAS_CONFLICT_PAGES, true)
302 }331 }
303332
304 /// Deletes conflict page `space` of page `of` as OneNote 2010 does: the manifest stops333 /// Deletes conflict page `space` of page `of` as OneNote 2010 does: the manifest stops
...@@ -322,21 +351,21 @@ impl<'a> Section<'a> {...@@ -322,21 +351,21 @@ impl<'a> Section<'a> {
322 }351 }
323 self.apply_changes(of, &[], BTreeMap::from([(manifest_id, manifest)]))?;352 self.apply_changes(of, &[], BTreeMap::from([(manifest_id, manifest)]))?;
324 if spaces.is_empty() {353 if spaces.is_empty() {
325 self.has_conflicts(of, false)?;354 self.flag_metadata(of, HAS_CONFLICT_PAGES, false)?;
326 }355 }
327 Ok(())356 Ok(())
328 }357 }
329358
330 /// Sets or clears `HasConflictPages` on page `of`'s metadata and on the section's copy359 /// Sets or clears a Boolean `property` on page `of`'s metadata and on the section's copy
331 /// of it, where its series keeps one.360 /// of it, where its series keeps one.
332 fn has_conflicts(&mut self, of: ExGuid, value: bool) -> Result<(), Failure> {361 fn flag_metadata(&mut self, of: ExGuid, property: u32, value: bool) -> Result<(), Failure> {
333 use crate::write::PropertyObject;362 use crate::write::PropertyObject;
334 let flag = |object: &crate::Object<'_>| -> Result<PropertyObject, Error> {363 let flag = |object: &crate::Object<'_>| -> Result<PropertyObject, Error> {
335 let mut object = PropertyObject::from_object(object)?;364 let mut object = PropertyObject::from_object(object)?;
336 if value {365 if value {
337 object.set(&[(0x88001d97, &[])])?;366 object.set(&[(property, &[])])?;
338 } else {367 } else {
339 object.remove(&[0x88001d97])?;368 object.remove(&[property])?;
340 }369 }
341 Ok(object)370 Ok(object)
342 };371 };
...@@ -399,7 +428,7 @@ fn stores_more_than_once(op: &Op) -> bool {...@@ -399,7 +428,7 @@ fn stores_more_than_once(op: &Op) -> bool {
399}428}
400429
401/// Why an op was not applied: a precondition it names, or what a writer refused.430/// Why an op was not applied: a precondition it names, or what a writer refused.
402enum Failure {431pub(crate) enum Failure {
403 Refused(OpError),432 Refused(OpError),
404 Rejected(Error),433 Rejected(Error),
405}434}
...@@ -524,7 +553,11 @@ impl<'a> Writer<'_, 'a> {...@@ -524,7 +553,11 @@ impl<'a> Writer<'_, 'a> {
524 (*child, target)553 (*child, target)
525 })554 })
526 .unzip();555 .unzip();
527 if current.iter().map(|(_, depth)| *depth).eq(depths.iter().copied()) {556 if current
557 .iter()
558 .map(|(_, depth)| *depth)
559 .eq(depths.iter().copied())
560 {
528 return Ok(());561 return Ok(());
529 }562 }
530 let (level, runs) = levels::runs(&children, &depths, is_cell)?;563 let (level, runs) = levels::runs(&children, &depths, is_cell)?;
...@@ -592,13 +625,15 @@ impl<'a> Writer<'_, 'a> {...@@ -592,13 +625,15 @@ impl<'a> Writer<'_, 'a> {
592 }625 }
593 PageOp::Color(color) => self.write(|page| {626 PageOp::Color(color) => self.write(|page| {
594 let [node] = crate::active::manifest_pages(&page.view)[..] else {627 let [node] = crate::active::manifest_pages(&page.view)[..] else {
595 return Err(invalid("The page has no page node to colour"));628 return Err(invalid("The page has no page node to color"));
596 };629 };
597 let mut object =630 let mut object =
598 crate::write::PropertyObject::from_object(&page.live.revision.objects[&node])?;631 crate::write::PropertyObject::from_object(&page.live.revision.objects[&node])?;
599 // OneNote's "No color" removes the property.632 // OneNote's "No color" removes the property.
600 match color {633 match color {
601 Some(color) => object.set(&[(crate::page::PAGE_COLOR, &color.to_le_bytes())])?,634 Some(color) => {
635 object.set(&[(crate::page::PAGE_COLOR, &color.to_le_bytes())])?
636 }
602 None => object.remove(&[crate::page::PAGE_COLOR])?,637 None => object.remove(&[crate::page::PAGE_COLOR])?,
603 }638 }
604 Ok(BTreeMap::from([(node, object)]))639 Ok(BTreeMap::from([(node, object)]))
...@@ -648,12 +683,9 @@ impl<'a> Writer<'_, 'a> {...@@ -648,12 +683,9 @@ impl<'a> Writer<'_, 'a> {
648 self.target(*paragraph)?;683 self.target(*paragraph)?;
649 let container = self.container(*paragraph)?;684 let container = self.container(*paragraph)?;
650 let base = self.level(container)?;685 let base = self.level(container)?;
651 let depth = level686 let depth = level.checked_sub(base).filter(|depth| *depth > 0).ok_or(
652 .checked_sub(base)687 OpError::Unsupported("A paragraph lies deeper than its parent"),
653 .filter(|depth| *depth > 0)688 )?;
654 .ok_or(OpError::Unsupported(
655 "A paragraph lies deeper than its parent",
656 ))?;
657 self.regroup(container, &[(*paragraph, depth)])689 self.regroup(container, &[(*paragraph, depth)])
658 }690 }
659 PageOp::Outline { object, edit } => {691 PageOp::Outline { object, edit } => {
...@@ -703,7 +735,9 @@ impl<'a> Writer<'_, 'a> {...@@ -703,7 +735,9 @@ impl<'a> Writer<'_, 'a> {
703 .get(id)735 .get(id)
704 .is_some_and(|owners| owners.iter().any(|owner| owner != paragraph))736 .is_some_and(|owners| owners.iter().any(|owner| owner != paragraph))
705 {737 {
706 return Err(OpError::Unsupported("A list node belongs to one paragraph").into());738 return Err(
739 OpError::Unsupported("A list node belongs to one paragraph").into()
740 );
707 }741 }
708 }742 }
709 let nodes = lists743 let nodes = lists
...@@ -833,7 +867,9 @@ impl<'a> Writer<'_, 'a> {...@@ -833,7 +867,9 @@ impl<'a> Writer<'_, 'a> {
833 let mut at = parent;867 let mut at = parent;
834 loop {868 loop {
835 if at == object {869 if at == object {
836 return Err(OpError::StructureChanged("A subtree cannot move inside itself").into());870 return Err(
871 OpError::StructureChanged("A subtree cannot move inside itself").into(),
872 );
837 }873 }
838 match self.page()?.parents.get(&at).map(Vec::as_slice) {874 match self.page()?.parents.get(&at).map(Vec::as_slice) {
839 Some([up]) => at = *up,875 Some([up]) => at = *up,
...@@ -844,9 +880,10 @@ impl<'a> Writer<'_, 'a> {...@@ -844,9 +880,10 @@ impl<'a> Writer<'_, 'a> {
844 Some(before) => {880 Some(before) => {
845 self.target(before)?;881 self.target(before)?;
846 if self.container(before)? != parent {882 if self.container(before)? != parent {
847 return Err(883 return Err(OpError::StructureChanged(
848 OpError::StructureChanged("The anchor is no child of the container").into(),884 "The anchor is no child of the container",
849 );885 )
886 .into());
850 }887 }
851 self.parent(before)?888 self.parent(before)?
852 }889 }
...@@ -877,9 +914,10 @@ impl<'a> Writer<'_, 'a> {...@@ -877,9 +914,10 @@ impl<'a> Writer<'_, 'a> {
877 Some(before) => {914 Some(before) => {
878 self.target(before)?;915 self.target(before)?;
879 if self.container(before)? != container {916 if self.container(before)? != container {
880 return Err(917 return Err(OpError::StructureChanged(
881 OpError::StructureChanged("The anchor is no child of the container").into(),918 "The anchor is no child of the container",
882 );919 )
920 .into());
883 }921 }
884 self.parent(before)?922 self.parent(before)?
885 }923 }
...@@ -920,8 +958,13 @@ impl<'a> Writer<'_, 'a> {...@@ -920,8 +958,13 @@ impl<'a> Writer<'_, 'a> {
920 .level958 .level
921 .checked_sub(parent_level)959 .checked_sub(parent_level)
922 .filter(|depth| *depth > 0)960 .filter(|depth| *depth > 0)
923 .ok_or(OpError::Unsupported("A paragraph lies deeper than its parent"))?;961 .ok_or(OpError::Unsupported(
924 targets.entry(parent).or_default().push((paragraph.id, depth));962 "A paragraph lies deeper than its parent",
963 ))?;
964 targets
965 .entry(parent)
966 .or_default()
967 .push((paragraph.id, depth));
925 self.paragraph(holder, anchor, paragraph, None)?;968 self.paragraph(holder, anchor, paragraph, None)?;
926 }969 }
927 for (parent, targets) in targets {970 for (parent, targets) in targets {
...@@ -966,13 +1009,16 @@ impl<'a> Writer<'_, 'a> {...@@ -966,13 +1009,16 @@ impl<'a> Writer<'_, 'a> {
966 match &paragraph.content {1009 match &paragraph.content {
967 ParagraphContent::Text(target) => {1010 ParagraphContent::Text(target) => {
968 if equation {1011 if equation {
969 return self.write(|page| content::equation_changes(page, text_id, &target.text));1012 return self
1013 .write(|page| content::equation_changes(page, text_id, &target.text));
970 }1014 }
971 let page = self.page()?;1015 let page = self.page()?;
972 let stored = crate::page::text_of(&page.view, &page.parents, text_id)?;1016 let stored = crate::page::text_of(&page.view, &page.parents, text_id)?;
973 let values = paragraph_fields(&stored, &target.text)?.values()?;1017 let values = paragraph_fields(&stored, &target.text)?.values()?;
974 if !values.is_empty() {1018 if !values.is_empty() {
975 self.write(|page| properties::paragraph_format_changes(page, text_id, &values))?;1019 self.write(|page| {
1020 properties::paragraph_format_changes(page, text_id, &values)
1021 })?;
976 }1022 }
977 let page = self.page()?;1023 let page = self.page()?;
978 let stored = crate::page::text_of(&page.view, &page.parents, text_id)?;1024 let stored = crate::page::text_of(&page.view, &page.parents, text_id)?;
...@@ -996,12 +1042,19 @@ impl<'a> Writer<'_, 'a> {...@@ -996,12 +1042,19 @@ impl<'a> Writer<'_, 'a> {
996 .rows1042 .rows
997 .iter()1043 .iter()
998 .flat_map(|row| &row.cells)1044 .flat_map(|row| &row.cells)
999 .map(|cell| (cell.id, (table::cell_indents(&cell.indents, None), cell.shading)))1045 .map(|cell| {
1046 (
1047 cell.id,
1048 (table::cell_indents(&cell.indents, None), cell.shading),
1049 )
1050 })
1000 .collect(),1051 .collect(),
1001 columns: &table.columns,1052 columns: &table.columns,
1002 borders: table.borders,1053 borders: table.borders,
1003 };1054 };
1004 self.write(|page| table::table_changes(page, table.id, Some(paragraph.id), &structure))?;1055 self.write(|page| {
1056 table::table_changes(page, table.id, Some(paragraph.id), &structure)
1057 })?;
1005 for cell in table.rows.iter().flat_map(|row| &row.cells) {1058 for cell in table.rows.iter().flat_map(|row| &row.cells) {
1006 self.cell(cell)?;1059 self.cell(cell)?;
1007 }1060 }
...@@ -1009,7 +1062,10 @@ impl<'a> Writer<'_, 'a> {...@@ -1009,7 +1062,10 @@ impl<'a> Writer<'_, 'a> {
1009 }1062 }
1010 ParagraphContent::Image(image) => {1063 ParagraphContent::Image(image) => {
1011 if image.layout.x.is_some() || image.layout.y.is_some() {1064 if image.layout.x.is_some() || image.layout.y.is_some() {
1012 return Err(OpError::Unsupported("A paragraph picture has no position of its own").into());1065 return Err(OpError::Unsupported(
1066 "A paragraph picture has no position of its own",
1067 )
1068 .into());
1013 }1069 }
1014 self.picture(image, Some(paragraph.id))1070 self.picture(image, Some(paragraph.id))
1015 }1071 }
...@@ -1065,7 +1121,11 @@ impl<'a> Writer<'_, 'a> {...@@ -1065,7 +1121,11 @@ impl<'a> Writer<'_, 'a> {
1065 self.free(stroke.id)?;1121 self.free(stroke.id)?;
1066 }1122 }
1067 let data = fresh()?;1123 let data = fresh()?;
1068 let strokes: Vec<_> = ink.strokes.iter().map(|stroke| (stroke.id, stroke)).collect();1124 let strokes: Vec<_> = ink
1125 .strokes
1126 .iter()
1127 .map(|stroke| (stroke.id, stroke))
1128 .collect();
1069 self.write(|page| content::ink_changes(page, ink, ink.id, data, &strokes, holder))1129 self.write(|page| content::ink_changes(page, ink, ink.id, data, &strokes, holder))
1070 }1130 }
10711131
...@@ -1104,7 +1164,9 @@ impl<'a> Writer<'_, 'a> {...@@ -1104,7 +1164,9 @@ impl<'a> Writer<'_, 'a> {
1104 .level1164 .level
1105 .checked_sub(parent_level)1165 .checked_sub(parent_level)
1106 .filter(|depth| *depth > 0)1166 .filter(|depth| *depth > 0)
1107 .ok_or(OpError::Unsupported("A paragraph lies deeper than its parent"))?;1167 .ok_or(OpError::Unsupported(
1168 "A paragraph lies deeper than its parent",
1169 ))?;
1108 self.paragraph(parent, None, paragraph, None)?;1170 self.paragraph(parent, None, paragraph, None)?;
1109 if parent == outline.id {1171 if parent == outline.id {
1110 targets.push((paragraph.id, depth));1172 targets.push((paragraph.id, depth));
...@@ -1128,8 +1190,9 @@ impl<'a> Writer<'_, 'a> {...@@ -1128,8 +1190,9 @@ impl<'a> Writer<'_, 'a> {
1128 let indents = content::measurement_bytes(&outline.indents, 4)?;1190 let indents = content::measurement_bytes(&outline.indents, 4)?;
1129 let id = outline.id;1191 let id = outline.id;
1130 self.write(|page| {1192 self.write(|page| {
1131 let mut node =1193 let mut node = crate::write::PropertyObject::from_object(
1132 crate::write::PropertyObject::from_object(&page.live.revision.objects[&id])?;1194 &page.live.revision.objects[&id],
1195 )?;
1133 node.set(&[(0x1c001c12, &indents)])?;1196 node.set(&[(0x1c001c12, &indents)])?;
1134 Ok(BTreeMap::from([(id, node)]))1197 Ok(BTreeMap::from([(id, node)]))
1135 })?;1198 })?;
...@@ -1137,13 +1200,17 @@ impl<'a> Writer<'_, 'a> {...@@ -1137,13 +1200,17 @@ impl<'a> Writer<'_, 'a> {
1137 }1200 }
1138 PageObject::Image(image) => {1201 PageObject::Image(image) => {
1139 if image.layout.x.is_none() || image.layout.y.is_none() {1202 if image.layout.x.is_none() || image.layout.y.is_none() {
1140 return Err(OpError::Unsupported("A new page-level picture needs a position").into());1203 return Err(
1204 OpError::Unsupported("A new page-level picture needs a position").into(),
1205 );
1141 }1206 }
1142 self.picture(image, None)?;1207 self.picture(image, None)?;
1143 }1208 }
1144 PageObject::Ink(ink) => self.ink(ink, None)?,1209 PageObject::Ink(ink) => self.ink(ink, None)?,
1145 PageObject::Title(_) | PageObject::Unsupported(_) => {1210 PageObject::Title(_) | PageObject::Unsupported(_) => {
1146 return Err(OpError::Unsupported("Titles and unsupported objects cannot be added").into());1211 return Err(
1212 OpError::Unsupported("Titles and unsupported objects cannot be added").into(),
1213 );
1147 }1214 }
1148 }1215 }
1149 if before.is_some() {1216 if before.is_some() {
...@@ -1178,13 +1245,13 @@ impl<'a> Writer<'_, 'a> {...@@ -1178,13 +1245,13 @@ impl<'a> Writer<'_, 'a> {
1178 .map(|row| (*row, page.view.nodes[row].children.clone()))1245 .map(|row| (*row, page.view.nodes[row].children.clone()))
1179 .collect();1246 .collect();
1180 let mut borders = *borders;1247 let mut borders = *borders;
1181 let template = rows1248 let template =
1182 .first()1249 rows.first()
1183 .and_then(|(_, cells)| cells.first())1250 .and_then(|(_, cells)| cells.first())
1184 .and_then(|cell| match &page.view.nodes[cell].kind {1251 .and_then(|cell| match &page.view.nodes[cell].kind {
1185 Kind::Cell { indents, .. } => Some(indents.clone()),1252 Kind::Cell { indents, .. } => Some(indents.clone()),
1186 _ => None,1253 _ => None,
1187 });1254 });
1188 let mut new_rows = BTreeSet::new();1255 let mut new_rows = BTreeSet::new();
1189 let mut new_cells: BTreeMap<ExGuid, (Vec<f32>, Option<u32>)> = BTreeMap::new();1256 let mut new_cells: BTreeMap<ExGuid, (Vec<f32>, Option<u32>)> = BTreeMap::new();
1190 let mut filled: Vec<&TableCell> = Vec::new();1257 let mut filled: Vec<&TableCell> = Vec::new();
...@@ -1195,17 +1262,22 @@ impl<'a> Writer<'_, 'a> {...@@ -1195,17 +1262,22 @@ impl<'a> Writer<'_, 'a> {
1195 )1262 )
1196 };1263 };
1197 match edit {1264 match edit {
1198 TableEdit::Rows { before, rows: added } => {1265 TableEdit::Rows {
1266 before,
1267 rows: added,
1268 } => {
1199 let at = match before {1269 let at = match before {
1200 Some(before) => rows1270 Some(before) => rows.iter().position(|(row, _)| row == before).ok_or(
1201 .iter()1271 OpError::StructureChanged("The anchor row is not in the table"),
1202 .position(|(row, _)| row == before)1272 )?,
1203 .ok_or(OpError::StructureChanged("The anchor row is not in the table"))?,
1204 None => rows.len(),1273 None => rows.len(),
1205 };1274 };
1206 for row in added {1275 for row in added {
1207 if row.cells.len() != columns.len() {1276 if row.cells.len() != columns.len() {
1208 return Err(OpError::Unsupported("Every table row has one cell per column").into());1277 return Err(OpError::Unsupported(
1278 "Every table row has one cell per column",
1279 )
1280 .into());
1209 }1281 }
1210 new_rows.insert(row.id);1282 new_rows.insert(row.id);
1211 for cell in &row.cells {1283 for cell in &row.cells {
...@@ -1245,13 +1317,18 @@ impl<'a> Writer<'_, 'a> {...@@ -1245,13 +1317,18 @@ impl<'a> Writer<'_, 'a> {
1245 return Err(OpError::TargetUnavailable(*row).into());1317 return Err(OpError::TargetUnavailable(*row).into());
1246 }1318 }
1247 if rows.is_empty() {1319 if rows.is_empty() {
1248 return Err(OpError::Unsupported("A table keeps a row; delete the table instead").into());1320 return Err(OpError::Unsupported(
1321 "A table keeps a row; delete the table instead",
1322 )
1323 .into());
1249 }1324 }
1250 }1325 }
1251 TableEdit::DeleteColumn(at) => {1326 TableEdit::DeleteColumn(at) => {
1252 let at = *at as usize;1327 let at = *at as usize;
1253 if at >= columns.len() || columns.len() == 1 {1328 if at >= columns.len() || columns.len() == 1 {
1254 return Err(OpError::Unsupported("Choose a column of a table with several").into());1329 return Err(
1330 OpError::Unsupported("Choose a column of a table with several").into(),
1331 );
1255 }1332 }
1256 columns.remove(at);1333 columns.remove(at);
1257 for (_, cells) in &mut rows {1334 for (_, cells) in &mut rows {
...@@ -1294,7 +1371,10 @@ impl<'a> Writer<'_, 'a> {...@@ -1294,7 +1371,10 @@ impl<'a> Writer<'_, 'a> {
1294 for row in &new_rows {1371 for row in &new_rows {
1295 self.free(*row)?;1372 self.free(*row)?;
1296 }1373 }
1297 if columns.iter().any(|c| !c.width.is_finite() || c.width < 36.0) {1374 if columns
1375 .iter()
1376 .any(|c| !c.width.is_finite() || c.width < 36.0)
1377 {
1298 return Err(OpError::Unsupported("Table columns are at least 36 points wide").into());1378 return Err(OpError::Unsupported("Table columns are at least 36 points wide").into());
1299 }1379 }
1300 let structure = Structure {1380 let structure = Structure {
...@@ -1348,7 +1428,11 @@ fn bare(paragraph: &PageParagraph) -> Result<(), OpError> {...@@ -1348,7 +1428,11 @@ fn bare(paragraph: &PageParagraph) -> Result<(), OpError> {
1348 if !table.tags.is_empty() {1428 if !table.tags.is_empty() {
1349 return Err(OpError::Unsupported("Set a table's tags with their own op"));1429 return Err(OpError::Unsupported("Set a table's tags with their own op"));
1350 }1430 }
1351 for paragraph in table.rows.iter().flat_map(|row| &row.cells).flat_map(|cell| &cell.paragraphs)1431 for paragraph in table
1432 .rows
1433 .iter()
1434 .flat_map(|row| &row.cells)
1435 .flat_map(|cell| &cell.paragraphs)
1352 {1436 {
1353 bare(paragraph)?;1437 bare(paragraph)?;
1354 }1438 }
...@@ -1364,7 +1448,9 @@ fn bare(paragraph: &PageParagraph) -> Result<(), OpError> {...@@ -1364,7 +1448,9 @@ fn bare(paragraph: &PageParagraph) -> Result<(), OpError> {
1364/// The field code OneNote stores before a hyperlink's label.1448/// The field code OneNote stores before a hyperlink's label.
1365fn field_code(target: &str) -> Result<String, Error> {1449fn field_code(target: &str) -> Result<String, Error> {
1366 if target.is_empty() || target.contains(['"', '\0', '\r', '\n', '\u{fddf}']) {1450 if target.is_empty() || target.contains(['"', '\0', '\r', '\n', '\u{fddf}']) {
1367 return Err(invalid("Choose a link target without quotes or line breaks"));1451 return Err(invalid(
1452 "Choose a link target without quotes or line breaks",
1453 ));
1368 }1454 }
1369 Ok(format!("\u{fddf}HYPERLINK \"{target}\""))1455 Ok(format!("\u{fddf}HYPERLINK \"{target}\""))
1370}1456}
...@@ -1449,6 +1535,8 @@ pub(crate) fn link_ops(...@@ -1449,6 +1535,8 @@ pub(crate) fn link_ops(
1449 clear: vec![TextProperty::Hyperlink, TextProperty::HyperlinkLabel],1535 clear: vec![TextProperty::Hyperlink, TextProperty::HyperlinkLabel],
1450 }),1536 }),
1451 }1537 }
1452 ops.retain(|op| !matches!(op, PageOp::Format { set, clear, .. } if set.is_empty() && clear.is_empty()));1538 ops.retain(
1539 |op| !matches!(op, PageOp::Format { set, clear, .. } if set.is_empty() && clear.is_empty()),
1540 );
1453 Ok(ops)1541 Ok(ops)
1454}1542}
crates/onestore/src/op/content.rs+16-9
...@@ -184,7 +184,11 @@ fn layout_values(stored: &Layout, layout: &Layout) -> Result<(Values, Vec<u32>),...@@ -184,7 +184,11 @@ fn layout_values(stored: &Layout, layout: &Layout) -> Result<(Values, Vec<u32>),
184 Ok((values, removed))184 Ok((values, removed))
185}185}
186186
187fn icon_size(values: &mut Values, size: Option<[f32; 2]>, message: &'static str) -> Result<(), Error> {187fn icon_size(
188 values: &mut Values,
189 size: Option<[f32; 2]>,
190 message: &'static str,
191) -> Result<(), Error> {
188 if let Some([width, height]) = size {192 if let Some([width, height]) = size {
189 if !(width.is_finite() && height.is_finite() && width > 0.0 && height > 0.0) {193 if !(width.is_finite() && height.is_finite() && width > 0.0 && height > 0.0) {
190 return Err(invalid(message));194 return Err(invalid(message));
...@@ -332,7 +336,11 @@ pub(crate) fn attachment_changes(...@@ -332,7 +336,11 @@ pub(crate) fn attachment_changes(
332 .unwrap_or("");336 .unwrap_or("");
333 let modified = crate::create::current_timestamps()?.0.to_le_bytes();337 let modified = crate::create::current_timestamps()?.0.to_le_bytes();
334 let mut values: Values = vec![(0x14001d7a, modified.to_vec())];338 let mut values: Values = vec![(0x14001d7a, modified.to_vec())];
335 icon_size(&mut values, attachment.size, "Attachment icon size must be positive")?;339 icon_size(
340 &mut values,
341 attachment.size,
342 "Attachment icon size must be positive",
343 )?;
336 values.push((0x14001c3b, 0x409_u32.to_le_bytes().to_vec()));344 values.push((0x14001c3b, 0x409_u32.to_le_bytes().to_vec()));
337 values.push((0x10001cfe, 0x409_u16.to_le_bytes().to_vec()));345 values.push((0x10001cfe, 0x409_u16.to_le_bytes().to_vec()));
338 values.push((0x1c001dcf, vec![0; 32]));346 values.push((0x1c001dcf, vec![0; 32]));
...@@ -412,7 +420,11 @@ pub(crate) fn attachment_edit_changes(...@@ -412,7 +420,11 @@ pub(crate) fn attachment_edit_changes(
412 }420 }
413 if attachment.size != stored.size {421 if attachment.size != stored.size {
414 match attachment.size {422 match attachment.size {
415 Some(_) => icon_size(&mut values, attachment.size, "Attachment icon size must be positive")?,423 Some(_) => icon_size(
424 &mut values,
425 attachment.size,
426 "Attachment icon size must be positive",
427 )?,
416 None => removed.extend([0x140034cd, 0x140034ce]),428 None => removed.extend([0x140034cd, 0x140034ce]),
417 }429 }
418 }430 }
...@@ -612,12 +624,7 @@ pub(crate) fn strokes_changes(...@@ -612,12 +624,7 @@ pub(crate) fn strokes_changes(
612 kept: &[ExGuid],624 kept: &[ExGuid],
613 added: &[(ExGuid, &InkStroke)],625 added: &[(ExGuid, &InkStroke)],
614) -> Result<Changes, Error> {626) -> Result<Changes, Error> {
615 let data = match active627 let data = match active.view.nodes.get(&container).map(|node| &node.kind) {
616 .view
617 .nodes
618 .get(&container)
619 .map(|node| &node.kind)
620 {
621 Some(crate::document::Kind::Ink {628 Some(crate::document::Kind::Ink {
622 data: Some(data), ..629 data: Some(data), ..
623 }) => *data,630 }) => *data,
crates/onestore/src/op/levels.rs+5-1
...@@ -37,7 +37,11 @@ pub(crate) type Runs = Vec<(u32, Vec<ExGuid>)>;...@@ -37,7 +37,11 @@ pub(crate) type Runs = Vec<(u32, Vec<ExGuid>)>;
3737
38/// The child level of a container whose children lie `depths` deeper than it, and its38/// The child level of a container whose children lie `depths` deeper than it, and its
39/// children in runs.39/// children in runs.
40pub(crate) fn runs(children: &[ExGuid], depths: &[u32], is_cell: bool) -> Result<(u32, Runs), Error> {40pub(crate) fn runs(
41 children: &[ExGuid],
42 depths: &[u32],
43 is_cell: bool,
44) -> Result<(u32, Runs), Error> {
41 let level = *depths45 let level = *depths
42 .iter()46 .iter()
43 .min()47 .min()
crates/onestore/src/op/lower.rs+46-17
...@@ -577,7 +577,9 @@ fn attributes(...@@ -577,7 +577,9 @@ fn attributes(
577 }577 }
578 let color =578 let color =
579 |value: u32| (value != 0xff000000).then(|| value.to_le_bytes()[..3].try_into().unwrap());579 |value: u32| (value != 0xff000000).then(|| value.to_le_bytes()[..3].try_into().unwrap());
580 value!(font, Font, |font: &String| TextAttribute::Font(font.clone()));580 value!(font, Font, |font: &String| TextAttribute::Font(
581 font.clone()
582 ));
581 value!(font_size, FontSize, |size: &f32| TextAttribute::FontSize(583 value!(font_size, FontSize, |size: &f32| TextAttribute::FontSize(
582 *size584 *size
583 ));585 ));
...@@ -690,9 +692,11 @@ pub(crate) fn same_tags(a: &[Tag], b: &[Tag]) -> bool {...@@ -690,9 +692,11 @@ pub(crate) fn same_tags(a: &[Tag], b: &[Tag]) -> bool {
690fn ordinary(text: &TextObject) -> bool {692fn ordinary(text: &TextObject) -> bool {
691 text.date_field.is_none()693 text.date_field.is_none()
692 && !text.text.text().contains('\u{fffc}')694 && !text.text.text().contains('\u{fffc}')
693 && text.text.spans().iter().all(|span| {695 && text
694 span.format.math != Some(true) && span.format.embedded_object != Some(true)696 .text
695 })697 .spans()
698 .iter()
699 .all(|span| span.format.math != Some(true) && span.format.embedded_object != Some(true))
696}700}
697701
698/// Whether a split at `offset` would divide a hyperlink, which the writer refuses.702/// Whether a split at `offset` would divide a hyperlink, which the writer refuses.
...@@ -936,7 +940,13 @@ impl Lowering {...@@ -936,7 +940,13 @@ impl Lowering {
936 continue;940 continue;
937 }941 }
938 let adopts_right = previous.text.text().is_empty();942 let adopts_right = previous.text.text().is_empty();
939 if after.id != if adopts_right { removed.id } else { previous.id } {943 if after.id
944 != if adopts_right {
945 removed.id
946 } else {
947 previous.id
948 }
949 {
940 continue;950 continue;
941 }951 }
942 self.emit(PageOp::Join {952 self.emit(PageOp::Join {
...@@ -1025,7 +1035,11 @@ impl Lowering {...@@ -1025,7 +1035,11 @@ impl Lowering {
1025 }1035 }
1026 let object = match object {1036 let object = match object {
1027 PageObject::Outline(outline) => {1037 PageObject::Outline(outline) => {
1028 if outline.paragraphs.iter().any(|p| old.paragraphs.contains_key(&p.id)) {1038 if outline
1039 .paragraphs
1040 .iter()
1041 .any(|p| old.paragraphs.contains_key(&p.id))
1042 {
1029 return Err(invalid(1043 return Err(invalid(
1030 "A new outline holds new paragraphs; move stored ones after",1044 "A new outline holds new paragraphs; move stored ones after",
1031 ));1045 ));
...@@ -1233,11 +1247,21 @@ impl Lowering {...@@ -1233,11 +1247,21 @@ impl Lowering {
1233 let mut deleted: Option<Vec<u32>> = None;1247 let mut deleted: Option<Vec<u32>> = None;
1234 let mut added: Option<Vec<u32>> = None;1248 let mut added: Option<Vec<u32>> = None;
1235 for row in &kept_rows {1249 for row in &kept_rows {
1236 let old_cells: Vec<ExGuid> = before.rows.iter().find(|r| r.id == *row).unwrap().cells1250 let old_cells: Vec<ExGuid> = before
1251 .rows
1252 .iter()
1253 .find(|r| r.id == *row)
1254 .unwrap()
1255 .cells
1237 .iter()1256 .iter()
1238 .map(|cell| cell.id)1257 .map(|cell| cell.id)
1239 .collect();1258 .collect();
1240 let new_cells: Vec<ExGuid> = after.rows.iter().find(|r| r.id == *row).unwrap().cells1259 let new_cells: Vec<ExGuid> = after
1260 .rows
1261 .iter()
1262 .find(|r| r.id == *row)
1263 .unwrap()
1264 .cells
1241 .iter()1265 .iter()
1242 .map(|cell| cell.id)1266 .map(|cell| cell.id)
1243 .collect();1267 .collect();
...@@ -1382,9 +1406,10 @@ impl Lowering {...@@ -1382,9 +1406,10 @@ impl Lowering {
1382 (ParagraphContent::Ink(stored), ParagraphContent::Ink(ink)) if stored != ink => {1406 (ParagraphContent::Ink(stored), ParagraphContent::Ink(ink)) if stored != ink => {
1383 self.strokes(stored, ink)?;1407 self.strokes(stored, ink)?;
1384 }1408 }
1385 (ParagraphContent::Attachment(stored), ParagraphContent::Attachment(attachment))1409 (
1386 if stored != attachment =>1410 ParagraphContent::Attachment(stored),
1387 {1411 ParagraphContent::Attachment(attachment),
1412 ) if stored != attachment => {
1388 self.emit(PageOp::Attachment {1413 self.emit(PageOp::Attachment {
1389 attachment: attachment.id,1414 attachment: attachment.id,
1390 filename: attachment.filename.clone(),1415 filename: attachment.filename.clone(),
...@@ -1520,8 +1545,8 @@ impl Lowering {...@@ -1520,8 +1545,8 @@ impl Lowering {
1520 {1545 {
1521 continue;1546 continue;
1522 }1547 }
1523 let stored =1548 let stored = stored
1524 stored.ok_or_else(|| invalid("An equation paragraph is missing after placement"))?;1549 .ok_or_else(|| invalid("An equation paragraph is missing after placement"))?;
1525 if stored.text != text.text {1550 if stored.text != text.text {
1526 self.emit(PageOp::Equation {1551 self.emit(PageOp::Equation {
1527 text: stored.id,1552 text: stored.id,
...@@ -1611,10 +1636,14 @@ impl Lowering {...@@ -1611,10 +1636,14 @@ impl Lowering {
1611 let current = model::paragraph(&self.current, *id)1636 let current = model::paragraph(&self.current, *id)
1612 .ok_or_else(|| invalid("A paragraph is missing after text edits"))?;1637 .ok_or_else(|| invalid("A paragraph is missing after text edits"))?;
1613 let same = paragraph.lists.len() == current.lists.len()1638 let same = paragraph.lists.len() == current.lists.len()
1614 && paragraph.lists.iter().zip(&current.lists).all(|(model, node)| {1639 && paragraph
1615 model == node1640 .lists
1616 && after.definitions.get(model) == self.current.definitions.get(node)1641 .iter()
1617 });1642 .zip(&current.lists)
1643 .all(|(model, node)| {
1644 model == node
1645 && after.definitions.get(model) == self.current.definitions.get(node)
1646 });
1618 if same {1647 if same {
1619 continue;1648 continue;
1620 }1649 }
crates/onestore/src/op/mod.rs+57-9
...@@ -5,13 +5,16 @@...@@ -5,13 +5,16 @@
5use crate::{5use crate::{
6 ExGuid, OutlineEdit, PageCreation, PageEdit, TextAttribute,6 ExGuid, OutlineEdit, PageCreation, PageEdit, TextAttribute,
7 document::{Layout, Tag},7 document::{Layout, Tag},
8 page::{Definition, InkStroke, Page, PageObject, PageParagraph, Paragraph, TableCell,8 page::{
9 TableColumn, TableRow},9 Definition, InkStroke, Page, PageObject, PageParagraph, Paragraph, TableCell, TableColumn,
10 TableRow,
11 },
10};12};
11use serde::{Deserialize, Serialize};13use serde::{Deserialize, Serialize};
12use std::ops::Range;14use std::ops::Range;
1315
14mod apply;16mod apply;
17pub(crate) use apply::Failure;
15pub(crate) mod content;18pub(crate) mod content;
16pub(crate) mod levels;19pub(crate) mod levels;
17pub(crate) mod lower;20pub(crate) mod lower;
...@@ -73,7 +76,10 @@ pub enum PageOp {...@@ -73,7 +76,10 @@ pub enum PageOp {
73 target: Option<String>,76 target: Option<String>,
74 },77 },
75 /// Rewrites an equation's linear text and spans whole.78 /// Rewrites an equation's linear text and spans whole.
76 Equation { text: ExGuid, math: Paragraph },79 Equation {
80 text: ExGuid,
81 math: Paragraph,
82 },
77 /// The page's date: its creation time, FILETIME, and the text each of the title's date83 /// The page's date: its creation time, FILETIME, and the text each of the title's date
78 /// and time fields shows for it, as OneNote 2010 writes both when the date changes.84 /// and time fields shows for it, as OneNote 2010 writes both when the date changes.
79 Date {85 Date {
...@@ -101,7 +107,10 @@ pub enum PageOp {...@@ -101,7 +107,10 @@ pub enum PageOp {
101 },107 },
102 /// Appends the right text to the left one and removes the right paragraph; an empty108 /// Appends the right text to the left one and removes the right paragraph; an empty
103 /// left text is replaced by the right text object.109 /// left text is replaced by the right text object.
104 Join { left: ExGuid, right: ExGuid },110 Join {
111 left: ExGuid,
112 right: ExGuid,
113 },
105 /// Moves a subtree before a direct child of `parent`, or last; `None` names the page,114 /// Moves a subtree before a direct child of `parent`, or last; `None` names the page,
106 /// whose children are outlines, pictures and ink. A paragraph keeps its level where it115 /// whose children are outlines, pictures and ink. A paragraph keeps its level where it
107 /// lies deeper than its new parent.116 /// lies deeper than its new parent.
...@@ -111,11 +120,19 @@ pub enum PageOp {...@@ -111,11 +120,19 @@ pub enum PageOp {
111 before: Option<ExGuid>,120 before: Option<ExGuid>,
112 },121 },
113 /// Removes a subtree.122 /// Removes a subtree.
114 Delete { object: ExGuid },123 Delete {
124 object: ExGuid,
125 },
115 /// Sets a paragraph's absolute outline level, regrouping its container.126 /// Sets a paragraph's absolute outline level, regrouping its container.
116 Level { paragraph: ExGuid, level: u32 },127 Level {
128 paragraph: ExGuid,
129 level: u32,
130 },
117 /// Outline position and width, or a paragraph's saved expansion state.131 /// Outline position and width, or a paragraph's saved expansion state.
118 Outline { object: ExGuid, edit: OutlineEdit },132 Outline {
133 object: ExGuid,
134 edit: OutlineEdit,
135 },
119 /// Paragraph formatting stored on a paragraph's text; `None` leaves a value as it is.136 /// Paragraph formatting stored on a paragraph's text; `None` leaves a value as it is.
120 Paragraph {137 Paragraph {
121 paragraph: ExGuid,138 paragraph: ExGuid,
...@@ -169,7 +186,10 @@ pub enum PageOp {...@@ -169,7 +186,10 @@ pub enum PageOp {
169 add: Vec<InkStroke>,186 add: Vec<InkStroke>,
170 remove: Vec<ExGuid>,187 remove: Vec<ExGuid>,
171 },188 },
172 Table { table: ExGuid, edit: TableEdit },189 Table {
190 table: ExGuid,
191 edit: TableEdit,
192 },
173}193}
174194
175#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]195#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
...@@ -205,7 +225,10 @@ pub enum SectionOp {...@@ -205,7 +225,10 @@ pub enum SectionOp {
205 Create(PageCreation),225 Create(PageCreation),
206 /// A page created with `creation` holding `page`'s content under the identities it226 /// A page created with `creation` holding `page`'s content under the identities it
207 /// carries (`Page::copy` gives fresh ones); the created title stays.227 /// carries (`Page::copy` gives fresh ones); the created title stays.
208 Import { creation: PageCreation, page: Page },228 Import {
229 creation: PageCreation,
230 page: Page,
231 },
209 /// A read-only conflict page of the listed page `of`, as OneNote 2010 keeps the version of232 /// A read-only conflict page of the listed page `of`, as OneNote 2010 keeps the version of
210 /// a page a merge could not take: created with `creation` (not placed, its author the user233 /// a page a merge could not take: created with `creation` (not placed, its author the user
211 /// whose version it is, a title only where `page` has one) and holding `page` as `Import`234 /// whose version it is, a title only where `page` has one) and holding `page` as `Import`
...@@ -223,6 +246,31 @@ pub enum SectionOp {...@@ -223,6 +246,31 @@ pub enum SectionOp {
223 /// The section's colour as COLORREF, kept in its own metadata; `None` is OneNote's246 /// The section's colour as COLORREF, kept in its own metadata; `None` is OneNote's
224 /// "no colour".247 /// "no colour".
225 Color(Option<u32>),248 Color(Option<u32>),
249 /// Makes a version of a page (`Section::versions`) its current state, as OneNote 2010's
250 /// Restore Version does; the page as it stood becomes the newest version. New objects take
251 /// identities from `guid`.
252 RestoreVersion {
253 page: ExGuid,
254 version: ExGuid,
255 guid: [u8; 16],
256 },
257 /// Deletes versions of a page, as OneNote 2010's Delete Version does; their revisions stay
258 /// stored.
259 DeleteVersions {
260 page: ExGuid,
261 versions: Vec<ExGuid>,
262 },
263}
264
265impl SectionOp {
266 /// `RestoreVersion` of `page`'s `version`, its new objects under a fresh identity.
267 pub fn restore(page: ExGuid, version: ExGuid) -> Result<Self, crate::Error> {
268 Ok(Self::RestoreVersion {
269 page,
270 version,
271 guid: crate::write::fresh_guid()?,
272 })
273 }
226}274}
227275
228/// A character property that `PageOp::Format` can clear so the text inherits it.276/// A character property that `PageOp::Format` can clear so the text inherits it.
crates/onestore/src/op/model.rs+102-41
...@@ -53,7 +53,12 @@ pub(crate) fn lists(page: &Page) -> Vec<(Path, ExGuid, &Vec<PageParagraph>)> {...@@ -53,7 +53,12 @@ pub(crate) fn lists(page: &Page) -> Vec<(Path, ExGuid, &Vec<PageParagraph>)> {
53 for (object, value) in page.objects.iter().enumerate() {53 for (object, value) in page.objects.iter().enumerate() {
54 let outlines: Vec<(Option<usize>, &Outline)> = match value {54 let outlines: Vec<(Option<usize>, &Outline)> = match value {
55 PageObject::Outline(outline) => vec![(None, outline)],55 PageObject::Outline(outline) => vec![(None, outline)],
56 PageObject::Title(title) => title.outlines.iter().enumerate().map(|(i, o)| (Some(i), o)).collect(),56 PageObject::Title(title) => title
57 .outlines
58 .iter()
59 .enumerate()
60 .map(|(i, o)| (Some(i), o))
61 .collect(),
57 _ => Vec::new(),62 _ => Vec::new(),
58 };63 };
59 for (title_outline, outline) in outlines {64 for (title_outline, outline) in outlines {
...@@ -88,24 +93,24 @@ pub(crate) fn list_at<'p>(page: &'p mut Page, path: &Path) -> &'p mut Vec<PagePa...@@ -88,24 +93,24 @@ pub(crate) fn list_at<'p>(page: &'p mut Page, path: &Path) -> &'p mut Vec<PagePa
88/// The list holding paragraph `id`, and its index there.93/// The list holding paragraph `id`, and its index there.
89fn find(page: &Page, id: ExGuid) -> Option<(Path, usize)> {94fn find(page: &Page, id: ExGuid) -> Option<(Path, usize)> {
90 lists(page).into_iter().find_map(|(path, _, list)| {95 lists(page).into_iter().find_map(|(path, _, list)| {
91 list.iter().position(|p| p.id == id).map(|index| (path, index))96 list.iter()
97 .position(|p| p.id == id)
98 .map(|index| (path, index))
92 })99 })
93}100}
94101
95/// The list a container's children lie in: its own for an outline or cell, the one102/// The list a container's children lie in: its own for an outline or cell, the one
96/// holding it for a paragraph.103/// holding it for a paragraph.
97fn container(page: &Page, id: ExGuid) -> Option<(Path, Option<usize>)> {104fn container(page: &Page, id: ExGuid) -> Option<(Path, Option<usize>)> {
98 lists(page)105 lists(page).into_iter().find_map(|(path, owner, list)| {
99 .into_iter()106 if owner == id {
100 .find_map(|(path, owner, list)| {107 Some((path, None))
101 if owner == id {108 } else {
102 Some((path, None))109 list.iter()
103 } else {110 .position(|p| p.id == id)
104 list.iter()111 .map(|index| (path, Some(index)))
105 .position(|p| p.id == id)112 }
106 .map(|index| (path, Some(index)))113 })
107 }
108 })
109}114}
110115
111/// The entries of `list` in the subtree of the paragraph at `index`.116/// The entries of `list` in the subtree of the paragraph at `index`.
...@@ -227,7 +232,8 @@ pub(crate) fn paragraph_text(page: &Page, text: ExGuid) -> Option<&TextObject> {...@@ -227,7 +232,8 @@ pub(crate) fn paragraph_text(page: &Page, text: ExGuid) -> Option<&TextObject> {
227}232}
228233
229fn text_mut(page: &mut Page, text: ExGuid) -> Result<&mut TextObject, Error> {234fn text_mut(page: &mut Page, text: ExGuid) -> Result<&mut TextObject, Error> {
230 let (path, index) = text_holder(page, text).ok_or_else(|| invalid("Text is not on the page"))?;235 let (path, index) =
236 text_holder(page, text).ok_or_else(|| invalid("Text is not on the page"))?;
231 Ok(list_at(page, &path)[index].text_mut().unwrap())237 Ok(list_at(page, &path)[index].text_mut().unwrap())
232}238}
233239
...@@ -269,7 +275,10 @@ pub(crate) fn replace_text(...@@ -269,7 +275,10 @@ pub(crate) fn replace_text(
269 let format = format_in(text, range.start)?.clone();275 let format = format_in(text, range.start)?.clone();
270 let removed = range.end - range.start;276 let removed = range.end - range.start;
271 if length - removed == 0 && with.is_empty() {277 if length - removed == 0 && with.is_empty() {
272 return Ok(Paragraph::new(String::new(), text.spans().last().unwrap().format.clone()));278 return Ok(Paragraph::new(
279 String::new(),
280 text.spans().last().unwrap().format.clone(),
281 ));
273 }282 }
274 let mut result = text.clone();283 let mut result = text.clone();
275 result.apply(crate::page::text::Edit {284 result.apply(crate::page::text::Edit {
...@@ -373,7 +382,10 @@ pub(crate) fn format_text(...@@ -373,7 +382,10 @@ pub(crate) fn format_text(
373 format382 format
374 };383 };
375 if text.text().is_empty() {384 if text.text().is_empty() {
376 return Ok(Paragraph::new(String::new(), apply(&text.spans()[0].format)));385 return Ok(Paragraph::new(
386 String::new(),
387 apply(&text.spans()[0].format),
388 ));
377 }389 }
378 let (start, end) = (text.byte_offset(range.start)?, text.byte_offset(range.end)?);390 let (start, end) = (text.byte_offset(range.start)?, text.byte_offset(range.end)?);
379 let mut runs = Vec::new();391 let mut runs = Vec::new();
...@@ -381,11 +393,19 @@ pub(crate) fn format_text(...@@ -381,11 +393,19 @@ pub(crate) fn format_text(
381 for span in text.spans() {393 for span in text.spans() {
382 for (from, to, selected) in [394 for (from, to, selected) in [
383 (previous, start.clamp(previous, span.end), false),395 (previous, start.clamp(previous, span.end), false),
384 (start.clamp(previous, span.end), end.clamp(previous, span.end), true),396 (
397 start.clamp(previous, span.end),
398 end.clamp(previous, span.end),
399 true,
400 ),
385 (end.clamp(previous, span.end), span.end, false),401 (end.clamp(previous, span.end), span.end, false),
386 ] {402 ] {
387 if from < to {403 if from < to {
388 let format = if selected { apply(&span.format) } else { span.format.clone() };404 let format = if selected {
405 apply(&span.format)
406 } else {
407 span.format.clone()
408 };
389 runs.push((text.text()[from..to].to_owned(), format));409 runs.push((text.text()[from..to].to_owned(), format));
390 }410 }
391 }411 }
...@@ -401,7 +421,9 @@ fn inserted(text: &Paragraph) -> Paragraph {...@@ -401,7 +421,9 @@ fn inserted(text: &Paragraph) -> Paragraph {
401 macro_rules! shared {421 macro_rules! shared {
402 ($field:ident, $type:ty, $store:expr) => {{422 ($field:ident, $type:ty, $store:expr) => {{
403 let first = spans[0].format.$field.unwrap_or_default();423 let first = spans[0].format.$field.unwrap_or_default();
404 (spans.iter().all(|s| s.format.$field.unwrap_or_default() == first)424 (spans
425 .iter()
426 .all(|s| s.format.$field.unwrap_or_default() == first)
405 && first != <$type>::default())427 && first != <$type>::default())
406 .then(|| $store(first))428 .then(|| $store(first))
407 }};429 }};
...@@ -485,8 +507,8 @@ fn insert(...@@ -485,8 +507,8 @@ fn insert(
485 before: Option<ExGuid>,507 before: Option<ExGuid>,
486 paragraphs: Vec<PageParagraph>,508 paragraphs: Vec<PageParagraph>,
487) -> Result<(), Error> {509) -> Result<(), Error> {
488 let (path, owner) =510 let (path, owner) = self::container(page, container)
489 self::container(page, container).ok_or_else(|| invalid("The container is not on the page"))?;511 .ok_or_else(|| invalid("The container is not on the page"))?;
490 let list = list_at(page, &path);512 let list = list_at(page, &path);
491 let at = match before {513 let at = match before {
492 Some(before) => list514 Some(before) => list
...@@ -519,7 +541,9 @@ fn relevel(list: &mut [PageParagraph], index: usize, level: u32) {...@@ -519,7 +541,9 @@ fn relevel(list: &mut [PageParagraph], index: usize, level: u32) {
519/// The title a page's writers store after changing it: the title text's first line when541/// The title a page's writers store after changing it: the title text's first line when
520/// it has one, which only an edit of that text changes, else the first line of body text.542/// it has one, which only an edit of that text changes, else the first line of body text.
521fn retitle(page: &mut Page, edited: Option<ExGuid>) {543fn retitle(page: &mut Page, edited: Option<ExGuid>) {
522 let line = |text: &str| text.trim_start().split('\r').next().unwrap_or_default().to_owned();544 let line = |text: &str| {
545 crate::edit::without_fields(text.trim_start().split('\r').next().unwrap_or_default())
546 };
523 match find_title_text(page) {547 match find_title_text(page) {
524 Some(title) => {548 Some(title) => {
525 let text = text_of(page, title);549 let text = text_of(page, title);
...@@ -545,7 +569,7 @@ fn automatic(page: &mut Page) {...@@ -545,7 +569,7 @@ fn automatic(page: &mut Page) {
545 .total_cmp(&b.y.unwrap_or(0.0))569 .total_cmp(&b.y.unwrap_or(0.0))
546 .then_with(|| a.x.unwrap_or(0.0).total_cmp(&b.x.unwrap_or(0.0)))570 .then_with(|| a.x.unwrap_or(0.0).total_cmp(&b.x.unwrap_or(0.0)))
547 });571 });
548 fn first(list: &[PageParagraph]) -> Option<&str> {572 fn first(list: &[PageParagraph]) -> Option<String> {
549 list.iter().find_map(|paragraph| match &paragraph.content {573 list.iter().find_map(|paragraph| match &paragraph.content {
550 ParagraphContent::Text(text) => {574 ParagraphContent::Text(text) => {
551 Some(crate::edit::automatic_title(text.text.text())).filter(|t| !t.is_empty())575 Some(crate::edit::automatic_title(text.text.text())).filter(|t| !t.is_empty())
...@@ -564,8 +588,7 @@ fn automatic(page: &mut Page) {...@@ -564,8 +588,7 @@ fn automatic(page: &mut Page) {
564 PageObject::Outline(outline) => first(&outline.paragraphs),588 PageObject::Outline(outline) => first(&outline.paragraphs),
565 _ => None,589 _ => None,
566 })590 })
567 .unwrap_or_default()591 .unwrap_or_default();
568 .to_owned();
569 page.title = title;592 page.title = title;
570}593}
571594
...@@ -650,7 +673,10 @@ fn interpret(page: &mut Page, op: &PageOp) -> Result<(), Error> {...@@ -650,7 +673,10 @@ fn interpret(page: &mut Page, op: &PageOp) -> Result<(), Error> {
650 let object = paragraph.text_mut().unwrap();673 let object = paragraph.text_mut().unwrap();
651 let previous = object.text.spans()[0].format.clone();674 let previous = object.text.spans()[0].format.clone();
652 let math = Paragraph::from_runs(math.spans().iter().scan(0, |start, span| {675 let math = Paragraph::from_runs(math.spans().iter().scan(0, |start, span| {
653 let run = (math.text()[*start..span.end].to_owned(), span.format.inherit(&inherited));676 let run = (
677 math.text()[*start..span.end].to_owned(),
678 span.format.inherit(&inherited),
679 );
654 *start = span.end;680 *start = span.end;
655 Some(run)681 Some(run)
656 }));682 }));
...@@ -671,7 +697,8 @@ fn interpret(page: &mut Page, op: &PageOp) -> Result<(), Error> {...@@ -671,7 +697,8 @@ fn interpret(page: &mut Page, op: &PageOp) -> Result<(), Error> {
671 right,697 right,
672 lists,698 lists,
673 } => {699 } => {
674 let (path, index) = text_holder(page, *text).ok_or_else(|| invalid("Text is not on the page"))?;700 let (path, index) =
701 text_holder(page, *text).ok_or_else(|| invalid("Text is not on the page"))?;
675 let definitions: Vec<_> = {702 let definitions: Vec<_> = {
676 let list = list_at(page, &path);703 let list = list_at(page, &path);
677 list[index].lists.clone()704 list[index].lists.clone()
...@@ -722,7 +749,8 @@ fn interpret(page: &mut Page, op: &PageOp) -> Result<(), Error> {...@@ -722,7 +749,8 @@ fn interpret(page: &mut Page, op: &PageOp) -> Result<(), Error> {
722 retitle(page, None);749 retitle(page, None);
723 }750 }
724 PageOp::Join { left, right } => {751 PageOp::Join { left, right } => {
725 let (path, index) = text_holder(page, *right).ok_or_else(|| invalid("Text is not on the page"))?;752 let (path, index) =
753 text_holder(page, *right).ok_or_else(|| invalid("Text is not on the page"))?;
726 let (left_path, left_index) =754 let (left_path, left_index) =
727 text_holder(page, *left).ok_or_else(|| invalid("Text is not on the page"))?;755 text_holder(page, *left).ok_or_else(|| invalid("Text is not on the page"))?;
728 let list = list_at(page, &path);756 let list = list_at(page, &path);
...@@ -778,7 +806,10 @@ fn interpret(page: &mut Page, op: &PageOp) -> Result<(), Error> {...@@ -778,7 +806,10 @@ fn interpret(page: &mut Page, op: &PageOp) -> Result<(), Error> {
778 }806 }
779 }807 }
780 }808 }
781 let run = (right_text.text.text()[previous..span.end].to_owned(), format);809 let run = (
810 right_text.text.text()[previous..span.end].to_owned(),
811 format,
812 );
782 previous = span.end;813 previous = span.end;
783 run814 run
784 }));815 }));
...@@ -849,7 +880,8 @@ fn interpret(page: &mut Page, op: &PageOp) -> Result<(), Error> {...@@ -849,7 +880,8 @@ fn interpret(page: &mut Page, op: &PageOp) -> Result<(), Error> {
849 retitle(page, None);880 retitle(page, None);
850 return Ok(());881 return Ok(());
851 };882 };
852 let (path, index) = find(page, *object).ok_or_else(|| invalid("The paragraph is not on the page"))?;883 let (path, index) =
884 find(page, *object).ok_or_else(|| invalid("The paragraph is not on the page"))?;
853 let list = list_at(page, &path);885 let list = list_at(page, &path);
854 let mut moved: Vec<PageParagraph> = list.drain(subtree(list, index)).collect();886 let mut moved: Vec<PageParagraph> = list.drain(subtree(list, index)).collect();
855 let into_cell = table_cells(page).contains(parent);887 let into_cell = table_cells(page).contains(parent);
...@@ -884,7 +916,8 @@ fn interpret(page: &mut Page, op: &PageOp) -> Result<(), Error> {...@@ -884,7 +916,8 @@ fn interpret(page: &mut Page, op: &PageOp) -> Result<(), Error> {
884 retitle(page, None);916 retitle(page, None);
885 }917 }
886 PageOp::Level { paragraph, level } => {918 PageOp::Level { paragraph, level } => {
887 let (path, index) = find(page, *paragraph).ok_or_else(|| invalid("The paragraph is not on the page"))?;919 let (path, index) = find(page, *paragraph)
920 .ok_or_else(|| invalid("The paragraph is not on the page"))?;
888 relevel(list_at(page, &path), index, *level);921 relevel(list_at(page, &path), index, *level);
889 }922 }
890 PageOp::Outline { object, edit } => match edit {923 PageOp::Outline { object, edit } => match edit {
...@@ -967,10 +1000,31 @@ fn interpret(page: &mut Page, op: &PageOp) -> Result<(), Error> {...@@ -967,10 +1000,31 @@ fn interpret(page: &mut Page, op: &PageOp) -> Result<(), Error> {
967 }1000 }
968 )*};1001 )*};
969 }1002 }
970 restyle!(bold, italic, underline, strike, superscript, subscript, hidden,1003 restyle!(
971 hyperlink, hyperlink_label, math, embedded_object, font, font_size,1004 bold,
972 color, highlight, language, alignment, rtl, space_before, space_after,1005 italic,
973 line_spacing, list_spacing, math_object);1006 underline,
1007 strike,
1008 superscript,
1009 subscript,
1010 hidden,
1011 hyperlink,
1012 hyperlink_label,
1013 math,
1014 embedded_object,
1015 font,
1016 font_size,
1017 color,
1018 highlight,
1019 language,
1020 alignment,
1021 rtl,
1022 space_before,
1023 space_after,
1024 line_spacing,
1025 list_spacing,
1026 math_object
1027 );
974 let run = (text.text.text()[previous..span.end].to_owned(), format);1028 let run = (text.text.text()[previous..span.end].to_owned(), format);
975 previous = span.end;1029 previous = span.end;
976 run1030 run
...@@ -1052,7 +1106,9 @@ fn interpret(page: &mut Page, op: &PageOp) -> Result<(), Error> {...@@ -1052,7 +1106,9 @@ fn interpret(page: &mut Page, op: &PageOp) -> Result<(), Error> {
1052 y: layout.y.map(stored),1106 y: layout.y.map(stored),
1053 max_width: layout.max_width.map(stored),1107 max_width: layout.max_width.map(stored),
1054 max_height: layout.max_height.map(stored),1108 max_height: layout.max_height.map(stored),
1055 width_set_by_user: layout.max_width.map(|_| layout.width_set_by_user == Some(true)),1109 width_set_by_user: layout
1110 .max_width
1111 .map(|_| layout.width_set_by_user == Some(true)),
1056 reserved_width: image.layout.reserved_width,1112 reserved_width: image.layout.reserved_width,
1057 };1113 };
1058 image.alt = alt.clone();1114 image.alt = alt.clone();
...@@ -1071,7 +1127,8 @@ fn interpret(page: &mut Page, op: &PageOp) -> Result<(), Error> {...@@ -1071,7 +1127,8 @@ fn interpret(page: &mut Page, op: &PageOp) -> Result<(), Error> {
1071 .map(|index| (path, index))1127 .map(|index| (path, index))
1072 })1128 })
1073 .ok_or_else(|| invalid("The attachment is not on the page"))?;1129 .ok_or_else(|| invalid("The attachment is not on the page"))?;
1074 let ParagraphContent::Attachment(stored_attachment) = &mut list_at(page, &path)[index].content1130 let ParagraphContent::Attachment(stored_attachment) =
1131 &mut list_at(page, &path)[index].content
1075 else {1132 else {
1076 unreachable!()1133 unreachable!()
1077 };1134 };
...@@ -1170,8 +1227,9 @@ fn interpret(page: &mut Page, op: &PageOp) -> Result<(), Error> {...@@ -1170,8 +1227,9 @@ fn interpret(page: &mut Page, op: &PageOp) -> Result<(), Error> {
11701227
1171/// Removes outlines a tree edit left without paragraphs, as the tree writer does.1228/// Removes outlines a tree edit left without paragraphs, as the tree writer does.
1172fn drop_emptied(page: &mut Page) {1229fn drop_emptied(page: &mut Page) {
1173 page.objects1230 page.objects.retain(
1174 .retain(|object| !matches!(object, PageObject::Outline(outline) if outline.paragraphs.is_empty()));1231 |object| !matches!(object, PageObject::Outline(outline) if outline.paragraphs.is_empty()),
1232 );
1175}1233}
11761234
1177/// The identity a read-only definition is stored under: an equal one the page holds, as1235/// The identity a read-only definition is stored under: an equal one the page holds, as
...@@ -1283,8 +1341,11 @@ fn text_of(page: &Page, text: ExGuid) -> String {...@@ -1283,8 +1341,11 @@ fn text_of(page: &Page, text: ExGuid) -> String {
1283 lists(page)1341 lists(page)
1284 .into_iter()1342 .into_iter()
1285 .find_map(|(_, _, list)| {1343 .find_map(|(_, _, list)| {
1286 list.iter()1344 list.iter().find_map(|p| {
1287 .find_map(|p| p.text().filter(|t| t.id == text).map(|t| t.text.text().to_owned()))1345 p.text()
1346 .filter(|t| t.id == text)
1347 .map(|t| t.text.text().to_owned())
1348 })
1288 })1349 })
1289 .unwrap_or_default()1350 .unwrap_or_default()
1290}1351}
crates/onestore/src/op/properties.rs+2-2
...@@ -258,8 +258,8 @@ pub(crate) fn style_changes(...@@ -258,8 +258,8 @@ pub(crate) fn style_changes(
258 let raw = &active.live.revision;258 let raw = &active.live.revision;
259 let mut changed = BTreeMap::new();259 let mut changed = BTreeMap::new();
260 if !raw.objects.contains_key(&style) {260 if !raw.objects.contains_key(&style) {
261 let definition =261 let definition = definition
262 definition.ok_or_else(|| invalid("A paragraph references a missing style definition"))?;262 .ok_or_else(|| invalid("A paragraph references a missing style definition"))?;
263 let Kind::Style { name } = &definition.kind else {263 let Kind::Style { name } = &definition.kind else {
264 return Err(invalid("A paragraph style must be a style definition"));264 return Err(invalid("A paragraph style must be a style definition"));
265 };265 };
crates/onestore/src/op/tests.rs+600-138
...@@ -15,21 +15,66 @@ use crate::{...@@ -15,21 +15,66 @@ use crate::{
15use std::collections::{BTreeMap, BTreeSet};15use std::collections::{BTreeMap, BTreeSet};
1616
17const SOURCES: &[(&str, &[u8])] = &[17const SOURCES: &[(&str, &[u8])] = &[
18 ("outline-edit", include_bytes!("../../../../corpus/outline-edit/before/notebook/synthetic.one")),18 (
19 ("paragraph-edit", include_bytes!("../../../../corpus/paragraph-edit/before/notebook/synthetic.one")),19 "outline-edit",
20 ("outline-tree", include_bytes!("../../../../corpus/outline-edit/tree/before/notebook/synthetic.one")),20 include_bytes!("../../../../corpus/outline-edit/before/notebook/synthetic.one"),
21 ("features", include_bytes!("../../../../corpus/m6/native-features-01/notebook/Features.one")),21 ),
22 ("tables", include_bytes!("../../../../corpus/table-edit/nested/cold/notebook/synthetic.one")),22 (
23 ("table-controls", include_bytes!("../../../../corpus/m6/native-table-controls-01/notebook/synthetic.one")),23 "paragraph-edit",
24 ("lists", include_bytes!("../../../../corpus/list-edit/cold/notebook/lists.one")),24 include_bytes!("../../../../corpus/paragraph-edit/before/notebook/synthetic.one"),
25 ("tags", include_bytes!("../../../../corpus/tag-edit/cold/notebook/tags.one")),25 ),
26 ("pictures", include_bytes!("../../../../corpus/picture-edit/native-page-level/notebook/pictures.one")),26 (
27 ("attachments", include_bytes!("../../../../corpus/attachment-edit/plain/cold/notebook/files.one")),27 "outline-tree",
28 ("ink", include_bytes!("../../../../corpus/ink-edit/drawing/cold/notebook/ink.one")),28 include_bytes!("../../../../corpus/outline-edit/tree/before/notebook/synthetic.one"),
29 ("math", include_bytes!("../../../../corpus/math-edit/native-editor/notebook/links.one")),29 ),
30 ("links", include_bytes!("../../../../corpus/link-edit/native-links/notebook/links.one")),30 (
31 ("paragraph-format", include_bytes!("../../../../corpus/paragraph-format/cold/notebook/synthetic.one")),31 "features",
32 ("enter-probe", include_bytes!("../../../../corpus/structural-probe/probe.one")),32 include_bytes!("../../../../corpus/m6/native-features-01/notebook/Features.one"),
33 ),
34 (
35 "tables",
36 include_bytes!("../../../../corpus/table-edit/nested/cold/notebook/synthetic.one"),
37 ),
38 (
39 "table-controls",
40 include_bytes!("../../../../corpus/m6/native-table-controls-01/notebook/synthetic.one"),
41 ),
42 (
43 "lists",
44 include_bytes!("../../../../corpus/list-edit/cold/notebook/lists.one"),
45 ),
46 (
47 "tags",
48 include_bytes!("../../../../corpus/tag-edit/cold/notebook/tags.one"),
49 ),
50 (
51 "pictures",
52 include_bytes!("../../../../corpus/picture-edit/native-page-level/notebook/pictures.one"),
53 ),
54 (
55 "attachments",
56 include_bytes!("../../../../corpus/attachment-edit/plain/cold/notebook/files.one"),
57 ),
58 (
59 "ink",
60 include_bytes!("../../../../corpus/ink-edit/drawing/cold/notebook/ink.one"),
61 ),
62 (
63 "math",
64 include_bytes!("../../../../corpus/math-edit/native-editor/notebook/links.one"),
65 ),
66 (
67 "links",
68 include_bytes!("../../../../corpus/link-edit/native-links/notebook/links.one"),
69 ),
70 (
71 "paragraph-format",
72 include_bytes!("../../../../corpus/paragraph-format/cold/notebook/synthetic.one"),
73 ),
74 (
75 "enter-probe",
76 include_bytes!("../../../../corpus/structural-probe/probe.one"),
77 ),
33];78];
3479
35/// FILETIME both writers take as the edit's time.80/// FILETIME both writers take as the edit's time.
...@@ -55,7 +100,15 @@ impl Rng {...@@ -55,7 +100,15 @@ impl Rng {
55 }100 }
56101
57 fn text(&mut self) -> String {102 fn text(&mut self) -> String {
58 let words = ["", "a", "Two words", "東京 🦀", "é\u{301}", "longer text here", " "];103 let words = [
104 "",
105 "a",
106 "Two words",
107 "東京 🦀",
108 "é\u{301}",
109 "longer text here",
110 " ",
111 ];
59 words[self.next() as usize % words.len()].to_owned()112 words[self.next() as usize % words.len()].to_owned()
60 }113 }
61}114}
...@@ -92,7 +145,10 @@ pub(super) fn normalize(page: &Page) -> Page {...@@ -92,7 +145,10 @@ pub(super) fn normalize(page: &Page) -> Page {
92 let mut page = page.clone();145 let mut page = page.clone();
93 let mut lists = BTreeMap::new();146 let mut lists = BTreeMap::new();
94 let mut canonical = BTreeMap::new();147 let mut canonical = BTreeMap::new();
95 let mut paths: Vec<_> = model::lists(&page).into_iter().map(|(path, _, _)| path).collect();148 let mut paths: Vec<_> = model::lists(&page)
149 .into_iter()
150 .map(|(path, _, _)| path)
151 .collect();
96 paths.sort_by_key(|path| format!("{path:?}"));152 paths.sort_by_key(|path| format!("{path:?}"));
97 let definitions = page.definitions.clone();153 let definitions = page.definitions.clone();
98 for path in &paths {154 for path in &paths {
...@@ -102,7 +158,9 @@ pub(super) fn normalize(page: &Page) -> Page {...@@ -102,7 +158,9 @@ pub(super) fn normalize(page: &Page) -> Page {
102 guid: [0xee; 16],158 guid: [0xee; 16],
103 n: 1 + canonical.len() as u32,159 n: 1 + canonical.len() as u32,
104 };160 };
105 let content = definitions.get(list).map_or(format!("{list:?}"), |d| format!("{d:?}"));161 let content = definitions
162 .get(list)
163 .map_or(format!("{list:?}"), |d| format!("{d:?}"));
106 let id = *canonical.entry(content).or_insert(next);164 let id = *canonical.entry(content).or_insert(next);
107 *list = *lists.entry(*list).or_insert(id);165 *list = *lists.entry(*list).or_insert(id);
108 }166 }
...@@ -185,26 +243,33 @@ fn as_written(after: &Page, before: &Page, stored: &Page) -> Page {...@@ -185,26 +243,33 @@ fn as_written(after: &Page, before: &Page, stored: &Page) -> Page {
185 fn adopt(after: &mut Value, stored: &Value, old: &BTreeSet<String>, new: bool) {243 fn adopt(after: &mut Value, stored: &Value, old: &BTreeSet<String>, new: bool) {
186 match (after, stored) {244 match (after, stored) {
187 (Value::Object(after), Value::Object(stored)) => {245 (Value::Object(after), Value::Object(stored)) => {
188 let new = new246 let new =
189 || matches!(after.get("id"), Some(Value::String(id)) if !old.contains(id));247 new || matches!(after.get("id"), Some(Value::String(id)) if !old.contains(id));
190 for (key, value) in after.iter_mut() {248 for (key, value) in after.iter_mut() {
191 let Some(written) = stored.get(key) else {249 let Some(written) = stored.get(key) else {
192 continue;250 continue;
193 };251 };
194 // Ink coordinates are stored at a coarser resolution.252 // Ink coordinates are stored at a coarser resolution.
195 let near = |a: &Value, b: &Value| {253 let near = |a: &Value, b: &Value| {
196 a.as_f64().zip(b.as_f64()).is_some_and(|(a, b)| (a - b).abs() < 0.05)254 a.as_f64()
255 .zip(b.as_f64())
256 .is_some_and(|(a, b)| (a - b).abs() < 0.05)
197 };257 };
198 let close_points = key == "points"258 let close_points = key == "points"
199 && value.as_array().zip(written.as_array()).is_some_and(|(a, b)| {259 && value
200 a.len() == b.len()260 .as_array()
201 && a.iter().zip(b).all(|(a, b)| {261 .zip(written.as_array())
202 a.as_array().zip(b.as_array()).is_some_and(|(a, b)| {262 .is_some_and(|(a, b)| {
203 a.len() == b.len() && a.iter().zip(b).all(|(a, b)| near(a, b))263 a.len() == b.len()
264 && a.iter().zip(b).all(|(a, b)| {
265 a.as_array().zip(b.as_array()).is_some_and(|(a, b)| {
266 a.len() == b.len()
267 && a.iter().zip(b).all(|(a, b)| near(a, b))
268 })
204 })269 })
205 })270 });
206 });271 let unset =
207 let unset = new && (value.is_null() || value.as_array().is_some_and(Vec::is_empty));272 new && (value.is_null() || value.as_array().is_some_and(Vec::is_empty));
208 if close_points || unset {273 if close_points || unset {
209 *value = written.clone();274 *value = written.clone();
210 } else {275 } else {
...@@ -223,7 +288,12 @@ fn as_written(after: &Page, before: &Page, stored: &Page) -> Page {...@@ -223,7 +288,12 @@ fn as_written(after: &Page, before: &Page, stored: &Page) -> Page {
223 let mut old = BTreeSet::new();288 let mut old = BTreeSet::new();
224 ids(&serde_json::to_value(before).unwrap(), &mut old);289 ids(&serde_json::to_value(before).unwrap(), &mut old);
225 let mut value = serde_json::to_value(after).unwrap();290 let mut value = serde_json::to_value(after).unwrap();
226 adopt(&mut value, &serde_json::to_value(stored).unwrap(), &old, false);291 adopt(
292 &mut value,
293 &serde_json::to_value(stored).unwrap(),
294 &old,
295 false,
296 );
227 let mut page: Page = serde_json::from_value(value).unwrap();297 let mut page: Page = serde_json::from_value(value).unwrap();
228 page.title = stored.title.clone();298 page.title = stored.title.clone();
229 unused_dropped(&mut page);299 unused_dropped(&mut page);
...@@ -323,11 +393,11 @@ pub(super) const FAMILIES: [Family; 33] = [...@@ -323,11 +393,11 @@ pub(super) const FAMILIES: [Family; 33] = [
323];393];
324394
325const PNG: &[u8] = &[395const PNG: &[u8] = &[
326 0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a, 0x00, 0x00, 0x00, 0x0d, 0x49, 0x48, 0x44,396 0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a, 0x00, 0x00, 0x00, 0x0d, 0x49, 0x48, 0x44, 0x52,
327 0x52, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x01, 0x08, 0x06, 0x00, 0x00, 0x00, 0x1f,397 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x01, 0x08, 0x06, 0x00, 0x00, 0x00, 0x1f, 0x15, 0xc4,
328 0x15, 0xc4, 0x89, 0x00, 0x00, 0x00, 0x0d, 0x49, 0x44, 0x41, 0x54, 0x78, 0x9c, 0x63, 0xf8,398 0x89, 0x00, 0x00, 0x00, 0x0d, 0x49, 0x44, 0x41, 0x54, 0x78, 0x9c, 0x63, 0xf8, 0xcf, 0xc0, 0xf0,
329 0xcf, 0xc0, 0xf0, 0x1f, 0x00, 0x05, 0x00, 0x01, 0xff, 0x89, 0x99, 0x3d, 0x1d, 0x00, 0x00,399 0x1f, 0x00, 0x05, 0x00, 0x01, 0xff, 0x89, 0x99, 0x3d, 0x1d, 0x00, 0x00, 0x00, 0x00, 0x49, 0x45,
330 0x00, 0x00, 0x49, 0x45, 0x4e, 0x44, 0xae, 0x42, 0x60, 0x82,400 0x4e, 0x44, 0xae, 0x42, 0x60, 0x82,
331];401];
332402
333fn id() -> ExGuid {403fn id() -> ExGuid {
...@@ -370,7 +440,9 @@ fn editable(page: &Page) -> Vec<model::Path> {...@@ -370,7 +440,9 @@ fn editable(page: &Page) -> Vec<model::Path> {
370 .objects440 .objects
371 .iter()441 .iter()
372 .filter_map(|o| match o {442 .filter_map(|o| match o {
373 PageObject::Title(title) => Some(title.outlines.iter().map(|o| o.id).collect::<Vec<_>>()),443 PageObject::Title(title) => {
444 Some(title.outlines.iter().map(|o| o.id).collect::<Vec<_>>())
445 }
374 _ => None,446 _ => None,
375 })447 })
376 .flatten()448 .flatten()
...@@ -493,7 +565,13 @@ pub(super) fn mutate(page: &mut Page, rng: &mut Rng, family: Family) -> bool {...@@ -493,7 +565,13 @@ pub(super) fn mutate(page: &mut Page, rng: &mut Rng, family: Family) -> bool {
493 0 => format.bold = Some(!format.bold.unwrap_or(false)),565 0 => format.bold = Some(!format.bold.unwrap_or(false)),
494 1 => format.italic = Some(!format.italic.unwrap_or(false)),566 1 => format.italic = Some(!format.italic.unwrap_or(false)),
495 2 => format.underline = Some(!format.underline.unwrap_or(false)),567 2 => format.underline = Some(!format.underline.unwrap_or(false)),
496 3 => format.font_size = Some(if format.font_size == Some(14.0) { 11.0 } else { 14.0 }),568 3 => {
569 format.font_size = Some(if format.font_size == Some(14.0) {
570 11.0
571 } else {
572 14.0
573 })
574 }
497 4 => format.color = Some(0x00ff),575 4 => format.color = Some(0x00ff),
498 _ => format.font = Some("Consolas".into()),576 _ => format.font = Some("Consolas".into()),
499 }577 }
...@@ -510,7 +588,10 @@ pub(super) fn mutate(page: &mut Page, rng: &mut Rng, family: Family) -> bool {...@@ -510,7 +588,10 @@ pub(super) fn mutate(page: &mut Page, rng: &mut Rng, family: Family) -> bool {
510 }588 }
511 _ => {589 _ => {
512 if range.is_empty()590 if range.is_empty()
513 || text.spans().iter().any(|s| s.format.hyperlink == Some(true))591 || text
592 .spans()
593 .iter()
594 .any(|s| s.format.hyperlink == Some(true))
514 {595 {
515 return false;596 return false;
516 }597 }
...@@ -579,7 +660,11 @@ pub(super) fn mutate(page: &mut Page, rng: &mut Rng, family: Family) -> bool {...@@ -579,7 +660,11 @@ pub(super) fn mutate(page: &mut Page, rng: &mut Rng, family: Family) -> bool {
579 .filter(|i| list.get(*i).is_none_or(|p| p.parent.is_none()))660 .filter(|i| list.get(*i).is_none_or(|p| p.parent.is_none()))
580 .collect();661 .collect();
581 let at = boundaries[rng.pick(boundaries.len()).unwrap()];662 let at = boundaries[rng.pick(boundaries.len()).unwrap()];
582 let count = if family == Family::Paste { 2 + rng.next() as usize % 3 } else { 1 };663 let count = if family == Family::Paste {
664 2 + rng.next() as usize % 3
665 } else {
666 1
667 };
583 let mut inserted = Vec::new();668 let mut inserted = Vec::new();
584 for i in 0..count {669 for i in 0..count {
585 let mut paragraph = text_paragraph(&rng.text(), template.clone());670 let mut paragraph = text_paragraph(&rng.text(), template.clone());
...@@ -612,7 +697,9 @@ pub(super) fn mutate(page: &mut Page, rng: &mut Rng, family: Family) -> bool {...@@ -612,7 +697,9 @@ pub(super) fn mutate(page: &mut Page, rng: &mut Rng, family: Family) -> bool {
612 return false;697 return false;
613 };698 };
614 let list = model::list_at(page, &path);699 let list = model::list_at(page, &path);
615 let top: Vec<usize> = (0..list.len()).filter(|i| list[*i].parent.is_none()).collect();700 let top: Vec<usize> = (0..list.len())
701 .filter(|i| list[*i].parent.is_none())
702 .collect();
616 match family {703 match family {
617 Family::Delete => {704 Family::Delete => {
618 if top.len() < 2 {705 if top.len() < 2 {
...@@ -645,7 +732,8 @@ pub(super) fn mutate(page: &mut Page, rng: &mut Rng, family: Family) -> bool {...@@ -645,7 +732,8 @@ pub(super) fn mutate(page: &mut Page, rng: &mut Rng, family: Family) -> bool {
645 }732 }
646 }733 }
647 Family::Position | Family::Width => {734 Family::Position | Family::Width => {
648 let mut outlines: Vec<&mut Outline> = outlines(page).into_iter().filter(|o| !o.title).collect();735 let mut outlines: Vec<&mut Outline> =
736 outlines(page).into_iter().filter(|o| !o.title).collect();
649 let Some(at) = rng.pick(outlines.len()) else {737 let Some(at) = rng.pick(outlines.len()) else {
650 return false;738 return false;
651 };739 };
...@@ -784,7 +872,11 @@ pub(super) fn mutate(page: &mut Page, rng: &mut Rng, family: Family) -> bool {...@@ -784,7 +872,11 @@ pub(super) fn mutate(page: &mut Page, rng: &mut Rng, family: Family) -> bool {
784 let paragraph = &list[index];872 let paragraph = &list[index];
785 let text = &paragraph.text().unwrap().text;873 let text = &paragraph.text().unwrap().text;
786 let end = text.utf16_offset(text.text().len()).unwrap();874 let end = text.utf16_offset(text.text().len()).unwrap();
787 if text.spans().iter().any(|s| s.format.hyperlink == Some(true)) {875 if text
876 .spans()
877 .iter()
878 .any(|s| s.format.hyperlink == Some(true))
879 {
788 return false;880 return false;
789 }881 }
790 let offsets: Vec<u32> = text882 let offsets: Vec<u32> = text
...@@ -822,14 +914,18 @@ pub(super) fn mutate(page: &mut Page, rng: &mut Rng, family: Family) -> bool {...@@ -822,14 +914,18 @@ pub(super) fn mutate(page: &mut Page, rng: &mut Rng, family: Family) -> bool {
822 let upper = &list[index];914 let upper = &list[index];
823 if lower.parent != upper.parent915 if lower.parent != upper.parent
824 || lower.level != upper.level916 || lower.level != upper.level
825 || list.iter().any(|p| p.parent == Some(upper.id) || p.parent == Some(lower.id))917 || list
918 .iter()
919 .any(|p| p.parent == Some(upper.id) || p.parent == Some(lower.id))
826 {920 {
827 return false;921 return false;
828 }922 }
829 let Some(lower_text) = lower.text().cloned() else {923 let Some(lower_text) = lower.text().cloned() else {
830 return false;924 return false;
831 };925 };
832 if lower_text.text.text().is_empty() || crate::page::Math::is_equation(&lower_text.text) {926 if lower_text.text.text().is_empty()
927 || crate::page::Math::is_equation(&lower_text.text)
928 {
833 return false;929 return false;
834 }930 }
835 let upper = list[index].text_mut().unwrap();931 let upper = list[index].text_mut().unwrap();
...@@ -859,7 +955,12 @@ pub(super) fn mutate(page: &mut Page, rng: &mut Rng, family: Family) -> bool {...@@ -859,7 +955,12 @@ pub(super) fn mutate(page: &mut Page, rng: &mut Rng, family: Family) -> bool {
859 if family == Family::Indent {955 if family == Family::Indent {
860 // Tab: the paragraph becomes the last child of its previous sibling.956 // Tab: the paragraph becomes the last child of its previous sibling.
861 let parent = list[index].parent;957 let parent = list[index].parent;
862 let Some(sibling) = list[..index].iter().rev().find(|p| p.parent == parent).map(|p| (p.id, p.level)) else {958 let Some(sibling) = list[..index]
959 .iter()
960 .rev()
961 .find(|p| p.parent == parent)
962 .map(|p| (p.id, p.level))
963 else {
863 return false;964 return false;
864 };965 };
865 list[index].parent = Some(sibling.0);966 list[index].parent = Some(sibling.0);
...@@ -912,7 +1013,13 @@ pub(super) fn mutate(page: &mut Page, rng: &mut Rng, family: Family) -> bool {...@@ -912,7 +1013,13 @@ pub(super) fn mutate(page: &mut Page, rng: &mut Rng, family: Family) -> bool {
912 }1013 }
913 Family::TableColumn => {1014 Family::TableColumn => {
914 let at = rng.pick(table.columns.len() + 1).unwrap();1015 let at = rng.pick(table.columns.len() + 1).unwrap();
915 table.columns.insert(at, TableColumn { width: 72.0, locked: false });1016 table.columns.insert(
1017 at,
1018 TableColumn {
1019 width: 72.0,
1020 locked: false,
1021 },
1022 );
916 for row in &mut table.rows {1023 for row in &mut table.rows {
917 row.cells.insert(at, new_cell(&indents));1024 row.cells.insert(at, new_cell(&indents));
918 }1025 }
...@@ -940,7 +1047,11 @@ pub(super) fn mutate(page: &mut Page, rng: &mut Rng, family: Family) -> bool {...@@ -940,7 +1047,11 @@ pub(super) fn mutate(page: &mut Page, rng: &mut Rng, family: Family) -> bool {
940 let cells: Vec<_> = table.rows.iter_mut().flat_map(|r| &mut r.cells).collect();1047 let cells: Vec<_> = table.rows.iter_mut().flat_map(|r| &mut r.cells).collect();
941 let len = cells.len();1048 let len = cells.len();
942 let cell = cells.into_iter().nth(rng.pick(len).unwrap()).unwrap();1049 let cell = cells.into_iter().nth(rng.pick(len).unwrap()).unwrap();
943 cell.shading = if cell.shading.is_some() { None } else { Some(0x00ccff) };1050 cell.shading = if cell.shading.is_some() {
1051 None
1052 } else {
1053 Some(0x00ccff)
1054 };
944 }1055 }
945 }1056 }
946 true1057 true
...@@ -948,7 +1059,9 @@ pub(super) fn mutate(page: &mut Page, rng: &mut Rng, family: Family) -> bool {...@@ -948,7 +1059,9 @@ pub(super) fn mutate(page: &mut Page, rng: &mut Rng, family: Family) -> bool {
948 Family::NewTable | Family::ParagraphPicture | Family::Attachment | Family::Equation => {1059 Family::NewTable | Family::ParagraphPicture | Family::Attachment | Family::Equation => {
949 let equation = page.objects.iter().find_map(|o| match o {1060 let equation = page.objects.iter().find_map(|o| match o {
950 PageObject::Outline(outline) => outline.paragraphs.iter().find_map(|p| {1061 PageObject::Outline(outline) => outline.paragraphs.iter().find_map(|p| {
951 p.text().filter(|t| crate::page::Math::is_equation(&t.text)).map(|t| t.text.clone())1062 p.text()
1063 .filter(|t| crate::page::Math::is_equation(&t.text))
1064 .map(|t| t.text.clone())
952 }),1065 }),
953 _ => None,1066 _ => None,
954 });1067 });
...@@ -959,7 +1072,13 @@ pub(super) fn mutate(page: &mut Page, rng: &mut Rng, family: Family) -> bool {...@@ -959,7 +1072,13 @@ pub(super) fn mutate(page: &mut Page, rng: &mut Rng, family: Family) -> bool {
959 paragraph.content = match family {1072 paragraph.content = match family {
960 Family::NewTable => ParagraphContent::Table(Table {1073 Family::NewTable => ParagraphContent::Table(Table {
961 id: id(),1074 id: id(),
962 columns: vec![TableColumn { width: 72.0, locked: false }; 2],1075 columns: vec![
1076 TableColumn {
1077 width: 72.0,
1078 locked: false
1079 };
1080 2
1081 ],
963 rows: (0..2)1082 rows: (0..2)
964 .map(|_| TableRow {1083 .map(|_| TableRow {
965 id: id(),1084 id: id(),
...@@ -1000,7 +1119,12 @@ pub(super) fn mutate(page: &mut Page, rng: &mut Rng, family: Family) -> bool {...@@ -1000,7 +1119,12 @@ pub(super) fn mutate(page: &mut Page, rng: &mut Rng, family: Family) -> bool {
1000 }1119 }
1001 };1120 };
1002 let boundaries: Vec<usize> = (0..=outline.paragraphs.len())1121 let boundaries: Vec<usize> = (0..=outline.paragraphs.len())
1003 .filter(|i| outline.paragraphs.get(*i).is_none_or(|p| p.parent.is_none()))1122 .filter(|i| {
1123 outline
1124 .paragraphs
1125 .get(*i)
1126 .is_none_or(|p| p.parent.is_none())
1127 })
1004 .collect();1128 .collect();
1005 let at = boundaries[rng.pick(boundaries.len()).unwrap()];1129 let at = boundaries[rng.pick(boundaries.len()).unwrap()];
1006 outline.paragraphs.insert(at, paragraph);1130 outline.paragraphs.insert(at, paragraph);
...@@ -1139,7 +1263,13 @@ pub(super) fn stored(source: &[u8], space: ExGuid, after: &Page, seed: u64) -> O...@@ -1139,7 +1263,13 @@ pub(super) fn stored(source: &[u8], space: ExGuid, after: &Page, seed: u64) -> O
1139 };1263 };
1140 let edit = Edit {1264 let edit = Edit {
1141 at: AT,1265 at: AT,
1142 ops: ops.iter().map(|op| Op::Page { space, op: op.clone() }).collect(),1266 ops: ops
1267 .iter()
1268 .map(|op| Op::Page {
1269 space,
1270 op: op.clone(),
1271 })
1272 .collect(),
1143 };1273 };
1144 if seeded(seed, || section.apply("Author", &edit)).is_err() {1274 if seeded(seed, || section.apply("Author", &edit)).is_err() {
1145 if section.page(space).unwrap() != before {1275 if section.page(space).unwrap() != before {
...@@ -1159,7 +1289,9 @@ pub(super) fn stored(source: &[u8], space: ExGuid, after: &Page, seed: u64) -> O...@@ -1159,7 +1289,9 @@ pub(super) fn stored(source: &[u8], space: ExGuid, after: &Page, seed: u64) -> O
1159 ));1289 ));
1160 }1290 }
1161 let Some(_) = seeded(seed, || section.seal().unwrap()) else {1291 let Some(_) = seeded(seed, || section.seal().unwrap()) else {
1162 return if live == before { Outcome::Unchanged } else {1292 return if live == before {
1293 Outcome::Unchanged
1294 } else {
1163 Outcome::Mismatch("the page changed and the seal stored nothing".into())1295 Outcome::Mismatch("the page changed and the seal stored nothing".into())
1164 };1296 };
1165 };1297 };
...@@ -1171,12 +1303,18 @@ pub(super) fn stored(source: &[u8], space: ExGuid, after: &Page, seed: u64) -> O...@@ -1171,12 +1303,18 @@ pub(super) fn stored(source: &[u8], space: ExGuid, after: &Page, seed: u64) -> O
1171 }1303 }
1172 let reread = read(&image, space);1304 let reread = read(&image, space);
1173 if reread != live {1305 if reread != live {
1174 return Outcome::Mismatch(format!("reads back otherwise: {}", first_difference(&live, &reread)));1306 return Outcome::Mismatch(format!(
1307 "reads back otherwise: {}",
1308 first_difference(&live, &reread)
1309 ));
1175 }1310 }
1176 let written = as_written(after, &before, &reread);1311 let written = as_written(after, &before, &reread);
1177 let (stored, wanted) = (normalize(&reread), normalize(&written));1312 let (stored, wanted) = (normalize(&reread), normalize(&written));
1178 if stored != wanted {1313 if stored != wanted {
1179 return Outcome::Mismatch(format!("stores another page: {}", first_difference(&wanted, &stored)));1314 return Outcome::Mismatch(format!(
1315 "stores another page: {}",
1316 first_difference(&wanted, &stored)
1317 ));
1180 }1318 }
1181 Outcome::Stored1319 Outcome::Stored
1182}1320}
...@@ -1187,7 +1325,16 @@ fn spans(text: &Paragraph) -> String {...@@ -1187,7 +1325,16 @@ fn spans(text: &Paragraph) -> String {
1187 let f = &span.format;1325 let f = &span.format;
1188 out.push_str(&format!(1326 out.push_str(&format!(
1189 " [{} b{:?} h{:?} l{:?}/{:?} c{:?} a{:?} s{:?} lang{:?} f{:?}]",1327 " [{} b{:?} h{:?} l{:?}/{:?} c{:?} a{:?} s{:?} lang{:?} f{:?}]",
1190 span.end, f.bold, f.hidden, f.hyperlink, f.hyperlink_label, f.color, f.alignment, f.space_before, f.language, f.font1328 span.end,
1329 f.bold,
1330 f.hidden,
1331 f.hyperlink,
1332 f.hyperlink_label,
1333 f.color,
1334 f.alignment,
1335 f.space_before,
1336 f.language,
1337 f.font
1191 ));1338 ));
1192 }1339 }
1193 out1340 out
...@@ -1243,7 +1390,8 @@ fn lowered_edits_store_the_edited_page() {...@@ -1243,7 +1390,8 @@ fn lowered_edits_store_the_edited_page() {
1243 };1390 };
1244 *tally.entry((family, key)).or_default() += 1;1391 *tally.entry((family, key)).or_default() += 1;
1245 if let Outcome::Mismatch(detail) = outcome {1392 if let Outcome::Mismatch(detail) = outcome {
1246 mismatches.push(format!("{name} page {p} {family:?} seed {seed}: {detail}"));1393 mismatches
1394 .push(format!("{name} page {p} {family:?} seed {seed}: {detail}"));
1247 }1395 }
1248 }1396 }
1249 }1397 }
...@@ -1279,7 +1427,25 @@ fn debug_case() {...@@ -1279,7 +1427,25 @@ fn debug_case() {
1279 model::apply(&mut page, op).unwrap();1427 model::apply(&mut page, op).unwrap();
1280 }1428 }
1281 for object in &page.objects {1429 for object in &page.objects {
1282 println!("{:?} {:?} {:?}", object.id(), object.layout(), match object { PageObject::Outline(o) => o.paragraphs.iter().filter_map(|p| p.text().map(|t| t.text.text().to_owned())).collect::<Vec<_>>(), PageObject::Title(t) => t.outlines.iter().flat_map(|o| &o.paragraphs).filter_map(|p| p.text().map(|t| format!("title {:?}", t.text.text()))).collect(), _ => vec![] });1430 println!(
1431 "{:?} {:?} {:?}",
1432 object.id(),
1433 object.layout(),
1434 match object {
1435 PageObject::Outline(o) => o
1436 .paragraphs
1437 .iter()
1438 .filter_map(|p| p.text().map(|t| t.text.text().to_owned()))
1439 .collect::<Vec<_>>(),
1440 PageObject::Title(t) => t
1441 .outlines
1442 .iter()
1443 .flat_map(|o| &o.paragraphs)
1444 .filter_map(|p| p.text().map(|t| format!("title {:?}", t.text.text())))
1445 .collect(),
1446 _ => vec![],
1447 }
1448 );
1283 }1449 }
1284 println!("{:?} -> {:?}", before.title, page.title);1450 println!("{:?} -> {:?}", before.title, page.title);
1285}1451}
...@@ -1334,7 +1500,11 @@ fn random_op(page: &Page, rng: &mut Rng) -> Option<PageOp> {...@@ -1334,7 +1500,11 @@ fn random_op(page: &Page, rng: &mut Rng) -> Option<PageOp> {
1334 text,1500 text,
1335 range,1501 range,
1336 set: vec![TextAttribute::Bold(rng.coin())],1502 set: vec![TextAttribute::Bold(rng.coin())],
1337 clear: if rng.next().is_multiple_of(3) { vec![TextProperty::Italic] } else { Vec::new() },1503 clear: if rng.next().is_multiple_of(3) {
1504 vec![TextProperty::Italic]
1505 } else {
1506 Vec::new()
1507 },
1338 },1508 },
1339 3 => PageOp::Split {1509 3 => PageOp::Split {
1340 text,1510 text,
...@@ -1357,7 +1527,9 @@ fn random_op(page: &Page, rng: &mut Rng) -> Option<PageOp> {...@@ -1357,7 +1527,9 @@ fn random_op(page: &Page, rng: &mut Rng) -> Option<PageOp> {
1357 6 => PageOp::Move {1527 6 => PageOp::Move {
1358 object: paragraph,1528 object: paragraph,
1359 parent: Some(container),1529 parent: Some(container),
1360 before: siblings.get(rng.pick(siblings.len().max(1)).unwrap_or(0)).copied(),1530 before: siblings
1531 .get(rng.pick(siblings.len().max(1)).unwrap_or(0))
1532 .copied(),
1361 },1533 },
1362 7 => PageOp::Level {1534 7 => PageOp::Level {
1363 paragraph,1535 paragraph,
...@@ -1377,7 +1549,10 @@ fn random_op(page: &Page, rng: &mut Rng) -> Option<PageOp> {...@@ -1377,7 +1549,10 @@ fn random_op(page: &Page, rng: &mut Rng) -> Option<PageOp> {
1377 language: None,1549 language: None,
1378 },1550 },
1379 10 => {1551 10 => {
1380 let right = texts.iter().skip_while(|(p, _, _)| *p != paragraph).nth(1)?;1552 let right = texts
1553 .iter()
1554 .skip_while(|(p, _, _)| *p != paragraph)
1555 .nth(1)?;
1381 PageOp::Join {1556 PageOp::Join {
1382 left: text,1557 left: text,
1383 right: right.1,1558 right: right.1,
...@@ -1505,7 +1680,10 @@ fn random_object_op(page: &Page, rng: &mut Rng) -> Option<PageOp> {...@@ -1505,7 +1680,10 @@ fn random_object_op(page: &Page, rng: &mut Rng) -> Option<PageOp> {
1505 };1680 };
1506 let edit = match rng.next() % 7 {1681 let edit = match rng.next() % 7 {
1507 0 => TableEdit::Rows {1682 0 => TableEdit::Rows {
1508 before: table.rows.get(rng.pick(table.rows.len() + 1)?).map(|r| r.id),1683 before: table
1684 .rows
1685 .get(rng.pick(table.rows.len() + 1)?)
1686 .map(|r| r.id),
1509 rows: vec![TableRow {1687 rows: vec![TableRow {
1510 id: id(),1688 id: id(),
1511 cells: table.columns.iter().map(|_| cell()).collect(),1689 cells: table.columns.iter().map(|_| cell()).collect(),
...@@ -1639,7 +1817,10 @@ fn random_ops_read_back_as_the_model_predicts() {...@@ -1639,7 +1817,10 @@ fn random_ops_read_back_as_the_model_predicts() {
1639 if ops.is_empty() {1817 if ops.is_empty() {
1640 continue;1818 continue;
1641 }1819 }
1642 let edit = Edit { at: AT + step as u64, ops };1820 let edit = Edit {
1821 at: AT + step as u64,
1822 ops,
1823 };
1643 // Text whose emptied final run keeps an insertion style the model cannot show.1824 // Text whose emptied final run keeps an insertion style the model cannot show.
1644 let hidden: Vec<ExGuid> = section1825 let hidden: Vec<ExGuid> = section
1645 .active(space)1826 .active(space)
...@@ -1658,7 +1839,9 @@ fn random_ops_read_back_as_the_model_predicts() {...@@ -1658,7 +1839,9 @@ fn random_ops_read_back_as_the_model_predicts() {
1658 applied += 1;1839 applied += 1;
1659 let mut predicted = before.clone();1840 let mut predicted = before.clone();
1660 for op in &edit.ops {1841 for op in &edit.ops {
1661 let Op::Page { op, .. } = op else { unreachable!() };1842 let Op::Page { op, .. } = op else {
1843 unreachable!()
1844 };
1662 model::apply(&mut predicted, op).unwrap();1845 model::apply(&mut predicted, op).unwrap();
1663 }1846 }
1664 let stored = section.page(space).unwrap();1847 let stored = section.page(space).unwrap();
...@@ -1671,7 +1854,9 @@ fn random_ops_read_back_as_the_model_predicts() {...@@ -1671,7 +1854,9 @@ fn random_ops_read_back_as_the_model_predicts() {
1671 matches!(op, Op::Page { op: PageOp::Style { paragraph, .. }, .. }1854 matches!(op, Op::Page { op: PageOp::Style { paragraph, .. }, .. }
1672 if model::paragraph(&before, *paragraph).is_some_and(|p| p.style.is_some()))1855 if model::paragraph(&before, *paragraph).is_some_and(|p| p.style.is_some()))
1673 });1856 });
1674 if (touches_hidden || restyles) && normalize(&predicted) != normalize(&stored) {1857 if (touches_hidden || restyles)
1858 && normalize(&predicted) != normalize(&stored)
1859 {
1675 unseen += 1;1860 unseen += 1;
1676 continue;1861 continue;
1677 }1862 }
...@@ -1680,7 +1865,11 @@ fn random_ops_read_back_as_the_model_predicts() {...@@ -1680,7 +1865,11 @@ fn random_ops_read_back_as_the_model_predicts() {
1680 .ops1865 .ops
1681 .iter()1866 .iter()
1682 .map(|op| match op {1867 .map(|op| match op {
1683 Op::Page { op, .. } => format!("{op:?}").split([' ', '{']).next().unwrap().to_owned(),1868 Op::Page { op, .. } => format!("{op:?}")
1869 .split([' ', '{'])
1870 .next()
1871 .unwrap()
1872 .to_owned(),
1684 Op::Section(_) => "Section".into(),1873 Op::Section(_) => "Section".into(),
1685 })1874 })
1686 .collect();1875 .collect();
...@@ -1693,7 +1882,12 @@ fn random_ops_read_back_as_the_model_predicts() {...@@ -1693,7 +1882,12 @@ fn random_ops_read_back_as_the_model_predicts() {
1693 "{}: {}",1882 "{}: {}",
1694 owner.n,1883 owner.n,
1695 list.iter()1884 list.iter()
1696 .map(|p| format!("{}<{:?}>@{}", p.id.n, p.parent.map(|p| p.n), p.level))1885 .map(|p| format!(
1886 "{}<{:?}>@{}",
1887 p.id.n,
1888 p.parent.map(|p| p.n),
1889 p.level
1890 ))
1697 .collect::<Vec<_>>()1891 .collect::<Vec<_>>()
1698 .join(" ")1892 .join(" ")
1699 )1893 )
...@@ -1701,7 +1895,13 @@ fn random_ops_read_back_as_the_model_predicts() {...@@ -1701,7 +1895,13 @@ fn random_ops_read_back_as_the_model_predicts() {
1701 .collect::<Vec<_>>()1895 .collect::<Vec<_>>()
1702 .join("\n")1896 .join("\n")
1703 };1897 };
1704 println!("ops {:?}\nbefore\n{}\nstored\n{}\npredicted\n{}", edit.ops, tree(&before), tree(&stored), tree(&predicted));1898 println!(
1899 "ops {:?}\nbefore\n{}\nstored\n{}\npredicted\n{}",
1900 edit.ops,
1901 tree(&before),
1902 tree(&stored),
1903 tree(&predicted)
1904 );
1705 }1905 }
1706 if mismatches.len() < 6 {1906 if mismatches.len() < 6 {
1707 println!(1907 println!(
...@@ -1715,7 +1915,10 @@ fn random_ops_read_back_as_the_model_predicts() {...@@ -1715,7 +1915,10 @@ fn random_ops_read_back_as_the_model_predicts() {
1715 Err(OpError::Failed(error)) => panic!("{name} step {step}: {error:?}"),1915 Err(OpError::Failed(error)) => panic!("{name} step {step}: {error:?}"),
1716 Err(_) => {1916 Err(_) => {
1717 refused += 1;1917 refused += 1;
1718 assert!(section.page(space).unwrap() == before, "{name} step {step}: a refused edit changed the page");1918 assert!(
1919 section.page(space).unwrap() == before,
1920 "{name} step {step}: a refused edit changed the page"
1921 );
1719 }1922 }
1720 }1923 }
1721 if step % 7 == 6 || step + 1 == steps {1924 if step % 7 == 6 || step + 1 == steps {
...@@ -1723,13 +1926,18 @@ fn random_ops_read_back_as_the_model_predicts() {...@@ -1723,13 +1926,18 @@ fn random_ops_read_back_as_the_model_predicts() {
1723 let image = section.image();1926 let image = section.image();
1724 let reopened = Section::open(&arena, image).unwrap();1927 let reopened = Section::open(&arena, image).unwrap();
1725 for space in &spaces {1928 for space in &spaces {
1726 assert!(reopened.page(*space).unwrap() == section.page(*space).unwrap(), "{name} step {step}: reopening changed a page");1929 assert!(
1930 reopened.page(*space).unwrap() == section.page(*space).unwrap(),
1931 "{name} step {step}: reopening changed a page"
1932 );
1727 }1933 }
1728 }1934 }
1729 }1935 }
1730 }1936 }
1731 }1937 }
1732 println!("{applied} edits applied, {refused} refused, {unseen} restyled or typed into a hidden insertion style");1938 println!(
1939 "{applied} edits applied, {refused} refused, {unseen} restyled or typed into a hidden insertion style"
1940 );
1733 let mut kinds = BTreeMap::new();1941 let mut kinds = BTreeMap::new();
1734 for kind in &mismatches {1942 for kind in &mismatches {
1735 *kinds.entry(kind.clone()).or_insert(0) += 1;1943 *kinds.entry(kind.clone()).or_insert(0) += 1;
...@@ -1745,7 +1953,8 @@ fn random_ops_read_back_as_the_model_predicts() {...@@ -1745,7 +1953,8 @@ fn random_ops_read_back_as_the_model_predicts() {
1745fn section_ops_leave_the_page_list_they_describe() {1953fn section_ops_leave_the_page_list_they_describe() {
1746 let sources: Vec<Vec<u8>> = vec![1954 let sources: Vec<Vec<u8>> = vec![
1747 crate::create_section("pages.one", "First", "Author").unwrap(),1955 crate::create_section("pages.one", "First", "Author").unwrap(),
1748 include_bytes!("../../../../corpus/page-lifecycle/04-nested/notebook/Lifecycle.one").to_vec(),1956 include_bytes!("../../../../corpus/page-lifecycle/04-nested/notebook/Lifecycle.one")
1957 .to_vec(),
1749 SOURCES[3].1.to_vec(),1958 SOURCES[3].1.to_vec(),
1750 ];1959 ];
1751 let mut applied = 0;1960 let mut applied = 0;
...@@ -1756,16 +1965,31 @@ fn section_ops_leave_the_page_list_they_describe() {...@@ -1756,16 +1965,31 @@ fn section_ops_leave_the_page_list_they_describe() {
1756 for step in 0..12u64 {1965 for step in 0..12u64 {
1757 let listed = section.pages().unwrap();1966 let listed = section.pages().unwrap();
1758 let spaces: Vec<ExGuid> = listed.iter().map(|(space, ..)| *space).collect();1967 let spaces: Vec<ExGuid> = listed.iter().map(|(space, ..)| *space).collect();
1759 let anchor = spaces.get(rng.pick(spaces.len()).unwrap_or(0)).copied().filter(|_| rng.coin());1968 let anchor = spaces
1969 .get(rng.pick(spaces.len()).unwrap_or(0))
1970 .copied()
1971 .filter(|_| rng.coin());
1760 let op = match step % 4 {1972 let op = match step % 4 {
1761 0 => SectionOp::Create(crate::PageCreation::new(anchor, Some("Created"), "Author").unwrap()),1973 0 => SectionOp::Create(
1974 crate::PageCreation::new(anchor, Some("Created"), "Author").unwrap(),
1975 ),
1762 1 => {1976 1 => {
1763 let Some(space) = spaces.get(rng.pick(spaces.len()).unwrap_or(0)).copied() else { continue };1977 let Some(space) = spaces.get(rng.pick(spaces.len()).unwrap_or(0)).copied()
1764 let level = if spaces[0] == space { 1 } else { 1 + (rng.next() % 2) as u32 };1978 else {
1979 continue;
1980 };
1981 let level = if spaces[0] == space {
1982 1
1983 } else {
1984 1 + (rng.next() % 2) as u32
1985 };
1765 SectionOp::Pages(vec![crate::PageEdit::set_level(space, level).unwrap()])1986 SectionOp::Pages(vec![crate::PageEdit::set_level(space, level).unwrap()])
1766 }1987 }
1767 2 => {1988 2 => {
1768 let Some(space) = spaces.get(rng.pick(spaces.len()).unwrap_or(0)).copied() else { continue };1989 let Some(space) = spaces.get(rng.pick(spaces.len()).unwrap_or(0)).copied()
1990 else {
1991 continue;
1992 };
1769 SectionOp::Pages(vec![crate::PageEdit::move_to(space, None, 1).unwrap()])1993 SectionOp::Pages(vec![crate::PageEdit::move_to(space, None, 1).unwrap()])
1770 }1994 }
1771 _ => {1995 _ => {
...@@ -1775,7 +1999,10 @@ fn section_ops_leave_the_page_list_they_describe() {...@@ -1775,7 +1999,10 @@ fn section_ops_leave_the_page_list_they_describe() {
1775 SectionOp::Delete(vec![spaces[rng.pick(spaces.len()).unwrap()]])1999 SectionOp::Delete(vec![spaces[rng.pick(spaces.len()).unwrap()]])
1776 }2000 }
1777 };2001 };
1778 let edit = Edit { at: AT, ops: vec![Op::Section(op.clone())] };2002 let edit = Edit {
2003 at: AT,
2004 ops: vec![Op::Section(op.clone())],
2005 };
1779 if section.apply("Author", &edit).is_err() {2006 if section.apply("Author", &edit).is_err() {
1780 assert_eq!(section.pages().unwrap(), listed, "{n} step {step} {op:?}");2007 assert_eq!(section.pages().unwrap(), listed, "{n} step {step} {op:?}");
1781 continue;2008 continue;
...@@ -1784,11 +2011,17 @@ fn section_ops_leave_the_page_list_they_describe() {...@@ -1784,11 +2011,17 @@ fn section_ops_leave_the_page_list_they_describe() {
1784 applied += 1;2011 applied += 1;
1785 let after = section.pages().unwrap();2012 let after = section.pages().unwrap();
1786 let order = |list: &[(ExGuid, String, u32)], without: ExGuid| -> Vec<ExGuid> {2013 let order = |list: &[(ExGuid, String, u32)], without: ExGuid| -> Vec<ExGuid> {
1787 list.iter().map(|(space, ..)| *space).filter(|space| *space != without).collect()2014 list.iter()
2015 .map(|(space, ..)| *space)
2016 .filter(|space| *space != without)
2017 .collect()
1788 };2018 };
1789 match &op {2019 match &op {
1790 SectionOp::Create(creation) => {2020 SectionOp::Create(creation) => {
1791 let at = after.iter().position(|(space, ..)| *space == creation.space()).unwrap();2021 let at = after
2022 .iter()
2023 .position(|(space, ..)| *space == creation.space())
2024 .unwrap();
1792 assert_eq!(after[at].1, "Created");2025 assert_eq!(after[at].1, "Created");
1793 assert_eq!(order(&after, creation.space()), spaces, "{n} step {step}");2026 assert_eq!(order(&after, creation.space()), spaces, "{n} step {step}");
1794 match anchor {2027 match anchor {
...@@ -1798,7 +2031,10 @@ fn section_ops_leave_the_page_list_they_describe() {...@@ -1798,7 +2031,10 @@ fn section_ops_leave_the_page_list_they_describe() {
1798 }2031 }
1799 SectionOp::Pages(edits) => {2032 SectionOp::Pages(edits) => {
1800 let space = edits[0].space();2033 let space = edits[0].space();
1801 let at = after.iter().position(|(listed, ..)| *listed == space).unwrap();2034 let at = after
2035 .iter()
2036 .position(|(listed, ..)| *listed == space)
2037 .unwrap();
1802 assert_eq!(after[at].2, edits[0].level(), "{n} step {step} {op:?}");2038 assert_eq!(after[at].2, edits[0].level(), "{n} step {step} {op:?}");
1803 if edits[0].position() == crate::PagePosition::Keep {2039 if edits[0].position() == crate::PagePosition::Keep {
1804 assert_eq!(order(&after, space), order(&listed, space));2040 assert_eq!(order(&after, space), order(&listed, space));
...@@ -1808,11 +2044,16 @@ fn section_ops_leave_the_page_list_they_describe() {...@@ -1808,11 +2044,16 @@ fn section_ops_leave_the_page_list_they_describe() {
1808 }2044 }
1809 }2045 }
1810 SectionOp::Delete(deleted) => {2046 SectionOp::Delete(deleted) => {
1811 assert_eq!(after.iter().map(|(s, ..)| *s).collect::<Vec<_>>(), order(&listed, deleted[0]));2047 assert_eq!(
1812 }2048 after.iter().map(|(s, ..)| *s).collect::<Vec<_>>(),
1813 SectionOp::Import { .. } | SectionOp::Conflict { .. } | SectionOp::Color(_) => {2049 order(&listed, deleted[0])
1814 unreachable!()2050 );
1815 }2051 }
2052 SectionOp::Import { .. }
2053 | SectionOp::Conflict { .. }
2054 | SectionOp::Color(_)
2055 | SectionOp::RestoreVersion { .. }
2056 | SectionOp::DeleteVersions { .. } => unreachable!(),
1816 }2057 }
1817 let image = section.image();2058 let image = section.image();
1818 let store = Store::parse(&image).unwrap();2059 let store = Store::parse(&image).unwrap();
...@@ -1830,7 +2071,11 @@ fn section_ops_leave_the_page_list_they_describe() {...@@ -1830,7 +2071,11 @@ fn section_ops_leave_the_page_list_they_describe() {
1830 .collect();2071 .collect();
1831 assert_eq!(reread, after, "{n} step {step}");2072 assert_eq!(reread, after, "{n} step {step}");
1832 for (space, ..) in &after {2073 for (space, ..) in &after {
1833 assert_eq!(read(&image, *space), section.page(*space).unwrap(), "{n} step {step}");2074 assert_eq!(
2075 read(&image, *space),
2076 section.page(*space).unwrap(),
2077 "{n} step {step}"
2078 );
1834 }2079 }
1835 }2080 }
1836 }2081 }
...@@ -1851,21 +2096,40 @@ fn an_imported_page_reads_back_as_its_copy() {...@@ -1851,21 +2096,40 @@ fn an_imported_page_reads_back_as_its_copy() {
1851 let arena = Arena::default();2096 let arena = Arena::default();
1852 let mut section = Section::open(&arena, target.clone()).unwrap();2097 let mut section = Section::open(&arena, target.clone()).unwrap();
1853 let import = |section: &mut Section<'_>, op: SectionOp| {2098 let import = |section: &mut Section<'_>, op: SectionOp| {
1854 section.apply("Author", &Edit { at: AT, ops: vec![Op::Section(op)] })2099 section.apply(
2100 "Author",
2101 &Edit {
2102 at: AT,
2103 ops: vec![Op::Section(op)],
2104 },
2105 )
1855 };2106 };
1856 import(&mut section, SectionOp::Create(creation.clone())).unwrap();2107 import(&mut section, SectionOp::Create(creation.clone())).unwrap();
1857 let created = section.page(creation.space()).unwrap();2108 let created = section.page(creation.space()).unwrap();
1858 let mut expected = created.clone();2109 let mut expected = created.clone();
1859 let arena = Arena::default();2110 let arena = Arena::default();
1860 let mut section = Section::open(&arena, target.clone()).unwrap();2111 let mut section = Section::open(&arena, target.clone()).unwrap();
1861 if import(&mut section, SectionOp::Import { creation: creation.clone(), page: copy.clone() }).is_err() {2112 if import(
2113 &mut section,
2114 SectionOp::Import {
2115 creation: creation.clone(),
2116 page: copy.clone(),
2117 },
2118 )
2119 .is_err()
2120 {
1862 continue;2121 continue;
1863 }2122 }
1864 section.seal().unwrap();2123 section.seal().unwrap();
1865 let image = section.image();2124 let image = section.image();
1866 let store = Store::parse(&image).unwrap();2125 let store = Store::parse(&image).unwrap();
1867 RevisionIndex::parse(&store).unwrap().validate_current().unwrap();2126 RevisionIndex::parse(&store)
1868 expected.objects.retain(|object| matches!(object, PageObject::Title(_)));2127 .unwrap()
2128 .validate_current()
2129 .unwrap();
2130 expected
2131 .objects
2132 .retain(|object| matches!(object, PageObject::Title(_)));
1869 expected.objects.extend(2133 expected.objects.extend(
1870 copy.objects2134 copy.objects
1871 .iter()2135 .iter()
...@@ -1876,16 +2140,24 @@ fn an_imported_page_reads_back_as_its_copy() {...@@ -1876,16 +2140,24 @@ fn an_imported_page_reads_back_as_its_copy() {
1876 let stored = read(&image, creation.space());2140 let stored = read(&image, creation.space());
1877 // New page children stack as the writers add them.2141 // New page children stack as the writers add them.
1878 let order: Vec<ExGuid> = stored.objects.iter().map(PageObject::id).collect();2142 let order: Vec<ExGuid> = stored.objects.iter().map(PageObject::id).collect();
1879 expected.objects.sort_by_key(|object| order.iter().position(|id| *id == object.id()));2143 expected
2144 .objects
2145 .sort_by_key(|object| order.iter().position(|id| *id == object.id()));
1880 let mut expected = as_written(&expected, &created, &stored);2146 let mut expected = as_written(&expected, &created, &stored);
1881 // An outline's height follows its content; no op carries it.2147 // An outline's height follows its content; no op carries it.
1882 for (object, stored) in expected.objects.iter_mut().zip(&stored.objects) {2148 for (object, stored) in expected.objects.iter_mut().zip(&stored.objects) {
1883 if let (PageObject::Outline(outline), PageObject::Outline(stored)) = (object, stored) {2149 if let (PageObject::Outline(outline), PageObject::Outline(stored)) =
2150 (object, stored)
2151 {
1884 outline.layout.max_height = stored.layout.max_height;2152 outline.layout.max_height = stored.layout.max_height;
1885 }2153 }
1886 }2154 }
1887 let (ours, theirs) = (normalize(&stored), normalize(&expected));2155 let (ours, theirs) = (normalize(&stored), normalize(&expected));
1888 assert!(ours == theirs, "{name}: {}", first_difference(&theirs, &ours));2156 assert!(
2157 ours == theirs,
2158 "{name}: {}",
2159 first_difference(&theirs, &ours)
2160 );
1889 imported += 1;2161 imported += 1;
1890 }2162 }
1891 }2163 }
...@@ -1902,7 +2174,15 @@ fn lower_range(before: &Page, after: &Page) -> Option<Result<Vec<PageOp>, crate:...@@ -1902,7 +2174,15 @@ fn lower_range(before: &Page, after: &Page) -> Option<Result<Vec<PageOp>, crate:
1902 match (a, b) {2174 match (a, b) {
1903 (PageObject::Outline(a), PageObject::Outline(b)) if a.id == b.id => {2175 (PageObject::Outline(a), PageObject::Outline(b)) if a.id == b.id => {
1904 if a.paragraphs != b.paragraphs {2176 if a.paragraphs != b.paragraphs {
1905 if changed.is_some() || (Outline { paragraphs: Vec::new(), ..a.clone() }) != (Outline { paragraphs: Vec::new(), ..b.clone() }) {2177 if changed.is_some()
2178 || (Outline {
2179 paragraphs: Vec::new(),
2180 ..a.clone()
2181 }) != (Outline {
2182 paragraphs: Vec::new(),
2183 ..b.clone()
2184 })
2185 {
1906 return None;2186 return None;
1907 }2187 }
1908 changed = Some((a, b));2188 changed = Some((a, b));
...@@ -1924,7 +2204,11 @@ fn lower_range(before: &Page, after: &Page) -> Option<Result<Vec<PageOp>, crate:...@@ -1924,7 +2204,11 @@ fn lower_range(before: &Page, after: &Page) -> Option<Result<Vec<PageOp>, crate:
1924 let (mut end_a, mut end_b) = (a.len() - suffix, b.len() - suffix);2204 let (mut end_a, mut end_b) = (a.len() - suffix, b.len() - suffix);
1925 // A range holds the children of what it changes.2205 // A range holds the children of what it changes.
1926 while end_a < a.len() {2206 while end_a < a.len() {
1927 let ids: Vec<ExGuid> = a[prefix..end_a].iter().chain(&b[prefix..end_b]).map(|p| p.id).collect();2207 let ids: Vec<ExGuid> = a[prefix..end_a]
2208 .iter()
2209 .chain(&b[prefix..end_b])
2210 .map(|p| p.id)
2211 .collect();
1928 if a[end_a].parent.is_some_and(|parent| ids.contains(&parent)) {2212 if a[end_a].parent.is_some_and(|parent| ids.contains(&parent)) {
1929 end_a += 1;2213 end_a += 1;
1930 end_b += 1;2214 end_b += 1;
...@@ -1932,23 +2216,49 @@ fn lower_range(before: &Page, after: &Page) -> Option<Result<Vec<PageOp>, crate:...@@ -1932,23 +2216,49 @@ fn lower_range(before: &Page, after: &Page) -> Option<Result<Vec<PageOp>, crate:
1932 break;2216 break;
1933 }2217 }
1934 }2218 }
1935 let outline = match before.objects.iter().find(|o| matches!(o, PageObject::Outline(o) if o.paragraphs == *a)) {2219 let outline = match before
2220 .objects
2221 .iter()
2222 .find(|o| matches!(o, PageObject::Outline(o) if o.paragraphs == *a))
2223 {
1936 Some(PageObject::Outline(outline)) => outline.id,2224 Some(PageObject::Outline(outline)) => outline.id,
1937 _ => return None,2225 _ => return None,
1938 };2226 };
1939 let mut definitions = before.definitions.clone();2227 let mut definitions = before.definitions.clone();
1940 definitions.extend(after.definitions.clone());2228 definitions.extend(after.definitions.clone());
1941 Some(lower(outline, &a[prefix..end_a], &b[prefix..end_b], b.get(end_b), &definitions))2229 Some(lower(
2230 outline,
2231 &a[prefix..end_a],
2232 &b[prefix..end_b],
2233 b.get(end_b),
2234 &definitions,
2235 ))
1942}2236}
19432237
1944/// Lowering the changed range of an outline stores what lowering the whole page does.2238/// Lowering the changed range of an outline stores what lowering the whole page does.
1945#[test]2239#[test]
1946fn a_range_lowers_as_its_page_does() {2240fn a_range_lowers_as_its_page_does() {
1947 let families = [2241 let families = [
1948 Family::Text, Family::Format, Family::Insert, Family::Paste, Family::Delete, Family::Move,2242 Family::Text,
1949 Family::Collapse, Family::List, Family::Tag, Family::Split, Family::Join, Family::Indent,2243 Family::Format,
1950 Family::Outdent, Family::Alignment, Family::Style, Family::Link, Family::NewTable,2244 Family::Insert,
1951 Family::ParagraphPicture, Family::Attachment, Family::Equation,2245 Family::Paste,
2246 Family::Delete,
2247 Family::Move,
2248 Family::Collapse,
2249 Family::List,
2250 Family::Tag,
2251 Family::Split,
2252 Family::Join,
2253 Family::Indent,
2254 Family::Outdent,
2255 Family::Alignment,
2256 Family::Style,
2257 Family::Link,
2258 Family::NewTable,
2259 Family::ParagraphPicture,
2260 Family::Attachment,
2261 Family::Equation,
1952 ];2262 ];
1953 let (mut same, mut total, mut smaller) = (0, 0, 0);2263 let (mut same, mut total, mut smaller) = (0, 0, 0);
1954 for (name, source) in SOURCES {2264 for (name, source) in SOURCES {
...@@ -1974,18 +2284,32 @@ fn a_range_lowers_as_its_page_does() {...@@ -1974,18 +2284,32 @@ fn a_range_lowers_as_its_page_does() {
1974 let mut section = Section::open(&arena, source.to_vec()).unwrap();2284 let mut section = Section::open(&arena, source.to_vec()).unwrap();
1975 let edit = Edit {2285 let edit = Edit {
1976 at: AT,2286 at: AT,
1977 ops: ops.iter().map(|op| Op::Page { space, op: op.clone() }).collect(),2287 ops: ops
2288 .iter()
2289 .map(|op| Op::Page {
2290 space,
2291 op: op.clone(),
2292 })
2293 .collect(),
1978 };2294 };
1979 section.apply("Author", &edit).map(|()| normalize(&section.page(space).unwrap()))2295 section
2296 .apply("Author", &edit)
2297 .map(|()| normalize(&section.page(space).unwrap()))
1980 };2298 };
1981 let (ours, theirs) = (write(&ranged), write(&whole));2299 let (ours, theirs) = (write(&ranged), write(&whole));
1982 match (ours, theirs) {2300 match (ours, theirs) {
1983 (Ok(ours), Ok(theirs)) => {2301 (Ok(ours), Ok(theirs)) => {
1984 assert!(ours == theirs, "{name} page {p} {family:?} seed {seed}: {}", first_difference(&theirs, &ours));2302 assert!(
2303 ours == theirs,
2304 "{name} page {p} {family:?} seed {seed}: {}",
2305 first_difference(&theirs, &ours)
2306 );
1985 same += 1;2307 same += 1;
1986 }2308 }
1987 (Err(_), Err(_)) => same += 1,2309 (Err(_), Err(_)) => same += 1,
1988 (ours, theirs) => panic!("{name} page {p} {family:?} seed {seed}: {ours:?} vs {theirs:?}"),2310 (ours, theirs) => {
2311 panic!("{name} page {p} {family:?} seed {seed}: {ours:?} vs {theirs:?}")
2312 }
1989 }2313 }
1990 let length = |ops: &[PageOp]| format!("{ops:?}").len();2314 let length = |ops: &[PageOp]| format!("{ops:?}").len();
1991 smaller += usize::from(length(&ranged) <= length(&whole));2315 smaller += usize::from(length(&ranged) <= length(&whole));
...@@ -2012,7 +2336,9 @@ fn keystroke_ops() {...@@ -2012,7 +2336,9 @@ fn keystroke_ops() {
2012 let image = std::fs::read(&path).unwrap();2336 let image = std::fs::read(&path).unwrap();
2013 let space = pages(&image)2337 let space = pages(&image)
2014 .into_iter()2338 .into_iter()
2015 .max_by_key(|space| read(&image, *space).objects.len() + format!("{:?}", read(&image, *space)).len())2339 .max_by_key(|space| {
2340 read(&image, *space).objects.len() + format!("{:?}", read(&image, *space)).len()
2341 })
2016 .unwrap();2342 .unwrap();
2017 let page = read(&image, space);2343 let page = read(&image, space);
2018 let outline = page2344 let outline = page
...@@ -2027,7 +2353,11 @@ fn keystroke_ops() {...@@ -2027,7 +2353,11 @@ fn keystroke_ops() {
2027 let middle = outline.paragraphs.len() / 2;2353 let middle = outline.paragraphs.len() / 2;
2028 let paragraph = &outline.paragraphs[middle];2354 let paragraph = &outline.paragraphs[middle];
2029 let text = paragraph.text().unwrap().id;2355 let text = paragraph.text().unwrap().id;
2030 println!("{path}: {} bytes, {} paragraphs in the outline", image.len(), outline.paragraphs.len());2356 println!(
2357 "{path}: {} bytes, {} paragraphs in the outline",
2358 image.len(),
2359 outline.paragraphs.len()
2360 );
20312361
2032 // Emitting: a keystroke is one op; a generic range replacement lowers its paragraphs.2362 // Emitting: a keystroke is one op; a generic range replacement lowers its paragraphs.
2033 let mut edited = paragraph.clone();2363 let mut edited = paragraph.clone();
...@@ -2087,7 +2417,11 @@ fn keystroke_ops() {...@@ -2087,7 +2417,11 @@ fn keystroke_ops() {
2087 seals.push(start.elapsed());2417 seals.push(start.elapsed());
2088 sizes.push((2418 sizes.push((
2089 transaction.append.len(),2419 transaction.append.len(),
2090 transaction.patches.iter().map(|(_, bytes)| bytes.len()).sum::<usize>(),2420 transaction
2421 .patches
2422 .iter()
2423 .map(|(_, bytes)| bytes.len())
2424 .sum::<usize>(),
2091 ));2425 ));
2092 }2426 }
2093 sizes.sort();2427 sizes.sort();
...@@ -2122,20 +2456,42 @@ fn op_gate_candidate() {...@@ -2122,20 +2456,42 @@ fn op_gate_candidate() {
2122 at.get()2456 at.get()
2123 };2457 };
2124 let apply = |section: &mut Section<'_>, op: Op| {2458 let apply = |section: &mut Section<'_>, op: Op| {
2125 let result = section.apply("Gate author", &Edit { at: next(), ops: vec![op] });2459 let result = section.apply(
2460 "Gate author",
2461 &Edit {
2462 at: next(),
2463 ops: vec![op],
2464 },
2465 );
2126 section.seal().unwrap();2466 section.seal().unwrap();
2127 result2467 result
2128 };2468 };
2129 let mut imported = Vec::new();2469 let mut imported = Vec::new();
2130 for (name, bytes) in SOURCES.iter().filter(|(name, _)| *name != "enter-probe") {2470 for (name, bytes) in SOURCES.iter().filter(|(name, _)| *name != "enter-probe") {
2131 let Some(space) = pages(bytes).first().copied() else { continue };2471 let Some(space) = pages(bytes).first().copied() else {
2472 continue;
2473 };
2132 let page = read(bytes, space);2474 let page = read(bytes, space);
2133 if page.objects.iter().any(|o| matches!(o, PageObject::Unsupported(_))) {2475 if page
2476 .objects
2477 .iter()
2478 .any(|o| matches!(o, PageObject::Unsupported(_)))
2479 {
2134 continue;2480 continue;
2135 }2481 }
2136 let Ok(copy) = page.copy() else { continue };2482 let Ok(copy) = page.copy() else { continue };
2137 let creation = crate::PageCreation::new(None, Some(&format!("Imported {name}")), "Gate author").unwrap();2483 let creation =
2138 if apply(&mut section, Op::Section(SectionOp::Import { creation: creation.clone(), page: copy })).is_ok() {2484 crate::PageCreation::new(None, Some(&format!("Imported {name}")), "Gate author")
2485 .unwrap();
2486 if apply(
2487 &mut section,
2488 Op::Section(SectionOp::Import {
2489 creation: creation.clone(),
2490 page: copy,
2491 }),
2492 )
2493 .is_ok()
2494 {
2139 imported.push(creation.space());2495 imported.push(creation.space());
2140 }2496 }
2141 }2497 }
...@@ -2163,10 +2519,16 @@ fn op_gate_candidate() {...@@ -2163,10 +2519,16 @@ fn op_gate_candidate() {
2163 .collect()2519 .collect()
2164 };2520 };
2165 let edited = fields(&before);2521 let edited = fields(&before);
2166 if fields(&after).iter().any(|(id, text)| edited.iter().any(|(e, old)| e == id && old != text)) {2522 if fields(&after)
2523 .iter()
2524 .any(|(id, text)| edited.iter().any(|(e, old)| e == id && old != text))
2525 {
2167 continue;2526 continue;
2168 }2527 }
2169 if section.apply_page("Gate author", next(), *space, &after).is_ok() {2528 if section
2529 .apply_page("Gate author", next(), *space, &after)
2530 .is_ok()
2531 {
2170 section.seal().unwrap();2532 section.seal().unwrap();
2171 edits += 1;2533 edits += 1;
2172 }2534 }
...@@ -2174,12 +2536,20 @@ fn op_gate_candidate() {...@@ -2174,12 +2536,20 @@ fn op_gate_candidate() {
2174 }2536 }
2175 // A created page: margins, an outline, and its first move adding the margin origin.2537 // A created page: margins, an outline, and its first move adding the margin origin.
2176 let creation = crate::PageCreation::new(None, Some("Created by ops"), "Gate author").unwrap();2538 let creation = crate::PageCreation::new(None, Some("Created by ops"), "Gate author").unwrap();
2177 apply(&mut section, Op::Section(SectionOp::Create(creation.clone()))).unwrap();2539 apply(
2540 &mut section,
2541 Op::Section(SectionOp::Create(creation.clone())),
2542 )
2543 .unwrap();
2178 let outline = Outline {2544 let outline = Outline {
2179 id: id(),2545 id: id(),
2180 title: false,2546 title: false,
2181 min_width: None,2547 min_width: None,
2182 layout: Layout { x: Some(36.0), y: Some(86.4), ..Default::default() },2548 layout: Layout {
2549 x: Some(36.0),
2550 y: Some(86.4),
2551 ..Default::default()
2552 },
2183 indents: Vec::new(),2553 indents: Vec::new(),
2184 paragraphs: vec![text_paragraph("Typed through ops", Format::default())],2554 paragraphs: vec![text_paragraph("Typed through ops", Format::default())],
2185 unsupported: Vec::new(),2555 unsupported: Vec::new(),
...@@ -2188,26 +2558,87 @@ fn op_gate_candidate() {...@@ -2188,26 +2558,87 @@ fn op_gate_candidate() {
2188 let text = outline.paragraphs[0].text().unwrap().id;2558 let text = outline.paragraphs[0].text().unwrap().id;
2189 let space = creation.space();2559 let space = creation.space();
2190 let page = |op| Op::Page { space, op };2560 let page = |op| Op::Page { space, op };
2191 apply(&mut section, page(PageOp::Add { object: PageObject::Outline(outline), before: None })).unwrap();2561 apply(
2192 apply(&mut section, page(PageOp::Outline { object: outline_id, edit: crate::OutlineEdit::Position { x: 72.0, y: 108.0 } })).unwrap();2562 &mut section,
2193 apply(&mut section, page(PageOp::Text { text, range: 5..5, with: " and sealed".into() })).unwrap();2563 page(PageOp::Add {
2194 apply(&mut section, page(PageOp::Link { text, range: 0..5, target: Some("https://example.invalid/ops".into()) })).unwrap();2564 object: PageObject::Outline(outline),
2195 let spaces: Vec<ExGuid> = section.pages().unwrap().into_iter().map(|(s, _, _)| s).collect();2565 before: None,
2196 apply(&mut section, Op::Section(SectionOp::Pages(vec![crate::PageEdit::move_to(space, Some(spaces[1]), 1).unwrap()]))).unwrap();2566 }),
2197 apply(&mut section, Op::Section(SectionOp::Pages(vec![crate::PageEdit::set_level(spaces[2], 2).unwrap()]))).unwrap();2567 )
2198 apply(&mut section, Op::Section(SectionOp::Delete(vec![*spaces.last().unwrap()]))).unwrap();2568 .unwrap();
2569 apply(
2570 &mut section,
2571 page(PageOp::Outline {
2572 object: outline_id,
2573 edit: crate::OutlineEdit::Position { x: 72.0, y: 108.0 },
2574 }),
2575 )
2576 .unwrap();
2577 apply(
2578 &mut section,
2579 page(PageOp::Text {
2580 text,
2581 range: 5..5,
2582 with: " and sealed".into(),
2583 }),
2584 )
2585 .unwrap();
2586 apply(
2587 &mut section,
2588 page(PageOp::Link {
2589 text,
2590 range: 0..5,
2591 target: Some("https://example.invalid/ops".into()),
2592 }),
2593 )
2594 .unwrap();
2595 let spaces: Vec<ExGuid> = section
2596 .pages()
2597 .unwrap()
2598 .into_iter()
2599 .map(|(s, _, _)| s)
2600 .collect();
2601 apply(
2602 &mut section,
2603 Op::Section(SectionOp::Pages(vec![
2604 crate::PageEdit::move_to(space, Some(spaces[1]), 1).unwrap(),
2605 ])),
2606 )
2607 .unwrap();
2608 apply(
2609 &mut section,
2610 Op::Section(SectionOp::Pages(vec![
2611 crate::PageEdit::set_level(spaces[2], 2).unwrap(),
2612 ])),
2613 )
2614 .unwrap();
2615 apply(
2616 &mut section,
2617 Op::Section(SectionOp::Delete(vec![*spaces.last().unwrap()])),
2618 )
2619 .unwrap();
2199 let image = section.image();2620 let image = section.image();
2200 let store = Store::parse(&image).unwrap();2621 let store = Store::parse(&image).unwrap();
2201 RevisionIndex::parse(&store).unwrap().validate_current().unwrap();2622 RevisionIndex::parse(&store)
2623 .unwrap()
2624 .validate_current()
2625 .unwrap();
2202 let count = section.pages().unwrap().len();2626 let count = section.pages().unwrap().len();
2203 std::fs::write(directory.join("ops.one"), &image).unwrap();2627 std::fs::write(directory.join("ops.one"), &image).unwrap();
2204 std::fs::write(2628 std::fs::write(
2205 directory.join("Open Notebook.onetoc2"),2629 directory.join("Open Notebook.onetoc2"),
2206 crate::create_table_of_contents("Open Notebook.onetoc2", &[("ops.one", store.header.file_id)]).unwrap(),2630 crate::create_table_of_contents(
2631 "Open Notebook.onetoc2",
2632 &[("ops.one", store.header.file_id)],
2633 )
2634 .unwrap(),
2207 )2635 )
2208 .unwrap();2636 .unwrap();
2209 std::fs::write(directory.join("expected-count.txt"), count.to_string()).unwrap();2637 std::fs::write(directory.join("expected-count.txt"), count.to_string()).unwrap();
2210 println!("{} imported pages, {edits} lowered edits, {count} pages", imported.len());2638 println!(
2639 "{} imported pages, {edits} lowered edits, {count} pages",
2640 imported.len()
2641 );
2211}2642}
22122643
2213/// OneNote 2010's own date change (`corpus/page-date/native`, COM `UpdatePageContent`2644/// OneNote 2010's own date change (`corpus/page-date/native`, COM `UpdatePageContent`
...@@ -2229,9 +2660,9 @@ fn changed_jcids(image: &[u8], space: ExGuid) -> Vec<u32> {...@@ -2229,9 +2660,9 @@ fn changed_jcids(image: &[u8], space: ExGuid) -> Vec<u32> {
2229 .objects2660 .objects
2230 .iter()2661 .iter()
2231 .filter(|(id, object)| {2662 .filter(|(id, object)| {
2232 old.objects.get(id).is_none_or(|old| {2663 old.objects
2233 format!("{:?}", old.data) != format!("{:?}", object.data)2664 .get(id)
2234 })2665 .is_none_or(|old| format!("{:?}", old.data) != format!("{:?}", object.data))
2235 })2666 })
2236 .map(|(_, object)| object.jcid)2667 .map(|(_, object)| object.jcid)
2237 .collect();2668 .collect();
...@@ -2256,10 +2687,19 @@ const DATED_AT: u64 = 133_961_211_000_000_000;...@@ -2256,10 +2687,19 @@ const DATED_AT: u64 = 133_961_211_000_000_000;
2256fn a_page_date_stores_what_onenote_stores() {2687fn a_page_date_stores_what_onenote_stores() {
2257 let spaces = pages(DATED);2688 let spaces = pages(DATED);
2258 let title = |space: &ExGuid| read(DATED, *space).title;2689 let title = |space: &ExGuid| read(DATED, *space).title;
2259 let kept = *spaces.iter().find(|space| title(space) == "Kept date").unwrap();2690 let kept = *spaces
2260 let changed = *spaces.iter().find(|space| title(space) == "Changed date").unwrap();2691 .iter()
2692 .find(|space| title(space) == "Kept date")
2693 .unwrap();
2694 let changed = *spaces
2695 .iter()
2696 .find(|space| title(space) == "Changed date")
2697 .unwrap();
2261 let native = changed_jcids(DATED, changed);2698 let native = changed_jcids(DATED, changed);
2262 assert_eq!(native, [0x20030, 0x6000c, 0x6000d, 0x6000d, 0x6000e, 0x6000e]);2699 assert_eq!(
2700 native,
2701 [0x20030, 0x6000c, 0x6000d, 0x6000d, 0x6000e, 0x6000e]
2702 );
22632703
2264 let arena = Arena::default();2704 let arena = Arena::default();
2265 let mut section = Section::open(&arena, DATED.to_vec()).unwrap();2705 let mut section = Section::open(&arena, DATED.to_vec()).unwrap();
...@@ -2278,7 +2718,13 @@ fn a_page_date_stores_what_onenote_stores() {...@@ -2278,7 +2718,13 @@ fn a_page_date_stores_what_onenote_stores() {
2278 let mut predicted = before.clone();2718 let mut predicted = before.clone();
2279 model::apply(&mut predicted, &op).unwrap();2719 model::apply(&mut predicted, &op).unwrap();
2280 section2720 section
2281 .apply("Author", &Edit { at: AT, ops: vec![Op::Page { space: kept, op }] })2721 .apply(
2722 "Author",
2723 &Edit {
2724 at: AT,
2725 ops: vec![Op::Page { space: kept, op }],
2726 },
2727 )
2282 .unwrap();2728 .unwrap();
2283 section.seal().unwrap();2729 section.seal().unwrap();
2284 let image = section.image();2730 let image = section.image();
...@@ -2288,20 +2734,36 @@ fn a_page_date_stores_what_onenote_stores() {...@@ -2288,20 +2734,36 @@ fn a_page_date_stores_what_onenote_stores() {
2288 assert_eq!(stored, predicted);2734 assert_eq!(stored, predicted);
2289 assert_eq!(changed_jcids(&image, kept), native);2735 assert_eq!(changed_jcids(&image, kept), native);
2290 let store = Store::parse(&image).unwrap();2736 let store = Store::parse(&image).unwrap();
2291 RevisionIndex::parse(&store).unwrap().validate_current().unwrap();2737 RevisionIndex::parse(&store)
2738 .unwrap()
2739 .validate_current()
2740 .unwrap();
22922741
2293 // The whole-page lowering reaches the same page through the same op.2742 // The whole-page lowering reaches the same page through the same op.
2294 let mut after = before.clone();2743 let mut after = before.clone();
2295 after.created = Some(DATED_AT);2744 after.created = Some(DATED_AT);
2296 model::apply(&mut after, &PageOp::Date { created: DATED_AT, fields }).unwrap();2745 model::apply(
2297 assert!(matches!(&lower_page(&before, &after).unwrap()[..], [PageOp::Date { .. }]));2746 &mut after,
2747 &PageOp::Date {
2748 created: DATED_AT,
2749 fields,
2750 },
2751 )
2752 .unwrap();
2753 assert!(matches!(
2754 &lower_page(&before, &after).unwrap()[..],
2755 [PageOp::Date { .. }]
2756 ));
22982757
2299 if let Some(directory) = std::env::var_os("ONESTORE_PAGE_DATE_EXPORT") {2758 if let Some(directory) = std::env::var_os("ONESTORE_PAGE_DATE_EXPORT") {
2300 let directory = std::path::Path::new(&directory);2759 let directory = std::path::Path::new(&directory);
2301 std::fs::create_dir_all(directory).unwrap();2760 std::fs::create_dir_all(directory).unwrap();
2302 std::fs::write(directory.join("pictures.one"), &image).unwrap();2761 std::fs::write(directory.join("pictures.one"), &image).unwrap();
2303 std::fs::copy(2762 std::fs::copy(
2304 concat!(env!("CARGO_MANIFEST_DIR"), "/../../corpus/page-date/native/notebook/Open Notebook.onetoc2"),2763 concat!(
2764 env!("CARGO_MANIFEST_DIR"),
2765 "/../../corpus/page-date/native/notebook/Open Notebook.onetoc2"
2766 ),
2305 directory.join("Open Notebook.onetoc2"),2767 directory.join("Open Notebook.onetoc2"),
2306 )2768 )
2307 .unwrap();2769 .unwrap();
crates/onestore/src/page/linear.rs created+741
...@@ -0,0 +1,741 @@
1//! Equations in the linear format of OneNote's equation editor (UnicodeMath): what Linear
2//! shows and what typing builds up. `Math::from_linear` reads the typed or shown text into the
3//! tree `Math::paragraph` stores; `Math::linear` writes a tree the way Linear shows it. Both
4//! follow the equation editor as observed (`corpus/math-edit/native-linear`): a space that
5//! ends a built object is consumed, other spaces stay, and an n-ary operator takes the rest
6//! of its row as its body.
7
8use super::Math;
9use super::math::{italic, plain};
10use super::text::Paragraph;
11use crate::document::Format;
12
13/// N-ary operators, which take limits and a body when a script follows them.
14const NARY: &[char] = &[
15 '∑', '∏', '∐', '∫', '∬', '∭', '∮', '∯', '∰', '⋀', '⋁', '⋂', '⋃', '⨀', '⨁', '⨂', '⨄', '⨅', '⨆',
16];
17
18/// Function names the equation editor sets upright; those that take a limit below take it
19/// from a subscript.
20const FUNCTIONS: &[&str] = &[
21 "lim", "max", "min", "sup", "inf", "det", "gcd", "sin", "cos", "tan", "sec", "csc", "cot",
22 "sinh", "cosh", "tanh", "log", "ln", "exp", "arg",
23];
24const LIMITS: &[&str] = &["lim", "max", "min", "sup", "inf", "det", "gcd"];
25
26/// The equation editor's control words: `\name` and the space after it become the symbol.
27const CONTROL_WORDS: &[(&str, char)] = &[
28 ("alpha", 'α'),
29 ("beta", 'β'),
30 ("gamma", 'γ'),
31 ("delta", 'δ'),
32 ("epsilon", 'ϵ'),
33 ("varepsilon", 'ε'),
34 ("zeta", 'ζ'),
35 ("eta", 'η'),
36 ("theta", 'θ'),
37 ("vartheta", 'ϑ'),
38 ("iota", 'ι'),
39 ("kappa", 'κ'),
40 ("lambda", 'λ'),
41 ("mu", 'μ'),
42 ("nu", 'ν'),
43 ("xi", 'ξ'),
44 ("pi", 'π'),
45 ("rho", 'ρ'),
46 ("sigma", 'σ'),
47 ("tau", 'τ'),
48 ("upsilon", 'υ'),
49 ("phi", 'ϕ'),
50 ("varphi", 'φ'),
51 ("chi", 'χ'),
52 ("psi", 'ψ'),
53 ("omega", 'ω'),
54 ("Gamma", 'Γ'),
55 ("Delta", 'Δ'),
56 ("Theta", 'Θ'),
57 ("Lambda", 'Λ'),
58 ("Xi", 'Ξ'),
59 ("Pi", 'Π'),
60 ("Sigma", 'Σ'),
61 ("Upsilon", 'Υ'),
62 ("Phi", 'Φ'),
63 ("Psi", 'Ψ'),
64 ("Omega", 'Ω'),
65 ("to", '→'),
66 ("rightarrow", '→'),
67 ("leftarrow", '←'),
68 ("times", '×'),
69 ("div", '÷'),
70 ("pm", '±'),
71 ("mp", '∓'),
72 ("infty", '∞'),
73 ("le", '≤'),
74 ("ge", '≥'),
75 ("ne", '≠'),
76 ("approx", '≈'),
77 ("equiv", '≡'),
78 ("sim", '∼'),
79 ("propto", '∝'),
80 ("cdot", '⋅'),
81 ("partial", '∂'),
82 ("nabla", '∇'),
83 ("in", '∈'),
84 ("notin", '∉'),
85 ("subset", '⊂'),
86 ("supset", '⊃'),
87 ("cup", '∪'),
88 ("cap", '∩'),
89 ("forall", '∀'),
90 ("exists", '∃'),
91 ("ll", '≪'),
92 ("gg", '≫'),
93 ("ldots", '…'),
94 ("cdots", '⋯'),
95 ("sum", '∑'),
96 ("prod", '∏'),
97 ("coprod", '∐'),
98 ("int", '∫'),
99 ("iint", '∬'),
100 ("iiint", '∭'),
101 ("oint", '∮'),
102 ("bigcap", '⋂'),
103 ("bigcup", '⋃'),
104 ("sqrt", '√'),
105 ("cbrt", '∛'),
106 ("qdrt", '∜'),
107 ("matrix", '■'),
108 ("eqarray", '█'),
109 ("box", '□'),
110 ("rect", '▭'),
111 ("overline", '¯'),
112 ("above", '┴'),
113 ("below", '┬'),
114 ("naryand", '▒'),
115 ("hat", '\u{302}'),
116 ("tilde", '\u{303}'),
117 ("bar", '\u{305}'),
118 ("dot", '\u{307}'),
119 ("ddot", '\u{308}'),
120 ("check", '\u{30c}'),
121 ("vec", '\u{20d7}'),
122];
123
124fn object(kind: u32, symbols: Vec<char>, columns: Option<u8>, arguments: Vec<Vec<Math>>) -> Math {
125 Math::Object {
126 kind,
127 symbols,
128 columns,
129 arguments,
130 }
131}
132
133fn accent(c: char) -> bool {
134 matches!(u32::from(c), 0x300..=0x36f | 0x20d0..=0x20ff)
135}
136
137fn closing(open: char) -> Option<char> {
138 Some(match open {
139 '(' => ')',
140 '[' => ']',
141 '{' => '}',
142 '⟨' => '⟩',
143 '〖' => '〗',
144 _ => return None,
145 })
146}
147
148struct Parser {
149 chars: Vec<char>,
150 at: usize,
151 /// Text still being typed: a construct whose last operand is missing stays as typed.
152 typing: bool,
153}
154
155impl Parser {
156 fn peek(&self) -> Option<char> {
157 self.chars.get(self.at).copied()
158 }
159
160 /// A space ending a built object triggered the build and is not kept.
161 fn built(&mut self, out: &mut Vec<Math>, node: Math) {
162 out.push(node);
163 if self.peek() == Some(' ') {
164 self.at += 1;
165 }
166 }
167
168 fn sequence(&mut self, stops: &[char]) -> Vec<Math> {
169 let mut out = Vec::new();
170 while let Some(c) = self.peek() {
171 if stops.contains(&c) {
172 break;
173 }
174 self.element(&mut out, stops);
175 }
176 out
177 }
178
179 fn element(&mut self, out: &mut Vec<Math>, stops: &[char]) {
180 let c = self.peek().unwrap();
181 let next = self.chars.get(self.at + 1).copied();
182 match c {
183 '/' => {
184 self.at += 1;
185 let denominator = stripped(self.run(stops));
186 if self.typing && denominator.is_empty() {
187 out.push(Math::Operator(c));
188 return;
189 }
190 let start = out
191 .iter()
192 .rposition(|node| matches!(node, Math::Operator(_)))
193 .map_or(0, |at| at + 1);
194 let numerator = stripped(out.split_off(start));
195 self.built(
196 out,
197 object(16, vec!['/'], None, vec![numerator, denominator]),
198 );
199 }
200 '^' | '_' => self.script(out),
201 '┴' | '┬' => {
202 self.at += 1;
203 let argument = self.operand();
204 if self.typing && argument.is_empty() {
205 out.push(Math::Operator(c));
206 return;
207 }
208 let base = out.pop().into_iter().collect();
209 let kind = if c == '┴' { 33 } else { 19 };
210 self.built(out, object(kind, vec![c], None, vec![base, argument]));
211 }
212 '\u{a0}' if next.is_some_and(accent) => self.at += 1,
213 c if accent(c) => {
214 self.at += 1;
215 if matches!(out.last(), Some(Math::Operator(' ' | '\u{a0}'))) {
216 out.pop();
217 }
218 let base = stripped(out.pop().into_iter().collect());
219 self.built(out, object(10, vec![c], None, vec![base]));
220 }
221 c if NARY.contains(&c) && matches!(next, Some('_' | '^' | '▒')) => {
222 self.at += 1;
223 let limits = self.at;
224 let (mut lower, mut upper) = (None, None);
225 loop {
226 match self.peek() {
227 Some('_') if lower.is_none() => {
228 self.at += 1;
229 lower = Some(self.operand());
230 }
231 Some('^') if upper.is_none() => {
232 self.at += 1;
233 upper = Some(self.operand());
234 }
235 _ => break,
236 }
237 }
238 if self.typing && [&lower, &upper].contains(&&Some(Vec::new())) {
239 self.at = limits;
240 out.push(Math::Operator(c));
241 return;
242 }
243 let body = match self.peek() {
244 Some('▒') => {
245 self.at += 1;
246 self.operand()
247 }
248 Some(' ') => {
249 self.at += 1;
250 self.sequence(stops)
251 }
252 _ => self.sequence(stops),
253 };
254 out.push(object(
255 21,
256 vec![c],
257 None,
258 vec![lower.unwrap_or_default(), upper.unwrap_or_default(), body],
259 ));
260 }
261 '〖' => {
262 self.at += 1;
263 let inner = self.sequence(&['〗']);
264 if self.peek() == Some('〗') {
265 self.at += 1;
266 }
267 out.extend(inner);
268 }
269 _ => {
270 let node = self.factor();
271 if matches!(node, Math::Object { .. }) {
272 self.built(out, node);
273 } else {
274 out.push(node);
275 }
276 }
277 }
278 }
279
280 fn script(&mut self, out: &mut Vec<Math>) {
281 let first = self.peek().unwrap();
282 self.at += 1;
283 let script = self.operand();
284 if self.typing && script.is_empty() {
285 out.push(Math::Operator(first));
286 return;
287 }
288 let base: Vec<Math> = out.pop().into_iter().collect();
289 let other = if first == '_' { '^' } else { '_' };
290 let node = if self.peek() == Some(other) {
291 self.at += 1;
292 let second = self.operand();
293 let (lower, upper) = if first == '_' {
294 (script, second)
295 } else {
296 (second, script)
297 };
298 object(30, vec!['_'], None, vec![base, lower, upper])
299 } else if first == '_'
300 && matches!(&base[..], [Math::Function(name)] if LIMITS.contains(&name.as_str()))
301 {
302 object(19, vec!['_'], None, vec![base, script])
303 } else {
304 let kind = if first == '^' { 31 } else { 29 };
305 object(kind, vec![first], None, vec![base, script])
306 };
307 self.built(out, node);
308 }
309
310 /// One factor as an object's argument: parentheses and the invisible grouping brackets
311 /// only group it.
312 fn operand(&mut self) -> Vec<Math> {
313 match self.peek() {
314 Some(open @ ('(' | '〖')) => {
315 let start = self.at;
316 self.at += 1;
317 let close = closing(open).unwrap();
318 let inner = self.sequence(&[close]);
319 if self.peek() == Some(close) {
320 self.at += 1;
321 } else if self.typing {
322 // A group still open is still being typed.
323 self.at = start;
324 return Vec::new();
325 }
326 inner
327 }
328 // An argument not typed yet stays empty for the caret.
329 Some(' ') | None => Vec::new(),
330 Some(_) => vec![self.factor()],
331 }
332 }
333
334 /// Juxtaposed factors with their scripts, as a fraction's denominator.
335 fn run(&mut self, stops: &[char]) -> Vec<Math> {
336 let mut out = Vec::new();
337 while let Some(c) = self.peek() {
338 if stops.contains(&c) {
339 break;
340 }
341 let starts = c.is_alphanumeric()
342 || closing(c).is_some()
343 || matches!(c, '√' | '∛' | '∜' | '■' | '█' | '□' | '▭' | '¯');
344 if matches!(c, '^' | '_') && !out.is_empty() {
345 self.script(&mut out);
346 } else if starts {
347 out.push(self.factor());
348 } else {
349 break;
350 }
351 }
352 out
353 }
354
355 fn factor(&mut self) -> Math {
356 let c = self.peek().unwrap();
357 self.at += 1;
358 if c.is_ascii_digit() {
359 let mut number = c.to_string();
360 while let Some(digit) = self.peek().filter(char::is_ascii_digit) {
361 number.push(digit);
362 self.at += 1;
363 }
364 return Math::Number(number);
365 }
366 if c.is_ascii_alphabetic() {
367 let word: String = self.chars[self.at - 1..]
368 .iter()
369 .take_while(|c| c.is_ascii_alphabetic())
370 .collect();
371 if FUNCTIONS.contains(&word.as_str()) {
372 self.at += word.len() - 1;
373 return Math::Function(word);
374 }
375 return Math::Identifier(c);
376 }
377 if c.is_alphabetic() {
378 return Math::Identifier(plain(c));
379 }
380 if let Some(close) = closing(c) {
381 let start = self.at;
382 let inner = self.sequence(&[close]);
383 if self.peek() == Some(close) {
384 self.at += 1;
385 return object(13, vec![c, close], None, vec![inner]);
386 }
387 self.at = start;
388 return Math::Operator(c);
389 }
390 match c {
391 '√' | '∛' | '∜' => {
392 let degree = match c {
393 '∛' => vec![Math::Number("3".into())],
394 '∜' => vec![Math::Number("4".into())],
395 _ => Vec::new(),
396 };
397 let (degree, radicand) = if c == '√' && self.peek() == Some('(') {
398 self.at += 1;
399 let first = self.sequence(&[')', '&']);
400 let parts = if self.peek() == Some('&') {
401 self.at += 1;
402 (first, self.sequence(&[')']))
403 } else {
404 (degree, first)
405 };
406 if self.peek() == Some(')') {
407 self.at += 1;
408 }
409 parts
410 } else {
411 (degree, self.operand())
412 };
413 if self.typing && radicand.is_empty() {
414 return Math::Operator(c);
415 }
416 object(25, vec!['√'], None, vec![degree, radicand])
417 }
418 '■' | '█' if self.peek() == Some('(') => {
419 self.at += 1;
420 let matrix = c == '■';
421 let mut rows = vec![Vec::new()];
422 loop {
423 let stops: &[char] = if matrix {
424 &[')', '&', '@']
425 } else {
426 &[')', '@']
427 };
428 let cell = self.sequence(stops);
429 rows.last_mut().unwrap().push(cell);
430 match self.peek() {
431 Some('&') => self.at += 1,
432 Some('@') => {
433 self.at += 1;
434 rows.push(Vec::new());
435 }
436 Some(_) => {
437 self.at += 1;
438 break;
439 }
440 None => break,
441 }
442 }
443 if matrix {
444 let columns = rows.iter().map(Vec::len).max().unwrap_or(1);
445 let arguments = rows
446 .into_iter()
447 .flat_map(|mut row| {
448 row.resize(columns, Vec::new());
449 row
450 })
451 .collect();
452 object(20, vec!['■'], u8::try_from(columns).ok(), arguments)
453 } else {
454 let arguments = rows.into_iter().flatten().collect();
455 object(15, vec!['█'], Some(1), arguments)
456 }
457 }
458 '□' | '▭' | '¯' => {
459 let kind = match c {
460 '□' => 11,
461 '▭' => 12,
462 _ => 23,
463 };
464 let inner = self.operand();
465 if self.typing && inner.is_empty() {
466 return Math::Operator(c);
467 }
468 object(kind, vec![c], None, vec![inner])
469 }
470 c => Math::Operator(c),
471 }
472 }
473}
474
475/// A lone parenthesized group as an argument only groups it.
476fn stripped(nodes: Vec<Math>) -> Vec<Math> {
477 match <[Math; 1]>::try_from(nodes) {
478 Ok(
479 [
480 Math::Object {
481 kind: 13,
482 symbols,
483 mut arguments,
484 ..
485 },
486 ],
487 ) if symbols == ['(', ')'] && arguments.len() == 1 => arguments.pop().unwrap(),
488 Ok([node]) => vec![node],
489 Err(nodes) => nodes,
490 }
491}
492
493/// Replaces control words with their symbols; an unknown one stays as typed.
494fn expand(text: &str) -> Vec<char> {
495 let chars: Vec<char> = text.chars().collect();
496 let mut out = Vec::with_capacity(chars.len());
497 let mut at = 0;
498 while at < chars.len() {
499 if chars[at] == '\\' {
500 let name: String = chars[at + 1..]
501 .iter()
502 .take_while(|c| c.is_ascii_alphabetic())
503 .collect();
504 if let Some((_, symbol)) = CONTROL_WORDS.iter().find(|(word, _)| *word == name) {
505 out.push(*symbol);
506 at += 1 + name.len();
507 if chars.get(at) == Some(&' ') {
508 at += 1;
509 }
510 continue;
511 }
512 }
513 out.push(chars[at]);
514 at += 1;
515 }
516 out
517}
518
519impl Math {
520 /// The equation typed or shown as `text` in the linear format, built up.
521 pub fn from_linear(text: &str) -> Vec<Math> {
522 Parser {
523 chars: expand(text),
524 at: 0,
525 typing: false,
526 }
527 .sequence(&[])
528 }
529
530 /// What the equation editor builds as `text` is typed: as [`Math::from_linear`], but a
531 /// construct still missing its last operand stays as typed, and an n-ary operator's
532 /// empty body waits for the caret.
533 pub fn typed(text: &str) -> Vec<Math> {
534 Parser {
535 chars: expand(text),
536 at: 0,
537 typing: true,
538 }
539 .sequence(&[])
540 }
541
542 /// The paragraph OneNote stores for an equation shown in the linear format: its linear
543 /// text in one run of math over `base`.
544 pub fn linear_paragraph(nodes: &[Math], base: &Format) -> Option<Paragraph> {
545 Some(Paragraph::new(Self::linear(nodes)?, Self::format(base)))
546 }
547
548 /// The linear format Linear shows for `nodes`, letters in mathematical italic; `None`
549 /// where an object has no linear form here.
550 pub fn linear(nodes: &[Math]) -> Option<String> {
551 let mut out = String::new();
552 write(nodes, &mut out)?;
553 Some(out)
554 }
555}
556
557fn write(nodes: &[Math], out: &mut String) -> Option<()> {
558 let mut after_object = false;
559 for node in nodes {
560 match node {
561 Math::Identifier(c) => out.push(italic(*c)),
562 Math::Function(name) => out.push_str(name),
563 Math::Number(n) => out.push_str(n),
564 // Reading consumes one space after an object.
565 Math::Operator(' ') if after_object => out.push_str(" "),
566 Math::Operator(c) => out.push(*c),
567 Math::Object {
568 kind,
569 symbols,
570 arguments,
571 columns,
572 } => write_object(*kind, symbols, *columns, arguments, out)?,
573 }
574 after_object = matches!(node, Math::Object { .. });
575 }
576 Some(())
577}
578
579/// An argument that reads back as one factor stands bare; others are parenthesized.
580fn argument(nodes: &[Math], out: &mut String) -> Option<()> {
581 let bare = match nodes {
582 [Math::Identifier(_) | Math::Number(_) | Math::Function(_)] => true,
583 [Math::Object { kind, symbols, .. }] => {
584 matches!(kind, 11 | 12 | 15 | 20 | 23 | 25) || *kind == 13 && symbols[..] != ['(', ')']
585 }
586 _ => false,
587 };
588 if bare {
589 return write(nodes, out);
590 }
591 out.push('(');
592 write(nodes, out)?;
593 out.push(')');
594 Some(())
595}
596
597/// A fraction's part stands bare when it reads back as one run of factors.
598fn part(nodes: &[Math], out: &mut String) -> Option<()> {
599 let bare = !nodes.is_empty()
600 && !matches!(nodes, [Math::Object { kind: 13, symbols, .. }] if symbols[..] == ['(', ')'])
601 && nodes.iter().all(|node| match node {
602 Math::Operator(_) => false,
603 Math::Object { kind, .. } => !matches!(kind, 10 | 16 | 19 | 21 | 33),
604 _ => true,
605 });
606 if bare {
607 return write(nodes, out);
608 }
609 out.push('(');
610 write(nodes, out)?;
611 out.push(')');
612 Some(())
613}
614
615fn write_object(
616 kind: u32,
617 symbols: &[char],
618 columns: Option<u8>,
619 arguments: &[Vec<Math>],
620 out: &mut String,
621) -> Option<()> {
622 match (kind, arguments) {
623 (31, [base, sup]) => {
624 argument(base, out)?;
625 out.push('^');
626 argument(sup, out)
627 }
628 (29, [base, sub]) => {
629 argument(base, out)?;
630 out.push('_');
631 argument(sub, out)
632 }
633 (30, [base, sub, sup]) => {
634 argument(base, out)?;
635 out.push('_');
636 argument(sub, out)?;
637 out.push('^');
638 argument(sup, out)
639 }
640 (16, [numerator, denominator]) => {
641 part(numerator, out)?;
642 out.push('/');
643 part(denominator, out)
644 }
645 (13, [inner]) => {
646 out.push(symbols.first().copied().unwrap_or('('));
647 write(inner, out)?;
648 out.push(symbols.get(1).copied().unwrap_or(')'));
649 Some(())
650 }
651 (21, [lower, upper, body]) => {
652 out.push(symbols.first().copied().unwrap_or('∑'));
653 if !lower.is_empty() {
654 out.push('_');
655 argument(lower, out)?;
656 }
657 if !upper.is_empty() {
658 out.push('^');
659 argument(upper, out)?;
660 }
661 out.push('▒');
662 match &body[..] {
663 [Math::Identifier(_) | Math::Number(_)] => write(body, out),
664 _ => {
665 out.push('〖');
666 write(body, out)?;
667 out.push('〗');
668 Some(())
669 }
670 }
671 }
672 (25, [degree, radicand]) => {
673 match &degree[..] {
674 [] => out.push('√'),
675 [Math::Number(n)] if n == "3" => out.push('∛'),
676 [Math::Number(n)] if n == "4" => out.push('∜'),
677 _ => {
678 out.push_str("√(");
679 write(degree, out)?;
680 out.push('&');
681 write(radicand, out)?;
682 out.push(')');
683 return Some(());
684 }
685 }
686 argument(radicand, out)
687 }
688 (19, [base, below]) => {
689 argument(base, out)?;
690 out.push(if symbols == ['_'] { '_' } else { '┬' });
691 argument(below, out)
692 }
693 (33, [base, above]) => {
694 argument(base, out)?;
695 out.push('┴');
696 argument(above, out)
697 }
698 (10, [base]) => {
699 argument(base, out)?;
700 out.push('\u{a0}');
701 out.push(*symbols.first()?);
702 Some(())
703 }
704 (11, [inner]) if symbols == ['□'] => {
705 out.push('□');
706 argument(inner, out)
707 }
708 (12, [inner]) if symbols == ['▭'] => {
709 out.push('▭');
710 argument(inner, out)
711 }
712 (23, [inner]) if symbols == ['¯'] => {
713 out.push('¯');
714 argument(inner, out)
715 }
716 (20, cells) => {
717 let width = usize::from(columns?.max(1));
718 out.push_str("■(");
719 for (index, cell) in cells.iter().enumerate() {
720 if index > 0 {
721 out.push(if index % width == 0 { '@' } else { '&' });
722 }
723 write(cell, out)?;
724 }
725 out.push(')');
726 Some(())
727 }
728 (15, rows) => {
729 out.push_str("█(");
730 for (index, row) in rows.iter().enumerate() {
731 if index > 0 {
732 out.push('@');
733 }
734 write(row, out)?;
735 }
736 out.push(')');
737 Some(())
738 }
739 _ => None,
740 }
741}
crates/onestore/src/page/link.rs+4-2
...@@ -71,7 +71,8 @@ pub struct InternalLink {...@@ -71,7 +71,8 @@ pub struct InternalLink {
71 pub object: Option<ExGuid>,71 pub object: Option<ExGuid>,
72}72}
7373
74/// The identities in an `onenote:#…` link, or `None` for any other URL.74/// The identities in an `onenote:` link, whatever path it names before the `#` (none as
75/// stored, the section file's as Copy Link to Page gives it), or `None` for any other URL.
75pub fn parse_internal_link(url: &str) -> Option<InternalLink> {76pub fn parse_internal_link(url: &str) -> Option<InternalLink> {
76 let guid = |value: &str| {77 let guid = |value: &str| {
77 format!("{value},0")78 format!("{value},0")
...@@ -79,7 +80,8 @@ pub fn parse_internal_link(url: &str) -> Option<InternalLink> {...@@ -79,7 +80,8 @@ pub fn parse_internal_link(url: &str) -> Option<InternalLink> {
79 .ok()80 .ok()
80 .map(|id| id.guid)81 .map(|id| id.guid)
81 };82 };
82 let parts: Vec<&str> = url.strip_prefix("onenote:#")?.split('&').collect();83 let (_, fragment) = url.strip_prefix("onenote:")?.split_once('#')?;
84 let parts: Vec<&str> = fragment.split('&').collect();
83 let value = |name: &str| {85 let value = |name: &str| {
84 parts.iter().find_map(|part| {86 parts.iter().find_map(|part| {
85 part.strip_prefix(name)87 part.strip_prefix(name)
crates/onestore/src/page/math.rs+99-41
...@@ -149,6 +149,10 @@ impl Math {...@@ -149,6 +149,10 @@ impl Math {
149 Math::Identifier(c) => tag(out, "mi", &c.to_string()),149 Math::Identifier(c) => tag(out, "mi", &c.to_string()),
150 Math::Function(name) => tag(out, "mi", name),150 Math::Function(name) => tag(out, "mi", name),
151 Math::Number(n) => tag(out, "mn", n),151 Math::Number(n) => tag(out, "mn", n),
152 // A parenthesis an object does not pair, as in linear text, is no fence.
153 Math::Operator(c @ ('(' | ')')) => {
154 out.push_str(&format!("<mml:mo fence=\"false\">{c}</mml:mo>"));
155 }
152 Math::Operator(c) => tag(out, "mo", &c.to_string()),156 Math::Operator(c) => tag(out, "mo", &c.to_string()),
153 Math::Object {157 Math::Object {
154 kind,158 kind,
...@@ -346,7 +350,7 @@ fn tag(out: &mut String, element: &str, content: &str) {...@@ -346,7 +350,7 @@ fn tag(out: &mut String, element: &str, content: &str) {
346350
347/// OneNote exports mathematical italic Latin letters as their plain letters and leaves every351/// OneNote exports mathematical italic Latin letters as their plain letters and leaves every
348/// other alphabet (Greek, double-struck, …) as stored.352/// other alphabet (Greek, double-struck, …) as stored.
349fn plain(c: char) -> char {353pub(crate) fn plain(c: char) -> char {
350 match u32::from(c) {354 match u32::from(c) {
351 code @ 0x1d434..=0x1d44d => char::from_u32(code - 0x1d434 + u32::from('A')).unwrap(),355 code @ 0x1d434..=0x1d44d => char::from_u32(code - 0x1d434 + u32::from('A')).unwrap(),
352 code @ 0x1d44e..=0x1d467 => char::from_u32(code - 0x1d44e + u32::from('a')).unwrap(),356 code @ 0x1d44e..=0x1d467 => char::from_u32(code - 0x1d44e + u32::from('a')).unwrap(),
...@@ -358,27 +362,32 @@ fn plain(c: char) -> char {...@@ -358,27 +362,32 @@ fn plain(c: char) -> char {
358impl Math {362impl Math {
359 /// The paragraph OneNote stores for `nodes`: linear text with object controls, every run363 /// The paragraph OneNote stores for `nodes`: linear text with object controls, every run
360 /// formatted the way the equation editor formats it (Cambria Math, italic, the math flags364 /// formatted the way the equation editor formats it (Cambria Math, italic, the math flags
361 /// and the math language over `base`), and the run data each object carries.365 /// and the math language over `base`; function names upright), and the run data each object
366 /// carries.
367 /// How the equation editor formats math typed over `base`, before it builds up: as a
368 /// run between objects.
369 pub fn format(base: &crate::document::Format) -> crate::document::Format {
370 let mut format = style(base);
371 format.math_object = Some(MathObject {
372 kind: PLAIN_RUN,
373 arguments: None,
374 columns: None,
375 symbols: Vec::new(),
376 });
377 format
378 }
379
362 pub fn paragraph(nodes: &[Math], base: &crate::document::Format) -> Paragraph {380 pub fn paragraph(nodes: &[Math], base: &crate::document::Format) -> Paragraph {
363 let mut style = base.clone();381 let style = style(base);
364 style.italic = Some(true);382 let mut runs = Vec::new();
365 style.hidden = Some(false);383 write_sequence(nodes, None, &mut runs);
366 style.hyperlink = Some(false);384 Paragraph::from_runs(runs.into_iter().map(|run| {
367 style.math = Some(true);
368 style.embedded_object = Some(true);
369 style.font = Some("Cambria Math".into());
370 style.language = Some(0x1007f);
371 let mut runs: Vec<(String, Option<MathObject>)> = Vec::new();
372 write_sequence(nodes, &mut runs);
373 Paragraph::from_runs(runs.into_iter().map(|(text, object)| {
374 let mut format = style.clone();385 let mut format = style.clone();
375 format.math_object = Some(object.unwrap_or(MathObject {386 if run.upright {
376 kind: PLAIN_RUN,387 format.italic = Some(false);
377 arguments: None,388 }
378 columns: None,389 format.math_object = Some(run.object);
379 symbols: Vec::new(),390 (run.text, format)
380 }));
381 (text, format)
382 }))391 }))
383 }392 }
384}393}
...@@ -386,12 +395,56 @@ impl Math {...@@ -386,12 +395,56 @@ impl Math {
386/// The object kind OneNote gives runs between objects.395/// The object kind OneNote gives runs between objects.
387const PLAIN_RUN: u32 = 0x9000_0000;396const PLAIN_RUN: u32 = 0x9000_0000;
388397
389fn write_sequence(nodes: &[Math], runs: &mut Vec<(String, Option<MathObject>)>) {398/// How the equation editor formats math over `base`.
399fn style(base: &crate::document::Format) -> crate::document::Format {
400 let mut style = base.clone();
401 style.italic = Some(true);
402 style.hidden = Some(false);
403 style.hyperlink = Some(false);
404 style.hyperlink_label = None;
405 style.math = Some(true);
406 style.embedded_object = Some(true);
407 style.font = Some("Cambria Math".into());
408 style.language = Some(0x1007f);
409 style
410}
411
412struct Run {
413 text: String,
414 object: MathObject,
415 upright: bool,
416 /// Text alone, which the control ending its argument may join.
417 leaf: bool,
418}
419
420/// Runs for `nodes`, an argument of an object of kind `within` or the equation itself. Text
421/// in an argument carries its object's kind, as the equation editor stores it.
422fn write_sequence(nodes: &[Math], within: Option<u32>, runs: &mut Vec<Run>) {
423 let object = || MathObject {
424 kind: within.unwrap_or(PLAIN_RUN),
425 arguments: None,
426 columns: None,
427 symbols: Vec::new(),
428 };
390 let mut leaf = String::new();429 let mut leaf = String::new();
430 let flush = |leaf: &mut String, runs: &mut Vec<Run>, upright: bool| {
431 if !leaf.is_empty() {
432 runs.push(Run {
433 text: std::mem::take(leaf),
434 object: object(),
435 upright,
436 leaf: true,
437 });
438 }
439 };
391 for node in nodes {440 for node in nodes {
392 match node {441 match node {
393 Math::Identifier(c) => leaf.push(italic(*c)),442 Math::Identifier(c) => leaf.push(italic(*c)),
394 Math::Function(name) => leaf.push_str(name),443 Math::Function(name) => {
444 flush(&mut leaf, runs, false);
445 leaf.push_str(name);
446 flush(&mut leaf, runs, true);
447 }
395 Math::Number(n) => leaf.push_str(n),448 Math::Number(n) => leaf.push_str(n),
396 Math::Operator(c) => leaf.push(*c),449 Math::Operator(c) => leaf.push(*c),
397 Math::Object {450 Math::Object {
...@@ -400,21 +453,22 @@ fn write_sequence(nodes: &[Math], runs: &mut Vec<(String, Option<MathObject>)>)...@@ -400,21 +453,22 @@ fn write_sequence(nodes: &[Math], runs: &mut Vec<(String, Option<MathObject>)>)
400 columns,453 columns,
401 arguments,454 arguments,
402 } => {455 } => {
403 if !leaf.is_empty() {456 flush(&mut leaf, runs, false);
404 runs.push((std::mem::take(&mut leaf), None));457 runs.push(Run {
405 }458 text: OBJECT_START.to_string(),
406 runs.push((459 object: MathObject {
407 OBJECT_START.to_string(),
408 Some(MathObject {
409 kind: *kind,460 kind: *kind,
410 arguments: Some(arguments.len() as u32),461 arguments: Some(arguments.len() as u32),
411 columns: *columns,462 columns: *columns,
412 symbols: symbols.clone(),463 symbols: symbols.clone(),
413 }),464 },
414 ));465 upright: false,
466 leaf: false,
467 });
415 for (index, argument) in arguments.iter().enumerate() {468 for (index, argument) in arguments.iter().enumerate() {
416 let before = runs.len();469 let before = runs.len();
417 write_sequence(argument, runs);470 write_sequence(argument, Some(*kind), runs);
471 let extended = runs.len() > before;
418 let control = if index + 1 == arguments.len() {472 let control = if index + 1 == arguments.len() {
419 OBJECT_END473 OBJECT_END
420 } else {474 } else {
...@@ -426,29 +480,33 @@ fn write_sequence(nodes: &[Math], runs: &mut Vec<(String, Option<MathObject>)>)...@@ -426,29 +480,33 @@ fn write_sequence(nodes: &[Math], runs: &mut Vec<(String, Option<MathObject>)>)
426 columns: None,480 columns: None,
427 symbols: Vec::new(),481 symbols: Vec::new(),
428 };482 };
429 let extended = runs.len() > before;
430 match runs.last_mut() {483 match runs.last_mut() {
431 Some((text, slot)) if extended && slot.is_none() => {484 Some(run) if extended && run.leaf && !run.upright => {
432 text.push(control);485 run.text.push(control);
433 *slot = Some(object);486 run.object = object;
487 run.leaf = false;
434 }488 }
435 _ => runs.push((control.to_string(), Some(object))),489 _ => runs.push(Run {
490 text: control.to_string(),
491 object,
492 upright: false,
493 leaf: false,
494 }),
436 }495 }
437 }496 }
438 }497 }
439 }498 }
440 }499 }
441 if !leaf.is_empty() {500 flush(&mut leaf, runs, false);
442 runs.push((leaf, None));
443 }
444}501}
445502
446/// The equation editor stores Latin letters as mathematical italics.503/// The equation editor stores Latin letters and lowercase Greek as mathematical italics.
447fn italic(c: char) -> char {504pub(crate) fn italic(c: char) -> char {
448 match c {505 match c {
449 'h' => '\u{210e}',506 'h' => '\u{210e}',
450 'A'..='Z' => char::from_u32(0x1d434 + u32::from(c) - u32::from('A')).unwrap(),507 'A'..='Z' => char::from_u32(0x1d434 + u32::from(c) - u32::from('A')).unwrap(),
451 'a'..='z' => char::from_u32(0x1d44e + u32::from(c) - u32::from('a')).unwrap(),508 'a'..='z' => char::from_u32(0x1d44e + u32::from(c) - u32::from('a')).unwrap(),
509 'α'..='ω' => char::from_u32(0x1d6fc + u32::from(c) - u32::from('α')).unwrap(),
452 c => c,510 c => c,
453 }511 }
454}512}
crates/onestore/src/page/mod.rs+28-10
...@@ -11,6 +11,7 @@ use std::{...@@ -11,6 +11,7 @@ use std::{
11};11};
1212
13pub mod ink;13pub mod ink;
14mod linear;
14pub mod link;15pub mod link;
15pub mod math;16pub mod math;
16pub mod text;17pub mod text;
...@@ -378,7 +379,11 @@ impl Page {...@@ -378,7 +379,11 @@ impl Page {
378 let mut selected = None;379 let mut selected = None;
379 for (space, id) in document.pages()? {380 for (space, id) in document.pages()? {
380 let revision = document.active(space)?;381 let revision = document.active(space)?;
381 if page_title(revision, id) == Some(title) {382 if page_title(revision, id)
383 .map(crate::edit::without_fields)
384 .as_deref()
385 == Some(title)
386 {
382 if selected.is_some() {387 if selected.is_some() {
383 return Err(Error {388 return Err(Error {
384 offset: 0,389 offset: 0,
...@@ -406,8 +411,8 @@ impl Page {...@@ -406,8 +411,8 @@ impl Page {
406 /// `copy`, renaming `ids`, identities on this page, to theirs on the copy.411 /// `copy`, renaming `ids`, identities on this page, to theirs on the copy.
407 pub fn copy_with(&self, ids: &mut [ExGuid]) -> Result<Self, Error> {412 pub fn copy_with(&self, ids: &mut [ExGuid]) -> Result<Self, Error> {
408 let invalid = |message| Error { offset: 0, message };413 let invalid = |message| Error { offset: 0, message };
409 let mut value = serde_json::to_value((self, &*ids))414 let mut value =
410 .map_err(|_| invalid("Page is not serializable"))?;415 serde_json::to_value((self, &*ids)).map_err(|_| invalid("Page is not serializable"))?;
411 let mut fresh: BTreeMap<String, String> = BTreeMap::new();416 let mut fresh: BTreeMap<String, String> = BTreeMap::new();
412 fn walk(417 fn walk(
413 value: &mut serde_json::Value,418 value: &mut serde_json::Value,
...@@ -495,7 +500,10 @@ impl Page {...@@ -495,7 +500,10 @@ impl Page {
495 PageObject::Title(title) => Some(title),500 PageObject::Title(title) => Some(title),
496 _ => None,501 _ => None,
497 })?;502 })?;
498 let outline = title.outlines.iter().find(|outline| Some(outline.id) == title.date)?;503 let outline = title
504 .outlines
505 .iter()
506 .find(|outline| Some(outline.id) == title.date)?;
499 let [first, second] = &outline.paragraphs[..] else {507 let [first, second] = &outline.paragraphs[..] else {
500 return None;508 return None;
501 };509 };
...@@ -503,7 +511,9 @@ impl Page {...@@ -503,7 +511,9 @@ impl Page {
503 let text = |object: &TextObject| object.text.text().to_owned();511 let text = |object: &TextObject| object.text.text().to_owned();
504 // OneNote writes the date first; a field marked as the other role says otherwise.512 // OneNote writes the date first; a field marked as the other role says otherwise.
505 Some(513 Some(
506 if first.date_field == Some(DateField::Time) || second.date_field == Some(DateField::Date) {514 if first.date_field == Some(DateField::Time)
515 || second.date_field == Some(DateField::Date)
516 {
507 [text(second), text(first)]517 [text(second), text(first)]
508 } else {518 } else {
509 [text(first), text(second)]519 [text(first), text(second)]
...@@ -539,7 +549,7 @@ impl Page {...@@ -539,7 +549,7 @@ impl Page {
539 _ => None,549 _ => None,
540 });550 });
541 (551 (
542 page_title(revision, id).unwrap_or_default().to_owned(),552 crate::edit::without_fields(page_title(revision, id).unwrap_or_default()),
543 level.unwrap_or(1),553 level.unwrap_or(1),
544 )554 )
545 }555 }
...@@ -577,7 +587,7 @@ impl Page {...@@ -577,7 +587,7 @@ impl Page {
577 .filter(|node| matches!(node.kind, Kind::Metadata { .. }))587 .filter(|node| matches!(node.kind, Kind::Metadata { .. }))
578 .and_then(|node| node.extra.first());588 .and_then(|node| node.extra.first());
579 let mut page = Self {589 let mut page = Self {
580 title: page_title(revision, id).unwrap_or_default().to_owned(),590 title: crate::edit::without_fields(page_title(revision, id).unwrap_or_default()),
581 identity: Self::identity_of(revision),591 identity: Self::identity_of(revision),
582 created: metadata592 created: metadata
583 .and_then(|fields| fields.iter().find(|field| field.id == 0x18001c65))593 .and_then(|fields| fields.iter().find(|field| field.id == 0x18001c65))
...@@ -604,7 +614,7 @@ impl Page {...@@ -604,7 +614,7 @@ impl Page {
604 FieldValue::Bytes(&[red, green, blue, alpha]) => {614 FieldValue::Bytes(&[red, green, blue, alpha]) => {
605 Ok(u32::from_le_bytes([red, green, blue, alpha]))615 Ok(u32::from_le_bytes([red, green, blue, alpha]))
606 }616 }
607 _ => Err(invalid("The page colour is damaged")),617 _ => Err(invalid("The page color is damaged")),
608 })618 })
609 .transpose()?,619 .transpose()?,
610 objects: Vec::new(),620 objects: Vec::new(),
...@@ -755,7 +765,11 @@ fn inherited_by(...@@ -755,7 +765,11 @@ fn inherited_by(
755 paragraph_style, ..765 paragraph_style, ..
756 } => {766 } => {
757 let style = match paragraph_style {767 let style = match paragraph_style {
758 Some(id) => revision.nodes.get(id).map(|n| n.format.clone()).unwrap_or_default(),768 Some(id) => revision
769 .nodes
770 .get(id)
771 .map(|n| n.format.clone())
772 .unwrap_or_default(),
759 None => Format::default(),773 None => Format::default(),
760 };774 };
761 node.format775 node.format
...@@ -805,7 +819,11 @@ pub(crate) fn text_of(...@@ -805,7 +819,11 @@ pub(crate) fn text_of(
805 let runs = revision.text_runs(text)?;819 let runs = revision.text_runs(text)?;
806 Ok(if runs.is_empty() {820 Ok(if runs.is_empty() {
807 let style = match paragraph_style {821 let style = match paragraph_style {
808 Some(id) => revision.nodes.get(id).map(|n| n.format.clone()).unwrap_or_default(),822 Some(id) => revision
823 .nodes
824 .get(id)
825 .map(|n| n.format.clone())
826 .unwrap_or_default(),
809 None => Format::default(),827 None => Format::default(),
810 };828 };
811 Paragraph::new(String::new(), node.format.inherit(&style).inherit(&format))829 Paragraph::new(String::new(), node.format.inherit(&style).inherit(&format))
crates/onestore/src/pages.rs+66-31
...@@ -38,7 +38,11 @@ pub(crate) fn metadata_guid(metadata: &crate::document::Element<'_>) -> Result<[...@@ -38,7 +38,11 @@ pub(crate) fn metadata_guid(metadata: &crate::document::Element<'_>) -> Result<[
38 .ok_or_else(|| invalid("Page metadata needs its page identifier"))38 .ok_or_else(|| invalid("Page metadata needs its page identifier"))
39}39}
4040
41pub(crate) fn set_references(object: &mut PropertyObject, property: u32, ids: &[ExGuid]) -> Result<(), Error> {41pub(crate) fn set_references(
42 object: &mut PropertyObject,
43 property: u32,
44 ids: &[ExGuid],
45) -> Result<(), Error> {
42 let mut references = Vec::new();46 let mut references = Vec::new();
43 for id in ids {47 for id in ids {
44 references.extend_from_slice(&object.reference(*id)?);48 references.extend_from_slice(&object.reference(*id)?);
...@@ -184,19 +188,29 @@ pub(crate) fn section_changes(...@@ -184,19 +188,29 @@ pub(crate) fn section_changes(
184 )188 )
185 };189 };
186 if element(&manifest)?.content.len() != 1 {190 if element(&manifest)?.content.len() != 1 {
187 return Err(invalid("Each page space must contain one page with a valid level"));191 return Err(invalid(
192 "Each page space must contain one page with a valid level",
193 ));
188 }194 }
189 Ok(PageParts {195 Ok(PageParts {
190 metadata: element(&stored)?,196 metadata: element(&stored)?,
191 manifest: (manifest, PropertyObject::from_object(&revision.objects[&manifest])?),197 manifest: (
192 stored_metadata: (stored, PropertyObject::from_object(&revision.objects[&stored])?),198 manifest,
199 PropertyObject::from_object(&revision.objects[&manifest])?,
200 ),
201 stored_metadata: (
202 stored,
203 PropertyObject::from_object(&revision.objects[&stored])?,
204 ),
193 })205 })
194 };206 };
195 Ok(page_changes(root, &view, &raw, &pages, parts, edits, removals)?207 Ok(
196 .unwrap_or_default()208 page_changes(root, &view, &raw, &pages, parts, edits, removals)?
197 .into_iter()209 .unwrap_or_default()
198 .filter(|(_, changes)| !changes.is_empty())210 .into_iter()
199 .collect())211 .filter(|(_, changes)| !changes.is_empty())
212 .collect(),
213 )
200}214}
201215
202/// The objects page moves, indentation and removals change in each space; none when the216/// The objects page moves, indentation and removals change in each space; none when the
...@@ -253,7 +267,10 @@ fn page_changes<'a>(...@@ -253,7 +267,10 @@ fn page_changes<'a>(
253 "Each page space must contain one page with a valid level",267 "Each page space must contain one page with a valid level",
254 ));268 ));
255 }269 }
256 if page_guids.insert(metadata_guid(&page.metadata)?, *sid).is_some() {270 if page_guids
271 .insert(metadata_guid(&page.metadata)?, *sid)
272 .is_some()
273 {
257 return Err(invalid("Pages must have distinct metadata identifiers"));274 return Err(invalid("Pages must have distinct metadata identifiers"));
258 }275 }
259 order.push(*sid);276 order.push(*sid);
...@@ -305,9 +322,7 @@ fn page_changes<'a>(...@@ -305,9 +322,7 @@ fn page_changes<'a>(
305 let sid = page_guids322 let sid = page_guids
306 .get(&metadata_guid(&view.nodes[id])?)323 .get(&metadata_guid(&view.nodes[id])?)
307 .filter(|sid| series.spaces.contains(sid))324 .filter(|sid| series.spaces.contains(sid))
308 .ok_or_else(|| {325 .ok_or_else(|| invalid("Metadata copy must identify a page in its series"))?;
309 invalid("Metadata copy must identify a page in its series")
310 })?;
311 if copies.insert(*sid, *id).is_some() && !mismatched {326 if copies.insert(*sid, *id).is_some() && !mismatched {
312 return Err(invalid("Each page needs its own ordinary metadata copy"));327 return Err(invalid("Each page needs its own ordinary metadata copy"));
313 }328 }
...@@ -334,11 +349,10 @@ fn page_changes<'a>(...@@ -334,11 +349,10 @@ fn page_changes<'a>(
334 order.remove(at);349 order.remove(at);
335 let position = match before {350 let position = match before {
336 None => order.len(),351 None => order.len(),
337 Some(before) => {352 Some(before) => order
338 order.iter().position(|sid| *sid == before).ok_or_else(|| {353 .iter()
339 invalid("The movement anchor is no longer in the section")354 .position(|sid| *sid == before)
340 })?355 .ok_or_else(|| invalid("The movement anchor is no longer in the section"))?,
341 }
342 };356 };
343 order.insert(position, edit.space);357 order.insert(position, edit.space);
344 }358 }
...@@ -351,7 +365,7 @@ fn page_changes<'a>(...@@ -351,7 +365,7 @@ fn page_changes<'a>(
351 if order == original_order && levels == original_levels {365 if order == original_order && levels == original_levels {
352 return Ok(None);366 return Ok(None);
353 }367 }
354 let mut groups: Vec<Vec<ExGuid>> = Vec::new();368 let mut groups: Vec<Vec<ExGuid>> = Vec::new();
355 for sid in order {369 for sid in order {
356 if levels[&sid] == 1 {370 if levels[&sid] == 1 {
357 groups.push(Vec::new());371 groups.push(Vec::new());
...@@ -405,8 +419,8 @@ fn page_changes<'a>(...@@ -405,8 +419,8 @@ fn page_changes<'a>(
405 )419 )
406 };420 };
407 children.push(id);421 children.push(id);
408 let membership_changed = old_id422 let membership_changed =
409 .is_none_or(|old| view.nodes[&old].spaces != spaces || stale.contains(&old));423 old_id.is_none_or(|old| view.nodes[&old].spaces != spaces || stale.contains(&old));
410 if !membership_changed424 if !membership_changed
411 && !spaces.iter().any(|sid| {425 && !spaces.iter().any(|sid| {
412 levels[sid] != original_levels[sid]426 levels[sid] != original_levels[sid]
...@@ -611,10 +625,7 @@ impl PageCreation {...@@ -611,10 +625,7 @@ impl PageCreation {
611 }625 }
612626
613 /// The root space's changes creating this page, and the new space's roots and objects.627 /// The root space's changes creating this page, and the new space's roots and objects.
614 pub(crate) fn changes(628 pub(crate) fn changes(&self, section: &mut crate::Section<'_>) -> Result<Creation, Error> {
615 &self,
616 section: &mut crate::Section<'_>,
617 ) -> Result<Creation, Error> {
618 self.validate()?;629 self.validate()?;
619 if section.revision(self.space()).is_ok() {630 if section.revision(self.space()).is_ok() {
620 return Err(invalid(631 return Err(invalid(
...@@ -636,7 +647,14 @@ impl PageCreation {...@@ -636,7 +647,14 @@ impl PageCreation {
636 section: &mut crate::Section<'_>,647 section: &mut crate::Section<'_>,
637 title: &str,648 title: &str,
638 level: u32,649 level: u32,
639 ) -> Result<(BTreeMap<u32, ExGuid>, crate::active::Changes, PropertyObject), Error> {650 ) -> Result<
651 (
652 BTreeMap<u32, ExGuid>,
653 crate::active::Changes,
654 PropertyObject,
655 ),
656 Error,
657 > {
640 if self.before.is_some() {658 if self.before.is_some() {
641 return Err(invalid("A conflict page is not placed in the page list"));659 return Err(invalid("A conflict page is not placed in the page list"));
642 }660 }
...@@ -683,7 +701,11 @@ impl PageCreation {...@@ -683,7 +701,11 @@ impl PageCreation {
683 };701 };
684 let text = |value: &str, role: u32| {702 let text = |value: &str, role: u32| {
685 // OneNote keeps Latin-1 text as TextExtendedAscii, other text as Unicode.703 // OneNote keeps Latin-1 text as TextExtendedAscii, other text as Unicode.
686 let content = match value.chars().map(|c| u8::try_from(c).ok()).collect::<Option<Vec<u8>>>() {704 let content = match value
705 .chars()
706 .map(|c| u8::try_from(c).ok())
707 .collect::<Option<Vec<u8>>>()
708 {
687 Some(latin) => (0x1c003498, latin),709 Some(latin) => (0x1c003498, latin),
688 None => (0x1c001c22, string(value)),710 None => (0x1c001c22, string(value)),
689 };711 };
...@@ -829,8 +851,11 @@ impl PageCreation {...@@ -829,8 +851,11 @@ impl PageCreation {
829 modified(),851 modified(),
830 (852 (
831 0x24001c20,853 0x24001c20,
832 [reference(14), self.date.iter().flat_map(|_| reference(20)).collect()]854 [
833 .concat(),855 reference(14),
856 self.date.iter().flat_map(|_| reference(20)).collect(),
857 ]
858 .concat(),
834 ),859 ),
835 (0x14001c14, 0_f32.to_le_bytes().to_vec()),860 (0x14001c14, 0_f32.to_le_bytes().to_vec()),
836 (0x14001c15, 0_f32.to_le_bytes().to_vec()),861 (0x14001c15, 0_f32.to_le_bytes().to_vec()),
...@@ -949,14 +974,24 @@ mod tests {...@@ -949,14 +974,24 @@ mod tests {
949 };974 };
950975
951 /// Page moves and removals through a section, sealed: the image they leave.976 /// Page moves and removals through a section, sealed: the image they leave.
952 fn edit_pages(source: &[u8], edits: &[PageEdit], removals: &[ExGuid]) -> Result<Vec<u8>, OpError> {977 fn edit_pages(
978 source: &[u8],
979 edits: &[PageEdit],
980 removals: &[ExGuid],
981 ) -> Result<Vec<u8>, OpError> {
953 let arena = crate::Arena::default();982 let arena = crate::Arena::default();
954 let mut section = crate::Section::open(&arena, source.to_vec()).map_err(OpError::Failed)?;983 let mut section = crate::Section::open(&arena, source.to_vec()).map_err(OpError::Failed)?;
955 let mut ops = vec![Op::Section(SectionOp::Pages(edits.to_vec()))];984 let mut ops = vec![Op::Section(SectionOp::Pages(edits.to_vec()))];
956 if !removals.is_empty() {985 if !removals.is_empty() {
957 ops.push(Op::Section(SectionOp::Delete(removals.to_vec())));986 ops.push(Op::Section(SectionOp::Delete(removals.to_vec())));
958 }987 }
959 section.apply("Author", &Edit { at: 133_700_000_000_000_000, ops })?;988 section.apply(
989 "Author",
990 &Edit {
991 at: 133_700_000_000_000_000,
992 ops,
993 },
994 )?;
960 section.seal().map_err(OpError::Failed)?;995 section.seal().map_err(OpError::Failed)?;
961 Ok(section.image())996 Ok(section.image())
962 }997 }
crates/onestore/src/protected/mod.rs+13-5
...@@ -416,7 +416,9 @@ impl UnlockedSection<'_> {...@@ -416,7 +416,9 @@ impl UnlockedSection<'_> {
416 ));416 ));
417 }417 }
418 if edit.ops.iter().any(|op| matches!(op, Op::Section(_))) {418 if edit.ops.iter().any(|op| matches!(op, Op::Section(_))) {
419 return Err(OpError::Unsupported("A protected section's page list is not edited"));419 return Err(OpError::Unsupported(
420 "A protected section's page list is not edited",
421 ));
420 }422 }
421 let twin = self.twin().map_err(failed)?;423 let twin = self.twin().map_err(failed)?;
422 let arena = crate::Arena::default();424 let arena = crate::Arena::default();
...@@ -435,7 +437,10 @@ impl UnlockedSection<'_> {...@@ -435,7 +437,10 @@ impl UnlockedSection<'_> {
435 .map(|space| Ok((*space, applied_index.resolve_active(*space)?)))437 .map(|space| Ok((*space, applied_index.resolve_active(*space)?)))
436 .collect::<std::result::Result<Vec<_>, crate::Error>>()438 .collect::<std::result::Result<Vec<_>, crate::Error>>()
437 .map_err(failed)?;439 .map_err(failed)?;
438 let edited: Vec<_> = edited.iter().map(|(space, after)| (*space, after)).collect();440 let edited: Vec<_> = edited
441 .iter()
442 .map(|(space, after)| (*space, after))
443 .collect();
439 let payloads = crate::write::declared_payloads(&applied_store)444 let payloads = crate::write::declared_payloads(&applied_store)
440 .into_iter()445 .into_iter()
441 .map(|guid| Ok((guid, applied_store.file_data(guid)?)))446 .map(|guid| Ok((guid, applied_store.file_data(guid)?)))
...@@ -444,12 +449,15 @@ impl UnlockedSection<'_> {...@@ -444,12 +449,15 @@ impl UnlockedSection<'_> {
444 let transaction = crate::write::squash(self.index, &edited, &payloads, Some(self))449 let transaction = crate::write::squash(self.index, &edited, &payloads, Some(self))
445 .map_err(failed)?450 .map_err(failed)?
446 .ok_or(OpError::Unsupported("The edit changes nothing"))?;451 .ok_or(OpError::Unsupported("The edit changes nothing"))?;
447 let written = crate::write::applied(self.index.store.data, Some(&transaction))452 let written =
448 .map_err(failed)?;453 crate::write::applied(self.index.store.data, Some(&transaction)).map_err(failed)?;
449 let store = crate::Store::parse(&written).map_err(failed)?;454 let store = crate::Store::parse(&written).map_err(failed)?;
450 let index = RevisionIndex::parse(&store).map_err(failed)?;455 let index = RevisionIndex::parse(&store).map_err(failed)?;
451 UnlockedSection::unlock(&index, Limits::default(), |metadata, _| {456 UnlockedSection::unlock(&index, Limits::default(), |metadata, _| {
452 self.keys.get(metadata).cloned().ok_or(Error::PasswordMismatch)457 self.keys
458 .get(metadata)
459 .cloned()
460 .ok_or(Error::PasswordMismatch)
453 })461 })
454 .map_err(|error| match error {462 .map_err(|error| match error {
455 Error::Invalid(error) => failed(error),463 Error::Invalid(error) => failed(error),
crates/onestore/src/section.rs+372-146
...@@ -9,13 +9,17 @@ use crate::{...@@ -9,13 +9,17 @@ use crate::{
9 page::Page,9 page::Page,
10 store::StoreState,10 store::StoreState,
11 write::{11 write::{
12 Appending, LiveRevision, Sealing, Written, chain_depth, check_transaction, declared,12 Appending, Commit, LiveRevision, PropertyObject, Sealing, Written, chain_depth,
13 replacement, unchanged,13 check_transaction, declared, label_payload, node, replacement, unchanged,
14 },14 },
15};15};
16use bumpalo::Bump;16use bumpalo::Bump;
17use std::collections::{BTreeMap, BTreeSet};17use std::collections::{BTreeMap, BTreeSet};
1818
19mod versions;
20pub(crate) use versions::HISTORY;
21pub use versions::PageVersion;
22
19type Result<T> = std::result::Result<T, Error>;23type Result<T> = std::result::Result<T, Error>;
2024
21/// Holds the bytes a `Section` reads: the image it opened and each transaction it sealed.25/// Holds the bytes a `Section` reads: the image it opened and each transaction it sealed.
...@@ -58,6 +62,58 @@ struct Space<'a> {...@@ -58,6 +62,58 @@ struct Space<'a> {
58 /// The active revision the file stores; none before a new space's first seal.62 /// The active revision the file stores; none before a new space's first seal.
59 rid: Option<ExGuid>,63 rid: Option<ExGuid>,
60 state: SpaceState<'a>,64 state: SpaceState<'a>,
65 /// The version a restore took the open revision from: the revision it depends on, which
66 /// the seal writes even unchanged.
67 restored: Option<ExGuid>,
68 /// Page states restores made versions of, which the seal writes before the open revision.
69 frozen: Vec<Frozen<'a>>,
70 history: Option<History<'a>>,
71 /// Revisions the seal labels as versions, with their contexts.
72 labels: Vec<(ExGuid, ExGuid)>,
73 /// The newest revision a seal appended to the space.
74 newest: Option<ExGuid>,
75}
76
77impl<'a> Space<'a> {
78 fn new(rid: Option<ExGuid>, state: SpaceState<'a>, history: Option<History<'a>>) -> Self {
79 Self {
80 rid,
81 state,
82 restored: None,
83 frozen: Vec::new(),
84 history,
85 labels: Vec::new(),
86 newest: None,
87 }
88 }
89}
90
91/// A page state a restore made a version of: a revision under the identity it chose.
92#[derive(Clone)]
93struct Frozen<'a> {
94 rid: ExGuid,
95 previous: Option<ExGuid>,
96 live: LiveRevision<'a>,
97 revised: Revised,
98}
99
100/// A revision `changed` committed: what it declares, and of those the objects whose bytes it
101/// stores and the objects new to the space.
102#[derive(Clone)]
103struct Revised {
104 commit: Commit,
105 replaced: BTreeSet<ExGuid>,
106 created: BTreeSet<ExGuid>,
107}
108
109/// A page's version history (`versions::HISTORY`), with the objects edits since the last
110/// seal replaced in it.
111#[derive(Clone)]
112struct History<'a> {
113 rid: ExGuid,
114 revision: ResolvedRevision<'a>,
115 depth: usize,
116 pending: BTreeMap<ExGuid, PropertyObject>,
61}117}
62118
63#[derive(Clone)]119#[derive(Clone)]
...@@ -105,50 +161,24 @@ impl<'a> Section<'a> {...@@ -105,50 +161,24 @@ impl<'a> Section<'a> {
105 let Some(rid) = space.labels.get(&(ExGuid::default(), 1)).copied() else {161 let Some(rid) = space.labels.get(&(ExGuid::default(), 1)).copied() else {
106 continue;162 continue;
107 };163 };
108 let resolved = index.resolve(*id, rid)?;164 let revision = bound(bytes, index.resolve(*id, rid)?);
109 let revision = ResolvedRevision {165 let history = space
110 roots: resolved.roots,166 .labels
111 objects: resolved167 .get(&(versions::HISTORY, 1))
112 .objects168 .map(|history| -> Result<History<'a>> {
113 .into_iter()169 Ok(History {
114 .map(|(id, object)| {170 rid: *history,
115 // The parse borrows `bytes` only as long as `store`; its slices lie171 revision: bound(bytes, index.resolve(*id, *history)?),
116 // in `bytes`, which the arena keeps.172 depth: chain_depth(&index, *id, *history),
117 let data = match object.data {173 pending: BTreeMap::new(),
118 ObjectData::Properties(data) => {
119 ObjectData::Properties(within(bytes, data))
120 }
121 ObjectData::Encrypted(data) => {
122 ObjectData::Encrypted(within(bytes, data))
123 }
124 ObjectData::File {
125 reference,
126 extension,
127 } => ObjectData::File {
128 reference: within(bytes, reference),
129 extension: within(bytes, extension),
130 },
131 };
132 let object = crate::Object {
133 jcid: object.jcid,
134 reference_count: object.reference_count,
135 data,
136 global_ids: object.global_ids,
137 };
138 (id, object)
139 })174 })
140 .collect(),175 })
176 .transpose()?;
177 let state = SpaceState::Stored {
178 revision,
179 depth: chain_depth(&index, *id, rid),
141 };180 };
142 spaces.insert(181 spaces.insert(*id, Space::new(Some(rid), state, history));
143 *id,
144 Space {
145 rid: Some(rid),
146 state: SpaceState::Stored {
147 revision,
148 depth: chain_depth(&index, *id, rid),
149 },
150 },
151 );
152 }182 }
153 let mut files = Vec::new();183 let mut files = Vec::new();
154 for node in store.lists.values().flat_map(|list| &list.nodes) {184 for node in store.lists.values().flat_map(|list| &list.nodes) {
...@@ -209,6 +239,15 @@ impl<'a> Section<'a> {...@@ -209,6 +239,15 @@ impl<'a> Section<'a> {
209 .filter_map(|(space, stored)| Some((*space, stored.rid?)))239 .filter_map(|(space, stored)| Some((*space, stored.rid?)))
210 }240 }
211241
242 /// Each object space with the revision this section last sealed in it, where one did, or
243 /// its active revision: a seal that changes only a page's versions leaves the page's
244 /// active revision as it was.
245 pub fn newest(&self) -> impl Iterator<Item = (ExGuid, ExGuid)> + '_ {
246 self.spaces
247 .iter()
248 .filter_map(|(space, stored)| Some((*space, stored.newest.or(stored.rid)?)))
249 }
250
212 /// The page an object space holds, with the changes applied since the last seal.251 /// The page an object space holds, with the changes applied since the last seal.
213 pub fn page(&self, space: ExGuid) -> Result<Page> {252 pub fn page(&self, space: ExGuid) -> Result<Page> {
214 let stored = self.spaces.get(&space).ok_or(Error {253 let stored = self.spaces.get(&space).ok_or(Error {
...@@ -303,10 +342,14 @@ impl<'a> Section<'a> {...@@ -303,10 +342,14 @@ impl<'a> Section<'a> {
303 /// own, so a page in another series than before was moved.342 /// own, so a page in another series than before was moved.
304 pub fn series(&mut self) -> Result<BTreeMap<ExGuid, ExGuid>> {343 pub fn series(&mut self) -> Result<BTreeMap<ExGuid, ExGuid>> {
305 let view = &self.active(self.root)?.view;344 let view = &self.active(self.root)?.view;
306 let section = view.roots.get(&1).and_then(|id| view.nodes.get(id)).ok_or(Error {345 let section = view
307 offset: 0,346 .roots
308 message: "Section root is unavailable",347 .get(&1)
309 })?;348 .and_then(|id| view.nodes.get(id))
349 .ok_or(Error {
350 offset: 0,
351 message: "Section root is unavailable",
352 })?;
310 Ok(section353 Ok(section
311 .children354 .children
312 .iter()355 .iter()
...@@ -375,7 +418,8 @@ impl<'a> Section<'a> {...@@ -375,7 +418,8 @@ impl<'a> Section<'a> {
375 title = alternate_title;418 title = alternate_title;
376 }419 }
377 }420 }
378 pages.push((space, title.unwrap_or_default(), level));421 let title = crate::edit::without_fields(&title.unwrap_or_default());
422 pages.push((space, title, level));
379 }423 }
380 Ok(pages)424 Ok(pages)
381 }425 }
...@@ -447,6 +491,15 @@ impl<'a> Section<'a> {...@@ -447,6 +491,15 @@ impl<'a> Section<'a> {
447 Ok(open)491 Ok(open)
448 }492 }
449493
494 /// Keeps a space as it is for the running edit to return to, the first time it changes it.
495 pub(crate) fn remember(&mut self, space: ExGuid) {
496 if let Some(undo) = &mut self.undo
497 && !undo.spaces.contains_key(&space)
498 {
499 undo.spaces.insert(space, self.spaces.get(&space).cloned());
500 }
501 }
502
450 /// Stores a writer's objects and payloads in a space until the next seal; false when503 /// Stores a writer's objects and payloads in a space until the next seal; false when
451 /// it stores nothing. An error leaves the section to be reopened.504 /// it stores nothing. An error leaves the section to be reopened.
452 pub(crate) fn apply_changes(505 pub(crate) fn apply_changes(
...@@ -456,11 +509,7 @@ impl<'a> Section<'a> {...@@ -456,11 +509,7 @@ impl<'a> Section<'a> {
456 changes: Changes,509 changes: Changes,
457 ) -> Result<bool> {510 ) -> Result<bool> {
458 self.usable()?;511 self.usable()?;
459 if let Some(undo) = &mut self.undo512 self.remember(space);
460 && !undo.spaces.contains_key(&space)
461 {
462 undo.spaces.insert(space, self.spaces.get(&space).cloned());
463 }
464 let arena = &self.arena.0;513 let arena = &self.arena.0;
465 self.broken = true;514 self.broken = true;
466 let open = self.editable(space)?;515 let open = self.editable(space)?;
...@@ -519,17 +568,12 @@ impl<'a> Section<'a> {...@@ -519,17 +568,12 @@ impl<'a> Section<'a> {
519 let all: Vec<ExGuid> = work.revision.objects.keys().copied().collect();568 let all: Vec<ExGuid> = work.revision.objects.keys().copied().collect();
520 work.settle(all)?;569 work.settle(all)?;
521 let page = ActivePage::open(space, work, FileType::Section, self.files())?;570 let page = ActivePage::open(space, work, FileType::Section, self.files())?;
522 self.spaces.insert(571 let state = SpaceState::Open(Box::new(Open {
523 space,572 file,
524 Space {573 page,
525 rid: None,574 pending,
526 state: SpaceState::Open(Box::new(Open {575 }));
527 file,576 self.spaces.insert(space, Space::new(None, state, None));
528 page,
529 pending,
530 })),
531 },
532 );
533 self.broken = false;577 self.broken = false;
534 Ok(())578 Ok(())
535 }579 }
...@@ -595,69 +639,124 @@ impl<'a> Section<'a> {...@@ -595,69 +639,124 @@ impl<'a> Section<'a> {
595 let mut appending = Appending::new(self.state.clone());639 let mut appending = Appending::new(self.state.clone());
596 let mut sealed = Vec::new();640 let mut sealed = Vec::new();
597 for (id, space) in &mut self.spaces {641 for (id, space) in &mut self.spaces {
598 let SpaceState::Open(open) = &mut space.state else {642 // A space's revisions and labels go in one fragment: page states restores made
599 continue;643 // versions of, the open revision, the version history, then the labels.
600 };644 let mut manifest = Vec::new();
601 if open.pending.is_empty() {645 for frozen in &space.frozen {
602 continue;646 appending.manifest(
647 &Sealing {
648 space: *id,
649 previous: frozen.previous,
650 new_space: false,
651 label: (ExGuid::default(), 1),
652 rid: Some(frozen.rid),
653 live: &frozen.live,
654 commit: &frozen.revised.commit,
655 replaced: &frozen.revised.replaced,
656 created: &frozen.revised.created,
657 },
658 &self.segments,
659 None,
660 None,
661 &mut manifest,
662 )?;
603 }663 }
604 let (work, file) = (&open.page.live, &mut open.file);664 let mut page = None;
605 let mut replacements = BTreeMap::new();665 if let SpaceState::Open(open) = &mut space.state
606 for object_id in &open.pending {666 && (!open.pending.is_empty() || space.restored.is_some())
607 if !work.is_reachable(*object_id) {667 {
608 continue;668 let file = &mut open.file;
609 }669 let restored = space.restored.is_some();
610 let object = &work.revision.objects[object_id];670 let written = match changed(arena, &open.page.live, &open.pending, file, restored)?
611 if let Some(stored) = file.revision.objects.get(object_id)
612 && unchanged(stored, object)?
613 {671 {
614 continue;672 Some(Revised {
673 commit,
674 replaced,
675 created,
676 }) => {
677 let (rid, stored) = appending.manifest(
678 &Sealing {
679 space: *id,
680 previous: space.restored.or(space.rid),
681 new_space: space.rid.is_none(),
682 label: (ExGuid::default(), 1),
683 rid: None,
684 live: file,
685 commit: &commit,
686 replaced: &replaced,
687 created: &created,
688 },
689 &self.segments,
690 None,
691 None,
692 &mut manifest,
693 )?;
694 if commit.checkpoint {
695 let all: Vec<ExGuid> = file.revision.objects.keys().copied().collect();
696 file.settle(all)?;
697 } else {
698 file.settle(commit.changed.iter().copied())?;
699 }
700 Some((rid, stored, commit))
701 }
702 None => None,
703 };
704 page = Some(written);
705 }
706 let mut history = None;
707 if let Some(stored) = &space.history
708 && !stored.pending.is_empty()
709 {
710 let mut live = LiveRevision::new(stored.revision.clone(), stored.depth)?;
711 let replacements = live.prepare(stored.pending.clone(), true)?;
712 if !replacements.is_empty() {
713 let replaced: BTreeSet<ExGuid> = replacements.keys().copied().collect();
714 let created = replaced
715 .iter()
716 .filter(|id| !live.revision.objects.contains_key(id))
717 .copied()
718 .collect();
719 let mut objects = Vec::new();
720 for (object_id, change) in replacements {
721 let bytes = arena.alloc_slice_copy(&change.bytes);
722 objects.push((object_id, declared(change.jcid, bytes, change.global_ids)?));
723 }
724 let commit = live.commit(objects)?;
725 let (rid, chunks) = appending.manifest(
726 &Sealing {
727 space: *id,
728 previous: Some(stored.rid),
729 new_space: false,
730 label: (versions::HISTORY, 1),
731 rid: None,
732 live: &live,
733 commit: &commit,
734 replaced: &replaced,
735 created: &created,
736 },
737 &self.segments,
738 None,
739 None,
740 &mut manifest,
741 )?;
742 if commit.checkpoint {
743 let all: Vec<ExGuid> = live.revision.objects.keys().copied().collect();
744 live.settle(all)?;
745 } else {
746 live.settle(commit.changed.iter().copied())?;
747 }
748 history = Some((rid, chunks, live, commit));
615 }749 }
616 replacements.insert(*object_id, replacement(*object_id, object)?);
617 }750 }
618 let replacements = file.prepare(replacements, true)?;751 for (rid, context) in &space.labels {
619 if replacements.is_empty() {752 manifest.push(node(0x5d, None, &label_payload(*rid, *context))?);
620 sealed.push((*id, None));
621 continue;
622 }753 }
623 let replaced: BTreeSet<ExGuid> = replacements.keys().copied().collect();754 if !manifest.is_empty() {
624 let created = replaced755 appending.close(*id, space.rid.is_none(), &manifest)?;
625 .iter()
626 .filter(|id| !file.revision.objects.contains_key(id))
627 .copied()
628 .collect();
629 let mut objects = Vec::new();
630 for (object_id, change) in replacements {
631 // The page holds these bytes already unless preparing remapped them.
632 let bytes = match work.revision.objects.get(&object_id).map(|o| o.data) {
633 Some(ObjectData::Properties(bytes)) if bytes == change.bytes => bytes,
634 _ => arena.alloc_slice_copy(&change.bytes),
635 };
636 objects.push((object_id, declared(change.jcid, bytes, change.global_ids)?));
637 }756 }
638 let commit = file.commit(objects)?;757 if page.is_some() || !manifest.is_empty() {
639 let (rid, stored) = appending.revision(758 sealed.push((*id, page, history));
640 &Sealing {
641 space: *id,
642 previous: space.rid,
643 new_space: space.rid.is_none(),
644 label: (ExGuid::default(), 1),
645 live: file,
646 commit: &commit,
647 replaced: &replaced,
648 created: &created,
649 },
650 &self.segments,
651 None,
652 None,
653 )?;
654 if commit.checkpoint {
655 let all: Vec<ExGuid> = file.revision.objects.keys().copied().collect();
656 file.settle(all)?;
657 } else {
658 file.settle(commit.changed.iter().copied())?;
659 }759 }
660 sealed.push((*id, Some((rid, stored, commit))));
661 }760 }
662 let payloads: Vec<_> = self761 let payloads: Vec<_> = self
663 .payloads762 .payloads
...@@ -674,20 +773,46 @@ impl<'a> Section<'a> {...@@ -674,20 +773,46 @@ impl<'a> Section<'a> {
674 }773 }
675 if let Some((transaction, state)) = &transaction {774 if let Some((transaction, state)) = &transaction {
676 let mut written = Vec::new();775 let mut written = Vec::new();
677 for (id, revision) in &sealed {776 let mut labels = Vec::new();
678 let (Some((rid, _, commit)), Some(space)) = (revision, self.spaces.get(id)) else {777 for (id, page, history) in &sealed {
679 continue;778 let space = &self.spaces[id];
680 };779 for frozen in &space.frozen {
681 let SpaceState::Open(open) = &space.state else {780 written.push(Written {
682 unreachable!()781 space: *id,
683 };782 rid: frozen.rid,
684 written.push(Written {783 previous: frozen.previous,
685 space: *id,784 label: (ExGuid::default(), 1),
686 rid: *rid,785 live: &frozen.live,
687 previous: space.rid,786 commit: &frozen.revised.commit,
688 live: &open.file,787 });
689 commit,788 }
690 });789 if let (Some(Some((rid, _, commit))), SpaceState::Open(open)) = (page, &space.state)
790 {
791 written.push(Written {
792 space: *id,
793 rid: *rid,
794 previous: space.restored.or(space.rid),
795 label: (ExGuid::default(), 1),
796 live: &open.file,
797 commit,
798 });
799 }
800 if let (Some((rid, _, live, commit)), Some(stored)) = (history, &space.history) {
801 written.push(Written {
802 space: *id,
803 rid: *rid,
804 previous: Some(stored.rid),
805 label: (versions::HISTORY, 1),
806 live,
807 commit,
808 });
809 }
810 labels.extend(
811 space
812 .labels
813 .iter()
814 .map(|(rid, context)| (*id, *rid, *context)),
815 );
691 }816 }
692 check_transaction(817 check_transaction(
693 &self.state,818 &self.state,
...@@ -695,6 +820,7 @@ impl<'a> Section<'a> {...@@ -695,6 +820,7 @@ impl<'a> Section<'a> {
695 transaction,820 transaction,
696 &self.segments,821 &self.segments,
697 &written,822 &written,
823 &labels,
698 &payloads,824 &payloads,
699 )?;825 )?;
700 let base = transaction.base.length;826 let base = transaction.base.length;
...@@ -703,23 +829,42 @@ impl<'a> Section<'a> {...@@ -703,23 +829,42 @@ impl<'a> Section<'a> {
703 self.patches.extend(transaction.patches.iter().cloned());829 self.patches.extend(transaction.patches.iter().cloned());
704 self.state = state.clone();830 self.state = state.clone();
705 self.declared.extend(payloads.iter().map(|(guid, _)| *guid));831 self.declared.extend(payloads.iter().map(|(guid, _)| *guid));
706 for (id, revision) in &sealed {832 let bind = |revision: &mut ResolvedRevision<'a>, stored: &[(ExGuid, crate::Chunk)]| {
707 let Some((rid, stored, _)) = revision else {
708 continue;
709 };
710 let space = self.spaces.get_mut(id).unwrap();
711 space.rid = Some(*rid);
712 let SpaceState::Open(open) = &mut space.state else {
713 unreachable!()
714 };
715 for (object_id, chunk) in stored {833 for (object_id, chunk) in stored {
716 let start = (chunk.offset - base) as usize;834 let start = (chunk.offset - base) as usize;
717 open.file.revision.objects.get_mut(object_id).unwrap().data =835 revision.objects.get_mut(object_id).unwrap().data =
718 ObjectData::Properties(&segment[start..start + chunk.length as usize]);836 ObjectData::Properties(&segment[start..start + chunk.length as usize]);
719 }837 }
838 };
839 for (id, page, history) in &mut sealed {
840 let space = self.spaces.get_mut(id).unwrap();
841 space.frozen.clear();
842 space.labels.clear();
843 if let (Some(Some((rid, stored, _))), SpaceState::Open(open)) =
844 (&*page, &mut space.state)
845 {
846 space.rid = Some(*rid);
847 space.restored = None;
848 space.newest = Some(*rid);
849 bind(&mut open.file.revision, stored);
850 }
851 if let Some((rid, stored, live, _)) = history.take() {
852 let mut revision = live.revision;
853 bind(&mut revision, &stored);
854 space.history = Some(History {
855 rid,
856 revision,
857 depth: live.depth,
858 pending: BTreeMap::new(),
859 });
860 space.newest = Some(rid);
861 }
720 }862 }
721 }863 }
722 for (id, revision) in sealed {864 for (id, page, _) in sealed {
865 let Some(revision) = page else {
866 continue;
867 };
723 let SpaceState::Open(open) = &mut self.spaces.get_mut(&id).unwrap().state else {868 let SpaceState::Open(open) = &mut self.spaces.get_mut(&id).unwrap().state else {
724 unreachable!()869 unreachable!()
725 };870 };
...@@ -759,6 +904,87 @@ impl<'a> Section<'a> {...@@ -759,6 +904,87 @@ impl<'a> Section<'a> {
759 }904 }
760}905}
761906
907/// Commits to `file` the objects `pending` names that `page` changed, with what the revision
908/// declares, replaces and creates; none when that changes nothing, unless `force`.
909fn changed<'a>(
910 arena: &'a Bump,
911 page: &LiveRevision<'a>,
912 pending: &BTreeSet<ExGuid>,
913 file: &mut LiveRevision<'a>,
914 force: bool,
915) -> Result<Option<Revised>> {
916 let mut replacements = BTreeMap::new();
917 for object_id in pending {
918 if !page.is_reachable(*object_id) {
919 continue;
920 }
921 let object = &page.revision.objects[object_id];
922 if let Some(stored) = file.revision.objects.get(object_id)
923 && unchanged(stored, object)?
924 {
925 continue;
926 }
927 replacements.insert(*object_id, replacement(*object_id, object)?);
928 }
929 let replacements = file.prepare(replacements, true)?;
930 if replacements.is_empty() && !force {
931 return Ok(None);
932 }
933 let replaced: BTreeSet<ExGuid> = replacements.keys().copied().collect();
934 let created = replaced
935 .iter()
936 .filter(|id| !file.revision.objects.contains_key(id))
937 .copied()
938 .collect();
939 let mut objects = Vec::new();
940 for (object_id, change) in replacements {
941 // The page holds these bytes already unless preparing remapped them.
942 let bytes = match page.revision.objects.get(&object_id).map(|o| o.data) {
943 Some(ObjectData::Properties(bytes)) if bytes == change.bytes => bytes,
944 _ => arena.alloc_slice_copy(&change.bytes),
945 };
946 objects.push((object_id, declared(change.jcid, bytes, change.global_ids)?));
947 }
948 Ok(Some(Revised {
949 commit: file.commit(objects)?,
950 replaced,
951 created,
952 }))
953}
954
955/// `resolved`, parsed from `bytes`, borrowing them for their lifetime.
956fn bound<'a>(bytes: &'a [u8], resolved: ResolvedRevision<'_>) -> ResolvedRevision<'a> {
957 ResolvedRevision {
958 roots: resolved.roots,
959 objects: resolved
960 .objects
961 .into_iter()
962 .map(|(id, object)| {
963 // The parse borrows `bytes` only as long as its store; its slices lie in
964 // `bytes`, which the arena keeps.
965 let data = match object.data {
966 ObjectData::Properties(data) => ObjectData::Properties(within(bytes, data)),
967 ObjectData::Encrypted(data) => ObjectData::Encrypted(within(bytes, data)),
968 ObjectData::File {
969 reference,
970 extension,
971 } => ObjectData::File {
972 reference: within(bytes, reference),
973 extension: within(bytes, extension),
974 },
975 };
976 let object = crate::Object {
977 jcid: object.jcid,
978 reference_count: object.reference_count,
979 data,
980 global_ids: object.global_ids,
981 };
982 (id, object)
983 })
984 .collect(),
985 }
986}
987
762/// `part`, a slice of `image`, with the image's lifetime.988/// `part`, a slice of `image`, with the image's lifetime.
763fn within<'a>(image: &'a [u8], part: &[u8]) -> &'a [u8] {989fn within<'a>(image: &'a [u8], part: &[u8]) -> &'a [u8] {
764 let start = part.as_ptr().addr() - image.as_ptr().addr();990 let start = part.as_ptr().addr() - image.as_ptr().addr();
crates/onestore/src/section/versions.rs created+492
...@@ -0,0 +1,492 @@
1//! Page versions, the earlier states of a page OneNote 2010 keeps (MS-ONE 2.1.17, 2.1.18).
2//! A version is an earlier revision of the page's own object space, current under a context
3//! of its own. The page's version history, the space's revision current under `HISTORY`,
4//! holds a proxy for each (`jcidVersionProxy`), newest first. `corpus/page-versions` records
5//! what OneNote 2010 writes.
6
7use super::{Frozen, Open, Section, SpaceState, changed, one_page};
8use crate::{
9 Error, ExGuid, FileType, Object, ObjectData, ResolvedRevision, RevisionIndex, Store,
10 active::{ActivePage, manifest_pages},
11 create::{author_properties, current_timestamps, properties},
12 document::{Element, Kind, Revision},
13 op::{Failure, OpError},
14 page::Page,
15 pages::set_references,
16 write::{LiveRevision, PropertyObject, chain_depth, compact, declared},
17};
18use std::{
19 collections::{BTreeMap, BTreeSet},
20 sync::Arc,
21};
22
23type Result<T> = std::result::Result<T, Error>;
24
25/// The context of a page's version history, {7111497F-1B6B-4209-9491-C98B04CF4C5A},1.
26pub(crate) const HISTORY: ExGuid = ExGuid {
27 guid: [
28 0x7f, 0x49, 0x11, 0x71, 0x6b, 0x1b, 0x09, 0x42, 0x94, 0x91, 0xc9, 0x8b, 0x04, 0xcf, 0x4c,
29 0x5a,
30 ],
31 n: 1,
32};
33
34/// A version's context is its revision's identity with these bits flipped, and its proxy's
35/// identity the context's with `PROXY` flipped, as OneNote 2010 derives both.
36const CONTEXT: [u8; 16] = [
37 0x7c, 0xfe, 0x52, 0x9d, 0xbc, 0x8f, 0x95, 0x44, 0x93, 0x00, 0x12, 0x9b, 0x57, 0x97, 0xa9, 0xea,
38];
39const PROXY: [u8; 16] = [
40 0xca, 0x0b, 0x44, 0x88, 0x8c, 0x11, 0x81, 0x0e, 0x32, 0xbf, 0x83, 0xdb, 0x8c, 0xb8, 0xb6, 0xb5,
41];
42
43fn flipped(mut id: ExGuid, salt: [u8; 16]) -> ExGuid {
44 for (byte, salt) in id.guid.iter_mut().zip(salt) {
45 *byte ^= salt;
46 }
47 id
48}
49
50/// An earlier state of a page, as the page list names it.
51#[derive(Debug, Clone, PartialEq, Eq)]
52pub struct PageVersion {
53 /// The context its revision is current under, which names it.
54 pub context: ExGuid,
55 /// FILETIME of its last change (`LastModifiedTimeStamp`).
56 pub modified: Option<u64>,
57 /// Who last changed it (`AuthorMostRecent`).
58 pub author: Option<String>,
59}
60
61fn element<'o>(object: &Object<'o>) -> Result<Element<'o>> {
62 Element::parse_with(object, FileType::Section, &mut |_| {
63 Err(Error {
64 offset: 0,
65 message: "Version metadata holds no payload",
66 })
67 })
68}
69
70/// When `revision`'s page was last changed and by whom, as its revision metadata says.
71fn last_change(revision: &ResolvedRevision<'_>) -> Result<(Option<u64>, Option<ExGuid>)> {
72 let Some(metadata) = revision
73 .roots
74 .get(&4)
75 .and_then(|id| revision.objects.get(id))
76 else {
77 return Ok((None, None));
78 };
79 let metadata = element(metadata)?;
80 let Kind::RevisionMetadata { modified_filetime } = metadata.kind else {
81 return Ok((None, None));
82 };
83 Ok((modified_filetime, metadata.latest_author))
84}
85
86impl<'a> Section<'a> {
87 /// The versions of each page that has them, in section order, each page's newest first as
88 /// OneNote 2010 lists them; read from the version histories alone.
89 pub fn versions(&mut self) -> Result<Vec<(ExGuid, Vec<PageVersion>)>> {
90 let mut listed = Vec::new();
91 for (page, ..) in self.pages()? {
92 let versions: Vec<PageVersion> = self
93 .listed(page)?
94 .into_iter()
95 .map(|(_, version)| version)
96 .collect();
97 if !versions.is_empty() {
98 listed.push((page, versions));
99 }
100 }
101 Ok(listed)
102 }
103
104 /// Page `space` as its version `context` holds it. Reads the sealed image: O(section).
105 pub fn version(&self, space: ExGuid, context: ExGuid) -> Result<Page> {
106 if !self
107 .listed(space)?
108 .iter()
109 .any(|(_, version)| version.context == context)
110 {
111 return Err(Error {
112 offset: 0,
113 message: "The page has no such version",
114 });
115 }
116 let stored = &self.spaces[&space];
117 if let Some(frozen) = stored
118 .frozen
119 .iter()
120 .find(|frozen| flipped(frozen.rid, CONTEXT) == context)
121 {
122 let files = self.files();
123 let pending = &self.payloads;
124 let (view, _) = Revision::parse(
125 space,
126 &frozen.live.revision,
127 FileType::Section,
128 &mut |guid| match pending.iter().find(|(id, _)| *id == guid) {
129 Some((_, payload)) => Ok(*payload),
130 None => files(guid),
131 },
132 )?;
133 return one_page(&view, &manifest_pages(&view));
134 }
135 let image = self.image();
136 let store = Store::parse(&image)?;
137 let index = RevisionIndex::parse(&store)?;
138 let rid = self.version_revision(&index, space, context)?;
139 let resolved = index.resolve(space, rid)?;
140 let (view, _) = Revision::parse(space, &resolved, FileType::Section, &mut |guid| {
141 store.file_data(guid)
142 })?;
143 one_page(&view, &manifest_pages(&view))
144 }
145
146 /// Page `space`'s versions, newest first, with their proxies.
147 fn listed(&self, space: ExGuid) -> Result<Vec<(ExGuid, PageVersion)>> {
148 let Some(history) = self
149 .spaces
150 .get(&space)
151 .and_then(|space| space.history.as_ref())
152 else {
153 return Ok(Vec::new());
154 };
155 let lookup = |id: &ExGuid| -> Result<Option<Object<'_>>> {
156 match history.pending.get(id) {
157 Some(object) => {
158 declared(object.jcid, &object.bytes, Arc::clone(&object.global_ids)).map(Some)
159 }
160 None => Ok(history.revision.objects.get(id).cloned()),
161 }
162 };
163 let Some(root) = history.revision.roots.get(&1) else {
164 return Ok(Vec::new());
165 };
166 let Some(root) = lookup(root)? else {
167 return Ok(Vec::new());
168 };
169 let mut versions = Vec::new();
170 for proxy in element(&root)?.children {
171 let Some(object) = lookup(&proxy)? else {
172 return Err(Error {
173 offset: 0,
174 message: "A version proxy is unavailable",
175 });
176 };
177 let node = element(&object)?;
178 let Kind::VersionProxy {
179 context,
180 modified_filetime,
181 } = node.kind
182 else {
183 return Err(Error {
184 offset: 0,
185 message: "A version history lists another kind of object",
186 });
187 };
188 let author = match node
189 .latest_author
190 .as_ref()
191 .map(lookup)
192 .transpose()?
193 .flatten()
194 {
195 Some(author) => match element(&author)?.kind {
196 Kind::Author { name } => name,
197 _ => None,
198 },
199 None => None,
200 };
201 versions.push((
202 proxy,
203 PageVersion {
204 context,
205 modified: modified_filetime,
206 author,
207 },
208 ));
209 }
210 Ok(versions)
211 }
212
213 /// The revision version `context` of `space` is, in `index` of the sealed image.
214 fn version_revision(
215 &self,
216 index: &RevisionIndex<'_>,
217 space: ExGuid,
218 context: ExGuid,
219 ) -> Result<ExGuid> {
220 let labelled = self.spaces[&space]
221 .labels
222 .iter()
223 .find(|(_, labelled)| *labelled == context)
224 .map(|(rid, _)| *rid);
225 labelled
226 .or_else(|| index.spaces.get(&space)?.labels.get(&(context, 1)).copied())
227 .ok_or(Error {
228 offset: 0,
229 message: "A version's revision is not stored",
230 })
231 }
232
233 /// Version `context` of `space` as a revision of the section's own bytes, with its
234 /// identity and chain depth. Reads the sealed image: O(section).
235 fn historic(
236 &self,
237 space: ExGuid,
238 context: ExGuid,
239 ) -> Result<(ExGuid, ResolvedRevision<'a>, usize)> {
240 if self.spaces[&space]
241 .frozen
242 .iter()
243 .any(|frozen| flipped(frozen.rid, CONTEXT) == context)
244 {
245 return Err(Error {
246 offset: 0,
247 message: "This version is still saving. Restore it in a moment.",
248 });
249 }
250 let image = self.image();
251 let store = Store::parse(&image)?;
252 let index = RevisionIndex::parse(&store)?;
253 let rid = self.version_revision(&index, space, context)?;
254 let resolved = index.resolve(space, rid)?;
255 // Object bytes lie in the section's segments, which outlive this image.
256 let resident = |part: &[u8]| -> Result<&'a [u8]> {
257 let offset = (part.as_ptr().addr() - image.as_ptr().addr()) as u64;
258 self.segments
259 .iter()
260 .rfind(|(start, _)| *start <= offset)
261 .and_then(|(start, segment)| {
262 let at = (offset - start) as usize;
263 segment.get(at..at + part.len())
264 })
265 .filter(|bytes| *bytes == part)
266 .ok_or(Error {
267 offset: offset as usize,
268 message: "A version's bytes lie outside the section",
269 })
270 };
271 let mut objects = BTreeMap::new();
272 for (id, object) in resolved.objects {
273 let data = match object.data {
274 ObjectData::Properties(data) => ObjectData::Properties(resident(data)?),
275 ObjectData::Encrypted(data) => ObjectData::Encrypted(resident(data)?),
276 ObjectData::File {
277 reference,
278 extension,
279 } => ObjectData::File {
280 reference: resident(reference)?,
281 extension: resident(extension)?,
282 },
283 };
284 objects.insert(
285 id,
286 Object {
287 jcid: object.jcid,
288 reference_count: object.reference_count,
289 data,
290 global_ids: object.global_ids,
291 },
292 );
293 }
294 let revision = ResolvedRevision {
295 roots: resolved.roots,
296 objects,
297 };
298 Ok((rid, revision, chain_depth(&index, space, rid)))
299 }
300
301 /// Restores version `context` of page `space` as OneNote 2010's Restore Version does: the
302 /// page as it stands becomes the newest version, and the page's next revision builds on
303 /// the version's, its revision metadata naming `author` now. New objects take identities
304 /// from `guid`.
305 pub(crate) fn restore(
306 &mut self,
307 author: &str,
308 space: ExGuid,
309 context: ExGuid,
310 guid: [u8; 16],
311 ) -> std::result::Result<(), Failure> {
312 if !self
313 .listed(space)?
314 .iter()
315 .any(|(_, version)| version.context == context)
316 {
317 return Err(OpError::TargetUnavailable(context).into());
318 }
319 let (from, revision, depth) = self.historic(space, context)?;
320 let (now, filetime) = current_timestamps()?;
321 let named = |n| ExGuid { guid, n };
322 self.remember(space);
323 let files = self.files();
324 let arena = &self.arena.0;
325 self.editable(space)?;
326 let stored = self.spaces.get_mut(&space).unwrap();
327 let SpaceState::Open(open) = &mut stored.state else {
328 unreachable!()
329 };
330 // The page as it stands: the stored revision, or its unsaved state as one of its own.
331 let (current, modified, who) = if open.pending.is_empty() && stored.restored.is_none() {
332 let rid = stored.rid.ok_or(Error {
333 offset: 0,
334 message: "A page never saved has no versions",
335 })?;
336 let (modified, who) = last_change(&open.file.revision)?;
337 let who = who
338 .and_then(|id| open.file.revision.objects.get(&id))
339 .cloned();
340 (rid, modified, who)
341 } else {
342 let mut live = open.file.clone();
343 let revised = changed(arena, &open.page.live, &open.pending, &mut live, true)?
344 .expect("a forced revision is written");
345 let rid = named(1);
346 stored.frozen.push(Frozen {
347 rid,
348 previous: stored.restored.or(stored.rid),
349 live,
350 revised,
351 });
352 (rid, None, None)
353 };
354 let label = flipped(current, CONTEXT);
355 stored.labels.push((current, label));
356 // Its proxy, newest in the history, names when and by whom it was last changed.
357 let proxy = flipped(label, PROXY);
358 let proxy_author = named(3);
359 let author_object = match &who {
360 Some(object) => PropertyObject::from_object(object)?,
361 None => PropertyObject {
362 jcid: 0x120001,
363 bytes: properties(&author_properties(author))?,
364 global_ids: Arc::new(BTreeMap::new()),
365 },
366 };
367 let author_object = PropertyObject {
368 global_ids: Arc::new(BTreeMap::from([(0, proxy_author.guid)])),
369 ..author_object
370 };
371 let table = BTreeMap::from([(0, proxy.guid), (1, proxy_author.guid), (2, label.guid)]);
372 let proxy_object = PropertyObject {
373 jcid: 0x6003d,
374 bytes: properties(&[
375 (0x14001d09, now.to_le_bytes().to_vec()),
376 (
377 0x18001d77,
378 modified.unwrap_or(filetime).to_le_bytes().to_vec(),
379 ),
380 (0x20001d79, compact(proxy_author, &table)?.to_vec()),
381 (0x3400347b, compact(label, &table)?.to_vec()),
382 ])?,
383 global_ids: Arc::new(table),
384 };
385 let listed: Vec<ExGuid> = self
386 .listed(space)?
387 .into_iter()
388 .map(|(proxy, _)| proxy)
389 .collect();
390 let stored = self.spaces.get_mut(&space).unwrap();
391 let history = stored.history.as_mut().ok_or(Error {
392 offset: 0,
393 message: "The page has no version history",
394 })?;
395 let root = *history.revision.roots.get(&1).ok_or(Error {
396 offset: 0,
397 message: "The version history has no content",
398 })?;
399 let mut root_object = match history.pending.get(&root) {
400 Some(object) => object.clone(),
401 None => PropertyObject::from_object(&history.revision.objects[&root])?,
402 };
403 let children: Vec<ExGuid> = std::iter::once(proxy).chain(listed).collect();
404 set_references(&mut root_object, 0x24001c20, &children)?;
405 history.pending.extend([
406 (root, root_object),
407 (proxy, proxy_object),
408 (proxy_author, author_object),
409 ]);
410 // The page's next revision builds on the version's.
411 let file = LiveRevision::new(revision, depth)?;
412 let page = ActivePage::open(space, file.clone(), FileType::Section, files)?;
413 stored.state = SpaceState::Open(Box::new(Open {
414 file,
415 page,
416 pending: BTreeSet::new(),
417 }));
418 stored.restored = Some(from);
419 // Its revision metadata names who restored it, and when.
420 let active = self.active(space)?;
421 let revision = &active.live.revision;
422 let mut changes = BTreeMap::new();
423 if let Some(id) = revision.roots.get(&4)
424 && revision.objects[id].jcid == 0x20044
425 {
426 let restorer = named(2);
427 let mut metadata = PropertyObject::from_object(&revision.objects[id])?;
428 let reference = metadata.reference(restorer)?;
429 metadata.set(&[
430 (0x18001d77, &filetime.to_le_bytes()),
431 (0x20001d79, &reference),
432 ])?;
433 changes.insert(*id, metadata);
434 changes.insert(
435 restorer,
436 PropertyObject {
437 jcid: 0x120001,
438 bytes: properties(&author_properties(author))?,
439 global_ids: Arc::new(BTreeMap::from([(0, restorer.guid)])),
440 },
441 );
442 }
443 self.apply_changes(space, &[], changes)?;
444 Ok(())
445 }
446
447 /// Deletes versions `contexts` of page `space` as OneNote 2010's Delete Version does: its
448 /// history stops listing them; their revisions stay stored. True when none is left.
449 pub(crate) fn unlist(
450 &mut self,
451 space: ExGuid,
452 contexts: &[ExGuid],
453 ) -> std::result::Result<bool, Failure> {
454 let listed = self.listed(space)?;
455 if let Some(context) = contexts.iter().find(|context| {
456 !listed
457 .iter()
458 .any(|(_, version)| version.context == **context)
459 }) {
460 return Err(OpError::TargetUnavailable(*context).into());
461 }
462 let kept: Vec<ExGuid> = listed
463 .iter()
464 .filter(|(_, version)| !contexts.contains(&version.context))
465 .map(|(proxy, _)| *proxy)
466 .collect();
467 self.remember(space);
468 let history = self
469 .spaces
470 .get_mut(&space)
471 .and_then(|space| space.history.as_mut())
472 .ok_or(Error {
473 offset: 0,
474 message: "The page has no version history",
475 })?;
476 let root = *history.revision.roots.get(&1).ok_or(Error {
477 offset: 0,
478 message: "The version history has no content",
479 })?;
480 let mut object = match history.pending.get(&root) {
481 Some(object) => object.clone(),
482 None => PropertyObject::from_object(&history.revision.objects[&root])?,
483 };
484 if kept.is_empty() {
485 object.remove(&[0x24001c20])?;
486 } else {
487 set_references(&mut object, 0x24001c20, &kept)?;
488 }
489 history.pending.insert(root, object);
490 Ok(kept.is_empty())
491 }
492}
crates/onestore/src/toc.rs+131-5
...@@ -64,19 +64,27 @@ pub fn edit_table_of_contents(...@@ -64,19 +64,27 @@ pub fn edit_table_of_contents(
64 let Some(Kind::Toc { entries, .. }) = revision.nodes.get(&root).map(|node| &node.kind) else {64 let Some(Kind::Toc { entries, .. }) = revision.nodes.get(&root).map(|node| &node.kind) else {
65 return Err(invalid("Missing notebook TOC root"));65 return Err(invalid("Missing notebook TOC root"));
66 };66 };
67 // Entries in stored order with their identities and filenames.67 // Entries in display order (their ordering numbers, not their place in the root's list)
68 let mut listed: Vec<(ExGuid, [u8; 16], String)> = Vec::new();68 // with their identities and filenames.
69 let mut stored = Vec::new();
69 for id in entries {70 for id in entries {
70 let Some(Kind::Toc {71 let Some(Kind::Toc {
71 filename,72 filename,
72 identity: Some(identity),73 identity: Some(identity),
74 order,
73 ..75 ..
74 }) = revision.nodes.get(id).map(|node| &node.kind)76 }) = revision.nodes.get(id).map(|node| &node.kind)
75 else {77 else {
76 return Err(invalid("Incomplete notebook TOC reference"));78 return Err(invalid("Incomplete notebook TOC reference"));
77 };79 };
78 listed.push((*id, *identity, filename.clone().unwrap_or_default()));80 stored.push((
81 order.unwrap_or(u32::MAX),
82 (*id, *identity, filename.clone().unwrap_or_default()),
83 ));
79 }84 }
85 stored.sort_by_key(|(order, _)| *order);
86 let mut listed: Vec<(ExGuid, [u8; 16], String)> =
87 stored.into_iter().map(|(_, entry)| entry).collect();
80 let mut created: BTreeMap<ExGuid, PropertyObject> = BTreeMap::new();88 let mut created: BTreeMap<ExGuid, PropertyObject> = BTreeMap::new();
81 let mut renamed = BTreeSet::new();89 let mut renamed = BTreeSet::new();
82 let mut color = None;90 let mut color = None;
...@@ -173,6 +181,13 @@ pub fn edit_table_of_contents(...@@ -173,6 +181,13 @@ pub fn edit_table_of_contents(
173 }181 }
174 }182 }
175 let listed = listed;183 let listed = listed;
184 // A rename or a colour keeps every number, gaps and ties included, as OneNote left them.
185 let resequence = edits.iter().any(|edit| {
186 matches!(
187 edit,
188 TocEdit::Add { .. } | TocEdit::Order(_) | TocEdit::Remove { .. }
189 )
190 });
176 let transaction = build_on(&index, &[], None, |index| {191 let transaction = build_on(&index, &[], None, |index| {
177 let raw = index.resolve_active(space)?;192 let raw = index.resolve_active(space)?;
178 let mut changed = BTreeMap::new();193 let mut changed = BTreeMap::new();
...@@ -190,9 +205,10 @@ pub fn edit_table_of_contents(...@@ -190,9 +205,10 @@ pub fn edit_table_of_contents(
190 Value::Bytes(b) => b.try_into().ok().map(u32::from_le_bytes),205 Value::Bytes(b) => b.try_into().ok().map(u32::from_le_bytes),
191 _ => None,206 _ => None,
192 });207 });
208 let order = order as u32 + 1;
193 let mut updates: Vec<(u32, Vec<u8>)> = Vec::new();209 let mut updates: Vec<(u32, Vec<u8>)> = Vec::new();
194 if stored_order != Some(order as u32 + 1) {210 if resequence && stored_order != Some(order) {
195 updates.push((0x14001cb9, (order as u32 + 1).to_le_bytes().to_vec()));211 updates.push((0x14001cb9, order.to_le_bytes().to_vec()));
196 }212 }
197 if renamed.contains(id) {213 if renamed.contains(id) {
198 updates.push((0x1c001d6b, crate::create::string(filename)));214 updates.push((0x1c001d6b, crate::create::string(filename)));
...@@ -219,3 +235,113 @@ pub fn edit_table_of_contents(...@@ -219,3 +235,113 @@ pub fn edit_table_of_contents(
219 check(&applied(source, transaction.as_ref())?, true)?;235 check(&applied(source, transaction.as_ref())?, true)?;
220 Ok(transaction)236 Ok(transaction)
221}237}
238
239#[cfg(test)]
240mod tests {
241 use super::*;
242
243 fn apply(source: &[u8], transaction: Option<Transaction>) -> Vec<u8> {
244 applied(source, transaction.as_ref()).unwrap()
245 }
246
247 /// Each entry's filename and ordering number, as the root lists them.
248 fn entries(image: &[u8]) -> Vec<(ExGuid, String, u32)> {
249 let store = Store::parse(image).unwrap();
250 let index = RevisionIndex::parse(&store).unwrap();
251 let document = Document::parse(&index).unwrap();
252 let revision = document.active(document.root).unwrap();
253 let Kind::Toc { entries, .. } = &revision.nodes[&revision.roots[&1]].kind else {
254 panic!()
255 };
256 entries
257 .iter()
258 .map(|id| match &revision.nodes[id].kind {
259 Kind::Toc {
260 filename, order, ..
261 } => (*id, filename.clone().unwrap(), order.unwrap()),
262 _ => panic!(),
263 })
264 .collect()
265 }
266
267 fn displayed(image: &[u8]) -> Vec<(String, u32)> {
268 let mut entries: Vec<_> = entries(image)
269 .into_iter()
270 .map(|(_, name, order)| (name, order))
271 .collect();
272 entries.sort_by_key(|(_, order)| *order);
273 entries
274 }
275
276 #[test]
277 fn edits_follow_the_ordering_numbers_not_the_list() {
278 let [a, b, c] = [[1; 16], [2; 16], [3; 16]];
279 let toc = crate::create_table_of_contents(
280 "Open Notebook.onetoc2",
281 &[("Video.one", a), ("Song.one", b), ("Album.one", c)],
282 )
283 .unwrap();
284 // Numbers out of the list's order, with a gap and a tie, as native TOCs carry them.
285 let dragged = {
286 let store = Store::parse(&toc).unwrap();
287 let index = RevisionIndex::parse(&store).unwrap();
288 let space = Document::parse(&index).unwrap().root;
289 let listed = entries(&toc);
290 let transaction = build_on(&index, &[], None, |index| {
291 let raw = index.resolve_active(space)?;
292 let mut changed = BTreeMap::new();
293 for ((id, _, _), order) in listed.iter().zip([7u32, 2, 2]) {
294 let mut object = PropertyObject::from_object(&raw.objects[id])?;
295 object.set(&[(0x14001cb9, &order.to_le_bytes())])?;
296 changed.insert(*id, object);
297 }
298 Ok(BTreeMap::from([(space, RevisionEdit::Update(changed))]))
299 })
300 .unwrap();
301 apply(&toc, transaction)
302 };
303 let renamed = apply(
304 &dragged,
305 edit_table_of_contents(
306 &dragged,
307 &[TocEdit::Rename {
308 identity: a,
309 filename: "Kitchen.one".into(),
310 }],
311 )
312 .unwrap(),
313 );
314 assert_eq!(
315 displayed(&renamed),
316 [
317 ("Song.one".into(), 2),
318 ("Album.one".into(), 2),
319 ("Kitchen.one".into(), 7)
320 ]
321 );
322 let added = apply(
323 &renamed,
324 edit_table_of_contents(
325 &renamed,
326 &[TocEdit::Add {
327 filename: "Lore.one".into(),
328 identity: [4; 16],
329 group: false,
330 }],
331 )
332 .unwrap(),
333 );
334 assert_eq!(
335 entries(&added)
336 .into_iter()
337 .map(|(_, name, order)| (name, order))
338 .collect::<Vec<_>>(),
339 [
340 ("Song.one".into(), 1),
341 ("Album.one".into(), 2),
342 ("Kitchen.one".into(), 3),
343 ("Lore.one".into(), 4)
344 ]
345 );
346 }
347}
crates/onestore/src/tree.rs+2-3
...@@ -458,9 +458,8 @@ mod tests {...@@ -458,9 +458,8 @@ mod tests {
458 .unwrap();458 .unwrap();
459 let paragraph = view.nodes[&outline].children[0];459 let paragraph = view.nodes[&outline].children[0];
460 let text = view.nodes[&paragraph].content[0];460 let text = view.nodes[&paragraph].content[0];
461 let deleted = |bytes: &[u8]| {461 let deleted =
462 edited(bytes, sid, vec![PageOp::Delete { object: paragraph }]).is_ok()462 |bytes: &[u8]| edited(bytes, sid, vec![PageOp::Delete { object: paragraph }]).is_ok();
463 };
464 for target in [page, outline, paragraph, text] {463 for target in [page, outline, paragraph, text] {
465 for property in [0x08001cde, 0x08001cb4, 0x08001cf9, 0x08001cb2] {464 for property in [0x08001cde, 0x08001cb4, 0x08001cf9, 0x08001cb2] {
466 for enabled in [false, true] {465 for enabled in [false, true] {
crates/onestore/src/write.rs+238-164
...@@ -76,6 +76,16 @@ pub(crate) fn node(id: u16, reference: Option<Reference>, payload: &[u8]) -> Res...@@ -76,6 +76,16 @@ pub(crate) fn node(id: u16, reference: Option<Reference>, payload: &[u8]) -> Res
76 Ok(bytes)76 Ok(bytes)
77}77}
7878
79/// A `RevisionRoleAndContextDeclarationFND` payload: `rid` current under `context` in the
80/// default role.
81pub(crate) fn label_payload(rid: ExGuid, context: ExGuid) -> Vec<u8> {
82 let mut payload = Vec::new();
83 rid.encode(&mut payload);
84 payload.extend_from_slice(&1_u32.to_le_bytes());
85 context.encode(&mut payload);
86 payload
87}
88
79pub(crate) fn append(data: &mut Vec<u8>, bytes: &[u8]) -> Result<Chunk> {89pub(crate) fn append(data: &mut Vec<u8>, bytes: &[u8]) -> Result<Chunk> {
80 let length = u64::from(u32::try_from(bytes.len()).map_err(|_| Error {90 let length = u64::from(u32::try_from(bytes.len()).map_err(|_| Error {
81 offset: 0,91 offset: 0,
...@@ -1213,6 +1223,7 @@ pub(crate) fn differing<'s, T: PartialEq>(before: &'s [T], after: &'s [T]) -> (&...@@ -1213,6 +1223,7 @@ pub(crate) fn differing<'s, T: PartialEq>(before: &'s [T], after: &'s [T]) -> (&
1213}1223}
12141224
1215/// A revision `LiveRevision::commit` stored.1225/// A revision `LiveRevision::commit` stored.
1226#[derive(Clone)]
1216pub(crate) struct Commit {1227pub(crate) struct Commit {
1217 /// Whether the revision declares every object, depending on none.1228 /// Whether the revision declares every object, depending on none.
1218 pub checkpoint: bool,1229 pub checkpoint: bool,
...@@ -1329,7 +1340,10 @@ fn build(...@@ -1329,7 +1340,10 @@ fn build(
1329 if validate {1340 if validate {
1330 index.validate_current()?;1341 index.validate_current()?;
1331 }1342 }
1332 applied(source, build_on(&index, payloads, protection, edit)?.as_ref())1343 applied(
1344 source,
1345 build_on(&index, payloads, protection, edit)?.as_ref(),
1346 )
1333}1347}
13341348
1335/// `write_revision` on a source the caller has parsed and validated.1349/// `write_revision` on a source the caller has parsed and validated.
...@@ -1484,6 +1498,7 @@ pub(crate) fn build_on(...@@ -1484,6 +1498,7 @@ pub(crate) fn build_on(
1484 previous: rid,1498 previous: rid,
1485 new_space,1499 new_space,
1486 label,1500 label,
1501 rid: None,
1487 live: &live,1502 live: &live,
1488 commit: &commit,1503 commit: &commit,
1489 replaced: &replaced,1504 replaced: &replaced,
...@@ -1578,6 +1593,8 @@ pub(crate) struct Sealing<'r, 'a> {...@@ -1578,6 +1593,8 @@ pub(crate) struct Sealing<'r, 'a> {
1578 pub new_space: bool,1593 pub new_space: bool,
1579 /// The context and role it is current under.1594 /// The context and role it is current under.
1580 pub label: (ExGuid, u32),1595 pub label: (ExGuid, u32),
1596 /// Its identity, where the caller chose one; a fresh one otherwise.
1597 pub rid: Option<ExGuid>,
1581 pub live: &'r LiveRevision<'a>,1598 pub live: &'r LiveRevision<'a>,
1582 pub commit: &'r Commit,1599 pub commit: &'r Commit,
1583 /// Objects whose bytes the revision stores rather than referencing stored bytes.1600 /// Objects whose bytes the revision stores rather than referencing stored bytes.
...@@ -1722,6 +1739,23 @@ impl Appending {...@@ -1722,6 +1739,23 @@ impl Appending {
1722 segments: &[(u64, &[u8])],1739 segments: &[(u64, &[u8])],
1723 protection: Option<&dyn Protection>,1740 protection: Option<&dyn Protection>,
1724 key: Option<&crate::Node<'_>>,1741 key: Option<&crate::Node<'_>>,
1742 ) -> Result<(ExGuid, Vec<(ExGuid, Chunk)>)> {
1743 let mut manifest = Vec::new();
1744 let written = self.manifest(sealing, segments, protection, key, &mut manifest)?;
1745 self.close(sealing.space, sealing.new_space, &manifest)?;
1746 Ok(written)
1747 }
1748
1749 /// Appends a revision's object groups and new data, and its manifest's nodes to
1750 /// `manifest` for `close`; returns its identity and where it stored each replaced
1751 /// object's bytes.
1752 pub(crate) fn manifest(
1753 &mut self,
1754 sealing: &Sealing<'_, '_>,
1755 segments: &[(u64, &[u8])],
1756 protection: Option<&dyn Protection>,
1757 key: Option<&crate::Node<'_>>,
1758 manifest: &mut Vec<Vec<u8>>,
1725 ) -> Result<(ExGuid, Vec<(ExGuid, Chunk)>)> {1759 ) -> Result<(ExGuid, Vec<(ExGuid, Chunk)>)> {
1726 let is_section = self.state.file_type == FileType::Section;1760 let is_section = self.state.file_type == FileType::Section;
1727 let Sealing {1761 let Sealing {
...@@ -1782,9 +1816,12 @@ impl Appending {...@@ -1782,9 +1816,12 @@ impl Appending {
1782 .or_default()1816 .or_default()
1783 .push((*id, object));1817 .push((*id, object));
1784 }1818 }
1785 let new_rid = ExGuid {1819 let new_rid = match sealing.rid {
1786 guid: fresh_guid()?,1820 Some(rid) => rid,
1787 n: 1,1821 None => ExGuid {
1822 guid: fresh_guid()?,
1823 n: 1,
1824 },
1788 };1825 };
1789 let mut start = Vec::new();1826 let mut start = Vec::new();
1790 new_rid.encode(&mut start);1827 new_rid.encode(&mut start);
...@@ -1806,7 +1843,6 @@ impl Appending {...@@ -1806,7 +1843,6 @@ impl Appending {
1806 if contextual {1843 if contextual {
1807 label.0.encode(&mut start);1844 label.0.encode(&mut start);
1808 }1845 }
1809 let mut manifest = Vec::new();
1810 if sealing.new_space {1846 if sealing.new_space {
1811 let mut payload = Vec::new();1847 let mut payload = Vec::new();
1812 space.encode(&mut payload);1848 space.encode(&mut payload);
...@@ -1997,8 +2033,13 @@ impl Appending {...@@ -1997,8 +2033,13 @@ impl Appending {
1997 }2033 }
1998 }2034 }
1999 manifest.push(node(0x1c, None, &[])?);2035 manifest.push(node(0x1c, None, &[])?);
2000 if sealing.new_space {2036 Ok((new_rid, stored))
2001 let (chunk, tail) = self.list(&manifest)?;2037 }
2038
2039 /// Adds a space's manifest nodes to its revision list, creating the space when `new`.
2040 pub(crate) fn close(&mut self, space: ExGuid, new: bool, manifest: &[Vec<u8>]) -> Result<()> {
2041 if new {
2042 let (chunk, tail) = self.list(manifest)?;
2002 let mut payload = Vec::new();2043 let mut payload = Vec::new();
2003 space.encode(&mut payload);2044 space.encode(&mut payload);
2004 let (chunk, _) = self.list(&[2045 let (chunk, _) = self.list(&[
...@@ -2013,10 +2054,10 @@ impl Appending {...@@ -2013,10 +2054,10 @@ impl Appending {
2013 offset: 0,2054 offset: 0,
2014 message: "Object space is absent from the root list",2055 message: "Object space is absent from the root list",
2015 })?;2056 })?;
2016 self.extend(&mut tail, &manifest)?;2057 self.extend(&mut tail, manifest)?;
2017 self.state.spaces.insert(space, tail);2058 self.state.spaces.insert(space, tail);
2018 }2059 }
2019 Ok((new_rid, stored))2060 Ok(())
2020 }2061 }
20212062
2022 /// Embeds payloads as file-data store objects referenced from the root file node list2063 /// Embeds payloads as file-data store objects referenced from the root file node list
...@@ -2221,12 +2262,14 @@ pub(crate) struct Written<'r, 'a> {...@@ -2221,12 +2262,14 @@ pub(crate) struct Written<'r, 'a> {
2221 pub space: ExGuid,2262 pub space: ExGuid,
2222 pub rid: ExGuid,2263 pub rid: ExGuid,
2223 pub previous: Option<ExGuid>,2264 pub previous: Option<ExGuid>,
2265 pub label: (ExGuid, u32),
2224 pub live: &'r LiveRevision<'a>,2266 pub live: &'r LiveRevision<'a>,
2225 pub commit: &'r Commit,2267 pub commit: &'r Commit,
2226}2268}
22272269
2228/// Reads back what `transaction` appends to the section `before` describes, whose bytes2270/// Reads back what `transaction` appends to the section `before` describes, whose bytes
2229/// `segments` hold, and checks it stores `revisions` and `payloads` as `after` expects:2271/// `segments` hold, and checks it stores `revisions`, then each space's `labels` (space,
2272/// revision, context), and `payloads` as `after` expects:
2230/// fragments link and decode to the logged counts under the logged checksum; each declared2273/// fragments link and decode to the logged counts under the logged checksum; each declared
2231/// object parses, names identities its group's table holds and objects reachable after its2274/// object parses, names identities its group's table holds and objects reachable after its
2232/// revision, carries its incremental reference count and, if read-only, its MD5; roots stay2275/// revision, carries its incremental reference count and, if read-only, its MD5; roots stay
...@@ -2237,6 +2280,7 @@ pub(crate) fn check_transaction(...@@ -2237,6 +2280,7 @@ pub(crate) fn check_transaction(
2237 transaction: &crate::Transaction,2280 transaction: &crate::Transaction,
2238 segments: &[(u64, &[u8])],2281 segments: &[(u64, &[u8])],
2239 revisions: &[Written<'_, '_>],2282 revisions: &[Written<'_, '_>],
2283 labels: &[(ExGuid, ExGuid, ExGuid)],
2240 payloads: &[([u8; 16], &[u8])],2284 payloads: &[([u8; 16], &[u8])],
2241) -> Result<()> {2285) -> Result<()> {
2242 let wrong = |message| Error { offset: 0, message };2286 let wrong = |message| Error { offset: 0, message };
...@@ -2332,14 +2376,18 @@ pub(crate) fn check_transaction(...@@ -2332,14 +2376,18 @@ pub(crate) fn check_transaction(
2332 }2376 }
2333 read(after.last)2377 read(after.last)
2334 };2378 };
2335 for written in revisions {2379 // Each space's list gains one fragment: its revisions in order, then its labels.
2336 let Written {2380 let mut spaces: Vec<ExGuid> = Vec::new();
2337 space,2381 for space in revisions
2338 rid,2382 .iter()
2339 live,2383 .map(|written| written.space)
2340 commit,2384 .chain(labels.iter().map(|(space, ..)| *space))
2341 ..2385 {
2342 } = *written;2386 if !spaces.contains(&space) {
2387 spaces.push(space);
2388 }
2389 }
2390 for space in spaces {
2343 let old = before.spaces.get(&space);2391 let old = before.spaces.get(&space);
2344 let new = after2392 let new = after
2345 .spaces2393 .spaces
...@@ -2359,166 +2407,190 @@ pub(crate) fn check_transaction(...@@ -2359,166 +2407,190 @@ pub(crate) fn check_transaction(
2359 .next()2407 .next()
2360 .ok_or(wrong("A revision manifest is truncated"))2408 .ok_or(wrong("A revision manifest is truncated"))
2361 };2409 };
2362 let mut node = next()?;2410 let mut fresh = old.is_none();
2363 if old.is_none() {2411 for written in revisions.iter().filter(|written| written.space == space) {
2412 let Written {
2413 rid,
2414 live,
2415 commit,
2416 label,
2417 ..
2418 } = *written;
2419 let mut node = next()?;
2420 if fresh {
2421 let mut c = Cursor {
2422 bytes: node.payload,
2423 offset: node.offset,
2424 };
2425 if node.id != 0x14 || c.exguid()? != space {
2426 return Err(wrong("A new space's revision list names another space"));
2427 }
2428 node = next()?;
2429 fresh = false;
2430 }
2364 let mut c = Cursor {2431 let mut c = Cursor {
2365 bytes: node.payload,2432 bytes: node.payload,
2366 offset: node.offset,2433 offset: node.offset,
2367 };2434 };
2368 if node.id != 0x14 || c.exguid()? != space {2435 let dependency = if commit.checkpoint {
2369 return Err(wrong("A new space's revision list names another space"));2436 ExGuid::default()
2437 } else {
2438 written
2439 .previous
2440 .ok_or(wrong("A dependent revision follows none"))?
2441 };
2442 let contextual = label.0 != ExGuid::default();
2443 if node.id != if contextual { 0x1f } else { 0x1e }
2444 || c.exguid()? != rid
2445 || c.exguid()? != dependency
2446 || c.read::<4>()? != label.1.to_le_bytes()
2447 || c.read::<2>()? != [0, 0]
2448 || (contextual && c.exguid()? != label.0)
2449 {
2450 return Err(wrong(
2451 "A revision starts with the wrong identity, dependency or label",
2452 ));
2370 }2453 }
2371 node = next()?;2454 let mut declared = BTreeSet::new();
2372 }2455 let mut roots = BTreeMap::new();
2373 let mut c = Cursor {2456 loop {
2374 bytes: node.payload,2457 let node = next()?;
2375 offset: node.offset,2458 match node.id {
2376 };2459 0xb0 => {
2377 let dependency = if commit.checkpoint {2460 let Some(Reference::NodeList(at)) = node.reference else {
2378 ExGuid::default()2461 return Err(wrong("An object group lacks its list"));
2379 } else {
2380 written
2381 .previous
2382 .ok_or(wrong("A dependent revision follows none"))?
2383 };
2384 if node.id != 0x1e
2385 || c.exguid()? != rid
2386 || c.exguid()? != dependency
2387 || c.read::<4>()? != 1_u32.to_le_bytes()
2388 || c.read::<2>()? != [0, 0]
2389 {
2390 return Err(wrong(
2391 "A revision starts with the wrong identity or dependency",
2392 ));
2393 }
2394 let mut declared = BTreeSet::new();
2395 let mut roots = BTreeMap::new();
2396 loop {
2397 let node = next()?;
2398 match node.id {
2399 0xb0 => {
2400 let Some(Reference::NodeList(at)) = node.reference else {
2401 return Err(wrong("An object group lacks its list"));
2402 };
2403 let bytes = read(at)?;
2404 let id = Fragment::parse(&bytes, at.offset as usize)?.id;
2405 let required = *counts
2406 .get(&id)
2407 .ok_or(wrong("An object group is not logged"))?;
2408 let group = nodes(&bytes, at, id, 0, required)?;
2409 let mut table = BTreeMap::new();
2410 let mut override_crc = u32::MAX;
2411 for item in &group {
2412 let mut c = Cursor {
2413 bytes: item.payload,
2414 offset: item.offset,
2415 };2462 };
2416 match item.id {2463 let bytes = read(at)?;
2417 0xb4 | 0x22 | 0x28 | 0xb8 => {}2464 let id = Fragment::parse(&bytes, at.offset as usize)?.id;
2418 0x24 => {2465 let required = *counts
2419 table.insert(u32::from_le_bytes(c.read()?), c.read::<16>()?);2466 .get(&id)
2420 }2467 .ok_or(wrong("An object group is not logged"))?;
2421 0xa5 | 0xc5 | 0x73 => {2468 let group = nodes(&bytes, at, id, 0, required)?;
2422 let id = c.compact(&table)?;2469 let mut table = BTreeMap::new();
2423 let object = live2470 let mut override_crc = u32::MAX;
2424 .revision2471 for item in &group {
2425 .objects2472 let mut c = Cursor {
2426 .get(&id)2473 bytes: item.payload,
2427 .ok_or(wrong("A revision declares an unknown object"))?;2474 offset: item.offset,
2428 let jcid = u32::from_le_bytes(c.read()?);2475 };
2429 let flags = if item.id == 0x73 {2476 match item.id {
2430 None2477 0xb4 | 0x22 | 0x28 | 0xb8 => {}
2431 } else {2478 0x24 => {
2432 Some(c.read::<1>()?[0])2479 table.insert(u32::from_le_bytes(c.read()?), c.read::<16>()?);
2433 };
2434 let count = u32::from_le_bytes(c.read()?);
2435 if !declared.insert(id)
2436 || jcid != object.jcid
2437 || count != object.reference_count
2438 {
2439 return Err(wrong(
2440 "A declaration disagrees with its object or its count",
2441 ));
2442 }
2443 override_crc =
2444 crc(override_crc, &count.to_le_bytes(), FileType::Section);
2445 if item.id == 0x73 {
2446 continue;
2447 }
2448 let Some(Reference::Data(at)) = item.reference else {
2449 return Err(wrong("An object declaration lacks its data"));
2450 };
2451 let data = read(at)?;
2452 let stored = crate::Object {
2453 jcid,
2454 reference_count: count,
2455 data: ObjectData::Properties(&data),
2456 global_ids: Arc::new(table.clone()),
2457 };
2458 let references = stored.references()?;
2459 let expected = u8::from(!references.objects.is_empty())
2460 | (u8::from(
2461 !references.object_spaces.is_empty()
2462 || !references.contexts.is_empty(),
2463 ) << 1);
2464 if object.data != ObjectData::Properties(&data)
2465 || flags != Some(expected)
2466 || (item.id == 0xc5) != (jcid & 0x100000 != 0)
2467 || (item.id == 0xc5 && c.read::<16>()? != md5::compute(&data).0)
2468 {
2469 return Err(wrong(
2470 "An object's stored bytes differ from its revision",
2471 ));
2472 }2480 }
2473 if live.is_reachable(id)2481 0xa5 | 0xc5 | 0x73 => {
2474 && !references2482 let id = c.compact(&table)?;
2475 .objects2483 let object =
2476 .iter()2484 live.revision.objects.get(&id).ok_or(wrong(
2477 .all(|target| live.is_reachable(*target))2485 "A revision declares an unknown object",
2478 {2486 ))?;
2479 return Err(wrong(2487 let jcid = u32::from_le_bytes(c.read()?);
2480 "An object references one its revision cannot reach",2488 let flags = if item.id == 0x73 {
2481 ));2489 None
2490 } else {
2491 Some(c.read::<1>()?[0])
2492 };
2493 let count = u32::from_le_bytes(c.read()?);
2494 if !declared.insert(id)
2495 || jcid != object.jcid
2496 || count != object.reference_count
2497 {
2498 return Err(wrong(
2499 "A declaration disagrees with its object or its count",
2500 ));
2501 }
2502 override_crc =
2503 crc(override_crc, &count.to_le_bytes(), FileType::Section);
2504 if item.id == 0x73 {
2505 continue;
2506 }
2507 let Some(Reference::Data(at)) = item.reference else {
2508 return Err(wrong("An object declaration lacks its data"));
2509 };
2510 let data = read(at)?;
2511 let stored = crate::Object {
2512 jcid,
2513 reference_count: count,
2514 data: ObjectData::Properties(&data),
2515 global_ids: Arc::new(table.clone()),
2516 };
2517 let references = stored.references()?;
2518 let expected = u8::from(!references.objects.is_empty())
2519 | (u8::from(
2520 !references.object_spaces.is_empty()
2521 || !references.contexts.is_empty(),
2522 ) << 1);
2523 if object.data != ObjectData::Properties(&data)
2524 || flags != Some(expected)
2525 || (item.id == 0xc5) != (jcid & 0x100000 != 0)
2526 || (item.id == 0xc5
2527 && c.read::<16>()? != md5::compute(&data).0)
2528 {
2529 return Err(wrong(
2530 "An object's stored bytes differ from its revision",
2531 ));
2532 }
2533 if live.is_reachable(id)
2534 && !references
2535 .objects
2536 .iter()
2537 .all(|target| live.is_reachable(*target))
2538 {
2539 return Err(wrong(
2540 "An object references one its revision cannot reach",
2541 ));
2542 }
2482 }2543 }
2544 _ => return Err(wrong("Unexpected node in an object group")),
2483 }2545 }
2484 _ => return Err(wrong("Unexpected node in an object group")),2546 }
2547 let node = next()?;
2548 let mut c = Cursor {
2549 bytes: node.payload,
2550 offset: node.offset,
2551 };
2552 if node.id != 0x84
2553 || c.read::<8>()? != [0; 8]
2554 || u32::from_le_bytes(c.read()?) != !override_crc
2555 {
2556 return Err(wrong("An object group's count overrides do not match"));
2485 }2557 }
2486 }2558 }
2487 let node = next()?;2559 0x5a => {
2488 let mut c = Cursor {2560 let mut c = Cursor {
2489 bytes: node.payload,2561 bytes: node.payload,
2490 offset: node.offset,2562 offset: node.offset,
2491 };2563 };
2492 if node.id != 0x842564 let id = c.exguid()?;
2493 || c.read::<8>()? != [0; 8]2565 roots.insert(u32::from_le_bytes(c.read()?), id);
2494 || u32::from_le_bytes(c.read()?) != !override_crc
2495 {
2496 return Err(wrong("An object group's count overrides do not match"));
2497 }2566 }
2567 0x1c => break,
2568 _ => return Err(wrong("Unexpected node in a revision manifest")),
2498 }2569 }
2499 0x5a => {2570 }
2500 let mut c = Cursor {2571 let expected: BTreeSet<ExGuid> = if commit.checkpoint {
2501 bytes: node.payload,2572 live.revision.objects.keys().copied().collect()
2502 offset: node.offset,2573 } else {
2503 };2574 commit.changed.clone()
2504 let id = c.exguid()?;2575 };
2505 roots.insert(u32::from_le_bytes(c.read()?), id);2576 if declared != expected
2506 }2577 || (commit.checkpoint && roots != live.revision.roots)
2507 0x1c => break,2578 || (!commit.checkpoint && !roots.is_empty())
2508 _ => return Err(wrong("Unexpected node in a revision manifest")),2579 {
2580 return Err(wrong(
2581 "A revision declares other objects or roots than it changed",
2582 ));
2509 }2583 }
2510 }2584 }
2511 let expected: BTreeSet<ExGuid> = if commit.checkpoint {2585 for (_, rid, context) in labels.iter().filter(|(labelled, ..)| *labelled == space) {
2512 live.revision.objects.keys().copied().collect()2586 let node = next()?;
2513 } else {2587 if node.id != 0x5d || node.payload != label_payload(*rid, *context) {
2514 commit.changed.clone()2588 return Err(wrong("A revision label names another revision or context"));
2515 };2589 }
2516 if declared != expected2590 }
2517 || (commit.checkpoint && roots != live.revision.roots)2591 if next().is_ok() {
2518 || (!commit.checkpoint && !roots.is_empty())
2519 {
2520 return Err(wrong(2592 return Err(wrong(
2521 "A revision declares other objects or roots than it changed",2593 "A revision list gains nodes no revision or label declares",
2522 ));2594 ));
2523 }2595 }
2524 }2596 }
...@@ -2534,8 +2606,10 @@ pub(crate) fn check_transaction(...@@ -2534,8 +2606,10 @@ pub(crate) fn check_transaction(
2534 )?;2606 )?;
2535 let created = revisions2607 let created = revisions
2536 .iter()2608 .iter()
2537 .filter(|written| !before.spaces.contains_key(&written.space))2609 .map(|written| written.space)
2538 .count();2610 .filter(|space| !before.spaces.contains_key(space))
2611 .collect::<BTreeSet<_>>()
2612 .len();
2539 let spaces = added.iter().filter(|node| node.id == 8).count();2613 let spaces = added.iter().filter(|node| node.id == 8).count();
2540 if spaces != created || added.iter().any(|node| !matches!(node.id, 8 | 0x90)) {2614 if spaces != created || added.iter().any(|node| !matches!(node.id, 8 | 0x90)) {
2541 return Err(wrong(2615 return Err(wrong(
crates/onestore/src/write/tests.rs+4-1
...@@ -130,7 +130,10 @@ fn document_insertions_and_formatting_respect_readonly_ancestors() {...@@ -130,7 +130,10 @@ fn document_insertions_and_formatting_respect_readonly_ancestors() {
130 })130 })
131 .unwrap();131 .unwrap();
132 let refused = |op| edited(&protected, sid, vec![op]).is_err();132 let refused = |op| edited(&protected, sid, vec![op]).is_err();
133 assert!(refused(insert(paragraph, text_paragraph("Nested", Format::default()))));133 assert!(refused(insert(
134 paragraph,
135 text_paragraph("Nested", Format::default())
136 )));
134 assert!(refused(PageOp::Split {137 assert!(refused(PageOp::Split {
135 text,138 text,
136 at: 1,139 at: 1,
crates/onestore/src/write/tests/multi.rs+1-2
...@@ -1,7 +1,6 @@...@@ -1,7 +1,6 @@
1use crate::{1use crate::{
2 ExGuid, RevisionIndex, Store,2 ExGuid, RevisionIndex, Store, Transaction,
3 document::{Document, Kind},3 document::{Document, Kind},
4 Transaction,
5 write::{PropertyObject, RevisionEdit, applied, revisions, write_revisions},4 write::{PropertyObject, RevisionEdit, applied, revisions, write_revisions},
6};5};
7use std::collections::BTreeMap;6use std::collections::BTreeMap;
crates/onestore/tests/commit.rs+9-2
...@@ -45,7 +45,8 @@ fn text_of(image: &[u8], space: ExGuid, object: ExGuid) -> String {...@@ -45,7 +45,8 @@ fn text_of(image: &[u8], space: ExGuid, object: ExGuid) -> String {
45 let index = RevisionIndex::parse(&store).unwrap();45 let index = RevisionIndex::parse(&store).unwrap();
46 index.validate_current().unwrap();46 index.validate_current().unwrap();
47 let document = Document::parse(&index).unwrap();47 let document = Document::parse(&index).unwrap();
48 let Kind::RichText { text, .. } = &document.spaces[&space].active().unwrap().nodes[&object].kind48 let Kind::RichText { text, .. } =
49 &document.spaces[&space].active().unwrap().nodes[&object].kind
49 else {50 else {
50 panic!()51 panic!()
51 };52 };
...@@ -76,7 +77,13 @@ fn repeated_appends_cross_log_fragments_and_expose_counter_tears() {...@@ -76,7 +77,13 @@ fn repeated_appends_cross_log_fragments_and_expose_counter_tears() {
76 };77 };
77 length = value.encode_utf16().count() as u32;78 length = value.encode_utf16().count() as u32;
78 section79 section
79 .apply("Author", &Edit { at: 133_700_000_000_000_000 + u64::from(count), ops: vec![Op::Page { space, op }] })80 .apply(
81 "Author",
82 &Edit {
83 at: 133_700_000_000_000_000 + u64::from(count),
84 ops: vec![Op::Page { space, op }],
85 },
86 )
80 .unwrap();87 .unwrap();
81 let transaction = section.seal().unwrap().unwrap();88 let transaction = section.seal().unwrap().unwrap();
82 let mut written = source.clone();89 let mut written = source.clone();
crates/onestore/tests/conflict.rs+30-11
...@@ -91,13 +91,15 @@ fn native_conflict_pages_are_listed_under_their_page_last_first() {...@@ -91,13 +91,15 @@ fn native_conflict_pages_are_listed_under_their_page_last_first() {
91 assert_eq!(conflicts.len(), 1);91 assert_eq!(conflicts.len(), 1);
92 let (page, listed) = &conflicts[0];92 let (page, listed) = &conflicts[0];
93 assert_eq!(*page, pages[0].0);93 assert_eq!(*page, pages[0].0);
94 let [ConflictPage {94 let [
95 space,95 ConflictPage {
96 title,96 space,
97 user,97 title,
98 created,98 user,
99 objects,99 created,
100 }] = listed.as_slice()100 objects,
101 },
102 ] = listed.as_slice()
101 else {103 else {
102 panic!("{listed:?}")104 panic!("{listed:?}")
103 };105 };
...@@ -282,7 +284,10 @@ fn a_conflict_page_is_stored_as_onenote_stores_one() {...@@ -282,7 +284,10 @@ fn a_conflict_page_is_stored_as_onenote_stores_one() {
282 std::fs::create_dir_all(output).unwrap();284 std::fs::create_dir_all(output).unwrap();
283 std::fs::write(output.join("synthetic.one"), &image).unwrap();285 std::fs::write(output.join("synthetic.one"), &image).unwrap();
284 std::fs::copy(286 std::fs::copy(
285 format!("{}/../../{notebook}/Open Notebook.onetoc2", env!("CARGO_MANIFEST_DIR")),287 format!(
288 "{}/../../{notebook}/Open Notebook.onetoc2",
289 env!("CARGO_MANIFEST_DIR")
290 ),
286 output.join("Open Notebook.onetoc2"),291 output.join("Open Notebook.onetoc2"),
287 )292 )
288 .unwrap();293 .unwrap();
...@@ -335,9 +340,20 @@ fn deleting_a_conflict_page_stores_what_onenote_stores() {...@@ -335,9 +340,20 @@ fn deleting_a_conflict_page_stores_what_onenote_stores() {
335 };340 };
336 let (manifest, metadata, flagged, space) = shapes(&image);341 let (manifest, metadata, flagged, space) = shapes(&image);
337 let (native_manifest, native_metadata, native_flagged, native_space) = shapes(&native);342 let (native_manifest, native_metadata, native_flagged, native_space) = shapes(&native);
338 assert_eq!(manifest.1.iter().filter(|id| **id != 0x3400347b).collect::<Vec<_>>(), [&0x24001c1f, &0x24003442]);
339 assert_eq!(343 assert_eq!(
340 native_manifest.1.iter().filter(|id| **id != 0x3400347b).collect::<Vec<_>>(),344 manifest
345 .1
346 .iter()
347 .filter(|id| **id != 0x3400347b)
348 .collect::<Vec<_>>(),
349 [&0x24001c1f, &0x24003442]
350 );
351 assert_eq!(
352 native_manifest
353 .1
354 .iter()
355 .filter(|id| **id != 0x3400347b)
356 .collect::<Vec<_>>(),
341 [&0x24001c1f, &0x24003442]357 [&0x24001c1f, &0x24003442]
342 );358 );
343 assert!(!metadata.1.contains(&0x88001d97) && !native_metadata.1.contains(&0x88001d97));359 assert!(!metadata.1.contains(&0x88001d97) && !native_metadata.1.contains(&0x88001d97));
...@@ -349,7 +365,10 @@ fn deleting_a_conflict_page_stores_what_onenote_stores() {...@@ -349,7 +365,10 @@ fn deleting_a_conflict_page_stores_what_onenote_stores() {
349 std::fs::create_dir_all(output).unwrap();365 std::fs::create_dir_all(output).unwrap();
350 std::fs::write(output.join("synthetic.one"), &image).unwrap();366 std::fs::write(output.join("synthetic.one"), &image).unwrap();
351 std::fs::copy(367 std::fs::copy(
352 format!("{}/../../{notebook}/Open Notebook.onetoc2", env!("CARGO_MANIFEST_DIR")),368 format!(
369 "{}/../../{notebook}/Open Notebook.onetoc2",
370 env!("CARGO_MANIFEST_DIR")
371 ),
353 output.join("Open Notebook.onetoc2"),372 output.join("Open Notebook.onetoc2"),
354 )373 )
355 .unwrap();374 .unwrap();
crates/onestore/tests/document.rs+5-5
...@@ -270,7 +270,7 @@ fn native_paragraph_breaks_preserve_every_stored_character() {...@@ -270,7 +270,7 @@ fn native_paragraph_breaks_preserve_every_stored_character() {
270}270}
271271
272#[test]272#[test]
273fn referenced_history_contexts_require_history_roots() {273fn referenced_history_contexts_require_history_roots_where_stored() {
274 let bytes = fs::read("../../corpus/m6/native-features-01/notebook/Features.one").unwrap();274 let bytes = fs::read("../../corpus/m6/native-features-01/notebook/Features.one").unwrap();
275 let store = Store::parse(&bytes).unwrap();275 let store = Store::parse(&bytes).unwrap();
276 let mut index = RevisionIndex::parse(&store).unwrap();276 let mut index = RevisionIndex::parse(&store).unwrap();
...@@ -306,16 +306,16 @@ fn referenced_history_contexts_require_history_roots() {...@@ -306,16 +306,16 @@ fn referenced_history_contexts_require_history_roots() {
306 Document::parse(&index).unwrap_err().message,306 Document::parse(&index).unwrap_err().message,
307 "History context does not resolve to version history"307 "History context does not resolve to version history"
308 );308 );
309 // OneNote's Copy Page To of a page version names a history it never writes
310 // (corpus/page-versions/native/step-03).
309 index311 index
310 .spaces312 .spaces
311 .get_mut(&sid)313 .get_mut(&sid)
312 .unwrap()314 .unwrap()
313 .labels315 .labels
314 .remove(&(context, 1));316 .remove(&(context, 1));
315 assert_eq!(317 let document = Document::parse(&index).unwrap();
316 Document::parse(&index).unwrap_err().message,318 assert!(!document.spaces[&sid].contexts.contains_key(&context));
317 "Document context has no revision"
318 );
319}319}
320320
321#[test]321#[test]
crates/onestore/tests/edit.rs+10-22
...@@ -181,9 +181,7 @@ fn snapshot_confirmation_needs_no_surviving_edit_target() {...@@ -181,9 +181,7 @@ fn snapshot_confirmation_needs_no_surviving_edit_target() {
181 write_limit: 17,181 write_limit: 17,
182 random: 911,182 random: 911,
183 };183 };
184 assert!(184 assert!(replace_text(SOURCE, ExGuid::default(), ExGuid::default(), 0..0, "").is_err());
185 replace_text(SOURCE, ExGuid::default(), ExGuid::default(), 0..0, "").is_err()
186 );
187 onestore::confirm(&mut disk, &onestore::Stamp::of(SOURCE).unwrap()).unwrap();185 onestore::confirm(&mut disk, &onestore::Stamp::of(SOURCE).unwrap()).unwrap();
188 assert_refreshed(SOURCE, &disk.durable);186 assert_refreshed(SOURCE, &disk.durable);
189 let flush = disk.operation;187 let flush = disk.operation;
...@@ -278,9 +276,7 @@ fn confirmation_rejects_a_changed_header_or_length_before_any_write() {...@@ -278,9 +276,7 @@ fn confirmation_rejects_a_changed_header_or_length_before_any_write() {
278 bytes: source.clone(),276 bytes: source.clone(),
279 events: Vec::new(),277 events: Vec::new(),
280 };278 };
281 let confirmed = |events: &[trace::Event]| {279 let confirmed = |events: &[trace::Event]| matches!(events, [trace::Event::Flush, trace::Event::Write(212, version), trace::Event::Flush] if version.len() == 40);
282 matches!(events, [trace::Event::Flush, trace::Event::Write(212, version), trace::Event::Flush] if version.len() == 40)
283 };
284 onestore::confirm(&mut disk, &base).unwrap();280 onestore::confirm(&mut disk, &base).unwrap();
285 assert!(confirmed(&disk.events));281 assert!(confirmed(&disk.events));
286 for changed in [96, 128, 212, 1023] {282 for changed in [96, 128, 212, 1023] {
...@@ -455,8 +451,7 @@ fn invalid_text_edits_never_touch_storage() {...@@ -455,8 +451,7 @@ fn invalid_text_edits_never_touch_storage() {
455 write_limit: 1024,451 write_limit: 1024,
456 random: 1,452 random: 1,
457 };453 };
458 let error =454 let error = commit_text(&mut disk, &source, sid, oid, range, replacement).unwrap_err();
459 commit_text(&mut disk, &source, sid, oid, range, replacement).unwrap_err();
460 assert_eq!(error.state, CommitState::NotCommitted);455 assert_eq!(error.state, CommitState::NotCommitted);
461 assert_eq!(disk.operation, 0);456 assert_eq!(disk.operation, 0);
462 assert_eq!(disk.durable, source);457 assert_eq!(disk.durable, source);
...@@ -497,8 +492,7 @@ fn cross_run_splices_match_a_character_model_and_preserve_history() {...@@ -497,8 +492,7 @@ fn cross_run_splices_match_a_character_model_and_preserve_history() {
497 let mut expected = before.clone();492 let mut expected = before.clone();
498 expected.splice(a..b, replacement.chars().map(|c| (c, before[a].1.clone())));493 expected.splice(a..b, replacement.chars().map(|c| (c, before[a].1.clone())));
499 let edited =494 let edited =
500 replace_text(SOURCE, sid, oid, offsets[a]..offsets[b], replacement)495 replace_text(SOURCE, sid, oid, offsets[a]..offsets[b], replacement).unwrap();
501 .unwrap();
502 assert_eq!(characters(&edited), expected, "characters {a}..{b}");496 assert_eq!(characters(&edited), expected, "characters {a}..{b}");
503 let current_store = Store::parse(&edited).unwrap();497 let current_store = Store::parse(&edited).unwrap();
504 assert_eq!(498 assert_eq!(
...@@ -519,8 +513,7 @@ fn cross_run_splices_match_a_character_model_and_preserve_history() {...@@ -519,8 +513,7 @@ fn cross_run_splices_match_a_character_model_and_preserve_history() {
519 }513 }
520 }514 }
521 }515 }
522 let cleared =516 let cleared = replace_text(SOURCE, sid, oid, 0..*offsets.last().unwrap(), "").unwrap();
523 replace_text(SOURCE, sid, oid, 0..*offsets.last().unwrap(), "").unwrap();
524 let typed = replace_text(&cleared, sid, oid, 0..0, "new").unwrap();517 let typed = replace_text(&cleared, sid, oid, 0..0, "new").unwrap();
525 let format = &before.last().unwrap().1;518 let format = &before.last().unwrap().1;
526 assert_eq!(519 assert_eq!(
...@@ -579,8 +572,7 @@ fn export_native_cross_run_edits() {...@@ -579,8 +572,7 @@ fn export_native_cross_run_edits() {
579 })572 })
580 .unwrap();573 .unwrap();
581 let range = range.start..range.end.min(length);574 let range = range.start..range.end.min(length);
582 let mut edited =575 let mut edited = replace_text(&source, sid, oid, range.clone(), replacement).unwrap();
583 replace_text(&source, sid, oid, range.clone(), replacement).unwrap();
584 if retype {576 if retype {
585 edited = replace_text(&edited, sid, oid, 0..0, "Retyped 🦀").unwrap();577 edited = replace_text(&edited, sid, oid, 0..0, "Retyped 🦀").unwrap();
586 }578 }
...@@ -648,8 +640,7 @@ fn automatic_titles_follow_native_line_and_utf16_limits() {...@@ -648,8 +640,7 @@ fn automatic_titles_follow_native_line_and_utf16_limits() {
648 };640 };
649 let (sid, oid) = check(&source, &expected);641 let (sid, oid) = check(&source, &expected);
650 let cleared =642 let cleared =
651 replace_text(&source, sid, oid, 0..text.encode_utf16().count() as u32, "")643 replace_text(&source, sid, oid, 0..text.encode_utf16().count() as u32, "").unwrap();
652 .unwrap();
653 check(&cleared, "");644 check(&cleared, "");
654 let written = replace_text(&cleared, sid, oid, 0..0, " new 🦀 name ").unwrap();645 let written = replace_text(&cleared, sid, oid, 0..0, " new 🦀 name ").unwrap();
655 check(&written, "new 🦀 name");646 check(&written, "new 🦀 name");
...@@ -803,8 +794,7 @@ fn title_text_and_navigation_caches_publish_together() {...@@ -803,8 +794,7 @@ fn title_text_and_navigation_caches_publish_together() {
803 for (source, write_limit) in [(source.as_slice(), 17), (checkpoint.as_slice(), 257)] {794 for (source, write_limit) in [(source.as_slice(), 17), (checkpoint.as_slice(), 257)] {
804 let before = state(source);795 let before = state(source);
805 for replacement in ["Renamed 🦀 日本語", ""] {796 for replacement in ["Renamed 🦀 日本語", ""] {
806 let edit =797 let edit = text_transaction(source, sid, *oid, 0..end, replacement).unwrap();
807 text_transaction(source, sid, *oid, 0..end, replacement).unwrap();
808 let disk = |fail_at| Disk {798 let disk = |fail_at| Disk {
809 visible: source.to_vec(),799 visible: source.to_vec(),
810 durable: source.to_vec(),800 durable: source.to_vec(),
...@@ -916,8 +906,7 @@ fn interrupted_text_commits_never_publish_mismatched_run_boundaries() {...@@ -916,8 +906,7 @@ fn interrupted_text_commits_never_publish_mismatched_run_boundaries() {
916 for seed in [1, 42] {906 for seed in [1, 42] {
917 let mut disk = make_disk(Some(at), seed);907 let mut disk = make_disk(Some(at), seed);
918 let error =908 let error =
919 commit_text(&mut disk, source, sid, oid, 0..23, "🐈 mixed edit")909 commit_text(&mut disk, source, sid, oid, 0..23, "🐈 mixed edit").unwrap_err();
920 .unwrap_err();
921 let observed = text_runs(&disk.durable, sid, oid);910 let observed = text_runs(&disk.durable, sid, oid);
922 match error.state {911 match error.state {
923 CommitState::NotCommitted => assert_eq!(observed, before),912 CommitState::NotCommitted => assert_eq!(observed, before),
...@@ -1018,8 +1007,7 @@ fn empty_and_legacy_native_text_gain_unicode_without_losing_existing_properties(...@@ -1018,8 +1007,7 @@ fn empty_and_legacy_native_text_gain_unicode_without_losing_existing_properties(
1018 for seed in [1, 42] {1007 for seed in [1, 42] {
1019 let mut interrupted = disk(Some(at), seed);1008 let mut interrupted = disk(Some(at), seed);
1020 let error =1009 let error =
1021 commit_text(&mut interrupted, &source, sid, oid, 0..0, "🦀 café ")1010 commit_text(&mut interrupted, &source, sid, oid, 0..0, "🦀 café ").unwrap_err();
1022 .unwrap_err();
1023 let actual = text_runs(&interrupted.durable, sid, oid);1011 let actual = text_runs(&interrupted.durable, sid, oid);
1024 match error.state {1012 match error.state {
1025 CommitState::NotCommitted => assert_eq!(actual, before),1013 CommitState::NotCommitted => assert_eq!(actual, before),
crates/onestore/tests/formatting.rs+154-17
...@@ -1,9 +1,9 @@...@@ -1,9 +1,9 @@
1#[path = "support/ops.rs"]
2mod ops;
3#[path = "support/current.rs"]1#[path = "support/current.rs"]
4mod current;2mod current;
5#[path = "support/disk.rs"]3#[path = "support/disk.rs"]
6mod disk;4mod disk;
5#[path = "support/ops.rs"]
6mod ops;
7#[path = "support/sweep.rs"]7#[path = "support/sweep.rs"]
8mod sweep;8mod sweep;
9use onestore::{9use onestore::{
...@@ -90,7 +90,16 @@ fn overlapping_unicode_format_edits_match_an_independent_character_model() {...@@ -90,7 +90,16 @@ fn overlapping_unicode_format_edits_match_an_independent_character_model() {
90 json!(0x00ff00_u32),90 json!(0x00ff00_u32),
91 ),91 ),
92 };92 };
93 let edit = ops::page_op(&source, sid, PageOp::Format { text: id, range: offsets[start]..offsets[end], set: std::slice::from_ref(&attribute).to_vec(), clear: Vec::new() })93 let edit = ops::page_op(
94 &source,
95 sid,
96 PageOp::Format {
97 text: id,
98 range: offsets[start]..offsets[end],
99 set: std::slice::from_ref(&attribute).to_vec(),
100 clear: Vec::new(),
101 },
102 )
94 .unwrap();103 .unwrap();
95 for (_, style) in &mut expected[start..end] {104 for (_, style) in &mut expected[start..end] {
96 style[key] = value.clone();105 style[key] = value.clone();
...@@ -101,7 +110,16 @@ fn overlapping_unicode_format_edits_match_an_independent_character_model() {...@@ -101,7 +110,16 @@ fn overlapping_unicode_format_edits_match_an_independent_character_model() {
101 "seed {seed}, step {step}"110 "seed {seed}, step {step}"
102 );111 );
103 assert_eq!(112 assert_eq!(
104 ops::page_op(edit.as_bytes(), sid, PageOp::Format { text: id, range: offsets[start]..offsets[end], set: vec![attribute], clear: Vec::new() })113 ops::page_op(
114 edit.as_bytes(),
115 sid,
116 PageOp::Format {
117 text: id,
118 range: offsets[start]..offsets[end],
119 set: vec![attribute],
120 clear: Vec::new()
121 }
122 )
105 .unwrap()123 .unwrap()
106 .as_bytes(),124 .as_bytes(),
107 edit.as_bytes()125 edit.as_bytes()
...@@ -123,9 +141,27 @@ fn overlapping_unicode_format_edits_match_an_independent_character_model() {...@@ -123,9 +141,27 @@ fn overlapping_unicode_format_edits_match_an_independent_character_model() {
123fn script_positions_are_exclusive_and_explicit_false_overrides_true() {141fn script_positions_are_exclusive_and_explicit_false_overrides_true() {
124 let source = onestore::create_section("script.one", "abc", "Author").unwrap();142 let source = onestore::create_section("script.one", "abc", "Author").unwrap();
125 let (sid, id) = target(&source);143 let (sid, id) = target(&source);
126 let superscript = ops::page_op(&source, sid, PageOp::Format { text: id, range: 0..3, set: vec![A::Superscript(true), A::Bold(true)], clear: Vec::new() })144 let superscript = ops::page_op(
145 &source,
146 sid,
147 PageOp::Format {
148 text: id,
149 range: 0..3,
150 set: vec![A::Superscript(true), A::Bold(true)],
151 clear: Vec::new(),
152 },
153 )
127 .unwrap();154 .unwrap();
128 let subscript = ops::page_op(superscript.as_bytes(), sid, PageOp::Format { text: id, range: 1..2, set: vec![A::Subscript(true), A::Bold(false)], clear: Vec::new() })155 let subscript = ops::page_op(
156 superscript.as_bytes(),
157 sid,
158 PageOp::Format {
159 text: id,
160 range: 1..2,
161 set: vec![A::Subscript(true), A::Bold(false)],
162 clear: Vec::new(),
163 },
164 )
129 .unwrap();165 .unwrap();
130 let chars = characters(subscript.as_bytes(), sid, id);166 let chars = characters(subscript.as_bytes(), sid, id);
131 assert_eq!(chars[0].1["superscript"], true);167 assert_eq!(chars[0].1["superscript"], true);
...@@ -140,9 +176,27 @@ fn script_positions_are_exclusive_and_explicit_false_overrides_true() {...@@ -140,9 +176,27 @@ fn script_positions_are_exclusive_and_explicit_false_overrides_true() {
140fn empty_paragraph_style_is_used_by_later_text_edits() {176fn empty_paragraph_style_is_used_by_later_text_edits() {
141 let source = onestore::create_section("empty.one", "", "Author").unwrap();177 let source = onestore::create_section("empty.one", "", "Author").unwrap();
142 let (sid, id) = target(&source);178 let (sid, id) = target(&source);
143 let formatted = ops::page_op(&source, sid, PageOp::Format { text: id, range: 0..0, set: vec![A::Italic(true), A::FontSize(18.0)], clear: Vec::new() })179 let formatted = ops::page_op(
180 &source,
181 sid,
182 PageOp::Format {
183 text: id,
184 range: 0..0,
185 set: vec![A::Italic(true), A::FontSize(18.0)],
186 clear: Vec::new(),
187 },
188 )
189 .unwrap();
190 let filled = ops::page_op(
191 formatted.as_bytes(),
192 sid,
193 PageOp::Text {
194 text: id,
195 range: 0..0,
196 with: "Added 🦀".into(),
197 },
198 )
144 .unwrap();199 .unwrap();
145 let filled = ops::page_op(formatted.as_bytes(), sid, PageOp::Text { text: id, range: 0..0, with: "Added 🦀".into() }).unwrap();
146 for (_, format) in characters(filled.as_bytes(), sid, id) {200 for (_, format) in characters(filled.as_bytes(), sid, id) {
147 assert_eq!(format["italic"], true);201 assert_eq!(format["italic"], true);
148 assert_eq!(format["font_size"], 18.0);202 assert_eq!(format["font_size"], 18.0);
...@@ -154,7 +208,19 @@ fn invalid_ranges_attributes_and_fields_are_rejected() {...@@ -154,7 +208,19 @@ fn invalid_ranges_attributes_and_fields_are_rejected() {
154 let (sid, id) = target(&source);208 let (sid, id) = target(&source);
155 for (start, end) in [(2, 3), (1, 2), (0, 8), (2, 2), (3, 1), (1, 1)] {209 for (start, end) in [(2, 3), (1, 2), (0, 8), (2, 2), (3, 1), (1, 1)] {
156 let range = start..end;210 let range = start..end;
157 assert!(ops::page_op(&source, sid, PageOp::Format { text: id, range, set: vec![A::Bold(true)], clear: Vec::new() }).is_err());211 assert!(
212 ops::page_op(
213 &source,
214 sid,
215 PageOp::Format {
216 text: id,
217 range,
218 set: vec![A::Bold(true)],
219 clear: Vec::new()
220 }
221 )
222 .is_err()
223 );
158 }224 }
159 for attributes in [225 for attributes in [
160 vec![],226 vec![],
...@@ -163,18 +229,63 @@ fn invalid_ranges_attributes_and_fields_are_rejected() {...@@ -163,18 +229,63 @@ fn invalid_ranges_attributes_and_fields_are_rejected() {
163 vec![A::Font("".into())],229 vec![A::Font("".into())],
164 vec![A::Font("a\0b".into())],230 vec![A::Font("a\0b".into())],
165 ] {231 ] {
166 assert!(ops::page_op(&source, sid, PageOp::Format { text: id, range: 0..4, set: attributes.to_vec(), clear: Vec::new() }).is_err());232 assert!(
233 ops::page_op(
234 &source,
235 sid,
236 PageOp::Format {
237 text: id,
238 range: 0..4,
239 set: attributes.to_vec(),
240 clear: Vec::new()
241 }
242 )
243 .is_err()
244 );
167 }245 }
168 for size in [f32::NAN, f32::INFINITY, 0.0, 5.5, 130.5, 144.0, 144.5, 12.1] {246 for size in [f32::NAN, f32::INFINITY, 0.0, 5.5, 130.5, 144.0, 144.5, 12.1] {
169 assert!(ops::page_op(&source, sid, PageOp::Format { text: id, range: 0..4, set: vec![A::FontSize(size)], clear: Vec::new() }).is_err());247 assert!(
248 ops::page_op(
249 &source,
250 sid,
251 PageOp::Format {
252 text: id,
253 range: 0..4,
254 set: vec![A::FontSize(size)],
255 clear: Vec::new()
256 }
257 )
258 .is_err()
259 );
170 }260 }
171 assert!(ops::page_op(&source, sid, PageOp::Format { text: ExGuid::default(), range: 0..4, set: vec![A::Bold(true)], clear: Vec::new() }).is_err());261 assert!(
262 ops::page_op(
263 &source,
264 sid,
265 PageOp::Format {
266 text: ExGuid::default(),
267 range: 0..4,
268 set: vec![A::Bold(true)],
269 clear: Vec::new()
270 }
271 )
272 .is_err()
273 );
172}274}
173#[test]275#[test]
174fn formatting_publication_faults_preserve_complete_old_or_new_styles() {276fn formatting_publication_faults_preserve_complete_old_or_new_styles() {
175 let source = onestore::create_section("atomic.one", "Before 🦀 after", "Author").unwrap();277 let source = onestore::create_section("atomic.one", "Before 🦀 after", "Author").unwrap();
176 let (sid, id) = target(&source);278 let (sid, id) = target(&source);
177 let edit = ops::page_op(&source, sid, PageOp::Format { text: id, range: 2..10, set: vec![A::Bold(true), A::Color(Some([8, 64, 128]))], clear: Vec::new() })279 let edit = ops::page_op(
280 &source,
281 sid,
282 PageOp::Format {
283 text: id,
284 range: 2..10,
285 set: vec![A::Bold(true), A::Color(Some([8, 64, 128]))],
286 clear: Vec::new(),
287 },
288 )
178 .unwrap();289 .unwrap();
179 let before = current::current(&source);290 let before = current::current(&source);
180 let after = current::current(edit.as_bytes());291 let after = current::current(edit.as_bytes());
...@@ -219,8 +330,17 @@ fn export_native_formatting_candidates() {...@@ -219,8 +330,17 @@ fn export_native_formatting_candidates() {
219 let mut manifest = Vec::new();330 let mut manifest = Vec::new();
220 let mut save =331 let mut save =
221 |name: &str, source: &[u8], sid, id, range: std::ops::Range<u32>, attributes: &[A]| {332 |name: &str, source: &[u8], sid, id, range: std::ops::Range<u32>, attributes: &[A]| {
222 let prepared =333 let prepared = ops::page_op(
223 ops::page_op(source, sid, PageOp::Format { text: id, range: range.clone(), set: attributes.to_vec(), clear: Vec::new() }).unwrap();334 source,
335 sid,
336 PageOp::Format {
337 text: id,
338 range: range.clone(),
339 set: attributes.to_vec(),
340 clear: Vec::new(),
341 },
342 )
343 .unwrap();
224 fs::write(output.join(format!("{name}.one")), prepared.as_bytes()).unwrap();344 fs::write(output.join(format!("{name}.one")), prepared.as_bytes()).unwrap();
225 manifest.push(345 manifest.push(
226 json!({"name":name,"space":sid,"object":id,"range":range,"attributes":attributes}),346 json!({"name":name,"space":sid,"object":id,"range":range,"attributes":attributes}),
...@@ -298,9 +418,26 @@ fn export_native_formatting_candidates() {...@@ -298,9 +418,26 @@ fn export_native_formatting_candidates() {
298 );418 );
299 let empty = onestore::create_section("empty.one", "", "Author").unwrap();419 let empty = onestore::create_section("empty.one", "", "Author").unwrap();
300 let (empty_sid, empty_id) = target(&empty);420 let (empty_sid, empty_id) = target(&empty);
301 let formatted = ops::page_op(&empty, empty_sid, PageOp::Format { text: empty_id, range: 0..0, set: vec![A::FontSize(18.0), A::Italic(true)], clear: Vec::new() })421 let formatted = ops::page_op(
422 &empty,
423 empty_sid,
424 PageOp::Format {
425 text: empty_id,
426 range: 0..0,
427 set: vec![A::FontSize(18.0), A::Italic(true)],
428 clear: Vec::new(),
429 },
430 )
302 .unwrap();431 .unwrap();
303 let typed = ops::page_op(formatted.as_bytes(), empty_sid, PageOp::Text { text: empty_id, range: 0..0, with: "Typed café 🦀".into() })432 let typed = ops::page_op(
433 formatted.as_bytes(),
434 empty_sid,
435 PageOp::Text {
436 text: empty_id,
437 range: 0..0,
438 with: "Typed café 🦀".into(),
439 },
440 )
304 .unwrap();441 .unwrap();
305 fs::write(output.join("empty-then-type.one"), typed.as_bytes()).unwrap();442 fs::write(output.join("empty-then-type.one"), typed.as_bytes()).unwrap();
306 let native = include_bytes!("../../../corpus/native-ink/20260905-ui/notebook/synthetic.one");443 let native = include_bytes!("../../../corpus/native-ink/20260905-ui/notebook/synthetic.one");
crates/onestore/tests/insertion.rs+58-16
...@@ -50,7 +50,13 @@ fn insertion(...@@ -50,7 +50,13 @@ fn insertion(
50 Place::Outline(x, y) => {50 Place::Outline(x, y) => {
51 let object = ops::outline(x, y, vec![paragraph]);51 let object = ops::outline(x, y, vec![paragraph]);
52 let id = object.id();52 let id = object.id();
53 (PageOp::Add { object, before: None }, id)53 (
54 PageOp::Add {
55 object,
56 before: None,
57 },
58 id,
59 )
54 }60 }
55 };61 };
56 let mut result = vec![Op::Page { space, op }];62 let mut result = vec![Op::Page { space, op }];
...@@ -161,7 +167,10 @@ fn atomic_formatted_insertions_match_a_character_model_and_preserve_history() {...@@ -161,7 +167,10 @@ fn atomic_formatted_insertions_match_a_character_model_and_preserve_history() {
161 Place::In(outline, None)167 Place::In(outline, None)
162 };168 };
163 let (ops, _, inserted) = insertion(sid, place, text, &formats);169 let (ops, _, inserted) = insertion(sid, place, text, &formats);
164 let edit = Edit { at: ops::now(), ops };170 let edit = Edit {
171 at: ops::now(),
172 ops,
173 };
165 let restored: Edit = serde_json::from_slice(&serde_json::to_vec(&edit).unwrap()).unwrap();174 let restored: Edit = serde_json::from_slice(&serde_json::to_vec(&edit).unwrap()).unwrap();
166 assert_eq!(restored, edit);175 assert_eq!(restored, edit);
167 let edited = ops::edited(&source, restored.ops).unwrap();176 let edited = ops::edited(&source, restored.ops).unwrap();
...@@ -308,7 +317,10 @@ fn export_native_formatted_insertions() {...@@ -308,7 +317,10 @@ fn export_native_formatted_insertions() {
308 .into_iter()317 .into_iter()
309 .map(|(label, attributes)| {318 .map(|(label, attributes)| {
310 let start = text[..text.find(label).unwrap()].encode_utf16().count() as u32;319 let start = text[..text.find(label).unwrap()].encode_utf16().count() as u32;
311 (start..start + label.encode_utf16().count() as u32, attributes)320 (
321 start..start + label.encode_utf16().count() as u32,
322 attributes,
323 )
312 })324 })
313 .collect()325 .collect()
314 };326 };
...@@ -322,7 +334,10 @@ fn export_native_formatted_insertions() {...@@ -322,7 +334,10 @@ fn export_native_formatted_insertions() {
322 };334 };
323 ops.push(Op::Page { space: sid, op });335 ops.push(Op::Page { space: sid, op });
324 }336 }
325 let edit = Edit { at: ops::now(), ops };337 let edit = Edit {
338 at: ops::now(),
339 ops,
340 };
326 let bytes = ops::edited(source, edit.ops.clone()).unwrap();341 let bytes = ops::edited(source, edit.ops.clone()).unwrap();
327 current::current(&bytes);342 current::current(&bytes);
328 let notebook = output.join(name);343 let notebook = output.join(name);
...@@ -348,7 +363,9 @@ fn paragraph_and_outline_insertions_publish_metadata_and_references_together() {...@@ -348,7 +363,9 @@ fn paragraph_and_outline_insertions_publish_metadata_and_references_together() {
348 (Place::Outline(144.0, 18.0), page, Some(144.0), Some(18.0)),363 (Place::Outline(144.0, 18.0), page, Some(144.0), Some(18.0)),
349 ] {364 ] {
350 let (ops, object, text) = insertion(sid, place, "First 🦀\rSecond", &[]);365 let (ops, object, text) = insertion(sid, place, "First 🦀\rSecond", &[]);
351 let transaction = ops::transaction(&source, "New author", ops).unwrap().unwrap();366 let transaction = ops::transaction(&source, "New author", ops)
367 .unwrap()
368 .unwrap();
352 let mut written = source.clone();369 let mut written = source.clone();
353 transaction.apply(&mut written).unwrap();370 transaction.apply(&mut written).unwrap();
354 let store = Store::parse(&written).unwrap();371 let store = Store::parse(&written).unwrap();
...@@ -405,7 +422,9 @@ fn repeated_insertions_and_formatting_share_immutable_objects() {...@@ -405,7 +422,9 @@ fn repeated_insertions_and_formatting_share_immutable_objects() {
405 let mut texts = Vec::new();422 let mut texts = Vec::new();
406 for _ in 0..12 {423 for _ in 0..12 {
407 let (ops, _, text) = insertion(sid, Place::In(outline, None), "Repeated paragraph", &[]);424 let (ops, _, text) = insertion(sid, Place::In(outline, None), "Repeated paragraph", &[]);
408 let transaction = ops::transaction(&source, "Same author", ops).unwrap().unwrap();425 let transaction = ops::transaction(&source, "Same author", ops)
426 .unwrap()
427 .unwrap();
409 transaction.apply(&mut source).unwrap();428 transaction.apply(&mut source).unwrap();
410 let bold = PageOp::Format {429 let bold = PageOp::Format {
411 text,430 text,
...@@ -413,9 +432,16 @@ fn repeated_insertions_and_formatting_share_immutable_objects() {...@@ -413,9 +432,16 @@ fn repeated_insertions_and_formatting_share_immutable_objects() {
413 set: vec![A::Bold(true), A::FontSize(20.0)],432 set: vec![A::Bold(true), A::FontSize(20.0)],
414 clear: Vec::new(),433 clear: Vec::new(),
415 };434 };
416 let transaction = ops::transaction(&source, "Same author", vec![Op::Page { space: sid, op: bold }])435 let transaction = ops::transaction(
417 .unwrap()436 &source,
418 .unwrap();437 "Same author",
438 vec![Op::Page {
439 space: sid,
440 op: bold,
441 }],
442 )
443 .unwrap()
444 .unwrap();
419 transaction.apply(&mut source).unwrap();445 transaction.apply(&mut source).unwrap();
420 texts.push(text);446 texts.push(text);
421 }447 }
...@@ -459,8 +485,12 @@ fn serialized_insertions_rebase_with_the_same_objects_and_preserve_remote_edits(...@@ -459,8 +485,12 @@ fn serialized_insertions_rebase_with_the_same_objects_and_preserve_remote_edits(
459 let source = onestore::create_section("rebase.one", "Original", "Author").unwrap();485 let source = onestore::create_section("rebase.one", "Original", "Author").unwrap();
460 let (sid, _, outline, paragraph, text) = targets(&source);486 let (sid, _, outline, paragraph, text) = targets(&source);
461 let formats = [(0..3, vec![A::Bold(true), A::Color(Some([12, 34, 56]))])];487 let formats = [(0..3, vec![A::Bold(true), A::Color(Some([12, 34, 56]))])];
462 let (ops, object, inserted) =488 let (ops, object, inserted) = insertion(
463 insertion(sid, Place::In(outline, Some(paragraph)), "Inserted", &formats);489 sid,
490 Place::In(outline, Some(paragraph)),
491 "Inserted",
492 &formats,
493 );
464 let encoded = serde_json::to_vec(&ops).unwrap();494 let encoded = serde_json::to_vec(&ops).unwrap();
465 let restored: Vec<Op> = serde_json::from_slice(&encoded).unwrap();495 let restored: Vec<Op> = serde_json::from_slice(&encoded).unwrap();
466 assert_eq!(ops, restored);496 assert_eq!(ops, restored);
...@@ -540,8 +570,12 @@ fn anchors_targets_identities_and_text_are_validated_before_publication() {...@@ -540,8 +570,12 @@ fn anchors_targets_identities_and_text_are_validated_before_publication() {
540 assert!(refused(&source, Place::In(outline, None), content));570 assert!(refused(&source, Place::In(outline, None), content));
541 }571 }
542 assert!(572 assert!(
543 ops::transaction(&source, "a\0b", insertion(sid, Place::In(outline, None), "Text", &[]).0)573 ops::transaction(
544 .is_err()574 &source,
575 "a\0b",
576 insertion(sid, Place::In(outline, None), "Text", &[]).0
577 )
578 .is_err()
545 );579 );
546 for place in [580 for place in [
547 Place::In(outline, Some(text)),581 Place::In(outline, Some(text)),
...@@ -610,13 +644,21 @@ fn insertion_publication_faults_expose_only_complete_graphs_and_title_caches() {...@@ -610,13 +644,21 @@ fn insertion_publication_faults_expose_only_complete_graphs_and_title_caches() {
610 &checkpoint,644 &checkpoint,
611 ] {645 ] {
612 let (sid, _, outline, paragraph, _) = targets(source);646 let (sid, _, outline, paragraph, _) = targets(source);
613 for place in [Place::In(outline, Some(paragraph)), Place::Outline(0.0, 0.0)] {647 for place in [
648 Place::In(outline, Some(paragraph)),
649 Place::Outline(0.0, 0.0),
650 ] {
614 let formats = [651 let formats = [
615 (0..3, vec![A::Bold(true), A::FontSize(18.0)]),652 (0..3, vec![A::Bold(true), A::FontSize(18.0)]),
616 (6..8, vec![A::Italic(true), A::Highlight(Some([255, 255, 0]))]),653 (
654 6..8,
655 vec![A::Italic(true), A::Highlight(Some([255, 255, 0]))],
656 ),
617 ];657 ];
618 let (ops, _, _) = insertion(sid, place, "First 🦀", &formats);658 let (ops, _, _) = insertion(sid, place, "First 🦀", &formats);
619 let edit = ops::transaction(source, "New author", ops).unwrap().unwrap();659 let edit = ops::transaction(source, "New author", ops)
660 .unwrap()
661 .unwrap();
620 let mut written = source.to_vec();662 let mut written = source.to_vec();
621 edit.apply(&mut written).unwrap();663 edit.apply(&mut written).unwrap();
622 let before = current::current(source);664 let before = current::current(source);
crates/onestore/tests/outline.rs+119-25
...@@ -67,7 +67,15 @@ fn geometry_and_expansion_preserve_unrelated_properties_objects_and_history() {...@@ -67,7 +67,15 @@ fn geometry_and_expansion_preserve_unrelated_properties_objects_and_history() {
67 for (name, sid, object, edit) in cases(&document) {67 for (name, sid, object, edit) in cases(&document) {
68 let restored = serde_json::from_value(serde_json::to_value(edit).unwrap()).unwrap();68 let restored = serde_json::from_value(serde_json::to_value(edit).unwrap()).unwrap();
69 assert_eq!(edit, restored);69 assert_eq!(edit, restored);
70 let prepared = ops::page_op(SOURCE, sid, PageOp::Outline { object, edit: restored }).unwrap();70 let prepared = ops::page_op(
71 SOURCE,
72 sid,
73 PageOp::Outline {
74 object,
75 edit: restored,
76 },
77 )
78 .unwrap();
71 let after_store = Store::parse(prepared.as_bytes()).unwrap();79 let after_store = Store::parse(prepared.as_bytes()).unwrap();
72 assert!(after_store.checksum_mismatches.is_empty());80 assert!(after_store.checksum_mismatches.is_empty());
73 let after_index = RevisionIndex::parse(&after_store).unwrap();81 let after_index = RevisionIndex::parse(&after_store).unwrap();
...@@ -144,7 +152,8 @@ fn geometry_and_expansion_preserve_unrelated_properties_objects_and_history() {...@@ -144,7 +152,8 @@ fn geometry_and_expansion_preserve_unrelated_properties_objects_and_history() {
144 assert_eq!(old_space.labels, after_index.spaces[space_id].labels);152 assert_eq!(old_space.labels, after_index.spaces[space_id].labels);
145 }153 }
146 }154 }
147 let again = ops::page_op(prepared.as_bytes(), sid, PageOp::Outline { object, edit }).unwrap();155 let again =
156 ops::page_op(prepared.as_bytes(), sid, PageOp::Outline { object, edit }).unwrap();
148 assert_eq!(again.as_bytes(), prepared.as_bytes());157 assert_eq!(again.as_bytes(), prepared.as_bytes());
149 }158 }
150}159}
...@@ -169,10 +178,17 @@ fn invalid_geometry_and_non_outline_targets_are_rejected() {...@@ -169,10 +178,17 @@ fn invalid_geometry_and_non_outline_targets_are_rejected() {
169 }178 }
170 for points in [-1.0, 0.0, 35.999] {179 for points in [-1.0, 0.0, 35.999] {
171 assert!(180 assert!(
172 ops::page_op(SOURCE, sid, PageOp::Outline { object, edit: Edit::Width {181 ops::page_op(
173 points,182 SOURCE,
174 user_set: false183 sid,
175 } })184 PageOp::Outline {
185 object,
186 edit: Edit::Width {
187 points,
188 user_set: false
189 }
190 }
191 )
176 .is_err()192 .is_err()
177 );193 );
178 }194 }
...@@ -181,8 +197,15 @@ fn invalid_geometry_and_non_outline_targets_are_rejected() {...@@ -181,8 +197,15 @@ fn invalid_geometry_and_non_outline_targets_are_rejected() {
181 for (id, node) in &view.nodes {197 for (id, node) in &view.nodes {
182 if !matches!(node.kind, Kind::Outline { .. }) {198 if !matches!(node.kind, Kind::Outline { .. }) {
183 assert!(199 assert!(
184 ops::page_op(SOURCE, sid, PageOp::Outline { object: *id, edit: Edit::Position { x: 72.0, y: 72.0 } })200 ops::page_op(
185 .is_err()201 SOURCE,
202 sid,
203 PageOp::Outline {
204 object: *id,
205 edit: Edit::Position { x: 72.0, y: 72.0 }
206 }
207 )
208 .is_err()
186 );209 );
187 }210 }
188 if matches!(node.kind, Kind::Title) {211 if matches!(node.kind, Kind::Title) {
...@@ -197,20 +220,55 @@ fn invalid_geometry_and_non_outline_targets_are_rejected() {...@@ -197,20 +220,55 @@ fn invalid_geometry_and_non_outline_targets_are_rejected() {
197 },220 },
198 Edit::Collapsed(true),221 Edit::Collapsed(true),
199 ] {222 ] {
200 assert!(ops::page_op(SOURCE, sid, PageOp::Outline { object: child, edit }).is_err());223 assert!(
224 ops::page_op(
225 SOURCE,
226 sid,
227 PageOp::Outline {
228 object: child,
229 edit
230 }
231 )
232 .is_err()
233 );
201 }234 }
202 }235 }
203 }236 }
204 if !matches!(node.kind, Kind::Paragraph { .. }) {237 if !matches!(node.kind, Kind::Paragraph { .. }) {
205 assert!(ops::page_op(SOURCE, sid, PageOp::Outline { object: *id, edit: Edit::Collapsed(true) }).is_err());238 assert!(
239 ops::page_op(
240 SOURCE,
241 sid,
242 PageOp::Outline {
243 object: *id,
244 edit: Edit::Collapsed(true)
245 }
246 )
247 .is_err()
248 );
206 }249 }
207 }250 }
208 assert!(251 assert!(
209 ops::page_op(SOURCE, ExGuid::default(), PageOp::Outline { object, edit: Edit::Collapsed(true) })252 ops::page_op(
210 .is_err()253 SOURCE,
254 ExGuid::default(),
255 PageOp::Outline {
256 object,
257 edit: Edit::Collapsed(true)
258 }
259 )
260 .is_err()
211 );261 );
212 assert!(262 assert!(
213 ops::page_op(SOURCE, sid, PageOp::Outline { object: ExGuid::default(), edit: Edit::Collapsed(true) }).is_err()263 ops::page_op(
264 SOURCE,
265 sid,
266 PageOp::Outline {
267 object: ExGuid::default(),
268 edit: Edit::Collapsed(true)
269 }
270 )
271 .is_err()
214 );272 );
215 }273 }
216}274}
...@@ -228,10 +286,17 @@ fn resizing_a_native_reserved_width_preserves_content() {...@@ -228,10 +286,17 @@ fn resizing_a_native_reserved_width_preserves_content() {
228 let space = &document.spaces[&sid];286 let space = &document.spaces[&sid];
229 let before = &space.revisions[&space.contexts[&ExGuid::default()]];287 let before = &space.revisions[&space.contexts[&ExGuid::default()]];
230 assert!(before.nodes[&object].layout.reserved_width.is_some());288 assert!(before.nodes[&object].layout.reserved_width.is_some());
231 let edit = ops::page_op(source, sid, PageOp::Outline { object, edit: Edit::Width {289 let edit = ops::page_op(
232 points: 144.0,290 source,
233 user_set: true,291 sid,
234 } })292 PageOp::Outline {
293 object,
294 edit: Edit::Width {
295 points: 144.0,
296 user_set: true,
297 },
298 },
299 )
235 .unwrap();300 .unwrap();
236 let after_store = Store::parse(edit.as_bytes()).unwrap();301 let after_store = Store::parse(edit.as_bytes()).unwrap();
237 let after_index = RevisionIndex::parse(&after_store).unwrap();302 let after_index = RevisionIndex::parse(&after_store).unwrap();
...@@ -279,8 +344,15 @@ fn outline_movement_updates_both_automatic_title_fields_without_changing_content...@@ -279,8 +344,15 @@ fn outline_movement_updates_both_automatic_title_fields_without_changing_content
279 (144.0, 36.0, "First"),344 (144.0, 36.0, "First"),
280 (0.0, 144.0, "First"),345 (0.0, 144.0, "First"),
281 ] {346 ] {
282 let edit = ops::page_op(&source, sid, PageOp::Outline { object: insertion, edit: Edit::Position { x, y } })347 let edit = ops::page_op(
283 .unwrap();348 &source,
349 sid,
350 PageOp::Outline {
351 object: insertion,
352 edit: Edit::Position { x, y },
353 },
354 )
355 .unwrap();
284 let saved = Store::parse(edit.as_bytes()).unwrap();356 let saved = Store::parse(edit.as_bytes()).unwrap();
285 let saved_index = RevisionIndex::parse(&saved).unwrap();357 let saved_index = RevisionIndex::parse(&saved).unwrap();
286 saved_index.validate_current().unwrap();358 saved_index.validate_current().unwrap();
...@@ -311,7 +383,14 @@ fn outline_movement_updates_both_automatic_title_fields_without_changing_content...@@ -311,7 +383,14 @@ fn outline_movement_updates_both_automatic_title_fields_without_changing_content
311 format!("{:?}", saved_index.resolve(sid, rid).unwrap())383 format!("{:?}", saved_index.resolve(sid, rid).unwrap())
312 );384 );
313 assert_eq!(385 assert_eq!(
314 ops::page_op(edit.as_bytes(), sid, PageOp::Outline { object: insertion, edit: Edit::Position { x, y } })386 ops::page_op(
387 edit.as_bytes(),
388 sid,
389 PageOp::Outline {
390 object: insertion,
391 edit: Edit::Position { x, y }
392 }
393 )
315 .unwrap()394 .unwrap()
316 .as_bytes(),395 .as_bytes(),
317 edit.as_bytes()396 edit.as_bytes()
...@@ -352,10 +431,17 @@ fn native_automatic_and_explicit_titles_follow_outline_movement() {...@@ -352,10 +431,17 @@ fn native_automatic_and_explicit_titles_follow_outline_movement() {
352 72.0431 72.0
353 },432 },
354 };433 };
355 candidate = ops::page_op(&candidate, sid, PageOp::Outline { object: outline, edit: change })434 candidate = ops::page_op(
356 .unwrap()435 &candidate,
357 .as_bytes()436 sid,
358 .to_vec();437 PageOp::Outline {
438 object: outline,
439 edit: change,
440 },
441 )
442 .unwrap()
443 .as_bytes()
444 .to_vec();
359 let store = Store::parse(&candidate).unwrap();445 let store = Store::parse(&candidate).unwrap();
360 let saved = RevisionIndex::parse(&store).unwrap();446 let saved = RevisionIndex::parse(&store).unwrap();
361 saved.validate_current().unwrap();447 saved.validate_current().unwrap();
...@@ -417,7 +503,15 @@ fn outline_publication_interruptions_reopen_as_complete_old_or_new_layout() {...@@ -417,7 +503,15 @@ fn outline_publication_interruptions_reopen_as_complete_old_or_new_layout() {
417 ),503 ),
418 (paragraph, Edit::Collapsed(true)),504 (paragraph, Edit::Collapsed(true)),
419 ] {505 ] {
420 let edit = ops::page_op(&source, sid, PageOp::Outline { object, edit: operation }).unwrap();506 let edit = ops::page_op(
507 &source,
508 sid,
509 PageOp::Outline {
510 object,
511 edit: operation,
512 },
513 )
514 .unwrap();
421 let before = current::current(&source);515 let before = current::current(&source);
422 let after = current::current(edit.as_bytes());516 let after = current::current(edit.as_bytes());
423 for write_limit in [17, 4096] {517 for write_limit in [17, 4096] {
crates/onestore/tests/page.rs+21-5
...@@ -117,7 +117,15 @@ fn created_pages_support_title_edits_and_body_insertion() {...@@ -117,7 +117,15 @@ fn created_pages_support_title_edits_and_body_insertion() {
117 Some(title.filter(|s| !s.is_empty()).unwrap_or("Body 🦀 é"))117 Some(title.filter(|s| !s.is_empty()).unwrap_or("Body 🦀 é"))
118 );118 );
119 if let Some(text) = intent.title_object() {119 if let Some(text) = intent.title_object() {
120 source = ops::page_op(&source, intent.space(), PageOp::Text { text, range: 0..title.unwrap().encode_utf16().count() as u32, with: "Renamed".into() })120 source = ops::page_op(
121 &source,
122 intent.space(),
123 PageOp::Text {
124 text,
125 range: 0..title.unwrap().encode_utf16().count() as u32,
126 with: "Renamed".into(),
127 },
128 )
121 .unwrap()129 .unwrap()
122 .as_bytes()130 .as_bytes()
123 .to_vec();131 .to_vec();
...@@ -359,10 +367,18 @@ fn native_changes_on_created_pages_accept_rust_followups() {...@@ -359,10 +367,18 @@ fn native_changes_on_created_pages_accept_rust_followups() {
359 let mut source = initial.to_vec();367 let mut source = initial.to_vec();
360 for (sid, title, body) in targets {368 for (sid, title, body) in targets {
361 for (object, prefix) in [(body, "Rust + "), (title, "Reviewed ")] {369 for (object, prefix) in [(body, "Rust + "), (title, "Reviewed ")] {
362 source = ops::page_op(&source, sid, PageOp::Text { text: object, range: 0..0, with: prefix.into() })370 source = ops::page_op(
363 .unwrap()371 &source,
364 .as_bytes()372 sid,
365 .to_vec();373 PageOp::Text {
374 text: object,
375 range: 0..0,
376 with: prefix.into(),
377 },
378 )
379 .unwrap()
380 .as_bytes()
381 .to_vec();
366 }382 }
367 let store = Store::parse(&source).unwrap();383 let store = Store::parse(&source).unwrap();
368 let index = RevisionIndex::parse(&store).unwrap();384 let index = RevisionIndex::parse(&store).unwrap();
crates/onestore/tests/page_deletion.rs+11-3
...@@ -171,8 +171,11 @@ fn removal_rejects_duplicate_missing_and_non_page_spaces_without_publication() {...@@ -171,8 +171,11 @@ fn removal_rejects_duplicate_missing_and_non_page_spaces_without_publication() {
171 );171 );
172 let written = ops::section_op(SOURCE, SectionOp::Delete([pages[0].0].to_vec())).unwrap();172 let written = ops::section_op(SOURCE, SectionOp::Delete([pages[0].0].to_vec())).unwrap();
173 assert!(173 assert!(
174 ops::section_op(written.as_bytes(), SectionOp::Delete([pages[1].0, pages[0].0].to_vec()))174 ops::section_op(
175 .is_err()175 written.as_bytes(),
176 SectionOp::Delete([pages[1].0, pages[0].0].to_vec())
177 )
178 .is_err()
176 );179 );
177}180}
178181
...@@ -427,7 +430,12 @@ fn external_payloads_and_their_historical_references_survive_file_removal() {...@@ -427,7 +430,12 @@ fn external_payloads_and_their_historical_references_survive_file_removal() {
427 assert_eq!(payloads.len(), 3);430 assert_eq!(payloads.len(), 3);
428 let path = root.join("synthetic.one");431 let path = root.join("synthetic.one");
429 std::fs::write(&path, &original).unwrap();432 std::fs::write(&path, &original).unwrap();
430 prepared.transaction.as_ref().unwrap().commit_file(&path).unwrap();433 prepared
434 .transaction
435 .as_ref()
436 .unwrap()
437 .commit_file(&path)
438 .unwrap();
431 let written = onestore::read_file(&path).unwrap();439 let written = onestore::read_file(&path).unwrap();
432 assert_eq!(written, prepared.as_bytes());440 assert_eq!(written, prepared.as_bytes());
433 let current_store = Store::parse(&written).unwrap();441 let current_store = Store::parse(&written).unwrap();
crates/onestore/tests/page_ink.rs-1
...@@ -6,7 +6,6 @@ use onestore::{...@@ -6,7 +6,6 @@ use onestore::{
6 page::{Ink, InkStroke, Page, PageObject, text::new_id},6 page::{Ink, InkStroke, Page, PageObject, text::new_id},
7};7};
88
9
10/// Two drawings made with the mouse in OneNote 2010 (`tools/native/ink.ahk`): a diamond and a9/// Two drawings made with the mouse in OneNote 2010 (`tools/native/ink.ahk`): a diamond and a
11/// diagonal line, whose native read reports their positions and sizes.10/// diagonal line, whose native read reports their positions and sizes.
12const NATIVE: &[u8] =11const NATIVE: &[u8] =
crates/onestore/tests/page_links.rs+47-2
...@@ -162,6 +162,8 @@ fn a_link_is_added_to_a_fresh_page_and_reads_back() {...@@ -162,6 +162,8 @@ fn a_link_is_added_to_a_fresh_page_and_reads_back() {
162 );162 );
163 let written = ops::saved(&source, space, &after).unwrap();163 let written = ops::saved(&source, space, &after).unwrap();
164 let stored = assert_same(written.as_slice(), space, &after);164 let stored = assert_same(written.as_slice(), space, &after);
165 // As OneNote titles `NATIVE_LINKS`'s page holding this paragraph: without the field code.
166 assert_eq!(stored.title, "Read about Rust the Rust site");
165 let mut stored = stored;167 let mut stored = stored;
166 let text = &body_paragraphs(&mut stored)[0].text().unwrap().text;168 let text = &body_paragraphs(&mut stored)[0].text().unwrap().text;
167 let flags: Vec<(Option<bool>, Option<bool>, Option<bool>)> = text169 let flags: Vec<(Option<bool>, Option<bool>, Option<bool>)> = text
...@@ -285,6 +287,24 @@ fn internal_links_match_what_onenote_stores() {...@@ -285,6 +287,24 @@ fn internal_links_match_what_onenote_stores() {
285 );287 );
286 assert_eq!(parse_internal_link(&stored[0]), None);288 assert_eq!(parse_internal_link(&stored[0]), None);
287 assert_eq!(parse_internal_link("onenote:#x&page-id={0}"), None);289 assert_eq!(parse_internal_link("onenote:#x&page-id={0}"), None);
290 // The Link dialog's page and Copy Link to Page name the section file before the `#`.
291 let ids = "section-id={0F1EFC25-DF7C-45CB-87A4-BDD03F42057E}&page-id={A9CCA318-A43E-4912-A1C7-F19F1EAF30CA}";
292 for url in [
293 format!("onenote:links.one#Page&{ids}&base-path=//C:/notebook"),
294 format!("onenote:///C:\\notebook\\links.one#Page&{ids}&end"),
295 ] {
296 let link = parse_internal_link(&url).unwrap();
297 assert_eq!(
298 (
299 link.section[..4].to_vec(),
300 link.page.map(|page| page[..4].to_vec())
301 ),
302 (
303 vec![0x25, 0xfc, 0x1e, 0x0f],
304 Some(vec![0x18, 0xa3, 0xcc, 0xa9])
305 )
306 );
307 }
288}308}
289309
290/// `ONESTORE_INTERNAL_LINK_EXPORT` names a new directory receiving the candidate for a cold310/// `ONESTORE_INTERNAL_LINK_EXPORT` names a new directory receiving the candidate for a cold
...@@ -295,7 +315,9 @@ fn a_page_links_to_another_page_and_its_paragraph() {...@@ -295,7 +315,9 @@ fn a_page_links_to_another_page_and_its_paragraph() {
295 let creation = onestore::PageCreation::new(None, Some("Link target"), "Author").unwrap();315 let creation = onestore::PageCreation::new(None, Some("Link target"), "Author").unwrap();
296 let with_target = ops::edited(316 let with_target = ops::edited(
297 &source,317 &source,
298 vec![onestore::op::Op::Section(onestore::op::SectionOp::Create(creation))],318 vec![onestore::op::Op::Section(onestore::op::SectionOp::Create(
319 creation,
320 ))],
299 )321 )
300 .unwrap();322 .unwrap();
301 let section = Store::parse(&with_target).unwrap().header.file_id;323 let section = Store::parse(&with_target).unwrap().header.file_id;
...@@ -354,7 +376,13 @@ fn a_page_links_to_another_page_and_its_paragraph() {...@@ -354,7 +376,13 @@ fn a_page_links_to_another_page_and_its_paragraph() {
354 })376 })
355 .unwrap();377 .unwrap();
356 let written = ops::saved(&with_target, space, &after).unwrap();378 let written = ops::saved(&with_target, space, &after).unwrap();
357 assert_same(written.as_slice(), space, &after);379 let stored = assert_same(written.as_slice(), space, &after);
380 // Titles show the link's label, not its field code, as OneNote stores them.
381 assert!(
382 stored.title.starts_with("Linking page Link target"),
383 "{}",
384 stored.title
385 );
358 if let Some(directory) = std::env::var_os("ONESTORE_INTERNAL_LINK_EXPORT") {386 if let Some(directory) = std::env::var_os("ONESTORE_INTERNAL_LINK_EXPORT") {
359 let directory = std::path::PathBuf::from(directory);387 let directory = std::path::PathBuf::from(directory);
360 std::fs::create_dir(&directory).unwrap();388 std::fs::create_dir(&directory).unwrap();
...@@ -367,3 +395,20 @@ fn a_page_links_to_another_page_and_its_paragraph() {...@@ -367,3 +395,20 @@ fn a_page_links_to_another_page_and_its_paragraph() {
367 .unwrap();395 .unwrap();
368 }396 }
369}397}
398
399/// A title stored with a link's field code, as an earlier writer left it
400/// (`corpus/link-edit/internal/candidate`), reads as the label the page shows.
401#[test]
402fn titles_read_without_link_field_codes() {
403 let bytes = include_bytes!("../../../corpus/link-edit/internal/candidate/links.one").to_vec();
404 let arena = onestore::Arena::default();
405 let mut section = onestore::Section::open(&arena, bytes.clone()).unwrap();
406 let titles: Vec<String> = section
407 .pages()
408 .unwrap()
409 .into_iter()
410 .map(|(_, title, _)| title)
411 .collect();
412 assert_eq!(titles, ["Linking page Link target", "Link target"]);
413 page_by_title(&bytes, "Linking page Link target");
414}
crates/onestore/tests/page_math.rs+267-15
...@@ -26,6 +26,10 @@ const EDITOR_3: &[u8] =...@@ -26,6 +26,10 @@ const EDITOR_3: &[u8] =
26 include_bytes!("../../../corpus/math-edit/native-editor-3/notebook/links.one");26 include_bytes!("../../../corpus/math-edit/native-editor-3/notebook/links.one");
27const EDITOR_3_READ: &str =27const EDITOR_3_READ: &str =
28 include_str!("../../../corpus/math-edit/native-editor-3/read/page-000.xml");28 include_str!("../../../corpus/math-edit/native-editor-3/read/page-000.xml");
29/// A fourth session repeating most of the others and adding an exponent, Greek letters, a
30/// fraction of sums, a Pythagorean sum, a limit applied as a function and braces.
31const EDITOR_4: &[u8] =
32 include_bytes!("../../../corpus/math-edit/native-editor-4/notebook/links.one");
2933
30fn native_mathml(read: &str) -> Vec<String> {34fn native_mathml(read: &str) -> Vec<String> {
31 read.split("<mml:math")35 read.split("<mml:math")
...@@ -70,22 +74,23 @@ fn rendered_equations(bytes: &[u8], title: &str) -> Vec<String> {...@@ -70,22 +74,23 @@ fn rendered_equations(bytes: &[u8], title: &str) -> Vec<String> {
70 if !Math::is_equation(&text.text) {74 if !Math::is_equation(&text.text) {
71 continue;75 continue;
72 }76 }
73 // Inline math follows ordinary text in the same paragraph; only the math runs77 // Inline math follows ordinary text in the same paragraph, and a line break
74 // form the equation.78 // divides an equation: each run of math is an equation of its own.
75 let mut at = 0;79 let mut at = 0;
76 let math_runs: Vec<(String, onestore::document::Format)> = text80 let mut zones: Vec<Vec<(String, onestore::document::Format)>> = vec![Vec::new()];
77 .text81 for span in text.text.spans() {
78 .spans()82 let piece = text.text.text()[at..span.end].to_owned();
79 .iter()83 at = span.end;
80 .map(|span| {84 if span.format.math == Some(true) {
81 let piece = text.text.text()[at..span.end].to_owned();85 zones.last_mut().unwrap().push((piece, span.format.clone()));
82 at = span.end;86 } else if !zones.last().unwrap().is_empty() {
83 (piece, span.format.clone())87 zones.push(Vec::new());
84 })88 }
85 .filter(|(_, format)| format.math == Some(true))89 }
86 .collect();90 for zone in zones.into_iter().filter(|zone| !zone.is_empty()) {
87 let equation = onestore::page::Paragraph::from_runs(math_runs);91 let equation = onestore::page::Paragraph::from_runs(zone);
88 rendered.push(Math::mathml(&Math::parse(&equation).unwrap()));92 rendered.push(Math::mathml(&Math::parse(&equation).unwrap()));
93 }
89 }94 }
90 }95 }
91 rendered96 rendered
...@@ -99,8 +104,17 @@ fn native_equations_render_to_the_mathml_onenote_exports() {...@@ -99,8 +104,17 @@ fn native_equations_render_to_the_mathml_onenote_exports() {
99 );104 );
100}105}
101106
107/// Enter inside equations (`tools/native_enter.py`): a line break dividing one, and an
108/// argument made an equation array.
109const ENTER: &[u8] = include_bytes!("../../../corpus/math-edit/native-enter/notebook/links.one");
110const ENTER_READ: &str = include_str!("../../../corpus/math-edit/native-enter/read/page-000.xml");
111
102#[test]112#[test]
103fn equation_editor_expressions_render_to_the_mathml_onenote_exports() {113fn equation_editor_expressions_render_to_the_mathml_onenote_exports() {
114 assert_eq!(
115 rendered_equations(ENTER, "Read about Rust the Rust site"),
116 native_mathml(ENTER_READ)
117 );
104 assert_eq!(118 assert_eq!(
105 rendered_equations(EDITOR, "Read about Rust the Rust site"),119 rendered_equations(EDITOR, "Read about Rust the Rust site"),
106 native_mathml(EDITOR_READ)120 native_mathml(EDITOR_READ)
...@@ -369,3 +383,241 @@ fn a_paragraph_emptied_of_its_inline_math_saves() {...@@ -369,3 +383,241 @@ fn a_paragraph_emptied_of_its_inline_math_saves() {
369 let document = Document::parse(&index).unwrap();383 let document = Document::parse(&index).unwrap();
370 assert_eq!(Page::from_space(&document, space).unwrap(), after);384 assert_eq!(Page::from_space(&document, space).unwrap(), after);
371}385}
386
387type MathRun = (String, Option<onestore::document::MathObject>, Option<bool>);
388
389/// The math runs of each equation paragraph on the page titled `title`: text, run data and
390/// whether the run is italic, which is what the equation editor decides.
391fn stored_equations(bytes: &[u8], title: &str) -> Vec<Vec<MathRun>> {
392 let store = Store::parse(bytes).unwrap();
393 let index = RevisionIndex::parse(&store).unwrap();
394 let document = Document::parse(&index).unwrap();
395 let page = document
396 .pages()
397 .unwrap()
398 .into_iter()
399 .map(|(space, _)| Page::from_space(&document, space).unwrap())
400 .find(|page| page.title == title)
401 .unwrap();
402 let mut equations = Vec::new();
403 for object in &page.objects {
404 let PageObject::Outline(outline) = object else {
405 continue;
406 };
407 for paragraph in &outline.paragraphs {
408 if let Some(text) = paragraph
409 .text()
410 .filter(|text| Math::is_equation(&text.text))
411 {
412 equations.push(runs(&text.text));
413 }
414 }
415 }
416 equations
417}
418
419fn runs(paragraph: &onestore::page::Paragraph) -> Vec<MathRun> {
420 let mut at = 0;
421 paragraph
422 .spans()
423 .iter()
424 .filter_map(|span| {
425 let text = paragraph.text()[at..span.end].to_owned();
426 at = span.end;
427 (span.format.math == Some(true))
428 .then(|| (text, span.format.math_object.clone(), span.format.italic))
429 })
430 .collect()
431}
432
433/// What OneNote 2010's equation editor stored for text typed into it, each followed by the
434/// space that builds it up, in the order the equations appear; `None` skips an equation the
435/// editor built in a way this reading does not follow.
436const TYPED: &[(&[u8], &[Option<&str>])] = &[
437 (
438 EDITOR,
439 &[
440 Some("a_1+b_2"),
441 Some("x_i^2"),
442 Some("(a+b)"),
443 Some("\\int_0^1 x dx"),
444 Some("\\sum_(i=1)^n i"),
445 ],
446 ),
447 (
448 EDITOR_2,
449 &[
450 Some("\\sqrt x+1"),
451 Some("\\cbrt(x)"),
452 Some("\\sqrt(n&x)"),
453 Some("a/b"),
454 Some("\\lim_(x\\to 0) f(x)"),
455 Some("\\prod_(k=1)^n k"),
456 Some("[a+b]"),
457 Some("\\overline(x)"),
458 Some("x\\hat"),
459 ],
460 ),
461 (
462 EDITOR_3,
463 &[
464 Some("\\matrix(1&2@3&4)"),
465 Some("\\eqarray(x&=1@y&=2)"),
466 Some("x\\above 2"),
467 Some("x\\below 2"),
468 Some("\\box(x)"),
469 Some("\\rect(x)"),
470 // `\underline` became an unrelated symbol.
471 None,
472 Some("\\iint x dx dy"),
473 Some("f(x)/(x^2+1)"),
474 Some("\\sum^n x"),
475 ],
476 ),
477 (
478 EDITOR_4,
479 &[
480 Some("a_1+b_2"),
481 // The editor was still starting this one and kept its placeholder.
482 None,
483 Some("\\sum_(i=1)^n i"),
484 Some("\\int_0^1 x dx"),
485 Some("\\sqrt x+1"),
486 Some("\\cbrt(x)"),
487 Some("\\sqrt(n&x)"),
488 Some("a/b"),
489 Some("\\prod_(k=1)^n k"),
490 Some("[a+b]"),
491 Some("\\overline(x)"),
492 Some("x\\hat"),
493 Some("\\matrix(1&2@3&4)"),
494 Some("\\eqarray(x&=1@y&=2)"),
495 Some("x\\above 2"),
496 Some("x\\below 2"),
497 Some("\\box(x)"),
498 Some("\\rect(x)"),
499 Some("f(x)/(x^2+1)"),
500 Some("\\sum^n x"),
501 Some("e^(x+1)"),
502 Some("\\alpha+\\beta"),
503 Some("(a+b)/(c+d)"),
504 Some("x^2+y^2=z^2"),
505 // A function applied to a limit, an object this reading does not build.
506 None,
507 Some("{a+b}"),
508 ],
509 ),
510];
511
512#[test]
513fn typed_linear_text_builds_up_as_the_equation_editor_stores_it() {
514 for (bytes, typed) in TYPED {
515 let stored = stored_equations(bytes, "Read about Rust the Rust site");
516 assert_eq!(stored.len(), typed.len());
517 for (stored, typed) in stored.iter().zip(*typed) {
518 let Some(typed) = typed else {
519 continue;
520 };
521 let built = Math::paragraph(
522 &Math::from_linear(&format!("{typed} ")),
523 &onestore::document::Format::default(),
524 );
525 assert_eq!(&runs(&built), stored, "{typed}");
526 }
527 }
528}
529
530/// Every native equation reads back from the linear text written for it, except the two
531/// the linear format here cannot write: a limit applied as a function and an empty
532/// equation's placeholder (`EDITOR_4`).
533#[test]
534fn native_equations_survive_their_linear_form() {
535 let mut unwritten = 0;
536 for (bytes, title) in [
537 (NATIVE, "Equation controls"),
538 (EDITOR, "Read about Rust the Rust site"),
539 (EDITOR_2, "Read about Rust the Rust site"),
540 (EDITOR_3, "Read about Rust the Rust site"),
541 (EDITOR_4, "Read about Rust the Rust site"),
542 ] {
543 for stored in stored_equations(bytes, title) {
544 let paragraph = onestore::page::Paragraph::from_runs(stored.iter().map(
545 |(text, object, italic)| {
546 let format = onestore::document::Format {
547 math: Some(true),
548 math_object: object.clone(),
549 italic: *italic,
550 ..Default::default()
551 };
552 (text.clone(), format)
553 },
554 ));
555 let Some(linear) = Math::linear(&Math::parse(&paragraph).unwrap()) else {
556 unwritten += 1;
557 continue;
558 };
559 let rebuilt = Math::paragraph(&Math::from_linear(&linear), &Default::default());
560 // Linear text without objects stores no run data until it is built up.
561 if stored.iter().all(|(_, object, _)| object.is_none()) {
562 assert_eq!(rebuilt.text(), paragraph.text());
563 continue;
564 }
565 assert_eq!(runs(&rebuilt), stored, "{linear}");
566 }
567 }
568 assert_eq!(unwritten, 2);
569}
570
571/// OneNote 2010's Linear on equations typed through its editor (`corpus/math-edit/native-linear`):
572/// the tree built from the typed text writes as the linear text OneNote showed and stored,
573/// and that text builds up to the same tree.
574#[test]
575fn equations_write_the_linear_text_onenote_shows() {
576 const LINEAR: &[u8] =
577 include_bytes!("../../../corpus/math-edit/native-linear/notebook/links.one");
578 let typed = [
579 "x^2+1",
580 "(a+b)/(c+d)",
581 "a_1+b_2",
582 "x_i^2",
583 "\\sum_(i=1)^n i",
584 "\\int_0^1 x dx",
585 "\\sqrt x+1",
586 "\\cbrt(x)",
587 "\\sqrt(n&x)",
588 "a/b",
589 "\\prod_(k=1)^n k",
590 "[a+b]",
591 "\\overline(x)",
592 "x\\hat",
593 "\\matrix(1&2@3&4)",
594 "\\eqarray(x&=1@y&=2)",
595 "x\\above 2",
596 "x\\below 2",
597 "\\box(x)",
598 "\\rect(x)",
599 "f(x)/(x^2+1)",
600 "\\sum^n x",
601 "e^(x+1)",
602 "\\alpha+\\beta",
603 ];
604 let shown: Vec<String> = stored_equations(LINEAR, "Read about Rust the Rust siteR")
605 .into_iter()
606 .map(|runs| runs.into_iter().map(|(text, ..)| text).collect())
607 .collect();
608 assert_eq!(shown.len(), typed.len());
609 for (shown, typed) in shown.iter().zip(typed) {
610 let built = Math::from_linear(&format!("{typed} "));
611 assert_eq!(
612 Math::linear(&built).as_deref(),
613 Some(shown.as_str()),
614 "{typed}"
615 );
616 let base = onestore::document::Format::default();
617 assert_eq!(
618 Math::paragraph(&Math::from_linear(shown), &base),
619 Math::paragraph(&built, &base),
620 "{typed}"
621 );
622 }
623}
crates/onestore/tests/page_movement.rs+66-15
...@@ -212,23 +212,52 @@ fn invalid_and_empty_page_edits_do_not_publish() {...@@ -212,23 +212,52 @@ fn invalid_and_empty_page_edits_do_not_publish() {
212 }212 }
213 assert!(PageEdit::move_to(ids[0], Some(ids[0]), 1).is_err());213 assert!(PageEdit::move_to(ids[0], Some(ids[0]), 1).is_err());
214 assert!(PageEdit::set_level(ExGuid::default(), 1).is_err());214 assert!(PageEdit::set_level(ExGuid::default(), 1).is_err());
215 assert_eq!(ops::section_op(SOURCE, SectionOp::Pages([].to_vec())).unwrap().as_bytes(), SOURCE);
216 assert_eq!(215 assert_eq!(
217 ops::section_op(SOURCE, SectionOp::Pages([PageEdit::set_level(ids[0], 1).unwrap()].to_vec()))216 ops::section_op(SOURCE, SectionOp::Pages([].to_vec()))
218 .unwrap()217 .unwrap()
219 .as_bytes(),218 .as_bytes(),
220 SOURCE219 SOURCE
221 );220 );
222 assert!(ops::section_op(SOURCE, SectionOp::Pages([PageEdit::set_level(ids[0], 2).unwrap()].to_vec())).is_err());221 assert_eq!(
222 ops::section_op(
223 SOURCE,
224 SectionOp::Pages([PageEdit::set_level(ids[0], 1).unwrap()].to_vec())
225 )
226 .unwrap()
227 .as_bytes(),
228 SOURCE
229 );
230 assert!(
231 ops::section_op(
232 SOURCE,
233 SectionOp::Pages([PageEdit::set_level(ids[0], 2).unwrap()].to_vec())
234 )
235 .is_err()
236 );
223 let duplicate = PageEdit::set_level(ids[3], 2).unwrap();237 let duplicate = PageEdit::set_level(ids[3], 2).unwrap();
224 assert!(ops::section_op(SOURCE, SectionOp::Pages([duplicate.clone(), duplicate].to_vec())).is_err());238 assert!(
239 ops::section_op(
240 SOURCE,
241 SectionOp::Pages([duplicate.clone(), duplicate].to_vec())
242 )
243 .is_err()
244 );
225 let missing = ExGuid {245 let missing = ExGuid {
226 guid: [0x77; 16],246 guid: [0x77; 16],
227 n: 1,247 n: 1,
228 };248 };
229 assert!(ops::section_op(SOURCE, SectionOp::Pages([PageEdit::set_level(missing, 1).unwrap()].to_vec())).is_err());
230 assert!(249 assert!(
231 ops::section_op(SOURCE, SectionOp::Pages([PageEdit::move_to(ids[3], Some(missing), 1).unwrap()].to_vec()))250 ops::section_op(
251 SOURCE,
252 SectionOp::Pages([PageEdit::set_level(missing, 1).unwrap()].to_vec())
253 )
254 .is_err()
255 );
256 assert!(
257 ops::section_op(
258 SOURCE,
259 SectionOp::Pages([PageEdit::move_to(ids[3], Some(missing), 1).unwrap()].to_vec())
260 )
232 .is_err()261 .is_err()
233 );262 );
234 let mut value = serde_json::to_value(PageEdit::set_level(ids[3], 2).unwrap()).unwrap();263 let mut value = serde_json::to_value(PageEdit::set_level(ids[3], 2).unwrap()).unwrap();
...@@ -253,8 +282,11 @@ fn moving_pages_preserves_ink_file_data_and_native_feature_objects() {...@@ -253,8 +282,11 @@ fn moving_pages_preserves_ink_file_data_and_native_feature_objects() {
253 .unwrap()282 .unwrap()
254 .as_bytes()283 .as_bytes()
255 .to_vec();284 .to_vec();
256 let moved =285 let moved = ops::section_op(
257 ops::section_op(&source, SectionOp::Pages([PageEdit::move_to(first, None, 1).unwrap()].to_vec())).unwrap();286 &source,
287 SectionOp::Pages([PageEdit::move_to(first, None, 1).unwrap()].to_vec()),
288 )
289 .unwrap();
258 assert_eq!(pages(moved.as_bytes()).last(), Some(&first));290 assert_eq!(pages(moved.as_bytes()).last(), Some(&first));
259 let stores = [&source, moved.as_bytes()].map(|bytes| Store::parse(bytes).unwrap());291 let stores = [&source, moved.as_bytes()].map(|bytes| Store::parse(bytes).unwrap());
260 let indexes = stores292 let indexes = stores
...@@ -338,7 +370,11 @@ fn metadata_copy_order_does_not_change_page_identity() {...@@ -338,7 +370,11 @@ fn metadata_copy_order_does_not_change_page_identity() {
338 "../../../corpus/page-lifecycle/page-edits/optional-cache/source-cold/notebook/Lifecycle.one"370 "../../../corpus/page-lifecycle/page-edits/optional-cache/source-cold/notebook/Lifecycle.one"
339 );371 );
340 let sid = pages(source)[3];372 let sid = pages(source)[3];
341 let edited = ops::section_op(source, SectionOp::Pages([PageEdit::set_level(sid, 2).unwrap()].to_vec())).unwrap();373 let edited = ops::section_op(
374 source,
375 SectionOp::Pages([PageEdit::set_level(sid, 2).unwrap()].to_vec()),
376 )
377 .unwrap();
342 let expected = include_bytes!(378 let expected = include_bytes!(
343 "../../../corpus/page-lifecycle/page-edits/optional-cache/candidate/Lifecycle.one"379 "../../../corpus/page-lifecycle/page-edits/optional-cache/candidate/Lifecycle.one"
344 );380 );
...@@ -374,8 +410,21 @@ fn native_edits_on_moved_pages_support_more_rust_changes() {...@@ -374,8 +410,21 @@ fn native_edits_on_moved_pages_support_more_rust_changes() {
374 let bodies: Vec<_> = view.nodes.iter().filter_map(|(id, node)|410 let bodies: Vec<_> = view.nodes.iter().filter_map(|(id, node)|
375 matches!(&node.kind, Kind::RichText { text, .. } if text == "Native body after a Rust page move.").then_some(*id)).collect();411 matches!(&node.kind, Kind::RichText { text, .. } if text == "Native body after a Rust page move.").then_some(*id)).collect();
376 let [body] = bodies.as_slice() else { panic!() };412 let [body] = bodies.as_slice() else { panic!() };
377 let moved = ops::section_op(source, SectionOp::Pages([PageEdit::set_level(sid, 2).unwrap()].to_vec())).unwrap();413 let moved = ops::section_op(
378 let edited = ops::page_op(moved.as_bytes(), sid, PageOp::Text { text: *body, range: 0..0, with: "Rust + ".into() }).unwrap();414 source,
415 SectionOp::Pages([PageEdit::set_level(sid, 2).unwrap()].to_vec()),
416 )
417 .unwrap();
418 let edited = ops::page_op(
419 moved.as_bytes(),
420 sid,
421 PageOp::Text {
422 text: *body,
423 range: 0..0,
424 with: "Rust + ".into(),
425 },
426 )
427 .unwrap();
379 let after_store = Store::parse(edited.as_bytes()).unwrap();428 let after_store = Store::parse(edited.as_bytes()).unwrap();
380 let after_index = RevisionIndex::parse(&after_store).unwrap();429 let after_index = RevisionIndex::parse(&after_store).unwrap();
381 let after = Document::parse(&after_index).unwrap();430 let after = Document::parse(&after_index).unwrap();
...@@ -533,9 +582,8 @@ fn repeated_nesting_retains_series_history_across_revision_checkpoints() {...@@ -533,9 +582,8 @@ fn repeated_nesting_retains_series_history_across_revision_checkpoints() {
533/// without them and gives the moved page one of its own.582/// without them and gives the moved page one of its own.
534#[test]583#[test]
535fn a_move_after_a_native_merge_drops_its_empty_series() {584fn a_move_after_a_native_merge_drops_its_empty_series() {
536 let merged = include_bytes!(585 let merged =
537 "../../../corpus/conflict-page/native-pages/merged/notebook/synthetic.one"586 include_bytes!("../../../corpus/conflict-page/native-pages/merged/notebook/synthetic.one");
538 );
539 let series = |bytes: &[u8]| {587 let series = |bytes: &[u8]| {
540 let store = Store::parse(bytes).unwrap();588 let store = Store::parse(bytes).unwrap();
541 let index = RevisionIndex::parse(&store).unwrap();589 let index = RevisionIndex::parse(&store).unwrap();
...@@ -561,7 +609,10 @@ fn a_move_after_a_native_merge_drops_its_empty_series() {...@@ -561,7 +609,10 @@ fn a_move_after_a_native_merge_drops_its_empty_series() {
561 let (id, spaces) = after.last().unwrap();609 let (id, spaces) = after.last().unwrap();
562 assert_eq!(spaces, &vec![ids[1]]);610 assert_eq!(spaces, &vec![ids[1]]);
563 assert!(before.iter().all(|(old, _)| old != id));611 assert!(before.iter().all(|(old, _)| old != id));
564 assert_eq!(pages(moved.as_bytes()), [&ids[..1], &ids[2..], &ids[1..2]].concat());612 assert_eq!(
613 pages(moved.as_bytes()),
614 [&ids[..1], &ids[2..], &ids[1..2]].concat()
615 );
565 if let Some(output) = std::env::var_os("ONESTORE_EMPTY_SERIES_EXPORT") {616 if let Some(output) = std::env::var_os("ONESTORE_EMPTY_SERIES_EXPORT") {
566 let output = std::path::Path::new(&output);617 let output = std::path::Path::new(&output);
567 std::fs::create_dir(output).unwrap();618 std::fs::create_dir(output).unwrap();
crates/onestore/tests/page_writer.rs+32-10
...@@ -242,16 +242,34 @@ fn formatting_matches_format_text() {...@@ -242,16 +242,34 @@ fn formatting_matches_format_text() {
242 let (space, page) = page_by_title(PARAGRAPHS, "Split style boundary");242 let (space, page) = page_by_title(PARAGRAPHS, "Split style boundary");
243 let (_, text, content) = text_ids(body(&page)[0])[0].clone();243 let (_, text, content) = text_ids(body(&page)[0])[0].clone();
244 let end = u32::try_from(content.encode_utf16().count()).unwrap();244 let end = u32::try_from(content.encode_utf16().count()).unwrap();
245 let reference = ops::page_op(PARAGRAPHS, space, PageOp::Format { text, range: 1..end.min(4), set: vec![245 let reference = ops::page_op(
246 TextAttribute::Bold(true),246 PARAGRAPHS,
247 TextAttribute::Color(Some([255, 0, 0])),247 space,
248 TextAttribute::FontSize(14.0),248 PageOp::Format {
249 ], clear: Vec::new() })249 text,
250 range: 1..end.min(4),
251 set: vec![
252 TextAttribute::Bold(true),
253 TextAttribute::Color(Some([255, 0, 0])),
254 TextAttribute::FontSize(14.0),
255 ],
256 clear: Vec::new(),
257 },
258 )
250 .unwrap();259 .unwrap();
251 let reference = ops::page_op(reference.as_bytes(), space, PageOp::Format { text, range: 0..1, set: vec![260 let reference = ops::page_op(
252 TextAttribute::Italic(true),261 reference.as_bytes(),
253 TextAttribute::Font("Consolas".into()),262 space,
254 ], clear: Vec::new() })263 PageOp::Format {
264 text,
265 range: 0..1,
266 set: vec![
267 TextAttribute::Italic(true),
268 TextAttribute::Font("Consolas".into()),
269 ],
270 clear: Vec::new(),
271 },
272 )
255 .unwrap();273 .unwrap();
256 oracle(PARAGRAPHS, space, reference.as_bytes());274 oracle(PARAGRAPHS, space, reference.as_bytes());
257}275}
...@@ -444,7 +462,11 @@ fn composed_edits_publish_one_transaction() {...@@ -444,7 +462,11 @@ fn composed_edits_publish_one_transaction() {
444 }],462 }],
445 );463 );
446 let insertion = insert(outline.id, Some(paragraphs[1].0), "Inserted", 1);464 let insertion = insert(outline.id, Some(paragraphs[1].0), "Inserted", 1);
447 let PageOp::Insert { paragraphs: inserted, .. } = &insertion else {465 let PageOp::Insert {
466 paragraphs: inserted,
467 ..
468 } = &insertion
469 else {
448 unreachable!()470 unreachable!()
449 };471 };
450 let inserted = inserted[0].text().unwrap().id;472 let inserted = inserted[0].text().unwrap().id;
crates/onestore/tests/paragraph.rs+50-12
...@@ -44,7 +44,13 @@ fn split(source: &[u8], space: ExGuid, text: ExGuid, at: u32) -> (Op, ExGuid, Ex...@@ -44,7 +44,13 @@ fn split(source: &[u8], space: ExGuid, text: ExGuid, at: u32) -> (Op, ExGuid, Ex
4444
45/// `source` after joining the paragraph of text `right` to the one of text `left`.45/// `source` after joining the paragraph of text `right` to the one of text `left`.
46fn join(source: &[u8], space: ExGuid, left: ExGuid, right: ExGuid) -> Result<Vec<u8>, OpError> {46fn join(source: &[u8], space: ExGuid, left: ExGuid, right: ExGuid) -> Result<Vec<u8>, OpError> {
47 ops::edited(source, vec![Op::Page { space, op: PageOp::Join { left, right } }])47 ops::edited(
48 source,
49 vec![Op::Page {
50 space,
51 op: PageOp::Join { left, right },
52 }],
53 )
48}54}
4955
50const SOURCE: &[u8] =56const SOURCE: &[u8] =
...@@ -445,10 +451,7 @@ fn splits_partition_native_paragraphs_at_every_scalar_boundary() {...@@ -445,10 +451,7 @@ fn splits_partition_native_paragraphs_at_every_scalar_boundary() {
445 assert_eq!(after.nodes[parent].children, children);451 assert_eq!(after.nodes[parent].children, children);
446 assert_eq!(after.nodes[&paragraph].content, [text]);452 assert_eq!(after.nodes[&paragraph].content, [text]);
447 assert!(after.nodes[&paragraph].children.is_empty());453 assert!(after.nodes[&paragraph].children.is_empty());
448 assert_eq!(454 assert_eq!(after.nodes[&new_paragraph].content, [new_text]);
449 after.nodes[&new_paragraph].content,
450 [new_text]
451 );
452 assert_eq!(455 assert_eq!(
453 after.nodes[&new_paragraph].children,456 after.nodes[&new_paragraph].children,
454 before.nodes[&paragraph].children457 before.nodes[&paragraph].children
...@@ -516,7 +519,17 @@ fn invalid_split_identities_and_title_targets_are_rejected() {...@@ -516,7 +519,17 @@ fn invalid_split_identities_and_title_targets_are_rejected() {
516 .0;519 .0;
517 let (op, ..) = split(SOURCE, sid, text, 1);520 let (op, ..) = split(SOURCE, sid, text, 1);
518 assert!(ops::transaction(SOURCE, "a\0b", vec![op.clone()]).is_err());521 assert!(ops::transaction(SOURCE, "a\0b", vec![op.clone()]).is_err());
519 let Op::Page { op: PageOp::Split { paragraph, right, lists, .. }, .. } = op else {522 let Op::Page {
523 op:
524 PageOp::Split {
525 paragraph,
526 right,
527 lists,
528 ..
529 },
530 ..
531 } = op
532 else {
520 unreachable!()533 unreachable!()
521 };534 };
522 for (text, at, paragraph) in [535 for (text, at, paragraph) in [
...@@ -525,7 +538,13 @@ fn invalid_split_identities_and_title_targets_are_rejected() {...@@ -525,7 +538,13 @@ fn invalid_split_identities_and_title_targets_are_rejected() {
525 (text, 1, ExGuid::default()),538 (text, 1, ExGuid::default()),
526 (text, 1, text),539 (text, 1, text),
527 ] {540 ] {
528 let op = PageOp::Split { text, at, paragraph, right, lists: lists.clone() };541 let op = PageOp::Split {
542 text,
543 at,
544 paragraph,
545 right,
546 lists: lists.clone(),
547 };
529 assert!(ops::page_edited(SOURCE, sid, vec![op]).is_err());548 assert!(ops::page_edited(SOURCE, sid, vec![op]).is_err());
530 }549 }
531 let mut titles = 0;550 let mut titles = 0;
...@@ -638,7 +657,9 @@ fn interrupted_splits_publish_a_complete_graph_or_retain_the_original() {...@@ -638,7 +657,9 @@ fn interrupted_splits_publish_a_complete_graph_or_retain_the_original() {
638 let checkpoint = checkpoint::pending(source, sid, text);657 let checkpoint = checkpoint::pending(source, sid, text);
639 for source in [source, &checkpoint] {658 for source in [source, &checkpoint] {
640 let (op, ..) = split(source, sid, text, 1);659 let (op, ..) = split(source, sid, text, 1);
641 let edit = ops::transaction(source, "Author", vec![op]).unwrap().unwrap();660 let edit = ops::transaction(source, "Author", vec![op])
661 .unwrap()
662 .unwrap();
642 let mut written = source.to_vec();663 let mut written = source.to_vec();
643 edit.apply(&mut written).unwrap();664 edit.apply(&mut written).unwrap();
644 let before = current::current(source);665 let before = current::current(source);
...@@ -698,7 +719,10 @@ fn export_native_paragraph_joins() {...@@ -698,7 +719,10 @@ fn export_native_paragraph_joins() {
698 })719 })
699 .unwrap()720 .unwrap()
700 .0;721 .0;
701 let op = Op::Page { space: sid, op: PageOp::Join { left, right } };722 let op = Op::Page {
723 space: sid,
724 op: PageOp::Join { left, right },
725 };
702 let transaction = ops::transaction(&source, "Rust join author", vec![op.clone()])726 let transaction = ops::transaction(&source, "Rust join author", vec![op.clone()])
703 .unwrap()727 .unwrap()
704 .unwrap();728 .unwrap();
...@@ -767,8 +791,16 @@ fn interrupted_joins_preserve_complete_graphs_and_empty_text_adoption() {...@@ -767,8 +791,16 @@ fn interrupted_joins_preserve_complete_graphs_and_empty_text_adoption() {
767 .unwrap();791 .unwrap();
768 let checkpoint = checkpoint::pending(original, sid, left);792 let checkpoint = checkpoint::pending(original, sid, left);
769 for source in [original, &empty, &checkpoint] {793 for source in [original, &empty, &checkpoint] {
770 let op = Op::Page { space: sid, op: PageOp::Join { left, right: right_text } };794 let op = Op::Page {
771 let edit = ops::transaction(source, "Join author", vec![op]).unwrap().unwrap();795 space: sid,
796 op: PageOp::Join {
797 left,
798 right: right_text,
799 },
800 };
801 let edit = ops::transaction(source, "Join author", vec![op])
802 .unwrap()
803 .unwrap();
772 let mut written = source.to_vec();804 let mut written = source.to_vec();
773 edit.apply(&mut written).unwrap();805 edit.apply(&mut written).unwrap();
774 let before = current::current(source);806 let before = current::current(source);
...@@ -828,7 +860,13 @@ fn joins_reject_invalid_identities_wrong_order_and_unrelated_pages() {...@@ -828,7 +860,13 @@ fn joins_reject_invalid_identities_wrong_order_and_unrelated_pages() {
828 ] {860 ] {
829 assert!(join(source, *sid, a, b).is_err());861 assert!(join(source, *sid, a, b).is_err());
830 }862 }
831 let op = Op::Page { space: *sid, op: PageOp::Join { left: *left, right: *right } };863 let op = Op::Page {
864 space: *sid,
865 op: PageOp::Join {
866 left: *left,
867 right: *right,
868 },
869 };
832 assert!(ops::transaction(source, "a\0b", vec![op]).is_err());870 assert!(ops::transaction(source, "a\0b", vec![op]).is_err());
833 let view = &space.revisions[&space.contexts[&ExGuid::default()]];871 let view = &space.revisions[&space.contexts[&ExGuid::default()]];
834 let paragraph = *view872 let paragraph = *view
crates/onestore/tests/protected.rs+4-1
...@@ -188,7 +188,10 @@ fn a_protected_page_edit_is_stored_under_the_section_key() {...@@ -188,7 +188,10 @@ fn a_protected_page_edit_is_stored_under_the_section_key() {
188 let ops = onestore::op::lower_page(&before, page).unwrap();188 let ops = onestore::op::lower_page(&before, page).unwrap();
189 let edit = Edit {189 let edit = Edit {
190 at: 134_000_000_000_000_000,190 at: 134_000_000_000_000_000,
191 ops: ops.into_iter().map(|op| Op::Page { space: *space, op }).collect(),191 ops: ops
192 .into_iter()
193 .map(|op| Op::Page { space: *space, op })
194 .collect(),
192 };195 };
193 let store = Store::parse(&bytes).unwrap();196 let store = Store::parse(&bytes).unwrap();
194 let index = RevisionIndex::parse(&store).unwrap();197 let index = RevisionIndex::parse(&store).unwrap();
crates/onestore/tests/shared_snapshot.rs+1-2
...@@ -198,8 +198,7 @@ fn published_snapshots_survive_interleaved_commit_io() {...@@ -198,8 +198,7 @@ fn published_snapshots_survive_interleaved_commit_io() {
198 bytes: source.clone(),198 bytes: source.clone(),
199 events: Vec::new(),199 events: Vec::new(),
200 };200 };
201 commit_text(&mut trace, &source, sid, oid, end..end, " [reader café 🦀]")201 commit_text(&mut trace, &source, sid, oid, end..end, " [reader café 🦀]").unwrap();
202 .unwrap();
203 let after = current(&trace.bytes);202 let after = current(&trace.bytes);
204 assert_ne!(before, after);203 assert_ne!(before, after);
205 let mut writes = Vec::new();204 let mut writes = Vec::new();
crates/onestore/tests/support/page_edits.rs+13-3
...@@ -5,8 +5,8 @@...@@ -5,8 +5,8 @@
55
6use onestore::{6use onestore::{
7 Arena, ExGuid, RevisionIndex, Section, Store,7 Arena, ExGuid, RevisionIndex, Section, Store,
8 op::{self, Op},
9 document::{Document, Format, Layout},8 document::{Document, Format, Layout},
9 op::{self, Op},
10 page::{10 page::{
11 Outline, Page, PageObject, PageParagraph, Paragraph, ParagraphContent,11 Outline, Page, PageObject, PageParagraph, Paragraph, ParagraphContent,
12 text::{Edit, new_id},12 text::{Edit, new_id},
...@@ -576,10 +576,20 @@ pub fn run(input: &[u8]) {...@@ -576,10 +576,20 @@ pub fn run(input: &[u8]) {
576 let mut section = Section::open(&arena, source.clone()).unwrap();576 let mut section = Section::open(&arena, source.clone()).unwrap();
577 let edit = op::Edit {577 let edit = op::Edit {
578 at: 134_000_000_000_000_000,578 at: 134_000_000_000_000_000,
579 ops: ops.iter().map(|op| Op::Page { space, op: op.clone() }).collect(),579 ops: ops
580 .iter()
581 .map(|op| Op::Page {
582 space,
583 op: op.clone(),
584 })
585 .collect(),
580 };586 };
581 if section.apply("Fuzz author", &edit).is_err() {587 if section.apply("Fuzz author", &edit).is_err() {
582 assert_eq!(section.page(space).unwrap(), before, "a refused edit changes nothing");588 assert_eq!(
589 section.page(space).unwrap(),
590 before,
591 "a refused edit changes nothing"
592 );
583 return;593 return;
584 }594 }
585 let mut predicted = before.clone();595 let mut predicted = before.clone();
crates/onestore/tests/support/page_schedule.rs+11-2
...@@ -94,7 +94,8 @@ pub fn run(input: &[u8]) {...@@ -94,7 +94,8 @@ pub fn run(input: &[u8]) {
94 let removed: Vec<_> = (0..count)94 let removed: Vec<_> = (0..count)
95 .map(|i| pages[(usize::from(step[2]) + i) % pages.len()].0)95 .map(|i| pages[(usize::from(step[2]) + i) % pages.len()].0)
96 .collect();96 .collect();
97 let prepared = ops::section_op(source, SectionOp::Delete(removed.to_vec())).unwrap();97 let prepared =
98 ops::section_op(source, SectionOp::Delete(removed.to_vec())).unwrap();
98 pages.retain(|page| !removed.contains(&page.0));99 pages.retain(|page| !removed.contains(&page.0));
99 for sid in removed {100 for sid in removed {
100 levels.remove(&sid);101 levels.remove(&sid);
...@@ -143,7 +144,15 @@ pub fn run(input: &[u8]) {...@@ -143,7 +144,15 @@ pub fn run(input: &[u8]) {
143 unreachable!()144 unreachable!()
144 };145 };
145 Some((146 Some((
146 ops::page_op(source, sid, PageOp::Text { text: object, range: 0..u32::try_from(original.encode_utf16().count()).unwrap(), with: text.into() })147 ops::page_op(
148 source,
149 sid,
150 PageOp::Text {
151 text: object,
152 range: 0..u32::try_from(original.encode_utf16().count()).unwrap(),
153 with: text.into(),
154 },
155 )
147 .unwrap(),156 .unwrap(),
148 Some((sid, object, text)),157 Some((sid, object, text)),
149 ))158 ))
crates/onestore/tests/support/tree_model.rs+18-7
...@@ -4,10 +4,7 @@ use onestore::{...@@ -4,10 +4,7 @@ use onestore::{
4 op::{Op, PageOp},4 op::{Op, PageOp},
5 page::Page,5 page::Page,
6};6};
7use std::{7use std::{collections::BTreeMap, sync::LazyLock};
8 collections::BTreeMap,
9 sync::LazyLock,
10};
118
12use crate::{current, disk, ops};9use crate::{current, disk, ops};
1310
...@@ -136,7 +133,12 @@ pub fn run(input: &[u8]) {...@@ -136,7 +133,12 @@ pub fn run(input: &[u8]) {
136 let destination = destinations[usize::from(step[4]) % destinations.len()];133 let destination = destinations[usize::from(step[4]) % destinations.len()];
137 // Outline groups are the writers' business: anchors are the paragraphs in them.134 // Outline groups are the writers' business: anchors are the paragraphs in them.
138 let mut children = Vec::new();135 let mut children = Vec::new();
139 let mut pending: Vec<ExGuid> = view.nodes[&destination].children.iter().rev().copied().collect();136 let mut pending: Vec<ExGuid> = view.nodes[&destination]
137 .children
138 .iter()
139 .rev()
140 .copied()
141 .collect();
140 while let Some(id) = pending.pop() {142 while let Some(id) = pending.pop() {
141 match view.nodes[&id].kind {143 match view.nodes[&id].kind {
142 Kind::OutlineGroup => pending.extend(view.nodes[&id].children.iter().rev()),144 Kind::OutlineGroup => pending.extend(view.nodes[&id].children.iter().rev()),
...@@ -155,7 +157,10 @@ pub fn run(input: &[u8]) {...@@ -155,7 +157,10 @@ pub fn run(input: &[u8]) {
155 (op, subtree.contains_key(&destination))157 (op, subtree.contains_key(&destination))
156 };158 };
157 let model = Page::from_space(&document, sid).unwrap();159 let model = Page::from_space(&document, sid).unwrap();
158 let ops = vec![Op::Page { space: sid, op: op.clone() }];160 let ops = vec![Op::Page {
161 space: sid,
162 op: op.clone(),
163 }];
159 let Ok(edit) = ops::apply(source, "Tree schedule", ops) else {164 let Ok(edit) = ops::apply(source, "Tree schedule", ops) else {
160 continue;165 continue;
161 };166 };
...@@ -170,7 +175,13 @@ pub fn run(input: &[u8]) {...@@ -170,7 +175,13 @@ pub fn run(input: &[u8]) {
170 if stored != predicted {175 if stored != predicted {
171 let (a, b) = (format!("{stored:?}"), format!("{predicted:?}"));176 let (a, b) = (format!("{stored:?}"), format!("{predicted:?}"));
172 let at = a.bytes().zip(b.bytes()).take_while(|(x, y)| x == y).count();177 let at = a.bytes().zip(b.bytes()).take_while(|(x, y)| x == y).count();
173 panic!("{op:?}\n stored …{}\n predicted …{}", &a[a.floor_char_boundary(at.saturating_sub(300))..a.floor_char_boundary((at + 200).min(a.len()))], &b[b.floor_char_boundary(at.saturating_sub(300))..b.floor_char_boundary((at + 200).min(b.len()))]);178 panic!(
179 "{op:?}\n stored …{}\n predicted …{}",
180 &a[a.floor_char_boundary(at.saturating_sub(300))
181 ..a.floor_char_boundary((at + 200).min(a.len()))],
182 &b[b.floor_char_boundary(at.saturating_sub(300))
183 ..b.floor_char_boundary((at + 200).min(b.len()))]
184 );
174 }185 }
175 let space = &after_document.spaces[&sid];186 let space = &after_document.spaces[&sid];
176 let after = &space.revisions[&space.contexts[&ExGuid::default()]];187 let after = &space.revisions[&space.contexts[&ExGuid::default()]];
crates/onestore/tests/table_growth.rs+1-3
...@@ -75,9 +75,7 @@ fn a_text_edit_stores_the_table_entries_it_names() {...@@ -75,9 +75,7 @@ fn a_text_edit_stores_the_table_entries_it_names() {
75 sparse |= (entries as u32) < span;75 sparse |= (entries as u32) < span;
76 }76 }
77 assert_eq!(77 assert_eq!(
78 ops::saved(&written, space, &page)78 ops::saved(&written, space, &page).unwrap().as_slice(),
79 .unwrap()
80 .as_slice(),
81 written79 written
82 );80 );
83 bytes = written;81 bytes = written;
crates/onestore/tests/tree.rs+8-2
...@@ -187,7 +187,10 @@ fn native_subtree_controls_match_with_preserved_fields_and_history() {...@@ -187,7 +187,10 @@ fn native_subtree_controls_match_with_preserved_fields_and_history() {
187 let Some((name, object, edit)) = operation(before, page) else {187 let Some((name, object, edit)) = operation(before, page) else {
188 continue;188 continue;
189 };189 };
190 let edit: Vec<Op> = edit.into_iter().map(|op| Op::Page { space: sid, op }).collect();190 let edit: Vec<Op> = edit
191 .into_iter()
192 .map(|op| Op::Page { space: sid, op })
193 .collect();
191 let prepared = ops::apply(source, "Tree author", edit.clone())194 let prepared = ops::apply(source, "Tree author", edit.clone())
192 .unwrap_or_else(|error| panic!("{name}: {error}"));195 .unwrap_or_else(|error| panic!("{name}: {error}"));
193 let after_store = Store::parse(prepared.as_bytes()).unwrap();196 let after_store = Store::parse(prepared.as_bytes()).unwrap();
...@@ -425,7 +428,10 @@ fn publication_interruptions_expose_complete_old_or_new_trees() {...@@ -425,7 +428,10 @@ fn publication_interruptions_expose_complete_old_or_new_trees() {
425 ) {428 ) {
426 continue;429 continue;
427 }430 }
428 let intent = intent.into_iter().map(|op| Op::Page { space: sid, op }).collect();431 let intent = intent
432 .into_iter()
433 .map(|op| Op::Page { space: sid, op })
434 .collect();
429 let edit = ops::apply(source, "Tree author", intent).unwrap();435 let edit = ops::apply(source, "Tree author", intent).unwrap();
430 let after = current::current(edit.as_bytes());436 let after = current::current(edit.as_bytes());
431 for write_limit in [17, 4096] {437 for write_limit in [17, 4096] {
crates/onestore/tests/versions.rs created+506
...@@ -0,0 +1,506 @@
1//! Page versions against OneNote 2010's own (`corpus/page-versions/native`): listing them,
2//! reading them, and Restore Version and Delete Version storing what OneNote stores.
3
4use onestore::{
5 Arena, ExGuid, ObjectData, PageVersion, PropertySets, RevisionIndex, Section, Store,
6 document::Document,
7 op::{Edit, Op, PageOp, SectionOp},
8 page::{Page, PageObject},
9};
10use std::collections::BTreeSet;
11
12const AT: u64 = 134_040_000_000_000_000;
13const NATIVE: &str = "corpus/page-versions/native";
14/// The version history's context.
15const HISTORY: &str = "{7111497F-1B6B-4209-9491-C98B04CF4C5A},1";
16
17fn read(path: &str) -> Vec<u8> {
18 std::fs::read(format!("{}/../../{path}", env!("CARGO_MANIFEST_DIR"))).unwrap()
19}
20
21fn step(n: u32) -> Vec<u8> {
22 read(&format!("{NATIVE}/step-{n:02}/notebook/History.one"))
23}
24
25fn id(text: &str) -> ExGuid {
26 text.parse().unwrap()
27}
28
29/// The page every step holds, and the version OneNote made when a second author edited it.
30fn page() -> ExGuid {
31 id("{55479BA2-FB3A-45EB-A43D-EDB93DA0593D},1")
32}
33
34fn texts(page: &Page) -> Vec<String> {
35 page.objects
36 .iter()
37 .flat_map(|object| match object {
38 PageObject::Outline(outline) => outline.paragraphs.clone(),
39 _ => Vec::new(),
40 })
41 .filter_map(|paragraph| paragraph.text().map(|text| text.text.text().to_owned()))
42 .collect()
43}
44
45fn versions(image: &[u8]) -> Vec<PageVersion> {
46 let arena = Arena::default();
47 let mut section = Section::open(&arena, image.to_vec()).unwrap();
48 section
49 .versions()
50 .unwrap()
51 .into_iter()
52 .find(|(listed, _)| *listed == page())
53 .map_or(Vec::new(), |(_, versions)| versions)
54}
55
56/// What a revision of `space` changed: its declared objects' types, sorted, with whether
57/// each was new.
58fn changed(index: &RevisionIndex<'_>, space: ExGuid, rid: ExGuid) -> Vec<(u32, bool)> {
59 let revision = index.resolve(space, rid).unwrap();
60 let base = index.spaces[&space].revisions[&rid]
61 .dependency
62 .map(|dependency| index.resolve(space, dependency).unwrap());
63 let mut changed: Vec<(u32, bool)> = revision
64 .objects
65 .iter()
66 .filter_map(|(id, object)| {
67 let old = base.as_ref().and_then(|base| base.objects.get(id));
68 let same = old.is_some_and(|old| {
69 old.reference_count == object.reference_count && old.data == object.data
70 });
71 (!same).then_some((object.jcid, old.is_none()))
72 })
73 .collect();
74 changed.sort();
75 changed
76}
77
78/// Property identities of an object's root set.
79fn properties(object: &onestore::Object<'_>) -> Vec<u32> {
80 let ObjectData::Properties(bytes) = object.data else {
81 panic!()
82 };
83 PropertySets::parse(bytes).unwrap().sets[0]
84 .iter()
85 .map(|property| property.id)
86 .collect()
87}
88
89/// How an edit left the page's revisions, from `before` to `after`: the page revision's
90/// dependency (named by what it was before), the history's, which revisions became versions,
91/// the declared object types of the page and history revisions, the history's children
92/// count and the proxy's properties, and whether the page metadata and the section's copy
93/// say the page has versions.
94#[derive(Debug, PartialEq)]
95struct Shape {
96 page_depends_on: &'static str,
97 history_depends_on_previous: bool,
98 labelled: Vec<&'static str>,
99 page_changed: Vec<(u32, bool)>,
100 history_changed: Vec<(u32, bool)>,
101 listed: usize,
102 proxy: Vec<u32>,
103 flagged: [bool; 2],
104}
105
106fn shape(before: &[u8], after: &[u8]) -> Shape {
107 let (old_store, new_store) = (Store::parse(before).unwrap(), Store::parse(after).unwrap());
108 let (old, new) = (
109 RevisionIndex::parse(&old_store).unwrap(),
110 RevisionIndex::parse(&new_store).unwrap(),
111 );
112 new.validate_current().unwrap();
113 Document::parse(&new).unwrap();
114 let space = page();
115 let history = id(HISTORY);
116 let (old_labels, new_labels) = (&old.spaces[&space].labels, &new.spaces[&space].labels);
117 let old_page = old_labels[&(ExGuid::default(), 1)];
118 let new_page = new_labels[&(ExGuid::default(), 1)];
119 let name = |rid: ExGuid| -> &'static str {
120 if rid == old_page {
121 "the page"
122 } else if old_labels.get(&(history, 1)) == Some(&rid) {
123 "the history"
124 } else if old_labels.iter().any(|((context, role), labelled)| {
125 *role == 1 && *context != ExGuid::default() && *labelled == rid
126 }) {
127 "a version"
128 } else {
129 "another revision"
130 }
131 };
132 let page_depends_on = if new_page == old_page {
133 "unchanged"
134 } else {
135 new.spaces[&space].revisions[&new_page]
136 .dependency
137 .map_or("none", name)
138 };
139 let (old_history, new_history) = (old_labels[&(history, 1)], new_labels[&(history, 1)]);
140 let labelled = new_labels
141 .iter()
142 .filter(|((context, role), _)| {
143 *role == 1
144 && *context != ExGuid::default()
145 && *context != history
146 && !old_labels.contains_key(&(*context, 1))
147 })
148 .map(|(_, rid)| name(*rid))
149 .collect();
150 let history_revision = new.resolve(space, new_history).unwrap();
151 let root = &history_revision.objects[&history_revision.roots[&1]];
152 let listed = root.references().unwrap().objects;
153 let proxy = listed.first().map_or(Vec::new(), |proxy| {
154 properties(&history_revision.objects[proxy])
155 });
156 let metadata = |index: &RevisionIndex<'_>| -> [bool; 2] {
157 let revision = index.resolve_active(space).unwrap();
158 let own = properties(&revision.objects[&revision.roots[&2]]).contains(&0x88003462);
159 // The section's copy, named by the page's space (MS-ONE 2.2.81).
160 let section = index.resolve_active(index.root).unwrap();
161 let copy = properties(&section.objects[&id("{77EF5B93-CD3A-0705-1329-3A15E7846CD5},1")])
162 .contains(&0x88003462);
163 [own, copy]
164 };
165 Shape {
166 page_depends_on,
167 history_depends_on_previous: new_history != old_history
168 && new.spaces[&space].revisions[&new_history].dependency == Some(old_history),
169 labelled,
170 page_changed: if new_page == old_page {
171 Vec::new()
172 } else {
173 changed(&new, space, new_page)
174 },
175 history_changed: if new_history == old_history {
176 Vec::new()
177 } else {
178 changed(&new, space, new_history)
179 },
180 listed: listed.len(),
181 proxy,
182 flagged: metadata(&new),
183 }
184}
185
186fn apply(image: &[u8], author: &str, ops: Vec<Op>) -> Vec<u8> {
187 let arena = Arena::default();
188 let mut section = Section::open(&arena, image.to_vec()).unwrap();
189 section.apply(author, &Edit { at: AT, ops }).unwrap();
190 section.seal().unwrap().unwrap();
191 section.image()
192}
193
194#[test]
195fn native_versions_are_listed_newest_first_with_their_authors() {
196 assert_eq!(versions(&step(1)), []);
197 let virtual_version = PageVersion {
198 context: id("{2F6B7C6D-DF95-0F56-085E-74357323A760},1"),
199 modified: Some(0x01dd4ee2641a3f00),
200 author: Some("virtual".into()),
201 };
202 assert_eq!(versions(&step(2)), std::slice::from_ref(&virtual_version));
203 // A Restore Version made the page as it stood the newest version.
204 let restored = versions(&step(4));
205 assert_eq!(
206 restored,
207 [
208 PageVersion {
209 context: id("{783B5BD2-65DB-082D-3AAC-47135BA38EA9},1"),
210 modified: Some(0x01dd4ee27e541d00),
211 author: Some("Other Person".into()),
212 },
213 virtual_version.clone(),
214 ]
215 );
216 assert_eq!(versions(&step(5)), [virtual_version]);
217 assert_eq!(versions(&step(6)), []);
218 // Optimizing keeps a version and the history as checkpoints.
219 assert_eq!(versions(&step(10)).len(), 1);
220}
221
222#[test]
223fn a_version_reads_as_the_page_it_was() {
224 let arena = Arena::default();
225 let section = Section::open(&arena, step(1)).unwrap();
226 let before = section.page(page()).unwrap();
227 let arena = Arena::default();
228 let mut section = Section::open(&arena, step(2)).unwrap();
229 let context = section.versions().unwrap()[0].1[0].context;
230 let version = section.version(page(), context).unwrap();
231 assert_eq!(version, before);
232 assert_eq!(texts(&version), ["First state. Typed by virtual."]);
233 assert_eq!(
234 texts(&section.page(page()).unwrap()),
235 ["First state. Typed by virtual.", "Second author line."]
236 );
237 assert!(section.version(page(), id(HISTORY)).is_err());
238}
239
240/// Restore Version stores what OneNote 2010's does (`step-03` to `step-04`): the page's next
241/// revision builds on the version's, the page as it stood becomes the newest version under
242/// the context its revision derives, the history lists it first, and the revision metadata
243/// names who restored it. `ONESTORE_VERSIONS_EXPORT` names a directory for the notebooks,
244/// for a cold read.
245#[test]
246fn restoring_a_version_stores_what_onenote_stores() {
247 let before = step(3);
248 let restored_version = versions(&before)[0].context;
249 let arena = Arena::default();
250 let section = Section::open(&arena, before.clone()).unwrap();
251 let standing = section.page(page()).unwrap();
252 let version = section.version(page(), restored_version).unwrap();
253 let ours = apply(
254 &before,
255 "Other Person",
256 vec![Op::Section(SectionOp::RestoreVersion {
257 page: page(),
258 version: restored_version,
259 guid: [7; 16],
260 })],
261 );
262 let expected = shape(&before, &step(4));
263 assert_eq!(expected.page_depends_on, "a version");
264 assert_eq!(shape(&before, &ours), expected);
265
266 let arena = Arena::default();
267 let mut reopened = Section::open(&arena, ours.clone()).unwrap();
268 assert_eq!(reopened.page(page()).unwrap().objects, version.objects);
269 let listed = reopened.versions().unwrap().remove(0).1;
270 assert_eq!(listed.len(), 2);
271 assert_eq!(listed[1].context, restored_version);
272 // As OneNote names it (`step-04`): the page's revision, its bits flipped by a salt.
273 let store = Store::parse(&before).unwrap();
274 let standing_rid = RevisionIndex::parse(&store)
275 .unwrap()
276 .active(page())
277 .unwrap();
278 let salt = |a: ExGuid, b: ExGuid| -> Vec<u8> {
279 a.guid.iter().zip(b.guid).map(|(a, b)| a ^ b).collect()
280 };
281 assert_eq!(
282 salt(listed[0].context, standing_rid),
283 salt(
284 id("{783B5BD2-65DB-082D-3AAC-47135BA38EA9},1"),
285 id("{E569A5AE-EA67-4CB8-A9AC-55880C342743},1")
286 )
287 );
288 assert_eq!(listed[0].author.as_deref(), Some("Other Person"));
289 assert_eq!(
290 reopened.version(page(), listed[0].context).unwrap(),
291 standing
292 );
293
294 if let Some(output) = std::env::var_os("ONESTORE_VERSIONS_EXPORT") {
295 export(&output, "restore", &ours);
296 }
297}
298
299/// Delete Version stores what OneNote 2010's does: the history stops listing the version
300/// (`step-04` to `step-05`), and once none is left the page's metadata and the section's copy
301/// stop saying it has versions (`step-05` to `step-06`).
302#[test]
303fn deleting_versions_stores_what_onenote_stores() {
304 let before = step(4);
305 let newest = versions(&before)[0].context;
306 let delete = |image: &[u8], versions: Vec<ExGuid>| {
307 apply(
308 image,
309 "Other Person",
310 vec![Op::Section(SectionOp::DeleteVersions {
311 page: page(),
312 versions,
313 })],
314 )
315 };
316 let ours = delete(&before, vec![newest]);
317 assert_eq!(shape(&before, &ours), shape(&before, &step(5)));
318 let emptied = delete(&step(5), vec![versions(&step(5))[0].context]);
319 let expected = shape(&step(5), &step(6));
320 assert_eq!(expected.flagged, [false, false]);
321 assert_eq!(shape(&step(5), &emptied), expected);
322 // Every version at once, as Delete All Versions in Section asks.
323 let all = versions(&before)
324 .into_iter()
325 .map(|version| version.context)
326 .collect();
327 assert_eq!(versions(&delete(&before, all)), []);
328 // A version already gone refuses the edit.
329 let arena = Arena::default();
330 let mut section = Section::open(&arena, step(5)).unwrap();
331 let refused = section.apply(
332 "Other Person",
333 &Edit {
334 at: AT,
335 ops: vec![Op::Section(SectionOp::DeleteVersions {
336 page: page(),
337 versions: vec![newest],
338 })],
339 },
340 );
341 assert_eq!(
342 refused,
343 Err(onestore::op::OpError::TargetUnavailable(newest))
344 );
345
346 if let Some(output) = std::env::var_os("ONESTORE_VERSIONS_EXPORT") {
347 export(&output, "delete", &ours);
348 export(&output, "delete-all", &emptied);
349 }
350}
351
352/// Edits not yet sealed become a version of their own when a restore follows them in the
353/// same batch, and restores stack: every state the page passed through stays listed.
354#[test]
355fn unsaved_edits_and_stacked_restores_become_versions() {
356 let before = step(2);
357 let arena = Arena::default();
358 let mut section = Section::open(&arena, before.clone()).unwrap();
359 let first = section.versions().unwrap()[0].1[0].context;
360 let text = section
361 .page(page())
362 .unwrap()
363 .objects
364 .iter()
365 .find_map(|object| match object {
366 PageObject::Outline(outline) => outline.paragraphs.last()?.text().map(|text| text.id),
367 _ => None,
368 });
369 let edit = |ops| Edit { at: AT, ops };
370 section
371 .apply(
372 "Clover Snow",
373 &edit(vec![Op::Page {
374 space: page(),
375 op: PageOp::Text {
376 text: text.unwrap(),
377 range: 0..6,
378 with: "Unsaved".into(),
379 },
380 }]),
381 )
382 .unwrap();
383 let edited = section.page(page()).unwrap();
384 let restore = |version, guid| {
385 edit(vec![Op::Section(SectionOp::RestoreVersion {
386 page: page(),
387 version,
388 guid,
389 })])
390 };
391 section
392 .apply("Clover Snow", &restore(first, [1; 16]))
393 .unwrap();
394 let listed = section.versions().unwrap()[0].1.clone();
395 assert_eq!(listed.len(), 2);
396 assert_eq!(section.version(page(), listed[0].context).unwrap(), edited);
397 // The unsaved state is still being saved, so it cannot be restored yet; the stored one can.
398 assert!(
399 section
400 .apply("Clover Snow", &restore(listed[0].context, [2; 16]))
401 .is_err()
402 );
403 section
404 .apply("Clover Snow", &restore(first, [3; 16]))
405 .unwrap();
406 assert_eq!(section.versions().unwrap()[0].1.len(), 3);
407 section.seal().unwrap().unwrap();
408 let image = section.image();
409
410 let arena = Arena::default();
411 let mut reopened = Section::open(&arena, image.clone()).unwrap();
412 let store = Store::parse(&image).unwrap();
413 let index = RevisionIndex::parse(&store).unwrap();
414 index.validate_current().unwrap();
415 Document::parse(&index).unwrap();
416 let listed = reopened.versions().unwrap().remove(0).1;
417 assert_eq!(listed.len(), 3);
418 let pages: Vec<Page> = listed
419 .iter()
420 .map(|version| reopened.version(page(), version.context).unwrap())
421 .collect();
422 let version = reopened.version(page(), first).unwrap();
423 assert_eq!(pages[1], edited);
424 assert_eq!(pages[0].objects, version.objects);
425 assert_eq!(reopened.page(page()).unwrap().objects, version.objects);
426 // The restored page edits on as any other.
427 let text = texts(&reopened.page(page()).unwrap());
428 assert_eq!(text, ["First state. Typed by virtual."]);
429}
430
431/// An edit that fails after a restore leaves the section as it was before the edit.
432#[test]
433fn a_failed_edit_undoes_its_restore() {
434 let before = step(2);
435 let arena = Arena::default();
436 let mut section = Section::open(&arena, before.clone()).unwrap();
437 let first = section.versions().unwrap()[0].1[0].context;
438 let page_before = section.page(page()).unwrap();
439 let refused = section.apply(
440 "Clover Snow",
441 &Edit {
442 at: AT,
443 ops: vec![
444 Op::Section(SectionOp::RestoreVersion {
445 page: page(),
446 version: first,
447 guid: [1; 16],
448 }),
449 Op::Section(SectionOp::DeleteVersions {
450 page: page(),
451 versions: vec![id(HISTORY)],
452 }),
453 ],
454 },
455 );
456 assert!(refused.is_err());
457 assert_eq!(section.page(page()).unwrap(), page_before);
458 assert_eq!(section.versions().unwrap()[0].1.len(), 1);
459 assert!(section.seal().unwrap().is_none());
460 assert!(section.image() == before);
461}
462
463#[test]
464fn private_notebook_versions_read() {
465 let path = format!(
466 "{}/../../corpus/private/current/Art/Album.one",
467 env!("CARGO_MANIFEST_DIR")
468 );
469 let Ok(image) = std::fs::read(path) else {
470 return;
471 };
472 let arena = Arena::default();
473 let mut section = Section::open(&arena, image).unwrap();
474 let listed = section.versions().unwrap();
475 assert!(
476 listed
477 .iter()
478 .map(|(_, versions)| versions.len())
479 .sum::<usize>()
480 >= 10
481 );
482 let contexts: BTreeSet<ExGuid> = listed
483 .iter()
484 .flat_map(|(_, versions)| versions.iter().map(|version| version.context))
485 .collect();
486 assert_eq!(
487 contexts.len(),
488 listed.iter().map(|(_, v)| v.len()).sum::<usize>()
489 );
490 for (page, versions) in &listed {
491 for version in versions {
492 section.version(*page, version.context).unwrap();
493 }
494 }
495}
496
497fn export(output: &std::ffi::OsStr, name: &str, image: &[u8]) {
498 let directory = std::path::Path::new(output).join(name);
499 std::fs::create_dir_all(&directory).unwrap();
500 std::fs::write(directory.join("History.one"), image).unwrap();
501 std::fs::write(
502 directory.join("Open Notebook.onetoc2"),
503 read(&format!("{NATIVE}/step-01/notebook/Open Notebook.onetoc2")),
504 )
505 .unwrap();
506}
tools/TESTING.md+10-4
...@@ -2,10 +2,12 @@...@@ -2,10 +2,12 @@
22
3## Public fixtures3## Public fixtures
44
5From a checkout with Rust and Python 3 plus Pillow installed:5From a checkout with Rust and [uv](https://docs.astral.sh/uv/), which provides
6Python 3.12 with Pillow and pdfplumber without installing anything system-wide:
67
7```sh8```sh
8python3 tools/check_public.py /absolute/path/to/new-results9uv run --no-project --python 3.12 --with pillow --with pdfplumber \
10 python tools/check_public.py /absolute/path/to/new-results
9```11```
1012
11The command runs formatting, all workspace features/targets, doctests, Clippy,13The command runs formatting, all workspace features/targets, doctests, Clippy,
...@@ -22,6 +24,11 @@ targets are versioned in this repository. Neither `corpus/private`, ignored...@@ -22,6 +24,11 @@ targets are versioned in this repository. Neither `corpus/private`, ignored
22`evidence`, existing `target` outputs, credentials nor running virtual machines24`evidence`, existing `target` outputs, credentials nor running virtual machines
23are required. Cargo's normal dependency cache may be reused.25are required. Cargo's normal dependency cache may be reused.
2426
27For the Python tests alone, end the same `uv run` with
28`python -m unittest discover -s tools -p 'test_*.py'`. Without pdfplumber they
29skip the PDF oracles (`test_pdf_format`, part of `test_document_oracle`) and
30say so; `check_public.py` refuses to start without it.
31
25Rust explicitly reports ignored lab tests and fixture generators. Those cases32Rust explicitly reports ignored lab tests and fixture generators. Those cases
26are **not** part of a successful public run. The Python suite tests native/lab33are **not** part of a successful public run. The Python suite tests native/lab
27harness logic using retained synthetic captures and mocks; it does not claim a34harness logic using retained synthetic captures and mocks; it does not claim a
...@@ -40,8 +47,7 @@ debug assertions, then every `fuzz/` target for the time limit. A failing round...@@ -40,8 +47,7 @@ debug assertions, then every `fuzz/` target for the time limit. A failing round
40leaves `soak/*.log` ending in the command that replays it; crash inputs stay in47leaves `soak/*.log` ending in the command that replays it; crash inputs stay in
41`fuzz/artifacts/<target>/`. On a Linux VM, install rustup's stable and nightly48`fuzz/artifacts/<target>/`. On a Linux VM, install rustup's stable and nightly
42toolchains, `cargo install cargo-fuzz`, and `build-essential pkg-config49toolchains, `cargo install cargo-fuzz`, and `build-essential pkg-config
43libfontconfig-dev`; copy the checkout together with the untracked `evidence/`,50libfontconfig-dev`; copy the checkout and run the script under `tmux`.
44which tests read, and run the script under `tmux`.
4551
46## Private and native verification52## Private and native verification
4753
tools/canvas/README.md+1-1
...@@ -48,7 +48,7 @@ snowbound title bar · toolbar · section frame · page tabs · the page as a...@@ -48,7 +48,7 @@ snowbound title bar · toolbar · section frame · page tabs · the page as a
4848
49## Desktop shell49## Desktop shell
5050
51`--notebook FOLDER` opens a notebook laid out as OneNote lays one out. The title bar shows the application's icon and the notebook's name; below it two rows of toolbar buttons follow the Home ribbon's Basic Text group, the first nine tags and zoom (only zoom acts yet). Readable top-level sections are tabs in the notebook's order, each leaning over the next at 45° with a faint highlight along its top, and the open one on top, casting a soft shadow and framing the page and the page tabs on the right in its colour, which shades slightly down the window; a tab that opens rises to meet the frame as its outline fades in. The frame's top corners and the page's corners are rounded concentric with the window's. The open page's tab is the page's colour and joins it with rounded inside corners; subpages are indented. The interface follows the system's light or dark appearance, and in the dark one the page's paper darkens and text left at OneNote's automatic colour turns light (except on a highlight), resolved when painting so nothing re-lays out; explicit colours, highlights, pictures and ink keep theirs, except that the background art of OneNote 2010's page templates, recognised by the picture's hash, paints from bundled vector recreations drawn for the paper. Sections take their stored colour's hue at a saturation and lightness chosen for each appearance. `--ui-font FONT_FILE_OR_FOLDER` sets the interface's typeface in place of the system's; fonts are not bundled. The section's colour is also the interface accent, easing to the next section's when another opens. The search box, New Page and the page-tab toggle sit above the page tabs. Command-F searches the page titles; Enter opens the first match and Escape clears it. A page that also changed on another computer shows Keep mine / Keep theirs above it. `--section FILE TITLE` opens one section the same way at that page. Section files must be regular files; discovery rejects symbolic links.51`--notebook FOLDER` opens a notebook laid out as OneNote lays one out. The title bar shows the application's icon and the notebook's name; below it two rows of toolbar buttons follow the Home ribbon's Basic Text group, the first nine tags and zoom (only zoom acts yet). Readable top-level sections are tabs in the notebook's order, each leaning over the next at 45° with a faint highlight along its top, and the open one on top, casting a soft shadow and framing the page and the page tabs on the right in its colour, which shades slightly down the window; a tab that opens rises to meet the frame as its outline fades in. The frame's top corners and the page's corners are rounded concentric with the window's. The open page's tab is the page's colour and joins it with rounded inside corners; subpages are indented. The interface follows the system's light or dark appearance, and in the dark one the page's paper darkens and text left at OneNote's automatic colour turns light (except on a highlight), resolved when painting so nothing re-lays out; explicit colours, highlights, pictures and ink keep theirs, except that the background art of OneNote 2010's page templates, recognised by the picture's hash, paints from bundled vector recreations drawn for the paper. Sections take their stored colour's hue at a saturation and lightness chosen for each appearance. The section's colour is also the interface accent, easing to the next section's when another opens. The search box, New Page and the page-tab toggle sit above the page tabs. Command-F searches the page titles; Enter opens the first match and Escape clears it. A page that also changed on another computer shows Keep mine / Keep theirs above it. `--section FILE TITLE` opens one section the same way at that page. Section files must be regular files; discovery rejects symbolic links.
5252
53Launched without a file, the app reopens the notebooks open when it last closed, at the one shown last, or shows "No notebooks open" with New Notebook and Open Existing. Settings (the open notebooks, the sidebar, recent fonts and the user name edits are stored under, which Settings… (⌘,; Options in the sidebar and Ctrl+, on Linux) changes and which starts as the account's full name) are JSON in `~/Library/Application Support/Snowbound/settings.json` or `$XDG_CONFIG_HOME/snowbound`; `--settings FILE` uses another file, `--notebook FOLDER` adds a notebook to them, and `--screenshot` never writes them. Open Existing takes a notebook folder, its `Open Notebook.onetoc2` or a section file, which opens in the notebook its folders make up. A notebook on a mounted SMB share opens through the embedded SMB client, signed in with the password the system keychain holds, and through the mount only when that fails. The notebook button left of the section tabs opens OneNote 2010's navigation bar: the open notebooks with their sections and section groups, the open section marked; dragging a row reorders a folder or, onto a group, moves into it, and right-clicking a row, a section tab or a page tab offers OneNote's commands (Rename, Delete to the notebook's recycle bin, New Section, New Section Group, Close This Notebook; Delete, New Page, Make Subpage, Promote Subpage). Page tabs drag to reorder. The `+` above the page tabs adds a page at the end of the section, titled with its date and time as OneNote titles one, and puts the caret in its title; until its body is typed in, templates offer our recreations of OneNote 2010's art (Ivy first), its page colours and Informal Meeting Notes' content, written as ops (`corpus/notebook-management`).53Launched without a file, the app reopens the notebooks open when it last closed, at the one shown last, or shows "No notebooks open" with New Notebook and Open Existing. Settings (the open notebooks, the sidebar, recent fonts and the user name edits are stored under, which Settings… (⌘,; Options in the sidebar and Ctrl+, on Linux) changes and which starts as the account's full name) are JSON in `~/Library/Application Support/Snowbound/settings.json` or `$XDG_CONFIG_HOME/snowbound`; `--settings FILE` uses another file, `--notebook FOLDER` adds a notebook to them, and `--screenshot` never writes them. Open Existing takes a notebook folder, its `Open Notebook.onetoc2` or a section file, which opens in the notebook its folders make up. A notebook on a mounted SMB share opens through the embedded SMB client, signed in with the password the system keychain holds, and through the mount only when that fails. The notebook button left of the section tabs opens OneNote 2010's navigation bar: the open notebooks with their sections and section groups, the open section marked; dragging a row reorders a folder or, onto a group, moves into it, and right-clicking a row, a section tab or a page tab offers OneNote's commands (Rename, Delete to the notebook's recycle bin, New Section, New Section Group, Close This Notebook; Delete, New Page, Make Subpage, Promote Subpage). Page tabs drag to reorder. The `+` above the page tabs adds a page at the end of the section, titled with its date and time as OneNote titles one, and puts the caret in its title; until its body is typed in, templates offer our recreations of OneNote 2010's art (Ivy first), its page colours and Informal Meeting Notes' content, written as ops (`corpus/notebook-management`).
5454
tools/check_public.py+1-1
...@@ -22,7 +22,7 @@ def main():...@@ -22,7 +22,7 @@ def main():
22 environment['PYTHONPATH'] = str(root / 'tools')22 environment['PYTHONPATH'] = str(root / 'tools')
23 environment['CARGO_TARGET_DIR'] = str(root / 'target')23 environment['CARGO_TARGET_DIR'] = str(root / 'target')
24 commands = [24 commands = [
25 ('python-dependencies', [sys.executable, '-c', 'import PIL']),25 ('python-dependencies', [sys.executable, '-c', 'import PIL, pdfplumber']),
26 ('format', ['cargo', 'fmt', '--all', '--check']),26 ('format', ['cargo', 'fmt', '--all', '--check']),
27 ('rust', ['cargo', 'test', '--locked', '--workspace', '--all-features', '--all-targets']),27 ('rust', ['cargo', 'test', '--locked', '--workspace', '--all-features', '--all-targets']),
28 ('doctests', ['cargo', 'test', '--locked', '--workspace', '--all-features', '--doc']),28 ('doctests', ['cargo', 'test', '--locked', '--workspace', '--all-features', '--doc']),
tools/document_model.py+2-1
...@@ -49,7 +49,8 @@ def ordered_pages(document):...@@ -49,7 +49,8 @@ def ordered_pages(document):
49def version_pages(document, sid, revision):49def version_pages(document, sid, revision):
50 manifest = revision['nodes'][revision['roots']['1']]['kind']50 manifest = revision['nodes'][revision['roots']['1']]['kind']
51 context = manifest['history']51 context = manifest['history']
52 if context is None:52 # A page OneNote copied from a version names a history it never wrote.
53 if context not in document['spaces'][sid]['contexts']:
53 return54 return
54 _, history = view(document, sid, context)55 _, history = view(document, sid, context)
55 root = history['nodes'][history['roots']['1']]56 root = history['nodes'][history['roots']['1']]
tools/native_enter.py created+67
...@@ -0,0 +1,67 @@
1"""Press Enter at the edges of and inside equations and links in OneNote 2010 on a
2Rust-written page and capture what it stores. The oracle for the canvas editor's Enter.
3
4Usage: native_enter.py NOTEBOOK_DIR OUTPUT_DIR
5"""
6import base64
7import json
8import sys
9from pathlib import Path
10import xml.etree.ElementTree as ET
11
12sys.path.insert(0, str(Path(__file__).resolve().parent))
13import native_runner as runner
14
15windows = runner.windows
16
17
18def text(value):
19 return 'SendText("%s")' % value.replace('"', '""')
20
21
22def interaction(name, output):
23 lines = [
24 # Enter at an equation's start moves it to a paragraph of its own.
25 text('Before '), 'Send("!=")', 'Sleep(600)', text('y^2+1'), 'Send(" ")', 'Sleep(700)',
26 'Send("{Home}")', 'Send("{Right 7}")', 'Sleep(300)', 'Send("{Enter}")', 'Sleep(600)',
27 # Enter inside the equation's own row breaks its line.
28 'Send("{End}")', 'Send("{Left 2}")', 'Sleep(300)', 'Send("{Enter}")', 'Sleep(600)',
29 # Enter inside a fraction's denominator makes it an equation array.
30 'Send("{End}{Enter}")', 'Send("!=")', 'Sleep(600)', text('(a+b)/(c+d)'), 'Send(" ")',
31 'Sleep(700)', 'Send("{Left 3}")', 'Sleep(300)', 'Send("{Enter}")', 'Sleep(600)',
32 # Enter at a link's start moves the link down whole (inside one, it follows the link).
33 'Send("^{End}{Enter}")', text('Select me please'), 'Send("+^{Left}+^{Left}")', 'Send("^k")',
34 'Sleep(1500)', text('https://example.com/selected'), 'Send("{Enter}")', 'Sleep(800)',
35 'Send("{End}")', 'Send("{Left 9}")', 'Sleep(300)', 'Send("{Enter}")', 'Sleep(600)',
36 ]
37 page = ET.parse(sorted((output / 'read').glob('page-*.xml'))[0]).getroot().attrib['ID']
38 script = '''ComObject("OneNote.Application").NavigateTo("PAGE_ID", "", false)
39WinWait("ahk_exe ONENOTE.EXE", , 30)
40WinActivate("ahk_exe ONENOTE.EXE")
41WinWaitActive("ahk_exe ONENOTE.EXE", , 30)
42WinMaximize("ahk_exe ONENOTE.EXE")
43Sleep(3000)
44CoordMode("Mouse", "Screen")
45Click(200, 140)
46Sleep(500)
47Send("{End}")
48Sleep(300)
49Send("{Enter}")
50Sleep(500)
51%s
52Sleep(2000)
53''' % '\n'.join(lines)
54 script = script.replace('PAGE_ID', page)
55 (output / 'enter.ahk').write_text(script)
56 result = windows.do_exec(script, target=name, timeout_ms=240000, shot_delay_ms=1500)
57 screenshot = result.pop('png_b64', None)
58 if screenshot:
59 (output / 'enter.png').write_bytes(base64.b64decode(screenshot))
60 (output / 'enter.json').write_text(json.dumps(result, indent=2))
61 if result.get('error') or result.get('exit') != 0:
62 raise RuntimeError(str(result))
63
64
65if __name__ == '__main__':
66 notebook, output = Path(sys.argv[1]), Path(sys.argv[2])
67 runner.capture(notebook, output, expected_pages=1, collect_notebook=True, interaction=interaction)
tools/native_format.py+2-1
...@@ -100,7 +100,8 @@ def compare_formats(space, nodes, native):...@@ -100,7 +100,8 @@ def compare_formats(space, nodes, native):
100 # Hidden runs and equation runs (exported as MathML) have no visible native text.100 # Hidden runs and equation runs (exported as MathML) have no visible native text.
101 if not style.get('hidden') and not style.get('math'):101 if not style.get('hidden') and not style.get('math'):
102 actual.extend((char, style) for char in text)102 actual.extend((char, style) for char in text)
103 if actual and actual[-1][0] == '\r':103 # A trailing CR ends the stored text; one between equations is a line break.
104 if actual and actual[-1][0] == '\r' and kind['text'].endswith('\r'):
104 actual.pop()105 actual.pop()
105 actual = [(projected, style) for char, style in actual for projected in project_text(char)]106 actual = [(projected, style) for char, style in actual for projected in project_text(char)]
106 if kind['text'] == '\u00a0' and not expected:107 if kind['text'] == '\u00a0' and not expected:
tools/native_links.py created+94
...@@ -0,0 +1,94 @@
1"""Type hyperlinks into OneNote 2010 on a Rust-written page and capture what it stores: URLs
2it links as they are typed, links made with the Link dialog (Ctrl+K), typing after a link and
3Remove Link. The oracle for the canvas editor's link editing.
4
5Usage: native_links.py NOTEBOOK_DIR OUTPUT_DIR
6"""
7import base64
8import json
9import sys
10from pathlib import Path
11import xml.etree.ElementTree as ET
12
13sys.path.insert(0, str(Path(__file__).resolve().parent))
14import native_runner as runner
15
16windows = runner.windows
17
18# Each typed URL sits between a word and a letter, one per paragraph; a space or Enter ends it.
19TYPED = [
20 'www.example.com',
21 'http://example.org/a?b=1',
22 'https://a.example/p.',
23 '(http://b.example/q)',
24 'ftp://c.example/f',
25 'mailto:me@example.com',
26 'me@example.com',
27 'example.com',
28 'onenote:#Page&section-id={x}',
29 'file://c:/windows',
30 '\\\\server\\share\\f',
31 'news:comp.lang',
32 'www.d.example,',
33 'HTTP://E.EXAMPLE/UP',
34 'https://f.example/a_b-c~d%20e#frag',
35]
36
37
38def text(value):
39 return 'SendText("%s")' % value.replace('"', '""')
40
41
42def interaction(name, output):
43 lines = []
44 for url in TYPED:
45 lines += [text('t ' + url), 'Send(" ")', text('z'), 'Send("{Enter}")', 'Sleep(300)']
46 # A URL ended by Enter, then a letter typed straight after the link it made.
47 lines += [text('see http://g.example/end'), 'Send("{Enter}")', 'Send("{Up}{End}")', text('Z'),
48 'Send("{Down}{End}")', 'Sleep(300)']
49 # The Link dialog on the word at the caret: its text is the word, the address as typed.
50 lines += [text('plain words here'), 'Send("^{Left}")', 'Send("^k")', 'Sleep(1500)',
51 text('example.net/x'), 'Send("{Enter}")', 'Sleep(800)', 'Send("{End}")', text('Q'),
52 'Send("{Enter}")', 'Sleep(300)']
53 # The Link dialog on a selection.
54 lines += [text('select me please'), 'Send("+^{Left}+^{Left}")', 'Send("^k")', 'Sleep(1500)',
55 text('https://example.com/selected'), 'Send("{Enter}")', 'Sleep(800)',
56 'Send("{End}{Enter}")', 'Sleep(300)']
57 # The Link dialog with no text: the address becomes the text; typing after it is plain.
58 lines += ['Send("^k")', 'Sleep(1500)', 'Send("!e")', 'Sleep(300)',
59 text('https://example.com/only'), 'Send("{Enter}")', 'Sleep(800)', text(' tail'),
60 'Send("{Enter}")', 'Sleep(300)']
61 # Remove Link from the context menu (Shift+F10, R) with the caret inside a typed link.
62 lines += [text('gone ftp://h.example/f'), 'Send(" ")', text('z'), 'Send("{Home}^{Right}^{Right}")',
63 'Send("+{F10}")', 'Sleep(900)', 'Send("r")', 'Sleep(800)']
64 page = ET.parse(sorted((output / 'read').glob('page-*.xml'))[0]).getroot().attrib['ID']
65 script = '''ComObject("OneNote.Application").NavigateTo("PAGE_ID", "", false)
66WinWait("ahk_exe ONENOTE.EXE", , 30)
67WinActivate("ahk_exe ONENOTE.EXE")
68WinWaitActive("ahk_exe ONENOTE.EXE", , 30)
69WinMaximize("ahk_exe ONENOTE.EXE")
70Sleep(3000)
71CoordMode("Mouse", "Screen")
72Click(200, 140)
73Sleep(500)
74Send("{End}")
75Sleep(300)
76Send("{Enter}")
77Sleep(500)
78%s
79Sleep(2000)
80''' % '\n'.join(lines)
81 script = script.replace('PAGE_ID', page)
82 (output / 'links.ahk').write_text(script)
83 result = windows.do_exec(script, target=name, timeout_ms=240000, shot_delay_ms=1500)
84 screenshot = result.pop('png_b64', None)
85 if screenshot:
86 (output / 'links.png').write_bytes(base64.b64decode(screenshot))
87 (output / 'links.json').write_text(json.dumps(result, indent=2))
88 if result.get('error') or result.get('exit') != 0:
89 raise RuntimeError(str(result))
90
91
92if __name__ == '__main__':
93 notebook, output = Path(sys.argv[1]), Path(sys.argv[2])
94 runner.capture(notebook, output, expected_pages=1, collect_notebook=True, interaction=interaction)
tools/native_versions.py created+219
...@@ -0,0 +1,219 @@
1#!/usr/bin/env python3
2"""OneNote 2010 page versions (Share, Page Versions): how they arise, what they store and
3how OneNote shows, restores, copies and deletes them.
4
5`observe OUTPUT` runs one disposable OneNote client on a new notebook: `virtual` writes a
6page, `Other Person` edits it (OneNote keeps the page as it was as a version), then OneNote's
7Copy Page To, Restore Version, Delete Version and Delete All Versions in Section run on it;
8two more authors' edits, a Delete Version of the only version and the author views (Hide
9Authors, Recent Edits, Find by Author) follow. The section after each step is
10`step-NN/notebook`, the screens `shots/`.
11
12`cold NOTEBOOK OUTPUT [--delete]` cold-opens a notebook whose page has versions in a fresh
13clone, shows them and opens the newest, screenshots each view, and with `--delete` deletes
14that version through its bar's menu and keeps the notebook OneNote leaves.
15"""
16import argparse
17import base64
18import json
19from pathlib import Path
20import sys
21import time
22import xml.etree.ElementTree as ET
23
24sys.path.insert(0, str(Path(__file__).resolve().parent))
25sys.path.insert(0, str(Path(__file__).resolve().parent / 'w7'))
26import native_runner as runner
27from native_runner import windows
28
29ROOT = Path(r'C:\one-tests\versions')
30SECTION = ROOT / 'Versions' / 'History.one'
31
32# Coordinates on OneNote 2010 maximized at 800x600 with its ribbon collapsed, from the
33# capture in corpus/page-versions/native: the Share tab, its Page Versions button and the
34# arrow under it, the information bar, the first body line, the first version listed under
35# the page and the page itself in the list.
36SHARE = (202, 36)
37PAGE_VERSIONS = (443, 80)
38PAGE_VERSIONS_MENU = (462, 108)
39DELETE_ALL_IN_SECTION = (525, 155)
40BAR = (320, 93)
41BODY = (160, 208)
42FIRST_VERSION = (710, 143)
43THE_PAGE = (700, 121)
44HIDE_AUTHORS = (383, 88)
45RECENT_EDITS = (296, 108)
46FIND_BY_AUTHOR = (338, 95)
47# Items of the version bar's menu.
48RESTORE, DELETE, COPY = (396, 106), (393, 128), (395, 150)
49
50# Reports any dialog OneNote raised, so a confirmation shows in the step's record.
51DIALOG = ('if WinExist("ahk_class #32770 ahk_exe ONENOTE.EXE") {\n'
52 ' FileAppend("dialog: " WinGetTitle() " | " WinGetText() "`n", "*", "UTF-8")\n'
53 '}\n')
54ACTIVATE = 'WinActivate("ahk_exe ONENOTE.EXE")\nCoordMode("Mouse", "Screen")\n'
55
56
57def click(point, wait=1000):
58 return f'Click({point[0]}, {point[1]})\nSleep({wait})\n'
59
60
61class Clone:
62 """A clone's desktop: `shot` runs an AutoHotkey step and keeps its screenshot."""
63
64 def __init__(self, name, output):
65 self.name, self.output = name, output
66 (output / 'shots').mkdir(parents=True, exist_ok=True)
67
68 def shot(self, label, script, delay=1500):
69 (self.output / 'shots' / f'{label}.ahk').write_text(script)
70 result = windows.do_exec(script, target=self.name, timeout_ms=180000, shot_delay_ms=delay)
71 screenshot = result.pop('png_b64', None)
72 if screenshot:
73 (self.output / 'shots' / f'{label}.png').write_bytes(base64.b64decode(screenshot))
74 (self.output / 'shots' / f'{label}.json').write_text(json.dumps(result, indent=2))
75 if result.get('error') or result.get('exit') != 0:
76 raise RuntimeError(f'{label}: {result}')
77 return result
78
79 def command(self, text, timeout_ms=120000):
80 runner.command(self.name, text, self.output, timeout_ms)
81
82 def keep(self, step, files=('History.one',)):
83 """Keeps the section once OneNote has stopped writing it: its size and time unchanged
84 for five seconds."""
85 self.command('powershell -NoProfile -Command "$f = \'%s\'; $last = \'\'; $still = 0; '
86 'for ($i = 0; $i -lt 120 -and $still -lt 10; $i++) { Start-Sleep -Milliseconds 500; '
87 '$item = Get-Item $f; $now = \'\' + $item.Length + $item.LastWriteTimeUtc.Ticks; '
88 'if ($now -eq $last) { $still++ } else { $still = 0; $last = $now } }"' % SECTION)
89 directory = self.output / f'step-{step:02}' / 'notebook'
90 directory.mkdir(parents=True)
91 for name in files:
92 result = windows.do_get(str(SECTION.parent / name), directory / name, self.name)
93 if result.get('error'):
94 raise RuntimeError(result['error'])
95
96
97def open_page(author):
98 """Closes OneNote, names the Office user `author` and opens the page."""
99 name, initials = author
100 return ('if WinExist("ahk_exe ONENOTE.EXE") {\n WinClose("ahk_exe ONENOTE.EXE")\n Sleep(4000)\n}\n'
101 'RunWait("cmd /c taskkill /IM ONENOTE.EXE /F", , "Hide")\nSleep(1000)\n'
102 f'RegWrite("{name}", "REG_SZ", "HKCU\\Software\\Microsoft\\Office\\Common\\UserInfo", "UserName")\n'
103 f'RegWrite("{initials}", "REG_SZ", "HKCU\\Software\\Microsoft\\Office\\Common\\UserInfo", "UserInitials")\n'
104 f'page := Trim(FileRead("{ROOT}\\page.txt", "UTF-8"), "`r`n ")\n'
105 'app := ComObject("OneNote.Application")\napp.NavigateTo(page)\n'
106 'WinWait("ahk_exe ONENOTE.EXE", , 20)\nWinMaximize("ahk_exe ONENOTE.EXE")\n'
107 + ACTIVATE + 'Sleep(3000)\n')
108
109
110def type_line(text, new_line=True):
111 return (ACTIVATE + click(BODY, 300) + ('Send("^{End}{Enter}")\n' if new_line else 'Send("{End}")\n')
112 + f'SendText("{text}")\nSleep(3000)\n')
113
114
115SETUP = r'''$ErrorActionPreference = 'Stop'
116$app = New-Object -ComObject OneNote.Application
117$notebook = ''
118$app.OpenHierarchy('%(root)s\Versions', '', [ref]$notebook, 1)
119$section = ''
120$app.OpenHierarchy('History.one', $notebook, [ref]$section, 3)
121$page = ''
122$app.CreateNewPage($section, [ref]$page, 0)
123$namespace = 'http://schemas.microsoft.com/office/onenote/2010/onenote'
124$app.UpdatePageContent('<one:Page xmlns:one="' + $namespace + '" ID="' + $page + '"><one:Title><one:OE><one:T>Versioned</one:T></one:OE></one:Title><one:Outline><one:Position x="36" y="86"/><one:OEChildren><one:OE><one:T>First state.</one:T></one:OE></one:OEChildren></one:Outline></one:Page>')
125$page | Out-File '%(root)s\page.txt' -Encoding UTF8
126'''
127
128VIRTUAL, OTHER = ('virtual', 'v'), ('Other Person', 'OP')
129
130
131def observe(output):
132 output.mkdir(parents=True, exist_ok=False)
133 with runner.clone(output) as name:
134 clone = Clone(name, output)
135 setup = output / 'setup.ps1'
136 setup.write_text(SETUP % {'root': ROOT})
137 clone.command(f'mkdir {ROOT}')
138 result = windows.do_put(setup, str(ROOT / 'setup.ps1'), name)
139 if result.get('error'):
140 raise RuntimeError(result['error'])
141 clone.command(f'powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File {ROOT}\\setup.ps1')
142 # One author: no version.
143 clone.shot('01-typed', open_page(VIRTUAL) + type_line(' Typed by virtual.', new_line=False))
144 clone.keep(1, ('History.one', 'Open Notebook.onetoc2'))
145 # A second author's edit keeps the page as it was as a version.
146 clone.shot('02-second-author', open_page(OTHER) + type_line('Second author line.'))
147 clone.keep(2)
148 clone.shot('02-menu', ACTIVATE + click(SHARE, 600) + click(PAGE_VERSIONS_MENU, 800))
149 clone.shot('02-shown', ACTIVATE + click((486, 129), 1500))
150 clone.shot('02-version', ACTIVATE + click(FIRST_VERSION, 1500))
151 clone.shot('02-version-menu', ACTIVATE + click(BAR, 1000))
152 # Copy Page To copies the version as a page at the section's end.
153 clone.shot('03-copy-dialog', ACTIVATE + click(COPY, 1500))
154 clone.shot('03-copied', 'WinActivate("Move or Copy Pages")\n' + click((289, 200), 500)
155 + click((474, 494), 3000), 2000)
156 clone.keep(3)
157 # Restore Version: the page as it stood becomes the newest version.
158 clone.shot('04-shown', ACTIVATE + click(THE_PAGE, 1000) + click(SHARE, 600) + click(PAGE_VERSIONS, 1200))
159 clone.shot('04-restored', ACTIVATE + click(FIRST_VERSION, 1200) + click(BAR, 800)
160 + click(RESTORE, 2500) + DIALOG, 2000)
161 clone.keep(4)
162 # Delete Version, without a confirmation.
163 clone.shot('05-newest', ACTIVATE + click(FIRST_VERSION, 1500))
164 clone.shot('05-deleted', ACTIVATE + click(BAR, 800) + click(DELETE, 2500) + DIALOG, 2000)
165 clone.keep(5)
166 # Delete All Versions in Section asks first.
167 clone.shot('06-confirm', ACTIVATE + click(SHARE, 600) + click(PAGE_VERSIONS_MENU, 800)
168 + click(DELETE_ALL_IN_SECTION, 1500) + DIALOG)
169 clone.shot('06-deleted-all', 'WinActivate("Microsoft OneNote ahk_class #32770")\nSend("!y")\nSleep(2500)\n', 2500)
170 clone.keep(6)
171 # Deleting the only version clears the page's HasVersionPages.
172 clone.shot('07-third-author-edit', open_page(VIRTUAL) + type_line('Third line by virtual.')
173 + click(SHARE, 600) + click(PAGE_VERSIONS, 1200))
174 clone.keep(7)
175 clone.shot('08-deleted-only', ACTIVATE + click(FIRST_VERSION, 1500) + click(BAR, 800)
176 + click(DELETE, 2500) + DIALOG, 2000)
177 clone.keep(8)
178 # Authors: Hide Authors off, Recent Edits, Find by Author.
179 clone.shot('09-authors', open_page(OTHER) + type_line('Fourth line by Other Person.')
180 + click(SHARE, 600) + click(HIDE_AUTHORS, 1500))
181 clone.shot('09-recent-edits', ACTIVATE + click(SHARE, 600) + click(RECENT_EDITS, 1200), 1000)
182 clone.shot('09-find-by-author', ACTIVATE + 'Send("{Esc}")\nSleep(400)\n' + click(SHARE, 900)
183 + click(FIND_BY_AUTHOR, 3000), 2500)
184 clone.keep(9)
185
186
187def cold(notebook, output, delete):
188 """Opens the page with versions, shows them, opens the newest and its menu; `delete`
189 then deletes it."""
190 def interaction(name, output):
191 clone = Clone(name, output)
192 page = ET.parse(sorted((output / 'read').glob('page-*.xml'))[0]).getroot().attrib['ID']
193 clone.shot('cold-main', runner.navigation_script(page) + 'Sleep(2000)\n')
194 clone.shot('cold-shown', ACTIVATE + click(SHARE, 600) + click(PAGE_VERSIONS, 1500))
195 clone.shot('cold-version', ACTIVATE + click(FIRST_VERSION, 1500))
196 clone.shot('cold-menu', ACTIVATE + click(BAR, 1000))
197 if delete:
198 clone.shot('cold-delete', ACTIVATE + click(DELETE, 2500) + DIALOG, 2000)
199 # OneNote writes the deletion in the background; give it time before the re-read.
200 time.sleep(20)
201 else:
202 clone.shot('cold-closed', ACTIVATE + 'Send("{Esc}")\nSleep(800)\n')
203 runner.capture(notebook, output, collect_notebook=True, interaction=interaction)
204
205
206if __name__ == '__main__':
207 parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
208 commands = parser.add_subparsers(dest='command', required=True)
209 observed = commands.add_parser('observe')
210 observed.add_argument('output', type=Path)
211 opened = commands.add_parser('cold')
212 opened.add_argument('notebook', type=Path)
213 opened.add_argument('output', type=Path)
214 opened.add_argument('--delete', action='store_true')
215 args = parser.parse_args()
216 if args.command == 'observe':
217 observe(args.output)
218 else:
219 cold(args.notebook, args.output, args.delete)
tools/test_document_oracle.py+8
...@@ -1,3 +1,4 @@...@@ -1,3 +1,4 @@
1from importlib.util import find_spec
1from pathlib import Path2from pathlib import Path
2import re3import re
3import runpy4import runpy
...@@ -9,9 +10,12 @@ import xml.etree.ElementTree as ET...@@ -9,9 +10,12 @@ import xml.etree.ElementTree as ET
910
10ROOT = Path(__file__).resolve().parent.parent11ROOT = Path(__file__).resolve().parent.parent
11compare = runpy.run_path(str(ROOT / 'tools/verify-document.py'))['compare']12compare = runpy.run_path(str(ROOT / 'tools/verify-document.py'))['compare']
13requires_pdfplumber = unittest.skipUnless(
14 find_spec('pdfplumber'), 'pdfplumber is not installed; tools/TESTING.md sets up the Python lane')
1215
1316
14class DocumentOracleTest(unittest.TestCase):17class DocumentOracleTest(unittest.TestCase):
18 @requires_pdfplumber
15 def test_locked_table_width_requires_the_native_value(self):19 def test_locked_table_width_requires_the_native_value(self):
16 fixture = ROOT / 'corpus/m6/native-structure-01'20 fixture = ROOT / 'corpus/m6/native-structure-01'
17 with TemporaryDirectory() as temporary:21 with TemporaryDirectory() as temporary:
...@@ -27,6 +31,7 @@ class DocumentOracleTest(unittest.TestCase):...@@ -27,6 +31,7 @@ class DocumentOracleTest(unittest.TestCase):
27 with self.assertRaisesRegex(AssertionError, 'Locked table column width differs'):31 with self.assertRaisesRegex(AssertionError, 'Locked table column width differs'):
28 compare(fixture / 'notebook', native)32 compare(fixture / 'notebook', native)
2933
34 @requires_pdfplumber
30 def test_native_2010_does_not_render_the_documented_cell_shading_control(self):35 def test_native_2010_does_not_render_the_documented_cell_shading_control(self):
31 import pdfplumber36 import pdfplumber
32 fixture = ROOT / 'corpus/m6/cell-shading-control-01/read/page-001'37 fixture = ROOT / 'corpus/m6/cell-shading-control-01/read/page-001'
...@@ -40,6 +45,7 @@ class DocumentOracleTest(unittest.TestCase):...@@ -40,6 +45,7 @@ class DocumentOracleTest(unittest.TestCase):
40 self.assertTrue(all(tuple(rect.get('non_stroking_color') or ()) != (1, 1, 0)45 self.assertTrue(all(tuple(rect.get('non_stroking_color') or ()) != (1, 1, 0)
41 for rect in table_page.rects))46 for rect in table_page.rects))
4247
48 @requires_pdfplumber
43 def test_rust_saved_expansion_is_honored_by_a_fresh_native_cache(self):49 def test_rust_saved_expansion_is_honored_by_a_fresh_native_cache(self):
44 import pdfplumber50 import pdfplumber
45 fixture = ROOT / 'corpus/m6/rust-collapse-control-01'51 fixture = ROOT / 'corpus/m6/rust-collapse-control-01'
...@@ -53,6 +59,7 @@ class DocumentOracleTest(unittest.TestCase):...@@ -53,6 +59,7 @@ class DocumentOracleTest(unittest.TestCase):
53 self.assertIn('Hidden child remains stored', text)59 self.assertIn('Hidden child remains stored', text)
54 self.assertIn('Grandchild', text)60 self.assertIn('Grandchild', text)
5561
62 @requires_pdfplumber
56 def test_native_expansion_changes_the_view_without_changing_notebook_bytes(self):63 def test_native_expansion_changes_the_view_without_changing_notebook_bytes(self):
57 import pdfplumber64 import pdfplumber
58 original = ROOT / 'corpus/m6/native-features-01/notebook/Features.one'65 original = ROOT / 'corpus/m6/native-features-01/notebook/Features.one'
...@@ -96,6 +103,7 @@ class DocumentOracleTest(unittest.TestCase):...@@ -96,6 +103,7 @@ class DocumentOracleTest(unittest.TestCase):
96 captures.append([ET.tostring(n) for n in equations])103 captures.append([ET.tostring(n) for n in equations])
97 self.assertEqual(*captures)104 self.assertEqual(*captures)
98105
106 @requires_pdfplumber
99 def test_rtl_xml_order_matches_native_visual_column_order(self):107 def test_rtl_xml_order_matches_native_visual_column_order(self):
100 import pdfplumber108 import pdfplumber
101 fixture = ROOT / 'corpus/m6/native-page-direction-03'109 fixture = ROOT / 'corpus/m6/native-page-direction-03'
tools/test_link_edit.py+32
...@@ -1,3 +1,4 @@...@@ -1,3 +1,4 @@
1import html
1import json2import json
2import re3import re
3from pathlib import Path4from pathlib import Path
...@@ -49,6 +50,37 @@ class LinkEditTest(unittest.TestCase):...@@ -49,6 +50,37 @@ class LinkEditTest(unittest.TestCase):
49 text = ' '.join(' '.join(oe.find('one:T', ns).text.split()) for oe in page.iter('{%s}OE' % ns['one']) if oe.find('one:T', ns) is not None)50 text = ' '.join(' '.join(oe.find('one:T', ns).text.split()) for oe in page.iter('{%s}OE' % ns['one']) if oe.find('one:T', ns) is not None)
50 self.assertRegex(text, r'^Linking page <a href="onenote:#Link%20target&amp;section-id=\{[0-9A-F-]{36}\}&amp;page-id=\{[0-9A-F-]{36}\}&amp;end&amp;base-path=C:\\one-tests\\runs\\capture\\notebook\\links.one">Link target</a>$')51 self.assertRegex(text, r'^Linking page <a href="onenote:#Link%20target&amp;section-id=\{[0-9A-F-]{36}\}&amp;page-id=\{[0-9A-F-]{36}\}&amp;end&amp;base-path=C:\\one-tests\\runs\\capture\\notebook\\links.one">Link target</a>$')
5152
53 def test_onenote_links_typed_urls_and_dialog_addresses(self):
54 """OneNote 2010's read of `native-typed`: a typed `www.` address opens over http,
55 and a Link dialog address typed without a scheme does too."""
56 text = (FIXTURE / 'native-typed/read/page-000.xml').read_text(encoding='utf-8-sig')
57 hrefs = [html.unescape(href) for href in re.findall(r'href="([^"]*)"', text)]
58 self.assertEqual(hrefs[1], 'http://www.example.com')
59 self.assertEqual(hrefs[11], 'http://www.d.example')
60 self.assertEqual(hrefs[15], 'http://example.net/x')
61 self.assertNotIn('me@example.com', hrefs)
62
63 def test_editor_links_and_equations_render_natively(self):
64 """What the canvas editor saved for typed URLs, the Link dialog, Remove Link and
65 equations typed after Alt+= reads back in a cold OneNote 2010 with the links and
66 MathML it meant (`crates/canvas/tests/links_equations.rs`)."""
67 row = FIXTURE / 'editor'
68 with TemporaryDirectory() as temporary:
69 read = Path(temporary) / 'read'
70 shutil.copytree(row / 'cold/read', read)
71 compare(row / 'candidate', read)
72 expected = json.loads((row / 'expected.json').read_text())
73 path = next(path for path in sorted((row / 'cold/read').glob('page-*.xml'))
74 if ET.parse(path).getroot().get('name').startswith('Linking page'))
75 page = ET.parse(path).getroot()
76 text = ''.join(oe.find('one:T', ns).text for oe in page.iter('{%s}OE' % ns['one']) if oe.find('one:T', ns) is not None)
77 hrefs = [html.unescape(href) for href in re.findall(r'href="([^"]*)"', text)]
78 self.assertEqual(hrefs, expected['hrefs'])
79 raw = path.read_text(encoding='utf-8-sig')
80 mathml = [re.sub(r'&#(\d+);', lambda m: chr(int(m.group(1))), body)
81 for body in re.findall(r'<mml:math[^>]*>(.*?)</mml:math>', raw, re.S)]
82 self.assertEqual(mathml, expected['mathml'])
83
5284
53if __name__ == '__main__':85if __name__ == '__main__':
54 unittest.main()86 unittest.main()
tools/test_math_edit.py+1
...@@ -24,6 +24,7 @@ class MathEditTest(unittest.TestCase):...@@ -24,6 +24,7 @@ class MathEditTest(unittest.TestCase):
24 self.assertEqual(len(exported_mathml(FIXTURE / 'native-editor/read/page-000.xml')), 5)24 self.assertEqual(len(exported_mathml(FIXTURE / 'native-editor/read/page-000.xml')), 5)
25 self.assertEqual(len(exported_mathml(FIXTURE / 'native-editor-2/read/page-000.xml')), 9)25 self.assertEqual(len(exported_mathml(FIXTURE / 'native-editor-2/read/page-000.xml')), 9)
26 self.assertEqual(len(exported_mathml(FIXTURE / 'native-editor-3/read/page-000.xml')), 10)26 self.assertEqual(len(exported_mathml(FIXTURE / 'native-editor-3/read/page-000.xml')), 10)
27 self.assertEqual(len(exported_mathml(FIXTURE / 'native-editor-4/read/page-000.xml')), 26)
2728
28 def test_rust_written_equations_export_the_expected_mathml(self):29 def test_rust_written_equations_export_the_expected_mathml(self):
29 with TemporaryDirectory() as temporary:30 with TemporaryDirectory() as temporary:
tools/test_page_versions.py created+103
...@@ -0,0 +1,103 @@
1"""Page versions (corpus/page-versions): what OneNote 2010 stores for them and does to them,
2and that it reads, lists and deletes the ones Rust writes."""
3import hashlib
4import json
5from pathlib import Path
6import runpy
7import shutil
8import subprocess
9from tempfile import TemporaryDirectory
10import unittest
11
12from document_model import EXPORTER, ordered_pages, version_pages, walk
13
14ROOT = Path(__file__).resolve().parent.parent
15FIXTURE = ROOT / 'corpus/page-versions'
16compare = runpy.run_path(str(ROOT / 'tools/verify-document.py'))['compare']
17HISTORY = '{7111497F-1B6B-4209-9491-C98B04CF4C5A},1'
18HAS_VERSION_PAGES = 0x88003462
19
20
21def model(notebook):
22 with TemporaryDirectory() as temporary:
23 output = Path(temporary) / 'model'
24 subprocess.run([EXPORTER, notebook / 'History.one', output], check=True, capture_output=True)
25 return json.loads((output / 'document.json').read_text())
26
27
28# The title's date and time fields, the same on every state of the page.
29DATE = {'Sunday, September 27, 2026', '5:42 PM'}
30
31
32def texts(space, root):
33 return [node['kind']['text'] for _, node in walk(space, root)
34 if node['kind']['type'] == 'RichText' and node['kind']['text'] and node['kind']['text'] not in DATE]
35
36
37def versions(document):
38 """Each listed page's text and versions, newest first: each version's author and text,
39 and whether its metadata says it has versions."""
40 listed = []
41 for sid, _, page, oid in ordered_pages(document):
42 own = []
43 history = document['spaces'][sid]['revisions'].get(document['spaces'][sid]['contexts'].get(HISTORY))
44 for _, _, revision, version, proxy in version_pages(document, sid, page):
45 author = history['nodes'][proxy['latest_author']]['kind']['name']
46 own.append((author, texts(revision, version)))
47 metadata = page['nodes'][page['roots']['2']]
48 flagged = any(field['id'] == HAS_VERSION_PAGES for fields in metadata['extra'] for field in fields)
49 listed.append((texts(page, oid), own, flagged))
50 return listed
51
52
53FIRST = 'First state. Typed by virtual.'
54SECOND = 'Second author line.'
55
56
57class PageVersionsTest(unittest.TestCase):
58 def test_onenote_keeps_a_version_when_another_author_edits(self):
59 native = FIXTURE / 'native'
60 self.assertEqual(versions(model(native / 'step-01/notebook')),
61 [(['Versioned', FIRST], [], False)])
62 page = ['Versioned', FIRST, SECOND]
63 self.assertEqual(versions(model(native / 'step-02/notebook')),
64 [(page, [('virtual', ['Versioned', FIRST])], True)])
65 # Copy Page To copies the version as a page of its own, without versions.
66 self.assertEqual(versions(model(native / 'step-03/notebook'))[1],
67 (['Versioned', FIRST], [], False))
68 # Restore Version: the page as it stood becomes the newest version.
69 self.assertEqual(versions(model(native / 'step-04/notebook'))[0],
70 (['Versioned', FIRST], [('Other Person', page), ('virtual', ['Versioned', FIRST])], True))
71 # Delete Version, then Delete All Versions in Section, which clears the flag.
72 self.assertEqual(versions(model(native / 'step-05/notebook'))[0][1], [('virtual', ['Versioned', FIRST])])
73 self.assertEqual(versions(model(native / 'step-06/notebook'))[0], (['Versioned', FIRST], [], False))
74 # Optimizing keeps versions.
75 self.assertEqual(len(versions(model(native / 'step-10/notebook'))[0][1]), 1)
76
77 def test_the_scripted_capture_repeats_it(self):
78 scripted = FIXTURE / 'native-scripted'
79 counts = [len(versions(model(scripted / f'step-{n:02}/notebook'))[0][1]) for n in range(1, 10)]
80 self.assertEqual(counts, [0, 1, 1, 2, 1, 0, 1, 0, 1])
81
82 def test_onenote_reads_a_rust_restore_and_deletes_its_version(self):
83 for name in ('restore', 'delete-all'):
84 source = json.loads((FIXTURE / f'cold-{name}/source.json').read_text())
85 written = hashlib.sha256((FIXTURE / f'candidate-{name}/History.one').read_bytes()).hexdigest()
86 self.assertIn(written, [item['sha256'] for item in source])
87 with TemporaryDirectory() as temporary:
88 read = Path(temporary) / 'read'
89 shutil.copytree(FIXTURE / f'cold-{name}/before-read', read)
90 compare(FIXTURE / f'candidate-{name}', read)
91 restored = versions(model(FIXTURE / 'candidate-restore'))[0]
92 self.assertEqual(restored[0], ['Versioned', FIRST])
93 self.assertEqual([author for author, _ in restored[1]], ['Other Person', 'virtual'])
94 self.assertEqual(versions(model(FIXTURE / 'candidate-delete-all'))[0], (['Versioned', FIRST], [], False))
95 # OneNote listed the Rust versions under the page, opened the newest under its bar and
96 # deleted it through the bar's menu, leaving the older one.
97 record = lambda name: json.loads((FIXTURE / f'cold-restore/shots/{name}.json').read_text())
98 self.assertEqual(record('cold-version')['win']['title'], 'Versioned - Microsoft OneNote')
99 after = versions(model(FIXTURE / 'cold-restore/notebook'))[0]
100 self.assertEqual([author for author, _ in after[1]], ['virtual'])
101
102if __name__ == '__main__':
103 unittest.main()
tools/test_pdf_format.py+5
...@@ -1,3 +1,4 @@...@@ -1,3 +1,4 @@
1from importlib.util import find_spec
1from pathlib import Path2from pathlib import Path
2import runpy3import runpy
3import unittest4import unittest
...@@ -6,7 +7,11 @@ from unittest.mock import patch...@@ -6,7 +7,11 @@ from unittest.mock import patch
67
7verify = runpy.run_path(str(Path(__file__).with_name('verify-document.py')))['verify_pdf_black']8verify = runpy.run_path(str(Path(__file__).with_name('verify-document.py')))['verify_pdf_black']
89
10requires_pdfplumber = unittest.skipUnless(
11 find_spec('pdfplumber'), 'pdfplumber is not installed; tools/TESTING.md sets up the Python lane')
912
13
14@requires_pdfplumber
10class PdfFormatOracleTest(unittest.TestCase):15class PdfFormatOracleTest(unittest.TestCase):
11 def test_unmapped_text_cannot_hide_missing_text_or_incorrect_highlighting(self):16 def test_unmapped_text_cannot_hide_missing_text_or_incorrect_highlighting(self):
12 chars = [{'text': c, 'x0': i * 2, 'x1': i * 2 + 1, 'top': 0, 'bottom': 1}17 chars = [{'text': c, 'x0': i * 2, 'x1': i * 2 + 1, 'top': 0, 'bottom': 1}
tools/test_recycle_bin_repair.py created+45
...@@ -0,0 +1,45 @@
1from pathlib import Path
2import unittest
3import xml.etree.ElementTree as ET
4
5ROOT = Path(__file__).resolve().parent.parent
6FIXTURE = ROOT / 'corpus/recycle-bin-repair'
7
8
9def tree(node):
10 out = []
11 for child in node:
12 tag = child.tag.rsplit('}', 1)[-1]
13 if tag in ('Section', 'SectionGroup', 'Page'):
14 flags = tuple(sorted(key for key in ('isRecycleBin', 'isDeletedPages', 'isInRecycleBin')
15 if child.get(key) == 'true'))
16 out.append((tag, child.get('name'), flags, tree(child)))
17 return out
18
19
20class RecycleBinRepairTest(unittest.TestCase):
21 def test_onenote_repairs_nothing_in_the_bin_a_page_delete_repaired(self):
22 candidates = sorted(path.relative_to(FIXTURE / 'candidate')
23 for path in (FIXTURE / 'candidate').rglob('*') if path.is_file())
24 self.assertEqual(candidates, sorted(path.relative_to(FIXTURE / 'cold/notebook')
25 for path in (FIXTURE / 'cold/notebook').rglob('*')
26 if path.is_file()))
27 for relative in candidates:
28 self.assertEqual((FIXTURE / 'cold/notebook' / relative).read_bytes(),
29 (FIXTURE / 'candidate' / relative).read_bytes(), str(relative))
30
31 def test_the_bin_holds_the_deleted_pages(self):
32 hierarchy = ET.parse(FIXTURE / 'cold/read/hierarchy.xml').getroot()
33 self.assertEqual(tree(hierarchy), [
34 ('Section', 'New Section 1', (), [('Page', 'Kept', (), [])]),
35 ('SectionGroup', 'OneNote_RecycleBin', ('isRecycleBin',), [
36 ('Section', 'Deleted Pages', ('isDeletedPages', 'isInRecycleBin'), [
37 ('Page', 'Kept', ('isInRecycleBin',), []),
38 ('Page', 'Kept', ('isInRecycleBin',), []),
39 ]),
40 ]),
41 ])
42
43
44if __name__ == '__main__':
45 unittest.main()
tools/verify-document.py+34-2
...@@ -82,7 +82,10 @@ def visible_text(node, space):...@@ -82,7 +82,10 @@ def visible_text(node, space):
82 text = ''.join(data[run['start'] * 2:run['end'] * 2].decode('utf-16-le') for run in kind['runs']82 text = ''.join(data[run['start'] * 2:run['end'] * 2].decode('utf-16-le') for run in kind['runs']
83 if not run['format'] or not (space['nodes'][run['format']]['format']['hidden']83 if not run['format'] or not (space['nodes'][run['format']]['format']['hidden']
84 or space['nodes'][run['format']]['format']['math']))84 or space['nodes'][run['format']]['format']['math']))
85 return "" if text == "\u00a0" else project_text(text.removesuffix('\r'))85 # A trailing CR ends the stored text; one inside it, between equations, is a line break.
86 if kind['text'].endswith('\r'):
87 text = text.removesuffix('\r')
88 return "" if text == "\u00a0" else project_text(text)
8689
8790
88ISF_X = bytes.fromhex('8f6a8a59c052a04b93afaf357411a561')91ISF_X = bytes.fromhex('8f6a8a59c052a04b93afaf357411a561')
...@@ -247,6 +250,33 @@ def compare_objects(space, roots, page, native_roots, assets, native_payloads, a...@@ -247,6 +250,33 @@ def compare_objects(space, roots, page, native_roots, assets, native_payloads, a
247 return count250 return count
248251
249252
253def exported_links(runs):
254 """Characters and links as OneNote 2010 exports them: an address without a scheme opens
255 over http, and URL text it finds in a page's text is a link, a removed link's included
256 (`corpus/link-edit/native-typed`)."""
257 def href(link):
258 if link is None or ':' in link.split('/')[0] or link.startswith('\\\\'):
259 return link
260 return 'http://' + link
261 out = [(char, href(link)) for char, link in runs]
262 start = 0
263 while start < len(out):
264 end = start
265 while end < len(out) and not out[end][0].isspace():
266 end += 1
267 word = ''.join(char for char, _ in out[start:end])
268 # The scheme stays with the text it links; trailing punctuation does not.
269 for scheme in ('http://', 'https://', 'ftp://', 'mailto:', 'news:'):
270 at = word.lower().find(scheme)
271 if at >= 0 and all(link is None for _, link in out[start:end]):
272 url = word[at:].rstrip('.,;:!?\'")]}')
273 for index in range(start + at, start + at + len(url)):
274 out[index] = (out[index][0], url)
275 break
276 start = end + 1
277 return out
278
279
250def compare(notebook, native, versions=None, password_file=None):280def compare(notebook, native, versions=None, password_file=None):
251 notebook = notebook.resolve(strict=True)281 notebook = notebook.resolve(strict=True)
252 native = native.resolve(strict=True)282 native = native.resolve(strict=True)
...@@ -396,12 +426,14 @@ def compare(notebook, native, versions=None, password_file=None):...@@ -396,12 +426,14 @@ def compare(notebook, native, versions=None, password_file=None):
396 # Native XML exports equation runs as MathML, not text (tools/test_math_edit.py compares that).426 # Native XML exports equation runs as MathML, not text (tools/test_math_edit.py compares that).
397 observed_runs = [(char, run['link']) for run in resolved_text[sid][rid][text_id]427 observed_runs = [(char, run['link']) for run in resolved_text[sid][rid][text_id]
398 if not run['format']['hidden'] and not run['format']['math'] for char in run['text']]428 if not run['format']['hidden'] and not run['format']['math'] for char in run['text']]
399 if observed_runs and observed_runs[-1][0] == '\r':429 stored = ''.join(run['text'] for run in resolved_text[sid][rid][text_id])
430 if observed_runs and observed_runs[-1][0] == '\r' and stored.endswith('\r'):
400 observed_runs.pop()431 observed_runs.pop()
401 observed_runs = [(projected, link) for char, link in observed_runs for projected in project_text(char)]432 observed_runs = [(projected, link) for char, link in observed_runs for projected in project_text(char)]
402 expected_links = [(char, style.get('link')) for char, style in expected_runs]433 expected_links = [(char, style.get('link')) for char, style in expected_runs]
403 if observed_runs == [('\u00a0', None)] and not expected_links:434 if observed_runs == [('\u00a0', None)] and not expected_links:
404 observed_runs = []435 observed_runs = []
436 observed_runs = exported_links(observed_runs)
405 assert observed_runs == expected_links, 'Resolved text or associated hyperlink differs'437 assert observed_runs == expected_links, 'Resolved text or associated hyperlink differs'
406 count, differences = compare_formats(space, text_nodes, native_runs)438 count, differences = compare_formats(space, text_nodes, native_runs)
407 formatting += count439 formatting += count