| author |  clover caruso <git@paperclover.net> 2026-09-12 20:05:15-07:00 |
| committer | clover caruso <git@paperclover.net> 2026-09-25 20:26:20-07:00 |
| log | 6478bd1e6e851e3c9bfe58640dfb03ffb64f09f8 |
| tree | 9cc3f2e634fecf28588e94d3ffeb37a5c780ed9d |
| parent | db49720208e16a932d4c13ebe1172ab872638ac7 |
| signature | Signed by SSH key SHA256:52mNGHRsVFBDED9IAX5pe+LRWUefqTbxEReunq21QvU |
feat: write equations through the page model; give new identities n 1
Math::paragraph builds the linear text and per-run object data from a Math
tree (Latin letters as mathematical italics, one run per object control,
the equation editor's Cambria Math italic style with the math flags and
language), and a new equation pass in the page writer rewrites a changed
equation paragraph the way OneNote stores one: the text, one style object
per distinct span format, run boundaries, the text-run data array naming
each run's inline object, and the language marker and flags every native
equation text object carries. tools/native_math.py types expressions
through the equation editor with AutoHotkey during a capture;
corpus/math-edit/native-editor adds subscripts, a sub-superscript,
parentheses, an integral and a sum to the MathML oracle, and the mapping
now covers them (n-ary operators take limits as scripts for integrals and
above/below otherwise; object arguments render in rows). The oracle tools
treat equation runs like hidden ones, since native XML exports them as
MathML rather than text.
Gating the ten-equation candidate found OneNote 2010 silently dropping an
outline element in roughly one build of five: deterministic per file,
structurally indistinguishable from the ones that rendered, and never seen
before because earlier rows added few paragraphs. OneNote never stores an
object as {guid},0, and with page::text::new_id giving n 1 every one of
eighteen builds and the written candidate rendered; corpus/math-edit/
native-drop keeps the dropped build, the cold read and a relocation
experiment. The transaction writer also keeps native files' 1 KiB
reservation after a transaction-log fragment that ends the file.
corpus/math-edit/written is the cold read of all ten expressions with a
screenshot; tools/test_math_edit.py checks it against the expected MathML.
Assisted-by: claude-fable-5.170 files changed, 1519 insertions(+), 28 deletions(-)
corpus/math-edit/README.md created+35
| ... | @@ -0,0 +1,35 @@ | ||
| 1 | # Native equation fixtures for the math model | ||
| 2 | |||
| 3 | `native-editor/` is OneNote 2010 typing five expressions through its equation | ||
| 4 | editor (Alt+=) on the Rust-authored link page, driven by | ||
| 5 | `tools/native_math.py` (`equations.ahk` is the AutoHotkey session, | ||
| 6 | `equations.png` the desktop afterwards): `a_1+b_2`, `x_i^2`, `(a+b)`, | ||
| 7 | `\int_0^1 x dx` and `\sum_(i=1)^n i`; `\sqrt(x+1)` did not build up and | ||
| 8 | left an empty paragraph. `before-read/` is the COM read before typing, | ||
| 9 | `read/` the read afterwards with each expression exported as MathML, and | ||
| 10 | `notebook/` the section OneNote saved: the linear text with U+FDD0/FDEE/FDEF | ||
| 11 | object controls and the text-run data naming each object's kind (29 | ||
| 12 | subscript, 30 sub-superscript, 13 brackets, 21 n-ary with its operator | ||
| 13 | symbol). `crates/onestore/tests/page_math.rs` renders every expression from | ||
| 14 | the stored section and compares it with the exported MathML byte for byte; | ||
| 15 | together with `corpus/m6/native-math-01` (superscripts, a fraction, inline | ||
| 16 | Greek) that covers the kinds `onestore::page::Math` claims. | ||
| 17 | |||
| 18 | `written/candidate` is the page writer's output for | ||
| 19 | `equations_are_written_and_read_back`: on a fresh section, every native | ||
| 20 | equation of both fixtures copied as a new paragraph, then two expressions | ||
| 21 | built from `Math` trees (`x^2+1` and a fraction with a superscript in its | ||
| 22 | denominator) through `Math::paragraph`. The writer stores each as OneNote | ||
| 23 | does: the linear text, one run per span with a Cambria Math italic style | ||
| 24 | carrying the math flags and the math language, the text-run data array | ||
| 25 | naming each run's object, and the math language marker on the text object. | ||
| 26 | `written/cold` is the OneNote 2010 read of that section: its exported MathML | ||
| 27 | equals `expected-mathml.json` (the native fixtures' exports plus the two | ||
| 28 | built expressions) and `page-000.png` shows the equations rendered. | ||
| 29 | `tools/test_math_edit.py` checks both without a VM. | ||
| 30 | |||
| 31 | Regenerate the editor fixture with `tools/native_math.py | ||
| 32 | corpus/link-edit/candidate OUTPUT` and the written one with | ||
| 33 | `ONESTORE_MATH_EXPORT` set while running the test, then | ||
| 34 | `tools/native_runner.py OUTPUT COLD --expected-pages 1 --collect-notebook | ||
| 35 | --screenshots`. | ||
corpus/math-edit/native-drop/README.md created+23
| ... | @@ -0,0 +1,23 @@ | ||
| 1 | # OneNote 2010 drops outline elements written with `{guid},0` identities | ||
| 2 | |||
| 3 | `dropped/candidate/four.one` is one of six identical-content builds of the | ||
| 4 | ten-equation page (`equations_are_written_and_read_back`); only this build | ||
| 5 | came back from OneNote 2010 with an empty outline element (`dropped/cold`: | ||
| 6 | the fifth equation `x_i^2` has no content, alignment or style). The object | ||
| 7 | bytes, identity tables, run data, styles, revision manifest and transaction | ||
| 8 | log are identical in shape to the builds that rendered, the drop is | ||
| 9 | deterministic for a given file, and across thirty builds written with the | ||
| 10 | model's `{random guid},0` identities seven lost one element each, always an | ||
| 11 | outline element, at varying positions. OneNote never stores an object as | ||
| 12 | `{guid},0` (its own objects are `{page guid},n` with `n` from 1); after | ||
| 13 | `page::text::new_id` moved to `n` 1, eighteen builds and the written | ||
| 14 | candidate (`../written`) rendered every paragraph. | ||
| 15 | |||
| 16 | `relocated/candidate` is the dropped build with only that element's 56-byte | ||
| 17 | data chunk copied to the end of the file and the declaration retargeted; it | ||
| 18 | rendered (`relocated/cold`), so the loss also depends on where the data | ||
| 19 | sits, which is consistent with the identity being mishandled in a | ||
| 20 | position-keyed structure rather than being rejected outright. The | ||
| 21 | transaction writer additionally keeps native files' 1 KiB reservation after | ||
| 22 | a transaction-log fragment that ends the file, so no object data abuts the | ||
| 23 | log. | ||
corpus/math-edit/native-drop/dropped/candidate/Open Notebook.onetoc2 created
| Binary files /dev/null and b/corpus/math-edit/native-drop/dropped/candidate/Open Notebook.onetoc2 differ | |||
corpus/math-edit/native-drop/dropped/candidate/four.one created
| Binary files /dev/null and b/corpus/math-edit/native-drop/dropped/candidate/four.one differ | |||
corpus/math-edit/native-drop/dropped/cold/page-000.xml created+3
| ... | @@ -0,0 +1,3 @@ | ||
| 1 | <?xml version="1.0"?> | ||
| 2 | <one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{749A596B-6FB2-019B-0CAA-218CC33BAB77}{1}{B0}" name="Equations below" dateTime="2026-09-13T02:12:13.000Z" lastModifiedTime="2026-09-13T02:15:26.000Z" pageLevel="1" lang="en-US"><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="Author" lastModifiedBy="Author" lastModifiedTime="2026-09-13T02:12:13.000Z" objectID="{9CB03CE2-4378-0217-314E-70F4BD14F5B5}{23}{B0}"><one:Position x="36.0" y="36.0" z="0"/><one:Size width="140.9678649902344" height="195.0500030517578"/><one:Indents><one:Indent level="0" indent="0.0"/></one:Indents><one:OEChildren><one:OE creationTime="2026-09-13T02:12:13.000Z" lastModifiedTime="2026-09-13T02:12:13.000Z" objectID="{9CB03CE2-4378-0217-314E-70F4BD14F5B5}{24}{B0}" alignment="left" style="font-family:Calibri;font-size:11.0pt"><one:T><![CDATA[Equations below]]></one:T></one:OE><one:OE author="Math author" lastModifiedBy="Math author" creationTime="2026-09-13T02:12:13.000Z" lastModifiedTime="2026-09-13T02:12:13.000Z" objectID="{84FD73AB-BBF7-06B0-368D-82BB9ED16F2B}{0}{B0}" alignment="left" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:msup><mml:mi>x</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mo>+</mml:mo><mml:msup><mml:mi>y</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mo>=</mml:mo><mml:msup><mml:mi>z</mml:mi><mml:mn>2</mml:mn></mml:msup></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Math author" lastModifiedBy="Math author" creationTime="2026-09-13T02:12:13.000Z" lastModifiedTime="2026-09-13T02:12:13.000Z" objectID="{41CA5519-AF22-0AC3-1FF3-A12BC7F7917C}{0}{B0}" alignment="left" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:mfrac><mml:mrow><mml:mi>a</mml:mi><mml:mo>+</mml:mo><mml:mi>b</mml:mi></mml:mrow><mml:mrow><mml:mi>c</mml:mi><mml:mo>+</mml:mo><mml:mi>d</mml:mi></mml:mrow></mml:mfrac></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Math author" lastModifiedBy="Math author" creationTime="2026-09-13T02:12:13.000Z" lastModifiedTime="2026-09-13T02:12:13.000Z" objectID="{A9B75797-45B7-0281-320B-B6DA8143F48F}{0}{B0}" alignment="left" style="font-size:11.0pt"><one:T><![CDATA[<span | ||
| 3 | style='font-family:Calibri' lang=en-US>Inline </span><!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>𝛼</mml:mi><mml:mo>+</mml:mo><mml:mi>𝛽</mml:mi></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Math author" lastModifiedBy="Math author" creationTime="2026-09-13T02:12:13.000Z" lastModifiedTime="2026-09-13T02:12:13.000Z" objectID="{5FBAB9BA-7A0A-068A-0EF3-C6DD41CF0F5E}{0}{B0}" alignment="left" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:msub><mml:mi>a</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo>+</mml:mo><mml:msub><mml:mi>b</mml:mi><mml:mn>2</mml:mn></mml:msub></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Math author" lastModifiedBy="Math author" creationTime="2026-09-13T02:12:13.000Z" lastModifiedTime="2026-09-13T02:12:13.000Z" objectID="{A8690883-1B55-0256-3483-B284DD3E0388}{0}{B0}"/><one:OE author="Math author" lastModifiedBy="Math author" creationTime="2026-09-13T02:12:13.000Z" lastModifiedTime="2026-09-13T02:12:13.000Z" objectID="{DDF12119-6FB9-0490-117F-0F969556B5E2}{0}{B0}" alignment="left" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:mfenced><mml:mrow><mml:mi>a</mml:mi><mml:mo>+</mml:mo><mml:mi>b</mml:mi></mml:mrow></mml:mfenced></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Math author" lastModifiedBy="Math author" creationTime="2026-09-13T02:12:13.000Z" lastModifiedTime="2026-09-13T02:12:13.000Z" objectID="{0097E7FF-CE7A-0A29-2221-1BCA7541FBB8}{0}{B0}" alignment="left" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:msubsup><mml:mo stretchy="false">∫</mml:mo><mml:mn>0</mml:mn><mml:mn>1</mml:mn></mml:msubsup><mml:mrow><mml:mi>x</mml:mi><mml:mo> </mml:mo><mml:mi>d</mml:mi><mml:mi>x</mml:mi><mml:mo> </mml:mo></mml:mrow></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Math author" lastModifiedBy="Math author" creationTime="2026-09-13T02:12:13.000Z" lastModifiedTime="2026-09-13T02:12:13.000Z" objectID="{742404F5-F245-0D37-3F8C-D96FEBF828EA}{0}{B0}" alignment="left" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:munderover><mml:mo stretchy="false">∑</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>n</mml:mi></mml:munderover><mml:mrow><mml:mi>i</mml:mi><mml:mo> </mml:mo></mml:mrow></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Math author" lastModifiedBy="Math author" creationTime="2026-09-13T02:12:13.000Z" lastModifiedTime="2026-09-13T02:12:13.000Z" objectID="{4A13F544-367B-0873-19ED-FD9F1AC12816}{0}{B0}" alignment="left" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:msup><mml:mi>x</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Math author" lastModifiedBy="Math author" creationTime="2026-09-13T02:12:13.000Z" lastModifiedTime="2026-09-13T02:12:13.000Z" objectID="{5D475655-5172-052B-3EF9-3A9C8F642970}{0}{B0}" alignment="left" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:mfrac><mml:mrow><mml:mi>a</mml:mi><mml:mo>+</mml:mo><mml:mi>b</mml:mi></mml:mrow><mml:mrow><mml:msup><mml:mi>c</mml:mi><mml:mn>2</mml:mn></mml:msup></mml:mrow></mml:mfrac></mml:math><![endif]-->]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page> | ||
corpus/math-edit/native-drop/relocated/candidate/Open Notebook.onetoc2 created
| Binary files /dev/null and b/corpus/math-edit/native-drop/relocated/candidate/Open Notebook.onetoc2 differ | |||
corpus/math-edit/native-drop/relocated/candidate/four.one created
| Binary files /dev/null and b/corpus/math-edit/native-drop/relocated/candidate/four.one differ | |||
corpus/math-edit/native-drop/relocated/cold/commands.jsonl created+3
| ... | @@ -0,0 +1,3 @@ | ||
| 1 | {"command": "powershell -NoProfile -Command \"Expand-Archive -LiteralPath C:\\one-tests\\transfer.zip -DestinationPath C:\\one-tests\\runs\\capture\\notebook\"", "exit": 0, "stdout": "", "stderr": "", "error": null} | ||
| 2 | {"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-18994D02 -ExpectedPages 1", "exit": 0, "stdout": "Read 1 sections and 1 pages.\r\n", "stderr": "", "error": null} | ||
| 3 | {"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null} | ||
corpus/math-edit/native-drop/relocated/cold/machine.json created+1
| ... | @@ -0,0 +1 @@ | ||
| 1 | {"name": "m6-18994d02", "hostname": "ONE-M6-18994D02"} | ||
corpus/math-edit/native-drop/relocated/cold/notebook/Open Notebook.onetoc2 created
| Binary files /dev/null and b/corpus/math-edit/native-drop/relocated/cold/notebook/Open Notebook.onetoc2 differ | |||
corpus/math-edit/native-drop/relocated/cold/notebook/four.one created
| Binary files /dev/null and b/corpus/math-edit/native-drop/relocated/cold/notebook/four.one differ | |||
corpus/math-edit/native-drop/relocated/cold/read/environment.json created+7
| ... | @@ -0,0 +1,7 @@ | ||
| 1 | { | ||
| 2 | "powershell": "5.1.14409.1005", | ||
| 3 | "schema": "xs2010", | ||
| 4 | "hostname": "ONE-M6-18994D02", | ||
| 5 | "cold": true, | ||
| 6 | "onenote": "14.0.4763.1000" | ||
| 7 | } | ||
corpus/math-edit/native-drop/relocated/cold/read/hierarchy.xml created+2
| ... | @@ -0,0 +1,2 @@ | ||
| 1 | <?xml version="1.0"?> | ||
| 2 | <one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{7BE43A65-667D-4023-9E00-AB2E51D0A1E2}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-13T02:26:45.000Z" color="#B49EDE"><one:Section name="four" ID="{0F192688-780C-0DCD-1E42-F056B20E4F54}{1}{B0}" path="C:\one-tests\runs\capture\notebook\four.one" lastModifiedTime="2026-09-13T02:26:45.000Z" color="#8AA8E4"><one:Page ID="{E7334301-54C6-0E41-23A6-A12ECC211196}{1}{B0}" name="Equations below" dateTime="2026-09-13T02:12:13.000Z" lastModifiedTime="2026-09-13T02:26:45.000Z" pageLevel="1"/></one:Section></one:Notebook> | ||
corpus/math-edit/native-drop/relocated/cold/read/page-000.xml created+3
| ... | @@ -0,0 +1,3 @@ | ||
| 1 | <?xml version="1.0"?> | ||
| 2 | <one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{E7334301-54C6-0E41-23A6-A12ECC211196}{1}{B0}" name="Equations below" dateTime="2026-09-13T02:12:13.000Z" lastModifiedTime="2026-09-13T02:26:45.000Z" pageLevel="1" style="font-family:Calibri;font-size:11.0pt" lang="en-US"><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="Author" lastModifiedBy="Author" lastModifiedTime="2026-09-13T02:12:13.000Z" objectID="{0F192688-780C-0DCD-1E42-F056B20E4F54}{23}{B0}"><one:Position x="36.0" y="36.0" z="0"/><one:Size width="140.9678649902344" height="208.9916534423828"/><one:Indents><one:Indent level="0" indent="0.0"/></one:Indents><one:OEChildren><one:OE creationTime="2026-09-13T02:12:13.000Z" lastModifiedTime="2026-09-13T02:12:13.000Z" objectID="{0F192688-780C-0DCD-1E42-F056B20E4F54}{24}{B0}" alignment="left"><one:T><![CDATA[Equations below]]></one:T></one:OE><one:OE author="Math author" lastModifiedBy="Math author" creationTime="2026-09-13T02:12:13.000Z" lastModifiedTime="2026-09-13T02:12:13.000Z" objectID="{175469C1-8083-096A-1981-021991CBD5CA}{0}{B0}" alignment="left" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:msup><mml:mi>x</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mo>+</mml:mo><mml:msup><mml:mi>y</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mo>=</mml:mo><mml:msup><mml:mi>z</mml:mi><mml:mn>2</mml:mn></mml:msup></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Math author" lastModifiedBy="Math author" creationTime="2026-09-13T02:12:13.000Z" lastModifiedTime="2026-09-13T02:12:13.000Z" objectID="{D2634F73-9456-0519-30FF-2189C8ED2B9D}{0}{B0}" alignment="left" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:mfrac><mml:mrow><mml:mi>a</mml:mi><mml:mo>+</mml:mo><mml:mi>b</mml:mi></mml:mrow><mml:mrow><mml:mi>c</mml:mi><mml:mo>+</mml:mo><mml:mi>d</mml:mi></mml:mrow></mml:mfrac></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Math author" lastModifiedBy="Math author" creationTime="2026-09-13T02:12:13.000Z" lastModifiedTime="2026-09-13T02:12:13.000Z" objectID="{3A1E4DFD-7EC3-0D5B-1D07-36788E594E6E}{0}{B0}" alignment="left" style="font-size:11.0pt"><one:T><![CDATA[<span | ||
| 3 | style='font-family:Calibri' lang=en-US>Inline </span><!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>𝛼</mml:mi><mml:mo>+</mml:mo><mml:mi>𝛽</mml:mi></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Math author" lastModifiedBy="Math author" creationTime="2026-09-13T02:12:13.000Z" lastModifiedTime="2026-09-13T02:12:13.000Z" objectID="{CC13A3D0-417E-0950-21FF-467F4ED5B5BF}{0}{B0}" alignment="left" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:msub><mml:mi>a</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo>+</mml:mo><mml:msub><mml:mi>b</mml:mi><mml:mn>2</mml:mn></mml:msub></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Math author" lastModifiedBy="Math author" creationTime="2026-09-13T02:12:13.000Z" lastModifiedTime="2026-09-13T02:12:13.000Z" objectID="{3BC012E9-2021-0D8C-1B8F-3226D224B969}{0}{B0}" alignment="left" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:msubsup><mml:mi>x</mml:mi><mml:mi>i</mml:mi><mml:mn>2</mml:mn></mml:msubsup></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Math author" lastModifiedBy="Math author" creationTime="2026-09-13T02:12:13.000Z" lastModifiedTime="2026-09-13T02:12:13.000Z" objectID="{4E583B73-54CD-0B4A-3E73-8F349A4C0F03}{0}{B0}" alignment="left" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:mfenced><mml:mrow><mml:mi>a</mml:mi><mml:mo>+</mml:mo><mml:mi>b</mml:mi></mml:mrow></mml:mfenced></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Math author" lastModifiedBy="Math author" creationTime="2026-09-13T02:12:13.000Z" lastModifiedTime="2026-09-13T02:12:13.000Z" objectID="{933EFD95-F50E-05F3-0D2D-9B687A5B4159}{0}{B0}" alignment="left" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:msubsup><mml:mo stretchy="false">∫</mml:mo><mml:mn>0</mml:mn><mml:mn>1</mml:mn></mml:msubsup><mml:mrow><mml:mi>x</mml:mi><mml:mo> </mml:mo><mml:mi>d</mml:mi><mml:mi>x</mml:mi><mml:mo> </mml:mo></mml:mrow></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Math author" lastModifiedBy="Math author" creationTime="2026-09-13T02:12:13.000Z" lastModifiedTime="2026-09-13T02:12:13.000Z" objectID="{E78D1E9F-C931-02ED-1080-59CDE4E2920B}{0}{B0}" alignment="left" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:munderover><mml:mo stretchy="false">∑</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>n</mml:mi></mml:munderover><mml:mrow><mml:mi>i</mml:mi><mml:mo> </mml:mo></mml:mrow></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Math author" lastModifiedBy="Math author" creationTime="2026-09-13T02:12:13.000Z" lastModifiedTime="2026-09-13T02:12:13.000Z" objectID="{D9BAEF2E-0D0F-07A9-36E1-7D3D15DB92F7}{0}{B0}" alignment="left" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:msup><mml:mi>x</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Math author" lastModifiedBy="Math author" creationTime="2026-09-13T02:12:13.000Z" lastModifiedTime="2026-09-13T02:12:13.000Z" objectID="{CEEE4C3F-6A06-0AF1-11F5-BA3E807E9391}{0}{B0}" alignment="left" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:mfrac><mml:mrow><mml:mi>a</mml:mi><mml:mo>+</mml:mo><mml:mi>b</mml:mi></mml:mrow><mml:mrow><mml:msup><mml:mi>c</mml:mi><mml:mn>2</mml:mn></mml:msup></mml:mrow></mml:mfrac></mml:math><![endif]-->]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page> | ||
corpus/math-edit/native-drop/relocated/cold/read/payloads.json created+3
| ... | @@ -0,0 +1,3 @@ | ||
| 1 | [ | ||
| 2 | |||
| 3 | ] | ||
| \ No newline at end of file | |||
corpus/math-edit/native-drop/relocated/cold/run.json created+18
| ... | @@ -0,0 +1,18 @@ | ||
| 1 | { | ||
| 2 | "notebook": "/private/tmp/m4/reloc", | ||
| 3 | "expected_pages": 1, | ||
| 4 | "author": null, | ||
| 5 | "author_timeout_seconds": 600, | ||
| 6 | "inspect": false, | ||
| 7 | "collect_notebook": true, | ||
| 8 | "base": { | ||
| 9 | "file": "win7-office-base.qcow2", | ||
| 10 | "format": "qcow2", | ||
| 11 | "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346", | ||
| 12 | "virtual_size": 68719476736 | ||
| 13 | }, | ||
| 14 | "scripts": { | ||
| 15 | "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331", | ||
| 16 | "read.ps1": "04013bfcccee40a17e2a225f9b9e96f40a3a8daad9e350609654f8eda3ccbb41" | ||
| 17 | } | ||
| 18 | } | ||
corpus/math-edit/native-drop/relocated/cold/scripts/cold.ps1 created+27
| ... | @@ -0,0 +1,27 @@ | ||
| 1 | param([Parameter(Mandatory=$true)][string]$Root, [string]$CloneHost = '') | ||
| 2 | Set-StrictMode -Version Latest | ||
| 3 | $ErrorActionPreference = 'Stop' | ||
| 4 | $root = [IO.Path]::GetFullPath($Root).TrimEnd('\') | ||
| 5 | if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') { | ||
| 6 | throw 'Choose a run directly below C:\one-tests\runs.' | ||
| 7 | } | ||
| 8 | if (Get-Process ONENOTE -ErrorAction SilentlyContinue) { throw 'Close OneNote before resetting its test cache.' } | ||
| 9 | $key = 'HKCU:\Software\Microsoft\Office\14.0\OneNote' | ||
| 10 | if ($CloneHost) { | ||
| 11 | if ($CloneHost -notmatch '^ONE-[A-Z0-9-]+$' -or [Environment]::MachineName -ne $CloneHost) { | ||
| 12 | throw 'The disposable clone hostname does not match this machine.' | ||
| 13 | } | ||
| 14 | New-Item "$key\Options\Paths" -Force | Out-Null | ||
| 15 | New-ItemProperty "$key\Options\Paths" -Name UnfiledNotesSection -PropertyType ExpandString -Value 'C:\one-tests\Loose.one' -Force | Out-Null | ||
| 16 | } elseif ((Get-ItemProperty "$key\Options\Paths").UnfiledNotesSection -ne 'C:\one-tests\Loose.one' -or | ||
| 17 | -not (Test-Path 'C:\one-tests\profile-original-cache')) { | ||
| 18 | throw 'Park the personal OneNote profile before resetting the test cache.' | ||
| 19 | } | ||
| 20 | $cache = Join-Path $env:LOCALAPPDATA 'Microsoft\OneNote\14.0' | ||
| 21 | $parked = Join-Path 'C:\one-tests\caches' ([IO.Path]::GetFileName($root)) | ||
| 22 | if (Test-Path $parked) { throw 'Choose a new run; its parked cache already exists.' } | ||
| 23 | New-Item -ItemType Directory -Path 'C:\one-tests\caches' -Force | Out-Null | ||
| 24 | if (Test-Path $cache) { Move-Item -LiteralPath $cache -Destination $parked } | ||
| 25 | if (Test-Path "$key\OpenNotebooks") { Remove-Item "$key\OpenNotebooks" -Recurse } | ||
| 26 | New-Item "$key\OpenNotebooks" | Out-Null | ||
| 27 | New-ItemProperty "$key\OpenNotebooks" -Name '1' -PropertyType String -Value "$root\notebook" | Out-Null | ||
corpus/math-edit/native-drop/relocated/cold/scripts/read.ps1 created+142
| ... | @@ -0,0 +1,142 @@ | ||
| 1 | param( | ||
| 2 | [Parameter(Mandatory=$true)][string]$Root, | ||
| 3 | [int]$ExpectedPages = -1, | ||
| 4 | [switch]$UseCurrentCache, | ||
| 5 | [switch]$Pdf, | ||
| 6 | [switch]$KeepOpen, | ||
| 7 | [string]$CloneHost = '' | ||
| 8 | ) | ||
| 9 | Set-StrictMode -Version Latest | ||
| 10 | $ErrorActionPreference = 'Stop' | ||
| 11 | $root = [IO.Path]::GetFullPath($Root).TrimEnd('\') | ||
| 12 | if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') { | ||
| 13 | throw 'Choose a run directly below C:\one-tests\runs.' | ||
| 14 | } | ||
| 15 | $notebook = Join-Path $root 'notebook' | ||
| 16 | $output = Join-Path $root 'read' | ||
| 17 | if (Test-Path $output) { throw 'Choose a new read destination.' } | ||
| 18 | if ($UseCurrentCache) { | ||
| 19 | if ((Get-ItemProperty 'HKCU:\Software\Microsoft\Office\14.0\OneNote\Options\Paths').UnfiledNotesSection -ne 'C:\one-tests\Loose.one') { | ||
| 20 | throw 'Park the personal OneNote profile before reading test notebooks.' | ||
| 21 | } | ||
| 22 | } else { | ||
| 23 | & "$PSScriptRoot\cold-current.ps1" -Root $root -CloneHost $CloneHost | ||
| 24 | } | ||
| 25 | New-Item -ItemType Directory -Path $output | Out-Null | ||
| 26 | $app = New-Object -ComObject OneNote.Application | ||
| 27 | $notebookId = '' | ||
| 28 | $failure = $null | ||
| 29 | try { | ||
| 30 | $app.OpenHierarchy($notebook, '', [ref]$notebookId, 0) | ||
| 31 | $process = Get-Process ONENOTE | ||
| 32 | @{ hostname = [Environment]::MachineName; onenote = $process.MainModule.FileVersionInfo.FileVersion; | ||
| 33 | powershell = $PSVersionTable.PSVersion.ToString(); schema = 'xs2010'; cold = (-not $UseCurrentCache.IsPresent) } | | ||
| 34 | ConvertTo-Json | Set-Content (Join-Path $output 'environment.json') -Encoding UTF8 | ||
| 35 | $sections = @() | ||
| 36 | foreach ($file in @(Get-ChildItem $notebook -Recurse | Where-Object { $_.Extension -eq '.one' })) { | ||
| 37 | $id = '' | ||
| 38 | $app.OpenHierarchy($file.FullName, '', [ref]$id, 0) | ||
| 39 | $sections += $id | ||
| 40 | } | ||
| 41 | $deadline = [DateTime]::UtcNow.AddSeconds(300) | ||
| 42 | $previous = '' | ||
| 43 | $lastChange = '' | ||
| 44 | $stableSince = [DateTime]::UtcNow | ||
| 45 | $settled = $false | ||
| 46 | do { | ||
| 47 | $pages = @{} | ||
| 48 | foreach ($section in $sections) { | ||
| 49 | $hierarchy = '' | ||
| 50 | $app.GetHierarchy($section, 4, [ref]$hierarchy, 1) | ||
| 51 | [xml]$xml = $hierarchy | ||
| 52 | foreach ($node in $xml.SelectNodes('//*[@path]')) { | ||
| 53 | if (-not $node.GetAttribute('path').StartsWith("$notebook\", [StringComparison]::OrdinalIgnoreCase)) { | ||
| 54 | throw 'OneNote opened a section outside the copied notebook.' | ||
| 55 | } | ||
| 56 | } | ||
| 57 | foreach ($node in $xml.SelectNodes('//*[local-name()="Page"]')) { | ||
| 58 | $id = $node.GetAttribute('ID') | ||
| 59 | $content = '' | ||
| 60 | $app.GetPageContent($id, [ref]$content, 1, 1) | ||
| 61 | $pages[$id] = $content | ||
| 62 | } | ||
| 63 | } | ||
| 64 | $signature = [String]::Join('|', @($pages.Keys | Sort-Object | ForEach-Object { $_ + $pages[$_] })) | ||
| 65 | if ($signature -ne $previous) { | ||
| 66 | $lastChange = $previous | ||
| 67 | $previous = $signature | ||
| 68 | $stableSince = [DateTime]::UtcNow | ||
| 69 | } | ||
| 70 | if ((($ExpectedPages -ge 0 -and $pages.Count -eq $ExpectedPages) -or | ||
| 71 | ($ExpectedPages -lt 0 -and $pages.Count -gt 0)) -and | ||
| 72 | ([DateTime]::UtcNow - $stableSince).TotalSeconds -ge 2) { $settled = $true; break } | ||
| 73 | Start-Sleep -Milliseconds 250 | ||
| 74 | } while ([DateTime]::UtcNow -lt $deadline) | ||
| 75 | if (-not $settled -or ($ExpectedPages -ge 0 -and $pages.Count -ne $ExpectedPages) -or ($ExpectedPages -lt 0 -and $pages.Count -eq 0)) { | ||
| 76 | [IO.File]::WriteAllText((Join-Path $output 'previous-signature.txt'), $lastChange, [Text.Encoding]::UTF8) | ||
| 77 | $index = 0 | ||
| 78 | foreach ($id in @($pages.Keys | Sort-Object)) { | ||
| 79 | [IO.File]::WriteAllText((Join-Path $output ('unsettled-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8) | ||
| 80 | $index++ | ||
| 81 | } | ||
| 82 | $hierarchy = '' | ||
| 83 | $app.GetHierarchy($notebookId, 4, [ref]$hierarchy, 1) | ||
| 84 | [IO.File]::WriteAllText((Join-Path $output 'unsettled-hierarchy.xml'), $hierarchy, [Text.Encoding]::UTF8) | ||
| 85 | throw "Expected $ExpectedPages stable pages; OneNote returned $($pages.Count), settled=$settled." | ||
| 86 | } | ||
| 87 | $index = 0 | ||
| 88 | $payloads = @() | ||
| 89 | foreach ($id in @($pages.Keys | Sort-Object)) { | ||
| 90 | [IO.File]::WriteAllText((Join-Path $output ('page-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8) | ||
| 91 | if ($Pdf) { | ||
| 92 | $pdfPath = Join-Path $output ('page-{0:d3}.pdf' -f $index) | ||
| 93 | $app.NavigateTo($id, '', $false) | ||
| 94 | $app.Publish($id, $pdfPath, 3, '') | ||
| 95 | if (-not (Test-Path $pdfPath) -or (Get-Item $pdfPath).Length -eq 0) { throw 'OneNote did not publish the page PDF.' } | ||
| 96 | } | ||
| 97 | [xml]$page = $pages[$id] | ||
| 98 | foreach ($file in $page.SelectNodes('//*[local-name()="InsertedFile" or local-name()="MediaFile"]')) { | ||
| 99 | $bytes = [IO.File]::ReadAllBytes($file.GetAttribute('pathCache')) | ||
| 100 | $hash = [BitConverter]::ToString([Security.Cryptography.SHA256]::Create().ComputeHash($bytes)).Replace('-', '').ToLowerInvariant() | ||
| 101 | [IO.File]::WriteAllBytes((Join-Path $output ($hash + '.attachment')), $bytes) | ||
| 102 | $payloads += @{ page = $id; object = $file.ParentNode.GetAttribute('objectID'); | ||
| 103 | kind = $file.LocalName; name = $file.GetAttribute('preferredName'); | ||
| 104 | sha256 = $hash; bytes = $bytes.Length } | ||
| 105 | } | ||
| 106 | $index++ | ||
| 107 | } | ||
| 108 | [IO.File]::WriteAllText((Join-Path $output 'payloads.json'), (ConvertTo-Json -InputObject $payloads -Depth 4), [Text.Encoding]::UTF8) | ||
| 109 | $all = '' | ||
| 110 | $app.GetHierarchy($notebookId, 4, [ref]$all, 1) | ||
| 111 | [xml]$finalTree = $all | ||
| 112 | $finalIds = @($finalTree.SelectNodes('//*[local-name()="Page"]') | ForEach-Object { $_.GetAttribute('ID') } | Sort-Object -Unique) | ||
| 113 | if ($finalIds.Count -ne $pages.Count -or @($finalIds | Where-Object { -not $pages.ContainsKey($_) }).Count -ne 0) { | ||
| 114 | throw 'The notebook hierarchy changed while collecting page evidence; repeat the cold read.' | ||
| 115 | } | ||
| 116 | [IO.File]::WriteAllText((Join-Path $output 'hierarchy.xml'), $all, [Text.Encoding]::UTF8) | ||
| 117 | Write-Output "Read $($sections.Count) sections and $($pages.Count) pages." | ||
| 118 | } catch { | ||
| 119 | $failure = $_ | ||
| 120 | [IO.File]::WriteAllText((Join-Path $output 'failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8) | ||
| 121 | throw | ||
| 122 | } finally { | ||
| 123 | try { | ||
| 124 | try { | ||
| 125 | if ($notebookId -and $CloneHost) { $app.SyncHierarchy($notebookId) } | ||
| 126 | if ($notebookId -and -not $KeepOpen -and (-not $UseCurrentCache -or $CloneHost)) { | ||
| 127 | $app.CloseNotebook($notebookId, $false) | ||
| 128 | } | ||
| 129 | } catch { | ||
| 130 | if ($null -eq $failure) { throw } | ||
| 131 | [IO.File]::WriteAllText((Join-Path $output 'cleanup-failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8) | ||
| 132 | } | ||
| 133 | } finally { | ||
| 134 | [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app) | ||
| 135 | } | ||
| 136 | $app = $null | ||
| 137 | [GC]::Collect() | ||
| 138 | [GC]::WaitForPendingFinalizers() | ||
| 139 | if (-not $UseCurrentCache -and -not $CloneHost) { | ||
| 140 | Get-Process ONENOTE -ErrorAction SilentlyContinue | Wait-Process -Timeout 10 | ||
| 141 | } | ||
| 142 | } | ||
corpus/math-edit/native-drop/relocated/cold/source.json created+14
| ... | @@ -0,0 +1,14 @@ | ||
| 1 | [ | ||
| 2 | { | ||
| 3 | "path": "Open Notebook.onetoc2", | ||
| 4 | "bytes": 1596, | ||
| 5 | "sha256": "3362ba38101c160a3089a7c35655071f3fefa99af400ee1b93ee9d2a6606382f", | ||
| 6 | "mtime_ns": 1789266331848783406 | ||
| 7 | }, | ||
| 8 | { | ||
| 9 | "path": "four.one", | ||
| 10 | "bytes": 14512, | ||
| 11 | "sha256": "70b21f337a82ff1ac8c6241a7c683f83282a863bdcf2aff70b36be6b9bf279ab", | ||
| 12 | "mtime_ns": 1789266316867267531 | ||
| 13 | } | ||
| 14 | ] | ||
corpus/math-edit/native-drop/relocated/cold/teardown.json created+1
| ... | @@ -0,0 +1 @@ | ||
| 1 | {"absent": true} | ||
corpus/math-edit/native-editor/before-read/environment.json created+7
| ... | @@ -0,0 +1,7 @@ | ||
| 1 | { | ||
| 2 | "powershell": "5.1.14409.1005", | ||
| 3 | "schema": "xs2010", | ||
| 4 | "hostname": "ONE-M6-E6AF28E4", | ||
| 5 | "cold": true, | ||
| 6 | "onenote": "14.0.4763.1000" | ||
| 7 | } | ||
corpus/math-edit/native-editor/before-read/hierarchy.xml created+2
| ... | @@ -0,0 +1,2 @@ | ||
| 1 | <?xml version="1.0"?> | ||
| 2 | <one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{7C6B7CB1-98A6-442C-A595-51A386DEDAD8}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-13T01:01:50.000Z" color="#B49EDE"><one:Section name="links" ID="{B06E7F4D-5B2D-0A13-3000-8A5BFA81A3AD}{1}{B0}" path="C:\one-tests\runs\capture\notebook\links.one" lastModifiedTime="2026-09-13T01:01:50.000Z" color="#8AA8E4"><one:Page ID="{1163B9C4-333A-052E-26AB-AA3B02622A56}{1}{B0}" name="Read about Rust HYPERLINK "https://example.invalid/rust"the Rust site" dateTime="2026-09-12T09:49:16.000Z" lastModifiedTime="2026-09-13T01:01:50.000Z" pageLevel="1"/></one:Section></one:Notebook> | ||
corpus/math-edit/native-editor/before-read/page-000.xml created+3
| ... | @@ -0,0 +1,3 @@ | ||
| 1 | <?xml version="1.0"?> | ||
| 2 | <one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{1163B9C4-333A-052E-26AB-AA3B02622A56}{1}{B0}" name="Read about Rust HYPERLINK "https://example.invalid/rust"the Rust site" dateTime="2026-09-12T09:49:16.000Z" lastModifiedTime="2026-09-13T01:01:50.000Z" pageLevel="1" style="font-family:Calibri;font-size:11.0pt" lang="en-US"><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="Author" lastModifiedBy="Author" lastModifiedTime="2026-09-12T09:49:16.000Z" objectID="{B06E7F4D-5B2D-0A13-3000-8A5BFA81A3AD}{23}{B0}"><one:Position x="36.0" y="36.0" z="0"/><one:Size width="198.3904571533203" height="13.4277172088623"/><one:Indents><one:Indent level="0" indent="0.0"/></one:Indents><one:OEChildren><one:OE creationTime="2026-09-12T09:49:16.000Z" lastModifiedTime="2026-09-12T09:49:16.000Z" objectID="{B06E7F4D-5B2D-0A13-3000-8A5BFA81A3AD}{24}{B0}" alignment="left"><one:T><![CDATA[Read about Rust <a | ||
| 3 | href="https://example.invalid/rust">the Rust site</a>]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page> | ||
corpus/math-edit/native-editor/before-read/payloads.json created+3
| ... | @@ -0,0 +1,3 @@ | ||
| 1 | [ | ||
| 2 | |||
| 3 | ] | ||
| \ No newline at end of file | |||
corpus/math-edit/native-editor/commands.jsonl created+6
| ... | @@ -0,0 +1,6 @@ | ||
| 1 | {"command": "powershell -NoProfile -Command \"Expand-Archive -LiteralPath C:\\one-tests\\transfer.zip -DestinationPath C:\\one-tests\\runs\\capture\\notebook\"", "exit": 0, "stdout": "", "stderr": "", "error": null} | ||
| 2 | {"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-E6AF28E4 -ExpectedPages 1 -KeepOpen", "exit": 0, "stdout": "Read 1 sections and 1 pages.\r\n", "stderr": "", "error": null} | ||
| 3 | {"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null} | ||
| 4 | {"command": "powershell -NoProfile -Command \"Rename-Item C:\\one-tests\\runs\\capture\\read before-read\"", "exit": 0, "stdout": "", "stderr": "", "error": null} | ||
| 5 | {"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-E6AF28E4 -UseCurrentCache -ExpectedPages -1", "exit": 0, "stdout": "Read 1 sections and 1 pages.\r\n", "stderr": "", "error": null} | ||
| 6 | {"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null} | ||
corpus/math-edit/native-editor/equations.ahk created+68
| ... | @@ -0,0 +1,68 @@ | ||
| 1 | ComObject("OneNote.Application").NavigateTo("{1163B9C4-333A-052E-26AB-AA3B02622A56}{1}{B0}", "", false) | ||
| 2 | WinWait("ahk_exe ONENOTE.EXE", , 30) | ||
| 3 | WinActivate("ahk_exe ONENOTE.EXE") | ||
| 4 | WinWaitActive("ahk_exe ONENOTE.EXE", , 30) | ||
| 5 | WinMaximize("ahk_exe ONENOTE.EXE") | ||
| 6 | Sleep(3000) | ||
| 7 | CoordMode("Mouse", "Screen") | ||
| 8 | Click(300, 140) | ||
| 9 | Sleep(500) | ||
| 10 | Send("{End}") | ||
| 11 | Sleep(300) | ||
| 12 | Send("{Enter}") | ||
| 13 | Sleep(500) | ||
| 14 | Send("!=") | ||
| 15 | Sleep(600) | ||
| 16 | SendText("a_1+b_2") | ||
| 17 | Send(" ") | ||
| 18 | Sleep(600) | ||
| 19 | Send("{End}") | ||
| 20 | Sleep(300) | ||
| 21 | Send("{Enter}") | ||
| 22 | Sleep(600) | ||
| 23 | Send("!=") | ||
| 24 | Sleep(600) | ||
| 25 | SendText("\sqrt(x+1)") | ||
| 26 | Send(" ") | ||
| 27 | Sleep(600) | ||
| 28 | Send("{End}") | ||
| 29 | Sleep(300) | ||
| 30 | Send("{Enter}") | ||
| 31 | Sleep(600) | ||
| 32 | Send("!=") | ||
| 33 | Sleep(600) | ||
| 34 | SendText("x_i^2") | ||
| 35 | Send(" ") | ||
| 36 | Sleep(600) | ||
| 37 | Send("{End}") | ||
| 38 | Sleep(300) | ||
| 39 | Send("{Enter}") | ||
| 40 | Sleep(600) | ||
| 41 | Send("!=") | ||
| 42 | Sleep(600) | ||
| 43 | SendText("(a+b)") | ||
| 44 | Send(" ") | ||
| 45 | Sleep(600) | ||
| 46 | Send("{End}") | ||
| 47 | Sleep(300) | ||
| 48 | Send("{Enter}") | ||
| 49 | Sleep(600) | ||
| 50 | Send("!=") | ||
| 51 | Sleep(600) | ||
| 52 | SendText("\int_0^1 x dx") | ||
| 53 | Send(" ") | ||
| 54 | Sleep(600) | ||
| 55 | Send("{End}") | ||
| 56 | Sleep(300) | ||
| 57 | Send("{Enter}") | ||
| 58 | Sleep(600) | ||
| 59 | Send("!=") | ||
| 60 | Sleep(600) | ||
| 61 | SendText("\sum_(i=1)^n i") | ||
| 62 | Send(" ") | ||
| 63 | Sleep(600) | ||
| 64 | Send("{End}") | ||
| 65 | Sleep(300) | ||
| 66 | Send("{Enter}") | ||
| 67 | Sleep(600) | ||
| 68 | Sleep(2000) | ||
corpus/math-edit/native-editor/equations.json created+13
| ... | @@ -0,0 +1,13 @@ | ||
| 1 | { | ||
| 2 | "exit": 0, | ||
| 3 | "stdout": "", | ||
| 4 | "stderr": "", | ||
| 5 | "w": 800, | ||
| 6 | "h": 600, | ||
| 7 | "error": null, | ||
| 8 | "win": { | ||
| 9 | "title": "Read about Rust the Rust site - Microsoft OneNote", | ||
| 10 | "class": "Framework::CFrame", | ||
| 11 | "dialog": false | ||
| 12 | } | ||
| 13 | } | ||
| \ No newline at end of file | |||
corpus/math-edit/native-editor/equations.png created
| Binary files /dev/null and b/corpus/math-edit/native-editor/equations.png differ | |||
corpus/math-edit/native-editor/machine.json created+1
| ... | @@ -0,0 +1 @@ | ||
| 1 | {"name": "m6-e6af28e4", "hostname": "ONE-M6-E6AF28E4"} | ||
corpus/math-edit/native-editor/notebook/Open Notebook.onetoc2 created
| Binary files /dev/null and b/corpus/math-edit/native-editor/notebook/Open Notebook.onetoc2 differ | |||
corpus/math-edit/native-editor/notebook/links.one created
| Binary files /dev/null and b/corpus/math-edit/native-editor/notebook/links.one differ | |||
corpus/math-edit/native-editor/read/environment.json created+7
| ... | @@ -0,0 +1,7 @@ | ||
| 1 | { | ||
| 2 | "powershell": "5.1.14409.1005", | ||
| 3 | "schema": "xs2010", | ||
| 4 | "hostname": "ONE-M6-E6AF28E4", | ||
| 5 | "cold": false, | ||
| 6 | "onenote": "14.0.4763.1000" | ||
| 7 | } | ||
corpus/math-edit/native-editor/read/hierarchy.xml created+2
| ... | @@ -0,0 +1,2 @@ | ||
| 1 | <?xml version="1.0"?> | ||
| 2 | <one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{7C6B7CB1-98A6-442C-A595-51A386DEDAD8}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-13T01:03:49.000Z" color="#B49EDE" isCurrentlyViewed="true"><one:Section name="links" ID="{B06E7F4D-5B2D-0A13-3000-8A5BFA81A3AD}{1}{B0}" path="C:\one-tests\runs\capture\notebook\links.one" lastModifiedTime="2026-09-13T01:03:49.000Z" color="#8AA8E4" isCurrentlyViewed="true"><one:Page ID="{1163B9C4-333A-052E-26AB-AA3B02622A56}{1}{B0}" name="Read about Rust the Rust site" dateTime="2026-09-12T09:49:16.000Z" lastModifiedTime="2026-09-13T01:03:49.000Z" pageLevel="1" isCurrentlyViewed="true"/></one:Section></one:Notebook> | ||
corpus/math-edit/native-editor/read/page-000.xml created+3
| ... | @@ -0,0 +1,3 @@ | ||
| 1 | <?xml version="1.0"?> | ||
| 2 | <one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{1163B9C4-333A-052E-26AB-AA3B02622A56}{1}{B0}" name="Read about Rust the Rust site" dateTime="2026-09-12T09:49:16.000Z" lastModifiedTime="2026-09-13T01:03:49.000Z" pageLevel="1" isCurrentlyViewed="true" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="Author" lastModifiedBy="Author" lastModifiedTime="2026-09-13T01:03:49.000Z" objectID="{B06E7F4D-5B2D-0A13-3000-8A5BFA81A3AD}{23}{B0}"><one:Position x="36.0" y="36.0" z="0"/><one:Size width="198.3904571533203" height="145.7231597900391"/><one:Indents><one:Indent level="0" indent="0.0"/></one:Indents><one:OEChildren><one:OE creationTime="2026-09-12T09:49:16.000Z" lastModifiedTime="2026-09-13T01:03:33.000Z" objectID="{B06E7F4D-5B2D-0A13-3000-8A5BFA81A3AD}{24}{B0}" alignment="left" style="font-family:Calibri;font-size:11.0pt"><one:T><![CDATA[Read about Rust <a | ||
| 3 | href="https://example.invalid/rust">the Rust site</a>]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-13T01:03:37.000Z" lastModifiedTime="2026-09-13T01:03:38.000Z" objectID="{A2764782-4E9B-4625-8DF8-41CD09100670}{12}{B0}" alignment="left" quickStyleIndex="0" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:msub><mml:mi>a</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo>+</mml:mo><mml:msub><mml:mi>b</mml:mi><mml:mn>2</mml:mn></mml:msub></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-13T01:03:38.000Z" lastModifiedTime="2026-09-13T01:03:38.000Z" objectID="{A2764782-4E9B-4625-8DF8-41CD09100670}{18}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-13T01:03:41.000Z" lastModifiedTime="2026-09-13T01:03:42.000Z" objectID="{A2764782-4E9B-4625-8DF8-41CD09100670}{21}{B0}" alignment="left" quickStyleIndex="0" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:msubsup><mml:mi>x</mml:mi><mml:mi>i</mml:mi><mml:mn>2</mml:mn></mml:msubsup></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-13T01:03:43.000Z" lastModifiedTime="2026-09-13T01:03:45.000Z" objectID="{A2764782-4E9B-4625-8DF8-41CD09100670}{24}{B0}" alignment="left" quickStyleIndex="0" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:mfenced><mml:mrow><mml:mi>a</mml:mi><mml:mo>+</mml:mo><mml:mi>b</mml:mi></mml:mrow></mml:mfenced></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-13T01:03:46.000Z" lastModifiedTime="2026-09-13T01:03:47.000Z" objectID="{A2764782-4E9B-4625-8DF8-41CD09100670}{27}{B0}" alignment="left" quickStyleIndex="0" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:msubsup><mml:mo stretchy="false">∫</mml:mo><mml:mn>0</mml:mn><mml:mn>1</mml:mn></mml:msubsup><mml:mrow><mml:mi>x</mml:mi><mml:mo> </mml:mo><mml:mi>d</mml:mi><mml:mi>x</mml:mi><mml:mo> </mml:mo></mml:mrow></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-13T01:03:48.000Z" lastModifiedTime="2026-09-13T01:03:49.000Z" objectID="{A2764782-4E9B-4625-8DF8-41CD09100670}{30}{B0}" alignment="left" quickStyleIndex="0" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:munderover><mml:mo stretchy="false">∑</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>n</mml:mi></mml:munderover><mml:mrow><mml:mi>i</mml:mi><mml:mo> </mml:mo></mml:mrow></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-13T01:03:49.000Z" lastModifiedTime="2026-09-13T01:03:49.000Z" objectID="{A2764782-4E9B-4625-8DF8-41CD09100670}{33}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page> | ||
corpus/math-edit/native-editor/read/payloads.json created+3
| ... | @@ -0,0 +1,3 @@ | ||
| 1 | [ | ||
| 2 | |||
| 3 | ] | ||
| \ No newline at end of file | |||
corpus/math-edit/native-editor/run.json created+18
| ... | @@ -0,0 +1,18 @@ | ||
| 1 | { | ||
| 2 | "notebook": "/Users/clo/dev/one/corpus/link-edit/candidate", | ||
| 3 | "expected_pages": 1, | ||
| 4 | "author": null, | ||
| 5 | "author_timeout_seconds": 600, | ||
| 6 | "inspect": false, | ||
| 7 | "collect_notebook": true, | ||
| 8 | "base": { | ||
| 9 | "file": "win7-office-base.qcow2", | ||
| 10 | "format": "qcow2", | ||
| 11 | "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346", | ||
| 12 | "virtual_size": 68719476736 | ||
| 13 | }, | ||
| 14 | "scripts": { | ||
| 15 | "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331", | ||
| 16 | "read.ps1": "04013bfcccee40a17e2a225f9b9e96f40a3a8daad9e350609654f8eda3ccbb41" | ||
| 17 | } | ||
| 18 | } | ||
corpus/math-edit/native-editor/scripts/cold.ps1 created+27
| ... | @@ -0,0 +1,27 @@ | ||
| 1 | param([Parameter(Mandatory=$true)][string]$Root, [string]$CloneHost = '') | ||
| 2 | Set-StrictMode -Version Latest | ||
| 3 | $ErrorActionPreference = 'Stop' | ||
| 4 | $root = [IO.Path]::GetFullPath($Root).TrimEnd('\') | ||
| 5 | if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') { | ||
| 6 | throw 'Choose a run directly below C:\one-tests\runs.' | ||
| 7 | } | ||
| 8 | if (Get-Process ONENOTE -ErrorAction SilentlyContinue) { throw 'Close OneNote before resetting its test cache.' } | ||
| 9 | $key = 'HKCU:\Software\Microsoft\Office\14.0\OneNote' | ||
| 10 | if ($CloneHost) { | ||
| 11 | if ($CloneHost -notmatch '^ONE-[A-Z0-9-]+$' -or [Environment]::MachineName -ne $CloneHost) { | ||
| 12 | throw 'The disposable clone hostname does not match this machine.' | ||
| 13 | } | ||
| 14 | New-Item "$key\Options\Paths" -Force | Out-Null | ||
| 15 | New-ItemProperty "$key\Options\Paths" -Name UnfiledNotesSection -PropertyType ExpandString -Value 'C:\one-tests\Loose.one' -Force | Out-Null | ||
| 16 | } elseif ((Get-ItemProperty "$key\Options\Paths").UnfiledNotesSection -ne 'C:\one-tests\Loose.one' -or | ||
| 17 | -not (Test-Path 'C:\one-tests\profile-original-cache')) { | ||
| 18 | throw 'Park the personal OneNote profile before resetting the test cache.' | ||
| 19 | } | ||
| 20 | $cache = Join-Path $env:LOCALAPPDATA 'Microsoft\OneNote\14.0' | ||
| 21 | $parked = Join-Path 'C:\one-tests\caches' ([IO.Path]::GetFileName($root)) | ||
| 22 | if (Test-Path $parked) { throw 'Choose a new run; its parked cache already exists.' } | ||
| 23 | New-Item -ItemType Directory -Path 'C:\one-tests\caches' -Force | Out-Null | ||
| 24 | if (Test-Path $cache) { Move-Item -LiteralPath $cache -Destination $parked } | ||
| 25 | if (Test-Path "$key\OpenNotebooks") { Remove-Item "$key\OpenNotebooks" -Recurse } | ||
| 26 | New-Item "$key\OpenNotebooks" | Out-Null | ||
| 27 | New-ItemProperty "$key\OpenNotebooks" -Name '1' -PropertyType String -Value "$root\notebook" | Out-Null | ||
corpus/math-edit/native-editor/scripts/read.ps1 created+142
| ... | @@ -0,0 +1,142 @@ | ||
| 1 | param( | ||
| 2 | [Parameter(Mandatory=$true)][string]$Root, | ||
| 3 | [int]$ExpectedPages = -1, | ||
| 4 | [switch]$UseCurrentCache, | ||
| 5 | [switch]$Pdf, | ||
| 6 | [switch]$KeepOpen, | ||
| 7 | [string]$CloneHost = '' | ||
| 8 | ) | ||
| 9 | Set-StrictMode -Version Latest | ||
| 10 | $ErrorActionPreference = 'Stop' | ||
| 11 | $root = [IO.Path]::GetFullPath($Root).TrimEnd('\') | ||
| 12 | if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') { | ||
| 13 | throw 'Choose a run directly below C:\one-tests\runs.' | ||
| 14 | } | ||
| 15 | $notebook = Join-Path $root 'notebook' | ||
| 16 | $output = Join-Path $root 'read' | ||
| 17 | if (Test-Path $output) { throw 'Choose a new read destination.' } | ||
| 18 | if ($UseCurrentCache) { | ||
| 19 | if ((Get-ItemProperty 'HKCU:\Software\Microsoft\Office\14.0\OneNote\Options\Paths').UnfiledNotesSection -ne 'C:\one-tests\Loose.one') { | ||
| 20 | throw 'Park the personal OneNote profile before reading test notebooks.' | ||
| 21 | } | ||
| 22 | } else { | ||
| 23 | & "$PSScriptRoot\cold-current.ps1" -Root $root -CloneHost $CloneHost | ||
| 24 | } | ||
| 25 | New-Item -ItemType Directory -Path $output | Out-Null | ||
| 26 | $app = New-Object -ComObject OneNote.Application | ||
| 27 | $notebookId = '' | ||
| 28 | $failure = $null | ||
| 29 | try { | ||
| 30 | $app.OpenHierarchy($notebook, '', [ref]$notebookId, 0) | ||
| 31 | $process = Get-Process ONENOTE | ||
| 32 | @{ hostname = [Environment]::MachineName; onenote = $process.MainModule.FileVersionInfo.FileVersion; | ||
| 33 | powershell = $PSVersionTable.PSVersion.ToString(); schema = 'xs2010'; cold = (-not $UseCurrentCache.IsPresent) } | | ||
| 34 | ConvertTo-Json | Set-Content (Join-Path $output 'environment.json') -Encoding UTF8 | ||
| 35 | $sections = @() | ||
| 36 | foreach ($file in @(Get-ChildItem $notebook -Recurse | Where-Object { $_.Extension -eq '.one' })) { | ||
| 37 | $id = '' | ||
| 38 | $app.OpenHierarchy($file.FullName, '', [ref]$id, 0) | ||
| 39 | $sections += $id | ||
| 40 | } | ||
| 41 | $deadline = [DateTime]::UtcNow.AddSeconds(300) | ||
| 42 | $previous = '' | ||
| 43 | $lastChange = '' | ||
| 44 | $stableSince = [DateTime]::UtcNow | ||
| 45 | $settled = $false | ||
| 46 | do { | ||
| 47 | $pages = @{} | ||
| 48 | foreach ($section in $sections) { | ||
| 49 | $hierarchy = '' | ||
| 50 | $app.GetHierarchy($section, 4, [ref]$hierarchy, 1) | ||
| 51 | [xml]$xml = $hierarchy | ||
| 52 | foreach ($node in $xml.SelectNodes('//*[@path]')) { | ||
| 53 | if (-not $node.GetAttribute('path').StartsWith("$notebook\", [StringComparison]::OrdinalIgnoreCase)) { | ||
| 54 | throw 'OneNote opened a section outside the copied notebook.' | ||
| 55 | } | ||
| 56 | } | ||
| 57 | foreach ($node in $xml.SelectNodes('//*[local-name()="Page"]')) { | ||
| 58 | $id = $node.GetAttribute('ID') | ||
| 59 | $content = '' | ||
| 60 | $app.GetPageContent($id, [ref]$content, 1, 1) | ||
| 61 | $pages[$id] = $content | ||
| 62 | } | ||
| 63 | } | ||
| 64 | $signature = [String]::Join('|', @($pages.Keys | Sort-Object | ForEach-Object { $_ + $pages[$_] })) | ||
| 65 | if ($signature -ne $previous) { | ||
| 66 | $lastChange = $previous | ||
| 67 | $previous = $signature | ||
| 68 | $stableSince = [DateTime]::UtcNow | ||
| 69 | } | ||
| 70 | if ((($ExpectedPages -ge 0 -and $pages.Count -eq $ExpectedPages) -or | ||
| 71 | ($ExpectedPages -lt 0 -and $pages.Count -gt 0)) -and | ||
| 72 | ([DateTime]::UtcNow - $stableSince).TotalSeconds -ge 2) { $settled = $true; break } | ||
| 73 | Start-Sleep -Milliseconds 250 | ||
| 74 | } while ([DateTime]::UtcNow -lt $deadline) | ||
| 75 | if (-not $settled -or ($ExpectedPages -ge 0 -and $pages.Count -ne $ExpectedPages) -or ($ExpectedPages -lt 0 -and $pages.Count -eq 0)) { | ||
| 76 | [IO.File]::WriteAllText((Join-Path $output 'previous-signature.txt'), $lastChange, [Text.Encoding]::UTF8) | ||
| 77 | $index = 0 | ||
| 78 | foreach ($id in @($pages.Keys | Sort-Object)) { | ||
| 79 | [IO.File]::WriteAllText((Join-Path $output ('unsettled-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8) | ||
| 80 | $index++ | ||
| 81 | } | ||
| 82 | $hierarchy = '' | ||
| 83 | $app.GetHierarchy($notebookId, 4, [ref]$hierarchy, 1) | ||
| 84 | [IO.File]::WriteAllText((Join-Path $output 'unsettled-hierarchy.xml'), $hierarchy, [Text.Encoding]::UTF8) | ||
| 85 | throw "Expected $ExpectedPages stable pages; OneNote returned $($pages.Count), settled=$settled." | ||
| 86 | } | ||
| 87 | $index = 0 | ||
| 88 | $payloads = @() | ||
| 89 | foreach ($id in @($pages.Keys | Sort-Object)) { | ||
| 90 | [IO.File]::WriteAllText((Join-Path $output ('page-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8) | ||
| 91 | if ($Pdf) { | ||
| 92 | $pdfPath = Join-Path $output ('page-{0:d3}.pdf' -f $index) | ||
| 93 | $app.NavigateTo($id, '', $false) | ||
| 94 | $app.Publish($id, $pdfPath, 3, '') | ||
| 95 | if (-not (Test-Path $pdfPath) -or (Get-Item $pdfPath).Length -eq 0) { throw 'OneNote did not publish the page PDF.' } | ||
| 96 | } | ||
| 97 | [xml]$page = $pages[$id] | ||
| 98 | foreach ($file in $page.SelectNodes('//*[local-name()="InsertedFile" or local-name()="MediaFile"]')) { | ||
| 99 | $bytes = [IO.File]::ReadAllBytes($file.GetAttribute('pathCache')) | ||
| 100 | $hash = [BitConverter]::ToString([Security.Cryptography.SHA256]::Create().ComputeHash($bytes)).Replace('-', '').ToLowerInvariant() | ||
| 101 | [IO.File]::WriteAllBytes((Join-Path $output ($hash + '.attachment')), $bytes) | ||
| 102 | $payloads += @{ page = $id; object = $file.ParentNode.GetAttribute('objectID'); | ||
| 103 | kind = $file.LocalName; name = $file.GetAttribute('preferredName'); | ||
| 104 | sha256 = $hash; bytes = $bytes.Length } | ||
| 105 | } | ||
| 106 | $index++ | ||
| 107 | } | ||
| 108 | [IO.File]::WriteAllText((Join-Path $output 'payloads.json'), (ConvertTo-Json -InputObject $payloads -Depth 4), [Text.Encoding]::UTF8) | ||
| 109 | $all = '' | ||
| 110 | $app.GetHierarchy($notebookId, 4, [ref]$all, 1) | ||
| 111 | [xml]$finalTree = $all | ||
| 112 | $finalIds = @($finalTree.SelectNodes('//*[local-name()="Page"]') | ForEach-Object { $_.GetAttribute('ID') } | Sort-Object -Unique) | ||
| 113 | if ($finalIds.Count -ne $pages.Count -or @($finalIds | Where-Object { -not $pages.ContainsKey($_) }).Count -ne 0) { | ||
| 114 | throw 'The notebook hierarchy changed while collecting page evidence; repeat the cold read.' | ||
| 115 | } | ||
| 116 | [IO.File]::WriteAllText((Join-Path $output 'hierarchy.xml'), $all, [Text.Encoding]::UTF8) | ||
| 117 | Write-Output "Read $($sections.Count) sections and $($pages.Count) pages." | ||
| 118 | } catch { | ||
| 119 | $failure = $_ | ||
| 120 | [IO.File]::WriteAllText((Join-Path $output 'failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8) | ||
| 121 | throw | ||
| 122 | } finally { | ||
| 123 | try { | ||
| 124 | try { | ||
| 125 | if ($notebookId -and $CloneHost) { $app.SyncHierarchy($notebookId) } | ||
| 126 | if ($notebookId -and -not $KeepOpen -and (-not $UseCurrentCache -or $CloneHost)) { | ||
| 127 | $app.CloseNotebook($notebookId, $false) | ||
| 128 | } | ||
| 129 | } catch { | ||
| 130 | if ($null -eq $failure) { throw } | ||
| 131 | [IO.File]::WriteAllText((Join-Path $output 'cleanup-failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8) | ||
| 132 | } | ||
| 133 | } finally { | ||
| 134 | [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app) | ||
| 135 | } | ||
| 136 | $app = $null | ||
| 137 | [GC]::Collect() | ||
| 138 | [GC]::WaitForPendingFinalizers() | ||
| 139 | if (-not $UseCurrentCache -and -not $CloneHost) { | ||
| 140 | Get-Process ONENOTE -ErrorAction SilentlyContinue | Wait-Process -Timeout 10 | ||
| 141 | } | ||
| 142 | } | ||
corpus/math-edit/native-editor/source.json created+14
| ... | @@ -0,0 +1,14 @@ | ||
| 1 | [ | ||
| 2 | { | ||
| 3 | "path": "Open Notebook.onetoc2", | ||
| 4 | "bytes": 1604, | ||
| 5 | "sha256": "6731c4214437fe1cb6aa347b0004542b7f7e4260f47773bf9ec07c4e894de47c", | ||
| 6 | "mtime_ns": 1789206821144296891 | ||
| 7 | }, | ||
| 8 | { | ||
| 9 | "path": "links.one", | ||
| 10 | "bytes": 6128, | ||
| 11 | "sha256": "0cfb356ed374fe919f73fb73fdbe5d66e0baa484d4119f70074d9f869d6db78a", | ||
| 12 | "mtime_ns": 1789206821116200533 | ||
| 13 | } | ||
| 14 | ] | ||
corpus/math-edit/native-editor/teardown.json created+1
| ... | @@ -0,0 +1 @@ | ||
| 1 | {"absent": true} | ||
corpus/math-edit/written/candidate/Open Notebook.onetoc2 created
| Binary files /dev/null and b/corpus/math-edit/written/candidate/Open Notebook.onetoc2 differ | |||
corpus/math-edit/written/candidate/expected-mathml.json created+12
| ... | @@ -0,0 +1,12 @@ | ||
| 1 | [ | ||
| 2 | "<mml:msup><mml:mi>x</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mo>+</mml:mo><mml:msup><mml:mi>y</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mo>=</mml:mo><mml:msup><mml:mi>z</mml:mi><mml:mn>2</mml:mn></mml:msup>", | ||
| 3 | "<mml:mfrac><mml:mrow><mml:mi>a</mml:mi><mml:mo>+</mml:mo><mml:mi>b</mml:mi></mml:mrow><mml:mrow><mml:mi>c</mml:mi><mml:mo>+</mml:mo><mml:mi>d</mml:mi></mml:mrow></mml:mfrac>", | ||
| 4 | "<mml:mi>𝛼</mml:mi><mml:mo>+</mml:mo><mml:mi>𝛽</mml:mi>", | ||
| 5 | "<mml:msub><mml:mi>a</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo>+</mml:mo><mml:msub><mml:mi>b</mml:mi><mml:mn>2</mml:mn></mml:msub>", | ||
| 6 | "<mml:msubsup><mml:mi>x</mml:mi><mml:mi>i</mml:mi><mml:mn>2</mml:mn></mml:msubsup>", | ||
| 7 | "<mml:mfenced><mml:mrow><mml:mi>a</mml:mi><mml:mo>+</mml:mo><mml:mi>b</mml:mi></mml:mrow></mml:mfenced>", | ||
| 8 | "<mml:msubsup><mml:mo stretchy=\"false\">∫</mml:mo><mml:mn>0</mml:mn><mml:mn>1</mml:mn></mml:msubsup><mml:mrow><mml:mi>x</mml:mi><mml:mo> </mml:mo><mml:mi>d</mml:mi><mml:mi>x</mml:mi><mml:mo> </mml:mo></mml:mrow>", | ||
| 9 | "<mml:munderover><mml:mo stretchy=\"false\">∑</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>n</mml:mi></mml:munderover><mml:mrow><mml:mi>i</mml:mi><mml:mo> </mml:mo></mml:mrow>", | ||
| 10 | "<mml:msup><mml:mi>x</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mo>+</mml:mo><mml:mn>1</mml:mn>", | ||
| 11 | "<mml:mfrac><mml:mrow><mml:mi>a</mml:mi><mml:mo>+</mml:mo><mml:mi>b</mml:mi></mml:mrow><mml:mrow><mml:msup><mml:mi>c</mml:mi><mml:mn>2</mml:mn></mml:msup></mml:mrow></mml:mfrac>" | ||
| 12 | ] | ||
| \ No newline at end of file | |||
corpus/math-edit/written/candidate/math.one created
| Binary files /dev/null and b/corpus/math-edit/written/candidate/math.one differ | |||
corpus/math-edit/written/cold/commands.jsonl created+3
| ... | @@ -0,0 +1,3 @@ | ||
| 1 | {"command": "powershell -NoProfile -Command \"Expand-Archive -LiteralPath C:\\one-tests\\transfer.zip -DestinationPath C:\\one-tests\\runs\\capture\\notebook\"", "exit": 0, "stdout": "", "stderr": "", "error": null} | ||
| 2 | {"command": "powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\\one-tests\\read-current.ps1 -Root C:\\one-tests\\runs\\capture -CloneHost ONE-M6-E32B5D30 -ExpectedPages 1 -KeepOpen", "exit": 0, "stdout": "Read 1 sections and 1 pages.\r\n", "stderr": "", "error": null} | ||
| 3 | {"command": "powershell -NoProfile -Command \"Compress-Archive -Force -Path C:\\one-tests\\runs\\capture\\* -DestinationPath C:\\one-tests\\captured.zip\"", "exit": 0, "stdout": "", "stderr": "", "error": null} | ||
corpus/math-edit/written/cold/machine.json created+1
| ... | @@ -0,0 +1 @@ | ||
| 1 | {"name": "m6-e32b5d30", "hostname": "ONE-M6-E32B5D30"} | ||
corpus/math-edit/written/cold/notebook/Open Notebook.onetoc2 created
| Binary files /dev/null and b/corpus/math-edit/written/cold/notebook/Open Notebook.onetoc2 differ | |||
corpus/math-edit/written/cold/notebook/expected-mathml.json created+12
| ... | @@ -0,0 +1,12 @@ | ||
| 1 | [ | ||
| 2 | "<mml:msup><mml:mi>x</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mo>+</mml:mo><mml:msup><mml:mi>y</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mo>=</mml:mo><mml:msup><mml:mi>z</mml:mi><mml:mn>2</mml:mn></mml:msup>", | ||
| 3 | "<mml:mfrac><mml:mrow><mml:mi>a</mml:mi><mml:mo>+</mml:mo><mml:mi>b</mml:mi></mml:mrow><mml:mrow><mml:mi>c</mml:mi><mml:mo>+</mml:mo><mml:mi>d</mml:mi></mml:mrow></mml:mfrac>", | ||
| 4 | "<mml:mi>𝛼</mml:mi><mml:mo>+</mml:mo><mml:mi>𝛽</mml:mi>", | ||
| 5 | "<mml:msub><mml:mi>a</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo>+</mml:mo><mml:msub><mml:mi>b</mml:mi><mml:mn>2</mml:mn></mml:msub>", | ||
| 6 | "<mml:msubsup><mml:mi>x</mml:mi><mml:mi>i</mml:mi><mml:mn>2</mml:mn></mml:msubsup>", | ||
| 7 | "<mml:mfenced><mml:mrow><mml:mi>a</mml:mi><mml:mo>+</mml:mo><mml:mi>b</mml:mi></mml:mrow></mml:mfenced>", | ||
| 8 | "<mml:msubsup><mml:mo stretchy=\"false\">∫</mml:mo><mml:mn>0</mml:mn><mml:mn>1</mml:mn></mml:msubsup><mml:mrow><mml:mi>x</mml:mi><mml:mo> </mml:mo><mml:mi>d</mml:mi><mml:mi>x</mml:mi><mml:mo> </mml:mo></mml:mrow>", | ||
| 9 | "<mml:munderover><mml:mo stretchy=\"false\">∑</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>n</mml:mi></mml:munderover><mml:mrow><mml:mi>i</mml:mi><mml:mo> </mml:mo></mml:mrow>", | ||
| 10 | "<mml:msup><mml:mi>x</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mo>+</mml:mo><mml:mn>1</mml:mn>", | ||
| 11 | "<mml:mfrac><mml:mrow><mml:mi>a</mml:mi><mml:mo>+</mml:mo><mml:mi>b</mml:mi></mml:mrow><mml:mrow><mml:msup><mml:mi>c</mml:mi><mml:mn>2</mml:mn></mml:msup></mml:mrow></mml:mfrac>" | ||
| 12 | ] | ||
| \ No newline at end of file | |||
corpus/math-edit/written/cold/notebook/math.one created
| Binary files /dev/null and b/corpus/math-edit/written/cold/notebook/math.one differ | |||
corpus/math-edit/written/cold/read/environment.json created+7
| ... | @@ -0,0 +1,7 @@ | ||
| 1 | { | ||
| 2 | "powershell": "5.1.14409.1005", | ||
| 3 | "schema": "xs2010", | ||
| 4 | "hostname": "ONE-M6-E32B5D30", | ||
| 5 | "cold": true, | ||
| 6 | "onenote": "14.0.4763.1000" | ||
| 7 | } | ||
corpus/math-edit/written/cold/read/hierarchy.xml created+2
| ... | @@ -0,0 +1,2 @@ | ||
| 1 | <?xml version="1.0"?> | ||
| 2 | <one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{0F4C5356-866C-415D-B89A-B0B56339CFBA}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-13T02:54:30.000Z" color="#B49EDE"><one:Section name="math" ID="{4B312AA6-4ECD-097F-23E2-16B705529D73}{1}{B0}" path="C:\one-tests\runs\capture\notebook\math.one" lastModifiedTime="2026-09-13T02:54:30.000Z" color="#8AA8E4"><one:Page ID="{CB1A98E0-B776-09C5-2716-9ED19E0C777D}{1}{B0}" name="Equations below" dateTime="2026-09-13T02:52:00.000Z" lastModifiedTime="2026-09-13T02:54:30.000Z" pageLevel="1"/></one:Section></one:Notebook> | ||
corpus/math-edit/written/cold/read/page-000.navigation.json created+13
| ... | @@ -0,0 +1,13 @@ | ||
| 1 | { | ||
| 2 | "exit": 0, | ||
| 3 | "stdout": "", | ||
| 4 | "stderr": "", | ||
| 5 | "w": 800, | ||
| 6 | "h": 600, | ||
| 7 | "error": null, | ||
| 8 | "win": { | ||
| 9 | "title": "Equations below - Microsoft OneNote", | ||
| 10 | "class": "Framework::CFrame", | ||
| 11 | "dialog": false | ||
| 12 | } | ||
| 13 | } | ||
| \ No newline at end of file | |||
corpus/math-edit/written/cold/read/page-000.png created
| Binary files /dev/null and b/corpus/math-edit/written/cold/read/page-000.png differ | |||
corpus/math-edit/written/cold/read/page-000.xml created+3
| ... | @@ -0,0 +1,3 @@ | ||
| 1 | <?xml version="1.0"?> | ||
| 2 | <one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{CB1A98E0-B776-09C5-2716-9ED19E0C777D}{1}{B0}" name="Equations below" dateTime="2026-09-13T02:52:00.000Z" lastModifiedTime="2026-09-13T02:54:30.000Z" pageLevel="1" style="font-family:Calibri;font-size:11.0pt" lang="en-US"><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="Author" lastModifiedBy="Author" lastModifiedTime="2026-09-13T02:52:00.000Z" objectID="{4B312AA6-4ECD-097F-23E2-16B705529D73}{23}{B0}"><one:Position x="36.0" y="36.0" z="0"/><one:Size width="140.9678649902344" height="208.9916534423828"/><one:Indents><one:Indent level="0" indent="0.0"/></one:Indents><one:OEChildren><one:OE creationTime="2026-09-13T02:52:00.000Z" lastModifiedTime="2026-09-13T02:52:00.000Z" objectID="{4B312AA6-4ECD-097F-23E2-16B705529D73}{24}{B0}" alignment="left"><one:T><![CDATA[Equations below]]></one:T></one:OE><one:OE author="Math author" lastModifiedBy="Math author" creationTime="2026-09-13T02:52:00.000Z" lastModifiedTime="2026-09-13T02:52:00.000Z" objectID="{D2D96111-4DB1-072C-2B68-B5B25890CEC5}{1}{B0}" alignment="left" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:msup><mml:mi>x</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mo>+</mml:mo><mml:msup><mml:mi>y</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mo>=</mml:mo><mml:msup><mml:mi>z</mml:mi><mml:mn>2</mml:mn></mml:msup></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Math author" lastModifiedBy="Math author" creationTime="2026-09-13T02:52:00.000Z" lastModifiedTime="2026-09-13T02:52:00.000Z" objectID="{271A41B3-0C22-0187-01C9-DF7E1994442D}{1}{B0}" alignment="left" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:mfrac><mml:mrow><mml:mi>a</mml:mi><mml:mo>+</mml:mo><mml:mi>b</mml:mi></mml:mrow><mml:mrow><mml:mi>c</mml:mi><mml:mo>+</mml:mo><mml:mi>d</mml:mi></mml:mrow></mml:mfrac></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Math author" lastModifiedBy="Math author" creationTime="2026-09-13T02:52:00.000Z" lastModifiedTime="2026-09-13T02:52:00.000Z" objectID="{65504139-6636-0A37-095E-73D450E3B4B5}{1}{B0}" alignment="left" style="font-size:11.0pt"><one:T><![CDATA[<span | ||
| 3 | style='font-family:Calibri' lang=en-US>Inline </span><!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>𝛼</mml:mi><mml:mo>+</mml:mo><mml:mi>𝛽</mml:mi></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Math author" lastModifiedBy="Math author" creationTime="2026-09-13T02:52:00.000Z" lastModifiedTime="2026-09-13T02:52:00.000Z" objectID="{7D882CCC-0B13-061E-0D61-EDFFD126FBBF}{1}{B0}" alignment="left" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:msub><mml:mi>a</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo>+</mml:mo><mml:msub><mml:mi>b</mml:mi><mml:mn>2</mml:mn></mml:msub></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Math author" lastModifiedBy="Math author" creationTime="2026-09-13T02:52:00.000Z" lastModifiedTime="2026-09-13T02:52:00.000Z" objectID="{25C98D90-28D4-0C33-341D-D74D9110541C}{1}{B0}" alignment="left" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:msubsup><mml:mi>x</mml:mi><mml:mi>i</mml:mi><mml:mn>2</mml:mn></mml:msubsup></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Math author" lastModifiedBy="Math author" creationTime="2026-09-13T02:52:00.000Z" lastModifiedTime="2026-09-13T02:52:00.000Z" objectID="{E7EBE421-A1AA-03F1-10E9-39885BEAB42C}{1}{B0}" alignment="left" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:mfenced><mml:mrow><mml:mi>a</mml:mi><mml:mo>+</mml:mo><mml:mi>b</mml:mi></mml:mrow></mml:mfenced></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Math author" lastModifiedBy="Math author" creationTime="2026-09-13T02:52:00.000Z" lastModifiedTime="2026-09-13T02:52:00.000Z" objectID="{EE1FB953-F398-0D15-35BA-F180971C0119}{1}{B0}" alignment="left" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:msubsup><mml:mo stretchy="false">∫</mml:mo><mml:mn>0</mml:mn><mml:mn>1</mml:mn></mml:msubsup><mml:mrow><mml:mi>x</mml:mi><mml:mo> </mml:mo><mml:mi>d</mml:mi><mml:mi>x</mml:mi><mml:mo> </mml:mo></mml:mrow></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Math author" lastModifiedBy="Math author" creationTime="2026-09-13T02:52:00.000Z" lastModifiedTime="2026-09-13T02:52:00.000Z" objectID="{C7DB884C-1F52-04E1-3F88-5598A9313529}{1}{B0}" alignment="left" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:munderover><mml:mo stretchy="false">∑</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>n</mml:mi></mml:munderover><mml:mrow><mml:mi>i</mml:mi><mml:mo> </mml:mo></mml:mrow></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Math author" lastModifiedBy="Math author" creationTime="2026-09-13T02:52:00.000Z" lastModifiedTime="2026-09-13T02:52:00.000Z" objectID="{C4552891-A700-0951-012A-8A8A692FCEA5}{1}{B0}" alignment="left" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:msup><mml:mi>x</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:math><![endif]-->]]></one:T></one:OE><one:OE author="Math author" lastModifiedBy="Math author" creationTime="2026-09-13T02:52:00.000Z" lastModifiedTime="2026-09-13T02:52:00.000Z" objectID="{E83223D3-665D-0CC9-064F-7F5A0CCED447}{1}{B0}" alignment="left" style="font-family:'Cambria Math';font-size:11.0pt"><one:T><![CDATA[<!--[if mathML]><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="block"><mml:mfrac><mml:mrow><mml:mi>a</mml:mi><mml:mo>+</mml:mo><mml:mi>b</mml:mi></mml:mrow><mml:mrow><mml:msup><mml:mi>c</mml:mi><mml:mn>2</mml:mn></mml:msup></mml:mrow></mml:mfrac></mml:math><![endif]-->]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page> | ||
corpus/math-edit/written/cold/read/payloads.json created+3
| ... | @@ -0,0 +1,3 @@ | ||
| 1 | [ | ||
| 2 | |||
| 3 | ] | ||
| \ No newline at end of file | |||
corpus/math-edit/written/cold/run.json created+18
| ... | @@ -0,0 +1,18 @@ | ||
| 1 | { | ||
| 2 | "notebook": "/private/tmp/m4/written", | ||
| 3 | "expected_pages": 1, | ||
| 4 | "author": null, | ||
| 5 | "author_timeout_seconds": 600, | ||
| 6 | "inspect": false, | ||
| 7 | "collect_notebook": true, | ||
| 8 | "base": { | ||
| 9 | "file": "win7-office-base.qcow2", | ||
| 10 | "format": "qcow2", | ||
| 11 | "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346", | ||
| 12 | "virtual_size": 68719476736 | ||
| 13 | }, | ||
| 14 | "scripts": { | ||
| 15 | "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331", | ||
| 16 | "read.ps1": "04013bfcccee40a17e2a225f9b9e96f40a3a8daad9e350609654f8eda3ccbb41" | ||
| 17 | } | ||
| 18 | } | ||
corpus/math-edit/written/cold/scripts/cold.ps1 created+27
| ... | @@ -0,0 +1,27 @@ | ||
| 1 | param([Parameter(Mandatory=$true)][string]$Root, [string]$CloneHost = '') | ||
| 2 | Set-StrictMode -Version Latest | ||
| 3 | $ErrorActionPreference = 'Stop' | ||
| 4 | $root = [IO.Path]::GetFullPath($Root).TrimEnd('\') | ||
| 5 | if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') { | ||
| 6 | throw 'Choose a run directly below C:\one-tests\runs.' | ||
| 7 | } | ||
| 8 | if (Get-Process ONENOTE -ErrorAction SilentlyContinue) { throw 'Close OneNote before resetting its test cache.' } | ||
| 9 | $key = 'HKCU:\Software\Microsoft\Office\14.0\OneNote' | ||
| 10 | if ($CloneHost) { | ||
| 11 | if ($CloneHost -notmatch '^ONE-[A-Z0-9-]+$' -or [Environment]::MachineName -ne $CloneHost) { | ||
| 12 | throw 'The disposable clone hostname does not match this machine.' | ||
| 13 | } | ||
| 14 | New-Item "$key\Options\Paths" -Force | Out-Null | ||
| 15 | New-ItemProperty "$key\Options\Paths" -Name UnfiledNotesSection -PropertyType ExpandString -Value 'C:\one-tests\Loose.one' -Force | Out-Null | ||
| 16 | } elseif ((Get-ItemProperty "$key\Options\Paths").UnfiledNotesSection -ne 'C:\one-tests\Loose.one' -or | ||
| 17 | -not (Test-Path 'C:\one-tests\profile-original-cache')) { | ||
| 18 | throw 'Park the personal OneNote profile before resetting the test cache.' | ||
| 19 | } | ||
| 20 | $cache = Join-Path $env:LOCALAPPDATA 'Microsoft\OneNote\14.0' | ||
| 21 | $parked = Join-Path 'C:\one-tests\caches' ([IO.Path]::GetFileName($root)) | ||
| 22 | if (Test-Path $parked) { throw 'Choose a new run; its parked cache already exists.' } | ||
| 23 | New-Item -ItemType Directory -Path 'C:\one-tests\caches' -Force | Out-Null | ||
| 24 | if (Test-Path $cache) { Move-Item -LiteralPath $cache -Destination $parked } | ||
| 25 | if (Test-Path "$key\OpenNotebooks") { Remove-Item "$key\OpenNotebooks" -Recurse } | ||
| 26 | New-Item "$key\OpenNotebooks" | Out-Null | ||
| 27 | New-ItemProperty "$key\OpenNotebooks" -Name '1' -PropertyType String -Value "$root\notebook" | Out-Null | ||
corpus/math-edit/written/cold/scripts/read.ps1 created+142
| ... | @@ -0,0 +1,142 @@ | ||
| 1 | param( | ||
| 2 | [Parameter(Mandatory=$true)][string]$Root, | ||
| 3 | [int]$ExpectedPages = -1, | ||
| 4 | [switch]$UseCurrentCache, | ||
| 5 | [switch]$Pdf, | ||
| 6 | [switch]$KeepOpen, | ||
| 7 | [string]$CloneHost = '' | ||
| 8 | ) | ||
| 9 | Set-StrictMode -Version Latest | ||
| 10 | $ErrorActionPreference = 'Stop' | ||
| 11 | $root = [IO.Path]::GetFullPath($Root).TrimEnd('\') | ||
| 12 | if ([IO.Path]::GetDirectoryName($root) -ne 'C:\one-tests\runs') { | ||
| 13 | throw 'Choose a run directly below C:\one-tests\runs.' | ||
| 14 | } | ||
| 15 | $notebook = Join-Path $root 'notebook' | ||
| 16 | $output = Join-Path $root 'read' | ||
| 17 | if (Test-Path $output) { throw 'Choose a new read destination.' } | ||
| 18 | if ($UseCurrentCache) { | ||
| 19 | if ((Get-ItemProperty 'HKCU:\Software\Microsoft\Office\14.0\OneNote\Options\Paths').UnfiledNotesSection -ne 'C:\one-tests\Loose.one') { | ||
| 20 | throw 'Park the personal OneNote profile before reading test notebooks.' | ||
| 21 | } | ||
| 22 | } else { | ||
| 23 | & "$PSScriptRoot\cold-current.ps1" -Root $root -CloneHost $CloneHost | ||
| 24 | } | ||
| 25 | New-Item -ItemType Directory -Path $output | Out-Null | ||
| 26 | $app = New-Object -ComObject OneNote.Application | ||
| 27 | $notebookId = '' | ||
| 28 | $failure = $null | ||
| 29 | try { | ||
| 30 | $app.OpenHierarchy($notebook, '', [ref]$notebookId, 0) | ||
| 31 | $process = Get-Process ONENOTE | ||
| 32 | @{ hostname = [Environment]::MachineName; onenote = $process.MainModule.FileVersionInfo.FileVersion; | ||
| 33 | powershell = $PSVersionTable.PSVersion.ToString(); schema = 'xs2010'; cold = (-not $UseCurrentCache.IsPresent) } | | ||
| 34 | ConvertTo-Json | Set-Content (Join-Path $output 'environment.json') -Encoding UTF8 | ||
| 35 | $sections = @() | ||
| 36 | foreach ($file in @(Get-ChildItem $notebook -Recurse | Where-Object { $_.Extension -eq '.one' })) { | ||
| 37 | $id = '' | ||
| 38 | $app.OpenHierarchy($file.FullName, '', [ref]$id, 0) | ||
| 39 | $sections += $id | ||
| 40 | } | ||
| 41 | $deadline = [DateTime]::UtcNow.AddSeconds(300) | ||
| 42 | $previous = '' | ||
| 43 | $lastChange = '' | ||
| 44 | $stableSince = [DateTime]::UtcNow | ||
| 45 | $settled = $false | ||
| 46 | do { | ||
| 47 | $pages = @{} | ||
| 48 | foreach ($section in $sections) { | ||
| 49 | $hierarchy = '' | ||
| 50 | $app.GetHierarchy($section, 4, [ref]$hierarchy, 1) | ||
| 51 | [xml]$xml = $hierarchy | ||
| 52 | foreach ($node in $xml.SelectNodes('//*[@path]')) { | ||
| 53 | if (-not $node.GetAttribute('path').StartsWith("$notebook\", [StringComparison]::OrdinalIgnoreCase)) { | ||
| 54 | throw 'OneNote opened a section outside the copied notebook.' | ||
| 55 | } | ||
| 56 | } | ||
| 57 | foreach ($node in $xml.SelectNodes('//*[local-name()="Page"]')) { | ||
| 58 | $id = $node.GetAttribute('ID') | ||
| 59 | $content = '' | ||
| 60 | $app.GetPageContent($id, [ref]$content, 1, 1) | ||
| 61 | $pages[$id] = $content | ||
| 62 | } | ||
| 63 | } | ||
| 64 | $signature = [String]::Join('|', @($pages.Keys | Sort-Object | ForEach-Object { $_ + $pages[$_] })) | ||
| 65 | if ($signature -ne $previous) { | ||
| 66 | $lastChange = $previous | ||
| 67 | $previous = $signature | ||
| 68 | $stableSince = [DateTime]::UtcNow | ||
| 69 | } | ||
| 70 | if ((($ExpectedPages -ge 0 -and $pages.Count -eq $ExpectedPages) -or | ||
| 71 | ($ExpectedPages -lt 0 -and $pages.Count -gt 0)) -and | ||
| 72 | ([DateTime]::UtcNow - $stableSince).TotalSeconds -ge 2) { $settled = $true; break } | ||
| 73 | Start-Sleep -Milliseconds 250 | ||
| 74 | } while ([DateTime]::UtcNow -lt $deadline) | ||
| 75 | if (-not $settled -or ($ExpectedPages -ge 0 -and $pages.Count -ne $ExpectedPages) -or ($ExpectedPages -lt 0 -and $pages.Count -eq 0)) { | ||
| 76 | [IO.File]::WriteAllText((Join-Path $output 'previous-signature.txt'), $lastChange, [Text.Encoding]::UTF8) | ||
| 77 | $index = 0 | ||
| 78 | foreach ($id in @($pages.Keys | Sort-Object)) { | ||
| 79 | [IO.File]::WriteAllText((Join-Path $output ('unsettled-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8) | ||
| 80 | $index++ | ||
| 81 | } | ||
| 82 | $hierarchy = '' | ||
| 83 | $app.GetHierarchy($notebookId, 4, [ref]$hierarchy, 1) | ||
| 84 | [IO.File]::WriteAllText((Join-Path $output 'unsettled-hierarchy.xml'), $hierarchy, [Text.Encoding]::UTF8) | ||
| 85 | throw "Expected $ExpectedPages stable pages; OneNote returned $($pages.Count), settled=$settled." | ||
| 86 | } | ||
| 87 | $index = 0 | ||
| 88 | $payloads = @() | ||
| 89 | foreach ($id in @($pages.Keys | Sort-Object)) { | ||
| 90 | [IO.File]::WriteAllText((Join-Path $output ('page-{0:d3}.xml' -f $index)), $pages[$id], [Text.Encoding]::UTF8) | ||
| 91 | if ($Pdf) { | ||
| 92 | $pdfPath = Join-Path $output ('page-{0:d3}.pdf' -f $index) | ||
| 93 | $app.NavigateTo($id, '', $false) | ||
| 94 | $app.Publish($id, $pdfPath, 3, '') | ||
| 95 | if (-not (Test-Path $pdfPath) -or (Get-Item $pdfPath).Length -eq 0) { throw 'OneNote did not publish the page PDF.' } | ||
| 96 | } | ||
| 97 | [xml]$page = $pages[$id] | ||
| 98 | foreach ($file in $page.SelectNodes('//*[local-name()="InsertedFile" or local-name()="MediaFile"]')) { | ||
| 99 | $bytes = [IO.File]::ReadAllBytes($file.GetAttribute('pathCache')) | ||
| 100 | $hash = [BitConverter]::ToString([Security.Cryptography.SHA256]::Create().ComputeHash($bytes)).Replace('-', '').ToLowerInvariant() | ||
| 101 | [IO.File]::WriteAllBytes((Join-Path $output ($hash + '.attachment')), $bytes) | ||
| 102 | $payloads += @{ page = $id; object = $file.ParentNode.GetAttribute('objectID'); | ||
| 103 | kind = $file.LocalName; name = $file.GetAttribute('preferredName'); | ||
| 104 | sha256 = $hash; bytes = $bytes.Length } | ||
| 105 | } | ||
| 106 | $index++ | ||
| 107 | } | ||
| 108 | [IO.File]::WriteAllText((Join-Path $output 'payloads.json'), (ConvertTo-Json -InputObject $payloads -Depth 4), [Text.Encoding]::UTF8) | ||
| 109 | $all = '' | ||
| 110 | $app.GetHierarchy($notebookId, 4, [ref]$all, 1) | ||
| 111 | [xml]$finalTree = $all | ||
| 112 | $finalIds = @($finalTree.SelectNodes('//*[local-name()="Page"]') | ForEach-Object { $_.GetAttribute('ID') } | Sort-Object -Unique) | ||
| 113 | if ($finalIds.Count -ne $pages.Count -or @($finalIds | Where-Object { -not $pages.ContainsKey($_) }).Count -ne 0) { | ||
| 114 | throw 'The notebook hierarchy changed while collecting page evidence; repeat the cold read.' | ||
| 115 | } | ||
| 116 | [IO.File]::WriteAllText((Join-Path $output 'hierarchy.xml'), $all, [Text.Encoding]::UTF8) | ||
| 117 | Write-Output "Read $($sections.Count) sections and $($pages.Count) pages." | ||
| 118 | } catch { | ||
| 119 | $failure = $_ | ||
| 120 | [IO.File]::WriteAllText((Join-Path $output 'failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8) | ||
| 121 | throw | ||
| 122 | } finally { | ||
| 123 | try { | ||
| 124 | try { | ||
| 125 | if ($notebookId -and $CloneHost) { $app.SyncHierarchy($notebookId) } | ||
| 126 | if ($notebookId -and -not $KeepOpen -and (-not $UseCurrentCache -or $CloneHost)) { | ||
| 127 | $app.CloseNotebook($notebookId, $false) | ||
| 128 | } | ||
| 129 | } catch { | ||
| 130 | if ($null -eq $failure) { throw } | ||
| 131 | [IO.File]::WriteAllText((Join-Path $output 'cleanup-failure.txt'), ($_ | Out-String), [Text.Encoding]::UTF8) | ||
| 132 | } | ||
| 133 | } finally { | ||
| 134 | [void][Runtime.InteropServices.Marshal]::FinalReleaseComObject($app) | ||
| 135 | } | ||
| 136 | $app = $null | ||
| 137 | [GC]::Collect() | ||
| 138 | [GC]::WaitForPendingFinalizers() | ||
| 139 | if (-not $UseCurrentCache -and -not $CloneHost) { | ||
| 140 | Get-Process ONENOTE -ErrorAction SilentlyContinue | Wait-Process -Timeout 10 | ||
| 141 | } | ||
| 142 | } | ||
corpus/math-edit/written/cold/source.json created+20
| ... | @@ -0,0 +1,20 @@ | ||
| 1 | [ | ||
| 2 | { | ||
| 3 | "path": "Open Notebook.onetoc2", | ||
| 4 | "bytes": 1596, | ||
| 5 | "sha256": "bc31e9f663f68420b4f7684641e2a75abefddd97256074bae432dff0fff5244c", | ||
| 6 | "mtime_ns": 1789267920902398086 | ||
| 7 | }, | ||
| 8 | { | ||
| 9 | "path": "expected-mathml.json", | ||
| 10 | "bytes": 1536, | ||
| 11 | "sha256": "7c0f17965424fbfee0f952b0293a2daf5fcd7040d50b7482e90785cc52192c84", | ||
| 12 | "mtime_ns": 1789267920927591231 | ||
| 13 | }, | ||
| 14 | { | ||
| 15 | "path": "math.one", | ||
| 16 | "bytes": 15424, | ||
| 17 | "sha256": "70b09e61eaca321b6aa555316b59769f282843c7710c4ca311f1b5d2d7ec42bd", | ||
| 18 | "mtime_ns": 1789267920871282689 | ||
| 19 | } | ||
| 20 | ] | ||
corpus/math-edit/written/cold/teardown.json created+1
| ... | @@ -0,0 +1 @@ | ||
| 1 | {"absent": true} | ||
crates/notebook/README.md+1-1
| ... | @@ -14,7 +14,7 @@ diffs the supplied model against the page stored in the supplied local snapshot, | ... | @@ -14,7 +14,7 @@ diffs the supplied model against the page stored in the supplied local snapshot, |
| 14 | writes the difference into the working image and queues one | 14 | writes the difference into the working image and queues one |
| 15 | `Operation::Page(PageIntent { before, after, author })`; `before` is the page the | 15 | `Operation::Page(PageIntent { before, after, author })`; `before` is the page the |
| 16 | edit started from and is the precondition reconciliation checks. Text, styles, | 16 | edit started from and is the precondition reconciliation checks. Text, styles, |
| 17 | paragraph formatting, hyperlinks (external and internal), bullets, numbering, note tags, table rows, columns, cell shading, nested tables, pictures (insertion in paragraphs or on the page, position, size, description), attachments, ink strokes, paragraph structure, | 17 | paragraph formatting, hyperlinks (external and internal), bullets, numbering, note tags, table rows, columns, cell shading, nested tables, pictures (insertion in paragraphs or on the page, position, size, description), attachments, ink strokes, equations, paragraph structure, |
| 18 | outline layout, insertions and deletions are all differences between `before` and | 18 | outline layout, insertions and deletions are all differences between `before` and |
| 19 | `after`; the library never sees editor operations. | 19 | `after`; the library never sees editor operations. |
| 20 | 20 |
crates/onestore/README.md+1-1
| ... | @@ -57,7 +57,7 @@ harness also accepts `--client-profile release`. | ... | @@ -57,7 +57,7 @@ harness also accepts `--client-profile release`. |
| 57 | | `ParagraphJoin`, `PreparedEdit::join` | Join adjacent ordinary text while preserving inherited character styles and native text-identity rules | | 57 | | `ParagraphJoin`, `PreparedEdit::join` | Join adjacent ordinary text while preserving inherited character styles and native text-identity rules | |
| 58 | | `TextAttribute`, `PreparedEdit::format` | Change character formatting over a UTF-16 range while sharing immutable styles; preserve unselected runs | | 58 | | `TextAttribute`, `PreparedEdit::format` | Change character formatting over a UTF-16 range while sharing immutable styles; preserve unselected runs | |
| 59 | | `OutlineEdit`, `PreparedEdit::outline` | Change ordinary outline position/width or a paragraph's saved expansion default, preserving identities and content | | 59 | | `OutlineEdit`, `PreparedEdit::outline` | Change ordinary outline position/width or a paragraph's saved expansion default, preserving identities and content | |
| 60 | | `PreparedEdit::page` | Publish an edited `page::Page` as one revision per changed space: text, character and paragraph formatting, hyperlinks (external and, via `page::link`, to pages, paragraphs and sections), bullets and numbering, note tags, table rows, columns, cell shading and indents, nested tables, inserted pictures (in paragraphs or on the page) and attachments, picture position, size and description, ink drawings and handwriting (strokes added and erased), paragraph insertion/split/join/move/deletion, outline insertion/deletion/position/width and saved collapse state, lowered onto the typed writers with the model's identities | | 60 | | `PreparedEdit::page` | Publish an edited `page::Page` as one revision per changed space: text, character and paragraph formatting, hyperlinks (external and, via `page::link`, to pages, paragraphs and sections), bullets and numbering, note tags, table rows, columns, cell shading and indents, nested tables, inserted pictures (in paragraphs or on the page) and attachments, picture position, size and description, ink drawings and handwriting (strokes added and erased), equations (built from `page::Math` trees), paragraph insertion/split/join/move/deletion, outline insertion/deletion/position/width and saved collapse state, lowered onto the typed writers with the model's identities | |
| 61 | | `TreeEdit`, `PreparedEdit::tree` | Move or delete a subtree on one page, normalize surviving containers, and replace an emptied table cell's paragraph atomically | | 61 | | `TreeEdit`, `PreparedEdit::tree` | Move or delete a subtree on one page, normalize surviving containers, and replace an emptied table cell's paragraph atomically | |
| 62 | | `PreparedEdit::commit`, `PreparedEdit::commit_file` | Publish the exact prepared image under caller-held exclusion or the conservative filesystem adapter | | 62 | | `PreparedEdit::commit`, `PreparedEdit::commit_file` | Publish the exact prepared image under caller-held exclusion or the conservative filesystem adapter | |
| 63 | | `read_file` | Read a snapshot under whole-file exclusion | | 63 | | `read_file` | Read a snapshot under whole-file exclusion | |
crates/onestore/src/page/math.rs+119-13
| ... | @@ -1,8 +1,9 @@ | ... | @@ -1,8 +1,9 @@ |
| 1 | //! Equations as OneNote stores them: a linear text where U+FDD0 opens an inline object, | 1 | //! Equations as OneNote stores them: a linear text where U+FDD0 opens an inline object, |
| 2 | //! U+FDEE separates its arguments and U+FDEF closes it, with the object's kind on the run | 2 | //! U+FDEE separates its arguments and U+FDEF closes it, with the object's kind on the run |
| 3 | //! data of the opening character. `Math::parse` builds the tree and `mathml` renders it the | 3 | //! data of the opening character. `Math::parse` builds the tree and `mathml` renders it the |
| 4 | //! way OneNote's own export does for the kinds a native fixture has verified (superscript | 4 | //! way OneNote's own export does for the kinds native fixtures have verified (superscript, |
| 5 | //! and fraction); other kinds follow the text object model's meaning without a native check. | 5 | //! subscript, both, fraction, brackets, n-ary operators with limits); radicals, under/over |
| 6 | //! limits and the rest follow the text object model's meaning without a native check. | ||
| 6 | 7 | ||
| 7 | use super::text::Paragraph; | 8 | use super::text::Paragraph; |
| 8 | use crate::document::MathObject; | 9 | use crate::document::MathObject; |
| ... | @@ -96,7 +97,7 @@ impl Math { | ... | @@ -96,7 +97,7 @@ impl Math { |
| 96 | arguments: frame.arguments, | 97 | arguments: frame.arguments, |
| 97 | }); | 98 | }); |
| 98 | } | 99 | } |
| 99 | c if c.is_whitespace() => {} | 100 | c if c.is_whitespace() => sequences.last_mut().unwrap().push(Math::Operator(' ')), |
| 100 | c if c.is_alphabetic() => sequences | 101 | c if c.is_alphabetic() => sequences |
| 101 | .last_mut() | 102 | .last_mut() |
| 102 | .unwrap() | 103 | .unwrap() |
| ... | @@ -133,11 +134,12 @@ impl Math { | ... | @@ -133,11 +134,12 @@ impl Math { |
| 133 | symbols, | 134 | symbols, |
| 134 | arguments, | 135 | arguments, |
| 135 | } => { | 136 | } => { |
| 137 | // A lone identifier, number or operator stands bare; anything else is a row. | ||
| 136 | let argument = |out: &mut String, index: usize| match arguments | 138 | let argument = |out: &mut String, index: usize| match arguments |
| 137 | .get(index) | 139 | .get(index) |
| 138 | .map(Vec::as_slice) | 140 | .map(Vec::as_slice) |
| 139 | { | 141 | { |
| 140 | Some([single]) => single.write(out), | 142 | Some([single]) if !matches!(single, Math::Object { .. }) => single.write(out), |
| 141 | Some(many) => { | 143 | Some(many) => { |
| 142 | out.push_str("<mml:mrow>"); | 144 | out.push_str("<mml:mrow>"); |
| 143 | for node in many { | 145 | for node in many { |
| ... | @@ -163,16 +165,26 @@ impl Math { | ... | @@ -163,16 +165,26 @@ impl Math { |
| 163 | (25, _) => wrapped(out, "msqrt", &[arguments.len() - 1]), | 165 | (25, _) => wrapped(out, "msqrt", &[arguments.len() - 1]), |
| 164 | (19, 2) => wrapped(out, "munder", &[0, 1]), | 166 | (19, 2) => wrapped(out, "munder", &[0, 1]), |
| 165 | (33, 2) => wrapped(out, "mover", &[0, 1]), | 167 | (33, 2) => wrapped(out, "mover", &[0, 1]), |
| 166 | (13, _) => { | 168 | // OneNote exports parentheses as default fences, whatever the pair. |
| 169 | (13, _) => wrapped(out, "mfenced", &[0]), | ||
| 170 | // Lower limit, upper limit, body; integrals take their limits as scripts, | ||
| 171 | // other operators above and below. | ||
| 172 | (21, 3) => { | ||
| 173 | let operator = symbols.first().copied().unwrap_or('∑'); | ||
| 174 | let element = if ('\u{222b}'..='\u{2233}').contains(&operator) { | ||
| 175 | "msubsup" | ||
| 176 | } else { | ||
| 177 | "munderover" | ||
| 178 | }; | ||
| 179 | out.push_str(&format!( | ||
| 180 | "<mml:{element}><mml:mo stretchy=\"false\">{operator}</mml:mo>" | ||
| 181 | )); | ||
| 182 | argument(out, 0); | ||
| 183 | argument(out, 1); | ||
| 184 | out.push_str(&format!("</mml:{element}>")); | ||
| 167 | out.push_str("<mml:mrow>"); | 185 | out.push_str("<mml:mrow>"); |
| 168 | if let Some(open) = symbols.first() { | 186 | for node in &arguments[2] { |
| 169 | tag(out, "mo", &open.to_string()); | 187 | node.write(out); |
| 170 | } | ||
| 171 | for index in 0..arguments.len() { | ||
| 172 | argument(out, index); | ||
| 173 | } | ||
| 174 | if let Some(close) = symbols.get(1) { | ||
| 175 | tag(out, "mo", &close.to_string()); | ||
| 176 | } | 188 | } |
| 177 | out.push_str("</mml:mrow>"); | 189 | out.push_str("</mml:mrow>"); |
| 178 | } | 190 | } |
| ... | @@ -199,6 +211,7 @@ fn tag(out: &mut String, element: &str, content: &str) { | ... | @@ -199,6 +211,7 @@ fn tag(out: &mut String, element: &str, content: &str) { |
| 199 | '<' => out.push_str("<"), | 211 | '<' => out.push_str("<"), |
| 200 | '&' => out.push_str("&"), | 212 | '&' => out.push_str("&"), |
| 201 | '>' => out.push_str(">"), | 213 | '>' => out.push_str(">"), |
| 214 | ' ' => out.push_str(" "), | ||
| 202 | c => out.push(c), | 215 | c => out.push(c), |
| 203 | } | 216 | } |
| 204 | } | 217 | } |
| ... | @@ -215,3 +228,96 @@ fn plain(c: char) -> char { | ... | @@ -215,3 +228,96 @@ fn plain(c: char) -> char { |
| 215 | _ => c, | 228 | _ => c, |
| 216 | } | 229 | } |
| 217 | } | 230 | } |
| 231 | |||
| 232 | impl Math { | ||
| 233 | /// The paragraph OneNote stores for `nodes`: linear text with object controls, every run | ||
| 234 | /// formatted the way the equation editor formats it (Cambria Math, italic, the math flags | ||
| 235 | /// and the math language over `base`), and the run data each object carries. | ||
| 236 | pub fn paragraph(nodes: &[Math], base: &crate::document::Format) -> Paragraph { | ||
| 237 | let mut style = base.clone(); | ||
| 238 | style.italic = Some(true); | ||
| 239 | style.hidden = Some(false); | ||
| 240 | style.hyperlink = Some(false); | ||
| 241 | style.math = Some(true); | ||
| 242 | style.embedded_object = Some(true); | ||
| 243 | style.font = Some("Cambria Math".into()); | ||
| 244 | style.language = Some(0x1007f); | ||
| 245 | let mut runs: Vec<(String, Option<MathObject>)> = Vec::new(); | ||
| 246 | write_sequence(nodes, &mut runs); | ||
| 247 | Paragraph::from_runs(runs.into_iter().map(|(text, object)| { | ||
| 248 | let mut format = style.clone(); | ||
| 249 | format.math_object = Some(object.unwrap_or(MathObject { | ||
| 250 | kind: PLAIN_RUN, | ||
| 251 | arguments: None, | ||
| 252 | symbols: Vec::new(), | ||
| 253 | })); | ||
| 254 | (text, format) | ||
| 255 | })) | ||
| 256 | } | ||
| 257 | } | ||
| 258 | |||
| 259 | /// The object kind OneNote gives runs between objects. | ||
| 260 | const PLAIN_RUN: u32 = 0x9000_0000; | ||
| 261 | |||
| 262 | fn write_sequence(nodes: &[Math], runs: &mut Vec<(String, Option<MathObject>)>) { | ||
| 263 | let mut leaf = String::new(); | ||
| 264 | for node in nodes { | ||
| 265 | match node { | ||
| 266 | Math::Identifier(c) => leaf.push(italic(*c)), | ||
| 267 | Math::Number(n) => leaf.push_str(n), | ||
| 268 | Math::Operator(c) => leaf.push(*c), | ||
| 269 | Math::Object { | ||
| 270 | kind, | ||
| 271 | symbols, | ||
| 272 | arguments, | ||
| 273 | } => { | ||
| 274 | if !leaf.is_empty() { | ||
| 275 | runs.push((std::mem::take(&mut leaf), None)); | ||
| 276 | } | ||
| 277 | runs.push(( | ||
| 278 | OBJECT_START.to_string(), | ||
| 279 | Some(MathObject { | ||
| 280 | kind: *kind, | ||
| 281 | arguments: Some(arguments.len() as u32), | ||
| 282 | symbols: symbols.clone(), | ||
| 283 | }), | ||
| 284 | )); | ||
| 285 | for (index, argument) in arguments.iter().enumerate() { | ||
| 286 | let before = runs.len(); | ||
| 287 | write_sequence(argument, runs); | ||
| 288 | let control = if index + 1 == arguments.len() { | ||
| 289 | OBJECT_END | ||
| 290 | } else { | ||
| 291 | ARGUMENT_SEPARATOR | ||
| 292 | }; | ||
| 293 | let object = MathObject { | ||
| 294 | kind: *kind, | ||
| 295 | arguments: (index > 0).then_some(index as u32), | ||
| 296 | symbols: Vec::new(), | ||
| 297 | }; | ||
| 298 | let extended = runs.len() > before; | ||
| 299 | match runs.last_mut() { | ||
| 300 | Some((text, slot)) if extended && slot.is_none() => { | ||
| 301 | text.push(control); | ||
| 302 | *slot = Some(object); | ||
| 303 | } | ||
| 304 | _ => runs.push((control.to_string(), Some(object))), | ||
| 305 | } | ||
| 306 | } | ||
| 307 | } | ||
| 308 | } | ||
| 309 | } | ||
| 310 | if !leaf.is_empty() { | ||
| 311 | runs.push((leaf, None)); | ||
| 312 | } | ||
| 313 | } | ||
| 314 | |||
| 315 | /// The equation editor stores Latin letters as mathematical italics. | ||
| 316 | fn italic(c: char) -> char { | ||
| 317 | match c { | ||
| 318 | 'h' => '\u{210e}', | ||
| 319 | 'A'..='Z' => char::from_u32(0x1d434 + u32::from(c) - u32::from('A')).unwrap(), | ||
| 320 | 'a'..='z' => char::from_u32(0x1d44e + u32::from(c) - u32::from('a')).unwrap(), | ||
| 321 | c => c, | ||
| 322 | } | ||
| 323 | } |
crates/onestore/src/page/text.rs+3-1
| ... | @@ -154,10 +154,12 @@ impl From<EditError> for crate::Error { | ... | @@ -154,10 +154,12 @@ impl From<EditError> for crate::Error { |
| 154 | } | 154 | } |
| 155 | 155 | ||
| 156 | /// A fresh random identity for a new paragraph or text object. | 156 | /// A fresh random identity for a new paragraph or text object. |
| 157 | /// A fresh identity with `n` 1: OneNote never stores an object as `{guid},0`, and OneNote | ||
| 158 | /// 2010 loses outline elements stored that way (corpus/math-edit/native-drop). | ||
| 157 | pub fn new_id() -> Result<ExGuid, EditError> { | 159 | pub fn new_id() -> Result<ExGuid, EditError> { |
| 158 | Ok(ExGuid { | 160 | Ok(ExGuid { |
| 159 | guid: crate::write::fresh_guid().map_err(|_| EditError::Identity)?, | 161 | guid: crate::write::fresh_guid().map_err(|_| EditError::Identity)?, |
| 160 | n: 0, | 162 | n: 1, |
| 161 | }) | 163 | }) |
| 162 | } | 164 | } |
| 163 | 165 |
crates/onestore/src/page/write.rs+163-1
| ... | @@ -1,7 +1,7 @@ | ... | @@ -1,7 +1,7 @@ |
| 1 | //! Publishes an edited page model by lowering the difference from the stored page onto | 1 | //! Publishes an edited page model by lowering the difference from the stored page onto |
| 2 | //! the typed writers, then squashing their transactions into one revision per space. | 2 | //! the typed writers, then squashing their transactions into one revision per space. |
| 3 | 3 | ||
| 4 | use super::{Image, Ink, Outline, Page, PageObject, PageParagraph, ParagraphContent, Table}; | 4 | use super::{Image, Ink, Math, Outline, Page, PageObject, PageParagraph, ParagraphContent, Table}; |
| 5 | use crate::{ | 5 | use crate::{ |
| 6 | Error, ExGuid, Insertion, ObjectData, OutlineEdit, ParagraphJoin, ParagraphSplit, PropertySets, | 6 | Error, ExGuid, Insertion, ObjectData, OutlineEdit, ParagraphJoin, ParagraphSplit, PropertySets, |
| 7 | RevisionIndex, Store, TextAttribute, TreeEdit, Value, | 7 | RevisionIndex, Store, TextAttribute, TreeEdit, Value, |
| ... | @@ -184,6 +184,48 @@ fn stroke_values(stroke: &super::InkStroke, index: u32, filetime: u64) -> Result | ... | @@ -184,6 +184,48 @@ fn stroke_values(stroke: &super::InkStroke, index: u32, filetime: u64) -> Result |
| 184 | ]) | 184 | ]) |
| 185 | } | 185 | } |
| 186 | 186 | ||
| 187 | /// The character-style properties a span format sets; paragraph-level fields stay on the | ||
| 188 | /// paragraph style. | ||
| 189 | fn style_values(format: &crate::document::Format) -> Values { | ||
| 190 | let mut values = Values::new(); | ||
| 191 | for (id, flag) in [ | ||
| 192 | (0x08001c04, format.bold), | ||
| 193 | (0x08001c05, format.italic), | ||
| 194 | (0x08001c06, format.underline), | ||
| 195 | (0x08001c07, format.strike), | ||
| 196 | (0x08001c08, format.superscript), | ||
| 197 | (0x08001c09, format.subscript), | ||
| 198 | (0x08001e16, format.hidden), | ||
| 199 | (0x08001e14, format.hyperlink), | ||
| 200 | (0x08001e19, format.hyperlink_label), | ||
| 201 | (0x08003401, format.math), | ||
| 202 | (0x08001e22, format.embedded_object), | ||
| 203 | ] { | ||
| 204 | if let Some(flag) = flag { | ||
| 205 | values.push((id | (u32::from(flag) << 31), Vec::new())); | ||
| 206 | } | ||
| 207 | } | ||
| 208 | if let Some(font) = &format.font { | ||
| 209 | values.push((0x1c001c0a, crate::create::string(font))); | ||
| 210 | } | ||
| 211 | if let Some(size) = format.font_size { | ||
| 212 | values.push(( | ||
| 213 | 0x10001c0b, | ||
| 214 | ((size * 2.0).round() as u16).to_le_bytes().to_vec(), | ||
| 215 | )); | ||
| 216 | } | ||
| 217 | if let Some(color) = format.color { | ||
| 218 | values.push((0x14001c0c, color.to_le_bytes().to_vec())); | ||
| 219 | } | ||
| 220 | if let Some(highlight) = format.highlight { | ||
| 221 | values.push((0x14001c0d, highlight.to_le_bytes().to_vec())); | ||
| 222 | } | ||
| 223 | if let Some(language) = format.language { | ||
| 224 | values.push((0x14001c3b, language.to_le_bytes().to_vec())); | ||
| 225 | } | ||
| 226 | values | ||
| 227 | } | ||
| 228 | |||
| 187 | fn picture_fixed_fields(stored: &Image, image: &Image) -> Result<(), Error> { | 229 | fn picture_fixed_fields(stored: &Image, image: &Image) -> Result<(), Error> { |
| 188 | if stored.id != image.id | 230 | if stored.id != image.id |
| 189 | || stored.bytes != image.bytes | 231 | || stored.bytes != image.bytes |
| ... | @@ -494,6 +536,7 @@ impl Lowering<'_> { | ... | @@ -494,6 +536,7 @@ impl Lowering<'_> { |
| 494 | self.split_and_join(&old, &new, &mut placed, &mut consumed)?; | 536 | self.split_and_join(&old, &new, &mut placed, &mut consumed)?; |
| 495 | self.place(&old, &new, &placed, &page_order)?; | 537 | self.place(&old, &new, &placed, &page_order)?; |
| 496 | self.delete(&old, &new, &consumed)?; | 538 | self.delete(&old, &new, &consumed)?; |
| 539 | self.edit_equations(&new)?; | ||
| 497 | self.edit_text(&new)?; | 540 | self.edit_text(&new)?; |
| 498 | self.edit_lists(after, &new)?; | 541 | self.edit_lists(after, &new)?; |
| 499 | self.edit_tags(after, &new)?; | 542 | self.edit_tags(after, &new)?; |
| ... | @@ -1679,6 +1722,125 @@ impl Lowering<'_> { | ... | @@ -1679,6 +1722,125 @@ impl Lowering<'_> { |
| 1679 | Ok(()) | 1722 | Ok(()) |
| 1680 | } | 1723 | } |
| 1681 | 1724 | ||
| 1725 | /// Rewrites every equation paragraph whose stored text object differs from the model, the | ||
| 1726 | /// way OneNote stores an equation: the linear text, one run per span with a style | ||
| 1727 | /// carrying the span's format, the run-data array naming each run's inline object, and | ||
| 1728 | /// the math language marker on the text object. | ||
| 1729 | fn edit_equations(&mut self, new: &View<'_>) -> Result<(), Error> { | ||
| 1730 | let current = self.current()?; | ||
| 1731 | let current = View::new(¤t)?; | ||
| 1732 | for (id, paragraph) in &new.paragraphs { | ||
| 1733 | let Some(text) = paragraph.text() else { | ||
| 1734 | continue; | ||
| 1735 | }; | ||
| 1736 | if !Math::is_equation(&text.text) { | ||
| 1737 | continue; | ||
| 1738 | } | ||
| 1739 | let stored = current | ||
| 1740 | .text(self.id(*id)) | ||
| 1741 | .ok_or_else(|| invalid("An equation paragraph is missing after placement"))?; | ||
| 1742 | if stored.text == text.text { | ||
| 1743 | continue; | ||
| 1744 | } | ||
| 1745 | let object = stored.id; | ||
| 1746 | if text.text.text().contains('\u{fffc}') { | ||
| 1747 | return Err(invalid("Equations cannot hold embedded objects")); | ||
| 1748 | } | ||
| 1749 | let encoded: Vec<u8> = text | ||
| 1750 | .text | ||
| 1751 | .text() | ||
| 1752 | .encode_utf16() | ||
| 1753 | .chain([0]) | ||
| 1754 | .flat_map(u16::to_le_bytes) | ||
| 1755 | .collect(); | ||
| 1756 | let mut ends = Vec::new(); | ||
| 1757 | let mut styles: Vec<Values> = Vec::new(); | ||
| 1758 | let mut sets = Vec::new(); | ||
| 1759 | let all_math = text | ||
| 1760 | .text | ||
| 1761 | .spans() | ||
| 1762 | .iter() | ||
| 1763 | .all(|s| s.format.math == Some(true)); | ||
| 1764 | for span in text.text.spans() { | ||
| 1765 | ends.extend(text.text.utf16_offset(span.end)?.to_le_bytes()); | ||
| 1766 | styles.push(style_values(&span.format)); | ||
| 1767 | sets.push(match &span.format.math_object { | ||
| 1768 | Some(object) => { | ||
| 1769 | let mut set = vec![(0x1400344f, object.kind.to_le_bytes().to_vec())]; | ||
| 1770 | if let Some(count) = object.arguments { | ||
| 1771 | set.push((0x14003450, count.to_le_bytes().to_vec())); | ||
| 1772 | } | ||
| 1773 | for (id, symbol) in [0x10003453, 0x10003454, 0x10003455] | ||
| 1774 | .into_iter() | ||
| 1775 | .zip(&object.symbols) | ||
| 1776 | { | ||
| 1777 | let unit = u16::try_from(u32::from(*symbol)) | ||
| 1778 | .map_err(|_| invalid("Math symbols are single UTF-16 units"))?; | ||
| 1779 | set.push((id, unit.to_le_bytes().to_vec())); | ||
| 1780 | } | ||
| 1781 | set | ||
| 1782 | } | ||
| 1783 | None => Vec::new(), | ||
| 1784 | }); | ||
| 1785 | } | ||
| 1786 | ends.truncate(ends.len() - 4); | ||
| 1787 | let has_objects = sets.iter().any(|set| !set.is_empty()); | ||
| 1788 | let modified = crate::create::current_timestamps()?.0.to_le_bytes(); | ||
| 1789 | let space = self.space; | ||
| 1790 | self.apply(|current| { | ||
| 1791 | crate::write::write_revision(current, space, |raw| { | ||
| 1792 | let mut changed = BTreeMap::new(); | ||
| 1793 | let mut target = PropertyObject::from_object(&raw.objects[&object])?; | ||
| 1794 | let mut references = Vec::new(); | ||
| 1795 | let mut created: Vec<(Values, ExGuid)> = Vec::new(); | ||
| 1796 | for values in &styles { | ||
| 1797 | let id = match created.iter().find(|(known, _)| known == values) { | ||
| 1798 | Some((_, id)) => *id, | ||
| 1799 | None => { | ||
| 1800 | let id = ExGuid { | ||
| 1801 | guid: crate::write::fresh_guid()?, | ||
| 1802 | n: 1, | ||
| 1803 | }; | ||
| 1804 | let mut style = PropertyObject { | ||
| 1805 | jcid: 0x12004d, | ||
| 1806 | bytes: crate::create::properties(values)?, | ||
| 1807 | global_ids: std::sync::Arc::new(BTreeMap::from([(0, id.guid)])), | ||
| 1808 | }; | ||
| 1809 | style.reference(id)?; | ||
| 1810 | changed.insert(id, style); | ||
| 1811 | created.push((values.clone(), id)); | ||
| 1812 | id | ||
| 1813 | } | ||
| 1814 | }; | ||
| 1815 | references.extend(target.reference(id)?); | ||
| 1816 | } | ||
| 1817 | target.remove(&[0x1c003498, 0x40003499])?; | ||
| 1818 | target.set(&[ | ||
| 1819 | (0x1c001c22, &encoded), | ||
| 1820 | (0x1c001e12, &ends), | ||
| 1821 | (0x24001e13, &references), | ||
| 1822 | (0x14001d7a, &modified), | ||
| 1823 | ])?; | ||
| 1824 | if has_objects { | ||
| 1825 | target.set_sets(0x40003499, 0x44000811, &sets)?; | ||
| 1826 | } | ||
| 1827 | if all_math { | ||
| 1828 | // The flags and language marker OneNote's equation editor leaves on | ||
| 1829 | // every equation text object. | ||
| 1830 | target.set(&[ | ||
| 1831 | (0x10001cfe, &0x7f_u16.to_le_bytes()), | ||
| 1832 | (0x14001c3e, &1u32.to_le_bytes()), | ||
| 1833 | (0x14001c84, &1u32.to_le_bytes()), | ||
| 1834 | ])?; | ||
| 1835 | } | ||
| 1836 | changed.insert(object, target); | ||
| 1837 | Ok(changed) | ||
| 1838 | }) | ||
| 1839 | })?; | ||
| 1840 | } | ||
| 1841 | Ok(()) | ||
| 1842 | } | ||
| 1843 | |||
| 1682 | fn edit_text(&mut self, new: &View<'_>) -> Result<(), Error> { | 1844 | fn edit_text(&mut self, new: &View<'_>) -> Result<(), Error> { |
| 1683 | let current = self.current()?; | 1845 | let current = self.current()?; |
| 1684 | let current = View::new(¤t)?; | 1846 | let current = View::new(¤t)?; |
crates/onestore/src/write.rs+11
| ... | @@ -782,6 +782,17 @@ pub(crate) fn write_revisions_with_payloads( | ... | @@ -782,6 +782,17 @@ pub(crate) fn write_revisions_with_payloads( |
| 782 | index.validate_current()?; | 782 | index.validate_current()?; |
| 783 | let changes = edit(&index)?; | 783 | let changes = edit(&index)?; |
| 784 | let mut output = source.to_vec(); | 784 | let mut output = source.to_vec(); |
| 785 | // Native files reserve 1 KiB per transaction-log fragment; a fragment that ends the file | ||
| 786 | // keeps that room before the first new chunk. | ||
| 787 | let tail = store.transaction_fragments.last().unwrap().chunk; | ||
| 788 | let reserved = usize::try_from(tail.offset + tail.length.max(1024) + 32).unwrap(); | ||
| 789 | if usize::try_from(tail.offset + tail.length) | ||
| 790 | .unwrap() | ||
| 791 | .next_multiple_of(8) | ||
| 792 | >= output.len() | ||
| 793 | { | ||
| 794 | output.resize(reserved, 0); | ||
| 795 | } | ||
| 785 | let mut maximum = store | 796 | let mut maximum = store |
| 786 | .transaction_fragments | 797 | .transaction_fragments |
| 787 | .iter() | 798 | .iter() |
crates/onestore/tests/page_math.rs+241-8
| ... | @@ -8,10 +8,13 @@ use onestore::{ | ... | @@ -8,10 +8,13 @@ use onestore::{ |
| 8 | /// MathML (`corpus/m6/native-math-01`). | 8 | /// MathML (`corpus/m6/native-math-01`). |
| 9 | const NATIVE: &[u8] = include_bytes!("../../../corpus/m6/native-math-01/notebook/synthetic.one"); | 9 | const NATIVE: &[u8] = include_bytes!("../../../corpus/m6/native-math-01/notebook/synthetic.one"); |
| 10 | const NATIVE_READ: &str = include_str!("../../../corpus/m6/native-math-01/read/page-000.xml"); | 10 | const NATIVE_READ: &str = include_str!("../../../corpus/m6/native-math-01/read/page-000.xml"); |
| 11 | /// Five more expressions typed through the equation editor by `tools/native_math.py`: | ||
| 12 | /// subscripts, a sub-superscript, parentheses, an integral and a sum with limits. | ||
| 13 | const EDITOR: &[u8] = include_bytes!("../../../corpus/math-edit/native-editor/notebook/links.one"); | ||
| 14 | const EDITOR_READ: &str = include_str!("../../../corpus/math-edit/native-editor/read/page-000.xml"); | ||
| 11 | 15 | ||
| 12 | fn native_mathml() -> Vec<String> { | 16 | fn native_mathml(read: &str) -> Vec<String> { |
| 13 | NATIVE_READ | 17 | read.split("<mml:math") |
| 14 | .split("<mml:math") | ||
| 15 | .skip(1) | 18 | .skip(1) |
| 16 | .map(|rest| { | 19 | .map(|rest| { |
| 17 | let body = &rest[rest.find('>').unwrap() + 1..rest.find("</mml:math>").unwrap()]; | 20 | let body = &rest[rest.find('>').unwrap() + 1..rest.find("</mml:math>").unwrap()]; |
| ... | @@ -30,9 +33,8 @@ fn native_mathml() -> Vec<String> { | ... | @@ -30,9 +33,8 @@ fn native_mathml() -> Vec<String> { |
| 30 | .collect() | 33 | .collect() |
| 31 | } | 34 | } |
| 32 | 35 | ||
| 33 | #[test] | 36 | fn rendered_equations(bytes: &[u8], title: &str) -> Vec<String> { |
| 34 | fn native_equations_render_to_the_mathml_onenote_exports() { | 37 | let store = Store::parse(bytes).unwrap(); |
| 35 | let store = Store::parse(NATIVE).unwrap(); | ||
| 36 | let index = RevisionIndex::parse(&store).unwrap(); | 38 | let index = RevisionIndex::parse(&store).unwrap(); |
| 37 | let document = Document::parse(&index).unwrap(); | 39 | let document = Document::parse(&index).unwrap(); |
| 38 | let page = document | 40 | let page = document |
| ... | @@ -40,7 +42,7 @@ fn native_equations_render_to_the_mathml_onenote_exports() { | ... | @@ -40,7 +42,7 @@ fn native_equations_render_to_the_mathml_onenote_exports() { |
| 40 | .unwrap() | 42 | .unwrap() |
| 41 | .into_iter() | 43 | .into_iter() |
| 42 | .map(|(space, _)| Page::from_space(&document, space).unwrap()) | 44 | .map(|(space, _)| Page::from_space(&document, space).unwrap()) |
| 43 | .find(|page| page.title == "Equation controls") | 45 | .find(|page| page.title == title) |
| 44 | .unwrap(); | 46 | .unwrap(); |
| 45 | let mut rendered = Vec::new(); | 47 | let mut rendered = Vec::new(); |
| 46 | for object in &page.objects { | 48 | for object in &page.objects { |
| ... | @@ -72,5 +74,236 @@ fn native_equations_render_to_the_mathml_onenote_exports() { | ... | @@ -72,5 +74,236 @@ fn native_equations_render_to_the_mathml_onenote_exports() { |
| 72 | rendered.push(Math::mathml(&Math::parse(&equation).unwrap())); | 74 | rendered.push(Math::mathml(&Math::parse(&equation).unwrap())); |
| 73 | } | 75 | } |
| 74 | } | 76 | } |
| 75 | assert_eq!(rendered, native_mathml()); | 77 | rendered |
| 78 | } | ||
| 79 | |||
| 80 | #[test] | ||
| 81 | fn native_equations_render_to_the_mathml_onenote_exports() { | ||
| 82 | assert_eq!( | ||
| 83 | rendered_equations(NATIVE, "Equation controls"), | ||
| 84 | native_mathml(NATIVE_READ) | ||
| 85 | ); | ||
| 86 | } | ||
| 87 | |||
| 88 | #[test] | ||
| 89 | fn equation_editor_expressions_render_to_the_mathml_onenote_exports() { | ||
| 90 | assert_eq!( | ||
| 91 | rendered_equations(EDITOR, "Read about Rust the Rust site"), | ||
| 92 | native_mathml(EDITOR_READ) | ||
| 93 | ); | ||
| 94 | } | ||
| 95 | |||
| 96 | fn object(kind: u32, symbols: &str, arguments: Vec<Vec<Math>>) -> Math { | ||
| 97 | Math::Object { | ||
| 98 | kind, | ||
| 99 | symbols: symbols.chars().collect(), | ||
| 100 | arguments, | ||
| 101 | } | ||
| 102 | } | ||
| 103 | |||
| 104 | fn built_equations() -> Vec<(Vec<Math>, &'static str)> { | ||
| 105 | vec![ | ||
| 106 | ( | ||
| 107 | vec![ | ||
| 108 | object( | ||
| 109 | 31, | ||
| 110 | "^", | ||
| 111 | vec![vec![Math::Identifier('x')], vec![Math::Number("2".into())]], | ||
| 112 | ), | ||
| 113 | Math::Operator('+'), | ||
| 114 | Math::Number("1".into()), | ||
| 115 | ], | ||
| 116 | "<mml:msup><mml:mi>x</mml:mi><mml:mn>2</mml:mn></mml:msup><mml:mo>+</mml:mo><mml:mn>1</mml:mn>", | ||
| 117 | ), | ||
| 118 | ( | ||
| 119 | vec![object( | ||
| 120 | 16, | ||
| 121 | "/", | ||
| 122 | vec![ | ||
| 123 | vec![ | ||
| 124 | Math::Identifier('a'), | ||
| 125 | Math::Operator('+'), | ||
| 126 | Math::Identifier('b'), | ||
| 127 | ], | ||
| 128 | vec![object( | ||
| 129 | 31, | ||
| 130 | "^", | ||
| 131 | vec![vec![Math::Identifier('c')], vec![Math::Number("2".into())]], | ||
| 132 | )], | ||
| 133 | ], | ||
| 134 | )], | ||
| 135 | "<mml:mfrac><mml:mrow><mml:mi>a</mml:mi><mml:mo>+</mml:mo><mml:mi>b</mml:mi></mml:mrow><mml:mrow><mml:msup><mml:mi>c</mml:mi><mml:mn>2</mml:mn></mml:msup></mml:mrow></mml:mfrac>", | ||
| 136 | ), | ||
| 137 | ] | ||
| 138 | } | ||
| 139 | |||
| 140 | #[test] | ||
| 141 | fn equations_built_from_the_tree_store_the_native_linear_form_and_parse_back() { | ||
| 142 | for (nodes, mathml) in built_equations() { | ||
| 143 | let paragraph = Math::paragraph(&nodes, &onestore::document::Format::default()); | ||
| 144 | assert!(Math::is_equation(¶graph)); | ||
| 145 | assert_eq!(Math::parse(¶graph).unwrap(), nodes); | ||
| 146 | assert_eq!(Math::mathml(&nodes), mathml); | ||
| 147 | } | ||
| 148 | let paragraph = Math::paragraph(&built_equations()[0].0, &Default::default()); | ||
| 149 | assert_eq!(paragraph.text(), "\u{fdd0}\u{1d465}\u{fdee}2\u{fdef}+1"); | ||
| 150 | let kinds: Vec<(Option<u32>, Option<u32>)> = paragraph | ||
| 151 | .spans() | ||
| 152 | .iter() | ||
| 153 | .map(|s| { | ||
| 154 | let object = s.format.math_object.as_ref().unwrap(); | ||
| 155 | (Some(object.kind), object.arguments) | ||
| 156 | }) | ||
| 157 | .collect(); | ||
| 158 | assert_eq!( | ||
| 159 | kinds, | ||
| 160 | [ | ||
| 161 | (Some(31), Some(2)), | ||
| 162 | (Some(31), None), | ||
| 163 | (Some(31), Some(1)), | ||
| 164 | (Some(0x9000_0000), None) | ||
| 165 | ] | ||
| 166 | ); | ||
| 167 | } | ||
| 168 | |||
| 169 | /// `ONESTORE_MATH_EXPORT` names a new directory receiving the candidate for a cold reopen: | ||
| 170 | /// every native equation copied onto a fresh page, then two built from the tree. | ||
| 171 | #[test] | ||
| 172 | fn equations_are_written_and_read_back() { | ||
| 173 | use onestore::page::{ParagraphContent, TextObject, text::new_id}; | ||
| 174 | let source = onestore::create_section("math.one", "Equations below", "Author").unwrap(); | ||
| 175 | let store = Store::parse(&source).unwrap(); | ||
| 176 | let index = RevisionIndex::parse(&store).unwrap(); | ||
| 177 | let document = Document::parse(&index).unwrap(); | ||
| 178 | let (space, _) = document.pages().unwrap()[0]; | ||
| 179 | let before = Page::from_space(&document, space).unwrap(); | ||
| 180 | let mut after = before.clone(); | ||
| 181 | let mut expected_mathml = Vec::new(); | ||
| 182 | let mut paragraphs = Vec::new(); | ||
| 183 | for (bytes, title, read) in [ | ||
| 184 | (NATIVE, "Equation controls", NATIVE_READ), | ||
| 185 | (EDITOR, "Read about Rust the Rust site", EDITOR_READ), | ||
| 186 | ] { | ||
| 187 | expected_mathml.extend(native_mathml(read)); | ||
| 188 | let store = Store::parse(bytes).unwrap(); | ||
| 189 | let index = RevisionIndex::parse(&store).unwrap(); | ||
| 190 | let document = Document::parse(&index).unwrap(); | ||
| 191 | let page = document | ||
| 192 | .pages() | ||
| 193 | .unwrap() | ||
| 194 | .into_iter() | ||
| 195 | .map(|(space, _)| Page::from_space(&document, space).unwrap()) | ||
| 196 | .find(|page| page.title == title) | ||
| 197 | .unwrap(); | ||
| 198 | for object in &page.objects { | ||
| 199 | let PageObject::Outline(outline) = object else { | ||
| 200 | continue; | ||
| 201 | }; | ||
| 202 | for paragraph in &outline.paragraphs { | ||
| 203 | if paragraph.text().is_some_and(|t| Math::is_equation(&t.text)) { | ||
| 204 | paragraphs.push(paragraph.clone()); | ||
| 205 | } | ||
| 206 | } | ||
| 207 | } | ||
| 208 | } | ||
| 209 | for (nodes, mathml) in built_equations() { | ||
| 210 | let mut paragraph = paragraphs[0].clone(); | ||
| 211 | paragraph.content = ParagraphContent::Text(TextObject { | ||
| 212 | id: new_id().unwrap(), | ||
| 213 | date_field: None, | ||
| 214 | text: Math::paragraph(&nodes, &Default::default()), | ||
| 215 | tags: Vec::new(), | ||
| 216 | }); | ||
| 217 | paragraphs.push(paragraph); | ||
| 218 | expected_mathml.push(mathml.to_owned()); | ||
| 219 | } | ||
| 220 | let outline = after | ||
| 221 | .objects | ||
| 222 | .iter_mut() | ||
| 223 | .find_map(|object| match object { | ||
| 224 | PageObject::Outline(outline) if !outline.title => Some(outline), | ||
| 225 | _ => None, | ||
| 226 | }) | ||
| 227 | .unwrap(); | ||
| 228 | for mut paragraph in paragraphs { | ||
| 229 | paragraph.id = new_id().unwrap(); | ||
| 230 | paragraph.parent = None; | ||
| 231 | paragraph.level = 1; | ||
| 232 | paragraph.lists.clear(); | ||
| 233 | paragraph.tags.clear(); | ||
| 234 | paragraph.style = None; | ||
| 235 | paragraph.collapsed = false; | ||
| 236 | if let ParagraphContent::Text(text) = &mut paragraph.content { | ||
| 237 | text.id = new_id().unwrap(); | ||
| 238 | text.tags.clear(); | ||
| 239 | // Paragraph-level spacing came from the source page's paragraph style. | ||
| 240 | let mut at = 0; | ||
| 241 | text.text = | ||
| 242 | onestore::page::Paragraph::from_runs(text.text.spans().iter().map(|span| { | ||
| 243 | let piece = text.text.text()[at..span.end].to_owned(); | ||
| 244 | at = span.end; | ||
| 245 | let mut format = span.format.clone(); | ||
| 246 | format.space_before = None; | ||
| 247 | format.space_after = None; | ||
| 248 | format.line_spacing = None; | ||
| 249 | (piece, format) | ||
| 250 | })); | ||
| 251 | } | ||
| 252 | outline.paragraphs.push(paragraph); | ||
| 253 | } | ||
| 254 | let written = onestore::PreparedEdit::page(&source, space, &after, "Math author").unwrap(); | ||
| 255 | let store = Store::parse(written.as_bytes()).unwrap(); | ||
| 256 | let index = RevisionIndex::parse(&store).unwrap(); | ||
| 257 | let document = Document::parse(&index).unwrap(); | ||
| 258 | let stored = Page::from_space(&document, space).unwrap(); | ||
| 259 | let mut expected = after.clone(); | ||
| 260 | expected.title = stored.title.clone(); | ||
| 261 | let body = |page: &Page| { | ||
| 262 | page.objects | ||
| 263 | .iter() | ||
| 264 | .find_map(|object| match object { | ||
| 265 | PageObject::Outline(outline) if !outline.title => Some(outline.paragraphs.clone()), | ||
| 266 | _ => None, | ||
| 267 | }) | ||
| 268 | .unwrap() | ||
| 269 | }; | ||
| 270 | for (index, (a, b)) in body(&stored).iter().zip(body(&expected)).enumerate() { | ||
| 271 | if a != &b { | ||
| 272 | let (ta, tb) = (a.text().unwrap(), b.text().unwrap()); | ||
| 273 | assert_eq!(ta.text.text(), tb.text.text(), "paragraph {index} text"); | ||
| 274 | for (sa, sb) in ta.text.spans().iter().zip(tb.text.spans()) { | ||
| 275 | assert_eq!(sa, sb, "paragraph {index} span"); | ||
| 276 | } | ||
| 277 | assert_eq!( | ||
| 278 | ta.text.spans().len(), | ||
| 279 | tb.text.spans().len(), | ||
| 280 | "paragraph {index} spans" | ||
| 281 | ); | ||
| 282 | assert_eq!(a, &b, "paragraph {index}"); | ||
| 283 | } | ||
| 284 | } | ||
| 285 | assert_eq!(stored, expected); | ||
| 286 | assert_eq!( | ||
| 287 | rendered_equations(written.as_bytes(), &stored.title), | ||
| 288 | expected_mathml | ||
| 289 | ); | ||
| 290 | if let Some(directory) = std::env::var_os("ONESTORE_MATH_EXPORT") { | ||
| 291 | let directory = std::path::PathBuf::from(directory); | ||
| 292 | std::fs::create_dir(&directory).unwrap(); | ||
| 293 | std::fs::write(directory.join("math.one"), written.as_bytes()).unwrap(); | ||
| 294 | std::fs::write( | ||
| 295 | directory.join("Open Notebook.onetoc2"), | ||
| 296 | onestore::create_table_of_contents( | ||
| 297 | "Open Notebook.onetoc2", | ||
| 298 | &[("math.one", store.header.file_id)], | ||
| 299 | ) | ||
| 300 | .unwrap(), | ||
| 301 | ) | ||
| 302 | .unwrap(); | ||
| 303 | std::fs::write( | ||
| 304 | directory.join("expected-mathml.json"), | ||
| 305 | serde_json::to_string_pretty(&expected_mathml).unwrap(), | ||
| 306 | ) | ||
| 307 | .unwrap(); | ||
| 308 | } | ||
| 76 | } | 309 | } |
tools/native_format.py+2-1
| ... | @@ -97,7 +97,8 @@ def compare_formats(space, nodes, native): | ... | @@ -97,7 +97,8 @@ def compare_formats(space, nodes, native): |
| 97 | for run in kind['runs']: | 97 | for run in kind['runs']: |
| 98 | style = {**base, **({k: v for k, v in space['nodes'][run['format']]['format'].items() if v is not None} if run['format'] else {})} | 98 | style = {**base, **({k: v for k, v in space['nodes'][run['format']]['format'].items() if v is not None} if run['format'] else {})} |
| 99 | text = source[run['start'] * 2:run['end'] * 2].decode('utf-16-le') | 99 | text = source[run['start'] * 2:run['end'] * 2].decode('utf-16-le') |
| 100 | if not style.get('hidden'): | 100 | # Hidden runs and equation runs (exported as MathML) have no visible native text. |
| 101 | if not style.get('hidden') and not style.get('math'): | ||
| 101 | actual.extend((char, style) for char in text) | 102 | actual.extend((char, style) for char in text) |
| 102 | if actual and actual[-1][0] == '\r': | 103 | if actual and actual[-1][0] == '\r': |
| 103 | actual.pop() | 104 | actual.pop() |
tools/native_math.py created+70
| ... | @@ -0,0 +1,70 @@ | ||
| 1 | """Author equations through OneNote 2010's equation editor on a Rust-written page and capture | ||
| 2 | the COM read, which exports each equation as MathML: the oracle for `onestore::page::Math`. | ||
| 3 | |||
| 4 | Usage: native_math.py NOTEBOOK_DIR OUTPUT_DIR | ||
| 5 | """ | ||
| 6 | import base64 | ||
| 7 | import json | ||
| 8 | import sys | ||
| 9 | from pathlib import Path | ||
| 10 | import xml.etree.ElementTree as ET | ||
| 11 | |||
| 12 | sys.path.insert(0, str(Path(__file__).resolve().parent)) | ||
| 13 | import native_runner as runner | ||
| 14 | |||
| 15 | windows = runner.windows | ||
| 16 | |||
| 17 | # Linear-format input, one equation per paragraph; a space builds each expression up. | ||
| 18 | EQUATIONS = [ | ||
| 19 | 'a_1+b_2', | ||
| 20 | '\\sqrt(x+1)', | ||
| 21 | 'x_i^2', | ||
| 22 | '(a+b)', | ||
| 23 | '\\int_0^1 x dx', | ||
| 24 | '\\sum_(i=1)^n i', | ||
| 25 | ] | ||
| 26 | |||
| 27 | |||
| 28 | def interaction(name, output): | ||
| 29 | lines = [] | ||
| 30 | for equation in EQUATIONS: | ||
| 31 | lines.append('Send("!=")') | ||
| 32 | lines.append('Sleep(600)') | ||
| 33 | lines.append('SendText("%s")' % equation.replace('"', '""')) | ||
| 34 | lines.append('Send(" ")') | ||
| 35 | lines.append('Sleep(600)') | ||
| 36 | lines.append('Send("{End}")') | ||
| 37 | lines.append('Sleep(300)') | ||
| 38 | lines.append('Send("{Enter}")') | ||
| 39 | lines.append('Sleep(600)') | ||
| 40 | page = ET.parse(sorted((output / 'read').glob('page-*.xml'))[0]).getroot().attrib['ID'] | ||
| 41 | script = '''ComObject("OneNote.Application").NavigateTo("PAGE_ID", "", false) | ||
| 42 | WinWait("ahk_exe ONENOTE.EXE", , 30) | ||
| 43 | WinActivate("ahk_exe ONENOTE.EXE") | ||
| 44 | WinWaitActive("ahk_exe ONENOTE.EXE", , 30) | ||
| 45 | WinMaximize("ahk_exe ONENOTE.EXE") | ||
| 46 | Sleep(3000) | ||
| 47 | CoordMode("Mouse", "Screen") | ||
| 48 | Click(300, 140) | ||
| 49 | Sleep(500) | ||
| 50 | Send("{End}") | ||
| 51 | Sleep(300) | ||
| 52 | Send("{Enter}") | ||
| 53 | Sleep(500) | ||
| 54 | %s | ||
| 55 | Sleep(2000) | ||
| 56 | ''' % '\n'.join(lines) | ||
| 57 | script = script.replace('PAGE_ID', page) | ||
| 58 | (output / 'equations.ahk').write_text(script) | ||
| 59 | result = windows.do_exec(script, target=name, timeout_ms=180000, shot_delay_ms=1500) | ||
| 60 | screenshot = result.pop('png_b64', None) | ||
| 61 | if screenshot: | ||
| 62 | (output / 'equations.png').write_bytes(base64.b64decode(screenshot)) | ||
| 63 | (output / 'equations.json').write_text(json.dumps(result, indent=2)) | ||
| 64 | if result.get('error') or result.get('exit') != 0: | ||
| 65 | raise RuntimeError(str(result)) | ||
| 66 | |||
| 67 | |||
| 68 | if __name__ == '__main__': | ||
| 69 | notebook, output = Path(sys.argv[1]), Path(sys.argv[2]) | ||
| 70 | runner.capture(notebook, output, expected_pages=1, collect_notebook=True, interaction=interaction) | ||
tools/test_math_edit.py created+37
| ... | @@ -0,0 +1,37 @@ | ||
| 1 | import json | ||
| 2 | from pathlib import Path | ||
| 3 | import re | ||
| 4 | import runpy | ||
| 5 | import shutil | ||
| 6 | from tempfile import TemporaryDirectory | ||
| 7 | import unittest | ||
| 8 | |||
| 9 | ROOT = Path(__file__).resolve().parent.parent | ||
| 10 | FIXTURE = ROOT / 'corpus/math-edit' | ||
| 11 | compare = runpy.run_path(str(ROOT / 'tools/verify-document.py'))['compare'] | ||
| 12 | |||
| 13 | |||
| 14 | def exported_mathml(path): | ||
| 15 | text = path.read_text(encoding='utf-8-sig') | ||
| 16 | out = [] | ||
| 17 | for body in re.findall(r'<mml:math[^>]*>(.*?)</mml:math>', text, re.S): | ||
| 18 | out.append(re.sub(r'&#(\d+);', lambda m: chr(int(m.group(1))), body)) | ||
| 19 | return out | ||
| 20 | |||
| 21 | |||
| 22 | class MathEditTest(unittest.TestCase): | ||
| 23 | def test_equation_editor_expressions_export_as_mathml(self): | ||
| 24 | self.assertEqual(len(exported_mathml(FIXTURE / 'native-editor/read/page-000.xml')), 5) | ||
| 25 | |||
| 26 | def test_rust_written_equations_export_the_expected_mathml(self): | ||
| 27 | with TemporaryDirectory() as temporary: | ||
| 28 | read = Path(temporary) / 'read' | ||
| 29 | shutil.copytree(FIXTURE / 'written/cold/read', read) | ||
| 30 | compare(FIXTURE / 'written/candidate', read) | ||
| 31 | expected = json.loads((FIXTURE / 'written/candidate/expected-mathml.json').read_text()) | ||
| 32 | self.assertEqual(exported_mathml(FIXTURE / 'written/cold/read/page-000.xml'), expected) | ||
| 33 | self.assertTrue((FIXTURE / 'written/cold/read/page-000.png').exists()) | ||
| 34 | |||
| 35 | |||
| 36 | if __name__ == '__main__': | ||
| 37 | unittest.main() | ||
tools/verify-document.py+5-2
| ... | @@ -78,8 +78,10 @@ def verify_pdf_black(path, paragraphs): | ... | @@ -78,8 +78,10 @@ def verify_pdf_black(path, paragraphs): |
| 78 | def visible_text(node, space): | 78 | def visible_text(node, space): |
| 79 | kind = node['kind'] | 79 | kind = node['kind'] |
| 80 | data = kind['text'].encode('utf-16-le') | 80 | data = kind['text'].encode('utf-16-le') |
| 81 | # Hidden runs and equation runs (exported as MathML) have no visible native text. | ||
| 81 | text = ''.join(data[run['start'] * 2:run['end'] * 2].decode('utf-16-le') for run in kind['runs'] | 82 | text = ''.join(data[run['start'] * 2:run['end'] * 2].decode('utf-16-le') for run in kind['runs'] |
| 82 | if not run['format'] or not space['nodes'][run['format']]['format']['hidden']) | 83 | if not run['format'] or not (space['nodes'][run['format']]['format']['hidden'] |
| 84 | or space['nodes'][run['format']]['format']['math'])) | ||
| 83 | return "" if text == "\u00a0" else project_text(text.removesuffix('\r')) | 85 | return "" if text == "\u00a0" else project_text(text.removesuffix('\r')) |
| 84 | 86 | ||
| 85 | 87 | ||
| ... | @@ -391,8 +393,9 @@ def compare(notebook, native, versions=None, password_file=None): | ... | @@ -391,8 +393,9 @@ def compare(notebook, native, versions=None, password_file=None): |
| 391 | text_ids = [oid for root in roots for oid, n in walk(space, root) | 393 | text_ids = [oid for root in roots for oid, n in walk(space, root) |
| 392 | if n['kind']['type'] == 'RichText' and not n['kind']['boilerplate']] | 394 | if n['kind']['type'] == 'RichText' and not n['kind']['boilerplate']] |
| 393 | for text_id, expected_runs in zip(text_ids, native_runs, strict=True): | 395 | for text_id, expected_runs in zip(text_ids, native_runs, strict=True): |
| 396 | # Native XML exports equation runs as MathML, not text (tools/test_math_edit.py compares that). | ||
| 394 | observed_runs = [(char, run['link']) for run in resolved_text[sid][rid][text_id] | 397 | observed_runs = [(char, run['link']) for run in resolved_text[sid][rid][text_id] |
| 395 | if not run['format']['hidden'] for char in run['text']] | 398 | if not run['format']['hidden'] and not run['format']['math'] for char in run['text']] |
| 396 | if observed_runs and observed_runs[-1][0] == '\r': | 399 | if observed_runs and observed_runs[-1][0] == '\r': |
| 397 | observed_runs.pop() | 400 | observed_runs.pop() |
| 398 | observed_runs = [(projected, link) for char, link in observed_runs for projected in project_text(char)] | 401 | observed_runs = [(projected, link) for char, link in observed_runs for projected in project_text(char)] |