authorgravatar for git@paperclover.netclover caruso <git@paperclover.net> 2026-09-08 20:51:37-07:00
committergravatar for git@paperclover.netclover caruso <git@paperclover.net> 2026-09-08 20:51:37-07:00
logcffb05d4803d38e6fbc9acef1aff140f067a5ede
treef9ca4542f1fd8285c787093f9ae0b861f9cbb350
parent50cc916d18c0cf6e5bf920d93cbfc83a8d827dec
signature Signed by SSH key SHA256:52mNGHRsVFBDED9IAX5pe+LRWUefqTbxEReunq21QvU

feat: publish revisions across existing object spaces atomically

Assemble revision manifests for multiple spaces behind one transaction-count update while retaining per-space histories, reference counts and checkpoints. Keep the public API unchanged. Verify a three-space page nesting against native OneNote and a fresh cold reopen. Cover every interrupted I/O, 514 paired revisions across the counter carry and both checkpoints, 130 core tests, 99 offline tests, 148 Python tests, Clippy, both iOS targets and 3,687 sanitizer executions. Fix property copying stopping early on multi-reference arrays; cover all three streams with direct and nested regression cases. Assisted-by: gpt-6-astra

18 files changed, 882 insertions(+), 354 deletions(-)

corpus/page-lifecycle/atomic-nesting/README.md created+21
...@@ -0,0 +1,21 @@
1# Atomic page nesting
2
3The candidate nests `child` and `grandchild` from `../03-renamed` beneath the
4`Same title` page, at levels 2 and 3. Section series order, section metadata
5copies and the two page metadata revisions publish in one transaction. Page
6bodies and prior revisions retain their identities and contents.
7
8`../04-nested` is the independent native control. `cold` is a fresh OneNote
9reopen of the Rust candidate; all active page and section objects compare exactly.
10The files differ in 48 header bytes, including file identity and version metadata;
11all bytes after the header are identical. Both images are retained. The capture's
12owned clone was removed; `provenance.json` records its source and teardown.
13
14Set `ONESTORE_PAGE_BATCH_OUTPUT` to a new directory and run the core unit test
15`write::tests::multi::nesting_publishes_section_order_and_page_levels_in_one_transaction`
16to export another candidate. Capture it with `tools/native_runner.py INPUT OUTPUT
17--expected-pages 9 --collect-notebook`. Run `tools.test_page_lifecycle` after
18building workspace examples to compare native navigation and document content.
19
20This fixture verifies the internal transaction writer. It does not define a
21public page operation, offline conflict policy or implicit subpage movement.
corpus/page-lifecycle/atomic-nesting/candidate/Lifecycle.one created
Binary files /dev/null and b/corpus/page-lifecycle/atomic-nesting/candidate/Lifecycle.one differ
corpus/page-lifecycle/atomic-nesting/cold/notebook/Lifecycle.one created
Binary files /dev/null and b/corpus/page-lifecycle/atomic-nesting/cold/notebook/Lifecycle.one differ
corpus/page-lifecycle/atomic-nesting/cold/read/hierarchy.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Notebook xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" name="notebook" nickname="notebook" ID="{F5B1F840-D34A-416F-B1FA-1B8E5493F8A1}{1}{B0}" path="C:\one-tests\runs\capture\notebook" lastModifiedTime="2026-09-09T03:56:15.000Z" color="#B49EDE"><one:Section name="Lifecycle" ID="{4A1FE443-51FD-0EA6-0E24-D52F8F7A58E7}{1}{B0}" path="C:\one-tests\runs\capture\notebook\Lifecycle.one" lastModifiedTime="2026-09-09T03:56:15.000Z" color="#91BAAE"><one:Page ID="{AA25BDB6-1E7D-06A1-1B39-45F756307A77}{1}{B0}" name="Untitled page" dateTime="2026-09-09T03:09:19.000Z" lastModifiedTime="2026-09-09T03:56:15.000Z" pageLevel="1"/><one:Page ID="{E048C445-8ED6-0F33-0F7A-45D6BBA0B159}{1}{B0}" name="Untitled page" dateTime="2026-09-09T03:09:19.000Z" lastModifiedTime="2026-09-09T03:56:15.000Z" pageLevel="1"/><one:Page ID="{C0470576-870E-0811-1FE7-83224012993C}{1}{B0}" name="Untitled page" dateTime="2026-09-09T03:09:19.000Z" lastModifiedTime="2026-09-09T03:56:15.000Z" pageLevel="1"/><one:Page ID="{D48B4C0F-81CB-4D47-A2E4-CB9EF846A6E4}{1}{B0}" name="Same title" dateTime="2026-09-09T03:09:27.000Z" lastModifiedTime="2026-09-09T03:56:15.000Z" pageLevel="1"/><one:Page ID="{8907679C-078D-4741-9C09-F34ABC6C3215}{1}{B0}" name="child" dateTime="2026-09-09T03:09:29.000Z" lastModifiedTime="2026-09-09T03:56:15.000Z" pageLevel="2"/><one:Page ID="{D4DBA55F-9D47-491E-802F-2BE696DE6EC3}{1}{B0}" name="grandchild" dateTime="2026-09-09T03:09:29.000Z" lastModifiedTime="2026-09-09T03:56:15.000Z" pageLevel="3"/><one:Page ID="{07185772-4ECB-4A13-BC88-87DC41A9838F}{1}{B0}" name="Renamed 🦋 é" dateTime="2026-09-09T03:09:30.000Z" lastModifiedTime="2026-09-09T03:56:15.000Z" pageLevel="1"/><one:Page ID="{6CFDADD2-7837-4022-8328-8587D6901795}{1}{B0}" name="Body trailing 🦀 é preserved" dateTime="2026-09-09T03:09:30.000Z" lastModifiedTime="2026-09-09T03:56:15.000Z" pageLevel="1"/><one:Page ID="{17B4C931-2D3E-4740-A892-ED86D2593AA4}{1}{B0}" name="Body automatic 🦀 é preserved" dateTime="2026-09-09T03:09:30.000Z" lastModifiedTime="2026-09-09T03:56:15.000Z" pageLevel="1"/></one:Section></one:Notebook>
corpus/page-lifecycle/atomic-nesting/cold/read/page-000.xml created+4
...@@ -0,0 +1,4 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{07185772-4ECB-4A13-BC88-87DC41A9838F}{1}{B0}" name="Renamed 🦋 é" dateTime="2026-09-09T03:09:30.000Z" lastModifiedTime="2026-09-09T03:56:15.000Z" pageLevel="1" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Title lang="en-US"><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-09T03:09:34.000Z" lastModifiedTime="2026-09-09T03:09:34.000Z" objectID="{2AE0B6C6-A870-4D63-AAF6-679E97FF06F0}{16}{B0}" alignment="left" quickStyleIndex="0" style="font-family:Calibri;font-size:17.0pt"><one:T><![CDATA[Renamed &#129419; é]]></one:T></one:OE></one:Title><one:Outline author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" lastModifiedTime="2026-09-09T03:09:30.000Z" objectID="{61979D6C-C04D-41F9-AFB6-F8FE349FB7CE}{39}{B0}"><one:Position x="36.0" y="86.4000015258789" z="0"/><one:Size width="170.2942504882812" height="13.42771339416504"/><one:OEChildren><one:OE creationTime="2026-09-09T03:09:30.000Z" lastModifiedTime="2026-09-09T03:09:30.000Z" objectID="{61979D6C-C04D-41F9-AFB6-F8FE349FB7CE}{40}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[<span
3style='font-weight:bold' lang=en-US>Body duplicate &#129408; é</span><span
4lang=en-US> </span><span style='font-style:italic' lang=x-none>preserved</span>]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/page-lifecycle/atomic-nesting/cold/read/page-001.xml created+4
...@@ -0,0 +1,4 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{17B4C931-2D3E-4740-A892-ED86D2593AA4}{1}{B0}" name="Body automatic 🦀 é preserved" dateTime="2026-09-09T03:09:30.000Z" lastModifiedTime="2026-09-09T03:56:15.000Z" pageLevel="1" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Outline author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" lastModifiedTime="2026-09-09T03:09:30.000Z" objectID="{A0CAB379-8594-48A1-AA50-36B8F10799B7}{14}{B0}"><one:Position x="36.0" y="14.40000057220459" z="0"/><one:Size width="174.3547973632812" height="13.42771625518799"/><one:OEChildren><one:OE creationTime="2026-09-09T03:09:30.000Z" lastModifiedTime="2026-09-09T03:09:30.000Z" objectID="{A0CAB379-8594-48A1-AA50-36B8F10799B7}{15}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[<span
3style='font-weight:bold' lang=en-US>Body automatic &#129408; é</span><span
4lang=en-US> </span><span style='font-style:italic' lang=x-none>preserved</span>]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/page-lifecycle/atomic-nesting/cold/read/page-002.xml created+4
...@@ -0,0 +1,4 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{6CFDADD2-7837-4022-8328-8587D6901795}{1}{B0}" name="Body trailing 🦀 é preserved" dateTime="2026-09-09T03:09:30.000Z" lastModifiedTime="2026-09-09T03:56:15.000Z" pageLevel="1" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Title lang="en-US"><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-09T03:09:34.000Z" lastModifiedTime="2026-09-09T03:09:34.000Z" objectID="{2973BC5F-757A-4590-96F9-39879A8C0725}{16}{B0}" alignment="left" quickStyleIndex="0" style="font-family:Calibri;font-size:17.0pt"><one:T><![CDATA[]]></one:T></one:OE></one:Title><one:Outline author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" lastModifiedTime="2026-09-09T03:09:30.000Z" objectID="{FDCC8DAF-57A7-4A12-B17D-E763F017FDD9}{39}{B0}"><one:Position x="36.0" y="86.4000015258789" z="0"/><one:Size width="160.1752014160156" height="13.42771339416504"/><one:OEChildren><one:OE creationTime="2026-09-09T03:09:30.000Z" lastModifiedTime="2026-09-09T03:09:30.000Z" objectID="{FDCC8DAF-57A7-4A12-B17D-E763F017FDD9}{40}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[<span
3style='font-weight:bold' lang=en-US>Body trailing &#129408; é</span><span
4lang=en-US> </span><span style='font-style:italic' lang=x-none>preserved</span>]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/page-lifecycle/atomic-nesting/cold/read/page-003.xml created+4
...@@ -0,0 +1,4 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{8907679C-078D-4741-9C09-F34ABC6C3215}{1}{B0}" name="child" dateTime="2026-09-09T03:09:29.000Z" lastModifiedTime="2026-09-09T03:56:15.000Z" pageLevel="2" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Title lang="en-US"><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-09T03:09:29.000Z" lastModifiedTime="2026-09-09T03:09:29.000Z" objectID="{035312EF-E5A0-43E9-B44E-EC1807D7EBB1}{28}{B0}" alignment="left" quickStyleIndex="0" style="font-family:Calibri;font-size:17.0pt"><one:T><![CDATA[child]]></one:T></one:OE></one:Title><one:Outline author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" lastModifiedTime="2026-09-09T03:09:30.000Z" objectID="{035312EF-E5A0-43E9-B44E-EC1807D7EBB1}{35}{B0}"><one:Position x="36.0" y="86.4000015258789" z="0"/><one:Size width="149.6102294921875" height="13.42771339416504"/><one:OEChildren><one:OE creationTime="2026-09-09T03:09:29.000Z" lastModifiedTime="2026-09-09T03:09:29.000Z" objectID="{035312EF-E5A0-43E9-B44E-EC1807D7EBB1}{36}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[<span
3style='font-weight:bold' lang=en-US>Body child &#129408; é</span><span
4lang=en-US> </span><span style='font-style:italic' lang=x-none>preserved</span>]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/page-lifecycle/atomic-nesting/cold/read/page-004.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{AA25BDB6-1E7D-06A1-1B39-45F756307A77}{1}{B0}" name="Untitled page" dateTime="2026-09-09T03:09:19.000Z" lastModifiedTime="2026-09-09T03:56:15.000Z" pageLevel="1"><one:QuickStyleDef index="0" name="PageTitle" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="17.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Title lang="en-US"><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-09T03:09:19.000Z" lastModifiedTime="2026-09-09T03:09:19.000Z" objectID="{6450CCAD-22C8-0EC6-30BE-1EC7CC43F688}{15}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[]]></one:T></one:OE></one:Title></one:Page>
corpus/page-lifecycle/atomic-nesting/cold/read/page-005.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{C0470576-870E-0811-1FE7-83224012993C}{1}{B0}" name="Untitled page" dateTime="2026-09-09T03:09:19.000Z" lastModifiedTime="2026-09-09T03:56:15.000Z" pageLevel="1"><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings></one:Page>
corpus/page-lifecycle/atomic-nesting/cold/read/page-006.xml created+4
...@@ -0,0 +1,4 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{D48B4C0F-81CB-4D47-A2E4-CB9EF846A6E4}{1}{B0}" name="Same title" dateTime="2026-09-09T03:09:27.000Z" lastModifiedTime="2026-09-09T03:56:15.000Z" pageLevel="1" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Title lang="en-US"><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-09T03:09:27.000Z" lastModifiedTime="2026-09-09T03:09:27.000Z" objectID="{D690D575-D56D-47D2-90DB-0191D6BDB859}{32}{B0}" alignment="left" quickStyleIndex="0" style="font-family:Calibri;font-size:17.0pt"><one:T><![CDATA[Same title]]></one:T></one:OE></one:Title><one:Outline author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" lastModifiedTime="2026-09-09T03:09:29.000Z" objectID="{D690D575-D56D-47D2-90DB-0191D6BDB859}{39}{B0}"><one:Position x="36.0" y="86.4000015258789" z="0"/><one:Size width="158.2952728271484" height="13.42771339416504"/><one:OEChildren><one:OE creationTime="2026-09-09T03:09:27.000Z" lastModifiedTime="2026-09-09T03:09:29.000Z" objectID="{D690D575-D56D-47D2-90DB-0191D6BDB859}{40}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[<span
3style='font-weight:bold' lang=en-US>Body parent &#129408; é</span><span
4lang=en-US> </span><span style='font-style:italic' lang=x-none>preserved</span>]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/page-lifecycle/atomic-nesting/cold/read/page-007.xml created+4
...@@ -0,0 +1,4 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{D4DBA55F-9D47-491E-802F-2BE696DE6EC3}{1}{B0}" name="grandchild" dateTime="2026-09-09T03:09:29.000Z" lastModifiedTime="2026-09-09T03:56:15.000Z" pageLevel="3" lang="en-US"><one:QuickStyleDef index="0" name="p" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="11.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Title lang="en-US"><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-09T03:09:30.000Z" lastModifiedTime="2026-09-09T03:09:30.000Z" objectID="{440E8EEC-9C7D-4AB1-BC2A-9B6EBF560D82}{32}{B0}" alignment="left" quickStyleIndex="0" style="font-family:Calibri;font-size:17.0pt"><one:T><![CDATA[grandchild]]></one:T></one:OE></one:Title><one:Outline author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" lastModifiedTime="2026-09-09T03:09:30.000Z" objectID="{440E8EEC-9C7D-4AB1-BC2A-9B6EBF560D82}{39}{B0}"><one:Position x="36.0" y="86.4000015258789" z="0"/><one:Size width="175.9715728759766" height="13.42771339416504"/><one:OEChildren><one:OE creationTime="2026-09-09T03:09:30.000Z" lastModifiedTime="2026-09-09T03:09:30.000Z" objectID="{440E8EEC-9C7D-4AB1-BC2A-9B6EBF560D82}{40}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[<span
3style='font-weight:bold' lang=en-US>Body grandchild &#129408; é</span><span
4lang=en-US> </span><span style='font-style:italic' lang=x-none>preserved</span>]]></one:T></one:OE></one:OEChildren></one:Outline></one:Page>
corpus/page-lifecycle/atomic-nesting/cold/read/page-008.xml created+2
...@@ -0,0 +1,2 @@
1<?xml version="1.0"?>
2<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2010/onenote" ID="{E048C445-8ED6-0F33-0F7A-45D6BBA0B159}{1}{B0}" name="Untitled page" dateTime="2026-09-09T03:09:19.000Z" lastModifiedTime="2026-09-09T03:56:15.000Z" pageLevel="1"><one:QuickStyleDef index="0" name="PageTitle" fontColor="automatic" highlightColor="automatic" font="Calibri" fontSize="17.0" spaceBefore="0.0" spaceAfter="0.0"/><one:PageSettings RTL="false" color="automatic"><one:PageSize><one:Automatic/></one:PageSize><one:RuleLines visible="false"/></one:PageSettings><one:Title lang="en-US"><one:OE author="virtual" authorInitials="V" lastModifiedBy="virtual" lastModifiedByInitials="V" creationTime="2026-09-09T03:09:19.000Z" lastModifiedTime="2026-09-09T03:09:19.000Z" objectID="{CFAC019E-2183-032A-2410-3D6897C66FDB}{15}{B0}" alignment="left" quickStyleIndex="0"><one:T><![CDATA[]]></one:T></one:OE></one:Title></one:Page>
corpus/page-lifecycle/atomic-nesting/provenance.json created+33
...@@ -0,0 +1,33 @@
1{
2 "source": "../03-renamed/notebook/Lifecycle.one",
3 "native_control": "../04-nested",
4 "generator": "write::tests::multi::nesting_publishes_section_order_and_page_levels_in_one_transaction",
5 "capture": "evidence/m10/page-batch-cold",
6 "run": {
7 "notebook": "/Users/clo/dev/one/evidence/m10/page-batch-candidate",
8 "expected_pages": 9,
9 "author": null,
10 "author_timeout_seconds": 600,
11 "inspect": false,
12 "collect_notebook": true,
13 "base": {
14 "file": "win7-office-base.qcow2",
15 "format": "qcow2",
16 "sha256": "a1a4f8fab782ee14885ff801ca2f6347c208fdcfc3513637096f314315c89346",
17 "virtual_size": 68719476736
18 },
19 "scripts": {
20 "cold.ps1": "c177fc72ae6c2634d186f5671a880de20b09f9716532c37c69cb13aa15c7e331",
21 "read.ps1": "04013bfcccee40a17e2a225f9b9e96f40a3a8daad9e350609654f8eda3ccbb41"
22 }
23 },
24 "machine": {
25 "name": "m6-3d4af21a",
26 "hostname": "ONE-M6-3D4AF21A"
27 },
28 "teardown": {
29 "absent": true
30 },
31 "candidate_sha256": "02fc858fb8854dbd14c1d3c58b62fdd653e51216872afc0cd8b8a4b94b651b9d",
32 "cold_sha256": "f7a0c74f1a10d5933b386a7da7fee1ead860190b8794dc0ad3788e36c50f14eb"
33}
crates/onestore/src/write.rs+385-354
...@@ -511,7 +511,7 @@ impl PropertyObject {...@@ -511,7 +511,7 @@ impl PropertyObject {
511 bytes: compact_ids,511 bytes: compact_ids,
512 offset: 0,512 offset: 0,
513 };513 };
514 while cursor.offset < cursor.bytes.len() {514 while !cursor.bytes.is_empty() {
515 let id = cursor.compact(&source.global_ids)?;515 let id = cursor.compact(&source.global_ids)?;
516 added[index].extend_from_slice(&target.reference(id)?);516 added[index].extend_from_slice(&target.reference(id)?);
517 }517 }
...@@ -586,6 +586,24 @@ pub(crate) fn write_revision(...@@ -586,6 +586,24 @@ pub(crate) fn write_revision(
586 source: &[u8],586 source: &[u8],
587 space: ExGuid,587 space: ExGuid,
588 edit: impl FnOnce(&crate::ResolvedRevision<'_>) -> Result<BTreeMap<ExGuid, PropertyObject>>,588 edit: impl FnOnce(&crate::ResolvedRevision<'_>) -> Result<BTreeMap<ExGuid, PropertyObject>>,
589) -> Result<Vec<u8>> {
590 write_revisions(source, |index| {
591 let rid = *index
592 .spaces
593 .get(&space)
594 .and_then(|space| space.labels.get(&(ExGuid::default(), 1)))
595 .ok_or(Error {
596 offset: 0,
597 message: "Object space has no active default revision",
598 })?;
599 let revision = index.resolve(space, rid)?;
600 Ok(BTreeMap::from([(space, edit(&revision)?)]))
601 })
602}
603
604pub(crate) fn write_revisions(
605 source: &[u8],
606 edit: impl FnOnce(&RevisionIndex<'_>) -> Result<BTreeMap<ExGuid, BTreeMap<ExGuid, PropertyObject>>>,
589) -> Result<Vec<u8>> {607) -> Result<Vec<u8>> {
590 let store = Store::parse(source)?;608 let store = Store::parse(source)?;
591 let is_section = store.header.file_type == FileType::Section;609 let is_section = store.header.file_type == FileType::Section;
...@@ -597,399 +615,415 @@ pub(crate) fn write_revision(...@@ -597,399 +615,415 @@ pub(crate) fn write_revision(
597 }615 }
598 let index = RevisionIndex::parse(&store)?;616 let index = RevisionIndex::parse(&store)?;
599 index.validate_current()?;617 index.validate_current()?;
600 let rid = *index618 let changes = edit(&index)?;
601 .spaces619 let mut output = source.to_vec();
602 .get(&space)620 let maximum = store
603 .and_then(|space| space.labels.get(&(ExGuid::default(), 1)))621 .transaction_fragments
604 .ok_or(Error {622 .iter()
605 offset: 0,623 .flat_map(|fragment| fragment.entries.chunks_exact(8))
606 message: "Object space has no active default revision",624 .map(|entry| u32::from_le_bytes(entry[..4].try_into().unwrap()))
607 })?;625 .max()
608 let mut revision = index.resolve(space, rid)?;626 .unwrap();
609 let reachable = revision.reachable()?;627 let mut counts = Vec::new();
610 let mut replacements = edit(&revision)?;628 for (space, mut replacements) in changes {
611 for (id, replacement) in &replacements {629 let rid = *index
612 if let Some(object) = revision.objects.get(id) {630 .spaces
613 if !reachable.contains(id) {631 .get(&space)
632 .and_then(|space| space.labels.get(&(ExGuid::default(), 1)))
633 .ok_or(Error {
634 offset: 0,
635 message: "Object space has no active default revision",
636 })?;
637 let mut revision = index.resolve(space, rid)?;
638 let reachable = revision.reachable()?;
639 for (id, replacement) in &replacements {
640 if let Some(object) = revision.objects.get(id) {
641 if !reachable.contains(id) {
642 return Err(Error {
643 offset: 0,
644 message: "Object is not reachable in the active revision",
645 });
646 }
647 if object.jcid & 0x100000 != 0 {
648 return Err(Error {
649 offset: 0,
650 message: "Read-only object requires a new identity",
651 });
652 }
653 if replacement.jcid != object.jcid
654 || !matches!(object.data, ObjectData::Properties(_))
655 {
656 return Err(Error {
657 offset: 0,
658 message: "An existing object's type cannot be changed",
659 });
660 }
661 } else if !is_section {
614 return Err(Error {662 return Err(Error {
615 offset: 0,663 offset: 0,
616 message: "Object is not reachable in the active revision",664 message: "New objects require a section file",
617 });665 });
618 }666 }
619 if object.jcid & 0x100000 != 0 {667 if replacement.jcid & 0x20000 == 0
668 || replacement.global_ids.keys().any(|i| *i > 0xffffff)
669 {
620 return Err(Error {670 return Err(Error {
621 offset: 0,671 offset: 0,
622 message: "Read-only object requires a new identity",672 message: "Invalid property object declaration",
623 });673 });
624 }674 }
625 if replacement.jcid != object.jcid || !matches!(object.data, ObjectData::Properties(_))675 compact(*id, &replacement.global_ids)?;
676 PropertySets::parse(&replacement.bytes)?;
677 }
678 // Native coalescing of duplicate readonly styles can leave dangling references.
679 let mut aliases = BTreeMap::new();
680 for (id, replacement) in &replacements {
681 if revision.objects.contains_key(id)
682 || replacement.jcid & 0x100000 == 0
683 || PropertySets::parse(&replacement.bytes)?
684 .sets
685 .iter()
686 .flatten()
687 .any(|p| matches!(p.value, Value::References { .. }))
626 {688 {
627 return Err(Error {689 continue;
628 offset: 0,
629 message: "An existing object's type cannot be changed",
630 });
631 }690 }
632 } else if !is_section {691 let existing = revision.objects.iter().find_map(|(other, object)| {
633 return Err(Error {692 (reachable.contains(other)
634 offset: 0,693 && object.jcid == replacement.jcid
635 message: "New objects require a section file",694 && object.data == ObjectData::Properties(&replacement.bytes))
695 .then_some(*other)
636 });696 });
637 }697 let existing = existing.or_else(|| {
638 if replacement.jcid & 0x20000 == 0 || replacement.global_ids.keys().any(|i| *i > 0xffffff) {698 replacements.range(..id).find_map(|(other, object)| {
639 return Err(Error {699 (object.jcid == replacement.jcid && object.bytes == replacement.bytes)
640 offset: 0,700 .then_some(*aliases.get(other).unwrap_or(other))
641 message: "Invalid property object declaration",701 })
642 });702 });
703 if let Some(existing) = existing {
704 aliases.insert(*id, existing);
705 }
643 }706 }
644 compact(*id, &replacement.global_ids)?;707 for id in aliases.keys() {
645 PropertySets::parse(&replacement.bytes)?;708 replacements.remove(id);
646 }
647 // Native coalescing of duplicate readonly styles can leave dangling references.
648 let mut aliases = BTreeMap::new();
649 for (id, replacement) in &replacements {
650 if revision.objects.contains_key(id)
651 || replacement.jcid & 0x100000 == 0
652 || PropertySets::parse(&replacement.bytes)?
653 .sets
654 .iter()
655 .flatten()
656 .any(|p| matches!(p.value, Value::References { .. }))
657 {
658 continue;
659 }
660 let existing = revision.objects.iter().find_map(|(other, object)| {
661 (reachable.contains(other)
662 && object.jcid == replacement.jcid
663 && object.data == ObjectData::Properties(&replacement.bytes))
664 .then_some(*other)
665 });
666 let existing = existing.or_else(|| {
667 replacements.range(..id).find_map(|(other, object)| {
668 (object.jcid == replacement.jcid && object.bytes == replacement.bytes)
669 .then_some(*aliases.get(other).unwrap_or(other))
670 })
671 });
672 if let Some(existing) = existing {
673 aliases.insert(*id, existing);
674 }709 }
675 }710 for object in replacements.values_mut() {
676 for id in aliases.keys() {711 let mut remapped = Vec::new();
677 replacements.remove(id);712 for property in PropertySets::parse(&object.bytes)?.sets.iter().flatten() {
678 }713 if let Value::References {
679 for object in replacements.values_mut() {714 stream: crate::IdStream::Objects,
680 let mut remapped = Vec::new();715 compact_ids,
681 for property in PropertySets::parse(&object.bytes)?.sets.iter().flatten() {716 } = property.value
682 if let Value::References {717 {
683 stream: crate::IdStream::Objects,718 for bytes in compact_ids.chunks_exact(4) {
684 compact_ids,719 let offset = bytes.as_ptr().addr() - object.bytes.as_ptr().addr();
685 } = property.value720 let id =
686 {721 crate::bytes::Cursor { bytes, offset }.compact(&object.global_ids)?;
687 for bytes in compact_ids.chunks_exact(4) {722 if let Some(existing) = aliases.get(&id) {
688 let offset = bytes.as_ptr().addr() - object.bytes.as_ptr().addr();723 remapped.push((offset, *existing));
689 let id = crate::bytes::Cursor { bytes, offset }.compact(&object.global_ids)?;724 }
690 if let Some(existing) = aliases.get(&id) {
691 remapped.push((offset, *existing));
692 }725 }
693 }726 }
694 }727 }
728 for (offset, id) in remapped {
729 let reference = object.reference(id)?;
730 object.bytes[offset..offset + 4].copy_from_slice(&reference);
731 }
695 }732 }
696 for (offset, id) in remapped {733 replacements.retain(|id, replacement| {
697 let reference = object.reference(id)?;734 !revision.objects.get(id).is_some_and(|object| {
698 object.bytes[offset..offset + 4].copy_from_slice(&reference);735 object.data == ObjectData::Properties(&replacement.bytes)
699 }736 && object.global_ids == replacement.global_ids
700 }737 })
701 replacements.retain(|id, replacement| {
702 !revision.objects.get(id).is_some_and(|object| {
703 object.data == ObjectData::Properties(&replacement.bytes)
704 && object.global_ids == replacement.global_ids
705 })
706 });
707 if replacements.is_empty() {
708 return Ok(source.to_vec());
709 }
710 let mut changed: BTreeSet<_> = replacements.keys().copied().collect();
711 for (id, replacement) in &replacements {
712 revision.objects.insert(
713 *id,
714 crate::Object {
715 jcid: replacement.jcid,
716 reference_count: 0,
717 data: ObjectData::Properties(&replacement.bytes),
718 global_ids: Arc::clone(&replacement.global_ids),
719 },
720 );
721 }
722 let incoming = revision.reference_counts()?;
723 if replacements.keys().any(|id| !incoming.contains_key(id)) {
724 return Err(Error {
725 offset: 0,
726 message: "Edited object is not reachable in the resulting revision",
727 });738 });
728 }739 if replacements.is_empty() {
729 for (id, object) in &mut revision.objects {740 continue;
730 if reachable.contains(id) || incoming.contains_key(id) {
731 let count = incoming.get(id).copied().unwrap_or(0);
732 if object.reference_count != count {
733 object.reference_count = count;
734 changed.insert(*id);
735 }
736 }741 }
737 }742 let mut changed: BTreeSet<_> = replacements.keys().copied().collect();
738743 for (id, replacement) in &replacements {
739 // Native cold-open fails on long dependency chains; cap their depth at 512.744 revision.objects.insert(
740 let checkpoint = std::iter::successors(Some(rid), |id| {745 *id,
741 index.spaces[&space].revisions[id].dependency746 crate::Object {
742 })747 jcid: replacement.jcid,
743 .nth(511)748 reference_count: 0,
744 .is_some();749 data: ObjectData::Properties(&replacement.bytes),
745 let selected: Vec<_> = revision750 global_ids: Arc::clone(&replacement.global_ids),
746 .objects751 },
747 .iter()752 );
748 .filter(|(id, _)| checkpoint || changed.contains(id))753 }
749 .collect();754 let incoming = revision.reference_counts()?;
750 let toc_table = if checkpoint && !is_section {755 if replacements.keys().any(|id| !incoming.contains_key(id)) {
751 if selected.iter().any(|(_, object)| {
752 object.jcid != 0x20001 || !matches!(object.data, ObjectData::Properties(_))
753 }) {
754 return Err(Error {756 return Err(Error {
755 offset: 0,757 offset: 0,
756 message: "TOC checkpoint requires table-of-contents property objects",758 message: "Edited object is not reachable in the resulting revision",
757 });759 });
758 }760 }
759 let guids: BTreeSet<_> = selected761 for (id, object) in &mut revision.objects {
762 if reachable.contains(id) || incoming.contains_key(id) {
763 let count = incoming.get(id).copied().unwrap_or(0);
764 if object.reference_count != count {
765 object.reference_count = count;
766 changed.insert(*id);
767 }
768 }
769 }
770
771 // Native cold-open fails on long dependency chains; cap their depth at 512.
772 let checkpoint = std::iter::successors(Some(rid), |id| {
773 index.spaces[&space].revisions[id].dependency
774 })
775 .nth(511)
776 .is_some();
777 let selected: Vec<_> = revision
778 .objects
760 .iter()779 .iter()
761 .flat_map(|(_, object)| object.global_ids.values().copied())780 .filter(|(id, _)| checkpoint || changed.contains(id))
762 .collect();781 .collect();
763 if guids.len() > 0x1000000 {782 let toc_table = if checkpoint && !is_section {
764 return Err(Error {783 if selected.iter().any(|(_, object)| {
765 offset: 0,784 object.jcid != 0x20001 || !matches!(object.data, ObjectData::Properties(_))
766 message: "Global ID table exceeds CompactID capacity",785 }) {
767 });786 return Err(Error {
768 }787 offset: 0,
769 Some(788 message: "TOC checkpoint requires table-of-contents property objects",
770 guids789 });
771 .into_iter()790 }
772 .enumerate()791 let guids: BTreeSet<_> = selected
773 .map(|(i, guid)| (u32::try_from(i).unwrap(), guid))792 .iter()
774 .collect::<BTreeMap<_, _>>(),793 .flat_map(|(_, object)| object.global_ids.values().copied())
775 )794 .collect();
776 } else {795 if guids.len() > 0x1000000 {
777 None796 return Err(Error {
778 };797 offset: 0,
779 let mut groups = BTreeMap::<_, Vec<_>>::new();798 message: "Global ID table exceeds CompactID capacity",
780 for (id, object) in selected {799 });
781 groups
782 .entry(toc_table.as_ref().unwrap_or(&object.global_ids))
783 .or_default()
784 .push((*id, object));
785 }
786 let mut output = source.to_vec();
787 let maximum = store
788 .transaction_fragments
789 .iter()
790 .flat_map(|fragment| fragment.entries.chunks_exact(8))
791 .map(|entry| u32::from_le_bytes(entry[..4].try_into().unwrap()))
792 .max()
793 .unwrap();
794 let new_rid = ExGuid {
795 guid: fresh_guid()?,
796 n: 1,
797 };
798 let mut start = Vec::new();
799 new_rid.encode(&mut start);
800 if checkpoint { ExGuid::default() } else { rid }.encode(&mut start);
801 if !is_section {
802 start.extend_from_slice(&0_u64.to_le_bytes());
803 }
804 start.extend_from_slice(&1_u32.to_le_bytes());
805 start.extend_from_slice(&0_u16.to_le_bytes());
806 let mut manifest = vec![node(if is_section { 0x1e } else { 0x1b }, None, &start)?];
807 let mut group_counts = Vec::new();
808 for (table, objects) in groups {
809 let mut payload = Vec::new();
810 let mut group = if is_section {
811 ExGuid {
812 guid: fresh_guid()?,
813 n: 1,
814 }800 }
815 .encode(&mut payload);801 Some(
816 vec![node(0xb4, None, &payload)?, node(0x22, None, &[])?]802 guids
803 .into_iter()
804 .enumerate()
805 .map(|(i, guid)| (u32::try_from(i).unwrap(), guid))
806 .collect::<BTreeMap<_, _>>(),
807 )
817 } else {808 } else {
818 vec![node(0x21, None, &[0])?]809 None
819 };810 };
820 for (id, guid) in table {811 let mut groups = BTreeMap::<_, Vec<_>>::new();
821 let mut entry = id.to_le_bytes().to_vec();812 for (id, object) in selected {
822 entry.extend_from_slice(guid);813 groups
823 group.push(node(0x24, None, &entry)?);814 .entry(toc_table.as_ref().unwrap_or(&object.global_ids))
815 .or_default()
816 .push((*id, object));
824 }817 }
825 group.push(node(0x28, None, &[])?);818 let new_rid = ExGuid {
826 let mut override_crc = u32::MAX;819 guid: fresh_guid()?,
827 for (id, object) in objects {820 n: 1,
828 let mut declaration = compact(id, table)?.to_vec();821 };
829 override_crc = crc(822 let mut start = Vec::new();
830 override_crc,823 new_rid.encode(&mut start);
831 &object.reference_count.to_le_bytes(),824 if checkpoint { ExGuid::default() } else { rid }.encode(&mut start);
832 FileType::Section,825 if !is_section {
833 );826 start.extend_from_slice(&0_u64.to_le_bytes());
834 match object.data {827 }
835 ObjectData::File {828 start.extend_from_slice(&1_u32.to_le_bytes());
836 reference,829 start.extend_from_slice(&0_u16.to_le_bytes());
837 extension,830 let mut manifest = vec![node(if is_section { 0x1e } else { 0x1b }, None, &start)?];
838 } => {831 for (table, objects) in groups {
839 declaration.extend_from_slice(&object.jcid.to_le_bytes());832 let mut payload = Vec::new();
840 declaration.extend_from_slice(&object.reference_count.to_le_bytes());833 let mut group = if is_section {
841 for bytes in [reference, extension] {834 ExGuid {
842 declaration.extend_from_slice(835 guid: fresh_guid()?,
843 &u32::try_from(bytes.len() / 2).unwrap().to_le_bytes(),836 n: 1,
844 );
845 declaration.extend_from_slice(bytes);
846 }
847 group.push(node(0x73, None, &declaration)?);
848 }837 }
849 ObjectData::Properties(bytes) => {838 .encode(&mut payload);
850 let references = object.references()?;839 vec![node(0xb4, None, &payload)?, node(0x22, None, &[])?]
851 let flags = u8::from(!references.objects.is_empty())840 } else {
852 | (u8::from(841 vec![node(0x21, None, &[0])?]
853 !references.object_spaces.is_empty() || !references.contexts.is_empty(),842 };
854 ) << 1);843 for (id, guid) in table {
855 let data = if toc_table.is_some() {844 let mut entry = id.to_le_bytes().to_vec();
856 let mut mapped = bytes.to_vec();845 entry.extend_from_slice(guid);
857 for property in PropertySets::parse(bytes)?.sets.iter().flatten() {846 group.push(node(0x24, None, &entry)?);
858 if let Value::References { compact_ids, .. } = property.value {847 }
859 let offset = compact_ids.as_ptr().addr() - bytes.as_ptr().addr();848 group.push(node(0x28, None, &[])?);
860 for (i, value) in compact_ids.chunks_exact(4).enumerate() {849 let mut override_crc = u32::MAX;
861 let value = u32::from_le_bytes(value.try_into().unwrap());850 for (id, object) in objects {
862 let target = ExGuid {851 let mut declaration = compact(id, table)?.to_vec();
863 guid: object.global_ids[&(value >> 8)],852 override_crc = crc(
864 n: value & 255,853 override_crc,
865 };854 &object.reference_count.to_le_bytes(),
866 mapped[offset + i * 4..offset + i * 4 + 4]855 FileType::Section,
867 .copy_from_slice(&compact(target, table)?);856 );
857 match object.data {
858 ObjectData::File {
859 reference,
860 extension,
861 } => {
862 declaration.extend_from_slice(&object.jcid.to_le_bytes());
863 declaration.extend_from_slice(&object.reference_count.to_le_bytes());
864 for bytes in [reference, extension] {
865 declaration.extend_from_slice(
866 &u32::try_from(bytes.len() / 2).unwrap().to_le_bytes(),
867 );
868 declaration.extend_from_slice(bytes);
869 }
870 group.push(node(0x73, None, &declaration)?);
871 }
872 ObjectData::Properties(bytes) => {
873 let references = object.references()?;
874 let flags = u8::from(!references.objects.is_empty())
875 | (u8::from(
876 !references.object_spaces.is_empty()
877 || !references.contexts.is_empty(),
878 ) << 1);
879 let data = if toc_table.is_some() {
880 let mut mapped = bytes.to_vec();
881 for property in PropertySets::parse(bytes)?.sets.iter().flatten() {
882 if let Value::References { compact_ids, .. } = property.value {
883 let offset =
884 compact_ids.as_ptr().addr() - bytes.as_ptr().addr();
885 for (i, value) in compact_ids.chunks_exact(4).enumerate() {
886 let value = u32::from_le_bytes(value.try_into().unwrap());
887 let target = ExGuid {
888 guid: object.global_ids[&(value >> 8)],
889 n: value & 255,
890 };
891 mapped[offset + i * 4..offset + i * 4 + 4]
892 .copy_from_slice(&compact(target, table)?);
893 }
868 }894 }
869 }895 }
870 }896 append(&mut output, &mapped)?
871 append(&mut output, &mapped)?897 } else if replacements.contains_key(&id) {
872 } else if replacements.contains_key(&id) {898 append(&mut output, bytes)?
873 append(&mut output, bytes)?899 } else {
874 } else {900 Chunk {
875 Chunk {901 offset: u64::try_from(
876 offset: u64::try_from(bytes.as_ptr().addr() - source.as_ptr().addr())902 bytes.as_ptr().addr() - source.as_ptr().addr(),
903 )
877 .unwrap(),904 .unwrap(),
878 length: u64::try_from(bytes.len()).unwrap(),905 length: u64::try_from(bytes.len()).unwrap(),
879 }906 }
880 };907 };
881 if is_section {
882 declaration.extend_from_slice(&object.jcid.to_le_bytes());
883 declaration.push(flags);
884 } else if checkpoint {
885 let body = 1_u64 | (u64::from(flags & 1) << 16);
886 declaration.extend_from_slice(&body.to_le_bytes()[..6]);
887 } else {
888 declaration.extend_from_slice(&u32::from(flags).to_le_bytes());
889 }
890 declaration.extend_from_slice(&object.reference_count.to_le_bytes());
891 let readonly = object.jcid & 0x100000 != 0;
892 if readonly {
893 declaration.extend_from_slice(&md5::compute(bytes).0);
894 }
895 group.push(node(
896 if is_section {908 if is_section {
897 if readonly { 0xc5 } else { 0xa5 }909 declaration.extend_from_slice(&object.jcid.to_le_bytes());
910 declaration.push(flags);
898 } else if checkpoint {911 } else if checkpoint {
899 0x2e912 let body = 1_u64 | (u64::from(flags & 1) << 16);
913 declaration.extend_from_slice(&body.to_le_bytes()[..6]);
900 } else {914 } else {
901 0x42915 declaration.extend_from_slice(&u32::from(flags).to_le_bytes());
902 },916 }
903 Some(Reference::Data(data)),917 declaration.extend_from_slice(&object.reference_count.to_le_bytes());
904 &declaration,918 let readonly = object.jcid & 0x100000 != 0;
905 )?);919 if readonly {
906 }920 declaration.extend_from_slice(&md5::compute(bytes).0);
907 ObjectData::Encrypted(_) => {921 }
908 return Err(Error {922 group.push(node(
909 offset: 0,923 if is_section {
910 message: "Encrypted objects cannot be checkpointed",924 if readonly { 0xc5 } else { 0xa5 }
911 });925 } else if checkpoint {
926 0x2e
927 } else {
928 0x42
929 },
930 Some(Reference::Data(data)),
931 &declaration,
932 )?);
933 }
934 ObjectData::Encrypted(_) => {
935 return Err(Error {
936 offset: 0,
937 message: "Encrypted objects cannot be checkpointed",
938 });
939 }
912 }940 }
913 }941 }
914 }
915 if is_section {
916 group.push(node(0xb8, None, &[])?);
917 let group_id = u32::try_from(group_counts.len())
918 .ok()
919 .and_then(|n| n.checked_add(1))
920 .and_then(|n| maximum.checked_add(n))
921 .ok_or(Error {
922 offset: 0,
923 message: "File-node list identities are exhausted",
924 })?;
925 let chunk = append_list(&mut output, group_id, &group)?;
926 group_counts.push((group_id, group.len()));
927 manifest.push(node(0xb0, Some(Reference::NodeList(chunk)), &payload)?);
928 let mut overrides = vec![0; 8];
929 overrides.extend_from_slice(&(!override_crc).to_le_bytes());
930 manifest.push(node(
931 0x84,
932 Some(Reference::Data(Chunk {
933 offset: u64::MAX,
934 length: 0,
935 })),
936 &overrides,
937 )?);
938 } else {
939 manifest.extend(group);
940 }
941 }
942 if checkpoint {
943 for (role, id) in &revision.roots {
944 let mut payload = Vec::new();
945 if is_section {942 if is_section {
946 id.encode(&mut payload);943 group.push(node(0xb8, None, &[])?);
944 let group_id = u32::try_from(counts.len())
945 .ok()
946 .and_then(|n| n.checked_add(1))
947 .and_then(|n| maximum.checked_add(n))
948 .ok_or(Error {
949 offset: 0,
950 message: "File-node list identities are exhausted",
951 })?;
952 let chunk = append_list(&mut output, group_id, &group)?;
953 counts.push((group_id, group.len()));
954 manifest.push(node(0xb0, Some(Reference::NodeList(chunk)), &payload)?);
955 let mut overrides = vec![0; 8];
956 overrides.extend_from_slice(&(!override_crc).to_le_bytes());
957 manifest.push(node(
958 0x84,
959 Some(Reference::Data(Chunk {
960 offset: u64::MAX,
961 length: 0,
962 })),
963 &overrides,
964 )?);
947 } else {965 } else {
948 payload.extend_from_slice(&compact(*id, toc_table.as_ref().unwrap())?);966 manifest.extend(group);
949 }967 }
950 payload.extend_from_slice(&role.to_le_bytes());
951 manifest.push(node(if is_section { 0x5a } else { 0x59 }, None, &payload)?);
952 }968 }
953 }969 if checkpoint {
954 manifest.push(node(0x1c, None, &[])?);970 for (role, id) in &revision.roots {
955 let root = store.list(store.header.root)?;971 let mut payload = Vec::new();
956 let space_node = root972 if is_section {
957 .nodes973 id.encode(&mut payload);
958 .iter()974 } else {
959 .find(|node| node.id == 8 && node.fields(&store).exguid() == Ok(space))975 payload.extend_from_slice(&compact(*id, toc_table.as_ref().unwrap())?);
960 .ok_or(Error {976 }
961 offset: 0,977 payload.extend_from_slice(&role.to_le_bytes());
962 message: "Object space is absent from the root list",978 manifest.push(node(if is_section { 0x5a } else { 0x59 }, None, &payload)?);
979 }
980 }
981 manifest.push(node(0x1c, None, &[])?);
982 let root = store.list(store.header.root)?;
983 let space_node = root
984 .nodes
985 .iter()
986 .find(|node| node.id == 8 && node.fields(&store).exguid() == Ok(space))
987 .ok_or(Error {
988 offset: 0,
989 message: "Object space is absent from the root list",
990 })?;
991 let space_list = space_node.referenced_list(&store)?;
992 let revision_node = space_list.iter().rfind(|node| node.id == 0x10).unwrap();
993 let Some(Reference::NodeList(manifest_reference)) = revision_node.reference else {
994 unreachable!()
995 };
996 let revision_list = store.list(manifest_reference)?;
997 let last_fragment = *revision_list.fragments.last().unwrap();
998 let list_start = usize::try_from(last_fragment.offset).unwrap();
999 let list_id =
1000 u32::from_le_bytes(source[list_start + 8..list_start + 12].try_into().unwrap());
1001 let manifest_chunk = append_list(&mut output, list_id, &manifest)?;
1002 let manifest_start = usize::try_from(manifest_chunk.offset).unwrap();
1003 let sequence = u32::try_from(revision_list.fragments.len()).map_err(|_| Error {
1004 offset: list_start,
1005 message: "File-node fragment sequences are exhausted",
963 })?;1006 })?;
964 let space_list = space_node.referenced_list(&store)?;1007 output[manifest_start + 12..manifest_start + 16].copy_from_slice(&sequence.to_le_bytes());
965 let revision_node = space_list.iter().rfind(|node| node.id == 0x10).unwrap();1008 let last_node = revision_list.nodes.last().unwrap();
966 let Some(Reference::NodeList(manifest_reference)) = revision_node.reference else {1009 let node_header = u32::from_le_bytes(
967 unreachable!()1010 source[last_node.offset..last_node.offset + 4]
968 };1011 .try_into()
969 let revision_list = store.list(manifest_reference)?;1012 .unwrap(),
970 let last_fragment = *revision_list.fragments.last().unwrap();1013 );
971 let list_start = usize::try_from(last_fragment.offset).unwrap();1014 let nodes_end = last_node.offset + usize::try_from((node_header >> 10) & 0x1fff).unwrap();
972 let list_id = u32::from_le_bytes(source[list_start + 8..list_start + 12].try_into().unwrap());1015 let tail = list_start + usize::try_from(last_fragment.length).unwrap() - 20;
973 let manifest_chunk = append_list(&mut output, list_id, &manifest)?;1016 if tail - nodes_end >= 4 {
974 let manifest_start = usize::try_from(manifest_chunk.offset).unwrap();1017 output[nodes_end..nodes_end + 4].copy_from_slice(&node(0xff, None, &[])?);
975 let sequence = u32::try_from(revision_list.fragments.len()).map_err(|_| Error {1018 }
976 offset: list_start,1019 output[tail..tail + 8].copy_from_slice(&manifest_chunk.offset.to_le_bytes());
977 message: "File-node fragment sequences are exhausted",1020 output[tail + 8..tail + 12].copy_from_slice(&(manifest_chunk.length as u32).to_le_bytes());
978 })?;1021
979 output[manifest_start + 12..manifest_start + 16].copy_from_slice(&sequence.to_le_bytes());1022 counts.push((list_id, revision_list.nodes.len() + manifest.len()));
980 let last_node = revision_list.nodes.last().unwrap();1023 }
981 let node_header = u32::from_le_bytes(1024 if counts.is_empty() {
982 source[last_node.offset..last_node.offset + 4]1025 return Ok(source.to_vec());
983 .try_into()
984 .unwrap(),
985 );
986 let nodes_end = last_node.offset + usize::try_from((node_header >> 10) & 0x1fff).unwrap();
987 let tail = list_start + usize::try_from(last_fragment.length).unwrap() - 20;
988 if tail - nodes_end >= 4 {
989 output[nodes_end..nodes_end + 4].copy_from_slice(&node(0xff, None, &[])?);
990 }1026 }
991 output[tail..tail + 8].copy_from_slice(&manifest_chunk.offset.to_le_bytes());
992 output[tail + 8..tail + 12].copy_from_slice(&(manifest_chunk.length as u32).to_le_bytes());
9931027
994 let transactions = store.header.transaction_count.checked_add(1).ok_or(Error {1028 let transactions = store.header.transaction_count.checked_add(1).ok_or(Error {
995 offset: 96,1029 offset: 96,
...@@ -999,10 +1033,7 @@ pub(crate) fn write_revision(...@@ -999,10 +1033,7 @@ pub(crate) fn write_revision(
999 let commit_byte = (31 - changed_bytes.leading_zeros()) / 8;1033 let commit_byte = (31 - changed_bytes.leading_zeros()) / 8;
1000 let ceiling = transactions | ((1_u32 << (commit_byte * 8)) - 1);1034 let ceiling = transactions | ((1_u32 << (commit_byte * 8)) - 1);
1001 let mut entries = Vec::new();1035 let mut entries = Vec::new();
1002 for (id, count) in group_counts1036 for (id, count) in counts {
1003 .into_iter()
1004 .chain([(list_id, revision_list.nodes.len() + manifest.len())])
1005 {
1006 entries.extend_from_slice(&id.to_le_bytes());1037 entries.extend_from_slice(&id.to_le_bytes());
1007 entries.extend_from_slice(1038 entries.extend_from_slice(
1008 &u32::try_from(count)1039 &u32::try_from(count)
...@@ -1088,6 +1119,6 @@ pub(crate) fn write_revision(...@@ -1088,6 +1119,6 @@ pub(crate) fn write_revision(
1088 output[228..236].copy_from_slice(&generation.to_le_bytes());1119 output[228..236].copy_from_slice(&generation.to_le_bytes());
1089 let written = Store::parse(&output)?;1120 let written = Store::parse(&output)?;
1090 let written_index = RevisionIndex::parse(&written)?;1121 let written_index = RevisionIndex::parse(&written)?;
1091 written_index.resolve(space, new_rid)?.reachable()?;1122 written_index.validate_current()?;
1092 Ok(output)1123 Ok(output)
1093}1124}
crates/onestore/src/write/tests.rs+68
...@@ -1,6 +1,74 @@...@@ -1,6 +1,74 @@
1use super::patch_properties;1use super::patch_properties;
2use crate::{PropertySets, Value, create::properties};2use crate::{PropertySets, Value, create::properties};
33
4mod multi;
5
6#[test]
7fn copying_reference_arrays_remaps_every_entry_in_each_stream() {
8 use super::PropertyObject;
9 use std::{collections::BTreeMap, sync::Arc};
10 for kind in [9_u32, 11, 13] {
11 for count in [0_u32, 1, 2, 3, 4, 8, 33] {
12 for nested in [false, true] {
13 let property = (kind << 26) | 1;
14 let references: Vec<_> = (0..count)
15 .flat_map(|i| (((i % 3) << 8) | i).to_le_bytes())
16 .collect();
17 let mut bytes = properties(&[(property, references)]).unwrap();
18 let selected = if nested {
19 let parsed = PropertySets::parse(&bytes).unwrap();
20 let offset = parsed.root_ids.as_ptr().addr() - bytes.as_ptr().addr() - 2;
21 bytes.truncate(bytes.len() - parsed.padding.len());
22 bytes.splice(offset..offset, [1, 0, 2, 0, 0, 0x44]);
23 bytes.resize(bytes.len().next_multiple_of(8), 0);
24 0x44000002
25 } else {
26 property
27 };
28 let source = PropertyObject {
29 jcid: 0x6000e,
30 bytes,
31 global_ids: Arc::new(BTreeMap::from([
32 (0, [1; 16]),
33 (1, [2; 16]),
34 (2, [3; 16]),
35 ])),
36 };
37 let mut target = PropertyObject {
38 jcid: 0x6000e,
39 bytes: properties(&[(0x14000003, 47_u32.to_le_bytes().to_vec())]).unwrap(),
40 global_ids: Arc::new(BTreeMap::from([
41 (0, [3; 16]),
42 (1, [9; 16]),
43 (2, [1; 16]),
44 ])),
45 };
46 target.copy_property(&source, selected).unwrap();
47 let parsed = PropertySets::parse(&target.bytes).unwrap();
48 assert_eq!(parsed.sets[0][0].value, Value::Bytes(&47_u32.to_le_bytes()));
49 let copied = parsed
50 .sets
51 .iter()
52 .flatten()
53 .find(|p| p.id == property)
54 .unwrap();
55 let Value::References { compact_ids, .. } = copied.value else {
56 panic!()
57 };
58 assert_eq!(compact_ids.len(), count as usize * 4);
59 for (i, compact) in compact_ids.chunks_exact(4).enumerate() {
60 let compact = u32::from_le_bytes(compact.try_into().unwrap());
61 assert_eq!(compact & 255, i as u32);
62 assert_eq!(target.global_ids[&(compact >> 8)], [(i % 3 + 1) as u8; 16]);
63 }
64 let before = target.bytes.clone();
65 target.copy_property(&source, selected).unwrap();
66 assert_eq!(target.bytes, before);
67 }
68 }
69 }
70}
71
4#[test]72#[test]
5fn document_insertions_and_formatting_respect_readonly_ancestors() {73fn document_insertions_and_formatting_respect_readonly_ancestors() {
6 use super::{PropertyObject, write_revision};74 use super::{PropertyObject, write_revision};
crates/onestore/src/write/tests/multi.rs created+302
...@@ -0,0 +1,302 @@
1use crate::{
2 ExGuid, RevisionIndex, Store,
3 document::{Document, Kind},
4 write::{PropertyObject, write_revisions},
5};
6use std::collections::BTreeMap;
7
8mod disk {
9 use crate as onestore;
10 include!("../../../tests/support/disk.rs");
11}
12
13mod current {
14 use crate as onestore;
15 include!("../../../tests/support/current.rs");
16}
17
18const SOURCE: &[u8] =
19 include_bytes!("../../../../../corpus/page-lifecycle/03-renamed/notebook/Lifecycle.one");
20
21fn nest(source: &[u8]) -> Vec<u8> {
22 write_revisions(source, |index| {
23 let document = Document::parse(index)?;
24 let pages = document.pages()?;
25 assert_eq!(pages.len(), 9);
26 let section = index.resolve(
27 index.root,
28 index.spaces[&index.root].labels[&(ExGuid::default(), 1)],
29 )?;
30 let view = &document.spaces[&index.root];
31 let view = &view.revisions[&view.contexts[&ExGuid::default()]];
32 let root = section.roots[&1];
33 let parent = *view.nodes[&root]
34 .children
35 .iter()
36 .find(|id| view.nodes[id].spaces.contains(&pages[3].0))
37 .unwrap();
38 let mut series = PropertyObject::from_object(&section.objects[&parent])?;
39 let mut spaces = Vec::new();
40 let mut metadata = Vec::new();
41 let mut replacements = BTreeMap::new();
42 let mut changes = BTreeMap::new();
43 for (ordinal, (sid, page)) in pages[3..6].iter().enumerate() {
44 spaces.extend_from_slice(&series.reference(*sid)?);
45 let raw = index.resolve(*sid, index.spaces[sid].labels[&(ExGuid::default(), 1)])?;
46 let id = raw.roots[&2];
47 let mut object = PropertyObject::from_object(&raw.objects[&id])?;
48 object.set(&[(0x14001dff, &((ordinal + 1) as u32).to_le_bytes())])?;
49 changes.insert(*sid, BTreeMap::from([(id, object)]));
50 let mut copy = PropertyObject::from_object(&raw.objects[&id])?;
51 copy.set(&[(0x14001dff, &((ordinal + 1) as u32).to_le_bytes())])?;
52 let mut copy_id = ExGuid {
53 guid: [
54 0x31, 0xc0, 0xa8, 0x22, 0, 0x36, 0xee, 0x42, 0xb7, 0x14, 0xd7, 0xac, 0xda,
55 0x24, 0x35, 0xe8,
56 ],
57 n: 1,
58 };
59 for (value, salt) in copy_id.guid.iter_mut().zip(page.guid) {
60 *value ^= salt;
61 }
62 copy.reference(copy_id)?;
63 metadata.extend_from_slice(&series.reference(copy_id)?);
64 replacements.insert(copy_id, copy);
65 }
66 let links = PropertyObject {
67 jcid: series.jcid,
68 bytes: crate::create::properties(&[(0x2c001d63, spaces)])?,
69 global_ids: std::sync::Arc::clone(&series.global_ids),
70 };
71 series
72 .copy_property(&links, 0x2c001d63)
73 .expect("copy spaces");
74 series
75 .set(&[(0x24003442, &metadata)])
76 .expect("set metadata links");
77 replacements.insert(parent, series);
78 let mut object = PropertyObject::from_object(&section.objects[&root])?;
79 let mut children = Vec::new();
80 for id in &view.nodes[&root].children {
81 if *id == parent
82 || !view.nodes[id]
83 .spaces
84 .iter()
85 .any(|sid| [pages[4].0, pages[5].0].contains(sid))
86 {
87 children.extend_from_slice(&object.reference(*id)?);
88 }
89 }
90 object.set(&[(0x24001c20, &children)])?;
91 replacements.insert(root, object);
92 changes.insert(index.root, replacements);
93 Ok(changes)
94 })
95 .unwrap()
96}
97
98#[test]
99fn nesting_publishes_section_order_and_page_levels_in_one_transaction() {
100 let written = nest(SOURCE);
101 let before = Store::parse(SOURCE).unwrap();
102 let after = Store::parse(&written).unwrap();
103 assert_eq!(
104 after.header.transaction_count,
105 before.header.transaction_count + 1
106 );
107 assert_eq!(after.header.generation, before.header.generation + 1);
108 let old = RevisionIndex::parse(&before).unwrap();
109 let new = RevisionIndex::parse(&after).unwrap();
110 let document = Document::parse(&new).unwrap();
111 let pages = document.pages().unwrap();
112 assert_eq!(pages, Document::parse(&old).unwrap().pages().unwrap());
113 let mut changed_spaces = 0;
114 for (sid, space) in &old.spaces {
115 changed_spaces += usize::from(space.labels != new.spaces[sid].labels);
116 for rid in space.revisions.keys() {
117 assert_eq!(
118 format!("{:?}", old.resolve(*sid, *rid).unwrap()),
119 format!("{:?}", new.resolve(*sid, *rid).unwrap())
120 );
121 }
122 }
123 assert_eq!(changed_spaces, 3);
124 for (ordinal, (sid, _)) in pages.iter().enumerate() {
125 let view = &document.spaces[sid];
126 let view = &view.revisions[&view.contexts[&ExGuid::default()]];
127 let Kind::Metadata { level, .. } = view.nodes[&view.roots[&2]].kind else {
128 panic!()
129 };
130 assert_eq!(
131 level,
132 Some(match ordinal {
133 4 => 2,
134 5 => 3,
135 _ => 1,
136 })
137 );
138 let old_view = old
139 .resolve(*sid, old.spaces[sid].labels[&(ExGuid::default(), 1)])
140 .unwrap();
141 let new_view = new
142 .resolve(*sid, new.spaces[sid].labels[&(ExGuid::default(), 1)])
143 .unwrap();
144 for (id, object) in &old_view.objects {
145 if *id != old_view.roots[&2] {
146 assert_eq!(format!("{object:?}"), format!("{:?}", new_view.objects[id]));
147 }
148 }
149 }
150 let root = &document.spaces[&document.root];
151 let root = &root.revisions[&root.contexts[&ExGuid::default()]];
152 assert_eq!(
153 root.nodes[&root.roots[&1]]
154 .children
155 .iter()
156 .map(|id| root.nodes[id].spaces.len())
157 .collect::<Vec<_>>(),
158 [1, 1, 1, 3, 1, 1, 1]
159 );
160 assert_eq!(nest(&written), written);
161 if let Some(output) = std::env::var_os("ONESTORE_PAGE_BATCH_OUTPUT") {
162 std::fs::create_dir(&output).unwrap();
163 std::fs::write(
164 std::path::Path::new(&output).join("Lifecycle.one"),
165 &written,
166 )
167 .unwrap();
168 }
169}
170
171#[test]
172fn interrupted_multi_space_publication_never_exposes_a_partial_nesting() {
173 let written = nest(SOURCE);
174 let old = current::current(SOURCE);
175 let new = current::current(&written);
176 assert_ne!(old, new);
177 for write_limit in [17, 4096] {
178 let mut complete = disk::Disk {
179 visible: SOURCE.to_vec(),
180 durable: SOURCE.to_vec(),
181 operation: 0,
182 fail_at: None,
183 write_limit,
184 random: 1951,
185 };
186 crate::commit::commit_bytes(&mut complete, SOURCE, &written).unwrap();
187 assert_eq!(complete.durable, written);
188 for fail_at in 1..=complete.operation {
189 let mut interrupted = disk::Disk {
190 visible: SOURCE.to_vec(),
191 durable: SOURCE.to_vec(),
192 operation: 0,
193 fail_at: Some(fail_at),
194 write_limit,
195 random: 1951 + fail_at as u64,
196 };
197 crate::commit::commit_bytes(&mut interrupted, SOURCE, &written).unwrap_err();
198 let observed = current::current(&interrupted.durable);
199 assert!(
200 observed == old || observed == new,
201 "{write_limit}:{fail_at}"
202 );
203 }
204 }
205}
206
207#[test]
208fn empty_batches_and_invalid_spaces_do_not_produce_an_edit() {
209 assert_eq!(
210 write_revisions(SOURCE, |_| Ok(BTreeMap::new())).unwrap(),
211 SOURCE
212 );
213 assert!(
214 write_revisions(SOURCE, |_| Ok(BTreeMap::from([(
215 ExGuid::default(),
216 BTreeMap::new()
217 )])))
218 .is_err()
219 );
220}
221
222#[test]
223fn repeated_multi_space_edits_cross_counter_carries_and_checkpoint_each_space() {
224 let mut source = crate::create_section("batch.one", "Preserved text", "Author").unwrap();
225 let initial = source.clone();
226 let initial_store = Store::parse(&initial).unwrap();
227 let initial_index = RevisionIndex::parse(&initial_store).unwrap();
228 let mut checkpoints = BTreeMap::new();
229 for step in 1_u32..=514 {
230 let written = write_revisions(&source, |index| {
231 let mut changes = BTreeMap::new();
232 for (sid, space) in &index.spaces {
233 let raw = index.resolve(*sid, space.labels[&(ExGuid::default(), 1)])?;
234 let (id, property) = if *sid == index.root {
235 (raw.roots[&2], 0x14001cbe)
236 } else {
237 (
238 *raw.objects
239 .iter()
240 .find(|(_, object)| object.jcid == 0x6000b)
241 .unwrap()
242 .0,
243 0x14001d7a,
244 )
245 };
246 let mut object = PropertyObject::from_object(&raw.objects[&id])?;
247 object.set(&[(property, &step.to_le_bytes())])?;
248 changes.insert(*sid, BTreeMap::from([(id, object)]));
249 }
250 Ok(changes)
251 })
252 .unwrap();
253 let store = Store::parse(&written).unwrap();
254 assert_eq!(
255 store.header.transaction_count,
256 initial_store.header.transaction_count + step
257 );
258 assert!(store.checksum_mismatches.is_empty());
259 let index = RevisionIndex::parse(&store).unwrap();
260 for (sid, space) in &index.spaces {
261 let rid = space.labels[&(ExGuid::default(), 1)];
262 if space.revisions[&rid].dependency.is_none() {
263 *checkpoints.entry(*sid).or_insert(0) += 1;
264 }
265 for old_rid in initial_index.spaces[sid].revisions.keys() {
266 assert_eq!(
267 format!("{:?}", initial_index.resolve(*sid, *old_rid).unwrap()),
268 format!("{:?}", index.resolve(*sid, *old_rid).unwrap())
269 );
270 }
271 }
272 if step == 1 || store.header.transaction_count == 256 || step == 512 {
273 let old = current::current(&source);
274 let new = current::current(&written);
275 let mut complete = disk::Disk {
276 visible: source.clone(),
277 durable: source.clone(),
278 operation: 0,
279 fail_at: None,
280 write_limit: 4096,
281 random: 1952,
282 };
283 crate::commit::commit_bytes(&mut complete, &source, &written).unwrap();
284 for fail_at in 1..=complete.operation {
285 let mut interrupted = disk::Disk {
286 visible: source.clone(),
287 durable: source.clone(),
288 operation: 0,
289 fail_at: Some(fail_at),
290 write_limit: 4096,
291 random: 1952 + fail_at as u64,
292 };
293 crate::commit::commit_bytes(&mut interrupted, &source, &written).unwrap_err();
294 let observed = current::current(&interrupted.durable);
295 assert!(observed == old || observed == new, "{step}:{fail_at}");
296 }
297 }
298 source = written;
299 }
300 assert_eq!(checkpoints.len(), 2);
301 assert!(checkpoints.values().all(|count| *count == 1));
302}
tools/test_page_lifecycle.py+41
...@@ -56,6 +56,47 @@ class PageLifecycleTest(unittest.TestCase):...@@ -56,6 +56,47 @@ class PageLifecycleTest(unittest.TestCase):
56 for oid in section['nodes'][section['roots']['1']]['children']])56 for oid in section['nodes'][section['roots']['1']]['children']])
57 self.assertEqual(groups[0], groups[1])57 self.assertEqual(groups[0], groups[1])
5858
59 def test_atomic_rust_nesting_matches_native_and_survives_cold_reopen(self):
60 fixture = FIXTURE / 'atomic-nesting'
61 self.assertEqual((fixture / 'candidate/Lifecycle.one').read_bytes()[1024:],
62 (fixture / 'cold/notebook/Lifecycle.one').read_bytes()[1024:])
63 with TemporaryDirectory() as temporary:
64 native_read = Path(temporary) / 'native/read'
65 cold_read = Path(temporary) / 'cold/read'
66 shutil.copytree(FIXTURE / '04-nested/read', native_read)
67 shutil.copytree(fixture / 'cold/read', cold_read)
68 models = []
69 for ordinal, source in enumerate((fixture / 'candidate', fixture / 'cold/notebook')):
70 output = Path(temporary) / str(ordinal)
71 subprocess.run([EXPORTER, source / 'Lifecycle.one', output], check=True)
72 models.append(json.loads((output / 'document.json').read_text()))
73 compare(source, native_read)
74 compare(source, cold_read)
75 pages = [list(ordered_pages(model)) for model in models]
76 expected = list(ordered_pages(self.models['04-nested', '']))
77 original = list(ordered_pages(self.models['03-renamed', '']))
78 self.assertEqual(len(pages[0]), 9)
79 for before, native, candidate, cold in zip(original, expected, *pages, strict=True):
80 sid, _, revision, page = candidate
81 self.assertEqual((sid, page), (before[0], before[3]))
82 self.assertEqual((sid, page), (cold[0], cold[3]))
83 self.assertEqual(dict(walk(revision, page)), dict(walk(before[2], before[3])))
84 self.assertEqual(revision['nodes'], cold[2]['nodes'])
85 self.assertEqual(revision['roots'], cold[2]['roots'])
86 metadata = revision['nodes'][revision['roots']['2']]['kind']
87 self.assertEqual(metadata, native[2]['nodes'][native[2]['roots']['2']]['kind'])
88 groups = []
89 for model in (self.models['04-nested', ''], *models):
90 _, revision = view(model, model['root'])
91 groups.append([revision['nodes'][oid]['spaces']
92 for oid in revision['nodes'][revision['roots']['1']]['children']])
93 self.assertEqual(groups[0], groups[1])
94 self.assertEqual(groups[1], groups[2])
95 _, candidate = view(models[0], models[0]['root'])
96 _, cold = view(models[1], models[1]['root'])
97 self.assertEqual(candidate['nodes'], cold['nodes'])
98 self.assertEqual(candidate['roots'], cold['roots'])
99
59 def test_native_page_styles_titles_order_and_parent_deletion(self):100 def test_native_page_styles_titles_order_and_parent_deletion(self):
60 phases = {phase: list(ordered_pages(self.models[phase, ''])) for phase in PHASES}101 phases = {phase: list(ordered_pages(self.models[phase, ''])) for phase in PHASES}
61 initial = [sid for sid, _, _, _ in phases['02-authored']]102 initial = [sid for sid, _, _, _ in phases['02-authored']]