Commit 8bcca78a05a23695cd0b16382164810da6d0c96e | snowbound

author	clover caruso <git@paperclover.net> 2026-09-08 14:28:35-07:00
committer	clover caruso <git@paperclover.net> 2026-09-08 14:28:35-07:00
log	8bcca78a05a23695cd0b16382164810da6d0c96e
tree	ced4fcdfe203aa93f0e6fe55540cd647d4d274e9
parent	fa4842b0710f130a915f5ae0cea755b17503825f
signature	Signed by SSH key SHA256:52mNGHRsVFBDED9IAX5pe+LRWUefqTbxEReunq21QvU

fix: reconcile native subtree normalization in offline deletion

Compare immutable referenced records by content and normalize absent versus empty child lists without weakening mutable identity or unknown-property checks. Preserve version-eight deletion observations until explicit conflict review.

Extend the shared native/Rust workload to ten document phases with independent graph and receipt verification. Retain the native regression and migration controls; offline tests, mixed twelve-client outage run, cold OneNote comparison, ASan schedules, Clippy and iOS checks pass.

Assisted-by: gpt-6-astra

13 files changed, 530 insertions(+), 56 deletions(-)

corpus
- outline-edit
  - README.md
  - empty-children
    before.one
    intent.json
    remote.one
crates
- onestore-offline
  - README.md
  - examples
    smb_offline_client.rs
  - src
    lib.rs
    tree.rs
  - tests
    cache.rs
    support
    tree_schedule.rs
    sync
    tree.rs
tools
- offline_document_history.py
- test_offline_document_history.py

corpus/outline-edit/README.md+7

...	...	@@ -1,5 +1,12 @@
1	1	# Native outline editing controls
2	2
	3	`empty-children` retains a mixed-client deletion regression: `before.one` exposes
	4	transaction 22 of the retained local image; `remote.one` was observed after native
	5	OneNote removed an empty child-list property and coalesced equivalent immutable
	6	styles. The retained intent addresses the same subtree in both files. The offline
	7	test requires deletion and its dependent edit to publish after reopening the
	8	cache, while an actual remote text change retains a content conflict.
	9
3	10	OneNote 2010 authored fourteen cases plus an unchanged source page. `before`,
4	11	`after`, and `cold` retain notebook bytes and independent native XML. The final
5	12	phase uses a fresh application cache. Cases preserve Unicode, hyperlinks, tags,

corpus/outline-edit/empty-children/before.one created

Binary files /dev/null and b/corpus/outline-edit/empty-children/before.one differ

corpus/outline-edit/empty-children/intent.json created+27

...	...	@@ -0,0 +1,27 @@
	1	[
	2	"{C9F8B62B-D016-44A6-BD6D-AD427C226A53},1",
	3	{
	4	"guid": [
	5	247,
	6	43,
	7	64,
	8	79,
	9	30,
	10	47,
	11	235,
	12	73,
	13	132,
	14	188,
	15	117,
	16	194,
	17	60,
	18	28,
	19	252,
	20	6
	21	],
	22	"object": "{91881B6E-F0D0-424E-917E-9D3BEF7D35C7},1",
	23	"placement": "Delete",
	24	"author": "Offline document writer",
	25	"created": 1473370703
	26	}
	27	]

corpus/outline-edit/empty-children/remote.one created

Binary files /dev/null and b/corpus/outline-edit/empty-children/remote.one differ

crates/onestore-offline/README.md+7-4

...	...	@@ -107,16 +107,19 @@ already satisfied move still requires guarded confirmation before acknowledgemen
107	107	Deletion compares the selected raw property graph, including referenced styles,
108	108	tags, unknown fields and internal attachments. Changed content produces
109	109	`ContentChanged`; property order, CompactID numbering and modification timestamps
110		do not affect that comparison. Missing targets retain `TargetUnavailable`.
	110	do not affect that comparison. Immutable records compare by content, and empty
	111	child lists compare equally to absent child lists. Mutable object identities
	112	remain significant. Missing targets retain `TargetUnavailable`.
111	113	`rebase_tree_conflict(id, local, remote)` reviews the original move/deletion against
112	114	both current images. An emptied cell's replacement paragraph/text identities must
113	115	remain the same before replay or review; later queued edits keep their targets.
114	116	Uncertain tree attempts require their original revision for confirmation, even
115	117	when an independent move or deletion has the same visible effect.
116	118
117		Recognized earlier caches migrate transactionally to version eight, which adds
118		tree intents and content conflicts. The migration retains
119		images, local IDs, publication attempts, conflicts,
	119	Recognized earlier caches migrate transactionally to version nine. Version-eight
	120	deletion observations retain their original identity-sensitive preconditions;
	121	explicit conflict review upgrades them to the immutable-content comparison.
	122	The migration retains images, local IDs, publication attempts, conflicts,
120	123	receipts and the autoincrement sequence; it does not reuse acknowledged IDs when
121	124	the pending queue is empty.
122	125

crates/onestore-offline/examples/smb_offline_client.rs+69-6

...	...	@@ -3,7 +3,7 @@ mod concurrent;
3	3
4	4	use onestore::{
5	5	CommitError, CommitState, ExGuid, Insertion, ParagraphJoin, ParagraphSplit, PreparedEdit,
6		RevisionIndex, Store, TextAttribute, document::Document,
	6	RevisionIndex, Store, TextAttribute, TreeEdit, document::Document,
7	7	};
8	8	use onestore_offline::{EditStatus, Error, Remote, Replica, SmbRemote};
9	9	use onestore_smb::{Client, Credentials};
...	...	@@ -41,7 +41,18 @@ impl DocumentView {
41	41	}
42	42	}
43	43
44		const DOCUMENT_OPERATIONS: [&str; 6] = ["insert", "format", "text", "split", "right_text", "join"];
	44	const DOCUMENT_OPERATIONS: [&str; 10] = [
	45	"insert",
	46	"format",
	47	"text",
	48	"split",
	49	"right_text",
	50	"nest",
	51	"unnest",
	52	"join",
	53	"tail_split",
	54	"delete",
	55	];
45	56
46	57	fn now() -> u128 {
47	58	SystemTime::now()
...	...	@@ -331,6 +342,8 @@ fn queue_document(
331	342	};
332	343	let end = u32::try_from(text.encode_utf16().count())?;
333	344	let split = ParagraphSplit::new(insertion.text_object(), end - 2, "Offline document writer")?;
	345	let tail_split =
	346	ParagraphSplit::new(insertion.text_object(), end + 1, "Offline document writer")?;
334	347	let join = ParagraphJoin::new(
335	348	insertion.text_object(),
336	349	split.text_object(),
...	...	@@ -344,14 +357,50 @@ fn queue_document(
344	357	let mut ids = [0; DOCUMENT_OPERATIONS.len()];
345	358	for (step, id) in ids.iter_mut().enumerate() {
346	359	let kind = DOCUMENT_OPERATIONS[step];
	360	let tree = match kind {
	361	"nest" => {
	362	let source = cache.snapshot()?;
	363	let store = Store::parse(&source)?;
	364	let index = RevisionIndex::parse(&store)?;
	365	let document = Document::parse(&index)?;
	366	let section = &document.spaces[&space];
	367	let view = &section.revisions[&section.contexts[&ExGuid::default()]];
	368	let paragraph = view
	369	.nodes
	370	.iter()
	371	.find(\|(_, node)\| node.content == [insertion.text_object()])
	372	.map(\|(id, _)\| *id)
	373	.ok_or("Missing inserted paragraph")?;
	374	Some(TreeEdit::move_to(
	375	split.object(),
	376	paragraph,
	377	None,
	378	"Offline document writer",
	379	)?)
	380	}
	381	"unnest" => Some(TreeEdit::move_to(
	382	split.object(),
	383	parent,
	384	None,
	385	"Offline document writer",
	386	)?),
	387	"delete" => Some(TreeEdit::delete(
	388	tail_split.object(),
	389	"Offline document writer",
	390	)?),
	391	_ => None,
	392	};
347	393	let range = match kind {
348	394	"text" => end - 3..end,
349	395	"split" => end - 2..end - 2,
	396	"tail_split" => end + 1..end + 1,
350	397	"right_text" => 0..2,
351	398	_ => 1..end - 2,
352	399	};
353	400	let target = if kind == "right_text" {
354	401	split.text_object()
	402	} else if kind == "delete" {
	403	tail_split.text_object()
355	404	} else {
356	405	insertion.text_object()
357	406	};
...	...	@@ -373,7 +422,9 @@ fn queue_document(
373	422	cache.edit_text(&source, space, target, range.clone(), replacement.unwrap())
374	423	}
375	424	"split" => cache.split(&source, space, &split),
	425	"tail_split" => cache.split(&source, space, &tail_split),
376	426	"join" => cache.join(&source, space, &join),
	427	"nest" \| "unnest" \| "delete" => cache.tree(&source, space, tree.as_ref().unwrap()),
377	428	_ => unreachable!(),
378	429	};
379	430	match result {
...	...	@@ -384,7 +435,8 @@ fn queue_document(
384	435	json!({"event":"local_document_commit","id":acknowledged,"operation":operation,"kind":kind,
385	436	"space":space.to_string(),"object":target.to_string(),"document":insertion.text_object().to_string(),
386	437	"text":text,"insertion":if kind=="insert" {Some(&insertion)} else {None},
387		"split":if kind=="split" {Some(&split)} else {None},
	438	"split":match kind { "split" => Some(&split), "tail_split" => Some(&tail_split), _ => None },
	439	"tree":tree,
388	440	"joined":if kind=="join" {Some(join.texts().map(\|id\| id.to_string()))} else {None},
389	441	"range":[range.start,range.end],"attributes":attributes,"replacement":replacement,
390	442	"started_us":started,"finished_us":now()})
...	...	@@ -555,7 +607,12 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
555	607	let remote = cache.remote_snapshot()?;
556	608	let current = view(&remote)?;
557	609	let onestore_offline::Operation::Text(edit) = &intent.operation else {
558		return Err("Expected text probe intents".into());
	610	return Err(format!(
	611	"Document intent {} requires review: {:?}",
	612	intent.id,
	613	cache.status(intent.id)?
	614	)
	615	.into());
559	616	};
560	617	let at = append_position(&edit.before, &current.text, &edit.replacement)
561	618	.ok_or("Append model disagrees with retained history")?;
...	...	@@ -713,9 +770,15 @@ mod tests {
713	770	cache = Replica::open(&path).unwrap();
714	771	}
715	772	let local = cache.snapshot().unwrap();
716		assert_eq!(cache.pending().unwrap().len(), 12);
	773	assert_eq!(
	774	cache.pending().unwrap().len(),
	775	2 * DOCUMENT_OPERATIONS.len()
	776	);
717	777	for (step, id) in ids.iter().enumerate() {
718		remote.remote.lost_reply = matches!(step % 6, 3 \| 5);
	778	remote.remote.lost_reply = matches!(
	779	DOCUMENT_OPERATIONS[step % DOCUMENT_OPERATIONS.len()],
	780	"split" \| "nest" \| "unnest" \| "join" \| "tail_split" \| "delete"
	781	);
719	782	let result = cache.sync_once(&mut remote);
720	783	let state = if result.is_err() {
721	784	assert!(matches!(

crates/onestore-offline/src/lib.rs+1-1

...	...	@@ -51,7 +51,7 @@ pub enum Error {
51	51	type Result<T> = std::result::Result<T, Error>;
52	52
53	53	const APPLICATION_ID: u32 = 0x4f4e454f;
54		const SCHEMA_VERSION: u32 = 8;
	54	const SCHEMA_VERSION: u32 = 9;
55	55
56	56	/// Text and its observed precondition, retained across cache reopen and rebasing.
57	57	#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]

crates/onestore-offline/src/tree.rs+199-21

...	...	@@ -4,13 +4,20 @@ use serde::{Deserialize, Serialize};
4	4	use sha2::{Digest, Sha256};
5	5	use std::collections::{BTreeMap, BTreeSet};
6	6
	7	#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
	8	#[serde(untagged, deny_unknown_fields)]
	9	enum Content {
	10	Legacy([u8; 32]),
	11	Semantic { sha256: [u8; 32] },
	12	}
	13
7	14	#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
8	15	#[serde(deny_unknown_fields)]
9	16	struct Observed {
10	17	path: Vec<(ExGuid, u8)>,
11	18	siblings: BTreeMap<ExGuid, bool>,
12	19	destination: Vec<(ExGuid, u8)>,
13		content: Option<[u8; 32]>,
	20	content: Option<Content>,
14	21	}
15	22
16	23	/// A subtree intent with observed placement, deletion content and replacement identities.
...	...	@@ -22,23 +29,41 @@ pub struct TreeEdit {
22	29	created: BTreeSet<ExGuid>,
23	30	}
24	31
25		fn fingerprint(store: &Store<'_>, raw: &ResolvedRevision<'_>, root: ExGuid) -> Result<[u8; 32]> {
	32	fn fingerprint(
	33	store: &Store<'_>,
	34	raw: &ResolvedRevision<'_>,
	35	root: ExGuid,
	36	legacy: bool,
	37	) -> Result<[u8; 32]> {
26	38	let mut objects = BTreeMap::new();
27		let mut pending = vec![root];
28		while let Some(id) = pending.pop() {
	39	let mut visiting = BTreeSet::new();
	40	let mut pending = vec![(root, false)];
	41	while let Some((id, complete)) = pending.pop() {
29	42	if objects.contains_key(&id) {
30	43	continue;
31	44	}
32	45	let object = &raw.objects[&id];
33	46	let references = object.references()?;
34		pending.extend(&references.objects);
	47	if !complete {
	48	if !visiting.insert(id) {
	49	return Err(io::Error::new(
	50	io::ErrorKind::InvalidData,
	51	"Subtree property references form a cycle",
	52	)
	53	.into());
	54	}
	55	pending.push((id, true));
	56	pending.extend(references.objects.iter().map(\|id\| (*id, false)));
	57	continue;
	58	}
	59	visiting.remove(&id);
35	60	let mut hash = Sha256::new();
36	61	hash.update(object.jcid.to_le_bytes());
37	62	match object.data {
38	63	ObjectData::Properties(bytes) => {
39	64	hash.update([0]);
40	65	let properties = PropertySets::parse(bytes)?;
41		let mut objects = references.objects.into_iter();
	66	let mut object_ids = references.objects.into_iter();
42	67	let mut spaces = references.object_spaces.into_iter();
43	68	let mut contexts = references.contexts.into_iter();
44	69	let mut fields = Vec::new();
...	...	@@ -54,20 +79,36 @@ fn fingerprint(store: &Store<'_>, raw: &ResolvedRevision<'_>, root: ExGuid) -> R
54	79	compact_ids,
55	80	} => {
56	81	let references = match stream {
57		IdStream::Objects => &mut objects,
	82	IdStream::Objects => &mut object_ids,
58	83	IdStream::ObjectSpaces => &mut spaces,
59	84	IdStream::Contexts => &mut contexts,
60	85	};
61	86	for reference in references.take(compact_ids.len() / 4) {
62		value.update(reference.guid);
63		value.update(reference.n.to_le_bytes());
	87	if !legacy && *stream == IdStream::Objects {
	88	let immutable =
	89	raw.objects[&reference].jcid & 0x100000 != 0;
	90	value.update([u8::from(immutable)]);
	91	if !immutable {
	92	value.update(reference.guid);
	93	value.update(reference.n.to_le_bytes());
	94	}
	95	value.update(objects[&reference]);
	96	} else {
	97	value.update(reference.guid);
	98	value.update(reference.n.to_le_bytes());
	99	}
64	100	}
65	101	}
66	102	Value::Sets(_) => {}
67	103	}
68		if property.id != 0x14001d7a {
69		values.insert(property.id, value);
	104	if property.id == 0x14001d7a
	105	\|\| (!legacy
	106	&& property.id == 0x24001c20
	107	&& matches!(&property.value, Value::References { compact_ids, .. } if compact_ids.is_empty()))
	108	{
	109	continue;
70	110	}
	111	values.insert(property.id, value);
71	112	}
72	113	fields.push(values);
73	114	}
...	...	@@ -106,6 +147,9 @@ fn fingerprint(store: &Store<'_>, raw: &ResolvedRevision<'_>, root: ExGuid) -> R
106	147	}
107	148	objects.insert(id, <[u8; 32]>::from(hash.finalize()));
108	149	}
	150	if !legacy {
	151	return Ok(objects[&root]);
	152	}
109	153	let mut hash = Sha256::new();
110	154	for (id, value) in objects {
111	155	hash.update(id.guid);
...	...	@@ -115,7 +159,12 @@ fn fingerprint(store: &Store<'_>, raw: &ResolvedRevision<'_>, root: ExGuid) -> R
115	159	Ok(hash.finalize().into())
116	160	}
117	161
118		fn observe(source: &[u8], space: ExGuid, intent: &onestore::TreeEdit) -> Result<Option<Observed>> {
	162	fn observe(
	163	source: &[u8],
	164	space: ExGuid,
	165	intent: &onestore::TreeEdit,
	166	legacy: bool,
	167	) -> Result<Option<Observed>> {
119	168	let store = Store::parse(source)?;
120	169	let index = RevisionIndex::parse(&store)?;
121	170	let document = Document::parse(&index)?;
...	...	@@ -168,7 +217,12 @@ fn observe(source: &[u8], space: ExGuid, intent: &onestore::TreeEdit) -> Result<
168	217	.collect(),
169	218	destination,
170	219	content: if intent.destination().is_none() {
171		Some(fingerprint(&store, &index.resolve(space, *rid)?, object)?)
	220	let sha256 = fingerprint(&store, &index.resolve(space, *rid)?, object, legacy)?;
	221	Some(if legacy {
	222	Content::Legacy(sha256)
	223	} else {
	224	Content::Semantic { sha256 }
	225	})
172	226	} else {
173	227	None
174	228	},
...	...	@@ -206,7 +260,7 @@ impl TreeEdit {
206	260	intent: &onestore::TreeEdit,
207	261	) -> Result<(Self, PreparedEdit<'a>)> {
208	262	let prepared = PreparedEdit::tree(source, space, intent)?;
209		let observed = observe(source, space, intent)?.ok_or_else(\|\| {
	263	let observed = observe(source, space, intent, false)?.ok_or_else(\|\| {
210	264	io::Error::new(
211	265	io::ErrorKind::InvalidData,
212	266	"Prepared subtree edit has no active target",
...	...	@@ -229,7 +283,13 @@ impl TreeEdit {
229	283	source: &'a [u8],
230	284	space: ExGuid,
231	285	) -> Result<std::result::Result<PreparedEdit<'a>, ConflictKind>> {
232		let Some(current) = observe(source, space, &self.intent)? else {
	286	let Some(current) = observe(
	287	source,
	288	space,
	289	&self.intent,
	290	matches!(self.observed.content, Some(Content::Legacy(_))),
	291	)?
	292	else {
233	293	return Ok(Err(ConflictKind::TargetUnavailable));
234	294	};
235	295	let prepared = match PreparedEdit::tree(source, space, &self.intent) {
...	...	@@ -304,7 +364,7 @@ mod tests {
304	364	guid: [2; 16],
305	365	n: 7,
306	366	};
307		let hash = \|reverse: bool, global: u32, timestamp: u32, change: usize\| {
	367	let hash = \|reverse: bool, global: u32, timestamp: u32, change: usize, immutable: bool\| {
308	368	let mut fields = vec![
309	369	(0x14001d7a, timestamp.to_le_bytes().to_vec()),
310	370	(0x20000001, vec![]),
...	...	@@ -317,6 +377,12 @@ mod tests {
317	377	set(&[(if change == 2 { 0x08000005 } else { 0x88000005 }, vec![])]),
318	378	),
319	379	];
	380	if change != 5 {
	381	fields.push((0x24001c20, 0_u32.to_le_bytes().to_vec()));
	382	}
	383	if change == 6 {
	384	fields.push((0x24000007, 0_u32.to_le_bytes().to_vec()));
	385	}
320	386	if reverse {
321	387	fields.reverse();
322	388	}
...	...	@@ -351,7 +417,7 @@ mod tests {
351	417	(
352	418	target,
353	419	Object {
354		jcid: 0x6000e,
	420	jcid: if immutable { 0x12004d } else { 0x6000e },
355	421	reference_count: 1,
356	422	data: ObjectData::Properties(&child_bytes),
357	423	global_ids: ids,
...	...	@@ -359,12 +425,124 @@ mod tests {
359	425	),
360	426	]),
361	427	};
362		fingerprint(&store, &raw, root).unwrap()
	428	fingerprint(&store, &raw, root, false).unwrap()
363	429	};
364		let original = hash(false, 0, 1, 0);
365		assert_eq!(original, hash(true, 137, 2, 0));
	430	let original = hash(false, 0, 1, 0, false);
	431	assert_eq!(original, hash(true, 137, 2, 0, false));
	432	assert_eq!(original, hash(true, 137, 2, 5, false));
	433	assert_ne!(original, hash(true, 137, 2, 6, false));
366	434	for change in 1..=4 {
367		assert_ne!(original, hash(true, 137, 2, change));
	435	assert_ne!(original, hash(true, 137, 2, change, false));
	436	}
	437	let immutable = hash(false, 0, 1, 0, true);
	438	assert_eq!(immutable, hash(true, 137, 2, 4, true));
	439	for change in 1..=3 {
	440	assert_ne!(immutable, hash(true, 137, 2, change, true));
	441	}
	442	}
	443
	444	#[test]
	445	fn version_eight_deletion_observations_survive_migration_and_upgrade_only_after_review() {
	446	struct Server(Vec<u8>);
	447	impl Remote for Server {
	448	fn read(&mut self) -> io::Result<Vec<u8>> {
	449	Ok(self.0.clone())
	450	}
	451	fn publish(
	452	&mut self,
	453	edit: &PreparedEdit<'_>,
	454	) -> std::result::Result<(), onestore::CommitError> {
	455	self.0 = edit.as_bytes().to_vec();
	456	Ok(())
	457	}
	458	fn confirm(
	459	&mut self,
	460	snapshot: &[u8],
	461	) -> std::result::Result<(), onestore::CommitError> {
	462	assert!(self.0 == snapshot);
	463	Ok(())
	464	}
	465	}
	466	let source = onestore::create_section("legacy.one", "AlphaOmega", "Author").unwrap();
	467	let store = Store::parse(&source).unwrap();
	468	let index = RevisionIndex::parse(&store).unwrap();
	469	let document = Document::parse(&index).unwrap();
	470	let (sid, _) = document.pages().unwrap()[0];
	471	let space = &document.spaces[&sid];
	472	let view = &space.revisions[&space.contexts[&ExGuid::default()]];
	473	let text = *view.nodes.iter().find(\|(_, node)\| matches!(&node.kind, Kind::RichText { text, .. } if text == "AlphaOmega")).unwrap().0;
	474	let directory = tempfile::tempdir().unwrap();
	475	let path = directory.path().join("legacy.sqlite");
	476	let cache = Replica::create(&path, &source).unwrap();
	477	cache
	478	.format(
	479	&source,
	480	sid,
	481	text,
	482	0..10,
	483	&[onestore::TextAttribute::Bold(true)],
	484	)
	485	.unwrap();
	486	let split = onestore::ParagraphSplit::new(text, 5, "Author").unwrap();
	487	cache
	488	.split(&cache.snapshot().unwrap(), sid, &split)
	489	.unwrap();
	490	let before = cache.snapshot().unwrap();
	491	let intent = onestore::TreeEdit::delete(split.object(), "Author").unwrap();
	492	let deletion = cache.tree(&before, sid, &intent).unwrap().unwrap();
	493	let dependent = cache
	494	.edit_text(&cache.snapshot().unwrap(), sid, text, 0..0, "Local ")
	495	.unwrap()
	496	.unwrap();
	497	let mut queue = cache.pending().unwrap();
	498	let Operation::Tree(edit) = &mut queue[2].operation else {
	499	panic!()
	500	};
	501	edit.observed = observe(&before, sid, &intent, true).unwrap().unwrap();
	502	let serialized = serde_json::to_string(&queue[2].operation).unwrap();
	503	let local = cache.snapshot().unwrap();
	504	drop(cache);
	505	let db = Connection::open(&path).unwrap();
	506	db.execute(
	507	"UPDATE edits SET operation=?1 WHERE id=?2",
	508	params![serialized, i64::try_from(deletion).unwrap()],
	509	)
	510	.unwrap();
	511	db.pragma_update(None, "user_version", 8).unwrap();
	512	drop(db);
	513	let cache = Replica::open(&path).unwrap();
	514	assert_eq!(cache.pending().unwrap(), queue);
	515	assert!(cache.snapshot().unwrap() == local);
	516	let mut server = Server(source);
	517	for _ in 0..2 {
	518	assert!(matches!(
	519	cache.sync_once(&mut server).unwrap(),
	520	Some((_, EditStatus::Published { .. }))
	521	));
	522	}
	523	assert_eq!(
	524	cache.sync_once(&mut server).unwrap(),
	525	Some((deletion, EditStatus::Conflict(ConflictKind::ContentChanged)))
	526	);
	527	cache
	528	.rebase_tree_conflict(deletion, &local, &server.0)
	529	.unwrap();
	530	let queue = cache.pending().unwrap();
	531	let Operation::Tree(edit) = &queue[0].operation else {
	532	panic!()
	533	};
	534	assert!(matches!(
	535	edit.observed.content,
	536	Some(Content::Semantic { .. })
	537	));
	538	for id in [deletion, dependent] {
	539	assert!(
	540	matches!(cache.sync_once(&mut server).unwrap(), Some((actual, EditStatus::Published { .. })) if actual == id)
	541	);
368	542	}
	543	assert_eq!(
	544	paragraph(&server.0, sid, text).unwrap().as_deref(),
	545	Some("Local Alpha")
	546	);
369	547	}
370	548	}

crates/onestore-offline/tests/cache.rs+1-1

...	...	@@ -797,7 +797,7 @@ fn unrecognized_persisted_operations_are_rejected_without_dropping_fields() {
797	797
798	798	#[test]
799	799	fn prior_schema_migrations_retain_queue_evidence_assets_and_enable_content_conflicts() {
800		for (version, ceiling) in [(5, 3), (6, 4), (7, 5)] {
	800	for (version, ceiling) in [(5, 3), (6, 4), (7, 5), (8, 6)] {
801	801	use onestore_offline::{ConflictKind, EditStatus, Recovery};
802	802	use sha2::{Digest, Sha256};
803	803	let directory = tempfile::tempdir().unwrap();

crates/onestore-offline/tests/support/tree_schedule.rs+26-1

...	...	@@ -105,6 +105,20 @@ impl Remote for Session<'_> {
105	105	.unwrap();
106	106	text.replace_range(0..1, &edit.replacement);
107	107	}
	108	Operation::Format(format) => {
	109	let [onestore::TextAttribute::FontSize(size)] = format.attributes.as_slice() else {
	110	panic!()
	111	};
	112	let store = Store::parse(edit.as_bytes()).unwrap();
	113	let index = RevisionIndex::parse(&store).unwrap();
	114	let document = Document::parse(&index).unwrap();
	115	let space = &document.spaces[&SOURCE.1];
	116	let view = &space.revisions[&space.contexts[&ExGuid::default()]];
	117	assert_eq!(
	118	view.text_runs(format.object).unwrap()[0].format.font_size,
	119	Some(*size)
	120	);
	121	}
108	122	_ => panic!(),
109	123	}
110	124	assert_eq!(rows(edit.as_bytes()), expected);
...	...	@@ -163,7 +177,7 @@ pub fn run(input: &[u8]) {
163	177	&TreeEdit::delete(local[target].0, "Offline").unwrap(),
164	178	)
165	179	.unwrap(),
166		_ => cache
	180	_ if step[3] & 1 == 0 => cache
167	181	.edit_text(
168	182	&snapshot,
169	183	*sid,
...	...	@@ -172,6 +186,17 @@ pub fn run(input: &[u8]) {
172	186	&char::from(b'A' + step[3] % 26).to_string(),
173	187	)
174	188	.unwrap(),
	189	_ => cache
	190	.format(
	191	&snapshot,
	192	*sid,
	193	local[target].1,
	194	0..1,
	195	&[onestore::TextAttribute::FontSize(
	196	12.0 + f32::from(step[3] % 20),
	197	)],
	198	)
	199	.unwrap(),
175	200	};
176	201	let next = cache.pending().unwrap();
177	202	assert_eq!(next[..pending.len()], pending);

crates/onestore-offline/tests/sync/tree.rs+105

...	...	@@ -100,6 +100,111 @@ fn move_preserves_remote_content_and_dependent_edits_through_reopen() {
100	100	assert_eq!(server.publications, 2);
101	101	}
102	102
	103	#[test]
	104	fn queued_format_split_and_delete_reconcile_equivalent_immutable_style_identities() {
	105	let (source, sid, _, _, texts) = fixture();
	106	let directory = tempfile::tempdir().unwrap();
	107	let path = directory.path().join("cache.sqlite");
	108	let cache = Replica::create(&path, &source).unwrap();
	109	cache
	110	.format(
	111	&source,
	112	sid,
	113	texts[0],
	114	0..8,
	115	&[onestore::TextAttribute::Bold(true)],
	116	)
	117	.unwrap();
	118	let split = onestore::ParagraphSplit::new(texts[0], 4, "Author").unwrap();
	119	cache
	120	.split(&cache.snapshot().unwrap(), sid, &split)
	121	.unwrap();
	122	cache
	123	.tree(
	124	&cache.snapshot().unwrap(),
	125	sid,
	126	&TreeEdit::delete(split.object(), "Author").unwrap(),
	127	)
	128	.unwrap();
	129	cache
	130	.edit_text(&cache.snapshot().unwrap(), sid, texts[0], 0..0, "Local ")
	131	.unwrap();
	132	let queue = cache.pending().unwrap();
	133	drop(cache);
	134	let mut server = Server::new(&source);
	135	for intent in queue {
	136	let cache = Replica::open(&path).unwrap();
	137	assert!(
	138	matches!(cache.sync_once(&mut server).unwrap(), Some((id, EditStatus::Published { .. })) if id == intent.id)
	139	);
	140	}
	141	assert_eq!(
	142	super::outline::node(&server.durable, sid, texts[0])["kind"]["text"],
	143	"Local Orig"
	144	);
	145	}
	146
	147	#[test]
	148	fn native_empty_child_list_normalization_preserves_deletion_and_its_dependents() {
	149	let source = include_bytes!("../../../../corpus/outline-edit/empty-children/before.one");
	150	let remote = include_bytes!("../../../../corpus/outline-edit/empty-children/remote.one");
	151	let (sid, intent): (ExGuid, TreeEdit) = serde_json::from_str(include_str!(
	152	"../../../../corpus/outline-edit/empty-children/intent.json"
	153	))
	154	.unwrap();
	155	let store = Store::parse(source).unwrap();
	156	let index = RevisionIndex::parse(&store).unwrap();
	157	let document = Document::parse(&index).unwrap();
	158	let space = &document.spaces[&sid];
	159	let view = &space.revisions[&space.contexts[&ExGuid::default()]];
	160	let tail = view.nodes[&intent.object()].content[0];
	161	let (outline_id, outline) = view
	162	.nodes
	163	.iter()
	164	.find(\|(_, node)\| node.children.contains(&intent.object()))
	165	.unwrap();
	166	let left = view.nodes[&outline.children[0]].content[0];
	167	for changed in [false, true] {
	168	let directory = tempfile::tempdir().unwrap();
	169	let path = directory.path().join("cache.sqlite");
	170	let cache = Replica::create(&path, source).unwrap();
	171	let deletion = cache.tree(source, sid, &intent).unwrap().unwrap();
	172	let dependent = cache
	173	.edit_text(&cache.snapshot().unwrap(), sid, left, 0..0, "Local ")
	174	.unwrap()
	175	.unwrap();
	176	let local = cache.snapshot().unwrap();
	177	drop(cache);
	178	let cache = Replica::open(&path).unwrap();
	179	let mut server = if changed {
	180	let edited = PreparedEdit::text(remote, sid, tail, 0..2, "Changed").unwrap();
	181	Server::new(edited.as_bytes())
	182	} else {
	183	Server::new(remote)
	184	};
	185	if changed {
	186	assert_eq!(
	187	cache.sync_once(&mut server).unwrap(),
	188	Some((deletion, EditStatus::Conflict(ConflictKind::ContentChanged)))
	189	);
	190	assert_eq!(cache.snapshot().unwrap(), local);
	191	assert_eq!(cache.pending().unwrap().len(), 2);
	192	assert_eq!(server.publications, 0);
	193	} else {
	194	for expected in [deletion, dependent] {
	195	assert!(
	196	matches!(cache.sync_once(&mut server).unwrap(), Some((id, EditStatus::Published { .. })) if id == expected)
	197	);
	198	}
	199	assert!(cache.pending().unwrap().is_empty());
	200	assert_eq!(server.publications, 2);
	201	assert!(!children(&server.durable, sid, *outline_id).contains(&intent.object()));
	202	let node = super::outline::node(&server.durable, sid, left);
	203	assert!(node["kind"]["text"].as_str().unwrap().starts_with("Local "));
	204	}
	205	}
	206	}
	207
103	208	#[test]
104	209	fn deletion_requires_review_of_remote_content_and_preserves_later_work() {
105	210	let (source, sid, outline, paragraphs, texts) = fixture();

tools/offline_document_history.py+46-19

...	...	@@ -18,7 +18,8 @@ def characters(observed):
18	18	def operation_kinds(events):
19	19	kinds = tuple(events[0].get('document_kinds', ('insert', 'format')))
20	20	assert kinds in (('insert', 'format'), ('insert', 'format', 'text'),
21		('insert', 'format', 'text', 'split', 'right_text', 'join')), 'Unknown document workload'
	21	('insert', 'format', 'text', 'split', 'right_text', 'join'),
	22	('insert', 'format', 'text', 'split', 'right_text', 'nest', 'unnest', 'join', 'tail_split', 'delete')), 'Unknown document workload'
22	23	return kinds
23	24
24	25
...	...	@@ -41,8 +42,9 @@ def document_history(logs, operations):
41	42	linked = {}
42	43	for intent, receipt in zip(edits, receipts, strict=True):
43	44	changed_targets = {intent['object']}
44		if intent['kind'] == 'split': changed_targets.add(identity(intent['split'], 2))
	45	if intent['kind'] in ('split', 'tail_split'): changed_targets.add(identity(intent['split'], 2))
45	46	if intent['kind'] == 'join': changed_targets = set(intent['joined'])
	47	if intent['kind'] in ('nest', 'unnest'): changed_targets = set()
46	48	attempts = [row for row in events if row['event'] == 'remote_attempt' and row['revision'] == receipt['revision']
47	49	and set(row.get('document_changes') or {}) == changed_targets]
48	50	if not attempts and intent['kind'] == 'format':
...	...	@@ -51,10 +53,12 @@ def document_history(logs, operations):
51	53	assert len(attempts) == 1, 'Document receipt lacks one publication attempt'
52	54	attempt, = attempts
53	55	assert attempt['state'] in ('Committed', 'Unknown'), 'Receipt identifies an unpublished document operation'
54		assert intent['object'] in attempt['documents'] and attempt['document_changes'] == {target: attempt['documents'].get(target) for target in changed_targets}, 'Document publication changed another target'
	56	assert (intent['object'] in attempt['documents']) == (intent['kind'] != 'delete'), 'Document publication retained or lost its target'
	57	assert attempt['document_changes'] == {target: attempt['documents'].get(target) for target in changed_targets}, 'Document publication changed another target'
55	58	successful = [row for row in events if row['event'] == 'remote_attempt'
56	59	and row['state'] in ('Committed', 'Unknown')
57		and row.get('document_changes') == attempt['document_changes']]
	60	and row.get('document_changes') == attempt['document_changes']
	61	and row.get('document_graph_changes') == attempt.get('document_graph_changes')]
58	62	assert successful == [attempt], 'Document intent was published or attempted uncertainly more than once'
59	63	assert intent['started_us'] <= attempt['started_us'] <= attempt['finished_us'] <= receipt['at_us'] and intent['started_us'] <= intent['finished_us'] <= receipt['at_us'], 'Document acknowledgement order is invalid'
60	64	if attempt['state'] == 'Unknown':
...	...	@@ -77,7 +81,7 @@ def document_history(logs, operations):
77	81	linked[intent['id']] = {**attempt, 'acknowledged_us': receipt['at_us'], 'receipt_revision': receipt['revision']}
78	82	assert {(row['revision'], row['started_us']) for row in events
79	83	if row['event'] == 'remote_attempt' and row['state'] in ('Committed', 'Unknown')
80		and row.get('document_changes')} == {
	84	and (row.get('document_changes') or row.get('document_graph_changes'))} == {
81	85	(attempt['revision'], attempt['started_us']) for attempt in linked.values()
82	86	}, 'A document publication lacks its recorded intent and receipt'
83	87	for at in range(0, len(edits), len(kinds)):
...	...	@@ -125,7 +129,8 @@ def document_history(logs, operations):
125	129	for state in states.values():
126	130	state['parts'] = [(paragraph, target, 0, len(state['characters']))]
127	131	if replaced and boundaries:
128		split, right_edit, join = boundaries
	132	split, right_edit, *following = boundaries
	133	join = next(row for row in following if row['kind'] == 'join')
129	134	assert events[0].get('document_graph') is True, 'Boundary workload requires structural observations'
130	135	assert all(row.get('document') == target and row['space'] == inserted['space']
131	136	for row in edits[at:at + len(kinds)]), 'Boundary operation lost its owning document'
...	...	@@ -137,17 +142,37 @@ def document_history(logs, operations):
137	142	assert right_edit['object'] == right and right_edit['range'] == [0, 2] and right_edit['replacement'] == 'B🦋', 'Dependent right edit differs from workload'
138	143	assert join['object'] == target and join['joined'] == [target, right], 'Join does not retain its original targets'
139	144	updated = final[:boundary] + [(char, *final[boundary][1:]) for char in 'B🦋'] + final[boundary+2:]
140		for row, value, parts in [
141		(split, final, [(paragraph, target, 0, boundary), (right_paragraph, right, boundary, len(final))]),
142		(right_edit, updated, [(paragraph, target, 0, boundary), (right_paragraph, right, boundary, len(updated))]),
143		(join, updated, [(paragraph, target, 0, len(updated))]),
144		]:
	145	split_parts = [(paragraph, target, 0, boundary), (right_paragraph, right, boundary, len(final))]
	146	edited_parts = [(paragraph, target, 0, boundary), (right_paragraph, right, boundary, len(updated))]
	147	stages = [(split, final, split_parts, {}), (right_edit, updated, edited_parts, {})]
	148	if len(following) > 1:
	149	nest, unnest, _, tail_split, deleted = following
	150	outline = identity(insertion, 1) if 'Outline' in insertion['placement'] else insertion['parent']
	151	for row, parent in [(nest, paragraph), (unnest, outline)]:
	152	tree = row['tree']
	153	assert row['object'] == target and tree['object'] == right_paragraph and tree['author'] == insertion['author'], 'Tree move addresses another subtree or author'
	154	assert tree['placement'] == {'Move': {'parent': parent, 'before': None}}, 'Tree destination differs from workload'
	155	stages += [(nest, updated, edited_parts, {right_paragraph: paragraph}),
	156	(unnest, updated, edited_parts, {})]
	157	stages.append((join, updated, [(paragraph, target, 0, len(updated))], {}))
	158	if len(following) > 1:
	159	tail = tail_split['split']
	160	tail_text, tail_paragraph = identity(tail, 2), identity(tail, 1)
	161	assert tail_split['object'] == tail['text'] == target and tail['author'] == insertion['author'], 'Tail split addresses another text or author'
	162	assert tail['offset'] == end+1 and tail_split['range'] == [end+1, end+1], 'Tail split boundary differs from workload'
	163	tree = deleted['tree']
	164	assert deleted['object'] == tail_text and tree['object'] == tail_paragraph and tree['author'] == insertion['author'], 'Deletion addresses another subtree or author'
	165	assert tree['placement'] == 'Delete', 'Deletion became a move'
	166	stages += [(tail_split, updated, [(paragraph, target, 0, len(updated)-1), (tail_paragraph, tail_text, len(updated)-1, len(updated))], {}),
	167	(deleted, updated[:-1], [(paragraph, target, 0, len(updated)-1)], {})]
	168	known_texts = {oid for _, _, pieces, _ in stages for _, oid, _, _ in pieces}
	169	for row, value, parts, parents in stages:
145	170	attempt = linked[row['id']]
146	171	assert list(states.values())[-1]['attempt']['finished_us'] <= attempt['started_us'], 'Boundary publication preceded its dependency'
147		assert set(attempt['documents']) & {target, right} == {oid for _, oid, _, _ in parts}, 'Boundary publication omitted or resurrected a text object'
	172	assert set(attempt['documents']) & known_texts == {oid for _, oid, _, _ in parts}, 'Boundary publication omitted or resurrected a text object'
148	173	for _, oid, start, stop in parts:
149	174	assert characters(attempt['documents'][oid]) == value[start:stop], 'Boundary publication differs from its local intent'
150		states[row['kind']] = {'characters': value, 'parts': parts, 'attempt': attempt}
	175	states[row['kind']] = {'characters': value, 'parts': parts, 'parents': parents, 'attempt': attempt}
151	176	allocated = {oid for state in states.values() for paragraph, text, _, _ in state['parts'] for oid in (paragraph, text)}
152	177	existing = {oid for document in documents.values() for state in document['states'].values()
153	178	for paragraph, text, _, _ in state['parts'] for oid in (paragraph, text)}
...	...	@@ -177,6 +202,8 @@ def document_history(logs, operations):
177	202	states = list(document['states'].values())
178	203	known_texts = {oid for state in states for _, oid, _, _ in state['parts']}
179	204	known_paragraphs = {oid for state in states for oid, _, _, _ in state['parts']}
	205	insertion = document['insertion']
	206	outline = identity(insertion, 1) if 'Outline' in insertion['placement'] else insertion['parent']
180	207	if is_read and row['started_us'] > states[0]['attempt']['acknowledged_us']:
181	208	assert target in observed, 'Reader missed an acknowledged insertion'
182	209	if target not in observed:
...	...	@@ -185,7 +212,9 @@ def document_history(logs, operations):
185	212	actual = {oid: characters(observed[oid]) for oid in known_texts & observed.keys()}
186	213	matches = [i for i, state in enumerate(states)
187	214	if actual == {oid: state['characters'][start:stop] for _, oid, start, stop in state['parts']}
188		and (not structural or known_paragraphs & graph.keys() == {oid for oid, _, _, _ in state['parts']})]
	215	and (not structural or (known_paragraphs & graph.keys() == {oid for oid, _, _, _ in state['parts']}
	216	and all(graph[p]['parent'] == state.get('parents', {}).get(p, outline)
	217	for p, _, _, _ in state['parts'])))]
189	218	assert matches, 'Reader observed partial or invented document content'
190	219	if is_read:
191	220	matches = [i for i in matches if i >= previous.get(target, 0)]
...	...	@@ -197,16 +226,14 @@ def document_history(logs, operations):
197	226	assert matches, 'Reader missed acknowledged document content'
198	227	previous[target] = min(matches)
199	228	if structural:
200		insertion = document['insertion']
201		outline = insertion['parent']
202	229	if 'Outline' in insertion['placement']:
203		outline = identity(insertion, 1)
204	230	expected_graph[outline] = {'parent': insertion['parent'], 'children': [], 'content': [],
205	231	'child_level': 1, 'position': insertion['placement']['Outline']}
206	232	assert outline in expected_graph, 'Snapshot omitted the inserted outline'
207	233	for paragraph, oid, _, _ in states[min(matches)]['parts']:
208		expected_graph[outline]['children'].append(paragraph)
209		expected_graph[paragraph] = {'parent': outline, 'children': [], 'content': [oid],
	234	parent = states[min(matches)].get('parents', {}).get(paragraph, outline)
	235	expected_graph[parent]['children'].append(paragraph)
	236	expected_graph[paragraph] = {'parent': parent, 'children': [], 'content': [oid],
210	237	'child_level': 1, 'position': None}
211	238	if structural:
212	239	assert graph == expected_graph, 'Snapshot contains partial, reordered or invented paragraph structure'

tools/test_offline_document_history.py+42-3

...	...	@@ -149,9 +149,48 @@ class DocumentHistoryTests(unittest.TestCase):
149	149	self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
150	150	events = [json.loads(line) for line in result.stdout.splitlines() if line.startswith('{')]
151	151	documents = document_history({'w0': events}, 2)
152		self.assertEqual(sum(len(document['states']) for document in documents.values()), 12)
153		self.assertEqual(sum(row['event'] == 'remote_attempt' and row['state'] == 'Unknown' for row in events), 4)
154		self.assertEqual(sum(row['event'] == 'remote_confirm' and row['state'] == 'Committed' for row in events), 4)
	152	self.assertEqual(sum(len(document['states']) for document in documents.values()), 20)
	153	self.assertEqual(sum(row['event'] == 'remote_attempt' and row['state'] == 'Unknown' for row in events), 12)
	154	self.assertEqual(sum(row['event'] == 'remote_confirm' and row['state'] == 'Committed' for row in events), 12)
	155	document = next(iter(documents.values()))
	156	states = document['states']
	157	for phase, next_phase, stale in [('nest', 'unnest', 'right_text'), ('delete', None, 'tail_split')]:
	158	read = copy.deepcopy(states[stale]['attempt'])
	159	started = states[phase]['attempt']['acknowledged_us'] + 1
	160	if next_phase:
	161	self.assertLess(started + 1, states[next_phase]['attempt']['started_us'])
	162	read.update(event='read', started_us=started, finished_us=started + 1)
	163	logs = {'w0': events, 'r0': [{'event': 'ready', 'document_graph': True}, read, {'event': 'done'}]}
	164	with self.subTest(phase=phase), self.assertRaisesRegex(AssertionError, 'missed acknowledged'):
	165	document_history(logs, 2)
	166	for mutation in ('nest-intent', 'unnest-intent', 'delete-intent', 'tail-boundary',
	167	'nest-children', 'delete-resurrection', 'graph-delta', 'extra-attempt'):
	168	changed = copy.deepcopy(events)
	169	intents = {row['kind']: row for row in changed if row['event'] == 'local_document_commit' and row['operation'] == 0}
	170	attempts = {kind: next(row for row in changed if row['event'] == 'remote_attempt' and row['revision'] == state['attempt']['revision'])
	171	for kind, state in states.items()}
	172	if mutation == 'nest-intent':
	173	intents['nest']['tree']['placement']['Move']['parent'] = intents['unnest']['tree']['placement']['Move']['parent']
	174	elif mutation == 'unnest-intent':
	175	intents['unnest']['tree']['placement']['Move']['before'] = intents['unnest']['tree']['object']
	176	elif mutation == 'delete-intent':
	177	intents['delete']['tree']['object'] = intents['nest']['tree']['object']
	178	elif mutation == 'tail-boundary':
	179	intents['tail_split']['split']['offset'] += 1
	180	elif mutation == 'nest-children':
	181	parent = intents['nest']['tree']['placement']['Move']['parent']
	182	attempts['nest']['document_graph'][parent]['children'] = []
	183	elif mutation == 'delete-resurrection':
	184	tail = intents['delete']['object']
	185	attempts['delete']['documents'][tail] = attempts['tail_split']['documents'][tail]
	186	elif mutation == 'graph-delta':
	187	attempts['nest']['document_graph_changes'] = {}
	188	else:
	189	extra = copy.deepcopy(attempts['nest'])
	190	extra.update(revision='unrecorded-tree-revision', state='Committed')
	191	changed.append(extra)
	192	with self.subTest(mutation=mutation), self.assertRaises(AssertionError):
	193	document_history({'w0': changed}, 2)
155	194
156	195	def test_boundary_histories_retain_every_intermediate_graph_and_retired_identity(self):
157	196	baseline = history(boundaries=True)