Commit 7f30bf34648c9ac7a69e74956b04546b9894a380 | snowbound

author	clover caruso <git@paperclover.net> 2026-09-12 17:09:11-07:00
committer	clover caruso <git@paperclover.net> 2026-09-25 20:26:20-07:00
log	7f30bf34648c9ac7a69e74956b04546b9894a380
tree	08a8c4f0e49181205989fdbdd0b98c72f9e8715e
parent	bcb13ba046299393ccadffa70cc2c57963ffc3ce
signature	Signed by SSH key SHA256:52mNGHRsVFBDED9IAX5pe+LRWUefqTbxEReunq21QvU

feat: read ink drawings and handwriting as strokes in page coordinates

The public MS-ONE text has no ink object; the layout matches what the
onenote_parser crate (MPL-2.0, reimplemented here rather than copied) and
Joplin's converter read: a container (jcid 0x60014) whose data node (0x2003b)
lists strokes (0x20047), each packing its coordinates as ISF multi-byte
first differences in HIMETRIC, one block per dimension of the style object's
(0x120048) GUID table, with pen width, height, colour, transparency and tip
on the style. The reader interprets those four kinds, and page::Ink decodes
every stroke into a polyline of absolute page points, keeping nested
containers as groups; ink is a page object or paragraph content, and the
writer treats it as fixed.

corpus/native-ink/cold-ui-ink is the oracle: two mouse-drawn drawings whose
native positions and sizes the decoded extents reproduce (OneNote reports a
size one HIMETRIC unit larger than the point extent). tests/page_ink.rs
checks the Rust decoding and tools/test_ink_read.py an independent Python
decoding of the same packets; verify-document.py now compares every ink
drawing's extent, projected through the page origin on left-to-right pages,
and reports differences as geometry entries. The canvas places its read-only
ink placeholder at the stroke extent.

Assisted-by: claude-fable-5.1

17 files changed, 521 insertions(+), 15 deletions(-)

crates
- canvas
  - src
    bin
    page_probe.rs
    document.rs
    editor.rs
    editor
    page.rs
    outline.rs
- onestore
  - README.md
  - src
    document.rs
    page
    ink.rs
    mod.rs
    write.rs
  - tests
    document.rs
    page_ink.rs
    page_model.rs
    page_tables.rs
    support
    page_edits.rs
tools
- test_ink_read.py
- verify-document.py

crates/canvas/src/bin/page_probe.rs+2

...	@@ -65,6 +65,8 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {	...	@@ -65,6 +65,8 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
65	PageObject::Image(image) => objects.push(json!({"kind": "image", "id": image.id,	65	PageObject::Image(image) => objects.push(json!({"kind": "image", "id": image.id,
66	"layout": image.layout, "bytes": image.bytes.as_ref().map(\|b\| b.len()),	66	"layout": image.layout, "bytes": image.bytes.as_ref().map(\|b\| b.len()),
67	"alt": image.alt, "background": image.background})),	67	"alt": image.alt, "background": image.background})),
		68	PageObject::Ink(ink) => objects.push(json!({"kind": "ink", "id": ink.id,
		69	"bounds": ink.bounds(), "strokes": ink.strokes.len(), "groups": ink.groups.len()})),
68	PageObject::Unsupported(object) => objects.push(json!({"kind": "unsupported",	70	PageObject::Unsupported(object) => objects.push(json!({"kind": "unsupported",
69	"id": object.id, "layout": object.layout, "jcid": object.jcid})),	71	"id": object.id, "layout": object.layout, "jcid": object.jcid})),
70	}	72	}

crates/canvas/src/document.rs+1

...	@@ -155,6 +155,7 @@ pub(crate) fn validate_nodes(	...	@@ -155,6 +155,7 @@ pub(crate) fn validate_nodes(
155	}	155	}
156	ParagraphContent::Image(_)	156	ParagraphContent::Image(_)
157	\| ParagraphContent::Attachment(_)	157	\| ParagraphContent::Attachment(_)
		158	\| ParagraphContent::Ink(_)
158	\| ParagraphContent::Unsupported(_) => {	159	\| ParagraphContent::Unsupported(_) => {
159	return Err(EditError::UnsupportedContent);	160	return Err(EditError::UnsupportedContent);
160	}	161	}

crates/canvas/src/editor.rs+3

crates/canvas/src/editor/page.rs+15-4

 @@ -80,10 +80,15 @@ impl ReadOnlyObject {
             [0.0; 2]
         };
         let layout = source.layout();
-        let x = layout.x.unwrap_or(0.0) + offset[0];
+        // Ink carries no layout of its own; its strokes say where it is.
-        let y = layout.y.unwrap_or(0.0) + offset[1];
+        let extent = match &source {
-        let width = layout.max_width.unwrap_or(160.0);
+            PageObject::Ink(ink) => ink.bounds(),
-        let height = layout.max_height.unwrap_or(42.0);
+            _ => None,
+        };
+        let x = extent.map_or(layout.x.unwrap_or(0.0), |e| e[0]) + offset[0];
+        let y = extent.map_or(layout.y.unwrap_or(0.0), |e| e[1]) + offset[1];
+        let width = extent.map_or(layout.max_width.unwrap_or(160.0), |e| e[2]);
+        let height = extent.map_or(layout.max_height.unwrap_or(42.0), |e| e[3]);
         if [x, y, width, height].iter().any(|v| !v.is_finite()) || width <= 0.0 || height <= 0.0 {
             return Err(EditorError::InvalidGeometry);
         }
 @@ -321,6 +326,12 @@ pub(crate) fn build(
                 };
                 objects.push(Content::Image(source));
             }
+            PageObject::Ink(_) => objects.push(Content::ReadOnly(ReadOnlyObject::new(
+                object,
+                page.margin_origin,
+                "Ink\nRead-only",
+                engine,
+            )?)),
             PageObject::Unsupported(_) => objects.push(Content::ReadOnly(ReadOnlyObject::new(
                 object,
                 page.margin_origin,

crates/canvas/src/outline.rs+1

...	@@ -376,6 +376,7 @@ impl OutlineLayout {	...	@@ -376,6 +376,7 @@ impl OutlineLayout {
376	}	376	}
377	ParagraphContent::Image(_)	377	ParagraphContent::Image(_)
378	\| ParagraphContent::Attachment(_)	378	\| ParagraphContent::Attachment(_)
		379	\| ParagraphContent::Ink(_)
379	\| ParagraphContent::Unsupported(_) => {	380	\| ParagraphContent::Unsupported(_) => {
380	return Err(LayoutError::UnsupportedContent);	381	return Err(LayoutError::UnsupportedContent);
381	}	382	}

crates/onestore/README.md+3-3

crates/onestore/src/document.rs+68-3

...	@@ -326,8 +326,41 @@ pub enum Kind<'a> {	...	@@ -326,8 +326,41 @@ pub enum Kind<'a> {
326	collapse_state: Option<u8>,	326	collapse_state: Option<u8>,
327	},	327	},
328	OutlineGroup,	328	OutlineGroup,
329	/// Native ink properties remain uninterpreted in the element's extra arena.	329	/// An ink drawing or handwriting container: strokes hang off `data`, nested containers off
330	Ink,	330	/// the element's content list. Scaling multiplies stroke coordinates.
		331	Ink {
		332	data: Option<ExGuid>,
		333	scale_x: Option<f32>,
		334	scale_y: Option<f32>,
		335	},
		336	InkData {
		337	strokes: Vec<ExGuid>,
		338	bounds: Option<[i32; 4]>,
		339	},
		340	/// One stroke: `path` holds ISF multi-byte first differences, one block per dimension of
		341	/// the style's dimension table, in HIMETRIC (1/2540 inch) page coordinates.
		342	InkStroke {
		343	path: Vec<u8>,
		344	style: Option<ExGuid>,
		345	bias: Option<u8>,
		346	language: Option<u16>,
		347	identity: Option<[u8; 16]>,
		348	index: Option<u32>,
		349	},
		350	/// Drawing attributes shared by strokes; `dimensions` holds 32-byte entries of GUID, lower
		351	/// and upper limit, and resolution.
		352	InkStyle {
		353	dimensions: Vec<u8>,
		354	width: Option<f32>,
		355	height: Option<f32>,
		356	color: Option<u32>,
		357	transparency: Option<u8>,
		358	pen_tip: Option<u8>,
		359	raster_operation: Option<u8>,
		360	antialiased: Option<bool>,
		361	fit_to_curve: Option<bool>,
		362	ignore_pressure: Option<bool>,
		363	},
331	RichText {	364	RichText {
332	text: String,	365	text: String,
333	runs: Vec<TextRun>,	366	runs: Vec<TextRun>,
...	@@ -993,7 +1026,39 @@ impl<'a> Element<'a> {	...	@@ -993,7 +1026,39 @@ impl<'a> Element<'a> {
993	level: f.u32(0x14001dff)?,	1026	level: f.u32(0x14001dff)?,
994	author: f.text(0x1c001d9e)?,	1027	author: f.text(0x1c001d9e)?,
995	},	1028	},
996	0x60014 => Kind::Ink,	1029	0x60014 => Kind::Ink {
		1030	data: f.one(0x20003415)?,
		1031	scale_x: f.float(0x14001c46, 1.0)?,
		1032	scale_y: f.float(0x14001c47, 1.0)?,
		1033	},
		1034	0x2003b => Kind::InkData {
		1035	strokes: f.refs(0x24003416, IdStream::Objects)?,
		1036	bounds: f.fixed::<16>(0x1c003418)?.map(\|b\| {
		1037	std::array::from_fn(\|i\| {
		1038	i32::from_le_bytes(b[i * 4..i * 4 + 4].try_into().unwrap())
		1039	})
		1040	}),
		1041	},
		1042	0x20047 => Kind::InkStroke {
		1043	path: f.bytes(0x1c00340b)?.unwrap_or_default().to_vec(),
		1044	style: f.one(0x20003409)?,
		1045	bias: f.u8(0x0c00341c)?,
		1046	language: f.u16(0x1000341b)?,
		1047	identity: f.fixed(0x1c00341a)?,
		1048	index: f.u32(0x14003419)?,
		1049	},
		1050	0x120048 => Kind::InkStyle {
		1051	dimensions: f.bytes(0x1c00340a)?.unwrap_or_default().to_vec(),
		1052	width: f.float(0x1400340d, 1.0)?,
		1053	height: f.float(0x1400340c, 1.0)?,
		1054	color: f.u32(0x1400340f)?,
		1055	transparency: f.u8(0x0c003414)?,
		1056	pen_tip: f.u8(0x0c003412)?,
		1057	raster_operation: f.u8(0x0c003413)?,
		1058	antialiased: f.boolean(0x0800340e)?,
		1059	fit_to_curve: f.boolean(0x08003410)?,
		1060	ignore_pressure: f.boolean(0x08003411)?,
		1061	},
997	0x20031 => Kind::SectionMetadata {	1062	0x20031 => Kind::SectionMetadata {
998	name: f.text(0x1c00349b)?,	1063	name: f.text(0x1c00349b)?,
999	color: f.u32(0x14001cbe)?,	1064	color: f.u32(0x14001cbe)?,

crates/onestore/src/page/ink.rs created+215

...	@@ -0,0 +1,215 @@
1	//! Ink drawings and handwriting: strokes as page-coordinate polylines in points, decoded from
2	//! the stroke packets OneNote stores (ISF multi-byte first differences in HIMETRIC).
3
4	use super::Error;
5	use crate::{
6	ExGuid,
7	document::{Kind, Layout, Revision},
8	};
9
10	const HIMETRIC_PER_POINT: f32 = 2540.0 / 72.0;
11	const DIMENSION_X: [u8; 16] = [
12	0x8f, 0x6a, 0x8a, 0x59, 0xc0, 0x52, 0xa0, 0x4b, 0x93, 0xaf, 0xaf, 0x35, 0x74, 0x11, 0xa5, 0x61,
13	];
14	const DIMENSION_Y: [u8; 16] = [
15	0x75, 0x9f, 0x3f, 0xb5, 0xe0, 0x04, 0x98, 0x44, 0xa7, 0xee, 0xc3, 0x0d, 0xbb, 0x5a, 0x90, 0x11,
16	];
17
18	#[derive(Clone, Debug, PartialEq, serde::Serialize, serde::Deserialize)]
19	pub struct Ink {
20	pub id: ExGuid,
21	pub layout: Layout,
22	pub strokes: Vec<InkStroke>,
23	/// Nested ink containers, as newer OneNote versions group handwriting.
24	pub groups: Vec<Ink>,
25	}
26
27	#[derive(Clone, Debug, PartialEq, serde::Serialize, serde::Deserialize)]
28	pub struct InkStroke {
29	pub id: ExGuid,
30	/// Absolute page coordinates in points.
31	pub points: Vec<[f32; 2]>,
32	/// Pen width and height in points.
33	pub width: f32,
34	pub height: f32,
35	/// COLORREF; absent means the window text colour.
36	pub color: Option<u32>,
37	/// 0 is opaque, 255 fully transparent.
38	pub transparency: Option<u8>,
39	/// 0 is a round (ball) tip, 1 a rectangle.
40	pub pen_tip: Option<u8>,
41	}
42
43	impl Ink {
44	pub(super) fn read(
45	revision: &Revision<'_>,
46	id: ExGuid,
47	node: &crate::document::Element<'_>,
48	) -> Result<Self, Error> {
49	let invalid = \|message\| Error { offset: 0, message };
50	let Kind::Ink {
51	data,
52	scale_x,
53	scale_y,
54	} = &node.kind
55	else {
56	unreachable!()
57	};
58	let scale = [scale_x.unwrap_or(1.0), scale_y.unwrap_or(1.0)];
59	let mut strokes = Vec::new();
60	if let Some(data) = data {
61	let data = revision
62	.nodes
63	.get(data)
64	.ok_or_else(\|\| invalid("Missing ink data"))?;
65	let Kind::InkData { strokes: ids, .. } = &data.kind else {
66	return Err(invalid("Ink data has the wrong type"));
67	};
68	for stroke_id in ids {
69	let stroke = revision
70	.nodes
71	.get(stroke_id)
72	.ok_or_else(\|\| invalid("Missing ink stroke"))?;
73	strokes.push(InkStroke::read(revision, *stroke_id, stroke, scale)?);
74	}
75	}
76	let mut groups = Vec::new();
77	for child in &node.content {
78	let element = revision
79	.nodes
80	.get(child)
81	.ok_or_else(\|\| invalid("Missing nested ink"))?;
82	if matches!(element.kind, Kind::Ink { .. }) {
83	groups.push(Ink::read(revision, *child, element)?);
84	}
85	}
86	Ok(Self {
87	id,
88	layout: node.layout.clone(),
89	strokes,
90	groups,
91	})
92	}
93
94	/// The extent of every stroke point, as `[left, top, width, height]` in points.
95	pub fn bounds(&self) -> Option<[f32; 4]> {
96	let mut extent: Option<[f32; 4]> = None;
97	for point in self
98	.strokes
99	.iter()
100	.flat_map(\|stroke\| stroke.points.iter())
101	.copied()
102	.chain(
103	self.groups
104	.iter()
105	.filter_map(Ink::bounds)
106	.flat_map(\|[x, y, w, h]\| [[x, y], [x + w, y + h]]),
107	)
108	{
109	extent = Some(match extent {
110	None => [point[0], point[1], point[0], point[1]],
111	Some([x0, y0, x1, y1]) => [
112	x0.min(point[0]),
113	y0.min(point[1]),
114	x1.max(point[0]),
115	y1.max(point[1]),
116	],
117	});
118	}
119	extent.map(\|[x0, y0, x1, y1]\| [x0, y0, x1 - x0, y1 - y0])
120	}
121	}
122
123	impl InkStroke {
124	fn read(
125	revision: &Revision<'_>,
126	id: ExGuid,
127	node: &crate::document::Element<'_>,
128	scale: [f32; 2],
129	) -> Result<Self, Error> {
130	let invalid = \|message\| Error { offset: 0, message };
131	let Kind::InkStroke { path, style, .. } = &node.kind else {
132	return Err(invalid("Ink stroke has the wrong type"));
133	};
134	let style = style
135	.and_then(\|style\| revision.nodes.get(&style))
136	.ok_or_else(\|\| invalid("Missing ink stroke style"))?;
137	let Kind::InkStyle {
138	dimensions,
139	width,
140	height,
141	color,
142	transparency,
143	pen_tip,
144	..
145	} = &style.kind
146	else {
147	return Err(invalid("Ink stroke style has the wrong type"));
148	};
149	if dimensions.len() % 32 != 0 \|\| dimensions.is_empty() {
150	return Err(invalid("Ink dimension table has an invalid length"));
151	}
152	let guids: Vec<&[u8]> = dimensions.chunks_exact(32).map(\|d\| &d[..16]).collect();
153	let axis = \|guid: [u8; 16]\| guids.iter().position(\|g\| **g == guid);
154	let (Some(x), Some(y)) = (axis(DIMENSION_X), axis(DIMENSION_Y)) else {
155	return Err(invalid("Ink dimension table lacks X and Y"));
156	};
157	let values = multi_byte(path).ok_or_else(\|\| invalid("Ink stroke packet is malformed"))?;
158	if values.len() % guids.len() != 0 {
159	return Err(invalid("Ink stroke packet does not cover its dimensions"));
160	}
161	let per_dimension = values.len() / guids.len();
162	let coordinates = \|dimension: usize, factor: f32\| {
163	let mut position = 0i64;
164	values[dimension * per_dimension..(dimension + 1) * per_dimension]
165	.iter()
166	.map(move \|delta\| {
167	position += delta;
168	position as f32 * factor / HIMETRIC_PER_POINT
169	})
170	};
171	let points = coordinates(x, scale[0])
172	.zip(coordinates(y, scale[1]))
173	.map(\|(x, y)\| [x, y])
174	.collect();
175	Ok(Self {
176	id,
177	points,
178	width: width.unwrap_or(0.0) / HIMETRIC_PER_POINT,
179	height: height.unwrap_or(0.0) / HIMETRIC_PER_POINT,
180	color: *color,
181	transparency: *transparency,
182	pen_tip: *pen_tip,
183	})
184	}
185	}
186
187	/// ISF multi-byte decoding: a count, then that many 7-bit little-endian varints whose low bit
188	/// is the sign.
189	fn multi_byte(bytes: &[u8]) -> Option<Vec<i64>> {
190	let mut cursor = 0;
191	let mut next = \|\| {
192	let mut value = 0u64;
193	let mut shift = 0;
194	loop {
195	let byte = *bytes.get(cursor)?;
196	cursor += 1;
197	if shift >= 64 {
198	return None;
199	}
200	value \|= u64::from(byte & 0x7f) << shift;
201	shift += 7;
202	if byte & 0x80 == 0 {
203	return Some(value);
204	}
205	}
206	};
207	let count = next()? >> 1;
208	let mut values = Vec::with_capacity(usize::try_from(count).ok()?.min(1 << 20));
209	for _ in 0..count {
210	let raw = next()?;
211	let magnitude = i64::try_from(raw >> 1).ok()?;
212	values.push(if raw & 1 == 1 { -magnitude } else { magnitude });
213	}
214	(cursor == bytes.len()).then_some(values)
215	}

crates/onestore/src/page/mod.rs+16

...	@@ -10,8 +10,11 @@ use std::{	...	@@ -10,8 +10,11 @@ use std::{
10	sync::Arc,	10	sync::Arc,
11	};	11	};
12		12
		13	pub mod ink;
13	pub mod link;	14	pub mod link;
14	pub mod text;	15	pub mod text;
		16
		17	pub use ink::{Ink, InkStroke};
15	pub(crate) mod write;	18	pub(crate) mod write;
16	pub use text::Paragraph;	19	pub use text::Paragraph;
17		20
...	@@ -45,6 +48,7 @@ pub enum PageObject {	...	@@ -45,6 +48,7 @@ pub enum PageObject {
45	Outline(Outline),	48	Outline(Outline),
46	Title(Title),	49	Title(Title),
47	Image(Image),	50	Image(Image),
		51	Ink(Ink),
48	Unsupported(Unsupported),	52	Unsupported(Unsupported),
49	}	53	}
50		54
...	@@ -54,6 +58,7 @@ impl PageObject {	...	@@ -54,6 +58,7 @@ impl PageObject {
54	Self::Outline(value) => value.id,	58	Self::Outline(value) => value.id,
55	Self::Title(value) => value.id,	59	Self::Title(value) => value.id,
56	Self::Image(value) => value.id,	60	Self::Image(value) => value.id,
		61	Self::Ink(value) => value.id,
57	Self::Unsupported(value) => value.id,	62	Self::Unsupported(value) => value.id,
58	}	63	}
59	}	64	}
...	@@ -62,6 +67,7 @@ impl PageObject {	...	@@ -62,6 +67,7 @@ impl PageObject {
62	Self::Outline(value) => &value.layout,	67	Self::Outline(value) => &value.layout,
63	Self::Title(value) => &value.layout,	68	Self::Title(value) => &value.layout,
64	Self::Image(value) => &value.layout,	69	Self::Image(value) => &value.layout,
		70	Self::Ink(value) => &value.layout,
65	Self::Unsupported(value) => &value.layout,	71	Self::Unsupported(value) => &value.layout,
66	}	72	}
67	}	73	}
...	@@ -71,6 +77,7 @@ impl PageObject {	...	@@ -71,6 +77,7 @@ impl PageObject {
71	Self::Outline(value) => &mut value.layout,	77	Self::Outline(value) => &mut value.layout,
72	Self::Title(value) => &mut value.layout,	78	Self::Title(value) => &mut value.layout,
73	Self::Image(value) => &mut value.layout,	79	Self::Image(value) => &mut value.layout,
		80	Self::Ink(value) => &mut value.layout,
74	Self::Unsupported(value) => &mut value.layout,	81	Self::Unsupported(value) => &mut value.layout,
75	}	82	}
76	}	83	}
...	@@ -116,6 +123,8 @@ pub enum ParagraphContent {	...	@@ -116,6 +123,8 @@ pub enum ParagraphContent {
116	Image(Image),	123	Image(Image),
117	/// An embedded file, as OneNote inserts attachments into outlines.	124	/// An embedded file, as OneNote inserts attachments into outlines.
118	Attachment(Attachment),	125	Attachment(Attachment),
		126	/// Handwriting held by the paragraph.
		127	Ink(Ink),
119	Unsupported(Unsupported),	128	Unsupported(Unsupported),
120	}	129	}
121		130
...	@@ -471,6 +480,10 @@ impl Page {	...	@@ -471,6 +480,10 @@ impl Page {
471	page.objects	480	page.objects
472	.push(PageObject::Image(Image::read(revision, id, node)?));	481	.push(PageObject::Image(Image::read(revision, id, node)?));
473	}	482	}
		483	Kind::Ink { .. } => {
		484	page.objects
		485	.push(PageObject::Ink(Ink::read(revision, id, node)?));
		486	}
474	_ => page.objects.push(PageObject::Unsupported(Unsupported {	487	_ => page.objects.push(PageObject::Unsupported(Unsupported {
475	id,	488	id,
476	jcid: node.jcid,	489	jcid: node.jcid,
...	@@ -601,6 +614,8 @@ fn read_paragraphs(	...	@@ -601,6 +614,8 @@ fn read_paragraphs(
601	ParagraphContent::Image(Image::read(revision, *content_id, content)?)	614	ParagraphContent::Image(Image::read(revision, *content_id, content)?)
602	} else if matches!(content.kind, Kind::Attachment { .. }) {	615	} else if matches!(content.kind, Kind::Attachment { .. }) {
603	ParagraphContent::Attachment(Attachment::read(revision, *content_id, content)?)	616	ParagraphContent::Attachment(Attachment::read(revision, *content_id, content)?)
		617	} else if matches!(content.kind, Kind::Ink { .. }) {
		618	ParagraphContent::Ink(Ink::read(revision, *content_id, content)?)
604	} else if matches!(content.kind, Kind::Table { .. }) {	619	} else if matches!(content.kind, Kind::Table { .. }) {
605	ParagraphContent::Table(read_table(	620	ParagraphContent::Table(read_table(
606	revision,	621	revision,
...	@@ -625,6 +640,7 @@ fn read_paragraphs(	...	@@ -625,6 +640,7 @@ fn read_paragraphs(
625	ParagraphContent::Table(table) => table.tags.as_slice(),	640	ParagraphContent::Table(table) => table.tags.as_slice(),
626	ParagraphContent::Image(_)	641	ParagraphContent::Image(_)
627	\| ParagraphContent::Attachment(_)	642	\| ParagraphContent::Attachment(_)
		643	\| ParagraphContent::Ink(_)
628	\| ParagraphContent::Unsupported(_) => &[],	644	\| ParagraphContent::Unsupported(_) => &[],
629	})	645	})
630	.filter_map(\|tag\| tag.definition.as_ref())	646	.filter_map(\|tag\| tag.definition.as_ref())

crates/onestore/src/page/write.rs+6-3

...	@@ -203,7 +203,7 @@ impl<'a> View<'a> {	...	@@ -203,7 +203,7 @@ impl<'a> View<'a> {
203	view.outline(outline)?;	203	view.outline(outline)?;
204	}	204	}
205	}	205	}
206	PageObject::Image(_) \| PageObject::Unsupported(_) => {}	206	PageObject::Image(_) \| PageObject::Ink(_) \| PageObject::Unsupported(_) => {}
207	}	207	}
208	}	208	}
209	Ok(view)	209	Ok(view)
...	@@ -366,6 +366,7 @@ impl Lowering<'_> {	...	@@ -366,6 +366,7 @@ impl Lowering<'_> {
366	.iter()	366	.iter()
367	.filter_map(\|object\| match object {	367	.filter_map(\|object\| match object {
368	PageObject::Unsupported(unsupported) => Some(format!("{unsupported:?}")),	368	PageObject::Unsupported(unsupported) => Some(format!("{unsupported:?}")),
		369	PageObject::Ink(ink) => Some(format!("{ink:?}")),
369	PageObject::Title(title) => Some(format!(	370	PageObject::Title(title) => Some(format!(
370	"{:?} {:?} {:?} {:?}",	371	"{:?} {:?} {:?} {:?}",
371	title.id,	372	title.id,
...	@@ -383,7 +384,7 @@ impl Lowering<'_> {	...	@@ -383,7 +384,7 @@ impl Lowering<'_> {
383	after_fixed.sort();	384	after_fixed.sort();
384	if before_fixed != after_fixed {	385	if before_fixed != after_fixed {
385	return Err(invalid(	386	return Err(invalid(
386	"Titles and unsupported objects cannot be edited through the page model",	387	"Titles, ink and unsupported objects cannot be edited through the page model",
387	));	388	));
388	}	389	}
389	for (id, outline) in &new.outlines {	390	for (id, outline) in &new.outlines {
...	@@ -672,7 +673,9 @@ impl Lowering<'_> {	...	@@ -672,7 +673,9 @@ impl Lowering<'_> {
672	let outlines: Vec<&Outline> = match object {	673	let outlines: Vec<&Outline> = match object {
673	PageObject::Outline(outline) => vec![outline],	674	PageObject::Outline(outline) => vec![outline],
674	PageObject::Title(title) => title.outlines.iter().collect(),	675	PageObject::Title(title) => title.outlines.iter().collect(),
675	PageObject::Image(_) \| PageObject::Unsupported(_) => Vec::new(),	676	PageObject::Image(_) \| PageObject::Ink(_) \| PageObject::Unsupported(_) => {
		677	Vec::new()
		678	}
676	};	679	};
677	for outline in outlines {	680	for outline in outlines {
678	containers.push(outline.id);	681	containers.push(outline.id);

crates/onestore/tests/document.rs+1-1

...	@@ -860,7 +860,7 @@ fn native_conflict_relationship_and_opaque_ink_are_retained() {	...	@@ -860,7 +860,7 @@ fn native_conflict_relationship_and_opaque_ink_are_retained() {
860	.values()	860	.values()
861	.flat_map(\|s\| s.revisions.values())	861	.flat_map(\|s\| s.revisions.values())
862	.flat_map(\|r\| r.nodes.values())	862	.flat_map(\|r\| r.nodes.values())
863	.filter(\|node\| matches!(node.kind, Kind::Ink))	863	.filter(\|node\| matches!(node.kind, Kind::Ink { .. }))
864	.collect();	864	.collect();
865	assert_eq!(ink.len(), 2);	865	assert_eq!(ink.len(), 2);
866	assert!(	866	assert!(

crates/onestore/tests/page_ink.rs created+76

...	@@ -0,0 +1,76 @@
1	use onestore::{
2	ExGuid, RevisionIndex, Store,
3	document::Document,
4	page::{Ink, Page, PageObject},
5	};
6
7	/// Two drawings made with the mouse in OneNote 2010 (`tools/native/ink.ahk`): a diamond and a
8	/// diagonal line, whose native read reports their positions and sizes.
9	const NATIVE: &[u8] =
10	include_bytes!("../../../corpus/native-ink/cold-ui-ink/notebook/synthetic.one");
11
12	fn first_page(bytes: &[u8]) -> (ExGuid, Page) {
13	let store = Store::parse(bytes).unwrap();
14	assert!(store.checksum_mismatches.is_empty());
15	let index = RevisionIndex::parse(&store).unwrap();
16	let document = Document::parse(&index).unwrap();
17	let (space, _) = document.pages().unwrap()[0];
18	(space, Page::from_space(&document, space).unwrap())
19	}
20
21	fn drawings(page: &Page) -> Vec<&Ink> {
22	page.objects
23	.iter()
24	.filter_map(\|object\| match object {
25	PageObject::Ink(ink) => Some(ink),
26	_ => None,
27	})
28	.collect()
29	}
30
31	fn close(actual: [f32; 4], expected: [f32; 4]) -> bool {
32	actual
33	.iter()
34	.zip(expected)
35	.all(\|(a, e)\| (a - e).abs() < 0.002)
36	}
37
38	#[test]
39	fn native_ink_drawings_read_as_strokes_in_page_coordinates() {
40	let (_, page) = first_page(NATIVE);
41	let found = drawings(&page);
42	let [diamond, line] = found.as_slice() else {
43	panic!("{:?}", found.len());
44	};
45	assert_eq!(diamond.strokes.len(), 1);
46	assert_eq!(diamond.strokes[0].points.len(), 253);
47	// OneNote reports a drawing's size one HIMETRIC unit larger than its point extent.
48	let unit = 72.0 / 2540.0;
49	let native = \|[x, y, w, h]: [f32; 4]\| [x, y, w + unit, h + unit];
50	let bounds = (
51	native(diamond.bounds().unwrap()),
52	native(line.bounds().unwrap()),
53	);
54	assert!(
55	close(bounds.0, [421.5118, 84.7559, 60.00945, 60.00944])
56	&& close(bounds.1, [504.0, 92.23936, 30.0189, 45.04252]),
57	"{bounds:?}"
58	);
59	let stroke = &diamond.strokes[0];
60	assert!(close(
61	[stroke.points[0][0], stroke.points[0][1], 0.0, 0.0],
62	[451.50235, 84.755905, 0.0, 0.0]
63	));
64	assert!((stroke.width - 35.0 * 72.0 / 2540.0).abs() < 1e-5);
65	assert_eq!(
66	(stroke.color, stroke.transparency, stroke.pen_tip),
67	(None, None, None)
68	);
69	assert!(diamond.groups.is_empty());
70	for stroke in [&diamond.strokes[0], &line.strokes[0]] {
71	assert!(stroke.points.windows(2).all(\|pair\| {
72	let [dx, dy] = [pair[1][0] - pair[0][0], pair[1][1] - pair[0][1]];
73	dx.abs() < 3.0 && dy.abs() < 3.0
74	}));
75	}
76	}

crates/onestore/tests/page_model.rs+2-1

...	@@ -27,6 +27,7 @@ fn identities(page: &Page) -> Vec<onestore::ExGuid> {	...	@@ -27,6 +27,7 @@ fn identities(page: &Page) -> Vec<onestore::ExGuid> {
27	ids.push(paragraph.id);	27	ids.push(paragraph.id);
28	match &paragraph.content {	28	match &paragraph.content {
29	ParagraphContent::Text(text) => ids.push(text.id),	29	ParagraphContent::Text(text) => ids.push(text.id),
		30	ParagraphContent::Ink(ink) => ids.push(ink.id),
30	ParagraphContent::Table(table) => {	31	ParagraphContent::Table(table) => {
31	ids.push(table.id);	32	ids.push(table.id);
32	for row in &table.rows {	33	for row in &table.rows {
...	@@ -59,7 +60,7 @@ fn identities(page: &Page) -> Vec<onestore::ExGuid> {	...	@@ -59,7 +60,7 @@ fn identities(page: &Page) -> Vec<onestore::ExGuid> {
59	ids.extend(outline.unsupported.iter().map(\|u\| u.id));	60	ids.extend(outline.unsupported.iter().map(\|u\| u.id));
60	}	61	}
61	}	62	}
62	PageObject::Image(_) \| PageObject::Unsupported(_) => {}	63	PageObject::Image(_) \| PageObject::Ink(_) \| PageObject::Unsupported(_) => {}
63	}	64	}
64	}	65	}
65	ids	66	ids

crates/onestore/tests/page_tables.rs+1

...	@@ -74,6 +74,7 @@ fn referenced(page: &Page) -> std::collections::BTreeSet<ExGuid> {	...	@@ -74,6 +74,7 @@ fn referenced(page: &Page) -> std::collections::BTreeSet<ExGuid> {
74	}	74	}
75	ParagraphContent::Image(_)	75	ParagraphContent::Image(_)
76	\| ParagraphContent::Attachment(_)	76	\| ParagraphContent::Attachment(_)
		77	\| ParagraphContent::Ink(_)
77	\| ParagraphContent::Unsupported(_) => {}	78	\| ParagraphContent::Unsupported(_) => {}
78	}	79	}
79	}	80	}

crates/onestore/tests/support/page_edits.rs+2

...	@@ -129,6 +129,7 @@ fn projection(page: &Page) -> String {	...	@@ -129,6 +129,7 @@ fn projection(page: &Page) -> String {
129	ParagraphContent::Attachment(attachment) => {	129	ParagraphContent::Attachment(attachment) => {
130	out.push_str(&format!(" attachment {}", attachment.id))	130	out.push_str(&format!(" attachment {}", attachment.id))
131	}	131	}
		132	ParagraphContent::Ink(ink) => out.push_str(&format!(" ink {}", ink.id)),
132	ParagraphContent::Unsupported(u) => {	133	ParagraphContent::Unsupported(u) => {
133	out.push_str(&format!(" unsupported {}", u.id))	134	out.push_str(&format!(" unsupported {}", u.id))
134	}	135	}
...	@@ -138,6 +139,7 @@ fn projection(page: &Page) -> String {	...	@@ -138,6 +139,7 @@ fn projection(page: &Page) -> String {
138	}	139	}
139	PageObject::Title(title) => out.push_str(&format!("title {}\n", title.id)),	140	PageObject::Title(title) => out.push_str(&format!("title {}\n", title.id)),
140	PageObject::Image(image) => out.push_str(&format!("image {}\n", image.id)),	141	PageObject::Image(image) => out.push_str(&format!("image {}\n", image.id)),
		142	PageObject::Ink(ink) => out.push_str(&format!("ink {}\n", ink.id)),
141	PageObject::Unsupported(u) => out.push_str(&format!("unsupported {}\n", u.id)),	143	PageObject::Unsupported(u) => out.push_str(&format!("unsupported {}\n", u.id)),
142	}	144	}
143	}	145	}

tools/test_ink_read.py created+42

...	@@ -0,0 +1,42 @@
1	import json
2	from pathlib import Path
3	import runpy
4	import subprocess
5	from tempfile import TemporaryDirectory
6	import unittest
7	import xml.etree.ElementTree as ET
8
9	from document_model import EXPORTER, ordered_pages
10	from native_xml import ns
11
12	ROOT = Path(__file__).resolve().parent.parent
13	FIXTURE = ROOT / 'corpus/native-ink/cold-ui-ink'
14	ink_extent = runpy.run_path(str(ROOT / 'tools/verify-document.py'))['ink_extent']
15
16
17	class InkReadTest(unittest.TestCase):
18	def test_decoded_ink_strokes_span_the_native_drawing_extents(self):
19	"""An independent decoding of the stored stroke packets reproduces the native positions and sizes."""
20	with TemporaryDirectory() as temporary:
21	exported = Path(temporary) / 'document'
22	subprocess.run([EXPORTER, FIXTURE / 'notebook/synthetic.one', exported], check=True)
23	document = json.loads((exported / 'document.json').read_text())
24	(_, _, revision, page_id), = ordered_pages(document)
25	source_page = revision['nodes'][page_id]
26	native, = (ET.parse(path).getroot() for path in sorted((FIXTURE / 'read').glob('page-*.xml')))
27	drawings = native.findall('one:InkDrawing', ns)
28	self.assertEqual(len(drawings), 2)
29	for drawing in drawings:
30	z = int(drawing.find('one:Position', ns).get('z'))
31	source = revision['nodes'][source_page['children'][z]]
32	self.assertEqual(source['kind']['type'], 'Ink')
33	position, size = drawing.find('one:Position', ns), drawing.find('one:Size', ns)
34	expected = [float(position.get('x')), float(position.get('y')),
35	float(size.get('width')) - 72 / 2540, float(size.get('height')) - 72 / 2540]
36	extent = ink_extent(revision, source)
37	for actual, wanted in zip(extent, expected):
38	self.assertAlmostEqual(actual, wanted, delta=0.002)
39
40
41	if __name__ == '__main__':
42	unittest.main()

tools/verify-document.py+67

...	@@ -83,6 +83,58 @@ def visible_text(node, space):	...	@@ -83,6 +83,58 @@ def visible_text(node, space):
83	return "" if text == "\u00a0" else project_text(text.removesuffix('\r'))	83	return "" if text == "\u00a0" else project_text(text.removesuffix('\r'))
84		84
85		85
		86	ISF_X = bytes.fromhex('8f6a8a59c052a04b93afaf357411a561')
		87	ISF_Y = bytes.fromhex('759f3fb5e0049844a7eec30dbb5a9011')
		88
		89
		90	def multi_byte(data):
		91	"""ISF multi-byte signed integers: a count, then 7-bit little-endian varints with the sign in bit 0."""
		92	values = []
		93	index = 0
		94	while index < len(data):
		95	value = shift = 0
		96	while True:
		97	byte = data[index]
		98	index += 1
		99	value \|= (byte & 0x7f) << shift
		100	shift += 7
		101	if not byte & 0x80:
		102	break
		103	values.append(-(value >> 1) if value & 1 else value >> 1)
		104	count, values = values[0], values[1:]
		105	assert count == len(values), 'Ink packet count differs'
		106	return values
		107
		108
		109	def ink_extent(space, container):
		110	"""Every stroke point of an ink container in points, as [left, top, width, height]."""
		111	points = []
		112	scale = [container['kind']['scale_x'] or 1.0, container['kind']['scale_y'] or 1.0]
		113	for stroke_id in space['nodes'][container['kind']['data']]['kind']['strokes']:
		114	stroke = space['nodes'][stroke_id]['kind']
		115	style = space['nodes'][stroke['style']]['kind']
		116	dimensions = bytes(style['dimensions'])
		117	guids = [dimensions[i:i + 16] for i in range(0, len(dimensions), 32)]
		118	values = multi_byte(bytes(stroke['path']))
		119	per_dimension = len(values) // len(guids)
		120	axes = []
		121	for guid, factor in ((ISF_X, scale[0]), (ISF_Y, scale[1])):
		122	start = guids.index(guid) * per_dimension
		123	position, coordinates = 0, []
		124	for delta in values[start:start + per_dimension]:
		125	position += delta
		126	coordinates.append(position * factor * 72 / 2540)
		127	axes.append(coordinates)
		128	points.extend(zip(*axes))
		129	for child in container['content']:
		130	nested = space['nodes'][child]
		131	if nested['kind']['type'] == 'Ink':
		132	x, y, w, h = ink_extent(space, nested)
		133	points.extend([(x, y), (x + w, y + h)])
		134	xs, ys = [p[0] for p in points], [p[1] for p in points]
		135	return [min(xs), min(ys), max(xs) - min(xs), max(ys) - min(ys)]
		136
		137
86	def compare_objects(space, roots, page, native_roots, assets, native_payloads, autofit):	138	def compare_objects(space, roots, page, native_roots, assets, native_payloads, autofit):
87	types = {'T': 'RichText', 'Image': 'Image', 'InsertedFile': 'Attachment', 'MediaFile': 'Attachment', 'Table': 'Table'}	139	types = {'T': 'RichText', 'Image': 'Image', 'InsertedFile': 'Attachment', 'MediaFile': 'Attachment', 'Table': 'Table'}
88	actual = [n for root in roots for _, n in walk(space, root)	140	actual = [n for root in roots for _, n in walk(space, root)
...	@@ -280,6 +332,21 @@ def compare(notebook, native, versions=None, password_file=None):	...	@@ -280,6 +332,21 @@ def compare(notebook, native, versions=None, password_file=None):
280	z = int(position.attrib['z'])	332	z = int(position.attrib['z'])
281	source = space['nodes'][source_page['children'][z]]	333	source = space['nodes'][source_page['children'][z]]
282	if source['kind']['type'] == 'Ink':	334	if source['kind']['type'] == 'Ink':
		335	if child.tag == '{%s}InkDrawing' % ns['one']:
		336	# Stroke coordinates are absolute; left-to-right pages report them
		337	# relative to the canonical margin origin, right-to-left pages as stored.
		338	extent = ink_extent(space, source)
		339	if not source_page['kind']['rtl']:
		340	for index, (axis, canonical_origin) in enumerate((('x', 36.0), ('y', 14.4))):
		341	origin = source_page['kind']['margin_origin_' + axis]
		342	extent[index] += canonical_origin - origin if origin is not None else 0
		343	size = child.find('one:Size', ns)
		344	# Native sizes are one HIMETRIC unit larger than the point extent.
		345	reported = [float(position.attrib['x']), float(position.attrib['y']), float(size.get('width')) - 72 / 2540, float(size.get('height')) - 72 / 2540]
		346	for index, key in enumerate(('x', 'y', 'width', 'height')):
		347	if abs(extent[index] - reported[index]) > 0.01:
		348	geometry.append({'file': str(relative), 'page': ordinal, 'z': z, 'axis': key,
		349	'stored': extent[index], 'native': reported[index]})
283	continue	350	continue
284	for axis in ('x', 'y'):	351	for axis in ('x', 'y'):
285	if source['layout'][axis] is not None:	352	if source['layout'][axis] is not None:

...	@@ -458,6 +458,7 @@ impl TextOutline {	...	@@ -458,6 +458,7 @@ impl TextOutline {
458	}	458	}
459	onestore::page::ParagraphContent::Image(_)	459	onestore::page::ParagraphContent::Image(_)
460	\| onestore::page::ParagraphContent::Attachment(_)	460	\| onestore::page::ParagraphContent::Attachment(_)
		461	\| onestore::page::ParagraphContent::Ink(_)
461	\| onestore::page::ParagraphContent::Unsupported(_) => {	462	\| onestore::page::ParagraphContent::Unsupported(_) => {
462	return Err(EditError::UnsupportedContent);	463	return Err(EditError::UnsupportedContent);
463	}	464	}
...	@@ -509,6 +510,7 @@ impl TextOutline {	...	@@ -509,6 +510,7 @@ impl TextOutline {
509	}	510	}
510	onestore::page::ParagraphContent::Image(_)	511	onestore::page::ParagraphContent::Image(_)
511	\| onestore::page::ParagraphContent::Attachment(_)	512	\| onestore::page::ParagraphContent::Attachment(_)
		513	\| onestore::page::ParagraphContent::Ink(_)
512	\| onestore::page::ParagraphContent::Unsupported(_) => {	514	\| onestore::page::ParagraphContent::Unsupported(_) => {
513	return Err(EditError::UnsupportedContent);	515	return Err(EditError::UnsupportedContent);
514	}	516	}
...	@@ -1699,6 +1701,7 @@ impl CanvasEditor {	...	@@ -1699,6 +1701,7 @@ impl CanvasEditor {
1699	}	1701	}
1700	onestore::page::ParagraphContent::Image(_)	1702	onestore::page::ParagraphContent::Image(_)
1701	\| onestore::page::ParagraphContent::Attachment(_)	1703	\| onestore::page::ParagraphContent::Attachment(_)
		1704	\| onestore::page::ParagraphContent::Ink(_)
1702	\| onestore::page::ParagraphContent::Unsupported(_) => {	1705	\| onestore::page::ParagraphContent::Unsupported(_) => {
1703	return Err(EditError::UnsupportedContent.into());	1706	return Err(EditError::UnsupportedContent.into());
1704	}	1707	}

...	@@ -43,7 +43,7 @@ harness also accepts `--client-profile release`.	...	@@ -43,7 +43,7 @@ harness also accepts `--client-profile release`.
43	\| `Object::file_reference`, `Store::file_data` \| Identify internal/external payloads and read internal payload bytes \|	43	\| `Object::file_reference`, `Store::file_data` \| Identify internal/external payloads and read internal payload bytes \|
44	\| `document::Document`, `Revision::text_runs` \| Interpret document objects and inherited text formatting while retaining unknown properties and revision identities \|	44	\| `document::Document`, `Revision::text_runs` \| Interpret document objects and inherited text formatting while retaining unknown properties and revision identities \|
45	\| `Document::active`, `Document::pages_in`, `Revision::parents`, `RevisionIndex::active` \| Resolve the active revision, the pages of a space and parent links without repeating the lookups \|	45	\| `Document::active`, `Document::pages_in`, `Revision::parents`, `RevisionIndex::active` \| Resolve the active revision, the pages of a space and parent links without repeating the lookups \|
46	\| `page::Page`, `page::Paragraph` \| Build an editable page model (title, outlines, paragraphs with coalesced text spans, tables, images) with stored identities; content outside the model is retained as `Unsupported` \|	46	\| `page::Page`, `page::Paragraph`, `page::Ink` \| Build an editable page model (title, outlines, paragraphs with coalesced text spans, tables, images, attachments, ink drawings and handwriting decoded to stroke polylines in page points) with stored identities; content outside the model is retained as `Unsupported` \|
47	\| `protected::UnlockedSection` (optional feature) \| Own decoded buffers for explicit known-password inspection; clear those buffers on drop; derived document strings/exports remain caller-owned \|	47	\| `protected::UnlockedSection` (optional feature) \| Own decoded buffers for explicit known-password inspection; clear those buffers on drop; derived document strings/exports remain caller-owned \|
48	\| `create_section` \| Create one page containing one plain-text paragraph and an author, including Unicode \|	48	\| `create_section` \| Create one page containing one plain-text paragraph and an author, including Unicode \|
49	\| `PageCreation`, `PreparedEdit::create_page` \| Add an empty top-level page and its section entry atomically, retaining page identities across retries \|	49	\| `PageCreation`, `PreparedEdit::create_page` \| Add an empty top-level page and its section entry atomically, retaining page identities across retries \|
...	@@ -174,8 +174,8 @@ cargo run --example random_edit -- /tmp/one-demo/synthetic.one /tmp/edited.one 4	...	@@ -174,8 +174,8 @@ cargo run --example random_edit -- /tmp/one-demo/synthetic.one /tmp/edited.one 4
174	The report contains readable pages, document JSON, assets, source identities and	174	The report contains readable pages, document JSON, assets, source identities and
175	coordinates. It preserves paragraph nesting, lists, tables, links and tags.	175	coordinates. It preserves paragraph nesting, lists, tables, links and tags.
176	Historical contexts, recycle-bin pages and default templates are represented	176	Historical contexts, recycle-bin pages and default templates are represented
177	separately. Native ink and structured equations retain their source data and	177	separately. Native ink is decoded to strokes; structured equations retain their
178	appear explicitly as opaque content. The report is a reading view; its native	178	source data and appear explicitly as opaque content. The report is a reading view; its native
179	PDF references supply the original canvas layout.	179	PDF references supply the original canvas layout.
180		180
181	Read and validate a snapshot before interpreting its graph:	181	Read and validate a snapshot before interpreting its graph: