| author | |
| committer | |
| log | 18a89c7d5115a98335626904ed081efcded373bb |
| tree | 95effb331b6d64071530c8cbccf65105f3bcf606 |
| parent | 5ef7269b5da51f5c9173f07bc7a7587fe5f46c83 |
| signature | Signed by SSH key SHA256:52mNGHRsVFBDED9IAX5pe+LRWUefqTbxEReunq21QvU |
A guest's batch whose request was in flight when its connection dropped ends uncertain, as the host may or may not have committed it. The queue then held that batch until someone reviewed it, but nothing in the app offers the review, so unless the host had in fact committed it, the section published nothing more and took in no one else's changes for good. A Live Share host lands a batch at most once: it applies none whose revisions the section holds, or which it landed itself since it started, as OneNote's Optimize drops a revision a later one supersedes (the page-versions corpus shows it), and it seals any merge under the guest's names. So the guest now sends an uncertain batch again on its next step, whether the section is open or kept in sync from the background, and the host applies it or confirms the copy it already holds. The batch stays uncertain until the host takes it; a refusal while OneNote holds the section only waits for the next step. Other remotes, whose commits a late write could repeat, still never replay one. Tests lose a batch's request together with the guest's relay connection and check that it lands once the guest is back, after the host refused it once while the section was held, and from the background for a closed section; a unit test drops a landed batch's revision as Optimize does, after the host deleted what it typed, and checks that sending it again brings nothing back. The cold-restart test now expects a batch sealed before its process died to publish without review.
- A Live Share guest's edits sent just as its connection dropped now publish once it reconnects, instead of holding up every later edit in that section.
Assisted-by: claude-opus-5.511 files changed, 410 insertions(+), 70 deletions(-)
arc/sync.md+5-2| ... | @@ -128,8 +128,11 @@ section. | ... | @@ -128,8 +128,11 @@ section. |
| 128 | Uncertainty is handled explicitly. A publication attempt is recorded before | 128 | Uncertainty is handled explicitly. A publication attempt is recorded before |
| 129 | the first network byte. If the reply is lost, the attempt confirms only when | 129 | the first network byte. If the reply is lost, the attempt confirms only when |
| 130 | the remote holds its revisions (or every page it changed, as it changed them). | 130 | the remote holds its revisions (or every page it changed, as it changed them). |
| 131 | Otherwise it waits as `AwaitingConfirmation`, and `release` resolves it | 131 | Otherwise it waits as `AwaitingConfirmation`. A Live Share host lands a batch at |
| 132 | explicitly: it exports a recovery archive first, then republishes or abandons. | 132 | most once, as it looks for the batch's revisions before applying it, so the next |
| 133 | step sends it the same batch again. Any other remote, where a late write could land | ||
| 134 | a second copy, waits for `release`, which exports a recovery archive first, then | ||
| 135 | republishes or abandons. | ||
| 133 | 136 | ||
| 134 | ## Merging | 137 | ## Merging |
| 135 | 138 |
crates/notebook/README.md+5-3| ... | @@ -164,9 +164,11 @@ merges back. | ... | @@ -164,9 +164,11 @@ merges back. |
| 164 | 164 | ||
| 165 | Publication attempts are recorded before network I/O. An attempt confirms only | 165 | Publication attempts are recorded before network I/O. An attempt confirms only |
| 166 | when the remote holds its revisions, or every page it changed as it changed them, | 166 | when the remote holds its revisions, or every page it changed as it changed them, |
| 167 | followed by `Remote::confirm`; otherwise it stays `AwaitingConfirmation` and is never | 167 | followed by `Remote::confirm`; otherwise it stays `AwaitingConfirmation`. A remote that |
| 168 | replayed. `release(id, archive, Resolution)` exports the queue first, then | 168 | accepts edits (a Live Share host) lands a batch at most once, so the next step sends it the |
| 169 | publishes the batch again (`Mine`) or abandons every unpublished edit (`Theirs`). | 169 | same batch again; any other remote never gets it again unreviewed. |
| 170 | `release(id, archive, Resolution)` exports the queue first, then publishes the batch again | ||
| 171 | (`Mine`) or abandons every unpublished edit (`Theirs`). | ||
| 170 | A durable `Published` receipt survives reopening. | 172 | A durable `Published` receipt survives reopening. |
| 171 | 173 | ||
| 172 | A schema-14 cache is converted when opened: it is exported to | 174 | A schema-14 cache is converted when opened: it is exported to |
crates/notebook/src/background.rs+13| ... | @@ -1105,6 +1105,19 @@ impl<R: Remote> Remote for Discovered<'_, R> { | ... | @@ -1105,6 +1105,19 @@ impl<R: Remote> Remote for Discovered<'_, R> { |
| 1105 | self.remote.confirm(base) | 1105 | self.remote.confirm(base) |
| 1106 | } | 1106 | } |
| 1107 | 1107 | ||
| 1108 | fn accepts_edits(&self) -> bool { | ||
| 1109 | self.remote.accepts_edits() | ||
| 1110 | } | ||
| 1111 | |||
| 1112 | fn publish_edits( | ||
| 1113 | &mut self, | ||
| 1114 | transaction: &onestore::Transaction, | ||
| 1115 | edits: &[crate::PendingEdit], | ||
| 1116 | revisions: &std::collections::BTreeMap<onestore::ExGuid, onestore::ExGuid>, | ||
| 1117 | ) -> std::result::Result<(), onestore::CommitError> { | ||
| 1118 | self.remote.publish_edits(transaction, edits, revisions) | ||
| 1119 | } | ||
| 1120 | |||
| 1108 | fn versions(&mut self) -> io::Result<Vec<crate::Version>> { | 1121 | fn versions(&mut self) -> io::Result<Vec<crate::Version>> { |
| 1109 | self.remote.versions() | 1122 | self.remote.versions() |
| 1110 | } | 1123 | } |
crates/notebook/src/live/share.rs+8-1| ... | @@ -14,7 +14,7 @@ use crate::{Error, Result, background::Reports, discover, session::Storage}; | ... | @@ -14,7 +14,7 @@ use crate::{Error, Result, background::Reports, discover, session::Storage}; |
| 14 | use onestore::{CommitError, CommitState, RevisionIndex, Stamp, Store, Transaction}; | 14 | use onestore::{CommitError, CommitState, RevisionIndex, Stamp, Store, Transaction}; |
| 15 | use sha2::{Digest, Sha256}; | 15 | use sha2::{Digest, Sha256}; |
| 16 | use std::{ | 16 | use std::{ |
| 17 | collections::{BTreeMap, HashMap}, | 17 | collections::{BTreeMap, HashMap, HashSet}, |
| 18 | io, | 18 | io, |
| 19 | path::PathBuf, | 19 | path::PathBuf, |
| 20 | sync::{ | 20 | sync::{ |
| ... | @@ -294,6 +294,9 @@ struct Served { | ... | @@ -294,6 +294,9 @@ struct Served { |
| 294 | puts: Mutex<ByGuest<Vec<u8>>>, | 294 | puts: Mutex<ByGuest<Vec<u8>>>, |
| 295 | guests: Mutex<BTreeMap<[u8; 16], Admitted>>, | 295 | guests: Mutex<BTreeMap<[u8; 16], Admitted>>, |
| 296 | writers: Mutex<HashMap<String, mpsc::SyncSender<batch::Waiting>>>, | 296 | writers: Mutex<HashMap<String, mpsc::SyncSender<batch::Waiting>>>, |
| 297 | /// The revisions guests' batches landed as, by page: OneNote's Optimize drops one a | ||
| 298 | /// later revision supersedes, and a batch sent again must still find it. | ||
| 299 | landed: Mutex<HashSet<(onestore::ExGuid, onestore::ExGuid)>>, | ||
| 297 | /// Hears the paths guests changed, as the host's own notebook should. | 300 | /// Hears the paths guests changed, as the host's own notebook should. |
| 298 | host: Mutex<Option<crate::session::Listener>>, | 301 | host: Mutex<Option<crate::session::Listener>>, |
| 299 | /// The share's room, to tell guests what changed. | 302 | /// The share's room, to tell guests what changed. |
| ... | @@ -1848,6 +1851,8 @@ pub struct HostedRemote { | ... | @@ -1848,6 +1851,8 @@ pub struct HostedRemote { |
| 1848 | path: String, | 1851 | path: String, |
| 1849 | /// The stamp last asked for, which an image the guest holds may already have. | 1852 | /// The stamp last asked for, which an image the guest holds may already have. |
| 1850 | seen: Option<Stamp>, | 1853 | seen: Option<Stamp>, |
| 1854 | /// The host refused the last batch, so the next step reads its image before sending | ||
| 1855 | /// edits. A stamp clears it, as an uncertain batch goes again regardless. | ||
| 1851 | rejected: bool, | 1856 | rejected: bool, |
| 1852 | #[cfg(target_arch = "wasm32")] | 1857 | #[cfg(target_arch = "wasm32")] |
| 1853 | pending: Option<web::Pending>, | 1858 | pending: Option<web::Pending>, |
| ... | @@ -1914,6 +1919,7 @@ impl crate::Remote for HostedRemote { | ... | @@ -1914,6 +1919,7 @@ impl crate::Remote for HostedRemote { |
| 1914 | } | 1919 | } |
| 1915 | 1920 | ||
| 1916 | fn stamp(&mut self) -> io::Result<Stamp> { | 1921 | fn stamp(&mut self) -> io::Result<Stamp> { |
| 1922 | self.rejected = false; | ||
| 1917 | let stamp = self.guest.stamp(&self.path)?; | 1923 | let stamp = self.guest.stamp(&self.path)?; |
| 1918 | self.seen = Some(stamp.clone()); | 1924 | self.seen = Some(stamp.clone()); |
| 1919 | Ok(stamp) | 1925 | Ok(stamp) |
| ... | @@ -2159,6 +2165,7 @@ mod tests { | ... | @@ -2159,6 +2165,7 @@ mod tests { |
| 2159 | puts: Mutex::default(), | 2165 | puts: Mutex::default(), |
| 2160 | guests: Mutex::default(), | 2166 | guests: Mutex::default(), |
| 2161 | writers: Mutex::default(), | 2167 | writers: Mutex::default(), |
| 2168 | landed: Mutex::default(), | ||
| 2162 | host: Mutex::default(), | 2169 | host: Mutex::default(), |
| 2163 | room: Mutex::default(), | 2170 | room: Mutex::default(), |
| 2164 | }); | 2171 | }); |
crates/notebook/src/live/share/batch.rs+140-9| ... | @@ -114,9 +114,10 @@ impl Served { | ... | @@ -114,9 +114,10 @@ impl Served { |
| 114 | let mut image = Arc::clone(&before); | 114 | let mut image = Arc::clone(&before); |
| 115 | let mut combined: Option<Transaction> = None; | 115 | let mut combined: Option<Transaction> = None; |
| 116 | let mut accepted = Vec::new(); | 116 | let mut accepted = Vec::new(); |
| 117 | let mut landing = Vec::new(); | ||
| 117 | for batch in waiting { | 118 | for batch in waiting { |
| 118 | match self.prepare_batch(path, &image, &batch) { | 119 | match self.prepare_batch(path, &image, &batch) { |
| 119 | Ok(Some(transaction)) => { | 120 | Ok((Some(transaction), revisions)) => { |
| 120 | let mut next = (*image).clone(); | 121 | let mut next = (*image).clone(); |
| 121 | let extended = | 122 | let extended = |
| 122 | transaction | 123 | transaction |
| ... | @@ -131,6 +132,7 @@ impl Served { | ... | @@ -131,6 +132,7 @@ impl Served { |
| 131 | match extended { | 132 | match extended { |
| 132 | Ok(()) => { | 133 | Ok(()) => { |
| 133 | image = Arc::new(next); | 134 | image = Arc::new(next); |
| 135 | landing.extend(revisions); | ||
| 134 | accepted.push(batch); | 136 | accepted.push(batch); |
| 135 | } | 137 | } |
| 136 | Err(error) => { | 138 | Err(error) => { |
| ... | @@ -138,7 +140,7 @@ impl Served { | ... | @@ -138,7 +140,7 @@ impl Served { |
| 138 | } | 140 | } |
| 139 | } | 141 | } |
| 140 | } | 142 | } |
| 141 | Ok(None) => accepted.push(batch), | 143 | Ok((None, _)) => accepted.push(batch), |
| 142 | Err(error) => { | 144 | Err(error) => { |
| 143 | let _ = batch.reply.send(failed(0, error)); | 145 | let _ = batch.reply.send(failed(0, error)); |
| 144 | } | 146 | } |
| ... | @@ -166,6 +168,7 @@ impl Served { | ... | @@ -166,6 +168,7 @@ impl Served { |
| 166 | }), | 168 | }), |
| 167 | }; | 169 | }; |
| 168 | if result.is_ok() { | 170 | if result.is_ok() { |
| 171 | self.landed.lock().unwrap().extend(landing); | ||
| 169 | self.tell_delta(path, &before, &image, None); | 172 | self.tell_delta(path, &before, &image, None); |
| 170 | self.keep(path, Arc::clone(&image)); | 173 | self.keep(path, Arc::clone(&image)); |
| 171 | self.changed(&[path.to_owned()]); | 174 | self.changed(&[path.to_owned()]); |
| ... | @@ -194,7 +197,7 @@ impl Served { | ... | @@ -194,7 +197,7 @@ impl Served { |
| 194 | path: &str, | 197 | path: &str, |
| 195 | image: &[u8], | 198 | image: &[u8], |
| 196 | batch: &Waiting, | 199 | batch: &Waiting, |
| 197 | ) -> Result<Option<Transaction>> { | 200 | ) -> Result<(Option<Transaction>, BTreeMap<ExGuid, ExGuid>)> { |
| 198 | if !self.guests.lock().unwrap().contains_key(&batch.peer) { | 201 | if !self.guests.lock().unwrap().contains_key(&batch.peer) { |
| 199 | return Err(refused( | 202 | return Err(refused( |
| 200 | io::ErrorKind::PermissionDenied, | 203 | io::ErrorKind::PermissionDenied, |
| ... | @@ -219,14 +222,15 @@ impl Served { | ... | @@ -219,14 +222,15 @@ impl Served { |
| 219 | } | 222 | } |
| 220 | let store = Store::parse(image)?; | 223 | let store = Store::parse(image)?; |
| 221 | let index = RevisionIndex::parse(&store)?; | 224 | let index = RevisionIndex::parse(&store)?; |
| 225 | let landed = self.landed.lock().unwrap(); | ||
| 222 | if edits.revisions.iter().all(|(space, revision)| { | 226 | if edits.revisions.iter().all(|(space, revision)| { |
| 223 | index | 227 | landed.contains(&(*space, *revision)) |
| 224 | .spaces | 228 | || (index.spaces.get(space)) |
| 225 | .get(space) | 229 | .is_some_and(|space| space.revisions.contains_key(revision)) |
| 226 | .is_some_and(|space| space.revisions.contains_key(revision)) | ||
| 227 | }) { | 230 | }) { |
| 228 | return Ok(None); | 231 | return Ok((None, edits.revisions)); |
| 229 | } | 232 | } |
| 233 | drop(landed); | ||
| 230 | let names = edits | 234 | let names = edits |
| 231 | .revisions | 235 | .revisions |
| 232 | .iter() | 236 | .iter() |
| ... | @@ -306,7 +310,7 @@ impl Served { | ... | @@ -306,7 +310,7 @@ impl Served { |
| 306 | "The batch needs a guest's commit", | 310 | "The batch needs a guest's commit", |
| 307 | )); | 311 | )); |
| 308 | } | 312 | } |
| 309 | Ok(transaction) | 313 | Ok((transaction, edits.revisions)) |
| 310 | } | 314 | } |
| 311 | } | 315 | } |
| 312 | 316 | ||
| ... | @@ -333,6 +337,7 @@ mod tests { | ... | @@ -333,6 +337,7 @@ mod tests { |
| 333 | puts: Mutex::default(), | 337 | puts: Mutex::default(), |
| 334 | guests: Mutex::default(), | 338 | guests: Mutex::default(), |
| 335 | writers: Mutex::default(), | 339 | writers: Mutex::default(), |
| 340 | landed: Mutex::default(), | ||
| 336 | host: Mutex::default(), | 341 | host: Mutex::default(), |
| 337 | room: Mutex::default(), | 342 | room: Mutex::default(), |
| 338 | }); | 343 | }); |
| ... | @@ -461,6 +466,132 @@ mod tests { | ... | @@ -461,6 +466,132 @@ mod tests { |
| 461 | assert_eq!(served.writers.lock().unwrap().len(), 1); | 466 | assert_eq!(served.writers.lock().unwrap().len(), 1); |
| 462 | } | 467 | } |
| 463 | 468 | ||
| 469 | /// OneNote's Optimize drops a revision a later one superseded, as the page-versions | ||
| 470 | /// corpus shows; a batch sent again after its revision went that way is not applied | ||
| 471 | /// again. | ||
| 472 | #[cfg(not(target_arch = "wasm32"))] | ||
| 473 | #[test] | ||
| 474 | fn a_batch_whose_revision_optimize_dropped_lands_once() { | ||
| 475 | let directory = tempfile::tempdir().unwrap(); | ||
| 476 | let folder = directory.path().join("notebook"); | ||
| 477 | std::fs::create_dir(&folder).unwrap(); | ||
| 478 | let path = "Garden.one"; | ||
| 479 | let base = onestore::create_section(path, "Original text", "Fixture").unwrap(); | ||
| 480 | std::fs::write(folder.join(path), &base).unwrap(); | ||
| 481 | let served = Arc::new(Served { | ||
| 482 | storage: crate::session::Notebook::open(&folder, directory.path().join("cache")) | ||
| 483 | .unwrap() | ||
| 484 | .into_storage(), | ||
| 485 | images: Mutex::default(), | ||
| 486 | snapshots: Mutex::default(), | ||
| 487 | puts: Mutex::default(), | ||
| 488 | guests: Mutex::default(), | ||
| 489 | writers: Mutex::default(), | ||
| 490 | landed: Mutex::default(), | ||
| 491 | host: Mutex::default(), | ||
| 492 | room: Mutex::default(), | ||
| 493 | }); | ||
| 494 | let peer = [1; 16]; | ||
| 495 | let (line, _outbound) = mpsc::channel(); | ||
| 496 | let (queue, _requests) = mpsc::sync_channel(1); | ||
| 497 | served.guests.lock().unwrap().insert( | ||
| 498 | peer, | ||
| 499 | Admitted { | ||
| 500 | hello: Arc::new(Hello { | ||
| 501 | peer, | ||
| 502 | ..Hello::new("Guest".into(), None).unwrap() | ||
| 503 | }), | ||
| 504 | line: crate::live::Line(line), | ||
| 505 | queue, | ||
| 506 | starts: BURST, | ||
| 507 | counted: Instant::now(), | ||
| 508 | held: Vec::new(), | ||
| 509 | }, | ||
| 510 | ); | ||
| 511 | let store = Store::parse(&base).unwrap(); | ||
| 512 | let index = RevisionIndex::parse(&store).unwrap(); | ||
| 513 | let document = onestore::document::Document::parse(&index).unwrap(); | ||
| 514 | let (space, text) = document | ||
| 515 | .spaces | ||
| 516 | .iter() | ||
| 517 | .find_map(|(space, object)| { | ||
| 518 | object.revisions[&object.contexts[&ExGuid::default()]] | ||
| 519 | .nodes | ||
| 520 | .iter() | ||
| 521 | .find_map(|(id, node)| { | ||
| 522 | matches!(node.kind, onestore::document::Kind::RichText { .. }) | ||
| 523 | .then_some((*space, *id)) | ||
| 524 | }) | ||
| 525 | }) | ||
| 526 | .unwrap(); | ||
| 527 | let typed = |range: std::ops::Range<u32>, with: &str| Edit { | ||
| 528 | at: 134_000_000_000_000_000, | ||
| 529 | ops: vec![Op::Page { | ||
| 530 | space, | ||
| 531 | op: PageOp::Text { | ||
| 532 | text, | ||
| 533 | range, | ||
| 534 | with: with.into(), | ||
| 535 | }, | ||
| 536 | }], | ||
| 537 | }; | ||
| 538 | let arena = Arena::default(); | ||
| 539 | let mut section = Section::open(&arena, base.clone()).unwrap(); | ||
| 540 | section.apply("Guest", &typed(13..13, "X")).unwrap(); | ||
| 541 | let transaction = section.seal().unwrap().unwrap(); | ||
| 542 | let request = Request { | ||
| 543 | path: path.into(), | ||
| 544 | stamp: Some(transaction.base().into()), | ||
| 545 | bytes: Some( | ||
| 546 | encode( | ||
| 547 | &Edits { | ||
| 548 | edits: vec![("Guest".into(), typed(13..13, "X"))], | ||
| 549 | revisions: section.newest().filter(|(at, _)| *at == space).collect(), | ||
| 550 | }, | ||
| 551 | LIMIT, | ||
| 552 | ) | ||
| 553 | .unwrap(), | ||
| 554 | ), | ||
| 555 | ..Request::default() | ||
| 556 | }; | ||
| 557 | assert!( | ||
| 558 | served | ||
| 559 | .batch(&peer, request.clone()) | ||
| 560 | .unwrap() | ||
| 561 | .failure | ||
| 562 | .is_none() | ||
| 563 | ); | ||
| 564 | |||
| 565 | // The host deletes what the batch typed, and Optimize keeps the page's newest | ||
| 566 | // revision alone: the file holds neither the text nor the batch's revision. | ||
| 567 | let arena = Arena::default(); | ||
| 568 | let mut optimized = Section::open(&arena, base.clone()).unwrap(); | ||
| 569 | optimized.apply("Guest", &typed(13..13, "X")).unwrap(); | ||
| 570 | optimized.apply("Host", &typed(13..14, "")).unwrap(); | ||
| 571 | let mut image = base; | ||
| 572 | optimized | ||
| 573 | .seal() | ||
| 574 | .unwrap() | ||
| 575 | .unwrap() | ||
| 576 | .apply(&mut image) | ||
| 577 | .unwrap(); | ||
| 578 | std::fs::write(folder.join(path), &image).unwrap(); | ||
| 579 | let replayed = served.batch(&peer, request).unwrap(); | ||
| 580 | assert!(replayed.failure.is_none(), "{:?}", replayed.failure); | ||
| 581 | let stored = std::fs::read(folder.join(path)).unwrap(); | ||
| 582 | let arena = Arena::default(); | ||
| 583 | let section = Section::open(&arena, stored).unwrap(); | ||
| 584 | let texts: Vec<String> = (section.page(space).unwrap().objects.iter()) | ||
| 585 | .filter_map(|object| match object { | ||
| 586 | PageObject::Outline(outline) => Some(outline.paragraphs.iter()), | ||
| 587 | _ => None, | ||
| 588 | }) | ||
| 589 | .flatten() | ||
| 590 | .filter_map(|paragraph| Some(paragraph.text()?.text.text().to_owned())) | ||
| 591 | .collect(); | ||
| 592 | assert_eq!(texts, ["Original text"]); | ||
| 593 | } | ||
| 594 | |||
| 464 | #[test] | 595 | #[test] |
| 465 | fn attachment_encoding_stops_at_the_upload_budget() { | 596 | fn attachment_encoding_stops_at_the_upload_budget() { |
| 466 | let id = ExGuid { | 597 | let id = ExGuid { |
crates/notebook/src/live/share/handover_tests.rs+1| ... | @@ -438,6 +438,7 @@ fn served() -> (tempfile::TempDir, Arc<Served>) { | ... | @@ -438,6 +438,7 @@ fn served() -> (tempfile::TempDir, Arc<Served>) { |
| 438 | puts: Mutex::default(), | 438 | puts: Mutex::default(), |
| 439 | guests: Mutex::default(), | 439 | guests: Mutex::default(), |
| 440 | writers: Mutex::default(), | 440 | writers: Mutex::default(), |
| 441 | landed: Mutex::default(), | ||
| 441 | host: Mutex::default(), | 442 | host: Mutex::default(), |
| 442 | room: Mutex::default(), | 443 | room: Mutex::default(), |
| 443 | }); | 444 | }); |
crates/notebook/src/live/share/membership.rs+1| ... | @@ -52,6 +52,7 @@ impl Host { | ... | @@ -52,6 +52,7 @@ impl Host { |
| 52 | puts: Mutex::default(), | 52 | puts: Mutex::default(), |
| 53 | guests: Mutex::default(), | 53 | guests: Mutex::default(), |
| 54 | writers: Mutex::default(), | 54 | writers: Mutex::default(), |
| 55 | landed: Mutex::default(), | ||
| 55 | host: Mutex::default(), | 56 | host: Mutex::default(), |
| 56 | room: Mutex::default(), | 57 | room: Mutex::default(), |
| 57 | }), | 58 | }), |
crates/notebook/src/live/share/web.rs+1| ... | @@ -90,6 +90,7 @@ impl crate::Remote for HostedRemote { | ... | @@ -90,6 +90,7 @@ impl crate::Remote for HostedRemote { |
| 90 | } | 90 | } |
| 91 | 91 | ||
| 92 | fn stamp(&mut self) -> io::Result<Stamp> { | 92 | fn stamp(&mut self) -> io::Result<Stamp> { |
| 93 | self.rejected = false; | ||
| 93 | let (guest, path) = (self.guest.clone(), self.path.clone()); | 94 | let (guest, path) = (self.guest.clone(), self.path.clone()); |
| 94 | let result = self | 95 | let result = self |
| 95 | .operation(async move { | 96 | .operation(async move { |
crates/notebook/src/sync.rs+36-3| ... | @@ -27,6 +27,8 @@ pub trait Remote { | ... | @@ -27,6 +27,8 @@ pub trait Remote { |
| 27 | fn accepts_edits(&self) -> bool { | 27 | fn accepts_edits(&self) -> bool { |
| 28 | false | 28 | false |
| 29 | } | 29 | } |
| 30 | /// Where the remote accepts edits, a batch published again with the same revisions, | ||
| 31 | /// however its first attempt ended, lands at most once. | ||
| 30 | fn publish_edits( | 32 | fn publish_edits( |
| 31 | &mut self, | 33 | &mut self, |
| 32 | transaction: &Transaction, | 34 | transaction: &Transaction, |
| ... | @@ -223,7 +225,7 @@ impl Replica { | ... | @@ -223,7 +225,7 @@ impl Replica { |
| 223 | /// Publishes the oldest unpublished batch, rebasing the queue first when the remote | 225 | /// Publishes the oldest unpublished batch, rebasing the queue first when the remote |
| 224 | /// changed. Reads the remote image only when its stamp moved; network I/O holds | 226 | /// changed. Reads the remote image only when its stamp moved; network I/O holds |
| 225 | /// synchronization ownership without holding the cache mutex. Uncertain attempts are | 227 | /// synchronization ownership without holding the cache mutex. Uncertain attempts are |
| 226 | /// never replayed. | 228 | /// replayed only to a remote that accepts edits. |
| 227 | /// Versions the remote keeps beside the file merge into it first, each published as one | 229 | /// Versions the remote keeps beside the file merge into it first, each published as one |
| 228 | /// more revision and then retired (`resolve.rs`). | 230 | /// more revision and then retired (`resolve.rs`). |
| 229 | pub fn sync_once(&self, remote: &mut impl Remote) -> Result<Synced> { | 231 | pub fn sync_once(&self, remote: &mut impl Remote) -> Result<Synced> { |
| ... | @@ -281,6 +283,19 @@ impl Replica { | ... | @@ -281,6 +283,19 @@ impl Replica { |
| 281 | let observed = awaited(remote, Remote::stamp) | 283 | let observed = awaited(remote, Remote::stamp) |
| 282 | .await | 284 | .await |
| 283 | .map_err(Error::RemoteIo)?; | 285 | .map_err(Error::RemoteIo)?; |
| 286 | let replay = match &state.blocked { | ||
| 287 | Some(blocked) if self.section.key.is_none() && remote.accepts_edits() => { | ||
| 288 | working::sealed(&*self.lock()?)? | ||
| 289 | .filter(|sealed| sealed.batch == blocked.batch) | ||
| 290 | .and_then(|sealed| sealed.transaction) | ||
| 291 | } | ||
| 292 | _ => None, | ||
| 293 | }; | ||
| 294 | if let (Some(blocked), Some(transaction)) = (&state.blocked, replay) { | ||
| 295 | return self | ||
| 296 | .publish_sealed(remote, blocked.batch, transaction, true, true, Vec::new()) | ||
| 297 | .await; | ||
| 298 | } | ||
| 284 | if let Some(blocked) = &state.blocked | 299 | if let Some(blocked) = &state.blocked |
| 285 | && observed == *state.remote.as_ref().unwrap_or(&state.base) | 300 | && observed == *state.remote.as_ref().unwrap_or(&state.base) |
| 286 | { | 301 | { |
| ... | @@ -419,8 +434,8 @@ impl Replica { | ... | @@ -419,8 +434,8 @@ impl Replica { |
| 419 | changed, | 434 | changed, |
| 420 | }); | 435 | }); |
| 421 | }; | 436 | }; |
| 422 | let id = newest(&*self.lock()?, batch)?; | ||
| 423 | let Some(transaction) = transaction else { | 437 | let Some(transaction) = transaction else { |
| 438 | let id = newest(&*self.lock()?, batch)?; | ||
| 424 | // Edits that changed nothing are published once the remote's image is durable. | 439 | // Edits that changed nothing are published once the remote's image is durable. |
| 425 | let base = base::base_stamp(&*self.lock()?)?; | 440 | let base = base::base_stamp(&*self.lock()?)?; |
| 426 | if let Err(error) = awaited(remote, |remote| remote.confirm(&base)).await { | 441 | if let Err(error) = awaited(remote, |remote| remote.confirm(&base)).await { |
| ... | @@ -441,6 +456,24 @@ impl Replica { | ... | @@ -441,6 +456,24 @@ impl Replica { |
| 441 | .execute("UPDATE batches SET attempted=0 WHERE id=?1", [batch])?; | 456 | .execute("UPDATE batches SET attempted=0 WHERE id=?1", [batch])?; |
| 442 | return Err(error.into()); | 457 | return Err(error.into()); |
| 443 | } | 458 | } |
| 459 | self.publish_sealed(remote, batch, transaction, batched, false, changed) | ||
| 460 | .await | ||
| 461 | } | ||
| 462 | |||
| 463 | /// Publishes the attempted `batch` sealed as `transaction`, rebasing on the remote's | ||
| 464 | /// image after where `batched`; `changed` holds the pages a rebase already changed. An | ||
| 465 | /// attempt made `again` stays uncertain until the remote takes it, as the first may | ||
| 466 | /// still land. | ||
| 467 | async fn publish_sealed( | ||
| 468 | &self, | ||
| 469 | remote: &mut impl Remote, | ||
| 470 | batch: i64, | ||
| 471 | transaction: Transaction, | ||
| 472 | batched: bool, | ||
| 473 | again: bool, | ||
| 474 | mut changed: Vec<ExGuid>, | ||
| 475 | ) -> Result<Synced> { | ||
| 476 | let id = newest(&*self.lock()?, batch)?; | ||
| 444 | let published = if self.section.key.is_none() && remote.accepts_edits() { | 477 | let published = if self.section.key.is_none() && remote.accepts_edits() { |
| 445 | let (edits, revisions) = { | 478 | let (edits, revisions) = { |
| 446 | let connection = self.lock()?; | 479 | let connection = self.lock()?; |
| ... | @@ -461,7 +494,7 @@ impl Replica { | ... | @@ -461,7 +494,7 @@ impl Replica { |
| 461 | }; | 494 | }; |
| 462 | match published { | 495 | match published { |
| 463 | Ok(()) => {} | 496 | Ok(()) => {} |
| 464 | Err(error) if error.state == CommitState::NotCommitted => { | 497 | Err(error) if error.state == CommitState::NotCommitted && !again => { |
| 465 | self.lock()? | 498 | self.lock()? |
| 466 | .execute("UPDATE batches SET attempted=0 WHERE id=?1", [batch])?; | 499 | .execute("UPDATE batches SET attempted=0 WHERE id=?1", [batch])?; |
| 467 | return Err(error.into()); | 500 | return Err(error.into()); |
crates/notebook/tests/live_batch.rs+192-51| ... | @@ -207,34 +207,6 @@ fn edit_receipts_survive_guest_and_host_process_death() { | ... | @@ -207,34 +207,6 @@ fn edit_receipts_survive_guest_and_host_process_death() { |
| 207 | stopped_at_boundary(&root); | 207 | stopped_at_boundary(&root); |
| 208 | } | 208 | } |
| 209 | let mut remote = HostedRemote::new(&guest, "Garden.one"); | 209 | let mut remote = HostedRemote::new(&guest, "Garden.one"); |
| 210 | if boundary == "unsent" { | ||
| 211 | let attempted = replica.status(id).unwrap(); | ||
| 212 | assert!(matches!( | ||
| 213 | attempted, | ||
| 214 | Some(EditStatus::AwaitingConfirmation { .. }) | ||
| 215 | )); | ||
| 216 | let pending = replica.pending().unwrap(); | ||
| 217 | let snapshot = replica.snapshot().unwrap(); | ||
| 218 | let original = std::fs::read(root.join("Garden/Garden.one")).unwrap(); | ||
| 219 | for _ in 0..3 { | ||
| 220 | replica.sync_once(&mut remote).unwrap(); | ||
| 221 | assert_eq!(replica.status(id).unwrap(), attempted); | ||
| 222 | assert_eq!(replica.pending().unwrap(), pending); | ||
| 223 | assert_eq!(replica.snapshot().unwrap(), snapshot); | ||
| 224 | assert_eq!( | ||
| 225 | std::fs::read(root.join("Garden/Garden.one")).unwrap(), | ||
| 226 | original | ||
| 227 | ); | ||
| 228 | } | ||
| 229 | let archive = root.join("before-review.sqlite"); | ||
| 230 | replica | ||
| 231 | .release(id, &archive, notebook::Resolution::Mine) | ||
| 232 | .unwrap(); | ||
| 233 | let archive = notebook::Recovery::open(archive).unwrap(); | ||
| 234 | assert_eq!(archive.status(id).unwrap(), attempted); | ||
| 235 | assert_eq!(archive.pending().unwrap(), pending); | ||
| 236 | assert_eq!(archive.snapshot().unwrap(), snapshot); | ||
| 237 | } | ||
| 238 | for _ in 0..5 { | 210 | for _ in 0..5 { |
| 239 | replica.sync_once(&mut remote).unwrap(); | 211 | replica.sync_once(&mut remote).unwrap(); |
| 240 | if matches!( | 212 | if matches!( |
| ... | @@ -421,29 +393,6 @@ fn edit_receipts_survive_guest_and_host_process_death() { | ... | @@ -421,29 +393,6 @@ fn edit_receipts_survive_guest_and_host_process_death() { |
| 421 | assert!(replica.pending().unwrap().is_empty()); | 393 | assert!(replica.pending().unwrap().is_empty()); |
| 422 | assert_eq!(replica.snapshot().unwrap(), result); | 394 | assert_eq!(replica.snapshot().unwrap(), result); |
| 423 | drop(replica); | 395 | drop(replica); |
| 424 | if boundary == "unsent" { | ||
| 425 | let archive = notebook::Recovery::open(root.join("before-review.sqlite")).unwrap(); | ||
| 426 | assert!(matches!( | ||
| 427 | archive.status(id).unwrap(), | ||
| 428 | Some(EditStatus::AwaitingConfirmation { .. }) | ||
| 429 | )); | ||
| 430 | assert_eq!(archive.pending().unwrap()[0].id, id); | ||
| 431 | assert_eq!(archive.pending().unwrap().len(), 1); | ||
| 432 | assert_eq!( | ||
| 433 | body(&archive.snapshot().unwrap()).2, | ||
| 434 | format!("{content}{INSERTION}") | ||
| 435 | ); | ||
| 436 | let database = rusqlite::Connection::open(root.join("before-review.sqlite")).unwrap(); | ||
| 437 | assert_eq!( | ||
| 438 | database | ||
| 439 | .query_row("PRAGMA integrity_check", [], |row| row.get::<_, String>(0)) | ||
| 440 | .unwrap(), | ||
| 441 | "ok" | ||
| 442 | ); | ||
| 443 | println!( | ||
| 444 | "unsent: cold uncertainty retained; explicit Mine review archived the original queue before publication" | ||
| 445 | ); | ||
| 446 | } | ||
| 447 | let database = rusqlite::Connection::open(&database).unwrap(); | 396 | let database = rusqlite::Connection::open(&database).unwrap(); |
| 448 | assert_eq!( | 397 | assert_eq!( |
| 449 | database | 398 | database |
| ... | @@ -535,3 +484,195 @@ fn concurrent_batches_keep_receipts_and_rebase_later_keystrokes() { | ... | @@ -535,3 +484,195 @@ fn concurrent_batches_keep_receipts_and_rebase_later_keystrokes() { |
| 535 | let mut section = onestore::Section::open(&arena, result).unwrap(); | 484 | let mut section = onestore::Section::open(&arena, result).unwrap(); |
| 536 | assert!(section.conflicts().unwrap().is_empty()); | 485 | assert!(section.conflicts().unwrap().is_empty()); |
| 537 | } | 486 | } |
| 487 | |||
| 488 | /// A batch whose request the relay loses with the guest's connection is uncertain; once the | ||
| 489 | /// guest is back, the host hears it again and it lands once. | ||
| 490 | #[test] | ||
| 491 | fn a_batch_lost_on_the_way_lands_once_the_guest_is_back() { | ||
| 492 | use notebook::{EditStatus, Error, Remote, live::share::Guest}; | ||
| 493 | use onestore::{CommitError, CommitState}; | ||
| 494 | |||
| 495 | let directory = tempfile::tempdir().unwrap(); | ||
| 496 | let folder = notebook(directory.path()); | ||
| 497 | let url = relay(Default::default()); | ||
| 498 | let sharing = Sharing::new("").unwrap(); | ||
| 499 | let host = host(&folder, &directory.path().join("host"), &sharing, &url); | ||
| 500 | let welcome = | ||
| 501 | notebook::live::share::join(hello("Grace"), &code(&host), "", None, Some(&url)).unwrap(); | ||
| 502 | let relay = PresenceRelay::new(&url, &welcome.room); | ||
| 503 | let guest = Guest::start( | ||
| 504 | hello("Grace"), | ||
| 505 | welcome.share, | ||
| 506 | welcome.secret, | ||
| 507 | None, | ||
| 508 | Some(&relay.url), | ||
| 509 | || {}, | ||
| 510 | ) | ||
| 511 | .unwrap(); | ||
| 512 | until("the host was never met", || guest.host().is_some()); | ||
| 513 | let file = folder.join("Garden.one"); | ||
| 514 | let original = std::fs::read(&file).unwrap(); | ||
| 515 | let (space, text, _) = server::text(&original); | ||
| 516 | let typed = |with: &str| Edit { | ||
| 517 | at: 134_000_000_000_000_000, | ||
| 518 | ops: vec![Op::Page { | ||
| 519 | space, | ||
| 520 | op: PageOp::Text { | ||
| 521 | text, | ||
| 522 | range: 0..0, | ||
| 523 | with: with.into(), | ||
| 524 | }, | ||
| 525 | }], | ||
| 526 | }; | ||
| 527 | let replica = Replica::open_or_create(directory.path().join("grace.sqlite"), None, || { | ||
| 528 | Ok(original.clone()) | ||
| 529 | }) | ||
| 530 | .unwrap(); | ||
| 531 | let remote = std::cell::RefCell::new(HostedRemote::new(&guest, "Garden.one")); | ||
| 532 | let first = replica.apply("Grace", typed("A")).unwrap(); | ||
| 533 | until("the first edit was never published", || { | ||
| 534 | let _ = replica.sync_once(&mut *remote.borrow_mut()); | ||
| 535 | matches!( | ||
| 536 | replica.status(first).unwrap(), | ||
| 537 | Some(EditStatus::Published { .. }) | ||
| 538 | ) | ||
| 539 | }); | ||
| 540 | until("the host's stamp was never known without asking", || { | ||
| 541 | let before = relay.state.lock().unwrap().access_bytes; | ||
| 542 | remote.borrow_mut().stamp().unwrap(); | ||
| 543 | relay.state.lock().unwrap().access_bytes == before | ||
| 544 | }); | ||
| 545 | |||
| 546 | relay.state.lock().unwrap().swallow = true; | ||
| 547 | let second = replica.apply("Grace", typed("B")).unwrap(); | ||
| 548 | assert!(matches!( | ||
| 549 | replica.sync_once(&mut *remote.borrow_mut()), | ||
| 550 | Err(Error::Remote(CommitError { | ||
| 551 | state: CommitState::Unknown, | ||
| 552 | .. | ||
| 553 | })) | ||
| 554 | )); | ||
| 555 | assert!(matches!( | ||
| 556 | replica.status(second).unwrap(), | ||
| 557 | Some(EditStatus::AwaitingConfirmation { .. }) | ||
| 558 | )); | ||
| 559 | // OneNote on the host holds the section as the batch comes again, and lets it go. The | ||
| 560 | // app's sync worker keeps its remote after such a refusal. | ||
| 561 | let held = std::fs::File::open(&file).unwrap(); | ||
| 562 | held.lock_shared().unwrap(); | ||
| 563 | let remote = std::cell::RefCell::new(HostedRemote::new(&guest, "Garden.one")); | ||
| 564 | until( | ||
| 565 | "the host never refused the batch while OneNote held it", | ||
| 566 | || { | ||
| 567 | matches!( | ||
| 568 | replica.sync_once(&mut *remote.borrow_mut()), | ||
| 569 | Err(Error::Remote(CommitError { | ||
| 570 | state: CommitState::NotCommitted, | ||
| 571 | .. | ||
| 572 | })) | ||
| 573 | ) | ||
| 574 | }, | ||
| 575 | ); | ||
| 576 | drop(held); | ||
| 577 | until("the uncertain batch never landed", || { | ||
| 578 | let _ = replica.sync_once(&mut *remote.borrow_mut()); | ||
| 579 | matches!( | ||
| 580 | replica.status(second).unwrap(), | ||
| 581 | Some(EditStatus::Published { .. }) | ||
| 582 | ) | ||
| 583 | }); | ||
| 584 | assert_eq!( | ||
| 585 | server::text(&std::fs::read(&file).unwrap()).2, | ||
| 586 | "BAOriginal text" | ||
| 587 | ); | ||
| 588 | } | ||
| 589 | |||
| 590 | /// A section closed while its batch is uncertain sends the batch again from the | ||
| 591 | /// background, which keeps a notebook's closed sections in sync. | ||
| 592 | #[test] | ||
| 593 | fn a_closed_section_s_uncertain_batch_lands_from_the_background() { | ||
| 594 | use notebook::{ | ||
| 595 | EditStatus, Error, Remote, | ||
| 596 | live::share::Guest, | ||
| 597 | session::{Background, Notebook}, | ||
| 598 | }; | ||
| 599 | use onestore::{CommitError, CommitState}; | ||
| 600 | |||
| 601 | let directory = tempfile::tempdir().unwrap(); | ||
| 602 | let folder = notebook(directory.path()); | ||
| 603 | let url = relay(Default::default()); | ||
| 604 | let sharing = Sharing::new("").unwrap(); | ||
| 605 | let host = host(&folder, &directory.path().join("host"), &sharing, &url); | ||
| 606 | let welcome = | ||
| 607 | notebook::live::share::join(hello("Grace"), &code(&host), "", None, Some(&url)).unwrap(); | ||
| 608 | let relay = PresenceRelay::new(&url, &welcome.room); | ||
| 609 | let guest = Guest::start( | ||
| 610 | hello("Grace"), | ||
| 611 | welcome.share, | ||
| 612 | welcome.secret, | ||
| 613 | None, | ||
| 614 | Some(&relay.url), | ||
| 615 | || {}, | ||
| 616 | ) | ||
| 617 | .unwrap(); | ||
| 618 | until("the host was never met", || guest.host().is_some()); | ||
| 619 | let mut notebook = | ||
| 620 | Notebook::open_hosted(Arc::clone(&guest), directory.path().join("grace")).unwrap(); | ||
| 621 | let file = folder.join("Garden.one"); | ||
| 622 | let original = std::fs::read(&file).unwrap(); | ||
| 623 | let (space, text, _) = server::text(&original); | ||
| 624 | let path = notebook.replica_path("Garden.one").unwrap(); | ||
| 625 | std::fs::create_dir_all(path.parent().unwrap()).unwrap(); | ||
| 626 | let replica = Replica::open_or_create(&path, None, || Ok(original.clone())).unwrap(); | ||
| 627 | let typed = |with: &str| Edit { | ||
| 628 | at: 134_000_000_000_000_000, | ||
| 629 | ops: vec![Op::Page { | ||
| 630 | space, | ||
| 631 | op: PageOp::Text { | ||
| 632 | text, | ||
| 633 | range: 0..0, | ||
| 634 | with: with.into(), | ||
| 635 | }, | ||
| 636 | }], | ||
| 637 | }; | ||
| 638 | let first = replica.apply("Grace", typed("A")).unwrap(); | ||
| 639 | until("the first edit was never published", || { | ||
| 640 | let _ = replica.sync_once(&mut HostedRemote::new(&guest, "Garden.one")); | ||
| 641 | matches!( | ||
| 642 | replica.status(first).unwrap(), | ||
| 643 | Some(EditStatus::Published { .. }) | ||
| 644 | ) | ||
| 645 | }); | ||
| 646 | until("the host's stamp was never known without asking", || { | ||
| 647 | let before = relay.state.lock().unwrap().access_bytes; | ||
| 648 | HostedRemote::new(&guest, "Garden.one").stamp().unwrap(); | ||
| 649 | relay.state.lock().unwrap().access_bytes == before | ||
| 650 | }); | ||
| 651 | relay.state.lock().unwrap().swallow = true; | ||
| 652 | let id = replica.apply("Grace", typed("B")).unwrap(); | ||
| 653 | assert!(matches!( | ||
| 654 | replica.sync_once(&mut HostedRemote::new(&guest, "Garden.one")), | ||
| 655 | Err(Error::Remote(CommitError { | ||
| 656 | state: CommitState::Unknown, | ||
| 657 | .. | ||
| 658 | })) | ||
| 659 | )); | ||
| 660 | drop(replica); | ||
| 661 | let background = Background::hosted(Arc::clone(&guest), || {}).unwrap(); | ||
| 662 | background.watch(notebook.replicas()); | ||
| 663 | // As the app does, the background is woken whenever the guest meets its host again. | ||
| 664 | let reached = std::cell::Cell::new(guest.host().is_some()); | ||
| 665 | until("the closed section's batch never landed", || { | ||
| 666 | let now = guest.host().is_some(); | ||
| 667 | if now && !reached.replace(now) { | ||
| 668 | background.wake(); | ||
| 669 | } | ||
| 670 | server::text(&std::fs::read(&file).unwrap()).2 == "BAOriginal text" | ||
| 671 | }); | ||
| 672 | background.stop(); | ||
| 673 | let replica = Replica::open(&path).unwrap(); | ||
| 674 | assert!(matches!( | ||
| 675 | replica.status(id).unwrap(), | ||
| 676 | Some(EditStatus::Published { .. }) | ||
| 677 | )); | ||
| 678 | } |
crates/notebook/tests/support/live.rs+8-1| ... | @@ -168,6 +168,8 @@ pub struct RelayState { | ... | @@ -168,6 +168,8 @@ pub struct RelayState { |
| 168 | pub blocked: bool, | 168 | pub blocked: bool, |
| 169 | pub access_connections: usize, | 169 | pub access_connections: usize, |
| 170 | pub access_bytes: usize, | 170 | pub access_bytes: usize, |
| 171 | /// The next bytes a guest sends for access are lost, and their connection with them. | ||
| 172 | pub swallow: bool, | ||
| 171 | sockets: Vec<(bool, std::net::TcpStream)>, | 173 | sockets: Vec<(bool, std::net::TcpStream)>, |
| 172 | stopped: bool, | 174 | stopped: bool, |
| 173 | } | 175 | } |
| ... | @@ -237,7 +239,12 @@ impl PresenceRelay { | ... | @@ -237,7 +239,12 @@ impl PresenceRelay { |
| 237 | break; | 239 | break; |
| 238 | } | 240 | } |
| 239 | if !presence { | 241 | if !presence { |
| 240 | counted.lock().unwrap().access_bytes += length; | 242 | let mut counted = counted.lock().unwrap(); |
| 243 | if std::mem::take(&mut counted.swallow) { | ||
| 244 | let _ = from.shutdown(Shutdown::Both); | ||
| 245 | break; | ||
| 246 | } | ||
| 247 | counted.access_bytes += length; | ||
| 241 | } | 248 | } |
| 242 | if to.write_all(&bytes[..length]).is_err() { | 249 | if to.write_all(&bytes[..length]).is_err() { |
| 243 | break; | 250 | break; |