| author | |
| committer | |
| log | 00efb7f2a201741ccea64df3120cf80bb4d0f6cd |
| tree | 327d51a3e2863f5172b34cda40dd861c7c0b22e1 |
| parent | 0be69d99f016301e65f1c6b8431e432c5f19f9cc |
| signature | Signed by SSH key SHA256:52mNGHRsVFBDED9IAX5pe+LRWUefqTbxEReunq21QvU |
Review fixes for the lost-upload stack (yvvmzxws, rpmtnlzk, toxstspw,
pswvkryr, ssnownqr). They share one change of design, so they land as one
commit rather than folded into each.
The images a section's publications keep now live beside its replica
("<replica>.published"), as the session chose it, instead of in a folder
guessed from whether a copy's folder had been labelled: the remotes the
session and the background make are handed the replica's path. A copy
found after the notebook opened kept its images under the original's
identity and replaced the original's; now each keeps its own. They move
with the replica (location::moved, adopt) and go with it (remove_replica),
so a password set elsewhere, which supersedes the replica, takes the
plaintext kept with it; closing a notebook, as the app does before renaming
it, keeps an offline copy whose images are not yet merged, as a waiting edit
keeps it, and only removing the notebook deletes them, and copies' folders are labelled as they are first
named, so a rename carries them. Android's provider keeps its images the
same way.
An upload iCloud Drive keeps as a conflict version is no longer also merged
from the store, which kept it beside the file twice: the store skips an image
a conflict version is, lists none while a conflict version cannot be read yet,
and drops it when that version is retired. A
confirmation, which rewrites only the header, carries the kept image along
rather than letting it list as lost and go. The store reads the file's stamp
only where it keeps something. The Mac names its versions by
SCDynamicStoreCopyComputerName instead of the deprecated NSHost, and its
iCloud functions take no cache, which pswvkryr had added only on macOS. A
share copy's replica is named as the session names a copy's
(session::replica_location) instead of by a second rule. Tests: the store's
own (a lost image lists once, after LAG, not beside a conflict version; a
confirmation; nothing read when nothing is kept), and on the phone an upload
undone then merged back after LAG, a phone-only notebook keeping nothing, a
password set elsewhere, a copy found while open, and the launch test now
asserting nothing kept beside the file and no stale image. The test
coordinator holds files per test, so parallel tests no longer take each
other's hold.
Assisted-by: claude-opus-5.517 files changed, 836 insertions(+), 410 deletions(-)
arc/sync.md+2-1| ... | @@ -265,7 +265,8 @@ mount reports none of another client's changes to inotify, so there each section | ... | @@ -265,7 +265,8 @@ mount reports none of another client's changes to inotify, so there each section |
| 265 | read every 15 seconds. On iOS, a folder on the device (Snowbound's own, or On My iPhone) has no | 265 | read every 15 seconds. On iOS, a folder on the device (Snowbound's own, or On My iPhone) has no |
| 266 | copies and learns of other apps' writes through file coordination (`NSFilePresenter`); one a | 266 | copies and learns of other apps' writes through file coordination (`NSFilePresenter`); one a |
| 267 | file provider keeps elsewhere, as iCloud Drive does, keeps copies and is checked every 15 | 267 | file provider keeps elsewhere, as iCloud Drive does, keeps copies and is checked every 15 |
| 268 | seconds. Closing a notebook deletes its copies, except any with edits still waiting. | 268 | seconds. Closing a notebook deletes its copies, except any with edits still waiting or images |
| 269 | of uploads it keeps (`notebook::published`). | ||
| 269 | 270 | ||
| 270 | ## iCloud Drive | 271 | ## iCloud Drive |
| 271 | 272 |
crates/mobile/src/library.rs+102-103| ... | @@ -20,7 +20,7 @@ use onestore::{ | ... | @@ -20,7 +20,7 @@ use onestore::{ |
| 20 | protected::Key, | 20 | protected::Key, |
| 21 | }; | 21 | }; |
| 22 | use std::{ | 22 | use std::{ |
| 23 | collections::{BTreeMap, HashMap}, | 23 | collections::{BTreeMap, BTreeSet, HashMap}, |
| 24 | ffi::{CString, c_char, c_void}, | 24 | ffi::{CString, c_char, c_void}, |
| 25 | io, | 25 | io, |
| 26 | path::{Path, PathBuf}, | 26 | path::{Path, PathBuf}, |
| ... | @@ -123,44 +123,51 @@ pub unsafe extern "C" fn sb_set_versions( | ... | @@ -123,44 +123,51 @@ pub unsafe extern "C" fn sb_set_versions( |
| 123 | 123 | ||
| 124 | /// A local section file read and published under the host's file coordination. One a file | 124 | /// A local section file read and published under the host's file coordination. One a file |
| 125 | /// provider keeps elsewhere, which syncs it whole with no lock, keeps each image this device | 125 | /// provider keeps elsewhere, which syncs it whole with no lock, keeps each image this device |
| 126 | /// publishes until the file holds it, and merges one another device's upload replaced back | 126 | /// publishes beside its replica until the file holds it, and merges one another device's |
| 127 | /// (`notebook::published`). | 127 | /// upload replaced back (`notebook::published`). |
| 128 | struct Coordinated { | 128 | struct Coordinated { |
| 129 | file: PathBuf, | 129 | file: PathBuf, |
| 130 | published: Option<Published>, | 130 | published: Option<Published>, |
| 131 | } | 131 | } |
| 132 | 132 | ||
| 133 | impl Coordinated { | 133 | impl Coordinated { |
| 134 | /// The file `file` of the notebook folder, or lone section, at `notebook`. | 134 | /// The section file `file`, its replica at `replica` where it has one. |
| 135 | fn new(file: &Path, notebook: &Path, cache: &Path, elsewhere: bool) -> Self { | 135 | fn new(file: &Path, replica: Option<&Path>, elsewhere: bool) -> Self { |
| 136 | let device = VERSIONS.get().map(|(.., device)| device.clone()); | 136 | let device = VERSIONS |
| 137 | let published = elsewhere | 137 | .get() |
| 138 | .then(|| Published::of(cache, notebook, file, device)) | 138 | .filter(|_| elsewhere) |
| 139 | .flatten(); | 139 | .map(|(.., device)| device); |
| 140 | Self { | 140 | Self { |
| 141 | file: file.to_owned(), | 141 | file: file.to_owned(), |
| 142 | published, | 142 | published: replica |
| 143 | .zip(device) | ||
| 144 | .map(|(replica, device)| Published::of(replica, device)), | ||
| 143 | } | 145 | } |
| 144 | } | 146 | } |
| 145 | } | 147 | } |
| 146 | 148 | ||
| 149 | /// The header and length of `file`, as the file remote's uncoordinated change detector reads | ||
| 150 | /// them. | ||
| 151 | fn stamp(file: &Path) -> io::Result<Stamp> { | ||
| 152 | coordinated(file, false, || { | ||
| 153 | use std::io::Read; | ||
| 154 | let mut opened = std::fs::File::open(file)?; | ||
| 155 | let mut header = [0; 1024]; | ||
| 156 | opened.read_exact(&mut header)?; | ||
| 157 | Ok(Stamp { | ||
| 158 | header, | ||
| 159 | length: opened.metadata()?.len(), | ||
| 160 | }) | ||
| 161 | })? | ||
| 162 | } | ||
| 163 | |||
| 147 | impl Remote for Coordinated { | 164 | impl Remote for Coordinated { |
| 148 | fn read(&mut self) -> io::Result<Vec<u8>> { | 165 | fn read(&mut self) -> io::Result<Vec<u8>> { |
| 149 | coordinated(&self.file, false, || onestore::read_file(&self.file))? | 166 | coordinated(&self.file, false, || onestore::read_file(&self.file))? |
| 150 | } | 167 | } |
| 151 | 168 | ||
| 152 | /// The header and length, as the file remote's uncoordinated change detector reads them. | ||
| 153 | fn stamp(&mut self) -> io::Result<Stamp> { | 169 | fn stamp(&mut self) -> io::Result<Stamp> { |
| 154 | coordinated(&self.file, false, || { | 170 | stamp(&self.file) |
| 155 | use std::io::Read; | ||
| 156 | let mut file = std::fs::File::open(&self.file)?; | ||
| 157 | let mut header = [0; 1024]; | ||
| 158 | file.read_exact(&mut header)?; | ||
| 159 | Ok(Stamp { | ||
| 160 | header, | ||
| 161 | length: file.metadata()?.len(), | ||
| 162 | }) | ||
| 163 | })? | ||
| 164 | } | 171 | } |
| 165 | 172 | ||
| 166 | fn publish(&mut self, transaction: &Transaction) -> std::result::Result<(), CommitError> { | 173 | fn publish(&mut self, transaction: &Transaction) -> std::result::Result<(), CommitError> { |
| ... | @@ -178,7 +185,12 @@ impl Remote for Coordinated { | ... | @@ -178,7 +185,12 @@ impl Remote for Coordinated { |
| 178 | 185 | ||
| 179 | fn confirm(&mut self, base: &Stamp) -> std::result::Result<(), CommitError> { | 186 | fn confirm(&mut self, base: &Stamp) -> std::result::Result<(), CommitError> { |
| 180 | coordinated(&self.file, true, || { | 187 | coordinated(&self.file, true, || { |
| 181 | onestore::confirm_file(&self.file, base) | 188 | onestore::confirm_file(&self.file, base)?; |
| 189 | if let Some(published) = &self.published { | ||
| 190 | let _ = std::fs::File::open(&self.file) | ||
| 191 | .and_then(|mut file| published.confirmed(Written::InPlace(&mut file), base)); | ||
| 192 | } | ||
| 193 | Ok(()) | ||
| 182 | }) | 194 | }) |
| 183 | .map_err(uncommitted)? | 195 | .map_err(uncommitted)? |
| 184 | } | 196 | } |
| ... | @@ -198,13 +210,9 @@ impl Remote for Coordinated { | ... | @@ -198,13 +210,9 @@ impl Remote for Coordinated { |
| 198 | let path = CString::new(self.file.as_os_str().as_encoded_bytes())?; | 210 | let path = CString::new(self.file.as_os_str().as_encoded_bytes())?; |
| 199 | versions(path.as_ptr(), found, (&raw mut listed).cast()); | 211 | versions(path.as_ptr(), found, (&raw mut listed).cast()); |
| 200 | } | 212 | } |
| 201 | if self.published.is_some() { | 213 | if let Some(published) = &self.published { |
| 202 | let stamp = self.stamp()?; | 214 | let lost = published.versions(|| stamp(&self.file), &listed)?; |
| 203 | listed.extend( | 215 | listed.extend(lost); |
| 204 | self.published | ||
| 205 | .iter() | ||
| 206 | .flat_map(|published| published.versions(&stamp)), | ||
| 207 | ); | ||
| 208 | } | 216 | } |
| 209 | Ok(listed) | 217 | Ok(listed) |
| 210 | } | 218 | } |
| ... | @@ -221,12 +229,14 @@ impl Remote for Coordinated { | ... | @@ -221,12 +229,14 @@ impl Remote for Coordinated { |
| 221 | && published.holds(id) | 229 | && published.holds(id) |
| 222 | { | 230 | { |
| 223 | if keep { | 231 | if keep { |
| 224 | let device = VERSIONS.get().map_or("this device", |(.., device)| device); | 232 | let beside = notebook::published::beside(&self.file, published.device()); |
| 225 | let beside = notebook::published::beside(&self.file, device); | ||
| 226 | coordinated(&beside, true, || std::fs::copy(id, &beside))??; | 233 | coordinated(&beside, true, || std::fs::copy(id, &beside))??; |
| 227 | } | 234 | } |
| 228 | return std::fs::remove_file(id); | 235 | return std::fs::remove_file(id); |
| 229 | } | 236 | } |
| 237 | if let Some(published) = &self.published { | ||
| 238 | let _ = published.retired(Path::new(id)); | ||
| 239 | } | ||
| 230 | let Some((_, retire, _)) = VERSIONS.get() else { | 240 | let Some((_, retire, _)) = VERSIONS.get() else { |
| 231 | return Ok(()); | 241 | return Ok(()); |
| 232 | }; | 242 | }; |
| ... | @@ -304,40 +314,38 @@ struct Joined { | ... | @@ -304,40 +314,38 @@ struct Joined { |
| 304 | 314 | ||
| 305 | /// A share notebook's sections as last listed, so they open from their replicas while the | 315 | /// A share notebook's sections as last listed, so they open from their replicas while the |
| 306 | /// server cannot be reached. | 316 | /// server cannot be reached. |
| 307 | #[derive(serde::Serialize, serde::Deserialize)] | 317 | #[derive(serde::Serialize, serde::Deserialize, Default)] |
| 308 | struct Listed { | 318 | pub(crate) struct Listed { |
| 309 | tabs: Vec<Tab>, | 319 | tabs: Vec<Tab>, |
| 310 | /// The name of each readable section's replica (`files`), by catalog path. | 320 | /// Each readable section's file identity in hex, which names its replica, by catalog path. |
| 311 | files: BTreeMap<String, String>, | 321 | pub(crate) files: BTreeMap<String, String>, |
| 322 | /// The sections that copy another, holding its identity, whose replicas are their own | ||
| 323 | /// (`session::replica_location`). | ||
| 324 | #[serde(default)] | ||
| 325 | pub(crate) copies: BTreeSet<String>, | ||
| 312 | } | 326 | } |
| 313 | 327 | ||
| 314 | fn hex(bytes: &[u8]) -> String { | 328 | fn hex(bytes: &[u8]) -> String { |
| 315 | bytes.iter().map(|byte| format!("{byte:02x}")).collect() | 329 | bytes.iter().map(|byte| format!("{byte:02x}")).collect() |
| 316 | } | 330 | } |
| 317 | 331 | ||
| 318 | /// The name of the replica of every readable section in `folder` and its groups, by catalog | 332 | /// Every readable section's file identity in `folder` and its groups, by catalog path, and |
| 319 | /// path, as `Notebook::replicas` lists a local notebook's: its file identity in hex, and for a | 333 | /// the copies among them, as `Notebook::replicas` lists a local notebook's. `Client::read` |
| 320 | /// copy of another section, which holds the same identity, its path besides, so that each | 334 | /// answers a protected section as it does one mid-write (WouldBlock), so syncing one would |
| 321 | /// keeps its own edits. `Client::read` answers a protected section as it does one mid-write | 335 | /// show it in use for good. |
| 322 | /// (WouldBlock), so syncing one would show it in use for good. | 336 | pub(crate) fn files(folder: &Folder, listed: &mut Listed) { |
| 323 | pub(crate) fn files(folder: &Folder, files: &mut BTreeMap<String, String>) { | ||
| 324 | for section in &folder.sections { | 337 | for section in &folder.sections { |
| 325 | if matches!(section.state, SectionState::Readable { .. }) { | 338 | if matches!(section.state, SectionState::Readable { .. }) { |
| 326 | let mut replica = hex(&section.file_id); | 339 | listed |
| 340 | .files | ||
| 341 | .insert(section.path.clone(), hex(&section.file_id)); | ||
| 327 | if section.copy { | 342 | if section.copy { |
| 328 | replica.push('-'); | 343 | listed.copies.insert(section.path.clone()); |
| 329 | replica.extend( | ||
| 330 | section | ||
| 331 | .path | ||
| 332 | .chars() | ||
| 333 | .map(|char| if char.is_alphanumeric() { char } else { '_' }), | ||
| 334 | ); | ||
| 335 | } | 344 | } |
| 336 | files.insert(section.path.clone(), replica); | ||
| 337 | } | 345 | } |
| 338 | } | 346 | } |
| 339 | for group in &folder.groups { | 347 | for group in &folder.groups { |
| 340 | self::files(group, files); | 348 | self::files(group, listed); |
| 341 | } | 349 | } |
| 342 | } | 350 | } |
| 343 | 351 | ||
| ... | @@ -473,11 +481,10 @@ impl Library { | ... | @@ -473,11 +481,10 @@ impl Library { |
| 473 | (None, Place::File(path.to_owned()), None) | 481 | (None, Place::File(path.to_owned()), None) |
| 474 | } else { | 482 | } else { |
| 475 | let mut notebook = Notebook::open(path, cache)?; | 483 | let mut notebook = Notebook::open(path, cache)?; |
| 476 | let (root, kept) = (path.to_owned(), cache.to_owned()); | ||
| 477 | let background = notebook.background_with( | 484 | let background = notebook.background_with( |
| 478 | local, | 485 | local, |
| 479 | !local, | 486 | !local, |
| 480 | move |file| Coordinated::new(file, &root, &kept, !local), | 487 | move |file, replica| Coordinated::new(file, replica, !local), |
| 481 | sync_woken, | 488 | sync_woken, |
| 482 | )?; | 489 | )?; |
| 483 | background.watch(notebook.replicas()); | 490 | background.watch(notebook.replicas()); |
| ... | @@ -523,7 +530,7 @@ impl Library { | ... | @@ -523,7 +530,7 @@ impl Library { |
| 523 | }; | 530 | }; |
| 524 | let reached = library.with_notebook(false, |_| Ok(())); | 531 | let reached = library.with_notebook(false, |_| Ok(())); |
| 525 | match library.listed() { | 532 | match library.listed() { |
| 526 | Some(listed) => library.watch(&listed.files), | 533 | Some(listed) => library.watch(&listed), |
| 527 | None => reached?, | 534 | None => reached?, |
| 528 | } | 535 | } |
| 529 | Ok(library) | 536 | Ok(library) |
| ... | @@ -619,17 +626,18 @@ impl Library { | ... | @@ -619,17 +626,18 @@ impl Library { |
| 619 | }) | 626 | }) |
| 620 | } | 627 | } |
| 621 | 628 | ||
| 622 | /// Has the background sync a share notebook's sections, `files` as `Listed` keeps them. | 629 | /// Has the background sync a share notebook's sections, as `listed` names them. |
| 623 | fn watch(&self, files: &BTreeMap<String, String>) { | 630 | fn watch(&self, listed: &Listed) { |
| 624 | let (Some(background), Some(replicas)) = (&self.background, self.share_replicas()) else { | 631 | let Some(background) = &self.background else { |
| 625 | return; | 632 | return; |
| 626 | }; | 633 | }; |
| 627 | background.watch( | 634 | background.watch( |
| 628 | files | 635 | listed |
| 629 | .iter() | 636 | .files |
| 630 | .map(|(path, replica)| Known { | 637 | .keys() |
| 638 | .map(|path| Known { | ||
| 631 | path: path.clone(), | 639 | path: path.clone(), |
| 632 | replica: Some(replicas.join(format!("{replica}.sqlite"))), | 640 | replica: self.share_replica(listed, path), |
| 633 | found: None, | 641 | found: None, |
| 634 | image: None, | 642 | image: None, |
| 635 | }) | 643 | }) |
| ... | @@ -637,13 +645,16 @@ impl Library { | ... | @@ -637,13 +645,16 @@ impl Library { |
| 637 | ); | 645 | ); |
| 638 | } | 646 | } |
| 639 | 647 | ||
| 640 | /// The cache folder of a share notebook's replicas. | 648 | /// The replica of the share section at catalog `path`, as `listed` names it: by its file |
| 641 | fn share_replicas(&self) -> Option<PathBuf> { | 649 | /// identity in the notebook's folder, a copy's in a folder of its own. |
| 650 | fn share_replica(&self, listed: &Listed, path: &str) -> Option<PathBuf> { | ||
| 642 | let Place::Share { server, root, .. } = &self.place else { | 651 | let Place::Share { server, root, .. } = &self.place else { |
| 643 | return None; | 652 | return None; |
| 644 | }; | 653 | }; |
| 645 | let location = notebook::location::smb(&server.address, &server.share, root); | 654 | let notebook = notebook::location::smb(&server.address, &server.share, root); |
| 646 | Some(notebook::location::folder(&self.cache, &location)) | 655 | let location = session::replica_location(&notebook, path, listed.copies.contains(path)); |
| 656 | let folder = notebook::location::claimed(&self.cache, &location).ok()?; | ||
| 657 | Some(folder.join(format!("{}.sqlite", listed.files.get(path)?))) | ||
| 647 | } | 658 | } |
| 648 | 659 | ||
| 649 | /// Where a share notebook's last listing is kept. | 660 | /// Where a share notebook's last listing is kept. |
| ... | @@ -757,13 +768,10 @@ impl Library { | ... | @@ -757,13 +768,10 @@ impl Library { |
| 757 | } | 768 | } |
| 758 | let listed = self.with_notebook(false, |notebook| { | 769 | let listed = self.with_notebook(false, |notebook| { |
| 759 | notebook.refresh()?; | 770 | notebook.refresh()?; |
| 760 | let mut listed = Listed { | 771 | let mut listed = Listed::default(); |
| 761 | tabs: Vec::new(), | ||
| 762 | files: BTreeMap::new(), | ||
| 763 | }; | ||
| 764 | tabs(notebook.catalog(), &self.unlocked(), &mut listed.tabs); | 772 | tabs(notebook.catalog(), &self.unlocked(), &mut listed.tabs); |
| 765 | if matches!(self.place, Place::Share { .. }) { | 773 | if matches!(self.place, Place::Share { .. }) { |
| 766 | files(notebook.catalog(), &mut listed.files); | 774 | files(notebook.catalog(), &mut listed); |
| 767 | } | 775 | } |
| 768 | if let (Place::Folder(_) | Place::Hosted(_) | Place::Provider(_), Some(background)) = | 776 | if let (Place::Folder(_) | Place::Hosted(_) | Place::Provider(_), Some(background)) = |
| 769 | (&self.place, &self.background) | 777 | (&self.place, &self.background) |
| ... | @@ -779,7 +787,7 @@ impl Library { | ... | @@ -779,7 +787,7 @@ impl Library { |
| 779 | Ok(listed) => { | 787 | Ok(listed) => { |
| 780 | std::fs::create_dir_all(listing.parent().ok_or("No cache")?)?; | 788 | std::fs::create_dir_all(listing.parent().ok_or("No cache")?)?; |
| 781 | std::fs::write(&listing, serde_json::to_vec(&listed)?)?; | 789 | std::fs::write(&listing, serde_json::to_vec(&listed)?)?; |
| 782 | self.watch(&listed.files); | 790 | self.watch(&listed); |
| 783 | Ok(listed.tabs) | 791 | Ok(listed.tabs) |
| 784 | } | 792 | } |
| 785 | Err(error) => Ok(self.listed().ok_or(error)?.tabs), | 793 | Err(error) => Ok(self.listed().ok_or(error)?.tabs), |
| ... | @@ -874,21 +882,19 @@ impl Library { | ... | @@ -874,21 +882,19 @@ impl Library { |
| 874 | notify: impl Fn() + Send + Sync + 'static, | 882 | notify: impl Fn() + Send + Sync + 'static, |
| 875 | ) -> Result<session::Section> { | 883 | ) -> Result<session::Section> { |
| 876 | let offline = self.offline.load(Ordering::Relaxed); | 884 | let offline = self.offline.load(Ordering::Relaxed); |
| 877 | let (cache, elsewhere) = (self.cache.clone(), self.elsewhere); | 885 | let elsewhere = self.elsewhere; |
| 878 | let connect = |notebook: &Path| { | 886 | let connect = move |file: &Path, replica: Option<&Path>| { |
| 879 | let notebook = notebook.to_owned(); | 887 | Ok(Coordinated::new(file, replica, elsewhere)) |
| 880 | move |file: &Path| Ok(Coordinated::new(file, &notebook, &cache, elsewhere)) | ||
| 881 | }; | 888 | }; |
| 882 | match &self.place { | 889 | match &self.place { |
| 883 | Place::File(file) => Ok(session::Section::open_with( | 890 | Place::File(file) => Ok(session::Section::open_with( |
| 884 | file, | 891 | file, |
| 885 | &self.cache, | 892 | &self.cache, |
| 886 | connect(file), | 893 | connect, |
| 887 | offline, | 894 | offline, |
| 888 | notify, | 895 | notify, |
| 889 | )?), | 896 | )?), |
| 890 | Place::Folder(root) => { | 897 | Place::Folder(_) => { |
| 891 | let connect = connect(root); | ||
| 892 | let notebook = self.notebook(); | 898 | let notebook = self.notebook(); |
| 893 | let notebook = notebook.as_ref().ok_or("A lone section has no notebook")?; | 899 | let notebook = notebook.as_ref().ok_or("A lone section has no notebook")?; |
| 894 | Ok(match self.unlocked().get(path) { | 900 | Ok(match self.unlocked().get(path) { |
| ... | @@ -908,14 +914,10 @@ impl Library { | ... | @@ -908,14 +914,10 @@ impl Library { |
| 908 | .as_ref() | 914 | .as_ref() |
| 909 | .ok_or("The notebook hasn’t been read")? | 915 | .ok_or("The notebook hasn’t been read")? |
| 910 | .replica_path(path)?, | 916 | .replica_path(path)?, |
| 911 | None => { | 917 | None => self |
| 912 | let replica = self | 918 | .listed() |
| 913 | .listed() | 919 | .and_then(|listed| self.share_replica(&listed, path)) |
| 914 | .and_then(|mut listed| listed.files.remove(path)) | 920 | .ok_or("The notebook hasn’t listed this section")?, |
| 915 | .ok_or("The notebook hasn’t listed this section")?; | ||
| 916 | let replicas = self.share_replicas().ok_or("Not a share")?; | ||
| 917 | replicas.join(format!("{replica}.sqlite")) | ||
| 918 | } | ||
| 919 | }; | 921 | }; |
| 920 | std::fs::create_dir_all(cache.parent().ok_or("No cache")?)?; | 922 | std::fs::create_dir_all(cache.parent().ok_or("No cache")?)?; |
| 921 | let replica = Replica::open_or_create(&cache, key.as_ref(), || { | 923 | let replica = Replica::open_or_create(&cache, key.as_ref(), || { |
| ... | @@ -2753,12 +2755,9 @@ mod tests { | ... | @@ -2753,12 +2755,9 @@ mod tests { |
| 2753 | onestore::create_section("Read.one", "Offline source content", "Fixture").unwrap(); | 2755 | onestore::create_section("Read.one", "Offline source content", "Fixture").unwrap(); |
| 2754 | std::fs::write(root.join("Read.one"), &bytes).unwrap(); | 2756 | std::fs::write(root.join("Read.one"), &bytes).unwrap(); |
| 2755 | let notebook = Notebook::open(&root, &cache).unwrap(); | 2757 | let notebook = Notebook::open(&root, &cache).unwrap(); |
| 2756 | let mut listed = Listed { | 2758 | let mut listed = Listed::default(); |
| 2757 | tabs: Vec::new(), | ||
| 2758 | files: BTreeMap::new(), | ||
| 2759 | }; | ||
| 2760 | tabs(notebook.catalog(), &HashMap::new(), &mut listed.tabs); | 2759 | tabs(notebook.catalog(), &HashMap::new(), &mut listed.tabs); |
| 2761 | files(notebook.catalog(), &mut listed.files); | 2760 | files(notebook.catalog(), &mut listed); |
| 2762 | let identity = listed.files["Read.one"].clone(); | 2761 | let identity = listed.files["Read.one"].clone(); |
| 2763 | drop(notebook); | 2762 | drop(notebook); |
| 2764 | 2763 | ||
| ... | @@ -2840,21 +2839,21 @@ mod tests { | ... | @@ -2840,21 +2839,21 @@ mod tests { |
| 2840 | std::fs::write(root.join(name), &bytes).unwrap(); | 2839 | std::fs::write(root.join(name), &bytes).unwrap(); |
| 2841 | } | 2840 | } |
| 2842 | let notebook = Notebook::open(&root, &cache).unwrap(); | 2841 | let notebook = Notebook::open(&root, &cache).unwrap(); |
| 2843 | let mut listed = Listed { | 2842 | let mut listed = Listed::default(); |
| 2844 | tabs: Vec::new(), | ||
| 2845 | files: BTreeMap::new(), | ||
| 2846 | }; | ||
| 2847 | tabs(notebook.catalog(), &HashMap::new(), &mut listed.tabs); | 2843 | tabs(notebook.catalog(), &HashMap::new(), &mut listed.tabs); |
| 2848 | files(notebook.catalog(), &mut listed.files); | 2844 | files(notebook.catalog(), &mut listed); |
| 2849 | drop(notebook); | 2845 | drop(notebook); |
| 2850 | let address = "127.0.0.1:0"; | 2846 | assert_eq!( |
| 2851 | let replicas = notebook::location::folder( | 2847 | listed.copies.iter().collect::<Vec<_>>(), |
| 2852 | &cache, | 2848 | ["Notes - Copy.one"] |
| 2853 | &notebook::location::smb(address, "agent", "Notebook"), | ||
| 2854 | ); | 2849 | ); |
| 2855 | std::fs::create_dir_all(&replicas).unwrap(); | 2850 | let address = "127.0.0.1:0"; |
| 2856 | for replica in listed.files.values() { | 2851 | let notebook = notebook::location::smb(address, "agent", "Notebook"); |
| 2857 | let _ = Replica::create(replicas.join(format!("{replica}.sqlite")), &bytes); | 2852 | for (path, identity) in &listed.files { |
| 2853 | let copy = listed.copies.contains(path); | ||
| 2854 | let location = session::replica_location(&notebook, path, copy); | ||
| 2855 | let replicas = notebook::location::claimed(&cache, &location).unwrap(); | ||
| 2856 | Replica::create(replicas.join(format!("{identity}.sqlite")), &bytes).unwrap(); | ||
| 2858 | } | 2857 | } |
| 2859 | std::fs::create_dir_all(cache.join("smb")).unwrap(); | 2858 | std::fs::create_dir_all(cache.join("smb")).unwrap(); |
| 2860 | std::fs::write( | 2859 | std::fs::write( |
crates/mobile/src/tests.rs+213-22| ... | @@ -1273,22 +1273,22 @@ fn a_share_notebook_syncs_only_its_readable_sections() { | ... | @@ -1273,22 +1273,22 @@ fn a_share_notebook_syncs_only_its_readable_sections() { |
| 1273 | .unwrap(); | 1273 | .unwrap(); |
| 1274 | let notebook = | 1274 | let notebook = |
| 1275 | notebook::session::Notebook::open(&root, directory.path().join("cache")).unwrap(); | 1275 | notebook::session::Notebook::open(&root, directory.path().join("cache")).unwrap(); |
| 1276 | let mut files = std::collections::BTreeMap::new(); | 1276 | let mut listed = library::Listed::default(); |
| 1277 | library::files(notebook.catalog(), &mut files); | 1277 | library::files(notebook.catalog(), &mut listed); |
| 1278 | assert_eq!(files.keys().collect::<Vec<_>>(), ["Empty.one"]); | 1278 | assert_eq!(listed.files.keys().collect::<Vec<_>>(), ["Empty.one"]); |
| 1279 | } | 1279 | } |
| 1280 | 1280 | ||
| 1281 | /// Paths the test coordinator was asked for, with whether to write. | 1281 | /// Paths the test coordinator was asked for, with whether to write. |
| 1282 | static COORDINATED: Mutex<Vec<(String, bool)>> = Mutex::new(Vec::new()); | 1282 | static COORDINATED: Mutex<Vec<(String, bool)>> = Mutex::new(Vec::new()); |
| 1283 | 1283 | ||
| 1284 | /// A file whose next coordinated access the test coordinator holds, as another app may: who | 1284 | /// Files whose next coordinated access the test coordinator holds, as another app may: who |
| 1285 | /// hears that it is held, and what lets it go. | 1285 | /// hears that each is held, and what lets it go. Tests running at once hold files of their own. |
| 1286 | type Held = ( | 1286 | type Held = ( |
| 1287 | String, | 1287 | String, |
| 1288 | std::sync::mpsc::Sender<()>, | 1288 | std::sync::mpsc::Sender<()>, |
| 1289 | std::sync::mpsc::Receiver<()>, | 1289 | std::sync::mpsc::Receiver<()>, |
| 1290 | ); | 1290 | ); |
| 1291 | static HELD: Mutex<Option<Held>> = Mutex::new(None); | 1291 | static HELD: Mutex<Vec<Held>> = Mutex::new(Vec::new()); |
| 1292 | 1292 | ||
| 1293 | extern "C" fn coordinator( | 1293 | extern "C" fn coordinator( |
| 1294 | path: *const c_char, | 1294 | path: *const c_char, |
| ... | @@ -1300,10 +1300,8 @@ extern "C" fn coordinator( | ... | @@ -1300,10 +1300,8 @@ extern "C" fn coordinator( |
| 1300 | COORDINATED.lock().unwrap().push((path.clone(), write)); | 1300 | COORDINATED.lock().unwrap().push((path.clone(), write)); |
| 1301 | let held = { | 1301 | let held = { |
| 1302 | let mut held = HELD.lock().unwrap(); | 1302 | let mut held = HELD.lock().unwrap(); |
| 1303 | match &*held { | 1303 | let at = held.iter().position(|(file, ..)| *file == path); |
| 1304 | Some((file, ..)) if *file == path => held.take(), | 1304 | at.map(|at| held.remove(at)) |
| 1305 | _ => None, | ||
| 1306 | } | ||
| 1307 | }; | 1305 | }; |
| 1308 | if let Some((_, entered, released)) = held { | 1306 | if let Some((_, entered, released)) = held { |
| 1309 | let _ = entered.send(()); | 1307 | let _ = entered.send(()); |
| ... | @@ -2043,7 +2041,9 @@ fn a_page_edited_while_it_moves_stays_with_its_edit() { | ... | @@ -2043,7 +2041,9 @@ fn a_page_edited_while_it_moves_stays_with_its_edit() { |
| 2043 | let (entered, holding) = std::sync::mpsc::channel(); | 2041 | let (entered, holding) = std::sync::mpsc::channel(); |
| 2044 | let (release, released) = std::sync::mpsc::channel(); | 2042 | let (release, released) = std::sync::mpsc::channel(); |
| 2045 | let empty = root.canonicalize().unwrap().join("Empty.one"); | 2043 | let empty = root.canonicalize().unwrap().join("Empty.one"); |
| 2046 | *HELD.lock().unwrap() = Some((empty.to_string_lossy().into_owned(), entered, released)); | 2044 | HELD.lock() |
| 2045 | .unwrap() | ||
| 2046 | .push((empty.to_string_lossy().into_owned(), entered, released)); | ||
| 2047 | let moving = Arc::clone(&library); | 2047 | let moving = Arc::clone(&library); |
| 2048 | let date = ["Thursday, October 1, 2026", "9:00 AM"]; | 2048 | let date = ["Thursday, October 1, 2026", "9:00 AM"]; |
| 2049 | let move_page = std::thread::spawn(move || { | 2049 | let move_page = std::thread::spawn(move || { |
| ... | @@ -4707,6 +4707,35 @@ fn every_page(root: &Path, cache: &Path) -> String { | ... | @@ -4707,6 +4707,35 @@ fn every_page(root: &Path, cache: &Path) -> String { |
| 4707 | shown | 4707 | shown |
| 4708 | } | 4708 | } |
| 4709 | 4709 | ||
| 4710 | /// The images the cache keeps of this device's publications. | ||
| 4711 | fn kept(cache: &Path) -> Vec<Vec<u8>> { | ||
| 4712 | let folders = std::fs::read_dir(cache.join("replicas")) | ||
| 4713 | .into_iter() | ||
| 4714 | .flatten(); | ||
| 4715 | folders | ||
| 4716 | .filter_map(|folder| std::fs::read_dir(folder.ok()?.path()).ok()) | ||
| 4717 | .flatten() | ||
| 4718 | .filter_map(|entry| { | ||
| 4719 | let path = entry.ok()?.path(); | ||
| 4720 | (path.extension()? == "published").then_some(path) | ||
| 4721 | }) | ||
| 4722 | .filter_map(|store| std::fs::read_dir(store).ok()) | ||
| 4723 | .flatten() | ||
| 4724 | .map(|image| std::fs::read(image.unwrap().path()).unwrap()) | ||
| 4725 | .collect() | ||
| 4726 | } | ||
| 4727 | |||
| 4728 | /// How many versions of a section are kept beside it in `root`, as "Name (iPhone).one". | ||
| 4729 | fn beside(root: &Path) -> usize { | ||
| 4730 | std::fs::read_dir(root) | ||
| 4731 | .unwrap() | ||
| 4732 | .filter(|entry| { | ||
| 4733 | let name = entry.as_ref().unwrap().file_name(); | ||
| 4734 | name.to_string_lossy().contains("(iPhone") | ||
| 4735 | }) | ||
| 4736 | .count() | ||
| 4737 | } | ||
| 4738 | |||
| 4710 | /// The titles and text of every page and conflict page each section file in `root` stores. | 4739 | /// The titles and text of every page and conflict page each section file in `root` stores. |
| 4711 | fn stored(root: &Path) -> String { | 4740 | fn stored(root: &Path) -> String { |
| 4712 | let mut text = String::new(); | 4741 | let mut text = String::new(); |
| ... | @@ -4741,13 +4770,11 @@ fn stored(root: &Path) -> String { | ... | @@ -4741,13 +4770,11 @@ fn stored(root: &Path) -> String { |
| 4741 | /// types on it, and a section the phone adds. The Mac's upload wins: iCloud Drive keeps the | 4770 | /// types on it, and a section the phone adds. The Mac's upload wins: iCloud Drive keeps the |
| 4742 | /// phone's file as a conflict version, or loses it. The phone's next launch opens, lists and | 4771 | /// phone's file as a conflict version, or loses it. The phone's next launch opens, lists and |
| 4743 | /// merges it back through the C surface as iOS does, its pages open and typed on in the | 4772 | /// merges it back through the C surface as iOS does, its pages open and typed on in the |
| 4744 | /// Reading View all the while. Nothing either device typed goes missing, merged or set aside | 4773 | /// Reading View all the while. Nothing either device typed goes missing, and nothing is kept |
| 4745 | /// whole. | 4774 | /// beside the file twice. |
| 4746 | #[test] | 4775 | #[test] |
| 4747 | fn an_icloud_notebook_merges_the_phones_upload_back_at_launch() { | 4776 | fn an_icloud_notebook_merges_the_phones_upload_back_at_launch() { |
| 4748 | unsafe { | 4777 | unsafe { library::sb_set_versions(cloud_versions, cloud_retire, c"iPhone".as_ptr()) }; |
| 4749 | library::sb_set_versions(cloud_versions, cloud_retire, c"Clover’s iPhone".as_ptr()) | ||
| 4750 | }; | ||
| 4751 | // Whether the devices' copies already share an edit of the page both type on, and | 4778 | // Whether the devices' copies already share an edit of the page both type on, and |
| 4752 | // whether iCloud Drive lost the phone's upload rather than keep it as a version. | 4779 | // whether iCloud Drive lost the phone's upload rather than keep it as a version. |
| 4753 | for (shared, lost) in [(false, false), (true, false), (false, true), (true, true)] { | 4780 | for (shared, lost) in [(false, false), (true, false), (false, true), (true, true)] { |
| ... | @@ -4825,7 +4852,7 @@ fn an_icloud_notebook_merges_the_phones_upload_back_at_launch() { | ... | @@ -4825,7 +4852,7 @@ fn an_icloud_notebook_merges_the_phones_upload_back_at_launch() { |
| 4825 | std::fs::rename(&file, &version).unwrap(); | 4852 | std::fs::rename(&file, &version).unwrap(); |
| 4826 | std::fs::copy(mac.join(&path), &file).unwrap(); | 4853 | std::fs::copy(mac.join(&path), &file).unwrap(); |
| 4827 | if !lost { | 4854 | if !lost { |
| 4828 | let device = "Clover’s iPhone".to_owned(); | 4855 | let device = "iPhone".to_owned(); |
| 4829 | let file = std::fs::canonicalize(&file).unwrap(); | 4856 | let file = std::fs::canonicalize(&file).unwrap(); |
| 4830 | CLOUD_VERSIONS.lock().unwrap().push((file, version, device)); | 4857 | CLOUD_VERSIONS.lock().unwrap().push((file, version, device)); |
| 4831 | } | 4858 | } |
| ... | @@ -4864,9 +4891,25 @@ fn an_icloud_notebook_merges_the_phones_upload_back_at_launch() { | ... | @@ -4864,9 +4891,25 @@ fn an_icloud_notebook_merges_the_phones_upload_back_at_launch() { |
| 4864 | unsafe { sb_insert(&mut view, c"After ".as_ptr()) }; | 4891 | unsafe { sb_insert(&mut view, c"After ".as_ptr()) }; |
| 4865 | sb_view_keep(&view); | 4892 | sb_view_keep(&view); |
| 4866 | } | 4893 | } |
| 4894 | // What the phone kept of its uploads goes once its file moves on, keeping them; only | ||
| 4895 | // what a file stands as stays kept. | ||
| 4896 | assert!( | ||
| 4897 | eventually(|| { | ||
| 4898 | section.shared.section.wake(); | ||
| 4899 | let files: Vec<Vec<u8>> = (std::fs::read_dir(&phone).unwrap()) | ||
| 4900 | .map(|entry| std::fs::read(entry.unwrap().path()).unwrap_or_default()) | ||
| 4901 | .collect(); | ||
| 4902 | kept(&folder("phone-cache")) | ||
| 4903 | .iter() | ||
| 4904 | .all(|image| files.contains(image)) | ||
| 4905 | }), | ||
| 4906 | "the phone's uploads stay kept (shared {shared}, lost {lost})" | ||
| 4907 | ); | ||
| 4867 | close_section(section); | 4908 | close_section(section); |
| 4868 | library.flush(Duration::from_secs(20)).unwrap(); | 4909 | library.flush(Duration::from_secs(20)).unwrap(); |
| 4869 | drop(library); | 4910 | drop(library); |
| 4911 | // Every upload merged: none is kept beside the file, nor twice. | ||
| 4912 | assert_eq!(beside(&phone), 0); | ||
| 4870 | 4913 | ||
| 4871 | let shown = every_page(&phone, &folder("phone-cache")); | 4914 | let shown = every_page(&phone, &folder("phone-cache")); |
| 4872 | for typed in [ | 4915 | for typed in [ |
| ... | @@ -4881,13 +4924,158 @@ fn an_icloud_notebook_merges_the_phones_upload_back_at_launch() { | ... | @@ -4881,13 +4924,158 @@ fn an_icloud_notebook_merges_the_phones_upload_back_at_launch() { |
| 4881 | } | 4924 | } |
| 4882 | } | 4925 | } |
| 4883 | 4926 | ||
| 4927 | /// A notebook Snowbound made in a scratch folder standing in for iCloud Drive's, opened as | ||
| 4928 | /// the phone opens one there, and its one section. | ||
| 4929 | fn cloud_notebook(directory: &Path) -> (PathBuf, PathBuf, String) { | ||
| 4930 | let (root, cache) = (directory.join("Notes"), directory.join("cache")); | ||
| 4931 | let creation = onestore::PageCreation::new(None, Some("Shared page"), "Clover").unwrap(); | ||
| 4932 | let notebook = | ||
| 4933 | notebook::session::Notebook::create(&root, &cache, 0x00f0_a0c0, &creation).unwrap(); | ||
| 4934 | let path = notebook.catalog().sections[0].path.clone(); | ||
| 4935 | (root, cache, path) | ||
| 4936 | } | ||
| 4937 | |||
| 4938 | /// Ages the images the cache keeps past the time reads get to catch up with an upload. | ||
| 4939 | fn aged(cache: &Path) { | ||
| 4940 | let folders = std::fs::read_dir(cache.join("replicas")).unwrap(); | ||
| 4941 | for folder in folders { | ||
| 4942 | for entry in std::fs::read_dir(folder.unwrap().path()).unwrap() { | ||
| 4943 | let store = entry.unwrap().path(); | ||
| 4944 | if store | ||
| 4945 | .extension() | ||
| 4946 | .is_none_or(|extension| extension != "published") | ||
| 4947 | { | ||
| 4948 | continue; | ||
| 4949 | } | ||
| 4950 | for image in std::fs::read_dir(store).unwrap() { | ||
| 4951 | let file = std::fs::File::options() | ||
| 4952 | .write(true) | ||
| 4953 | .open(image.unwrap().path()) | ||
| 4954 | .unwrap(); | ||
| 4955 | file.set_modified(std::time::SystemTime::now() - Duration::from_secs(61)) | ||
| 4956 | .unwrap(); | ||
| 4957 | } | ||
| 4958 | } | ||
| 4959 | } | ||
| 4960 | } | ||
| 4961 | |||
| 4962 | /// iCloud Drive puts back the file as it stood before the phone's upload, as when another | ||
| 4963 | /// device's upload made on that state wins: the phone's edits merge back once reads have had a | ||
| 4964 | /// minute to catch up with its upload, not before. | ||
| 4965 | #[test] | ||
| 4966 | fn an_upload_icloud_undid_merges_back_once_reads_have_caught_up() { | ||
| 4967 | unsafe { library::sb_set_versions(cloud_versions, cloud_retire, c"iPhone".as_ptr()) }; | ||
| 4968 | let directory = tempfile::tempdir().unwrap(); | ||
| 4969 | let (root, cache, path) = cloud_notebook(directory.path()); | ||
| 4970 | let library = cloud_library(&root, &cache); | ||
| 4971 | let section = cloud_section(&library, &path); | ||
| 4972 | let page = listed_ids(&section)[0]; | ||
| 4973 | type_on(&section, page, "Before "); | ||
| 4974 | assert!(section.flush(Duration::from_secs(20))); | ||
| 4975 | let before = std::fs::read(root.join(&path)).unwrap(); | ||
| 4976 | type_on(&section, page, "Phone words "); | ||
| 4977 | assert!(section.flush(Duration::from_secs(20))); | ||
| 4978 | std::fs::write(root.join(&path), before).unwrap(); | ||
| 4979 | for _ in 0..20 { | ||
| 4980 | section.shared.section.wake(); | ||
| 4981 | std::thread::sleep(Duration::from_millis(50)); | ||
| 4982 | } | ||
| 4983 | assert!(!stored(&root).contains("Phone words")); | ||
| 4984 | aged(&cache); | ||
| 4985 | assert!(eventually(|| { | ||
| 4986 | section.shared.section.wake(); | ||
| 4987 | stored(&root).contains("Phone words") | ||
| 4988 | })); | ||
| 4989 | close_section(section); | ||
| 4990 | } | ||
| 4991 | |||
| 4992 | /// A notebook on this device alone loses no upload, and keeps nothing of its publications. | ||
| 4993 | #[test] | ||
| 4994 | fn a_notebook_on_the_phone_alone_keeps_no_uploads() { | ||
| 4995 | unsafe { library::sb_set_versions(cloud_versions, cloud_retire, c"iPhone".as_ptr()) }; | ||
| 4996 | let directory = tempfile::tempdir().unwrap(); | ||
| 4997 | let (root, cache, path) = cloud_notebook(directory.path()); | ||
| 4998 | let library = Arc::new(Library::open(&root, &cache, true).unwrap()); | ||
| 4999 | let section = cloud_section(&library, &path); | ||
| 5000 | type_on(&section, listed_ids(&section)[0], "Phone words "); | ||
| 5001 | assert!(section.flush(Duration::from_secs(20))); | ||
| 5002 | assert!(stored(&root).contains("Phone words")); | ||
| 5003 | assert!(kept(&cache).is_empty()); | ||
| 5004 | close_section(section); | ||
| 5005 | } | ||
| 5006 | |||
| 5007 | /// Another device sets a password on the section: the file is written anew under a new | ||
| 5008 | /// identity, and the plaintext the phone kept of its uploads goes with the replica it backed. | ||
| 5009 | #[test] | ||
| 5010 | fn a_password_set_elsewhere_takes_the_plaintext_kept() { | ||
| 5011 | unsafe { library::sb_set_versions(cloud_versions, cloud_retire, c"iPhone".as_ptr()) }; | ||
| 5012 | let directory = tempfile::tempdir().unwrap(); | ||
| 5013 | let (root, cache, path) = cloud_notebook(directory.path()); | ||
| 5014 | let library = cloud_library(&root, &cache); | ||
| 5015 | let section = cloud_section(&library, &path); | ||
| 5016 | type_on(&section, listed_ids(&section)[0], "Secret words "); | ||
| 5017 | close_section(section); | ||
| 5018 | assert!(!kept(&cache).is_empty()); | ||
| 5019 | |||
| 5020 | let other = directory.path().join("other"); | ||
| 5021 | let copied = std::process::Command::new("cp") | ||
| 5022 | .arg("-R") | ||
| 5023 | .arg(&root) | ||
| 5024 | .arg(&other) | ||
| 5025 | .status(); | ||
| 5026 | assert!(copied.unwrap().success()); | ||
| 5027 | let mut notebook = | ||
| 5028 | notebook::session::Notebook::open(&other, directory.path().join("other-cache")).unwrap(); | ||
| 5029 | notebook | ||
| 5030 | .set_password(&path, None, Some("password")) | ||
| 5031 | .unwrap(); | ||
| 5032 | drop(notebook); | ||
| 5033 | for entry in std::fs::read_dir(&other).unwrap() { | ||
| 5034 | let entry = entry.unwrap(); | ||
| 5035 | if entry.path().is_file() { | ||
| 5036 | std::fs::copy(entry.path(), root.join(entry.file_name())).unwrap(); | ||
| 5037 | } | ||
| 5038 | } | ||
| 5039 | let tabs = library.tabs().unwrap(); | ||
| 5040 | assert!(tabs.iter().any(|tab| tab.path == path && tab.locked)); | ||
| 5041 | assert!(kept(&cache).is_empty()); | ||
| 5042 | } | ||
| 5043 | |||
| 5044 | /// A copy of a section made while the notebook is open keeps what it publishes of its own, | ||
| 5045 | /// beside its own replica, never in place of the original's. | ||
| 5046 | #[test] | ||
| 5047 | fn a_copy_found_while_open_keeps_its_own_uploads() { | ||
| 5048 | unsafe { library::sb_set_versions(cloud_versions, cloud_retire, c"iPhone".as_ptr()) }; | ||
| 5049 | let directory = tempfile::tempdir().unwrap(); | ||
| 5050 | let (root, cache, path) = cloud_notebook(directory.path()); | ||
| 5051 | let library = cloud_library(&root, &cache); | ||
| 5052 | let section = cloud_section(&library, &path); | ||
| 5053 | type_on(&section, listed_ids(&section)[0], "Original words "); | ||
| 5054 | close_section(section); | ||
| 5055 | let original = kept(&cache); | ||
| 5056 | assert_eq!(original.len(), 1); | ||
| 5057 | std::fs::copy(root.join(&path), root.join("Copy.one")).unwrap(); | ||
| 5058 | assert!( | ||
| 5059 | library | ||
| 5060 | .tabs() | ||
| 5061 | .unwrap() | ||
| 5062 | .iter() | ||
| 5063 | .any(|tab| tab.path == "Copy.one") | ||
| 5064 | ); | ||
| 5065 | let copy = cloud_section(&library, "Copy.one"); | ||
| 5066 | type_on(&copy, listed_ids(&copy)[0], "Copy words "); | ||
| 5067 | close_section(copy); | ||
| 5068 | let images = kept(&cache); | ||
| 5069 | assert_eq!(images.len(), 2); | ||
| 5070 | assert!(images.contains(&original[0])); | ||
| 5071 | assert!(images.contains(&std::fs::read(root.join("Copy.one")).unwrap())); | ||
| 5072 | } | ||
| 5073 | |||
| 4884 | /// The phone's upload to an iCloud Drive notebook is lost after the phone renamed the | 5074 | /// The phone's upload to an iCloud Drive notebook is lost after the phone renamed the |
| 4885 | /// notebook: the images its publications keep move with it, and the next launch merges it back. | 5075 | /// notebook: the images its publications keep move with it, and the next launch merges it back. |
| 4886 | #[test] | 5076 | #[test] |
| 4887 | fn an_upload_lost_after_the_notebook_is_renamed_merges_back() { | 5077 | fn an_upload_lost_after_the_notebook_is_renamed_merges_back() { |
| 4888 | unsafe { | 5078 | unsafe { library::sb_set_versions(cloud_versions, cloud_retire, c"iPhone".as_ptr()) }; |
| 4889 | library::sb_set_versions(cloud_versions, cloud_retire, c"Clover’s iPhone".as_ptr()) | ||
| 4890 | }; | ||
| 4891 | let directory = tempfile::tempdir().unwrap(); | 5079 | let directory = tempfile::tempdir().unwrap(); |
| 4892 | let folder = |name: &str| directory.path().join(name); | 5080 | let folder = |name: &str| directory.path().join(name); |
| 4893 | let (phone, mac, cache) = (folder("Notes"), folder("mac"), folder("phone-cache")); | 5081 | let (phone, mac, cache) = (folder("Notes"), folder("mac"), folder("phone-cache")); |
| ... | @@ -4916,7 +5104,8 @@ fn an_upload_lost_after_the_notebook_is_renamed_merges_back() { | ... | @@ -4916,7 +5104,8 @@ fn an_upload_lost_after_the_notebook_is_renamed_merges_back() { |
| 4916 | let section = cloud_section(&library, &path); | 5104 | let section = cloud_section(&library, &path); |
| 4917 | type_on(&section, page, typed); | 5105 | type_on(&section, page, typed); |
| 4918 | close_section(section); | 5106 | close_section(section); |
| 4919 | library.flush(Duration::from_secs(20)).unwrap(); | 5107 | // As the app closes a notebook before renaming it (Lists.swift). |
| 5108 | library.close(Duration::from_secs(20)).unwrap(); | ||
| 4920 | } | 5109 | } |
| 4921 | 5110 | ||
| 4922 | // Renamed in the app, as Arranging.swift renames a notebook's folder. | 5111 | // Renamed in the app, as Arranging.swift renames a notebook's folder. |
| ... | @@ -5025,7 +5214,9 @@ fn main_thread_calls_return_while_icloud_holds_the_section_file() { | ... | @@ -5025,7 +5214,9 @@ fn main_thread_calls_return_while_icloud_holds_the_section_file() { |
| 5025 | let file = root.canonicalize().unwrap().join(&path); | 5214 | let file = root.canonicalize().unwrap().join(&path); |
| 5026 | let (entered, held) = std::sync::mpsc::channel(); | 5215 | let (entered, held) = std::sync::mpsc::channel(); |
| 5027 | let (release, released) = std::sync::mpsc::channel(); | 5216 | let (release, released) = std::sync::mpsc::channel(); |
| 5028 | *HELD.lock().unwrap() = Some((file.to_string_lossy().into_owned(), entered, released)); | 5217 | HELD.lock() |
| 5218 | .unwrap() | ||
| 5219 | .push((file.to_string_lossy().into_owned(), entered, released)); | ||
| 5029 | type_on(&section, space, "Typed while iCloud downloads "); | 5220 | type_on(&section, space, "Typed while iCloud downloads "); |
| 5030 | sb_library_sync_now(&library); | 5221 | sb_library_sync_now(&library); |
| 5031 | held.recv_timeout(Duration::from_secs(10)).unwrap(); | 5222 | held.recv_timeout(Duration::from_secs(10)).unwrap(); |
crates/notebook/src/background.rs+23-19| ... | @@ -144,7 +144,7 @@ impl Background { | ... | @@ -144,7 +144,7 @@ impl Background { |
| 144 | ) -> Result<Self> | 144 | ) -> Result<Self> |
| 145 | where | 145 | where |
| 146 | R: Remote, | 146 | R: Remote, |
| 147 | B: FnMut(&str) -> R, | 147 | B: FnMut(&str, Option<&Path>) -> R, |
| 148 | L: FnMut(String) -> LF, | 148 | L: FnMut(String) -> LF, |
| 149 | LF: Future<Output = io::Result<Vec<Entry>>>, | 149 | LF: Future<Output = io::Result<Vec<Entry>>>, |
| 150 | { | 150 | { |
| ... | @@ -262,7 +262,7 @@ impl Background { | ... | @@ -262,7 +262,7 @@ impl Background { |
| 262 | continue; | 262 | continue; |
| 263 | }; | 263 | }; |
| 264 | let (queued, outcome) = step_async( | 264 | let (queued, outcome) = step_async( |
| 265 | &mut bind(&path), | 265 | &mut bind(&path, replica.as_deref()), |
| 266 | replica.as_deref(), | 266 | replica.as_deref(), |
| 267 | seen.as_deref(), | 267 | seen.as_deref(), |
| 268 | current, | 268 | current, |
| ... | @@ -360,7 +360,7 @@ impl Background { | ... | @@ -360,7 +360,7 @@ impl Background { |
| 360 | Err(error) => return Err(error), | 360 | Err(error) => return Err(error), |
| 361 | }; | 361 | }; |
| 362 | let (bound, folder) = (Arc::clone(&client), root.clone()); | 362 | let (bound, folder) = (Arc::clone(&client), root.clone()); |
| 363 | let bind = move |path: &str| { | 363 | let bind = move |path: &str, _: Option<&Path>| { |
| 364 | let file = match folder.as_str() { | 364 | let file = match folder.as_str() { |
| 365 | "" => path.to_owned(), | 365 | "" => path.to_owned(), |
| 366 | root => format!("{root}/{path}"), | 366 | root => format!("{root}/{path}"), |
| ... | @@ -394,7 +394,9 @@ impl Background { | ... | @@ -394,7 +394,9 @@ impl Background { |
| 394 | move |reports| { | 394 | move |reports| { |
| 395 | guest.watch(reports)?; | 395 | guest.watch(reports)?; |
| 396 | let (bound, listing) = (Arc::clone(&guest), Arc::clone(&guest)); | 396 | let (bound, listing) = (Arc::clone(&guest), Arc::clone(&guest)); |
| 397 | let bind = move |path: &str| crate::live::share::HostedRemote::new(&bound, path); | 397 | let bind = move |path: &str, _: Option<&Path>| { |
| 398 | crate::live::share::HostedRemote::new(&bound, path) | ||
| 399 | }; | ||
| 398 | let list = move |folder: String| { | 400 | let list = move |folder: String| { |
| 399 | let listing = listing.clone(); | 401 | let listing = listing.clone(); |
| 400 | async move { listing.entries_async(&folder).await } | 402 | async move { listing.entries_async(&folder).await } |
| ... | @@ -418,7 +420,9 @@ impl Background { | ... | @@ -418,7 +420,9 @@ impl Background { |
| 418 | true, | 420 | true, |
| 419 | move |_| { | 421 | move |_| { |
| 420 | let (bound, listing) = (Arc::clone(&documents), Arc::clone(&documents)); | 422 | let (bound, listing) = (Arc::clone(&documents), Arc::clone(&documents)); |
| 421 | let bind = move |path: &str| crate::provider::ProviderRemote::new(&bound, path); | 423 | let bind = move |path: &str, replica: Option<&Path>| { |
| 424 | crate::provider::ProviderRemote::new(&bound, path, replica) | ||
| 425 | }; | ||
| 422 | let list = move |folder: String| std::future::ready(listing.entries(&folder)); | 426 | let list = move |folder: String| std::future::ready(listing.entries(&folder)); |
| 423 | Ok(((bind, list), false)) | 427 | Ok(((bind, list), false)) |
| 424 | }, | 428 | }, |
| ... | @@ -909,17 +913,17 @@ async fn list<F: Future<Output = io::Result<Vec<Entry>>>>( | ... | @@ -909,17 +913,17 @@ async fn list<F: Future<Output = io::Result<Vec<Entry>>>>( |
| 909 | listings | 913 | listings |
| 910 | } | 914 | } |
| 911 | 915 | ||
| 912 | /// Deletes the replica at `replica` if it holds nothing unpublished and no one holds it. | 916 | /// Deletes the replica at `replica` if it holds nothing unpublished and no one holds it. One |
| 917 | /// whose section keeps images of its publications stays as an edit waiting keeps it: an image | ||
| 918 | /// may be an upload the file lost, which merges back once the notebook opens again. | ||
| 913 | fn discard(replica: &Path) { | 919 | fn discard(replica: &Path) { |
| 914 | if matches!( | 920 | if !crate::published::keeps(replica) |
| 915 | crate::closed(replica).and_then(|held| crate::peek(&held)), | 921 | && matches!( |
| 916 | Ok((_, 0)) | 922 | crate::closed(replica).and_then(|held| crate::peek(&held)), |
| 917 | ) { | 923 | Ok((_, 0)) |
| 918 | for suffix in ["-wal", "-shm", ""] { | 924 | ) |
| 919 | let mut file = replica.as_os_str().to_owned(); | 925 | { |
| 920 | file.push(suffix); | 926 | let _ = crate::remove_replica(replica); |
| 921 | let _ = fs::remove_file(file); | ||
| 922 | } | ||
| 923 | } | 927 | } |
| 924 | } | 928 | } |
| 925 | 929 | ||
| ... | @@ -1448,7 +1452,7 @@ mod tests { | ... | @@ -1448,7 +1452,7 @@ mod tests { |
| 1448 | move |reports| { | 1452 | move |reports| { |
| 1449 | armed.send(reports).unwrap(); | 1453 | armed.send(reports).unwrap(); |
| 1450 | let (files, listed) = (remote.clone(), remote.clone()); | 1454 | let (files, listed) = (remote.clone(), remote.clone()); |
| 1451 | let bind = move |path: &str| File(files.clone(), path.to_owned()); | 1455 | let bind = move |path: &str, _: Option<&Path>| File(files.clone(), path.to_owned()); |
| 1452 | let list = move |folder: String| std::future::ready(Ok(listed.list(&folder))); | 1456 | let list = move |folder: String| std::future::ready(Ok(listed.list(&folder))); |
| 1453 | Ok(((bind, list), true)) | 1457 | Ok(((bind, list), true)) |
| 1454 | }, | 1458 | }, |
| ... | @@ -1524,7 +1528,7 @@ mod tests { | ... | @@ -1524,7 +1528,7 @@ mod tests { |
| 1524 | move |reports| { | 1528 | move |reports| { |
| 1525 | armed.send(reports).unwrap(); | 1529 | armed.send(reports).unwrap(); |
| 1526 | let (files, listed) = (remote.clone(), remote.clone()); | 1530 | let (files, listed) = (remote.clone(), remote.clone()); |
| 1527 | let bind = move |path: &str| File(files.clone(), path.to_owned()); | 1531 | let bind = move |path: &str, _: Option<&Path>| File(files.clone(), path.to_owned()); |
| 1528 | let list = move |folder: String| std::future::ready(Ok(listed.list(&folder))); | 1532 | let list = move |folder: String| std::future::ready(Ok(listed.list(&folder))); |
| 1529 | Ok(((bind, list), true)) | 1533 | Ok(((bind, list), true)) |
| 1530 | }, | 1534 | }, |
| ... | @@ -1575,7 +1579,7 @@ mod tests { | ... | @@ -1575,7 +1579,7 @@ mod tests { |
| 1575 | false, | 1579 | false, |
| 1576 | move |_| { | 1580 | move |_| { |
| 1577 | let (files, listed) = (remote.clone(), remote.clone()); | 1581 | let (files, listed) = (remote.clone(), remote.clone()); |
| 1578 | let bind = move |path: &str| File(files.clone(), path.to_owned()); | 1582 | let bind = move |path: &str, _: Option<&Path>| File(files.clone(), path.to_owned()); |
| 1579 | let list = move |folder: String| std::future::ready(Ok(listed.list(&folder))); | 1583 | let list = move |folder: String| std::future::ready(Ok(listed.list(&folder))); |
| 1580 | Ok(((bind, list), true)) | 1584 | Ok(((bind, list), true)) |
| 1581 | }, | 1585 | }, |
| ... | @@ -1674,7 +1678,7 @@ mod tests { | ... | @@ -1674,7 +1678,7 @@ mod tests { |
| 1674 | move |reports| { | 1678 | move |reports| { |
| 1675 | armed.send(reports).unwrap(); | 1679 | armed.send(reports).unwrap(); |
| 1676 | let (files, listed) = (remote.clone(), remote.clone()); | 1680 | let (files, listed) = (remote.clone(), remote.clone()); |
| 1677 | let bind = move |path: &str| File(files.clone(), path.to_owned()); | 1681 | let bind = move |path: &str, _: Option<&Path>| File(files.clone(), path.to_owned()); |
| 1678 | let list = move |folder: String| std::future::ready(Ok(listed.list(&folder))); | 1682 | let list = move |folder: String| std::future::ready(Ok(listed.list(&folder))); |
| 1679 | Ok(((bind, list), true)) | 1683 | Ok(((bind, list), true)) |
| 1680 | }, | 1684 | }, |
crates/notebook/src/lib.rs+22| ... | @@ -153,6 +153,23 @@ impl Drop for Released { | ... | @@ -153,6 +153,23 @@ impl Drop for Released { |
| 153 | } | 153 | } |
| 154 | } | 154 | } |
| 155 | 155 | ||
| 156 | /// Deletes the closed replica at `replica`, with the files SQLite keeps beside it and the | ||
| 157 | /// images its section's publications keep (`published`). | ||
| 158 | pub(crate) fn remove_replica(replica: &Path) -> io::Result<()> { | ||
| 159 | for suffix in ["", "-wal", "-shm"] { | ||
| 160 | let mut file = replica.as_os_str().to_owned(); | ||
| 161 | file.push(suffix); | ||
| 162 | match fs::remove_file(file) { | ||
| 163 | Err(error) if error.kind() != io::ErrorKind::NotFound => return Err(error), | ||
| 164 | _ => {} | ||
| 165 | } | ||
| 166 | } | ||
| 167 | match fs::remove_dir_all(published::folder(replica)) { | ||
| 168 | Err(error) if error.kind() != io::ErrorKind::NotFound => Err(error), | ||
| 169 | _ => Ok(()), | ||
| 170 | } | ||
| 171 | } | ||
| 172 | |||
| 156 | impl Replica { | 173 | impl Replica { |
| 157 | /// Seeds a new cache from a validated section image, refusing any existing path. | 174 | /// Seeds a new cache from a validated section image, refusing any existing path. |
| 158 | /// An initialization error preserves the created file for inspection. | 175 | /// An initialization error preserves the created file for inspection. |
| ... | @@ -286,6 +303,11 @@ impl Replica { | ... | @@ -286,6 +303,11 @@ impl Replica { |
| 286 | lock(&self.section.cache) | 303 | lock(&self.section.cache) |
| 287 | } | 304 | } |
| 288 | 305 | ||
| 306 | /// Where the cache is. | ||
| 307 | pub fn path(&self) -> Option<PathBuf> { | ||
| 308 | self.lock().ok()?.path().map(PathBuf::from) | ||
| 309 | } | ||
| 310 | |||
| 289 | /// Hands `request` to the section thread. | 311 | /// Hands `request` to the section thread. |
| 290 | fn send(&self, request: working::Request) -> Result<()> { | 312 | fn send(&self, request: working::Request) -> Result<()> { |
| 291 | self.section.send(request) | 313 | self.section.send(request) |
crates/notebook/src/location.rs+33-49| ... | @@ -41,29 +41,37 @@ pub fn smb(address: &str, share: &str, root: &str) -> String { | ... | @@ -41,29 +41,37 @@ pub fn smb(address: &str, share: &str, root: &str) -> String { |
| 41 | format!("smb://{address}/{share}/{}", root.trim_matches('/')) | 41 | format!("smb://{address}/{share}/{}", root.trim_matches('/')) |
| 42 | } | 42 | } |
| 43 | 43 | ||
| 44 | /// Moves the replicas of the sections at `from` to `to`, and the images their publications | 44 | /// Moves the replicas of the sections at `from` to `to`, with the images their publications |
| 45 | /// keep (`published`), as when the app renames or moves a notebook; with them those of every | 45 | /// keep (`published`), as when the app renames or moves a notebook; with them those of every |
| 46 | /// location within it, as a copy of a section keeps its own. Each replica `to` already holds | 46 | /// location within it, as a copy of a section keeps its own. Each replica `to` already holds |
| 47 | /// for the same section stays. Every replica moved must be closed. | 47 | /// for the same section stays. Every replica moved must be closed. |
| 48 | pub fn moved(cache: &Path, from: &str, to: &str) -> io::Result<()> { | 48 | pub fn moved(cache: &Path, from: &str, to: &str) -> io::Result<()> { |
| 49 | let within = format!("{from}/"); | 49 | let inner = within(cache, from); |
| 50 | let inner: Vec<String> = fs::read_dir(cache.join("replicas")) | ||
| 51 | .into_iter() | ||
| 52 | .flatten() | ||
| 53 | .filter_map(|entry| fs::read_to_string(entry.ok()?.path().join(LOCATION)).ok()) | ||
| 54 | .filter(|location| location.starts_with(&within)) | ||
| 55 | .collect(); | ||
| 56 | move_folder(cache, from, to)?; | 50 | move_folder(cache, from, to)?; |
| 57 | for location in inner { | 51 | for (_, location) in inner { |
| 58 | move_folder( | 52 | move_folder( |
| 59 | cache, | 53 | cache, |
| 60 | &location, | 54 | &location, |
| 61 | &format!("{to}/{}", &location[within.len()..]), | 55 | &format!("{to}{}", &location[from.len()..]), |
| 62 | )?; | 56 | )?; |
| 63 | } | 57 | } |
| 64 | Ok(()) | 58 | Ok(()) |
| 65 | } | 59 | } |
| 66 | 60 | ||
| 61 | /// The folders of the locations within `location`, labelled, with their locations. | ||
| 62 | fn within(cache: &Path, location: &str) -> Vec<(PathBuf, String)> { | ||
| 63 | let within = format!("{location}/"); | ||
| 64 | fs::read_dir(cache.join("replicas")) | ||
| 65 | .into_iter() | ||
| 66 | .flatten() | ||
| 67 | .filter_map(|entry| { | ||
| 68 | let folder = entry.ok()?.path(); | ||
| 69 | let label = fs::read_to_string(folder.join(LOCATION)).ok()?; | ||
| 70 | label.starts_with(&within).then_some((folder, label)) | ||
| 71 | }) | ||
| 72 | .collect() | ||
| 73 | } | ||
| 74 | |||
| 67 | fn move_folder(cache: &Path, from: &str, to: &str) -> io::Result<()> { | 75 | fn move_folder(cache: &Path, from: &str, to: &str) -> io::Result<()> { |
| 68 | let old = folder(cache, from); | 76 | let old = folder(cache, from); |
| 69 | if fs::metadata(&old).is_err() { | 77 | if fs::metadata(&old).is_err() { |
| ... | @@ -78,50 +86,22 @@ fn move_folder(cache: &Path, from: &str, to: &str) -> io::Result<()> { | ... | @@ -78,50 +86,22 @@ fn move_folder(cache: &Path, from: &str, to: &str) -> io::Result<()> { |
| 78 | adopt(&old.join(&name), &new.join(&name))?; | 86 | adopt(&old.join(&name), &new.join(&name))?; |
| 79 | } | 87 | } |
| 80 | } | 88 | } |
| 81 | if let Ok(images) = fs::read_dir(old.join(PUBLISHED)) { | ||
| 82 | fs::create_dir_all(new.join(PUBLISHED))?; | ||
| 83 | for entry in images { | ||
| 84 | let name = entry?.file_name(); | ||
| 85 | let image = new.join(PUBLISHED).join(&name); | ||
| 86 | if fs::metadata(&image).is_err() { | ||
| 87 | fs::rename(old.join(PUBLISHED).join(&name), image)?; | ||
| 88 | } | ||
| 89 | } | ||
| 90 | let _ = fs::remove_dir(old.join(PUBLISHED)); | ||
| 91 | } | ||
| 92 | tidy(&old); | 89 | tidy(&old); |
| 93 | Ok(()) | 90 | Ok(()) |
| 94 | } | 91 | } |
| 95 | 92 | ||
| 96 | /// The folder of a replica folder holding the images its sections' publications keep. | ||
| 97 | pub const PUBLISHED: &str = "published"; | ||
| 98 | |||
| 99 | /// The replica folder of the section file at location `file` in the notebook at location | ||
| 100 | /// `notebook`: the file's own where one was claimed, as a copy's and a lone section's are, | ||
| 101 | /// else the notebook's. | ||
| 102 | pub fn section_folder(cache: &Path, notebook: &str, file: &str) -> PathBuf { | ||
| 103 | let own = folder(cache, file); | ||
| 104 | if fs::metadata(own.join(LOCATION)).is_ok() { | ||
| 105 | own | ||
| 106 | } else { | ||
| 107 | folder(cache, notebook) | ||
| 108 | } | ||
| 109 | } | ||
| 110 | |||
| 111 | /// Deletes what the cache keeps of the notebook at `location`, as when the app deletes the | 93 | /// Deletes what the cache keeps of the notebook at `location`, as when the app deletes the |
| 112 | /// notebook: its replicas, edits waiting in them included, its sections' copies' and its | 94 | /// notebook: its replicas, edits waiting in them included, its sections' copies' and its |
| 113 | /// listing. Every replica must be closed. | 95 | /// listing. Every replica must be closed. |
| 114 | pub fn forget(cache: &Path, location: &str) -> io::Result<()> { | 96 | pub fn forget(cache: &Path, location: &str) -> io::Result<()> { |
| 115 | let within = format!("{location}/"); | 97 | let own = folder(cache, location); |
| 116 | if let Ok(entries) = fs::read_dir(cache.join("replicas")) { | 98 | let ours = fs::read_to_string(own.join(LOCATION)).is_ok_and(|label| label == location); |
| 117 | for entry in entries { | 99 | for folder in ours.then_some(own).into_iter().chain( |
| 118 | let folder = entry?.path(); | 100 | within(cache, location) |
| 119 | let ours = fs::read_to_string(folder.join(LOCATION)) | 101 | .into_iter() |
| 120 | .is_ok_and(|label| label == location || label.starts_with(&within)); | 102 | .map(|(folder, _)| folder), |
| 121 | if ours { | 103 | ) { |
| 122 | fs::remove_dir_all(&folder)?; | 104 | fs::remove_dir_all(&folder)?; |
| 123 | } | ||
| 124 | } | ||
| 125 | } | 105 | } |
| 126 | match fs::remove_file(crate::session::listing(cache, location)) { | 106 | match fs::remove_file(crate::session::listing(cache, location)) { |
| 127 | Err(error) if error.kind() != io::ErrorKind::NotFound => Err(error), | 107 | Err(error) if error.kind() != io::ErrorKind::NotFound => Err(error), |
| ... | @@ -130,7 +110,7 @@ pub fn forget(cache: &Path, location: &str) -> io::Result<()> { | ... | @@ -130,7 +110,7 @@ pub fn forget(cache: &Path, location: &str) -> io::Result<()> { |
| 130 | } | 110 | } |
| 131 | 111 | ||
| 132 | /// The folder for `location`, made and labelled with it where it was not. | 112 | /// The folder for `location`, made and labelled with it where it was not. |
| 133 | fn claimed(cache: &Path, location: &str) -> io::Result<PathBuf> { | 113 | pub fn claimed(cache: &Path, location: &str) -> io::Result<PathBuf> { |
| 134 | let folder = folder(cache, location); | 114 | let folder = folder(cache, location); |
| 135 | let label = folder.join(LOCATION); | 115 | let label = folder.join(LOCATION); |
| 136 | if fs::metadata(&label).is_err() { | 116 | if fs::metadata(&label).is_err() { |
| ... | @@ -223,7 +203,8 @@ fn follows(file: &Stamp, base: &Stamp) -> bool { | ... | @@ -223,7 +203,8 @@ fn follows(file: &Stamp, base: &Stamp) -> bool { |
| 223 | ) | 203 | ) |
| 224 | } | 204 | } |
| 225 | 205 | ||
| 226 | /// Moves the replica at `from`, with the write-ahead log a crash may have left, to `to`. | 206 | /// Moves the replica at `from`, with the write-ahead log a crash may have left and the images |
| 207 | /// its publications keep, to `to`. | ||
| 227 | fn adopt(from: &Path, to: &Path) -> io::Result<()> { | 208 | fn adopt(from: &Path, to: &Path) -> io::Result<()> { |
| 228 | for suffix in ["-wal", "-shm", ""] { | 209 | for suffix in ["-wal", "-shm", ""] { |
| 229 | let (mut source, mut target) = (from.as_os_str().to_owned(), to.as_os_str().to_owned()); | 210 | let (mut source, mut target) = (from.as_os_str().to_owned(), to.as_os_str().to_owned()); |
| ... | @@ -234,7 +215,10 @@ fn adopt(from: &Path, to: &Path) -> io::Result<()> { | ... | @@ -234,7 +215,10 @@ fn adopt(from: &Path, to: &Path) -> io::Result<()> { |
| 234 | moved => moved?, | 215 | moved => moved?, |
| 235 | } | 216 | } |
| 236 | } | 217 | } |
| 237 | Ok(()) | 218 | match fs::rename(crate::published::folder(from), crate::published::folder(to)) { |
| 219 | Err(error) if error.kind() != io::ErrorKind::NotFound => Err(error), | ||
| 220 | _ => Ok(()), | ||
| 221 | } | ||
| 238 | } | 222 | } |
| 239 | 223 | ||
| 240 | /// Removes a replica folder that holds no replica, with the owner lock a crash may have left | 224 | /// Removes a replica folder that holds no replica, with the owner lock a crash may have left |
crates/notebook/src/provider.rs+25-29| ... | @@ -6,7 +6,7 @@ | ... | @@ -6,7 +6,7 @@ |
| 6 | 6 | ||
| 7 | use crate::{ | 7 | use crate::{ |
| 8 | Error, Result, discover, | 8 | Error, Result, discover, |
| 9 | published::{Published, Written}, | 9 | published::{Published, Written, whole}, |
| 10 | session::Storage, | 10 | session::Storage, |
| 11 | }; | 11 | }; |
| 12 | use onestore::{CommitError, CommitIo, CommitState, Stamp, Store, Transaction}; | 12 | use onestore::{CommitError, CommitIo, CommitState, Stamp, Store, Transaction}; |
| ... | @@ -14,7 +14,7 @@ use std::{ | ... | @@ -14,7 +14,7 @@ use std::{ |
| 14 | collections::BTreeMap, | 14 | collections::BTreeMap, |
| 15 | fs::File, | 15 | fs::File, |
| 16 | io::{self, Read, Seek, SeekFrom, Write}, | 16 | io::{self, Read, Seek, SeekFrom, Write}, |
| 17 | path::PathBuf, | 17 | path::{Path, PathBuf}, |
| 18 | sync::{Arc, Mutex, MutexGuard, PoisonError}, | 18 | sync::{Arc, Mutex, MutexGuard, PoisonError}, |
| 19 | }; | 19 | }; |
| 20 | 20 | ||
| ... | @@ -78,12 +78,6 @@ impl Documents { | ... | @@ -78,12 +78,6 @@ impl Documents { |
| 78 | format!("provider:{}", self.root) | 78 | format!("provider:{}", self.root) |
| 79 | } | 79 | } |
| 80 | 80 | ||
| 81 | /// Where this device keeps the images it published until the documents are seen holding | ||
| 82 | /// them: a provider without compare-and-swap may let another device's upload replace one. | ||
| 83 | fn published(&self) -> PathBuf { | ||
| 84 | crate::location::folder(&self.cache, &self.location()).join(crate::location::PUBLISHED) | ||
| 85 | } | ||
| 86 | |||
| 87 | fn id(&self, path: &str) -> io::Result<String> { | 81 | fn id(&self, path: &str) -> io::Result<String> { |
| 88 | if path.is_empty() { | 82 | if path.is_empty() { |
| 89 | return Ok(self.root.clone()); | 83 | return Ok(self.root.clone()); |
| ... | @@ -288,15 +282,6 @@ fn lock<T>(mutex: &Mutex<T>) -> MutexGuard<'_, T> { | ... | @@ -288,15 +282,6 @@ fn lock<T>(mutex: &Mutex<T>) -> MutexGuard<'_, T> { |
| 288 | mutex.lock().unwrap_or_else(PoisonError::into_inner) | 282 | mutex.lock().unwrap_or_else(PoisonError::into_inner) |
| 289 | } | 283 | } |
| 290 | 284 | ||
| 291 | fn whole(file: &mut File, limit: usize) -> io::Result<Vec<u8>> { | ||
| 292 | let mut bytes = Vec::new(); | ||
| 293 | file.take(limit as u64 + 1).read_to_end(&mut bytes)?; | ||
| 294 | if bytes.len() > limit { | ||
| 295 | return Err(io::ErrorKind::FileTooLarge.into()); | ||
| 296 | } | ||
| 297 | Ok(bytes) | ||
| 298 | } | ||
| 299 | |||
| 300 | /// A document as a commit writes it: its seekable descriptor, or its image to write whole. | 285 | /// A document as a commit writes it: its seekable descriptor, or its image to write whole. |
| 301 | enum Io { | 286 | enum Io { |
| 302 | File(File), | 287 | File(File), |
| ... | @@ -349,21 +334,22 @@ impl CommitIo for Io { | ... | @@ -349,21 +334,22 @@ impl CommitIo for Io { |
| 349 | } | 334 | } |
| 350 | 335 | ||
| 351 | /// A section a document provider keeps, as its replica publishes to it, each image it | 336 | /// A section a document provider keeps, as its replica publishes to it, each image it |
| 352 | /// publishes kept until the document holds it (`published.rs`). | 337 | /// publishes kept beside the replica until the document holds it (`published.rs`). |
| 353 | pub(crate) struct ProviderRemote { | 338 | pub(crate) struct ProviderRemote { |
| 354 | documents: Arc<Documents>, | 339 | documents: Arc<Documents>, |
| 355 | path: String, | 340 | path: String, |
| 356 | published: Published, | 341 | /// None for a section with no replica yet, which publishes nothing. |
| 342 | published: Option<Published>, | ||
| 357 | /// The image the step read whole for a stamp, which its read takes. | 343 | /// The image the step read whole for a stamp, which its read takes. |
| 358 | read: Option<Vec<u8>>, | 344 | read: Option<Vec<u8>>, |
| 359 | } | 345 | } |
| 360 | 346 | ||
| 361 | impl ProviderRemote { | 347 | impl ProviderRemote { |
| 362 | pub(crate) fn new(documents: &Arc<Documents>, path: &str) -> Self { | 348 | pub(crate) fn new(documents: &Arc<Documents>, path: &str, replica: Option<&Path>) -> Self { |
| 363 | Self { | 349 | Self { |
| 364 | documents: Arc::clone(documents), | 350 | documents: Arc::clone(documents), |
| 365 | path: path.to_owned(), | 351 | path: path.to_owned(), |
| 366 | published: Published::new(documents.published(), Some(documents.device.clone())), | 352 | published: replica.map(|replica| Published::of(replica, &documents.device)), |
| 367 | read: None, | 353 | read: None, |
| 368 | } | 354 | } |
| 369 | } | 355 | } |
| ... | @@ -372,10 +358,8 @@ impl ProviderRemote { | ... | @@ -372,10 +358,8 @@ impl ProviderRemote { |
| 372 | /// put back and read again: the newest image kept that the torn bytes begin. Any other | 358 | /// put back and read again: the newest image kept that the torn bytes begin. Any other |
| 373 | /// document that does not parse is left as it is. | 359 | /// document that does not parse is left as it is. |
| 374 | fn repair(&self, torn: &[u8]) -> io::Result<Vec<u8>> { | 360 | fn repair(&self, torn: &[u8]) -> io::Result<Vec<u8>> { |
| 375 | let image = self | 361 | let published = self.published.as_ref().ok_or(io::ErrorKind::WouldBlock)?; |
| 376 | .published | 362 | let image = published.torn(torn)?.ok_or(io::ErrorKind::WouldBlock)?; |
| 377 | .torn(torn)? | ||
| 378 | .ok_or(io::ErrorKind::WouldBlock)?; | ||
| 379 | let id = self.documents.id(&self.path)?; | 363 | let id = self.documents.id(&self.path)?; |
| 380 | self.documents.writable(&id)?; | 364 | self.documents.writable(&id)?; |
| 381 | self.documents | 365 | self.documents |
| ... | @@ -420,14 +404,23 @@ impl crate::Remote for ProviderRemote { | ... | @@ -420,14 +404,23 @@ impl crate::Remote for ProviderRemote { |
| 420 | self.documents.write( | 404 | self.documents.write( |
| 421 | &self.path, | 405 | &self.path, |
| 422 | |io| transaction.commit(io), | 406 | |io| transaction.commit(io), |
| 423 | |written| self.published.keep(written, transaction), | 407 | |written| match &self.published { |
| 408 | Some(published) => published.keep(written, transaction), | ||
| 409 | None => Ok(()), | ||
| 410 | }, | ||
| 424 | ) | 411 | ) |
| 425 | } | 412 | } |
| 426 | 413 | ||
| 427 | fn confirm(&mut self, base: &Stamp) -> std::result::Result<(), CommitError> { | 414 | fn confirm(&mut self, base: &Stamp) -> std::result::Result<(), CommitError> { |
| 428 | self.read = None; | 415 | self.read = None; |
| 429 | self.documents | 416 | self.documents.write( |
| 430 | .write(&self.path, |io| onestore::confirm(io, base), |_| Ok(())) | 417 | &self.path, |
| 418 | |io| onestore::confirm(io, base), | ||
| 419 | |written| match &self.published { | ||
| 420 | Some(published) => published.confirmed(written, base), | ||
| 421 | None => Ok(()), | ||
| 422 | }, | ||
| 423 | ) | ||
| 431 | } | 424 | } |
| 432 | 425 | ||
| 433 | /// The images kept of this document other than the one it stands as | 426 | /// The images kept of this document other than the one it stands as |
| ... | @@ -441,7 +434,10 @@ impl crate::Remote for ProviderRemote { | ... | @@ -441,7 +434,10 @@ impl crate::Remote for ProviderRemote { |
| 441 | (stamp, image) = (repaired_stamp, Some(repaired)); | 434 | (stamp, image) = (repaired_stamp, Some(repaired)); |
| 442 | } | 435 | } |
| 443 | self.read = image; | 436 | self.read = image; |
| 444 | Ok(self.published.versions(&stamp)) | 437 | match &self.published { |
| 438 | Some(published) => published.versions(|| Ok(stamp), &[]), | ||
| 439 | None => Ok(Vec::new()), | ||
| 440 | } | ||
| 445 | } | 441 | } |
| 446 | 442 | ||
| 447 | fn version(&mut self, id: &str) -> io::Result<Vec<u8>> { | 443 | fn version(&mut self, id: &str) -> io::Result<Vec<u8>> { |
crates/notebook/src/provider/tests.rs+19-13| ... | @@ -899,8 +899,8 @@ fn a_provider_that_lists_but_cannot_open_opens_sections_from_their_replicas() { | ... | @@ -899,8 +899,8 @@ fn a_provider_that_lists_but_cannot_open_opens_sections_from_their_replicas() { |
| 899 | section.close().unwrap(); | 899 | section.close().unwrap(); |
| 900 | } | 900 | } |
| 901 | 901 | ||
| 902 | /// The images a device keeps of what it published, for every notebook in its cache. | 902 | /// Where a device keeps the images it published, beside each replica in its cache. |
| 903 | fn kept_images(fake: &Fake) -> Vec<String> { | 903 | fn stores(fake: &Fake) -> Vec<std::path::PathBuf> { |
| 904 | let cache = fake | 904 | let cache = fake |
| 905 | .store | 905 | .store |
| 906 | .folder | 906 | .folder |
| ... | @@ -908,7 +908,21 @@ fn kept_images(fake: &Fake) -> Vec<String> { | ... | @@ -908,7 +908,21 @@ fn kept_images(fake: &Fake) -> Vec<String> { |
| 908 | .join(format!("cache-{}", fake.device)); | 908 | .join(format!("cache-{}", fake.device)); |
| 909 | let folders = fs::read_dir(cache.join("replicas")).into_iter().flatten(); | 909 | let folders = fs::read_dir(cache.join("replicas")).into_iter().flatten(); |
| 910 | folders | 910 | folders |
| 911 | .filter_map(|folder| fs::read_dir(folder.ok()?.path().join("published")).ok()) | 911 | .filter_map(|folder| fs::read_dir(folder.ok()?.path()).ok()) |
| 912 | .flatten() | ||
| 913 | .map(|entry| entry.unwrap().path()) | ||
| 914 | .filter(|path| { | ||
| 915 | path.extension() | ||
| 916 | .is_some_and(|extension| extension == "published") | ||
| 917 | }) | ||
| 918 | .collect() | ||
| 919 | } | ||
| 920 | |||
| 921 | /// The images a device keeps of what it published, for every notebook in its cache. | ||
| 922 | fn kept_images(fake: &Fake) -> Vec<String> { | ||
| 923 | stores(fake) | ||
| 924 | .into_iter() | ||
| 925 | .filter_map(|store| fs::read_dir(store).ok()) | ||
| 912 | .flatten() | 926 | .flatten() |
| 913 | .map(|entry| entry.unwrap().file_name().to_string_lossy().into_owned()) | 927 | .map(|entry| entry.unwrap().file_name().to_string_lossy().into_owned()) |
| 914 | .filter(|name| name.ends_with(".one")) | 928 | .filter(|name| name.ends_with(".one")) |
| ... | @@ -962,16 +976,8 @@ fn a_publication_another_devices_upload_replaced_merges_back() { | ... | @@ -962,16 +976,8 @@ fn a_publication_another_devices_upload_replaced_merges_back() { |
| 962 | 976 | ||
| 963 | /// Ages the images a device keeps past `LAG`, as if reads had long had time to catch up. | 977 | /// Ages the images a device keeps past `LAG`, as if reads had long had time to catch up. |
| 964 | fn aged(fake: &Fake) { | 978 | fn aged(fake: &Fake) { |
| 965 | let cache = fake | 979 | for store in stores(fake) { |
| 966 | .store | 980 | for image in fs::read_dir(store).into_iter().flatten() { |
| 967 | .folder | ||
| 968 | .path() | ||
| 969 | .join(format!("cache-{}", fake.device)); | ||
| 970 | for folder in fs::read_dir(cache.join("replicas")).unwrap() { | ||
| 971 | for image in fs::read_dir(folder.unwrap().path().join("published")) | ||
| 972 | .into_iter() | ||
| 973 | .flatten() | ||
| 974 | { | ||
| 975 | let file = File::options() | 981 | let file = File::options() |
| 976 | .write(true) | 982 | .write(true) |
| 977 | .open(image.unwrap().path()) | 983 | .open(image.unwrap().path()) |
crates/notebook/src/published.rs+253-52| ... | @@ -1,8 +1,9 @@ | ... | @@ -1,8 +1,9 @@ |
| 1 | //! The images this device publishes to section files a provider syncs whole, with no lock or | 1 | //! The images this device publishes to section files a provider syncs whole, with no lock or |
| 2 | //! compare-and-swap, as iCloud Drive and Android's document providers do: another device's | 2 | //! compare-and-swap, as iCloud Drive and Android's document providers do: another device's |
| 3 | //! upload can replace this one's. Each image is kept until the file is seen holding it, and | 3 | //! upload can replace this one's. Each image is kept beside the section's replica until the |
| 4 | //! one the file lost is listed as a `Remote::versions` version, so that it merges back as a | 4 | //! file is seen holding it, and one the file lost is listed as a `Remote::versions` version, |
| 5 | //! conflict version does (`resolve.rs`). | 5 | //! so that it merges back as a conflict version does (`resolve.rs`). They move and go with |
| 6 | //! the replica (`location::moved`, `remove_replica`). | ||
| 6 | 7 | ||
| 7 | use crate::Version; | 8 | use crate::Version; |
| 8 | use onestore::{Stamp, Transaction}; | 9 | use onestore::{Stamp, Transaction}; |
| ... | @@ -23,13 +24,13 @@ pub enum Written<'a> { | ... | @@ -23,13 +24,13 @@ pub enum Written<'a> { |
| 23 | Image(&'a [u8]), | 24 | Image(&'a [u8]), |
| 24 | } | 25 | } |
| 25 | 26 | ||
| 26 | /// The images this device published to the files of one place, kept in `folder` and named | 27 | /// The images this device published to one section file, named `<document>-<image>-<base>.one`: |
| 27 | /// `<document>-<image>-<base>.one`: the file's identity, then the digests of the image's stamp | 28 | /// the file's identity, then the digests of the image's stamp and of the stamp of the image it |
| 28 | /// and of the stamp of the image it was published on. Version ids are their paths. | 29 | /// was published on. Version ids are their paths. |
| 29 | pub struct Published { | 30 | pub struct Published { |
| 30 | folder: PathBuf, | 31 | folder: PathBuf, |
| 31 | /// This device, as the conflict pages a lost publication makes name it. | 32 | /// This device, as the conflict pages a lost publication makes name it. |
| 32 | device: Option<String>, | 33 | device: String, |
| 33 | } | 34 | } |
| 34 | 35 | ||
| 35 | struct Kept { | 36 | struct Kept { |
| ... | @@ -40,21 +41,22 @@ struct Kept { | ... | @@ -40,21 +41,22 @@ struct Kept { |
| 40 | at: SystemTime, | 41 | at: SystemTime, |
| 41 | } | 42 | } |
| 42 | 43 | ||
| 44 | /// The folder beside the replica at `replica` holding its section's images. | ||
| 45 | pub(crate) fn folder(replica: &Path) -> PathBuf { | ||
| 46 | replica.with_extension("published") | ||
| 47 | } | ||
| 48 | |||
| 43 | impl Published { | 49 | impl Published { |
| 44 | pub fn new(folder: impl Into<PathBuf>, device: Option<String>) -> Self { | 50 | /// The images kept of the section whose replica is at `replica`. |
| 51 | pub fn of(replica: &Path, device: impl Into<String>) -> Self { | ||
| 45 | Self { | 52 | Self { |
| 46 | folder: folder.into(), | 53 | folder: folder(replica), |
| 47 | device, | 54 | device: device.into(), |
| 48 | } | 55 | } |
| 49 | } | 56 | } |
| 50 | 57 | ||
| 51 | /// The images kept of the section file `file` of the notebook folder, or lone section, at | 58 | pub fn device(&self) -> &str { |
| 52 | /// `notebook`, which `cache` holds the replicas of: beside the section's replica, so that | 59 | &self.device |
| 53 | /// they move with the notebook (`location::moved`). | ||
| 54 | pub fn of(cache: &Path, notebook: &Path, file: &Path, device: Option<String>) -> Option<Self> { | ||
| 55 | use crate::location::{PUBLISHED, local, section_folder}; | ||
| 56 | let folder = section_folder(cache, &local(notebook).ok()?, &local(file).ok()?); | ||
| 57 | Some(Self::new(folder.join(PUBLISHED), device)) | ||
| 58 | } | 60 | } |
| 59 | 61 | ||
| 60 | fn kept(&self) -> Vec<Kept> { | 62 | fn kept(&self) -> Vec<Kept> { |
| ... | @@ -75,6 +77,19 @@ impl Published { | ... | @@ -75,6 +77,19 @@ impl Published { |
| 75 | .collect() | 77 | .collect() |
| 76 | } | 78 | } |
| 77 | 79 | ||
| 80 | fn put(&self, image: &[u8], name: String, replacing: Option<Kept>) -> io::Result<()> { | ||
| 81 | crate::fs::create_dir_all(&self.folder)?; | ||
| 82 | let mut kept = tempfile::NamedTempFile::new_in(&self.folder)?; | ||
| 83 | kept.write_all(image)?; | ||
| 84 | kept.as_file().sync_all()?; | ||
| 85 | kept.persist(self.folder.join(name)) | ||
| 86 | .map_err(|error| error.error)?; | ||
| 87 | if let Some(kept) = replacing { | ||
| 88 | crate::fs::remove_file(kept.path)?; | ||
| 89 | } | ||
| 90 | Ok(()) | ||
| 91 | } | ||
| 92 | |||
| 78 | /// Keeps the image `transaction` published, in place of the one it was published on: | 93 | /// Keeps the image `transaction` published, in place of the one it was published on: |
| 79 | /// that kept image with the transaction applied where this device kept it, else read back. | 94 | /// that kept image with the transaction applied where this device kept it, else read back. |
| 80 | pub fn keep(&self, written: Written<'_>, transaction: &Transaction) -> io::Result<()> { | 95 | pub fn keep(&self, written: Written<'_>, transaction: &Transaction) -> io::Result<()> { |
| ... | @@ -93,31 +108,46 @@ impl Published { | ... | @@ -93,31 +108,46 @@ impl Published { |
| 93 | } | 108 | } |
| 94 | (Written::InPlace(file), None) => { | 109 | (Written::InPlace(file), None) => { |
| 95 | file.seek(SeekFrom::Start(0))?; | 110 | file.seek(SeekFrom::Start(0))?; |
| 96 | let mut image = Vec::new(); | 111 | whole(file, crate::MAX_FILE_BYTES)? |
| 97 | file.take(crate::MAX_FILE_BYTES as u64 + 1) | ||
| 98 | .read_to_end(&mut image)?; | ||
| 99 | if image.len() > crate::MAX_FILE_BYTES { | ||
| 100 | return Err(io::ErrorKind::FileTooLarge.into()); | ||
| 101 | } | ||
| 102 | image | ||
| 103 | } | 112 | } |
| 104 | }; | 113 | }; |
| 105 | let name = format!("{document}-{}-{base}.one", digest(&transaction.stamp())); | 114 | let name = format!("{document}-{}-{base}.one", digest(&transaction.stamp())); |
| 106 | crate::fs::create_dir_all(&self.folder)?; | 115 | self.put(&image, name, on) |
| 107 | let mut kept = tempfile::NamedTempFile::new_in(&self.folder)?; | 116 | } |
| 108 | kept.write_all(&image)?; | 117 | |
| 109 | kept.as_file().sync_all()?; | 118 | /// Follows the image kept as `base` through a confirmation (`onestore::confirm`), which |
| 110 | kept.persist(self.folder.join(name)) | 119 | /// rewrites only the header: the image `written` now begins with. |
| 111 | .map_err(|error| error.error)?; | 120 | pub fn confirmed(&self, written: Written<'_>, base: &Stamp) -> io::Result<()> { |
| 112 | if let Some(kept) = on { | 121 | let image = digest(base); |
| 113 | crate::fs::remove_file(kept.path)?; | 122 | let Some(kept) = (self.kept().into_iter()).find(|kept| kept.image == image) else { |
| 123 | return Ok(()); | ||
| 124 | }; | ||
| 125 | let mut header = [0; 1024]; | ||
| 126 | match written { | ||
| 127 | Written::InPlace(file) => { | ||
| 128 | file.seek(SeekFrom::Start(0))?; | ||
| 129 | file.read_exact(&mut header)?; | ||
| 130 | } | ||
| 131 | Written::Image(image) => { | ||
| 132 | header.copy_from_slice(image.get(..1024).ok_or(io::ErrorKind::UnexpectedEof)?) | ||
| 133 | } | ||
| 114 | } | 134 | } |
| 115 | Ok(()) | 135 | let mut confirmed = crate::fs::read(&kept.path)?; |
| 136 | confirmed | ||
| 137 | .get_mut(..1024) | ||
| 138 | .ok_or(io::ErrorKind::InvalidData)? | ||
| 139 | .copy_from_slice(&header); | ||
| 140 | let stamp = Stamp { | ||
| 141 | header, | ||
| 142 | length: confirmed.len() as u64, | ||
| 143 | }; | ||
| 144 | let name = format!("{}-{}-{}.one", kept.document, digest(&stamp), kept.base); | ||
| 145 | self.put(&confirmed, name, Some(kept)) | ||
| 116 | } | 146 | } |
| 117 | 147 | ||
| 118 | /// The newest image kept that `torn` begins, as a provider writing in place leaves this | 148 | /// The newest image kept that `torn` begins, as a provider writing in place leaves this |
| 119 | /// device's upload cut off. | 149 | /// device's upload cut off. |
| 120 | pub fn torn(&self, torn: &[u8]) -> io::Result<Option<Vec<u8>>> { | 150 | pub(crate) fn torn(&self, torn: &[u8]) -> io::Result<Option<Vec<u8>>> { |
| 121 | let mut newest = None; | 151 | let mut newest = None; |
| 122 | for kept in self.kept() { | 152 | for kept in self.kept() { |
| 123 | let image = crate::fs::read(&kept.path)?; | 153 | let image = crate::fs::read(&kept.path)?; |
| ... | @@ -128,28 +158,60 @@ impl Published { | ... | @@ -128,28 +158,60 @@ impl Published { |
| 128 | Ok(newest.map(|(_, image)| image)) | 158 | Ok(newest.map(|(_, image)| image)) |
| 129 | } | 159 | } |
| 130 | 160 | ||
| 131 | /// The images kept of the file `stamp` stamps, other than the one it stands as; one whose | 161 | /// The images kept other than the one the file stands as, by `stamp`, which is read only |
| 132 | /// upload the file still stands before is listed only once reads have had `LAG` to catch up. | 162 | /// where any is kept; one whose upload the file still stands before is listed only once |
| 133 | pub fn versions(&self, stamp: &Stamp) -> Vec<Version> { | 163 | /// reads have had `LAG` to catch up. One the provider keeps as a conflict version too, in |
| 164 | /// `conflicts`, is left to that version, so that it is not merged, or kept beside, twice; | ||
| 165 | /// none is listed while a conflict version cannot be read. | ||
| 166 | pub fn versions( | ||
| 167 | &self, | ||
| 168 | stamp: impl FnOnce() -> io::Result<Stamp>, | ||
| 169 | conflicts: &[Version], | ||
| 170 | ) -> io::Result<Vec<Version>> { | ||
| 171 | let kept = self.kept(); | ||
| 172 | if kept.is_empty() { | ||
| 173 | return Ok(Vec::new()); | ||
| 174 | } | ||
| 175 | let stamp = stamp()?; | ||
| 134 | // A header mid-write names no file. | 176 | // A header mid-write names no file. |
| 135 | let Some(document) = document(stamp) else { | 177 | let Some(document) = document(&stamp) else { |
| 136 | return Vec::new(); | 178 | return Ok(Vec::new()); |
| 137 | }; | 179 | }; |
| 138 | let current = digest(stamp); | 180 | let current = digest(&stamp); |
| 139 | let lagging = |kept: &Kept| { | 181 | let lagging = |kept: &Kept| { |
| 140 | kept.base == current && kept.at.elapsed().is_ok_and(|elapsed| elapsed < LAG) | 182 | kept.base == current && kept.at.elapsed().is_ok_and(|elapsed| elapsed < LAG) |
| 141 | }; | 183 | }; |
| 142 | (self.kept().into_iter()) | 184 | // A conflict version not read yet, as one iCloud Drive has not brought down, may be |
| 143 | .filter(|kept| kept.document == document && kept.image != current && !lagging(kept)) | 185 | // one of these: they wait for it, rather than be merged or kept beside the file twice. |
| 186 | let Ok(conflicts) = (conflicts.iter()) | ||
| 187 | .map(|version| Ok(digest(&file_stamp(Path::new(&version.id))?))) | ||
| 188 | .collect::<io::Result<Vec<String>>>() | ||
| 189 | else { | ||
| 190 | return Ok(Vec::new()); | ||
| 191 | }; | ||
| 192 | Ok(kept | ||
| 193 | .into_iter() | ||
| 194 | .filter(|kept| { | ||
| 195 | kept.document == document | ||
| 196 | && kept.image != current | ||
| 197 | && !lagging(kept) | ||
| 198 | && !conflicts.contains(&kept.image) | ||
| 199 | }) | ||
| 144 | .map(|kept| Version { | 200 | .map(|kept| Version { |
| 145 | id: kept.path.to_string_lossy().into_owned(), | 201 | id: kept.path.to_string_lossy().into_owned(), |
| 146 | device: self.device.clone(), | 202 | device: Some(self.device.clone()), |
| 147 | }) | 203 | }) |
| 148 | .collect() | 204 | .collect()) |
| 149 | } | 205 | } |
| 150 | 206 | ||
| 151 | pub fn device(&self) -> Option<&str> { | 207 | /// The provider's conflict version at `version` is retired, merged or kept beside the |
| 152 | self.device.as_deref() | 208 | /// file: an image kept here that it is goes with it, so that it is not merged again. |
| 209 | pub fn retired(&self, version: &Path) -> io::Result<()> { | ||
| 210 | let image = digest(&file_stamp(version)?); | ||
| 211 | for kept in self.kept().into_iter().filter(|kept| kept.image == image) { | ||
| 212 | crate::fs::remove_file(kept.path)?; | ||
| 213 | } | ||
| 214 | Ok(()) | ||
| 153 | } | 215 | } |
| 154 | 216 | ||
| 155 | /// Whether version `id` is an image kept here. | 217 | /// Whether version `id` is an image kept here. |
| ... | @@ -158,18 +220,48 @@ impl Published { | ... | @@ -158,18 +220,48 @@ impl Published { |
| 158 | } | 220 | } |
| 159 | } | 221 | } |
| 160 | 222 | ||
| 223 | /// The stamp of the file at `path`: its header and length. | ||
| 224 | fn file_stamp(path: &Path) -> io::Result<Stamp> { | ||
| 225 | let mut file = File::open(path)?; | ||
| 226 | let mut header = [0; 1024]; | ||
| 227 | file.read_exact(&mut header)?; | ||
| 228 | Ok(Stamp { | ||
| 229 | header, | ||
| 230 | length: file.metadata()?.len(), | ||
| 231 | }) | ||
| 232 | } | ||
| 233 | |||
| 234 | /// Whether the section whose replica is at `replica` keeps images of its publications. | ||
| 235 | pub(crate) fn keeps(replica: &Path) -> bool { | ||
| 236 | crate::fs::read_dir(folder(replica)).is_ok_and(|mut images| images.next().is_some()) | ||
| 237 | } | ||
| 238 | |||
| 161 | /// Where a version of `file` that cannot merge is kept beside it, as iCloud Drive keeps a | 239 | /// Where a version of `file` that cannot merge is kept beside it, as iCloud Drive keeps a |
| 162 | /// conflict version: "Name (Device).one", numbered past any name taken. | 240 | /// conflict version: "Name (Device).one", numbered past any name taken. |
| 163 | pub fn beside(file: &Path, device: &str) -> PathBuf { | 241 | pub fn beside(file: &Path, device: &str) -> PathBuf { |
| 164 | let stem = file.file_stem().unwrap_or_default().to_string_lossy(); | 242 | let stem = file.file_stem().unwrap_or_default().to_string_lossy(); |
| 165 | (1..) | 243 | let mut n = 1; |
| 166 | .map(|n| match n { | 244 | loop { |
| 245 | let name = match n { | ||
| 167 | 1 => format!("{stem} ({device}).one"), | 246 | 1 => format!("{stem} ({device}).one"), |
| 168 | n => format!("{stem} ({device} {n}).one"), | 247 | n => format!("{stem} ({device} {n}).one"), |
| 169 | }) | 248 | }; |
| 170 | .map(|name| file.with_file_name(name)) | 249 | let beside = file.with_file_name(name); |
| 171 | .find(|beside| std::fs::symlink_metadata(beside).is_err()) | 250 | if std::fs::symlink_metadata(&beside).is_err() { |
| 172 | .unwrap_or_default() | 251 | return beside; |
| 252 | } | ||
| 253 | n += 1; | ||
| 254 | } | ||
| 255 | } | ||
| 256 | |||
| 257 | /// All of `file`, at most `limit` bytes. | ||
| 258 | pub(crate) fn whole(file: &mut File, limit: usize) -> io::Result<Vec<u8>> { | ||
| 259 | let mut bytes = Vec::new(); | ||
| 260 | file.take(limit as u64 + 1).read_to_end(&mut bytes)?; | ||
| 261 | if bytes.len() > limit { | ||
| 262 | return Err(io::ErrorKind::FileTooLarge.into()); | ||
| 263 | } | ||
| 264 | Ok(bytes) | ||
| 173 | } | 265 | } |
| 174 | 266 | ||
| 175 | /// The hex of a stamp's digest, which names an image this device keeps. | 267 | /// The hex of a stamp's digest, which names an image this device keeps. |
| ... | @@ -189,3 +281,112 @@ fn document(stamp: &Stamp) -> Option<String> { | ... | @@ -189,3 +281,112 @@ fn document(stamp: &Stamp) -> Option<String> { |
| 189 | fn hex(bytes: &[u8]) -> String { | 281 | fn hex(bytes: &[u8]) -> String { |
| 190 | bytes.iter().map(|byte| format!("{byte:02x}")).collect() | 282 | bytes.iter().map(|byte| format!("{byte:02x}")).collect() |
| 191 | } | 283 | } |
| 284 | |||
| 285 | #[cfg(test)] | ||
| 286 | mod tests { | ||
| 287 | use super::*; | ||
| 288 | use onestore::op::{Edit, Op, PageOp}; | ||
| 289 | |||
| 290 | /// A section file in a scratch folder, its image, and the transaction one edit of it | ||
| 291 | /// publishes. | ||
| 292 | fn edited(folder: &Path) -> (PathBuf, Vec<u8>, Transaction) { | ||
| 293 | let file = folder.join("Notes.one"); | ||
| 294 | let image = onestore::create_section("Notes.one", "Text", "Author").unwrap(); | ||
| 295 | std::fs::write(&file, &image).unwrap(); | ||
| 296 | let arena = onestore::Arena::default(); | ||
| 297 | let mut section = onestore::Section::open(&arena, image.clone()).unwrap(); | ||
| 298 | let space = section.pages().unwrap()[0].0; | ||
| 299 | let edit = Edit { | ||
| 300 | at: crate::now(), | ||
| 301 | ops: vec![Op::Page { | ||
| 302 | space, | ||
| 303 | op: PageOp::Color(Some(0x00ab_cdef)), | ||
| 304 | }], | ||
| 305 | }; | ||
| 306 | section.apply("Author", &edit).unwrap(); | ||
| 307 | (file, image, section.seal().unwrap().unwrap()) | ||
| 308 | } | ||
| 309 | |||
| 310 | fn published(folder: &Path, file: &Path, transaction: &Transaction) -> Published { | ||
| 311 | let published = Published::of(&folder.join("replica.sqlite"), "iPhone"); | ||
| 312 | transaction.commit_file(file).unwrap(); | ||
| 313 | let mut opened = File::open(file).unwrap(); | ||
| 314 | published | ||
| 315 | .keep(Written::InPlace(&mut opened), transaction) | ||
| 316 | .unwrap(); | ||
| 317 | published | ||
| 318 | } | ||
| 319 | |||
| 320 | #[test] | ||
| 321 | fn nothing_kept_reads_no_stamp() { | ||
| 322 | let folder = tempfile::tempdir().unwrap(); | ||
| 323 | let published = Published::of(&folder.path().join("replica.sqlite"), "iPhone"); | ||
| 324 | let listed = published.versions(|| unreachable!("no stamp is read"), &[]); | ||
| 325 | assert!(listed.unwrap().is_empty()); | ||
| 326 | } | ||
| 327 | |||
| 328 | /// The image this device published lists once the file stands otherwise: once reads have | ||
| 329 | /// had `LAG` to catch up where the file stands as the image it was published on, and not | ||
| 330 | /// where the provider keeps it as a conflict version itself. | ||
| 331 | #[test] | ||
| 332 | fn a_lost_image_lists_once() { | ||
| 333 | let folder = tempfile::tempdir().unwrap(); | ||
| 334 | let (file, base, transaction) = edited(folder.path()); | ||
| 335 | let published = published(folder.path(), &file, &transaction); | ||
| 336 | let listed = | ||
| 337 | |conflicts: &[Version]| published.versions(|| file_stamp(&file), conflicts).unwrap(); | ||
| 338 | assert!(listed(&[]).is_empty()); | ||
| 339 | let ours = std::fs::read(&file).unwrap(); | ||
| 340 | std::fs::write(&file, base).unwrap(); | ||
| 341 | assert!(listed(&[]).is_empty()); | ||
| 342 | let aged = File::options() | ||
| 343 | .write(true) | ||
| 344 | .open(&published.kept()[0].path) | ||
| 345 | .unwrap(); | ||
| 346 | aged.set_modified(SystemTime::now() - LAG - Duration::from_secs(1)) | ||
| 347 | .unwrap(); | ||
| 348 | assert_eq!(listed(&[]).len(), 1); | ||
| 349 | // A conflict version not here yet could be this image: it waits for it. | ||
| 350 | let missing = Version { | ||
| 351 | id: file | ||
| 352 | .with_file_name("Not here.one") | ||
| 353 | .to_string_lossy() | ||
| 354 | .into_owned(), | ||
| 355 | device: None, | ||
| 356 | }; | ||
| 357 | assert!(listed(&[missing]).is_empty()); | ||
| 358 | let version = file.with_file_name("Notes (iPhone).one"); | ||
| 359 | std::fs::write(&version, ours).unwrap(); | ||
| 360 | let conflict = Version { | ||
| 361 | id: version.to_string_lossy().into_owned(), | ||
| 362 | device: Some("iPhone".into()), | ||
| 363 | }; | ||
| 364 | assert!(listed(std::slice::from_ref(&conflict)).is_empty()); | ||
| 365 | published.retired(&version).unwrap(); | ||
| 366 | assert!(published.kept().is_empty()); | ||
| 367 | } | ||
| 368 | |||
| 369 | /// A confirmation rewrites the header of the image last published: the kept image follows | ||
| 370 | /// it, rather than listing as lost and going, which would leave it unprotected. | ||
| 371 | #[test] | ||
| 372 | fn a_confirmation_keeps_the_image_kept() { | ||
| 373 | let folder = tempfile::tempdir().unwrap(); | ||
| 374 | let (file, _, transaction) = edited(folder.path()); | ||
| 375 | let published = published(folder.path(), &file, &transaction); | ||
| 376 | let base = file_stamp(&file).unwrap(); | ||
| 377 | onestore::confirm_file(&file, &base).unwrap(); | ||
| 378 | assert_ne!(file_stamp(&file).unwrap(), base); | ||
| 379 | let mut opened = File::open(&file).unwrap(); | ||
| 380 | published | ||
| 381 | .confirmed(Written::InPlace(&mut opened), &base) | ||
| 382 | .unwrap(); | ||
| 383 | let kept = published.kept(); | ||
| 384 | assert_eq!(kept.len(), 1); | ||
| 385 | assert_eq!( | ||
| 386 | std::fs::read(&kept[0].path).unwrap(), | ||
| 387 | std::fs::read(&file).unwrap() | ||
| 388 | ); | ||
| 389 | let listed = published.versions(|| file_stamp(&file), &[]).unwrap(); | ||
| 390 | assert!(listed.is_empty()); | ||
| 391 | } | ||
| 392 | } |
crates/notebook/src/session.rs+69-53| ... | @@ -626,7 +626,7 @@ impl Notebook { | ... | @@ -626,7 +626,7 @@ impl Notebook { |
| 626 | _ => continue, | 626 | _ => continue, |
| 627 | }; | 627 | }; |
| 628 | claims | 628 | claims |
| 629 | .entry(replica_location(&location, section)) | 629 | .entry(replica_location(&location, &section.path, section.copy)) |
| 630 | .or_default() | 630 | .or_default() |
| 631 | .push((identity, &section.path)); | 631 | .push((identity, &section.path)); |
| 632 | } | 632 | } |
| ... | @@ -1017,7 +1017,7 @@ impl Notebook { | ... | @@ -1017,7 +1017,7 @@ impl Notebook { |
| 1017 | self.catalog | 1017 | self.catalog |
| 1018 | .sections() | 1018 | .sections() |
| 1019 | .filter(|section| section.copy) | 1019 | .filter(|section| section.copy) |
| 1020 | .map(|section| replica_location(&from, section)), | 1020 | .map(|section| replica_location(&from, &section.path, section.copy)), |
| 1021 | ); | 1021 | ); |
| 1022 | for location in locations { | 1022 | for location in locations { |
| 1023 | let entries = match fs::read_dir(crate::location::folder(&self.cache, &location)) { | 1023 | let entries = match fs::read_dir(crate::location::folder(&self.cache, &location)) { |
| ... | @@ -1592,11 +1592,13 @@ impl Notebook { | ... | @@ -1592,11 +1592,13 @@ impl Notebook { |
| 1592 | }); | 1592 | }); |
| 1593 | let _ = self.refresh(); | 1593 | let _ = self.refresh(); |
| 1594 | drop(replica); | 1594 | drop(replica); |
| 1595 | // Unless something else opened it meanwhile. | 1595 | // Unless something else opened it meanwhile, or it keeps images of uploads the old |
| 1596 | // file may have lost, which name that file and stay with its replica. | ||
| 1596 | if let Ok(file) = file | 1597 | if let Ok(file) = file |
| 1597 | && crate::closed(&file).is_ok() | 1598 | && crate::closed(&file).is_ok() |
| 1599 | && !crate::published::keeps(&file) | ||
| 1598 | { | 1600 | { |
| 1599 | let _ = remove_replica(&file); | 1601 | let _ = crate::remove_replica(&file); |
| 1600 | } | 1602 | } |
| 1601 | Ok(Moved { | 1603 | Ok(Moved { |
| 1602 | path: copy, | 1604 | path: copy, |
| ... | @@ -1817,7 +1819,7 @@ impl Notebook { | ... | @@ -1817,7 +1819,7 @@ impl Notebook { |
| 1817 | replica.apply_async(&author, edit).await?; | 1819 | replica.apply_async(&author, edit).await?; |
| 1818 | } | 1820 | } |
| 1819 | drop(old); | 1821 | drop(old); |
| 1820 | remove_replica(&path)?; | 1822 | crate::remove_replica(&path)?; |
| 1821 | } | 1823 | } |
| 1822 | } | 1824 | } |
| 1823 | Ok(key) | 1825 | Ok(key) |
| ... | @@ -1893,7 +1895,7 @@ impl Notebook { | ... | @@ -1893,7 +1895,7 @@ impl Notebook { |
| 1893 | for (replica, held, queued) in self.superseded(&locked) { | 1895 | for (replica, held, queued) in self.superseded(&locked) { |
| 1894 | drop(held); | 1896 | drop(held); |
| 1895 | if queued == 0 { | 1897 | if queued == 0 { |
| 1896 | let _ = remove_replica(&replica); | 1898 | let _ = crate::remove_replica(&replica); |
| 1897 | } | 1899 | } |
| 1898 | } | 1900 | } |
| 1899 | } | 1901 | } |
| ... | @@ -1944,7 +1946,7 @@ impl Notebook { | ... | @@ -1944,7 +1946,7 @@ impl Notebook { |
| 1944 | return Err(error); | 1946 | return Err(error); |
| 1945 | } | 1947 | } |
| 1946 | drop(held); | 1948 | drop(held); |
| 1947 | remove_replica(&replica)?; | 1949 | crate::remove_replica(&replica)?; |
| 1948 | let parent = self.folder(folder)?; | 1950 | let parent = self.folder(folder)?; |
| 1949 | if let Some(identity) = Entry::of(parent, section).identity { | 1951 | if let Some(identity) = Entry::of(parent, section).identity { |
| 1950 | self.edit_toc( | 1952 | self.edit_toc( |
| ... | @@ -1980,9 +1982,15 @@ impl Notebook { | ... | @@ -1980,9 +1982,15 @@ impl Notebook { |
| 1980 | } | 1982 | } |
| 1981 | 1983 | ||
| 1982 | /// The `location::folder` holding the replica of `section`. | 1984 | /// The `location::folder` holding the replica of `section`. |
| 1985 | /// A copy's is labelled with its location as it is first named, as one found after the | ||
| 1986 | /// notebook opened is, so that it moves and goes with the notebook (`location::moved`). | ||
| 1983 | fn replica_folder(&self, section: &discover::Section) -> PathBuf { | 1987 | fn replica_folder(&self, section: &discover::Section) -> PathBuf { |
| 1984 | let location = replica_location(&self.storage.location(), section); | 1988 | let location = replica_location(&self.storage.location(), &section.path, section.copy); |
| 1985 | crate::location::folder(&self.cache, &location) | 1989 | match section.copy { |
| 1990 | true => crate::location::claimed(&self.cache, &location) | ||
| 1991 | .unwrap_or_else(|_| crate::location::folder(&self.cache, &location)), | ||
| 1992 | false => crate::location::folder(&self.cache, &location), | ||
| 1993 | } | ||
| 1986 | } | 1994 | } |
| 1987 | 1995 | ||
| 1988 | /// Every readable section, for `Background::watch`, handing on the files the last | 1996 | /// Every readable section, for `Background::watch`, handing on the files the last |
| ... | @@ -2012,16 +2020,21 @@ impl Notebook { | ... | @@ -2012,16 +2020,21 @@ impl Notebook { |
| 2012 | copies: bool, | 2020 | copies: bool, |
| 2013 | notify: impl Fn() + Send + 'static, | 2021 | notify: impl Fn() + Send + 'static, |
| 2014 | ) -> Result<Background> { | 2022 | ) -> Result<Background> { |
| 2015 | self.background_with(watched, copies, |file| FileRemote(file.to_owned()), notify) | 2023 | self.background_with( |
| 2024 | watched, | ||
| 2025 | copies, | ||
| 2026 | |file, _| FileRemote(file.to_owned()), | ||
| 2027 | notify, | ||
| 2028 | ) | ||
| 2016 | } | 2029 | } |
| 2017 | 2030 | ||
| 2018 | /// `background`, reaching each section file through the remote `remote` makes for it | 2031 | /// `background`, reaching each section file through the remote `remote` makes for it and |
| 2019 | /// (`section_with`). | 2032 | /// its replica, where it has one (`section_with`). |
| 2020 | pub fn background_with<R: Remote + 'static>( | 2033 | pub fn background_with<R: Remote + 'static>( |
| 2021 | &self, | 2034 | &self, |
| 2022 | watched: bool, | 2035 | watched: bool, |
| 2023 | copies: bool, | 2036 | copies: bool, |
| 2024 | remote: impl Fn(&Path) -> R + Clone + Send + 'static, | 2037 | remote: impl Fn(&Path, Option<&Path>) -> R + Clone + Send + 'static, |
| 2025 | notify: impl Fn() + Send + 'static, | 2038 | notify: impl Fn() + Send + 'static, |
| 2026 | ) -> Result<Background> { | 2039 | ) -> Result<Background> { |
| 2027 | let Some(root) = self.root.clone() else { | 2040 | let Some(root) = self.root.clone() else { |
| ... | @@ -2035,7 +2048,8 @@ impl Notebook { | ... | @@ -2035,7 +2048,8 @@ impl Notebook { |
| 2035 | copies, | 2048 | copies, |
| 2036 | move |_| { | 2049 | move |_| { |
| 2037 | let (folder, remote) = (root.clone(), remote.clone()); | 2050 | let (folder, remote) = (root.clone(), remote.clone()); |
| 2038 | let bind = move |path: &str| remote(&folder.join(path)); | 2051 | let bind = |
| 2052 | move |path: &str, replica: Option<&Path>| remote(&folder.join(path), replica); | ||
| 2039 | let mut local = discover::Local::open(&root)?; | 2053 | let mut local = discover::Local::open(&root)?; |
| 2040 | let list = move |folder: String| { | 2054 | let list = move |folder: String| { |
| 2041 | use discover::Source; | 2055 | use discover::Source; |
| ... | @@ -2056,18 +2070,18 @@ impl Notebook { | ... | @@ -2056,18 +2070,18 @@ impl Notebook { |
| 2056 | ) -> Result<Section> { | 2070 | ) -> Result<Section> { |
| 2057 | self.section_with( | 2071 | self.section_with( |
| 2058 | path, | 2072 | path, |
| 2059 | |file| Ok(FileRemote(file.to_owned())), | 2073 | |file, _| Ok(FileRemote(file.to_owned())), |
| 2060 | offline, | 2074 | offline, |
| 2061 | notify, | 2075 | notify, |
| 2062 | ) | 2076 | ) |
| 2063 | } | 2077 | } |
| 2064 | 2078 | ||
| 2065 | /// `section`, reading and publishing through the remote `connect` makes for its file | 2079 | /// `section`, reading and publishing through the remote `connect` makes for its file and |
| 2066 | /// (`Section::open_with`). | 2080 | /// its replica, which it reads through before there is one (`Section::open_with`). |
| 2067 | pub fn section_with<R: Remote + 'static>( | 2081 | pub fn section_with<R: Remote + 'static>( |
| 2068 | &self, | 2082 | &self, |
| 2069 | path: &str, | 2083 | path: &str, |
| 2070 | connect: impl FnMut(&Path) -> io::Result<R> + Send + 'static, | 2084 | connect: impl FnMut(&Path, Option<&Path>) -> io::Result<R> + Send + 'static, |
| 2071 | offline: bool, | 2085 | offline: bool, |
| 2072 | notify: impl Fn() + Send + 'static, | 2086 | notify: impl Fn() + Send + 'static, |
| 2073 | ) -> Result<Section> { | 2087 | ) -> Result<Section> { |
| ... | @@ -2085,7 +2099,7 @@ impl Notebook { | ... | @@ -2085,7 +2099,7 @@ impl Notebook { |
| 2085 | self.section_unlocked_with( | 2099 | self.section_unlocked_with( |
| 2086 | path, | 2100 | path, |
| 2087 | key, | 2101 | key, |
| 2088 | |file| Ok(FileRemote(file.to_owned())), | 2102 | |file, _| Ok(FileRemote(file.to_owned())), |
| 2089 | offline, | 2103 | offline, |
| 2090 | notify, | 2104 | notify, |
| 2091 | ) | 2105 | ) |
| ... | @@ -2096,7 +2110,7 @@ impl Notebook { | ... | @@ -2096,7 +2110,7 @@ impl Notebook { |
| 2096 | &self, | 2110 | &self, |
| 2097 | path: &str, | 2111 | path: &str, |
| 2098 | key: &protected::Key, | 2112 | key: &protected::Key, |
| 2099 | connect: impl FnMut(&Path) -> io::Result<R> + Send + 'static, | 2113 | connect: impl FnMut(&Path, Option<&Path>) -> io::Result<R> + Send + 'static, |
| 2100 | offline: bool, | 2114 | offline: bool, |
| 2101 | notify: impl Fn() + Send + 'static, | 2115 | notify: impl Fn() + Send + 'static, |
| 2102 | ) -> Result<Section> { | 2116 | ) -> Result<Section> { |
| ... | @@ -2107,7 +2121,7 @@ impl Notebook { | ... | @@ -2107,7 +2121,7 @@ impl Notebook { |
| 2107 | &self, | 2121 | &self, |
| 2108 | path: &str, | 2122 | path: &str, |
| 2109 | key: Option<&protected::Key>, | 2123 | key: Option<&protected::Key>, |
| 2110 | connect: impl FnMut(&Path) -> io::Result<R> + Send + 'static, | 2124 | connect: impl FnMut(&Path, Option<&Path>) -> io::Result<R> + Send + 'static, |
| 2111 | offline: bool, | 2125 | offline: bool, |
| 2112 | notify: impl Fn() + Send + 'static, | 2126 | notify: impl Fn() + Send + 'static, |
| 2113 | ) -> Result<Section> { | 2127 | ) -> Result<Section> { |
| ... | @@ -2146,7 +2160,7 @@ impl Notebook { | ... | @@ -2146,7 +2160,7 @@ impl Notebook { |
| 2146 | return Section::start( | 2160 | return Section::start( |
| 2147 | file, | 2161 | file, |
| 2148 | replica, | 2162 | replica, |
| 2149 | move || connect(&remote), | 2163 | move || connect(&remote, Some(&cache)), |
| 2150 | CHECK, | 2164 | CHECK, |
| 2151 | offline, | 2165 | offline, |
| 2152 | notify, | 2166 | notify, |
| ... | @@ -2412,27 +2426,14 @@ fn catalog_path(folder: &str, name: &str) -> String { | ... | @@ -2412,27 +2426,14 @@ fn catalog_path(folder: &str, name: &str) -> String { |
| 2412 | /// The location keying the replica of `section` in the notebook at `notebook`: the notebook's, | 2426 | /// The location keying the replica of `section` in the notebook at `notebook`: the notebook's, |
| 2413 | /// so that a section renamed or moved within it keeps its replica, or for a copy of another | 2427 | /// so that a section renamed or moved within it keeps its replica, or for a copy of another |
| 2414 | /// section of the notebook its own file's. | 2428 | /// section of the notebook its own file's. |
| 2415 | fn replica_location(notebook: &str, section: &discover::Section) -> String { | 2429 | pub fn replica_location(notebook: &str, path: &str, copy: bool) -> String { |
| 2416 | if section.copy { | 2430 | if copy { |
| 2417 | format!("{notebook}/{}", section.path) | 2431 | format!("{notebook}/{path}") |
| 2418 | } else { | 2432 | } else { |
| 2419 | notebook.to_owned() | 2433 | notebook.to_owned() |
| 2420 | } | 2434 | } |
| 2421 | } | 2435 | } |
| 2422 | 2436 | ||
| 2423 | /// Deletes the closed replica at `replica`, with the files SQLite keeps beside it. | ||
| 2424 | fn remove_replica(replica: &Path) -> io::Result<()> { | ||
| 2425 | for suffix in ["", "-wal", "-shm"] { | ||
| 2426 | let mut file = replica.as_os_str().to_owned(); | ||
| 2427 | file.push(suffix); | ||
| 2428 | match fs::remove_file(file) { | ||
| 2429 | Err(error) if error.kind() != io::ErrorKind::NotFound => return Err(error), | ||
| 2430 | _ => {} | ||
| 2431 | } | ||
| 2432 | } | ||
| 2433 | Ok(()) | ||
| 2434 | } | ||
| 2435 | |||
| 2436 | /// How often an open section's file is checked where nothing watches it. | 2437 | /// How often an open section's file is checked where nothing watches it. |
| 2437 | const CHECK: Duration = Duration::from_secs(2); | 2438 | const CHECK: Duration = Duration::from_secs(2); |
| 2438 | 2439 | ||
| ... | @@ -2598,7 +2599,7 @@ impl Section { | ... | @@ -2598,7 +2599,7 @@ impl Section { |
| 2598 | Self::open_with( | 2599 | Self::open_with( |
| 2599 | file, | 2600 | file, |
| 2600 | cache, | 2601 | cache, |
| 2601 | |file| Ok(FileRemote(file.to_owned())), | 2602 | |file, _| Ok(FileRemote(file.to_owned())), |
| 2602 | offline, | 2603 | offline, |
| 2603 | notify, | 2604 | notify, |
| 2604 | ) | 2605 | ) |
| ... | @@ -2610,7 +2611,7 @@ impl Section { | ... | @@ -2610,7 +2611,7 @@ impl Section { |
| 2610 | pub fn open_with<R: Remote + 'static>( | 2611 | pub fn open_with<R: Remote + 'static>( |
| 2611 | file: impl AsRef<Path>, | 2612 | file: impl AsRef<Path>, |
| 2612 | cache: impl AsRef<Path>, | 2613 | cache: impl AsRef<Path>, |
| 2613 | connect: impl FnMut(&Path) -> io::Result<R> + Send + 'static, | 2614 | connect: impl FnMut(&Path, Option<&Path>) -> io::Result<R> + Send + 'static, |
| 2614 | offline: bool, | 2615 | offline: bool, |
| 2615 | notify: impl Fn() + Send + 'static, | 2616 | notify: impl Fn() + Send + 'static, |
| 2616 | ) -> Result<Self> { | 2617 | ) -> Result<Self> { |
| ... | @@ -2626,12 +2627,12 @@ impl Section { | ... | @@ -2626,12 +2627,12 @@ impl Section { |
| 2626 | file: PathBuf, | 2627 | file: PathBuf, |
| 2627 | key: Option<&protected::Key>, | 2628 | key: Option<&protected::Key>, |
| 2628 | replica: impl FnOnce(&[u8; 16], &[u8]) -> Result<PathBuf>, | 2629 | replica: impl FnOnce(&[u8; 16], &[u8]) -> Result<PathBuf>, |
| 2629 | mut connect: impl FnMut(&Path) -> io::Result<R> + Send + 'static, | 2630 | mut connect: impl FnMut(&Path, Option<&Path>) -> io::Result<R> + Send + 'static, |
| 2630 | interval: Duration, | 2631 | interval: Duration, |
| 2631 | offline: bool, | 2632 | offline: bool, |
| 2632 | notify: impl Fn() + Send + 'static, | 2633 | notify: impl Fn() + Send + 'static, |
| 2633 | ) -> Result<Self> { | 2634 | ) -> Result<Self> { |
| 2634 | let source = connect(&file)?.read()?; | 2635 | let source = connect(&file, None)?.read()?; |
| 2635 | let store = Store::parse(&source)?; | 2636 | let store = Store::parse(&source)?; |
| 2636 | let identity = RevisionIndex::parse(&store)?.root; | 2637 | let identity = RevisionIndex::parse(&store)?.root; |
| 2637 | let cache = replica(&identity.guid, &source)?; | 2638 | let cache = replica(&identity.guid, &source)?; |
| ... | @@ -2640,7 +2641,7 @@ impl Section { | ... | @@ -2640,7 +2641,7 @@ impl Section { |
| 2640 | Self::start( | 2641 | Self::start( |
| 2641 | file, | 2642 | file, |
| 2642 | replica, | 2643 | replica, |
| 2643 | move || connect(&remote), | 2644 | move || connect(&remote, Some(&cache)), |
| 2644 | interval, | 2645 | interval, |
| 2645 | offline, | 2646 | offline, |
| 2646 | notify, | 2647 | notify, |
| ... | @@ -2717,10 +2718,15 @@ impl Section { | ... | @@ -2717,10 +2718,15 @@ impl Section { |
| 2717 | offline: bool, | 2718 | offline: bool, |
| 2718 | notify: impl Fn() + Send + 'static, | 2719 | notify: impl Fn() + Send + 'static, |
| 2719 | ) -> Result<Self> { | 2720 | ) -> Result<Self> { |
| 2721 | let cache = replica.path(); | ||
| 2720 | Self::start( | 2722 | Self::start( |
| 2721 | PathBuf::from(&path), | 2723 | PathBuf::from(&path), |
| 2722 | replica, | 2724 | replica, |
| 2723 | move || Ok(crate::provider::ProviderRemote::new(&documents, &path)), | 2725 | move || { |
| 2726 | let remote = | ||
| 2727 | crate::provider::ProviderRemote::new(&documents, &path, cache.as_deref()); | ||
| 2728 | Ok(remote) | ||
| 2729 | }, | ||
| 2724 | Background::UNWATCHED, | 2730 | Background::UNWATCHED, |
| 2725 | offline, | 2731 | offline, |
| 2726 | notify, | 2732 | notify, |
| ... | @@ -2738,7 +2744,11 @@ impl Section { | ... | @@ -2738,7 +2744,11 @@ impl Section { |
| 2738 | let (cache, location) = (documents.cache.clone(), documents.location()); | 2744 | let (cache, location) = (documents.cache.clone(), documents.location()); |
| 2739 | let cache = cache.as_path(); | 2745 | let cache = cache.as_path(); |
| 2740 | let file = PathBuf::from(&documents.root); | 2746 | let file = PathBuf::from(&documents.root); |
| 2741 | let remote = move || Ok(crate::provider::ProviderRemote::new(&documents, "")); | 2747 | let remote = move |replica: Option<&Path>| { |
| 2748 | Ok(crate::provider::ProviderRemote::new( | ||
| 2749 | &documents, "", replica, | ||
| 2750 | )) | ||
| 2751 | }; | ||
| 2742 | let notify = Arc::new(notify); | 2752 | let notify = Arc::new(notify); |
| 2743 | let (connect, woken) = (remote.clone(), Arc::clone(&notify)); | 2753 | let (connect, woken) = (remote.clone(), Arc::clone(&notify)); |
| 2744 | let replicas = claimed(cache, &location); | 2754 | let replicas = claimed(cache, &location); |
| ... | @@ -2746,7 +2756,7 @@ impl Section { | ... | @@ -2746,7 +2756,7 @@ impl Section { |
| 2746 | file.clone(), | 2756 | file.clone(), |
| 2747 | None, | 2757 | None, |
| 2748 | replicas, | 2758 | replicas, |
| 2749 | move |_| connect(), | 2759 | move |_, replica| connect(replica), |
| 2750 | Background::UNWATCHED, | 2760 | Background::UNWATCHED, |
| 2751 | offline, | 2761 | offline, |
| 2752 | move || woken(), | 2762 | move || woken(), |
| ... | @@ -2761,8 +2771,8 @@ impl Section { | ... | @@ -2761,8 +2771,8 @@ impl Section { |
| 2761 | match (replicas.next(), replicas.next()) { | 2771 | match (replicas.next(), replicas.next()) { |
| 2762 | (Some(only), None) => Self::start( | 2772 | (Some(only), None) => Self::start( |
| 2763 | file, | 2773 | file, |
| 2764 | Replica::open(only)?, | 2774 | Replica::open(&only)?, |
| 2765 | remote, | 2775 | move || remote(Some(&only)), |
| 2766 | Background::UNWATCHED, | 2776 | Background::UNWATCHED, |
| 2767 | offline, | 2777 | offline, |
| 2768 | move || notify(), | 2778 | move || notify(), |
| ... | @@ -2872,7 +2882,9 @@ impl Section { | ... | @@ -2872,7 +2882,9 @@ impl Section { |
| 2872 | 2882 | ||
| 2873 | /// Follows a section file moved within its notebook, keeping its replica and queued edits. | 2883 | /// Follows a section file moved within its notebook, keeping its replica and queued edits. |
| 2874 | pub fn relocate(&mut self, file: impl AsRef<Path>) -> Result<()> { | 2884 | pub fn relocate(&mut self, file: impl AsRef<Path>) -> Result<()> { |
| 2875 | self.relocate_with(fs::absolute(file)?, |file| Ok(FileRemote(file.to_owned()))) | 2885 | self.relocate_with(fs::absolute(file)?, |file, _| { |
| 2886 | Ok(FileRemote(file.to_owned())) | ||
| 2887 | }) | ||
| 2876 | } | 2888 | } |
| 2877 | 2889 | ||
| 2878 | /// `relocate`, with a host's publication target. Cancellation leaves any in-flight step | 2890 | /// `relocate`, with a host's publication target. Cancellation leaves any in-flight step |
| ... | @@ -2880,17 +2892,21 @@ impl Section { | ... | @@ -2880,17 +2892,21 @@ impl Section { |
| 2880 | pub fn relocate_with<R: Remote + 'static>( | 2892 | pub fn relocate_with<R: Remote + 'static>( |
| 2881 | &mut self, | 2893 | &mut self, |
| 2882 | file: impl AsRef<Path>, | 2894 | file: impl AsRef<Path>, |
| 2883 | mut connect: impl FnMut(&Path) -> io::Result<R> + Send + 'static, | 2895 | mut connect: impl FnMut(&Path, Option<&Path>) -> io::Result<R> + Send + 'static, |
| 2884 | ) -> Result<()> { | 2896 | ) -> Result<()> { |
| 2885 | let file = file.as_ref().to_owned(); | 2897 | let file = file.as_ref().to_owned(); |
| 2886 | let target = file.clone(); | 2898 | let (target, replica) = (file.clone(), self.replica.path()); |
| 2887 | let previous = self.worker.take(); | 2899 | let previous = self.worker.take(); |
| 2888 | if let Some(worker) = &previous { | 2900 | if let Some(worker) = &previous { |
| 2889 | worker.cancel(); | 2901 | worker.cancel(); |
| 2890 | } | 2902 | } |
| 2891 | self.observed = Arc::new(Mutex::new(None)); | 2903 | self.observed = Arc::new(Mutex::new(None)); |
| 2892 | self.worker = | 2904 | self.worker = Some(self.start_worker( |
| 2893 | Some(self.start_worker(move || connect(&target), CHECK, false, previous.as_ref())?); | 2905 | move || connect(&target, replica.as_deref()), |
| 2906 | CHECK, | ||
| 2907 | false, | ||
| 2908 | previous.as_ref(), | ||
| 2909 | )?); | ||
| 2894 | self.file = file; | 2910 | self.file = file; |
| 2895 | Ok(()) | 2911 | Ok(()) |
| 2896 | } | 2912 | } |
crates/notebook/src/session/tests.rs+1-1| ... | @@ -691,7 +691,7 @@ fn initially_offline_workers_keep_the_original_queue_until_explicitly_woken() { | ... | @@ -691,7 +691,7 @@ fn initially_offline_workers_keep_the_original_queue_until_explicitly_woken() { |
| 691 | fs::rename(&file, &target).unwrap(); | 691 | fs::rename(&file, &target).unwrap(); |
| 692 | let counter = Arc::clone(&calls); | 692 | let counter = Arc::clone(&calls); |
| 693 | section | 693 | section |
| 694 | .relocate_with(&target, move |file| { | 694 | .relocate_with(&target, move |file, _| { |
| 695 | counter[0].fetch_add(1, Ordering::SeqCst); | 695 | counter[0].fetch_add(1, Ordering::SeqCst); |
| 696 | Ok(Counted { | 696 | Ok(Counted { |
| 697 | file: FileRemote(file.to_owned()), | 697 | file: FileRemote(file.to_owned()), |
crates/notebook/tests/session.rs+3-3| ... | @@ -1360,7 +1360,7 @@ fn a_moved_section_keeps_its_replica_while_the_old_remote_is_stalled() { | ... | @@ -1360,7 +1360,7 @@ fn a_moved_section_keeps_its_replica_while_the_old_remote_is_stalled() { |
| 1360 | let mut section = Section::open_with( | 1360 | let mut section = Section::open_with( |
| 1361 | &file, | 1361 | &file, |
| 1362 | directory.path().join("cache"), | 1362 | directory.path().join("cache"), |
| 1363 | move |file| { | 1363 | move |file, _| { |
| 1364 | let finishing = (!std::mem::replace(&mut opening, false)).then(|| finished.clone()); | 1364 | let finishing = (!std::mem::replace(&mut opening, false)).then(|| finished.clone()); |
| 1365 | if finishing.is_some() { | 1365 | if finishing.is_some() { |
| 1366 | connected.send(()).unwrap(); | 1366 | connected.send(()).unwrap(); |
| ... | @@ -1460,7 +1460,7 @@ fn an_open_section_the_background_holds_checks_its_file_when_the_watch_reports_i | ... | @@ -1460,7 +1460,7 @@ fn an_open_section_the_background_holds_checks_its_file_when_the_watch_reports_i |
| 1460 | let section = notebook | 1460 | let section = notebook |
| 1461 | .section_with( | 1461 | .section_with( |
| 1462 | "Second.one", | 1462 | "Second.one", |
| 1463 | move |file| Ok(Counted(file.to_owned(), Arc::clone(&counter))), | 1463 | move |file, _| Ok(Counted(file.to_owned(), Arc::clone(&counter))), |
| 1464 | false, | 1464 | false, |
| 1465 | || {}, | 1465 | || {}, |
| 1466 | ) | 1466 | ) |
| ... | @@ -1544,7 +1544,7 @@ fn the_background_takes_each_file_discovery_read_rather_than_reading_it_again() | ... | @@ -1544,7 +1544,7 @@ fn the_background_takes_each_file_discovery_read_rather_than_reading_it_again() |
| 1544 | let mut notebook = two_sections(directory.path()); | 1544 | let mut notebook = two_sections(directory.path()); |
| 1545 | let reads = Arc::new(AtomicUsize::new(0)); | 1545 | let reads = Arc::new(AtomicUsize::new(0)); |
| 1546 | let counter = Arc::clone(&reads); | 1546 | let counter = Arc::clone(&reads); |
| 1547 | let remote = move |file: &Path| Reads(file.to_owned(), Arc::clone(&counter)); | 1547 | let remote = move |file: &Path, _: Option<&Path>| Reads(file.to_owned(), Arc::clone(&counter)); |
| 1548 | let synced = |background: &notebook::session::Background| { | 1548 | let synced = |background: &notebook::session::Background| { |
| 1549 | let status = background.status(); | 1549 | let status = background.status(); |
| 1550 | status.len() == 2 && status.iter().all(|(_, status)| status.synced.is_some()) | 1550 | status.len() == 2 && status.iter().all(|(_, status)| status.synced.is_some()) |
crates/snowbound/Cargo.toml+1-1| ... | @@ -65,7 +65,7 @@ web-sys = { version = "0.3", features = ["console", "Document", "Element", "Html | ... | @@ -65,7 +65,7 @@ web-sys = { version = "0.3", features = ["console", "Document", "Element", "Html |
| 65 | libc = "0.2" | 65 | libc = "0.2" |
| 66 | objc2 = "0.5.2" | 66 | objc2 = "0.5.2" |
| 67 | objc2-app-kit = { version = "0.2.2", default-features = false, features = ["NSAlert", "NSButton", "NSCell", "NSControl", "NSView", "NSResponder", "NSWindow", "NSApplication", "NSEvent", "NSMenu", "NSMenuItem", "NSColor", "NSColorSpace", "NSDatePicker", "NSDatePickerCell", "NSOpenPanel", "NSPanel", "NSSavePanel", "NSWorkspace"] } | 67 | objc2-app-kit = { version = "0.2.2", default-features = false, features = ["NSAlert", "NSButton", "NSCell", "NSControl", "NSView", "NSResponder", "NSWindow", "NSApplication", "NSEvent", "NSMenu", "NSMenuItem", "NSColor", "NSColorSpace", "NSDatePicker", "NSDatePickerCell", "NSOpenPanel", "NSPanel", "NSSavePanel", "NSWorkspace"] } |
| 68 | objc2-foundation = { version = "0.2.2", default-features = false, features = ["NSString", "NSThread", "NSAttributedString", "NSDate", "NSCalendar", "NSDateFormatter", "NSGeometry", "NSLocale", "NSPathUtilities", "NSTimeZone", "NSURL", "NSArray", "NSRange", "NSError", "NSFileCoordinator", "NSFilePresenter", "NSFileVersion", "NSFileManager", "NSHost", "NSNotification", "NSOperation", "block2"] } | 68 | objc2-foundation = { version = "0.2.2", default-features = false, features = ["NSString", "NSThread", "NSAttributedString", "NSDate", "NSCalendar", "NSDateFormatter", "NSGeometry", "NSLocale", "NSPathUtilities", "NSTimeZone", "NSURL", "NSArray", "NSRange", "NSError", "NSFileCoordinator", "NSFilePresenter", "NSFileVersion", "NSFileManager", "NSNotification", "NSOperation", "block2"] } |
| 69 | block2 = "0.5.1" | 69 | block2 = "0.5.1" |
| 70 | 70 | ||
| 71 | [target.'cfg(not(any(target_os = "macos", target_arch = "wasm32")))'.dependencies] | 71 | [target.'cfg(not(any(target_os = "macos", target_arch = "wasm32")))'.dependencies] |
crates/snowbound/src/icloud_linux.rs-3| ... | @@ -24,7 +24,6 @@ pub fn drive() -> Option<PathBuf> { | ... | @@ -24,7 +24,6 @@ pub fn drive() -> Option<PathBuf> { |
| 24 | pub fn section( | 24 | pub fn section( |
| 25 | notebook: &Notebook, | 25 | notebook: &Notebook, |
| 26 | _: &Path, | 26 | _: &Path, |
| 27 | _: &Path, | ||
| 28 | path: &str, | 27 | path: &str, |
| 29 | key: Option<&onestore::protected::Key>, | 28 | key: Option<&onestore::protected::Key>, |
| 30 | offline: bool, | 29 | offline: bool, |
| ... | @@ -47,8 +46,6 @@ pub fn lone_section( | ... | @@ -47,8 +46,6 @@ pub fn lone_section( |
| 47 | 46 | ||
| 48 | pub fn background( | 47 | pub fn background( |
| 49 | notebook: &Notebook, | 48 | notebook: &Notebook, |
| 50 | _: &Path, | ||
| 51 | _: &Path, | ||
| 52 | notify: impl Fn() + Send + 'static, | 49 | notify: impl Fn() + Send + 'static, |
| 53 | ) -> Result<Background, notebook::Error> { | 50 | ) -> Result<Background, notebook::Error> { |
| 54 | notebook.background(false, true, notify) | 51 | notebook.background(false, true, notify) |
crates/snowbound/src/icloud_macos.rs+61-49| ... | @@ -14,8 +14,6 @@ use objc2::{ | ... | @@ -14,8 +14,6 @@ use objc2::{ |
| 14 | rc::Retained, | 14 | rc::Retained, |
| 15 | runtime::{AnyObject, NSObject, NSObjectProtocol, ProtocolObject}, | 15 | runtime::{AnyObject, NSObject, NSObjectProtocol, ProtocolObject}, |
| 16 | }; | 16 | }; |
| 17 | #[allow(deprecated)] | ||
| 18 | use objc2_foundation::NSHost; | ||
| 19 | use objc2_foundation::{ | 17 | use objc2_foundation::{ |
| 20 | NSFileCoordinator, NSFileCoordinatorReadingOptions, NSFileCoordinatorWritingOptions, | 18 | NSFileCoordinator, NSFileCoordinatorReadingOptions, NSFileCoordinatorWritingOptions, |
| 21 | NSFileManager, NSFilePresenter, NSFileVersion, NSNotification, NSNotificationCenter, | 19 | NSFileManager, NSFilePresenter, NSFileVersion, NSNotification, NSNotificationCenter, |
| ... | @@ -249,6 +247,44 @@ pub struct Coordinated { | ... | @@ -249,6 +247,44 @@ pub struct Coordinated { |
| 249 | published: Option<Published>, | 247 | published: Option<Published>, |
| 250 | } | 248 | } |
| 251 | 249 | ||
| 250 | /// This Mac's name as Sharing shows it, as iCloud Drive names its conflict versions. | ||
| 251 | fn computer() -> &'static str { | ||
| 252 | static NAME: OnceLock<String> = OnceLock::new(); | ||
| 253 | NAME.get_or_init(|| { | ||
| 254 | #[link(name = "SystemConfiguration", kind = "framework")] | ||
| 255 | unsafe extern "C" { | ||
| 256 | fn SCDynamicStoreCopyComputerName( | ||
| 257 | store: *const std::ffi::c_void, | ||
| 258 | encoding: *mut u32, | ||
| 259 | ) -> *mut NSString; | ||
| 260 | } | ||
| 261 | // SAFETY: a null store asks the system's; the name, a CFString the caller owns, is | ||
| 262 | // toll-free bridged to NSString. | ||
| 263 | let name = unsafe { | ||
| 264 | Retained::from_raw(SCDynamicStoreCopyComputerName( | ||
| 265 | std::ptr::null(), | ||
| 266 | std::ptr::null_mut(), | ||
| 267 | )) | ||
| 268 | }; | ||
| 269 | name.map_or_else(|| "Mac".to_owned(), |name| name.to_string()) | ||
| 270 | }) | ||
| 271 | } | ||
| 272 | |||
| 273 | /// The header and length of `file`, once iCloud Drive has it here. | ||
| 274 | fn stamp(file: &Path) -> io::Result<Stamp> { | ||
| 275 | present(file)?; | ||
| 276 | coordinated(file, false, || { | ||
| 277 | use std::io::Read; | ||
| 278 | let mut opened = std::fs::File::open(file)?; | ||
| 279 | let mut header = [0; 1024]; | ||
| 280 | opened.read_exact(&mut header)?; | ||
| 281 | Ok(Stamp { | ||
| 282 | header, | ||
| 283 | length: opened.metadata()?.len(), | ||
| 284 | }) | ||
| 285 | })? | ||
| 286 | } | ||
| 287 | |||
| 252 | fn uncommitted(error: io::Error) -> CommitError { | 288 | fn uncommitted(error: io::Error) -> CommitError { |
| 253 | CommitError { | 289 | CommitError { |
| 254 | state: CommitState::NotCommitted, | 290 | state: CommitState::NotCommitted, |
| ... | @@ -257,19 +293,11 @@ fn uncommitted(error: io::Error) -> CommitError { | ... | @@ -257,19 +293,11 @@ fn uncommitted(error: io::Error) -> CommitError { |
| 257 | } | 293 | } |
| 258 | 294 | ||
| 259 | impl Coordinated { | 295 | impl Coordinated { |
| 260 | /// The file `file` of the notebook folder, or lone section, at `notebook`, whose replicas | 296 | /// The section file `file`, its replica at `replica` where it has one. |
| 261 | /// `cache` holds. | 297 | pub fn new(file: &Path, replica: Option<&Path>) -> Self { |
| 262 | pub fn new(file: &Path, notebook: &Path, cache: &Path) -> Self { | ||
| 263 | static DEVICE: OnceLock<Option<String>> = OnceLock::new(); | ||
| 264 | // The computer's name as Sharing shows it, as iCloud names this Mac's versions; the | ||
| 265 | // Network framework that deprecates NSHost has no such name. | ||
| 266 | #[allow(deprecated)] | ||
| 267 | let device = DEVICE.get_or_init(|| { | ||
| 268 | unsafe { NSHost::currentHost().localizedName() }.map(|name| name.to_string()) | ||
| 269 | }); | ||
| 270 | Self { | 298 | Self { |
| 271 | file: file.to_owned(), | 299 | file: file.to_owned(), |
| 272 | published: Published::of(cache, notebook, file, device.clone()), | 300 | published: replica.map(|replica| Published::of(replica, computer())), |
| 273 | } | 301 | } |
| 274 | } | 302 | } |
| 275 | 303 | ||
| ... | @@ -301,17 +329,7 @@ impl Remote for Coordinated { | ... | @@ -301,17 +329,7 @@ impl Remote for Coordinated { |
| 301 | } | 329 | } |
| 302 | 330 | ||
| 303 | fn stamp(&mut self) -> io::Result<Stamp> { | 331 | fn stamp(&mut self) -> io::Result<Stamp> { |
| 304 | present(&self.file)?; | 332 | stamp(&self.file) |
| 305 | coordinated(&self.file, false, || { | ||
| 306 | use std::io::Read; | ||
| 307 | let mut file = std::fs::File::open(&self.file)?; | ||
| 308 | let mut header = [0; 1024]; | ||
| 309 | file.read_exact(&mut header)?; | ||
| 310 | Ok(Stamp { | ||
| 311 | header, | ||
| 312 | length: file.metadata()?.len(), | ||
| 313 | }) | ||
| 314 | })? | ||
| 315 | } | 333 | } |
| 316 | 334 | ||
| 317 | fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> { | 335 | fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> { |
| ... | @@ -330,7 +348,12 @@ impl Remote for Coordinated { | ... | @@ -330,7 +348,12 @@ impl Remote for Coordinated { |
| 330 | 348 | ||
| 331 | fn confirm(&mut self, base: &Stamp) -> Result<(), CommitError> { | 349 | fn confirm(&mut self, base: &Stamp) -> Result<(), CommitError> { |
| 332 | coordinated(&self.file, true, || { | 350 | coordinated(&self.file, true, || { |
| 333 | onestore::confirm_file(&self.file, base) | 351 | onestore::confirm_file(&self.file, base)?; |
| 352 | if let Some(published) = &self.published { | ||
| 353 | let _ = std::fs::File::open(&self.file) | ||
| 354 | .and_then(|mut file| published.confirmed(Written::InPlace(&mut file), base)); | ||
| 355 | } | ||
| 356 | Ok(()) | ||
| 334 | }) | 357 | }) |
| 335 | .map_err(uncommitted)? | 358 | .map_err(uncommitted)? |
| 336 | } | 359 | } |
| ... | @@ -346,13 +369,9 @@ impl Remote for Coordinated { | ... | @@ -346,13 +369,9 @@ impl Remote for Coordinated { |
| 346 | }) | 369 | }) |
| 347 | }) | 370 | }) |
| 348 | .collect(); | 371 | .collect(); |
| 349 | if self.published.is_some() { | 372 | if let Some(published) = &self.published { |
| 350 | let stamp = self.stamp()?; | 373 | let lost = published.versions(|| stamp(&self.file), &listed)?; |
| 351 | listed.extend( | 374 | listed.extend(lost); |
| 352 | self.published | ||
| 353 | .iter() | ||
| 354 | .flat_map(|published| published.versions(&stamp)), | ||
| 355 | ); | ||
| 356 | } | 375 | } |
| 357 | Ok(listed) | 376 | Ok(listed) |
| 358 | } | 377 | } |
| ... | @@ -382,12 +401,14 @@ impl Remote for Coordinated { | ... | @@ -382,12 +401,14 @@ impl Remote for Coordinated { |
| 382 | && published.holds(id) | 401 | && published.holds(id) |
| 383 | { | 402 | { |
| 384 | if keep { | 403 | if keep { |
| 385 | let device = published.device().unwrap_or("this Mac"); | 404 | let beside = notebook::published::beside(&self.file, published.device()); |
| 386 | let beside = notebook::published::beside(&self.file, device); | ||
| 387 | coordinated(&beside, true, || std::fs::copy(id, &beside))??; | 405 | coordinated(&beside, true, || std::fs::copy(id, &beside))??; |
| 388 | } | 406 | } |
| 389 | return std::fs::remove_file(id); | 407 | return std::fs::remove_file(id); |
| 390 | } | 408 | } |
| 409 | if let Some(published) = &self.published { | ||
| 410 | let _ = published.retired(Path::new(id)); | ||
| 411 | } | ||
| 391 | let version = match self.conflict(id) { | 412 | let version = match self.conflict(id) { |
| 392 | Ok(version) => version, | 413 | Ok(version) => version, |
| 393 | // Another device resolved it first. | 414 | // Another device resolved it first. |
| ... | @@ -419,7 +440,6 @@ impl Remote for Coordinated { | ... | @@ -419,7 +440,6 @@ impl Remote for Coordinated { |
| 419 | pub fn section( | 440 | pub fn section( |
| 420 | notebook: &Notebook, | 441 | notebook: &Notebook, |
| 421 | root: &Path, | 442 | root: &Path, |
| 422 | cache: &Path, | ||
| 423 | path: &str, | 443 | path: &str, |
| 424 | key: Option<&onestore::protected::Key>, | 444 | key: Option<&onestore::protected::Key>, |
| 425 | offline: bool, | 445 | offline: bool, |
| ... | @@ -428,8 +448,7 @@ pub fn section( | ... | @@ -428,8 +448,7 @@ pub fn section( |
| 428 | if !notebook.replica_path(path)?.exists() { | 448 | if !notebook.replica_path(path)?.exists() { |
| 429 | fetch(&root.join(path))?; | 449 | fetch(&root.join(path))?; |
| 430 | } | 450 | } |
| 431 | let (root, cache) = (root.to_owned(), cache.to_owned()); | 451 | let connect = |file: &Path, replica: Option<&Path>| Ok(Coordinated::new(file, replica)); |
| 432 | let connect = move |file: &Path| Ok(Coordinated::new(file, &root, &cache)); | ||
| 433 | let section = match key { | 452 | let section = match key { |
| 434 | Some(key) => notebook.section_unlocked_with(path, key, connect, offline, notify)?, | 453 | Some(key) => notebook.section_unlocked_with(path, key, connect, offline, notify)?, |
| 435 | None => notebook.section_with(path, connect, offline, notify)?, | 454 | None => notebook.section_with(path, connect, offline, notify)?, |
| ... | @@ -458,10 +477,7 @@ pub fn lone_section( | ... | @@ -458,10 +477,7 @@ pub fn lone_section( |
| 458 | let section = Section::open_with( | 477 | let section = Section::open_with( |
| 459 | file, | 478 | file, |
| 460 | cache, | 479 | cache, |
| 461 | { | 480 | |file, replica| Ok(Coordinated::new(file, replica)), |
| 462 | let cache = cache.to_owned(); | ||
| 463 | move |file: &Path| Ok(Coordinated::new(file, file, &cache)) | ||
| 464 | }, | ||
| 465 | offline, | 481 | offline, |
| 466 | notify, | 482 | notify, |
| 467 | )?; | 483 | )?; |
| ... | @@ -475,13 +491,9 @@ pub fn lone_section( | ... | @@ -475,13 +491,9 @@ pub fn lone_section( |
| 475 | /// presenter or the folder's watch reports it. | 491 | /// presenter or the folder's watch reports it. |
| 476 | pub fn background( | 492 | pub fn background( |
| 477 | notebook: &Notebook, | 493 | notebook: &Notebook, |
| 478 | root: &Path, | ||
| 479 | cache: &Path, | ||
| 480 | notify: impl Fn() + Send + 'static, | 494 | notify: impl Fn() + Send + 'static, |
| 481 | ) -> Result<Background, notebook::Error> { | 495 | ) -> Result<Background, notebook::Error> { |
| 482 | let (root, cache) = (root.to_owned(), cache.to_owned()); | 496 | notebook.background_with(false, true, Coordinated::new, notify) |
| 483 | let remote = move |file: &Path| Coordinated::new(file, &root, &cache); | ||
| 484 | notebook.background_with(false, true, remote, notify) | ||
| 485 | } | 497 | } |
| 486 | 498 | ||
| 487 | struct Ivars { | 499 | struct Ivars { |
| ... | @@ -655,7 +667,7 @@ mod tests { | ... | @@ -655,7 +667,7 @@ mod tests { |
| 655 | let notebook = Notebook::create(&root, &cache, Notebook::NEW_COLOR, &page).unwrap(); | 667 | let notebook = Notebook::create(&root, &cache, Notebook::NEW_COLOR, &page).unwrap(); |
| 656 | assert!(ubiquitous(&root)); | 668 | assert!(ubiquitous(&root)); |
| 657 | let path = notebook.catalog().sections[0].path.clone(); | 669 | let path = notebook.catalog().sections[0].path.clone(); |
| 658 | let section = section(&notebook, &root, &cache, &path, None, false, || {}).unwrap(); | 670 | let section = section(&notebook, &root, &path, None, false, || {}).unwrap(); |
| 659 | let (space, ..) = section.pages().unwrap()[0].clone(); | 671 | let (space, ..) = section.pages().unwrap()[0].clone(); |
| 660 | let title = section | 672 | let title = section |
| 661 | .page(space) | 673 | .page(space) |
| ... | @@ -687,7 +699,7 @@ mod tests { | ... | @@ -687,7 +699,7 @@ mod tests { |
| 687 | .unwrap(); | 699 | .unwrap(); |
| 688 | // Wakes skip the pause in typing, as leaving the window does. | 700 | // Wakes skip the pause in typing, as leaving the window does. |
| 689 | let file = root.join(&path); | 701 | let file = root.join(&path); |
| 690 | let mut remote = Coordinated::new(&file, &root, &cache); | 702 | let mut remote = Coordinated::new(&file, None); |
| 691 | let deadline = std::time::Instant::now() + Duration::from_secs(30); | 703 | let deadline = std::time::Instant::now() + Duration::from_secs(30); |
| 692 | loop { | 704 | loop { |
| 693 | section.wake(); | 705 | section.wake(); |
| ... | @@ -725,7 +737,7 @@ mod tests { | ... | @@ -725,7 +737,7 @@ mod tests { |
| 725 | let notebook = Notebook::create(&root, &cache, Notebook::NEW_COLOR, &page).unwrap(); | 737 | let notebook = Notebook::create(&root, &cache, Notebook::NEW_COLOR, &page).unwrap(); |
| 726 | let path = notebook.catalog().sections[0].path.clone(); | 738 | let path = notebook.catalog().sections[0].path.clone(); |
| 727 | let file = root.join(&path); | 739 | let file = root.join(&path); |
| 728 | let section = section(&notebook, &root, &cache, &path, None, false, || {}).unwrap(); | 740 | let section = section(&notebook, &root, &path, None, false, || {}).unwrap(); |
| 729 | let space = section.pages().unwrap()[0].0; | 741 | let space = section.pages().unwrap()[0].0; |
| 730 | let stored = || { | 742 | let stored = || { |
| 731 | let arena = onestore::Arena::default(); | 743 | let arena = onestore::Arena::default(); |
crates/snowbound/src/library.rs+8-11| ... | @@ -413,7 +413,7 @@ impl Library { | ... | @@ -413,7 +413,7 @@ impl Library { |
| 413 | } | 413 | } |
| 414 | let mut notebook = Notebook::open(location, cache); | 414 | let mut notebook = Notebook::open(location, cache); |
| 415 | let (background, watch) = match &mut notebook { | 415 | let (background, watch) = match &mut notebook { |
| 416 | Ok(notebook) => local_background(notebook, location, cache), | 416 | Ok(notebook) => local_background(notebook, location), |
| 417 | Err(_) => (None, None), | 417 | Err(_) => (None, None), |
| 418 | }; | 418 | }; |
| 419 | let name = display_name(cache, location).unwrap_or_else(|| file_name(Path::new(location))); | 419 | let name = display_name(cache, location).unwrap_or_else(|| file_name(Path::new(location))); |
| ... | @@ -819,7 +819,7 @@ impl Library { | ... | @@ -819,7 +819,7 @@ impl Library { |
| 819 | 819 | ||
| 820 | /// A notebook `Notebook::create` just made in the folder at `location`. | 820 | /// A notebook `Notebook::create` just made in the folder at `location`. |
| 821 | pub fn created(location: &str, mut notebook: Notebook, cache: &Path) -> Self { | 821 | pub fn created(location: &str, mut notebook: Notebook, cache: &Path) -> Self { |
| 822 | let (background, watch) = local_background(&mut notebook, location, cache); | 822 | let (background, watch) = local_background(&mut notebook, location); |
| 823 | Self { | 823 | Self { |
| 824 | background, | 824 | background, |
| 825 | watch, | 825 | watch, |
| ... | @@ -902,7 +902,7 @@ impl Library { | ... | @@ -902,7 +902,7 @@ impl Library { |
| 902 | #[cfg(feature = "live")] | 902 | #[cfg(feature = "live")] |
| 903 | if let Some(joined) = &self.joined { | 903 | if let Some(joined) = &self.joined { |
| 904 | let guest = Arc::clone(&joined.guest); | 904 | let guest = Arc::clone(&joined.guest); |
| 905 | section.relocate_with(path, move |file| { | 905 | section.relocate_with(path, move |file, _| { |
| 906 | Ok(notebook::live::share::HostedRemote::new( | 906 | Ok(notebook::live::share::HostedRemote::new( |
| 907 | &guest, | 907 | &guest, |
| 908 | &file.to_string_lossy(), | 908 | &file.to_string_lossy(), |
| ... | @@ -921,7 +921,7 @@ impl Library { | ... | @@ -921,7 +921,7 @@ impl Library { |
| 921 | root => format!("{root}/{path}"), | 921 | root => format!("{root}/{path}"), |
| 922 | }; | 922 | }; |
| 923 | let server = Arc::clone(server); | 923 | let server = Arc::clone(server); |
| 924 | section.relocate_with(file, move |file| { | 924 | section.relocate_with(file, move |file, _| { |
| 925 | Ok(notebook::SmbRemote::new( | 925 | Ok(notebook::SmbRemote::new( |
| 926 | server.connect()?, | 926 | server.connect()?, |
| 927 | file.to_string_lossy(), | 927 | file.to_string_lossy(), |
| ... | @@ -935,9 +935,8 @@ impl Library { | ... | @@ -935,9 +935,8 @@ impl Library { |
| 935 | let file = Path::new(&self.location).join(path); | 935 | let file = Path::new(&self.location).join(path); |
| 936 | #[cfg(target_os = "macos")] | 936 | #[cfg(target_os = "macos")] |
| 937 | if self.in_icloud() { | 937 | if self.in_icloud() { |
| 938 | let (root, cache) = (PathBuf::from(&self.location), self.cache.clone()); | 938 | section.relocate_with(file, |file, replica| { |
| 939 | section.relocate_with(file, move |file| { | 939 | Ok(crate::icloud::Coordinated::new(file, replica)) |
| 940 | Ok(crate::icloud::Coordinated::new(file, &root, &cache)) | ||
| 941 | })?; | 940 | })?; |
| 942 | } else { | 941 | } else { |
| 943 | section.relocate(file)?; | 942 | section.relocate(file)?; |
| ... | @@ -1025,7 +1024,6 @@ impl Library { | ... | @@ -1025,7 +1024,6 @@ impl Library { |
| 1025 | crate::icloud::section( | 1024 | crate::icloud::section( |
| 1026 | notebook, | 1025 | notebook, |
| 1027 | root, | 1026 | root, |
| 1028 | &self.cache, | ||
| 1029 | path, | 1027 | path, |
| 1030 | key.as_ref(), | 1028 | key.as_ref(), |
| 1031 | offline(), | 1029 | offline(), |
| ... | @@ -1535,13 +1533,12 @@ fn read_tag_art(notebook: &Notebook) -> TagArt { | ... | @@ -1535,13 +1533,12 @@ fn read_tag_art(notebook: &Notebook) -> TagArt { |
| 1535 | fn local_background( | 1533 | fn local_background( |
| 1536 | notebook: &mut Notebook, | 1534 | notebook: &mut Notebook, |
| 1537 | location: &str, | 1535 | location: &str, |
| 1538 | cache: &Path, | ||
| 1539 | ) -> (Option<Arc<Background>>, Option<Arc<crate::watch::Watch>>) { | 1536 | ) -> (Option<Arc<Background>>, Option<Arc<crate::watch::Watch>>) { |
| 1540 | let reports = Arc::new(OnceLock::new()); | 1537 | let reports = Arc::new(OnceLock::new()); |
| 1541 | let watch = watcher(location, Arc::clone(&reports)); | 1538 | let watch = watcher(location, Arc::clone(&reports)); |
| 1542 | let copies = !crate::watch::on_this_computer(Path::new(location)); | 1539 | let copies = !crate::watch::on_this_computer(Path::new(location)); |
| 1543 | let background = match crate::icloud::ubiquitous(Path::new(location)) { | 1540 | let background = match crate::icloud::ubiquitous(Path::new(location)) { |
| 1544 | true => crate::icloud::background(notebook, Path::new(location), cache, notify_background), | 1541 | true => crate::icloud::background(notebook, notify_background), |
| 1545 | false => notebook.background(watch.is_some(), copies, notify_background), | 1542 | false => notebook.background(watch.is_some(), copies, notify_background), |
| 1546 | }; | 1543 | }; |
| 1547 | let Some(background) = background | 1544 | let Some(background) = background |
| ... | @@ -1772,7 +1769,7 @@ mod tests { | ... | @@ -1772,7 +1769,7 @@ mod tests { |
| 1772 | let section = Section::open_with( | 1769 | let section = Section::open_with( |
| 1773 | &file, | 1770 | &file, |
| 1774 | root.join("cache"), | 1771 | root.join("cache"), |
| 1775 | move |file| { | 1772 | move |file, _| { |
| 1776 | Ok(Stalled { | 1773 | Ok(Stalled { |
| 1777 | file: file.to_owned(), | 1774 | file: file.to_owned(), |
| 1778 | entered: Arc::clone(&entered), | 1775 | entered: Arc::clone(&entered), |
crates/snowbound/src/undo.rs+1-1| ... | @@ -1872,7 +1872,7 @@ mod tests { | ... | @@ -1872,7 +1872,7 @@ mod tests { |
| 1872 | let arrived = Arc::clone(b.replica()); | 1872 | let arrived = Arc::clone(b.replica()); |
| 1873 | let (entered, holding) = std::sync::mpsc::channel(); | 1873 | let (entered, holding) = std::sync::mpsc::channel(); |
| 1874 | let (release, released) = std::sync::mpsc::channel::<()>(); | 1874 | let (release, released) = std::sync::mpsc::channel::<()>(); |
| 1875 | b.relocate_with(fixture.folder.join("Notebook/B.one"), move |_| { | 1875 | b.relocate_with(fixture.folder.join("Notebook/B.one"), move |_, _| { |
| 1876 | let _ = entered.send(()); | 1876 | let _ = entered.send(()); |
| 1877 | let _ = released.recv(); | 1877 | let _ = released.recv(); |
| 1878 | Err::<Unreached, _>(std::io::ErrorKind::TimedOut.into()) | 1878 | Err::<Unreached, _>(std::io::ErrorKind::TimedOut.into()) |