Migrating to new Mac with large iCloud Library - hints, experiences, wise words sought!

A place for users to ask each other questions, make suggestions, and discuss Bookends.
Post Reply
macgarvin
Posts: 16
Joined: Mon Oct 15, 2018 9:44 am

Migrating to new Mac with large iCloud Library - hints, experiences, wise words sought!

Post by macgarvin »

I'm doing an orderly (non-panic situation!) transfer to a new Mac M3 powerbook. I've done it via a fresh full Time Capsule back-up, and then migrated from that to the virgin out-of-the-box Mac. Done yesterday over an afternoon and evening, which was pleasantly fast considering it involved over 2TB of data.

The only thing that is taking time (I wouldn't call it an issue yet) is Bookends, with the library stored in iCloud. Having been using a Mac since 1986, and accumulating pdfs pretty much since they were invented, it includes over 30,000 references and there are over 18,000 pdfs of papers, books etc. in the iCloud Bookends folder.

Immediately after the migration, all the pdfs seemed to be displayed in the Bookends iCloud folder. But now, after closing and re-opening, and doing various restarts to do with installing other software/ MacOS updates etc., I just see a few files (different each time I restart the Mac) along with inconstant indications of progress regarding update progress for iCloud folders.

By way of background information, there is plenty of space for all the iCloud files to have been stored locally on the original MacBook Pro, on the 2TB iCloud account, and on the new MacBook Pro.

I'm aware of a recent discussion regarding a Bookends IOS issue when the Bookend iCloud folder has more than 10,000 files.

viewtopic.php?t=5933

For now, being aware that fiddling around with things while in the middle of a big synchronisation is not a good idea, I'm just letting the new Mac grind away synchronising - or not.

However if anyone else has experience at doing a migration like this, and has hints, experiences, or wise words, I'd be interested in hearing them, as might people in future doing big migrations.

Malcolm
Jon
Site Admin
Posts: 10073
Joined: Tue Jul 13, 2004 6:27 pm
Location: Bethesda, MD
Contact:

Re: Migrating to new Mac with large iCloud Library - hints, experiences, wise words sought!

Post by Jon »

If I understand you correctly, your seeing a problem with the PDFs in iCloud Drive, nothing to do with Bookends per se. Is that correct? If so, could you say a bit more about what you mean by "I just see a few files (different each time I restart the Mac)"?

Jon
Sonny Software
macgarvin
Posts: 16
Joined: Mon Oct 15, 2018 9:44 am

Re: Migrating to new Mac with large iCloud Library - hints, experiences, wise words sought!

Post by macgarvin »

Thanks for responding Jon
Jon wrote: Thu Dec 21, 2023 8:34 am If I understand you correctly, your seeing a problem with the PDFs in iCloud Drive, nothing to do with Bookends per se. Is that correct? If so, could you say a bit more about what you mean by "I just see a few files (different each time I restart the Mac)"?
Not quite - there is a separate issue with Bookends, which I'll describe in a moment - I didn't want to overload the first post.

Dealing first with "I just see a few files (different each time I restart the Mac)"?"

On the new Mac, after around 18 hours I see e.g.

New Mac - Bookends iCloud Folder before restart.jpg
New Mac - Bookends iCloud Folder before restart.jpg (138.06 KiB) Viewed 8941 times
and after a restart I see e.g.

New Mac - Bookends iCloud Folder after restart.jpg
New Mac - Bookends iCloud Folder after restart.jpg (159.38 KiB) Viewed 8941 times
(In this case no files show up, sometimes a handful of other files show up, different from the previous occasion). You can also see a 'Bookends temp' folder.

Meanwhile the iCloud Bookends folder on the Original Mac still appears as I expec in my normal 'Date Created' order, with files appearing in the order that I have recently added them. However if I view in 'Date Added' order I can see ancient files being listed as newly added in recent hours (although only in the cloud), so there seems to be activity going on.

Original Mac - Date Added Order.jpeg
Original Mac - Date Added Order.jpeg (168.21 KiB) Viewed 8941 times


With Regard to a separate issue with Bookends on the New Mac

Currently I can't open my library on the new Mac. If I click on either the Bookends Icon, or on my library file (in Library>Applications Support>Bookends) after a long delay, ca. 10 minutes, I see the message:-

"Do you want to create a new reference library?
Reference information must be stored in a library. Press Create to make a new library. Note that y0u can make a new library at any time with File -> New Library."
Followed by the buttons 'Not Now' or 'Create'

I had assumed that this was to do with the incomplete synchronisation in iCloud, but perhaps not.
Meanwhile the library is working fine on the original Mac.

Sorry that this seems so convoluted - I was half expecting that someone would answer ' It's fine, just be patient, it will sort itself out with time".
Jon
Site Admin
Posts: 10073
Joined: Tue Jul 13, 2004 6:27 pm
Location: Bethesda, MD
Contact:

Re: Migrating to new Mac with large iCloud Library - hints, experiences, wise words sought!

Post by Jon »

The file syncing is handled entirely by Apple, so I can't comment except to say it's probably just slow and you're seeing artifacts related to that.

As for Bookends, though, when you do a File -> New Library -> Download From Cloud, the download will take time if you have many thousands of references. But when complete you should be able to open it. Confirm that the entire library was downloaded by comparing it's size to the one on the other Mac (they won't be identical at first for a variety of reasons, but you'll see if the sizes are grossly different).

Jon
Sonny Software
macgarvin
Posts: 16
Joined: Mon Oct 15, 2018 9:44 am

Re: Migrating to new Mac with large iCloud Library - hints, experiences, wise words sought!

Post by macgarvin »

Jon - I've been working with Apple on this for the last couple of days. We've done a couple of complete reinstalls using Migration Assistant/Timecapsule backups, and tried opening the new Mac in safe mode, along with various other checks, without resolving the problem.

The senior advisor at Apple asked me to ask you whether are there any third party apps (virus protection etc. VPN, anything else) that Bookends needs to be uninstalled, rather than just temporarily disabled?).

Meanwhile, when I opened Bookends up on an iMac, on which I had not used Bookends for a year, the iCloud Bookends folder updated a few GB of new pdfs (for the iMac) very quickly, at 100-200 lbs and synced in a hour or so.

iCloud appears to be a mystery known to God!

Malcolm
Jon
Site Admin
Posts: 10073
Joined: Tue Jul 13, 2004 6:27 pm
Location: Bethesda, MD
Contact:

Re: Migrating to new Mac with large iCloud Library - hints, experiences, wise words sought!

Post by Jon »

Why makes you think he does?

So I can be clear about the current status:

1. PDF syncing using iCloud Drive isn't working properly. As I said, that had nothing to do with Bookends. You can remove Bookends from your system and the iCloud Drive file sync behavior wouldn't change. So I'm afraid I can't shed any light on that -- except to say this isn't the first time we've seen iCloud Drive behave oddly. I don't know if this was mentioned in this long thread, but if you're not syncing with Bookends for iOS you don't have to use iCloud Drive for your PDFs. You can use any file sync service (I see you have Dropbox). Just make sure you tell Bookends where the default attachment folder is and Bookends will find the PDFs if you move them.

2. Is Bookends still crashing on launch? Is so, there are a few things you might try.

Jon
Sonny Software
macgarvin
Posts: 16
Joined: Mon Oct 15, 2018 9:44 am

Re: Migrating to new Mac with large iCloud Library - hints, experiences, wise words sought!

Post by macgarvin »

Thanks Jon
There seems to have been some lurking corruption on my original Mac, that was getting carried over with Migration Assistant onto the new Mac, and only then revealing itself in this behaviour.

So I did a factory reset on the new Mac, and didn't uses Migration Assistant (selectively using the Time Capsule back up instead).

iCloud then synced correctly. On Bookeds I then unsynced the Library from iClouds, and then resynced it, and it is back up and running correctly.
Malcolm
Post Reply