Bookends can't find attachment - but it exists with a slightly different filename. Automatic fix?

Users asking other users for AppleScripts that work with Bookends.
Post Reply
paranous79
Posts: 15
Joined: Sat Aug 05, 2017 11:06 am

Bookends can't find attachment - but it exists with a slightly different filename. Automatic fix?

Post by paranous79 »

Hi,

I must have changed some settings a year or two ago in how Bookends stores attachment names. Ever since then, I now have many attachments that Bookends cannot find, even though they are in the Bookends folder, but they have their older filename format. I'll provide an example (please see the attached screenshot for the reference 'Golinski_1988...' that I have highlighted): you can see that Bookends can't find the attachment and that's because it is looking for an attachment name that is slightly different, especially in terms of 'long' vs. 'short' underscores and some extra dashes. They are very similar but not exact. This is true for hundreds, perhaps thousands, of my attachments, so naturally I don't want to manually re-attach each of these.

__
Screenshot 2024-02-13 at 10.27.23.png
Screenshot 2024-02-13 at 10.27.23.png (125.9 KiB) Viewed 797 times
__

Can anyone think of an automatic way to fix this? How, in other words, do I tell Bookends to look for filenames in the current format - not the one with which they were originally attached, if that makes sense?

Basically, is there a more fine-grained way to control how Bookends tries to find these orphan attachments by filename? If I could tell it to look for a slightly different format, it could automatically find all these orphaned attachments. I know that if I manually change the filename in my attachment folder to match what Bookends is looking for, it does automatically 'update' and find the attachment...

Any insight/help would be very appreciated! I have been trying to fix this for more than a year...

-JW
Jon
Site Admin
Posts: 10054
Joined: Tue Jul 13, 2004 6:27 pm
Location: Bethesda, MD
Contact:

Re: Bookends can't find attachment - but it exists with a slightly different filename. Automatic fix?

Post by Jon »

The file name is the "unique id", so if it's off by even a space Bookends won't match it (anymore than a bank will match your credit card number if it's off by a single digit).

There is, however, one possibility that might help. Make a COPY of your library (as backup) and try Refs -> Get PDF -> Find & Attach Local PDF. Select the folder containing your attachments (loose, not in subfolders) and hope for the best. Bookends uses Spotlight to try to match reference metadata with PDF metadata (name is irrelevant, I think). Success rate isn't 100%, of course, but it will probably find and attach some. Please try that and post here about the result.

Jon
Sonny Software
paranous79
Posts: 15
Joined: Sat Aug 05, 2017 11:06 am

Re: Bookends can't find attachment - but it exists with a slightly different filename. Automatic fix?

Post by paranous79 »

Thanks for the quick response.

Here's what happened. I was able to find the Golinski file using that method but it was a pretty manual process. I had to sift through 72 'matching' PDFs to find the right one, but it was there:
__
Screenshot 2024-02-13 at 19.49.59.png
Screenshot 2024-02-13 at 19.49.59.png (94.44 KiB) Viewed 781 times
__

Then my original reference had 2 attachments, the 'dead' one and the new, correct one, which I chose:
__
Screenshot 2024-02-13 at 19.50.25.png
Screenshot 2024-02-13 at 19.50.25.png (13.54 KiB) Viewed 781 times
__

Finally, I detached the 'dead' attachment:

__
Screenshot 2024-02-13 at 19.50.41.png
Screenshot 2024-02-13 at 19.50.41.png (28.75 KiB) Viewed 781 times
__

Given this process, it's probably faster to simply open up my Bookends folder and manually identify the correct attachment and re-attach it, followed by detachment of the 'dead' one.

I also tried the Get Local PDF command on the next reference after the one above, and - while it found 66 options - none of them were correct. Given the range of options it provides, Spotlight seems fairly unreliable (nothing you can do about this, of course).

So, I think we're possibly back where we started - there is no automatic way to tell Bookends the 'correct' filename format for 'orphan' attachments. I have the gut feeling there is a creative solution here, but not sure what that is. Perhaps the ability to right-click on a reference and do a search of the default attachments folder using the string that is listed in the 'Can't find...' statement?

(Another timesaver in this context would be a command that tells Bookends to automatically *detach* all attachments that are linked to Orphan references.)
paranous79
Posts: 15
Joined: Sat Aug 05, 2017 11:06 am

Re: Bookends can't find attachment - but it exists with a slightly different filename. Automatic fix?

Post by paranous79 »

Or another idea: using the 'Can't find...' filename for the dead attachment to 'rename' the modified name of the existing attachment in the Bookends folder (which Bookends can't find...). This would probably require some AppleScript - fine - but I'd first need to know how to get the 'Can't find...'filename string from Bookends (it's not 'selectable', for instance).

Thanks!
JW
Jon
Site Admin
Posts: 10054
Joined: Tue Jul 13, 2004 6:27 pm
Location: Bethesda, MD
Contact:

Re: Bookends can't find attachment - but it exists with a slightly different filename. Automatic fix?

Post by Jon »

Bookends already has the option to remove (detach) the names of orphaned attachments from the database. This is in Refs -> Global Change.

As for getting the names of attachments, you can do that via an AppleScript. You can also create a format that outputs attachments names, if you want something you can copy/paste. You open the formatted reference display pane on the bottom to do that. You can also have an Attachment column in the list So there are lots of ways to get these.

Another feature is that if you right click on the "can't find" display you get the option to Reattach. If you do this, Bookends will replace the name it couldn't find with the correct one. It's still done on a case-by-case basis, but it save a detach step each time.

Finally, as to the Spotlight issue, what you see isn't typical of what I do (so many false positive matches). I wonder if it could be due to the fact that most of your names use underlines instead of spaces. Or if your Spotlight database needs to be rebuilt (google for that). And I assume that you specified the folder containing the attachments, not the entire HD. Just mentioning that.

Jon
Sonny Software
paranous79
Posts: 15
Joined: Sat Aug 05, 2017 11:06 am

Re: Bookends can't find attachment - but it exists with a slightly different filename. Automatic fix?

Post by paranous79 »

Hi Jon,

Is there any way to interact with Bookends via Python? Or is it all done through AppleScript? Any future plans for this?

I've got a script I'm working on that combines some AppleScript with Python to do what I want - more on that in a bit - but it would be much simpler to simply work through Python. The Python script, for instance, does a great job of matching the Bookends 'missing' attachment file path with similar (but not identical) existing attachments (actually, their filenames, to be more precise) in my Bookends folder, thereby automating a big chunk of the replacement process. It is also more accurate than the 'Find & Attach Local PDF' command, when done through Apple's Spotlight.

Best,
JW
Jon
Site Admin
Posts: 10054
Joined: Tue Jul 13, 2004 6:27 pm
Location: Bethesda, MD
Contact:

Re: Bookends can't find attachment - but it exists with a slightly different filename. Automatic fix?

Post by Jon »

I don't know what hybrid approach you have come up with, but you can run Python in an AppleScript using the osascript command. I expect you are aware of this, but if not (and for others reading this) googling for this yields a lot of information. You can also do this via Automator, I believe. Do your file name processing in Python and use AppleScript to update the Bookends database. Or vice versa, get the attachment names in the Bookends database (easy via AppleScript) and update the corresponding file name in the Finder via you Python script.

If you don't mind sharing, I'd be curious to hear how the Python script does its magic.

Jon
Sonny Software
paranous79
Posts: 15
Joined: Sat Aug 05, 2017 11:06 am

Re: Bookends can't find attachment - but it exists with a slightly different filename. Automatic fix?

Post by paranous79 »

I ended up just writing an AppleScript that called my Python script ('match_attachments.py') from within it, using the following AS code:

Code: Select all


if pubAttachments is not "" then
					set pythonScriptPath to "(Local user path would go here)_match_attachments.py"
					set firstAttachmentPath to pubAttachments
					set pythonCommand to "python3 " & quoted form of pythonScriptPath & " " & quoted form of firstAttachmentPath 
For the Python script itself, see below. Here's the version I used to automatically replace the attachment path in Bookends for missing attachments (Can't find file...), so long as the filename in my Bookends folder was sufficiently similar. On a 'similarity scale' between 0 and 1, I found about 0.8 to be a good enough matching balance (but of course, one is free to change this at will, which offers flexibility), without being too restrictive but also not letting in too many false positives (though a few did creep in - always a good idea to verify - but I would say it worked with 95% accuracy).

It uses Python's built-in 'difflib' to do the matching (using SequenceMatcher).

I first made it with the ability to ask the user to approve filenames if they fell below the automatic threshold of 0.8 (which were automatically accepted) but above a prompt threshold of 0.55 or so. But I ended up not using this, which is why the comment below reads 'DID NOT IMPLEMENT'. But it worked, when I was trying it out.

Finally, it uses the command line string value that the AppleScript sends to it (if len(sys.argv) > 1:
search_query = sys.argv[1]) as the search query for the matching. And it prints the matching attachment path back to the console which is picked up as 'matchResult' by the AppleScript:

Code: Select all

try
	set matchResult to do shell script pythonCommand
	if matchResult is not "None" and matchResult does not start with "Error" then
		set attachments of aPub to matchResult
		-- Add the successfully updated attachment path to the success list
		set end of successList to matchResult
		
-JW
___

Code: Select all

# Import statements (if any)
import sys
from difflib import SequenceMatcher
import os
import time

# Function and class definitions (if any)

def absolute_similarity(search_query, candidate):
    """
    Calculate the absolute similarity score between two strings.

    :param search_query: The query string.
    :param candidate: The candidate string for comparison.
    :return: Similarity score as a float between 0 and 1.
    """
    return SequenceMatcher(None, search_query, candidate).ratio()

# Assuming 'attachment_folder_path' is the path to your Bookends attachment folder
attachment_folder_path = '.../Library/Mobile Documents/iCloud~com~sonnysoftware~bot/Documents' # USE YOUR OWN local Bookends attachment folder path
attachment_files = [f for f in os.listdir(attachment_folder_path) if
                    os.path.isfile(os.path.join(attachment_folder_path, f))]

# Define your absolute similarity threshold for automatic acceptance
auto_accept_threshold = 0.80  # Adjust as needed

# DID NOT IMPLEMENT: Lower threshold for showing a prompt - you might still consider scores above a certain lower limit
prompt_threshold = 0.55  # Adjust as needed

if __name__ == "__main__":
    # Accept the search query filename from command-line arguments
    if len(sys.argv) > 1:
        search_query = sys.argv[1]

        highest_score = 0
        best_match = None
        for candidate in attachment_files:
            score = absolute_similarity(search_query, candidate)
            if score > highest_score:
                highest_score = score
                best_match = candidate

        # Check if the highest score is above the threshold
        if highest_score >= auto_accept_threshold:
            correct_path = best_match  # Save the relative path of the matching file

        else:
            correct_path = "None"  # No match found

        print(correct_path)  # Print the correct path value (either correct path or 'None') to the console for further processing in the main AppleScript

    else:
        print("Error: No search query provided.")
Jon
Site Admin
Posts: 10054
Joined: Tue Jul 13, 2004 6:27 pm
Location: Bethesda, MD
Contact:

Re: Bookends can't find attachment - but it exists with a slightly different filename. Automatic fix?

Post by Jon »

Thank you very much for posting this. I'm sure others will benefit from it, too. I wonder if you'd be willing to post this to the AppleScript section of the forum? That's where people looking for such solutions would naturally go, and since there is less traffic it won't be lost among many other threads, which is what will happen to this one with time. If you'd prefer, I can create a new thread there myself and copy/paste the individual parts of your post.

Jon
Sonny Software
paranous79
Posts: 15
Joined: Sat Aug 05, 2017 11:06 am

Re: Bookends can't find attachment - but it exists with a slightly different filename. Automatic fix?

Post by paranous79 »

Sure, you are welcome to do that.

-JW
paranous79
Posts: 15
Joined: Sat Aug 05, 2017 11:06 am

Re: Bookends can't find attachment - but it exists with a slightly different filename. Automatic fix?

Post by paranous79 »

For anyone reading this thread at a later date - I did not upload my full solution that combines both the AppleScript and the Python script. I could do so, if there is interest (I'd have to modify things to ensure it works for a user besides myself). I have no idea if this is a widespread issue or not. Feel free to respond to this thread if you want me to do this at some point (time is always in short supply).

When it works, it does save an incredible amount of time, if the problem affects hundreds or thousands of your attachments.

-JW
Post Reply