AppleScript to Export Notes in OPML

Users asking other users for AppleScripts that work with Bookends.
dave83
Posts: 38
Joined: Fri Oct 02, 2015 8:55 am

AppleScript to Export Notes in OPML

Post by dave83 » Fri Oct 02, 2015 3:31 pm

I'm a new user to Bookends and looking for a capability (script) to extract my note cards for selected references in OPML format. My intent is to then import these notes into the research folder in Scrivener. I am not versed in AppleScript to write (develop) this myself, so looking for either an exisiting script or work with someone to develop an exisiting script.
Cheers,
Dave

Farcas
Posts: 14
Joined: Sat Feb 15, 2014 10:20 pm

Re: AppleScript to Export Notes in OPML

Post by Farcas » Wed Nov 18, 2015 12:10 pm

Brilliant idea. I second the request for a script as I'm in a similar boat - not very good with scripting my automation.



Thanks!

Farcas
Posts: 14
Joined: Sat Feb 15, 2014 10:20 pm

Re: AppleScript to Export Notes in OPML

Post by Farcas » Sat Dec 12, 2015 2:53 am

+1

At present all I can do is copy and paste - plain text. It would be very nice to have little more from copy/paste and/or a script to pull from the data a bite better into a word processor. My preference is Mellel.


Thanks!

dave83
Posts: 38
Joined: Fri Oct 02, 2015 8:55 am

Re: AppleScript to Export Notes in OPML

Post by dave83 » Mon Feb 08, 2016 6:41 pm

Jon,

So I spent the weekend learning AppleScript (to a very limited degree) and have the basic foundation of my scrip to extract the notes into an OPML file. I believe I understand how Bookends stores the notes; specifically the header, tags, and page reference using the #, %, and @ designators. To ensure I am parsing the note cards correctly, I want to understand if there are any variances to these rules. I used pages 89-92 of the user guide as my reference point for these rules.

1) If there is a header, the first character of the note will be a #, otherwise the note does not have a header.

2) The header text will begin immediately after the header designator (#) or page number reference (see rule 3 below).

3) Page numbers are designated with the @ character and the page numbers immediately follow and the page reference ends with a space.

4) The page designator (@) will either be the first or second character of the note; depending on whether the note has a header (see rule 1 above).

5) Tags designator (%) is either in the first or last line of the note (after any headers and page references) and the tag designator will be the first character in that line

6) Mulitple tags are separated by a single space and each tag starts with the designator (%)

7) Tag lists terminate with an ASCII 13 (return) -- I am assuming this

Furthermore,
A) How are quotes (highlighted text from the PDF copied/made into a Bookends note) designated?

B) When I extract the notes field using the RFLD command, is the formating removed or do you use markdown commands for bold (<b>), italics, (<I>), etc.

Thanks for your help. I have all the basic funcationaliy down and am now working on the tweaking the script to better format the resulting notes in Scrivener. I also have similar questions for them so I can ensure I put the headers in the correct OPML strings. Once I have this is completed, I'll do some error check and then run tests. Then I post here for general use.
Cheers,
Dave

Jon
Site Admin
Posts: 8356
Joined: Tue Jul 13, 2004 6:27 pm
Location: Bethesda, MD
Contact:

Re: AppleScript to Export Notes in OPML

Post by Jon » Tue Feb 09, 2016 2:21 pm

Your summary looks largely correct. Re #7, Return is ASCII 10 (actually linefeed) in OS X.

Quotes begin with >, and end with a Return. If you have multiple Returns in the same quote, put > in front of the first word after the Return.

RFLD does not return styled text information. To obtain styled text (as RTF, not markup) you'd have to use GUID and create a format that you could parse for individual fields.

Jon
Sonny Software

KirtWilson
Posts: 3
Joined: Thu Jul 21, 2016 5:08 pm

Re: AppleScript to Export Notes in OPML

Post by KirtWilson » Sat Jul 23, 2016 1:35 pm

Dave,

Did you ever finish your OPML script? If you did, would you mind sharing it here?

Thank you,

Kirt

dave83
Posts: 38
Joined: Fri Oct 02, 2015 8:55 am

Re: AppleScript to Export Notes in OPML

Post by dave83 » Sat Jul 30, 2016 1:10 pm

Kirt,

Not quite. Most functionality is done, but working through error conditions. Due to heavy work commitments through August, I do not expect to get back to it until then. I will post and share once done.
Cheers,
Dave

dave83
Posts: 38
Joined: Fri Oct 02, 2015 8:55 am

Re: AppleScript to Export Notes in OPML

Post by dave83 » Mon Jul 31, 2017 10:09 pm

To support my research workflow, in created an AppleScript to extract selected references and associated from Bookends into an OPML file which can be imported into Scrivener's Research folder. Below is a screenshot showing the notecard structure in the binder, the top level notecard in the left editor pane, the subordinate notecards in the right editor pane, and the comments in the inspector.
Scrivener Screenshot v2.jpg
Scrivener Screenshot v2.jpg (141.75 KiB) Viewed 5745 times
This script converts Bookends references and associated notes into an OPML structured file which can then be imported into Scrivener's research folder. Each reference is a top level card which contains the Title, Author, Date, Type, Publisher, Abstract, and Bookends citation key. If there are notes associated with a reference, each note creates its own subordinate note card with the Page number (if any), note header, quotes, comments, and keywoards (tags). This allows you to individually review each comment and change its status (label) within Scrivener.

The script does some error checking as follows:
  • Strips images from notes
    Converts double quotes (") to single quotes (')
    Converts ampersand (&) to the word "and"
The script is written very modularly so that it can easily be adapted based on changes in OPML syntax, the need to add additional "bad characters", or changes in the Bookends reference or note delimiters or event calls. I know of one issue related to Bookends reference types and have reached out to Jon for some assistance.

Code: Select all

--Script to Export Bookends Notes to OPML file v1.0
--Written by Dave Glogowski
--29 July 2017
--
--This script converts Bookends references and associated notes into an OPML structured file which can then be imported into Scrivener's research folder.
--Each reference is a top level card which contains the Title, Author, Date, Type, Publisher, Abstract, and Bookends citation key
--If there are notes associated with a reference, each note creates its own subordinate note card with the Page number (if any), note header, quotes, 
--comments, and keywoards (tags).  This allows you to individually review each comment and change its status (label) within Scrivener.
--
--The script does some error checking as follows:
--  - Strips images from notes
--  - Converts double quotes (") to single quotes (')
--  - Converts ampersand (&) to the word "and"
--
--The script is written very modularly so that it can easily be adapted based on changes in OPML syntax, the need to add additional "bad characters", or 
--changes in the Bookends reference or note delimiters or event calls
--
--
--
--Variable Setup------------------------------------------------------------------------
--Set Counters
set nbr_references to 0
set nbr_notes to 0

--Set Control Variables
set remove_headers to true
set userCanceled to false

--Set Old Text Delimiters
set tid to AppleScript's text item delimiters
set note_delimiter to (ASCII character 10) & (ASCII character 10)
set comma to ","

--set Bookends Delimiters
set be_page_nbr_delimiter to "@"
set be_header_delimiter to "#"
set be_tag_delimiter to "%"
set be_quote_delimiter to ">"
set be_cite_delimiter to ";"

--set image content tags
set open_image_tag to "<iimg>"
set end_image_tag to "</iimg>"

--Set OPML Text variables
set xml_version to "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>" & return
set opml_version to "<opml version=\"1.0\">" & return
set opml_close to "</opml>" & return
set opml_header to tab & "<head>" & return
set opml_title to tab & tab & "<title>Bookends to Scrivener OPML File</title>" & return
set opml_date to tab & tab & "<dateCreated>" & (current date) & "</dateCreated>" & return
set opml_header_close to tab & "</head>" & return
set opml_body to tab & "<body>" & return
set opml_body_close to tab & "</body>" & return
set opml_outline to tab & tab & "<outline text=\""
set opml_outline_close to tab & tab & "</outline>" & return
set opml_notes to "  _note=\""

--File Name and Setup-----------------------------------------------------------------
--Ask user to remove header only Bookends notes
try
	set AlertResult to display alert "Remove Bookends Header ONLY note cards?" buttons {"No", "Yes"} default button "Yes" giving up after 5
end try

if button returned of AlertResult is "No" then set remove_headers to false


--Request file name and path from user
set target_file_name to "BE-to-OPML.opml"

try
	set data_file to choose file name with prompt "Please select destination file and folder." default name target_file_name default location (path to desktop folder)
	
on error errMsg number errNum
	if errNum = -128 then
		display dialog "User Cancelled Get File - Goodbye!"
		return false
	else
		display dialog "Try to Close Access to:" & data_file & "Error:" & errNum & return & errMsg
		close access data_file
		return false
	end if
	
end try

--Test if .opml extension was provided, if not then append to file name Actually just force .opml extension
try
	set clean_name to data_file as string
	if clean_name contains "." then
		set file_name to text 1 thru ((offset of "." in clean_name) - 1) of clean_name
		set clean_name to file_name & ".opml"
	else
		set clean_name to clean_name & ".opml"
	end if
	set data_file to clean_name
	
on error errMsg number errNum
	display dialog "Error with validaing file extension" & return & "Data Name: " & data_file & return & "Clean name: " & clean_name
	return false
end try

--Write OPML Version, Headers, and Open Body statements to File---------------
my write_to_file(xml_version, data_file, false)
my write_to_file(opml_version, data_file, true)
my write_to_file(opml_header, data_file, true)
my write_to_file(opml_title, data_file, true)
my write_to_file(opml_date, data_file, true)
my write_to_file(opml_header_close, data_file, true)
my write_to_file(opml_body, data_file, true)


--Interaction with Bookends-----------------------------------------------------------
tell application "Bookends"
	activate
	
	--get ids for selected bookends references 
	set selected_ids to «event ToySRUID» "Selection"
	
	--test to make sure user selected Bookends references
	if selected_ids is "" then
		display dialog "No Boookends References were selected." & return & return & "Please select 1 or more references and restrart"
		return false
	end if
	
	set selected_ids to words of selected_ids
	
	--get the cite_keys from Bookends
	set the clipboard to ""
	tell application "System Events" to keystroke "y" using command down
	delay 0.05
	set be_cite_key to the clipboard
	
	--strip citation key brackets
	set cite_length to length of be_cite_key
	set cite_list to text 2 thru (cite_length - 1) of be_cite_key
	set AppleScript's text item delimiters to be_cite_delimiter
	set cite_keys to text items of cite_list
	set AppleScript's text item delimiters to tid
	
	--Process Handiling For Each Reference------------------------------------------
	--get citation, quotes, comments, and tags for of the selected references
	repeat with i from 1 to length of selected_ids
		
		--set variables and counters
		set ref_id to item i of selected_ids
		set ref_nbr to ref_id
		set nbr_references to nbr_references + 1
		
		--For each reference build the top level OPML outline 
		--with the Author, Date, Bookends Reference Number, Title, Abstract, etc
		
		--Author and Date Processing
		--extract author and dates from citation key
		set cite_key_item to text item i of cite_keys
		set AppleScript's text item delimiters to comma
		set cite_key_components to text items of cite_key_item
		set AppleScript's text item delimiters to tid
		set key_component_count to count of items in cite_key_components
		
		set no_date to false
		set ref_date to «event ToySRFLD» ref_id given string:"thedate"
		if ref_date is "" then set no_date to true
		
		if key_component_count = 3 then
			set ref_author to first item of cite_key_components
			set ref_date to second item of cite_key_components
		end if
		
		if key_component_count = 2 then
			if no_date then
				set ref_author to first item of cite_key_components
				set ref_date to " Undated"
			else
				set ref_author to "No Author(s)"
				set ref_date to first item of cite_key_components
			end if
		end if
		
		if key_component_count = 1 then
			set ref_author to "No Author(s)"
			set ref_date to " Undated"
		end if
		
		--strip leading blank space from cite key, author and date		
		if first character of ref_author is space then set ref_author to characters 2 thru -1 of ref_author
		if first character of ref_date is space then set ref_date to characters 2 thru -1 of ref_date
		if first character of cite_key_item is space then set cite_key_item to characters 2 thru -1 of cite_key_item
		
		set ref_author to my replace_bad_characters(ref_author)
		
		--Index Card Processing (Title, Journal, etc)
		
		--Reference Title
		set ref_title to «event ToySRFLD» ref_id given string:"title"
		if ref_title is "" then
			set ref_title to "No Title"
		else
			set ref_title to my replace_bad_characters(ref_title)
		end if
		
		--Reference Type (Journal, Book, etc)
		--set ref_type to "undefined type"
		
		set ref_type to «event ToySGUID» ref_id given string:"RIS"
		if ref_type is "" then
			set ref_type to "undefined type"
		else
			set ref_type to second word of ref_type as text
			set ref_type to my replace_bad_characters(ref_type)
		end if
		
		--Reference Publisher
		set ref_publisher to «event ToySRFLD» ref_id given string:"publisher"
		if ref_publisher is "" then
			set ref_publisher to "undefined publisher"
		else
			set ref_publisher to my replace_bad_characters(ref_publisher)
		end if
		
		--Reference Abstract
		set ref_abstract to «event ToySRFLD» ref_id given string:"abstract"
		if ref_abstract is "" then
			set ref_abstract to "Abstract not provided"
		else
			set ref_abstract to my replace_bad_characters(ref_abstract)
		end if
		
		--Form the Reference Card OPML Outline Statement
		set ref_text to opml_outline & ref_author & ", " & ref_date & " (ID:" & ref_nbr & ")\"" & ¬
			opml_notes & ref_title & return & ¬
			ref_author & return & ¬
			ref_date & "   " & ref_type & " - " & ref_publisher & return & ¬
			"------" & return & ¬
			"Abstract: " & return & ref_abstract & "\">" & return as text
		
		my write_to_file(ref_text, data_file, true)
		
		--Process Handiling For Each Note within Each Reference------------		
		--get notes from this reference
		set ref_notes to «event ToySRFLD» ref_id given string:"notes"
		
		--extract each note and create separate note card (subordinate outline)		
		set AppleScript's text item delimiters to note_delimiter
		set ref_notes to text items of ref_notes
		set AppleScript's text item delimiters to tid
		
		repeat with p from 1 to length of ref_notes
			--reset variables
			set header_only to false
			set keywords to false
			set quotes to false
			set ref_note_header to " "
			set ref_page_nbr to "##"
			set ref_note_title to " "
			set ref_note_text to " "
			set ref_tags to " "
			set ref_quote to " "
			set ref_note_quote to " "
			set ref_note_comment to ""
			
			-- ref_note_item is an individual note within the note stream
			set ref_note_item to item p of ref_notes
			
			--parse note_item for Bookends header and page number
			if ref_note_item is not "" then
				
				set ref_note_list to paragraphs of ref_note_item
				
				--test and process headers	
				if first character in ref_note_item is be_header_delimiter then
					--header
					set ref_note_header to first paragraph in ref_note_item
					--determine if header only and set note contents to rest of notes
					if (count of ref_note_list) > 1 then
						set ref_note_text to text (second paragraph of ref_note_item) thru -1 of ref_note_item
					else
						set ref_note_text to "Header Only"
						if remove_headers is true then set header_only to true
					end if
					--set header, but first test if header also includes page number
					if second character in ref_note_item is be_page_nbr_delimiter then
						--with page number
						set ref_page_nbr to "@" & first word of ref_note_item
						set ref_note_header to text (second word of ref_note_header) thru -1 of ref_note_header
					else
						--without page number
						set ref_note_header to text (first word of ref_note_header) thru -1 of ref_note_header
					end if
				else
					--no header, set title to untitled note and set contents to all of note
					set ref_note_header to "Untitled Note"
					set ref_note_text to ref_note_item
				end if
				
				--test for page numbers		
				if first character in ref_note_item is be_page_nbr_delimiter then
					set ref_page_nbr to "@" & first word of ref_note_item
					set ref_note_text to text (second word of ref_note_item) thru -1 of ref_note_item
				end if
				
				--form the note card title
				set ref_note_title to ref_page_nbr & " - " & ref_note_header
				
				--test note title for well formed contents (bad opml characters)
				set ref_note_title to my replace_bad_characters(ref_note_title)
				
				--Process Handling For Each Line (Paragraph) within Each Note
				--extract each line (paragraph) of the note		
				set AppleScript's text item delimiters to (ASCII character 10)
				set ref_note_text to text items of ref_note_text
				set AppleScript's text item delimiters to tid
				
				repeat with n from 1 to length of ref_note_text
					
					--get and process each paragraph (segement) of the note
					set ref_note_body to item n of ref_note_text
					
					--test notes for well formed contents (ie. no images)
					if (open_image_tag is in ref_note_body) then
						set image_start to (offset of open_image_tag in ref_note_body)
						set image_end to (offset of end_image_tag in ref_note_body) + (length of end_image_tag) - 1
						set first_half to text 1 thru image_start of ref_note_body
						set last_half to text from image_end to -1 of ref_note_body
						set ref_note_body to first_half & " -- Graphics Image Removed --" & last_half
					end if
					
					--test notes for well formed contents (bad opml characters)
					set ref_note_body to my replace_bad_characters(ref_note_body)
					
					if ref_note_body is not "" then
						
						--clear temp vars
						set temp_tag to ""
						set temp_quote to ""
						set no_comment_flag to false
						
						--test for tags
						if first character in ref_note_body is be_tag_delimiter then
							set temp_tags to ref_note_body
							set AppleScript's text item delimiters to be_tag_delimiter
							set temp_tags to text items of temp_tags
							set AppleScript's text item delimiters to space
							set temp_tags to temp_tags as text
							set AppleScript's text item delimiters to tid
							set ref_tags to ref_tags & temp_tags
							set no_comment_flag to true
							set keywords to true
						end if
						
						
						--test for bookends quotes				
						if first character in ref_note_body is be_quote_delimiter then
							set temp_quote to ref_note_body
							set temp_quote to text (second character of temp_quote) thru -1 of temp_quote
							set ref_note_quote to ref_quote & temp_quote
							set no_comment_flag to true
							set quotes to true
						end if
						
						--form note comments
						if no_comment_flag is false then ¬
							set ref_note_comment to ref_note_comment & ref_note_body & return
						
					end if
					
				end repeat
				
				--form the note card OPML statements
				set ref_key to "Keywords: " & ref_tags & return & return
				set ref_quote to "Quote(s): " & return & ref_note_quote & return & return
				set ref_comment to "Comments: " & return & ref_note_comment & return
				if ref_page_nbr is "##" then
					set ref_cite_key to "Citation Key: {" & cite_key_item & "}" & return
				else
					set ref_cite_key to "Citation Key: {" & cite_key_item & ref_page_nbr & "}" & return
				end if
				
				set note_card to "" as text
				if keywords is true then set note_card to note_card & ref_key
				if quotes is true then set note_card to note_card & ref_quote
				set note_card to note_card & ref_comment & "----------" & return & ref_cite_key
				
				set ref_note_contents to tab & opml_outline & ref_note_title & "\"" & opml_notes & note_card & "\"/>" & return as text
				
				--write contents of note
				if header_only is false then
					my write_to_file(ref_note_contents, data_file, true)
					set nbr_notes to nbr_notes + 1
				end if
			end if
		end repeat
		my write_to_file(opml_outline_close, data_file, true)
	end repeat
end tell

my write_to_file(opml_body_close, data_file, true)
my write_to_file(opml_close, data_file, true)

display notification "Complete" & return & return & "Exported " & nbr_references & " References and " & nbr_notes & " Notes"
return true

--end of script **************************************************

--subroutine area ************************************************
--write_to_this_file subroutine
on write_to_file(this_data, target_file, append_data)
	try
		set target_file to target_file as string
		set open_target_file to open for access target_file with write permission
		if append_data is false then set eof of the open_target_file to 0
		write this_data as «class utf8» to open_target_file starting at eof
		close access open_target_file
		return true
		
	on error errMsg number errNum
		if errNum = -49 then
			close access target_file
			set open_target_file to open for access target_file with write permission
			if append_data is false then set eof of the open_target_file to 0
			write this_data as «class utf8» to open_target_file starting at eof
			close access open_target_file
			return true
		else
			display dialog "Write Subroutine Error:" & return & "Target_File: " & target_file & return & "Open_Target_File: " & open_target_file & return & "Error: " & errNum & " - " & errMsg
			close access open_target_file
			return false
		end if
	end try
end write_to_file


--test for well formed contents (bad opml characters) subroutine
on replace_bad_characters(input_string)
	try
		--set arrays for error checking
		set bad_opml_char_array to {"\"", "&"}
		set good_opml_char_array to {"'", "and"}
		set input_string to input_string as string
		set output_string to ""
		set good_char to ""
		
		repeat with x from 1 to count of characters in input_string
			set good_char to character x of input_string
			repeat with y from 1 to length of bad_opml_char_array
				if good_char is equal to item y of bad_opml_char_array then set good_char to item y of good_opml_char_array
			end repeat
			set output_string to output_string & good_char
		end repeat
		return output_string
		
	on error errMsg number errNum
		display dialog "Error replacing bad OPML characters.  Error Number: " & errNum & " - " & errMsg
		return false
	end try
end replace_bad_characters
Cheers,
Dave

dave83
Posts: 38
Joined: Fri Oct 02, 2015 8:55 am

Re: AppleScript to Export Notes in OPML

Post by dave83 » Thu Aug 03, 2017 10:50 am

Based on some feedback from a user on the Literature & Latte forum, I've updated the script to remove the keyboard command (CMD-Y) which caused the unintended consequence of overwriting your Scrivener text. The revised script is provided at the bottom of this reply.

Additionally, there was some concern about just using the default temporary citation format (Author, Date, #Unique ID). I use the default temporary citation format as its the most common. The rationale for including the temporary citation is this allows me to work off my iPad in Scrivener and insert the citation keys by copying them from the notecard into the text. If you use another temporary citation format, then you can still use this default temporary citation format and do a Proofreading Scan (see Bookends User Guide v12.7, pg 225-226) to convert the temporary citation to your preferred temporary citation prior to doing a full Scan.

Code: Select all

--Script to Export Bookends Notes to OPML file v1.0
--Written by Dave Glogowski
--29 July 2017
--
--This script converts Bookends references and associated notes into an OPML structured file which can then be imported into Scrivener's research folder.
--Each reference is a top level card which contains the Title, Author, Date, Type, Publisher, Abstract, and Bookends citation key
--If there are notes associated with a reference, each note creates its own subordinate note card with the Page number (if any), note header, quotes, 
--comments, and keywoards (tags).  This allows you to individually review each comment and change its status (label) within Scrivener.
--
--The script does some error checking as follows:
--  - Strips images from notes
--  - Converts double quotes (") to single quotes (')
--  - Converts ampersand (&) to the word "and"
--
--The script is written very modularly so that it can easily be adapted based on changes in OPML syntax, the need to add additional "bad characters", or 
--changes in the Bookends reference or note delimiters or event calls
--
--
--
--Variable Setup------------------------------------------------------------------------
--Set Counters
set nbr_references to 0
set nbr_notes to 0

--Set Control Variables
set remove_headers to true
set userCanceled to false

--Set Old Text Delimiters
set tid to AppleScript's text item delimiters
set note_delimiter to (ASCII character 10) & (ASCII character 10)
set comma to ","
set date_separators to {"/", " ", ".", "-"}
set digits to {"0", "1", "2", "3", "4", "5", "6", "7", "8", "9"}
set era_first_digit to {"1", "2"}


--set Bookends Delimiters
set be_page_nbr_delimiter to "@"
set be_header_delimiter to "#"
set be_tag_delimiter to "%"
set be_quote_delimiter to ">"
set be_cite_delimiter to ";"

--set image content tags
set open_image_tag to "<iimg>"
set end_image_tag to "</iimg>"

--Set OPML Text variables
set xml_version to "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>" & return
set opml_version to "<opml version=\"1.0\">" & return
set opml_close to "</opml>" & return
set opml_header to tab & "<head>" & return
set opml_title to tab & tab & "<title>Bookends to Scrivener OPML File</title>" & return
set opml_date to tab & tab & "<dateCreated>" & (current date) & "</dateCreated>" & return
set opml_header_close to tab & "</head>" & return
set opml_body to tab & "<body>" & return
set opml_body_close to tab & "</body>" & return
set opml_outline to tab & tab & "<outline text=\""
set opml_outline_close to tab & tab & "</outline>" & return
set opml_notes to "  _note=\""

--File Name and Setup-----------------------------------------------------------------
--Ask user to remove header only Bookends notes
try
	set AlertResult to display alert "Remove Bookends Header ONLY note cards?" buttons {"No", "Yes"} default button "Yes" giving up after 5
end try

if button returned of AlertResult is "No" then set remove_headers to false


--Request file name and path from user
set target_file_name to "BE-to-OPML.opml"

try
	set data_file to choose file name with prompt "Please select destination file and folder." default name target_file_name default location (path to desktop folder)
	
on error errMsg number errNum
	if errNum = -128 then
		display dialog "User Cancelled Get File - Goodbye!"
		return false
	else
		display dialog "Try to Close Access to:" & data_file & "Error:" & errNum & return & errMsg
		close access data_file
		return false
	end if
	
end try

--Test if .opml extension was provided, if not then append to file name Actually just force .opml extension
try
	set clean_name to data_file as string
	if clean_name contains "." then
		set file_name to text 1 thru ((offset of "." in clean_name) - 1) of clean_name
		set clean_name to file_name & ".opml"
	else
		set clean_name to clean_name & ".opml"
	end if
	set data_file to clean_name
	
on error errMsg number errNum
	display dialog "Error with validaing file extension" & return & "Data Name: " & data_file & return & "Clean name: " & clean_name
	return false
end try

--Write OPML Version, Headers, and Open Body statements to File---------------
my write_to_file(xml_version, data_file, false)
my write_to_file(opml_version, data_file, true)
my write_to_file(opml_header, data_file, true)
my write_to_file(opml_title, data_file, true)
my write_to_file(opml_date, data_file, true)
my write_to_file(opml_header_close, data_file, true)
my write_to_file(opml_body, data_file, true)


--Interaction with Bookends-----------------------------------------------------------
tell application "Bookends"
	activate
	
	--get ids for selected bookends references 
	set selected_ids to «event ToySRUID» "Selection"
	
	--test to make sure user selected Bookends references
	if selected_ids is "" then
		display dialog "No Boookends References were selected." & return & return & "Please select 1 or more references and restrart"
		return false
	end if
	
	set selected_ids to words of selected_ids
	
	--Process Handiling For Each Reference------------------------------------------
	--get citation, quotes, comments, and tags for of the selected references
	repeat with i from 1 to length of selected_ids
		
		--set variables and counters
		set ref_id to item i of selected_ids
		set ref_nbr to ref_id
		set nbr_references to nbr_references + 1
		
		--FOR EACH REFERENCE BUILD THE TOP LEVEL OPML OUTLINE 
		
		--Reference Author or Editor		
		set ref_author to «event ToySRFLD» ref_id given string:"authors"
		if ref_author is "" then set ref_author to «event ToySRFLD» ref_id given string:"editors"
		
		if ref_author is "" then
			set ref_title to "No Authors or Editors"
		else
			set AppleScript's text item delimiters to linefeed
			set author_list to every text item of ref_author
			set author_list_count to length of author_list
			set AppleScript's text item delimiters to comma
			set first_author to (first item of author_list as string)
			set remaining_authors to every text item of first_author
			if (last character of first_author) is comma then set first_author to first_author's text 1 thru -2
			if author_list_count = 1 then
				set ref_author to (first item of remaining_authors as string)
			else if author_list_count = 2 then
				set second_author to (second item of author_list as string)
				set final_authors to every text item of second_author
				set ref_author to (first item of remaining_authors as string) & " and " & (first item of final_authors as string)
			else if author_list_count > 2 then
				set ref_author to (first item of remaining_authors as string) & " et al."
			end if
			set ref_author to my replace_bad_characters(ref_author)
		end if
		set AppleScript's text item delimiters to tid
		
		
		--Reference Year		
		set ref_date to «event ToySRFLD» ref_id given string:"thedate"
		if ref_date is "" then
			set ref_date to "Undated"
		else
			set ref_date to my replace_bad_characters(ref_date)
			set got_year to false
			repeat with d in date_separators
				set AppleScript's text item delimiters to d
				set date_list to every text item of ref_date
				set AppleScript's text item delimiters to tid
				repeat with k in date_list
					if length of k is 4 then
						if first character of k is in era_first_digit then
							set ref_year to k
							set got_year to true
						end if
					end if
					if got_year then exit repeat
				end repeat
				if got_year then exit repeat
			end repeat
			if got_year then set ref_date to ref_year
		end if
		
		
		--Reference Title
		set ref_title to «event ToySRFLD» ref_id given string:"title"
		if ref_title is "" then
			set ref_title to "No Title"
		else
			set ref_title to my replace_bad_characters(ref_title)
		end if
		
		
		--Reference Type (Journal, Book, etc)
		set ref_type to «event ToySGUID» ref_id given string:"RIS"
		if ref_type is "" then
			set ref_type to "undefined type"
		else
			set ref_type to second word of ref_type as text
			set ref_type to my replace_bad_characters(ref_type)
		end if
		
		
		--Reference Publisher
		set ref_publisher to «event ToySRFLD» ref_id given string:"publisher"
		if ref_publisher is "" then
			set ref_publisher to "undefined publisher"
		else
			set ref_publisher to my replace_bad_characters(ref_publisher)
		end if
		
		
		--Reference Abstract
		set ref_abstract to «event ToySRFLD» ref_id given string:"abstract"
		if ref_abstract is "" then
			set ref_abstract to "Abstract not provided"
		else
			set ref_abstract to my replace_bad_characters(ref_abstract)
		end if
		
		
		--Build Cite Key
		set cite_key to ref_author & ", " & ref_date & ", #" & ref_nbr
		
		
		--Form the Reference Card OPML Outline Statement
		set ref_text to opml_outline & ref_author & ", " & ref_date & " (ID:" & ref_nbr & ")\"" & ¬
			opml_notes & ref_title & return & ¬
			ref_author & return & ¬
			ref_date & "   " & ref_type & " - " & ref_publisher & return & ¬
			"------" & return & ¬
			"Abstract: " & return & ref_abstract & "\">" & return as text
		
		my write_to_file(ref_text, data_file, true)
		
		--Process Handiling For Each Note within Each Reference------------		
		--get notes from this reference
		set ref_notes to «event ToySRFLD» ref_id given string:"notes"
		
		--extract each note and create separate note card (subordinate outline)		
		set AppleScript's text item delimiters to note_delimiter
		set ref_notes to text items of ref_notes
		set AppleScript's text item delimiters to tid
		
		repeat with p from 1 to length of ref_notes
			--reset variables
			set header_only to false
			set keywords to false
			set quotes to false
			set ref_note_header to " "
			set ref_page_nbr to "##"
			set ref_note_title to " "
			set ref_note_text to " "
			set ref_tags to " "
			set ref_quote to " "
			set ref_note_quote to " "
			set ref_note_comment to ""
			
			-- ref_note_item is an individual note within the note stream
			set ref_note_item to item p of ref_notes
			
			--parse note_item for Bookends header and page number
			if ref_note_item is not "" then
				
				set ref_note_list to paragraphs of ref_note_item
				
				--test and process headers	
				if first character in ref_note_item is be_header_delimiter then
					--header
					set ref_note_header to first paragraph in ref_note_item
					--determine if header only and set note contents to rest of notes
					if (count of ref_note_list) > 1 then
						set ref_note_text to text (second paragraph of ref_note_item) thru -1 of ref_note_item
					else
						set ref_note_text to "Header Only"
						if remove_headers is true then set header_only to true
					end if
					--set header, but first test if header also includes page number
					if second character in ref_note_item is be_page_nbr_delimiter then
						--with page number
						set ref_page_nbr to "@" & first word of ref_note_item
						set ref_note_header to text (second word of ref_note_header) thru -1 of ref_note_header
					else
						--without page number
						set ref_note_header to text (first word of ref_note_header) thru -1 of ref_note_header
					end if
				else
					--no header, set title to untitled note and set contents to all of note
					set ref_note_header to "Untitled Note"
					set ref_note_text to ref_note_item
				end if
				
				--test for page numbers		
				if first character in ref_note_item is be_page_nbr_delimiter then
					set ref_page_nbr to "@" & first word of ref_note_item
					set ref_note_text to text (second word of ref_note_item) thru -1 of ref_note_item
				end if
				
				--form the note card title
				set ref_note_title to ref_page_nbr & " - " & ref_note_header
				
				--test note title for well formed contents (bad opml characters)
				set ref_note_title to my replace_bad_characters(ref_note_title)
				
				--Process Handling For Each Line (Paragraph) within Each Note
				--extract each line (paragraph) of the note		
				set AppleScript's text item delimiters to (ASCII character 10)
				set ref_note_text to text items of ref_note_text
				set AppleScript's text item delimiters to tid
				
				repeat with n from 1 to length of ref_note_text
					
					--get and process each paragraph (segement) of the note
					set ref_note_body to item n of ref_note_text
					
					--test notes for well formed contents (ie. no images)
					if (open_image_tag is in ref_note_body) then
						set image_start to (offset of open_image_tag in ref_note_body)
						set image_end to (offset of end_image_tag in ref_note_body) + (length of end_image_tag) - 1
						set first_half to text 1 thru image_start of ref_note_body
						set last_half to text from image_end to -1 of ref_note_body
						set ref_note_body to first_half & " -- Graphics Image Removed --" & last_half
					end if
					
					--test notes for well formed contents (bad opml characters)
					set ref_note_body to my replace_bad_characters(ref_note_body)
					
					if ref_note_body is not "" then
						
						--clear temp vars
						set temp_tag to ""
						set temp_quote to ""
						set no_comment_flag to false
						
						--test for tags
						if first character in ref_note_body is be_tag_delimiter then
							set temp_tags to ref_note_body
							set AppleScript's text item delimiters to be_tag_delimiter
							set temp_tags to text items of temp_tags
							set AppleScript's text item delimiters to space
							set temp_tags to temp_tags as text
							set AppleScript's text item delimiters to tid
							set ref_tags to ref_tags & temp_tags
							set no_comment_flag to true
							set keywords to true
						end if
						
						
						--test for bookends quotes				
						if first character in ref_note_body is be_quote_delimiter then
							set temp_quote to ref_note_body
							set temp_quote to text (second character of temp_quote) thru -1 of temp_quote
							set ref_note_quote to ref_quote & temp_quote
							set no_comment_flag to true
							set quotes to true
						end if
						
						--form note comments
						if no_comment_flag is false then ¬
							set ref_note_comment to ref_note_comment & ref_note_body & return
						
					end if
					
				end repeat
				
				--form the note card OPML statements
				set ref_key to "Keywords: " & ref_tags & return & return
				set ref_quote to "Quote(s): " & return & ref_note_quote & return & return
				set ref_comment to "Comments: " & return & ref_note_comment & return
				if ref_page_nbr is "##" then
					set ref_cite_key to "Citation Key: {" & cite_key & "}" & return
				else
					set ref_cite_key to "Citation Key: {" & cite_key & ref_page_nbr & "}" & return
				end if
				
				set note_card to "" as text
				if keywords is true then set note_card to note_card & ref_key
				if quotes is true then set note_card to note_card & ref_quote
				set note_card to note_card & ref_comment & "----------" & return & ref_cite_key
				
				set ref_note_contents to tab & opml_outline & ref_note_title & "\"" & opml_notes & note_card & "\"/>" & return as text
				
				--write contents of note
				if header_only is false then
					my write_to_file(ref_note_contents, data_file, true)
					set nbr_notes to nbr_notes + 1
				end if
			end if
		end repeat
		my write_to_file(opml_outline_close, data_file, true)
	end repeat
end tell

my write_to_file(opml_body_close, data_file, true)
my write_to_file(opml_close, data_file, true)

display notification "Complete" & return & return & "Exported " & nbr_references & " References and " & nbr_notes & " Notes"
return true

--end of script **************************************************

--subroutine area ************************************************
--write_to_this_file subroutine
on write_to_file(this_data, target_file, append_data)
	try
		set target_file to target_file as string
		set open_target_file to open for access target_file with write permission
		if append_data is false then set eof of the open_target_file to 0
		write this_data as «class utf8» to open_target_file starting at eof
		close access open_target_file
		return true
		
	on error errMsg number errNum
		if errNum = -49 then
			close access target_file
			set open_target_file to open for access target_file with write permission
			if append_data is false then set eof of the open_target_file to 0
			write this_data as «class utf8» to open_target_file starting at eof
			close access open_target_file
			return true
		else
			display dialog "Write Subroutine Error:" & return & "Target_File: " & target_file & return & "Open_Target_File: " & open_target_file & return & "Error: " & errNum & " - " & errMsg
			close access open_target_file
			return false
		end if
	end try
end write_to_file


--test for well formed contents (bad opml characters) subroutine
on replace_bad_characters(input_string)
	try
		--set arrays for error checking
		set bad_opml_char_array to {"\"", "&"}
		set good_opml_char_array to {"'", "and"}
		set input_string to input_string as string
		set output_string to ""
		set good_char to ""
		
		repeat with x from 1 to count of characters in input_string
			set good_char to character x of input_string
			repeat with y from 1 to length of bad_opml_char_array
				if good_char is equal to item y of bad_opml_char_array then set good_char to item y of good_opml_char_array
			end repeat
			set output_string to output_string & good_char
		end repeat
		return output_string
		
	on error errMsg number errNum
		display dialog "Error replacing bad OPML characters.  Error Number: " & errNum & " - " & errMsg
		return false
	end try
end replace_bad_characters
Cheers,
Dave

iandol
Posts: 299
Joined: Fri Jan 25, 2008 2:31 pm

Re: AppleScript to Export Notes in OPML

Post by iandol » Thu Aug 03, 2017 9:16 pm

Dear Dave, this is now working for me. I'm also completely happy that you use the default temporary citation as part of the content: I misunderstood your first script code and as it was not working for me thought you were using it to extract the first author and year from. I can do my own post processing to transform from {author, year, #id} to Pandoc format easily.

You done a great job on this, thank you!

iandol
Posts: 299
Joined: Fri Jan 25, 2008 2:31 pm

Re: AppleScript to Export Notes in OPML

Post by iandol » Fri Aug 04, 2017 12:58 am

One small update: if you use "&#xA;" (newline in XML) you will get line feeds that formats things better in Scrivener. e.g.

Code: Select all

set ref_text to opml_outline & ref_author & ", " & ref_date & " (ID:" & ref_nbr & ")\"" & ¬
			opml_notes & ref_title & " &#xA;" & return & ¬
			ref_author & " &#xA;" & return & ¬
			ref_date & "   " & ref_type & " - " & ref_publisher & " &#xA;" & return & ¬
			"------" & " &#xA;" & return & ¬
			"Abstract: " & ref_abstract & "\">" & return as text
I've slightly modified this so I get the DOI and a URL, and remove the author and date as it is already in the title.

Also, you have your own keywords system, and do not use the Bookends keywords (line 341)?

AsafKeller
Posts: 64
Joined: Thu Jul 05, 2007 1:07 pm

Re: AppleScript to Export Notes in OPML

Post by AsafKeller » Fri Aug 04, 2017 8:03 am

Thanks much for this, David! I'm trying to figure out how to use PMID instead of reference ID, but I am stumped. Would appreciate a hint! Thanks.

dave83
Posts: 38
Joined: Fri Oct 02, 2015 8:55 am

Re: AppleScript to Export Notes in OPML

Post by dave83 » Fri Aug 04, 2017 9:23 am

iandol,
Also, you have your own keywords system, and do not use the Bookends keywords (line 341)?
No, I use the Bookends keywords. They are embedded in within the notes and preceded with the percent sign (%).

Thanks for the improvements. When you added the URI, does Scrivener recognize it; thus allowing you to link to the reference?
Cheers,
Dave

dave83
Posts: 38
Joined: Fri Oct 02, 2015 8:55 am

Re: AppleScript to Export Notes in OPML

Post by dave83 » Fri Aug 04, 2017 9:43 am

AsafKeller,

I would not use it to replace the Bookends Unique ID (ref_nbr in my script) since this is used to retrieve the precise reference in Bookends and its associated information. Additionally, I would continue to use the ref_nbr in the temporary citation as again this is just a key into the Bookends database during the scans.

Now, if you want to add the PMID to the output (note cards), then the challenge is extracting it from Bookends. Currently the script uses AppleEvents to communicate with Bookends and is limited to extracting the following fields [Bookends Guide v12.7, p25]:
The field names are: authors, title, editors, journal, volume, pages, thedate, publisher, location, url, title2, abstract, notes, user1...user20.
iandol was able to extract the DOI, but I don't have his code to see how this was done. iandol -- can you share? The PMID could possibly be extracted in a similar manner.
Cheers,
Dave

Jon
Site Admin
Posts: 8356
Joined: Tue Jul 13, 2004 6:27 pm
Location: Bethesda, MD
Contact:

Re: AppleScript to Export Notes in OPML

Post by Jon » Fri Aug 04, 2017 9:52 am

The PMID is in, by default, field

user18

The DOI is in field

user17

Jon
Sonny Software

Post Reply