Speak Instapaper Posts — Part 2

In the last part we built a useful workflow that would open a given number of unread article from your Instapaper feed. But we stopped short of the goal, to convert the text of the articles to speech files.

If you look into the library of Automator actions there is one with the promising name “Get Text from Webpage.” However this will extract all the text, usually including all the menus, ads and all the other detritus that clutters webpages these days. The latest version of Safari (( Safari 5, as I write this )) has a functionality called “Reader,” which removes all this clutter and allows the user to focus on just the text. Unfortunately, the “Reader” functionality in Safari is not scriptable.

But before Safari had “Reader” there was the Readability javascriptlet from Arclab90 which does very much the same thing. Since Safari’s AppleScript dictionary allows us to execute arbitrary JavaScript against a webpage, we can use that to extract the relevant text from the article. That saves us from having to recreate the logic of the Readabilty scriptlet in AppleScript.

Do the following with the workflow we built in Part 1:

  • duplicate the Workflow file and name the copy: Speak Instapaper Articles to iTunes
  • remove the last action “New Safari Documents” from the workflow (( there is a bug in Safari’s AppleScript implementation where document references from freshly created web documents will go stale once the page is loaded. This also affects the “New Safari Documents” action. We will work around this bug in our AppleScript))
  • add a new empty “Run AppleScript” action at the end of the workflow and enter the following code:
on run {input, parameters}
	-- uses the 'Readability' javascript from
	-- http://lab.arc90.com/experiments/readability/
	set readabilityScript to "javascript:(function(){readConvertLinksToFootnotes=false;readStyle='style-newspaper';readSize='size-medium';readMargin='margin-medium';_readability_script=document.createElement('script');_readability_script.type='text/javascript';_readability_script.src='http://lab.arc90.com/experiments/readability/js/readability.js?x='+(Math.random());document.documentElement.appendChild(_readability_script);_readability_css=document.createElement('link');_readability_css.rel='stylesheet';_readability_css.href='http://lab.arc90.com/experiments/readability/css/readability.css';_readability_css.type='text/css';_readability_css.media='all';document.documentElement.appendChild(_readability_css);_readability_print_css=document.createElement('link');_readability_print_css.rel='stylesheet';_readability_print_css.href='http://lab.arc90.com/experiments/readability/css/readability-print.css';_readability_print_css.media='print';_readability_print_css.type='text/css';document.getElementsByTagName('head')[0].appendChild(_readability_print_css);})();"
	set output to {}
	tell application "Safari"
		repeat with x in input
			set theURL to contents of x
			make new document with properties {URL:theURL}
			delay 0.5
			repeat until ( (do JavaScript "document.readyState;" in document of window 1) is equal to "complete")
				delay 0.5
			end repeat
			set d to document of window 1
			do JavaScript readabilityScript in d
			delay 3
			repeat until ( (do JavaScript "document.readyState;" in d) is equal to "complete")
				delay 1
			end repeat
			set thetext to text of d
			-- remove first three and last four paragraphs since these are Readability links
			set AppleScript's text item delimiters to return
			set thetext to (paragraphs 4 through -5 of thetext) as text
			close d
			set output to output & {thetext}
		end repeat
	end tell
	return output
end run

Let’s slowly go through this code:

  • first we setup variable to store the Readabilty javascript code.
  • then we initialize a list output to store the results.
  • then we loop through the items that were passed into the action in the input variable. In this case the items are the URLs of the Instapaper posts.
  • set theURL to contents of x
    this de-references the iterator variable. Due to some oddities of the AppleScript language this is usually a wise thing to do in a repeat loop.
  • make new document with properties {URL:theURL}
    delay 0.5

    we tell Safari to open a new document with the given URL and pause for a while to let Safari start loading

  • repeat until ( (do JavaScript "document.readyState;" in document of window 1) is equal to "complete")
        delay 0.5
    end repeat
    set d to document of window 1

    We have to wait until the page is completely loaded before we can apply the Readability script against the page. Unfortunately Safari does not expose the state of the page (loading or complete) to AppleScript. This is however exposed to the JavaScript DOM within the page and we can access DOM information from AppleScript with the do Javascript event. So we poll the document.readyState attribute in Javascript until it reports complete. Then we remember a reference to this document in a variable. ((Safari has a bug where a AppleScript reference to document will change while it is loading, resulting in broken references. All this is a clumsy, but effective workaround.))

  • now we can execute the Readability script against the page:
    do JavaScript readabilityScript in d
    delay 3
    repeat until ( (do JavaScript "document.readyState;" in d) is equal to "complete")
    	delay 1
    end repeat

    We use the same DOM trick to wait until Safari is done.

  • Now the text property of the document contains the cleaned up text of the article. We can extract that, remove some extra lines that Readabilty inserts, close the Safari window and append the text as its own element to the output list.
    set thetext to text of d
    -- remove first three and last four paragraphs since these are Readability links
    set AppleScript's text item delimiters to return
    set thetext to (paragraphs 4 through -5 of thetext) as text
    close d
    set output to output & {thetext}

This would be a good time to save the workflow, and do a test run. You can show the results of the workflow in Automator to see if the text is extracted properly. Readability is not perfect and does not work on all pages, but the success rate is quite high.

The remaining work of converting the text into audio is very straightforward. Add the following workflow actions:

  • Text to Audio File
  • Import File into iTunes
  • Add Songs to Playlist ((You want to create a specific playlist for these files in iTunes))

And then you are done. You can also download the complete Workflow.

Start Screen Sharing or Apple Remote Desktop Sessions with Spotlight

Screen Sharing is a really useful tool in Mac OS X. Most people use it locally and select the Computer from the Sharing area in the Finder sidebar. You can also connect Screen Sharing to a remote host. In Finder select “Connect to Server” from the “Go” menu and enter


which will connect Screen Sharing to the address. ((It will use VNC on TCP port 5900 in case you have connection issues.))

You could add the vnc URI to the favorites in the “Connect to Server” dialog, but there is a better way: Screen Sharing remembers the last connections in ~/Library/Application Support/Screen Sharing/. There you will find the hosts you have connected to as .vncloc files. Find the host(s) you use most frequently and copy them to the Desktop or your Documents folder. ((anywhere Spotlight will index)) Then rename them to just the hostname or another descriptor. You can now double-click to initiate the Screen Sharing connection. But even better: you can invoke Spotlight, start typing the hostname and the vncloc file should be right there. No matter what you are doing the remote session is just a few keystrokes away.

However, if you prefer to use Apple Remote Desktop over Screen Sharing, this will not work. ARD does not open vncloc files. However, ARD is scriptable, so we can build a workaround. Even better ARD supports Automator, so we don’t even need to write code.

  • open Automator. From the template chooser, select “Application.”
  • add the action “Choose Remote Computer” and select a computer to connect to. ((The computer has to be already known to the local Remote Desktop to appear in this list.))
  • add the “Observe Computers” action next. ((curiously enough there is no “Control Computers” action, but switching from observe to control is only a single click. This may be so you cannot accidentally invoke a remote control session.))
  • you’re done. Save this Automator applet and give it the name of the computer.
  • you can start this applet by starting to type the computer’s name in Spotlight and you will a remote observe session in Remote Desktop.

Create more applets for each host you frequently use. if you select multiple computers in the first action, you will get the nice “multi observe” window in Remote Desktop. Or you can replace the “Choose Remote Computers” action with a “Choose Computer Lists” action.

Speak Instapaper Posts — Part 1

I saw this in my Twitter stream the other day:

You know what I want? A text-to-speech plugin for @instapaper so, while commuting to/from work, I can listen to the stuff I find at work.

— @seankaiser

That shouldn’t be too hard, shouldn’t it?

First we have to get the unread articles from Instapaper. If you go instapaper, log in, and go to your unread articles, you can see the RSS button in the URL field in Safari. To get to the RSS feed in Automator, do the following:

  • Open Automator, create a new Workflow
  • Open www.instapaper.com/u in Safari and drag the link from the URL bar to the empty Automator Workflow window. Automator will create an new “Get Specified URLs” action with the Instapaper URL unread in it
  • Add a “Get Feeds from URLs” action next
  • Add a “Get Link URLs from Articles” action. Unselect the “only in the same domain” option
  • Finally add the “New Safari Documents” action.
  • The workflow already does something useful. Save as “Open unread Instapaper articles”

If you ares anything like me this workflow will open quite a large number of pages. I think Instapaper limits the RSS feed to 25. That’s still a lot of new Safari tabs/windows you are opening there. We want to add an action that restricts the number of items passed through it. Surprisingly there is none in the default actions, but this is fairly easy to add. Insert a new “Run AppleScript” action before the “New Safari Documents” action and replace the default code with the following:

on run {input, parameters}
	set maxNum to 3
	-- filters all but the first maxNum items from the articles, change as appropriate
	-- enter '-1' or remove this action entirely to get all urls
	if (count of input) > maxNum then
		set output to items 1 through maxNum of input
		set output to input
	end if
	return output
end run

This will only pass through the first maxNum of items passed into it, regardless of type. You can change maxNum to fit your taste and/or needs. You can also set maxNum to -1 if you want to pass all items without removing the AppleScript action.

Save again and try running it. The next step will be to filter the actual text out of the web page which will be a little tougher and the main topic of Part 2.

Enable Some Extra Services

It’s Thanksgiving here in the US. To keep you happy with minimal effort on my side I’ll give you a whole bunch of services to explore, without me (or you) having to write any of them.

Open System Preferences, select the Keyboard preference pane, select the Keyboard Shortcuts Pane and then from the list on the left select “Services.”

There you will find a long list of pre-installed services many of which are disabled by default. Go through the list and enable those that sound promising. “Get Result of AppleScript” and “Add to iTunes as Spoken Track” are two of my favorites.

Any services you have built yourself will also appear in this list. You can disable or re-enable them to keep your context menu trim.

This pane is also where you assign or change keyboard shortcuts. So if there is a service that you use frequently you can further optimize your workflow with a keystroke.

On man Pages

Note: update for macOS Ventura

So you’re writing this email explaining to a customer or colleague on how to do some really cool thing (say hide a file in the Finder) in Terminal. The command for that is chflags, but of course you can’t remember the exact syntax. So you open Terminal and write man chflags and find the correct options.1

However reading longer man pages (try ssh or bash) in the Terminal can be kind of painful. I’m sure some of you have encountered this command before:

man -t chflags | open -f -a "Preview"

which uses the -t flag to pass the output to groff and generate a postscript file which we then pipe into the Preview app, using open‘s -f option to pipe the stdin into a file to open in a GUI app. Preview will then convert the postscript to PDF and display the result.

I think this started to work in Tiger and you should immediately go and add this command to your shell’s profile.2 Which is nice but you still have to make the roundtrip to the Terminal.3

Enter Snow Leopard Automator Services. Open Automator. Create a new service. Leave the settings to work on ‘text’ in ‘any application’. Search for the ‘Run Shell Script’ action and double click to add to the workflow. Leave the Shell at ‘/bin/bash’ but set the ‘Pass Input’ option to ‘as arguments.’

Replace the default code with

man -t "$1" | open -f -a /Applications/Preview.app
Screenshot of the workflow in Automator

Save the Service and give it a nice name, such as “Open Man Page.”

Then in any application4 you can ctrl/right/double-finger click on a word and “Open Man Page” will be an option in the menu.5 You can even go to System Preferences -> Keyboard -> and add a keyboard shortcut to the command.6 If any other command in the man page strikes your curiosity, just ctrl/right/double-finger click the word in Preview and select “Open Man Page” again.

Another rarely known but quite useful trick is that you can create hyperlinks to man pages with the x-man-page://command URL.7 This will open the man page in man in a new Terminal window. This is especially useful in IM sessions.

  1. chflags [no]hidden /path/to/file
  2. for me the relevant line looks like: function preman() { man -t "$@" | open -f -a "Preview" ;}
  3. though you could save the resulting PDF and attach it to the email, especially if you have a customer who can’t find his or her way around the command line.
  4. well… most…
  5. it may be under a under an extra level “Services” Menu. Some applications do not show Services in the context menu, but all will show the “Services” Menu under the Application Menu
  6. I usually don’t bother since I immediately forget any new keystrokes I assign. Your memory may work better.
  7. I once had somebody freak out on me because I made him open Terminal on his Mac with this…