Monday, January 30, 2017

Processing the Selected Text via Script

It seems like it should be easy to run the selected text in any app through a shell script, but I’ve run into a surprising number of issues doing this. I’ve long used ThisService to create system services out of shell scripts that sort lines, smarten quotes, and apply title case. Then I would assign keyboard shortcuts in System Preferences and have easy access to the scripts from any app.

Over the last few months, keyboard shortcuts for system services (and PDF services) have become unreliable. They still show up in System Preferences, but they usually don’t work. Often, System Preferences forgets them. It also forgets which services I’ve enabled and disabled, hiding my favorite scripts and bringing back dozens of commands that I never use. This makes the Services contextual menu unwieldy. The Keyboard preferences pane is awkward to use, so that it takes a long time to go through the list checking and unchecking the appropriate boxes.

My first thought was to use LaunchBar, which I knew could get and replace the selected text (either using a script or a service). However, this takes multiple steps: long Command-Space to load the text, Tab, type the name of the script/service, Command-Shift-C to copy the result text and paste it back. This is fine for occasionally used scripts, where keyboard shortcuts would not be practical, but not for ones I use many times per day.

How about using FastScripts to assign keyboard shortcuts and run the scripts? Unfortunately, most applications are not scriptable enough to access the selected text. I wrote a script to use GUI scripting to invoke my service from the Services menu:

my runServiceNamed("Title Case")

on runServiceNamed(_serviceName)
    tell application "System Events"
        set _frontApp to first application process whose frontmost is true
        tell _frontApp
            set _appMenu to menu 2 of menu bar 1
            set _servicesMenu to menu 1 of menu item "Services" of _appMenu
            set _menuItem to menu item _serviceName of _servicesMenu
            click _menuItem
        end tell
    end tell
end runServiceNamed

However, GUI scripting the menu bar doesn’t seem to work in MarsEdit, one of the apps where I would use the scripts most frequently.

I ended up giving up on services entirely and used GUI scripting and the clipboard to get and set the selected text:

my process("/Users/mjt/Library/Application Support/BBEdit/Text Filters/titlecase.py")

on process(_scriptPath)
    set _savedClipboard to the clipboard

    -- Copy selected text
    tell application "System Events" to keystroke "c" using {command down}
    delay 1 -- Without this, the clipboard may have stale data.

    -- Clipboard has Mac line breaks, but script requires Unix.
    set _script to "pbpaste | tr '\\r' '\\n'  | " & _scriptPath's quoted form & " | pbcopy"
    do shell script _script

    -- Paste to replace selected text
    tell application "System Events" to keystroke "v" using {command down}
    delay 1 -- Without this, may restore clipboard before pasting.

    set the clipboard to _savedClipboard
end process

This is ugly and has some downsides: there are delays necessitated by the GUI scripting, and certain types of clipboard data are not preserved. But I can assign a single keyboard shortcut, which FastScripts won’t forget, and it works reliably.

Note: I originally wrote the script to use the clipboard instead of pbcopy and pbpaste. The latter are better because that way the text doesn’t end up in an AppleScript variable. Getting the text from AppleScript into stdin seems to require using either echo, which is subject to the shell’s command length limit, or writing to a temporary file.

16 Comments RSS · Twitter

Shameless plug: Have a look at iClip. I'm the (not original) author of it and it does work similarly, but with one advantage: You can define your hot keys for each script you want to use to manipulate what you just copied. It would work like this:

You select your text and Copy it, using cmd-C. iClip records the clipboard within half a second. iClip then invokes a special user-editable script which has a handler for the new script. You can now use iClip's scripting commands to get the text of the new clipping, process and update it. Then you can tell iClip to paste the updated clipping back to the App (which is the same as you do with System Events). Or, instead of applying this to every clipping iClip records, you can assign a hot key for executing a custom script that performs the modification and Paste steps, meaning you'd press cmd-C to Copy, then press your custom hot key to perform your operation and have it pasted back.

Let me know privately if you want to try it (I'm also a happy user of SpamSieve, BTW).

Worth using Keyboard Maestro to access the menus if you're having trouble with GUI scripting. My MarsEdit scripts used to do GUI scripting with AppleEvents but I switched a few years back to just keeping the main logic in the script and the control flow in Keyboard Maestro. It was just easier to fix if menus changed or the like.

I recently had the same need and ended up using LaunchBar for it. It's possible for it to work in a much less cumbersome way than the procedure you described. For example, I only have to double tap Command, type one or two characters from the script name I want to run, and hit enter. Here's a couple tips to get it working more efficiently:

- Instead of a long Command-Space to load text, go to LaunchBar's prefs -> shortcuts tab, then choose a keyboard combo for Instant Send. I've set it to Double Command, so that a double tap instantly brings up LaunchBar with my text selection ready to act on and no need to press Tab.

- There's no need to Command-Shift-C to manually copy/paste after running the script. Instead, just end your script with:

tell application "LaunchBar"
    perform action "Copy and Paste" with string outputString
end tell

- I've saved my text processing scripts as LaunchBar actions. Here's one as an example (note, I'm a total AppleScript novice so this is sure to look hacky to experienced eyes). This prefixes the lines of text you've selected with a - in order to format as a Markdown unordered list: https://www.dropbox.com/s/tdkpxdmi0w7n82q/Unordered%20List.lbaction.zip?dl=0

Getting the text from AppleScript into stdin seems to require using either echo, which is subject to the shell’s command length limit, or writing to a temporary file.

AppleScript can read from stdin via Cocoa and AppleScript-ObjC. It's one of many useful basic everyday commands I packaged as a proposed set of "standard libraries" a year ago; see the File library's Shell Script Support suite.

I also tried to rally AS users to file Radar requests for Apple to adopt those libraries for 10.13 (e.g. by duping my original ticket); a few did, a few whined, and most still won't know they exist. But I just don't have time or energy to keep trying to save AppleScript from itself any more. If anyone else wants to run with it they're free to try, though they'll need to bloody hurry. As it is I expect AppleScript's goose is already 100% cooked inside Apple; this and SwiftAutomation were just a chance to keep the current system useful a bit longer, and maybe give Apple reason to think at least some of it could be worth salvaging, or at least learning from, before they blindly toss the whole lot for Shiny New AI Fail.

Shane Stanley

> certain types of clipboard data are not preserved

With a bit more code they can be:

use AppleScript version "2.4"
use scripting additions
use framework "Foundation"
use framework "AppKit"

on process(_scriptPath)
    set oldClip to my fetchStorableClipboard()
    
    -- Copy selected text
    tell application "System Events" to keystroke "c" using {command down}
    delay 1 -- Without this, the clipboard may have stale data.
    
    -- Clipboard has Mac line breaks, but script requires Unix.
    set _script to "pbpaste | tr '\\r' '\\n'  | " & _scriptPath's quoted form & " | pbcopy"
    do shell script _script
    
    -- Paste to replace selected text
    tell application "System Events" to keystroke "v" using {command down}
    delay 1 -- Without this, may restore clipboard before pasting.
    
    -- restore original
    my putOnClipboard:oldClip
end process

on fetchStorableClipboard()
    set aMutableArray to current application's NSMutableArray's array() -- used to store contents
    -- get the pasteboard and then its pasteboard items
    set thePasteboard to current application's NSPasteboard's generalPasteboard()
    -- loop through pasteboard items
    repeat with anItem in thePasteboard's pasteboardItems()
        -- make a new pasteboard item to store existing item's stuff
        set newPBItem to current application's NSPasteboardItem's alloc()'s init()
        -- get the types of data stored on the pasteboard item
        set theTypes to anItem's |types|()
        -- for each type, get the corresponding data and store it all in the new pasteboard item
        repeat with aType in theTypes
            set theData to (anItem's dataForType:aType)'s mutableCopy()
            if theData is not missing value then
                (newPBItem's setData:theData forType:aType)
            end if
        end repeat
        -- add new pasteboard item to array
        (aMutableArray's addObject:newPBItem)
    end repeat
    return aMutableArray
end fetchStorableClipboard

on putOnClipboard:theArray
    -- get pasteboard
    set thePasteboard to current application's NSPasteboard's generalPasteboard()
    -- clear it, then write new contents
    thePasteboard's clearContents()
    thePasteboard's writeObjects:theArray
end putOnClipboard:

@Nigel Looks cool. Thanks. It turns out that LaunchBar also has a paste in frontmost application command in its dictionary.

@has I’m talking about the other way: having “do shell script” write to stdin. I guess I could use ASOC and NSTask…

@Shane Awesome. I keep forgetting that ASOC works from regular scripts now.

BetterTouchTool is very good for attaching various actions to key strokes as well as touch bar gestures.

Another option is Keyboard Maestro. I appreciate all the suggestions for other apps (and am happy to document them here), but for simplicity I am trying to stick with the ones that I already have.

Shane Stanley

@Michael You can also reduce that first delay to a minimum, which might reduce some of the irritation:

on process(_scriptPath)
    set oldClip to my fetchStorableClipboard()
    
    set thePasteboard to current application's NSPasteboard's generalPasteboard()
    set theCount to thePasteboard's changeCount()
    -- Copy selected text
    tell application "System Events" to keystroke "c" using {command down}
    -- Check for changed clipboard
    repeat 20 times
        if thePasteboard's changeCount() is not theCount then exit repeat
        delay 0.1
    end repeat

First off: I’m one of the developers of LaunchBar. Thanks to Nigel to point out Instant Send and the other tips!

In addition to what Nigel wrote before, there’s another way you could speed this up. Whenever you have a text item selected in LaunchBar, you can press ⇧↩ to paste that text in the frontmost application.
Even better, if you have a text item selected and send that to a text processing action (by using Instant Send as you suggested or Send To by pressing ⇥), that ⇧↩ executes that action first, then pastes the result in the frontmost application.

An example to make things clear:
- Select some text in any app
- Hold ⌘Space or use your shortcut for Instant Send (I use a single Option tap)
- Type an abbreviation for the text conversion option you want, e.g. CTT for Convert to Titlecase
- Hit ⇧↩

Note that the text conversion action can be anything you want that returns some text, e.g. the built-in actions, but also simple scripts or full LaunchBar Actions written in whatever scripting language you want. So if you find a need for something like that you can write an Action in JavaScript with just this:

function run(argument) {
    return argument + " Foo!";
}

@Michael: Derp. Gotcha.

NSTask is problematic as it now uses closures to notify completion, and ASOC never added support for those. (That's why I didn't do a library for that.) But if you don't care about output, or are willing to trust to the old deprecated ObjC 1.x APIs, or are in a position where you can use a custom ObjC wrapper class as a shim between the new APIs and ASOC then it's certainly an option for passing data from AS to stdin.

@Marco Thanks. It sounds like that’s the same number of steps that I had, but with ⇧↩ instead of Command-Shift-C. Right now I’m experimenting with using a single tap for Instant Send and scripts that automatically insert their results. I like it so far.

@has I guess you’re talking about the NSFileHandle APIs rather than NSTask itself. I’m still using the notifications in my Objective-C code because I heard from multiple people that the closure APIs were buggy.

@Michael thanks for the paste in frontmost application tip. Much nicer not having my scripts overwrite the clipboard.

@Marco great to know about ⇧↩ !

The alternative, much better interface to selecting services was Services Manager (http://www.macosxautomation.com/services/servicesmanager/). Unfortunately it's garbage collected so it breaks on 10.12, and given Sal's no longer at Apple I don't expect it to ever be updated.

Unfortunately I'm also seeing that service selections no longer "stick" in 10.12. Bugs from prior OS versions, e.g., having to pull down the Services menu the first time you use an app before the keyboard equivalent activates, have not been fixed either.

[…] Custom keyboard shortcuts don’t work and/or disappear. […]

[…] Michael Tsai – Blog – Processing the Selected Text via Script […]

Leave a Comment