-dealloc

Keyboard Adventures

2012-04-26T00:00:00-04:00

The best utility for making custom keyboard layouts on Mac OS X has long been Ukelele (sic). Sure, Apple came up with an XML format for keyboard layouts, but when you want a variant of an existing keyboard, it’s a lot easier to just copy the existing keyboard in Ukelele and modify the keys you want to change.

A few years ago I took a class on Phonetics and Phonology, and found myself needing to type in IPA quite often. So I fired up Ukelele and made a copy of the standard Dvorak keyboard and got to work. I didn’t change any of the regular keys, but I added a number of IPA characters for when you hold Option, and turned my numpad into an approximation of the human vowel space.

(mouse over if you’ve forgotten what regular Dvorak looks like)

What are all those orange keys? Those are dead keys, usually used to make accented characters and co-opted by me for typing characters linguistically similar to the first one.¹ Ukelele can handle these as well.

To use your custom keyboard, just save the keylayout file from Ukelele in a “Keyboard Layouts” folder inside your Library folder. You can even give your keyboard an icon for the menu bar switcher by putting a 16x16 icon file in the folder, as long as it’s in ICNS format and has the same name as the layout file.

All that was good enough for me until Mac OS X Lion came around.

Press and Hold

When Lion came out, there was a new feature, clearly inspired by iOS: the ability to hold a key for a bit and get a popover of variants. This generated quite a bit of outcry from a number of people who apparently need to type repeated letters a lot. I, on the other hand, think it’s a pretty cool feature.

It’d be really nice to have configurable press-and-hold options in the same way that we have configurable keyboard layouts with configurable dead keys, right? Having taken a year of Mandarin Chinese, it might be useful if, say, the first four variants of “e” corresponded to the four tones in Mandarin. But there’s nowhere in the keylayout spec that includes this feature.

Fortunately, someone’s already solved this problem. For maybe 95% of the world, Lri’s solution is good enough: it works, and it’s fairly extensible. There was just one problem: my custom Dvorak IPA keyboard layout wasn’t getting the press-and-hold popover at all.

The Text Input System

The OS X text input system is a complicated beast, but then again text input is pretty complicated in and of itself. Keyboard layouts are well and good for many alphabetic languages, but for something like Japanese or Chinese you need a bit more power. For this, there are programs called input methods, which handle, say, the translation from Latin characters to Chinese ones with the corresponding Mandarin pronunciation.²

The press-and-hold feature is implemented as an input method that listens for held-down key events. Unlike the Japanese or Chinese input methods, the press-and-hold feature is not exclusive, and does not appear in the input menu. Other than that, though, the text input system treats it like any other input method, and in theory you could write my own input method that did exactly the same thing. (More on this later…)

Underneath the nice interface for writing input methods is the Text Input Source Services layer. This is a CoreFoundation-compatible C API for the text input system — essentially a programmatic interface to the input menu. That means it has to handle both the complex input methods like Japanese and the humble keyboard layouts for QWERTY, Dvorak, and my “Dvorak IPA”.

In order to bring the keylayout files in line with the way OS X plugins are structured, and more importantly to make them localizable, there’s a second way to provide keyboard layouts to the text input system: bundles containing keylayout files. This is also described alongside the XML format, and shows a pretty standard bundle layout. The main benefit is being able to provide a strings file for the localized keyboard names, but there’s also the added effects of having the icon and the keyboard layout in a single package, and being able to distribute multiple keyboard layouts together if you want.

None of that was going to solve my problem, though. It wasn’t until I looked in the actual header file for the Text Input Source Services that I discovered there was more to writing keyboard layouts, or for that matter input methods, than existed in Apple’s online documentation. Specifically:

For Leopard, there are two new keys for use in plists to provide information that supports the Text Input Sources functions above (these keys will be ignored in earlier systems):

"TISInputSourceID" - a key to specify the InputSourceID, a reverse-DNS-style string meant to uniquely identify any input source. If this key is not specified, the Text Input Sources functions will attempt to construct an InputSourceID from other information.

"TISIntendedLanguage" - a key to specify the primary language which the input source is intended to input. If there is none - as with the Unicode Hex Input key layout, for example - this key need not be specified. The language is indicated by a string in in the format described by BCP 47 (the successor to RFC 3066).

In the Info.plist file, the value for the CFBundleIdentifier key must be a string that includes ".keyboardlayout."; typically this might be something like "com.companyname.keyboardlayout.MyKeyboardLayouts". (Before Leopard, it was required to be a string that began "com.apple.keyboardlayout", even for keyboard layouts not supplied by Apple).

A dictionary of properties for each key layout in the bundle should be provided using a key of the form "KLInfo_keylayoutname" (even if keylayoutname includes spaces or punctuation). This dictionary is where to specify the keys "TISInputSourceID" and "TISIntendedLanguage" and their associated values.

"TISInputSourceID" note: For keyboard layouts this should typically be something like "com.companyname.keylayout.keylayoutname``”. If this key is not specified, an InputSourceID will be constructed by combining _bundleID_ + “.keylayout.”` + keylayoutname.

If the keyboard layouts in the above example were intended to input Azerbaijani in Latin script, then the Info.plist entries could be:
<key>KLInfo_MyLayoutOne</key>
<dict>
  <key>TISInputSourceID</key>
  <string>com.companyname.keylayout.MyLayoutOne</string>
  <key>TISIntendedLanguage</key>
  <string>az-Latn</string>
</dict>
<key>KLInfo_MyLayoutTwo</key>
<dict>
  <key>TISInputSourceID</key>
  <string>com.companyname.keylayout.MyLayoutTwo</string>
  <key>TISIntendedLanguage</key>
  <string>az-Latn</string>
</dict>

Cocoa programmers will probably be smirking at the Text Input Source Services’ abuse of Info.plist: it’s not that this information shouldn’t be here, but that it shouldn’t use *prefixed* keys. The usual way to do this is an array of dictionaries with a "name" key of some kind. Also, the keyboard names are localized in InfoPlist.strings, which is supposed to only be for data in Info.plist.

There is more information on how input methods use these keys, but the main point is that this information isn’t available anywhere online. And though Ukelele now has an option to save in bundle format, it doesn’t have a way to set the magic language key — which, it turns out, is the key to getting the press-and-hold feature to work with custom keyboard layouts. By manually editing the Info.plist file and specifying that my keyboard had an intended language of "en", then logging out and back in, I got exactly what I wanted: press-and-hold accents in Dvorak IPA mode.

I’m planning to send an e-mail to the developer of Ukelele to add this feature, and I’ve filed rdar://11327265 with Apple about the lack of documentation here. Meanwhile, I’ve copied the relevant parts of the headers here in the hopes that it’ll show up on [search engine] next time someone has this problem.

Custom Press and Hold

So, this ought to be enough, right? I managed to get the accents popover to show up for my custom keyboard layout, and I know how to change the set of variants. If I was just interested in that, I’d be done, right?

…Except I don’t really like messing with system files, especially not ones in the /System folder that you’re really not supposed to touch. Because system apps and plugins are signed now, modifying resources inside PressAndHold.app could disable the feature altogether. (Clearly it doesn’t right now, but that could change.) On a more practical note, any system update that touches PressAndHold.app could overwrite your changes, which is at least a bit annoying.

I said before that it’s possible to write your own input method that does everything PressAndHold.app does. But…

Problem:

Some applications do about 90% of what I want.

Solution:

Develop my own applications.

Better Solution:

Patch the application myself…

That comes from the developer of SIMBL, a program specifically designed for injecting code into existing OS X applications. While this can be a HUGE security risk, it’s also what allows me to build a plugin like Keystone that runs inside Safari. In this case, I decided to make a plugin that ran inside PressAndHold.app. In a fit of grandeur I called it MagicPressAndHold.

The actual implementation of that plug-in wasn’t so hard. Using the class-dump utility (and, uh, breaking Apple’s Terms of Service a bit), I found that there’s a single method that serves as the entry point for accented character suggestions: -accentsStringForString:language:. So I just had to inject my own implementation of that method to gain control over the set of variant characters.

I ran into a slight snag because PressAndHold.app doesn’t run like a normal Mac OS X application. Instead, it’s launched by the text input system as necessary and doesn’t show up in NSWorkspace’s usual “application launched” notifications. That means SIMBL doesn’t get a chance to look at it. Manually sending SIMBL’s “please load me” AppleEvent to PressAndHold.app did the trick.

tell application "PressAndHold" to inject SIMBL into Snow Leopard

And it worked!

Of course, this isn’t so useful yet. There’s no preferences UI, and it isn’t automatically loaded. But it’s a working proof-of-concept, and the code is available on GitHub.

Postscript: Chronology

The progression from “custom keyboard layout” to “hacking into a system plugin for the text input system” may have seemed pretty overwhelming already, but the true story is even less of a straight line. In truth, I first heard about the press-and-hold feature when everyone started complaining about it. Like I said, when I first upgraded to Lion myself, I thought it was pretty cool.

Then I spent months never using it, and not just because I was on a custom keyboard layout. I just forgot about it.

A few days ago I came across it again on someone’s blog, made it to the StackOverflow post I mentioned already, and found about about PressAndHold.app. What was different about my custom keyboard layout from Apple’s? Unfortunately, Apple’s keyboard layouts aren’t stored in a user-readable format anymore, so I couldn’t do much more than check the output of Ukelele. I tried saying my keyboard layout was a Roman keyboard instead of Unicode, but that had no effect. I tried using a bundle instead of just the keylayout file, but that had no effect. So I started hacking into PressAndHold.app to try to figure out what was going on.

(That was fun. Pro tip: don’t attach a text-based debugger to a text input method — you lose all ability to type after a few seconds. Not sure why the delay.)

I started hacking together MagicPressAndHold, taking my usual approach of simply logging arguments and return values in the methods I wanted to edit. In this case, that was -accentsStringForString:language:…and I noticed that my Dvorak IPA keyboard layout had no language (an empty string). What was up with that?

I sort of took it for granted at first that keylayout files can’t have associated languages, since there’s nothing in the XML file that indicates a language. So my first pass was to assume the system language if there isn’t one associated with the keyboard, and that worked.

Why continue? Well, it was an accident. I started looking ahead to how I could have the variant characters be different depending on which keyboard was active. And the way you find out which keyboard is active is by using the Text Input Source Services.

And that’s where the “secret” language keys are documented. So after getting MagicPressAndHold in a working state, I backtracked to my keyboard layout bundle, trying a number of experiments in the configuration. What I didn’t realize is that most of them don’t take effect unless you log out and back in again, but the simplest thing—a TISIntendedLanguage of "en"—did work in the end.

And MagicPressAndHold? Well, without a good way to inject itself into PressAndHold (which quits after not being used for a while, and launches again on demand), it’s probably going to remain an experiment, and I don’t plan to do any serious work on it.

For example, typing Option-R gives me retroflex variants of /t/, /d/, and /s/.↩
There are actually two other kinds of input to the Cocoa text system: “ink” inputs, which allow you to, say, draw a Chinese character instead of typing its Romanization, and “palette” inputs, which include the Keyboard Viewer and Character Picker palettes. You can create these too, but that’s beyond the scope of this post.↩

Mail Aliases

2012-04-10T00:00:00-04:00

Even though I’m the developer of Webmailer, I mostly use Apple Mail for e-mail these days. Sometimes it’s nice to be able to send e-mails using different identities; for Belkadan Software tech support I’m “Jordy Rose”, but for school-related communication I’d prefer my name to show up as “Jordan Rose” and use my school e-mail address. Mail supports this, but doesn’t actually have a preference for it without creating separate accounts.¹ You have to edit the settings file manually.

So I wrote Mail Aliases, a little app that can edit this data for you. It’s not particularly polished, but it might come in handy for some of you. Note that this comes WITHOUT WARRANTY, so if you’re at all worried, back up your Mail account settings in ~/Library/Mail/V2/MailData/Accounts.plist.

Mail Aliases has been tested on Mac OS X v10.7 (Lion), but should work with 10.6 (Snow Leopard) as well. You can check out the source on Github.

By the way, if you wondering why the blog has been silent so long, there are two reasons. First, I’ve been moving away from Cocoa development into lower-level stuff, specifically contributing to Clang in my free time. (Don’t worry, Webmailer and Keystone are still being supported!) Second, I spent several months after graduation teaching English abroad, and during that time I tried to stay away from software for a while. I’m hoping to start posting again soon about nifty programming things I come across, but (like every other blog on the internet) no promises!

You can have multiple e-mail addresses associated with a single account by separating them with commas, but you can only have one “Full Name” per account.↩

Header Anchors: A Safari Extension

2011-08-14T00:00:00-04:00

Ever want to link to a specific part of a web page? Well-designed web pages have anchors at the start of each section, which can serve as targets of links.¹ You’ve probably seen URLs that end in #something; that fragment identifier, or “frag-id”, refers to a specific anchor on the page.

What’s annoying, though, is that even if an author includes them, they’re hard to discover. Sometimes there’s a table of contents, sometimes not. What I really wanted was a way to just click on the nearest header and grab the link. So I wrote “Header Anchors”.

Header Anchors is a Safari extension which turns any header into a link if it has an associated anchor. It won’t look like a link (that’s my preferred design choice) but if you mouse over a header that has an anchor, the cursor will change to the usual “click here” pointer. You can then grab or copy the URL as usual.

The quickest way to test this is to go to a Wikipedia article. Each section on the page has a header (such as “References”) which has an associated anchor. With Header Anchors installed, clicking on “References” will show you the URL that refers to that part of the page in your location bar.

Download Header Anchors. Requires Safari 5.

Header Anchors will never access personal data, other than any preferences it might have in the future. Header Anchors will never access the internet; it does all its work in the browser.

You can check out the source on Github. If anyone wants to port this to Chrome or Firefox, be my guest…and please tell me about it, so I can link to your port!

The term “anchor” is why the HTML tag for linking is called a; its other use was to introduce an anchor on the page. Today I think the standard way to do that is just to use an id attribute.↩

"Little Big Details"

2011-08-10T00:00:00-04:00

I just discovered the site Little Big Details (via a UI designer’s Twitter feed…only I can’t remember whose).

Yes, yes, a thousand times yes. These sorts of details are what separate functional software from good software. This is the sort of thing I was talking about in “Priorities”. This is something that I think Apple gets right more than most people, but something which is good for everyone.

Whenever there’s a place in your program where you can leave things as they are, or make things easier on the user, and you’re not on a deadline, the right choice is always make things easier on the user.

Weak Linking

2011-07-29T00:00:00-04:00

When you compile a program that uses external libraries or frameworks, the last step (or a step near the end, at least) is to hook up all of the functions, etc. you use in your program to their implementations in the libraries. This is called “linking”.¹

A while back, Apple realized that when they added new features to their frameworks (usually with the release of each new OS version), people might want to take advantage of them, but remain backwards-compatible with old OSs. So they added a feature called weak linking. Apple wasn’t the first to realize that this might be a problem, and I don’t know if they were the first to come up with weak linking. (Heck, it wasn’t even their first time; apparently there was a similar capability in Mac OS Classic’s Code Fragment Manager.) But that’s where I heard about it first.

Here’s the way you use it (taken from Apple’s guide at the link above):

If a weakly linked symbol is not available in a framework, the linker sets the address of the symbol to NULL. You can check this address in your code using code similar to the following:
extern int MyWeakLinkedFunction() __attribute__((weak_import));
int main() {
    int result = 0;
    if (MyWeakLinkedFunction != NULL) {
        result = MyWeakLinkedFunction();
    }
    return result;
}
Note: When checking for the existence of a symbol, you must explicitly compare it to NULL or nil in your code. You cannot use the negation operator ( ! ) to negate the address of the symbol.

I’m not sure why that last note is in there; I’m pretty sure it would work anyway with a modern GCC or Clang. But I haven’t tested it.

Anyway, pretty cool, right? If you want to take advantage of something that only exists on 10.6 (say, associated objects) you can do that without sacrificing backwards compatibility.

Functions are all well and good, but what about other symbols, though? Webmailer uses the NSImageNameStatusAvailable constant to show the standard “online” image from iChat, using it to show which destination is active. I was trying to use it like this:

NSImage *activeImage;
// No need to redeclare it as weak-linked: AvailabilityMacros.h takes care of it.
if (NSImageNameStatusAvailable != NULL) {
    activeImage = [[NSImage imageNamed:NSImageNameStatusAvailable] copy];
} else {
    // Fall back to an image in our bundle.
    activeImage = [[NSImage alloc] initWithContentsOfFile:[bundle pathForImageResource:@"active"]];
}

And it wasn’t working. Despite my check to see if the image name was NULL, it still seemed to be unconditionally loading the image.

See the problem?

Let’s look at the instructions again:

If a weakly linked symbol is not available in a framework, the linker sets the address of the symbol to NULL.

OH! Right. You need to be able to tell the difference between a variable whose value is NULL, and a variable that doesn’t actually exist at all. For a function pointer, this doesn’t matter: when you say the name of a function like objc_getAssociatedObject, the compiler treats it like &objc_getAssociatedObject. But that’s not true for NSImageNameStatusAvailable, which is just a global constant, a pointer to an NSString.

Here’s what I should have written:

if (&NSImageNameStatusAvailable != NULL)

And indeed that works. Must be careful in the future!

By the way, Greg Parker has a good article on weakly-linked classes, but since this only works back to the first version of the framework that was compiled with weakly-linked classes, you can only use them with Mac OS X v10.7+ or iOS 3.1+. *sigh* Before that, you’ll have to use the old way of NSClassFromString to see if a class is available at runtime.

Postscript: How does it work?

In regular code, you know that it’s not possible for the address of a variable to be NULL. Clearly, if the variable exists, it lives somewhere in memory, and that address is guaranteed not to be 0 by the C standard.² But with weakly-linked symbols, the variable might not exist, and so it’s actually quite appropriate that its “address” might be NULL.

How does it work? It’s basically like a C++ reference: a weakly-linked symbol X actually has a “shadow” symbol with a name like X$non_lazy_ptr. Any time you refer to X, the compiler replaces it internally with *X$non_lazy_ptr.

And how does X$non_lazy_ptr get its value? When you load your bundle, there are two secret functions called dyld_stub_binding_helper and __dyld_func_lookup. My (weak) understanding is that the first one will go through all of your dynamically-linked symbols and use the second one to find the correct addresses to “fill in” the “shadow pointers”.

It’s basically a very small form of self-modifying code, but it makes weak linking possible. Pretty cool, huh?

Of course, this is greatly simplified from what a linker really does. If you’re a CS major, you’ve probably learned at least a little about linkers, and if not, well, chances are you won’t be needing to write your own any time soon. If you’re still looking for an explanation, here’s one that I found on the first page of Google results for “what is a linker”.↩
Well, actually, that might not be true. What’s guaranteed is that if you cast the integer 0 to a pointer value, you get the null pointer, and that the null pointer does not point to any valid object, and that the null pointer has a conditional value of false. But I don’t think it says that the representation of a null pointer has to be 0, which means that doing a C++ reinterpret_cast on a 0 could conceivably give you a valid pointer. I think.

In practice, pretty much everyone uses 0 as the representation of null pointers.↩