This is probably a good move for the vast majority of Mac users, but for folks with even a small amount of interest in tinkering with the configuration files and caches of various applications, it’s an outright nuisance.

The nuisance is exacerbated by Apple’s decision to build in the “make invisible” action into every single dot-update to the OS. So if you’ve taken the trouble to undo Apple’s work and make the folder visible again, you’ll notice that when you updated from 10.7.3 to 10.7.4, poof!, it’s gone again.

I solved this very early on for my own needs by adding the command-line instructions for re-showing the Library folder to my Terminal configuration script. Every time I open a new Terminal window, the Library folder is aggressively set to visible again. In practice, this has meant that since 10.7 shipped, I’ve never been bothered once by Apple’s disappearing act.

For less Terminal-intensive Mac users, this solution might not be sufficient. If you, or somebody you love, truly wants the Library folder to be forever visible in the Finder, a good trick is to add a script Login item to do the deed. Here are detailed instructions for achieving this:

1. Download UnhideLibrary. Make sure it unzips to a script application file after downloading. You’ll need to keep this file around, so I recommend storing it somewhere in your home folder.
2. Open System Preferences.
3. Click the Users & Groups icon.
4. Select the Login Items tab for appropriate user.
5. Click the + icon and select the downloaded UnhideLibrary script item.
6. Log out and log back in again.
7. Never curse Apple’s Library folder policy again.

The script file is just a few lines of AppleScript code to invoke the appropriate shell script. I originally used a plain shell script file but Thomas Borowski noticed it sometimes opens as a regular document at login time. He also wrote his own post essentially stating all the things I’m now repeating here!

So long Library folder, see you again, automatically, on 10.7.5.

Apple has been threatening for almost a year that apps for sale in the Mac App Store will need to embrace the Application Sandbox, or else further updates to the apps will not be accepted. The deadline for adopting the sandbox has slipped several times, but it currently rests at June 1, 2012. That’s only a few weeks away, and comes just ahead of Apple’s annual developer conference.

I’ve written a few rants about the Application Sandbox, culminating in my February 2012 piece imploring Apple to “Fix the Sandbox.” Slowly but surely, they are improving the technology that drives the sandboxing features of Mac OS X, but by June 1, it appears that many classes of application will still be “unsandboxable” under the current permissions model supplied by Apple.

The shortage of permissions, or “entitlements” in sandbox lingo, has always been at the root of my concerns. Especially because of the political move requiring existing Mac App Store apps to adopt the sandbox, it is easy to imagine a features bloodbath come June, or whenever the requirement goes into place. When Apple announced the postponement to June 1, they also took care to make assurances that bug-fix updates would still be allowed for non-sandboxed apps, which is a nice break, but will still leave many apps to die on the vine in lieu of suitable sandboxing entitlements for the features various apps provide.

Apple publicly documents the list of entitlements available to Mac software developers. One of my major concerns has always been that because the list of entitlements doesn’t cover all the nuanced, legitimate tasks that an app might want to perform on a user’s behalf, that Application Sandbox ran the risk of permanently forbidding certain types of application behavior from being conducted in the sandbox.

The high-level list of Application Sandbox permissions is intentionally coarser-grained than the lower-level “sandbox facility” which ultimately imposes the restrictions. Some developers criticize Apple for failing to embrace the sanbdbox with their own apps, while expecting developers to embrace it for the Mac App Store. On Mac OS X 10.7.4, the two notably sandboxed apps from Apple remain Preview and TextEdit, two apps that would be relatively simple to sandbox compared to a large number of complex, 3rd party products.

But Apple has applied a good deal of sandboxing to their code. For example, take a look at /usr/share/sandbox/ and /System/Library/Sandbox/Profiles/ for examples a variety of “.sb” files that specify the sandbox restrictions of a variety of system services and tools. These files are expressed in SBPL, or the Sandbox Policy Language, which is a scheme-based language used to express the fine-grained permission the sandbox facility is capable of controlling. See my “Sandbox Corners” for a bit more about the lower-level sandbox facility and SBPL files.

Today I decided to take a closer look at one SBPL file of particular interest: /System/Library/Sandbox/Profiles/application.sb. This is the SBPL file that, so far as I can tell, translates the high-level “entitlements” of the Application Sandbox into corresponding lower-level policy expressions for the sandbox facility. For example:

(if (entitlement "com.apple.security.network.client")
    (allow network-outbound (remote ip)))

You can get a taste for how the high level “I’m a network client” translates specifically to “allow out bound ip traffic” at the sandbox facility level. Other high-level rules express much more complex logic. For example, the “I want to use the printer” entitlement translates to a variety of low-level permissions to communicate with printer-system daemons, and read from printer configuration files on the system.

But by far the most interesting entitlement of all is one that I found at the bottom of application.sb. It’s not documented on Apple’s site, and from what I can tell this blog post will become the first Google match on the term. The entitlement is “com.apple.security.temporary-exception.sbpl”, and the comment above it reads simply: “Big Red Button”.

;; Big Red Button
(sandbox-array-entitlement
  "com.apple.security.temporary-exception.sbpl"
  (lambda (string)
    (let* ((port (open-input-string string))
           (sbpl (read port)))
      (eval sbpl))))

This temporary entitlement enables a high-level Application Sandbox-restricted application to supply, along with whatever other high-level entitlements it requests, an entitlement that brings with it as a parameter a literal SBPL program, that will be evaluated and thus applied by the lower-level sandboxing facility.

In short: the Big Red Button gives Apple an out.

Whatever mistakes they make in the devising of high-level entitlements can be theoretically undone after-the-fact by supplying developers with special Big Red Button entitlements that pass along specific permissions to the lower-level sandbox, liberating the application to do whatever important task it needs to do.

Of course, the fact that this facility exists does not say anything about whether Apple will indulge its use. And just because you, as a developer, could use this entitlement key to empower your sandboxed app to do practically anything, it doesn’t mean that Apple’s App Store reviewers would look kindly upon it. In fact, I’m almost positive that at this point, any developer who submits a sandboxed app with this entitlement will have to have already conversed extensively with Apple about the need for such a transgression.

But the entitlement is there, and that makes me breathe a little easier. The Application Sandbox is, so far as I can tell, technically capable of granting whatever permission any app could reasonably need. The only obstacle, and it’s a big one at that, is the political challenge of App Store approval.

Update: John Brayton made an interesting point in the comments, that regardless of MAS policy, the Big Red Button might be useful to non-MAS developers who wish to adopt the Application Sandbox , but can’t manage to squeeze all their functionality into the confines of its entitlements. I originally thought this made good sense, but then realized how risky a move this would be in practice. Because the entitlement in question here has not been documented by Apple, and is furthermore only listed in an implementation file for the system itself, the behavior of this entitlement can’t be relied upon.

Indeed, above I suggested that the entitlement would grant developers the power to enable virtually any capability in their sandboxed apps, but we have no idea how Apple has actually implemented support for the entitlement inside Apple. They could very well have a special case inside the SBPL language evaluator itself that looks for and rejects scripts that it doesn’t recognize as its own.

The application.sb file even has a strong disclaimer to this effect:

WARNING: The sandbox rules in this file currently constitute
Apple System Private Interface and are subject to change at any time and
without notice. The contents of this file are also auto-generated and
not user editable; it may be overwritten at any time.

In other words: you can’t count on the details of this file remaining stable from release to release. It would be foolish to base the behavior of any shipping app on entitlements listed there, but not documented by Apple. I think the Big Red Button is a very interesting find, and I’m very curious about it. But that’s how it should be considered: as a curiosity.

When I released MarsEdit 3.5 a few weeks ago, it included some very significant overhaul to MarsEdit’s handling of rich text formatting, including new support for customized formatting macros in Rich Text mode.

Given the large number of changes, I was not too surprised that a few annoying bugs snuck in as well. In particular, it became unreliable to apply some formatting options like heading and block quote in Rich Text mode. I took the opportunity while fixing those things to take care of a bunch of other little gotchas, including some more compatibility tweaks for the forthcoming Mountain Lion 10.8 release.

• Fix the preview window to recognize and substitute template placeholders in the section
• Fix a bug with Squarespace and Expression Engine not prompting for username/password on new blog
• Restore ability to set “Warn when no title” option on Blogger blogs
• Fix a bug that prevented some blog-configuration options from being selected after switching system type
• Restore ability to start a Header or Blockquote before typing content
• Pressing return twice to exit Preformatted works again
• Improve support for future releases of Mac OS X
  • Address an issue with icon view when resizing the Media Manager window
  • Avoid crashes caused by conflicting system libraries

Rich-Text Formatting Macros

When MarsEdit 3.0 was released two years ago, it marked a major transformation in the product. Previously the app was completely focused on providing a means to edit blog posts in HTML, Markdown, or other text based markup languages. With 3.0, I added the long-requested “WYSWIYG” rich-text editing that many customers prefer when working with web content.

But given the considerable work to achieve MarsEdit’s rich text editor, there were some loose ends. One of those loose ends had to do with the very powerful formatting macros available in plain-text mode. These formatting macros are completely user-customizable and allow users to add their own markup templates for virtually any kind of purpose, filling in the details with values from the editor such as the selection, the pasteboard, etc. Various complexities prevented this powerful formatting macro concept from being available to users in rich-text mode.

With MarsEdit 3.5, custom formatting macros are finally available in rich-text mode. I have already discovered some edge cases that don’t work perfectly, but for inserting arbitrary HTML snippets or “wrapping” the selected text in the rich editor with a particular style, these custom macros are very useful. Starting with this release, you can also customize the keyboard shortcut for any formatting macros, whether they are built-in or custom.

The possibilities are limitless, but to illustrate with a simple example, imagine you commonly find yourself wanting green text in your blog posts. The following formatting macro will take whatever text happens to be on the pasteboard, and insert it into the edited post with HTML markup for coloring it green:

Lion Full-Screen Mode

Shortly after Lion 10.7 was released, I added nominal support for full-screen mode to MarsEdit. The problem was, it wasn’t very useful for real-world writing scenarios. My mistakes? I merely zoomed the post editor window to occupy the entire height and width of the screen. Most users found this awkwardly huge editor space impossible to work comfortably in, so found themselves avoiding the full-screen mode, even though they liked the idea in concept.

I made a relatively simple tweak for 3.5 that should improve things dramatically. When you enter full-screen mode on Lion, the current width of your editor is preserved, and only the height is zoomed to occupy the full screen space on your Mac. Here’s a shrunken example for the MarsEdit post editor I’m using to write this entry:

Of course, it’s easy to imagine a number of ways that full-screen could be even more improved. Incorporating the preview window optionally into the space, offering to hide the Title and other fields, etc. But this is a step in the right direction.

Tumblr Tweaks

I won’t mince words: as a developer, I have a love-hate relationship with Tumblr. The service is immensely popular but in many regards their API (even the newer one, which I admit I’m not using yet) falls short when it comes to providing the services MarsEdit users expect. For example, it’s extremely frustrating that it remains impossible to upload images apart from photo posts. Over the years since I first added support for Tumblr, I have often wondered how much effort I should bother put in, when the service doesn’t seem too interested in catering to offline editing clients. But that doesn’t excuse the fact that many important features are supported by the API, and I haven’t had time to implement support for them yet in MarsEdit. I’m starting to chip away at those now.

MarsEdit 3.5 supports publishing entries to Tumblr as server-side drafts. I decided to tackle this one now because it’s one of the most frequently requested features from my Tumblr-using customers, and because it’s possible to implement correctly with Tumblr’s API. I tried to tackle “queue” type posts with this release as well, but ran into issues with the API that forced me to put that back on the shelf for now.

I also made some minor adjustments to the post editor UI for Tumblr to prevent some issues with Quote and Chat style posts.

Everything Else

I managed to fit a lot of little fixes and nagging shortcomings for this release. It’s hard to quantify all of those little details with much more than a bullet line in a list, so I’m listing the complete list of changes below. Hopefully you’ll just find this release to feel overall more finished and refined than before, and I hope to continue that trend on into the future.

• Highlighted Enhancements
  • Improved support for Lion full-screen mode
  • Custom Formatting Macros now work in Rich Text mode as well as HTML mode
  • Support for editing Tumblr Draft posts
• Post Editor enhancements
  • A number of improvements to Rich Text editor to address minor formatting bugs
  • Unified Formatting Macro menu for HTML and Rich Text modes
  • Draggging an image to the post editor now causes inserted image to go to expected target location in text
  • Fixed performance issues that could cause very slow typing in some configurations
  • Fixed keyboard-based text selection in the HTML editor to avoid getting stuck on line endings
  • Fix a slight mismatch in background color between Rich and HTML Text editing modes
  • Work around hanging issues with External Editing in some apps
• Tumblr-specific enhancemeents
  • Added support for server-stored Drafts
  • Fixed Word Count feature to work properly with Tumblr Quote and Chat posts
  • Revised Tumblr interface for Chat-style posts to prevent formatting problems
• Other changes
  • Added support for browsing Lightroom 4 libraries
  • New support for unicode characters in URLs
  • Bug fixes for improved Media Manager reliability
  • Fixed an issue where uploaded attachments would fail without prompting for password on authentication errors
  • Added support for authenticated proxy servers
  • Prevent a crash that occurred when working with Blog.de system’s categories
  • Improved XML sanitizing to increase compatibility with malformed blog content
  • Fix a bug that prevented locally added tags from becoming part of the tag-suggestion history

I focused again on puzzle printing improvements for this update. Significantly, Black Ink now supports an option to position the puzzle grid on the bottom-left or bottom-right of the page. This is a more natural solving position for many people, because it allows you to keep more of the clues visible while writing answers into the grid.

Here’s a taste of how the printing configuration and preview panel looks in 1.5:

Complete list of changes:

• Support new options for printing puzzle grid at the bottom of the page
• Cleaner printing layout for fit-on-one-sheet printing
• Improve speed of printing puzzles

The first cipher I’d suggest you consider besides bcrypt is PBKDF2. It’s ubiquitous and time-tested with an academic pedigree from RSA Labs, you know, the guys who invented much of the cryptographic ecosystem we use today.

I was a little fuzzy on the distinction between encryption techniques such as AES, and the technology being discussed here, which is known as a key derivation function. Let’s break it down. With an encryption technique like AES you can use a large (e.g. 128 bits), difficult to guess private key to encrypt and decrypt data. But as a human, you can’t reasonably be expected to type in a random, 128-bit key in by hand when you want to access your data. The key derivation function is the code that takes your relatively easily-remembered password and derives a suitably monstrous, unpredictably random key from it. The quality and uncrackability of that key derivation is what Tony is questioning here.

I don’t know enough about encryption to have my own informed opinion about this. I tend to rely on the collective wisdom of the software industry, or on high-level service providers such as Apple, to suitably safeguard sensitive data in my apps. Tony included Apple’s FileVault full-disk-encryption in the list of technologies that use PBKDF2, which lent the technique an air of superiority in my mind. I know some of the folks behind Apple’s disk encryption, and they are careful, smart engineers.

I rely on FileVault for protection of my documents. But like most folks, I rely on Apple’s Keychain for the protection of passwords. I’m keenly interested to know if the Keychain is as secure as it reasonably can be, because I store not only my own passwords in it, but also e.g. my users’ blogging passwords in their respective keychains.

AgileBits, developers of the popular secure-storage app 1Password, made a conscious decision not to use Apple’s Keychain. They cite a variety of compelling reasons, including Keychain’s alleged use of a somewhat outdated encryption technique called Triple DES. Agile has written extensively about the design of their own keychain, in which they confirm that they are using PBKDF2 to derive their encryption keys.

I’m confident that Apple’s Keychain is secure for all practical purposes, but it is just sort of irksome if they are not adopting the very best protection that Mac-money can buy. Unable to find suitably authoritative documentation on the matter, I took to Apple’s open source for libsecurity_keychain, the library through which the Keychain’s data is managed. My reading of the source code for a function called SecKeyDeriveFromPassword, does show that Apple is indeed using PBKDF2 to generate the key.

On 10.7.3 they are, at least. The SecKeyDeriveFromPassword API was new to 10.7, taking over for the older CSSM_DeriveKey. Perhaps the default behavior of that function did not use PBKDF2. In any case, it sure sounds as if on top of Tony’s urging, FileVault’s use, and 1Password’s adoption of PBKDF2, Apple’s decision to use it as the mechanism in their latest versions of the Keychain only adds to the impression that it’s a fine choice.

Many developers already choose to sign software using self-generated signing certificates. I wrote many years ago about the value of doing this for applications that access information from the user’s Keychain. With an unsigned application, every new version of your app needs the user’s approval to access the same keychain item (e.g. a blog password). But with a signed application, Apple allows a new version of the app from the same signature authority to continue accessing the keychain without nagging the user.

You might think that switching to Developer ID would be as simple as switching from the self-generated certificate to a certificate from Apple. I certainly assumed it would be, and so apparently did The Omni Group when they shipped a version of OmniFocus signed with their Developer ID certificate.

But there’s a catch. As The Omni Group’s own Ken Case pointed out on Twitter, simply substituting the Apple certificate will lead to code signature validation failure on 10.6 and earlier systems:

OmniFocus for Mac v1.10.1 is now available for those who were running into the keychain bug in our Developer ID builds on pre-Lion systems.

— Ken Case (@kcase) March 16, 2012

What is the cause for this failure? I asked Ken and he kindly went into more detail over Twitter. I decided to elaborate on his response and to do some more research of my own. I’m sharing a summary of my findings here for the reference of other developers.

Identity Confirmation

Every item in the keychain has individual settings for how it may be accessed in the future. The desirable Keychain behavior of allowing an app continued access to keychain items painstakingly documented in an Apple tech note, but the gist of it is that after gaining approval to access a keychain item from the user, an app will be allowed to continue accessing that item if it is in fact the same app it claims to be. The thinking here is, if the user indicated to the system that she trusts MarsEdit, then any future application that can prove it is also MarsEdit should inherit that same trust.

An app is determined to be itself by meeting its designated requirement (DR). The DR is the list of policies, as specified within the code signature, for determining the authenticity of the app. The code signing process generates a default DR that is suitable for most applications. You can use the codesign command-line tool to examine the DR of any signed application on your Mac. For brevity, I’m omitting all output here except the DR:

% codesign -d -r- /Applications/MarsEdit.app

designated => identifier "com.red-sweater.marsedit" and
certificate root = H"3a3292a0f7b9b5f5492d956d8f561621fe446e51"

This makes sense. The app should be considered to be “MarsEdit” if it has the right bundle ID, and is properly signed by a certificate whose trust chain leads to a certificate with the listed SHA1 fingerprint. In this case, the certificate listed is the self-generated certificate I made a few years ago for Red Sweater’s apps. Since the certificate is embedded into the app by the code-signing process, the resuilting app is imminently verifiable and will “satisfy its own DR” on any Mac:

% codesign -dv /Applications/MarsEdit.app

/Applications/MarsEdit3.4.4.app: valid on disk
/Applications/MarsEdit3.4.4.app: satisfies its Designated Requirement

I never had a role in choosing the DR for MarsEdit. It was the default policy installed into my app by the codesign utility. I didn’t care about the DR, I just cared that users weren’t nagged about accessing the keychain on successive updates to my app. Thanks to Apple’s sensible defaults, it did in fact “just work.”

A Betrayal Of Trust

When I switched over to Apple’s Developer ID certificate, I continued to assume that Apple’s default signing behaviors would lead to reasonable behavior with regard the Keychain. And, for the most part, it does. Here is the DR for a version of MarsEdit that was signed with Apple’s certificate:

designated => identifier "com.red-sweater.marsedit" and
anchor apple generic and
certificate 1[field.1.2.840.113635.100.6.2.6] /* exists */ and
certificate leaf[field.1.2.840.113635.100.6.1.13] /* exists */ and
certificate leaf[subject.OU] = "493CVA9A35"

Here the requirements take a little more decoding to make sense of. I’m learning as I go here but I’ll try to translate. The first line is pretty obvious, specifying the bundle ID. The other lines proceed to state requirements of the certificates in the signing trust chain. It identifies an anchor, certificate 1, and a leaf, so it expects three certificates in the chain in all. And specifically for these certificates it expects:

1. The root certificate must be an “apple generic” certificate. This is a special certificate designation specifically documented by Apple as being any certificate “for code signed by Apple, including code signed using a signing certificate issued by Apple to other developers.” I take this to mean that the root must either be an Apple certificate or a Developer Developer ID Certification Authority certificate that Apple provided to me for signing my app.
2. Certificate 1 (the certificate in the middle of the trust chain) must have a particular field, which I was able to verify on my Mac exists in the “Developer ID Certification Authority” certificate.
3. The leaf (my private Developer ID certificate) must have a particular field, which I again verified exists in my certificate.
4. The leaf must also have a specific subject value of “493CVA9A35″, which I take to be the unique ID Apple assigned to me for my certificate.

In short, the application must be signed with my specific Developer ID certificate, that certificate must be trusted by a certificate that has a special field indicating it’s a Developer ID authority, and that certificate must be trusted by Apple.

You can examine the certificate trust chain for a signed app with codesign:

% codesign -dvvv MarsEdit.app

Executable=/Volumes/daniel/Desktop/newmars/MarsEdit.app/Contents/MacOS/MarsEdit
Identifier=com.red-sweater.marsedit
Format=bundle with Mach-O universal (i386 x86_64)
CodeDirectory v=20100 size=4953 flags=0x0(none) hashes=241+3 location=embedded
Hash type=sha1 size=20
CDHash=f20bbe82230831579e301989faf2956a0142e838
Signature size=4234
Authority=Developer ID Application: Red Sweater Software
Authority=Developer ID Certification Authority
Authority=Apple Root CA
Signed Time=Mar 17, 2012 9:01:47 PM
Info.plist entries=31
Sealed Resources rules=4 files=149
Internal requirements count=1 size=260

I’ve highlighted the certificate chain in green. In this case, the trust chain has my Developer ID certificate as the leaf, the Apple Developer ID authority a step removed, and Apple’s Root CA a step above that. This all meshes nicely with the designated requirement rules we outlined above. So what’s the catch?

The catch is that the intermediate certificate is not likely to actually exist on any customer’s Mac. Because that’s a special certificate provided by Apple to developers, it’s only likely to be installed in the keychains of other developers. So when the Keychain goes to validate the app, it does not meet the designated requirement, so it is not trusted to inherit the access to the keychain.

A little bit that I still don’t completely understand is that apps that are signed in this way are allowed access on 10.7, even if the intermediate certificate is missing. I did just a small amount of research into this, but I think this has something to do with the new “SystemPolicy” file located at /var/db/SystemPolicy. It’s an sqlite3 file that you can freely examine. It’s where the system stores information about all the Gatekeeper signed apps on your Mac, but also where it maintains a list of what I think are high-level security overrides. Examining the SystemPolicy file on my 10.7 machine, I find a couple interesting “authority” elements:

INSERT INTO "authority" VALUES(4,1,'anchor apple generic and
certificate 1[field.1.2.840.113635.100.6.2.6] exists and
certificate leaf[field.1.2.840.113635.100.6.1.13] exists',
1,NULL,0.0,'Developer Seed',NULL,0,NULL);

INSERT INTO "authority" VALUES(5,2,'anchor apple generic and
certificate 1[field.1.2.840.113635.100.6.2.6] exists and
certificate leaf[field.1.2.840.113635.100.6.1.14] exists',
1,NULL,0.0,'Developer Seed',NULL,0,NULL);

Since Apple’s security code is open sourced, you can actually poke around at libsecurity_codesigning and try to make some sense of it. I don’t want to get too sucked into this curiosity, but I was able to confirm that the items above specify “authority” policies for running applications, and for installing applications. I was also able to confirm these are included in the source code to libSecurity, so they weren’t added on my system as a result of anything I did.

The naming of those items, “Developer Seed”, is pretty interesting. I speculate that Apple knows that the certificate chain for Developer ID-signed applications will not validate, and so they added these high level overrides in 10.7 so that we can test our signed applications, and so that customers can run them without incident. The only problem is there is no such workaround on 10.6 and earlier systems.

Getting Back To Basics

In order for our Developer ID-signed applications to work as expected on 10.6 or earlier, the designated requirement has to be met. Since the odds of the default DR being met are low, we need to override Apple’s default DR and install a simpler one that self-verifies with rules more along the lines of the way our self-generated certificate did it.

As Ken Case pointed out in his tweets, you can do this by specifying the requirements information explictily on the command line:

% codesign -f --sign "Developer ID"
  -r='designated => certificate leaf H"xxx" and
  identifier "com.red-sweater.marsedit"' MarsEdit.app

(Split into three lines for readability on my blog. It should all be on one line.)

This will sign the app using your Keychain-installed Developer ID certificate from Apple, but it will specify the given explicit DR. The xxx string is replaced by the SHA1 fingerprint for your Developer ID signing certificate, which you can find by examining the certificate in the Keychain Access app and looking at the very bottom.

Now, when the Keychain validates the designated requirement on either Mac OS X 10.7 or 10.6, it passes the test.

A Proper Solution

I have some concerns about working around the problem using this custom, dumbed down designated requirement. Sure, it’s no “dumber” than the self-generated signature’s DR, but what concerns me is whether Apple’s support for Gatekeeper in 10.8 will make assumptions about an app’s, ahem, Gateworthiness, based on some of those semi-magical certificate field checks in the problematic DR. If I’m correct in my guesses about how those SystemPolicy entries work, then there is already “magical” behavior happening in 10.7 for Developer ID signed apps. Who’s to say there won’t be continued magic going forward?

I’m not sure what the right fix is. Maybe it’s just this, dumbing down the requirements string. But it would be nice if Apple’s default code-signing requirements “just worked”…

Hey, wait. Why does this version of my app built with Xcode 4.3 “just work”? All of my shipping apps are currently built using Xcode 3 on a 10.6 machine. All of the examples above are based on analysis of an app that has been code-signed in that environment. But when I look at a version of my app built Mac OS X 10.7, and with Xcode 4.3, the requirements are a little different:

designated => anchor apple generic and
identifier "com.red-sweater.marsedit" and
(certificate leaf[field.1.2.840.113635.100.6.1.9] /* exists */ or
certificate 1[field.1.2.840.113635.100.6.2.6] /* exists */ and
certificate leaf[field.1.2.840.113635.100.6.1.13] /* exists */ and
certificate leaf[subject.OU] = "493CVA9A35")

I’ve highlighted in green the part that is different from my old DR. Where it used to require that intermediate Developer ID authority certificate that is missing from most Macs, it now requires either that certificate OR that a field in my Developer ID signing certificate exists. This effectively restores the passing status of the DR on any Mac, whether it has an intermediate installed, or whether or not there are any alleged funky overrides going on in SystemPolicy.

So, my recommended fix either to use Xcode 4.3 to build your app, or mimic Xcode 4.3′s code signing behavior by stealing its codesign arguments (look in the build log after building on 4.3) and applying them verbatim in your build process.

I’ve learned a whole lot about code signing and certificates today. If you got to the bottom of this post, I hope that you have too. In any case, this should give you a good idea of a situation you need to be aware of, and test against: if you are signing your app with Developer ID, and need to run on 10.6 or earlier, make sure the Keychain is still kind to your app.

Update: Now With More Facts

My conclusions above were based on some false assumptions on my part about how designated requirements are synthesized and how the certificate validation process actually works. Based on some valuable comments from Chris Suter and Evan Schoenberg I think it’s fair to say the root problem here is that “default” DR synthesis is actually just a placeholder that allows the host system to insert whatever default it thinks is suitable for the app. This explains a weird thing that I thought may have just been a bug: that the designated requirement showed up differently on 10.6 than on 10.7. It turns out this reflects the fact that if you don’t specify a specific DR, the system will do it’s best to make sense of your app (maybe based on its signatures to some extent?).

So, I’m not 100% fact based yet, but I think the new moral of the story is: if you need to run earlier than 10.7, specify your DR literally because the default DR generated for a Developer ID certificiate will not work perfectly on 10.6 and earlier.

Update: Now With Even More Facts

My friends at Fetch Softworks went even deeper on this issue, discovering that Apple’s own code-signing defaults for Mac App Store submissions offer a useful clue for a better, more generalized designated-requirement for Gatekeeper-signed apps. In a nutshell: you want to key the designated requirement off of your developer ID being listed in the certificate, rather than a precise fingerprint of a certificate.

This release is important for MarsEdit customers who use the integrated Flickr browser to insert images into blog posts. In particular, if you tend to use the “Medium” size embed, this update is critical to restore the app’s ability to insert photos of that size.

What happened is Flickr added some new image sizes: “Large Square” and “Small 320″. MarsEdit didn’t know what to make of either of these, and ended up calling each of them “Medium” too. So customers were seeing three medium items in the menu, and it was only ever possible to select the first one.

This update fixes the problem at hand and further fixes the way MarsEdit handled Flickr embed sizes so that should they add additional items or rename the existing ones, MarsEdit should adapt more elegantly.

The tech industry has been absorbed by the ad industry, and vice versa.

However, there is, imho, still room for a tech industry that is not merged with the ad industry.

I’ll take this a step further: advertising is on the way out. Technology loathes a middle-man, and advertising as an industry is the king of all middle-men. The purpose of advertising is to connect customers with companies, so as to facilitate a transfer of money in exchange for goods or services. As time goes by, customers and companies will be more and more capable of achieving this on their own.

In the history of the world so far, there has been considerable opportunity for advertisers to misguide customers, and to lure their money toward products or services that can be framed as perfect for them, even when they are not. That’s the art and the holy grail of advertising. But going forward, technology will offer customers and companies the tools to connect effortlessly, optimizing for compatibility without the help of the bogus, outdated advertising system.

Most of us base purchasing decisions on vague hunches derived from a mix of advertising influences, word-of-mouth, and the relative trendiness of a product. But more and more as customers we are cutting out the advertising middle-man, in favor of systems based on education and trust. Amazon is a good example of this. With the notable exception of their Kindle line of products, they have little concern about which products their customers buy. It only matters that they buy things, and that they buy things often. They provide detailed product information, and allow honest, often scathing reviews. The goal is for customers to make self-serving decisions. In this case, defying the advertisers’ best interests is in Amazon’s best interest as well.

Extrapolate the technology-assisted consumption process out over the next 10, 50, 100 years, and I have a hard time imagining a meaningful role for conventional advertising. If I search Google for “lawnmower,” it’s not interesting that some tractor company has paid Google for the privilege of putting their brand’s information at the top of the list. At some point in the future, customers will assume that companies who choose to advertise conventionally are afraid of the outcome when consulting various self-empowering resources. Where am I more likely to search for “lawnmower?” If I want to know what a lawnmower is, Google. If I want to know which lawnmower to buy? Amazon, or another site that strives to empower customers, not advertisers.

I do worry about what happens to some of our beloved, advertising-driven services. We’ve all grown accustomed to the subsidization of news reporting and analysis. In recent decades, advertising has crept further into our lives, even subsidizing municipal infrastructures such as public transit. What impact will the end of advertising have on these important services?

In the old world, technology for connecting customers directly to companies did not exist, so companies were satisfied in buying advertising. It is tool that serves to expose customers to the concept of a product, and to crudely attempt to educate them about the suitability of the product for their purposes.

In the new world, mass-exposure will be replaced by social networking, and education will be not only replaced by, but massively bolstered by trusted systems such as Amazon’s review database, Consumer Reports, and other much better stuff that is presumably coming in the future. Presumably? It has to be coming, and it has to be better, because everything’s riding on it.

Everything’s riding on it because this is the salvation for current advertising-subsidized industries. They will shift from being exposure-focused, to education-focused. Amazon, Apple, and many others already offer affiliate systems that reward anybody who can produce a sale. The old way to produce a sale is by blasting customers with unwanted information until you happen upon something that sticks. The new way is to provide customers with a trustworthy, opt-in system for determining what’s best for the customer. To stay alive in the changing world, these subsidized industries will change their business plans, or go out of business.

Earlier today, before I even followed Brent’s link to Dave’s piece, I read this short, thought-provoking essay, allegedly by the graffiti artist Banksy. Here is an excerpt that I think is pertinent to my predictions here:

They have access to the most sophisticated technology the world has ever seen and they bully you with it. They are The Advertisers and they are laughing at you.

…

You owe the companies nothing. Less than nothing, you especially don’t owe them any courtesy. They owe you. They have re-arranged the world to put themselves in front of you. They never asked for your permission, don’t even start asking for theirs.

As a businessman who is dedicated to my own commercial success, I embrace the challenge of getting the word out to potential customers. I will shout my message from the rooftops to anybody who will listen. But only to those who will listen. I don’t want to annoy, interrupt, cajole, or appeal to a customer’s feelings of inferiority. I don’t want a customer to choose my product over a competitor’s unless it’s better for them. In short: I want a future without advertising, where my products sell themselves through word-of-mouth and through trusted systems that educate customers about making the right choice for them. Not what’s right for companies, and certainly, so long as they’re still around, not what’s right for advertisers.

I highly recommend this update for anybody who has installed the Safari 5.2 beta release on Lion, or who is running a developer preview of a future OS X release.

MarsEdit 3.4.3

• Fixes to improve with the Safari 5.2 beta
• Fixes to improve with a future OS X update
• Prevent a possible hang while parsing Lightroom libraries