This is probably a good move for the vast majority of Mac users, but for folks with even a small amount of interest in tinkering with the configuration files and caches of various applications, it’s an outright nuisance.

The nuisance is exacerbated by Apple’s decision to build in the “make invisible” action into every single dot-update to the OS. So if you’ve taken the trouble to undo Apple’s work and make the folder visible again, you’ll notice that when you updated from 10.7.3 to 10.7.4, poof!, it’s gone again.

I solved this very early on for my own needs by adding the command-line instructions for re-showing the Library folder to my Terminal configuration script. Every time I open a new Terminal window, the Library folder is aggressively set to visible again. In practice, this has meant that since 10.7 shipped, I’ve never been bothered once by Apple’s disappearing act.

For less Terminal-intensive Mac users, this solution might not be sufficient. If you, or somebody you love, truly wants the Library folder to be forever visible in the Finder, a good trick is to add a script Login item to do the deed. Here are detailed instructions for achieving this:

1. Download UnhideLibrary. Make sure it unzips to a script application file after downloading. You’ll need to keep this file around, so I recommend storing it somewhere in your home folder.
2. Open System Preferences.
3. Click the Users & Groups icon.
4. Select the Login Items tab for appropriate user.
5. Click the + icon and select the downloaded UnhideLibrary script item.
6. Log out and log back in again.
7. Never curse Apple’s Library folder policy again.

The script file is just a few lines of AppleScript code to invoke the appropriate shell script. I originally used a plain shell script file but Thomas Borowski noticed it sometimes opens as a regular document at login time. He also wrote his own post essentially stating all the things I’m now repeating here!

So long Library folder, see you again, automatically, on 10.7.5.

Apple has been threatening for almost a year that apps for sale in the Mac App Store will need to embrace the Application Sandbox, or else further updates to the apps will not be accepted. The deadline for adopting the sandbox has slipped several times, but it currently rests at June 1, 2012. That’s only a few weeks away, and comes just ahead of Apple’s annual developer conference.

I’ve written a few rants about the Application Sandbox, culminating in my February 2012 piece imploring Apple to “Fix the Sandbox.” Slowly but surely, they are improving the technology that drives the sandboxing features of Mac OS X, but by June 1, it appears that many classes of application will still be “unsandboxable” under the current permissions model supplied by Apple.

The shortage of permissions, or “entitlements” in sandbox lingo, has always been at the root of my concerns. Especially because of the political move requiring existing Mac App Store apps to adopt the sandbox, it is easy to imagine a features bloodbath come June, or whenever the requirement goes into place. When Apple announced the postponement to June 1, they also took care to make assurances that bug-fix updates would still be allowed for non-sandboxed apps, which is a nice break, but will still leave many apps to die on the vine in lieu of suitable sandboxing entitlements for the features various apps provide.

Apple publicly documents the list of entitlements available to Mac software developers. One of my major concerns has always been that because the list of entitlements doesn’t cover all the nuanced, legitimate tasks that an app might want to perform on a user’s behalf, that Application Sandbox ran the risk of permanently forbidding certain types of application behavior from being conducted in the sandbox.

The high-level list of Application Sandbox permissions is intentionally coarser-grained than the lower-level “sandbox facility” which ultimately imposes the restrictions. Some developers criticize Apple for failing to embrace the sanbdbox with their own apps, while expecting developers to embrace it for the Mac App Store. On Mac OS X 10.7.4, the two notably sandboxed apps from Apple remain Preview and TextEdit, two apps that would be relatively simple to sandbox compared to a large number of complex, 3rd party products.

But Apple has applied a good deal of sandboxing to their code. For example, take a look at /usr/share/sandbox/ and /System/Library/Sandbox/Profiles/ for examples a variety of “.sb” files that specify the sandbox restrictions of a variety of system services and tools. These files are expressed in SBPL, or the Sandbox Policy Language, which is a scheme-based language used to express the fine-grained permission the sandbox facility is capable of controlling. See my “Sandbox Corners” for a bit more about the lower-level sandbox facility and SBPL files.

Today I decided to take a closer look at one SBPL file of particular interest: /System/Library/Sandbox/Profiles/application.sb. This is the SBPL file that, so far as I can tell, translates the high-level “entitlements” of the Application Sandbox into corresponding lower-level policy expressions for the sandbox facility. For example:

(if (entitlement "com.apple.security.network.client")
    (allow network-outbound (remote ip)))

You can get a taste for how the high level “I’m a network client” translates specifically to “allow out bound ip traffic” at the sandbox facility level. Other high-level rules express much more complex logic. For example, the “I want to use the printer” entitlement translates to a variety of low-level permissions to communicate with printer-system daemons, and read from printer configuration files on the system.

But by far the most interesting entitlement of all is one that I found at the bottom of application.sb. It’s not documented on Apple’s site, and from what I can tell this blog post will become the first Google match on the term. The entitlement is “com.apple.security.temporary-exception.sbpl”, and the comment above it reads simply: “Big Red Button”.

;; Big Red Button
(sandbox-array-entitlement
  "com.apple.security.temporary-exception.sbpl"
  (lambda (string)
    (let* ((port (open-input-string string))
           (sbpl (read port)))
      (eval sbpl))))

This temporary entitlement enables a high-level Application Sandbox-restricted application to supply, along with whatever other high-level entitlements it requests, an entitlement that brings with it as a parameter a literal SBPL program, that will be evaluated and thus applied by the lower-level sandboxing facility.

In short: the Big Red Button gives Apple an out.

Whatever mistakes they make in the devising of high-level entitlements can be theoretically undone after-the-fact by supplying developers with special Big Red Button entitlements that pass along specific permissions to the lower-level sandbox, liberating the application to do whatever important task it needs to do.

Of course, the fact that this facility exists does not say anything about whether Apple will indulge its use. And just because you, as a developer, could use this entitlement key to empower your sandboxed app to do practically anything, it doesn’t mean that Apple’s App Store reviewers would look kindly upon it. In fact, I’m almost positive that at this point, any developer who submits a sandboxed app with this entitlement will have to have already conversed extensively with Apple about the need for such a transgression.

But the entitlement is there, and that makes me breathe a little easier. The Application Sandbox is, so far as I can tell, technically capable of granting whatever permission any app could reasonably need. The only obstacle, and it’s a big one at that, is the political challenge of App Store approval.

Update: John Brayton made an interesting point in the comments, that regardless of MAS policy, the Big Red Button might be useful to non-MAS developers who wish to adopt the Application Sandbox , but can’t manage to squeeze all their functionality into the confines of its entitlements. I originally thought this made good sense, but then realized how risky a move this would be in practice. Because the entitlement in question here has not been documented by Apple, and is furthermore only listed in an implementation file for the system itself, the behavior of this entitlement can’t be relied upon.

Indeed, above I suggested that the entitlement would grant developers the power to enable virtually any capability in their sandboxed apps, but we have no idea how Apple has actually implemented support for the entitlement inside Apple. They could very well have a special case inside the SBPL language evaluator itself that looks for and rejects scripts that it doesn’t recognize as its own.

The application.sb file even has a strong disclaimer to this effect:

WARNING: The sandbox rules in this file currently constitute
Apple System Private Interface and are subject to change at any time and
without notice. The contents of this file are also auto-generated and
not user editable; it may be overwritten at any time.

In other words: you can’t count on the details of this file remaining stable from release to release. It would be foolish to base the behavior of any shipping app on entitlements listed there, but not documented by Apple. I think the Big Red Button is a very interesting find, and I’m very curious about it. But that’s how it should be considered: as a curiosity.

The first cipher I’d suggest you consider besides bcrypt is PBKDF2. It’s ubiquitous and time-tested with an academic pedigree from RSA Labs, you know, the guys who invented much of the cryptographic ecosystem we use today.

I was a little fuzzy on the distinction between encryption techniques such as AES, and the technology being discussed here, which is known as a key derivation function. Let’s break it down. With an encryption technique like AES you can use a large (e.g. 128 bits), difficult to guess private key to encrypt and decrypt data. But as a human, you can’t reasonably be expected to type in a random, 128-bit key in by hand when you want to access your data. The key derivation function is the code that takes your relatively easily-remembered password and derives a suitably monstrous, unpredictably random key from it. The quality and uncrackability of that key derivation is what Tony is questioning here.

I don’t know enough about encryption to have my own informed opinion about this. I tend to rely on the collective wisdom of the software industry, or on high-level service providers such as Apple, to suitably safeguard sensitive data in my apps. Tony included Apple’s FileVault full-disk-encryption in the list of technologies that use PBKDF2, which lent the technique an air of superiority in my mind. I know some of the folks behind Apple’s disk encryption, and they are careful, smart engineers.

I rely on FileVault for protection of my documents. But like most folks, I rely on Apple’s Keychain for the protection of passwords. I’m keenly interested to know if the Keychain is as secure as it reasonably can be, because I store not only my own passwords in it, but also e.g. my users’ blogging passwords in their respective keychains.

AgileBits, developers of the popular secure-storage app 1Password, made a conscious decision not to use Apple’s Keychain. They cite a variety of compelling reasons, including Keychain’s alleged use of a somewhat outdated encryption technique called Triple DES. Agile has written extensively about the design of their own keychain, in which they confirm that they are using PBKDF2 to derive their encryption keys.

I’m confident that Apple’s Keychain is secure for all practical purposes, but it is just sort of irksome if they are not adopting the very best protection that Mac-money can buy. Unable to find suitably authoritative documentation on the matter, I took to Apple’s open source for libsecurity_keychain, the library through which the Keychain’s data is managed. My reading of the source code for a function called SecKeyDeriveFromPassword, does show that Apple is indeed using PBKDF2 to generate the key.

On 10.7.3 they are, at least. The SecKeyDeriveFromPassword API was new to 10.7, taking over for the older CSSM_DeriveKey. Perhaps the default behavior of that function did not use PBKDF2. In any case, it sure sounds as if on top of Tony’s urging, FileVault’s use, and 1Password’s adoption of PBKDF2, Apple’s decision to use it as the mechanism in their latest versions of the Keychain only adds to the impression that it’s a fine choice.

Many developers already choose to sign software using self-generated signing certificates. I wrote many years ago about the value of doing this for applications that access information from the user’s Keychain. With an unsigned application, every new version of your app needs the user’s approval to access the same keychain item (e.g. a blog password). But with a signed application, Apple allows a new version of the app from the same signature authority to continue accessing the keychain without nagging the user.

You might think that switching to Developer ID would be as simple as switching from the self-generated certificate to a certificate from Apple. I certainly assumed it would be, and so apparently did The Omni Group when they shipped a version of OmniFocus signed with their Developer ID certificate.

But there’s a catch. As The Omni Group’s own Ken Case pointed out on Twitter, simply substituting the Apple certificate will lead to code signature validation failure on 10.6 and earlier systems:

OmniFocus for Mac v1.10.1 is now available for those who were running into the keychain bug in our Developer ID builds on pre-Lion systems.

— Ken Case (@kcase) March 16, 2012

What is the cause for this failure? I asked Ken and he kindly went into more detail over Twitter. I decided to elaborate on his response and to do some more research of my own. I’m sharing a summary of my findings here for the reference of other developers.

Identity Confirmation

Every item in the keychain has individual settings for how it may be accessed in the future. The desirable Keychain behavior of allowing an app continued access to keychain items painstakingly documented in an Apple tech note, but the gist of it is that after gaining approval to access a keychain item from the user, an app will be allowed to continue accessing that item if it is in fact the same app it claims to be. The thinking here is, if the user indicated to the system that she trusts MarsEdit, then any future application that can prove it is also MarsEdit should inherit that same trust.

An app is determined to be itself by meeting its designated requirement (DR). The DR is the list of policies, as specified within the code signature, for determining the authenticity of the app. The code signing process generates a default DR that is suitable for most applications. You can use the codesign command-line tool to examine the DR of any signed application on your Mac. For brevity, I’m omitting all output here except the DR:

% codesign -d -r- /Applications/MarsEdit.app

designated => identifier "com.red-sweater.marsedit" and
certificate root = H"3a3292a0f7b9b5f5492d956d8f561621fe446e51"

This makes sense. The app should be considered to be “MarsEdit” if it has the right bundle ID, and is properly signed by a certificate whose trust chain leads to a certificate with the listed SHA1 fingerprint. In this case, the certificate listed is the self-generated certificate I made a few years ago for Red Sweater’s apps. Since the certificate is embedded into the app by the code-signing process, the resuilting app is imminently verifiable and will “satisfy its own DR” on any Mac:

% codesign -dv /Applications/MarsEdit.app

/Applications/MarsEdit3.4.4.app: valid on disk
/Applications/MarsEdit3.4.4.app: satisfies its Designated Requirement

I never had a role in choosing the DR for MarsEdit. It was the default policy installed into my app by the codesign utility. I didn’t care about the DR, I just cared that users weren’t nagged about accessing the keychain on successive updates to my app. Thanks to Apple’s sensible defaults, it did in fact “just work.”

A Betrayal Of Trust

When I switched over to Apple’s Developer ID certificate, I continued to assume that Apple’s default signing behaviors would lead to reasonable behavior with regard the Keychain. And, for the most part, it does. Here is the DR for a version of MarsEdit that was signed with Apple’s certificate:

designated => identifier "com.red-sweater.marsedit" and
anchor apple generic and
certificate 1[field.1.2.840.113635.100.6.2.6] /* exists */ and
certificate leaf[field.1.2.840.113635.100.6.1.13] /* exists */ and
certificate leaf[subject.OU] = "493CVA9A35"

Here the requirements take a little more decoding to make sense of. I’m learning as I go here but I’ll try to translate. The first line is pretty obvious, specifying the bundle ID. The other lines proceed to state requirements of the certificates in the signing trust chain. It identifies an anchor, certificate 1, and a leaf, so it expects three certificates in the chain in all. And specifically for these certificates it expects:

1. The root certificate must be an “apple generic” certificate. This is a special certificate designation specifically documented by Apple as being any certificate “for code signed by Apple, including code signed using a signing certificate issued by Apple to other developers.” I take this to mean that the root must either be an Apple certificate or a Developer Developer ID Certification Authority certificate that Apple provided to me for signing my app.
2. Certificate 1 (the certificate in the middle of the trust chain) must have a particular field, which I was able to verify on my Mac exists in the “Developer ID Certification Authority” certificate.
3. The leaf (my private Developer ID certificate) must have a particular field, which I again verified exists in my certificate.
4. The leaf must also have a specific subject value of “493CVA9A35″, which I take to be the unique ID Apple assigned to me for my certificate.

In short, the application must be signed with my specific Developer ID certificate, that certificate must be trusted by a certificate that has a special field indicating it’s a Developer ID authority, and that certificate must be trusted by Apple.

You can examine the certificate trust chain for a signed app with codesign:

% codesign -dvvv MarsEdit.app

Executable=/Volumes/daniel/Desktop/newmars/MarsEdit.app/Contents/MacOS/MarsEdit
Identifier=com.red-sweater.marsedit
Format=bundle with Mach-O universal (i386 x86_64)
CodeDirectory v=20100 size=4953 flags=0x0(none) hashes=241+3 location=embedded
Hash type=sha1 size=20
CDHash=f20bbe82230831579e301989faf2956a0142e838
Signature size=4234
Authority=Developer ID Application: Red Sweater Software
Authority=Developer ID Certification Authority
Authority=Apple Root CA
Signed Time=Mar 17, 2012 9:01:47 PM
Info.plist entries=31
Sealed Resources rules=4 files=149
Internal requirements count=1 size=260

I’ve highlighted the certificate chain in green. In this case, the trust chain has my Developer ID certificate as the leaf, the Apple Developer ID authority a step removed, and Apple’s Root CA a step above that. This all meshes nicely with the designated requirement rules we outlined above. So what’s the catch?

The catch is that the intermediate certificate is not likely to actually exist on any customer’s Mac. Because that’s a special certificate provided by Apple to developers, it’s only likely to be installed in the keychains of other developers. So when the Keychain goes to validate the app, it does not meet the designated requirement, so it is not trusted to inherit the access to the keychain.

A little bit that I still don’t completely understand is that apps that are signed in this way are allowed access on 10.7, even if the intermediate certificate is missing. I did just a small amount of research into this, but I think this has something to do with the new “SystemPolicy” file located at /var/db/SystemPolicy. It’s an sqlite3 file that you can freely examine. It’s where the system stores information about all the Gatekeeper signed apps on your Mac, but also where it maintains a list of what I think are high-level security overrides. Examining the SystemPolicy file on my 10.7 machine, I find a couple interesting “authority” elements:

INSERT INTO "authority" VALUES(4,1,'anchor apple generic and
certificate 1[field.1.2.840.113635.100.6.2.6] exists and
certificate leaf[field.1.2.840.113635.100.6.1.13] exists',
1,NULL,0.0,'Developer Seed',NULL,0,NULL);

INSERT INTO "authority" VALUES(5,2,'anchor apple generic and
certificate 1[field.1.2.840.113635.100.6.2.6] exists and
certificate leaf[field.1.2.840.113635.100.6.1.14] exists',
1,NULL,0.0,'Developer Seed',NULL,0,NULL);

Since Apple’s security code is open sourced, you can actually poke around at libsecurity_codesigning and try to make some sense of it. I don’t want to get too sucked into this curiosity, but I was able to confirm that the items above specify “authority” policies for running applications, and for installing applications. I was also able to confirm these are included in the source code to libSecurity, so they weren’t added on my system as a result of anything I did.

The naming of those items, “Developer Seed”, is pretty interesting. I speculate that Apple knows that the certificate chain for Developer ID-signed applications will not validate, and so they added these high level overrides in 10.7 so that we can test our signed applications, and so that customers can run them without incident. The only problem is there is no such workaround on 10.6 and earlier systems.

Getting Back To Basics

In order for our Developer ID-signed applications to work as expected on 10.6 or earlier, the designated requirement has to be met. Since the odds of the default DR being met are low, we need to override Apple’s default DR and install a simpler one that self-verifies with rules more along the lines of the way our self-generated certificate did it.

As Ken Case pointed out in his tweets, you can do this by specifying the requirements information explictily on the command line:

% codesign -f --sign "Developer ID"
  -r='designated => certificate leaf H"xxx" and
  identifier "com.red-sweater.marsedit"' MarsEdit.app

(Split into three lines for readability on my blog. It should all be on one line.)

This will sign the app using your Keychain-installed Developer ID certificate from Apple, but it will specify the given explicit DR. The xxx string is replaced by the SHA1 fingerprint for your Developer ID signing certificate, which you can find by examining the certificate in the Keychain Access app and looking at the very bottom.

Now, when the Keychain validates the designated requirement on either Mac OS X 10.7 or 10.6, it passes the test.

A Proper Solution

I have some concerns about working around the problem using this custom, dumbed down designated requirement. Sure, it’s no “dumber” than the self-generated signature’s DR, but what concerns me is whether Apple’s support for Gatekeeper in 10.8 will make assumptions about an app’s, ahem, Gateworthiness, based on some of those semi-magical certificate field checks in the problematic DR. If I’m correct in my guesses about how those SystemPolicy entries work, then there is already “magical” behavior happening in 10.7 for Developer ID signed apps. Who’s to say there won’t be continued magic going forward?

I’m not sure what the right fix is. Maybe it’s just this, dumbing down the requirements string. But it would be nice if Apple’s default code-signing requirements “just worked”…

Hey, wait. Why does this version of my app built with Xcode 4.3 “just work”? All of my shipping apps are currently built using Xcode 3 on a 10.6 machine. All of the examples above are based on analysis of an app that has been code-signed in that environment. But when I look at a version of my app built Mac OS X 10.7, and with Xcode 4.3, the requirements are a little different:

designated => anchor apple generic and
identifier "com.red-sweater.marsedit" and
(certificate leaf[field.1.2.840.113635.100.6.1.9] /* exists */ or
certificate 1[field.1.2.840.113635.100.6.2.6] /* exists */ and
certificate leaf[field.1.2.840.113635.100.6.1.13] /* exists */ and
certificate leaf[subject.OU] = "493CVA9A35")

I’ve highlighted in green the part that is different from my old DR. Where it used to require that intermediate Developer ID authority certificate that is missing from most Macs, it now requires either that certificate OR that a field in my Developer ID signing certificate exists. This effectively restores the passing status of the DR on any Mac, whether it has an intermediate installed, or whether or not there are any alleged funky overrides going on in SystemPolicy.

So, my recommended fix either to use Xcode 4.3 to build your app, or mimic Xcode 4.3′s code signing behavior by stealing its codesign arguments (look in the build log after building on 4.3) and applying them verbatim in your build process.

I’ve learned a whole lot about code signing and certificates today. If you got to the bottom of this post, I hope that you have too. In any case, this should give you a good idea of a situation you need to be aware of, and test against: if you are signing your app with Developer ID, and need to run on 10.6 or earlier, make sure the Keychain is still kind to your app.

Update: Now With More Facts

My conclusions above were based on some false assumptions on my part about how designated requirements are synthesized and how the certificate validation process actually works. Based on some valuable comments from Chris Suter and Evan Schoenberg I think it’s fair to say the root problem here is that “default” DR synthesis is actually just a placeholder that allows the host system to insert whatever default it thinks is suitable for the app. This explains a weird thing that I thought may have just been a bug: that the designated requirement showed up differently on 10.6 than on 10.7. It turns out this reflects the fact that if you don’t specify a specific DR, the system will do it’s best to make sense of your app (maybe based on its signatures to some extent?).

So, I’m not 100% fact based yet, but I think the new moral of the story is: if you need to run earlier than 10.7, specify your DR literally because the default DR generated for a Developer ID certificiate will not work perfectly on 10.6 and earlier.

Update: Now With Even More Facts

My friends at Fetch Softworks went even deeper on this issue, discovering that Apple’s own code-signing defaults for Mac App Store submissions offer a useful clue for a better, more generalized designated-requirement for Gatekeeper-signed apps. In a nutshell: you want to key the designated requirement off of your developer ID being listed in the certificate, rather than a precise fingerprint of a certificate.

The tech industry has been absorbed by the ad industry, and vice versa.

However, there is, imho, still room for a tech industry that is not merged with the ad industry.

I’ll take this a step further: advertising is on the way out. Technology loathes a middle-man, and advertising as an industry is the king of all middle-men. The purpose of advertising is to connect customers with companies, so as to facilitate a transfer of money in exchange for goods or services. As time goes by, customers and companies will be more and more capable of achieving this on their own.

In the history of the world so far, there has been considerable opportunity for advertisers to misguide customers, and to lure their money toward products or services that can be framed as perfect for them, even when they are not. That’s the art and the holy grail of advertising. But going forward, technology will offer customers and companies the tools to connect effortlessly, optimizing for compatibility without the help of the bogus, outdated advertising system.

Most of us base purchasing decisions on vague hunches derived from a mix of advertising influences, word-of-mouth, and the relative trendiness of a product. But more and more as customers we are cutting out the advertising middle-man, in favor of systems based on education and trust. Amazon is a good example of this. With the notable exception of their Kindle line of products, they have little concern about which products their customers buy. It only matters that they buy things, and that they buy things often. They provide detailed product information, and allow honest, often scathing reviews. The goal is for customers to make self-serving decisions. In this case, defying the advertisers’ best interests is in Amazon’s best interest as well.

Extrapolate the technology-assisted consumption process out over the next 10, 50, 100 years, and I have a hard time imagining a meaningful role for conventional advertising. If I search Google for “lawnmower,” it’s not interesting that some tractor company has paid Google for the privilege of putting their brand’s information at the top of the list. At some point in the future, customers will assume that companies who choose to advertise conventionally are afraid of the outcome when consulting various self-empowering resources. Where am I more likely to search for “lawnmower?” If I want to know what a lawnmower is, Google. If I want to know which lawnmower to buy? Amazon, or another site that strives to empower customers, not advertisers.

I do worry about what happens to some of our beloved, advertising-driven services. We’ve all grown accustomed to the subsidization of news reporting and analysis. In recent decades, advertising has crept further into our lives, even subsidizing municipal infrastructures such as public transit. What impact will the end of advertising have on these important services?

In the old world, technology for connecting customers directly to companies did not exist, so companies were satisfied in buying advertising. It is tool that serves to expose customers to the concept of a product, and to crudely attempt to educate them about the suitability of the product for their purposes.

In the new world, mass-exposure will be replaced by social networking, and education will be not only replaced by, but massively bolstered by trusted systems such as Amazon’s review database, Consumer Reports, and other much better stuff that is presumably coming in the future. Presumably? It has to be coming, and it has to be better, because everything’s riding on it.

Everything’s riding on it because this is the salvation for current advertising-subsidized industries. They will shift from being exposure-focused, to education-focused. Amazon, Apple, and many others already offer affiliate systems that reward anybody who can produce a sale. The old way to produce a sale is by blasting customers with unwanted information until you happen upon something that sticks. The new way is to provide customers with a trustworthy, opt-in system for determining what’s best for the customer. To stay alive in the changing world, these subsidized industries will change their business plans, or go out of business.

Earlier today, before I even followed Brent’s link to Dave’s piece, I read this short, thought-provoking essay, allegedly by the graffiti artist Banksy. Here is an excerpt that I think is pertinent to my predictions here:

They have access to the most sophisticated technology the world has ever seen and they bully you with it. They are The Advertisers and they are laughing at you.

…

You owe the companies nothing. Less than nothing, you especially don’t owe them any courtesy. They owe you. They have re-arranged the world to put themselves in front of you. They never asked for your permission, don’t even start asking for theirs.

As a businessman who is dedicated to my own commercial success, I embrace the challenge of getting the word out to potential customers. I will shout my message from the rooftops to anybody who will listen. But only to those who will listen. I don’t want to annoy, interrupt, cajole, or appeal to a customer’s feelings of inferiority. I don’t want a customer to choose my product over a competitor’s unless it’s better for them. In short: I want a future without advertising, where my products sell themselves through word-of-mouth and through trusted systems that educate customers about making the right choice for them. Not what’s right for companies, and certainly, so long as they’re still around, not what’s right for advertisers.

I am excited about Gatekeeper not only because it will improve security on the Mac but because of how it will achieve this goal. Apple, as the authority in the OS X environment, will convey information to Mac users about who developed a particular application, empowering them to protect themselves. Compare this to the status quo of the App Store, where security is completely out of users’ hands, and Apple uses its discretion to protect users from software it judges unfit for consumption.

Who vs. What

Simply establishing the identities of software developers is a major step for increasing security, because bad actors can either be immediately shut down, or at least prevented from further propagating on the platform. If “Hawt Dawg Industries” is discovered to be a malware developer, Apple can flip a switch and any user who trusts Apple’s opinion about such things can automatically prevent their Macs from trusting software from that vendor.

If somebody knocks on your door in the middle of the night, the first thing you’re liable to ask is “Who are you?” That’s Gatekeeper. Sometimes, the “who” is all the information you need. But if there’s any doubt, the next bit of information you’ll pry for is “What do you want?” That’s the sandbox. At least, it’s what the sandbox will be, after Apple fixes it.

The Broken Sandbox

At its best sandboxing is a means for app developers to faithfully state their intentions in a manner that can be evaluated by users, and also be reliably enforced by the operating system. So if your new “Fun on Facebook” app declares its intention is to connect to the web, you might judiciously allow it. If it says it needs to write files to the root of the filesystem, you’d be wise to search for another app.

Sandboxing on the Mac works by providing developers with a standardized list of “entitlements” which are clear descriptions of things it would like to do on your Mac. Examples include: access the internet, read files from your Pictures folder, print things on your printer.

The number one broken thing about sandboxing as it stands today, is the list of entitlements is simply too limited. Many apps on the App Store, including my own, will need to have their functionality considerably diminished, or in some cases made outright useless, in order to accommodate the available list of entitlements that sandboxing offers.

To stretch the stranger-at-your-door metaphor a little further, imagine the visitor is your trusted plumber, who’s come to fix a leaking pipe. In response to “What do you want?” he’s a bit tongue-tied. There’s no “entitlement” for fixing pipes, so he’s forced to say “I’m here for a chat.” When you reluctantly let him in the door he informs you that actually, due to recent legal changes, he’s no longer allowed to fix your pipes.

The impending state of the Mac App Store is very much like this. A great number of apps provide useful services to grateful customers, but as those services don’t fit the mould of Apple’s sandboxing entitlements, they will be effectively barred from the store within a few weeks. If you want to hire somebody to “fix the leaky pipe,” you’ll have to look outside the store, where apps are not sandboxed at all, and where Apple is in no position to improve users’ knowledge about the “what” of an app.

Saving Face

Gatekeeper is extremely simple, yet is likely to be extremely effective. Some exasperated developers who have been frustrated by the sandbox limitations are hopeful that all this attention on Gatekeeper might indicate a change of heart on Apple’s part. Will they see the error of their ways and ditch sandboxing in favor of Gatekeeper’s elegance?

One problem with this approach is that Apple would appear as though it had stumbled in its strategy. It spent the greater part of a year selling the idea of sandboxing and all of its merits, then two weeks before its grand debut, jumps ship for a completely different approach? Smooth move, Apple.

But a more important problem is that abandoning sandboxing would mean the significant engineering investment, both by Apple and by developers who have refactored their apps to satisfy sandboxing requirements, would have been a waste. There is such great value in sandboxing technology, we just need to finish the job of mining it out.

What should Apple do about all this? Gatekeeper and the Mac App Sandbox are both technologies that stand to improve security on the Mac by labeling apps with useful information about their developers and their functionality. The extent to which security is improved is very much tied to how widely adopted these technologies are. If the vast majority of developers agree to sign their apps with Gatekeeper certificates, then the vast majority of users will leave the Gatekeeper “safe” mode enabled.

Embrace, Expand, and Empower

Apple should embrace the utility of sandboxing by shifting their focus away from sandboxing only Mac App Store titles, to a strategy that would sandbox virtually every Mac app, inside the store or out. Given the current limitations of sandboxing, a significant number of developers will not adopt the technology, so its usefulness to users and to the security of the platform will be diminished. Apple can turn that around so that sandboxing is a worthy counterpart to Gatekeeper, and a technology that any developer in his or her right mind would feel foolish not to incorporate.

To increase adoption, Apple should expand the current list of entitlements until it covers every reasonable behavior that users expect from Mac apps. A good test for this is any app that is currently available in the Mac App Store. Having been approved by Apple’s own reviewers, and purchased by Apple’s own customers, the merit of these apps should be considered implicit. If a Mac App Store app’s reasonable behavior cannot be achieved in the confines of the sandbox, it should be considered a sandboxing bug, and a new entitlement should be added.

Finally, Apple should take a cue from its own Gatekeeper approach. By incorporating sandbox information about apps into the forthcoming app security preference pane, they will empower users to understand application intentions. Along with the proposed options controlling the “who” of apps, users would be given reasonable defaults pertaining to the “what” of apps. For power users, these settings would be configurable on an entitlement-by-entitlement basis. The sheer transparency will be yet another motivation for developers to adopt sandbox, and for users to demand sandboxing from their developers.

Imagine a future where the majority of Mac apps are signed with Gatekeeper certificates, and an accurate list of entitlements. Users will be protected by smart default settings, and by the knowledge of who their apps come from, as well as what they intend to do. Developers will be protected from their own unintentionally destructive mistakes, and from impostors selling software purported to be authentic. And Apple? Apple will be remembered as the huge, clever computer company that solved the software security problem on two fronts, without pissing off developers or customers. Much.

(This piece was inspired by a lunchtime chat with my friend Paul Kafasis. Thanks, Paul!)

I’m one of those computer programmers who downplays the difficulty of the profession, because “if I can do it, anybody can do it!” On the other hand, I have faced challenges that made me question whether I’m vaguely qualified for the job. What it boils down to is that programming is both incredibly simple and impossibly hard, like so many important things in life.

There was a time when nobody knew how to write literary prose. The geniuses who invented it shared their special tool with a few friends, and they relished in their private, elite communications. Eventually monks, politicians, and academics joined the club. Now, we judge a society’s overall level of intellectual advancement by the literacy rate: the percentage of people who have learned to read and write.

Literacy isn’t about becoming a Hemingway or a Chabon. It’s about learning the basic tools to get a job done. I think programming — coding — is much the same. You don’t have to be the world’s best programmer to develop a means of expressing yourself, of solving a problem, of making something happen. If you’re lucky, you’ll be a  genius, but you start out with the basics.

Long ago, it would have been ridiculous to assume a whole society could be judged by its ability to read and write prose. It feels ridiculous now, to assume that we might use computer programming as a similar benchmark. Yet it may happen.

Did you always mean to learn another language, but never did? By all means, learn Spanish, French, or Chinese. But learn to code, too.

And yet John and MG are totally right. You either see it or you don’t. This is egalitarian, relating to all facets of life, in every nuanced area of preference or priority. For whatever details a given person appreciates and values, far more people will be disinterested and be unlikely to even distinguish differences. How about those Android aficionados? They’ll point to the flexibility afforded by true multitasking, freedom to install unapproved apps, etc. They shake their heads at silly iPhone lovers, hold their phones up high and take pride in these qualities. To them, these are the finer points. This is the “polish.” The rest of us just don’t see it.

For many of us who make, use, or write about software for a living, polish is all about removing from the software as many jarring behaviors as possible. Sweating the small stuff. It’s exactly the details like the persistently stuttering scrolling that MG points out that continue to make Android products appear less polished to us. It’s seriously unnerving. It’s a big freaking deal to us, while other people just don’t see it.

It doesn’t have to relate to expense, and isn’t restricted to a premium class of product. It’s also, of course, not restricted to vision. I can imagine some of my wine-loving friends holding up a $15 bottle of something precious they’d discovered, while expressing disdain for a $200 bottle of swill that somebody else just adores. Nor does it need to be something “high class.” I’m sure a number of hard-working farmworkers could explain to me in agonizing detail why I picked the absolute worst rake and shovel for my garden.

If you’ve got a taste for something, a nose for something, an eye for something, an ear for something, a feel for something, and you find a product that soothes that sense, then you have a special gift: the ability to cast judgement on inferior efforts. Other folks? They’ll either sense it too, or they won’t.

Judging by the massive response on Twitter in support of her opinion, I think that most people tend to agree. However, I think that the vagueness of her description of the incident leaves much to the imagination, and is causing people to leap to condemnation of the talk, the conference, and the industry. I am not saying the condemnation is unwarranted, but my feelings about this particular event are complicated, and I think yours might be too, if you knew more of the details.

The talk in question was titled The Ten Dirty Words and How To Use Them. The talk was given by my friend Andy Lee, and his synopsis from the conference session descriptions reads:

In 2003, I came up with “The Top Ten Cocoa Words That Sound Dirty But Aren’t.” By finding APIs via this arbitrary way, we talk a random stroll through Cocoa, which can stimulate curiosity and lead to new discoveries and new questions. What would you guess NSInsertionPosition is for? (I incorrectly guessed text editing.) It can also be worthwhile reviewing familiar ground. We all autorelease — some of us every day — but it may still be possible to learn a thing or two about it. I will talk about the proper use of each word in the list.

To give you a more specific idea of the offensiveness of the API method names that Andy discussed, take a look at his blog post listing the API methods under discussion. [Update: MacTech has just posted slides from the talk.]

The genius of this talk is it takes a running gag in the Cocoa community, that occasional API names here and there were unfortunately named, and runs with that gag as a scaffolding for exploring the specific APIs in more detail. As for the sexual jokes, I think they basically write themselves in the individual minds of the audience. As anybody who has sat through an all-too-dry conference talk about the specific technical blah, blah, blah of any subject can attest, it is generally a good idea to inject some humor into a talk’s structure.

So was the humor in this case too much, or too vulgar? I’m sure that Brittany was not the only person in the room who was offended by the talk. On the other hand, as the one “comic relief” talk in a 3 day schedule that contained more than its fair share of professionalism, I think it’s probably fair to say that some members of the audience were relieved to have a chance to laugh about something.

Injecting humor into any talk is dangerous. Especially with a diverse crowd, you are liable to offend somebody. Jokes of a sexual nature are even more dangerous. Even if the joke is not sexist, per se, there is a strong possibility that members of the audience will take offense because of sexual taboos in society. I also imagine the discussion of sex in a strongly gender-imbalanced setting will make members of the minority gender more uncomfortable than the rest of the room.

But neglecting to inject humor is also dangerous. The Mac and iOS communities have a strong tradition of humor in our conferences. Many of us feel annoyed and cheated by a conference if there isn’t a bit of liveliness. So, it goes both ways. It’s important to process this incident carefully to understand how it went wrong. Was the problem that there was a session with a noticeably lower level of “seriousness” than the other sessions? Was it that the session’s jokes were of a sexual nature? Or was it that the sexual nature of the talk’s jokes were not made clear enough to the audience before it took place?

I think what is happening on the web now is many people are seizing onto the angle of Brittany’s complaint that most resonates with their own frustrations about the conduct of our community. This is a valuable reaction and a good opening for further exploration. But what isn’t useful is the large number of people who are condemning the conference and the speaker without significant information about the context of what happened. I hope that I have at least helped to clarify that to some extent. If you still feel that blanket condemnation is the appropriate response, then I’m happier with your opinion now that you’ve read mine.

C++ is a horrible language. It’s made more horrible by the fact that a lot of substandard programmers use it, to the point where it’s much much easier to generate total and utter crap with it.

Brent affirms his support while paying homage to plain-old C:

But I will admit to an enduring love of C. I still think of C not as C but as the language.

I loved C. Emphasis on the past-tense. As object-oriented programming concepts became popular, those of us who were programming in C or similar procedural languages had to find new, object-oriented languages to fulfill our needs. At the time, I chose C++. Or I should say, I had C++ forced upon me. Because C++ popularized the notion of object-oriented programming, it was the only choice presented to many programmers.

Because I ended up at Apple, the time I spent with C++ was thankfully short. They were fortuitously behind the times at the dawn of my career, and wound up taking a different path while my career was ascending.

Objective-C was Apple’s response to object-oriented programming, and continues to be the lingua-franca for programmers on Macs, iPhones and iPads. I loved C. I love C. But it always fell short for me. It lacked something. Objective-C fixed that. I hope I never have to program in C++ again. But tellingly, I also hope I never have to program in C again.

There are lots of great features from Python, Ruby, or JavaScript, that I’d love to see incorporated into Objective-C. By no means is it perfect. But for its elegance, and for the fact that it fulfills many of the requirements of object-oriented programming, while maintaining the familiar simplicity of C, it presently earns the title of the language for me.

[Update Nov 7: I've had a lot of reaction to my claim above that Objective-C was "Apple's response to object-oriented programming." This makes it sound like Apple invented the language, and they didn't. But they have done more to popularize and promote it than anybody else. I stand by the meaning of it being "what they bring to the table," when it comes to object-oriented programming].