A place to keep my thoughts on programming

August 12, 2012 geek, Happenstance

Key management strategies for Happenstance

Fundamental to the design of Happenstance is the idea that everything you publish, messages and meta-data, is cryptographically signed. The need for the signature stems from the distributed nature of the system. I.e. as messages are published and copied to any number of aggregators to be included in any number of feeds, there needs to be a way to know that the message you are reading came from the claimed author, without having to read all messages directly on the author's feed.

The challenge this brings with it, especially when striving for consumer adoption, is that the goals of accessibility/usability and security can be at odds. For this reason, I've designed (and am currently refinining/revising) the use of public key infrastructure (PKI) in such a fashion that private key compromise does not fatally affect the integrity of an author's identity.

A full explanation of how content is signed can be found in the Signing Spec. The short version is that feeds provide RSA public keys, and all message blocks are signed with an RSA private key in base64-encoded RSA-SHA256 format creating a _sig key in the message block like this:

  _sig: {
    name: 'key2',
    sig: 'xfZ4DmrcLbz8qPJoTwYg ...'

The pitfalls of signing content in a service environment

In order for content to be signed, a private key has to be used. If anyone else gets their hand on your private key, they could fake messages appearing to come from you. While the fakes won't hold up to scrutiny, as I will explain below, it's still a trust issue. So the private key needs to be protected. Before I get into how a compromised key can be dealt with, let me explain why PKI for Happenstance may violate normal best practices for dealing with private keys, creating greater risk of compromise.

Ideally only you would have your private key, preferably encrypted and only decrypted in memory for the purpose of signing. That implies that all authoring has to happen client side. For some clients and users that is certainly feasible, but for non-technical users used to web applications the added level of complexity may be too cumbersome. These users have come to expect the experience provided by walled garden services with their implicit trust via authority (not that that trust can't be and hasn't been compromised).

Using a service to author and sign your messages will expose your private key to attack vectors one way or another. But since a hosted authoring experience can remove the complexity of key management, I've tried to enumerate some approaches to reduce the dangers, while simultaneously designing the ecosystem to survive the compromise of keys.

Service stores user encrypted key, sends it to the client for signing

Note that all strategies assume that communication with the host always happens over HTTPS.

The first option keeps the key encrypted by the user's passphrase at the server and sends it to the client at authoring time. The client side application can then prompt the user to decrypt it, sign content and send the signed content back. Basically, the authoring experience is implemented on the client, but hides the complexity of key management.


  • Need the password for every message (could cache in memory, which has dangers of its own)
  • Need a client that can handle encrypted RSA keys and can sign with them. At the time of this writing, I've found javascript for signing but only via unencrypted keys. I'm sure this could be addressed.
  • The encrypted key is frequently sent over the wire and its use by the client application may be exploited
  • If the service is compromised, the passphrase for each key stored at the server must be cracked, which isn't outside the realm of processing power these days.

It should be noted that this is the only approach in which the service provider might be considered untrusted, since they never get the passphrase. However they still have the key, so they could crack it or compromise the client authoring experience (since they provide it) to capture the key, so the service provider still needs to be trusted.

Service stores user encrypted key, requires password at signing

This approach moves authoring to the server, which means that the passphrase to decrypt the key must be sent to server. This makes the client a lot thinner. Less moving parts, less chance for errors or attack vectors on the client and once again the complexity of key management is hidden.


  • Need the password for every message (server could cache in memory, again introducing new dangers)
  • Passphrase is frequently sent over the wire
  • If the service is compromised, the passphrase for each key stored at the server must be cracked, which isn't outside the realm of processing power these days
  • Even worse, if the service is compromised, it could be made to capture the passphrases as well

You might think you could just use the password the user uses for login as the key passphrase, but in a properly designed system, the login password is stored in a one way hashed form, i.e. the service has no way to "decrypt" the password, it just hashes the incoming password and compares hashes. The private key, however, even though it is part of an asymetric encryption scheme is itself encrypted symmetrically, i.e. you need the same passphrase you used to encrypt the private key to decrypt it in order to use it. So the key's passphrase must be available for every signing event.

Service stores key without user encryption, signs on behalf of user without additional authentication

This final approach is the most user friendly but wrests the control over the private key from the user. The server keeps the key in a form in which it can use it to sign messages on the behalf of the user without any additional authentication. The entire key management mechanism is hidden.


  • Complete reliance on the service to protect the key in case of service compromise

This one seems to require more trust in the service provider, but in reality each version is vulnerable if the service is a bad actor or compromised. However this last method removes all need for the private key to ever be communicated between client and server. While the security of the key is now completely in the hand of the service provider, much like credit card storage, there are established practices for securely storing and using such data, such as via some form of one-way vault (usually a dedicated server not on the public network that accepts keys and can sign messages, but has no mechanism for retrieving the keys). Given that Happenstance allows for multiple keys and key expiration, a service provider that never divulges the private key to the user is likely to be the most secure and convenient approach.

Mitigating the effects of compromised keys

Since all methods (even you storing your own key and never giving it out) can be compromised, the focus really should be on mitigating the effects a compromise can have. Since Happenstance does not actually encrypt data to hide it from prying eyes, but only uses it authenticate data in the clear, a compromised key's only use is for impersonation.

A compromised key could be used to push fake posts into the ecosystem in one of two ways: Create mentions to deliver messages directly to users impersonating the owner of the key, or by "re-posting" a faked post to their followers. Fortunately, a compromised key does not mean that the user has lost control over their feed and the user could simply generate a new key and remove the existing one from their feed meta data. This way any aggregator would immediately detect it as a fake when validating incoming messages.

The side-effect of key revocation is that all messages currently in flight and older messages being re-checked would also become invalid. To guard against this, a message failing validation should be re-fetched via its canonical uri and be re-validated. This way, upon removing a compromised key, all previously signed messages can be re-signed with the a new key.

Multiple keys and dealing with loss of access to a key

Because loss of access to a key or compromise through personal or service provider negligence are realities that cannot be fully eliminated, it is important that Happenstance messaging can mitigate the loss of a key without compromising identity.

Keys being compromised isn't the only way that an author might have for discontinuing the use of a key. A service provider could never have given out the private key (as suggested above). Or a service provider went out of business and the status of their key storage is unknown. Or the author forgot their passphrase, or lost the only back-up of a key they used in client side authoring environment. etc.

In addition a user may have need for using multiple different keys due to using multiple authoring environments with different private key strategies. To support multiple keys, public keys are represented in the feed meta data like this:

  public_keys: {
    key2: {
      key: '-----BEGIN PUBLIC KEY----- ...'
    key1: {
      key: '-----BEGIN PUBLIC KEY----- ...',
      expired: '2012-07-30T11:31:00Z'

Revocation of a key is simply the removal of the key from the author meta-data public_keys hash and re-signing all content signed by that key. Alternatively, a key can also be expired with the expired key, meaning that any message signed by that key after that time should no longer be trusted.

Expiration is best for keys that are no longer accessible, rather than compromised, since faked messages with old timestamps could be created.

You seriously want me to treat my keys as expendable?

Hopefully those better versed and more serious about the application of PKI infrastructure haven't stopped reading before this. I know that as a PKI best practice it does go against established norms for guarding private keys.

So to answer the above question: No, I do not. Happenstance is a spec and the implementation details are completely open. I'm simply providing strategies that I personally consider secure enough and which do not hinder broad user adoption. But it does not preclude anyone from creating authoring environments that only work on local, encrypted keys that are never shared. That's the point of Happenstance being broken up into small interoperating parts rather than being a large platform with lots of hard API requirements. Let the ecosystem and users decide which implementations strategy best suits their comfort level.

Leave a comment