We're proud to be recognized as a Leader in the 2024 Gartner®️ Magic Quadrant™️ for Collaborative Work ManagementGet the report
Note: The following assumes some knowledge of public key cryptography, and specifically RSA. You may wish to read up on this beforehand.
The global conversation that has accompanied Edward Snowden and PRISM has forced us all to recognize that eavesdropping by an unwanted third party is not just theory but reality. The voluntary shutdowns by Silent Circle and Lavabit highlight that service providers must think very carefully about the possible ways information might be extracted from their system, regardless of their best intentions.
Our customers’ data privacy is of utmost importance to Asana. Being a cloud-based teamwork app, we’re asking our customers to trust not only our software with their mission-critical data and communications, but also our infrastructure and our team. It’s important for us to protect all the channels through which data comes into Asana, in addition to protecting our infrastructure and datastore itself.
TLS (formerly known as SSL) is the technology used to encrypt most applications on the world wide web. SSL was the first way of providing security to web traffic. Secure is the ‘S’ in HTTPS. TLS adds x509 for authentication and delivers a symmetric key over the public-private key encrypted channel in order to speed things up. In most TLS connections the integrity of the server’s private key is the root of the security of the connection. This root is very strong and is the industry standard for exchanging sensitive information including banking information; however, if the key is compromised, whether by force of law or force of math, all sessions are exposed by that single key.
We’ve recently improved the security of Asana by making perfect forward secrecy available on all browsers that support it. We exchange a new secret for every HTTPS session. These secrets are not kept on our servers longer than they’re needed, so they’re more difficult to provide by simple legal request. Additionally, they each require a similar force of math to decrypt a single session that would have previously decrypted all captured customer traffic. It is important to place more barriers in the way of attackers, because compute power increases every day, and attacks that cost millions of dollars now may cost significantly less in the future.
Perfect forward secrecy uses Diffie-Hellman Key exchange to agree on a session key. Both parties begin with a public number and mix-in their own secret number. The mixed numbers are then exchanged, and then each mixes their own secret number in again. The order in which the two secrets are mixed in does not matter, so both parties end up with the same secret at the end. An eavesdropper only has access to the original number and the two mixed numbers that were exchanged. With this information its a difficult math problem to obtain the secret key. If you’d like more explanation of the math, head on over to wikipedia.
In addition to providing perfect forward secrecy, we’d like Asana to be as secure as possible against known attacks on TLS and be compatible with most browsers. When a TLS connection is started, both ends agree on what ciphers to use for the connection. The server has ultimate say of what cipher suite both will use, but this option needs to be turned on in most server software. Once we have control over the cipher suite used in a connection, we need to be able to make the best choice. This is where it gets tricky, as many of the things we want end up being contradictory.
We’d like to offer perfect forward secrecy to all browsers that support it, so our key exchange algorithm should be Diffie-Hellman (DHE or ECDHE), but we’d also like TLS to be performant. DHE key exchange is slow, so we should avoid it. In the end, we prefer the elliptic curve variant (ECDHE), but fall back to DHE and eventually to RSA in order to support older browsers.
We’d like to use RC4 as a stream cipher to mitigate the BEAST attack. This measure would protect some browsers who do not yet have the fix. However, we’d like not to use RC4 because academic attacks continue to reduce its effectiveness. In the end we chose to be future-oriented here. The BEAST attack will eventually be eliminated by a browser fix, which is implemented on Chrome, Firefox, IE, and on the most recent version of Safari distributed with OSX Mavericks. The current academic attacks on RC4 are not going away, and TLS connections that use RC4 will only get weaker as more attacks are found.
On our front-end, this was fairly simple to deploy. Our load-balancers use Nginx to terminate TLS, and then pass the unencrypted connection on to HAProxy. HAProxy in turn delivers it to a web server. We have eight cores on the load balancer, and Nginx spreads the TLS load out over seven cores, while HAProxy runs on the remaining core.
Forward secret ciphers only resulted in ~20% more cpu time spent on TLS per request. In order to get this performance, we enabled SSL session caching in Nginx so that most connections do not incur a full SSL handshake, but all SSL sessions expire after ten minutes. The increase in CPU usage can be a road-block to deploying TLS forward secrecy, but we believe the additional security and product confidence is well worth this overhead.
Our load balancers run the long-term-support version of Ubuntu, and the included OpenSSL library has all the features we want. This made implementation simple. We settled on an Nginx configuration with a cipher string based off this excellent post by Hynek Schlawack, which offers the best trade-off between compatibility, security, and speed.
There were many tools that helped us iterate and test this configuration. Qualys has an excellent tool for evaluating the security of an SSL installation, and a great tool for seeing what ciphers your browser supports. Their articles are an excellent jumping off point for further research and understanding. And, we’d like to give special thanks to Hynek Schlawack again, for his work summarizing the security issues facing those who deploy TLS.
The security landscape is always evolving, and we’ll be revisiting this configuration on a regular basis to ensure we’re choosing the best options available. If you see any issue with our site’s security we’d like to know about it. Contact us at [email protected]. If you’re a security professional, please check out our Bug Bounty Program.