TLS/SSL uses asymmetric encryption algorithm for authentication and deriving the session key and symmetric algorithm to encrypt the data for its speed. For the user data to go through the encryption tunnel, a TLS handshake must take place to authenticate the peer and generate the common session key for data encryption. The diagram below describes how TLS negotiation works at the high level:
Client Hello is the first message sent by the client to the server in the TLS/SSL session setup sequence. It typically contains the ciphers and extensions supported by the client.
In response to Client Hello, the server sends back the following messages:
- Server Hello: Contains the cipher the server picked from the list provided by the client based on its preference.
- Server Certificate: Contains the server’s certificate and its CA if configured so.
- [Client Certificate Request]: Optionally, the server can request the client certificate for authentication, which usually is not used.
- Server Hello Done: Concludes the server-client handshake.
In response to Server Hello, Server Certificate, [Client Certificate Request] and Server Hello Done, the client sends back the following messages:
- [Client Certificate Request], [Certificate Verify]: If the server requests the client certificate, the client will send its own certificate and a Certificate Verify message which is a signature over the previous handshake message using its certificate related private key.
- Client Key Exchange: Usually contains a pre-master key which is encrypted using the server's public key obtained from its certificate.
- Change Cipher Spec: A message to notify the server about the start of data authentication and encryption.
- Finished: A message encrypted with the new key is sent to determine if the server is able to decrypt the message and the negotiation was successful.
In response to [Client Certificate], Client Key Exchange, [Certificate Verify], Change Cipher Spec, Finished, the server sends back a Change Cipher Spec to confirm the start of data authentication and encryption. The server also sends its own Finished message encrypted using the common session key. If the client can read this message then the negotiation is successfully completed.
From now on, all the communication between the client and server is encrypted.
The "client" and "server" described above are roles in a specific session. The same device may change roles in different sessions. For example, when the FortiMail unit receives email from either a client or another sending MTA, the FortiMail unit acts as the TLS server. When the FortiMail unit relays email to the next hop receiving MTA, it acts as a TLS client. Nonetheless, some applications always act as a TLS client or server, but not both. For example, a web browser always acts as a TLS client and a web server always acts as a TLS server.