SYN cookies are a method by which server administrators can prevent a form of denial of service (DoS) attack against a server through a method known as SYN flooding. This type of attack utilizes the process by which a connection between a client and host is established, known as a three-way handshake, to cause the host to have an excessive number of client requests, freezing or crashing the system. Such attacks have largely become outdated, however, through methods such as the use of SYN cookies that circumvent them. These cookies do not present a security threat or risk to either the host or clients and do not cause connectivity issues or problems.
The way in which SYN cookies function is built on the basic way by which many servers and users, or host and client systems, connect to each other. This process is known as a three-way handshake and begins when the client system sends a request to connect to the host system. The request is called a synchronize message or SYN and is received by the host system. This host system then acknowledges that the SYN has been received by sending an acknowledgement, or SYN-ACK message, back to the client.
Once the client system receives this SYN-ACK message, then a final ACK message is sent back by the client to the host. When the host system receives this final ACK, then it allows the client to access the system and can then receive additional SYN requests from other clients. Most host servers have a fairly small queue for SYN requests, usually only eight at any one time.
The form of DoS attack known as SYN flooding uses this to overwhelm a host system. This is done by sending a SYN message, to which a SYN-ACK is sent by the host in response, but the final ACK message is not sent by the client, keeping a position in the queue open. If this is done properly during a SYN flood attack, the entire queue becomes occupied by these unanswered requests and is unable to accept new requests from legitimate clients.
SYN cookies help circumvent this type of attack by allowing a host to act as though it has a larger queue than it truly has. In case of a SYN flood attack, the host can use SYN cookies to send a SYN-ACK to a client, but it eliminates the SYN entry for that client. This basically allows the host to function as though no SYN was ever received.
Once this SYN-ACK with SYN cookies is received by the client, however, the corresponding ACK sent back to the host includes data regarding the original SYN-ACK. The host can then use this ACK and the included SYN cookies to reconstruct the original SYN-ACK and the appropriate entry for that original request. Once this is done, the client can be allowed to connect to the host, but the entire process effectively circumvented the queue that may otherwise be occupied by a SYN flood attack.