Secure Shell (SSH) is a cryptographic protocol that securely transports data over an unsecured network (see RFC 4253).
The ssh-agent refers to a component of the OpenSSH software, which is used to cache (store temporarily) passphrases that are used to control access to ssh keys. Once a passphrase was cached, on subsequent use of the same key (within a given time), ssh-agent provides the passphrase to the authentication system, thus relieving the user of the necessity to enter the passphrase multiple times.
The SSH agent handles signing of authentication data for you. When authenticating to a server, you are required to prove that you are, well, 'you'.
Besides using passwords, this prove can be performed through a public/private key pair (commonly called the ssh keys) and asymmetrical encryption: The host sends a data packet (a so called challenge) to the client. This packet is encrypted that using the account's public key. The client decodes the packet using the private key an sends the result back to the server. If it matches the original data packet, the user proved that he owns the private key.
As a security measure most people sensibly encrypt their private keys with a passphrase, so that the private key can not be used by anybody who obtains the file. This means that any authentication attempt will require you to enter this passphrase. This can be tedious though (a typical developer may need to authenticate hundredths of times a day), so the ssh agent caches they key for you and you only need to enter the password once, when the agent wants to decrypt it. The key is held in memory, but is flushed after some time when it is not usesd (e.g. with a flush time of eight hours, the passphrase needs to be enterd only once per day).
In fact the SSH agent never hands these keys to client programs directly, but merely offers to decrypt the data packet on their behalf and returns the signed data. A side benefit of this is that you can use your private key even with programs you don't fully trust.
This will provide the user with a pair of files, e.g. id_rsa and id_rsa.pub. The pub file will then be added to the ssh server, while the private file remains in a folder on the user's computer (under linux this file will get file system permissions that makes it only accessible by the owner, e.g. chmod 600 id_rsa).
After generating the key, you can add it to ssh-agent via the folloing command:
Another feature related to the ssh-agent is agent forwarding. This applies to situations where you connect to a server via ssh from within another ssh session. E.g. you connect to server-A and on the shell you get there, type another ssh command to connect to server-B, rather than connecting to server-B directly from your computer.
In this situation, if ssh-key authentication is used and all hosts accept the same key, it would be necessary to store the private key on server-A as well (rahter than just keeping it on the user's workstation). The reason for this is, that when connecting to server-B, the server-A (which runs the ssh command that connects to server-B) will need the private key to decrypt the challenge (see above).
Agent-forwarding solves this problem by forwarding the challenge request back to the user's workstation, where the ssh-agent running there can sign the challenge on behalf of server-A.