The bitcoin client stores, in the wallet, the transactions that affect coins it manages. If it sees a new transaction, it checks to see if that transaction affects any accounts stored in the wallet. If so, the wallet is updated. Similarly, if the chain is reorganized, the wallet is updated.

However, the client doesn't check to make sure the wallet is synchronized to the current block chain. To fix this, the rescan command can be used. This causes the client, on startup, to go through every single block it has downloaded, search for transactions affecting accounts in the wallet, and update the wallet's transaction store and balances to reflect that transaction.

There are a few cases when you definitely need to rescan. For example:

If you restore a backed up wallet, it will be missing any transactions in blocks you downloaded after you backed up the wallet. Since the client won't receive those blocks (since it already has them) the normal wallet update logic won't fix this. You will need to rescan.

If you modify your wallet, say by adding keys to it or removing transactions from it, the wallet update logic won't see the transactions. So if you use a program other than the client to modify your wallet, you will need to rescan.

Theoretically, it shouldn't matter whether you download all the blocks and then rescan or vice-versa. The client won't download blocks while it's rescanning. And newly-downloaded blocks should update the wallet anyway. The logic is basically that this ensures the rescan process sees the blocks containing any missing transactions, rather than the regular block update logic. (They should both work. But if you need to rescan, we know something's wrong anyway.)

This is also useful if you are importing more than 1 private key into your wallet:

bitcoind importprivkey 1zbcvs.... "My Label" false
bitcoind importprivkey 1zbcvs.... "My Label" false
bitcoind importprivkey 1zbcvs.... "My Label" false
bitcoind stop

# One of the following
bitcoind -rescan 
bitcoin-qt -rescan