Actually, that is exactly what happens, and it is the reason why synchronizing the block chain is so slow: we download all blocks, with all transaction history in it, and build and maintain an index that states which transaction output was spent or not, and even where it was spent.

As of 0.8, the reference client will switch to a new model where no such index is maintained anymore. Instead, it will maintain a copy of the set of transactions, but only with their not-yet-spent outputs, which is much smaller than the full data (around 150 MiB as of writing, december 10, 2012).