Please do not use a web wallet if you are planning on sitting on your coins for 10 years. When you send coins to a web wallet or to any address you did not generate yourself, you are giving the spending power to another person. Just like when you deposit $100 in the bank they will use it for other things and in most certainty return you a different $100 should you go and withdraw it.
Download the original bitcoin-qt client and open it, it will begin to download blocks, but this is not important if you are planning on just sitting on your coin. On the first run the client will also generate some addresses for you, showing you the first one on the "Receive" tab. You can have your newly purchased coins sent to this address.
There are plenty of security steps to safeguard against loss, please read carefully and understand that if you lose access to your coins or if a mischievous person gains access to them, they will likely be gone forever.
It is strongly recommended you encrypt your wallet asap. Please pick a very long passphrase you can remember, at least 6 to 8 words. Be aware that forgetting this passphrase will likely result in the permanent loss of your coins (there is no "Forgot Password" option). If you have strongly encrypted your wallet, then it is pretty safe to put it in as many places as you can, since cracking a random 6 to 8 word password would take an unreasonably long time using traditional password cracking methods.
You can find your wallet file on windows by opening the file explorer and typing %appdata% in the path (where you usually see c:\user\documents, etc), then selecting the folder named "Bitcoin". Inside this folder you will see a file called wallet.dat, this file can be placed in this folder on any other computer, giving this computer the ability to see and spend the coin associated with address on this wallet.
Should you decide not to encrypt your wallet, you can still copy it to a USB and put it in a safety deposit box, or somewhere else secure, but please be aware that any person who has access to an unencrypted wallet.dat can easily load it to their computer and send your bitcoin off into cyber-space.
1I would advice against online wallets for long time storage. Too risky. OP would be much better off with a wallet on a usb stick. – jgyou – 2013-12-31T23:11:50.363
I've downloaded bitcoin qt for windows from sourceforge. I'll try following the instructions for storing on flash drives, in "Bitcoin Explained Like You’re Five: Part 4 – Securing Your Wallet" on http://chrispacia.wordpress.com/2013/09/29/bitcoin-explained-like-youre-five-part-4-securing-your-wallet/ Please say if I've got something wrong or if there's a better way. I might also get a hardware wallet later as they seem more foolproof.
– coinfused – 2014-01-02T15:28:21.170You should be good! – jgyou – 2014-01-02T22:09:03.843