In the United States, by far the easiest way to get up and running is with Coinbase. It's very easy to get up and running - anyone who has a bank account can buy bitcoins with Coinbase by providing them with a bank account number and routing number. Aside from being a Coinbase customer, I'm not affiliated with them.
Since this is how I bought my own bitcoin stash in the past, here's what you need to do to buy bitcoins on Coinbase:
1) Register an account on Coinbase.com.
2) Provide your bank account number and routing number and wait a few days for them to send you a small amount of money to your account so they can verify that you actually control it.
3) Confirm the transactions they sent.
4) Decide how many bitcoins you want to buy, and initiate the transaction from Coinbase.com.
That's it. The bitcoins will arrive in the account about five business days later, if my transaction with Coinbase is typical.
I have to address what you said about Bitcoin being anonymous, however. Bitcoin is not anonymous, it is far less anonymous than cash in that all transactions are stored forever in the blockchain. In that sense, it is far less anonymous than cash since with bitcoin, if someone knows your bitcoin address, they can track what you spent your bitcoins on, what the recipient spent your bitcoins on, and so on. Unless you use a mixer, bitcoins are definitely not anonymous.
4Bitcoin is not anonymous, all transactions are public. The only thing private is your identity behind your addresses. So if you send or receive bitcoin to someone who know who you are they will able to trace the transactions you do but with only that address. You can get anonymity if you mix the address you use. – Gopoi – 2013-01-26T20:40:11.273