If I open an account on, say, mtgox, can I immediately buy bitcoins and if I can, can I then transfer the very bitcoins I just bought to my own personal wallet (the one that I'll generate later on offline and put its private part in a safe).

Yes, that's a basic Bitcoin withdrawal.

As a side-question: where are the bitcoins that I buy on mtgox stored? In a wallet belonging to mtgox?

Yes, exactly.

You have to verify your identity and address with Mt. Gox and Bitstamp before you are able to withdraw coins or USD. BTC-E does not have that requirement for small amounts but it is a lot harder to get money into. So overall there is no "quick" way to get into bitcoin. You have to put in the time to get verified and also to learn what bitcoin is and how it works. It is unlike any other currency so if you don't know how it works you are liable to loose a lot of money. For example exchange accounts get hacked all the time so you never leave a large amount there and using an online wallet like blockchain.info is also asking for trouble.