Assuming the miner as an up-to-date version of the blockchain, he can compute the difficulty from the data about the previous blocks. An explanation of how the difficulty is computed can be found in this other answer.

Given a specific chain, indeed, the computation of the difficulty always gives the same result. Every miner may compute this new difficulty by himself and the result will be the same since the computation is based on the 2016 blocks slot. There is no need to know the hashrate of the network, it is enough to know how much time it was needed to generate the 2016 blocks by looking at their timestamps.

Additionally, block headers also encode the target threshold through the nBits field. The reason can be found in this answer.

PS: In order for a block to be considered valid, its timestamp must also be valid, that is, it must be:

greater than the median timestamp of previous 11 blocks, and less than the network-adjusted time + 2 hours.

So the effects of the "attack" are very constrained.

Because the difficulty calculation is deterministic, everyone can calculate the difficulty independently. Given that they have come to consensus on the blockchain (which blocks in which order), they should all calculate the same value. In this way, the protocol never relies on trusting anyone about what the correct difficulty is.

Also, keep in mind that it's not just miners that would have to be fooled into accepting a block with insufficient work. Full nodes also verify blocks as they receive them, and will reject blocks that do not have nonces that meet their own calculated difficulty requirement. Since full nodes play an important role in relaying blocks, this should not be overlooked.

If you'd like to look at how the difficulty calculation is performed, it's in the code at: https://github.com/bitcoin/bitcoin/blob/master/src/pow.cpp

To address the remark in the PS, i.e.

P.S. Assume also an adversary solves the proof-of-work and then generates a new block but forges the block's times-stamp to trick the network at the time of updating target. Since the target will be updated using the blocks' time-stamp.

That is indeed possible, it's called "timejacking" or the time traveller attack, you can find out more about it here:

http://culubas.blogspot.de/2011/05/timejacking-bitcoin_802.html

or here:

bitcoins from the future

It's done through consensus. You can read about how it's calculated and also look at the source code as well. You can find the answer in the developer's guide: https://bitcoin.org/en/developer-guide#proof-of-work

I believe the actual code is in bitcoin/src/chain.h

Excerpt:

New blocks will only be added to the block chain if their hash is at least as challenging as a difficulty value expected by the consensus protocol. Every 2,016 blocks, the network uses timestamps stored in each block header to calculate the number of seconds elapsed between generation of the first and last of those last 2,016 blocks. The ideal value is 1,209,600 seconds (two weeks).