There are several definitions with overlapping meanings.
The first is perhaps best called extinct blocks. These are blocks that were produced by building on an block that is no longer the active tip of the chain. Some nodes may have considered it to be the best block at some point, but they switched to another chain which does not contain the relevant block anymore. They are valid, verified, and their ancestry up to the genesis block is fully known - they're just not currently 'active'. They are sometimes called stale blocks (typically in the context of mining software realizing it built on old data) or orphan blocks. The latter name originates from the fact that payouts from extinct blocks are denoted as "orphaned" in the reference client (referring to the fact that their coinbase transactions are now orphaned).
However, there also exist real orphan blocks, with orphan in its original meaning of "having no parent". These are blocks received by a node that does not have its entire ancestry (yet) and thus cannot be validated. Nodes keep such blocks in memory, while asking their peers to fill in the gap of their history. The client does not show these, so when people talk about orphan blocks, they are most likely referring to extinct blocks. Note that since Bitcoin Core v0.10, there are no such orphan blocks anymore, due to a significant change in the download mechanism.
Terminology is confusing here :)
Got to this thread from Twitter. My suggestion is "lonesome block". – Phuoc Do – 2019-08-29T18:52:08.997
Nice explanation Pieter, could you please elaborate on details of current download mechanism that Bitcoin Core uses? I wonder how it solves orphan block problem. – eugenekr – 2018-03-19T16:46:17.147
4@eugenkr The headers are downloaded and validated first, before the client even requests block data. As a result, it will never receive blocks whose parents it doesn't know about. – Pieter Wuille – 2018-03-19T17:11:17.457