The computer has to be on while you are mining. As part of the way mining works, you need block headers as your input data. For solo mining, these block headers are computed from transactions and other blocks coming from other peers on the network, and you need to update that data frequently. (If another miner finds a block and you don't update your header to link to it, any block you find after that will be orphaned and likely rejected, making your mining a waste of time and energy.) For pooled mining, you need updates for similar reasons, except that the updated headers come from the pool operator.

For most such devices, collecting this data from the Internet is handled by the host computer, so it has to be on and running the whole time.

It is possible to use a small low-power computer for the host. One popular choice is the Raspberry Pi.