Wallets are used to store crypto off of third party "accounts", allowing you to HODL, or bank, your own money.
Crypto wallets come in many forms. Your "account" on an exchange or with a financial company such as PayPal or Fidelity Investments can be considered a wallet, because the crypto ledgered there is held there for your benefit. However, unlike in banking and investments, crypto allows you to move control of your money instantly off exchanges or to receive money from third parties directly to your own software or hardware wallet address.
Actually, because the blockchain is an electronic ledger that simply moves coins between owners identified by their electronic “keys”, coins never actually leave the blockchain. Setting up a hardware or software wallet establishes private keys, which allows access to the coins associated with the corresponding “public keys” on the blockchain. As long as the wallet that holds the keys is kept safely, the private keys of the wallet owner cannot be used by anyone else to access the coins assigned to their public keys on the blockchain.
Typically, hardware wallets are more secure than software wallets, because they can be disconnected (air-gapped) from any network, thus allowing you to keep your private keys safe from hackers. Of course, the physical loss of a wallet can lead to loss of access to your coins, but careful setup of your wallet and safe storage of recovery phrases used to configure the wallet will minimize that risk.