The problem that mining solves is the problem of providing secure transactions without a central authority. There is value to solving these problems because otherwise, there would be no way to securely exchange Bitcoins.

Bitcoin uses proof of work as its means of solving the double spend problem without a central authority. If I try to send the same Bitcoin to two different accounts, the miners choose one transaction or the other and use it as the basis of the computations they do. Because nobody else can outcompute all the Bitcoin miners, this makes it possible for others to determine which transaction "wins". Without this mechanism, there would be no way to resolve conflicting transactions.

Because this proof of work costs money to do, it's not likely that it could be accomplished without some incentive. Mining rewards provide this incentive.

Miners are essentially putting a notary stamp on a batch of transactions. That's all they are needed for.

But how do you prevent from having a corrupt notary? Bitcoin does this by having tens of thousands of potential notaries and one of them will happen to be the lucky one that gets to do the stamp. The lucky one is the one who happens to solve the problem. All the potential notaries try to solve the puzzle over and over, but it will take about ten minutes for one to become successful.

Others will see the solution, and if it is valid, will continue on to the next puzzle. This builds a chain of batches.

So to go back in history to change the batches of transactions requires having to redo all the problem solving attempts that occurred previously. The further back you want to go, the more problem solving work you need to perform.

So the problem being solved is a system that makes it enormously expensive to change history. That's the job that miners are paid to provide.

No, no-one is paying for the solutions to the math problems. In fact, they are only math problems in the same sense that watching a Youtube video is doing a computation.

Miners are checking transactions of other users. The Bitcoin protocol is built in such a way that this process sometimes creates new bitcoins. The creation of new bitcoins is also regulated by the protocol - only 21 million can be created, and the last Bitcoin will be created around 2140. The creation of coins is distinct from the value of the coins, which come from their usefulness, supply and demand.

There is another cryptocurrency, Primecoin (XPM), which uses calculation of prime numbers in the "proof of work" process of the currency. These prime numbers do have potential scientific value, but the value of Primecoin is not linked to the value of the prime numbers. The prime numbers are publicly available, and value again comes from supply, demand and usefulness. The goal is that the computation involved in "proof of work", which must be done anyway, should have a side-benefit.

Mining essentially tries to brute force decrypt an encryption. This is not so much a complex mathematical problem as rather systematically trying a multitude of potential solutions until one fits the current prerequisites (context and difficulty).

The purpose of mining is to validate the transactions of the network. The more computational effort is applied to mining, the more resilient is the network against attacks. Therefore the Bitcoin protocol has been designed to dispense rewards among the miners to incentivize behavior that is beneficial to the network – whoever finds the winning solution is allowed to transmit a number of newly created coins to an address of his choice.