Probability might seem like a simple concept, there there are different probability interpretations. The frequentist school believes that chance is objective, and the Bayesians say that probability is just a measure of someone's subjective belief.
The frequentists say that they are more scientific because they are more objective. The Bayesians say that they are more scientific because they more fully use the available info.
Mathematically, the whole idea of chance is a useful fiction. It is just a way of using formulas for thinking about uncertainty. There is no genuine uncertainty in math. A random variable is just a function on some sample space, and the formulas are equally valid for any interpretation.
Coin tosses are considered random for the purpose of doing controlled experiments. It does not matter to the experiment if some theoretical analysis of Newtonian forces on the coin is able to predict the coin being heads or tails. The causal factors on the coin will be statistically independent of whatever is being done in the experiment. There is no practical difference between the coin being random and being statistically independent from whatever else is being measured.
It is sometimes argued that radioactive decay is truly random, but there is really no physical evidence that it is any more random than coin tosses. We can measure the half-life of potassium, but not predict individual decays. According to our best theories, a potassium-40 nucleus consists of 120 quarks bouncing around a confined region. Maybe if we understood the strong interaction better and had precise data for the wave function, we could predict the decay.
The half-life of potassium-40 is about a billion years, so any precise prediction seems extremely unlikely. But we do not know that it is any different from putting dice in a box and shaking it for a billion years.
All fields of science seek to quantify counterfactuals, and so they use probabilities. They may use frequentist or Bayesian statistics, and may debate which is the better methodology. Only quantum physicists try to raise the issue to one of fundamental reality, and argue whether the probability is psi-ontic or psi-epistemic. The terms come from philosophy, where ontology is about what is real, and epistemology is about knowledge. So the issue is whether the wave function psi is used to calculate probabilities that are real, or that are about our knowledge of the system.
It seems like a stupid philosophical point, but the issue causes endless confusion about Schroedinger's cat and other paradoxes. Physicist N. David Mermin argues that these paradoxes disappear if you take a Bayesian/psi-epistemic view, as was common among the founders of quantum mechanics 80 years ago. He previously argued that quantum probability was objective, like what Karl Popper called "propensity". That is the idea that probability is something physical, but nobody has been able to demonstrate that there is any such thing.
Max Tegmark in the March 12, 2014 episode of Through the Wormhole uses multiple universes to deny randomness:
Luck and randomness aren't real. Some things feel random, but that's just how it subjectively feels whenever you get cloned. And you get cloned all the time. ... There is no luck, just cloning.There are more and more physicists who say this nonsense, but there is not a shred of evidence that anyone ever gets cloned. There is just a silly metaphysical argument that probabilities do not exist because all possible counterfactuals are real in some other universe. These universes do not interact with each other, so there can be no way to confirm it.
Scott Aaronson argues that the essence of quantum mechanics is that probabilities can be negative. But the probabilities are not really negative. The wave function values can be positive, negative, complex, spinor, or vector, and they can be used to calculate probabilities, but those probabilities are never negative.
There is no experiment to tell us whether the probabilities are real. It is not a scientific question. Even tho the Bayesian view solves a lot of problems, as Mermin says, most physicists today insist that phenomena like radioactive decay and spin quantization prove that the probabilities are real.
Quantum mechanics supposedly makes essential use of probabilities. But that is only Born's interpretation. Probabilities are no more essential to quantum mechanics than to any other branch of science, as I explained here.