Batteries have three elements - an anode, a cathode, and an electrolyte.
The anode is the "-" electrode, the cathode is the "+" electrode, and the electrolyte is in-between. The word "electrode" means an electrical conductor where energy leaves or enters the battery.
In a battery, chemical reactions cause electrons to gather at the anode. Because like charges repel like charges, these electrons want to find fewer electrons and are driven through the circuit to the positive terminal in the battery (aka the cathode). They could just pop back through the battery to the cathode, but the electrolyte prevents that from happening.
When a battery runs low and then goes dead, it's because the chemical process that is creating the charge has run its course and converted the original chemicals, and the battery is depleted.