A cell’s voltage will change with how much energy it stores, but if you keep applying current to force more charge to move you can cause voltages to be quite far outside of the proper range. However you don’t want to do this as at minimum you are damaging the materials in the cell, or worse, cause a significant safety hazard where the cell could catch on fire.

You can look at the Discharge Curve of a cell which compares voltage vs capacity, as a rule of thumb, essentially the steeper the curve changes, the more damage you are doing to the cell by operating in the range.