"Too loud", "too quiet" opinions: what are the factors?

Sorry if the post is off topic for this forum.
I have put a player in a store a few weeks ago. People who work there keep changing the volume on the player although it uses track replaygain and plays at random from a specific library.
Not only they change volume but they have one single complain about the system: it's either too soft or too loud. People's opinions vary in time: it's not always "too loud" for the same person.

My personal take on this is that the ambient noise in the store varies "a lot" between moments there are little customers and rush hour, while the player keeps pushing audio at the same volume; if this is true, I think something like an ambient microphone would help adjust player volume automatically.

However I have no idea how I would make that really work, and before starting I'd like to verify if this is the reason for these people's swinging opinion about music volume.
Hence my question: what makes people say music is too loud or too soft?

I hope this is clear and not off-topic. If so I am eager to get your inputs.