So, I'm opening up this question to everyone to answer. How far back should lore go? Is it dependent on the story, or is there a set average? How do you know you've gone too far? Alternatively, how do you know when you don't have enough?
Ok, it really depends on the type of story you're writing. My answer is not contained to book narratives, given my "specialty" is game narratives, so this is suitable for any story you wanna write in any media :) Also, this is just MY OPINION, not a strict set of rules
It depends on the focus of the story. Ask yourself: is my story character-based or plot-based? I like to use the 70-30 rule, focus on producing 70% of the story based on one and 30% on the other, for example, keep 70% of your writing about the characters and their inner worlds and 30% about the world they live in, or vice-versa.
If it's character based, it makes for a better story to keep the lore short, and add the mystery spark in the lore and plot of the world, since you wanna develop and showcase the internal world of your charaters, therefore, cluttering the story with lore and a heavy outer plot might make it too complicated and hard to follow, the reader/player might get confused with so many timelines and pieces of information seemingly unrelated being woven together. In this case, develop what is necessary, no extra embellishments needed, for example, "centuries ago" is relevant to the story, but "when humans first inhabited this land" is prooooobably not.
The opposite is equally valid, maybe the sense of wonder you want to evoke is through an ethereal, other-worldly universe, with stable languages and religions, consistent laws and magic and determined social structures. In this case, you DO want to create a long set of lores, and keep in mind you should start with at least 1 millennium ago. That, or what is enough to explain the "why" your world is how it is, a good rule of thumb is to ask:
-why is the society in this world the way it is?
-why are the religions in this world the way they are?
-why is the magic in this world the way it is?
-why is the law in this world the way it is?
-why is the tecnology in this world the way it is?
-why are the clothes in this world the way they are?
If you prefer, replace "why is ___ the way it is?" with "how did ___ become this way?" if it's easier to answer this way!
in this scenerio, your characters don't have to have a super detailed backstory, just enough to make them drive the plot and not be 2-dimensional. You don't have to tell people about how your main character's abusive great-grandma shaped their grandpa to be abusive which made their mom want to stop the cycle and then shaped your MC to be a world-saver and that's why they're the MC, unless that abusive great-grandma was the queen who started the war between MC's country and another country, and therefore changed how traditions were carried and had an effect on the enviroment around your MC.
To make it clear, NONE of these are rules, they are just my take and advice, there's no "right way" to write Lore. I hope this was helpful somehow :]