Tolkien was a life-long tinkerer -- it was his way of working and he was never satisfied with what he'd done. The only way anything he wrote could approach finality was when it was published -- and woe betide the publisher wanting a new edition!
Incidentally, this is why LotR and the Silmarillion are so much more canonical than the dozen or so volumes of collected notes that have been published. The notes are inconsistent with themselves in many ways, and stories and musing on background frequently exist in multiple versions, at most one of which matches the world of LotR and the Silmarillion.
C. S. Lewis gave credit to the Inklings for encouraging Tolkien stop rewriting and get LotR to the publisher, but he never claimed credit for the work itself, saying:
No one ever influenced Tolkien -- you might as well try to influence a Bandersnatch. We listened to his work, but could affect it only by encouragement. He has only two reactions to criticism; either he begins the whole work over again from the beginning or else takes no notice at all.
(My emphasis.)
It's also worth reading his "Leaf by Niggle" which is more than a bit autobiographical. Niggle is a painter who is (like Tolkien) a procrastinator who is obsessively painting a great tree, but must make every leaf a masterpiece and his life becomes devoted to perfecting this painting to the exclusion of everything else. Tom Shippey saw "Leaf by Niggle" as autobiographical.
If you read the latest book of Tolkien's notes, The Nature of Middle-earth by Carl Hostetter, you can see his obsessive tinkering at work as he tries, for example, to reconcile what he's said about Elves' lives and the nature of Arda with itself, leading him to consider huge changes to his chronology and world.
Personally, I can only say that Tolkien's obsessive artistry created a truly glorious book in Lord of the Rings, and Christopher Tolkien's restraint and clear judgement with the The Silmarillion salvaged a masterpiece out of the astonishingly scattered leaves JRRT left behind.
