This is up to the DM
Interestingly enough, this is one of those cases where I think "I'm sure there's something in the rules for interacting with objects regarding adventuring gear!" but the rules are strangely silent. The only text for bullseye lanterns specifically is
A bullseye lantern casts bright light in a 60-foot cone and dim light for an additional 60 feet. Once lit, it burns for 6 hours on a flask (1 pint) of oil.
The other texts for lamps and hooded lanterns, and even torches, are not helpful either. The only slight clue is torches saying
If you make a melee attack with a burning torch and hit, it deals 1 fire damage.
You would need to be holding a torch to do this, but lanterns don't have this feature so ultimately this is unhelpful.
The section on Interacting with Objects is equally silent, in fact more or less states that it is up to the DM.
A character's interaction with objects in an environment is often simple to resolve in the game. The player tells the DM that his or her character is doing something, ... and the DM describes what, if anything happens.
If you are in combat, it would require the Use an Object action to light and attach the lantern to your belt or whatever. Whether that would be two separate Use an Object actions would be up to the DM.
So, at the end of the day, it is up to the DM whether or not you can. In my opinion, and from the real world, this will not break anything. You need a free hand to actually light it, but beyond that it simply produces light. Having it on your belt might make it sway a bit more, but not enough to really matter unless you're sprinting, which would be its own edge case to deal with as a DM.