The same way they decide on every other kinetic action: Rules of Engagement and Rules for the Use of Force.
Also, self defense. Roughly: if someone is currently shooting you, you can generally shoot back to make them stop. I don't know of any reason that would apply with UFOs, but if the Martians attack tomorrow, self defense would be a factor.
Generally, Rules of Engagement (ROE) for United States forces are the rules around what you can and cannot do to accomplish some mission outside the USA. Wikipedia has a good summary, with an example (link). Generally for any deployment, you would have a blanket policy on ROE (like the example in the link), which supports international-law obligations and, perhaps more importantly, supports the overarching mission. I think there are weird corner cases where ROE could apply inside the US borders (like invasion, maybe?), but the good rule of thumb is that ROE is for outside the US. It is very easy to imagine that in something like Desert Storm, shooting down UFOs would be entirely at the pilot's discretion (or wing leader, or whoever has local command). In something like the Somalia Relief operation in the '90s, it would be very surprising if UFOs could be shot down.
The base instruction for this is CJCSI 3121.01, which establishes standing rules of engagement which apply whenever something more specific is not available.
Inside US borders there is a similar document, but with a different purpose. Where ROE are focused on legally and effectively achieving some operational or strategic objective, Rules for the Use of Force (RUF) focus on DoD use of force in a supporting role (like drug interdictions, where some Navy aircraft might be acting subsidiary to the Coast Guard). I am told this focus is to ensure compliance with the posse comitatus act, but I don't have any detailed insight. There is no underlying or cohesive framework for RUF, which makes it hard to provide a general answer.
In either case: ROE and RUF policy is controlled by the Joint Chiefs of Staff. Changes or augmentations to ROE/RUF can be proposed by local commanders and must be approved by SECDEF. It is exceedingly unlikely (and probably illegal) for RUF to give the military authority to shoot down weather balloons different from, say, whatever authority the Coast Guard has.
So:
If the UFO is attacking in a way that makes self-defense
viable and valid, the decision to shoot it down can be made very
locally.
If outside the US, consult the relevant ROE.
If inside the US, and operating as part of some other agency,
consult the RUF.
Outside the US, ROE tend to allow leeway for commanders to make decisions in line with force protection and their overall goals. Inside the US, I presume that anything other than strictly-defensive actions would need SECDEF approval, as it is essentially SECDEF policy that prevents engagement.
(I am unclear on what happens if SECDEF and the president disagree in a serious way, e.g. if Biden said "shoot" and Austin said "no". At that point, the UFO is probably a lower priority.)
@cpast corrected an error:
Minor correction: Per JP 3-27, air and maritime homeland defense
missions are under the ROE framework instead of the RUF framework.
Land homeland defense operations within US territory are RUF, but this
air mission generally wouldn’t be.
Additional reading here.