There's another aspect to the supply and demand problem that hasn't been discussed yet.
In 2020, the demand for bikes and associated items ramped up because of COVID and the lockdowns. At this point, some companies ramped up production by a lot. Some held back, reasoning that demand would revert to normal-ish levels after the pandemic.
Wahoo is definitely one of the companies that did the former. However, in 2022 and 2023, their debt ratings were repeatedly cut, and they defaulted in 2023 (they reportedly restructured their debt, terms not public, and are continuing operations). I think this is part of why we are seeing some steep discounts on Wahoo smart trainers (although competition from the Zwift Hub is also part of it).
My understanding is that Shimano took the latter course and held back on expansions. Thus, there has been a general shortage of Shimano parts. Their largest OEM customers get priority on parts allocations, which definitely seems unfair to everyone else, but is probably inevitable in business.
The OP's situation may be further complicated by being on a rim brake bike. Shimano did commit verbally (in interviews after the launch of their latest generation of groupsets) to producing 11s 105 and Ultegra parts for some time. With these groups, you have a split between the rim and hydraulic disc groups, so they have to cover two combinations of shifters and brakes. And new rim brake bikes are not being offered due to lack of demand, plus I don't think it's that common for people to buy groupsets at retail to revamp older bikes. So, Shimano might not have a lot of 11s rim brake 105 or Ultegra groups on hand at all - they might have reasoned that the bulk of the demand would be for the hydraulic versions.
A different way to phrase juhist's contention is this: Shimano and everyone else in the cycling industry do seek to maximize their profits. They perceive that the demand for newer rim brake, mechanical shifting groupsets is low to nonexistent. For better or worse, the bike market has changed. It's not that there's a conspiracy to force everyone onto disc brakes. It's that when offered the choice, consumers tend to choose disc brakes. When offered the choice, they often take electronic shifting, and that means that a lot of newer road bikes have internal routing that makes mech shifting work less well, plus electronic shifting is heading downmarket rapidly. For example, SRAM just released Apex AXS, which is the same level as Shimano Tiagra. That's the stuff that goes on a lot of entry-level bikes! SRAM released a Rival AXS group last year, and Shimano correspondingly released an electronic version of 105.
When juhist said that
... Shimano has just detected that there are a huge number of cyclists with lots of money in their pockets ...
This might not be entirely wrong. We know that income inequality in the West has been increasing markedly. This means that willingness to pay for the fanciest bikes may have also increased markedly, which might have played some role in how the sport has developed technically. However, it will also price a lot more people out of the market, and this was a relatively expensive sport to begin with. Bike companies need to pay attention to the lower end of the market as well.
Without endorsing the notion that there's a conspiracy theory, consumers at the entry level of the sport may sometimes be better served by rim brakes. This is because if you cut enough corners, the mechanical disc brakes and lower-quality housing you see at the entry level frequently (but not always!) produce poor braking, even compared to entry level rim brakes. Furthermore, road bikes definitely don't need the extra dry braking power that discs give, and many people don't ride in the wet.