I'm not an engineer or a historian, so I'm going to go off my memory and, where I can find it, companies' stated rationale for making a change.
Bolt circle diameter
The larger the BCD, the bigger the minimum chainring size. 41 teeth is the absolute smallest chainring you can fit on a 144 BCD. For the 130mm BCD used by Shimano and many third party cranks for performance road bikes in the 2000s, the smallest possible chainring is 38t. Some roadies in the 2000s might have scoffed if you pointed out that this makes for steep gearing, and I might have been one of those folks. This was totally wrong. Frankly, it was also stupid. Cyclists span a pretty broad range of physical ability, and we want to make sure the gears aren't too big for people.
The advantage of the 110mm BCD is that it can accept chainrings as small as 33t, or sometimes 32t (as I recall, you may have to make modifications to the spider and/or chainrings to enable this). Possibly, rings with a 110mm BCD might be more flexible than larger rings, all else equal. I recall some users raising this as an objection to compact cranks and chainrings. However, you can compensate with the chainring design, discussed later.
Focusing on road bikes, I don't believe Shimano outright changed to the 110mm BCD on road bikes for some time. I believe they defaulted to 130mm BCD all the way until the 7900 generation of Dura Ace and the corresponding Ultegra and 105 versions (models 6700 and 5600), but they offered cranksets with 110mm BCDs (i.e. compact cranksets) as options. You'd add 50 to the model number to designate this option (e.g. the 6750 crankset was the compact one). I think that in performance road bikes, FSA may have been the first manufacturer to offer compact cranksets; I recall some riders using them on some stages at the 2004 Tour de France, but they weren't necessarily a common option.
I recall that Campagnolo was roughly contemporaneous with Shimano in offering compact cranksets as an option. In any case, these companies offered compact cranksets as an option because there was an increasing demand for them. Riders were getting sick of grinding away in their 39t chainring with a 25-28t big cog. I don't recall if SRAM led Shimano and Campagnolo in offering 110mm options or if it was roughly contemporaneous.
1x, 2x, 3x
Aside from the restrictions on the minimum chainring size, I think you could make a 3x crankset out of any BCD if you really wanted to - although, it would be very strange to have your granny ring be 41t (if you used the 144mm BCD). 3x has become a very rare option, however. Front shifting was always harder with a 3x crankset, and given the increased spread of cogs, road bikes have defaulted to 2x cranks. Most road bikes can approximately replicate the total range of an older 3x setup with 2x and big cassettes.
However, there is a growing demand for sub-compact cranks, i.e. cranks with gearing lower than 50/34 compact chainrings, which requires a correspondingly smaller BCD. The big 3 component groups have lagged at responding to this demand, especially on traditional road bikes. To my memory, Campagnolo was the first to do so, albeit they only have 48/32t chainrings on their relatively premium Chorus groupset, and no other groupset. Shimano followed with more affordable options, albeit they're gravel specific (i.e. the GRX groups). SRAM has also moved in this direction with the Force AXS Wide group, which is a premium groupset.
4- vs 5-arm cranksets
I don't know why most road cranksets had been 5-arm cranks, but I recall that Shimano moved to 4-arm road cranksets starting with the Dura Ace 9000 groupset, released in 2012. This Road.cc review states their rationale:
The idea is that the arms are positioned from an engineering standpoint to transfer power as best they can – taking the positions in the pedal stroke where we tend to apply most power into account – while allowing Shimano to reduce their number of arms from five to four, and thereby save weight.
Shimano reckon this is as stiff as previously, but lighter (the 7900 version, including bottom bracket, was 725g; this version, including bottom bracket, is 683g). Did I notice any difference in the rigidity when riding? Nope, certainly not.
The bolt circle diameter is 110mm whether you go for a standard (something like a 53-39T combination) or a compact (50-34T) – which makes things easier. You could ride standard chainrings at home, for example, and swap to smaller chainrings for a week in The Alps, for example, using the same cranks.
I'm not sure what to make of the rationale, but the 4-arm 110mm BCD enables Shimano to offer standard road gearing, semi-compact, and standard compact (53/39, 52/36, and 50/34 respectively) on the same BCD. This means they don't have to manufacture two types of crankarms with different BCDs. I am not even sure who really needs 53/39 or larger chainrings these days aside from professional road cyclists. In any case, Campagnolo and SRAM moved to 4 arms after Shimano. It appears that all of the big 3 manufacturers, and some smaller ones as well, are able to make 110mm BCD chainrings sufficiently stiff in traditional road chainring sizes. If a chainring is flexy, then if front shifting creates any side loads on the chainring, that could impair the shift quality.
As a side note, it's commonly accepted that all else equal, larger chainrings reduce drivetrain friction. This is because the chain bends less than with a smaller ring. Professional road cyclists sometimes use big rings larger than 53t for time trials. Additionally, some SRAM teams have custom 54/41 chainrings; there's some suspicion that this is to reduce drivetrain friction over the stock 50/37 option, and that these teams are basically not using their 10t cogs. The gains involved are unlikely to be meaningful to amateurs, and I think most amateur riders would prefer a wider gear range anyway.
Direct mount?
All cranksets have crank arms, a spider, and chainrings. From the 1990s to today, Shimano and Campagnolo have made the crankarms and the spider together.
However, some companies separate the spider from the crankarms. In theory, you could have a crank where you need to buy the arms, the spider, and the chainrings separately. The TA Carmina is one such crank, but this is a rare option. A more common alternative arrangement is for the chainring and spider to be combined, and then you would attach this to a splined interface on the arms, i.e. direct mount cranks. I'm not sure when or where this started on the road, but Cannondale may have originated the concept on the road. This may have been a more common design on MTBs. Currently, SRAM appears to have moved to either direct mount chainrings or a separate spider and chainrings with the Red and Force AXS groups released in 2019. SRAM also offers an option to integrate a power meter into the spider.