Yes, they can
There are no rules saying that a bard cannot initiate her bardic performance while underwater, or that you cannot use abilities with audible components while underwater. On the contrary, we know that you can speak normally while underwater, because sound travels farther underwater, as evidenced in Blood of the Sea (pg.28):
Since sound and speech travel farther through water, aquatic creatures tend to talk in a lower, firmer tone in their native environment. This doesn’t always work in the air, and aquatic adventurers new to the surface may have a hard time making themselves heard. Land dwellers may accuse their aquatic companions of whispering or muttering. Some aquatic creatures tend to overcompensate for their habit of speaking quietly by shouting; locathahs, in particular, enjoy raising their voices to make themselves heard (though this is a benefit to surfacers who don’t like to get close to the fishy-smelling creatures anyway). After several weeks or months on land, however, most aquatic travelers learn to speak at a moderate volume.
And again, Aquatic Adventures (pg.45), when explaining how spells work differently while underwater:
If the spell has a verbal component,
expelling the air needed to incant the spell reduces the
creature’s remaining number of rounds of breath by
3, (...) and a creature can’t finish the spell if it doesn’t have enough
breath left. Spells that have verbal components and casting times of swift or immediate actions instead decrease the caster’s remaining number of rounds of breath by 1.
Another evidence of that is the Sea Singer bard archetype, which is a bard who is at home next to the water, nor the Aquatic sorcerer bloodline, which is themed as being descendant of creatures from the ocean, have any abilities modifying how audible components work. While they have several water-themed abilities, none of them modifies the requirement of audible components or affects their bardic performance differently while being underwater.
Also note that neither creatures with swim speed or amphibious creatures have any rule to "allow" them to use audible components underwater, because they don't need to have such ability.
For spells, you only require concentration checks if you cannot breath underwater. Otherwise, that's not even an issue:
Spellcasting Underwater: Casting spells while submerged can be difficult for those who cannot breathe underwater. A creature that cannot breathe water must make a concentration check (DC 15 + spell level) to cast a spell underwater (this is in addition to the caster level check to successfully cast a fire spell underwater). Creatures that can breathe water are unaffected and can cast spells normally. Some spells might function differently underwater, subject to GM discretion.
So, if spells, which are normally more restricted than supernatural abilities (such as bardic performance), work normally underwater, why wouldn't a water-breathing bard be able to sing just fine?
This is all assuming her bardic performance even requires singing, playing an instrument or oratory, if their performance only has visual components, the only restriction might be in the murkiness of the water, which won't affect if they can do it, but the maximum distance they could possibly affect their allies (or enemies).
From Aquatic Adventures (pg.46):
Perception checks to see something are always made at a minimum –2 penalty underwater, and often a greater penalty when the water is murky (flowing water, such as in a current or river, is always at least somewhat murky). The maximum distance most creatures can see underwater varies from
40 to 320 feet in clear water (4d8 × 10) and from 10 to
80 feet in murky water (1d8 × 10).