There are 19th century post-civil war examples of both state and federal officials (at least one federal judge) being removed from office after taking office under the 14th Amendment provision referenced, and there has been one post-January 6, 2021 example of a local government official being removed from office on that ground.
Josh Blackman's position writing for The Volokh Conspiracy that argues that the president is not an officer of the United States, would be a minority view on that issue.
According to the Congressional Research Service (a non-partisan research agency of Congress):
According to the text of Section 3, the bar against office-holding
applies to Members of Congress, officers of the United States, members
of state legislatures, and state executive or judicial officers, who
previously swore an oath to support the Constitution of the United
States and later break that oath by committing the acts mentioned. The
offices to which such persons are then barred include seats in
Congress, membership in the Electoral College, and any civil or
military office under the United States or any state. Although not
expressly referenced, the bar appears historically to have applied to
judgeships.
There is an argument that because the President is not covered
explicitly by the provision, the presidency itself is exempt from the
disqualification. In contrast, the Impeachment Clause of the
Constitution explicitly applies to the “President, Vice President and
all civil Officers of the United States,” which suggests that the
President might not be a “civil Officer of the United States” whose
oath of office would subject him to possible disqualification.
However, it may be more likely that the office of the President is
included as an office under the United States (unlike Members of
Congress and electors, which may be why they are expressly included),
so that any person subject to the disqualification is ineligible to
serve as President.
One scholar notes that the drafting history of Section 3 of the
Fourteenth Amendment suggests that the office of the President is
covered:
"During the debate on Section Three, one Senator asked why
ex-Confederates “may be elected President or Vice President of the
United States, and why did you all omit to exclude them? I do not
understand them to be excluded from the privilege of holding the two
highest offices in the gift of the nation.” Another Senator replied
that the lack of specific language on the Presidency and
VicePresidency was irrelevant: “Let me call the Senator’s attention to
the words ‘or hold any office, civil or military, under the United
States.’”"
In the January 2021 article of impeachment against President Donald
Trump, the House of Representatives, citing Section 3 of the
Fourteenth Amendment, appears to have presumed that the
Disqualification Clause would operate as a bar against President Trump
continuing to serve as President, presumably due to his previous oath
of office and his alleged “incite[ment of] violence against the
Government of the United States.”
But since it has never happened before, it is hard to know how a fact pattern alleging that a sitting President is an insurrectionist who is not eligible to serve would play out.
Courts might be reluctant to rule that the President was ineligible, but it isn't obviously a non-justiciable issue any more than, for example, the constitutional requirement that the President be at least 35 years of age.
Also, contrary to the implication in the answer of Rich Smith to this question, neither an indictment or conviction of a relevant crime would be necessary, although it would be sufficient if there was a conviction.
A civil action to disqualify the President from holding office could move forward even if a criminal action was pending, although the civil court could also elect to stay its proceedings pending the outcome of the criminal case.
The decision to remove someone from office because they lack of the qualifications to hold that office would be decided in a civil quo warranto writ petition and not directly in a criminal trial. Many of the people removed from office for lack of qualifications under the insurrection clause were not criminally convicted but were removed by a judge in a civil proceeding.
According to the same CRS source:
Section 3 of the Fourteenth Amendment does not expressly require a
criminal conviction, and historically, one was not necessary.
Reconstruction Era federal prosecutors brought civil actions in court
to oust officials linked to the Confederacy, and Congress in some
cases took action to refuse to seat Members.
Congress last used Section 3 of the Fourteenth Amendment in 1919 to
refuse to seat a socialist Congressman accused of having given aid and
comfort to Germany during the First World War, irrespective of the
Amnesty Act. The Congressman, Victor Berger, was eventually seated at
a subsequent Congress after the Supreme Court threw out his espionage
conviction for judicial bias.
Recently, various groups and
organizations have challenged the eligibility of certain candidates
running for Congress, arguing that the candidates’ alleged involvement
in the events surrounding the January 6, 2021, breach of the Capitol
render them ineligible for office. No challenges have to date resulted
in the disqualification of any congressional candidate. A New Mexico
state court, however, has removed Otero County Commissioner Couy
Griffin from office and prohibited him from seeking or holding any
future office based on his participation in, and preparation for, the
January 6 interruption of the election certification.
A judgment against the President in a civil lawsuit regarding his qualifications, would be sufficient to remove him from office, without the necessity of an indictment, conviction, or impeachment.
This does still leave some open questions as to how the civil action would proceed. Most notably, who would have standing to sue, what the burden of proof would be, and so on. But the precedents from petitions for writs of quo warranto for other offices would fill in many of the procedural details. The process would probably be similar to the one set forth in the District of Columbia's municipal code (and there is a plausible although probably not a winning argument that it is directly applicable to an action to remove the U.S. President from office).
A proper party could also probably petition for an original writ on the matter from the U.S. Supreme Court, which has original writ jurisdiction under the All Writs Act, even though it very rarely exercises that authority. Original writ procedure in the U.S. Supreme Court under U.S. Supreme Court Rule 20 is discussed here.
