In rare cases, none of the party's candidates may have a majority of delegates going into the convention. The convention is then considered "contested." Delegates will pick their presidential nominee through one or more rounds of voting.
In the first round of voting:
Pledged delegates usually have to vote for the candidate they were awarded to at the start of the convention.
Unpledged delegates can vote for any candidate.
Superdelegates in the Democratic Party cannot vote in the first round of a contested convention. But they can vote in the first round of a convention in which a candidate already has enough delegates through primaries and caucuses to get the nomination.
In the rare instance that no nominee wins in the first round, the convention is considered "brokered." The pledged delegates may choose any candidate in later rounds of voting. Superdelegates can also vote in these later rounds.
Balloting continues until one candidate receives the required majority to win the nomination.
https://www.usa.gov/national-conventions
My understanding is that if the ruling comes before the convention, and the Republican party recognizes the ruling and disqualifies Trump, then in the first round of voting, all votes for Trump would be considered invalid. If this results in no one getting a majority, then in further rounds Trump's delegates would be free to vote for anyone they want.
If the ruling happens after the convention, I'm sure the Republican party could come up with some emergency justification for picking a different candidate, but I really don't see the Supreme Court disqualifying him after the convention, both because at that point I think it would be hard to find five justices willing to be that disruptive to the electoral process, and because I don't see them sitting on it for six months.