Sort of yes, sort of no.
There's writing style and then there's linguistic grammar. There's prescription (what you should do) and description (what people actually do). Those two sentences are not necessarily parallel)
People often use conjunctions at the beginning of sentences. 'And', 'but', 'or'. Also 'moreover', 'inasmuch', 'nonetheless'. That's just plain description. But are just a lot of people in error? A lot of famous and supposedly good (but probably overrated) writers do it.
Capulet: But woo her, gentle Paris, get her heart.
(but this is poetry where... well... not any thing goes, but a sure lot of it does, just to fit the meter).
"My dear Mr. Bennet," said his lady to him one day, "have you heard that Netherfield Park is let at last?"
Mr. Bennet replied that he had not.
"But it is," returned she; "for Mrs. Long has just been here, and she told me all about it."
But this is reported speech. Who knows what shipyard patois Austen is attempting to record?
But a writer should be careful not to construct too many of his sentences after this pattern
And they are exemplars of all that is both correct (grammatically) and tasteful (stylistically) in modern American writing.
Stylistically, though, it sounds funny. Shouldn't a conjunction have something parallel before it in the same sentence? But frankly it's probably more jarring in many circumstances to not use the conjunction especially when it adds meaningful contrast.
Introducing a sentence with a conjunction like and/but/or is grammatically OK in English. People do it all the time in formal and informal speech and they do it in a rule-based, consistent fashion (no one follows the conjunction with an adverb, that would be perverse. And how!).
If you are writing for a newspaper or for a journal article or a paper for school or some other place that tends towards the formal, it is advised not to do it because, and this is the subtle part, some people think it is a rule (also it is a bit informal and gimmicky style marker). Like singular 'they', ending a sentence with a preposition, or comma splicing, these things have been judged to be poor style and to be avoided because of our refined esthetic sensibilities (like wearing jeans that became ripped naturally: that is abhorrent when you can be buying them pre-ripped by professionals who know how to do it right).
In other news, I'm not saying anything new.
Sure, you probably shouldn't use it too much (which is a general advice on any stylistic peculiarity). But every so often it's totally OK.