31

People constantly ask how to parse a block of HTML using a regular expression.
A popular response is simply "you can't". Is that an appropriate answer on its own?

A few recent examples:

Answer1 - This was the leading answer until I've posted a link on the PHP chat room, asking for better answers.
Answer2 - Close vote as duplicate (?).
Answer3 - Got closed as duplicate although it has good answers.
There are also many examples of people simple posting the link, say "don't do it" and walk away.

I'm not the only one who's annoyed: here's someone, and another one.

Are people overreacting? Are my standards too high?

7
  • I'm not sure I have asked a question, but I'd like to hear some thoughts :)
    – Kobi
    Commented Dec 23, 2010 at 18:08
  • 21
    You think you are overreacting?
    – Grace Note StaffMod
    Commented Dec 23, 2010 at 18:18
  • 2
    possible duplicate of Is "Don't do it" a valid answer?
    – Jon Seigel
    Commented Dec 23, 2010 at 18:27
  • 3
    @Jon - Actually, I'd like to talk specifically about this subject, if that's all right, specifically about how to handle answers that don't add much besides that link (on which you have to scroll 10 answers to get to relevant code, which I doubt anyone will past the Paris Hilton answer).
    – Kobi
    Commented Dec 23, 2010 at 18:37
  • For the record: I hardly ever agree with Eric Raymond, but he makes some good points in esr.ibiblio.org/?p=1387. I still love the bobince post, though :)
    – balpha StaffMod
    Commented Dec 23, 2010 at 21:18
  • 1
  • I think this is partly cos different people seem to mean different things by parsing, some use a more formal, syntactic analysis meaning, others seem to include lexical analysis as well
    – jk.
    Commented Sep 3, 2011 at 11:08

6 Answers 6

13

The neverending influx of these horrible questions is catalyst for the smartassistance from some Stackoverflowers. Or to put it in nicer words: it's a meme. And people have simply given up being helpful, because it seemingly doesn't make a difference.

I've recently come across a couple of examples too. People are posting the link to @bobince's fun answer in a not overly relevant manner. - Don't get me wrong, I like it being linked from reddit twice a year, and I might even have upvoted it myself. - But in that specific Q&A there were a couple of way more objective analyzations of the html-with-regex topic. But those go completely ignored by anchoring the anti-regex rant in the run to post the funny comment.

Maybe someone could comment on that. Is that link commonly posted because it's funny? Do you have a bookmark? Or a favorite? Do you remember it that well to find it with the site search?

Anyway, I don't think it's generally intended to be help the asker. It does not explain why regular expressions are mostly unfit for the task. And then people even post it when regular expressions would indeed be an acceptable tool for a simplistic extraction (e.g. src from imgs).

Practically there is no way to make these questions go away. New and even seasoned users post it every now and then. That the anti-regex meme is permeating SO won't stop it. I think it's entirely caused by outdated PHP tutorials on the web, and books (all PHP books should be burned). - But I don't think it's beneficial to post the fun link. There is a very very very remote possibility to reduce the number of htmlregex questions by offering a reasonable explanation instead. [link] It's IMO as simple as contrasting the complexity of regex approaches with phpQuery/QueryPath/etc:

 preg_match("#<address[^>]*>([^<>]+)</a#ims", $html, $m);

 qp($html)->find("address")->text();

Obviously: more work, less fun. But it's not as if we didn't have better reference questions for the dreaded html-vs-regex topic: How to parse HTML with PHP?


That link: Not helping. What do we do about it? Commenting doesn't help. Is it allowed to flag it for not being contributing?

Or can we maybe have it added to the very same blocklist where LMGTFY lives? I do think it belongs there.

Wikipedia style: Add an objectivizing info block to bobince's fun answer to deter from overzealous linking?

50

There's a lot of prejudice against parsing HTML with regex, but the vast majority (I'd say over 80%) of these regex questions are simple pattern matching questions - for which a DOM parser is overkill. So, I actually have started voting down answers that say "You can't". Not only they are not helpful, but plain wrong. Parse HTML with regex? Yes, you can!

Further aggravating is that people started closing these questions as a "duplicate" of a generic question on how to parse HTML, which has as the accepted answer a bunch of links to various DOM parsers. It is essentially the same thing as telling the OP to google DOM Parsers.

Suggesting the use of DOM parsers for those is a moronic idea, akin to telling people to use jQuery for anything Javascript. We are not teaching the OP to "catch fish," but simply telling them "hey, here are some cruise missiles you can use to blow fish to oblivion. Go figure out which one and how to use them."

Instead of parroting the "hey use a DOM parser" for pretty much every regex+HTML question, a better thing to do would be link them to a regex guide (in addition to providing a valid answer). Only when you understand regex then you can apply your judgement to see if it's an appropriate tool for your problem and decide if you need a DOM parser or not.

5
  • 1
    you're my guy.. I sometimes (when I really don't know javascript solution (everything's much simplier in jQuery)) suggest user to use jQuery, but I at least provide code to DO so
    – genesis
    Commented Sep 2, 2011 at 22:49
  • 12
    I think comparing recommending a HTML parser to recommending jQuery is utterly ridiculous. As Tchrist says in his epic answer on parsing HTML with regex: Sure, it isn’t easy, but it is possible! ... And trying to do so is a terrible waste of time, because good parsing classes exist which you should use for this task.
    – Pekka
    Commented Sep 3, 2011 at 6:34
  • 9
    Pure pattern matching may work for the HTML example the OP shows, but will fail for the real-world case the next day where the attributes are in different order, are in uppercase or lowercase, tags are self-closing... That's why I say it's still true that if the OP isn't in it for the learning, "use a HTML parser" is still the Correct Answer™​.
    – Pekka
    Commented Sep 3, 2011 at 6:36
  • 3
    I disagree with some of that. Most question come from people who don't know the right tool, and they should be pointed in the right direction. However, I would always include working code in the answer. It usually amount to 3 lines of code, and it is much better than "parroting", as you say, "you can't use a parser". (I used "duckspeak" before, but I may overdid it :))
    – Kobi
    Commented Sep 3, 2011 at 8:04
  • 1
    You failed to realize that the questions on SO are mostly just examples, targeting a small piece of an application. In real world regexes are really fragile against updates of the incoming format, and HARD to maintain. Would you like to get overhanded a project where developers parse all that html with regexes or would you like a simple to read DOM parser code?. Also in real world scenarios it is likely that multiple values should be extracted from the document, would you say it is not a overhead to run a regex every time over the whole document?
    – hek2mgl
    Commented Jul 3, 2014 at 9:25
33

Linking to The Response is the worst kind of self-congratulatory unhelpfulness. That big block of funny is only funny to those of us who understand why it's funny. To the guy who comes in with a rep of 47, whose only tool is PHP, and who may well have never done any programming before, The Repsonse is useless at best, and humiliating at worst. Posting a link to The Response tells the poster "HA HA, you don't know any better, and I'm going to rub your face in it." It's drive-by mockery, and it's cruel.

So, no, linking to that is not helpful. When we went through this in the past dealing with escaping strings embedded in dynamically-built SQL commands, I created http://bobby-tables.com so that we could have a canonical place to point people at. I'm currently working on a website that we can point people who ask about parsing HTML along the same lines, and I welcome help from anyone who's interested. Email me at [email protected] if you'd like to join in. I need examples of the right way to parse HTML in myriad different languages.

The website is http://htmlparsing.com/ and I've started putting out the bits and pieces I've collected.

1
  • 4
    +1: While I disagree with your opening statement I applaud you doing something about the problem.
    – johnsyweb
    Commented Feb 4, 2012 at 0:43
8

I find this whole topic interesting - and not just because I seem to have been made a poster child of it...

Those who are offended by the posting of "that link" (something I do not do, by the way!) possibly don't realize that their sometimes violent reactions to the posting of that link probably both perpetuates and exacerbates the problem.

In my own answer linked in this question, I was waiting for feedback from the OP to see where he wanted to go, before posting a code sample that wasn't likely to get used anyway. That way, I didn't have a 'physically large' answer that could arguably be seen as being somewhat off-topic without knowing in advance that the OP was interested in seeing such examples.

The OP did ask for such examples at one point - but by then, the asker of this question had already posted a snarky, one-line criticism of my answer. (Should I start a question here about that practice?)... and the OP's comment asking for examples... included a link to a page that did just that.

The OP ended up sticking with Regular Expressions. That's fine by me. The asker of this question criticized my semi-non-answer. Also fine by me. This question itself has also earned me some more reputation on that semi-non-answer. Also... fine by me.


For the record; I think "that link" isn't useful at all, and I think posting it is just plain silly. I do not happen to be a member of the "Never use Regexes for HTML" crowd, but I do think that using an HTML parser is generally better.

But I think it's interesting to have apparently become a 'poster boy' for something that I didn't actually do... ;)

3
  • Also interesting: The asker of this question actually did not even post an answer in the question being addressed here. Seems like that would have been the best way to make the case... Commented Dec 23, 2010 at 19:03
  • First, thanks. Looking back, I was too harsh. You've posted the first answer, and you were suggesting something the OP obviously didn't know, but did like eventually and did choose the parser - the accepted answer shows that. In fact, it seems we almost agree - posting a link isn't helpful, but I think your kind answer isn't helpful either. Again, thanks for taking the time.
    – Kobi
    Commented Dec 23, 2010 at 19:08
  • I didn't post an answer because I couldn't. I tried for 20 minutes to mind a site like ideone, which allows PHP + Dom parser + load external html, but couldn't. I don't know PHP and don't have ready on my computer, but I'm sure I can write a few lines of code in most languages, and so can you! That is my point exactly - in a typical question (not this one though), you need 2 lines of code to load the document, one line of xpath, and you're done. It is the best way to make the case.
    – Kobi
    Commented Dec 23, 2010 at 19:11
7

I suspect most people who ask such questions genuinely aren't even remotely aware of the possibilty that regexes shouldn't aren't the way to go, so such a link can be useful. I think Bobince's answer is easier to digest than a mathematician's proof of HTML not being regular.

4

Addressing only the appropriateness of "You can't" answers:

An unadorned "You can't" is often correct, is not very helpful (which if you read the tool-tip is the metric for up votes). Accordingly such answer should be voted neither up nor down.

1
  • 1
    I've started voting down those answers. Not only they are not helpful, but plain wrong. Parse HTML with regex? Yes, you can! Commented Sep 2, 2011 at 22:35

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .