I thought the whole point of XHTML is that non-well-formed documents should generate an error instead of being unpredictably interpreted according to each implementation's whims.
And if you really insist on doing that, why not just parse XHTML as HTML? HTML5 parsing rules can already handle some XHTML-like constructs; it's what browsers do when they're served XHTML as text/html. If it's good enough for them, it should be good enough for you.
In E-books, which IIUC this library was designed to parse: very frequently. The ePub3 specification actually specifies the XHTML serialization of HTML5 as its content serialization, so this is a correct eBook. Ironically, this means that most eBooks are not actually valid HTML5. We had another eBook reader (Sigil) that used Gumbo and ran into this issue.
On the web: very rarely. I've actually never seen a self-closing <title /> tag, but I've seen other cases where docs that are actually XML are served with a text/html MIME type and this creates pathological DOM structures.
(I have a funny story from Gumbo's testing that illustrates this. I ran across a document that was actually some XML dialect with 45,000 self-closing elements in a row, but was served with a text/html MIME type. Since HTML5 doesn't recognize self-closing elements that aren't in the spec, this created a DOM tree with 45,000 levels of nesting. Gumbo could handle this because it uses an iterative state machine for parsing, but my testing code did recursive descent on this and choked. I posted it on MemeGen - Google's internal water-cooler website - with a link to the offending web page ... and then got a few emails from other Googlers about how it was kinda rude of me to crash their browsers. It turns out Chrome couldn't handle the page, and would die with a stack overflow when viewing it.)
I encounter self closed <title> tags in malformed XHTML files all the time. In fact I encounter them so often, I used to use a dedicated sanitization pass before passing int he html to html5lib, for that reason alone.
Ha, okay! I tried. I really did. I feel for whomever has to read this spec.
I'm assuming the tokenizer successfully tokenizes through "<title", and that we're in the "tag name state"[1]; from there, we see the space, that takes us to the "before attribute name state"[2]; then we consume the solidus, and switch to the "self-closing start tag state"[3]; we see the ">" and the spec says,
> Set the self-closing flag of the current tag token. Switch to the data state. Emit the current tag token.
So, now the tokenizer is:
<title /></head>
^ about to consume this
and we're in the "data" state, which is approximately correct for about to consume a </head>, so that's looking okay. (I'll also point out that the more Englishy parts that talk about start tags[4] only say that "if the element is one of the void elements, or if the element is a foreign element, then there may be a single "/" (U+002F) character" but don't really elaborate on what to do if it isn't.)
So, let's say the tokenizer escapes okay. The parser is another story…
Let's assume at this point we're in the "in head" insertion mode[5]; we get a start tag from the tokenizer as per above. This state tells us,
> A start tag whose tag name is "title"
> Follow the generic RCDATA element parsing algorithm.
but that algorithm seems to be directed at the tokenizer, which runs a bit counter to
> The input to the tree construction stage is a sequence of tokens from the tokenization stage.
Regardless, the self-closing flag on the start tag that got emitted doesn't get acknowledged[6], which seems to cause a "parse error". But what should a parser do with that parse error? How should we behave? If we follow the link, we end up with
> The error handling for parse errors is well-defined (that's the processing rules described throughout this specification)
…which, no spec, no you don't.
The best I've got is that the tokenizer and tree construction stages aren't independent; that is, if we go back to the bit where we're supposed to be following the "generic RCDATA element parsing algorithm" — which is under tree construction, it states,
> switch the tokenizer to the RCDATA state.
which seems to heavily imply that we adjust the tokenizer's state from within tree construction. Okay. So if we follow the algorithm for RCDATA, noting that the tokenizer was about to consume </head>, AFAICT, it gets very quickly into the "RCDATA end tag open state" after eating the </ from </head>, then "RCDATA end tag name state", then eventually consumes the > and we hit:
> "If the current end tag token is an appropriate end tag token, then switch to the data state and emit the current tag token. Otherwise, treat it as per the "anything else" entry below."
So what's an appropriate end tag token?
> An appropriate end tag token is an end tag token whose tag name matches the tag name of the last start tag to have been emitted from this tokenizer,
The last tag we emitted was that weird <title/> thing, so no, this is not an appropriate end tag, and we skip to "anything else"
> Switch to the RCDATA state. Emit a U+003C LESS-THAN SIGN character token, a U+002F SOLIDUS character token, and a character token for each of the characters in the temporary buffer (in the order they were added to the buffer). Reconsume the current input character.
Uh-oh. From here, my mental parse just loops around in the RCDATA state; the title of the document ends up being "</head><body><p>foo".
I threw up a simple webserver to host that string: Both Chrome and Firefox agree! But as a parser, I feel abused.
The HTML5 algorithm could be a bit clearer here, but I think it does specify a well formed result, and additionally would flag this document as having a parse error (for an unacknowledged self-closing tag, but hilariously? oddly?, not for the rampant consumption of tags by the RCDATA algorithm…)
But that seems to be what the spec says to do. I'm going to go out on a limb and guess that ebook readers don't follow the spec, and that if you attempted to, people would (incorrectly) blame your software, even if it's doing the right thing, putting you between a rock and a hard place because the ebook writers can't write well-formed markup? (I really feel for you here, if that's correct; ideally, you could just emit a blank book and tell the ebook writer to fix it…; maybe if you emitted a warning at the bottom that stated "This eBook contained errors; a best-effort has been made to correct it; if you experience problems with this book, notify the publisher.") But, then, my SO shows me plenty of just simply typographical errors in her ebooks, and she can't even see the markup.
