Parsing robustness

[Traumflug]

Fallout of a hacking Sunday. Tackles #7567 and #8417.

Tested against my own project, only, as I'm not aware how the testsuite works.

Not so sure about commit 0d91578, "don't shortcut on previously seen elements", as I'm not sure how much performance impact it has. It solves handling ignored tags, but probably needs either removal of then dead code or a different solution for dealing with ignored tags appearing multiple times.

[CLAassistant]

All committers have signed the CLA.

[Traumflug]

In case somebody wants to enhance the testsuite, here are additional cases:

<script type="text/javascript">
    for (var i = 0; i < imgDefer.length; i++) {
        doNothing();
    }
</script>

<script
  type="text/javascript"
>
  if (a<b && a>c) {
    haha();
  }
</script>

<form
	action="[URL]"
	method="post"
>

<hr	id=a>

Yes, newlines and tab vs. space matters :-)

Failure of each is recognized by non-alphabetical characters appearing in the hugo_stats.json.

[bep]

Tested against my own project, only, as I'm not aware how the testsuite works.

go test ./publisher

[bep]

Thanks for this.

There are failing tests. It looks mostly to be elements inside pre/textarea, which I don't see any reason to collect (but please prove me wrong).

Also, it would be great if you could

1. Add some test cases for the cases this PR is supposed to fix.
2. Run the benchmark inside the ./publisher package and compare it to master (see below)

name                     old time/op    new time/op    delta
ClassCollectorWriter-16    21.2µs ± 2%    47.3µs ± 1%  +123.21%  (p=0.029 n=4+4)

name                     old alloc/op   new alloc/op   delta
ClassCollectorWriter-16    35.3kB ± 0%    86.4kB ± 0%  +144.95%  (p=0.029 n=4+4)

name                     old allocs/op  new allocs/op  delta
ClassCollectorWriter-16       155 ± 0%       329 ± 0%  +112.26%  (p=0.029 n=4+4)

Note that I don't mind it getting slower if really needed, but I fail to understand the motivation behind the removal of elementSet, which I suspect is the reason why it is now so much slower and memory hungry.

[bep]

Can you check whether the latest commits re this in the master branch covers your needs?

[Traumflug]

Thanks for the hint on how to run these testcases easily. I just pushed a reviewed set of commits, rebased on current master.

There are failing tests. It looks mostly to be elements inside pre/textarea, which I don't see any reason to collect

Well, the simple reason for these failures was me removing special handling of pre/textarea, but not knowing how to remove these tests as well.

That said, removing these was a mistake, I misunderstood the intention of this code earlier. It's not about not collecting non-styleable tags, but about ignoring stuff inside them. Accordingly, I melted down the first commit to just adding another two such tags with preformatted code. Where <title> isn't exactly preformatted, but of no use for styling anyways.

Can you check whether the latest commits re this in the master branch covers your needs?

Well, some, but not all. This time, knowing how to deal with these tests, I managed to find a testcase for each. Committed testcases separately to allow seeing them failing. Luckily, I also found a fix for each.

Also one or two testcases which fell into the keyboard somehow, without a matching reported issue I'd be aware of :-)

Note that I don't mind it getting slower if really needed, but I fail to understand the motivation behind the removal of elementSet

This time I moved the commit with this removal, along with a commit with a demonstrating (previously failing, now working) testcase just before it, all to the end, to allow (cherry-)picking earlier commits.

The problem to solve appears when such a preformatted tag appears twice and contains a < (happens rarely in testcases, often in real world usage). First round everything is fine and dandy. On the second round, the processing time saver misses the part where w.inPreTag gets set. Accordingly, the following parsing doesn't recognize the preformatted tag as such and treats stuff inside the preformatted tag like normal tags.

Perhaps you have a better idea on how to get the best of both, quick processing as well as recognizing repeated preformatted tags.

[ghost]

I just rebased the pull request branch to tag v0.83.1 and force-pushed it.

[bep]

Thanks, but I've already spent a considerably amount of time simplifying this in another. Would welcome any failed test cases after that PR is merged.

[github-actions]

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.