How about using path weight instead of class weight? #80

Hi @buriy, do you wake up? It seems I always post when you go to sleep, the dammit timezone...

Ok, I come up a thought :

List all node absolute path with class and id, calculate path score by positiveRe and negativeRe.
Such as:

body/div/p[1]
body/div/div[1].article/p[1]
body/div/div[1].article/div.comment/p
body/div/div[1].article/div.content/p
body/div/div[1].article/div/p
body/div/div[2]/div.comment/p.content

How to calculate score:

1. search each path from right to left,
2. sum all (distance * distance_punishment * negative weight or positive weight)

distance is current node depth minus ancestor depth which match negativeRe or positiveRe
PS: The bigger distance need make a bigger punishment I think, the correct parameter is hard to determine now, but I think this way can work.

Let's set
positiveRe weight to 25,
negativeRe weight to -25,
distance_punishment to 1

Example:

body/div/div[1].article/p[1]

score : 1 (p[1] depth - div[1].article depth) * 25 = 25

body/div/div[1].article/div.comment/p
order: div[1].article: positiveRe, div.comment: negativeRe

score : 1 (p depth - div.comment depth) * -25 + 1/2 * 25 = -12.5

body/div/div[2]/div.comment/p.content

score : 1 (p depth to itself is 1, I forget to set it ...)* 25 + 1 *-25 = 0

Maybe this approach would give an element with comment class with a positive score. But you could imagine that a comment can't be the only element under article,

Such as it has other bother like :

body/div/div[1].article/div.content/p
body/div/div[1].article/div/p

their's score must higher than comment.

However, what if article only has a child node with class comment with positive score ? Congratulation ! Just means that site's naming has big problem (abuse English word) , but we still find a good candidate! (at least a more confident candidate if they don't misunderstand what article means)

And, as I see, current approach do not give a score to html5 tag such as article, this approach cover this . Calculate node score from deep to shallow would work well with this too.

But , due to lack of data, I can't test it now....