首页 文章

xpath和htmlagilitypack迭代类似的节点

提问于
浏览 次
0

我正在抓取的HTML是在下面 . 它包含一个帖子和2个回复：

<div class="share_buttons noprint">...</div>

<strong>Dan</strong> Says:

<span class="small soft"><time datetime="2009-10-05T02:27:38Z">Sun, Oct 04 '09, 7:27 PM</time></span>
<div class="quote_top">&nbsp;</div>
<div class="quote_item">Hello all, this is my original post.
</div>

<form class="action_heading noprint">
<strong>Page</strong> 
...
</form>

<div class="post_number" id="r_140626">1</div>
<strong>AnnieMae</strong> Says:

<span class="small soft"><time datetime="2009-10-05T02:30:27Z">Sun, Oct 04 '09, 7:30 PM</time></span>
<div class="quote_top clear_float">&nbsp;</div>
<div class="quote_item">What do you think of it?
</div>

<div class="post_number" id="r_140627">2</div>
<strong>Thomas77</strong> Says:

<span class="small soft"><time datetime="2009-10-05T02:32:32Z">Sun, Oct 04 '09, 7:32 PM</time></span>
<div class="quote_top clear_float">&nbsp;</div>
<div class="quote_item">Not really sure, can't see this pic?

</div>

所以我已经想出如何获得原帖...

'get AUTHOR and DATE of original post
Dim divOriginalPostAuthor As HtmlNode = threadDoc.DocumentNode.SelectSingleNode("//div[@class='share_buttons noprint']/following-sibling::strong")
Dim divOriginalPostDate As HtmlNode = threadDoc.DocumentNode.SelectSingleNode("//div[@class='share_buttons noprint']/following-sibling::span/time")

Dim strDate As String = divOriginalPostDate.InnerText.Trim
strDate = strDate.Remove(0, InStr(strDate, ", ")).Trim
strDate = Replace(strDate, "'", 20)
Dim strAuthor As String = (divOriginalPostAuthor.InnerText).Trim
dtPosted = CDate(strDate)
divOriginalPostText = threadDoc.DocumentNode.SelectSingleNode("//div[@class='share_buttons noprint']/following-sibling::div[@class='quote_item']")

现在我只想弄清楚如何获得回复......我正在考虑获取当前行位置，如下所示：

Dim currentNodePosition As Integer = threadDoc.DocumentNode.SelectSingleNode("//form[@class='action_heading noprint']").Line

然后使用它来迭代回复，因为我增加当前行位置 . 让我觉得棘手的想法是，回复中没有一个“容器”html元素供我立即收集....任何想法？

asp.net vb.net xpath html-agility-pack

1 回答

  • 0

    只是为了记录，我想出了这一点，并想为将来需要它的人发布答案 .

    'then get thread replies
Dim nodesPostNumber As HtmlNodeCollection = threadDoc.DocumentNode.SelectNodes("//form[@class='action_heading noprint']/following-sibling::div[contains(@id, 'r_')]")
Dim replies As New List(Of ThreadReply)

If Not nodesPostNumber Is Nothing Then

Dim intNumberOfReplies As Integer = nodesPostNumber.Count
For i = 1 To intNumberOfReplies
    Dim nodeReplyDate As HtmlNode = threadDoc.DocumentNode.SelectSingleNode("//form[@class='action_heading noprint']/following-sibling::span[@class='small soft' and position()=" + i.ToString + "]")
    Dim strXPathForDate As String = nodeReplyDate.XPath
    Dim strReplyText As String = threadDoc.DocumentNode.SelectSingleNode(strXPathForDate + "/following-sibling::div[@class='quote_item']").InnerHtml
    strReplyText = Left(strReplyText, InStr(strReplyText, "<div class=""noprint""") - 1)
    Dim strReplyAuthor As String = threadDoc.DocumentNode.SelectSingleNode(nodeReplyDate.XPath + "/preceding-sibling::strong").InnerText
    Dim strReplyDate As String = nodeReplyDate.InnerText.Trim
    strReplyDate = strReplyDate.Remove(0, InStr(strReplyDate, ", ")).Trim
    strReplyDate = Replace(strReplyDate, "'", 20)
    strReplyDate = Replace(strReplyDate, "via mobile", "")
    Dim thisReply As New ThreadReply With {.Author = strReplyAuthor, .DatePosted = strReplyDate, .ThreadID = thisThread.ThreadID, .Text = strReplyText}
    replies.Add(thisReply)
Next
End If

    因此，它是关于“抓取”用于1个回复的节点并再次在xpath中使用它以确保您只获得在您抓取的节点之后出现的回复 . 我这样做是通过使用HTMLNode.Xpath为您提供任何给定HTMLAgilityPack.htmlnode的xpath字符串，然后添加“/ following-sibling” .

    回复于

相关问题