抓取网站数据,插入Excel单元格,然后继续下一步

时间:2019-03-18 01:38:23

标签: excel vba web-scraping innertext

我的项目是在tax和mot网站上插入一个汽车登记证,单击按钮,加载页面,然后获取日期。

我遇到的一个问题是要在一个强大的li元素中提取数据,这是我需要在两个单元格中输入的税款日期和日期。

Sub searchbot()

'dimension (declare or set aside memory for) our variables
    Dim objIE As InternetExplorer 'special object variable representing the IE browser

    Dim liEle As HTMLLinkElement 'special object variable for an <li> (link) element
    Dim pEle As HTMLLinkElement 'special object variable for an <a> (link) element

    Dim y As Integer 'integer variable we'll use as a counter

'''''''''''''''''''''''''''''''''''''''''''
'open internet

    'initiating a new instance of Internet Explorer and asigning it to objIE
    Set objIE = New InternetExplorer

    'make IE browser visible (False would allow IE to run in the background)
    objIE.Visible = True

'''''''''''''''''''''''''''''''''''''''''''
'open tax/mot page

    'navigate IE to this web page (a pretty neat search engine really)
    objIE.Navigate "https://vehicleenquiry.service.gov.uk/"

    'make IE browser visible (False would allow IE to run in the background)
    objIE.Visible = True

    'wait here a few seconds while the browser is busy
    Do While objIE.Busy = True Or objIE.ReadyState <> 4: DoEvents: Loop

'''''''''''''''''''''''''''''''''''''''''''
'enter details in to page

    'in the search box put cell "b2" value, the word "in" and cell "C" value
    objIE.Document.getElementById("Vrm").Value = _
    Sheets("INPUT & DATA RESULTS").Range("F3").Value

    'wait here a few seconds while the browser is busy
    Do While objIE.Busy = True Or objIE.ReadyState <> 4: DoEvents: Loop

'''''''''''''''''''''''''''''''''''''''''''''''''''''

'click the 'Continue' button
objIE.Document.getElementsByClassName("button")(0).Click

'wait here a few seconds while the browser is busy
    Do While objIE.Busy = True Or objIE.ReadyState <> 4: DoEvents: Loop

'''''''''''''''''''''''''''''''''''''''''''''''''''''

'click the 'Yes' button
objIE.Document.getElementById("Correct_True").Click

'wait here a few seconds while the browser is busy
    Do While objIE.Busy = True Or objIE.ReadyState <> 4: DoEvents: Loop

'''''''''''''''''''''''''''''''''''''''''''''''''''''

'click the 'Continue' button
objIE.Document.getElementsByClassName("button")(0).Click

'wait here a few seconds while the browser is busy
    Do While objIE.Busy = True Or objIE.ReadyState <> 4: DoEvents: Loop

'above works
'''''''''''''''''''''''''''''''''''''''''''''''''''''
'''''''''''''''''''''''''''''''''''''''''''''''''''''
'''''''''''''''''''''''''''''''''''''''''''''''''''''
'''''''''''''''''''''''''''''''''''''''''''''''''''''
'''''''''''''''''''''''''''''''''''''''''''''''''''''
'''''''''''''''''''''''''''''''''''''''''''''''''''''
'''''''''''''''''''''''''''''''''''''''''''''''''''''
'''''''''''''''''''''''''''''''''''''''''''''''''''''
'''''''''''''''''''''''''''''''''''''''''''''''''''''

'HELP FROM HERE PLEASE

'take tax and mot dates and insert in to cells next to each other
'the first search result will go in row 2
y = 2

'TAKE TAX EXPIRY DATE AND PUT IN CELL
'I have tried reading up on extracting data from li elements, parent and child elements but struggling
For Each itemEle In objIE.Document.getElementsByClassName("top-section-list")
data = itemEle.getElementsByTagName("li")(0).innerText


'TAKE MOT EXPIRY DATE AND PUT IN CELL
'I have tried reading up on extracting data from li elements, parent and child elements but struggling
For Each itemEle In objIE.Document.getElementsByClassName("top-section-list")
data = itemEle.getElementsByTagName("li")(0).innerText


'increment our row counter, so the next result goes below
y = y + 1

'repeat times cells have car regs in
'Next
'take next car reg and do the same as above until there are no cells in rows with a car reg
Next
Range("A3").Value = data


'''''''''''''''''''''''''''''''''''''''''''
'close the browser
objIE.Quit

'''''''''''''''''''''''''''''''''''''''''''
'exit our SearchBot subroutine and start new row for new website data
End Sub

我是一名欺诈调查员,试图自学VBA。

2 个答案:

答案 0 :(得分:0)

此网页的结构非常简单,只有一个元素class = status-bar,在其中,您要查找的两个信息位于类型strong的标记中。

因此,无需循环,您可以简单地执行此操作(在编写“上述作品”的位置之后):

'TAX EXPIRY DATE:
TaxExpiryDate = objIE.Document.getElementsByClassName("status-bar")(0).getElementsByTagName("strong")(0).innerText
'MOT EXPIRY DATE:
MotExpiryDate = objIE.Document.getElementsByClassName("status-bar")(0).getElementsByTagName("strong")(1).innerText

然后,您可以将变量TaxExpiryDateMotExpiryDate放在您想要的位置(例如Range("A1").Value = TaxExpiryDate)。

变量本身包含<strong>标记的简单内容:

Tax due:
01 July 2019

如果只想获取日期,可以使用Split()作为分隔符来vbNewLine,然后获取拆分的第二部分:

'IN TWO LINES FOR BETTER CODE READIBILITY:
TaxExpiryDate = objIE.Document.getElementsByClassName("status-bar")(0).getElementsByTagName("strong")(0).innerText
TaxExpiryDate = Split(TaxExpiryDate, vbNewLine)(1)

'IN ONE LINE FOR SHORTER CODE:
TaxExpiryDate = Split(objIE.Document.getElementsByClassName("status-bar")(0).getElementsByTagName("strong")(0).innerText, vbNewLine)(1)

答案 1 :(得分:0)

您想要的项目位于strong粗体)标记中,并且是页面上的前两个,因此您可以使用strong的更快的CSS选择器来执行

Dim items As Object, i As Long, taxInfo As String, motInfo As String
Set items = ie.document.querySelectorAll("strong")
taxInfo = items.item(0).innerText
motInfo = items.item(1).innerText

仅针对日期执行

taxInfo = Replace$(items.item(0).innerText,"Tax due: ",vbNullString)
motInfo = Replace$(items.item(1).innerText,"Expires: ",vbNullString)

这与使用CSS选择器的情况类似,现代选择器针对CSS选择器进行了优化,因此速度更快。 #是ID选择器。我使用了定时等待,以确保存在用于输入注册的搜索框。如果没有找到车辆,则进行初步检查。

Option Explicit   
'VBE > Tools > References:
' Microsoft Internet Controls
Public Sub CheckTax()
    Dim ie As InternetExplorer, searchBox As Object, t As Date, ws As Worksheet
    Const MAX_WAIT_SEC As Long = 20
    Dim inputValues(), i As Long

    Set ie = New InternetExplorer
    Set ws = ThisWorkbook.Worksheets("INPUT & DATA RESULTS")
    inputValues = Application.Transpose(ws.Range("F3:F5").Value) '<=change range here for range containing values to lookup
    With ie
        .Visible = True

        For i = LBound(inputValues) To UBound(inputValues)
            .Navigate2 "https://vehicleenquiry.service.gov.uk/"

            While .Busy Or .readyState < 4: DoEvents: Wend
            t = Timer
            Do
                On Error Resume Next
                Set searchBox = .document.querySelector("#Vrm")
                On Error GoTo 0
                If Timer - t > MAX_WAIT_SEC Then Exit Do
            Loop While searchBox Is Nothing

            If searchBox Is Nothing Then
                Exit Sub
            Else
                searchBox.Focus
                searchBox.Value = inputValues(i)
            End If

            .document.querySelector(".button").Click

            While .Busy Or .readyState < 4: DoEvents: Wend

            If .document.querySelectorAll("h3").Length > 0 Then
                ws.Cells(i + 2, "G") = "Vehicle details could not be found"
                ws.Cells(i + 2, "H") = "Vehicle details could not be found"
            Else
                t = Timer
                Do
                    If Timer - t > MAX_WAIT_SEC Then Exit Do
                Loop While ie.document.querySelectorAll("#Correct_True").Length = 0

                ie.document.querySelector("#Correct_True").Click
                While .Busy Or .readyState < 4: DoEvents: Wend
                .document.querySelector(".button").Click

                While .Busy Or .readyState < 4: DoEvents: Wend

                Dim items As Object, taxInfo As String, motInfo As String
                t = Timer
                Do
                    On Error Resume Next
                    Set items = ie.document.querySelectorAll("strong")
                    On Error GoTo 0
                    If Timer - t > MAX_WAIT_SEC Then Exit Do
                Loop While items.Length = 0

                'taxInfo = items.item(0).innerText
                'motInfo = items.item(1).innerText

                'Debug.Print taxInfo, motInfo

                taxInfo = Replace$(items.item(0).innerText, "Tax due: ", vbNullString)
                motInfo = Replace$(items.item(1).innerText, "Expires: ", vbNullString)

                ws.Cells(i + 2, "G") = taxInfo
                ws.Cells(i + 2, "H") = motInfo
            End If
            Set searchBox = Nothing: Set items = Nothing
        Next
        .Quit
    End With
End Sub