using VB to search the web? 2

[kaylabear]

Hello,

I require a point in the right direction as I am unsure as to how this topic works.

I need to build a VB application that will open Internet Explorer (but not actually open the browser window) and search millions of predetermined URL's, searching for a specific line of text and then paste that line of text into my database.

Is it possible to this using Visual Basic and if so how would I go about getting started?

All of your your tips and ideas are greatly appreciated thank you.

 

[JESTAR]

In VB6, you can automatically add a browser to your program. This is the code it generates....

Public StartingAddress As String
Dim mbDontNavigateNow As Boolean
Private Sub Form_Load()
On Error Resume Next
Me.Show
tbToolBar.Refresh
Form_Resize


cboAddress.Move 50, lblAddress.Top + lblAddress.Height + 15


If Len(StartingAddress) > 0 Then
cboAddress.Text = StartingAddress
cboAddress.AddItem cboAddress.Text
'try to navigate to the starting address
timTimer.Enabled = True
brwWebBrowser.Navigate StartingAddress
End If


End Sub



Private Sub brwWebBrowser_DownloadComplete()
On Error Resume Next
Me.Caption = brwWebBrowser.LocationName
End Sub


Private Sub brwWebBrowser_NavigateComplete2(ByVal pDisp As Object, URL As Variant)
On Error Resume Next
Dim i As Integer
Dim bFound As Boolean
Me.Caption = brwWebBrowser.LocationName
For i = 0 To cboAddress.ListCount - 1
If cboAddress.List(i) = brwWebBrowser.LocationURL Then
bFound = True
Exit For
End If
Next i
mbDontNavigateNow = True
If bFound Then
cboAddress.RemoveItem i
End If
cboAddress.AddItem brwWebBrowser.LocationURL, 0
cboAddress.ListIndex = 0
mbDontNavigateNow = False
End Sub


Private Sub cboAddress_Click()
If mbDontNavigateNow Then Exit Sub
timTimer.Enabled = True
brwWebBrowser.Navigate cboAddress.Text
End Sub


Private Sub cboAddress_KeyPress(KeyAscii As Integer)
On Error Resume Next
If KeyAscii = vbKeyReturn Then
cboAddress_Click
End If
End Sub


Private Sub Form_Resize()
On Error Resume Next
cboAddress.Width = Me.ScaleWidth - 100
brwWebBrowser.Width = Me.ScaleWidth - 100
brwWebBrowser.Height = Me.ScaleHeight - (picAddress.Top + picAddress.Height) - 100
End Sub


Private Sub timTimer_Timer()
If brwWebBrowser.Busy = False Then
timTimer.Enabled = False
Me.Caption = brwWebBrowser.LocationName
Else
Me.Caption = "Working..."
End If
End Sub


Private Sub tbToolBar_ButtonClick(ByVal Button As Button)
On Error Resume Next


timTimer.Enabled = True


Select Case Button.Key
Case "Back"
brwWebBrowser.GoBack
Case "Forward"
brwWebBrowser.GoForward
Case "Refresh"
brwWebBrowser.Refresh
Case "Home"
brwWebBrowser.GoHome
Case "Search"
brwWebBrowser.GoSearch
Case "Stop"
timTimer.Enabled = False
brwWebBrowser.Stop
Me.Caption = brwWebBrowser.LocationName
End Select


End Sub

I'm not sure where you can go from here, maybe one of the more experienced programmers (Dale/Rob etc.) can take the lead.

Hope I helped.

 

[sfvb]

If you're doing this in Excel, try the following:[ul][li]go to the Visual Basic Editor (Alt+F11)
[/li][li]Insert a User Form
[/li][li]If the ToolBox is not visible then View the Toolbox.
[/li][li]Right Click on one of the ToolBox controls and choose "Additional Controls..."
[/li][li]From the list of Available Controls, mark the "Microsoft Web Browser" and click on the "OK" button.
[/li][li]The WebBrowser object is now available from the ToolBox. Choose the WebBrowser object and add it to the UserForm.
[/li][li]Insert a Module
[/li][li]Add the code from below into the module
[/li][li]From Excel run the TestWebSearch macro
[/li][/ul]What the code should do is get the web page that you're reading right now, and search the page for the sentence that starts I require a point, once it finds it, it should display the whole sentence.

Hope this helps.

Sub TestWebSearch()
   Dim strTxt        As String
   Dim intBegPos     As Integer
   Dim intEndPos     As Integer

   UserForm1.WebBrowser1.Navigate "[URL unfurl="true"]http://www.tek-tips.com/gviewthread.cfm/lev2/4/lev3/32/pid/707/qid/399215"[/URL]
   Do Until UserForm1.WebBrowser1.ReadyState = READYSTATE_COMPLETE
      DoEvents
   Loop
   
   strTxt = UserForm1.WebBrowser1.Document.documentElement.outerText
   intBegPos = InStr(1, strTxt, "I require a point")
   If intBegPos > 0 Then
      intEndPos = InStr(intBegPos, strTxt, vbCr)
      If intEndPos > 0 Then
         MsgBox Mid(strTxt, intBegPos, intEndPos - intBegPos)
      Else
         MsgBox Mid(strTxt, intBegPos)
      End If
   Else
      MsgBox "Didn't find the string."
   End If
End Sub

 

[x3]

I didn't know I could add controls to my toolbox. Thanks.

 

[TimMyth]

My issue is along similar lines. I have embedded a webbrowser into my Excel XP document, and have created the appropriate Home, back, forward, etc buttons to control it. It works nicely. However, I am having problems getting the formated text from the web page into my Excel sheet. I would like for it to automatically paste the text onto the Excel sheet in the same format used on the web page. The page has several tables, and text color is important.

I tried using worksheets("sheet1&quot.range("a1&quot = WebBrowser.Document.documentelement.outertext, but that only puts the entire web page text into cell A1. Using range("a1:z50&quot puts the entire web page text into cells A1:Z50.

I have also tried using a RichTextBox, but the entire page doesn't display in the box, instead only the first portion of the page appears. The other problem with the RichTextBox is that the text loses its color, which leads me to beleive that .documentelement.outertext will only provide the text and not the formatting.

I could just require the user to physically copy and paste the page to sheet, but thats not a very elegant solution.