Help with IDhttp and Thread

[clubreseau]

Hi,

Here is what I would like to do.

a simple Form with a Tlistbox which would contain several URLs with a TButton and a TMemo.

To make it simple, I try as quickly as possible to go to all the URLs of my listbox and take the source page of each URL in my listbox and if in the page the word Welcome is there. then insert the url in the Tmemo

I have watched several tutorials that talk about Thread and Parallel Programming Library I can't seem to do this. I'm trying from this tutorial

https://stackoverflow.com/questions/37123765/how-to-use-threads-with-idhttp-in-delphi-10

but it doesn't work.

Could someone give me a piece of code that will do the job.

Thank you and sorry for my english

 

[mirtheil]

What do you mean it doesn't work? How is it failing? What debugging have you tried? What code do you currently have?

Mirtheil

http://www.mirtheil.com

 

[clubreseau]

I use this

procedure TForm1.Button1Click(Sender: TObject);
var
  i:integer;
  lpath:string;

begin
  for i := 0 to ListBox1.Items.Count-1 do
  begin
    lPath := ListBox1.Items.Strings[i];
    download(lPath);
   end;
 end;

 procedure TForm1.Download(lpath: string);
 begin
    TTask.Create(
      procedure
      var
        HTTP : TidHTTP;
        IdSSL: TIdSSLIOHandlerSocketOpenSSL;
        content : String;
        URI: TIdURI;
        fmatch : TMatchCollection;
        regex2 : TRegEx;
        fmatchcount:string;
      begin
        HTTP := TIdHTTP.Create(nil);
        try
          HTTP.ReadTimeout := 30000;
          HTTP.HandleRedirects := True;
          IdSSL := TIdSSLIOHandlerSocketOpenSSL.Create(HTTP);
          IdSSL.SSLOptions.Method := sslvTLSv1_2;
          IdSSL.SSLOptions.Mode := sslmClient;
          HTTP.IOHandler := IdSSL;
          HTTP.AllowCookies:=true;
          HTTP.Request.UserAgent := 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2227.0 Safari/537.36';
          HTTP.Request.Accept := 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8';
          HTTP.Request.AcceptLanguage := 'en-GB,en;q=0.5';
          HTTP.Request.Connection := 'keep-alive';
          HTTP.Request.ContentType := 'application/x-[URL unfurl="true"]www-form-urlencoded';[/URL]
          HTTP.CookieManager := TIdCookieManager.Create(HTTP);
          URI := TIdURI.Create('[URL unfurl="true"]http://localhost');[/URL]
          try
            HTTP.CookieManager.AddServerCookie('habari=10283454', URI);
          finally
            URI.Free;
          end;
          Content := HTTP.Get(lpath);
        finally
          TThread.queue(TThread.Current,
          procedure
          begin

              regex2 := TRegEx.Create('Welcome', [roIgnoreCase]);

    fmatch := regex2.Matches(Content);
    fmatchcount:=IntToStr(fmatch.Count);


  if fmatchcount > '0' then

           listbox2.items.Add(lpath);
        
            //listbox2.Items.Add(lpath);
          end);
        end;

      end
    ).Start;
  end;

Like this its work. its paste the url 1 by 1 in the listbox1, the problem its slown how I can make it faster.

 

[mirtheil]

What part is slow? How slow is slow? If it's the HTTP GET, what does Postman or some other tool for issuing an HTTP show for the time / speed? What debugging have you done? If you don't know which part is slow, you should debug line by line until you find the one that is slow.

Mirtheil

http://www.mirtheil.com

 

[clubreseau]

the source page in memo take 1 second to 2 second.

How I can make it faster ?

use 3 or 4 thread at the same time ?

 

[mirtheil]

the source page in memo take 1 second to 2 second." -- What do you mean?
How long does it take to issue a GET using Postman or another HTTP tool? For example,

https://cnn.com

takes 218ms (and

https://cbc.ca

takes 223ms) to issue the GET and get a response using Postman. How long does it take in your code?

Mirtheil

http://www.mirtheil.com

 

[clubreseau]

forget about this.

HOW to use 4 url at the same time and paste the source on different memo or string dont care.

 

[whosrdaddy]

you must perform the downloads in a separate thread,
actually your download code is running in context of main thread and will block the UI.

-----------------------------------------------------
Helping people is my job...

 

[clubreseau]

whosrdaddy can you show me a piece of code for example.

Thank you

 

[whosrdaddy]

https://stackoverflow.com/a/37126614/800214

-----------------------------------------------------
Helping people is my job...