Delphi ignores code order 4

[mischmasch]

Hello,

I'd like to display a short message (e.g. 'Please wait ...') on Panel before I start some long calculations. My code structure looks like below

//1_check data
{...}

//2_show message on panel: 'Pleas wait ...'
{...}

//3_make calculation
{...}

//4_export result to excel
{...}

//5_change message on panel: ''
{...}

Everything works ok, but there is a problem with correct order of steps.
When I execute program I see, that program makes calculations (it's diffucult calculations so program doesn't respond for a while), then dispalys message on panel (it should make it before calculations) and then exports result to excel and changes message on panel.
Even if in code editor, everything go step by step, finally result of Step 2 is after Step 3.
I've tried to use threads but unsuccessfully.

Any idea why calculations are made before displaying message?

Thank you in advance

 

[whosrdaddy]

thats because the long calculations block the main VCL thread and disrupt window message processing, like the WM_PAINT message destined to redraw the panel.
Threading is the best solution for this sort of situations.
A poor mans solution is to add Application.processmessages after each operation (or add it in the calculation loop for a more responsive GUI).

Cheers,
Daddy

-----------------------------------------------------
What You See Is What You Get
Never underestimate tha powah of tha google!

 

[djjd47130]

Application.ProcessMessages is a great way to prevent this, but it's very costly and must be used carefully. Especially if you are using it in a timer, it can cause the program to become unresponsive, because it can do the process messages faster than it can do the actual code. Therefore, if I have to put it in a timer, I wrap that timer so it can't execute more than once at a time.

Basically, Application.ProcessMessages makes sure all commands to Windows (especially display of controls) are completed before continuing to the rest of the code. If you excessively use it, your program will be sluggish, yet will properly show all controls properly upon each use. Not enough of it will make your program fast, but may become unresponsive if you're doing a big task. It's especially useful in really heavy loops, so your program doesn't get recognized as "Not Responding" by Windows. The Not Responding messages scares a lot of people, but really it's just because your program is busy doing something and can't reply back to Windows. Windows doesn't think the program is running properly because it's not replying. So you need to use a little Application.ProcessMessages at the end of each loop, or at least at the end of every heavy process as you see fit, to make sure it doesn't become unresponsive.

This is just to understand the importance of proper usage of it below. If this protection was not put on the Timer, then the more you use ProcessMessages, the deeper this timer would get frozen. Try it yourself, put a heavy process in it, then when you slap in the Application.ProcessMessages without this protection, you will see the procedure will be executed over and over, and will overlap its self, and could result in a Stack Overflow. If you try to terminate the program, it won't end immediately because it's executing the same procedure possibly hundreds or thousands of times in a row. This is all thanks to the Application.ProcessMessages. The command tells Windows that it's "OK" to execute the timer again. So you need this protection around it...

[b]type[/b]
  TfrmTest = [b]class[/b](TForm)
    MyTimer: TTimer;
    [b]procedure[/b] TimerExecute(Sender: TObject);
    [b]procedure[/b] FormCreate(Sender: TObject);
  [b]private[/b]
    fExecutingTimer: Bool;  [navy][i]//Used for checking if MyTimer is already executing or not
[/i][/navy]  [b]end[/b];

..........

[b]procedure[/b] TfrmTest.FormCreate(Sender: TObject);
[b]begin[/b]
  fExecutingTimer:= False;  [navy][i]//Make sure this is set to False by default
[/i][/navy][b]end[/b];

[b]procedure[/b] TfrmTest.TimerExecute(Sender: TObject);
[b]begin[/b]
  [b]if[/b] [b]not[/b] fExecutingTimer [b]then[/b] [b]begin[/b]  [navy][i]//Make sure MyTimer not already executing TimerExecute by checking fExecutingTimer
[/i][/navy]    fExecutingTimer:= True;  [navy][i]//Set fExecutingTimer to True so MyTimer won't execute twice in a row
[/i][/navy]    [b]try[/b]
      MyLabel.Caption:= [navy]'Calculating...'[/navy];
      Application.ProcessMessages; [navy][i]//Makes sure message gets displayed first
[/i][/navy]
      [navy][i]//This is where you put all 
[/i][/navy]
      [navy][i]//your code which you don't 
[/i][/navy]
      [navy][i]//want executed more than 
[/i][/navy]
      [navy][i]//once at a time
[/i][/navy]
      MyLabel.Caption:= [navy]'Finished Calculating.'[/navy];
      Application.ProcessMessages; [navy][i]//Makes sure message gets displayed first
[/i][/navy]    [b]finally[/b]
      fExecutingTimer:= False;  [navy][i]//Reset this variable to false safely, in case an exception may happen in your code
[/i][/navy]    [b]end[/b];
  [b]end[/b];
[b]end[/b];

JD Solutions

 

[Griffyn]

@djjd47130: Are you sure wrapping the TTimer is necessary? My understanding and experience is that the TTimer.TimerExecute event is called from the main VCL thread, therefore it cannot be executed again whilst the event is still executing, because it's all one thread.

 

[djjd47130]

Trust me, I've had apps crash for that exact reason, I've done tests and confirmed it can overlap.

JD Solutions

 

[djjd47130]

Well, that is if your Interval is smaller than the time it takes to complete your code in the procedure

JD Solutions

 

[Griffyn]

TIL that TTimer.TimerExecute is simply killed if the timer interval is reached while it's still processing.

The code below, with a TTimer.Inteval set to 1000 ms, will simply show a sequence of 1 2 3 1 2 3 1 2 3... in the TMemo.

[b]procedure[/b] TForm1.Timer1Timer(Sender: TObject);
[b]var[/b]
  c : Integer;
[b]begin[/b]
  [b]for[/b] c := [purple]1[/purple] [b]to[/b] [purple]5[/purple] [b]do[/b]
  [b]begin[/b]
    Memo1.Lines.Add(IntToStr(c));
    Application.ProcessMessages;
    Sleep([purple]500[/purple]);
  [b]end[/b];
[b]end[/b];

 

[djjd47130]

I just re-tested and re-confirmed, unfortunately using a heavy component which I'm not about to publish. Like I said, try it yourself. Here's the scenario:
- Create a timer with interval of 1 ms
- Create a global integer to track number of processes
- On app start, set this integer to 0
- In the OnTimer event,
--- Add 1 to the integer
--- Display integer value in label
--- Call Application.ProcessMessages
--- Do something very heavy and time consuming
--- Subtract 1 from the integer
--- Display integer value in label
--- Call Application.ProcessMessages

Then, watching this variable will clearly show you that it's being call many times in a row. I just got it to execute over 400 times in a row loading the same large image file over and over, and would have grown larger if I hadn't terminated it.


JD Solutions

 

[djjd47130]

If that's the case Griffyn, then it's even worse, because if I had created an object in that timer, you mean that object won't get free'd, it would simply get ignored? If that's true, that's an even worse problem.

JD Solutions

 

[djjd47130]

Actually, no that problem is not what you think, because the 3 4 and 5 never even have a chance. When the second execution takes place, the first one doesn't get killed, but gets paused until the second one finishes. Then the third gets called in the meantime, and the first and second are paused. When the 3rd one finishes (which if a 4th is called, never will), then the 2nd will continue, then when that finishes, the 1st will finish. Your little test there only proved half the story, you never let it run to see it finish filling in the 3 4 and 5.

JD Solutions

 

[djjd47130]

Adding a few lines to a TMemo is a very lightweight task, btw. Must take 0.0005 ms to complete. Try loading a huge image from a file and drawing it to your form inside that loop of 1 to 5 and count how many times it's running, like mentioned above. Try this out...

Create a form with 1 timer and 1 label, and declare 1 integer variable...

...
  lblStatus: TLabel;
  Timer1: TTimer;
  public
    fRunning: Integer;
...
[b]procedure[/b] TfrmListViewTest.FormCreate(Sender: TObject);
[b]begin[/b]
  fRunning:= 0;  //Set to 0 by default
  //Next 3 lines forcefully show the main form even when it's not yet loaded all the way
  Self.Show;
  Self.BringToFront;
  Application.ProcessMessages;
  Timer1.Enabled:= True;  //Timer was disabled by default, enable it at the end of FormCreate
[b]end[/b];

[b]procedure[/b] TfrmListViewTest.Timer1Timer(Sender: TObject);
[b]const[/b]
  FN = [navy]'D:\Media\Pictures\GForce\Toxic.bmp'[/navy]; //Replace with some huge bitmap image of yours
[b]var[/b]
  X: Integer;
  B: TBitmap;
[b]begin[/b]
  fRunning:= fRunning + [navy]1[/navy];
  lblStatus.Caption:= IntToStr(fRunning)+[navy]' running at a time'[/navy];
  Application.ProcessMessages;
  [b]try[/b]
    [b]for[/b] X:= [navy]1[/navy] [b]to[/b] [navy]5[/navy] [b]do[/b] [b]begin[/b]
      B:= TBitmap.Create;
      [b]try[/b]
        B.LoadFromFile(FN);
        Canvas.StretchDraw(Rect(X,X,Width,Height), B);
        Application.ProcessMessages;
      [b]finally[/b]
        B.Free;
      [b]end[/b];
    [b]end[/b];
  [b]finally[/b]
    lblStatus.Caption:= IntToStr(fRunning)+[navy]' running at a time'[/navy];
    Application.ProcessMessages;
    fRunning:= fRunning - [navy]1[/navy];
  [b]end[/b];
[b]end[/b];

JD Solutions

 

[djjd47130]

And, at the same time you're watching that number climb up there, check out your Task Manager and watch that little green line of your memory grow higher too. Eventually, that line will reach almost the top and you will get:

"Project XXXXX.exe raised exception class EOSError with message 'System error. Code: 8. Not enough storage is available to process this command'. Process stopped. Use Step or Run to continue."

Meaning, your memory got sucked up by an overactive timer.

JD Solutions

 

[whosrdaddy]

JD,

the behaviour you are seeing is perfectly normal.
a windows Timer just posts WM_TIMER messages to itself to the interval that you have set.
The reason I don't like Application.Processmessages is that people don't understand the windows message pump and use it in the wrong context like you are doing now.
I'll try to explain it here in detail.

For this I created a new project in Delphi (just select new->VCL forms application) here is the source code (.pas + dfm)

.pas -> 

unit Unit1;

interface

uses
  Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
  Dialogs, ExtCtrls, StdCtrls;

type
  TForm1 = class(TForm)
    Btn_good: TButton;
    Btn_bad: TButton;
    Lbl_timer: TLabel;
    procedure FormCreate(Sender: TObject);
    procedure FormDestroy(Sender: TObject);
    procedure Btn_goodClick(Sender: TObject);
    procedure Btn_badClick(Sender: TObject);
  private
    { Private declarations }
    MyTimer : TTimer;
    Counter : Integer;
    procedure GoodTimerEvent(Sender: TObject);
    procedure BadTimerEvent(Sender: TObject);
  public
    { Public declarations }
  end;

var
  Form1: TForm1;

implementation

{$R *.dfm}

function LongCalculationLoop : double;

var i : integer;

begin
 for i :=0 to 1000000 do
   Result := Sqrt(i);
end;

procedure TForm1.BadTimerEvent(Sender: TObject);
begin
 Inc(Counter);
 Lbl_timer.Caption := Format('Counter is now: %d', [Counter]);
 LongCalculationLoop;
 Application.ProcessMessages;
 Counter := 0;
end;

procedure TForm1.GoodTimerEvent(Sender: TObject);
begin
 Inc(Counter);
 Lbl_timer.Caption := Format('Counter is now: %d', [Counter]);
 LongCalculationLoop;
 Counter := 0;
end;

procedure TForm1.Btn_badClick(Sender: TObject);
begin
 Counter := 0;
 MyTimer.Enabled := False;
 MyTimer.OnTimer := BadTimerEvent;
 MyTimer.Enabled := True;
end;

procedure TForm1.Btn_goodClick(Sender: TObject);
begin
 Counter := 0;
 MyTimer.Enabled := False;
 MyTimer.OnTimer := GoodTimerEvent;
 MyTimer.Enabled := True;
end;

procedure TForm1.FormCreate(Sender: TObject);
begin
 MyTimer := TTimer.Create(nil);
 MyTimer.Interval := 10;
end;

procedure TForm1.FormDestroy(Sender: TObject);
begin
 FreeAndNil(MyTimer);
end;

end.


.dfm ->

object Form1: TForm1
  Left = 0
  Top = 0
  Caption = 'Form1'
  ClientHeight = 79
  ClientWidth = 208
  Color = clBtnFace
  Font.Charset = DEFAULT_CHARSET
  Font.Color = clWindowText
  Font.Height = -11
  Font.Name = 'Tahoma'
  Font.Style = []
  OldCreateOrder = False
  OnCreate = FormCreate
  OnDestroy = FormDestroy
  PixelsPerInch = 96
  TextHeight = 13
  object Lbl_timer: TLabel
    Left = 20
    Top = 12
    Width = 6
    Height = 13
    Caption = '0'
  end
  object Btn_good: TButton
    Left = 20
    Top = 44
    Width = 75
    Height = 25
    Caption = 'Good'
    TabOrder = 0
    OnClick = Btn_goodClick
  end
  object Btn_bad: TButton
    Left = 101
    Top = 44
    Width = 75
    Height = 25
    Caption = 'Bad'
    TabOrder = 1
    OnClick = Btn_badClick
  end
end

now click on the 'good' button and you will see the text "the counter is now: 1"
clicking on the 'bad' button and you will see the counter value fluctuating.
huh what is going here??
Please remember that a VCL forms application is single threaded (and a TTimer component does not change that fact).
The understand what is going on here, we need to take a look under the delphi hood.

we start with the .dpr file source code:

program Project1;

uses
  Forms,
  Unit1 in 'Unit1.pas' {Form1};

{$R *.res}

begin
  Application.Initialize;
  Application.CreateForm(TForm1, Form1);
  Application.Run;
end.

not much to see here:
- some initializing
- our main form is autocreated
- aplication is run

but what does Application.Run really do? let's find out:

small note: this is the source code for D2006, it may vary from version to version.

procedure TApplication.Run;
begin
  FRunning := True;
  try
    AddExitProc(DoneApplication);
    if FMainForm <> nil then
    begin
      case CmdShow of
        SW_SHOWMINNOACTIVE: FMainForm.FWindowState := wsMinimized;
        SW_SHOWMAXIMIZED: MainForm.WindowState := wsMaximized;
      end;
      if FShowMainForm then
        if FMainForm.FWindowState = wsMinimized then
          Minimize else
          FMainForm.Visible := True;
      repeat
        try
          HandleMessage;
        except
          HandleException(Self);
        end;
      until Terminated;
    end;
  finally
    FRunning := False;
  end;
end;

we see a repeat loop:
Handle a windows message until the application is terminated.
That's it, that's all that is needed to run a windows application.
What the HandleMessage procedure in reality does is read the Application's message queue.

function TApplication.ProcessMessage(var Msg: TMsg): Boolean;
var
  Unicode: Boolean;
  Handled: Boolean;
  MsgExists: Boolean;
begin
  Result := False;
  if PeekMessage(Msg, 0, 0, 0, PM_NOREMOVE) then
  begin
    Unicode := (Msg.hwnd <> 0) and IsWindowUnicode(Msg.hwnd);
    if Unicode then
      MsgExists := PeekMessageW(Msg, 0, 0, 0, PM_REMOVE)
    else
      MsgExists := PeekMessage(Msg, 0, 0, 0, PM_REMOVE);
    if not MsgExists then Exit;
    Result := True;
    if Msg.Message <> WM_QUIT then
    begin
      Handled := False;
      if Assigned(FOnMessage) then FOnMessage(Msg, Handled);
      if not IsPreProcessMessage(Msg) and not IsHintMsg(Msg) and
        not Handled and not IsMDIMsg(Msg) and
        not IsKeyMsg(Msg) and not IsDlgMsg(Msg) then
      begin
        TranslateMessage(Msg);
        if Unicode then
          DispatchMessageW(Msg)
        else
          DispatchMessage(Msg);
      end;
    end
    else
      FTerminate := True;
  end;
end;

procedure TApplication.HandleMessage;
var
  Msg: TMsg;
begin
  if not ProcessMessage(Msg) then Idle(Msg);
end;

procedure TApplication.ProcessMessages;
var
  Msg: TMsg;
begin
  while ProcessMessage(Msg) do {loop};
end;

Handlemessage calls Processmessage.
Processmessage reads the message queue (PeekMessage) and if there is a message in the queue it will dispatch the message (DispatchMessage).
DispatchMessage will in fact forward the message to the destined control (form, button, timer, ...)

I included Application.ProcessMessages and as you can see it is a bit different than HandleMessage.
HandleMessage will take 1 message out of the queue (if there is any).
ProcessMessages will take messages out of the queue UNTIL it is empty.

Now what does this mean for our little application?
When creating and activating a Timer in windows, windows will send WM_TIMER messages at the specified interval.
Delphi will get the WM_TIMER message in it's queue and will dispatch the message to the TTimer object.
Our TTimer object will fire the onTimer event when that happens. That's all, nothing more, nothing less...

when we activate the 'good' timer the application will behave like this:

application.run ->
handlemessage (WM_TIMER in queue) ->
processmessage -> TForm1.GoodTimerEvent
-> increase counter
-> set label
-> longcalculation
-> reset counter
handlemessage (WM_TIMER in queue) ->
processmessage -> TForm1.GoodTimerEvent
-> increase counter
-> set label
-> longcalculation
-> reset counter
...
and so on

when we activate the 'bad' timer we will see this

application.run ->
handlemessage (WM_TIMER in queue) ->
processmessage -> TForm1.BadTimerEvent
-> increase counter
-> set label
-> longcalculation
-> ProcessMessages
handlemessage (WM_TIMER in queue) ->
processmessage -> TForm1.BadTimerEvent
-> increase counter
-> set label
-> longcalculation
-> ProcessMessages
handlemessage (WM_TIMER in queue) ->
processmessage -> TForm1.BadTimerEvent
...
-> reset counter
-> reset counter


See what's happening here?
the key is that if you have a long calculation in your timer event (longer than the specified Timer interval),
WM_timer messages will start filling the up the application's message queue and in extreme conditions (like JD's code) the
Application's stack will be filled with calls to the timerevent until there is no more room in the stack and our nice little app will crash...

There I hope this clears things up for everyone.
As you can see Application.ProcessMessages can easily fuck up things if used in the wrong context...

Cheers,
Daddy




-----------------------------------------------------
What You See Is What You Get
Never underestimate tha powah of tha google!

 

[whosrdaddy]

For reference, if you need to know more about windows messaging, this is a good starting point:

http://msdn.microsoft.com/en-us/library/ms632586(v=VS.85).aspx

Cheers,
Daddy

-----------------------------------------------------
What You See Is What You Get
Never underestimate tha powah of tha google!

 

[djjd47130]

Thanks Daddy, great tutorial, but I do understand (at least 89%) the workings of it. Just the part of how the entire application runs on this cycle was way over my head. Of course computers run on processor ticks, each of which processing the next message in each running thread. I'd rather not get that detailed, I doubt I'll be working on that level any time soon.

However I actually just finished posting a new FAQ on this particular matter, and your description above would be great there too in the Performance section.

I'm no genious, but slowly climbing my way up there...

JD Solutions

 

[djjd47130]

Sorry for all the confusion, mischmasch, lemme sum it up for ya..
When you do commands such as changing the caption of a label or panel, or adding data to a list, or otherwise doing something which requires something visual to be changed on the form, it sends a message to windows. This message gets stuck in a queue, where when everything's functioning fluidly, you won't see any issue. However when you start doing heavier work like exporting to excel, and doing some big calculation, this will require you to forcefully process this queue of messages waiting to be sent to Windows. This queue is processed by calling Application.ProcessMessages. When this queue is processed completely, Windows will recognize this application as properly responding. When your code finishes in your app, this queue is automatically processed. But if your procedure is dragging long, the queue can back up quickly. No harm in letting it back up, except your program will be unresponsive. Just if you need, call Application.ProcessMessages to forcefully show this data. That's all.

JD Solutions

 

[whosrdaddy]

Just if you need, call Application.ProcessMessages to forcefully show this data. That's all.

but you must be carefull using this command as it can get ugly very fast when wrongly used(like I pointed out in the previous post).
I stand by my original answer,
threading is the best longterm solution, although it requires even more prudence and skill to get it right....

/Daddy

-----------------------------------------------------
What You See Is What You Get
Never underestimate tha powah of tha google!

 

[djjd47130]

Thanks for that reminder Daddy. I'm sure threading is certainly the way to go, for the hardcore coders like yourself, but for intermediate people like me, that's like trying to fly an F14 when the most I can do is ride a motorcycle.

Like I mentioned above, I wrote an FAQ on this matter, view it here:

http://www.tek-tips.com/faqs.cfm?fid=7482

JD Solutions