ProcessMessages again
ProcessMessages again
The topic was brought up by Brian Hamilton a few posts below, and Dalija Prasnikar explained how code that is calling ProcessMessages is reentrant in two different ways.
I had to deal with a ProcessMessages problem days ago, let me explain:
Within a loop I create thumbnail images of all the samples in the current batch of samples. Each sample in the batch is loaded, the 3D image rendered, and the small bitmap copied out of the viewport and inserted into the larger bitmap, see picture.
Viewport.InvalidateRect(Viewport.ClipRect);
Viewport.Repaint;
Application.ProcessMessages;
Viewport.Context.CopyToBitmap(SmallBitmap, SmallRectangle);
Without calling ProcessMessages, the Context will not be ready for CopyToBitmap. At least I did not find a better way of doing it.
There is also a keyboard event handler in the application which triggers the thumbnail creation process when the user types a question mark.
To complete the description, let me say that the viewport is made smaller before the loop is entered and the whole thing completes when the user clicks/taps on a thumbnail.
On Windows, it works. The keyboard message was the only message in the queue and removed before calling the event handler. Hopefully, ProcessMessages will only ever find the stuff in the queue that is supposed to be there, maybe nothing, I am not quite
sure about that. (Not sure why I have to call ProcessMessages in the first place.)
However on iOS, it appears the keyboard message has not yet been removed from the message loop when the event handler is called. So, ProcessMessages will enter the program into an endless loop. I can set a breakpoint in the event handler. It breaks there whenever I call F9. Whether this is indeed endless or will somehow be stoppable is unclear and does not matter.
I use a Bluetooth keyboard with my application on iOS, but if you just tap a button or rectangle, it is still the same problem.
The solution was not to call the thing from the event handler. I just take a note and trigger the process when the application goes idle. When OnIdle fires, the message queue is supposed to be empty, this seems to be reliable.
I searched the FMX units for usage of ProcessMessages and found several.
Also I thought about whether or not I should include the thumbnail feature in the release of the app. Have not decided yet. Much of what I have said is based on observation and speculation. I did not see what messages are in the message loop or who removes them at what time.
As long as ProcessMessages is with us in the main thread of the application, make sure you block reentrance of your critical area with a flag, and think before you do stuff from within the event handler.
My observation, that the message is not removed from the queue early enough on iOS, is it something you can confirm, and is it perhaps possible for Embarcadero to improve on this?
Application.ProcessMessage is wrap from cocoa framework NSRunLoop.
ReplyDeleteAccording to the ios SDK document,
https://developer.apple.com/library/mac/documentation/cocoa/reference/foundation/classes/nsrunloop_class/reference/reference.html
it says:
Manually removing all known input sources and timers from the run loop is not a guarantee that the run loop will exit. OS X can install and remove additional input sources as needed to process requests targeted at the receiver’s thread. Those sources could therefore prevent the run loop from exiting.
If you want the run loop to terminate, you shouldn't use this method. Instead, use one of the other run methods and also check other arbitrary conditions of your own, in a loop. A simple example would be:
BOOL shouldKeepRunning = YES; // global
NSRunLoop *theRL = [NSRunLoop currentRunLoop];
while (shouldKeepRunning && [theRL runMode:NSDefaultRunLoopMode beforeDate:[NSDate distantFuture]]);
So, I think you have to make your own FLAG in somewhere to stop Application.ProcessMessage loop.
e.g. if (Flag=True) Applicaton.ProcessMessage ....
Next version of Delphi they should mark Application.ProcessMessages as deprecated...
ReplyDeleteI just tried to reproduce the problem in a simple test case. The idea was to make a cool blinking-rectangle type of application with the help of ProcessMessages. But no, the simple test app works. Have to use OnKeyDown on iOS, because nothing arrives in OnKeyUp. Then, when I press Shift in order to produce a ? I am notified of Shift being pressed multiple times, before I have a chance to press the question mark key. (My real application did handle this all right).
ReplyDeleteNow I have the simple test app ready and it works on Windows and iPod touch. Next I added stuff and I am now dealing with a Retina effect. When I copy the thumbnails from viewport to large bitmap I get only the top-left quarter. (iPad mini does not have Retina).
When I am done with Retina, I will reenable the process messages problem and have a second look. My OnIdle solution serves me well in my real app.
Regarding the flag, it is simple.
procedure TFormMain.ApplicationEventsIdle(Sender: TObject; var Done: Boolean);
begin
if Flag then
DoStuff
Done := True;
end;
Why do I have to call ProcessMessages? Maybe because Viewport.InvalidateRect and/or Viewport.Repaint put something into the queue, and maybe only on some platform? I am not sure.
My code does call InvalidateRect and Repaint, typically after processing keystrokes.
Or is it the upgrade from 7.1.1 to 7.1.2 or the bluetooth keyboard? Tried to connect my apple bluetooth keyboard to iPod touch but did not get a connection. It is currently connected to iPad. So I will rule out a few simple things and report back here very short when I am done.
Gustav Schubert thanks for sharing.
ReplyDeleteA reproducable test case below for iOS devices with Apple Wireless Keyboard attached.
ReplyDeleteKeys a, b, c are good cases.
Key d will enter the application into the unwanted loop effect.
While you see the noise, try any key to exit the loop.
Embarcadero Delphi XE4 Update 1
iPad mini with iOS 7.1.2
unit FrmMain;
Interface
uses
System.SysUtils,
System.Types,
System.UITypes,
System.UIConsts,
System.Classes,
FMX.Types,
FMX.Forms,
FMX.Edit,
FMX.Objects,
FMX.StdCtrls,
FMX.Viewport3D,
FMX.Layouts,
FMX.Controls;
type
TFormMain = class(TForm)
procedure FormCreate(Sender: TObject);
procedure FormDestroy(Sender: TObject);
private
bm: TBitmap;
ra: TRect;
TileSize: Integer;
Flag: Boolean;
WantPM: Boolean;
IsRetina: Boolean;
FocusEdit: TEdit;
Rectangle: TRectangle;
Viewport: TViewport3D;
procedure ApplicationEventsIdle(Sender: TObject; var Done: Boolean);
procedure FocusEditKeyDown(Sender: TObject; var Key: Word; var KeyChar: Char;
Shift: TShiftState);
procedure DoStuff;
procedure CreateThumbnail(i: Integer);
function GetTileColor(Index: Integer): TAlphaColor;
procedure Init;
end;
var
FormMain: TFormMain;
implementation
{$R *.fmx}
procedure TFormMain.FormCreate(Sender: TObject);
begin
TileSize := 100;
IsRetina := False;
if IsRetina then
TileSize := 2 * TileSize;
bm := TBitmap.Create(TileSize, TileSize);
ra := Rect(0, 0, TileSize, TileSize);
Init;
FocusEdit.SetFocus;
Application.OnIdle := ApplicationEventsIdle;
end;
procedure TFormMain.FormDestroy(Sender: TObject);
begin
bm.Free;
end;
procedure TFormMain.Init;
begin
FocusEdit := TEdit.Create(self);
FocusEdit.Position.X := 10;
FocusEdit.Position.Y := 10;
FocusEdit.Width := 40;
FocusEdit.OnKeyDown := FocusEditKeyDown;
FocusEdit.Visible := True;
FocusEdit.Enabled := True;
AddObject(FocusEdit);
Rectangle := TRectangle.Create(self);
Rectangle.Position.X := 100;
Rectangle.Position.Y := 10;
Rectangle.Width := 50;
Rectangle.Height := 50;
AddObject(Rectangle);
Viewport := TViewport3D.Create(self);
Viewport.Parent := self;
Viewport.UsingDesignCamera := False;
Viewport.Position.X := (ClientWidth - TileSize) / 2;
Viewport.Position.Y := (ClientHeight - TileSize) / 2;
Viewport.Width := TileSize;
Viewport.Height := TileSize;
end;
procedure TFormMain.ApplicationEventsIdle(Sender: TObject; var Done: Boolean);
begin
if Flag then
DoStuff;
Done := True;
end;
{ Want to control iOS app with keystrokes from Apple Wireless Keyboard. }
procedure TFormMain.FocusEditKeyDown(Sender: TObject; var Key: Word; var KeyChar: Char;
Shift: TShiftState);
begin
{ skip if problems anticipated }
// if not (KeyChar = 'd') then
FocusEdit.Text := ''; //<-- critical
if KeyChar = 'a' then
begin
{ test out idle processing }
Flag := True
end
else if KeyChar = 'b' then
begin
{ no problem }
DoStuff
end
else if KeyChar = 'c' then
begin
{ run the stuff with ProcessMessaging from OnIdle}
WantPM := True;
Flag := True;
end
else if KeyChar = 'd' then
begin
{ run it directly }
WantPM := True;
DoStuff;
end;
end;
procedure TFormMain.DoStuff;
var
i: Integer;
begin
Flag := False;
if WantPM then
ReplyDeletebegin
WantPM := False;
for i := 1 to 9 do
CreateThumbnail(i);
end;
if Rectangle.Fill.Color = claCornflowerblue then
Rectangle.Fill.Color := claCrimson
else
Rectangle.Fill.Color := claCornflowerblue;
end;
procedure TFormMain.CreateThumbnail(i: Integer);
begin
Viewport.Color := GetTileColor(i);
Viewport.InvalidateRect(Viewport.ClipRect);
Viewport.Repaint;
Application.ProcessMessages;
Viewport.Context.CopyToBitmap(bm, ra);
end;
function TFormMain.GetTileColor(Index: Integer): TAlphaColor;
begin
case Index of
1: result := claRed;
2: result := claGreen;
3: result := claBlue;
4: result := claCyan;
5: result := claMagenta;
6: result := claYellow;
7: result := claWhite;
8: result := claBlack;
9: result := claGray;
else
result := claWhite;
end;
end;
end.
Qing-Shan Xiao Indeed. This is what Gustav Schubert and all other UI builders should do. Process the logic in a separate thread, and have the UI thread perform rendering. Sometimes hard, but the only way to make things stable.
ReplyDeleteHi +Qing-Shan Xiao,
ReplyDeleteI use the TViewport3D.Context.CopyToBitmap function in a loop. Every time with a different 3D image just rendered by the graphics hardware of the iPad mini. On Windows, I can use the same code, but DirectX10 Software rendering is responsible. The 3D Viewport is a visible UI component. UI components should be used in the main thread of the application. As long as you do this you will be on the safe side of things. I found that if OnIdle (idle processing) can be used to solve a problem, it is usually a quick and reliable thing to try out. Using idle processing you can easily control (reduce) the number of times a task is carried out, without a timer, and also decouple the task from the event handler currently running.
I have used idle processing in the past, e.g. I am using a timer in my VCL application. The event handler of the timer is empty. The only purpose of the timer is to trigger idle processing, in case the user does not move the mouse any more. This way idle processing is a workable solution.
You could use the test case above to try out idle processing. Do you have XE6? Are you able to test on a device with a Bluetooth Keyboard?
Gustav Schubert how well does OnIdle work in FMX? I've had some high-CPU usage usage with that in the past using VCL (not sure which Delphi version, as that code got refactored a long time ago)
ReplyDeleteIdle processing does work in FMX. I wondered if it was available at all. But it is the same as before, from a developer perspective. High-CPU usage as you can see it in a CPU-Monitor window does not always mean what you see. While the app will run through OnIdle often, it will be immediately available when the user starts to interact with it. Other applications are not affected. That is my experience. Have indeed built applications on top of OnIdle. Networked applications, which must give priority to the network stuff, but still have a UI. The painting of a StringGrid on the form has no priority there and is throttled down to once a second at most. Sometimes it is easier to paint everything in a timed manner instead of knowing what changed. Binding will have its own complexity and overhead. When I say once a second, that is when the user only looks on the screen. If the user enters something in the Grid, it is safe and responsive. No problem with the network being active behind the scene. The network components are using threads, I do my work in the main thread, as does the StringGrid. The limiting factor on FMX is not idle processing or networking, it is the StringGrid, or the Memo.
ReplyDeleteJeroen Wiert Pluimers Without ProcessMessages it does not render the new image in time. If Embarcadero deprecates ProcessMessages, I will have a warning in my application!
ReplyDeleteMy logic has click speed. The rendering takes time, especially with lighting enabled. There is one case, when I add one 3D Text, and rendering slows down dramatically. Therefor I use the 3D Text only to create nice screenshots with copyright info accross the image.
As to threading, I knew a developer who added threading to a small and working prototype application I made for him. That was the first thing he added. He was never able to debug the application properly, it used to crash. The app had a feature, which could only be started, not stopped and then restarted. Users always killed and restarted the app. This was a long time ago, but I still appreciate an application which is debuggable, source code which is comparable, references that are traceable. If I control click and be taken to the interface, I am not where I wanted to go.
I wonder if someone will examine my test case, and show how to solve it with threading. The job however is to confirm the issue and find out why the test app is bothered with the same KeyChar again and again. Not you Jeroen, you are exused of course.
This is a little iPad app, ideal for starters, with some interest in high school mathematics. FMX at its best. Enjoyable. A zero thread zone!
:)
Gustav Schubert (: Omni Thread library for the win (;
ReplyDeleteok, I have a guess about the meaning of the emoticons. Will look it up.
ReplyDeleteApplication.OnKeyStrokeReceived(KeyChar: Char)
ReplyDelete(Want to give a summary on the essentials)
TViewport3D does not respond to keyboard on iPad.
How would you enable it?
My first quick attempt uses an Edit field.
Someone will write a component.
It will perhaps optimize for Bluetooth,
and work without the Edit.
{ Control iPad app with keystrokes from Bluetooth }
procedure TFormMain.FocusEditKeyDown(
Sender: TObject;
var Key: Word; var KeyChar: Char;
Shift: TShiftState);
begin
//don't write novels
//just respond to keystokes,
//(the ones available on Apple keyboard)
//skip the trimming if problems anticipated
//if not KeyChar = 'd' then
FocusEdit.Text := ''; //<-- critical ?
if KeyChar = 'a' then
begin
{ test out idle processing, cool }
Flag := True
end
else if KeyChar = 'b' then
begin
{ no problem }
DoStuff
end
else if KeyChar = 'c' then
begin
{ run stuff with ProcessMessages from OnIdle}
WantPM := True;
Flag := True;
end
else if KeyChar = 'd' then
begin
{ dare to run it directly }
WantPM := True;
DoStuff;
end
end;
The problem only shows if ProcessMessages is called in a loop several times. There have been other effects reported in FMX which only occur if triggered a certain number of times. There is a slight chance that something can be learned from the study of this case. Maybe you can come up with a real solution to the original problem.
:)
Gustav Schubert I ever used visual studio c++ for game programming. Almost game programs render graphics in IDLE block. But since everything render by our own, we don't need to do with any WM_PAINT message. Normal application does deal with WM_PAINT, and if we put some drawing instructions in idle block, drawing may not smooth when user input mouse messages. I guess this is why you creatively use empty timer handler to interrupt user continuous inputs. It not bad idea!
ReplyDeleteJeroen Wiert Pluimers About CPU get high, there's a magic API : sleep(1); Just put that inside idle block. CPU won't complain anymore. :)
Gustav Schubert I have XE6, but unfortunately, I do not have BT keyboard to test. :(
Jeroen Wiert Pluimers Yes, I know that. But up to 15 milliseconds in normal application, I think its NOT a big deal WHEN windows get idle. Just think about when user has some mouse / keyboard inputs, OnIlde won't be execute, so that no sleep 15 ms happened. Just only when application without interact with user, it will get into idle, but at this time, drawing some blinking pictures with sleep 15 ms interval, which means about 60 FPS (frame per second), is very smooth to human. Please rethink it. :)
ReplyDeleteQing-Shan Xiao For audio, you can notice the difference between 60Hz and 1000Hz very well (;
ReplyDeleteI could set up a second viewport, place it outside the screen area, make it visible, and render the thumbnails there, in a thread. When the thread finishes, it should notify the main thread that the large bitmap is ready.
ReplyDeleteI would lose the progress bar aspect of my current solution though. Have not seen a more entertaining one. As of now it is much like a watch with transparent housing. The user can see what is going on inside. The threaded solution should only be an option, maybe implemented as a future. It looks as if I had to call ProcessMessages in the thread. So the thread needs its own message queue?
The thumbnail feature has variations, e.g. on the desktop the user can choose to create a series of (high resolution) images with scripted parameter changes. To watch the application at work is fun and arguably better than watching a standard progress bar while waiting. My users have time to do it properly! They may want to interrupt the process it if it takes too long or if it is clear that the parameters need to be adjusted.
Anyway, I am interested to explore the threaded approach. Feel free to provide insight. Will not report back without seeing an example. Provide an example and I may try it out. I may even talk positive about it, if it meets all possible criteria.
You mentioned games
ReplyDeleteI know, a real game has a game loop, and gamers are concerned about frames/second. In contrast, my application sits idle and does nothing by default. Only when the user moves the mouse or uses the scroll wheel (my favorite), something is rendered. The interesting thing is that the normal computer today is fast enough to do it live. We have click speed, or mouse move speed, or scroll speed. I can feel if it is fast enough or a bit slow. When it is slow, I have to move the mouse in a less aggressive manner, or choose a lower resolution (less vertices /
triangles in the mesh), which renders way faster. The lowest resolution is very fast. Idle processing helps to reduce/compact/accumulate the flow of input messages. Rotations are fast, can be done directly by the graphics hardware. Mesh manipulations are slow, I need to control how often this is done. But mesh manipulation is at the core of this app. There is only one mesh, it changes in real time, scaring thing, insane!
On the other end, high resolution images can be rendered. I takes time, but only the result matters, no threading needed. So far I did not talk about the creation of models for 3D printing.
The game idea is that the user can explore the model, which has a real physical and mathematical meaning. The user changes parameters. Alternatively you can get updated parameters from a sensor via the network. I hope that people see that it is fast enough and find real
applications for the application. Let's say they use gnuplot and need something with higher resolution, and with a hardcoded model that is live.
Games: I am on the lookout for good game examples, need one for each platform / language, which shows how to render a single 3D mesh. My plan is to break the game loop and reuse the graphics pipeline as a starting point to port my application over. I need to see from the example how
keyboard and mouse can be used. I am ready for Java, Swift, and JavaScript (Smart Mobile), but do not have a timeline.
Hence, my project can be seen as a prototype. I were able to do it with Delphi and will continue with Delphi. But the stuff that I have prepared should swiftly port over to any 3D platform. I am convinced that this holds true. Two applications need to meet. You did C++ game development? You need a new project to explore Swift? You want the project to be open source? If I like it I may jump on it and add a model.
(I already consumed OpenGL ES 2 for Android, A Quick-Start Guide, by Kevin Brothaler, because I needed to know a little more about shaders. So I will be able to do changes, but cannot create the game from scratch.)
The game is only one component. There is a bigger picture. How the data is fed to the application, there are so many ways. I intend to keep my way of doing this closed.
:)
I have updated my previous comment with the XE4 test form.
ReplyDeleteReproducable when and only when Apple Wireless Keyboard is connected.
Tested on iPod touch with Retina and iPad mini without Retina, both on iOS 7.2.1.
Is it worth a QC?
If so, please confirm with latest Delphi first.