We're using a third-party dll provided by a hardware manufacturer, apparently the developers thought it's a great idea to show messages from dll in stead of just returning an error code.
We're using a third-party dll provided by a hardware manufacturer, apparently the developers thought it's a great idea to show messages from dll in stead of just returning an error code.
The problem:
- dll calls MessageBoxA whenever it feels like, we don't know all cases in which it display a message, the application is a kiosk, therefore it's unacceptable
Partial solution:
- currently, using https://code.google.com/p/delphi-detours-library/ we can intercept all calls to MessageBoxA and log them, however, ideally, we would want to hook just the call from the dll or somehow, figure out that the caller is the dll
Any help is appreciated!
The problem:
- dll calls MessageBoxA whenever it feels like, we don't know all cases in which it display a message, the application is a kiosk, therefore it's unacceptable
Partial solution:
- currently, using https://code.google.com/p/delphi-detours-library/ we can intercept all calls to MessageBoxA and log them, however, ideally, we would want to hook just the call from the dll or somehow, figure out that the caller is the dll
Any help is appreciated!
Depending on the use-case you could either run a thread which automatically answers / closes the message boxes. Or (evil!) you patch the dll, disabling calls to the MessageDialog functions.
ReplyDeleteOliver Funcke thank you for your input:
ReplyDeletethread, auto-click : we've solved it a bit more elegant by patching the MessageBoxA(windows api)
patch dll : this isn't very useful, if tomorrow we get a hotfix, we need to patch that as well, can't just replace the older version...
What about that API intercept toolkit that was posted a few weeks back?
ReplyDeletein theory you could do some kind of stack trace, and then walk through the caller list...
ReplyDeleteBut hmmm, I think jclDebug.pas also contained a function to get the caller, and based on the address you could check if it is from the dll. So yes it is possible but no direct ready to use code for it
Lars Fosdal unless it's DDetours(posted link above), I don't know which toolkit you're referring to
ReplyDeleteThat was the one I was thinking of.
ReplyDeleteDorin Duminica You wouldn't do this manually, of course. A small patching tool to search and destroy could automtize that.
ReplyDeleteBut, isn't it possible to the provider of the dll that these messages can be switched off optionally?
quick search (no demo yet :):
ReplyDeletevar p := jclDebug.Caller();
var dll_or_exe := jclDebug.ModuleOfAddr(p)
Does the DLL open the dialog with a parent hWnd ?
ReplyDeleteLars Fosdal not sure which toolkit you're referring to
ReplyDeleteOliver Funcke there's no option in documentation or anything remotely connected as an exported function
André Mussche do I have to install the whole JCL pack for that? ):
Lars Fosdal yes, it does pass a hwnd value different than "GetDesktopWindow()", but, we don't know all the cases in which it will show the dialog and can't rely that it will always pass one.
ReplyDeleteYou can path the DLL to use USER31.DLL instead of USER32.DLL then you create this DLL with an empty MessageBoxA function and all the functions required by the DLL
ReplyDelete---
library USER31;
uses
Windows;
function MessageBoxA(hWnd: HWND; lpText, lpCaption: PAnsiChar; uType: UINT): Integer; stdcall;
begin
// do nothing
end;
export
// export all what required by the DLL directly from Windows.pas
MessageBoxA;
begin
end.
yes and no: I made an extract of the files needed for jclDebug for my AsmProfiler, so you can also get (somewhat older) files from my dir: https://code.google.com/p/asmprofiler/source/browse/trunk/EXT/jcl/
ReplyDeleteJust put a breakpoint at the start of your version of MessageBoxA API, wait to hit it when an error occurs and look at the callstack. If there is no callstack that leads to the function you seek show CPU and hit Run Until Return (Shift+F8) and Step Over (F8) or Trace Into (F7). This will land you at the function that shows the error messages, scroll up until you locate the prolog code for this function and you will have the address of the function you need to hook. You will also have to find the types of the parameters if any and the calling conversion. If the function you seek is before the function you are just hit Run Until Return and repeat the rest of the steps.
ReplyDeleteInstead of trying to patch and tweak the dll, have you considered just ask the supplier to modify this dll, if necessary by reminding him the incongruity of having put a GUI in a dll?
ReplyDeleteI guess what you would need is API hooking like you can do with madCodeHook as described here: http://help.madshi.net/ApiCodeHooking.htm
ReplyDeleteAs others pointed out:
ReplyDelete1. Make the DLL call a patched version of user32 that redirects MessageBoxA.
2. Contact the vendor. He might not be aware of the problem. We have quite a good experience with that approach.