HEEEELP!
HEEEELP!
Using TComPort to communicate with hardware device, we're experiencing a lot of incomplete packet responses, especially under Windows 7, Windows XP is better, but we're doing it under W7.
When using 1024bytes for input and output buffer, we're losing about 20% of the packet data(the end of it), when using a 24bytes input and output buffer, it seems to loose less and less frequently.
The packets are around 14bytes max and we're clearing both the input and output buffer after each request and response.
Anyone got any idea?
UPDATE:
using Delphi 2010
Using TComPort to communicate with hardware device, we're experiencing a lot of incomplete packet responses, especially under Windows 7, Windows XP is better, but we're doing it under W7.
When using 1024bytes for input and output buffer, we're losing about 20% of the packet data(the end of it), when using a 24bytes input and output buffer, it seems to loose less and less frequently.
The packets are around 14bytes max and we're clearing both the input and output buffer after each request and response.
Anyone got any idea?
UPDATE:
using Delphi 2010
Very odd, but in these days, it is necessary to ask: Is the port inside the box? Or is it a USB COM port device?
ReplyDeleteIt's using USB
ReplyDeleteThat could be a problem. Some USB serial adapters work better than others. The issue is packet fragmentation. If you need to support USB COM port devices, and cannot control which brands and models, then you need to rework your coding to allow for possibly long gaps in message packets.
ReplyDeletenot sure I understand, can you please rephrase it?
ReplyDeleteI did a project with a ComPort last year and I ended up with ComPort Library: http://comport.sf.net. It saved me a lot of trouble with TComPort and comes with configuration dialogs etc. Very easy to use.
ReplyDeleteMaybe something for you as well?
I have been using TCommPortDriver in ComDrv32.pas
ReplyDeleteii is free for more then 15 years without any problems
Look for it on the net.
P.S use it with up to Delphi 7 otherwise you will have problems !!!
And may be this is your case? xe and delphi <then xe
Birger Jansen that's exactly the one we're using.
ReplyDeletesorry, forgot to mention, Delphi 2010
ReplyDeleteOK. As I understand it, the USB device is apparently polled, and so it delivers whatever is in its buffer at the time. Your message protocol may have a specific termination character for a packet, but the USB device knows nothing of this. Anyway, the external device sendds its 14 bytes, and after maybe 10 bytes are received by the USB device, its driver collects the buffer content and passes it to you. The rest of the bytes would, we hope, be delivered in the next sample.
ReplyDeleteBill Meyer got'cha, right now, there's no time to figure these things out, but will investigate really soon.
ReplyDeleteThere are USB analyzers available, which may help. I have not used TCommPort (my favorite is TSerial), but if you are receiving on an event-driven basis, and have a timeout value in place, it may be that increasing the timeout will solve the problem. Many years ago, I also had a problem in a system using BDE (D2), in which the BDE operation would apparently hog the CPU for 40-60 mS at a time, and we would then simply drop characters.
ReplyDeleteBill Meyer the communication is in sync
ReplyDeleteWriteBytes
Wait for minimum required bytes in buffer
ReadBytes
P.S. the hardware guys said that the hw unit is responding within 0.5ms
ReplyDeleteAs i said i tried using it with xe the unicode problems did
ReplyDeletenot work for me so i revert back to Delphi 6!!!
shlomo abuisak I'm not writing "chars", I'm writing "bytes" directly.
ReplyDeleteWe use TMS Asynch32 in many of our projects without problems. Unfortunately it is not free, but I have found over the past few years that it is very stably. Whether using Onboard/USB/Ethernet COM Redirectors. http://www.tmssoftware.com/site/async32.asp
ReplyDeleteTake care when running any serial component within a thread. We used to use TCiaComPort and this would open a hidden window for processing messages that caused us great grief :)
I have never used TComPort, does it fire an event when bytes are received or on a time out?
Guys the problem is XE and below max 7.
ReplyDeleteThe components is proven!!!
If the project will be compiled with 7 and under there will not be a any problem !!! this is checked already.
Sean Bertinotti yeah, TComPort has events, but we're not waiting for events, we're waiting for a minimum of bytes to be available in the input buffer, then grab those, parse the packet and act accordingly.
ReplyDeleteshlomo abuisak Well, that's disturbing! I assume when you say it's the Delphi version, all tests have been done on the same computer, same OS?
ReplyDeleteDorin i took some of my 6 rs232 proj to xe and went back!!!!
ReplyDeleteGoogle is not helping. I know that some years ago, Atmel had some info on using USB adapters with their (then) RS-232 based development cards for the AVR. That was where I first learned of the fragmentation issues. However, these days most of the hits I get are with respect to Raspberry Pi, which seems to have some issues in that area. But if all is well with the code built with D7... not sure what to suggest.
ReplyDeleteDorin Duminica Can you check the XE version on a computer with an internal COM port? It would reduce somewhat the range of variables in the problem.
ReplyDeleteDorin i mite have a solution for you. i saw that i did use with xe
ReplyDeleteCPort.pas with TComPort it seems to work ok on xe.
It is free
(****************************************************
* ComPort Library ver. 4.11 *
* for Delphi 5, 6, 7, 2007-2010,XE and *
* C++ Builder 3, 4, 5, 6 *
* written by Dejan Crnila, 1998 - 2002 *
* maintained by Lars B. Dybdahl, 2003 *
* Homepage: http://comport.sf.net/ *
* *
* Brian Gochnauer Oct 2010 *
* Removed ansi references for backward compat *
* Made unicode ready *
***************************************************)
shlomo abuisak thank you, but that's exactly what I already use and have issues with, now, I don't know for sure if it's a problem with the component or with windows, driver, etc.
ReplyDeleteI am using TCiaCompPort for years without ever loosing any chanracter. It is interrupt driven so it is very hard to loose anything.
ReplyDeletewww.mestdagh.biz/soft_components.html
François Piette it seems that we can reduce the number of invalid/incomplete packets by raising the minimum number of expected bytes, clearing input and output buffers before sending request and by closing the com port after response received, now we get 1 failure in about 20-30 requests...
ReplyDeleteMichael Thuma I have little hw knowledge, when I asked about the configuration required for flow control, I was told to disable all, for some reason, I didn't ask why... will have to investigate further... currently we've got most of the things buttoned down(for the time being).
Thank you all for your input!!
Dorin Duminica
ReplyDeleteMake sure you have the proper driver for the USB to serial installed. My own experience in this field was really painful, and it took me a lot of experimentation to find the right driver - the latest ones seem to work worse then older ones. Essentially there is just one USB to serial chip manufacturer which also provides the drivers but the devices are sold under many different brands. If you want I can look up the driver that worked for me and send it to you tomorrow.
We don't use USB to Serial if we can avoid it, and we go to great lengths to avoid it.
ReplyDeleteOff-topic: Isn't it strange that there is so few USB based scanners? You'd think that was a no-brainer.
ReplyDeleteLars Fosdal I once thought a SCSI scanner was a good investment. Then they didn't update drivers when XP shipped. Lately, there are few scanners at low prices, probably because there arfe so many all-in-ones. The whole scanner business is a little funky. Glad I am not in it!
ReplyDeleteYou could easily put flash memory in the USB device that would solve the buffering needs.
ReplyDeleteMichael Thuma Oer the original message, Dorin used 1024 byte bufferes, and had huge data loss. With 24 byte buffers, less data loss. So although buffering is a factor, there is something counterintuitive about it. As to why anything more than a CreateFile is used, in my own experience, there were other issues of interest.
ReplyDeleteMichael Thuma But really, there is little excuse these days for failing to handle any serial baud rate up to at least 115K. Anything higher can justifiably be denied in the absence of an RS-422/485 adapter card, and won't function anyway, beyond a very few feet on RS-232. Any USB converter which can't handle 115K without intervention in adjusting IRQ selections should be tossed.
ReplyDeleteThe USB converter should stream to in-device file, then raise IRQ when file is complete.
ReplyDeleteLars Fosdal Ah, but the converter knows nothing of the protocol, so how does it recognize "complete"?
ReplyDeleteNo data for n milliseconds? Most scanners understand what they scan, up to and including validation.
ReplyDeleteLars Fosdal If the other end is using a USB converter, too, that can be problematic. ;) But how do you notify the USB converter (which is supposed to be transparent to the operation in the first place) to monitor the interval of quiet time?
ReplyDeleteLars Fosdal And that brings us back to my reply to Michael about why not just use CreateFile....
ReplyDeleteWell, I'm not so sure I'd want an USB device to create files on my disk. If it's in their own storage - fine - but not in mine. One thing is access rights, another is the risk of a runaway device.
ReplyDeleteMichael Thuma Those of us who came from CP/M knew painfully well the problems in the 8250 UART which became the standard for the PC. The main one was that if you wanted to run at more than 9600 baud, you really were forced to use an interrupt-driven solution, because of internal problems in the chip which resulted in uncertain behavior in the polled status bits, and therefore to dropped characters. Admittedly, the hardware these days no longer suffers such problems, in the very few motherboards which still have COM ports. (A whole other discussion which makes me foam at the mouth.) In many industrial systems, the communications run at 38.4KBaud or higher. Badly implemented drivers can cause real headaches. Also, depending on milliseconds of idle time, when you do not have control over the physical devices, is fraught with peril. At 38.4Kb, you are getting new chars/bytes every 1/4 mS (8 bits plus 1 start and 1 stop). In embedded systems, it is quite reasonable to say that 5mS of idle time is a message terminator. Don't bank on making that work with Windows.
ReplyDeleteWe mostly use Intermec scanners, and they are quite capable and even programmable. They still come with serial devices though, or as Bluetooth Wedge scanners.
ReplyDeleteMichael Thuma Figuring out how the controller behaves is challenge enough, when you have full control. And in Windows, through a driver, it becomes an impenetrable mystery, subject to intuition and observed behaviors.
ReplyDeleteLars Fosdal I think the reason we don't see USB scanners is that they would need more horsepower and large internal buffers. Scanning seems to be a synchronous business, and that is not a good fit with USB.
ReplyDeleteWell, the Intermec we use is an imaging scanner.
ReplyDeletehttp://www.intermec.com/products/scansr61bex/index.aspx
which already have a lot of horsepower.
We use it for 3" tall Code 128 barcode labels up to six pallets high shelves, in -26C with frosting. Pretty clever devices, but not available with a USB interface.
Curiosa from the DOS age - My trusty interrupt driven serial port routines from 1988: http://goo.gl/hmXBG :)
Note the capitalized keywords. It took me years to unlearn that, after syntax highlighting was introduced :)
Dorin Duminica Is the code top secret? If not why not post some of it somewhere?
ReplyDeleteFtdi usb to serial chipset is reliable. Many other USB chipset or drivers are unreliable.
ReplyDeleteW
jeff weir it's my customer's IP, can't share without approval.
ReplyDeletewill let you guys know when I find the issue, thought it's something easy to spot...
OK, figured it out, it's all my fault... what is annoying is that it is so simple:
ReplyDelete- basically, at each command sent to hardware, nothing can continue until the hardware responds(or timeout kicks in which usually means that we got a problem), so here's what happens(this is the max I can share)
procedure ComWriteReadRaw(AOutBytes: TBytes; out AInBytes: TBytes);
var
//...
LDataLen: Integer;
begin
//...
// Here's where we send bytes to COM port
LDataLen := Length(AOutBytes);
ComPort1.Write(AOutBytes[0], LDataLen);
repeat
LDataLen := ComPort1.InputCount; // how many bytes in input buffer?
if LDataLen > 0 then begin // <- ERROR
// ...
SetLength(AInBytes, LDataLen);
ComPort1.Read(AInBytes[0], LDataLen);
// ...
end;
until ;
//...
end;
NOTE: this method does NOT return the control to the caller unless we get a response from hardware or a timeout occurs.
Now, if you look at "ERROR", you can see that I should have set the minimum required amount of bytes to a higher value... anyhu', we've figured out the minimum amount of bytes that the hardware will always respond with and based on command, increase it...
So, all my fault... the reason for the error is that the input buffer might only have half of the bytes we actually need when we query for LDataLen > 0...
Found it a bit earlier, but wasn't sure that this is the only place where the issue exists... oh well.
Thank you all for your input, I enjoyed reading all your comments and I've learned a few more things from you!