OH GOD NOT THIS THING AGAIN. Did you READ it? I wonder if it was you who cited this paper to me in a discussion regarding the speed of Delphi. Read the paper... one of the "benchmarks" is AN EMPTY LOOP and another is PRINTING HELLO WORLD 5000 TIMES.
Then for more fun, look up the journal. Guess what? It's one of those places that will print any paper for a fee. Then for some more fun check out the author and their publications. At least the last time I checked, every single paper they'd ever published was in a pay-to-publish journal.
"Guess who fares very very well?" I love this comment?
"Guess who didn't read the paper with a critical eye?" is my riposte.
Second para from the introduction,
"Further, most of these programming languages are object-oriented programming languages [1]. The languages C# 2013, Delphi XE6 are managed languages. The Python is unmanaged language. Memory method of unmanaged programming languages is not automatic. Therefore they are not safe."
I'm amazed anybody got past this point of the "paper".
Our love for and hate of Delphi should not be governed by junk like this. I would prefer that Chad Hower removed the whole thing and don't let this crap propagate.
The managed vs unmanaged appears to be a simple accidental transpose, ie typo but embarrassing. Not a critical flaw.
I fully agree that its not very comprehensive but it does test basic sorts etc. Hello World might be stupid but it establishes a shared workload inside a loop that should be rather common among all to better focus on the measurement of the loop overhead itself.
The blank loop? Thats only one of about half a dozen things there. Certainly not the core mark. And again useful to establish and compare against the others.
Would be great to see someone update/redo something similar.
Joseph Mitzen No I did not post it in reply to you anywhere. I just ran across it in someone elses comment somewhere else.
Chad Hower That's not a simple translation or grammar error (which are big snafus in a paper anyways). Those are plain wrong and ignorant statements that gives you enough information to not take that thing seriously. That implies this paper wasn't reviewed by a proper peer.
An empty loop in a benchmark is not useful, at all.
Chad Hower I assume you mean allocation cost. Why would you count that? This is a benchmark about the compute code. And for LU the pattern is to decompose once, and back solve multiple times. Also, when such functions are important, the matrices are large and the compute time dominates. Finally, the big weakness of Delphi memory management is actually when multi threading where its very hard to find a good MM for large processor count machines, NUMA machines. I wrote my own to deal with this, but in any case you preallocate to avoid allocation cost becoming significant.
Of course if you mean the cost to access memory, that's invariably determined by the hardware.
And if you mean memory usage in terms of how much memory then it's obviously identical everywhere. For real world scenarios the overhead is always trivial.
And just for what it is worth, this example comes from a real program, used around the world to design offshore structures, and led to me porting a chunk of code from Pascal to C for the clear performance gain.
Chad Hower where did David said that "ram is the same"? David is pointing out that if you are measuring compute code, you shouldn't need to measure RAM. And that phrase about the memory usage means, at least in my understanding, that a properly designed test should use about the same memory no matter what language you use - the overhead imposed by your compiler shouldn't matter.
One would hope there are not other factors. So do you want to continue to post assumptions (which are probably valid) and spend more time than it would take to just show the results?
If you are that tight on time, fine say it. But info is info and if you firmly believe in your assumptions, then I simply don't see why the resistance.
Joseph Mitzen That is probably one of the most absurd statements I've seen today. He can't post simple memory measurements or else its like the earth being flat?
Chad Hower you are missing the point completely. For the class of problems being tested in the linked "paper" memory usage is irrelevant. And Windows Task Manager is not a good measurement tool.
Chad Hower I'm stating it as a known fact that in all the languages under consideration, arrays of double precision values are represented identically. Contiguous arrays of IEEE754 double precision values.
Chad Hower He's proven via logic that memory usage can't be different. You're insisting he do the measurement anyway.
I used to have a boss who always said "There might be a perfectly logical reason for that but you just don't know what it is" every time I found something our company was doing that I considered dumb. While technically true, I believed he was only saying it because he never, ever wanted to criticize anything the company did. I dubbed it "Schwartzman's Law" after him.
One time after hearing it used once too often, I left a meeting and created "Mitzen's Corollary To Schwartzman's Law":
"There may well be a perfectly logical reason for something and one may simply not know what it is, but if no one who cites Schwartzman's Law can suggest what that reason might be, it probably doesn't exist."
I'm invoking a version of Mitzen's Corollary To Schwartzman's Law here. You need to suggest a plausible reason why memory use might be different in order to justify making measurements for something that otherwise would seem determined purely through logical argument.
Otherwise I'm reminded when I got cited Schwartzman's Law after explaining to Schwartzman that our company had to be printing the billing address rather than the stated ship from address on our bill of lading forms. I kept arguing and got back, "How do you KNOW?" I lost my temper and blurted out in frustration, "Because they can't be shipping steel garbage cans out of a 3rd floor New York City office suite on Broadway!"
At our next meeting, Schwartzman informs me that "You were right by the way"; yes, he'd actually made some phone calls to verify that no one was chucking garbage cans out of a 3rd floor window to a waiting delivery truck below. I had to grip the arm rests of my chair to keep from saying something I'd regret.
Of course task manager is a bad way to measure RAM. You know an even worse way to measure ram? By guessing.
There is more than just floating math. It would be interesting to see the diff in RAM between C and Delphi and it should be brain dead simple for you to do since you did the original code.
Chad Hower "There is more than just floating math. " Did you look at his code? Some floating point arrays and a few integers; that's it. There are no other data structures that might result in memory differences.
This is a direct response to the paper that is the subject of this post. The author implemented Sci-Mark and found that both C# and Java beat Delphi handily. In a long-running discussion at a now-defunct Delphi blog, readers were able to get Delphi to match or just beat C#, but only at the expense of turning the Delphi code into something that resembled C more than Pascal. They could not beat the Java code.
Delphi,. C++, C#, FreePascal benchmark with prime numbers:
OH GOD NOT THIS THING AGAIN. Did you READ it? I wonder if it was you who cited this paper to me in a discussion regarding the speed of Delphi. Read the paper... one of the "benchmarks" is AN EMPTY LOOP and another is PRINTING HELLO WORLD 5000 TIMES.
ReplyDeleteThen for more fun, look up the journal. Guess what? It's one of those places that will print any paper for a fee. Then for some more fun check out the author and their publications. At least the last time I checked, every single paper they'd ever published was in a pay-to-publish journal.
This paper is worthless.
"Guess who fares very very well?" I love this comment?
ReplyDelete"Guess who didn't read the paper with a critical eye?" is my riposte.
Second para from the introduction,
"Further, most of these programming languages are object-oriented programming languages [1]. The languages C# 2013, Delphi XE6 are managed languages. The Python is unmanaged language. Memory method of unmanaged programming languages is not automatic. Therefore they are not safe."
I'm amazed anybody got past this point of the "paper".
Our love for and hate of Delphi should not be governed by junk like this.
ReplyDeleteI would prefer that Chad Hower removed the whole thing and don't let this crap propagate.
Cool but need an update!
ReplyDeleteThe managed vs unmanaged appears to be a simple accidental transpose, ie typo but embarrassing. Not a critical flaw.
ReplyDeleteI fully agree that its not very comprehensive but it does test basic sorts etc. Hello World might be stupid but it establishes a shared workload inside a loop that should be rather common among all to better focus on the measurement of the loop overhead itself.
The blank loop? Thats only one of about half a dozen things there. Certainly not the core mark. And again useful to establish and compare against the others.
Would be great to see someone update/redo something similar.
Joseph Mitzen No I did not post it in reply to you anywhere. I just ran across it in someone elses comment somewhere else.
Chad Hower That's not a simple translation or grammar error (which are big snafus in a paper anyways). Those are plain wrong and ignorant statements that gives you enough information to not take that thing seriously. That implies this paper wasn't reviewed by a proper peer.
ReplyDeleteAn empty loop in a benchmark is not useful, at all.
Please let that thing go quietly.
This one is about SmallTalk
ReplyDelete, but lists Delphi and it ranks well as well in different criteria.
smalltalkrenaissance.wordpress.com - Smalltalk’s Proven Productivity
Chad Hower https://plus.google.com/103246155735524926641/posts/MKTLEZoyE5R
ReplyDeleteplus.google.com - Following on from the recent discussion (https://plus.google.com/118000060693...
David Heffernan Would be interesting to see memory usage in those tests too to see if the other article has any merit on that point.
ReplyDeleteChad Hower we are operating on large floating point arrays. Memory usage will be identical in all languages.
ReplyDeleteOnly if you arent counting general overhead of the compiler and libraries which are the reason for measuring it.
ReplyDeleteChad Hower I assume you mean allocation cost. Why would you count that? This is a benchmark about the compute code. And for LU the pattern is to decompose once, and back solve multiple times. Also, when such functions are important, the matrices are large and the compute time dominates. Finally, the big weakness of Delphi memory management is actually when multi threading where its very hard to find a good MM for large processor count machines, NUMA machines. I wrote my own to deal with this, but in any case you preallocate to avoid allocation cost becoming significant.
ReplyDeleteOf course if you mean the cost to access memory, that's invariably determined by the hardware.
And if you mean memory usage in terms of how much memory then it's obviously identical everywhere. For real world scenarios the overhead is always trivial.
And just for what it is worth, this example comes from a real program, used around the world to design offshore structures, and led to me porting a chunk of code from Pascal to C for the clear performance gain.
ReplyDeleteBecause thats why we run tests. To confirm or deny our theories. You say ram is the same, so there is no reason not to post results to back it.
ReplyDeleteChad Hower where did David said that "ram is the same"? David is pointing out that if you are measuring compute code, you shouldn't need to measure RAM. And that phrase about the memory usage means, at least in my understanding, that a properly designed test should use about the same memory no matter what language you use - the overhead imposed by your compiler shouldn't matter.
ReplyDeleteLeonardo Herrera "And if you mean memory usage in terms of how much memory then it's obviously identical everywhere. "
ReplyDeleteWhats the big deal? Seriously? Its that hard to measure memory and support/deny assumptions?
Chad Hower an array of double precision values is represented identically in all of the languages we are talking about. Or do you disagree?
ReplyDeleteOne would hope there are not other factors. So do you want to continue to post assumptions (which are probably valid) and spend more time than it would take to just show the results?
ReplyDeleteIf you are that tight on time, fine say it. But info is info and if you firmly believe in your assumptions, then I simply don't see why the resistance.
Chad Hower Because it's like asking someone to prove the Earth isn't flat or 1+1 isn't three.
ReplyDeleteJoseph Mitzen That is probably one of the most absurd statements I've seen today. He can't post simple memory measurements or else its like the earth being flat?
ReplyDeleteChad Hower you are missing the point completely. For the class of problems being tested in the linked "paper" memory usage is irrelevant. And Windows Task Manager is not a good measurement tool.
ReplyDeleteChad Hower I'm stating it as a known fact that in all the languages under consideration, arrays of double precision values are represented identically. Contiguous arrays of IEEE754 double precision values.
ReplyDeleteChad Hower He's proven via logic that memory usage can't be different. You're insisting he do the measurement anyway.
ReplyDeleteI used to have a boss who always said "There might be a perfectly logical reason for that but you just don't know what it is" every time I found something our company was doing that I considered dumb. While technically true, I believed he was only saying it because he never, ever wanted to criticize anything the company did. I dubbed it "Schwartzman's Law" after him.
One time after hearing it used once too often, I left a meeting and created "Mitzen's Corollary To Schwartzman's Law":
"There may well be a perfectly logical reason for something and one may simply not know what it is, but if no one who cites Schwartzman's Law can suggest what that reason might be, it probably doesn't exist."
I'm invoking a version of Mitzen's Corollary To Schwartzman's Law here. You need to suggest a plausible reason why memory use might be different in order to justify making measurements for something that otherwise would seem determined purely through logical argument.
Otherwise I'm reminded when I got cited Schwartzman's Law after explaining to Schwartzman that our company had to be printing the billing address rather than the stated ship from address on our bill of lading forms. I kept arguing and got back, "How do you KNOW?" I lost my temper and blurted out in frustration, "Because they can't be shipping steel garbage cans out of a 3rd floor New York City office suite on Broadway!"
At our next meeting, Schwartzman informs me that "You were right by the way"; yes, he'd actually made some phone calls to verify that no one was chucking garbage cans out of a 3rd floor window to a waiting delivery truck below. I had to grip the arm rests of my chair to keep from saying something I'd regret.
Of course task manager is a bad way to measure RAM. You know an even worse way to measure ram? By guessing.
ReplyDeleteThere is more than just floating math. It would be interesting to see the diff in RAM between C and Delphi and it should be brain dead simple for you to do since you did the original code.
Chad Hower " That is probably one of the most absurd statements I've seen today."
ReplyDeleteI get that a lot.
Chad Hower "There is more than just floating math. " Did you look at his code? Some floating point arrays and a few integers; that's it. There are no other data structures that might result in memory differences.
ReplyDeleteHere are some additional benchmarks:
ReplyDeletewebandlife.blogspot.com - C# performance v.s. Delphi performance
This is a direct response to the paper that is the subject of this post. The author implemented Sci-Mark and found that both C# and Java beat Delphi handily. In a long-running discussion at a now-defunct Delphi blog, readers were able to get Delphi to match or just beat C#, but only at the expense of turning the Delphi code into something that resembled C more than Pascal. They could not beat the Java code.
Delphi,. C++, C#, FreePascal benchmark with prime numbers:
http://blog.digitaltundra.com/?p=348
Here Lennart Aasenden is depressed to see Javascript outperforming Delphi:
https://web.archive.org/web/20110404193045/https://delphimax.wordpress.com/2011/03/29/delphi-beaten-by-javascript
Here's Asbjørn Heid implementing Tiger Hash in Delphi, Java, C# and C++:
https://plus.google.com/+Asbj%C3%B8rnHeid/posts/87hBxYehavM
Chad Hower There is no difference because the arrays are represented identically in memory.
ReplyDeleteChad Hower "There is more than just floating math" - at least in the paper you provided, no, there is not, and that's what we are trying to tell you.
ReplyDelete