Marco Cantù In the older versions of Delphi, it would be a enumeration set. But since you are asking the question, I will take it that is not the case here...
Assuming it's a new type of dynamic array syntax, would it create a new array all the time or only if the reference count is right, just append? (Copy on write)
Eric Grange Isn't appending to a dynamic array not a copy process anyway? Basically it is a SetLength and Move operation, where the SetLength will most likely allocate new memory and copy the content.
Uwe Raabe strings have some compiler magic for concatenation, the question if you prefer is if dynamic arrays would get that concatenation magic as well or just "dumb" copy + append
Asbjørn Heid array comprehension and improved arrays would simplify or eliminate most generics. Main advantage being you don't have to create or manage them, and you don't have any overhead when accessing elements, plus compiler support for them can open up automatic parallelization opportunities library-based generics can't achieve .
Eric Grange Array comprehension, you mean similar to Python's list comprehension? If so I'm all for that, but a lot of my generic code doesn't operate with arrays like that. For example, I don't see how array comprehension will help with TArray.Sort or similar calls.
Asbjørn Heid sorting is one area where generics have limited value since the comparison function can't be generic outside of trivial cases, and for those generic sorting will have sub par performance. For sorting and advanced stuff you need templates, or you're going to compromise with both ugly code and lower performance.
Eric Grange Most of my code these days is not performance sensitive, so I prefer things which help me be more productive. Using generics is a big help as such, and not having to type all the type parameters explicitly would help as well. But by all means, array comprehension would help too.
Asbjørn Heid generics are fairly inconvenient compared to built array/list comprehension, which can go beyond python's approach. And templates are more convenient to use (they understand operator overloading)
Eric Grange Templates are a horrible mess, and there's a good reason why nobody (within epsilon) has implemented them outside of C++. If you want proper sorting of generics without horrible IComparer hacks, the way to do it right would be to introduce an Operator constraint, which I've been asking for since D2009.
Mason Wheeler The template approach is actually very common these days: all dynamically typed languages are for all practical purposes using "templates everywhere and all the time", even if they do not claim it that way, that is what they are doing, applying the same code to multiple concrete types.
Both templates are generics have the same goal, but the key difference is that with generics, the onus of typing correctness falls on full, explicit and static declaration by the developer, while with generics the onus of typing correctness falls on the compiler and/or runtime engine.
Eric Grange Yes, with templates, the onus of getting static typing correct falls on the programmer--where it belongs in a statically typed language--as far as it does for the rest of the language. (Though they can be type inferenced, just like anything.)
Whereas with templates, you take something that's nominally a statically-typed language and without any warning drop a big steaming pile of duck typing right in the middle of it, with all of its attendant headaches. You take one of the most significant advantages of a static compiler--good, robust early error detection--and throw it right out the window in the middle of one of the most complex phases of the compilation process, which is why C++ template error messages tend to be horrendous, incomprehensible masses of compiler-vomit. (And yes, I know about clang. I also know it took decades to come up with it.)
Marco Cantù is "+" it smart with copy-on-write? and is it smart like for strings where multiple concatenations get grouped together? (give them a finger and they'll take the arm)
Not sure, honestly, if it creates a new array and concatenates them, instead. Because you can also write: di := di + di; and many other combinations...
I'm asking because the compiler is very capable when doing that kind of operations for strings.
Whereas when using operator overloading, each operator is taken isolation as syntax sugar over a function call, and thus it becomes quickly very inefficient. Besides custom initialization, operator overloading limitations are one of the reasons why it is impossible to recreate a very efficient ANSI string type from scratch for recent Delphis, and why it was impossible to recreate an efficient UNICODE string type pre-Delphi 2009.
If the new syntax has first-class operator support from the compiler like strings do, then it is great news, because it is impossible to recreate that support in libraries.
Eric Grange Copy-On-Write semantics for string is a very specific support. Yes, it can help performance, no denial. But COW helps performance only in some specific scenarios, most string operations work regardless...
Marco Cantù COW helps with concatenation and local strings, but what I'm referring is also the "StrNCat" ability, ie. for a string the compiler will recognize
di := di + xxx + yyy;
and compile it to something like "di.Append(xxx, yyy)" would be, and it will also compile
di := xxx + yyy + zzz;
as something like "di := StrNCat(xxx, yyy, zzz)" would be.
Those last two mechanism aren't related to COW, but can help tremendously vs a library using operator-overloading, which can only manage to get code compiler as
di := Concat( Concat( di, xxx), yyy);
The above involves extra memory allocations, copies, reference count management, and an implicit "Schlemiel the Painter algorithm" (ie. performance degrades faster than the number of operators, and for strings/arrays the complexity is O(n²) with n the number of operators)
In contrast for strings, the StrNCat combine concatenations and appends into a single call, with a single allocation, and a single copy (complexity stays O(n)) Of course what can be said for '+' can also applies to all other operators when processing vectors, matrixes, etc.
There really are gems on the string-support side of the Delphi compiler, would be nice if they were available not just for strings.
If done right, a souped up dynamic array could replace favorably TList<> and TStack<> in many cases, and not just in terms of performance, but also in terms of convenience: you "naturally" don't have to create a dynamic array and you don't have to free it either.
It can also be great basis for automatic parallelization, as the compiler can "know" things about dynamic arrays in ways it can't when using generics (not without ugly and fragile peepholes optimizations anyway), so it opens up opportunities for automatic SIMD and for automatic "parallel for".
And if you're still reading, a breaking but very powerful change would be to make dynamic arrays into true reference types (indirections), currently they are half-way between reference and value types, and that limits their usefulness, as whenever you resize an array, there is a chance (a chance!) that it will be copied, and that all existing "references" will point to the "old" array before the resize.
Marco Cantù In the older versions of Delphi, it would be a enumeration set. But since you are asking the question, I will take it that is not the case here...
ReplyDeletemost likely an array in XE7? (:
ReplyDeleteThis is currently valid syntax for type "set of byte", or sets of byte subranges that include 4 and 5.
ReplyDeleteBased on image #3 from https://plus.google.com/+SamShaw_tw/posts/cPk1ptBp5Rj?cfem=1 I am guessing that in XE7, this will also be valid syntax for type "array of integer" and related types.
I'm surprised that Marco doesn't know LOL
ReplyDeletesomething to replace
ReplyDeletes := s + #4#5
My immediate reaction was a set (of byte for example as Mason Wheeler said). Then again my Delphi is a bit rusty :)
ReplyDeleteset of byte or a similar subrange.
ReplyDeleteMarco Cantù 2D Vector here?
ReplyDeleteAssuming it's a new type of dynamic array syntax, would it create a new array all the time or only if the reference count is right, just append? (Copy on write)
ReplyDeleteEric Grange Isn't appending to a dynamic array not a copy process anyway? Basically it is a SetLength and Move operation, where the SetLength will most likely allocate new memory and copy the content.
ReplyDeleteWhile I'm all for better dynamic array support, I'm really longing for better type inference when it comes to generics.
ReplyDeleteUwe Raabe strings have some compiler magic for concatenation, the question if you prefer is if dynamic arrays would get that concatenation magic as well or just "dumb" copy + append
ReplyDeleteAsbjørn Heid array comprehension and improved arrays would simplify or eliminate most generics. Main advantage being you don't have to create or manage them, and you don't have any overhead when accessing elements, plus compiler support for them can open up automatic parallelization opportunities library-based generics can't achieve .
ReplyDeletedi could be any type - eventually resulting in errors when compiling :-)
ReplyDeleteEric Grange Array comprehension, you mean similar to Python's list comprehension? If so I'm all for that, but a lot of my generic code doesn't operate with arrays like that. For example, I don't see how array comprehension will help with TArray.Sort or similar calls.
ReplyDeleteAsbjørn Heid sorting is one area where generics have limited value since the comparison function can't be generic outside of trivial cases, and for those generic sorting will have sub par performance. For sorting and advanced stuff you need templates, or you're going to compromise with both ugly code and lower performance.
ReplyDeleteBoth could be any properties of two different objects or records, referenced by the beloved with.
ReplyDeleteEric Grange Most of my code these days is not performance sensitive, so I prefer things which help me be more productive. Using generics is a big help as such, and not having to type all the type parameters explicitly would help as well. But by all means, array comprehension would help too.
ReplyDeleteAsbjørn Heid generics are fairly inconvenient compared to built array/list comprehension, which can go beyond python's approach. And templates are more convenient to use (they understand operator overloading)
ReplyDeleteEric Grange On the other hand, we have generics and we all know how long it takes to get new features in: http://qc.embarcadero.com/wc/qcmain.aspx?d=740.
ReplyDeleteEric Grange Templates are a horrible mess, and there's a good reason why nobody (within epsilon) has implemented them outside of C++. If you want proper sorting of generics without horrible IComparer hacks, the way to do it right would be to introduce an Operator constraint, which I've been asking for since D2009.
ReplyDeleteMason Wheeler The template approach is actually very common these days: all dynamically typed languages are for all practical purposes using "templates everywhere and all the time", even if they do not claim it that way, that is what they are doing, applying the same code to multiple concrete types.
ReplyDeleteBoth templates are generics have the same goal, but the key difference is that with generics, the onus of typing correctness falls on full, explicit and static declaration by the developer, while with generics the onus of typing correctness falls on the compiler and/or runtime engine.
Eric Grange Yes, with templates, the onus of getting static typing correct falls on the programmer--where it belongs in a statically typed language--as far as it does for the rest of the language. (Though they can be type inferenced, just like anything.)
ReplyDeleteWhereas with templates, you take something that's nominally a statically-typed language and without any warning drop a big steaming pile of duck typing right in the middle of it, with all of its attendant headaches. You take one of the most significant advantages of a static compiler--good, robust early error detection--and throw it right out the window in the middle of one of the most complex phases of the compilation process, which is why C++ template error messages tend to be horrendous, incomprehensible masses of compiler-vomit. (And yes, I know about clang. I also know it took decades to come up with it.)
I see the discussion diverged a little, that's good. In any case the answer I was hinting at is:
ReplyDeletevar
di: array of Integer;
and, yes, you'll need to have Delphi XE7 to compile it...
Marco Cantù is "+" it smart with copy-on-write? and is it smart like for strings where multiple concatenations get grouped together? (give them a finger and they'll take the arm)
ReplyDeleteDynamic arrays have no COW implementation... at least for now. So it is not that smart.
ReplyDeleteSorry, yes no COW, meant for "resize in place", ie. if "di" is referenced nowehere else, then the snippet will be equivalent to
ReplyDeleten := Length(di)
SetLength(di, n+2);
di[n-2] := 4;
di[n-1] := 5;
and won't generate a new array, just resize it.
Not sure, honestly, if it creates a new array and concatenates them, instead. Because you can also write:
ReplyDeletedi := di + di;
and many other combinations...
I'm asking because the compiler is very capable when doing that kind of operations for strings.
ReplyDeleteWhereas when using operator overloading, each operator is taken isolation as syntax sugar over a function call, and thus it becomes quickly very inefficient. Besides custom initialization, operator overloading limitations are one of the reasons why it is impossible to recreate a very efficient ANSI string type from scratch for recent Delphis, and why it was impossible to recreate an efficient UNICODE string type pre-Delphi 2009.
If the new syntax has first-class operator support from the compiler like strings do, then it is great news, because it is impossible to recreate that support in libraries.
Eric Grange Copy-On-Write semantics for string is a very specific support. Yes, it can help performance, no denial. But COW helps performance only in some specific scenarios, most string operations work regardless...
ReplyDeleteMarco Cantù COW helps with concatenation and local strings, but what I'm referring is also the "StrNCat" ability, ie. for a string the compiler will recognize
ReplyDeletedi := di + xxx + yyy;
and compile it to something like "di.Append(xxx, yyy)" would be, and it will also compile
di := xxx + yyy + zzz;
as something like "di := StrNCat(xxx, yyy, zzz)" would be.
Those last two mechanism aren't related to COW, but can help tremendously vs a library using operator-overloading, which can only manage to get code compiler as
di := Concat( Concat( di, xxx), yyy);
The above involves extra memory allocations, copies, reference count management, and an implicit "Schlemiel the Painter algorithm" (ie. performance degrades faster than the number of operators, and for strings/arrays the complexity is O(n²) with n the number of operators)
In contrast for strings, the StrNCat combine concatenations and appends into a single call, with a single allocation, and a single copy (complexity stays O(n))
Of course what can be said for '+' can also applies to all other operators when processing vectors, matrixes, etc.
There really are gems on the string-support side of the Delphi compiler, would be nice if they were available not just for strings.
Answering my own question: there is a DynArrayCatN, so that's nice, but... there is some horrible code involved before the DynArrayCatN...
ReplyDeleteSo I guess we have a good basis, but room for improvement in dynamic array concatenation.... I think it is a relevant feature.
ReplyDeleteYes, very relevant IMHO.
ReplyDeleteIf done right, a souped up dynamic array could replace favorably TList<> and TStack<> in many cases, and not just in terms of performance, but also in terms of convenience: you "naturally" don't have to create a dynamic array and you don't have to free it either.
It can also be great basis for automatic parallelization, as the compiler can "know" things about dynamic arrays in ways it can't when using generics (not without ugly and fragile peepholes optimizations anyway), so it opens up opportunities for automatic SIMD and for automatic "parallel for".
And if you're still reading, a breaking but very powerful change would be to make dynamic arrays into true reference types (indirections), currently they are half-way between reference and value types, and that limits their usefulness, as whenever you resize an array, there is a chance (a chance!) that it will be copied, and that all existing "references" will point to the "old" array before the resize.
Eric Grange If only I would find the same level of array support in Delphi that does exist in your DWS/SmartPascal version, I would be delighted. :)
ReplyDelete