I need to download a binary file > 2GB (about 3.5GB) on Win64.
I need to download a binary file > 2GB (about 3.5GB) on Win64.
I know TMemoryStream etc is broken, because it uses LongInt instead of NativeInt.
Is there a fix to the RTL that I can download. I know I can write my own TMemoryStream, but I'd much rather use a patch to the RTL that fixes in all places where the streams are used.
I'm using Tokyo BTW, very miffed that this is still not fixed.
Yes, I know TMemoryStream is an anti-pattern and I agree: https://stackoverflow.com/questions/34801278/delphi-out-of-memory-error-when-i-load-file-to-memory-stream
But right now I just need to have TMemoryStream fixed.
Even using TFileStream I have to load the file in chucks:
procedure TForm2.BtnOpenFileClick(Sender: TObject);
var
MS: TFileStream;
begin
if FileOpenDialog1.Execute then begin
SetLength(Buffer, _2GB);
MS:= TFileStream.Create(FileOpenDialog1.FileName, fmOpenRead);
try
MS.Read(Buffer, _2GB div 2);
MS.Read(Buffer, _2GB div 2, _2GB div 2);
finally
MS.Free;
end;
end;
Form2.Caption:= 'File loaded';
end;
How silly is this in 2018?
I know TMemoryStream etc is broken, because it uses LongInt instead of NativeInt.
Is there a fix to the RTL that I can download. I know I can write my own TMemoryStream, but I'd much rather use a patch to the RTL that fixes in all places where the streams are used.
I'm using Tokyo BTW, very miffed that this is still not fixed.
Yes, I know TMemoryStream is an anti-pattern and I agree: https://stackoverflow.com/questions/34801278/delphi-out-of-memory-error-when-i-load-file-to-memory-stream
But right now I just need to have TMemoryStream fixed.
Even using TFileStream I have to load the file in chucks:
procedure TForm2.BtnOpenFileClick(Sender: TObject);
var
MS: TFileStream;
begin
if FileOpenDialog1.Execute then begin
SetLength(Buffer, _2GB);
MS:= TFileStream.Create(FileOpenDialog1.FileName, fmOpenRead);
try
MS.Read(Buffer, _2GB div 2);
MS.Read(Buffer, _2GB div 2, _2GB div 2);
finally
MS.Free;
end;
end;
Form2.Caption:= 'File loaded';
end;
How silly is this in 2018?
Map the file in memory. There won't be any limit, and random access will be very fast. See e.g. how our TMemoryMap class does it in https://synopse.info/files/html/api-1.18/SynCommons.html#TMEMORYMAP There is a https://synopse.info/files/html/api-1.18/SynCommons.html#TSYNMEMORYSTREAMMAPPED which defined a TStream over a mapped file.
ReplyDeletedownload from where ?
ReplyDeleteif one needs a huge file totally in memory - usually one uses "memory-mapped files"
A. Bouchez Cute! Thanks. I occasionally have some customers in need and I do try to provide alternatives to purchasing our code for simple chores like this.
ReplyDelete