Virtual Pascal

Classic Object Pascal for free

There seems a limitation with the RTL using the seek() function on an untyped file, which was written to be > 2GB.  The buffer size was set to prevent overflow of the record counter (Longint), so that is not the problem. After invoking seek() just past the 2GB record size sum, get I/O error 131.   Anyone know of how to deal with this problem on the VP compiler?

Views: 525

Reply to This

Replies to This Discussion

Seek offsets are often signed, since they can also have a different position=0 point (beginning file, end file, current position)

VP afaik has no int64, so 32-bit integer is as large as it goes.


Or maybe check if it is "comp", the 64-bit floating point integer. (strange, I know).  Try putting a value in a comp, and passing that to seek().

Thank you for the reply and interest.  I will note that the record count passed to seek does not exceed a 32 bit integer.  This is because the file was Reset with a record size > 1 byte and the file, though size > 2 GB, should be seekable.  In any case, I wonder if you know of any other RTL functionallity that will get the job done, per the int64 data type you describe.  Interestingly, as you may know, there is something like this in Free Pascal Compiler and, naturally, also part of the Windows API.  Is there anyway to port the code from FPC or the Windows API (perhaps from kernel32.dll, etc.), so as to extend the capability of  VPC?

Well, development (even minor RTL work) on VP seems to be mostly halted for 5+ years already. Or at least the publically visible part of that.

SInce I/O error 131 is the Windows error for negative searchindex, I assume you use Windows.  I guess it means that the multiplication that VP does to create a file index from record size and offset in records is 32-bit signed. I guess in theory it could be done in Comp. (and that is only if my guess work is right, and you don't get into multiplatform trouble with other targets)

IIRC the winapi seek takes a record of two 32-bit values, so it doesn't need int64. But it would require to get the handle from the filetype, and of course be a major inconvenience. (and it might only fix the immediately visible problem, it might not be last)  Flogging dead horse and all that.

More logically would be to simply use FPC or Delphi, specially for Windows I couldn't think a reason not to. Or to repartition the problem so that e.g. you use more files and avoid this limit.

But I'm slightly biassed towards FPC as you can see by my avatar picture.

Thanks Macro for the discussion.  On a similar path, how does/mightt FPC implement this in its RTL.  I have downloaded the source code (system.pas unit and include files as well), but the code for seek (int64 file pointer or such) is not presenting itself as far as I may tell.  The closest thing, in my estimation, my be some linked code per {$L  xxxx.o}, which may be helpful.  I am asking this only from the thought process of adding similar functionality, myself, to VPC.  I have never done this, and wonder if you may offer advice on the steps to do this and or manner to port the FPC code (assuming it is available?).  Thanks. 

The code is in the rtl/win directory (which is shared between win32 and win64).

Do_seek  in rtl/win/sysfile.inc


type
  tint64rec = record
    low, high: dword;
  end;

procedure do_seek(handle: thandle; pos: Int64);
var
  posHigh: LongInt;
begin
  posHigh := tint64rec(pos).high;
  if (SetFilePointer(handle, pos, @posHigh, FILE_BEGIN)=-1) and
  { return value of -1 is valid unless GetLastError is non-zero }
    (GetLastError <> 0) then
  begin
    errno := GetLastError;
    Errno2InoutRes;
  end;
end;


Which is called back from generic file routines in inc/file.inc 

Procedure Seek(var f:File;Pos:Int64);[IOCheck];

Note a few things:

- everything is in int64, which VP doesn't have (COMP might be doable).

- The setfilepointer function (which is the winapi function called on windows for this) takes a 32-bit value (the "pos" argument is automatically scaled down) and a pointer to the upper dword of the pos parameter as another parameter

- I don't know how to rebuilds VP rtl, nor have I seen reports of sb doing it in recent years.

This problem won't be solved by poaching code from other places, but must be fixed by patching and rebuilding VP rtl.  Typedfile handling is magic, and such things are highly compiler and RTL setup dependent, and can't be copy and pasted from somewhere else.

Finding the setfilepointer call in the VP rtl would be central to this, and see if the parameter with the high dword is passed, and to track if this offset is calculated in 64-bit.

This is as far as I can help you, I don't know much about VP internals.

Macro, thanks for the insight of the FPC rtl (from the expert himself).  Just out of interest in learning how this (any?) compiler is developed: Is the SetFilePointer (FPC internal name) an external function, derived itself from the kernel32.dll or similar library module?  If so what is the name of FPC unit/code (perhaps this is the greatest ancestor of the system.pas unit and like) where this happens?  You say I need to recompille the RTL in VPC, but, I can't, instead, mount the appropriate library module (i.e. statically link import library, use external function prototypes, etc.)  directly in my code?  Your response and explanations have intrigued my interest at the very least.  Maybe if the answers to the above questions are hidden or too complicated, then might it be easier to port code to FPC, in your opinion?   

It's one line in rtl/win/sysos.inc (a simple grep could have told you):

 function SetFilePointer(l1,l2 : thandle;l3 : pointer;l4 : longint) : longint;
     {$ifdef wince}cdecl{$else}stdcall{$endif};external KernelDLL name 'SetFilePointer';
 

KernelDLL is a string constant and contains 'kernel32.dll' for win32/64

thandle is as large as pointer (32-bit on 32-bit).

Like Delphi, FPC doesn't need anything else, import stubs are automatically generated if needed. Since VP has an own assembler/linker, I assume it works the same, but don't know for sure.

But the trouble is that you must get the RTL to call that properly, not your code, or you are entering a minefield.

I'm can't answer the question about moving to FPC, since I don't know why you chose VP. (and if that is long ago, why you stuck with it so long).

Thanks for your patience and the explanation on how a rtl is setup.  When you mention the idea, of one's own higher level code linking to the kerneldll library, as a minefield analogy, does this, simply, mean that to work with the winApi is difficult (i.e., hard to get the interface working just right), or is there an access/stability issue when accessing this particular type of modules directly?  I use VPC as a nice tool for my type of problems (mostly scientific computing, etc.), and it has served me fine until I have, currently, come up the limits of memory and, apparently, disk file management.  It sounds like FPC is more modern update compiler (that has, hopefully, kept pace with other mainstream compilers: Is the capability there to develop modern console applications) and I will try to port my code over: probably to the win64 target...

No, it is not the winapi access but using winapi seek and VP rtl file I/O that is the minefield. IOW if you switch to winapi, you should switch ALL to winapi.

But nearly the whole windows subset of VP will be supported by FPC, and way, way more.

Just install Lazarus (even if you plan console applications, it is still the main, most userfriendly IDE for FPC), and enter the 21st century :-)  And subscribe to the Lazarus forum for questions, I'm there too.

RSS

© 2022   Created by Allan Mertner.   Powered by

Report an Issue  |  Terms of Service