gphugef.pas

Advanced Data Import Component Suite for Borland Delphi and C++ Builder allows you to import your da

{$B-,H+,J+,Q-,T-,X+}

unit GPHugeF;

{$I QImport3VerCtrl.Inc}

(*:Interface to 64-bit file functions with some added functionality.
   @author Primoz Gabrijelcic
   @desc <pre>

This software is distributed under the BSD license.

Copyright (c) 2006, Primoz Gabrijelcic
All rights reserved.

Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
- Redistributions of source code must retain the above copyright notice, this
  list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright notice,
  this list of conditions and the following disclaimer in the documentation
  and/or other materials provided with the distribution.
- The name of the Primoz Gabrijelcic may not be used to endorse or promote
  products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

   Author           : Primoz Gabrijelcic
   Creation date    : 1998-09-15
   Last modification: 2006-08-17
   Version          : 4.0a
</pre>*)(*
   History:
     4.0a: 2006-08-17
       - Fixed compilation problems under D6-D2005.
     4.0: 2006-08-14
       - TGpHugeFile
         - Added new constructors CreateW and CreateExW that use Unicode encoding for the
           file name.
         - FileName property changed to WideString.
       - TGpHugeFileStream
         - Added new constructor CreateW that uses Unicode encoding for the file name.
         - FileName property changed to WideString.
     3.12a: 2006-04-25
       - Don't use FileDateToDateTime on Delphi 2006 and newer.
     3.12: 2004-03-24
       - Added parameters diskLockTimeout and diskRetryDelay to the TGpHugeFileStream
         constructor. Those parameters are passed to the TGpHugeFile.RewriteEx/ResetEx.
         Default value for those parameters - 0 - is the same that was previously sent to
         the RewriteEx/ResetEx.
     3.11: 2003-11-21
       - TGpHugeFileStream.Create constructor got parameter desiredShareMode, which is
         passed to the nested TGpHugeFile.CreateEx. Default value for this parameter -
         $FFFF - is the same that was previously sent to the CreateEx. Therefore, no old
         code should become broken because of this change.
     3.10: 2003-05-14
       - Compatible with Delphi 7.
     3.09a: 2003-02-12
       - Faster TGpHugeFile.Eof.
     3.09: 2003-02-12
       - Eof function added.
       - Seek in buffered write mode was not working. Fixed.
     3.08a: 2002-10-14
       - TGpHugeFileStream.Create failed when append mode was used and file did not exist.
         Fixed.
     3.08: 2002-04-24
       - File handle exposed through the Handle property.
       - Added THFOpenOption hfoCompressed.
     3.07a: 2001-12-15
       - Updated to compile with Delphi 6.
     3.07: 2001-07-02
       - Added TGpHugeFile.FileDate setter.
     3.06b: 2001-06-27
       - TGpHugeFile.FileSize function was returning wrong result when file was open for
         buffered write access.
     3.06a: 2001-06-24
       - Modified CreateEx behaviour - if DesiredShareMode is not set and file is open in
         GENERIC_READ mode, sharing will be set to FILE_SHARE_READ.
     3.06: 2001-06-22
       - Added parameter DesiredShareMode to the CreateEx constructor.
     3.05: 2001-02-27
       - Modified Reset and Rewrite methods to open file in buffered mode by default.
     3.04: 2001-01-31
       - All raised exceptions now have HelpContext set. All possible HelpContext values
         are enumerated in 'const' section at the very beginning of the unit. Thanks to
         Peter Evans for the suggestion.
     3.03: 2000-10-18
       - Fixed bugs in hfoCloseOnEOF support in TGpHugeFile.
     3.02: 2000-10-12
       - Fixed bugs in hfoCloseOnEOF support in TGpHugeFileStream.
     3.01: 2000-10-06
       - TGpHugeFileStream constructor now accepts THFOpenOptions parameter, which is
         passed to TGpHugeFile ResetEx/RewriteEx. Default open mode for stream files is
         now hfoBuffered.
       - TGpHugeFileStream constructor parameters are simpler - FlagsAndAttributes and
         DesiredAccess parameters are no longer present.
       - Added TGpHugeFileStream.CreateFromHandle constructor accepting instance of
         TGpHugeFile, which is then used for all stream access. This TGpHugeFile instance
         must already be created and open (Reset, Rewrite). It will not be destroyed in
         TGpHugeFileStream destructor.
       - Added read-only property TGpHugeFileStream.FileName.
       - Added read-only property TGpHugeFileStream.WindowsError.
       - Fully documented.
       - All language-dependant string constants moved to resourcestring section.
     3.0: 2000-10-03
       - Created TGpHugeFileStream - descendant of TStream that wraps TGpHugeFile.
         Although it does not support huge files fully (because of TStream limitations),
         you could still use it as a buffered file stream.
     2.33: 2000-09-04
       - TGpHugeFile now exposes WindowsError property, which is set to last Windows error
         wherever it is checked for.
     2.32: 2000-08-01
       - All raised exceptions converted to EGpHugeFile exceptions.
       - All windows exceptions are now caught and converted to EGpHugeFile exceptions.
       - If file is open in read buffered mode *and* is then seeked past Eof
         (Seek(FileSize)) *and* is then written into, it will switch to write buffered
         mode (previous versions of GpHugeFile raised exception under those conditions).
     2.31: 2000-05-15
       - Call to Truncate is now allowed in buffered write mode. It will cause buffer to
         be flushed, though.
     2.30a: 2000-05-15
       - Fix introduced in 2.29a sometimes caused BlockRead to return error even when
         there was some data present. This only happened when file was open for reading
         (via Reset) and then extended with BlockWrite.
     2.30: 2000-05-12
       - New property: IsBuffered. Returns true if file is open in buffered mode.
     2.29a: 2000-05-02
       - While reading near end of (buffered) file, ReadFile API was called much too
         often. Fixed.
     2.29: 2000-04-14
       - Added new ResetEx/RewriteEx parameter - waitObject. If not equal to zero,
         TGpHugeFile will check it periodically in the wait loop. If object becomes
         signalled, TGpHugeFile will stop trying to open the file and will return an
         error.
     2.28: 2000-04-12
       - Added new THFOpenOption: hfoCanCreate. Set it to allow ResetEx to create file
         when it does not exist.
     2.27: 2000-04-02
       - Added property FileDate.
     2.26a: 2000-03-07
       - Fixed bug in hfoCloseOnEOF processing.
     2.26: 2000-03-03
       - Added THFOpenOption hfoCloseOnEOF. If specified in a call to ResetEx TGpHugeFile
         will close file handle as soon as last block is read from the file. This will
         free file for other programs while main program may still read data from
         TGpHugeFile's buffer. {*}
         After the end of file is reached (and handle is closed):
           - FilePos may be used.
           - FileSize may be used.
           - Seek and BlockRead may be used as long as the request can be fulfilled from
             the buffer.
         Use of this option is not recommended when access to the file is random. {*}
         It was designed to use with sequential access to the file. hfoCloseOnEOF is
         ignored if hfoBuffered is not set. hfoCloseOnEOF is ignored if used in RewriteEx.
         {*} hfoCloseOnEOF can cope with a program that alternately calls BlockRead and
         Seek requests. When BlockRead reaches Eof, this condition will be marked but file
         handle will not be closed yet. Only when BlockRead is called again, file will be
         closed, but only if between those calls Seek did not invalidate the buffer (Seek
         that can be fulfilled from the buffer is OK). This works with programs that load
         a small buffer and then Seek somewhere in the middle of this buffer (like Readln
         function in TGpTextFile class).
     2.25a: 2000-02-19
       - Fixed bug where TGpHugeFile.Reset would create a file if file did not exist
         before. Thanks to Peter Evans for finding the bug and solution.
     2.25: 1999-12-29
       - Changed implementation of TGpHugeFile.ResetEx and TGpHugeFile.RewriteEx (called
         from all Reset* and Rewrite* functions). Before the change, they were closing and
         reopening the file - not a very good idea if you share a file between
         applications.
     2.24e: 1999-12-22
       - Fixed broken TGpHugeFile.IsOpen. Thanks for Phil Hodgson for finding this bug.
     2.24d: 1999-11-22
       - Fixed small problem in file access routines. They would continue trying to access
         a file event if returned error was not sharing or locking error.
     2.24c: 1999-11-20
       - Behaviour changed. If you open file with GENERIC_READ access, sharing mode will
         be set to FILE_SHARE_READ. 2.24b and older set sharing mode to 0 in all
         occasions.
     2.24b: 1999-11-06
       - Added (again) ResetBuffered and RewriteBuffered;
     2.24a: 1999-11-03
       - Fixed Reset and Rewrite.
     2.24: 1999-11-02
       - ResetBuffered and RewriteBuffered renamed to ResetEx and RewriteEx.
       - Parameters diskLockTimeout and diskRetryDelay added to ResetEx and RewriteEx.
     2.23: 1999-10-28
       - Compiles with D5.
     2.22: 1999-06-14
       - Better error reporting.
     2.21: 1998-12-21
       - Better error checking.
     2.2: 1998-12-14
       - New function IsOpen.
       - Lots of SetLastError(0) calls added.
     2.12: 1998-10-28
       - CreateEx enhanced.
     2.11: 1998-10-14
       - Error reporting in Block*Unsafe enhanced.
     2.1: 1998-10-13
       - FilePos works in buffered mode.
       - Faster FilePos in unbuffered mode.
       - Seek works in read buffered mode.
         - In FILE_FLAG_NO_BUFFERING mode Seek works only when offset is on a sector
           boundary.
         - Truncate works in read buffered mode (untested).
         - Dependance on MSString removed.
     2.0: 1998-10-08
       - Win32 API error checking.
       - Sequential access buffering (ResetBuffered, RewriteBuffered).
       - Buffered files can be safely accessed in FILE_FLAG_NO_BUFFERING mode.
       - New procedures BlockReadUnsafe, BlockWriteUnsafe.
     1.1: 1998-10-05
       - CreateEx constructor added.
         - can specify attributes (for example FILE_FLAG_SEQUENTIAL_SCAN)
       - D4 compatible.
     1.0: 1998-09-15
       - First published version.
*)

{$DEFINE D3PLUS}
{$DEFINE D4PLUS}
{$DEFINE D6PLUS}
{$DEFINE D7PLUS}
{$IFDEF VER100}{$UNDEF D4PLUS}{$UNDEF D6PLUS}{$UNDEF D7PLUS}{$ENDIF}
{$IFDEF VER110}{$UNDEF D6PLUS}{$UNDEF D7PLUS}{$ENDIF}
{$IFDEF VER120}{$UNDEF D6PLUS}{$UNDEF D7PLUS}{$ENDIF}
{$IFDEF VER130}{$UNDEF D6PLUS}{$UNDEF D7PLUS}{$ENDIF}
{$IFDEF VER140}{$UNDEF D7PLUS}{$ENDIF}
{$IFDEF CONDITIONALEXPRESSIONS}
  {$IF (RTLVersion >= 18)} // Delphi 2006 or newer
    {$DEFINE D10PLUS}
  {$IFEND}
{$ENDIF}

interface

uses
  SysUtils,
  Windows,
  Classes;

// HelpContext values for all raised exceptions
const
  //:Exception was handled and converted to EGpHugeFile but was not expected and is not categorised.
  hcHFUnexpected              = 1000;
  //:Windows error.
  hcHFWindowsError            = 1001;
  //:Unknown Windows error.
  hcHFUnknownWindowsError     = 1002;
  //:Invalid block size.
  hcHFInvalidBlockSize        = 1003;
  //:Invalid file handle.
  hcHFInvalidHandle           = 1004;
  //:Failed to allocate buffer.
  hcHFFailedToAllocateBuffer  = 1005;
  //:Write operation encountered while in buffered read mode.
  hcHFWriteInBufferedReadMode = 1006;
  //:Read operation encountered while in buffered write mode.
  hcHFReadInBufferedWriteMode = 1007;
  //:Unexpected end of file.
  hcHFUnexpectedEOF           = 1008;
  //:Write failed - not all data was saved.
  hcHFWriteFailed             = 1009;
  //:Invalid 'mode' parameter passed to Seek function.
  hcHFInvalidSeekMode         = 1010;

  CAutoShareMode = $FFFF;

type
  {:Alias for int64 so it is Delphi-version-independent (as much as that is
    possible at all).
  }
  HugeInt = LONGLONG;

  {:Base exception class for all exceptions raised in TGpHugeFile and
    descendants.
  }
  EGpHugeFile       = class(Exception);

  {:Base exception class for exceptions created in TGpHugeFileStream.
  }
  EGpHugeFileStream = class(EGpHugeFile);

  {:Result of TGpHugeFile reset and rewrite methods.
    @enum hfOK         File opened successfully.
    @enum hfFileLocked Access to file failed because it is already open and
                       compatible sharing is not allowed.
    @enum hfError      Other file access errors (file/path not found...).
   }
  THFError = (hfOK, hfFileLocked, hfError);

  {:TGpHugeFile reset/rewrite options.
    @enum hfoBuffered   Open file in buffered mode. Buffer size is either
                        default (BUF_SIZE, currently 64 KB) or specified by the
                        caller in ResetEx or RewriteEx methods.
    @enum hfoLockBuffer Buffer must be locked (Windows require that for direct
                        access files (FILE_FLAG_NO_BUFFERING) to work
                        correctly).
    @enum hfoCloseOnEOF Valid only when file is open for reading. If set,
                        TGpHugeFile will close file handle as soon as last block
                        is read from the file. This will free file for other
                        programs while main program may still read data from
                        TGpHugeFile's buffer. (*)                                <br>
                        After the end of file is reached (and handle is closed): <ul><li>
                          FilePos may be used.                                   </li><li>
                          FileSize may be used.                                  </li><li>
                          Seek and BlockRead may be used as long as the request
                          can be fulfilled from the buffer.                      </li></ul><br>
                        Use of this option is not recommended when access to the
                        file is random. (*) It was designed to use with
                        sequential or almost sequential access to the file.
                        hfoCloseOnEOF is ignored if hfoBuffered is not set.
                        hfoCloseOnEOF is ignored if used in RewriteEx.           <br>
                        (*) hfoCloseOnEOF can cope with a program that
                        alternately calls BlockRead and Seek requests. When
                        BlockRead reaches Eof, this condition will be marked but
                        file handle will not be closed yet. When BlockRead is
                        called again, file will be closed, but only if between
                        those calls Seek did not invalidate the buffer (Seek
                        that can be fulfilled from the buffer is OK). This works
                        with programs that load a small buffer and then Seek
                        somewhere in the middle of this buffer (like Readln
                        function in TGpTextFile class does).
    @enum hfoCanCreate  Reset is allowed to create a file if it doesn't exist.   <br>
    @enum hfoCompressed Valid only when file is opened for writing. Will try
                        to set the "compressed" attribute (when running on NT
                        and file is on NTFS drive).
  }
  THFOpenOption  = (hfoBuffered, hfoLockBuffer, hfoCloseOnEOF, hfoCanCreate,
                    hfoCompressed);

  {:Set of all TGpHugeFile reset/rewrite options.
  }
  THFOpenOptions = set of THFOpenOption;