COPY TO not following SET DEFAULT TO in xHb (solved)

[hua]

WIth the latest xHarbour, I noticed when I create a dbf file using COPY TO, it didn't follow the path I set in SET DEFAULT TO.
This will create a Dos error 2 at the next line that attempt to USE the dbf because USE will search in SET DEFAULT TO path

Most likely this is due to something at my environment since I don't see others reporting it

Anyone has any idea on how to find what's going on?
Thanks

[Rick Lipkin]

Hua

I create a folder on each local workstation hard drive and I always look for that folder when I start my application .. C:\Dbtmp and that is where I create all my temp files like .dbfs for reports ... so I always know where I can create and delete my temp files.

Rick Lipkin

[hua]

Thanks for the reply Rick.
It's not the storage scheme that is in doubt here since this is a pre-existing software.

It's just that this error pops up when I recompiled with the latest xHarbour. I don't get this error when I tested with latest Harbour but I got a different error with it.

[karinha]

A small example, can you show it via programming?

Gracias, tks.

Regards, saludos.

[karinha]

Code: Select all | Expand

#include "FiveWin.ch"

FUNCTION Main()

   USE Sales NEW

   COPY TO C:\TEMP\TempHua

RETURN NIL
 

Regards, saludos.

[hua]

As a workaround, I amended my COPY TO command as such

Code: Select all | Expand

COPY TO (AddDefPath(cTmpFile)) FOR &( cFilter )
USE (cTmpFile) NEW
*--------------------------------------------
function AddDefPath(cDbf)
  local cPath := set(_SET_DEFAULT)
return cPath+"\"+cDbf

[karinha]

Code: Select all | Expand

// C:\FWH\SAMPLES\HUATO2.PRG

#Include "FiveWin.ch"

ANNOUNCE RDDSYS
REQUEST DBFCDX, DBFFPT

FUNCTION Main()

   LOCAL cPath, cTmpFile, cDbfCopy, cFilter

   cPath    := "C:\TEMP\"
   cTmpFile := "SALES.DBF"
   cDbfCopy := "TEMPHUA.DBF"
   cFilter  := "DATE"

   IF .NOT. lIsDir( cPath )

      MsgInfo( cPath + " not found" )

      RETURN NIL

   ENDIF

   IF FILE( "C:\TEMP\TEMPHUA.DBF" )

      DELETE FILE( "C:\TEMP\TEMPHUA.DBF" )

   ENDIF

   USE ( cTmpFile ) NEW

   COPY FIELDS &cFilter TO ( cPath ) + cDbfCopy

   CLOSE DATABASES

   IF FILE( "C:\TEMP\TEMPHUA.DBF" )

      MsgInfo( "I'm a good copy.", "Listo:" )

   ENDIF

RETURN NIL

// FIN / END - kapiabafwh@gmail.com

Regards, saludos.

[Enrico Maria Giordano]

WIth the latest xHarbour, I noticed when I create a dbf file using COPY TO, it didn't follow the path I set in SET DEFAULT TO.
This will create a Dos error 2 at the next line that attempt to USE the dbf because USE will search in SET DEFAULT TO path

Most likely this is due to something at my environment since I don't see others reporting it

Anyone has any idea on how to find what's going on?
Thanks

There is nothing in the docs indicating that it should follow SET DEFAULT TO path. And, by the way, there was no significant changes in the xHarbour source code related to that command:

https://github.com/xHarbour-org/xharbou ... /ChangeLog

Anyway, if you can provide me the full xHarbour build that works fine for you and a simple PRG example to test here, I'll investigate the problem.

[Otto]

Hello Hua,

I recently spent some time working on direct access to the DBF.

You could build a much more powerful COPY TO command yourself. Where "Filter" is in the code, you can easily add which fields you want to export, and where "Search" is, any condition.

Target and format can also be easily added.

Best regards,
Otto

Code: Select all | Expand

// DBFToTXT
#include "FiveWin.ch"


PROCEDURE Main
    LOCAL cFilePath := "c:\fwh\samples\Data\database.dbf"
    LOCAL cName := "clark"
    LOCAL aResult

    aResult := FindNameInDbf(cFilePath, cName)
    
    IF !EMPTY(aResult)
        WriteRecordsToFile(aResult, "c:\fwh\samples\Data\output.txt")
    ENDIF
    
RETURN

FUNCTION FindNameInDbf(cFilePath, cName)
    LOCAL nHandle := FOPEN(cFilePath)
    LOCAL cHeader := SPACE(32)
    LOCAL nHeaderSize, nRecordSize, nNumRecords
    LOCAL aFieldDescriptors := {}
    LOCAL aFieldOffsets := {}
    LOCAL nOffset := 0
    LOCAL cFieldDescriptor, cFieldName
    LOCAL nFieldLength
    LOCAL nNameOffset, nNameLength
    LOCAL aMatchingRecords := {}
    LOCAL cRecord, cExtractedName 
    LOCAL hField, hRecordData
    LOCAL i, j
    LOCAL cFieldValue
    LOCAL hFieldDescriptor := { => }
    LOCAL nFound := 0
    LOCAL cFileData
     LOCAL nVersion
     LOCAL nYear
     LOCAL nMonth
     LOCAL nDay
     LOCAL LastUpdate
     
     
    Msginfo("Start Suche")

    IF nHandle == -1
        ? "Konnte die Datei nicht öffnen."
        RETURN {}
    ENDIF

    // Read entire file into memory
    cFileData := MEMOREAD(cFilePath)

    // Header lesen
    cHeader := LEFT(cFileData, 32)
    
    // Byte-Interpretation der Header-Daten
    nNumRecords := (ASC(SUBSTR(cHeader, 5, 1)) + (ASC(SUBSTR(cHeader, 6, 1)) * 256) + (ASC(SUBSTR(cHeader, 7, 1)) * 65536) + (ASC(SUBSTR(cHeader, 8, 1)) * 16777216))
    nHeaderSize := (ASC(SUBSTR(cHeader, 9, 1)) + (ASC(SUBSTR(cHeader, 10, 1)) * 256))
    nRecordSize := (ASC(SUBSTR(cHeader, 11, 1)) + (ASC(SUBSTR(cHeader, 12, 1)) * 256))

    // Felddeskriptoren lesen
    FOR i := 33 TO nHeaderSize STEP 32
        cFieldDescriptor := SUBSTR(cFileData, i, 32)
        IF ASC(LEFT(cFieldDescriptor, 1)) == 13
            EXIT
        ENDIF
        cFieldName := RTRIM(SUBSTR(cFieldDescriptor, 1, 11))
        nFieldLength := ASC(SUBSTR(cFieldDescriptor, 17, 1))
        AADD(aFieldDescriptors, { "name" => cFieldName, "length" => nFieldLength })
    NEXT

    // Feld-Offsets berechnen
    FOR i := 1 TO LEN(aFieldDescriptors)
        hFieldDescriptor := aFieldDescriptors[i]
        AADD(aFieldOffsets, { hFieldDescriptor["name"], nOffset, hFieldDescriptor["length"] })
        nOffset += hFieldDescriptor["length"]
    NEXT
   
    nNameOffset := AScan(aFieldOffsets, { |a| LEFT(a[1], 10) = "LAST" })
    nNameLength := aFieldOffsets[nNameOffset, 3]


    // FILTER, welche Felder
    aFieldDescriptors := {}
    AADD(aFieldDescriptors, { "name" => "FIRST", "length" => 20 })
    AADD(aFieldDescriptors, { "name" => "LAST", "length" => 20 })
    xbrowse(aFieldDescriptors)



    // Process records
    FOR i := 1 TO nNumRecords
         cRecord := SUBSTR(cFileData, nHeaderSize + (i - 1) * nRecordSize + 1, nRecordSize)
       cExtractedName :=   ALLTRIM(LOWER( SUBSTR(cRecord, aFieldOffsets[nNameOffset, 2] + 1, nNameLength) ))
      
 
    // Search    
    
    IF cExtractedName = cName 
            nFound += 1

            hRecordData := { "recno" => i }
            nOffset := 0

            FOR j := 1 TO LEN(aFieldDescriptors)
                hField := aFieldDescriptors[j]
                
                cFieldValue := (SUBSTR(cRecord, nOffset + 2, hField["length"]))
                hRecordData[hField["name"]] := cFieldValue
                nOffset += hField["length"]
            NEXT

            AADD(aMatchingRecords, hRecordData)
            
     ENDIF

    NEXT

    xbrowse(aMatchingRecords)

    RETURN(aMatchingRecords)
    
 FUNCTION WriteRecordsToFile(aRecords, cFilePath)
    LOCAL nHandle := FCREATE(cFilePath)
    LOCAL cLine
    LOCAL hRecord
    LOCAL cFieldName
    LOCAL cValue

    IF nHandle == -1
        ? "Konnte die Datei nicht erstellen."
        RETURN NIL
    ENDIF

    FOR EACH hRecord IN aRecords

        cLine := ""
        
        FOR cFieldName := 1 TO LEN(hRecord)
       
            IF HGetKeyAt(hRecord, cFieldName) != "recno"
                cValue := hRecord[HGetKeyAt(hRecord, cFieldName)]
                cLine += cValue + CHR(9) // Tab-separated

               
            ENDIF
        NEXT
        cLine := RTRIM(cLine) + CRLF
        FWRITE(nHandle, cLine)
    NEXT

    FCLOSE(nHandle)
    ? "Datei erfolgreich erstellt: ", cFilePath
RETURN NIL

// Funktion zum rechtsbündigen Auffüllen eines Strings auf eine bestimmte Länge
FUNCTION PadR(cText, nLength)
RETURN SUBSTR(cText + SPACE(nLength), 1, nLength)



 

[Otto]

The design of the report selection I will make for myself as in this gif.

[karinha]

Meister Otto, ausgezeichnet! Glückwunsch!
Master Otto, excellent! Congratulations!

Code: Select all | Expand

// C:\FWH\SAMPLES\DBFTOTXT.PRG

#include "FiveWin.ch"

PROCEDURE Main

   LOCAL cFilePath := "..\samples\Data\database.dbf"
   LOCAL cName     := "clark"
   LOCAL aResult

   aResult := FindNameInDbf( cFilePath, cName )

   IF .NOT. Empty( aResult )

      WriteRecordsToFile( aResult, "..\samples\Data\output.txt" )

   ENDIF

RETURN NIL

FUNCTION FindNameInDbf( cFilePath, cName )

   LOCAL nHandle := FOpen( cFilePath )
   LOCAL cHeader := Space( 32 )
   LOCAL nHeaderSize, nRecordSize, nNumRecords
   LOCAL aFieldDescriptors := {}
   LOCAL aFieldOffsets := {}
   LOCAL nOffset := 0
   LOCAL cFieldDescriptor, cFieldName
   LOCAL nFieldLength
   LOCAL nNameOffset, nNameLength
   LOCAL aMatchingRecords := {}
   LOCAL cRecord, cExtractedName
   LOCAL hField, hRecordData
   LOCAL i, j
   LOCAL cFieldValue
   LOCAL hFieldDescriptor := { => }
   LOCAL nFound := 0
   LOCAL cFileData
   LOCAL nVersion
   LOCAL nYear
   LOCAL nMonth
   LOCAL nDay
   LOCAL LastUpdate

   // Msginfo( "Start Suche" ) // Start search
   Msginfo( "Start search" )   // German Language

   IF nHandle == -1

      ? "Konnte die Datei nicht öffnen." // Could not open the file.
      ? "Could not open the file." // German Language

      RETURN {} // ??? NIL ?

   ENDIF

   // Read entire file into memory
   cFileData := MemoRead( cFilePath )

   // Header lesen
   cHeader := Left( cFileData, 32 )

   // Byte-Interpretation der Header-Daten
   nNumRecords := ( Asc( SubStr( cHeader, 5, 1 ) ) + ( Asc( SubStr( cHeader, 6, 1 ) ) * 256 ) + ( Asc( SubStr( cHeader, 7, 1 ) ) * 65536 ) + ( Asc( SubStr( cHeader, 8, 1 ) ) * 16777216 ) )
   nHeaderSize := ( Asc( SubStr( cHeader, 9, 1 ) ) + ( Asc( SubStr( cHeader, 10, 1 ) ) * 256 ) )
   nRecordSize := ( Asc( SubStr( cHeader, 11, 1 ) ) + ( Asc( SubStr( cHeader, 12, 1 ) ) * 256 ) )

   // Felddeskriptoren lesen
   FOR i := 33 TO nHeaderSize STEP 32

      cFieldDescriptor := SubStr( cFileData, i, 32 )

      IF Asc( Left( cFieldDescriptor, 1 ) ) == 13
         EXIT
      ENDIF

      cFieldName := RTrim( SubStr( cFieldDescriptor, 1, 11 ) )
      nFieldLength := Asc( SubStr( cFieldDescriptor, 17, 1 ) )

      AAdd( aFieldDescriptors, { "name" => cFieldName, "length" => nFieldLength } )

   NEXT

   // Feld-Offsets berechnen
   FOR i := 1 TO Len( aFieldDescriptors )

      hFieldDescriptor := aFieldDescriptors[ i ]

      AAdd( aFieldOffsets, { hFieldDescriptor[ "name" ], nOffset, hFieldDescriptor[ "length" ] } )

      nOffset += hFieldDescriptor[ "length" ]

   NEXT

   nNameOffset := AScan( aFieldOffsets, {| a | Left( a[ 1 ], 10 ) = "LAST" } )
   nNameLength := aFieldOffsets[ nNameOffset, 3 ]

   // FILTER, welche Felder
   aFieldDescriptors := {}

   AAdd( aFieldDescriptors, { "name" => "FIRST", "length" => 20 } )
   AAdd( aFieldDescriptors, { "name" => "LAST", "length" => 20 } )

   xbrowse( aFieldDescriptors )

   // Process records
   FOR i := 1 TO nNumRecords

      cRecord := SubStr( cFileData, nHeaderSize + ( i - 1 ) * nRecordSize + 1, nRecordSize )
      cExtractedName :=   AllTrim( Lower( SubStr( cRecord, aFieldOffsets[ nNameOffset, 2 ] + 1, nNameLength ) ) )

      // Search

      IF cExtractedName = cName

         nFound += 1

         hRecordData := { "recno" => i }
         nOffset     := 0

         FOR j := 1 TO Len( aFieldDescriptors )

            hField := aFieldDescriptors[ j ]

            cFieldValue := ( SubStr( cRecord, nOffset + 2, hField[ "length" ] ) )

            hRecordData[ hField[ "name" ] ] := cFieldValue

            nOffset += hField[ "length" ]

         NEXT

         AAdd( aMatchingRecords, hRecordData )

      ENDIF

   NEXT

   xbrowse( aMatchingRecords )

RETURN( aMatchingRecords )

FUNCTION WriteRecordsToFile( aRecords, cFilePath )

   LOCAL nHandle := FCreate( cFilePath )
   LOCAL cLine
   LOCAL hRecord
   LOCAL cFieldName
   LOCAL cValue

   IF nHandle == -1

      // ? "Konnte die Datei nicht erstellen." // Could not create the file.
      ? "Could not create the file."           // German Language

      RETURN NIL

   ENDIF

   FOR EACH hRecord IN aRecords

      cLine := ""

      FOR cFieldName := 1 TO Len( hRecord )

         IF HGetKeyAt( hRecord, cFieldName ) != "recno"

            cValue := hRecord[ HGetKeyAt( hRecord, cFieldName ) ]

            cLine += cValue + Chr( 9 ) // Tab-separated

         ENDIF

      NEXT

      cLine := RTrim( cLine ) + CRLF

      FWrite( nHandle, cLine )

   NEXT

   FClose( nHandle )

   // ? "Datei erfolgreich erstellt: ", cFilePath // File created successfully:
   ? "File created successfully:: ", cFilePath

RETURN NIL
// Funktion zum rechtsbündigen Auffüllen eines Strings auf eine bestimmte Länge
FUNCTION PadR( cText, nLength )

RETURN SubStr( cText + Space( nLength ), 1, nLength )

// FIN / END
 

Gracias, tks.

Regards, saludos.

[hua]

Hi Enrico,

There is nothing in the docs indicating that it should follow SET DEFAULT TO path. And, by the way, there was no significant changes in the xHarbour source code related to that command:

https://github.com/xHarbour-org/xharbou ... /ChangeLog

Anyway, if you can provide me the full xHarbour build that works fine for you and a simple PRG example to test here, I'll investigate the problem.

The COPY TO behaviour is based on this SET DEFAULT TO's description from Clipper's Norton Guide

SET DEFAULT sets the drive and directory where the application program
creates and saves files, with the exception of temporary files and files
created with the low-level file functions.

SET DEFAULT does not change the DOS drive and directory. When
attempting to access files, the DEFAULT drive and directory are searched
first. To set additional search paths for file access, use SET PATH.

I'll try to create a self-contained example to see whether I can replicate the buggy behaviour

[Otto]

direct access

This direct access to the DBF files has potential.

It makes it very easy to quickly perform complex database queries.

I did a speed test for COPY TO against the test database with the 200,000 records.

Direct access is 46 ms faster, or 5.58%.

I will now perform further tests with my own data.

[Enrico Maria Giordano]

Anyway, if you can provide me the full xHarbour build that works fine for you and a simple PRG example to test here, I'll investigate the problem.

Ok, I managed to replicate the problem and found that with Harbour it works fine. I'll try to fix the bug, thank you.

[Enrico Maria Giordano]

I think I found the bug:

Code: Select all | Expand

2009-08-27 20:53 UTC-0430 Ron Pinkas <ron.pinkas/at/xharbour.com>
   * source/rdd/dbcmd.c
     * Hack to __DBCOPY() to explicitly prefix target file with the current path
       to to avoid inadvertent override of file in the SET PATH folder.

       /*
          NOTE: RDD authors please review if this is the correct place for such hack.
        */

I'm going to borrow the correct code from Harbour.