Automated download of pictures

[Enrico Maria Giordano]

Thank you.
GETURLTOFILE() is working with http: but not https:

This is not true. It would work if tipssl were linked (and opensll DLL's were available).

URLDOWNLOADTOFILE() is working well with all and is also much faster than GETURLTOFILE().

I was not aware the URLDOWNLOADTOFILE() is much faster than GETURLTOFILE(). Are you made any real test? Can you share the timing result?

The actual reason for my testing is to find a faster alternative to the existing function WebPageContents(). Our requirement is to directly read the contents of the URL into memory.

I use this (but it needs opensll as well):

Code: Select all  Expand view

FUNCTION GETURL( cUrl )

    LOCAL cHtm := ""

    LOCAL oCli

    TRY
        oCli = TIPClientHttp():New( cUrl )

        IF EMPTY( oCli ) THEN BREAK

        oCli:nConnTimeout = -1

        IF !oCli:Open() THEN BREAK

        cHtm = oCli:ReadAll()

        IF EMPTY( oCli:cReply ) .OR. !( "OK" $ UPPER( oCli:cReply ) )
            cHtm = ""
        ENDIF
    CATCH
    END

    IF !EMPTY( oCli ) THEN oCli:Close()

    RETURN cHtm

EMG

[nageswaragunupudi]

I was not aware the URLDOWNLOADTOFILE() is much faster than GETURLTOFILE(). Are you made any real test? Can you share the timing result?

I used the Tajmahal.jpg ( http not https )

GETURLTOFILE() --> 3.1 to 3.3 secs
Both
URLDOWNLOADTOFILE() and
MEMOWRIT( cFile, webpagecontents( curl ) ) --> 0.06 to 0.09 secs

Code: Select all  Expand view

function testweb4()

   local cURL := "http://cdn.history.com/sites/2/2015/04/hith-eiffel-tower-iStock_000016468972Large.jpg"
   local cFile := "test.jpg"
   local nSecs

   FERASE( cFile )
   URLDOWNLOADTOFILE( 0, cURL, cFile ) // Ignore time taken for the first time

   nSecs := SECONDS()
   GETURLTOFILE( cURL, cFile )
   ? SECONDS() - nSecs, File( cFile ) // --> 3.23

   nSecs := SECONDS()
   URLDOWNLOADTOFILE( 0, cURL, cFile )
   ? SECONDS() - nSecs, File( cFile ) // --> 0.07

   nSecs := SECONDS()
   MEMOWRIT( cFile, webpagecontents( curl ) )
   ? SECONDS() - nSecs, File( cFile )  // --> 0.07

   if File( cFile )
      XIMAGE( cFile )
   endif

return nil

[Enrico Maria Giordano]

That's only a cache effect. Try calling this before download:

Code: Select all  Expand view

DLL FUNCTION DELETEURLCACHEENTRY( cUrl AS LPSTR ) AS BOOL;
    PASCAL FROM "DeleteUrlCacheEntryA" LIB "wininet.dll"

EMG

[nageswaragunupudi]

Mr EMG

You are right.
Without the benefit of cache, GETURLTOFILE() is faster than the other two.
Another observation is that GETURLTOFILE() does not take advantage of cache and other two take advantage of cache if available

[Enrico Maria Giordano]

Mr EMG

You are right.
Without the benefit of cache, GETURLTOFILE() is faster than the other two.
Another observation is that GETURLTOFILE() does not take advantage of cache and other two take advantage of cache if available

That's true. But use of cache means that the file is not really downloaded, even if it's changed. So use it with caution.

EMG

[nageswaragunupudi]

That's true. But use of cache means that the file is not really downloaded, even if it's changed. So use it with caution.

Yes. Thanks