Using the WebBrowser control as an image "control"

[Olaf Doschke]

This is a follow up for the thread thread184-1803757

The suspicion of vernspace there is that the VFP image control has a memory leak problem if used extensively with larger images (a few MB) and may be related to stretch="isometric".
I couldn't confirm, but the obvious test of such suspicion is using an alternative control and maybe the MS Office Forms Image ActiveX control would be the simplest candidate, where available.

I said it would be far fetched to use the MS WebBrowser control, as it likely has an even worse memory footprint. It creates a dependency, AFAIR with the shdocview.dll, VFP viewcode tool says ieframe.dll. It's simpler to use an ActiveX control that has a dedicated redistributable setup. What to add for Webbrowser control might depend on IE and Windows version.

Anyway, I [highlight #FCE94F]DON'T[/highlight] recommend this, and this demo code also shows one of the much simpler problems before going deeper into the technical problems: Using isometric stretching of an image you will need to put this webbrowser image control on your form in the exact right aspect ratio size, as it has no way to render any excess margin pixels transparent. That's literally an easy to overlook VFP image control feature: When there are some excess pixels as the WxH of the VFP image control is not the image aspect ratio, VFP can render these pixels transparent (BackStyle is transparent by default). The WebBrowser won't.

I still wanted to post the result, as it may help someone to use an image type only browsers support. It will expect a URL in its Picture property, to use a local file instead of a URL prepend the file name with "file://" essentialy and you can also use this with image files not hosted somewhere. I wanted to make the URI capability a feature at least, as it's a disadvantage on the other side anyway, as an image now also needs a download and doesn't show immediately.

Changing the picture at runtime also is not covered by this. The HTML for image display is generated at INIT() time and so requires the wbimage.Picture property to be set at design time. A runtime change won't change the picture.

One of the other few advantages besides being able to use internet images where internet connection is available and the image types browsers support, but VFP doesn't is, that the pictures render better, if you ask me. Especially when the real image size is larger than the WebBrowser control and this the image is downsized, VFPs isometric stretch does not result in the same quality. You can also nicely resized and resampled images using GDI+, but that's a whole different story and may also be an image control replacement I suggested in the PictureVal thread already. It would be more intense work, though.

If you want to use this, subclass the OleControl based on the Webbrowser control and add in the few new properties, Init(), Refresh(), and Destroy() event code, instead of using the class as a PRG. In the visual class and form designer, you won't see the image at design time, but it's far easier to size and anchor the control, any control, visually than by code. The PRG code is only for easier posting.

Bye, Olaf.

o = CreateObject("testform")
o.show(1)

Define Class testform as form
   Height = 325
   Width = 500
   Add Object img1 as wbimage With;
      Picture = '[URL unfurl="true"]https://picsum.photos/400/300',;[/URL]
      Top = 50,;
      Left= 50
EndDefine


**************************************************
*-- Class:        wbimage
*-- ParentClass:  olecontrol
*-- BaseClass:    olecontrol
*-- Time Stamp:   07/04/20 06:45:07 PM
*-- OLEObject = C:\Windows\SysWOW64\ieframe.dll
*
Define Class wbimage As OleControl
   OleClass = 'Shell.Explorer.2'
   Height = 225
   Width = 400
   *-- Specifies the graphics file or field to display on the control.
   Picture = ("")
   *-- html for this image "control" (as it's a webbrowser control under the hood)
   imagehtml = ""
   Name = "wbimage"

   Procedure Refresh
      *** ActiveX Control Method ***
      NoDefault
   Endproc

   Procedure Init
      Local lcHTML, lcHTMLFile

      TEXT To lcHTML NoShow TextMerge
      <!DOCTYPE html>
      <html lang="en">
        <head>
          <meta charset="utf-8" />
          <meta http-equiv="X-UA-Compatible" content="IE=edge; chrome=1" />
         <title>HTML img</title>
         <style>
      * {
         margin:0;
         border:0;
         padding:0;
      }

      .fullviewport {
        display:block;
        min-width:100%;
        min-height:100%;
        width:100%;
        height:100%;
        background-color:rgba(128,128,128,0.0);
        overflow: hidden;
      }

      .imageblock {
        display: table;
        width:100vw;
        height:100vh;
        text-align: center;
        white-space: nowrap;
      }

      .centered {
        display: table-cell;
        vertical-align: middle;
         }
         </style>
        </head>
        <body class="fullviewport">
        <div  class="imageblock">
          <div class="centered">
             <img src="<<This.Picture>>" style="max-width:100vw; max-height:100vh;">
          </div>
        </div>
        </body>
      </html>
      ENDTEXT

      This.imagehtml = Addbs(Getenv("TEMP"))+Sys(2015)+".html"

      Strtofile(lcHTML,This.imagehtml)

      This.navigate2("file://"+This.imagehtml)
   Endproc

   Procedure Destroy
      Try
         Erase (This.imagehtml)
      Catch
         *
      Endtry
   Endproc
Enddefine
*
*-- EndDefine: wbimage
**************************************************

Olaf Doschke Software Engineering

https://www.doschke.name

 

[Olaf Doschke]

To illustrate the problem with excess pixels, this is how the test form will look like, using the above code (the image will randomly vary):

The image size has aspect-ratio 4:3 and the WebBrowser based image "control" has an aspect-ratio 16:9, therefore you have the white margin sides. VFP's image control would render those margins in the greyish form tone.

There might be a way to make these two rectangles transparent by getting the HWND of the web-browser control and Windows API functions for WinForm transparency, but I haven't investigated, as it's much simpler to size the control as necessary. You could even do so automatically getting image width and height from GDI+ analyzing the image file and computing the effective size and margins, but then it would be easier to go for GDI+ (GDIplusX) all the way to render the image on the form canvas directly and have more control and means for manual garbage collection.

Bye, Olaf.

Olaf Doschke Software Engineering

https://www.doschke.name

 

[Vlad Smith]

You can also find related info here

https://wellmage.com

 

[vernpace]

Olaf,

Interesting proof of concept. We found a solution to our problem stated in thread184-1803757. You are correct as the issue is with the stretch = "isometric" property. Image width and height (not image size in bytes) are a problem if they greatly exceed the width and height of the image canvas. For instance, a 5400x3600 image will render nicely on a 1343x855 image canvas. However, as mentioned before, there is some kind of memory issue which slows down mouse events. Our solution is to use GDIPlusX to resize the image to that of the canvas. Here is an example:

PUBLIC toForm
LOCAL lcImage

toForm = CREATEOBJECT("cfrmpix")
toForm.Show()

lcImage = GETPICT()

IF FILE(lcImage)
   toForm.Setimage(lcImage)
ENDIF

DEFINE CLASS cfrmpix AS form

    Height = 855
    Width = 1343
    ShowWindow = 2
    DoCreate = .T.
    AutoCenter = .T.
    Caption = "Image Test"
    AlwaysOnTop = .T.
    BackColor = RGB(255,255,255)
    Name = "cfrmpix"

    ADD OBJECT image AS image WITH ;
        Anchor = 15, ;
        Stretch = 1, ;
        Height = 855, ;
        Left = 0, ;
        Top = 0, ;
        Visible = .F., ;
        Width = 1343, ;
        Name = "Image"

    PROCEDURE resizeimage
        LPARAMETERS tcFileSource AS String, tcFileDestination AS String, tiMaxWidth AS Integer, tiMaxHeight AS Integer
        LOCAL lcFileExt, loImage, liWidth, liHeight, liDecimals, liFactor, liWidthNew, liHeightNew

        WITH _Screen.System.Drawing 
             loImage  = .Bitmap.FromFile(tcFileSource, .T.)
             liWidth  = loImage.Size.Width
             liHeight = loImage.Size.Height

             loImage.Destroy()

             lcFileExt = UPPER(JUSTEXT(tcFileDestination))
             liDecimals = SET("Decimals")
             SET DECIMALS TO 4

             liFactor = MIN(tiMaxWidth/liWidth, tiMaxHeight/liHeight)
             liWidthNew = INT(liWidth  * liFactor)
             liHeightNew = INT(liHeight * liFactor)

             SET DECIMALS TO (liDecimals)

        *!*  Load the original Image 
             LOCAL loSrcImage as xfcBitmap 
             loSrcImage = .Bitmap.New(tcFileSource) 

        *!*  Create a New Image with the desired size 
             LOCAL loResized as xfcBitmap 
             loResized = .Bitmap.New(liWidthNew, liHeightNew, .Imaging.PixelFormat.Format32bppARGB)

        *!*  Set the image resolution to be the same as the original 
             loResized.SetResolution(loSrcImage.HorizontalResolution, loSrcImage.VerticalResolution) 

        *!*  Obtain a Graphics object to get the rights to draw on it 
             LOCAL loGfx as xfcGraphics 
             loGfx = .Graphics.FromImage(loResized) 

        *!*  Set some properties to ensure to have a better quality of image 
             loGfx.CompositingQuality = .Drawing2D.CompositingQuality.HighQuality
             loGfx.InterpolationMode = .Drawing2D.InterpolationMode.HighQualityBicubic 
             loGfx.SmoothingMode = .Drawing2D.SmoothingMode.HighQuality 

        *!*  Draw the source image on the new image at the desired dimensions 
             loGfx.DrawImage(loSrcImage, 0, 0, liWidthNew, liHeightNew) 

        *!*  Save the resized image 
             DO CASE
                CASE lcFileExt == "BMP"
                     loResized.Save(tcFileDestination, .Imaging.ImageFormat.Bmp)

                CASE lcFileExt == "GIF"
                     loResized.Save(tcFileDestination, .Imaging.ImageFormat.Gif)

                CASE INLIST(lcFileExt, "JPG", "JPEG")
                     loResized.Save(tcFileDestination, .Imaging.ImageFormat.Jpeg)

                CASE lcFileExt == "PNG"
                     loResized.Save(tcFileDestination, .Imaging.ImageFormat.Png)

                CASE INLIST(lcFileExt, "TIF", "TIFF")
                     loResized.Save(tcFileDestination, .Imaging.ImageFormat.Tiff)
                
             ENDCASE

             loResized.Destroy()

        ENDWITH
    ENDPROC


    PROCEDURE setimage
        LPARAMETERS tcImage AS String
        LOCAL lcImageDestination

        lcImageDestination = ADDBS(SYS(2023)) + SYS(2015) + "." + JUSTEXT(tcImage)

        ThisForm.ResizeImage(tcImage, lcImageDestination, 1343, 855)
        ThisForm.Image.PictureVal = FILETOSTR(lcImageDestination)
        DOEVENTS FORCE
        SYS(1104)
        ThisForm.Image.Visible = .T.
        ERASE(lcImageDestination)
    ENDPROC


    PROCEDURE Init
        DO system.app
    ENDPROC


ENDDEFINE

Please make sure to specify the correct path to system.app in the form's Init. The cool thing about this is the superb quality when resizing the form. The GDIPlusX code does a good job of downsizing AND upsizing images while minimizing unwanted artifacts. Extremely small images should not be used with this.

 

[vernpace]

Here is a 4288x2848 10.5MB JPG of The Great Wall of China. GDIPlusX downsized (optimized) the image to 1287x855 188KB to fit the image canvas. When the form is resized, the image control (Anchor = 15) resizes the image perfectly maintaining aspect ratio and quality - this is due to the use of PictureVal and Stretch = Isometric. Using Picture will not yield the same results.

 

[Olaf Doschke]

OK, so if I understand you right, you once size it down, and further resizing is done with the image control stretch mode, right?
I don't see why that resizing mechanism would differ with PictureVal or Picture, ie when you save the GDIplusX resized picture and use it as the file for the picture property, this should work as well.


So, do you still have the memory leak problems with that handling?


That reminds me, in conjunction with a website I once though with all the different device sizes I'd best determine display size in a bootstrap situation and then generate an exactly fitting image size. And actually someone teached me it's not worth too much effort. Yes, you save a little bandwidth for very large images. But as you wouldn't use raw uncompressed images on the web, what's true is

1. Image compression is good even with very high-resolution images. Well, especially with high resolution images.
2. The HTML img control is really good at resizing in most browsers. Also, like you observed, not only downsizing. So there is no need to let a webserver compute very individual images.

But yes, it's worth to at least only compute a few sensible sizes to pick from, one that is perhaps 3/4 display width times 3/4 display height is usually enough, considering exact display size for the special case of actually making it a full-size background, of course.

And in a desktop application, you have the advantage you don't need to bring a picture there.

Back on the question: No matter if you leave the stretching=resizing to the control or would also do it with GDIplusX, the image will be resized and repainted with every form resize event, besides repainting after something was clipped in front by another window and then becomes visible again, that's constantly done with the mouse pointer moving in front. And it doesn't degrade Windows over the time of a session.

I'd bet on image controls, including the VFP image control, do this without causing a GDI resource or memory leak. That's very fundamental base functionality and would constantly drive the OS to crash, if it wouldn't be done cleanly.

The thought that seems plausible is a difference between Picture and Pictureval is that Picture continually refers back to a file, which would stress the hard drive, but obviously it'll still only load the image from disc once, either from a separate file or from the EXE file with images included in the build. So the only detail may be what Mike already addressed spot on in the first answer, there is a difference if you simply assign FILETOSTR(image) to the PictureVal property or let it bind to a Blob field. and the Blob (or binary Memo) would be the thing that keeps the memory consumption lower than directly assigning the binary image content as PuctureVal property. I can't really put my fingers on why keeping a dbf/cursor field in memory would be more efficient and less prone to memory leaks than how a property stores its value, but perhaps the answer is in the first paragraph of Christof Wollenhaupt's explanation of the name table index VFP uses to handle to which memory names map:

http://www.foxpert.com/docs/howfoxproworks.en.htm

In Visual FoxPro we can name various items. To those items, Visual FoxPro assigns something called a name. These items are variables ([highlight #FCE94F]not properties[/highlight]), array names, procedures, functions, aliases of tables, field names and objects (not classes).

So properties are not a first citizen, they're just child nodes of some object that will be in the name table, ie the form object. Maybe it's that. And then you only will need to do what you do, resize an image to a sensible size for SYSMETRIC(1) x SYSMETRIC(2) or the image control max/start WIDTH x HEIGHT, create a cursor with a blob field, store the value into that and assign Image.PictureVal = "cursoralias.blobname" to be done with it.

And if you then still have a leak it must be your GDIPlusX resizing, even when you only do that once per image used in the application session and it only causes a leak every time another image is shown, not with every resize of the forms. It could be enough to bring down an application used throughout the whole day.

Bye, Olaf.

Olaf Doschke Software Engineering

https://www.doschke.name

 

[Olaf Doschke]

Btw, I don't see you use Dispose methods in your resize code, and in comparison here's what I use for resizing images with GDIplusX (incorporated your newwidth/height logic, mine was differently aiming for a target size):

#INCLUDE gdiplusconstants.h
LPARAMETERS tcFileSource AS String, tcFileDestination AS String, tiMaxWidth AS Integer, tiMaxHeight AS Integer

Local loImg, loBmp, loGraphics, lnPic, lnNewWidth, lnNewHeight, lnOldWidth, lnOldHeight
Local liFactor
loImg = _Screen.System.Drawing.Bitmap.FromFile(tcFileSource,.T.)

lnOldWidth  = loImg.Size.Width
lnOldHeight = loImg.Size.Height

liFactor = MIN(tiMaxWidth/lnOldWidth, tiMaxHeight/lnOldHeight)
lnNewWidth = INT(liWidth  * liFactor)
lnNewHeight = INT(liHeight * liFactor)

loResized = _Screen.System.Drawing.Bitmap.New(lnNewWidth,lnNewHeight,0,PixelFormat24bppRGB)
loResized.SetResolution(loImg.HorizontalResolution,loImg.VerticalResolution)
loGraphics = _Screen.System.Drawing.Graphics.FromImage(loResized)
loGraphics.DrawImage(loImg,0,0,loResized.Size.Width,loResized.Size.Height,0,0,loImg.Size.Width,loImg.Size.Height,UnitPixel)

       *!*  Save the resized image 
             DO CASE
                CASE lcFileExt == "BMP"
                     loResized.Save(tcFileDestination, .Imaging.ImageFormat.Bmp)

                CASE lcFileExt == "GIF"
                     loResized.Save(tcFileDestination, .Imaging.ImageFormat.Gif)

                CASE INLIST(lcFileExt, "JPG", "JPEG")
                     loResized.Save(tcFileDestination, .Imaging.ImageFormat.Jpeg)

                CASE lcFileExt == "PNG"
                     loResized.Save(tcFileDestination, .Imaging.ImageFormat.Png)

                CASE INLIST(lcFileExt, "TIF", "TIFF")
                     loResized.Save(tcFileDestination, .Imaging.ImageFormat.Tiff)
                
             ENDCASE

loGraphics.[highlight #FCE94F]Dispose()[/highlight]
loImg.[highlight #FCE94F]Dispose()[/highlight]
loResized.[highlight #FCE94F]Dispose()[/highlight]

You call destroy, that doesn't dispose of memory nor does it destroy the object, just like Pageframe.Page2.Activate() doesn't activate page 2. The VFP objects just have a destroy() event like any VFP objects, this doesn't come from GDI+, nor does it call Dispose, also not the base behavior of the objects. Destroy will happen when the variables are releases. And they are released, no matter if private or local. But the Dispose has to happen in advance. Just logical, you can only call a method on a still existing object. The last thing before destroy. And VFP's destroy does not do so, nor the native destructor of GDI+.

You need to call the Dispose methods, as GDI+ is lazy with garbage collection and within a VFP process that's mainöly single-threaded, not much in that direction will happen.

I batch process images with my routine, I have a few minor differences, ie a separate savepic() method to which I pass loResized, but essentially this way I resize images has never caused me memory trouble or degrading of application responsiveness or something like that. Indeed I just batch process images and then use them for the web, so there's also not much of a long duration application session. But I do think to .Dispose() makes the difference.

Bye, Olaf.

Olaf Doschke Software Engineering

https://www.doschke.name