Array question - Need help from a Memory??, Heap??, Stack?? guru 2

[SBendBuckeye]

Hello,

I am primarily a VB programmer, not an API guy although I have worked with the APIs some. If the answer to this is so obvious it is either trivial or ridiculously impossible, please enjoy a chuckle or belly laugh at my expense! So here we go...

One of the things that bugs me about VB is that I cannot return or set an entire row of a given 2 dimensional array.
I was thinking about this the other night when I couldn't sleep and it occurred to me that if an array is (I think) a contiguous block of memory, then I should be able to map it using offset logic. But I don't work at that level so I don't know about how the memory is laid out.

To simplify things, let's say I have a 4 by 8 array named aMatrix with the following long integer data in it:

Row1: 1 10 100 1000
Row2: 2 20 200 2000
Row3: 3 30 300 3000
Row4: 4 40 400 4000
Row5: 5 50 500 5000
Row6: 6 60 600 6000
Row7: 7 70 700 7000
Row8: 8 80 800 8000

Let's also say I have a 4 by 1 array named aRow with the following long integer data in it:

Row1: 0 0 0 0

Now if arrays are just memory offsets, it seems like I should be able to do something like the following:

1. Calculate aMatrix Row5's beginning point in memory
2. Calculate aMatrix Row Length (Length of Long * 4)
3. Copy memory block offset from aMatrix calculated in 1
and 2 above into array aRow so that after the operation
aRow would contain 5 50 500 5000
4. It seems as if 3 should work because both blocks of
memory are the same length (eg Length of Long * 4)
5. If this is doable, does aRow need to be initialized or
just dimensioned to match an aMatrix row

If the above is doable, can it be generalized even further into this scenario...

Let's use array aMatrix as above. Now let's define a User Defined Type as below:

Type UDT
lng1 As Long
lng2 As Long
lng3 As Long
lng4 As Long
End Type

aUTD(7) As UDT

Again, if arrays and user defined types are contiguous blocks of memory, could I do something like this:

1. Calculate aMatrix beginning point in memory
2. Calculate aMatrix Length (Length of Long * 4 * Rows)
3. Copy memory block offset from aMatrix calculated in 1
and 2 above into array aUDT
4. It seems as if 3 should work because both blocks of
memory are the same length (eg Length of Long * 4 * Rows)
5. If this is doable, does aUDT need to be initialized or
just dimensioned to match aMatrix

Now the last set of questions (I aplogize for the long windedness of this all).

1. If either of the above is doable, how would I handle variants or strings, either of which can be variable in length? Or would I need to handle them individually?

Am I even in the ballpark with these concepts? If so, can I do it with APIs? If so, can someone point me? I've already downloaded the API guide from

http://www.AllAPI

and dug around in it so I'm willing to learn. Thanks to everyone who took the time to wade through the above and for any ideas and/or suggestions!


Have a great day!

j2consulting@yahoo.com

 

[MichaelRed]

hmmmmmmmmmmmmmm ... mmmmmmmmmm ... mmmmmmmmmm


Way off the beaten track (or your question(s). Why. Where ever the Array is originated, and however it is used, a give row could easily be selected in VB(A) nad the elements examined / manipulated. Like wise, the originating source could easily generate an array of Udt's and these could also be examined / manipiulated vis simple index values. To accompluish your transform seems rather like going the long way around the barn. Personally, I see little difference in the Array and UDT use, except that for the Array, ALL elements are of the same Type. While there are real differences, they would normally be of concern in handling large sets of values, even on some of the slower machines in commercial use today.




MichaelRed
m.red@att.net

Searching for employment in all the wrong places

 

[SBendBuckeye]

Hello Michael,

Thanks for the Saturday response. I know you can process a 2 dimensional array similar to the following:

For lngRow = LBound(MyArray, 2) To UBound(MyArray, 2)
For lngCol = LBound(MyArray, 1) To UBound(MyArray, 1)
'Process here
Next lngCol
Next lngRow

What I was looking for was a way to grab the whole Nth row without having to process the columns one at a time to extract them. Enjoy the rest of your weekend.

Have a great day!

j2consulting@yahoo.com

 

[strongm]

VB arrays are really SafeArrays. Try looking them up on MSDN.

 

[Hypetia]

BSendBuckEye, things coming to mind and the thoughts giving you sleepless nigths are certainly true. These things, I mean arrays and UDTs go exactly this way.

You can manipulate arrays, UDTs, strings and other data types using pointers and treat them as contiguous blocks of memory. This sometimes gives advantages like fast data transfers and access.

People generally say that VB cannot access the data at lower levels using memory pointers. But this is not true. VB contains a set of functions which enable it to effeciently manipulate the data using pointer arithmatic at low levels, but high speeds.

VB has a hidden module whose name is also "_HiddenModule" as it is not visible in Object Browser by default. This hidden module is only visible if the "Show Hidden Members" option is selected in the Object Browser.

This hidden module contains 3 very useful hidden functions named ObjPtr, StrPtr and VarPtr. These three functions return the pointer to an object, a string and a variable, respectively. In addition to these pointer functions, you can use the CopyMemory API to copy block of data from one memory location to another given their pointers and size of data to be copied/moved.

The following code example demonstrates all this. Just run this code step by step and see what happens.
___
[tt]
Option Explicit
Option Base 1 'default the lower index of all arrays to 1
Private Declare Sub CopyMemory Lib "kernel32" Alias "RtlMoveMemory" (Destination As Any, Source As Any, ByVal Length As Long)
Private Type UDT
lng1 As Long
lng2 As Long
lng3 As Long
lng4 As Long
End Type
Private Sub Form_Load()
'create the aMatrix array
Dim aMatrix(4, 8) As Long

'fill the aMatrix array
Dim Row As Integer, Col As Integer
For Row = 1 To 8
For Col = 1 To 4
aMatrix(Col, Row) = Row * 10 ^ (Col - 1)
Next
Next

'verify the contents of the aMatrix in
'the debug window they should be like this:

'Row1: 1 10 100 1000
'Row2: 2 20 200 2000
'Row3: 3 30 300 3000
'Row4: 4 40 400 4000
'Row5: 5 50 500 5000
'Row6: 6 60 600 6000
'Row7: 7 70 700 7000
'Row8: 8 80 800 8000
Debug.Print "contents of aMatrix"
For Row = 1 To 8
Debug.Print aMatrix(1, Row); aMatrix(2, Row); aMatrix(3, Row); aMatrix(4, Row)
Next

'now create the aRow() array
'this could also be a simple 1-dimensional
'array like this: Dim aRow(4) As Long
'but I created a 2-dimensional array as you
'explicitly mentioned a "4 by 1 array" in your post
Dim aRow(4, 1) As Long

'you don't need to fill the aRow array
'because its contents are already initialized to 0

'verify the contents of the aRow
Debug.Print "contents of aRow"
Debug.Print aRow(1, 1); aRow(2, 1); aRow(3, 1); aRow(4, 1)

'now the actual memory block copy operation.
Dim Pointer1 As Long, Pointer2 As Long, DataLength As Long

'1. Calculate aMatrix Row5's beginning point in memory
Pointer1 = VarPtr(aMatrix(1, 5))
'2. Calculate aMatrix Row Length (Length of Long * 4)
'this is obviously 16 as the size of a Long is 4.

'you can also calculate this length with the help of pointers
'by subtracting the starting address of two consecutive rows
'datalength=(row2 address) - (row1 address)
DataLength = VarPtr(aMatrix(1, 2)) - VarPtr(aMatrix(1, 1))

'3. Copy memory block offset from aMatrix calculated in 1
'and 2 above into array aRow so that after the operation
'aRow would contain 5 50 500 5000

'get the starting address of aRow
Pointer2 = VarPtr(aRow(1, 1))

'copy the data from aMatrix offset to aRow
CopyMemory ByVal Pointer2, ByVal Pointer1, DataLength

'at this moment, the row 5 of aMatrix is copied to aRow
'this can be verified. (5 50 500 5000)
Debug.Print "contents of aRow after copying aMatrix row 5"
Debug.Print aRow(1, 1); aRow(2, 1); aRow(3, 1); aRow(4, 1)
'---------------------------------------------------------
'the second thing with UDTs is also perfectly 'doable'
'let's see this
'create the UDT array.
'not aUDT(7) because it is a 1-based array
Dim aUDT(8) As UDT

'1. Calculate aMatrix beginning point in memory
Pointer1 = VarPtr(aMatrix(1, 1))

'2. Calculate aMatrix Length (Length of Long * 4 * Rows)
'this comes out to be 4 * 4 * 8 = 128 bytes this, also
'can be calculated using pointers. the formula is:
'datalength = (address of last element) - (address of first element) + (size of last element)
DataLength = VarPtr(aMatrix(4, 8)) - VarPtr(aMatrix(1, 1)) + Len(aMatrix(4, 8))
'the above also evaluates to 128.

'3. Copy memory block offset from aMatrix calculated in 1 and 2 above into array aUDT
Pointer2 = VarPtr(aUDT(1))
CopyMemory ByVal Pointer2, ByVal Pointer1, DataLength

'at this stage the entire aMatrix is copied to aUDT.
'let's see if it is true
Debug.Print "contents of aUDT after copying aMatrix"
For Row = 1 To 8
Debug.Print aUDT(Row).lng1; aUDT(Row).lng2; aUDT(Row).lng3; aUDT(Row).lng4
Next
'this will prove that contents of aMatrix are copied to aUDT!
End
End Sub[/tt]

 

[SBendBuckeye]

THANKS, Hypetia!

This is exactly the type of thing I was looking for. I knew that arrays were contiguous blocks of data in memory so I was pretty sure it could be done, I just wasn't sure how. Your example works like a charm and is extremely helpful because of the extra documentation you were kind enough to provide. Sorry I can only give you one star for your efforts.

The only thing your example did not address was string handling which I know is much trickier due to the length variation of strings. Am I correct when saying you would just use dynamic arrays instead and process them one row at a time with Redim Preserve as opposed to processing the entire block? Also, where do you go to find out things like the above? Thanks again for an excellent post!

Have a great day!

j2consulting@yahoo.com

 

[SBendBuckeye]

One other thing I didn't notice the first time. What purpose does the End line just before End Sub serve? Thanks!

Have a great day!

j2consulting@yahoo.com

 

[strongm]

Note that Hypetia's solution relies on at least one assumption - the way in which a SAFEARRAY orders its data (there is no specific reason why it might not store info column*row thather than row*column)

 

[Hypetia]

Arrays of strings?

Yes, strings are handled differently from other data types.
The reason for this is that strings variables are already pointers to data (strings) stored in memory.

A variable-length string in an array or UDT always occupies 4 bytes. These 4 bytes store the actual location where the string data is located. You can verify it from the following code fragment.
___
[tt]
Option Explicit
Private Type UDT
S1 As String
S2 As String
S3 As String
End Type
Private Sub Form_Load()
'checking size of strings in an array
Dim A(3) As String
A(1) = "SBendBuckeye"
A(2) = "MichaelRead"
A(3) = "strongm"

Debug.Print "Size of A(1) ="; VarPtr(A(2)) - VarPtr(A(1)) '4 (not 12)
Debug.Print "Size of A(2) ="; VarPtr(A(3)) - VarPtr(A(2)) 'also 4 (not 11)

'checking size of strings in a UDT
Dim aUDT As UDT
aUDT.S1 = "SBendBuckeye"
aUDT.S2 = "MichaelRead"
aUDT.S3 = "strongm"

Debug.Print Len(aUDT)
'gives 4 + 4 + 4 = 12 instead of 12 + 11 + 7 (= 30)
End Sub[/tt]
___

So there is no question about the "size" of variable-length strings, dynamic arrays, ReDim Preserve and things like that. Variable-length strings variables always occupy 4 bytes/string.

On the other hand, if fixed-length strings are used in UDTs, the actual string data is stored in the UDT variable instead of its 4-byte pointer only.
___
[tt]
Option Explicit
Private Type UDT
S1 As String * 15
S2 As String * 15
S3 As String * 15
End Type
Private Sub Form_Load()
Dim aUDT As UDT
aUDT.S1 = "SBendBuckeye"
aUDT.S2 = "MichaelRead"
aUDT.S3 = "strongm"

Debug.Print Len(aUDT)
'gives 15 + 15 + 15 = 45 instead of 12 or 30
End Sub[/tt]
___

Once again note that size of the string is fixed regardless of its contents. It is always the actual size of the fixed-length string no matter you assing a smaller or a larger value to the string variable.

If you obtain a variable pointer to a string using the VarPtr function, it will give you the location of this 4-byte variable, not the actual string data. To obtain the pointer to the actual data, you use StrPtr function, which returns the pointer value stored at these 4 bytes which is the actual location of string data.

Also note that the address of a string variable never changes in its life. But the location of the data pointed by it may change its value as you assign new values to a string.
___
[tt]
Private Sub Form_Load()
Dim S As String
S = "this"

Debug.Print "Location of string variable ="; VarPtr(S)
Debug.Print "Location of string data ="; StrPtr(S)

S = "that"
Debug.Print "Location of string variable ="; VarPtr(S)
Debug.Print "Location of string data ="; StrPtr(S)
End Sub[/tt]
___

You will notice that the value returned by StrPtr function will change when a new value is stored in the string but the return value of VarPtr will never change. The data at the old location (where "this" was stored) is invalidated as soon as the new value "that" is stored in the string.

See also thread222-740682 for a brief discussion on strings, pointers and the difference between StrPtr and VarPtr function.

And follow the link about strings, pointers and API that I posted in the above thread. Here I post it again.

http://msdn.microsoft.com/library/en-us/dnw32dev/html/ora_apiprog6_topic1.asp

It contains some excellent reading material regarding string storage and management in VB and ANSI/Unicode conversion which take place when strings are passed to APIs.

I hope that all the above stuff makes some sense and will not further confuse you.
I feel as if I am finding it difficult to explain all this stuff and making it more complicated so I stop right now.

...

The End statement in my previous post just terminates the program when all is done. If you remove this statement, the program will continue to run after the completion of Form_Load event and the form will show up (which was not needed so I simply terminated the program).

Thanks for the star.

 

[SBendBuckeye]

Thanks a lot guys! I don't know that I followed everything the first time, but enough that I can reread it a few times to figure out what is going on.

Hypetia - Thanks again for more explanations, excellent!

Strongm - Can you force a SAFEARRAY into either r*c or c*r storage and if not, how do you determine which it is?

Anyone, even though I can follow the above discussion I have never seen anything at this level in my reading. What does anyone suggest besides Knowledge Base articles, anything at all? Thanks again for your interest!

Have a great day!

j2consulting@yahoo.com

 

[strongm]

No, no.My point is that Hypetia's code works because of the way that SAFEARRAYS store stuff. Often, understanding the underpinnings of the world of Windows is key to getting good results.

In many ways the move to infrastructures such as .NET (and, to be honest, even COM) hides what is really happening from the programmer, and I often think this can be detrimental. IMHO understanding how/why things work is much better than simply following presecribed rules.

 

[MichaelRed]

strongm,

" ... hides what is really happening from the programmer ... "

... so you think that learning assembler for the CPU will improve the understanding of many / most programmers?

Having started my 'computer' life with machine code (Octal and Hex on DEC PDP series and Systems Engineering Lab processors), I have often thought this to be the case, but seldom hear anyone else express it (at least not "recently&quot.





MichaelRed
m.red@att.net

Searching for employment in all the wrong places

 

[elHollandes]

Hi all,

A little late reading this discussion that handles part of what I am working on at the moment. Anyway, some comments:

SBendBuckeye : Check out the MSDN Lybary books section and espacially the book called "visual basic hardcore". This is written for VB5 and the author doesn't want to update it for VB6, but it helpt me a lot. The book comes with a CD that holds among other things an API type lybary. Since the author is't going to publish an updated version, this libary is availeble on Internet. I don't remember the URL, but searge for the authors name Bruce McKinney and you'll find it.

MichalRed and Strongm : Sure it'll help a programmer to understand programming better it he understands how the CPU is working, how the operating system is interpretating that and how his programming language is working with the operating system. Starting with 6510 machine language I miss a good source for this kind of information in the modern programming world. Encapsuling is nice, but it hides so much that I feel a bit blindfolded some times.

Well, I go on investigating a bit on arrays, collections and objects. I'll start a new thread when I have something that I can put into words.

 

[SBendBuckeye]

I revisited this thread and thought I would pull all of Hypetia's code into one block for the benefit of others who may be learning as much as I did from it. Thanks again!

Option Explicit

Private Declare Sub CopyMemory Lib "kernel32" Alias "RtlMoveMemory" (Destination As Any, Source As Any, ByVal Length As Long)

Private Type UDT_Long
lng1 As Long
lng2 As Long
lng3 As Long
lng4 As Long
End Type

Private Type UDT_String
S1 As String
S2 As String
S3 As String
End Type

Private Type UDT_FixedString
S1 As String * 15
S2 As String * 15
S3 As String * 15
End Type

Private Sub Form_Load()
Call CopyArray
Call CopyArray2
End
End Sub

Private Sub CopyArray()
Dim aUDT_Long(8) As UDT_Long
Dim aMatrix(4, 8) As Long, aRow(4, 1) As Long
Dim Row As Integer, Col As Integer
Dim SourcePtr As Long, TargetPtr As Long, DataLength As Long

'Fill aMatrix array
For Row = 1 To 8
For Col = 1 To 4
aMatrix(Col, Row) = Row * 10 ^ (Col - 1)
Next Col
Next Row

'Verify contents of aMatrix in debug window - Should be
'Row1: 1 10 100 1000
'Row2: 2 20 200 2000
'Row3: 3 30 300 3000
'Row4: 4 40 400 4000
'Row5: 5 50 500 5000
'Row6: 6 60 600 6000
'Row7: 7 70 700 7000
'Row8: 8 80 800 8000
Debug.Print "Contents of aMatrix"
For Row = 1 To 8
Debug.Print "Row: " & Row; " Values: "; aMatrix(1, Row); aMatrix(2, Row); aMatrix(3, Row); aMatrix(4, Row)
Next Row

'Now actual memory block copy operation

'1. Calculate aMatrix Row5's beginning point in memory
SourcePtr = VarPtr(aMatrix(1, 5))

'2. Calculate aMatrix Row Length (Length of Long * 4)
'Length is obviously 16 in example as size of a Long = 4
'Length can be calculated with the help of pointers by
'subtracting starting addresses of two consecutive rows
'eg DataLength = (row2 address) - (row1 address)
DataLength = VarPtr(aMatrix(1, 2)) - VarPtr(aMatrix(1, 1))

'3. Get starting address of aRow
TargetPtr = VarPtr(aRow(1, 1))

'4. Copy memory block offset from aMatrix calculated in 1
'and 2 above into array aRow so that after the operation
'aRow would contain 5 50 500 5000

'Copy data from aMatrix offset to aRow
CopyMemory ByVal TargetPtr, ByVal SourcePtr, DataLength

Debug.Print "Contents of aRow after copying aMatrix row 5"
Debug.Print aRow(1, 1); aRow(2, 1); aRow(3, 1); aRow(4, 1)

'---------------------------------------------------------
'Second thing with UDTs is also perfectly 'doable'

'1. Calculate aMatrix beginning point in memory
SourcePtr = VarPtr(aMatrix(1, 1))

'2. Calculate aMatrix Length (Length of Long * 4 * Rows)
'This comes out to be 4 * 4 * 8 = 128 bytes - This also
'can be calculated using pointers - the formula is:
'DataLength = (address of last element) - _
' (address of first element) + (size of last element)
DataLength = VarPtr(aMatrix(4, 8)) - VarPtr(aMatrix(1, 1)) + Len(aMatrix(4, 8))

'3. Get starting address of aUDT_Long
TargetPtr = VarPtr(aUDT_Long(1))

'4. Copy memory block offset from aMatrix into array aUDT_Long
CopyMemory ByVal TargetPtr, ByVal SourcePtr, DataLength

Debug.Print "Contents of aUDT_Long after copying aMatrix"
For Row = 1 To 8
Debug.Print "Row: " & Row; " Values: "; aUDT_Long(Row).lng1; aUDT_Long(Row).lng2; aUDT_Long(Row).lng3; aUDT_Long(Row).lng4
Next Row
'this will prove that contents of aMatrix are copied to aUDT_Long!

End Sub

Private Sub CopyArray2()
'checking size of strings in an array
Dim A(3) As String
A(1) = "SBendBuckeye"
A(2) = "MichaelRead"
A(3) = "strongm"

Debug.Print "Size of A(1) ="; VarPtr(A(2)) - VarPtr(A(1)) '4 (not 12)
Debug.Print "Size of A(2) ="; VarPtr(A(3)) - VarPtr(A(2)) 'also 4 (not 11)

'checking size of strings in a UDT
Dim aUDT_String As UDT_String
aUDT_String.S1 = "SBendBuckeye"
aUDT_String.S2 = "MichaelRead"
aUDT_String.S3 = "strongm"

'gives 4 + 4 + 4 = 12 instead of 12 + 11 + 7 (= 30)
Debug.Print "Length of aUDT_String = "; Len(aUDT_String)

Dim aUDT_FixedString As UDT_FixedString
aUDT_FixedString.S1 = "SBendBuckeye"
aUDT_FixedString.S2 = "MichaelRead"
aUDT_FixedString.S3 = "strongm"

'gives 15 + 15 + 15 = 45 instead of 12 or 30
Debug.Print "Length of aUDT_FixedString = "; Len(aUDT_FixedString)

Dim S As String
S = "this"

Debug.Print
Debug.Print S
Debug.Print "Location of string variable ="; VarPtr(S)
Debug.Print "Location of string data ="; StrPtr(S)

S = "that"
Debug.Print S
Debug.Print "Location of string variable ="; VarPtr(S)
Debug.Print "Location of string data ="; StrPtr(S)

End Sub

Have a great day!

j2consulting@yahoo.com

 

[Gila47ac]

The posts so far cover the movement of data FROM a matrix to a row, but there's more to the story. Matthew Curland's book Advanced Visual Basic 6 talks about the hidden flags that are associated with arrays (i.e., a VB array is more than a contiguous block of memory). My copy is at the office, and it's been a long time since I read the relevant chapter. But you need to be aware of those flags if you are going to modify the contents of an array using CopyMemory, especially an array that is passed as a parameter to your code.

Bruce McKinney's book Hardcore Visual Basic: Version 5.0 has good descriptions of the internals of VB strings (BSTR), the undocumented pointer functions that Hypetia described, how to create linked lists, etc.

 

[strongm]

Also we've covered SAFEARRAYs a number of times in the Visual Basic(Microsoft): Version 5 & 6 Forum (forum222)