I need a fast cache format 1

[qbasicking]

Here's the specifics of what I need: A graphics format. One that can be altered one pixil at a time, and doesn't need to be loaded to do it...OPEN-EDIT-CLOSE
I need to cache the contents of a window.
I need it to be somewhat fast, maybe fast enough to run on a 486 or 386, but I need it to hold 630*430 pixils
It also needs to be somewhat compact, the formats that I've come up with were huge (1.x MB).

Bitmaps and JPEGS would be nice, but I can't figure out how to edit them one pixil at a time.

When i load them, I need to be able to load only parts of it at a time too, sometimes maybe only a few pixils.

Heres some formats I've tried, but were to bulky and big:
PUT #1, 1, pxl1
PUT #1, 2, pxl2
PUT #1, 3, pxl3
...

PUT #1, 1, pxl1 + pxl2 + pxl3...

Any ideas would be helpful, thanks in advace.

 

[trollacious]

If you're using 256 colors on the screen, you can build a string with CHR$(pix) and it will store in half the memory that integers will. As well, you can create an array to hold the pixels in before converting that to a string. Disk reads and writes are slow, so the fewer you have to do, the faster the program will be. In QuickBASIC, I generally use 1K or 2K strings per disk read or write. If you read one pixel at a time from the screen and store that in the array, then your array will be 307,200 bytes long working with a full 640 x 480 screen. If you use 16 bit color, then you'd be best off converting the color of each pixel to an RGB 3-byte or 6-byte string and storing it that way. You could access the array/string at any pixel this way to manipulate colors, like you want to do.

 

[hardkor1001110]

editing bitmaps one pixel at a time isn't hard, just remember bitmaps are flipped on disk so (0,0) is (0, Height) on disk, (0,Height) is (0,0) on disk. Bitmaps also need padding if the width isn't evenly divisible by 4. 24 bit bmps are the easiest because the raw data always starts at file position 55. If you're using 8-bit bitmaps you'll need to read a bitmap specification document so you will know where to find the offset to the raw data. You will probably need to mess around with bitmaps a little to get used to the odd(but probably good reason) way they're stored. For manipulating a lot of pixels I'd recommend loading the data from disk into a Byte array, do what you want, and then save it back. JPG is a pretty sophisiticated compression algorithim so using it to do any kind of editing is inefficient if speed is an issue...

 

[logiclrd]

Hardkor1001110 is referring not to bitmaps in general, but to the standard BMP DIB format used by Windows. In general, any format that stores the data uncompressed is a bitmap format.

JPEG is not an option for what you want because it uses so-called "lossy compression" -- that is, the data that is stored is not actually identical to the original data, and if you load and save repeatedly, the data will become increasingly distorted.

By far the fastest way to access files on disk with QB is [tt]GET[/tt] and [tt]PUT[/tt]. I recommend you use an array of [tt]STRING * 1[/tt], and then use [tt]GET[/tt] and [tt]PUT[/tt] to read & write the entire array. This will allow you to do more than one pixel operation per read/write. Writing a single byte to disk requires that the entire sector be read, the one byte be modified, and then the entire sector written back, and QuickBASIC does no caching for you. Sectors are 512 bytes on most modern drives.

To say much more than this, I would need to know precisely what you are doing with the data on disk.

 

[qbasicking]

I started using GET and PUT, but they were unbelievably slow, on my 386 it took 5 minutes to load an image the size of what I want. They were also more than a megabyte each.

I can't store all of them in arrays, for the arrays for one window will be enourmous, and my program allows 20 windows!

 

[qbasicking]

I started using GET and PUT, but they were unbelievably slow, on my 386 it took 5 minutes to load an image the size of what I want. They were also more than a megabyte each.

I can't store all of them in arrays, for the arrays for one window will be enourmous, and my program allows 20 windows!

Bitmaps were one of my original ideas, but I couldn't figure out how to maipulate them in the way that I want. I have some experience working with bitmaps, but not very much, and only about 10% of them load correctly in the program that I wrote.

 

[logiclrd]

First of all, [tt]GET[/tt] and [tt]PUT[/tt] with individual items are quite slow. Since no caching is performed, reading two consecutive bytes one at a time with [tt]GET[/tt] results in the same sector being read twice from the hard drive. Since there are 512 bytes in a sector, reading one byte at a time results in 512 times as much data transfer than reading one sector at a time.

Secondly, from the resolution I will assume that you are using [tt]SCREEN 12[/tt]. As I recently mentioned in another thread, all VGA graphics modes other than mode 13h are planar, which means that the bits making up a single pixel are split across multiple planes. This means that to write a single pixel in [tt]SCREEN 12[/tt], 4 planes must be "exposed" in video memory, an entire byte must be read from each one, one bit of the byte changed, and then the byte stored back. This is an immensely inefficient operation. [tt]GET[/tt] and [tt]PUT[/tt], the graphics commands (not file access), optimize this by writing out the data to one entire plane at a time, rather than one entire pixel at a time, which requires only 4 bank switches. This is all very technical, but it should give you the gist of why you found [tt]GET[/tt] in conjunction with [tt]PSET[/tt] very slow in [tt]SCREEN 12[/tt]. One thing you could do as a very minimum is to break the windows up into one-line chunks, and then use the graphics [tt]GET[/tt] and [tt]PUT[/tt] to update the screen. Then, using the disk [tt]GET[/tt] and [tt]PUT[/tt] would result in only one operation per row, each one of which would read at most 2 sectors. Of course, updating individual pixels would be slightly tricky, because each pixel is 4 bits wide, and the graphical [tt]GET/PUT[/tt] memory format is "packed", meaning that no bits are wasted. As a result, you will need to determine whether the pixel you are updating is in the lower or upper 4 bits of the corresponding byte. However, this is not really hard to do. You just have to think about it.

 

[qbasicking]

I can't used GET and PUT, it takes a huge amount of code and time to maipulate them in the way that I want.
All of the windows will have a default size of 630*430, but since the windows are scalable, they won't be entire visible all the time.

 

[logiclrd]

Well, I did some testing, and it seems that in [tt]SCREEN 12[/tt], the [tt]GET/PUT[/tt] format is planar just like video memory. This certainly explains how they can be so fast, in comparison with all other graphics routines for that mode. It also makes manipulating the graphics essentially impossible, which restricts you to [tt]PSET[/tt] and [tt]POINT()[/tt], which are both hideously slow, especially in [tt]SCREEN 12[/tt]. You may not be able to achieve what you want in this video mode in QuickBASIC. Unfortunately, all the video modes other than [tt]SCREEN 13[/tt] are planar, which leaves you with a low resolution being your only hope of a fast overlapped windowing system, without going to assembler routines or a more capable language/environment.

 

[qbasicking]

assembly? I though of that too, but you can't use assembly in the SUBs, which is where all of this handling is done.

I've been doing extensive testing with GET and PUT (graphics). I wanted to save it, which I can do, but I still can't edit it nor can I just show part of it, I haven't decifered the weird binary code.

 

[qbasicking]

Thanks you anyway, you game me a great idea of how to load the background image. GET it, save it using PRINT #.

Good tip for anyone, you need a image format that doesn't require what mine does, use PRINT # and INPUT #, they are faster and are more compact.

 

[logiclrd]

Um, there's nothing weird about the graphics [tt]GET/PUT[/tt] format. The first WORD is the number of bits wide each scan is, the second WORD is the number of pixels vertically. Following that is the packed data for the pixels. In [tt]SCREEN 13[/tt], each scan consists of all of the pixels for a row. In all other modes, each row consists of multiple scans, one for each bit of the pixels. So, if you grab a section of [tt]SCREEN 12[/tt] that is 8 pixels wide and 1 pixel tall, the corresponding graphical [tt]GET/PUT[/tt] data is as follows:
[tt]
Offset Data Meaning
0 00000000-00000100 8 pixels wide
2 00000000-00000001 1 pixel tall
4 abcdefgh least significant bit of each pixel
5 abcdefgh next significant bit of each pixel
6 abcdefgh next-to-most significant bit of each pixel
7 abcdefgh most significant bit of each pixel
^^^^^^^^
|||||||`- pixel (7,0)
||||||`-- pixel (6,0)
|||||`--- pixel (5,0)
||||`---- pixel (4,0)
|||`----- pixel (3,0)
||`------ pixel (2,0)
|`------- pixel (1,0)
`-------- pixel (0,0)
[/tt]
You can see how the pixels are spread across multiple planes. This makes the format essentially useless for what you want to do (unless you are using [tt]SCREEN 13[/tt]).

The other [tt]GET/PUT[/tt], the pair used for file access, are MUCH, MUCH faster than [tt]PRINT[/tt] and [tt]INPUT$()[/tt]. If you've managed to write code that shows them to be slower, then you have managed to use the file-access [tt]GET/PUT[/tt] in about the least efficient manner possible. They are most efficient reading & writing large blocks of data. I have written a number of file handling programs, and [tt]GET/PUT[/tt] are blazing fast when using 2,048-byte blocks -- [tt]GET[/tt] can easily read several megabytes per second. Since [tt]GET[/tt] uses a pre-formed string, there is no overhead caused by creating a new string descriptor and allocating memory for it, like there is with [tt]INPUT$()[/tt]. Trust me, [tt]GET/PUT[/tt] are faster.

 

[qbasicking]

GET and PUT are faster, but INPUT and PRINT are more compact. I made two fonts, the same exact same thing. The one using INPUT and PRINT was 20KB, while the GET/PUT was a staggering 1MB

 

[logiclrd]

Um, [tt]GET[/tt] and [tt]PUT[/tt] are not a format, they are simply a means of accessing a file on the binary level. If you are creating huge files with [tt]PUT[/tt], then you are not using it correctly.

 

[qbasicking]

How do I make them smaller?
Here is the format that I used that created huge files:

FOR l = 1 TO 128 '128 ASCII characters
FOR x = 1 TO 6 '6 pixils wide
FOR y = 1 TO 8 '8 pixils high
p = p + 1
PUT #1, p, pixil(l,x,y) 'Pre-drawn pixil
NEXT
NEXT
NEXT
'Totaled about 1000KB


Here's the one I am currently using:

FOR l = 1 TO 128 '128 ASCII characters
FOR x = 1 TO 6 '6 pixils wide
FOR y = 1 TO 8 '8 pixils high
INPUT #1 pixil(l,x,y) 'Pre-drawn pixil
NEXT
NEXT
NEXT
Totaled about 30 KB

remember my question was about picture formats, i am just using my font as an example.

 

[logiclrd]

You could try something like this:
[tt]
SUB saveFont(font%(), filename$)
numFontChars% = UBOUND(font%, 1) - LBOUND(font%, 1) + 1
fontCharWidth% = UBOUND(font%, 2) - LBOUND(font%, 2) + 1
fontCharHeight% = UBOUND(font%, 3) - LBOUND(font%, 3) + 1
fontString$ = SPACE$(numFontChars% * fontCharWidth% * fontCharHeight%)

offset% = 0
FOR l% = LBOUND(font%, 1) TO UBOUND(font%, 1)
FOR x% = LBOUND(font%, 2) TO UBOUND(font%, 2)
FOR y% = LBOUND(font%, 3) TO UBOUND(font%, 3)
offset% = offset% + 1
MID$(fontString$, offset%, 1) = CHR$(font%(l%, x%, y%))
NEXT y%
NEXT x%
NEXT l%

ff% = FREEFILE
OPEN filename$ FOR OUTPUT AS #ff%
CLOSE #ff%
OPEN filename$ FOR BINARY AS #ff%
PUT #ff%, , numFontChars%
PUT #ff%, , fontCharWidth%
PUT #ff%, , fontCharHeight%
PUT #ff%, , fontString$
CLOSE #ff%
END SUB

SUB loadFont(font%(), filename$)
ff% = FREEFILE
OPEN filename$ FOR BINARY AS #ff%
GET #ff%, , numFontChars%
GET #ff%, , fontCharWidth%
GET #ff%, , fontCharHeight%

fontString$ = SPACE$(numFontChars% * fontCharWidth% * fontCharHeight%)

GET #ff%, , fontString$
CLOSE #ff%

REDIM font%(numFontChars% - 1, fontCharWidth% - 1, fontCharHeight% - 1)

offset% = 0
FOR l% = 0 TO numFontChars% - 1
FOR x% = 0 TO fontCharWidth% - 1
FOR y% = 0 TO fontCharHeight% - 1
offset% = offset% + 1
' Could be replaced with DEF SEG/PEEK() to read string
' directly from memory -- enhanced speed and less noise
' on the string heap
'
' If you want to do this, then first read my FAQ on
' segments & offsets, and then look up the VARSEG()
' and SADD() functions (QB45) or SSEG() and SADD()
' (QB71).
font%(l%, x%, y%) = ASC(MID$(fontString$, offset%, 1))
NEXT y%
NEXT x%
NEXT l%
END SUB
[/tt]
Whew, that was fun

 

[qbasicking]

That looks phenominally complicated, that really saves space? More than INPUT #, I ran your code and it was still thrice as big as my INPUT # code, and that faster much faster than it, only a second or two was the difference on my 386.
Had fun writing that though ;-) ? Yeah, fonts are fun. Try writing a GUI, I'm writing one and it's the most fun I've ever had in Qbasic.

 

[logiclrd]

The only way that the file could be smaller is if you are applying compression techniques to the other format.

 

[qbasicking]

But I don't know any, tht was what my original question was about (kinda)

 

[qbasicking]

can somebody explain to me why this doesn't work:

SCREEN 12
DIM icon(30)
GET (1,1)-(16,16), icon
OPEN file$ FOR OUTPUT AS #1
FOR a = 1 to 30
PRINT #1, icon(a)
NEXT
CLOSE

'then later

SCREEN 12
DIM icon(30)
OPEN file$ FOR INPUT AS #1
FOR a = 1 to 30
INPUT #1, icon(a)
NEXT
PUT (1,1), icon, PSET

why do I get an illegal function call, it should work.