Extensible Storage - GZIP

[richard.schaffranek]

Hello,

I would need to store some information in Extensible storage, unfortunalty the 16mb limit is a little small, so the idea was to simply gzip it to a string as demonstrated here : https://www.codeproject.com/Articles/27203/GZipStream-Compress-Decompress-a-String...

Issue is that sometimes it wokrs, sometimes it doesn't...

when I save to extensible storage I save this:

"‹\b\0\0\0\0\0\0ì½ÙrÛÈ–. + more  "

when I read it again I get:

"‹\b"

Which of course gives me no valid result when Unzipping again.

Any insights more than welcome...

[RPTHOMAS108]

Not sure what ES is doing on unmanaged side but the term 'null terminated string' comes to mind due to where the string stops.

Historically in null terminated strings the null is the end of the string and the next character is for a different allocation. This is how strings were stored in memory and often still are when you do P/Invoke etc. you need to know where the buffer is and how many characters it contains. So it is possible an unmanaged method is stopping at /0 for a similar reason.

What could be used instead of /0 perhaps? If you have compressed string to bytes stream you could use a form of serialization to store it? You will not get much benefit if your original content was not text. You can store a byte array also (AddArrayField with Byte).

I remember the times when all we needed was a 1.44MB disk to transfer things. With more space comes more ways to fill it.

[richard.schaffranek]

Hmm, ja that seams to be the case, however modified the compression a bit and converted the compressed byte stream to a base 64 string, this works. Except that if I have multiple files open and try to use the same schema on both files the redrived Entity Is always invalid..

[jeremy_tammik]

So, you have discovered the difference between a string and binary data!

Well done!

[jeremy_tammik]

Maybe take a look at the topic group:

https://thebuildingcoder.typepad.com/blog/about-the-author.html#5.23

Especially this hint on handling large amounts of data in estorage:

https://thebuildingcoder.typepad.com/blog/2011/06/extensible-storage-features.html#2

[jeremy_tammik]

You could also convert your zip file to an array of (binary) byte, short or long values instead of a (text) string...

[richard.schaffranek]

Not sure what you mean, the other file was also a string but obviously somehow it couldn't be handled...

[richard.schaffranek]

Yeah well 1.44 MB ... 

https://eeggs.com/images/items/4104.full.jpg