Loading ZipFile via a MemoryBlock

[nbacklund]

I’m trying to create a ZipFile from a MemoryBlock that’s in turn loaded from a File. The reason I want to go via the MemoryBlock is that I’m planning to encrypt the original file.

Loading the ZipFile directly from the File works as expected, so I’m assuming that my file is OK.

I’m loading the MemoryBlock in a constructor as follows:

File dataFile(FILE_PATH);
bool doesFileExist = dataFile.existsAsFile();
jassert(doesFileExist);
_rawData = new MemoryBlock(dataFile.getSize());
bool didFileLoad = dataFile.loadFileAsData(*_rawData);
jassert(didFileLoad);

None of the assertions fail. The only thing that seems strange is that this goes extremely fast, even though the file is about 300MB in size. So I’m already suspecting that something might be wrong.

Later on, in a separate thread, I try to create a ZipFile from this MemoryBlock and extract some data from it:

MemoryInputStream *memStream = new MemoryInputStream(*_rawData, false);
_zipFile = new ZipFile(*memStream);
const int numEntries = _zipFile->getNumEntries();
jassert(numEntries == 766);

Again, the assertion doesn’t fail, so it seems I really do get 766 entries in my ZipFile (running it in the debugger also seems to confirm this). Great! But… When I look closer at the ZipEntries in the debugger, it turns out that each one has the name “” and is 0 bytes in size…

I’m trying to figure out where I went wrong? Did I forget a step somewhere?

[railjonrogut]

Try not preallocating the size of the MemoryBlock… if memory (no pun intended) serves… loadFileAsData() appends to the MemoryBlock… so you get a memoryBlock twice the size that’s expected.

Rail

[nbacklund]

Yes, that certainly did the trick, thank you so much!

Cheers

[nbacklund]

And a followup question:

Now I’m trying to encrypt and decrypt the zipfile using Blowfish. I encrypted it using bcrypt on macOS, with compression turned off. According to the man page it always hashes the passphrase to 448 bits.

I tried decrypting it using the BlowFish class in the following way, appending to the constructor code in my first post:

String key = PASS_PHRASE;
BlowFish bf = BlowFish(&key, 448);
bf.decrypt(_rawData->getData(), _rawData->getSize());

Nothing crashes, but the resulting memory block is no longer recognizable as a zip file – numEntries is now 0.

So I’m not sure if I’m using the BlowFish class in the right way, if there’s a compatibility issue between bcrypt and BlowFish, or something else?

[railjonrogut]

I haven’t used bcrypt - so can’t confirm or deny if it’ll work… but check out this thread:

BlowFish String Encryption

I am a little bit confused, I figured that using: String::toHexString (textToEncrypt.getHexValue64()); would return a hex string. I realize that getHexValue64() returns an int64, but I thought that once that it is passed into the static method String::toHexString(), then a hex string would return. Though I probably misunderstood what the IntegerType Number parameter of the toHexString() method meant. Also, thanks for the tip, I’ll look into the MemoryOutputStream class. Yeah, my general is…

Rail

[nbacklund]

Thank you! Although I’m afraid that thread didn’t shed much more light on the subject for me.

Maybe I have to write a separate Juce app to encrypt the file instead of using bcrypt? I would expect the Blowfish implementation to be standardized in some way, so I’m still suspecting I’m doing something wrong, maybe I’m adding the key in the wrong format or something like that?

[railjonrogut]

How are you passing the key to the BlowFish constructor? You can write a test encrypt and decrypt test app to check your code… shouldn’t be too hard.

Rail

[nbacklund]

I pass the key as a pointer to a string which just contains the passphrase in clear text.

String key ("theSecretPassPhrase");
BlowFish bf(&key, 448);

The bcrypt man page says the following:

Passphrases may be between 8 and 56 characters. Regardless of the passphrase size, the key
is hashed internally to 448 bits - the largest keysize supported by the blowfish algorithm.
However, it is still wise to use a strong passphrase.

I’m not sure if the BlowFish class hashes the key, too? I just assumed that it did, since you also state the number of bits in the constructor. But the BlowFish class reference isn’t very verbose, so I’m not sure. I may have to dig into the source code to se if I can “decipher” it (heh).

But yeah, maybe using juce::BlowFish to encrypt the file is a safer bet? I’ll probably try that out tomorrow.

[railjonrogut]

It doesn’t take a String* as a parameter:

BlowFish (const void* keyData, int keyBytes);

check out that other thread for an example.

Perhaps make a feature request for a new BlowFish constructor which takes a String reference.

Rail

[railjonrogut]

BTW - If you code using Microsoft tools you shouldn’t start your variable/identifiers with an underscore.

Rail

[nbacklund]

I created a commandline tool to encrypt the file now, it was pretty easy!
However, I’m still not having any luck decrypting the file in my audio plugin afterwards.

Here’s the relevant parts of the code I use to encrypt the file in my commandline tool:

ScopedPointer<MemoryBlock> block = new MemoryBlock();
sourceFile.loadFileAsData(*block);
BlowFish bf(passKey.toUTF8(), static_cast<int>(passKey.getNumBytesAsUTF8()));
bf.encrypt(*block);
FileOutputStream fos(destFile);
fos.write(block, block->getSize());
fos.flush();

And here’s the corresponding code in my audio plugin:

_rawData = new MemoryBlock();
dataFile.loadFileAsData(*_rawData);
BlowFish bf(passKey.toUTF8(), static_cast<int>(passKey.getNumBytesAsUTF8()));
bf.decrypt(*_rawData);

I’ve double checked that I’m using the same pass phrase for both conversions

But the data is still not recognizable as a valid zip file! So I’m probably still missing something?

[jules]

Don’t know what your actual mistake is, but when I see java-style code like this (i.e. lots of totally unnecessary use of “new”) it doesn’t inspire much confidence that you’re managing lifetimes properly. Maybe start by learning how to properly allocate objects in C++, and maybe by fixing that aspect of your code you’ll also find a silly mistake somewhere.

[otristan]

@nbacklund

FileOutputStream fos(destFile);
fos.write(block, block->getSize());
fos.flush();

Obvisouly you don’t want to write block pointer but block actual data content

FileOutputStream fos(destFile);
fos.write(block->getData(), block->getSize());
fos.flush();

[nbacklund]

Oops, I missed that! It’s working fine now, thank you!

[jules]

It may be working fine, but it’s a mistake that the compiler would have choked on if you hadn’t heap-allocated that object. MemoryBlock is a textbook case of something that you should never ever heap-allocate!

[daniel]

I wonder, if similar to the final keyword there is some trickery to mark classes, that are not meant to be heap allocated, like MemoryBlock, Image, String, var etc…

Maybe a JUCE_PREVENT_HEAP_ALLOCATION like

private:
  void *operator new( size_t );
  void operator delete( void* );
  void *operator new[]( size_t );
  void operator delete[]( void* );

…just a thought to save beginners and supports time…

[jules]

Yes… The annoying thing is that although a beginner should never heap-allocate a whole bunch of classes like that, experts will end up hitting obscure edge-cases where they do need to do this for a good reason, so this would get in their way!