How to securely store encrypted files on the server?

I'm working on a messenger with E2E encryption of messages (per room keys). It's a very old project of mine, the thing is to be able to add an extra layer of security on top of the SSL and also to be able to provide at least some security without it.

But unfortunately, my knowledge in cryptography and algorithms is very poor (though I regularly read something about it for 3-4 years, the project is developed even for longer time).

I know basic things about symmetric/asymetric ciphers, certificates, Feistel networks, how 3DES, IDEA and similar algorithms work, how SSL works in basic terms. I also used to implement Diffie-Hellman algorithm with my own computing library, but I can't use it for now because the underlying library is still a bit buggy.

My encryption algorithm that works with messages uses some technique of key strengthening, but I am pretty sure that this is not enough to securely store and sent large files.

The hole idea of the app is not only to secure from a third party (only room members know their passwords, the server knows nothing about it), but also to secure data from server admins and hosters.

So now my question is the following. Imagine I am an evil person and have access to the underlying network channel, and no SSL is used. I somehow guess what file is being sent (by it's encrypted name length, or by the file size, it can be easily done if a file is some popular piece of software, for example). I can use some padding before encryption to conceal original size, but it won't really help, I guess.

So I guessed which file is being sent. I have the original, unencrypted file. Suppose our file is an exe/mp3 file at least 5-10 Mb long. But then I can just get the whole key data just by saving the ciphertext and used positions in subkeys in a big hashtable, because even after key strengthening my overall number of unique positions for applying is around 2000 * N bytes, where N is the length of the room password. So the files larger than 6-8K will not only compromise their contents, but also compromise the whole room!

Of course after that I won't get the original room password and even its real length, but I will have enough data (if the file is 1Mb or more) to decrypt anything in this room, period.

I know I can use Diffie-Hellman here and use temporary session keys which will be random on each session, but that does not help from the attack from the server side: web hoster has all the data before it is session-encrypted.

So,

1. What is my best way to go here? How should I modify/change my algorithms, ciphers, and key lengths?
2. What is the approximate strength of common ciphers such as AES if being used to encrypt large binary files and not small chunks of data is SSL? How many unique positions after strengthening phase are there in total from one 512 bit, 1024 bit key and so on?
3. I know that IVs are used in such ciphers, but I don't really know is there a way to store the IV used in encryption process in such a way that it cannot be read by anyone except the room members (also the app user should be able to use any device/browser to download and decrypt file). In case of my algorithm (it's not mine, of course, I just made some improvements) a random integer/character is used as a random seed, but it is appended to the ciphertext at the end of encryption, so I don't see how it can make the hole scheme better.

https://stackoverflow.com/questions/65785294/how-to-securely-store-encrypted-files-on-the-server January 19, 2021 at 11:59AM