Privacy-preserving neural networks learning
Applying machine learning to problems that include medical, financial, or other types of sensitive data requires not only accurate predictions, but careful attention to maintaining data privacy and security. Legal and ethical requirements prevent the use of cloud-based machine learning for this kind of task . This article presents a method for converting trained neural networks that can be applied to encrypted data. This allows the data owner to send their data in encrypted form to a [cloud] service that contains a neural network.Encryption ensures that the data remains confidential because the cloud does not have access to the keys needed to decrypt. Nevertheless, the cloud service is able to apply the neural network to the encrypted data, to generate encrypted forecasts, and also to return them in encrypted form to the copyright holder. These encrypted predictions can be sent back to the owner of the secret key that can decrypt them.herefore, the cloud service does not receive any information about the raw data or about the forecast made by it. Consider a hospital that would like to use the cloud service to predict the likelihood of a patient being hospitalized again over the next 30 days in order to improve the quality of care and reduce costs. Due to the ethical and legal requirements regarding the confidentiality of patient information, the hospital may be prohibited from using such a service. This article presents the way in which a hospital can use this service without sacrificing patient confidentiality.In the proposed protocol, the hospital encrypts personal information and sends it in encrypted form to the cloud. The cloud is able to generate a forecast for the encrypted data records and send back the results that the hospital can decipher and read. The encryption scheme uses the public key for encryption and the private key (private key) for decryption. It is important to note that the cloud does not have access to the private key, so it cannot decrypt the data and cannot decrypt the prediction. The only information he receives during this process is that he performed the forecast on behalf of the hospital.When using a homomorphic encryption scheme, you can encrypt your data in advance before entering it into the machine learning model. This procedure will allow you to make private and secure forecasts without the need to establish trust between the data owner and the service provider. This can be applied in areas such as healthcare, finance, business, etc. One of the encryption schemes, which has a homomorphic property, is the Paillier encryption scheme, first introduced in 1999 by Pascal Paillier. This encryption scheme has proven its semantic security against plaintext attacks.Paillier encryption scheme supports the addition of ciphertexts and the multiplication of ciphertext by plain text . With this property, we can do matrix multiplications and vector additions. Similar to the fact that the trained model of a small neural network can be simplified to matrix multiplication and vector addition (The principle of the neural network in cryptography can be found by clicking on the link ), therefore, this model can accept input data encrypted using an encryption scheme.
Applying Paillier Encryption Scheme
Paillier encryption scheme is an asymmetric encryption scheme. It uses as the public-key,where - 𝑛 is a product of two big prime numbers and , and is an element of such that its order is a multiple of . The two prime numbers and are used as a secret-key. One version of this encryption scheme sets equals to and requires to be a prime of equal length (in their binary representation). The requirement assures that has an inverse modulo . The inverse of is denoted by and used as the secret-key . The scheme is summarized in [Figure 1] . Suppose that two plaintexts and are encrypted to and where . We can see that
Because of this property, Paillier encryption scheme is called a partially homomorphic encryption scheme. This property can be generalize to addition of plaintexts
We will make use of these two properties to build a system where a machine learning model can take an input encrypted by Paillier encryption scheme and produce an output encrypted with the same key.
The Machine Learning Model
First, we will train a neural network model with description given as below:
2.First hidden layer: fully connected (dense) layer with z2 neurons and identity function as the activation function.
3.Second hidden layer: fully connected layer with z3 neurons, also having identity function as the activation function.
4.Output layer with z4 neurons corresponding to z4 classes of the MNIST database .
5.There are (z1 × z2) + (z2 × z3) + (z3 × z4) weights parameter and z2 + z3 + z4 biases parameter that can be optimized in the training process. This model can be written in a form of matrix multiplications as below:
- and respectively is a column vector of length z1, z2, z3 and z4.
- and respectively is a column vector of length z2, z3, z4.
- and respectively is a matrix of size z2 х z1, z3 х z2, z4 х z3.
- or is the output of the model in the form of a column vector of length z4.
After the training phase, the weights and biases will have a fixed value, so the model can then be simplified as .
is a matrix of size z4 х z1 and a column vector of length z4 respectively . Because of the softmax activation function in the output layer, this model cannot take an input encrypted with Paillier encryption scheme yet, softmax function is not a function that only consists of addition and scalar multiplication. However, we will see that the model is able to predict the input correctly without computing the softmax function in the output layer. Neural networks model determine the inputs’ class by looking at the index of the maximum value of .On the other hand, softmax function dosen’t change the order of its input, for example if the input vector is и we see that the elements of. In general, suppose that and we can assume that since is greater than 1 we can see that finally since is positive then тthus we have proven that in general case the elements of and have the same order. We can now safely drop the softmax function in the output layer of the model. The model is now simplified to
Suppose that Alice have successfully trained a model that has a structure like [(3)] which she then simplify to [(4)]. She then builds a web application so that people can access it easily via internet. Bob wants to use the model with his private data . The output that Bob looking for . Because is private, Bob will encrypt it first to which he then inputs it to Alices’ app. In order to get , the app have to compute
which equals to
Therefore, using equation [(5)] Alices’ app can outputs , when given as input. Bob can decrypt this this output to ,the value that he was looking for from the trained model. This way, Bob can still obtain the value that he was looking for from Alices’ model without having to give his private data to Alice. Furthermore, Alice, as the application admin, won’t be able to read the output— which may be confidential as well—since it is in an encrypted form. [Figure 2] gives an illustration on how this concept works
Paillier encryption scheme is homomorphic with addition and scalar multiplication of ciphertexts. Using the weight and dimensions of the simplified model, you can form an encrypted prediction for sensitive data.
- Machine learning
- Neural networks in cryptography
- Public key
- Private key
- Homomorphic encryption scheme
- Pailier encryption
- Asymmetric encryption schemes
Perekos.V.A. Moscow, 2019