# Privacy-preserving neural networks learning

## Applying Paillier Encryption Scheme

Figure 1 - Paillier encryption scheme.

Paillier encryption scheme is an asymmetric encryption scheme. It uses as the public-key,where - đ is a product of two big prime numbers and , and is an element of such that its order is a multiple of . The two prime numbers and are used as a secret-key. One version of this encryption scheme sets equals to and requires to be a prime of equal length (in their binary representation). The requirement assures that has an inverse modulo . The inverse of is denoted by and used as the secret-key [5]. The scheme is summarized in [Figure 1] . Suppose that two plaintexts and are encrypted to and where . We can see that

Since has order in and then

Furthermore, if , then

Because of this property, Paillier encryption scheme is called a partially homomorphic encryption scheme. This property can be generalize to addition of plaintexts

Now suppose that is a positive integer then

Since has order in and

Furthermore, if then

We will make use of these two properties to build a system where a machine learning model can take an input encrypted by Paillier encryption scheme and produce an output encrypted with the same key.

## The Machine Learning Model

First, we will train a neural network model with description given as below:

1.Input layer with z1 neurons corresponding to flattened array of the MNIST [4] image data.

2.First hidden layer: fully connected (dense) layer with z2 neurons and identity function as the activation function.

3.Second hidden layer: fully connected layer with z3 neurons, also having identity function as the activation function.

4.Output layer with z4 neurons corresponding to z4 classes of the MNIST database .

5.There are (z1 Ă z2) + (z2 Ă z3) + (z3 Ă z4) weights parameter and z2 + z3 + z4 biases parameter that can be optimized in the training process. This model can be written in a form of matrix multiplications as below:

(3)

ĐłĐ´Đľ:

• and respectively is a column vector of length z1, z2, z3 and z4.
• and respectively is a column vector of length z2, z3, z4.
• and respectively is a matrix of size z2 Ń z1, z3 Ń z2, z4 Ń z3.
• or is the output of the model in the form of a column vector of length z4.

After the training phase, the weights and biases will have a fixed value, so the model can then be simplified as .

where:

is a matrix of size z4 Ń z1 and a column vector of length z4 respectively [6]. Because of the softmax activation function in the output layer, this model cannot take an input encrypted with Paillier encryption scheme yet, softmax function is not a function that only consists of addition and scalar multiplication. However, we will see that the model is able to predict the input correctly without computing the softmax function in the output layer. Neural networks model determine the inputsâ class by looking at the index of the maximum value of .On the other hand, softmax function dosenât change the order of its input, for example if the input vector is Đ¸ we see that the elements of. In general, suppose that and we can assume that since is greater than 1 we can see that finally since is positive then Ńthus we have proven that in general case the elements of and have the same order. We can now safely drop the softmax function in the output layer of the model. The model is now simplified to

(4)

Suppose that Alice have successfully trained a model that has a structure like [(3)] which she then simplify to [(4)]. She then builds a web application so that people can access it easily via internet. Bob wants to use the model with his private data . The output that Bob looking for . Because is private, Bob will encrypt it first to which he then inputs it to Alicesâ app. In order to get , the app have to compute

Figure 2 - Summary.

(5)

which equals to

Therefore, using equation [(5)] Alicesâ app can outputs , when given as input. Bob can decrypt this this output to ,the value that he was looking for from the trained model. This way, Bob can still obtain the value that he was looking for from Alicesâ model without having to give his private data to Alice. Furthermore, Alice, as the application admin, wonât be able to read the outputâ which may be confidential as wellâsince it is in an encrypted form. [Figure 2] gives an illustration on how this concept works

## Conclusion

Paillier encryption scheme is homomorphic with addition and scalar multiplication of ciphertexts. Using the weight and dimensions of the simplified model, you can form an encrypted prediction for sensitive data.

## Bibliography

Perekos.V.A. Moscow, 2019