I had a chance to work on a project in which data was encrypted and shared between a Python program on the server side and a Silverlight .NET Framework application on the client side. Both programming environments offer a rich set of libraries for doing data encryption. On the Python side I chose to use the excellent
PyCrypto library. At the time that I wrote the code I was using version 2.1 of PyCrypto and version 2.7 of Python. PyCrypto is in the public domain from the way I understand the license files. On the .NET side, the System.Security.Cryptography namespace provides the classes needed. In a Silverlight application the System.Security.Cryptography namespace is much, much smaller than in the full .NET Framework. Thus AES would have to be the encryption algorithm used.
AES is the Rijndael symmetric algorithm with a fixed block size (128 bits) and iteration count. For the
Silverlight version of this algorithm the cipher mode and padding mode are CBC and PKCS7 respectively and can not be changed. Padding is used to ensure that the data to be encrypted is a multiple of the block size.
Here is an example of encrypting the data in Python. You can find the code in this article on
GitHub. Obviously, you will need to have
Python and PyCrypto installed to use it.
1 from Crypto.Cipher import AES
2 from pkcs7 import PKCS7Encoder
3 import base64
4
5 key = 'your key 16bytes'
6 # 16 byte initialization vector
7 iv = '1234567812345678'
8
9 aes = AES.new(key, AES.MODE_CBC, iv)
10 encoder = PKCS7Encoder()
11
12 text = 'This is my plain text'
13
14 # pad the plain text according to PKCS7
15 pad_text = encoder.encode(text)
16 # encrypt the padding text
17 cipher = aes.encrypt(pad_text)
18 # base64 encode the cipher text for transport
19 enc_cipher = base64.b64encode(cipher)
20
21 print enc_cipher
The data to be encrypted is first run through the padding encoder. This will ensure that the data is a multiple of the block size (16 bytes in this case). Note that even if the data is already the correct size it is still padded. The data is always padded thus the padding must always need to be removed. This alleviates the programmer from having to know if the data is padded or not. This program will print the following encoded string: ZeYXkFf8wPbvzdC91V4adwx4U56o2zMMOathdDYuBOE=
The PKCS7 padding code is built in to the Silverlight AesManaged class. On the Python side, I had to write this my self. PKCS7 is described in
RFC 2315 and is actually very simple, here is the code.
1 import binascii
2 import StringIO
3
4 class PKCS7Encoder(object):
5 '''
6 RFC 2315: PKCS#7 page 21
7 Some content-encryption algorithms assume the
8 input length is a multiple of k octets, where k > 1, and
9 let the application define a method for handling inputs
10 whose lengths are not a multiple of k octets. For such
11 algorithms, the method shall be to pad the input at the
12 trailing end with k - (l mod k) octets all having value k -
13 (l mod k), where l is the length of the input. In other
14 words, the input is padded at the trailing end with one of
15 the following strings:
16
17 01 -- if l mod k = k-1
18 02 02 -- if l mod k = k-2
19 .
20 .
21 .
22 k k ... k k -- if l mod k = 0
23
24 The padding can be removed unambiguously since all input is
25 padded and no padding string is a suffix of another. This
26 padding method is well-defined if and only if k < 256;
27 methods for larger k are an open issue for further study.
28 '''
29 def __init__(self, k=16):
30 self.k = k
31
32 ## @param text The padded text for which the padding is to be removed.
33 # @exception ValueError Raised when the input padding is missing or corrupt.
34 def decode(self, text):
35 '''
36 Remove the PKCS#7 padding from a text string
37 '''
38 nl = len(text)
39 val = int(binascii.hexlify(text[-1]), 16)
40 if val > self.k:
41 raise ValueError('Input is not padded or padding is corrupt')
42
43 l = nl - val
44 return text[:l]
45
46 ## @param text The text to encode.
47 def encode(self, text):
48 '''
49 Pad an input string according to PKCS#7
50 '''
51 l = len(text)
52 output = StringIO.StringIO()
53 val = self.k - (l % self.k)
54 for _ in xrange(val):
55 output.write('%02x' % val)
56 return text + binascii.unhexlify(output.getvalue())
Remember that each time the AES key is used to encrypt/decrypt a block of data, the internal state of the key changes. Thus each time you are preparing encrypted data to transmit to the server you must create a new AES key object from the original key and iv or that data will not be properly transformed. The server will also have to create a new AES key with the original data as well. If you do not, you will have a hard time keeping the server and client AES keys in sync and data will not be transformed properly on either side.
Decryption on the Silverlight side is very straight forward. I am presenting this here as a console based program but there should not be any relevant difference when used in a Silverlight program.
1 using System;
2 using System.Text;
3 using System.Security.Cryptography;
4
5 namespace AesTest
6 {
7 class Program
8 {
9 static void Main(string[] args)
10 {
11 // This was the output of our Python program.
12 string enc_cipher = "ZeYXkFf8wPbvzdC91V4adwx4U56o2zMMOathdDYuBOE=";
13
14 var textEncoder = new UTF8Encoding();
15
16 // defaults to CBC and PKCS7
17 var aes = new AesManaged();
18 aes.Key = textEncoder.GetBytes("your key 16bytes");
19 aes.IV = textEncoder.GetBytes("1234567812345678");
20
21 var decryptor = aes.CreateDecryptor();
22 var cipher = Convert.FromBase64String(enc_cipher);
23 var text_bytes = decryptor.TransformFinalBlock(cipher, 0, cipher.Length);
24
25 var text = textEncoder.GetString(text_bytes);
26 // Should print 'This is my plain text'
27 Console.WriteLine(text);
28 }
29 }
30 }
It should be easy for any programmer to reverse the process so the the Silverlight client encrypts data that is then decrypted on the python side. The hard part here is the age old problem of how you share the key to the symmetric encryption algorithm. For this we used RSA public key cryptography.
PyCrypto once again comes to the rescue as it includes RSA encryption classes, for the most part. Once again I had to write a padding encoder/decoder to be used along with the RSA classes. RSA can use a variety of padding schemes like PKCS#1 v1.5, and OAEP (optimal asymmetric encryption padding), with OAEP being recommended for new applications. I wrapped the Crypto.PublicKey.RSA class in my own RSAkey class shown below. As you can see in the code the data is padded, encrypted, and then base64 encoded. The data can easily be transmitted via web protocols when it is base64 encoded.
1 from Crypto.PublicKey import RSA
2 from pkcs1 import OAEPEncoder
3 import base64
4 import binascii
5 import os
6 import pickle
7
8 class RSAKey(object):
9
10 def __init__(self, keybitsize, encoder=OAEPEncoder()):
11 self._encoder = encoder
12 self._keysize = keybitsize
13 self._key = RSA.generate(keybitsize, os.urandom)
14
15 @property
16 def key(self):
17 return self._key
18
19 @property
20 def key_size(self):
21 return self._keysize
22
23 ## Get the public RSA key used to encrypt data as
24 # an XML string.
25 # @param xml_format True if the key should be returned as XML.
26 # If False, the key is returned as a base64 encoded pickled
27 # Python object.
28 # @return: An XML string representation of the
29 # public RSA key. Each node is a base64
30 # encoded string. It has the following
31 # structure.
32 # \<RSAKeyValue\>
33 # \<Exponent\>AQAB\</Exponent\>
34 # \<Modulus\>some data\</Modulus\>
35 # \</RSAKeyValue\>
36 def public_key(self, xml_format):
37 pkey = self._key.publickey()
38
39 if xml_format:
40 # Pads with leading zeros if needed.
41 def ensure_length(hexstr):
42 if len(hexstr) % 2 != 0:
43 return '0' + hexstr
44 else:
45 return hexstr
46 # make an encoded child node
47 def add_child(tag, n):
48 str_n = ensure_length('%x' % n)
49 n_bytes = binascii.unhexlify(str_n)
50
51 sub = et.SubElement(root, tag)
52 sub.text = base64.b64encode(n_bytes)
53
54 root = et.Element('RSAKeyValue')
55 add_child('Exponent', pkey.e)
56 add_child('Modulus', pkey.n)
57 return tostring(root)
58 else:
59 return base64.b64encode(pickle.dumps(pkey))
60
61 ## Encrypt data with the public RSA key.
62 # @param data The data to be encrypted
63 # @return A base64 encoded string that is the encrypted data.
64 def encrypt(self, data):
65 enc_data = self._encoder.encode(data, keybits=self._keysize)
66 cipher = self._key.encrypt(enc_data, '')
67 return base64.b64encode(cipher[0])
68
69 ## Decrypt data with the private RSA key.
70 # @param encoded_cipher A base64 encoded string of encrypted data.
71 # @return The decrypted data as a string.
72 def decrypt(self, encoded_cipher):
73 cipher = base64.b64decode(encoded_cipher)
74 enc_data = self._key.decrypt(cipher)
75 data = self._encoder.decode(enc_data)
76 return data
Following are the classes I wrote that implement two of the padding schemes (OAEP, PKCS#1 v1.5)described in
rfc 2437. Coming from a C background, I find it hard to work with binary data in python. i know that there must be more efficient ways to handle binary data. The python standard library modules cStringIo, binascii and struct come in very handy.
You may notice that some of the variable names in the following code are not ideal (sorry Uncle Bob!). They actually reflect the names used within the RFC to make it easier for someone to follow along with the RFC document. Also, for those of you that do not know Python well, an '_' prefix to a field name in a class is the convention for specifying a private class field. When a Python programmer sees a field like this: self._hash_length, it is clear (to Python programmers) that the author intended this to be private. This is just a convention that you must enforce upon yourself as the Python language will allow you to access the field via an instance of the class.
1 import binascii
2 import cStringIO
3 import hashlib
4 import os
5 import struct
6
7 class PKCS1Error(RuntimeError):
8 '''
9 Base class for PKCS1 encoding/decoding errors.
10 Error of this or derived classes should be caught
11 by the calling code and then a generic error message
12 should be returned to the caller.
13 '''
14 pass
15
16 class DecoderError(PKCS1Error):
17 '''
18 Raised when a decoding error has been detected.
19 '''
20 pass
21
22 class EncoderError(PKCS1Error):
23 '''
24 Raise when an encoding error has been detected.
25 '''
26 pass
27
28
29 class PKCSAuxiliary(object):
30 '''
31 Auxiliary functions used in RFC 2437
32 '''
33
34 def __init__(self):
35 self._hash_length = None
36
37 @property
38 def hash_length(self):
39 if not self._hash_length:
40 hasher = self.create_hasher()
41 self._hash_length = hasher.digest_size
42
43 return self._hash_length
44
45 @staticmethod
46 def create_hasher():
47 return hashlib.sha1()
48
49 @staticmethod
50 def compute_hash(data, hex_digest=False):
51 hasher = PKCSAuxiliary.create_hasher()
52 hasher.update(data)
53 if hex_digest:
54 return hasher.hex_digest()
55 else:
56 return hasher.digest()
57
58 def mgf(self, seed, length):
59 '''
60 RFC 2437 page 28 MFG1
61 '''
62 counter = 0
63 output = cStringIO.StringIO()
64 try:
65 limit = length / self.hash_length
66 while counter <= limit:
67 C = self.i2osp(counter)
68 output.write(self.compute_hash(seed + C))
69 counter += 1
70
71 raw_mask = output.getvalue()
72 if len(raw_mask) < length:
73 raise PKCS1Error("MGF: mask too long")
74 finally:
75 output.close()
76
77 mask = raw_mask[:length]
78 return mask
79
80 def i2osp(self, x):
81 '''
82 RFC 2437 page 6 I2OSP
83 Special case where length = 4
84 '''
85 if x > 256 ** 4:
86 raise PKCS1Error("I2OSP: integer too large")
87
88 sp = (
89 int((x >> 24) & 0xff),
90 int((x >> 16) & 0xff),
91 int((x >> 8) & 0xff),
92 int((x >> 0) & 0xff)
93 )
94
95 return struct.pack('BBBB', *sp)
96
97 @staticmethod
98 def xor(a, b):
99 '''
100 RFC 2437 bitwise exclusive-or of two octet strings.
101 page 23
102 '''
103 if len(a) != len(b):
104 raise PKCS1Error("XOR: invalid input lengths")
105
106 output = cStringIO.StringIO()
107
108 try:
109 for i in xrange(len(a)):
110 x = int(binascii.hexlify(a[i]), 16)
111 y = int(binascii.hexlify(b[i]), 16)
112 output.write('%02x' % (x ^ y))
113
114 data = output.getvalue()
115
116 finally:
117 output.close()
118
119 return binascii.unhexlify(data)
120
121
122 class OAEPEncoder(PKCSAuxiliary):
123 '''
124 RFC 2437 9.1.1 EME-OAEP PKCS1-v2.0
125 9.1.1.1 EME-OAEP-ENCODE
126 9.1.1.2 EME-OAEP-DECODE
127 '''
128
129 def __init__(self):
130 super(OAEPEncoder, self).__init__()
131
132
133 def encode(self, msg, salt='', keybits=1024):
134 k = keybits / 8
135 if len(msg) > (k - 2 - 2 * self.hash_length):
136 raise EncoderError("EME-OAEP: message too long")
137
138 emLen = k - 1
139 if (emLen < (2 * self.hash_length + 1) or
140 len(msg) > (emLen - 1 - 2 * self.hash_length)):
141 raise EncoderError("EME-OAEP: message too long")
142
143 pslen = emLen - len(msg) - 2 * self.hash_length - 1
144 output = cStringIO.StringIO()
145 try:
146 for _ in xrange(pslen):
147 output.write('%02x' % 0)
148 ps = binascii.unhexlify(output.getvalue())
149 assert len(ps) == pslen, "PS: invalid length"
150 finally:
151 output.close()
152
153 shash = self.compute_hash(salt)
154 dbout = cStringIO.StringIO()
155 try:
156 dbout.write(shash)
157 dbout.write(ps)
158 dbout.write('\x01')
159 dbout.write(msg)
160 db = dbout.getvalue()
161 finally:
162 dbout.close()
163
164 seed = os.urandom(self.hash_length)
165 assert len(seed) == self.hash_length
166
167 dbMask = self.mgf(seed, emLen - self.hash_length)
168 maskedDB = self.xor(db, dbMask)
169 seedMask = self.mgf(maskedDB, self.hash_length)
170 maskedSeed = self.xor(seed, seedMask)
171 emout = cStringIO.StringIO()
172 try:
173 emout.write(maskedSeed)
174 emout.write(maskedDB)
175 emsg = emout.getvalue()
176 finally:
177 emout.close()
178 return emsg
179
180
181 def decode(self, emsg, salt=''):
182 if len(emsg) < (2 * self.hash_length + 1):
183 raise DecoderError("EME-OAEP: decoding error")
184
185 maskedSeed = emsg[:self.hash_length]
186 maskedDB = emsg[self.hash_length:]
187 seedMask = self.mgf(maskedDB, self.hash_length)
188 seed = self.xor(maskedSeed, seedMask)
189 dbMask = self.mgf(seed, len(emsg) - self.hash_length)
190 db = self.xor(maskedDB, dbMask)
191 shash = self.compute_hash(salt)
192
193 db_shash = db[:self.hash_length]
194 if db_shash != shash:
195 raise DecoderError("EME-OAEP: decoding error")
196
197 index = db.find('\x01', self.hash_length)
198 if - 1 == index:
199 raise DecoderError("EME-OAEP: decoding error")
200
201 return db[index + 1:]
202
203
204
205 class PKCS1v1_5Encoder(object):
206 '''
207 RFC 2437 9.1.2 EME-PKCS1-v1_5
208
209 9.1.2.1 EME-PKCS1-v1_5-ENCODE
210 9.1.2.2 EME-PKCS1-v1_5-DECODE
211 '''
212
213 def encode(self, msg, keybits=1024):
214 emLen = keybits / 8 - 1
215 if len(msg) > (emLen - 10):
216 raise EncoderError("PKCS1-V1.5: message too long")
217
218 ps = self.rnd_non_zero(emLen - len(msg) - 2)
219 assert len(ps) >= 8, "PKCS1-V1.5: invalid PS"
220
221 emout = cStringIO.StringIO()
222 try:
223 emout.write('\x02')
224 emout.write(ps)
225 emout.write('\x00')
226 emout.write(msg)
227 emsg = emout.getvalue()
228 finally:
229 emout.close()
230
231 return emsg
232
233
234 def decode(self, emsg):
235 if len(emsg) < 10:
236 raise DecoderError("PKCS1-V1.5: decoding error")
237
238 if '\x02' != emsg[0]:
239 raise DecoderError("PKCS1-V1.5: decoding error")
240
241 index = emsg.find('\x00')
242 if - 1 == index:
243 raise DecoderError("PKCS1-V1.5: decoding error")
244
245 ps = emsg[1:index]
246 if len(ps) < 8:
247 raise DecoderError("PKCS1-V1.5: decoding error")
248
249 return emsg[index + 1:]
250
251
252 @staticmethod
253 def rnd_non_zero(length):
254 rnd = os.urandom(length)
255 while - 1 != rnd.find('\x00'):
256 rnd = rnd.replace('\x00', os.urandom(1))
257 return rnd
Silverlight does not provide any RSA cryptography classes in the its version of the .NET framework. Fortunately, the
Scrypt project exists and provides a nice RSA library for Silverlight (version 3+) and windows phone 7! the Scrypt project is licensed under the Microsoft public license (ms-pl). the RSA.RSACrypto class has an interface that is very similar to that of System.Security.Cryptography.RSACryptoServiceProvider in the full .NET framework.
In my project the Silverlight client would obtain the RSA public key in XML format via a web service call. This would be used to encrypt login credentials. During that process an AES key would be generated for the login session. Any sensitive data would then be encrypted by the AES key shared between the Python server and the Silverlight client. Here is an example of the encryption on the Silverlight side.
1 using System;
2 using System.Text;
3 using RSA;
4
5 // ... other code ...
6
7 // in this example, e is the GetPublicKeyCompletedEventArgs
8 // parameter from an asynchronous web service call
9 var pkey = new RSACrypto();
10 pkey.FromXmlString(e.Result.GetPublicKeyResult);
11
12 StringBuilder output = new StringBuilder();
13 // fill output with some data...
14 Byte[] raw_data = Encoding.UTF8.GetBytes(output.ToString());
15 var cipher = pkey.Encrypt(raw_data);
16 var encodedCipher = Convert.ToBase64String(cipher);
17
18 // now encodedCipher is ready to be transported
19 // to the Python server.
Remember, with RSA public key cryptography you can only encrypt up to keysize / 8 – 1 bytes of data. Thus if you have a key that is 128 bytes (1024 bits), you can only encrypt up to 127 bytes (1016 bits) of data (depending on the implementation details, maybe less). The padding schemes used with RSA ensure that the data you are to encrypt is exactly keysize / 8 – 1 bytes in length. Thus if your data is short, like a 32 byte AES key (16 byte key, 16 byte iv for 256 bits), the padding scheme will pad out the data to keysize / 8 – 1 bytes before the data is encrypted.
This was a fun project to work on. I learned a great deal and had a blast. I appreciate all of the hard work that others have put in to make the excellent encryption libraries that exist and are freely available. Maybe some of you will find my padding encoders useful, Cheers!
