Skip to content

Commit e380eee

Browse files
authored
feat: [NET-1445] quantum resistant key exchange using ML-KEM (#3060)
## Summary This PR introduces the ability to use (and to require the use of) `ML-KEM` for key exchange instead of `RSA`. The former is considered quantum resistant, while the latter is not. This PR does not change the defaults, but introduces a new config option to enable the use of the new quantum resistant algorithm. Implementing the new algorithm itself was straight forward, but certain refactoring was necessary to use it. Most of the line changes in this PR are actually related to that refactoring instead of the core feature itself. **Backwards compatibility** The change is backwards compatible with the default settings, ie. the change is not breaking as such. However, the new feature can obviously only be used if both the publisher and subscriber have the new version. Backwards compatibility with the previous version was tested manually, and the results are as expected - ie. key exchange works with default (non-quantum-secure) settings, but fails if `requireQuantumResistantKeyExchange` is `true`, as the other side running the old version doesn't support the new algorithm. ## Changes - Implement encryption and decryption using `ML-KEM` in `EncryptionUtil` - Refactor interfaces in `EncryptionUtil` to be unified across different algorithms - Add a new `StreamrClient` config option under the `encryption` block: `requireQuantumResistantKeyExchange: <boolean>` (default: false) - Update `SubscriberKeyExchange` and `PublisherKeyExchange` to read that config option and act accordingly - Add to `GroupKeyRequest` and `GroupKeyResponse` protobuf messages an `encryptionType` field, the value of which is an `AsymmetricEncryptionType` enum. For backwards compatibility, the absence of this field is interpreted to imply RSA. - Refactor away the technical debt of `OldGroupKeyRequest` and `OldGroupKeyResponse` vs. `NewGroupKeyRequest` and `NewGroupKeyResponse` mess and get rid of `GroupKeyRequestTranslator`. Now, the "new" group key classes (generated from protobuf) are used everywhere, and the old hand-rolled ones have been deleted. - Add unit test cases for `EncryptionUtil` and integration test cases for `SubscriberKeyExchange` and `PublisherKeyExchange` - Add a section to the end-to-end encryption page in the docs, describing the new feature - Add a `--quantum` flag to the CLI tool to flip on the new switch ## Limitations and future improvements - Refactoring away the `OldStreamMessage` / `NewStreamMessage` / `StreamMessageTranslator` debt is out of scope, the old vs. new classes issue was only cleaned up in relation to the group key classes, as only they were relevant to the feature. - In the future, encryption preferences could come from stream metadata by default, instead of explicit local config ## Checklist before requesting a review - [x] Is this a breaking change? If it is, be clear in summary. **-> (No)** - [x] Read through code myself one more time. - [x] Make sure any and all `TODO` comments left behind are meant to be left in. - [x] Has reasonable passing test coverage? - [x] Updated changelog if applicable. - [x] Updated documentation if applicable.
1 parent be2f310 commit e380eee

42 files changed

Lines changed: 748 additions & 534 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ Changes before Tatum release are not documented in this file.
1313
#### Added
1414

1515
- Add new storage node address `STREAMR_STORAGE_NODE_ADDRESS` (https://github.com/streamr-dev/network/pull/3020)
16+
- Added support for quantum secure key exchange using ML-KEM (https://github.com/streamr-dev/network/pull/3060)
1617

1718
#### Changed
1819

docs/docs/streamr-network/security/end-to-end-encryption.md

Lines changed: 28 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,12 +5,38 @@ sidebar_position: 3
55
# End-to-end encryption
66
Confidentiality of events published on a stream can be guaranteed with end-to-end encryption. The publisher generates a AES-256 symmetric encryption key and encrypts the messages before publishing them to the network. As the publisher fully controls who can access their data, they are also responsible for communicating the key to subscribers - usually via the key exchange mechanism described below.
77

8+
The following algorithms are currently available:
9+
- Message encryption: AES-256
10+
- Key exchange: RSA (default), ML-KEM (experimental)
11+
812
## Key exchange
913
The subscribers need the symmetric group key in order to decrypt the data. They automatically obtain this key by performing a key exchange with the publisher. The key exchange happens using asymmetric encryption:
1014

11-
- Both the publisher and subscriber generate a temporary RSA key pair to be used for the key exchange
15+
- Both the publisher and subscriber generate a temporary asymmetric key pair (RSA or ML-KEM, depending on configuration) to be used for the key exchange
1216
- The subscriber sends a key request to the publisher, containing the subscriber's public key, signed with the subscriber's Ethereum key
13-
- The publisher checks from the on-chain access control registry whether that subscriber should be able to access the stream, and if it does, the publisher responds with the AES symmetric key, encrypted with the publisher's RSA key for the subscriber's RSA key, and signs with the publisher's Ethereum key.
17+
- The publisher checks from the on-chain access control registry whether that subscriber should be able to access the stream, and if it does, the publisher responds by encrypting the AES symmetric key required to unlock the data. The key is encrypted with the publisher's temporary private key for the subscriber's temporary public key, and signed with the publisher's Ethereum key.
18+
19+
## Quantum security
20+
As an experimental feature, Streamr Network allows quantum resistant cryptographic algorithms to be used instead of traditional ones where applicable. Here's an overview of supported algorithms with commentary from the quantum security point of view:
21+
- Data encryption: AES-256 (quantum resistant, used by default)
22+
- Key exchange: ML-KEM-1024 (quantum resistant, available via config option), RSA (not quantum resistant, currently used by default)
23+
- Signatures: ECDSA with secp256k1 curve (not quantum resistant, used by default). Quantum resistant alternatives coming soon
24+
25+
## Quantum resistant key exchange
26+
The ML-KEM-1024 based key exchange works as follows. As ML-KEM can only be used to generate a shared secret between the publisher and subscriber, by itself it's not sufficient to allow an arbitrary key to be transferred from the publisher to a subscriber. Therefore, the ML-KEM shared secret is used to derive an AES-256 'wrapper' key using HKDF. The wrapper key is obtained by both the publisher and subscriber by repeating the same key derivation starting with the shared secret. The wrapper key is used to encrypt and decrypt the actual data encryption key, which is the key being exchanged. All algorithms involved in this procedure are considered quantum resistant, therefore making the entirety of the key exchange quantum resistant.
27+
28+
To start using the ML-KEM based key exchange, pass the following configuration to `StreamrClient` on *subscribers*:
29+
30+
```
31+
const streamr = new StreamrClient({
32+
encryption: {
33+
requireQuantumResistantKeyExchange: true
34+
},
35+
// ...
36+
})
37+
```
38+
39+
Publishers will automatically respond to key requests based on what algorithm the subscriber requests, so configuring publishers with the above is not necessary. However, if you do set the above config on publishers, they will *only* respond to key requests using ML-KEM, and will ignore requests for RSA. Note that both publishers and subscribers need to have a recent version of the Streamr libraries to use the quantum secure key exchange.
1440

1541
## Publisher liveness
1642
To perform the key exchange with the subscribers, the publisher must be online and present in the Network. As the Streamr Network deals with real-time messages, publishers are often constantly online. However, it may happen that the publisher has disappeared since publishing the data, making those messages inaccessible to subscribers who have yet to receive the key. This is a consequence of the data publisher being in full control of who can access their data on the Network.

package-lock.json

Lines changed: 25 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

packages/cli-tools/src/client.ts

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,12 +5,16 @@ import { getConfig } from './config'
55

66
export const getClientConfig = (commandOptions: Options, overridenOptions: StreamrClientConfig = {}): StreamrClientConfig => {
77
const configFileJson = getConfig(commandOptions.config)?.client
8-
const environmentOptions = { environment: commandOptions.env }
9-
const authenticationOptions = (commandOptions.privateKey !== undefined) ? { auth: { privateKey: commandOptions.privateKey } } : undefined
8+
const environmentOptions: StreamrClientConfig = { environment: commandOptions.env }
9+
const authenticationOptions: StreamrClientConfig =
10+
(commandOptions.privateKey !== undefined) ? { auth: { privateKey: commandOptions.privateKey } } : {}
11+
const encryptionOptions: StreamrClientConfig =
12+
(commandOptions.quantum === true) ? { encryption: { requireQuantumResistantKeyExchange: true } } : {}
1013
return merge(
1114
configFileJson,
1215
environmentOptions,
1316
authenticationOptions,
17+
encryptionOptions,
1418
overridenOptions
1519
)
1620
}

packages/cli-tools/src/command.ts

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ export interface Options {
88
privateKey?: string
99
config?: string
1010
env?: EnvironmentId
11+
quantum?: boolean
1112
}
1213

1314
export const createCommand = (): commander.Command => {
@@ -34,6 +35,7 @@ export const createClientCommand = (
3435
.option('--config <file>', 'read connection and authentication settings from a config file')
3536
.option('--env <environmentId>', `use pre-defined environment (${formEnumArgValueDescription(ENVIRONMENT_IDS, DEFAULT_ENVIRONMENT_ID)})`,
3637
createFnParseEnum('env', ENVIRONMENT_IDS))
38+
.option('--quantum', 'use and require quantum secure algorithms where available')
3739
.action(async (...args: any[]) => {
3840
const commandOptions = args[args.length - 1].opts()
3941
try {

packages/sdk/package.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -87,6 +87,7 @@
8787
"dependencies": {
8888
"@babel/runtime": "^7.26.7",
8989
"@babel/runtime-corejs3": "^7.26.7",
90+
"@noble/post-quantum": "^0.4.0",
9091
"@protobuf-ts/runtime": "^2.8.2",
9192
"@protobuf-ts/runtime-rpc": "^2.8.2",
9293
"@streamr/config": "^5.5.2",

packages/sdk/src/Config.ts

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -355,6 +355,15 @@ export interface StreamrClientConfig {
355355
* requested via the standard Streamr key-exchange.
356356
*/
357357
rsaKeyLength?: number
358+
359+
/**
360+
* If true, ML-KEM-1024 will be used for key exchange instead of RSA.
361+
* If true on subscribers, they will send key requests specifying an ML-KEM public key instead of an RSA one.
362+
* If true on publishers, they will *only* respond to key requests specifying an ML-KEM public key.
363+
* If false or undefined on publishers, they will respond to key requests with either RSA or ML-KEM
364+
* depending on what the subscriber requests.
365+
*/
366+
requireQuantumResistantKeyExchange?: boolean
358367
}
359368

360369
contracts?: {

packages/sdk/src/config.schema.json

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -350,6 +350,10 @@
350350
"rsaKeyLength": {
351351
"type": "number",
352352
"default": 4096
353+
},
354+
"requireQuantumResistantKeyExchange": {
355+
"type": "boolean",
356+
"default": false
353357
}
354358
},
355359
"default": {}

packages/sdk/src/encryption/EncryptionUtil.ts

Lines changed: 127 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,28 +1,146 @@
11
import crypto, { CipherKey } from 'crypto'
2+
import { ml_kem1024 } from '@noble/post-quantum/ml-kem'
3+
import { randomBytes } from '@noble/post-quantum/utils'
24
import { StreamMessageAESEncrypted } from '../protocol/StreamMessage'
35
import { StreamrClientError } from '../StreamrClientError'
46
import { GroupKey } from './GroupKey'
7+
import { AsymmetricEncryptionType } from '@streamr/trackerless-network'
8+
import { binaryToUtf8 } from '@streamr/utils'
9+
import { getSubtle } from '../utils/crossPlatformCrypto'
510

611
export const INITIALIZATION_VECTOR_LENGTH = 16
712

13+
const INFO = Buffer.from('streamr-key-exchange')
14+
const KEM_CIPHER_LENGTH_BYTES = 1568
15+
const KDF_SALT_LENGTH_BYTES = 64
16+
817
// eslint-disable-next-line @typescript-eslint/no-extraneous-class
918
export class EncryptionUtil {
10-
private static validateRSAPublicKey(publicKey: crypto.KeyLike): void | never {
11-
const keyString = typeof publicKey === 'string' ? publicKey : publicKey.toString('utf8')
12-
if (typeof keyString !== 'string' || !keyString.startsWith('-----BEGIN PUBLIC KEY-----')
19+
/**
20+
* Public API for asymmetric encryption, unified interface across the different AsymmetricEncryptionTypes
21+
*/
22+
static async encryptForPublicKey(plaintext: Uint8Array, publicKey: Uint8Array, type: AsymmetricEncryptionType): Promise<Buffer> {
23+
if (type === AsymmetricEncryptionType.ML_KEM) {
24+
return this.encryptWithMLKEMPublicKey(plaintext, publicKey)
25+
}
26+
if (type === AsymmetricEncryptionType.RSA) {
27+
return this.encryptWithRSAPublicKey(plaintext, publicKey)
28+
}
29+
throw new Error(`Unexpected encryption type: ${type}`)
30+
}
31+
32+
static async decryptWithPrivateKey(cipher: Uint8Array, privateKey: Uint8Array, type: AsymmetricEncryptionType): Promise<Buffer> {
33+
if (type === AsymmetricEncryptionType.ML_KEM) {
34+
return this.decryptWithMLKEMPrivateKey(cipher, privateKey)
35+
}
36+
if (type === AsymmetricEncryptionType.RSA) {
37+
return this.decryptWithRSAPrivateKey(cipher, privateKey)
38+
}
39+
throw new Error(`Unexpected encryption type: ${type}`)
40+
}
41+
42+
/**
43+
* RSA
44+
*/
45+
private static toRSAPublicKeyString(publicKey: Uint8Array): string {
46+
// RSA publicKey passed around in string format for legacy reasons
47+
const keyString = binaryToUtf8(publicKey)
48+
if (!keyString.startsWith('-----BEGIN PUBLIC KEY-----')
1349
|| !keyString.endsWith('-----END PUBLIC KEY-----\n')) {
14-
throw new Error('"publicKey" must be a PKCS#8 RSA public key in the PEM format')
50+
throw new Error('"publicKey" must be an RSA public key (SPKI) in PEM format, encoded in UTF-8')
51+
}
52+
return keyString
53+
}
54+
55+
private static toRSAPrivateKeyString(privateKey: Uint8Array): string {
56+
// RSA privateKey passed around in string format for legacy reasons
57+
const keyString = binaryToUtf8(privateKey)
58+
if (!keyString.startsWith('-----BEGIN PRIVATE KEY-----')
59+
|| !keyString.endsWith('-----END PRIVATE KEY-----\n')) {
60+
throw new Error('"privateKey" must be a PKCS#8 RSA private key in PEM format, encoded in UTF-8')
1561
}
62+
return keyString
1663
}
1764

18-
static encryptWithRSAPublicKey(plaintextBuffer: Uint8Array, publicKey: crypto.KeyLike): Buffer {
19-
this.validateRSAPublicKey(publicKey)
20-
const ciphertextBuffer = crypto.publicEncrypt(publicKey, plaintextBuffer)
65+
private static encryptWithRSAPublicKey(plaintextBuffer: Uint8Array, publicKey: Uint8Array): Buffer {
66+
const keyString = this.toRSAPublicKeyString(publicKey)
67+
const ciphertextBuffer = crypto.publicEncrypt(keyString, plaintextBuffer)
2168
return ciphertextBuffer
2269
}
2370

24-
static decryptWithRSAPrivateKey(ciphertext: Uint8Array, privateKey: crypto.KeyLike): Buffer {
25-
return crypto.privateDecrypt(privateKey, ciphertext)
71+
private static decryptWithRSAPrivateKey(ciphertext: Uint8Array, privateKey: Uint8Array): Buffer {
72+
const keyString = this.toRSAPrivateKeyString(privateKey)
73+
return crypto.privateDecrypt(keyString, ciphertext)
74+
}
75+
76+
/**
77+
* ML-KEM
78+
*/
79+
private static async deriveAESWrapperKey(sharedSecret: Uint8Array, kdfSalt: Uint8Array): Promise<Uint8Array> {
80+
const subtle = getSubtle()
81+
const keyMaterial = await subtle.importKey(
82+
'raw',
83+
sharedSecret,
84+
{ name: 'HKDF' },
85+
false,
86+
['deriveKey']
87+
)
88+
89+
const derivedKey = await subtle.deriveKey(
90+
{
91+
name: 'HKDF',
92+
hash: 'SHA-512',
93+
salt: kdfSalt,
94+
info: INFO
95+
},
96+
keyMaterial,
97+
{ name: 'AES-CTR', length: 256 },
98+
true,
99+
['encrypt', 'decrypt']
100+
)
101+
102+
const exportedKey = await subtle.exportKey('raw', derivedKey)
103+
return new Uint8Array(exportedKey)
104+
}
105+
106+
private static async encryptWithMLKEMPublicKey(plaintextBuffer: Uint8Array, publicKey: Uint8Array): Promise<Buffer> {
107+
// Encapsulate to get kemCipher and shared secret
108+
// The recipient will be able to derive sharedSecret using privateKey and kemCipher
109+
const { cipherText: kemCipher, sharedSecret } = ml_kem1024.encapsulate(publicKey)
110+
111+
if (kemCipher.length !== KEM_CIPHER_LENGTH_BYTES) {
112+
throw new Error(`Expected KEM cipher to be ${KEM_CIPHER_LENGTH_BYTES}, but it was ${kemCipher.length} bytes`)
113+
}
114+
115+
// Derive an AES wrapping key from the shared secret using HKDF
116+
// The recipient will be able to repeat this computation to derive the same key
117+
const kdfSalt = randomBytes(KDF_SALT_LENGTH_BYTES)
118+
const wrappingAESKey = await this.deriveAESWrapperKey(sharedSecret, kdfSalt)
119+
120+
// Encrypt plaintext with the AES wrapping key
121+
const aesEncryptedPlaintext = this.encryptWithAES(plaintextBuffer, Buffer.from(wrappingAESKey))
122+
123+
// Concatenate the deliverables into a binary package
124+
return Buffer.concat([kemCipher, kdfSalt, aesEncryptedPlaintext])
125+
}
126+
127+
private static async decryptWithMLKEMPrivateKey(cipherPackage: Uint8Array, privateKey: Uint8Array): Promise<Buffer> {
128+
// Split the cipherPackage, see encryptWithMLKEMPublicKey how it's constructed
129+
let pos = 0
130+
const kemCipher = cipherPackage.slice(0, KEM_CIPHER_LENGTH_BYTES)
131+
pos += KEM_CIPHER_LENGTH_BYTES
132+
const kdfSalt = cipherPackage.slice(pos, pos + KDF_SALT_LENGTH_BYTES)
133+
pos += KDF_SALT_LENGTH_BYTES
134+
const aesEncryptedPlaintext = cipherPackage.slice(pos)
135+
136+
// Derive the shared secret using the private key and kemCipher
137+
const sharedSecret = ml_kem1024.decapsulate(kemCipher, privateKey)
138+
139+
// Derive the wrappingAESKey
140+
const wrappingAESKey = await this.deriveAESWrapperKey(sharedSecret, kdfSalt)
141+
142+
// Decrypt the aesEncryptedPlaintext
143+
return this.decryptWithAES(aesEncryptedPlaintext, Buffer.from(wrappingAESKey))
26144
}
27145

28146
/*

packages/sdk/src/encryption/GroupKey.ts

Lines changed: 5 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
import crypto from 'crypto'
2-
import { EncryptedGroupKey } from '../protocol/EncryptedGroupKey'
2+
import { EncryptedGroupKey } from '@streamr/trackerless-network'
33
import { uuid } from '../utils/uuid'
44
import { EncryptionUtil } from './EncryptionUtil'
55
export class GroupKeyError extends Error {
@@ -70,7 +70,10 @@ export class GroupKey {
7070

7171
/** @internal */
7272
encryptNextGroupKey(nextGroupKey: GroupKey): EncryptedGroupKey {
73-
return new EncryptedGroupKey(nextGroupKey.id, EncryptionUtil.encryptWithAES(nextGroupKey.data, this.data))
73+
return {
74+
id: nextGroupKey.id,
75+
data: EncryptionUtil.encryptWithAES(nextGroupKey.data, this.data)
76+
}
7477
}
7578

7679
/** @internal */
@@ -81,11 +84,4 @@ export class GroupKey {
8184
)
8285
}
8386

84-
/** @internal */
85-
static decryptRSAEncrypted(encryptedKey: EncryptedGroupKey, rsaPrivateKey: string): GroupKey {
86-
return new GroupKey(
87-
encryptedKey.id,
88-
EncryptionUtil.decryptWithRSAPrivateKey(encryptedKey.data, rsaPrivateKey)
89-
)
90-
}
9187
}

0 commit comments

Comments
 (0)