Handle invalid OMEMO bundles better: invalid signature on device key

[chrisballinger]

Somehow it's possible to publish an invalid bundle, but there's not a good way to know that other people are having trouble parsing it. We need a way to:

• Figure out if your own keys are messed up
• Show errors for other people if their keys are messed up
• (automatically?) republish new keys when your bundle is messed up

[afriedmanGlacier]

@chrisballinger, do you know where it fails when this (invalid bundle) happens? Right now, sometimes it fails for us in fetching the bundle for a device id, but other times it fails in getting a chain_key (I'm not sure what that is) in session_cipher.c, which causes SignalSessionCipher.encryptData to return nil and OTROMEMOSignalCoordinator.encryptAndSendMessage to return nil.

[chrisballinger]

Here's an example of a prekey bundle that fails.

<iq xmlns="jabber:client" id="F0D665E8-FE0F-4F28-9388-6F1F448C0E47" type="result" to="alice@example.com/chatsecure63531" from="bob@example.com">
  <pubsub xmlns="http://jabber.org/protocol/pubsub">
    <items node="eu.siacs.conversations.axolotl.bundles:1364390939">
      <item id="1">
        <bundle xmlns="eu.siacs.conversations.axolotl">
          <signedPreKeyPublic signedPreKeyId="1934732210">Bc9TOYv2lM63DQrSeLZUSe1M6JduulYIAnfLF0DZJI5V</signedPreKeyPublic>
          <signedPreKeySignature>eeW/q6ZVdF+uIfEMT/VnEEHmqc9zp9lfS7OST8q+C6gjSF5Vd/5tTPfvwPQsBgEFSj+Vy2+bu4Hmw3mJtSAniw==</signedPreKeySignature>
          <identityKey>Bd/9tSfIPPZiHYXfxYFfvuDNLZiu3F+/ZG7C8iOJoIxE</identityKey>
          <prekeys>
            <preKeyPublic preKeyId="17">BVgol2k9D89w1KX+3La4t45/pZFq+T2jJLmihj+MU+cY</preKeyPublic>
            <preKeyPublic preKeyId="14">BegAUQXMjjRYqAQ2invDfoPAHTmTTzDdaMavruMIfPdZ</preKeyPublic>
            <preKeyPublic preKeyId="30">BVNm2Jld7AFJWf4v+wEoJMoz6T8YJPMklDttxbxbZjxc</preKeyPublic>
            <preKeyPublic preKeyId="77">BXjfnhp3yKcZz2kN2n8eYmnsuQ8tdyxQedcEsP8UB0Ir</preKeyPublic>
            <preKeyPublic preKeyId="68">BRFR0zgNGzQdCGviFhtIHgURIzJInu5NyUeU8gzjMsYx</preKeyPublic>
            <preKeyPublic preKeyId="99">BSkS2RHa22KOhWPLNyxim8J8EULwFoMJkcCkw6fk0qM/</preKeyPublic>
            <preKeyPublic preKeyId="3">BVf0/9NEk+w4uLIt3Af2KVl2meZK5MUdbcsNO211+dRa</preKeyPublic>
            <preKeyPublic preKeyId="32">BQanIevASxDSFbwi87IAQHV3eE33q1NXxzzqj7qeh08d</preKeyPublic>
            <preKeyPublic preKeyId="58">BQPV18k9fkxwiyrVIlkek5UoJQGHxTUL42D/U6BMnP9o</preKeyPublic>
            <preKeyPublic preKeyId="28">BcW5PH9ozMAw3ceX4J2ML1ZToZhuYumwFoD3i6MOQGEW</preKeyPublic>
            <preKeyPublic preKeyId="84">BXpE4jXrbbQ+J4A/o70R1BkfAO/cqgOeCGtKYTyU7YMP</preKeyPublic>
            <preKeyPublic preKeyId="6">BXwsAI6fUixM7Op2mk1vXfDt1MxkrfhrSu8NWgWc8OhM</preKeyPublic>
            <preKeyPublic preKeyId="50">BYXpDs0SQ6qK9+nDP4v8gkK73ftXXYhAmZVlXTl2tL0o</preKeyPublic>
            <preKeyPublic preKeyId="12">BYE3OgiOMbO1hISoMpTeWevqRiKKaPjNGpsMT0OIDAEI</preKeyPublic>
            <preKeyPublic preKeyId="23">BXhYwuh1lINDsvMbM3fli5N7u7gZtrdDxX5EDv9UDtcd</preKeyPublic>
            <preKeyPublic preKeyId="25">BRH1cZn8bQ0Ts50Bs2sv7Baudt0G9a2Vsq1TnZQiO6Eo</preKeyPublic>
            <preKeyPublic preKeyId="90">BXlPH29mvyGgXxVu3RPoAUHI816vEfYswF/9v17mP2FN</preKeyPublic>
            <preKeyPublic preKeyId="5">BTHMg4Ez8YKR882Ov7jq6OXLgsrvxh6JNk1/64hrMu1X</preKeyPublic>
            <preKeyPublic preKeyId="78">BbclBwL7gVgrrTO4ZwYEsydsoMjIIrnOxk5mZOSn839Y</preKeyPublic>
            <preKeyPublic preKeyId="45">BUlt2QKpAfp8LbnHRKMNieCWOGUUg+u7ISASW7CoLn0a</preKeyPublic>
            <preKeyPublic preKeyId="63">BVqgeffrJJpaJ4vnV8PzXPRuX32EkpR5aQZZX6yLhUEc</preKeyPublic>
            <preKeyPublic preKeyId="11">BcMgvAiaFf1bKJiQi/l9R32ljUlVLuZt2+MM4R99XoIt</preKeyPublic>
            <preKeyPublic preKeyId="37">Bf49TxkGy6PYu7TZ5l5H2ofJ4dqxjXsj4bSFZ1WbCvEe</preKeyPublic>
            <preKeyPublic preKeyId="2">BS/T8qMafufISTq+79Yk3n0m4T7Eux6hkuc2NT2A+s1/</preKeyPublic>
            <preKeyPublic preKeyId="98">Bdbk28yWm9YXpxFD7gPn4wlsMu4OFr6kR68jXjBzM00i</preKeyPublic>
            <preKeyPublic preKeyId="27">BV1n/AAfxZpY/plZXR3ATJHDfryvMeaB4UucdVhlJkFr</preKeyPublic>
            <preKeyPublic preKeyId="71">BbL8GqqlDS8F4RsXNvcvlDfw1j/3knvhh8/ysO75qboh</preKeyPublic>
            <preKeyPublic preKeyId="87">BR0NlJd2MosLrcZ09TbPoF1bKqENq2P0qGgoWxKq638h</preKeyPublic>
            <preKeyPublic preKeyId="76">BfZ4TPSHEaRn4aZMdQnR/Nim7yEg5lkDXvZzkXqq29Ne</preKeyPublic>
            <preKeyPublic preKeyId="21">BZSzrssHvJtuLh3mxQzSQey7J0JTUi1Hd/mHAWA+3JUA</preKeyPublic>
            <preKeyPublic preKeyId="9">BYODpcPqBYamzvuRGjNy1mA1dAn3sB6Q2nqSBGUadKhS</preKeyPublic>
            <preKeyPublic preKeyId="44">BUqk4NVjWvDzZnJvX6Ez6cGIIPGz+9Aczg4E3Wjcank9</preKeyPublic>
            <preKeyPublic preKeyId="35">Bfiq5tU7ogqWJm8k2Y4o9oXGzuoNhhBWav2a/iBD7RQ+</preKeyPublic>
            <preKeyPublic preKeyId="100">BfCsHGjD09XdYpJyxc/vkz1gnwZwPd2S/cSAk0QurnNN</preKeyPublic>
            <preKeyPublic preKeyId="15">BSRBdpbnXJB9WdApemI4HI3ul3dyaeCiFlxyHRygctVA</preKeyPublic>
            <preKeyPublic preKeyId="29">Bfc76xzZ2n86Shm6BaUo7pFQnSSc8y1X4oZg+pG6J2sY</preKeyPublic>
            <preKeyPublic preKeyId="52">BchJlVzcjELfkMYbNNNg7i1MX3ppFAZMAxgkILFPfFt5</preKeyPublic>
            <preKeyPublic preKeyId="85">BawnMh3ie//q08LJIn4kCa4txz64XnrZKowrDT/JQ705</preKeyPublic>
            <preKeyPublic preKeyId="56">BZEho4bj2RxwYd3YSAfIZrCymgrNBgieKBIjF31NC8dM</preKeyPublic>
            <preKeyPublic preKeyId="97">BcEtTra9/VP44/08kcA6aqqJWdnBViMNgL/xQabWK010</preKeyPublic>
            <preKeyPublic preKeyId="89">BUmf1GkvJxZEUVVno+BqSRL85F/+yg23cjk+Z4xeH1Yu</preKeyPublic>
            <preKeyPublic preKeyId="22">BRekEYs6t7Lem2/2+j687D8O/+v7Qyq4fQ5VxEAtCtpJ</preKeyPublic>
            <preKeyPublic preKeyId="18">BSy87LJFK9cqF0dlUOHf403pIRW8DrWWzNE7QW0wMQZv</preKeyPublic>
            <preKeyPublic preKeyId="53">BT6P5TlbpS5y6mFcf3WBSjuFjtgim7jOoe7PKM9V8B5B</preKeyPublic>
            <preKeyPublic preKeyId="62">Bd8/5zqPtbWJ/Ilkr95MHf5119VAqib8zCsBKLa3ApwC</preKeyPublic>
            <preKeyPublic preKeyId="36">BTzBT0zpohuvsWQRSoJhkpwV6LflGrtBycMcTTUVFJhW</preKeyPublic>
            <preKeyPublic preKeyId="57">BbZtjFH86FM8VSQ+pHhDClmMurKajZvLvfRsIM/4h2Bw</preKeyPublic>
            <preKeyPublic preKeyId="69">BerLeJonLrTJIFRCvksuoJ0a+AlRnhoRr18adbwGHFkB</preKeyPublic>
            <preKeyPublic preKeyId="91">BZ67PFCHXGtXOBJhk+lOfRyuXjES5VLaj2FK5tegpKRb</preKeyPublic>
            <preKeyPublic preKeyId="41">BahE5MLGkzOpsjcpdAmO+Dh56B2cQ3p5TD2Rwsxpq4Zu</preKeyPublic>
            <preKeyPublic preKeyId="93">BSEK3/fLgKGJFPRjV8KUREYgfMkICOjNgTxgKZLwvX0v</preKeyPublic>
            <preKeyPublic preKeyId="20">Bd7rg84qFNn+5hEFP5/koBZNp17Im29Dfi5/jwhBMzwA</preKeyPublic>
            <preKeyPublic preKeyId="49">BTgU7LKPaNX4IqrXK9rcqtFwABeZAV23cu5pGl2lqswX</preKeyPublic>
            <preKeyPublic preKeyId="38">BUXT1/8F8mCRZ8cyq0zPk2O8tDwoGpC7+rc0W4YPxZ1k</preKeyPublic>
            <preKeyPublic preKeyId="74">BbJaQEAXeZdcwswZunsgCfYOxdf6x9UcCxKoD1p4wydT</preKeyPublic>
            <preKeyPublic preKeyId="42">BUwfZpUjB0eLfQ64+y+PedwJkL2I4dNWk2V5db2+CZl2</preKeyPublic>
            <preKeyPublic preKeyId="47">BZ+f7iKAS+4rBeJp2fM/KLVYm+H2CLPRcRGno9LAivZ2</preKeyPublic>
            <preKeyPublic preKeyId="59">BWZBuyCD15Rju/edPcWJWEygkLb12mlFL9n2NrO7mMRq</preKeyPublic>
            <preKeyPublic preKeyId="39">Bdc6BrWhmk34dxXF2v+2ekePxX6/AlcYyBq4qUNsJQYA</preKeyPublic>
            <preKeyPublic preKeyId="34">BdlTotdWpTektj86mVwEXG6JVQ4D7ZhKHSEuiKfDDbxX</preKeyPublic>
            <preKeyPublic preKeyId="46">BaGsnlr+IYxxfEmRF+BlW+2K8nD08PPfINTsr12ObItX</preKeyPublic>
            <preKeyPublic preKeyId="67">BXGdBaakCYbaCpCD6dHhqRC6wioaXxPpKV9ioOnhAaVN</preKeyPublic>
            <preKeyPublic preKeyId="95">Bfv1h3m+ytFxTw4YhqBrIKHkUdLUw7YeIsoHEd1YGGFg</preKeyPublic>
            <preKeyPublic preKeyId="81">Bdp82Fbaz2437FJt71hthxWSCWMBIw/8VnRufjYc8Q53</preKeyPublic>
            <preKeyPublic preKeyId="96">BX1zZ1cHGi1OQb+eGjliSxZmbPQ5E6ZglBs8pc4oV6Y4</preKeyPublic>
            <preKeyPublic preKeyId="7">BQiIT+E/28/rO11Ybxr+NQ8rmSVa9GYWGXpCdP8vHbEV</preKeyPublic>
            <preKeyPublic preKeyId="43">BXZeTX/osvhnESiAcy7R7/Q3Q6ELEgE9Nmtm0l3VwUE8</preKeyPublic>
            <preKeyPublic preKeyId="16">BTmr2dRoHuuNCyfXA72+W3ypDpZwSr2zTaqyJfMamCZD</preKeyPublic>
            <preKeyPublic preKeyId="51">BRqGezS5f/Z7DUU5iDWM3ibtZNRLfY/i31BPl4oQRVMZ</preKeyPublic>
            <preKeyPublic preKeyId="64">BV+jWFzmBa8DDYsZe4RC/tZ4VxZC6/sID1AeExow+MRm</preKeyPublic>
            <preKeyPublic preKeyId="31">BfNyXR8CfkLYJ8zox1+nxE5Pato6jzqXENBrMG0l9w4R</preKeyPublic>
            <preKeyPublic preKeyId="26">BZ1iht/gCzd6AAf3vqdtTjNw/kBUytcU1glfaM+Ix59j</preKeyPublic>
            <preKeyPublic preKeyId="61">BdCa0lBFNvdcSIYEae67wN8+pCE8invYBp0NX9sgaoB2</preKeyPublic>
            <preKeyPublic preKeyId="66">BQ/C5XLGhgC91fA6fOGdT5QSKNtUfsvag6dTkk16hQAW</preKeyPublic>
            <preKeyPublic preKeyId="73">BR2SNNob68yM8/gX8aewG4cMlOWeZvQS7YVcB4t9irkB</preKeyPublic>
            <preKeyPublic preKeyId="8">BUDy6UH9Ax9Wb0Hq2EQAJpo8Vm2X4i0MR5qWuP5xwShR</preKeyPublic>
            <preKeyPublic preKeyId="79">BRm00b60h2APaYeK/tw5+9RFsPT/jnbZ59F9NAuXpW5j</preKeyPublic>
            <preKeyPublic preKeyId="65">BeEni84T85BElZRa1nPrQ7FYBZpLPCZLgtwQhP3r0VZP</preKeyPublic>
            <preKeyPublic preKeyId="40">BRc9cWnxXHZbJ1sRfDMa/zePC2NT0jzPCYpZovL3XP45</preKeyPublic>
            <preKeyPublic preKeyId="24">BdGt9xlKNgHzxNDOL5Y4nFnAoPr621HuZ2MTZp+NdZoG</preKeyPublic>
            <preKeyPublic preKeyId="83">BS0jUl1QQhzXOoJn56uFMGL0qYTT5Oh60g0vs/oSAvca</preKeyPublic>
            <preKeyPublic preKeyId="82">BfPH6haNAvjeWOHJocog8Xx+i5VEu1LJrij/xD13yA5c</preKeyPublic>
            <preKeyPublic preKeyId="13">BeL6YbmwyiAs0lv06dB0hG29+k4Ittmhb4OOi7TeU9Rb</preKeyPublic>
            <preKeyPublic preKeyId="72">BctSvnzXSXF4VZOXwrW9bB6Q5J0/u4vN7YWtoMAQi9JI</preKeyPublic>
            <preKeyPublic preKeyId="70">BZ2tmu4UWTmfYi7x0DjQpYRVmkEwholwocw1YsA+Cqls</preKeyPublic>
            <preKeyPublic preKeyId="19">Be7C8m65ek9RdCyhH9Hg6kfTfL2BbH0bEVo2NfNh9lx1</preKeyPublic>
            <preKeyPublic preKeyId="88">BYEV4b/WIcGy7oguoN0C4aEJlisplIx6FJ35X3ncDi9V</preKeyPublic>
            <preKeyPublic preKeyId="60">BXfGJqB/mp+ZI9faJkfeTtG+3ybFxEZaDUUuEZVYX/Yd</preKeyPublic>
            <preKeyPublic preKeyId="86">BaKqoLh2zXb+AfqW7jfA/1uHIN1UEnaGKXb6oBLjPQln</preKeyPublic>
            <preKeyPublic preKeyId="1">BWPfL0mpnQUAx2Jnpb9r97iAB/UPAv4bucY2FMCFqtFl</preKeyPublic>
            <preKeyPublic preKeyId="80">BfcOuP4ocKKFMuC+sAxFob/+qrBZngTWs1NWyYW2Sc04</preKeyPublic>
            <preKeyPublic preKeyId="10">BZrYSLsCceCc6zNCWfQWF2EvcQSa0Bp2xqxXuf3ikfAZ</preKeyPublic>
            <preKeyPublic preKeyId="75">BWZz/7gasx/yXDII7OUdW113I6DFeFvqf7my0mdyG4ko</preKeyPublic>
            <preKeyPublic preKeyId="55">Bfsy1DLHCZZhQ8/i5vXhFG4ILEq2NdsQWNYJmWSgFWJ5</preKeyPublic>
            <preKeyPublic preKeyId="48">BQhMMpzVL9I/gXDTpe+roEHA+24CS5sxOx7Iua44QwQj</preKeyPublic>
            <preKeyPublic preKeyId="33">Benj4YHC+4d9TTaseqIxNQkh1erwVt25h9L5uyrQ7Gl1</preKeyPublic>
            <preKeyPublic preKeyId="54">BVMO38GN4AJw/Wtnmg6Up0N9g8JKFLqiLJ9sNCILDhMi</preKeyPublic>
            <preKeyPublic preKeyId="94">BWjnEmFwVbW7xPySZ2f+E1WC6cF02+UvoqpsP1MBbjoK</preKeyPublic>
            <preKeyPublic preKeyId="4">BS4dVwGvtNyAezc3AhdE9SLAkj72iJXDcJlNqYqdVq4G</preKeyPublic>
            <preKeyPublic preKeyId="92">BciGwpIU/PxH5xjsgIzRmUf3r33t3yOfG6wtbbZJ1cUI</preKeyPublic>
          </prekeys>
        </bundle>
      </item>
    </items>
  </pubsub>
</iq>

[chrisballinger]

@afriedmanGlacier I think you'll like bfcdb74

@chrisballinger, please push the fix to master and release it. It would be good to wait less time. Conversations can't send a OMEMO message to ChatSecure.

[chrisballinger]

@participante0 I'll probably cherry-pick bfcdb74 and 09b2a92 into master and push a build tomorrow.

[afriedmanGlacier]

@chrisballinger awesome, I'll look at it today, thanks!!!

[afriedmanGlacier]

@chrisballinger Ummmmm, I can't seem to get this to work.

Using our eJabber 16.09 server, I ran this through the xcode debugger and it is definitely doing a much better job exchanging keys and not failing in the same place, but now it is failing on the client receiving end here:

OMEMOModule.didReceiveMessage ->
OTROMEMOSignalCoordinator.processKeyData ->
OTRAccountSignalEncryptionManager.decryptFromAddress ->
SignalSessionCipher.decryptCiphertext returns nil ->
session_cipher.c session_cipher_decrypt_pre_key_signal_message, result = -1005,
session_cipher.c session_cipher_decrypt_from_record_and_signal_message, result = -1005, (cannot find a valid session)

Just in case something funky was happening on the server, I initialized accounts on talker.to (Prosody) and had the same result. After quite a few tests, at least I am always seeing the same result. Keys are successfully exchanged, and message received but not processed.

Are you not having this issue?

In case it is relevant, here's what I did. I started from scratch with a git clone on master and then did git checkout on the commit you referenced (bfcdb74). I figured out that I also needed to install Carthage and do a carthage update. Is it possible that something isn't playing well from doing the build from scratch from Master and then checking out the commit?

[chrisballinger]

What you're seeing isn't necessarily a problem if the message was encrypted to another key. OMEMO messages can contain many parallel sessions (one for each device, or even more in a group chat). Can the session be initiated from one of the sides? I'm guessing you're seeing messages sent to a stale key?

…

On Mon, Apr 17, 2017 at 7:15 AM, afriedmanGlacier ***@***.***> wrote: @chrisballinger <https://github.com/chrisballinger> Ummmmm, I can't seem to get this to work. Using our eJabber 16.09 server, I ran this through the xcode debugger and it is definitely doing a much better job exchanging keys and not failing in the same place, but now it is failing on the client receiving end here: OMEMOModule.didReceiveMessage -> OTROMEMOSignalCoordinator.processKeyData -> OTRAccountSignalEncryptionManager.decryptFromAddress -> SignalSessionCipher.decryptCiphertext returns nil -> session_cipher.c session_cipher_decrypt_pre_key_signal_message, result = -1005, session_cipher.c session_cipher_decrypt_from_record_and_signal_message, result = -1005, (cannot find a valid session) Just in case something funky was happening on the server, I initialized accounts on talker.to (Prosody) and had the same result. Are you not having this issue? In case it is relevant, here's what I did. I started from scratch with a git clone on master and then did git checkout on the commit you referenced ( bfcdb74 <bfcdb74>). I figured out that I also needed to install Carthage and do a carthage update. Is it possible that something isn't playing well from doing the build from scratch from Master and then checking out the commit? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#725 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAfqH_la2ofkdqDwnSDm8_pp-4zFKDrpks5rw3QFgaJpZM4Mo5xq> .

[afriedmanGlacier]

In many tests today, I haven't successfully received one OMEMO message. Would that happen repetitively like that? I created new users multiple times just to try to rule out various possible issues. I am using the same two iOS 7 devices for each test. I've tried initiating from both sides multiple times

[afriedmanGlacier]

Oh, and as far as stale keys, each time I have looked at the Profiles for each user/device (not every test, but several times), they have shown one OMEMO key for themselves and one for the other user. And the keys appear to match. When you refer to "messages sent to a stale key," a stale key would show up in the profile, right?