How to debug hermes internal buffer error in a react-native app

[roni-castro-shipt]

I've got this crash on iOS when opening the app, and that just happens in an app generated in CI, never locally. It uses hermes with react-native 0.75.5, and we have already tried clearing all possible cache in CI and checking the versions.

RCTFatalException: Compiling JS failed: Buffer is smaller than the size stated in the file header. Expected at least 18950956 bytes but got 18950955 bytes: Compiling JS failed: Buffer is smaller than the size stated in the file header. Expected at least 18950956 bytes but got 18950955 bytes

0  CoreFoundation +0x2d5f8    ___exceptionPreprocess
1  libobjc.A.dylib +0x31240   _objc_exception_throw
2  MyApp +0x36b86c     RCTFatal (RCTAssert.m:147:7)
3  MyApp +0x3869ac     __28-[RCTCxxBridge handleError:]_block_invoke (RCTCxxBridge.mm:1130:5)
4  libdispatch.dylib +0x2244  __dispatch_call_block_and_release
5  libdispatch.dylib +0x3fa4  __dispatch_client_callout
6  libdispatch.dylib +0x12a30 __dispatch_main_queue_drain
7  libdispatch.dylib +0x12648 __dispatch_main_queue_callback_4CF
8  CoreFoundation +0x79bc8    ___CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__
9  CoreFoundation +0x761bc    ___CFRunLoopRun
10 CoreFoundation +0xc8280    _CFRunLoopRunSpecific
11 GraphicsServices +0x14bc   _GSEventRunModal
12 UIKitCore +0x3ee670        -[UIApplication _run]
13 UIKitCore +0x14e84         _UIApplicationMain
14 MyApp +0x881c       main (main.m:14:12)
15 dyld +0x2fde4              start

I understand that the error message originates from this line, but I’m trying to figure out the root cause of the buffering issue and how to debug it, considering this code is part of Hermes and isn’t visible in Xcode, and this issue was never reported in react-native/hermes.

We've identified that applying the change below in a specific project file resolves the crash, but it doesn't make sense to me why reordering something would fix a hermes compilation buffer issue.

To investigate further, I decompiled the main.jsbundle using hbc-decompiler from both the crashing app version and one where the fix has been applied. I then compared the changes, and those are the only modifications that seem to resolve the crash.

➜  Downloads git diff --no-index --color 212.js 213.js | tee diff.txt
diff --git a/212.js b/213.js
index 65d337c..1579855 100644
--- a/212.js
+++ b/213.js
@@ -413951,7 +413951,7 @@ case 295:
         r2['ID_INVALID_ERROR_MODAL'] = r1;
         r1 = {'TITLE': 'Enter ID info', 'NAME_LABEL': 'First and last name', 'NAME_PLACEHOLDER': 'E.g. John Doe', 'DATE_OF_BIRTH_LABEL': 'Date of birth', 'DATE_PLACEHOLDER': 'MM/DD/YYYY', 'ID_NUMBER_LABEL': 'ID number', 'ID_NUMBER_PLACEHOLDER': 'Enter ID number', 'EXPIRATION_DATE_LABEL': 'Expiration date', 'NEXT': 'Next', 'FULL_NAME_REQUIRED': 'Enter first and last name', 'INVALID_DATE': 'Enter date as MM/DD/YYYY', 'INVALID_ID_NUMBER': 'Enter a valid ID number - no special characters'};
         r2['ID_MANUAL_CHECK_MODAL'] = r1;
-        r1 = {'SUGGEST_SUBSTITUTE': 'Suggest a substitute', 'CHOOSE_SUBSTITUTE': 'Choose a substitute', 'DO_NOT_SUB': 'Do not substitute', 'CHOOSE_FOR_MEMBER': 'Choose for member', 'CONTACT_MEMBER': 'Contact member', 'CONTACT_MEMBER_NAME': 'Contact ${0}', 'CONTACT_TYPE': '${0} member', 'CONTACT_SUBTITLE': null, 'CHOOSE_SUB_SUBTITLE': 'trusts your substitution choice.', 'DO_NOT_SUB_SUBTITLE': 'requested that you do not substitute.', 'CALL': 'Call', 'MESSAGE': 'Message'};
+        r1 = {'SUGGEST_SUBSTITUTE': 'Suggest a substitute', 'DO_NOT_SUB': 'Do not substitute', 'CHOOSE_SUBSTITUTE': 'Choose a substitute', 'CHOOSE_FOR_MEMBER': 'Choose for member', 'CONTACT_MEMBER': 'Contact member', 'CONTACT_MEMBER_NAME': 'Contact ${0}', 'CONTACT_TYPE': '${0} member', 'CONTACT_SUBTITLE': null, 'CHOOSE_SUB_SUBTITLE': 'trusts your substitution choice.', 'DO_NOT_SUB_SUBTITLE': 'requested that you do not substitute.', 'CALL': 'Call', 'MESSAGE': 'Message'};
         r4 = {'ORIGINAL': 'would like to chat with you if the item is not available', 'EXPERIMENT': 'wants to chat if their item is not available.'};
         r1['CONTACT_SUBTITLE'] = r4;
         r2['NOT_FOUND'] = r1;

The diff above formatted to make it easier to notice the difference.

[tmikov]

What happens when you pass the "bad" bundle to the hermes CLI?

hermes bad.jsbundle

[roni-castro-shipt]

I get some errors for both the bad and working main.jsbundle. The challenge is that it's in bytecode, making it hard to fully comprehend all the errors.

main.jsbundle:1:1: error: Invalid UTF-8 continuation byte
���`
main.jsbundle:1:1: error: unrecognized Unicode character \ufffd
���`
main.jsbundle:1:2: error: unrecognized Unicode character \u1f
���`
main.jsbundle:1:3: error: Invalid UTF-8 lead byte 0xbc
���`
main.jsbundle:1:3: error: unrecognized Unicode character \ufffd
���`
main.jsbundle:1:4: error: unrecognized Unicode character \u3
���`
main.jsbundle:1:5: error: Invalid UTF-8 continuation byte
���`
main.jsbundle:1:5: error: unrecognized Unicode character \ufffd
���`
main.jsbundle:1:6: error: unrecognized Unicode character \u3
���`
main.jsbundle:1:7: error: unrecognized Unicode character \u19
���`
main.jsbundle:1:8: error: unrecognized Unicode character \u1f
���`
main.jsbundle:1:14: error: Invalid UTF-8 continuation byte
���`
main.jsbundle:1:15: error: Invalid UTF-8 continuation byte
���`
main.jsbundle:1:16: error: Invalid UTF-8 lead byte 0xfc
���`
main.jsbundle:1:18: error: Invalid UTF-8 lead byte 0xaa
���`
main.jsbundle:1:19: error: Invalid UTF-8 lead byte 0x8e
���`
main.jsbundle:1:20: error: Invalid UTF-8 lead byte 0xf8
���`
main.jsbundle:1:21: error: Invalid UTF-8 continuation byte
���`
main.jsbundle:1:25: error: Invalid UTF-8 lead byte 0xfc
���`
main.jsbundle:1:27: error: Invalid UTF-8 continuation byte
���`
<unknown>:0: error: too many errors emitted

[tmikov]

Ah, you need to pass -b, otherwise it doesn't know it is bytecode (bytecode is expected to have the extension .hbc).

hermes -b bad.jsbundle

[roni-castro-shipt]

Got it. I tried with -b, and I got this for both

Uncaught Invariant Violation: __fbBatchedBridgeConfig is not set, cannot invoke native modules
    at invariant (address at main.jsbundle:1:131950)
    at anonymous (address at main.jsbundle:1:139654)
    at loadModuleImplementation (address at main.jsbundle:1:102746)
    at guardedLoadModule (address at main.jsbundle:1:102287)
    at metroRequire (address at main.jsbundle:1:101909)
    at requireModule (address at main.jsbundle:1:139168)
    at get (address at main.jsbundle:1:139331)
    at anonymous (address at main.jsbundle:1:228439)
    at loadModuleImplementation (address at main.jsbundle:1:102746)
    at guardedLoadModule (address at main.jsbundle:1:102287)
    at metroRequire (address at main.jsbundle:1:101909)
    at anonymous (address at main.jsbundle:1:227403)
    at loadModuleImplementation (address at main.jsbundle:1:102746)
    at guardedLoadModule (address at main.jsbundle:1:102287)
    at metroRequire (address at main.jsbundle:1:101909)
    at anonymous (address at main.jsbundle:1:224718)
    at loadModuleImplementation (address at main.jsbundle:1:102746)
    at guardedLoadModule (address at main.jsbundle:1:102287)
    at metroRequire (address at main.jsbundle:1:101909)
    at anonymous (address at main.jsbundle:1:224408)
    at loadModuleImplementation (address at main.jsbundle:1:102746)
    at guardedLoadModule (address at main.jsbundle:1:102287)
    at metroRequire (address at main.jsbundle:1:101909)
    at anonymous (address at main.jsbundle:1:221151)
    at loadModuleImplementation (address at main.jsbundle:1:102746)
    at guardedLoadModule (address at main.jsbundle:1:102244)
    at metroRequire (address at main.jsbundle:1:101909)
    at global (address at main.jsbundle:1:101450)

This comes from here

[tmikov]

This confirms that the bundle itself is correct. It is probably 18950956 bytes, as expected.

This points conclusively to a bug in React Native when loading the bundle. 0.75 is pretty old, so this doesn't surprise me. This is most likely caused by RN incorrectly stripping the last byte of the "bad" bundle, because it happens to be zero.

You can verify this by inspecting the last byte:

xxd -p -s -1 -l 1 main.jsbundle

An easy workaround is to append something non zero to the end:

echo -n " " >> main.jsbundle

[roni-castro-shipt]

That's very helpful! That's is happening when I run xxd -p -s -1 -l 1 main.jsbundle
For bad bundle

00

Good bundle

86

I'll try merging that code with the RN 0.76 update we have to get your fix, and see if that solves it.

[roni-castro-shipt]

React Native 0.76 fixed the problem we were facing. Thank you for all the support!

[neildhar]

I recall there was a bug in RN where the bytecode bundle could get shortened by a byte if the last byte is 0. This was fixed in RN 0.76 by facebook/react-native@bb3c51d

[roni-castro-shipt]

Thanks for sharing that.
I can see that commit in 0.76+