POST to /new containing CRLF leads to unusable document

[azure9]

when I follow the instruction in #673
curl -XPOST -H 'Content-Type: text/markdown' --data-binary "@public/docs/features.md" http://localhost:3000/new

It's done without error.
But when I try to edit this created document, it can't be edited, on server side it shows

app_1       | 2018-05-15T13:29:59.279Z - error:  Error: The operation's base length must be equal to the string's length.
app_1       |     at TextOperation.ot.TextOperation.TextOperation.apply (/hackmd/lib/ot/text-operation.js:184:13)
app_1       |     at WrappedOperation.ot.WrappedOperation.WrappedOperation.apply (/hackmd/lib/ot/wrapped-operation.js:16:31)
app_1       |     at EditorSocketIOServer.ot.Server.Server.receiveOperation (/hackmd/lib/ot/server.js:33:33)
app_1       |     at EditorSocketIOServer.onOperation (/hackmd/lib/ot/editor-socketio-server.js:107:33)
app_1       |     at /hackmd/lib/ot/editor-socketio-server.js:51:22
app_1       |     at EditorSocketIOServer.ifMayEdit [as mayWrite] (/hackmd/lib/realtime.js:659:10)
app_1       |     at Socket.<anonymous> (/hackmd/lib/ot/editor-socketio-server.js:45:14)
app_1       |     at emitThree (events.js:136:13)
app_1       |     at Socket.emit (events.js:217:7)
app_1       |     at /hackmd/node_modules/socket.io/lib/socket.js:513:12
app_1       |     at _combinedTickCallback (internal/process/next_tick.js:131:7)
app_1       |     at process._tickCallback (internal/process/next_tick.js:180:9)

[SISheogorath]

Can't really reproduce over here. Does it happen every time you do it or only once? (means it's gone when you reload or recreate the document)

[azure9]

It happens every time when I use post /new api with --data-binary option, without the --data-binary option, the created document is editable, when I add this option, the problem shows. I use the image on docker-hub, test with latest docker image, the problem still exist.

[SISheogorath]

I just tried it with the docker image myself and still can't reproduce it.

What browser are you using? What database backend are you using?

[ccoenen]

you stated that it works without the --data-binary - is there any particular reason why you need to use this flag?

[azure9]

I use chrome latest version and postgres as backend server, ubuntu 16.04 on server side, and config a nginx with websocket proxy to forward the requests to docker. I use docker image from https://github.com/hackmdio/docker-hackmd/blob/master/README.md and only changed the version to 2 to let ubuntu docker-compose work properly, settings are all default.
Without the —data-binary option, the markdown content will be in one line, no \r or \n will be in the document, and the document is messy.
Umm by the way there’s utf8 character in markdown document.

[azure9]

Ah, I locate the root cause, when there's \r\n in the post content, the error shows up.

[ccoenen]

I can't replicate the error. I created two files (one with LF, one with CRLF) and posted them like this:

curl -XPOST -H "Content-Type: text/markdown" --data-binary "@ipsum-utf8-lf.txt" "https://hackmd-ce.herokuapp.com/new"
Found. Redirecting to https://hackmd-ce.herokuapp.com/dadF-s0aQSGYKNgEqetT4g

curl -XPOST -H "Content-Type: text/markdown" --data-binary "@ipsum-utf8-crlf.txt" "https://hackmd-ce.herokuapp.com/new"
Found. Redirecting to https://hackmd-ce.herokuapp.com/G6kWT9iHRRqtFw0CZp1OYA

Both ended up fine, they are indistinguishable.

[azure9]

You just reproduce the bug, the bug is in https://hackmd-ce.herokuapp.com/G6kWT9iHRRqtFw0CZp1OYA

When you trying to modify the content, the error shows up.

[ccoenen]

You're right! Thanks for getting back to this. I can edit the first (LF only) but the second behaves weird (CRLF)!