No JavaScript Benchmarks?

[ohenepee]

Would love to see how the built-in JSON and Protobuf fare against colfer. Its kinda unfair to see C, Go, and Java make it to the benchmark games and JS stays at home. Could it be that the JS version couldn't live up to the Colfer performance promise?

[pascaldekloe]

JavaScript was added to give browsers access to backed data so that no conversion would be needed. NodeJS changes all that and I know little about it's performance characteristics. Don't expect peak results yet it should be easy to make it fast.

So how does one make JavaScript benchmarks work? Would you care to help?

[lewisdiamond]

I just tried running it, I could provide the code but I can't marshal + unmarshal and get the same result. It's also more than 2x slower than JSON in my benchmark.

[pascaldekloe]

Please share @lewisdiamond. I'll be happy to make it fast. No problem. You can make a pull request or mail to pascal@quies.net.

[pascaldekloe]

I have a few days off so the sooner the better @lewisdiamond. No need for a nice delivery. I just need a little help with the benchmark setup.

[lewisdiamond]

@pascaldekloe Well the problem I have right now is that the JS code generated by colf either doesn't work or I don't use it properly. Otherwise you just need the benchmark npm package and use it like this.

const Benchmark = require('benchmark');
const suite = new Benchmark.Suite;
const data = { //Data goes here }
const colfer = new b.bench.Bench(data) // Is this how it's supposed to be used?
const marshaled = colfer.marshal()
const stringified = JSON.stringify(data)

suite.add('Colfer > marshal', function() {
    colfer.marshal()
})
    .add('JSON > stringify', function() {
        JSON.stringify(data)
    })
    .add('Colfer > unmarshal', function() {
        colfer.unmarshal(marshaled)
    })
    .add('JSON > parse', function() {
        JSON.parse(stringified)
    })
    .on('cycle', (event) => {
        console.log(String(event.target))
    })
    .run({async: true})
    

[lewisdiamond]

Also, consider making the generated Javascript code actually just load a C version compiled to WASM.

[pascaldekloe]

Yes WASM sounds really cool yet that's another issue/feature.

I'm not sure where the b. in new b.bench.Bench(data) comes from. If you made a type called foo in a package called bar then it should read new bar.Foo(data).

Also, I just fixed issue #40 with an int64 bug. If you use that datatype please try version 1.7.2 and/or Maven plugin 1.11.2.

[lewisdiamond]

Sorry I omitted it.
b is just const b = require('./MyGeneratedColferFile.js')

[pascaldekloe]

Ah, then yes, your setup seems right.

I downloaded benchmark.js including its dependency lodash.js and I populated test.js with the following.

const Lodash = require('./lodash.js');
const Benchmark = require('./benchmark.js');

console.log(Benchmark.Suite);

For some reason node test.js prints "undefined". Do you understand why @lewisdiamond?

[lewisdiamond]

Did you download benchmark.js from there? https://raw.githubusercontent.com/bestiejs/benchmark.js/master/benchmark.js

 $ node
> const benchmark = require('./benchmark')
undefined
> benchmark.Suite
{ [Function: Suite] options: { name: undefined } }

$ node --version
v9.3.0

[pascaldekloe]

That exact file yes. Now I know this is supposed to work. Thanks for helping out.

% node --version
v8.9.4
% node
> const benchmark = require('./benchmark');
undefined
> benchmark.Suite
undefined
> benchmark
{ [Function: Benchmark]
  runInContext: [Function: runInContext],
  Benchmark: [Circular] }

[lewisdiamond]

It's likely because you don't have the lodash dependency.

const Lodash = require('./lodash.js');

That does not fulfill the dependency requirement.
They get lodash like this:

var _ = context && context._ || require('lodash') || root._;

Try this:

root._ = require('./lodash')
const Benchmark = require('benchmark')

Alternatively, use npm or yarn:

$cd my_directory
$npm init //or yarn init
$npm install lodash //or yarn add lodash
$npm install benchmark //or yarn add benchmark

then you can just do this:

const Benchmark = require('benchmark') //notice no leading ./

[pascaldekloe]

That root import did the trick and the suggested framework plays nicely. I can start tuning now.