v2.2.1 frontend poorly handles very large list of nodes

[justparking]

When first loading nodel browser list, the browser locks up while it processes the nodes. In my case, >2500 nodes to be listed.

[justparking]

Does this also "fix" node selection when updating bindings?

[justparking]

I'm not sure the latest commit ([8e1fece]) really helped much. In particular, updating bindings is still very poor. A filter on binding names might be a good starting point and make updating very large lists (e.g. Dashboards) a lot easier.

[scroix]

I'm not sure if this would play a factor but just the rendering on the browser alone appears to cause some chugging.

Here's an example of a very large activity log measured in pixels.

[mcartmel]

Yes, I've actually created a fork to test a new mechanism for handling nodes with large volumes of activity. Please feel free to test!

[scroix]

Oh -- that's awesome. Thank you for pointing it out.

@paxofsnax We should test the dashboard on this build. It's the ultimate test.

[mcartmel]

Oh -- that's awesome. Thank you for pointing it out.

@paxofsnax We should test the dashboard on this build. It's the ultimate test.

Any feedback on this at all?

[justparking]

Technical Nodel interfaces aside, I will state for the record that it's very clear that the "new mechanism" has a noticeable impact on the load time of the Dashboard-type Frontend interfaces (and possibly the update speed after load).

In a test I did on a reasonably large dashboard the "new mechanism" took >16 secs to load fully. Using previous versions it takes <3 secs to load the same large dashboard.

Using the technical Nodel interfaces the "new mechanism" is generally a bit quicker doing some things but it's hard to do a side-by-side comparison because the load behaviour is so different.

Regardless of my last point, I think it would be ideal if the existing Dashboard-type Frontends are excluded from any optimisations made by the "new mechanism". There has never been any issue with their load times and performance. In fact, they've always been impressively fast and efficient and it'd be a shame to lose that.

[mcartmel]

I've modified the "new mechanism" to be less aggressive, but retained the reduction in the number of long running operations (which were causing the lack of responsiveness). I also found a more efficient way to buffer the stream of incoming state changes. Load time is significantly improved in my tests.

Please feel free to test and comment further.

[scroix]

I'm not sure if the build had issues, but I tried compiling a binary with the new mechanism from here:

https://github.com/mcartmel/nodel/tree/feature_uispeed

I had some errors immediately after the launch.

2020-06-29 10:01:57.075 WARN [org.nodel.jyhost.NodelHostHTTPD_0] [pool_Nano HTTP_2] Unexpected exception during REST operation.
java.lang.NullPointerException
at org.nodel.reflection.Serialisation.serialise(Serialisation.java:698)
at org.nodel.reflection.Serialisation.serialise(Serialisation.java:681)
at org.nodel.reflection.Serialisation.serialise(Serialisation.java:674)
at org.nodel.jyhost.NodelHostHTTPD.serve(NodelHostHTTPD.java:323)
at org.nodel.host.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:634)
at org.nodel.threading.ThreadPool.threadMain(ThreadPool.java:303)
at org.nodel.threading.ThreadPool.access$000(ThreadPool.java:26) 
at org.nodel.threading.ThreadPool$1.run(ThreadPool.java:213)                                       

r415 loads the "dashboard" succesfully, albeit slowly (10s).

It might've been an issue with my build? The .jar was missing a revision number.