Great Dashboard

[dougcsd]

I have been administrating a local Typesense-Server Cluster for a little over 2 years using curl commands exclusively, which is a bit cumbersome to say the least. I was getting ready to update to v29, and decided to see what administrative tools were available and came across your dashboard.

Fantastic Work!

Your desktop made checking server status, versions, and statistics a snap, While upgrading, I was able to quickly switch between servers using the history feature to check versions, server loads, etc, and most importantly to compare metrics across each node in almost real time. This allowed me to compare the performance of the updated nodes against the old nodes immediately after each update.

My feature recommendations:

1. Only feature I didn't find was support for administrating stem dictionaries. Would love to see that feature added in the future, similar to how you handled synonyms, but not urgent. When you do, probably should add the "stem_dictionary" field to your form schema editor, but that change can be done with the JSON schema editor in your desktop already.

2. Not absolutely necessary, but I think a great feature to add would be a "Cluster Status" tab. I think it would probably be a relatively easy feature to implement since it would mostly re-use your existing server status column code. The idea is to add a screen with a select/add to cluster from the existing history list (top of page box option?). You can name and save selected clusters if you want to enhance this feature idea beyond a simple group selection. The user selects multiple servers and can see the same status column you already generate displayed in a side by side format (one column for each server in the cluster). You would want to poll them in parallel as close as possible to optimize the comparison time frame window for each polling if possible. Didn't look at your code, but assuming you used libcurl, you can use curl_multi if you aren't already, to launch these queries in parallel.

Surprised Typesense has not promoted your desktop from their website as it really makes administrating local servers much easier. Guess they don't want to promote something that might lead people away from their cloud service, but most people using a cloud service wouldn't do a local install even with a dashboard..

Thanks again for all your hard work,
Doug

[bfritscher]

Hi Doug,
Thanks for your kind words and taking the time to share your experience.

1. For the stem_dictionnary it is clearly something we can add ( I add new typesense feature, as I need them or as they are requested ) Implement UI for Stemming feature #108
2. Cluster server monitoring is a bit out of scope for this app and may be better done with monitoring apps like Grafana or other. I do not have experience with typesense clusters. Do you need to send the api write commands to te leader or any node? Is there a way through the api to now which node is part of which cluster?

Regards,
Boris

[dougcsd]

Boris,

Clusters work the same as a stand alone nodes as far as functionality. The only difference is changes on any node (Follower or Leader) is propagated through the Leader to all the followers. It is not like MySQL where you have a master/slave architecture and writes are limited to one server. Shutting down the current leader results in a re-election among the remaining nodes to nominate a new Leader. The statistics displayed on your summary page from these calls are limited to the node being queried.

I haven't seen a way to get a list of nodes (active or configured) in Typesense documentation. Since I wrote my last message I looked at your code a bit, and see you are using their API, so you can ignore my curl comment. I envisioned this as you can select multiple nodes from your saved history and your application displays just the right sidebar info for each node (remove the CPU meters/percentage from the page) in a side by side layout. With the API you are using, you can't make these calls in parallel, but since most people are likely running this local, the delay between serial calls would be minimal and give a good idea of how the nodes are performing in relation to each other.

The idea here is to see performance of each node (XXXX/s variables) side by side. I am generally looking at queries per second per node and latency of those queries when comparing the performance of the nodes.

FYI - I found one issue. In newer versions (at least 25+), you can use an int64 as a default sort index in your schema. You code cleared this when I tried to modify a field. Not sure if this is an API bug or in your code. I didn't dig that deep. It was limited to int32 in older versions from what I have read. I think they made this change because dates are a common sort index, and most people use an int64 to hold dates.

Thanks again,
Doug

[dougcsd]

FYI in case you want to try a cluster setup. You need to add a node config file to define the list of nodes in your cluster. ie:

172.22.130.234:8107:8108,172.22.131.199:8107:8108,172.22.132.113:8107:8108

This is a comma separated list of servers in the format: <ip address>:<peering port:>:<api port>

In typesense.ini your general config is as follows with the last 3 lines defining the cluster variables:

api-address = 0.0.0.0
api-port = 8108
data-dir = /usr/local/typesense/data
log-dir = /usr/local/typesense/log

; HA Cluster config
peering-subnet = 172.22.130.0/21
peering-port = 8107
nodes=/etc/typesense/nodes

You can change the ports/dirs around to run multiple nodes on the same host for your tests if you don't have separate servers to test on. This config has three nodes for a basic n+1.

Good luck.

[bfritscher]

@dougcsd I relased v2.3.0 with support for Stemming dictionnaries. I identified some problems with the api I documented here typesense/typesense#2585 Did you use the api with curl and have some other insights?

[dougcsd]

Tried 2.30, and it looks good. Thanks for the update. I have only loaded/deleted complete dictionaries in the form of .jsonl files. on my end so far. I used command line "curl" commands to do this, so your implementation supports both the operations I am currently using. I saw your comments on failing upserts. Would have been nice to have a line item editor, but they need to fix that issue in their API first.

Hate to say this as I like Typesense on the whole, but the areas it seems to be lacking in are Administrative functions and up to date documentation. I assume this is because the development team is focused on search features., so I am hopeful this will improve in the future. Your dictionary upsert issue and the int64 default index are perfect examples. Their documentation page still says only int32, but I found a commit where this was changed to support int64 several versions ago.

Anyways thanks again for the update. The dashboard you have created is a much appreciated and useful tool.

Doug

[bfritscher]

Hi, @dougcsd v2.4 has cluster support, You can group nodes from history via the pen edit tool

Kind Regards,
Boris

[dougcsd]

Boris,

Fantastic. Works perfect. I can see everything going on in my cluster at a glance. Thanks for all your hard work. I like the cluster node quick select shortcut links you added to the standard status page column as well. Very easy to jump between nodes once the cluster is defined. The new health indicator you added is nice as well. The integration into the existing layout is very clean.

Thanks again,
Doug