Implement position file change notification

[kentcdodds]

I'm trying to implement improved consistency and one approach I started with but don't like is polling the file until it's been updated to the user session's txid. I'd rather be notified when the file is updated so I can respond quicker and don't have to poll. So I'm trying to use chokidar on that file and it doesn't seem to be working. I'm thinking that because that file didn't show up in ls -al until something fancy was done maybe it's not a real file? So maybe it doesn't work with file watching tools like chokidar?

Is there any way to be notified when the -pos file is updated?

[benbjohnson]

@kentcdodds The position file isn't really an actual file. LiteFS fakes it so it reads the litefs.DB.Pos() value and formats it when read from a handle:

litefs/fuse/pos_node.go

Lines 49 to 65 in 6ecc549

	func (n *PosNode) Read(ctx context.Context, req *fuse.ReadRequest, resp *fuse.ReadResponse) error {
	pos := n.db.Pos()
	data := fmt.Sprintf("%016x/%016x\n", pos.TXID, pos.PostApplyChecksum)
	if req.Offset >= int64(len(data)) {
	return io.EOF
	}
	// Size buffer to offset/size.
	data = data[req.Offset:]
	if len(data) > req.Size {
	data = data[:req.Size]
	}
	resp.Data = []byte(data)
	return nil
	}

I think I just need to implement fuse/fs.Poller on the position node. Currently it returns ENOSYS (which just says it's not implemented):

litefs/fuse/pos_node.go

Lines 49 to 65 in 6ecc549

	func (n *PosNode) Read(ctx context.Context, req *fuse.ReadRequest, resp *fuse.ReadResponse) error {
	pos := n.db.Pos()
	data := fmt.Sprintf("%016x/%016x\n", pos.TXID, pos.PostApplyChecksum)
	if req.Offset >= int64(len(data)) {
	return io.EOF
	}
	// Size buffer to offset/size.
	data = data[req.Offset:]
	if len(data) > req.Size {
	data = data[:req.Size]
	}
	resp.Data = []byte(data)
	return nil
	}

I'll take a look and see how difficult it would be to implement. Hopefully it's relatively easy.

[benbjohnson]

@kentcdodds I did some more research and it looks like FUSE doesn't implement file change notification (e.g. inotify). There are plans to add an event stream (#20) and that'll be accessible via something like a unix socket. The position data will be available through that handle so I'm going to close this issue in favor of #20.

[kentcdodds]

Bummer. Back to polling then. I've subscribed to the other issue. Thanks!

[benbjohnson]

@kentcdodds This is an old issue but I just wanted to let you know that I did end up adding an event stream (#401) so you can get immediate notification of the replication position & primary state changes from LiteFS. That's probably not as big of a deal now that we have the LiteFS proxy but I figured I'd give you a heads up.

[kentcdodds]

Thanks!