UI watch mode that automatically opens/updates panels from external IDE

[mofojed]

As a developer using an external IDE (such as VSCode or IntelliJ) in my workflow, I would like a browser mode that automatically opens/updates/closes panels as code is run in a session. I do not the Console, Command History, Log, etc. as those are all in my IDE already.

Desired workflow:

1. Set up IntelliJ to use a venv with deephaven and deephaven_server installed
   a. Create venv locally - python -m venv .venv
   b. Under File -> Project Structure -> Platform Settings -> SDKs, selection Python Home Path to be .venv/bin/python
2. Open up the Python Console and start a server:

from deephaven_server import Server
s = Server(port=10500, jvm_args=["-DAuthHandlers=io.deephaven.auth.AnonymousAuthenticationHandler"])
s.start()

3. Open a browser to http://localhost:10500/watch (whatever the URL should be ... /watch? /live?). Should be an app with an empty layout (settings accessible etc).
4. From a Python file in IntelliJ, enter some basic code, then right click and select "Run File in Python Console", e.g.

from deephaven import empty_table
t = empty_table(100).update("x=i")

5. The table should open up in the browser
6. Change the code in your file and re-run:

from deephaven import empty_table
t = empty_table(100).update("x=i*2")
t2 = t.update("y=x*i")

7. The browser should update with the newly created t2, and t should update to show the new value entered.

[mofojed]

You can also use pydeephaven to connect to an existing server rather than creating a new one in the Python console. Run this in the Python console first before running any other code:

import code
from pydeephaven import Session

port = 10000
session = Session(port=port)

class Repl(code.InteractiveConsole):
    def runsource(
        self, source: str, filename: str = "<input>", symbol: str = "single"
    ) -> bool:
        session.run_script(source)
        return True


repl = Repl()
repl.interact(banner=f"Connected to Deephaven on port {port}.", exitmsg="Disconnected from Deephaven.")

It will capture the code executed when running "Execute selection in Python console" and run it on the server it is connected to.

[dsmmcken]

You can also use pydeephaven to connect to an existing server rather than creating a new one in the Python console. Run this in the Python console first before running any other code:

We should try wrapping that up in a deephaven server -i or --interactive call.

For enterprise I wonder if we can support an authenticated version too?

[dsmmcken]

Open a browser to http://localhost:10500/watch (whatever the URL should be ... /watch? /live?). Should be an app with an empty layout (settings accessible etc).

Think it should be a special page or just something that a regular console does? Auto open panels executed from anywhere? I thought nate's old console branch did that.

[dsmmcken]

exectuted commands could also write back iframe urls or something each element:

>>> a = empty_table...
>>> b = ....
a , b

[mofojed]

You can also use pydeephaven to connect to an existing server rather than creating a new one in the Python console.
We should try wrapping that up in a deephaven server -i or --interactive call.

deephaven server starts up a new server, whereas this would be using pydeephaven to connect to an existing server... more importantly, we need to run that in the Python console that IntelliJ opens, so we can't wrap it with a CLI call necessarily. But yes, it would be nice if there was something in pydeephaven perhaps that did that, e.g.

from pydeephaven import start_repl
start_repl(port=10000, ...)

For enterprise I wonder if we can support an authenticated version too?

For Enterprise, we'd be looking at using the session manager: https://deephaven.io/enterprise/docs/coreplus/coreplus-python-client/

I didn't bother trying an example for that yet, but you should be able to do the same. Annoying to have to remember which to use/where, and the syntax being different for these ways to connect. Maybe could shorten it to something like:

from deephaven_enterprise.client import start_repl
start_repl(connection_info="https://deephaven-host:8000/iris/connection.json", key="...", options={...})

[bmingles]

Depending on how this is implemented, this might also take care of Opened panels should match the state of panels in the panels menu #1924