The Pluto package has a few functions that allow you to programmatically control Pluto. This page contains an overview of the public API exposed by the Pluto.jl package. We have also factored out the packages Malt.jl, ExpressionExplorer.jl and PlutoDependencyExplorer.jl. These packages contain useful functionality that is not documented here.

Overview

You can use ServerSession and SessionActions (module) for opening notebooks, shutting down, reading results and exporting to a file. With run you can serve a web interface of a ServerSession on localhost.

The Notebook and Cell structs are used to store notebook state. Their fields are not public API, but they are mostly stable in practice. You can read our tests, like this one to see how to interact with notebooks.

There is public API to change bond values programmatically: get_bond_names and set_bond_values_reactive, to trigger reactivity.

The functions frontmatter_html and generate_html let you export a notebook to a standalone HTML file. (A PDF file is just a browser print of the HTML file, see PlutoPDF.jl.) See also frontmatter. You can use notebook_to_js to get a ‘statefile’. You can use pack and unpack to access Pluto’s MsgPack implementation. Firebasey contains Pluto’s deep-state diffing algorithm (like immer.js).

Pluto contains functions to programmatically manage the package environment contained in a Pluto notebook: reset_notebook_environment, update_notebook_dependencies, activate_notebook_environment and will_use_pluto_pkg.

There are some utility functions to work with Pluto notebook file extensions: pluto_file_extensions, without_pluto_file_extension and is_pluto_notebook.

SessionActions

These functions are in the Pluto.SessionActions module. Everything in Pluto.SessionActions is public API, and covered by semver. Available functions are:

  • SessionActions.open
  • SessionActions.open_url
  • SessionActions.new
  • SessionActions.shutdown
  • SessionActions.move

ServerSession

All functionality in SessionActions requires a Pluto.ServerSession object. You can create a session with the Pluto.ServerSession() constructor:

session = Pluto.ServerSession()

You can also specify the keyword argument options to change the Configuration. For example:

session = Pluto.ServerSession(;
        options=Pluto.Configuration.from_flat_kwargs(
            auto_reload_from_file=true,
            etc...
        )
    )

Opening notebooks

Once you have a ServerSession, you can open notebooks in it. You can use open and open_url.

open

You can use open to open a notebook from a local file path, and add it to the session.

open(session::ServerSession, path::AbstractString; 
    run_async::Bool=true, 
    as_sample::Bool=false, 
    execution_allowed::Bool=true,
    notebook_id::UUID=uuid1()
    clear_frontmatter::Bool=false,
)

Useful keyword arguments:

  • execution_allowed: If false, the notebook will launch in “Safe Preview” mode. This is the default behavior when opening notebooks using the Pluto web interface.
  • run_async: If true, the function will block until all cells in the notebook have finished executing. If false, the function will return immediately after starting the notebook, and cells will execute in the background.
  • as_sample: If true, Pluto will not open the path, but create a new temporary copy of the notebook at path and open that instead.
  • clear_frontmatter: If true, any frontmatter in the notebook file will be removed.

open_url

You can use open_url to download a notebook from the web and open it.

function open_url(session::ServerSession, url::AbstractString; kwargs...)

The same keyword arguments as open are supported.

new

You can use new to create a new empty notebook in the session.

new(session::ServerSession;
    run_async=true,
    notebook_id::UUID=uuid1()
)

See open for the meaning of run_async.

More actions

shutdown

You can use shutdown to shut down a session.

shutdown(session::ServerSession, notebook::Notebook;
    keep_in_session::Bool=false,
    async::Bool=false,
    verbose::Bool=true
)

Keyword arguments:

  • keep_in_session: If false, the notebook will also be removed from session.notebooks.
  • async: If true, the function will return quickly, and the shutdown will happen in the background.
  • verbose: If true, warning messages might be logged.

move

You can use move to change where a notebook is stored.

move(session::ServerSession, notebook::Notebook, newpath::String)

run

Use Pluto.run to start Pluto. This is how most people start Pluto:

run(; kwargs...)

Keyword arguments: check out Configuration.

If you created your own ServerSession, you can use run! and run to start the web interface for that session.

run(session::ServerSession)
run!(session::ServerSession)

run! will start the server, and return a RunningPlutoServer once it’s ready. You can Base.wait on it to block until the server stops (with Ctrl+C), or Base.close to shut it down. run is a shorthand for Base.wait ∘ run!.

Exporting notebooks

Generate the HTML export contents of a notebook:

generate_html(notebook::Notebook)::String

This includes frontmatter HTML head content, but you can also generate this separately with:

frontmatter_html(fm::Dict{String,Any})::String

fm can be the output of Pluto.frontmatter.

Statefile

A core part of how Pluto works is its state synchronization (between the Julia server and the connected web browsers). All parties work on a shared “statefile” representation of the notebook. This is a JSON-like structure that contains all information about the notebook (cells, code, outputs, logs, timings, etc). Read more about it in Pluto’s source code, by reading the docstring of Pluto.Firebasey.

To get the statefile of a Notebook, use:

notebook_to_js(notebook::Notebook)::Dict{String,Any}

The output structure is easy to work with in another language (like JavaScript). Its structure is not covered by semver.

More info about Pluto's state management (Firebasey docstring)

State management in Pluto

Aka: how do the server and clients stay in sync?

A Pluto notebook session has state: with this, we mean:

  1. The input and ouput of each cell, the cell order, and more metadata about the notebook and cells [state]

This state needs to be synchronised between the server and all clients (we support multiple synchronised clients), and note that:

  • Either side wants to update the state. Generally, a client will update cell inputs, the server will update cell outputs.

  • Both sides want to react to state updates

  • The server is in Julia, the clients are in JS

  • This is built on top of our websocket+msgpack connection, but that doesn't matter too much

We do this by implementing something similar to how you use Google Firebase: there is one shared state object, any party can mutate it, and it will synchronise to all others automatically. The state object is a nested structure of mutable Dicts, with immutable ints, strings, bools, arrays, etc at the endpoints.

Some cool things are:

  • Our system uses object diffing, so only changes to the state are actually tranferred over the network. But you can use it as if the entire state is sent around constantly.

  • In the frontend, the shared state is part of the react state, i.e. shared state updates automatically trigger visual updates.

  • Within the client, state changes take effect instantly, without waiting for a round trip to the server. This means that when you add a cell, it shows up instantly.

Diffing is done using immer.js (frontend) and src/webserver/Firebasey.jl (server). We wrote Firebasey ourselves to match immer's functionality, and the cool thing is: it is a Pluto notebook! Since Pluto notebooks are .jl files, we can just include it in our module.

The shared state object is generated by notebook_to_js. Take a look! The Julia server orchestrates this firebasey stuff. For this, we keep a copy of the latest state of each client on the server (see current_state_for_clients). When anything changes to the Julia state (e.g. when a cell finished running), we call send_notebook_changes!, which will call notebook_to_js to compute the new desired state object. For each client, we diff the new state to their last known state, and send them the difference.

Responding to changes made by a client

When a client updates the shared state object, we want the server to react to that change by taking an action. Which action to take depends on which field changes. For example, when state["path"] changes, we should rename the notebook file. When state["cell_inputs"][a_cell_id]["code"] changes, we should reparse and analyze that cel, etc. This location of the change, e.g. "cell_inputs/<a_cell_id>/code" is called the path of the change.

effects_of_changed_state define these pattern-matchers. We use a Wildcard() to take the place of any key, see Wildcard, and we use the change/update/patch inside the given function.

Not everything uses the shared state (yet)

Besides :update_notebook, you will find more functions in responses that respond to classic 'client requests', such as :reshow_cell and :shutdown_notebook. Some of these requests get a direct response, like the list of autocomplete options to a :complete request (in src/webserver/REPLTools.jl). On the javascript side, these direct responses can be awaited, because every message has a unique ID.

state

Two other meanings of state could be:

  1. The reactivity data: the parsed AST (Expr) of each cell, which variables are defined or referenced by which cells, in what order will cells run?

  2. The state of the Julia process: i.e. which variables are defined, which packages are imported, etc.

The first two (1 & 2) are stored in a Notebook struct, remembered by the server process (Julia). (In fact, (2) is entirely described by (1), but we store it for performance reasons.) I included (3) for completeness, but it is not stored by us, we hope to control and minimize (3) by keeping track of (1) and (2).

pack and unpack

You can use Pluto.pack and Pluto.unpack to serialize and deserialize data (like the statefile) using MsgPack (with a couple of Pluto extensions and optimizations).

pack(x::Any)::Vector{UInt8}
unpack(data::Vector{UInt8})::Any

Firebasey

Firebasey is Pluto’s deep-state diffing algorithm (like immer.js). It is used to efficiently synchronize notebook state between the Julia server and the web browsers. To see simple examples of Firebasey usage (and its public API), check out the PlutoSliderServer.jl source code.

Firebasey is written as a Pluto notebook, and you can open src/webserver/Firebasey.jl in Pluto to see many more examples.

Bonds

You can use get_bond_names to get the list of variables names that are bound with @bind:

get_bond_names(session::ServerSession, notebook::Notebook)::Set{String}

With set_bond_values_reactive you can change the values of bound variables to trigger a reactive run in a Notebook. Note that the values should be those before transformation.

# set the new values
notebook.bonds[:x] = Dict("value" => 1)
notebook.bonds[:y] = Dict("value" => "two")

set_bond_values_reactive(;
    session::ServerSession, notebook::Notebook,
    # names of the bonds that are changed
    bound_sym_names::AbstractVector{Symbol},
    # just leave this as default
    is_first_values::AbstractVector{Bool}=[false for x in bound_sym_names],
    # if `false`, the function will block until all reactive cells have run
    run_async::Bool=true,
)::TopologicalOrder

Package management

Pluto contains functions to programmatically manage the package environment contained in a Pluto notebook. These functions are:

reset_notebook_environment

reset_notebook_environment(notebook_path::String; keep_project::Bool=false, backup::Bool=true)

Remove the embedded Project.toml and Manifest.toml from a notebook file, modifying the notebook file. If keep_project is true, only Manifest.toml will be deleted. A backup of the notebook file is created by default.

update_notebook_environment

update_notebook_environment(notebook_path::String; backup::Bool=true, level::Pkg.UpgradeLevel=Pkg.UPLEVEL_MAJOR)

Call Pkg.update in the package environment embedded in a notebook file, modifying the notebook file. A Pkg.UpgradeLevel can be passed to the level keyword argument. A backup file is created by default.

activate_notebook_environment

activate_notebook_environment(notebook_path::String; show_help::Bool=true)::Nothing

Activate the package environment embedded in a notebook file, for interactive use. This will allow you to use the Pkg REPL and Pkg commands to modify the environment, and any changes you make will be automatically saved in the notebook file.

More help will be displayed if show_help is true.

Limitations:

  • Shut down the notebook before using this functionality.

  • Non-interactive use is limited, use the functional form instead, or insert sleep calls after modifying the environment.

Info

This functionality works using file watching. A dummy repository contains a copy of the embedded tomls and gets activated, and the notebook file is updated when the dummy repository changes.

activate_notebook_environment(f::Function, notebook_path::String)

Temporarily activate the package environment embedded in a notebook file, for use inside scripts. Inside your function f, you can use Pkg commands to modify the environment, and any changes you make will be automatically saved in the notebook file after your function finishes. Not thread-safe.

This method is best for scripts that update notebook files. For interactive use, the method activate_notebook_environment(notebook_path::String) is recommended.

Example

Pluto.activate_notebook_environment("notebook.jl") do
    Pkg.add("Example")
end

Now the file "notebook.jl" was updated!

Warning

This function uses the private method Pkg.activate(f::Function, path::String). This API might not be available in future Julia versions. 🤷

will_use_pluto_pkg

will_use_pluto_pkg(notebook_path::String)::Bool

Will this notebook use the Pluto package manager? false means that the notebook contains Pkg.activate or another deactivator.

File extensions

Pluto notebook files are usually .jl, but there are some other “official” extensions like .plutojl, .pluto.jl and more. There are some utility functions:

# List of "official" Pluto file extensions
pluto_file_extensions::Vector{String}

# Remove Pluto file extension from path
without_pluto_file_extension(path::String)::String

## Does the path end with a pluto file extension (like `.jl` or `.pluto.jl`) and does the first line say `### A Pluto.jl notebook ###`?
is_pluto_notebook(path::String)::Bool
Edit this page