Frontmatter

""" The full list of keyword arguments that can be passed to [`Pluto.run`](@ref) (or [`Pluto.Configuration.from_flat_kwargs`](@ref)) is divided into four categories. Take a look at the documentation for: - [`Pluto.Configuration.CompilerOptions`](@ref) defines the command line arguments for notebook `julia` processes. - [`Pluto.Configuration.ServerOptions`](@ref) configures the HTTP server. - [`Pluto.Configuration.SecurityOptions`](@ref) configures the authentication options for Pluto's HTTP server. Change with caution. - [`Pluto.Configuration.EvaluationOptions`](@ref) is used internally during Pluto's testing. Note that Pluto is designed to be _zero-configuration_, and most users should not (have to) change these settings. Most 'customization' can be achieved using Julia's wide range of packages! That being said, the available settings are useful if you are using Pluto in a special environment, such as docker, mybinder, etc. """ module Configuration using Configurations # https://github.com/Roger-luo/Configurations.jl import ..Pluto: tamepath safepwd() = try pwd() catch e @warn "pwd() failure" exception=(e, catch_backtrace()) homedir() end # Using a ref to avoid fixing the pwd() output during the compilation phase. We don't want this value to be baked into the sysimage, because it depends on the `pwd()`. We do want to cache it, because the pwd might change while Pluto is running. const pwd_ref = Ref{String}() function notebook_path_suggestion() pwd_val = if isassigned(pwd_ref) pwd_ref[] else safepwd() end preferred_dir = startswith(Sys.BINDIR, pwd_val) ? homedir() : pwd_val # so that it ends with / or \ string(joinpath(preferred_dir, "")) end function __init__() pwd_ref[] = safepwd() end const ROOT_URL_DEFAULT = nothing const BASE_URL_DEFAULT = "/" const HOST_DEFAULT = "127.0.0.1" const PORT_DEFAULT = nothing const PORT_HINT_DEFAULT = 1234 const LAUNCH_BROWSER_DEFAULT = true const DISMISS_UPDATE_NOTIFICATION_DEFAULT = false const DISMISS_MOTIVATIONAL_QUOTES = false const SHOW_FILE_SYSTEM_DEFAULT = true const ENABLE_AI_EDITOR_FEATURES_DEFAULT = true const DISABLE_WRITING_NOTEBOOK_FILES_DEFAULT = false const AUTO_RELOAD_FROM_FILE_DEFAULT = false const AUTO_RELOAD_FROM_FILE_COOLDOWN_DEFAULT = 0.4 const AUTO_RELOAD_FROM_FILE_IGNORE_PKG_DEFAULT = false const NOTEBOOK_DEFAULT = nothing const SIMULATED_LAG_DEFAULT = 0.0 const SIMULATED_PKG_LAG_DEFAULT = 0.0 const INJECTED_JAVASCRIPT_DATA_URL_DEFAULT = "data:text/javascript;base64," const ON_EVENT_DEFAULT = function(a) #= @info "$(typeof(a))" =# end """ ServerOptions([; kwargs...]) The HTTP server options. See [`SecurityOptions`](@ref) for additional settings. # Keyword arguments - `host::String = "$HOST_DEFAULT"` Set to `"127.0.0.1"` (default) to run on *localhost*, which makes the server available to your computer and the local network (LAN). Set to `"0.0.0.0"` to make the server available to the entire network (internet). - `port::Union{Nothing,Integer} = $PORT_DEFAULT` When specified, this port will be used for the server. - `port_hint::Integer = $PORT_HINT_DEFAULT` If the other setting `port` is not specified, then this setting (`port_hint`) will be used as the starting point in finding an available port to run the server on. - `launch_browser::Bool = $LAUNCH_BROWSER_DEFAULT` - `dismiss_update_notification::Bool = $DISMISS_UPDATE_NOTIFICATION_DEFAULT` If `false`, the Pluto frontend will check the Pluto.jl github releases for any new recommended updates, and show a notification if there are any. If `true`, this is disabled. - `dismiss_motivational_quotes::Bool = $DISMISS_MOTIVATIONAL_QUOTES` If `true`, motivational quotes on error messages won't be shown. - `show_file_system::Bool = $SHOW_FILE_SYSTEM_DEFAULT` - `enable_ai_editor_features::Bool = $ENABLE_AI_EDITOR_FEATURES_DEFAULT` Enable or disable LLM-powered editor features - `notebook_path_suggestion::String = notebook_path_suggestion()` - `disable_writing_notebook_files::Bool = $DISABLE_WRITING_NOTEBOOK_FILES_DEFAULT` - `auto_reload_from_file::Bool = $AUTO_RELOAD_FROM_FILE_DEFAULT` Watch notebook files for outside changes and update running notebook state automatically - `auto_reload_from_file_cooldown::Real = $AUTO_RELOAD_FROM_FILE_COOLDOWN_DEFAULT` Experimental, will be removed - `auto_reload_from_file_ignore_pkg::Bool = $AUTO_RELOAD_FROM_FILE_IGNORE_PKG_DEFAULT` Experimental flag, will be removed - `notebook::Union{Nothing,String} = $NOTEBOOK_DEFAULT` Optional path of notebook to launch at start - `simulated_lag::Real=$SIMULATED_LAG_DEFAULT` (internal) Extra lag to add to our server responses. Will be multiplied by `0.5 + rand()`. - `simulated_pkg_lag::Real=$SIMULATED_PKG_LAG_DEFAULT` (internal) Extra lag to add to operations done by Pluto's package manager. Will be multiplied by `0.5 + rand()`. - `injected_javascript_data_url::String = "$INJECTED_JAVASCRIPT_DATA_URL_DEFAULT"` (internal) Optional javascript injectables to the front-end. Can be used to customize the editor, but this API is not meant for general use yet. - `on_event::Function = $ON_EVENT_DEFAULT` - `root_url::Union{Nothing,String} = $ROOT_URL_DEFAULT` This setting is used to specify the root URL of the Pluto server, but this setting is *only* used to customize the launch message (*"Go to http://localhost:1234/ in your browser"*). You can probably ignore this and use `base_url` instead. - `base_url::String = "$BASE_URL_DEFAULT"` This (advanced) setting is used to specify a subpath at which the Pluto server will run, it should be a path starting and ending with a '/'. E.g. with `base_url = "/hello/world/"`, the server will run at `http://localhost:1234/hello/world/`, and you edit a notebook at `http://localhost:1234/hello/world/edit?id=...`. """ @option mutable struct ServerOptions root_url::Union{Nothing,String} = ROOT_URL_DEFAULT base_url::String = BASE_URL_DEFAULT host::String = HOST_DEFAULT port::Union{Nothing,Integer} = PORT_DEFAULT port_hint::Integer = PORT_HINT_DEFAULT launch_browser::Bool = LAUNCH_BROWSER_DEFAULT dismiss_update_notification::Bool = DISMISS_UPDATE_NOTIFICATION_DEFAULT dismiss_motivational_quotes::Bool = DISMISS_MOTIVATIONAL_QUOTES show_file_system::Bool = SHOW_FILE_SYSTEM_DEFAULT enable_ai_editor_features::Bool = ENABLE_AI_EDITOR_FEATURES_DEFAULT notebook_path_suggestion::String = notebook_path_suggestion() disable_writing_notebook_files::Bool = DISABLE_WRITING_NOTEBOOK_FILES_DEFAULT auto_reload_from_file::Bool = AUTO_RELOAD_FROM_FILE_DEFAULT auto_reload_from_file_cooldown::Real = AUTO_RELOAD_FROM_FILE_COOLDOWN_DEFAULT auto_reload_from_file_ignore_pkg::Bool = AUTO_RELOAD_FROM_FILE_IGNORE_PKG_DEFAULT notebook::Union{Nothing,String,Vector{<:String}} = NOTEBOOK_DEFAULT simulated_lag::Real = SIMULATED_LAG_DEFAULT simulated_pkg_lag::Real = SIMULATED_PKG_LAG_DEFAULT injected_javascript_data_url::String = INJECTED_JAVASCRIPT_DATA_URL_DEFAULT on_event::Function = ON_EVENT_DEFAULT end const REQUIRE_SECRET_FOR_OPEN_LINKS_DEFAULT = true const REQUIRE_SECRET_FOR_ACCESS_DEFAULT = true const WARN_ABOUT_UNTRUSTED_CODE_DEFAULT = true """ SecurityOptions([; kwargs...]) Security settings for the HTTP server. # Arguments - `require_secret_for_open_links::Bool = $REQUIRE_SECRET_FOR_OPEN_LINKS_DEFAULT` Whether the links `http://localhost:1234/open?path=/a/b/c.jl` and `http://localhost:1234/open?url=http://www.a.b/c.jl` should be protected. Use `true` for almost every setup. Only use `false` if Pluto is running in a safe container (like mybinder.org), where arbitrary code execution is not a problem. - `require_secret_for_access::Bool = $REQUIRE_SECRET_FOR_ACCESS_DEFAULT` If `false`, you do not need to use a `secret` in the URL to access Pluto: you will be authenticated by visiting `http://localhost:1234/` in your browser. An authentication cookie is still used for access (to prevent XSS and deceptive links or an img src to `http://localhost:1234/open?url=badpeople.org/script.jl`), and is set automatically, but this request to `/` is protected by cross-origin policy. Use `true` on a computer used by multiple people simultaneously. Only use `false` if necessary. - `warn_about_untrusted_code::Bool = $WARN_ABOUT_UNTRUSTED_CODE_DEFAULT` Should the Pluto GUI show warning messages about executing code from an unknown source, e.g. when opening a notebook from a URL? When `false`, notebooks will still open in Safe mode, but there is no scary message when you run it. **Leave these options on `true` for the most secure setup.** Note that Pluto is quickly evolving software, maintained by designers, educators and enthusiasts  not security experts. If security is a serious concern for your application, then we recommend running Pluto inside a container and verifying the relevant security aspects of Pluto yourself. """ @option mutable struct SecurityOptions require_secret_for_open_links::Bool = REQUIRE_SECRET_FOR_OPEN_LINKS_DEFAULT require_secret_for_access::Bool = REQUIRE_SECRET_FOR_ACCESS_DEFAULT warn_about_untrusted_code::Bool = WARN_ABOUT_UNTRUSTED_CODE_DEFAULT end const RUN_NOTEBOOK_ON_LOAD_DEFAULT = true const WORKSPACE_USE_DISTRIBUTED_DEFAULT = true const WORKSPACE_USE_DISTRIBUTED_STDLIB_DEFAULT = nothing const LAZY_WORKSPACE_CREATION_DEFAULT = false const CAPTURE_STDOUT_DEFAULT = true const WORKSPACE_CUSTOM_STARTUP_EXPR_DEFAULT = nothing """ EvaluationOptions([; kwargs...]) Options to change Pluto's evaluation behaviour during internal testing and by downstream packages. These options are not intended to be changed during normal use. - `run_notebook_on_load::Bool = $RUN_NOTEBOOK_ON_LOAD_DEFAULT` When running a notebook (not in Safe mode), should all cells evaluate immediately? Warning: this is only for internal testing, and using it will lead to unexpected behaviour and hard-to-reproduce notebooks. It's not the Pluto way! - `workspace_use_distributed::Bool = $WORKSPACE_USE_DISTRIBUTED_DEFAULT` Whether to start notebooks in a separate process. Setting this to `false` is only meant for very advanced users, many features will be broken or behave unexpectedly (inlcuding anything related to package loading or interrupts). - `workspace_use_distributed_stdlib::Bool? = $WORKSPACE_USE_DISTRIBUTED_STDLIB_DEFAULT` Should we use the Distributed stdlib to run processes, instead of the new Malt.jl runner? You can use `true` to get the old behaviour. `nothing` means: determine automatically (which is currently `false`). - `lazy_workspace_creation::Bool = $LAZY_WORKSPACE_CREATION_DEFAULT` - `capture_stdout::Bool = $CAPTURE_STDOUT_DEFAULT` - `workspace_custom_startup_expr::Union{Nothing,String} = $WORKSPACE_CUSTOM_STARTUP_EXPR_DEFAULT` An expression to be evaluated in the workspace process before running notebook code. Warning: this will mean that your notebooks are not reproducible. """ @option mutable struct EvaluationOptions run_notebook_on_load::Bool = RUN_NOTEBOOK_ON_LOAD_DEFAULT workspace_use_distributed::Bool = WORKSPACE_USE_DISTRIBUTED_DEFAULT workspace_use_distributed_stdlib::Union{Bool,Nothing} = WORKSPACE_USE_DISTRIBUTED_STDLIB_DEFAULT lazy_workspace_creation::Bool = LAZY_WORKSPACE_CREATION_DEFAULT capture_stdout::Bool = CAPTURE_STDOUT_DEFAULT workspace_custom_startup_expr::Union{Nothing,String} = WORKSPACE_CUSTOM_STARTUP_EXPR_DEFAULT end const COMPILE_DEFAULT = nothing const PKGIMAGES_DEFAULT = nothing const COMPILED_MODULES_DEFAULT = nothing const SYSIMAGE_DEFAULT = nothing const SYSIMAGE_NATIVE_CODE_DEFAULT = nothing const BANNER_DEFAULT = nothing const DEPWARN_DEFAULT = nothing const OPTIMIZE_DEFAULT = nothing const MIN_OPTLEVEL_DEFAULT = nothing const INLINE_DEFAULT = nothing const CHECK_BOUNDS_DEFAULT = nothing const MATH_MODE_DEFAULT = nothing const STARTUP_FILE_DEFAULT = "no" const HISTORY_FILE_DEFAULT = "no" const HEAP_SIZE_HINT_DEFAULT = nothing function roughly_the_number_of_physical_cpu_cores() # https://gist.github.com/fonsp/738fe244719cae820245aa479e7b4a8d threads = Sys.CPU_THREADS num_threads_is_maybe_doubled_for_marketing = Sys.ARCH === :x86_64 if threads == 1 1 elseif threads == 2 || threads == 3 2 elseif num_threads_is_maybe_doubled_for_marketing # This includes: # - intel hyperthreading # - Apple ARM efficiency cores included in the count (when running the x86 executable) threads  2 else threads end end function default_number_of_threads() env_value = get(ENV, "JULIA_NUM_THREADS", "") all(isspace, env_value) ? roughly_the_number_of_physical_cpu_cores() : env_value end """ CompilerOptions([; kwargs...]) These options will be passed as command line argument to newly launched processes. See [the Julia documentation on command-line options](https://docs.julialang.org/en/v1/manual/command-line-options/). # Arguments - `compile::Union{Nothing,String} = $COMPILE_DEFAULT` - `pkgimages::Union{Nothing,String} = $PKGIMAGES_DEFAULT` - `compiled_modules::Union{Nothing,String} = $COMPILED_MODULES_DEFAULT` - `sysimage::Union{Nothing,String} = $SYSIMAGE_DEFAULT` - `sysimage_native_code::Union{Nothing,String} = $SYSIMAGE_NATIVE_CODE_DEFAULT` - `banner::Union{Nothing,String} = $BANNER_DEFAULT` - `depwarn::Union{Nothing,String} = $DEPWARN_DEFAULT` - `optimize::Union{Nothing,Int} = $OPTIMIZE_DEFAULT` - `min_optlevel::Union{Nothing,Int} = $MIN_OPTLEVEL_DEFAULT` - `inline::Union{Nothing,String} = $INLINE_DEFAULT` - `check_bounds::Union{Nothing,String} = $CHECK_BOUNDS_DEFAULT` - `math_mode::Union{Nothing,String} = $MATH_MODE_DEFAULT` - `heap_size_hint::Union{Nothing,String} = $HEAP_SIZE_HINT_DEFAULT` - `startup_file::Union{Nothing,String} = "$STARTUP_FILE_DEFAULT"` By default, the startup file isn't loaded in notebooks. - `history_file::Union{Nothing,String} = "$HISTORY_FILE_DEFAULT"` By default, the history isn't loaded in notebooks. - `threads::Union{Nothing,String,Int} = default_number_of_threads()` """ @option mutable struct CompilerOptions compile::Union{Nothing,String} = COMPILE_DEFAULT pkgimages::Union{Nothing,String} = PKGIMAGES_DEFAULT compiled_modules::Union{Nothing,String} = COMPILED_MODULES_DEFAULT sysimage::Union{Nothing,String} = SYSIMAGE_DEFAULT sysimage_native_code::Union{Nothing,String} = SYSIMAGE_NATIVE_CODE_DEFAULT banner::Union{Nothing,String} = BANNER_DEFAULT depwarn::Union{Nothing,String} = DEPWARN_DEFAULT optimize::Union{Nothing,Int} = OPTIMIZE_DEFAULT min_optlevel::Union{Nothing,Int} = MIN_OPTLEVEL_DEFAULT inline::Union{Nothing,String} = INLINE_DEFAULT check_bounds::Union{Nothing,String} = CHECK_BOUNDS_DEFAULT math_mode::Union{Nothing,String} = MATH_MODE_DEFAULT heap_size_hint::Union{Nothing,String} = HEAP_SIZE_HINT_DEFAULT # notebook specified options # the followings are different from # the default julia compiler options startup_file::Union{Nothing,String} = STARTUP_FILE_DEFAULT history_file::Union{Nothing,String} = HISTORY_FILE_DEFAULT threads::Union{Nothing,String,Int} = default_number_of_threads() end """ Collection of all settings that configure a Pluto session. `ServerSession` contains a `Configuration`. """ @option struct Options server::ServerOptions = ServerOptions() security::SecurityOptions = SecurityOptions() evaluation::EvaluationOptions = EvaluationOptions() compiler::CompilerOptions = CompilerOptions() end function from_flat_kwargs(; root_url::Union{Nothing,String} = ROOT_URL_DEFAULT, base_url::String = BASE_URL_DEFAULT, host::String = HOST_DEFAULT, port::Union{Nothing,Integer} = PORT_DEFAULT, port_hint::Integer = PORT_HINT_DEFAULT, launch_browser::Bool = LAUNCH_BROWSER_DEFAULT, dismiss_update_notification::Bool = DISMISS_UPDATE_NOTIFICATION_DEFAULT, dismiss_motivational_quotes::Bool = DISMISS_MOTIVATIONAL_QUOTES, show_file_system::Bool = SHOW_FILE_SYSTEM_DEFAULT, enable_ai_editor_features::Bool = ENABLE_AI_EDITOR_FEATURES_DEFAULT, notebook_path_suggestion::String = notebook_path_suggestion(), disable_writing_notebook_files::Bool = DISABLE_WRITING_NOTEBOOK_FILES_DEFAULT, auto_reload_from_file::Bool = AUTO_RELOAD_FROM_FILE_DEFAULT, auto_reload_from_file_cooldown::Real = AUTO_RELOAD_FROM_FILE_COOLDOWN_DEFAULT, auto_reload_from_file_ignore_pkg::Bool = AUTO_RELOAD_FROM_FILE_IGNORE_PKG_DEFAULT, notebook::Union{Nothing,String,Vector{<:String}} = NOTEBOOK_DEFAULT, simulated_lag::Real = SIMULATED_LAG_DEFAULT, simulated_pkg_lag::Real = SIMULATED_PKG_LAG_DEFAULT, injected_javascript_data_url::String = INJECTED_JAVASCRIPT_DATA_URL_DEFAULT, on_event::Function = ON_EVENT_DEFAULT, require_secret_for_open_links::Bool = REQUIRE_SECRET_FOR_OPEN_LINKS_DEFAULT, require_secret_for_access::Bool = REQUIRE_SECRET_FOR_ACCESS_DEFAULT, warn_about_untrusted_code::Bool = WARN_ABOUT_UNTRUSTED_CODE_DEFAULT, run_notebook_on_load::Bool = RUN_NOTEBOOK_ON_LOAD_DEFAULT, workspace_use_distributed::Bool = WORKSPACE_USE_DISTRIBUTED_DEFAULT, workspace_use_distributed_stdlib::Union{Bool,Nothing} = WORKSPACE_USE_DISTRIBUTED_STDLIB_DEFAULT, lazy_workspace_creation::Bool = LAZY_WORKSPACE_CREATION_DEFAULT, capture_stdout::Bool = CAPTURE_STDOUT_DEFAULT, workspace_custom_startup_expr::Union{Nothing,String} = WORKSPACE_CUSTOM_STARTUP_EXPR_DEFAULT, compile::Union{Nothing,String} = COMPILE_DEFAULT, pkgimages::Union{Nothing,String} = PKGIMAGES_DEFAULT, compiled_modules::Union{Nothing,String} = COMPILED_MODULES_DEFAULT, sysimage::Union{Nothing,String} = SYSIMAGE_DEFAULT, sysimage_native_code::Union{Nothing,String} = SYSIMAGE_NATIVE_CODE_DEFAULT, banner::Union{Nothing,String} = BANNER_DEFAULT, depwarn::Union{Nothing,String} = DEPWARN_DEFAULT, optimize::Union{Nothing,Int} = OPTIMIZE_DEFAULT, min_optlevel::Union{Nothing,Int} = MIN_OPTLEVEL_DEFAULT, inline::Union{Nothing,String} = INLINE_DEFAULT, check_bounds::Union{Nothing,String} = CHECK_BOUNDS_DEFAULT, math_mode::Union{Nothing,String} = MATH_MODE_DEFAULT, heap_size_hint::Union{Nothing,String} = HEAP_SIZE_HINT_DEFAULT, startup_file::Union{Nothing,String} = STARTUP_FILE_DEFAULT, history_file::Union{Nothing,String} = HISTORY_FILE_DEFAULT, threads::Union{Nothing,String,Int} = default_number_of_threads(), ) server = ServerOptions(; root_url, base_url, host, port, port_hint, launch_browser, dismiss_update_notification, dismiss_motivational_quotes, show_file_system, enable_ai_editor_features, notebook_path_suggestion, disable_writing_notebook_files, auto_reload_from_file, auto_reload_from_file_cooldown, auto_reload_from_file_ignore_pkg, notebook, simulated_lag, simulated_pkg_lag, injected_javascript_data_url, on_event, ) security = SecurityOptions(; require_secret_for_open_links, require_secret_for_access, warn_about_untrusted_code, ) evaluation = EvaluationOptions(; run_notebook_on_load, workspace_use_distributed, workspace_use_distributed_stdlib, lazy_workspace_creation, capture_stdout, workspace_custom_startup_expr, ) compiler = CompilerOptions(; compile, pkgimages, compiled_modules, sysimage, sysimage_native_code, banner, depwarn, optimize, min_optlevel, inline, check_bounds, math_mode, heap_size_hint, startup_file, history_file, threads, ) return Options(; server, security, evaluation, compiler) end function _merge_notebook_compiler_options(notebook, options::CompilerOptions)::CompilerOptions if notebook.compiler_options === nothing return options end kwargs = Dict{Symbol,Any}() for each in fieldnames(CompilerOptions) # 1. not specified by notebook options # 2. general notebook specified options if getfield(notebook.compiler_options, each) === nothing kwargs[each] = getfield(options, each) else kwargs[each] = getfield(notebook.compiler_options, each) end end return CompilerOptions(; kwargs...) end function _convert_to_flags(options::CompilerOptions)::Vector{String} option_list = String[] for name in fieldnames(CompilerOptions) flagname = string("--", replace(String(name), "_" => "-")) value = getfield(options, name) if value !== nothing push!(option_list, string(flagname, "=", value)) end end return option_list end end """ Start a notebook server using: ```julia julia> Pluto.run() ``` Have a look at the FAQ: https://github.com/fonsp/Pluto.jl/wiki """ module Pluto if isdefined(Base, :Experimental) && isdefined(Base.Experimental, Symbol("@max_methods")) @eval Base.Experimental.@max_methods 1 end import Markdown import RelocatableFolders: @path const ROOT_DIR = normpath(joinpath(@__DIR__, "..")) const FRONTEND_DIR = @path(joinpath(ROOT_DIR, "frontend")) const FRONTEND_DIST_DIR = let dir = joinpath(ROOT_DIR, "frontend-dist") isdir(dir) ? @path(dir) : FRONTEND_DIR end const frontend_dist_exists = FRONTEND_DIR !== FRONTEND_DIST_DIR const SAMPLE_DIR = @path(joinpath(ROOT_DIR, "sample")) const RUNNER_DIR = @path(joinpath(ROOT_DIR, "src", "runner")) function project_relative_path(root, xs...) root == joinpath("src", "runner") ? joinpath(RUNNER_DIR, xs...) : root == "frontend-dist" && frontend_dist_exists ? joinpath(FRONTEND_DIST_DIR, xs...) : root == "frontend" ? joinpath(FRONTEND_DIR, xs...) : root == "sample" ? joinpath(SAMPLE_DIR, xs...) : normpath(joinpath(pkgdir(Pluto), root, xs...)) end import Pkg import Scratch include_dependency("../Project.toml") const PLUTO_VERSION = pkgversion(@__MODULE__) const PLUTO_VERSION_STR = "v$(string(PLUTO_VERSION))" const JULIA_VERSION_STR = "v$(string(VERSION))" import PlutoDependencyExplorer: PlutoDependencyExplorer, TopologicalOrder, NotebookTopology, ExprAnalysisCache, ImmutableVector, ExpressionExplorerExtras, topological_order, all_cells, disjoint, where_assigned, where_referenced using ExpressionExplorer include("./notebook/path helpers.jl") include("./notebook/Export.jl") include("./Configuration.jl") include("./evaluation/Tokens.jl") include("./evaluation/Throttled.jl") include("./runner/PlutoRunner/src/PlutoRunner.jl") include("./packages/PkgCompat.jl") include("./webserver/Status.jl") include("./notebook/Cell.jl") include("./notebook/Notebook.jl") include("./notebook/saving and loading.jl") include("./notebook/frontmatter.jl") include("./notebook/Events.jl") include("./webserver/Session.jl") include("./webserver/PutUpdates.jl") include("./analysis/Parse.jl") include("./analysis/is_just_text.jl") include("./analysis/DependencyCache.jl") include("./analysis/MoreAnalysis.jl") include("./evaluation/WorkspaceManager.jl") include("./evaluation/MacroAnalysis.jl") include("./packages/IOListener.jl") include("./packages/precompile_isolated.jl") include("./packages/Packages.jl") include("./packages/PkgUtils.jl") include("./evaluation/Run.jl") include("./evaluation/RunBonds.jl") module DownloadCool include("./webserver/data_url.jl") end include("./webserver/MsgPack.jl") include("./webserver/SessionActions.jl") include("./webserver/Static.jl") include("./webserver/Authentication.jl") include("./webserver/Router.jl") include("./webserver/Dynamic.jl") include("./webserver/REPLTools.jl") include("./webserver/WebServer.jl") const reset_notebook_environment = PkgUtils.reset_notebook_environment const update_notebook_environment = PkgUtils.update_notebook_environment const activate_notebook_environment = PkgUtils.activate_notebook_environment const will_use_pluto_pkg = PkgUtils.will_use_pluto_pkg export reset_notebook_environment export update_notebook_environment export activate_notebook_environment export will_use_pluto_pkg include("./precompile.jl") const pluto_boot_environment_path = Ref{String}() function julia_compat_issue(short::String) if short == "1.12" "Check [https://github.com/fonsp/Pluto.jl/issues/3005](https://github.com/fonsp/Pluto.jl/issues/3005)" else "Search [github.com/fonsp/Pluto.jl/issues](https://github.com/fonsp/Pluto.jl/issues) for `Julia $short`" end end function warn_julia_compat() if VERSION > v"1.11.9999" short = "$(VERSION.major).$(VERSION.minor)" msg = "# WARNING: Unsupported Julia version\nPluto (`$(PLUTO_VERSION)`) is running on a new version of Julia (`$(VERSION)`). Support for Julia $short will be added in a later Pluto release.\n\nYou can try:\n$( " 1. Update Pluto using `Pkg.update(\"Pluto\")`.\n" * " 1. If there is no newer version of Pluto yet, then you can **help us develop it**! _Julia $short compatibility takes a lot of work, and we would really appreciate your help! $(julia_compat_issue(short)) to see what still needs to be done. Not all compatibility issues are known  play around and try running `Pkg.test(\"Pluto\")`_.\n\n")$( VERSION.prerelease === () && VERSION.build === () ? "" : # if using a build/prerelease, then the user is using a future Julia version that we don't support yet. "!!! note\n\tPreview versions of Julia are not fully supported by Pluto.\n\n" )" println() println() display(Markdown.parse(msg)) println() println() end bad_depots = filter(d -> !isabspath(expanduser(d)), DEPOT_PATH) if !isempty(bad_depots) @error """Pluto: The provided depot path is not an absolute path. Pluto will not be able to run correctly. Did you recently change the DEPOT path setting? Change your setting to use an absolute path. Do you not know what this means? Please get in touch! https://github.com/fonsp/Pluto.jl/issues """ bad_depots DEPOT_PATH end end function __init__() pluto_boot_environment_name = "pluto-boot-environment-$(VERSION)-$(PLUTO_VERSION)" pluto_boot_environment_path[] = Scratch.@get_scratch!(pluto_boot_environment_name) # Print a welcome banner if (get(ENV, "JULIA_PLUTO_SHOW_BANNER", "1") != "0" && get(ENV, "CI", "") != "true" && isinteractive()) # Print the banner only once per version, if there isn't # yet a file for this version in banner_shown scratch space. # (Using the Pluto version as the filename enables later # version-specific "what's new" messages.) fn = joinpath(Scratch.@get_scratch!("banner_shown"), PLUTO_VERSION_STR) if !isfile(fn) @info """ Welcome to Pluto $(PLUTO_VERSION_STR)  Start a notebook server using: julia> Pluto.run() Have a look at the FAQ: https://github.com/fonsp/Pluto.jl/wiki """ # create empty file to indicate that we've shown the banner write(fn, ""); end end warn_julia_compat() end end using PrecompileTools: PrecompileTools PrecompileTools.@compile_workload begin nb = Pluto.Notebook([ Pluto.Cell("""md"Hello *world*" """) Pluto.Cell("""[f(x)]""") Pluto.Cell("""x = 1""") Pluto.Cell( """ function f(z::Integer) z / 123 end """) Pluto.Cell( """ "asdf" begin while false local p = 123 try [(x,a...) for x in (a for a in b)] A.B.hello() do z @gensym z (z) -> z/:(z / z) end catch end end end """ ) ]) let topology = Pluto.updated_topology(nb.topology, nb, nb.cells) # Our reactive sorting algorithm. Pluto.topological_order(topology, topology.cell_order) end # let # io = IOBuffer() # # Notebook file format. # Pluto.save_notebook(io, nb) # seekstart(io) # Pluto.load_notebook_nobackup(io, "whatever.jl") # end let state1 = Pluto.notebook_to_js(nb) state2 = Pluto.notebook_to_js(nb) # MsgPack Pluto.unpack(Pluto.pack(state1)) # State diffing Pluto.Firebasey.diff(state1, state2) end s = Pluto.ServerSession(; options=Pluto.Configuration.from_flat_kwargs( disable_writing_notebook_files=true, workspace_use_distributed=false, auto_reload_from_file=false, run_notebook_on_load=false, lazy_workspace_creation=true, capture_stdout=false, ) ) end using PrecompileSignatures: @precompile_signatures @precompile_signatures(Pluto) import UUIDs: UUID """ Gets a dictionary of all symbols and the respective cells which are dependent on the given cell. Changes in the given cell cause re-evaluation of these cells. Note that only direct dependents are given here, not indirect dependents. """ function downstream_cells_map(cell::Cell, topology::NotebookTopology)::Dict{Symbol,Vector{Cell}} defined_symbols = let node = topology.nodes[cell] node.definitions  Iterators.filter(!_is_anon_function_name, node.funcdefs_without_signatures) end return Dict{Symbol,Vector{Cell}}( sym => PlutoDependencyExplorer.where_referenced(topology, Set([sym])) for sym in defined_symbols ) end _is_anon_function_name(s::Symbol) = startswith(String(s), "__ExprExpl_anon__") """ Gets a dictionary of all symbols and the respective cells on which the given cell depends. Changes in these cells cause re-evaluation of the given cell. Note that only direct dependencies are given here, not indirect dependencies. """ function upstream_cells_map(cell::Cell, topology::NotebookTopology)::Dict{Symbol,Vector{Cell}} referenced_symbols = topology.nodes[cell].references return Dict{Symbol,Vector{Cell}}( sym => PlutoDependencyExplorer.where_assigned(topology, Set([sym])) for sym in referenced_symbols ) end "Fills cell dependency information for display in the GUI" function update_dependency_cache!(cell::Cell, topology::NotebookTopology) cell.cell_dependencies = CellDependencies( downstream_cells_map(cell, topology), upstream_cells_map(cell, topology), PlutoDependencyExplorer.cell_precedence_heuristic(topology, cell), ) end "Fills dependency information on notebook and cell level." function update_dependency_cache!(notebook::Notebook, topology::NotebookTopology=notebook.topology) notebook._cached_topological_order = topological_order(notebook) if notebook._cached_cell_dependencies_source !== topology notebook._cached_cell_dependencies_source = topology for cell in all_cells(topology) update_dependency_cache!(cell, topology) end notebook._cached_cell_dependencies = Dict{UUID,Dict{String,Any}}( id => Dict{String,Any}( "cell_id" => cell.cell_id, "downstream_cells_map" => Dict{String,Vector{UUID}}( String(s) => cell_id.(r) for (s, r) in cell.cell_dependencies.downstream_cells_map ), "upstream_cells_map" => Dict{String,Vector{UUID}}( String(s) => cell_id.(r) for (s, r) in cell.cell_dependencies.upstream_cells_map ), "precedence_heuristic" => cell.cell_dependencies.precedence_heuristic, ) for (id, cell) in notebook.cells_dict ) end end module MoreAnalysis export bound_variable_connections_graph import ..Pluto import ..Pluto: Cell, Notebook, NotebookTopology, ExpressionExplorer, ExpressionExplorerExtras, PlutoDependencyExplorer import PlutoDependencyExplorer: all_cells "Return whether any cell references the given symbol. Used for the @bind mechanism." function is_referenced_anywhere(topology::NotebookTopology, sym::Symbol)::Bool any(all_cells(topology)) do cell sym  topology.nodes[cell].references end end @deprecate is_referenced_anywhere(notebook::Notebook, topology::NotebookTopology, sym::Symbol) is_referenced_anywhere(topology, sym) "Return whether any cell defines the given symbol. Used for the @bind mechanism." function is_assigned_anywhere(topology::NotebookTopology, sym::Symbol)::Bool any(all_cells(topology)) do cell sym  topology.nodes[cell].definitions end end @deprecate is_assigned_anywhere(notebook::Notebook, topology::NotebookTopology, sym::Symbol) is_assigned_anywhere(topology, sym) "Find all subexpressions of the form `@bind symbol something`, and extract the `symbol`s." function find_bound_variables(expr) found = Set{Symbol}() _find_bound_variables!(found, ExpressionExplorerExtras.maybe_macroexpand_pluto(expr; recursive=true, expand_bind=false)) found end function _find_bound_variables!(found::Set{Symbol}, expr::Expr) if expr.head === :macrocall && expr.args[1] === Symbol("@bind") && length(expr.args) == 4 && expr.args[3] isa Symbol push!(found, expr.args[3]) _find_bound_variables!(found, expr.args[4]) elseif expr.args === :quote found else for a in expr.args _find_bound_variables!(found, a) end end end function _find_bound_variables!(found::Set{Symbol}, expr::Any) end "Return the given cells, and all cells that depend on them (recursively)." function downstream_recursive( topology::NotebookTopology, from::Union{Vector{Cell},Set{Cell}}, )::Set{Cell} found = Set{Cell}(empty(from)) _downstream_recursive!(found, topology, from) found end @deprecate downstream_recursive( notebook::Notebook, topology::NotebookTopology, from::Union{Vector{Cell},Set{Cell}}, ) downstream_recursive( topology, from, ) function _downstream_recursive!( found::Set{Cell}, topology::NotebookTopology, from::Vector{Cell}, )::Nothing for cell in from one_down = PlutoDependencyExplorer.where_referenced(topology, cell) for next in one_down if next  found push!(found, next) _downstream_recursive!(found, topology, Cell[next]) end end end end "Return all cells that are depended upon by any of the given cells." function upstream_recursive( topology::NotebookTopology, from::Union{Vector{Cell},Set{Cell}}, )::Set{Cell} found = Set{Cell}(empty(from)) _upstream_recursive!(found, topology, from) found end @deprecate upstream_recursive( notebook::Notebook, topology::NotebookTopology, from::Union{Vector{Cell},Set{Cell}}, ) upstream_recursive( topology, from, ) function _upstream_recursive!( found::Set{Cell}, topology::NotebookTopology, from::Vector{Cell}, )::Nothing for cell in from references = topology.nodes[cell].references for upstream in PlutoDependencyExplorer.where_assigned(topology, references) if upstream  found push!(found, upstream) _upstream_recursive!(found, topology, Cell[upstream]) end end end end "All cells that can affect the outcome of changing the given variable." function codependents(topology::NotebookTopology, var::Symbol)::Set{Cell} assigned_in = filter(all_cells(topology)) do cell var  topology.nodes[cell].definitions end downstream = collect(union!(downstream_recursive(topology, assigned_in), assigned_in)) downupstream = union!(upstream_recursive(topology, downstream), assigned_in) end @deprecate codependents(notebook::Notebook, topology::NotebookTopology, var::Symbol) codependents(topology, var) "Return a `Dict{Symbol,Vector{Symbol}}` where the _keys_ are the bound variables of the notebook. For each key (a bound symbol), the value is the list of (other) bound variables whose values need to be known to compute the result of setting the bond." function bound_variable_connections_graph(topology::NotebookTopology)::Dict{Symbol,Vector{Symbol}} bound_variables = union(map(all_cells(topology)) do cell find_bound_variables(topology.codes[cell].parsedcode) end...) Dict{Symbol,Vector{Symbol}}( var => let cells = codependents(topology, var) defined_there = union!(Set{Symbol}(), (topology.nodes[c].definitions for c in cells)...) # Set([var])  collect((defined_there  bound_variables)) end for var in bound_variables ) end @deprecate bound_variable_connections_graph(notebook::Notebook) bound_variable_connections_graph(notebook.topology) end # This is how we go from a String of cell code to a Julia `Expr` that can be executed. import ExpressionExplorer import Markdown "Generate a file name to be given to the parser (will show up in stack traces)." pluto_filename(notebook::Notebook, cell::Cell)::String = notebook.path * "#==#" * string(cell.cell_id) "Is Julia new enough to support filenames in parsing?" const can_insert_filename = (Base.parse_input_line("1;2") != Base.parse_input_line("1\n2")) """ Parse the code from `cell.code` into a Julia expression (`Expr`). Equivalent to `Meta.parse_input_line` in Julia v1.3, no matter the actual Julia version. 1. Turn multiple expressions into an error expression. 2. Fix some `LineNumberNode` idiosyncrasies to be more like modern Julia. 3. Will always produce an expression of the form: `Expr(:toplevel, LineNumberNode(..), root)`. It gets transformed (i.e. wrapped) into this form if needed. A `LineNumberNode` contains a line number and a file name. We use the cell UUID as a 'file name', which makes the stack traces easier to interpret. (Otherwise it would be impossible to tell from which cell a stack frame originates.) Not all Julia versions insert these `LineNumberNode`s, so we insert it ourselves if Julia doesn't. 4. Apply `preprocess_expr` (below) to `root` (from rule 2). """ function parse_custom(notebook::Notebook, cell::Cell)::Expr # 1. raw = if can_insert_filename filename = pluto_filename(notebook, cell) ex = Base.parse_input_line(cell.code, filename=filename) if Meta.isexpr(ex, :toplevel) # if there is more than one expression: if count(a -> !(a isa LineNumberNode), ex.args) > 1 Expr(:error, "extra token after end of expression\n\nBoundaries: $(expression_boundaries(cell.code))") else ex end else ex end else # Meta.parse returns the "extra token..." like we want, but also in cases like "\n\nx = 1\n# comment", so we need to do the multiple expressions check ourselves after all parsed1, next_ind1 = Meta.parse(cell.code, 1, raise=false) parsed2, next_ind2 = Meta.parse(cell.code, next_ind1, raise=false) if parsed2 === nothing # only whitespace or comments after the first expression parsed1 else Expr(:error, "extra token after end of expression\n\nBoundaries: $(expression_boundaries(cell.code))") end end # 2. filename = pluto_filename(notebook, cell) if !can_insert_filename fix_linenumbernodes!(raw, filename) end # 3. topleveled = if ExpressionExplorer.is_toplevel_expr(raw) raw else Expr(:toplevel, LineNumberNode(1, Symbol(filename)), raw) end # 4. Expr(topleveled.head, topleveled.args[1], preprocess_expr(topleveled.args[2])) end "Old Julia versions insert some `LineNumberNode`s with `:none` as filename, which are useless and break stack traces, so we replace those." function fix_linenumbernodes!(ex::Expr, actual_filename) for (i, a) in enumerate(ex.args) if a isa Expr fix_linenumbernodes!(a, actual_filename) elseif a isa LineNumberNode if a.file === nothing || a.file == :none ex.args[i] = LineNumberNode(a.line, Symbol(actual_filename)) end end end end fix_linenumbernodes!(::Any, actual_filename) = nothing """Get the list of string indices that denote expression boundaries. # Examples `expression_boundaries("sqrt(1)") == [ncodeunits("sqrt(1)") + 1]` `expression_boundaries("sqrt(1)\n\n123") == [ncodeunits("sqrt(1)\n\n") + 1, ncodeunits("sqrt(1)\n\n123") + 1]` """ function expression_boundaries(code::String, start=1)::Vector{<:Integer} expr, next = Meta.parse(code, start, raise=false) if next <= ncodeunits(code) [next, expression_boundaries(code, next)...] else [next] end end """ Make some small adjustments to the `expr` to make it work nicely inside a timed, wrapped expression: 1. If `expr` is a `:toplevel` expression (this is the case iff the expression is a combination of expressions using semicolons, like `a = 1; b` or `123;`), then it gets turned into a `:block` expression. The reason for this transformation is that `:toplevel` does not return/relay the output of its last argument, unlike `begin`, `let`, `if`, etc. (But we want it!) 2. If `expr` is a `:module` expression, wrap it in a `:toplevel` block - module creation needs to be at toplevel. Rule 1. is not applied. 3. If `expr` is a `:(=)` expression with a curly assignment, wrap it in a `:const` to allow execution - see https://github.com/fonsp/Pluto.jl/issues/517 4. If `expr` failed to parse, it has head in (:incomplete, :error) and is replaced with a call to PlutoRunner.throw_syntax_error which will render diagnostics in a frontend compatible manner (Pluto.jl#2526). """ function preprocess_expr(expr::Expr) if expr.head === :toplevel Expr(:block, expr.args...) elseif expr.head === :module Expr(:toplevel, expr) elseif expr.head === :(=) && (expr.args[1] isa Expr && expr.args[1].head == :curly) Expr(:const, expr) elseif expr.head === :incomplete || expr.head === :error Expr(:call, :(PlutoRunner.throw_syntax_error), expr.args...) else expr end end # for expressions that are just values, like :(1) or :(x) preprocess_expr(val::Any) = val """ Does this `String` contain a single expression? If this function returns `false`, then Pluto will show a "multiple expressions in one cell" error in the editor. !!! compat "Pluto 0.20.5" This function is new in Pluto 0.20.5. """ function is_single_expression(s::String) n = Pluto.Notebook([Pluto.Cell(s)]) e = parse_custom(n, n.cells[1]) bad = Meta.isexpr(e, :toplevel, 2) && Meta.isexpr(e.args[2], :call, 2) && e.args[2].args[1] == :(PlutoRunner.throw_syntax_error) && e.args[2].args[2] isa String && startswith(e.args[2].args[2], "extra token after end of expression") return !bad end function updated_topology(old_topology::NotebookTopology{Cell}, notebook::Notebook, updated_cells) get_code_str(cell::Cell) = cell.code get_code_expr(cell::Cell) = parse_custom(notebook, cell) PlutoDependencyExplorer.updated_topology( old_topology, notebook.cells, updated_cells; get_code_str, get_code_expr, get_cell_disabled=is_disabled, ) end const md_and_friends = [ # Text Symbol("@md_str"), Symbol("@html_str"), :getindex, ] """Does the cell only contain md"..." and html"..."? This is used to run these cells first.""" function is_just_text(topology::NotebookTopology, cell::Cell)::Bool # https://github.com/fonsp/Pluto.jl/issues/209 node = topology.nodes[cell] ((isempty(node.definitions) && isempty(node.funcdefs_with_signatures) && node.references  md_and_friends) || (length(node.references) == 2 && :PlutoRunner in node.references && Symbol("PlutoRunner.throw_syntax_error") in node.references)) && no_loops(ExpressionExplorerExtras.maybe_macroexpand_pluto(topology.codes[cell].parsedcode; recursive=true)) end function no_loops(ex::Expr) if ex.head === :while || ex.head === :for || ex.head === :comprehension || ex.head === :generator || ex.head === :try || ex.head === :quote || ex.head === :module false else all(no_loops, ex.args) end end no_loops(x) = true # Macro Analysis & Topology Resolution (see https://github.com/fonsp/Pluto.jl/pull/1032) import .WorkspaceManager: macroexpand_in_workspace const lazymap = Base.Generator function defined_variables(topology::NotebookTopology, cells) lazymap(cells) do cell topology.nodes[cell].definitions end end function defined_functions(topology::NotebookTopology, cells) lazymap(cells) do cell ((cell.cell_id, namesig.name) for namesig in topology.nodes[cell].funcdefs_with_signatures) end end is_macro_identifier(symbol::Symbol) = startswith(string(symbol), "@") function with_new_soft_definitions(topology::NotebookTopology, cell::Cell, soft_definitions) old_node = topology.nodes[cell] new_node = union!(ReactiveNode(), old_node, ReactiveNode(soft_definitions=soft_definitions)) NotebookTopology( codes=topology.codes, nodes=merge(topology.nodes, Dict(cell => new_node)), unresolved_cells=topology.unresolved_cells, cell_order=topology.cell_order, disabled_cells=topology.disabled_cells, ) end collect_implicit_usings(topology::NotebookTopology, cell::Cell) = ExpressionExplorerExtras.collect_implicit_usings(topology.codes[cell].module_usings_imports) function cells_with_deleted_macros(old_topology::NotebookTopology, new_topology::NotebookTopology) old_macros = mapreduce(c -> defined_macros(old_topology, c), union!, all_cells(old_topology); init=Set{Symbol}()) new_macros = mapreduce(c -> defined_macros(new_topology, c), union!, all_cells(new_topology); init=Set{Symbol}()) removed_macros = setdiff(old_macros, new_macros) PlutoDependencyExplorer.where_referenced(old_topology, removed_macros) end "Returns the set of macros names defined by this cell" defined_macros(topology::NotebookTopology, cell::Cell) = defined_macros(topology.nodes[cell]) defined_macros(node::ReactiveNode) = union!(filter(is_macro_identifier, node.funcdefs_without_signatures), filter(is_macro_identifier, node.definitions)) # macro definitions can come from imports "Tells whether or not a cell can 'unlock' the resolution of other cells" function can_help_resolve_cells(topology::NotebookTopology, cell::Cell) cell_code = topology.codes[cell] cell_node = topology.nodes[cell] macros = defined_macros(cell_node) !isempty(cell_code.module_usings_imports.usings) || (!isempty(macros) && any(calls -> !disjoint(calls, macros), topology.nodes[c].macrocalls for c in topology.unresolved_cells)) end # Sorry couldn't help myself - DRAL abstract type Result end struct Success <: Result result end struct Failure <: Result error end struct Skipped <: Result end """We still have 'unresolved' macrocalls, use the current and maybe previous workspace to do macro-expansions. You can optionally specify the roots for the current reactive run. If a cell macro contains only macros that will be re-defined during this reactive run, we don't expand yet and expect the `can_help_resolve_cells` function above to be true for the cell defining the macro, triggering a new topology resolution without needing to fallback to the previous workspace. """ function resolve_topology( session::ServerSession, notebook::Notebook, unresolved_topology::NotebookTopology, old_workspace_name::Symbol; current_roots::Vector{Cell}=Cell[], )::NotebookTopology sn = (session, notebook) function macroexpand_cell(cell) function try_macroexpand(module_name::Symbol=Symbol("")) success, result = macroexpand_in_workspace(sn, unresolved_topology.codes[cell].parsedcode, cell.cell_id, module_name) if success Success(result) else Failure(result) end end result = try_macroexpand() if result isa Success result else if (result.error isa LoadError && result.error.error isa UndefVarError) || result.error isa UndefVarError try_macroexpand(old_workspace_name) else result end end end function analyze_macrocell(cell::Cell) if unresolved_topology.nodes[cell].macrocalls  ExpressionExplorerExtras.can_macroexpand return Skipped() end result = macroexpand_cell(cell) if result isa Success (expr, computer_id) = result.result expanded_node = ExpressionExplorer.compute_reactive_node(ExpressionExplorerExtras.pretransform_pluto(expr)) function_wrapped = ExpressionExplorerExtras.can_be_function_wrapped(expr) Success((expanded_node, function_wrapped, computer_id)) else result end end run_defined_macros = mapreduce(c -> defined_macros(unresolved_topology, c), union!, current_roots; init=Set{Symbol}()) # create new node & new codes for macrocalled cells new_nodes = Dict{Cell,ReactiveNode}() new_codes = Dict{Cell,ExprAnalysisCache}() still_unresolved_nodes = Set{Cell}() for cell in unresolved_topology.unresolved_cells if unresolved_topology.nodes[cell].macrocalls  run_defined_macros # Do not try to expand if a newer version of the macro is also scheduled to run in the # current run. The recursive reactive runs will take care of it. push!(still_unresolved_nodes, cell) continue end result = try if will_run_code(notebook) analyze_macrocell(cell) else Failure(ErrorException("shutdown")) end catch error @error "Macro call expansion failed with a non-macroexpand error" exception=(error,catch_backtrace()) cell.code Failure(error) end if result isa Success (new_node, function_wrapped, forced_expr_id) = result.result union!(new_node.macrocalls, unresolved_topology.nodes[cell].macrocalls) union!(new_node.references, new_node.macrocalls) new_nodes[cell] = new_node # set function_wrapped to the function wrapped analysis of the expanded expression. new_codes[cell] = ExprAnalysisCache(unresolved_topology.codes[cell]; forced_expr_id, function_wrapped) elseif result isa Skipped # Skipped because it has already been resolved during ExpressionExplorer. else @debug "Could not resolve" result cell.code push!(still_unresolved_nodes, cell) end end all_nodes = merge(unresolved_topology.nodes, new_nodes) all_codes = merge(unresolved_topology.codes, new_codes) new_unresolved_cells = if length(still_unresolved_nodes) == length(unresolved_topology.unresolved_cells) # then they must equal, and we can skip creating a new one to preserve identity: unresolved_topology.unresolved_cells else PlutoDependencyExplorer.ImmutableSet(still_unresolved_nodes; skip_copy=true) end NotebookTopology(; nodes=all_nodes, codes=all_codes, unresolved_cells=new_unresolved_cells, cell_order=unresolved_topology.cell_order, disabled_cells=unresolved_topology.disabled_cells, ) end """Tries to add information about macro calls without running any code, using knowledge about common macros. So, the resulting reactive nodes may not be absolutely accurate. If you can run code in a session, use `resolve_topology` instead. """ function static_macroexpand(topology::NotebookTopology, cell::Cell) new_node = ExpressionExplorer.compute_reactive_node( ExpressionExplorerExtras.pretransform_pluto( ExpressionExplorerExtras.maybe_macroexpand_pluto( topology.codes[cell].parsedcode; recursive=true ) ) ) union!(new_node.macrocalls, topology.nodes[cell].macrocalls) new_node end "The same as `resolve_topology` but does not require custom code execution, only works with a few `Base` & `PlutoRunner` macros" function static_resolve_topology(topology::NotebookTopology) new_nodes = Dict{Cell,ReactiveNode}(cell => static_macroexpand(topology, cell) for cell in topology.unresolved_cells) all_nodes = merge(topology.nodes, new_nodes) NotebookTopology( nodes=all_nodes, codes=topology.codes, unresolved_cells=topology.unresolved_cells, cell_order=topology.cell_order, disabled_cells=topology.disabled_cells, ) end import REPL: ends_with_semicolon import .Configuration import .Throttled import ExpressionExplorer: is_joined_funcname import UUIDs: UUID """ Run given cells and all the cells that depend on them, based on the topology information before and after the changes. """ function run_reactive!( session::ServerSession, notebook::Notebook, old_topology::NotebookTopology, new_topology::NotebookTopology, roots::Vector{Cell}; save::Bool=true, deletion_hook::Function = WorkspaceManager.move_vars, user_requested_run::Bool = true, bond_value_pairs=zip(Symbol[],Any[]), )::TopologicalOrder topological_order = withtoken(notebook.executetoken) do run_reactive_core!( session, notebook, old_topology, new_topology, roots; save, deletion_hook, user_requested_run, bond_value_pairs, ) end try_event_call(session, NotebookExecutionDoneEvent(notebook, user_requested_run)) return topological_order end """ Run given cells and all the cells that depend on them, based on the topology information before and after the changes. !!! warning You should probably not call this directly and use `run_reactive!` instead. """ function run_reactive_core!( session::ServerSession, notebook::Notebook, old_topology::NotebookTopology, new_topology::NotebookTopology, roots::Vector{Cell}; save::Bool=true, deletion_hook::Function = WorkspaceManager.move_vars, user_requested_run::Bool = true, already_run::Vector{Cell} = Cell[], bond_value_pairs=zip(Symbol[],Any[]), )::TopologicalOrder @assert !isready(notebook.executetoken) "run_reactive_core!() was called with a free notebook.executetoken." @assert will_run_code(notebook) old_workspace_name, _ = WorkspaceManager.bump_workspace_module((session, notebook)) # A state sync will come soon from this function, so let's delay anything coming from the status_tree listener, see https://github.com/fonsp/Pluto.jl/issues/2978 Throttled.force_throttle_without_run(notebook.status_tree.update_listener_ref[]) run_status = Status.report_business_started!(notebook.status_tree, :run) Status.report_business_started!(run_status, :resolve_topology) cell_status = Status.report_business_planned!(run_status, :evaluate) if !PlutoDependencyExplorer.is_resolved(new_topology) unresolved_topology = new_topology new_topology = notebook.topology = resolve_topology(session, notebook, unresolved_topology, old_workspace_name; current_roots = setdiff(roots, already_run)) # update cache and save notebook because the dependencies might have changed after expanding macros update_dependency_cache!(notebook) end # find (indirectly) skipped-as-script cells and update their status update_skipped_cells_dependency!(notebook, new_topology) removed_cells = setdiff(all_cells(old_topology), all_cells(new_topology)) roots = vcat(roots, removed_cells) # by setting the reactive node and expression caches of deleted cells to "empty", we are essentially pretending that those cells still exist, but now have empty code. this makes our algorithm simpler. new_topology = PlutoDependencyExplorer.exclude_roots(new_topology, removed_cells) # find (directly and indirectly) deactivated cells and update their status indirectly_deactivated = collect(topological_order_cached(new_topology, collect(new_topology.disabled_cells); allow_multiple_defs=true, skip_at_partial_multiple_defs=true)) for cell in indirectly_deactivated cell.running = false cell.queued = false cell.depends_on_disabled_cells = true end new_topology = PlutoDependencyExplorer.exclude_roots(new_topology, indirectly_deactivated) # save the old topological order - we'll delete variables assigned from its # and re-evalutate its cells unless the cells have already run previously in the reactive run old_order = topological_order_cached(old_topology, roots) old_runnable = setdiff(old_order.runnable, already_run) to_delete_vars = union!(Set{Symbol}(), defined_variables(old_topology, old_runnable)...) to_delete_funcs = union!(Set{Tuple{UUID,FunctionName}}(), defined_functions(old_topology, old_runnable)...) new_roots = setdiff(union(roots, keys(old_order.errable)), indirectly_deactivated) # get the new topological order new_order = topological_order_cached(new_topology, new_roots) new_runnable = setdiff(new_order.runnable, already_run) to_run = setdiff!( union(new_runnable, old_runnable), indirectly_deactivated, keys(new_order.errable) )::Vector{Cell} # TODO: think if old error cell order matters # change the bar on the sides of cells to "queued" for cell in to_run cell.queued = true cell.depends_on_disabled_cells = false end for (cell, error) in new_order.errable cell.running = false cell.queued = false cell.depends_on_disabled_cells = false relay_reactivity_error!(cell, error) end # Save the notebook. In most cases, this is the only time that we save the notebook, so any state changes that influence the file contents (like `depends_on_disabled_cells`) should be behind this point. (More saves might happen if a macro expansion or package using happens.) save && save_notebook(session, notebook) # Send intermediate updates to the clients at most 20 times / second during a reactive run. (The effective speed of a slider is still unbounded, because the last update is not throttled.) # flush_send_notebook_changes_throttled, send_notebook_changes_throttled = Throttled.throttled(1.0 / 20; runtime_multiplier=2.0) do # We will do a state sync now, so that means that we can delay the status_tree state sync loop, see https://github.com/fonsp/Pluto.jl/issues/2978 Throttled.force_throttle_without_run(notebook.status_tree.update_listener_ref[]) # State sync: send_notebook_changes!(ClientRequest(; session, notebook)) end send_notebook_changes_throttled() Status.report_business_finished!(run_status, :resolve_topology) Status.report_business_started!(cell_status) for i in eachindex(to_run) Status.report_business_planned!(cell_status, Symbol(i)) end # delete new variables that will be defined by a cell unless this cell has already run in the current reactive run to_delete_vars = union!(to_delete_vars, defined_variables(new_topology, new_runnable)...) to_delete_funcs = union!(to_delete_funcs, defined_functions(new_topology, new_runnable)...) # delete new variables in case a cell errors (then the later cells show an UndefVarError) new_errable = keys(new_order.errable) to_delete_vars = union!(to_delete_vars, defined_variables(new_topology, new_errable)...) to_delete_funcs = union!(to_delete_funcs, defined_functions(new_topology, new_errable)...) cells_to_macro_invalidate = Set{UUID}(c.cell_id for c in cells_with_deleted_macros(old_topology, new_topology)) cells_to_js_link_invalidate = Set{UUID}(c.cell_id for c in union!(Set{Cell}(), to_run, new_errable, indirectly_deactivated)) module_imports_to_move = reduce(all_cells(new_topology); init=Set{Expr}()) do module_imports_to_move, c c  to_run && return module_imports_to_move usings_imports = new_topology.codes[c].module_usings_imports for (using_, isglobal) in zip(usings_imports.usings, usings_imports.usings_isglobal) isglobal || continue push!(module_imports_to_move, using_) end module_imports_to_move end if will_run_code(notebook) to_delete_funcs_simple = Set{Tuple{UUID,Tuple{Vararg{Symbol}}}}((id, name.parts) for (id,name) in to_delete_funcs) deletion_hook((session, notebook), old_workspace_name, nothing, to_delete_vars, to_delete_funcs_simple, module_imports_to_move, cells_to_macro_invalidate, cells_to_js_link_invalidate; to_run) # `deletion_hook` defaults to `WorkspaceManager.move_vars` end foreach(v -> delete!(notebook.bonds, v), to_delete_vars) local any_interrupted = false for (i, cell) in enumerate(to_run) Status.report_business_started!(cell_status, Symbol(i)) cell.queued = false cell.running = true # Important to not use empty! here because AppendonlyMarker requires a new array identity. # Eventually we could even make AppendonlyArray to enforce this but idk if it's worth it. yadiyadi. cell.logs = Vector{Dict{String,Any}}() send_notebook_changes_throttled() if (skip = any_interrupted || notebook.wants_to_interrupt || !will_run_code(notebook)) relay_reactivity_error!(cell, InterruptException()) else run = run_single!( (session, notebook), cell, new_topology.nodes[cell], new_topology.codes[cell]; user_requested_run = (user_requested_run && cell  roots), capture_stdout = session.options.evaluation.capture_stdout, ) any_interrupted |= run.interrupted # Support one bond defining another when setting both simultaneously in PlutoSliderServer # https://github.com/fonsp/Pluto.jl/issues/1695 # set the redefined bound variables to their original value from the request defs = notebook.topology.nodes[cell].definitions set_bond_value_pairs!(session, notebook, Iterators.filter(((sym,val),) -> sym  defs, bond_value_pairs)) end cell.running = false Status.report_business_finished!(cell_status, Symbol(i), !skip && !run.errored) defined_macros_in_cell = defined_macros(new_topology, cell) |> Set{Symbol} # Also set unresolved the downstream cells using the defined macros if !isempty(defined_macros_in_cell) new_topology = PlutoDependencyExplorer.set_unresolved(new_topology, PlutoDependencyExplorer.where_referenced(new_topology, defined_macros_in_cell)) end implicit_usings = collect_implicit_usings(new_topology, cell) if !PlutoDependencyExplorer.is_resolved(new_topology) && can_help_resolve_cells(new_topology, cell) notebook.topology = new_new_topology = resolve_topology(session, notebook, new_topology, old_workspace_name) if !isempty(implicit_usings) new_soft_definitions = WorkspaceManager.collect_soft_definitions((session, notebook), implicit_usings) notebook.topology = new_new_topology = with_new_soft_definitions(new_new_topology, cell, new_soft_definitions) end # update cache and save notebook because the dependencies might have changed after expanding macros update_dependency_cache!(notebook) save && save_notebook(session, notebook) return run_reactive_core!(session, notebook, new_topology, new_new_topology, to_run; save, deletion_hook, user_requested_run, already_run = to_run[1:i]) elseif !isempty(implicit_usings) new_soft_definitions = WorkspaceManager.collect_soft_definitions((session, notebook), implicit_usings) notebook.topology = new_new_topology = with_new_soft_definitions(new_topology, cell, new_soft_definitions) # update cache and save notebook because the dependencies might have changed after expanding macros update_dependency_cache!(notebook) save && save_notebook(session, notebook) return run_reactive_core!(session, notebook, new_topology, new_new_topology, to_run; save, deletion_hook, user_requested_run, already_run = to_run[1:i]) end end notebook.wants_to_interrupt = false Status.report_business_finished!(run_status) flush(send_notebook_changes_throttled) return new_order end """ ```julia set_bond_value_pairs!(session::ServerSession, notebook::Notebook, bond_value_pairs::Vector{Tuple{Symbol, Any}}) ``` Given a list of tuples of the form `(bound variable name, (untransformed) value)`, assign each (transformed) value to the corresponding global bound variable in the notebook workspace. `bond_value_pairs` can also be an iterator. """ function set_bond_value_pairs!(session::ServerSession, notebook::Notebook, bond_value_pairs) for (bound_sym, new_value) in bond_value_pairs WorkspaceManager.eval_in_workspace((session, notebook), :($(bound_sym) = Main.PlutoRunner.transform_bond_value($(QuoteNode(bound_sym)), $(new_value)))) end end run_reactive_async!(session::ServerSession, notebook::Notebook, to_run::Vector{Cell}; kwargs...) = run_reactive_async!(session, notebook, notebook.topology, notebook.topology, to_run; kwargs...) function run_reactive_async!(session::ServerSession, notebook::Notebook, old::NotebookTopology, new::NotebookTopology, to_run::Vector{Cell}; run_async::Bool=true, kwargs...)::Union{Task,TopologicalOrder} maybe_async(run_async) do run_reactive!(session, notebook, old, new, to_run; kwargs...) end end function maybe_async(f::Function, async::Bool) run_task = @asynclog f() if async run_task else fetch(run_task) end end "Run a single cell non-reactively, set its output, return run information." function run_single!( session_notebook::Union{Tuple{ServerSession,Notebook},WorkspaceManager.Workspace}, cell::Cell, reactive_node::ReactiveNode, expr_cache::ExprAnalysisCache; user_requested_run::Bool=true, capture_stdout::Bool=true, ) run = WorkspaceManager.eval_format_fetch_in_workspace( session_notebook, expr_cache.parsedcode, cell.cell_id; ends_with_semicolon = ends_with_semicolon(cell.code), function_wrapped_info = expr_cache.function_wrapped ? (filter(!is_joined_funcname, reactive_node.references), reactive_node.definitions) : nothing, forced_expr_id = expr_cache.forced_expr_id, known_published_objects = collect(keys(cell.published_objects)), user_requested_run, capture_stdout, ) set_output!(cell, run, expr_cache; persist_js_state=!user_requested_run) if session_notebook isa Tuple && run.process_exited session_notebook[2].process_status = ProcessStatus.no_process end return run end function set_output!(cell::Cell, run, expr_cache::ExprAnalysisCache; persist_js_state::Bool=false) cell.output = CellOutput( body=run.output_formatted[1], mime=run.output_formatted[2], rootassignee=if ends_with_semicolon(expr_cache.code) nothing else try ExpressionExplorer.get_rootassignee(expr_cache.parsedcode) catch _ # @warn "Error in get_rootassignee" expr=expr_cache.parsedcode nothing end end, last_run_timestamp=time(), persist_js_state=persist_js_state, has_pluto_hook_features=run.has_pluto_hook_features, ) cell.published_objects = let old_published = cell.published_objects new_published = run.published_objects for (k,v) in old_published if haskey(new_published, k) new_published[k] = v end end new_published end cell.runtime = run.runtime cell.errored = run.errored cell.running = cell.queued = false end function clear_output!(cell::Cell) cell.output = CellOutput() cell.published_objects = Dict{String,Any}() cell.runtime = nothing cell.errored = false cell.running = cell.queued = false end "Send `error` to the frontend without backtrace. Runtime errors are handled by `WorkspaceManager.eval_format_fetch_in_workspace` - this function is for Reactivity errors." function relay_reactivity_error!(cell::Cell, error::Exception) body, mime = PlutoRunner.format_output(CapturedException(error, [])) cell.output = CellOutput( body=body, mime=mime, rootassignee=nothing, last_run_timestamp=time(), persist_js_state=false, ) cell.published_objects = Dict{String,Any}() cell.runtime = nothing cell.errored = true end will_run_code(notebook::Notebook) = notebook.process_status  (ProcessStatus.ready, ProcessStatus.starting) will_run_pkg(notebook::Notebook) = notebook.process_status !== ProcessStatus.waiting_for_permission "Do all the things!" function update_save_run!( session::ServerSession, notebook::Notebook, cells::Vector{Cell}; save::Bool=true, run_async::Bool=false, prerender_text::Bool=false, clear_not_prerenderable_cells::Bool=false, auto_solve_multiple_defs::Bool=false, on_auto_solve_multiple_defs::Function=identity, kwargs... ) old = notebook.topology new = notebook.topology = updated_topology(old, notebook, cells) # macros are not yet resolved # _assume `auto_solve_multiple_defs == false` if you want to skip some details_ if auto_solve_multiple_defs to_disable_dict = cells_to_disable_to_resolve_multiple_defs(old, new, cells) if !isempty(to_disable_dict) to_disable = keys(to_disable_dict) @debug "Using augmented topology" cell_id.(to_disable) foreach(c -> set_disabled(c, true), to_disable) cells = union(cells, to_disable) # need to update the topology because the topology also keeps track of disabled cells new = notebook.topology = updated_topology(new, notebook, to_disable) end on_auto_solve_multiple_defs(to_disable_dict) end update_dependency_cache!(notebook) save && save_notebook(session, notebook) # _assume `prerender_text == false` if you want to skip some details_ to_run_online = cells if prerender_text # this code block will run cells that only contain text offline, i.e. on the server process, before doing anything else # this makes the notebook load a lot faster - the front-end does not have to wait for each output, and perform costly reflows whenever one updates # "A Workspace on the main process, used to prerender markdown before starting a notebook process for speedy UI." original_pwd = try pwd(); catch; end offline_workspace = WorkspaceManager.make_workspace( ( ServerSession(), notebook, ), is_offline_renderer=true, ) to_run_offline = filter(c -> !c.running && is_just_text(new, c), cells) for cell in to_run_offline run_single!(offline_workspace, cell, new.nodes[cell], new.codes[cell]) end isnothing(original_pwd) || cd(original_pwd) to_run_online = setdiff(cells, to_run_offline) clear_not_prerenderable_cells && foreach(clear_output!, to_run_online) send_notebook_changes!(ClientRequest(; session, notebook)) end # this setting is not officially supported (default is `true`), so you can skip this block when reading the code if !session.options.evaluation.run_notebook_on_load && prerender_text # these cells do something like settings up an environment, we should always run them setup_cells = filter(notebook.cells) do c PlutoDependencyExplorer.cell_precedence_heuristic(notebook.topology, c) < DEFAULT_PRECEDENCE_HEURISTIC end # for the remaining cells, clear their topology info so that they won't run as dependencies old = notebook.topology to_remove = setdiff(to_run_online, setup_cells) notebook.topology = NotebookTopology( nodes=PlutoDependencyExplorer.setdiffkeys(old.nodes, to_remove), codes=PlutoDependencyExplorer.setdiffkeys(old.codes, to_remove), unresolved_cells=setdiff(old.unresolved_cells, to_remove), cell_order=old.cell_order, disabled_cells=setdiff(old.disabled_cells, to_remove), ) # and don't run them to_run_online = to_run_online  setup_cells end maybe_async(run_async) do topological_order = withtoken(notebook.executetoken) do run_code = !( isempty(to_run_online) && session.options.evaluation.lazy_workspace_creation ) && will_run_code(notebook) if run_code # this will trigger the notebook process to start. @async makes it run in the background, so that sync_nbpkg (below) can start running in parallel. # Some notes: # - @async is enough, we don't need multithreading because the notebook runs in a separate process. # - sync_nbpkg manages the notebook package environment using Pkg on this server process. This means that sync_nbpkg does not need the notebook process at all, and it can run in parallel, before it has even started. @async WorkspaceManager.get_workspace((session, notebook)) end if will_run_pkg(notebook) # downloading and precompiling packages from the General registry is also arbitrary code execution sync_nbpkg(session, notebook, old, new; save=(save && !session.options.server.disable_writing_notebook_files), take_token=false ) end if run_code # not async because that would be double async run_reactive_core!(session, notebook, old, new, to_run_online; save, kwargs...) # run_reactive_async!(session, notebook, old, new, to_run_online; deletion_hook=deletion_hook, run_async=false, kwargs...) end end try_event_call( session, NotebookExecutionDoneEvent(notebook, get(kwargs, :user_requested_run, true)) ) topological_order end end update_save_run!(session::ServerSession, notebook::Notebook, cell::Cell; kwargs...) = update_save_run!(session, notebook, [cell]; kwargs...) update_run!(args...; kwargs...) = update_save_run!(args...; save=false, kwargs...) function cells_to_disable_to_resolve_multiple_defs(old::NotebookTopology, new::NotebookTopology, cells::Vector{Cell})::Dict{Cell,Any} # keys are cells to disable # values are the reason why to_disable_and_why = Dict{Cell,Any}() for cell in cells new_node = new.nodes[cell] fellow_assigners_old = filter!(c -> !PlutoDependencyExplorer.is_disabled(old, c), PlutoDependencyExplorer.where_assigned(old, new_node)) fellow_assigners_new = filter!(c -> !PlutoDependencyExplorer.is_disabled(new, c), PlutoDependencyExplorer.where_assigned(new, new_node)) if length(fellow_assigners_new) > length(fellow_assigners_old) other_definers = setdiff(fellow_assigners_new, (cell,)) # we want cell to be the only element of cells that defines this varialbe, i.e. all other definers must have been created previously if disjoint(cells, other_definers) # all fellow cells (including the current cell) should meet some criteria: all_fellows_are_simple_enough = all(fellow_assigners_new) do c node = new.nodes[c] # all must be true: return ( length(node.definitions) == 1 && # for more than one defined variable, we might confuse the user, or disable more things than we want to. disjoint(node.references, node.definitions) && # avoid self-reference like `x = x + 1` isempty(node.funcdefs_without_signatures) && node.macrocalls  (Symbol("@bind"),) # allow no macros (except for `@bind`) ) end if all_fellows_are_simple_enough for c in other_definers # if the cell is already disabled (indirectly), then we don't need to disable it. probably. if !c.depends_on_disabled_cells to_disable_and_why[c] = (cell_id(cell), only(new.nodes[c].definitions)) end end end end end end to_disable_and_why end function notebook_differences(from::Notebook, to::Notebook) from_cells = from.cells_dict to_cells = to.cells_dict ( # it's like D3 joins: https://observablehq.com/@d3/learn-d3-joins#cell-528 added = setdiff(keys(to_cells), keys(from_cells)), removed = setdiff(keys(from_cells), keys(to_cells)), changed = let remained = keys(from_cells)  keys(to_cells) filter(remained) do id from_cells[id].code != to_cells[id].code || from_cells[id].metadata != to_cells[id].metadata end end, folded_changed = any(from_cells[id].code_folded != to_cells[id].code_folded for id in keys(from_cells) if haskey(to_cells, id)), order_changed = from.cell_order != to.cell_order, nbpkg_changed = !is_nbpkg_equal(from.nbpkg_ctx, to.nbpkg_ctx), ) end notebook_differences(from_filename::String, to_filename::String) = notebook_differences(load_notebook_nobackup(from_filename), load_notebook_nobackup(to_filename)) """ Read the notebook file at `notebook.path`, and compare the read result with the notebook's current state. Any changes will be applied to the running notebook, i.e. code changes are run, removed cells are removed, etc. Returns `false` if the file could not be parsed, `true` otherwise. """ function update_from_file(session::ServerSession, notebook::Notebook; kwargs...)::Bool include_nbpg = !session.options.server.auto_reload_from_file_ignore_pkg just_loaded = try load_notebook_nobackup(notebook.path) catch e @error "Skipping hot reload because loading the file went wrong" exception=(e,catch_backtrace()) return false end::Notebook d = notebook_differences(notebook, just_loaded) added = d.added removed = d.removed changed = d.changed # @show added removed changed cells_changed = !(isempty(added) && isempty(removed) && isempty(changed)) folded_changed = d.folded_changed order_changed = d.order_changed nbpkg_changed = d.nbpkg_changed something_changed = cells_changed || folded_changed || order_changed || (include_nbpg && nbpkg_changed) if something_changed @info "Reloading notebook from file and applying changes!" notebook.last_hot_reload_time = time() end for c in added notebook.cells_dict[c] = just_loaded.cells_dict[c] end for c in removed delete!(notebook.cells_dict, c) end for c in changed notebook.cells_dict[c].code = just_loaded.cells_dict[c].code notebook.cells_dict[c].metadata = just_loaded.cells_dict[c].metadata end for c in keys(notebook.cells_dict)  keys(just_loaded.cells_dict) notebook.cells_dict[c].code_folded = just_loaded.cells_dict[c].code_folded end notebook.cell_order = just_loaded.cell_order notebook.metadata = just_loaded.metadata if include_nbpg && nbpkg_changed @info "nbpkgs not equal" (notebook.nbpkg_ctx isa Nothing) (just_loaded.nbpkg_ctx isa Nothing) if (notebook.nbpkg_ctx isa Nothing) != (just_loaded.nbpkg_ctx isa Nothing) @info "nbpkg status changed, overriding..." notebook.nbpkg_ctx = just_loaded.nbpkg_ctx notebook.nbpkg_install_time_ns = just_loaded.nbpkg_install_time_ns else @info "Old new project" PkgCompat.read_project_file(notebook) PkgCompat.read_project_file(just_loaded) @info "Old new manifest" PkgCompat.read_manifest_file(notebook) PkgCompat.read_manifest_file(just_loaded) write(PkgCompat.project_file(notebook), PkgCompat.read_project_file(just_loaded)) write(PkgCompat.manifest_file(notebook), PkgCompat.read_manifest_file(just_loaded)) end notebook.nbpkg_restart_required_msg = "Yes, because the file was changed externally and the embedded Pkg changed." end if something_changed update_save_run!(session, notebook, Cell[notebook.cells_dict[c] for c in union(added, changed)]; kwargs...) # this will also update nbpkg if needed end return true end function update_skipped_cells_dependency!(notebook::Notebook, topology::NotebookTopology=notebook.topology) skipped_cells = filter(is_skipped_as_script, notebook.cells) indirectly_skipped = collect(topological_order_cached(topology, skipped_cells)) for cell in notebook.cells cell.depends_on_skipped_cells = false end for cell in indirectly_skipped cell.depends_on_skipped_cells = true end end function update_disabled_cells_dependency!(notebook::Notebook, topology::NotebookTopology=notebook.topology) disabled_cells = filter(is_disabled, notebook.cells) indirectly_disabled = collect(topological_order_cached(topology, disabled_cells)) for cell in notebook.cells cell.depends_on_disabled_cells = false end for cell in indirectly_disabled cell.depends_on_disabled_cells = true end end import LRUCache const _cache_for_topological_order = LRUCache.LRU{UInt, TopologicalOrder{Cell}}(; maxsize = 10) function topological_order_cached(topology::NotebookTopology, roots::AbstractVector{Cell}; kwargs...) h = hash(( # the `topology` object is designed to be `===` the same if the cell inputs dont change. So we should use objectid here objectid(topology), # we don't just hash `roots` directly because thats quite a lot of work objectid.(roots), # we hash the kwargs kwargs)) get!(_cache_for_topological_order, h) do topological_order(topology, roots; kwargs...) end end function set_bond_values_reactive(; session::ServerSession, notebook::Notebook, bound_sym_names::AbstractVector{Symbol}, is_first_values::AbstractVector{Bool}=[false for x in bound_sym_names], initiator=nothing, kwargs... )::Union{Task,TopologicalOrder} # filter out the bonds that don't need to be set syms_to_set = first.( Iterators.filter(zip(bound_sym_names, is_first_values) |> collect) do (bound_sym, is_first_value) new_value = notebook.bonds[bound_sym].value variable_exists = Pluto.MoreAnalysis.is_assigned_anywhere(notebook.topology, bound_sym) if !variable_exists # a bond was set while the cell is in limbo state # we don't need to do anything return false end # TODO: Not checking for any dependents now # any_dependents = Pluto.MoreAnalysis.is_referenced_anywhere(notebook.topology, bound_sym) # fix for https://github.com/fonsp/Pluto.jl/issues/275 # if `Base.get` was defined to give an initial value (read more about this in the Interactivity sample notebook), then we want to skip the first value sent back from the bond. (if `Base.get` was not defined, then the variable has value `missing`) # Check if the variable does not already have that value. # because if the initial value is already set, then we don't want to run dependent cells again. eq_tester = :(try !ismissing($bound_sym) && ($bound_sym == Main.PlutoRunner.transform_bond_value($(QuoteNode(bound_sym)), $(new_value))) === true catch; false end) # not just a === comparison because JS might send back the same value but with a different type (Float64 becomes Int64 in JS when it's an integer. The `=== true` check handles cases like `[missing] == [123]`, which returns `missing`, not `true` or `false`.) if is_first_value && will_run_code(notebook) && WorkspaceManager.eval_fetch_in_workspace((session, notebook), eq_tester) return false end return true end )::Vector{Symbol} if isempty(syms_to_set) || !will_run_code(notebook) send_notebook_changes!(ClientRequest(; session, notebook, initiator)) return TopologicalOrder(notebook.topology, Cell[], Dict{Cell,PlutoDependencyExplorer.ReactivityError}()) end new_values = Any[notebook.bonds[bound_sym].value for bound_sym in syms_to_set] bond_value_pairs = zip(syms_to_set, new_values) syms_to_set_set = Set{Symbol}(syms_to_set) function custom_deletion_hook((session, notebook)::Tuple{ServerSession,Notebook}, old_workspace_name, new_workspace_name, to_delete_vars::Set{Symbol}, methods_to_delete, module_imports_to_move, cells_to_macro_invalidate, cells_to_js_link_invalidate; to_run) to_delete_vars = union(to_delete_vars, syms_to_set_set) # also delete the bound symbols WorkspaceManager.move_vars( (session, notebook), old_workspace_name, new_workspace_name, to_delete_vars, methods_to_delete, module_imports_to_move, cells_to_macro_invalidate, cells_to_js_link_invalidate, syms_to_set_set, ) set_bond_value_pairs!(session, notebook, zip(syms_to_set, new_values)) end to_reeval = PlutoDependencyExplorer.where_referenced(notebook.topology, syms_to_set_set) run_reactive_async!(session, notebook, to_reeval; deletion_hook=custom_deletion_hook, save=false, user_requested_run=false, run_async=false, bond_value_pairs, kwargs...) end """ Returns the names of all defined bonds """ function get_bond_names(session::ServerSession, notebook::Notebook) cells_bond_names = map(notebook.cell_order) do cell_id WorkspaceManager.get_bond_names((session,notebook), cell_id) end union!(Set{Symbol}(), cells_bond_names...) end """ Returns the set of all possible values for the binded variable `n` as returned by the widget implementation using `AbstractPlutoDingetjes.possible_bond_values(element)`. This API is meant to be used by PlutoSliderServer. """ possible_bond_values(session::ServerSession, notebook::Notebook, name::Symbol) = WorkspaceManager.possible_bond_values((session,notebook), name) """ Optimized version of `length  possible_bond_values`. """ possible_bond_values_length(session::ServerSession, notebook::Notebook, name::Symbol) = WorkspaceManager.possible_bond_values((session,notebook), name; get_length=true) module Throttled import Base.Threads struct ThrottledFunction f::Function timeout::Real runtime_multiplier::Float64 tlock::ReentrantLock iscoolnow::Ref{Bool} run_later::Ref{Bool} last_runtime::Ref{Float64} end "Run the function now" function Base.flush(tf::ThrottledFunction) lock(tf.tlock) do tf.run_later[] = false tf.last_runtime[] = @elapsed result = tf.f() result end end "Start the cooldown period. If at the end, a run_later[] is set, then we run the function and schedule the next cooldown period." function schedule(tf::ThrottledFunction) # if the last runtime was quite long, increase the sleep period to match. Timer(tf.timeout + tf.last_runtime[] * tf.runtime_multiplier) do _t if tf.run_later[] flush(tf) schedule(tf) else tf.iscoolnow[] = true end end end function (tf::ThrottledFunction)() if tf.iscoolnow[] tf.iscoolnow[] = false flush(tf) schedule(tf) else tf.run_later[] = true end nothing end """ throttled(f::Function, timeout::Real) Return a function that when invoked, will only be triggered at most once during `timeout` seconds. The throttled function will run as much as it can, without ever going more than once per `wait` duration. This throttle is 'leading' and has some other properties that are specifically designed for our use in Pluto, see the tests. Inspired by FluxML See: https://github.com/FluxML/Flux.jl/blob/8afedcd6723112ff611555e350a8c84f4e1ad686/src/utils.jl#L662 """ function throttled(f::Function, timeout::Real; runtime_multiplier::Float64=0.0) tlock = ReentrantLock() iscoolnow = Ref(false) run_later = Ref(false) last_runtime = Ref(0.0) tf = ThrottledFunction(f, timeout, runtime_multiplier, tlock, iscoolnow, run_later, last_runtime) # we initialize hot, and start the cooldown period immediately schedule(tf) return tf end """ Given a throttled function, skip any pending run if hot (but let the cooldown period continue), or start the cooldown period if cool. This forces the throttled function to not fire for a little while. Argument should be the first function returned by `throttled`. """ function force_throttle_without_run(tf::ThrottledFunction) tf.run_later[] = false if tf.iscoolnow[] tf.iscoolnow[] = false schedule(tf) end end force_throttle_without_run(::Function) = nothing """ simple_leading_throttle(f, delay::Real) Return a function that when invoked, will only be triggered at most once during `timeout` seconds. The throttled function will run as much as it can, without ever going more than once per `wait` duration. Compared to [`throttled`](@ref), this simple function only implements [leading](https://css-tricks.com/debouncing-throttling-explained-examples/) throttling and accepts function with arbitrary number of positional and keyword arguments. """ function simple_leading_throttle(f, delay::Real) last_time = 0.0 return function(args...;kwargs...) now = time() if now - last_time > delay last_time = now f(args...;kwargs...) end end end end "A `Token` can only be held by one async process at one time. Use `Base.take!(token)` to claim the token, `Base.put!(token)` to give the token back." struct Token c::Channel{Nothing} Token() = let c = Channel{Nothing}(1) push!(c, nothing) new(c) end end Base.take!(token::Token) = Base.take!(token.c) Base.put!(token::Token) = Base.put!(token.c, nothing) Base.isready(token::Token) = Base.isready(token.c) Base.wait(token::Token) = Base.put!(token.c, Base.take!(token.c)) function withtoken(f::Function, token::Token) take!(token) result = try f() finally put!(token) end result end ### "Track whether some task needs to be done. `request!(requestqueue)` will make sure that the task is done at least once after calling it. Multiple calls might get bundled into one." mutable struct RequestQueue is_processing::Bool c::Channel{Nothing} RequestQueue() = new(false, Channel{Nothing}(1)) end "Give a function (with no arguments) that should be called after a request." function process(f::Function, requestqueue::RequestQueue) @assert !requestqueue.is_processing requestqueue.is_processing = true while true take!(requestqueue.c) f() end end function request!(queue::RequestQueue) if isready(queue.c) push!(queue.c, nothing) end end "Like @async except it prints errors to the terminal. " macro asynclog(expr) quote @async begin # because this is being run asynchronously, we need to catch exceptions manually try $(esc(expr)) catch ex bt = stacktrace(catch_backtrace()) showerror(stderr, ex, bt) rethrow(ex) end end end end module WorkspaceManager import UUIDs: UUID, uuid1 import ..Pluto import ..Pluto: Configuration, Notebook, Cell, ProcessStatus, ServerSession, ExpressionExplorer, pluto_filename, Token, withtoken, tamepath, project_relative_path, putnotebookupdates!, UpdateMessage import ..Pluto.Status import ..Pluto.PkgCompat import ..Configuration: CompilerOptions, _merge_notebook_compiler_options, _convert_to_flags import ..Pluto.ExpressionExplorer: FunctionName import ..PlutoRunner import Malt import Malt.Distributed """ Contains the Julia process to evaluate code in. Each notebook gets at most one `Workspace` at any time, but it can also have no `Workspace` (it cannot `eval` code in this case). """ Base.@kwdef mutable struct Workspace worker::Malt.AbstractWorker notebook_id::UUID discarded::Bool=false remote_log_channel::Union{Distributed.RemoteChannel,AbstractChannel} module_name::Symbol dowork_token::Token=Token() nbpkg_was_active::Bool=false has_executed_effectful_code::Bool=false is_offline_renderer::Bool=false original_LOAD_PATH::Vector{String}=String[] original_ACTIVE_PROJECT::Union{Nothing,String}=nothing end const SN = Tuple{ServerSession, Notebook} "These expressions get evaluated whenever a new `Workspace` process is created." process_preamble() = quote Base.exit_on_sigint(false) const pluto_boot_environment_path = $(Pluto.pluto_boot_environment_path[]) include($(project_relative_path(joinpath("src", "runner"), "Loader.jl"))) ENV["GKSwstype"] = "nul" ENV["JULIA_REVISE_WORKER_ONLY"] = "1" end const active_workspaces = Dict{UUID,Task}() "Set of notebook IDs that we will never make a process for again." const discarded_workspaces = Set{UUID}() "Create a workspace for the notebook, optionally in the main process." function make_workspace((session, notebook)::SN; is_offline_renderer::Bool=false)::Workspace workspace_business = is_offline_renderer ? Status.Business(name=:gobble) : Status.report_business_started!(notebook.status_tree, :workspace) create_status = Status.report_business_started!(workspace_business, :create_process) Status.report_business_planned!(workspace_business, :init_process) is_offline_renderer || (notebook.process_status = ProcessStatus.starting) WorkerType = if is_offline_renderer || !session.options.evaluation.workspace_use_distributed Malt.InProcessWorker elseif something( session.options.evaluation.workspace_use_distributed_stdlib, false ) Malt.DistributedStdlibWorker else Malt.Worker end @debug "Creating workspace process" notebook.path length(notebook.cells) try worker = create_workspaceprocess(WorkerType; compiler_options=_merge_notebook_compiler_options(notebook, session.options.compiler), status=create_status) Status.report_business_finished!(workspace_business, :create_process) init_status = Status.report_business_started!(workspace_business, :init_process) Status.report_business_started!(init_status, Symbol(1)) Status.report_business_planned!(init_status, Symbol(2)) Status.report_business_planned!(init_status, Symbol(3)) Status.report_business_planned!(init_status, Symbol(4)) let s = session.options.evaluation.workspace_custom_startup_expr s === nothing || Malt.remote_eval_wait(worker, Meta.parseall(s)) end Malt.remote_eval_wait(worker, quote PlutoRunner.notebook_id[] = $(notebook.notebook_id) end) remote_log_channel = Malt.worker_channel(worker, quote channel = Channel{Any}(10) Main.PlutoRunner.setup_plutologger( $(notebook.notebook_id), channel ) channel end) run_channel = Malt.worker_channel(worker, :(Main.PlutoRunner.run_channel)) module_name = create_emptyworkspacemodule(worker) original_LOAD_PATH, original_ACTIVE_PROJECT = Malt.remote_eval_fetch(worker, :(Base.LOAD_PATH, Base.ACTIVE_PROJECT[])) workspace = Workspace(; worker, notebook_id=notebook.notebook_id, remote_log_channel, module_name, original_LOAD_PATH, original_ACTIVE_PROJECT, is_offline_renderer, ) Status.report_business_finished!(init_status, Symbol(1)) Status.report_business_started!(init_status, Symbol(2)) @async start_relaying_logs((session, notebook), remote_log_channel) @async start_relaying_self_updates((session, notebook), run_channel) cd_workspace(workspace, notebook.path) Status.report_business_finished!(init_status, Symbol(2)) Status.report_business_started!(init_status, Symbol(3)) use_nbpkg_environment((session, notebook), workspace) Status.report_business_finished!(init_status, Symbol(3)) Status.report_business_started!(init_status, Symbol(4)) # TODO: precompile 1+1 with display # sleep(3) eval_format_fetch_in_workspace(workspace, Expr(:toplevel, LineNumberNode(-1), :(1+1)), uuid1(); code_is_effectful=false) Status.report_business_finished!(init_status, Symbol(4)) Status.report_business_finished!(workspace_business, :init_process) Status.report_business_finished!(workspace_business) is_offline_renderer || if notebook.process_status == ProcessStatus.starting notebook.process_status = ProcessStatus.ready end return workspace catch e Status.report_business_finished!(workspace_business, false) notebook.process_status = ProcessStatus.no_process rethrow(e) end end function use_nbpkg_environment((session, notebook)::SN, workspace=nothing) enabled = notebook.nbpkg_ctx !== nothing workspace.nbpkg_was_active == enabled && return workspace = workspace !== nothing ? workspace : get_workspace((session, notebook)) workspace.discarded && return workspace.nbpkg_was_active = enabled if workspace.worker isa Malt.InProcessWorker # Not supported return end new_LP = enabled ? ["@", "@stdlib"] : workspace.original_LOAD_PATH new_AP = enabled ? PkgCompat.env_dir(notebook.nbpkg_ctx) : workspace.original_ACTIVE_PROJECT Malt.remote_eval_wait(workspace.worker, quote copy!(LOAD_PATH, $(new_LP)) Base.ACTIVE_PROJECT[] = $(new_AP) end) end function start_relaying_self_updates((session, notebook)::SN, run_channel) while true try next_run_uuid = take!(run_channel) cell_to_run = notebook.cells_dict[next_run_uuid] Pluto.run_reactive!(session, notebook, notebook.topology, notebook.topology, Cell[cell_to_run]; user_requested_run=false) catch e if !isopen(run_channel) break end @error "Failed to relay self-update" exception=(e, catch_backtrace()) end end end function start_relaying_logs((session, notebook)::SN, log_channel) update_throttled = Pluto.Throttled.throttled(0.1) do Pluto.send_notebook_changes!(Pluto.ClientRequest(; session, notebook)) end while true try next_log::Dict{String,Any} = take!(log_channel) fn = next_log["file"] match = findfirst("#==#", fn) # Show the log at the currently running cell, which is given by running_cell_id = next_log["cell_id"]::UUID # Some logs originate from outside of the running code, through function calls. Some code here to deal with that: begin source_cell_id = if match !== nothing # The log originated from within the notebook UUID(fn[match[end]+1:end]) else # The log originated from a function call defined outside of the notebook. # Show the log at the currently running cell, at "line -1", i.e. without line info. next_log["line"] = -1 running_cell_id end if running_cell_id != source_cell_id # The log originated from a function in another cell of the notebook # Show the log at the currently running cell, at "line -1", i.e. without line info. next_log["line"] = -1 end end source_cell = get(notebook.cells_dict, source_cell_id, nothing) running_cell = get(notebook.cells_dict, running_cell_id, nothing) display_cell = if running_cell === nothing || (source_cell !== nothing && source_cell.output.has_pluto_hook_features) source_cell else running_cell end @assert !isnothing(display_cell) # this handles the use of published_to_js inside logs: objects that were newly published during the rendering of the log args. merge!(display_cell.published_objects, next_log["new_published_objects"]) delete!(next_log, "new_published_objects") push!(display_cell.logs, next_log) Pluto.@asynclog update_throttled() catch e if !isopen(log_channel) break end @error "Failed to relay log" exception=(e, catch_backtrace()) end end end "Call `cd(\$path)` inside the workspace. This is done when creating a workspace, and whenever the notebook changes path." function cd_workspace(workspace, path::AbstractString) eval_in_workspace(workspace, quote cd(dirname($path)) end) end "Create a new empty workspace. Return the `(old, new)` workspace names as a tuple of `Symbol`s." function bump_workspace_module(session_notebook::SN) workspace = get_workspace(session_notebook) old_name = workspace.module_name new_name = workspace.module_name = create_emptyworkspacemodule(workspace.worker) old_name, new_name end function get_bond_names(session_notebook::SN, cell_id) workspace = get_workspace(session_notebook) Malt.remote_eval_fetch(workspace.worker, quote PlutoRunner.get_bond_names($cell_id) end) end function possible_bond_values(session_notebook::SN, n::Symbol; get_length::Bool=false) workspace = get_workspace(session_notebook) Malt.remote_eval_fetch(workspace.worker, quote PlutoRunner.possible_bond_values($(QuoteNode(n)); get_length=$(get_length)) end) end function create_emptyworkspacemodule(worker::Malt.AbstractWorker)::Symbol Malt.remote_eval_fetch(worker, quote PlutoRunner.increment_current_module() end) end # NOTE: this function only start a worker process using given # compiler options, it does not resolve paths for notebooks # compiler configurations passed to it should be resolved before this function create_workspaceprocess(WorkerType; compiler_options=CompilerOptions(), status::Status.Business=Status.Business())::Malt.AbstractWorker if WorkerType === Malt.InProcessWorker worker = WorkerType() if !(isdefined(Main, :PlutoRunner) && Main.PlutoRunner isa Module) # we make PlutoRunner available in Main, right now it's only defined inside this Pluto module. Malt.remote_eval_wait(Main, worker, quote PlutoRunner = $(PlutoRunner) end) end else Status.report_business_started!(status, Symbol("Starting process")) Status.report_business_planned!(status, Symbol("Loading notebook boot environment")) worker = WorkerType(; exeflags=_convert_to_flags(compiler_options)) Status.report_business_finished!(status, Symbol("Starting process")) Status.report_business_started!(status, Symbol("Loading notebook boot environment")) Malt.remote_eval_wait(worker, process_preamble()) # so that we NEVER break the workspace with an interrupt  Malt.remote_eval(worker, quote while true try wait() catch end end end) end Status.report_business_finished!(status) worker end """ Return the `Workspace` of `notebook`; will be created if none exists yet. If `allow_creation=false`, then `nothing` is returned if no workspace exists, instead of creating one. """ function get_workspace(session_notebook::SN; allow_creation::Bool=true)::Union{Nothing,Workspace} session, notebook = session_notebook if notebook.notebook_id in discarded_workspaces @debug "This should not happen" notebook.process_status error("Cannot run code in this notebook: it has already shut down.") end task = if !allow_creation get(active_workspaces, notebook.notebook_id, nothing) else get!(active_workspaces, notebook.notebook_id) do  = Pluto.@asynclog make_workspace(session_notebook) yield();  end end isnothing(task) ? nothing : fetch(task) end get_workspace(workspace::Workspace; kwargs...)::Workspace = workspace "Try our best to delete the workspace. `Workspace` will have its worker process terminated." function unmake_workspace(session_notebook::SN; async::Bool=false, verbose::Bool=true, allow_restart::Bool=true) session, notebook = session_notebook workspace = get_workspace(session_notebook; allow_creation=false) workspace === nothing && return workspace.discarded = true allow_restart || push!(discarded_workspaces, notebook.notebook_id) filter!(p -> fetch(p.second).worker != workspace.worker, active_workspaces) t = @async begin interrupt_workspace(workspace; verbose=false) Malt.stop(workspace.worker) end async || wait(t) nothing end function workspace_exception_result(ex::Union{Base.IOError, Malt.TerminatedWorkerException, Distributed.ProcessExitedException}, workspace::Workspace) ( output_formatted=PlutoRunner.format_output(CapturedException(ex, [])), errored=true, interrupted=true, process_exited=true && !workspace.discarded, # don't report a process exit if the workspace was discarded on purpose runtime=nothing, published_objects=Dict{String,Any}(), has_pluto_hook_features=false, ) end workspace_exception_result(exs::CompositeException, workspace::Workspace) = workspace_exception_result(first(exs.exceptions), workspace) function workspace_exception_result(ex::Exception, workspace::Workspace) if ex isa InterruptException || (ex isa Malt.RemoteException && occursin("InterruptException", ex.message)) @info "Found an interrupt!" ex ( output_formatted=PlutoRunner.format_output(CapturedException(InterruptException(), [])), errored=true, interrupted=true, process_exited=false, runtime=nothing, published_objects=Dict{String,Any}(), has_pluto_hook_features=false, ) else @error "Unkown error during eval_format_fetch_in_workspace" ex ( output_formatted=PlutoRunner.format_output(CapturedException(ex, [])), errored=true, interrupted=true, process_exited=!Malt.isrunning(workspace.worker) && !workspace.discarded, # don't report a process exit if the workspace was discarded on purpose runtime=nothing, published_objects=Dict{String,Any}(), has_pluto_hook_features=false, ) end end """ Evaluate expression inside the workspace - output is fetched and formatted, errors are caught and formatted. Returns formatted output and error flags. `expr` has to satisfy `ExpressionExplorer.is_toplevel_expr`. """ function eval_format_fetch_in_workspace( session_notebook::Union{SN,Workspace}, expr::Expr, cell_id::UUID; ends_with_semicolon::Bool=false, function_wrapped_info::Union{Nothing,Tuple}=nothing, forced_expr_id::Union{PlutoRunner.ObjectID,Nothing}=nothing, known_published_objects::Vector{String}=String[], user_requested_run::Bool=true, capture_stdout::Bool=true, code_is_effectful::Bool=true, )::PlutoRunner.FormattedCellResult workspace = get_workspace(session_notebook) is_on_this_process = workspace.worker isa Malt.InProcessWorker # if multiple notebooks run on the same process, then we need to `cd` between the different notebook paths if session_notebook isa Tuple use_nbpkg_environment(session_notebook, workspace) end # Run the code  # A try block (on this process) to catch an InterruptException take!(workspace.dowork_token) workspace.has_executed_effectful_code |= code_is_effectful early_result = try Malt.remote_eval_wait(workspace.worker, quote PlutoRunner.run_expression( getfield(Main, $(QuoteNode(workspace.module_name))), $(QuoteNode(expr)), $(workspace.notebook_id), $cell_id, $function_wrapped_info, $forced_expr_id; user_requested_run=$user_requested_run, capture_stdout=$(capture_stdout && !is_on_this_process), ) end) put!(workspace.dowork_token) nothing catch e # Don't use a `finally` because the token needs to be back asap for the interrupting code to pick it up. put!(workspace.dowork_token) workspace_exception_result(e, workspace) end if early_result === nothing format_fetch_in_workspace(workspace, cell_id, ends_with_semicolon, known_published_objects; capture_stdout) else early_result end end "Evaluate expression inside the workspace - output is not fetched, errors are rethrown. For internal use." function eval_in_workspace(session_notebook::Union{SN,Workspace}, expr) workspace = get_workspace(session_notebook) Malt.remote_eval_wait(workspace.worker, quote Core.eval($(workspace.module_name), $(QuoteNode(expr))) end) nothing end function format_fetch_in_workspace( session_notebook::Union{SN,Workspace}, cell_id, ends_with_semicolon, known_published_objects::Vector{String}=String[], showmore_id::Union{PlutoRunner.ObjectDimPair,Nothing}=nothing; capture_stdout::Bool=true, )::PlutoRunner.FormattedCellResult workspace = get_workspace(session_notebook) # Instead of fetching the output value (which might not make sense in our context, # since the user can define structs, types, functions, etc), # we format the cell output on the worker, and fetch the formatted output. withtoken(workspace.dowork_token) do try Malt.remote_eval_fetch(workspace.worker, quote PlutoRunner.formatted_result_of( $(workspace.notebook_id), $cell_id, $ends_with_semicolon, $known_published_objects, $showmore_id, getfield(Main, $(QuoteNode(workspace.module_name))); capture_stdout=$capture_stdout, ) end) catch e workspace_exception_result(CompositeException([e]), workspace) end end end function collect_soft_definitions(session_notebook::SN, modules::Set{Expr}) workspace = get_workspace(session_notebook) Malt.remote_eval_fetch(workspace.worker, quote PlutoRunner.collect_soft_definitions($(workspace.module_name), $modules) end) end function macroexpand_in_workspace(session_notebook::SN, macrocall, cell_id, module_name = Symbol(""); capture_stdout::Bool=true)::Tuple{Bool, Any} workspace = get_workspace(session_notebook) module_name = module_name === Symbol("") ? workspace.module_name : module_name Malt.remote_eval_fetch(workspace.worker, quote try (true, PlutoRunner.try_macroexpand($(module_name), $(workspace.notebook_id), $(cell_id), $(macrocall |> QuoteNode); capture_stdout=$(capture_stdout))) catch error # We have to be careful here, for example a thrown `MethodError()` will contain the called method and arguments. # which normally would be very useful for debugging, but we can't serialize it! # So we make sure we only serialize the exception we know about, and string-ify the others. if error isa UndefVarError (false, UndefVarError(error.var)) elseif error isa LoadError && error.error isa UndefVarError (false, UndefVarError(error.error.var)) else (false, ErrorException(sprint(showerror, error))) end end end) end "Evaluate expression inside the workspace - output is returned. For internal use." function eval_fetch_in_workspace(session_notebook::Union{SN,Workspace}, expr) workspace = get_workspace(session_notebook) Malt.remote_eval_fetch(workspace.worker, quote Core.eval($(workspace.module_name), $(QuoteNode(expr))) end) end function do_reimports(session_notebook::Union{SN,Workspace}, module_imports_to_move::Set{Expr}) workspace = get_workspace(session_notebook) Malt.remote_eval_wait(workspace.worker, quote PlutoRunner.do_reimports($(workspace.module_name), $module_imports_to_move) end) end """ Move variables to a new module. A given set of variables to be 'deleted' will not be moved to the new module, making them unavailable. """ function move_vars( session_notebook::Union{SN,Workspace}, old_workspace_name::Symbol, new_workspace_name::Union{Nothing,Symbol}, to_delete::Set{Symbol}, methods_to_delete::Set{Tuple{UUID,Tuple{Vararg{Symbol}}}}, module_imports_to_move::Set{Expr}, cells_to_macro_invalidate::Set{UUID}, cells_to_js_link_invalidate::Set{UUID}, keep_registered::Set{Symbol}=Set{Symbol}(); kwargs... ) workspace = get_workspace(session_notebook) new_workspace_name = something(new_workspace_name, workspace.module_name) Malt.remote_eval_wait(workspace.worker, quote PlutoRunner.move_vars( $(QuoteNode(old_workspace_name)), $(QuoteNode(new_workspace_name)), $to_delete, $methods_to_delete, $module_imports_to_move, $cells_to_macro_invalidate, $cells_to_js_link_invalidate, $keep_registered, ) end) end function move_vars( session_notebook::Union{SN,Workspace}, to_delete::Set{Symbol}, methods_to_delete::Set{Tuple{UUID,Tuple{Vararg{Symbol}}}}, module_imports_to_move::Set{Expr}, cells_to_macro_invalidate::Set{UUID}, cells_to_js_link_invalidate::Set{UUID}; kwargs... ) move_vars( session_notebook, bump_workspace_module(session_notebook)..., to_delete, methods_to_delete, module_imports_to_move, cells_to_macro_invalidate, cells_to_js_link_invalidate; kwargs... ) end """ ```julia poll(query::Function, timeout::Real=Inf64, interval::Real=1/20)::Bool ``` Keep running your function `query()` in intervals until it returns `true`, or until `timeout` seconds have passed. `poll` returns `true` if `query()` returned `true`. If `timeout` seconds have passed, `poll` returns `false`. # Example ```julia vals = [1,2,3] @async for i in 1:5 sleep(1) vals[3] = 99 end poll(8 #= seconds =#) do vals[3] == 99 end # returns `true` (after 5 seconds)! ### @async for i in 1:5 sleep(1) vals[3] = 5678 end poll(2 #= seconds =#) do vals[3] == 5678 end # returns `false` (after 2 seconds)! ``` """ function poll(query::Function, timeout::Real=Inf64, interval::Real=1/20) start = time() while time() < start + timeout if query() return true end sleep(interval) end return false end "Force interrupt (SIGINT) a workspace, return whether successful" function interrupt_workspace(session_notebook::Union{SN,Workspace}; verbose=true)::Bool workspace = get_workspace(session_notebook; allow_creation=false) if !(workspace isa Workspace) # verbose && @info "Can't interrupt this notebook: it is not running." return false end if poll(() -> isready(workspace.dowork_token), 2.0, 5/100) verbose && println("Cell finished, other cells cancelled!") return true end if (workspace.worker isa Malt.DistributedStdlibWorker) && Sys.iswindows() verbose && @warn "Stopping cells is not yet supported on Windows, but it will be soon!\n\nYou can already try out this new functionality with:\n\nPluto.run(workspace_use_distributed_stdlib=false)\n\nLet us know what you think!" return false end if isready(workspace.dowork_token) verbose && @info "Tried to stop idle workspace - ignoring." return true end # You can force kill a julia process by pressing Ctrl+C five times  # But this is not very consistent, so we will just keep pressing Ctrl+C until the workspace isn't running anymore. # TODO: this will also kill "pending" evaluations, and any evaluations started within 100ms of the kill. A global "evaluation count" would fix this. # TODO: listen for the final words of the remote process on stdout/stderr: "Force throwing a SIGINT" try verbose && @info "Sending interrupt to process $(summary(workspace.worker))" Malt.interrupt(workspace.worker) if poll(() -> isready(workspace.dowork_token), 5.0, 5/100) verbose && println("Cell interrupted!") return true end verbose && println("Still running... starting sequence") while !isready(workspace.dowork_token) for _ in 1:5 verbose && print("  ") Malt.interrupt(workspace.worker) sleep(0.18) if isready(workspace.dowork_token) break end end sleep(1.5) end verbose && println() verbose && println("Cell interrupted!") true catch e if !(e isa KeyError) @warn "Interrupt failed for unknown reason" showerror(e, stacktrace(catch_backtrace())) end false end end end import UUIDs: UUID, uuid1 # Hello!  Check out the `Cell` struct. const METADATA_DISABLED_KEY = "disabled" const METADATA_SHOW_LOGS_KEY = "show_logs" const METADATA_SKIP_AS_SCRIPT_KEY = "skip_as_script" # Make sure to keep this in sync with DEFAULT_CELL_METADATA in ../frontend/components/Editor.js const DEFAULT_CELL_METADATA = Dict{String, Any}( METADATA_DISABLED_KEY => false, METADATA_SHOW_LOGS_KEY => true, METADATA_SKIP_AS_SCRIPT_KEY => false, ) Base.@kwdef struct CellOutput body::Union{Nothing,String,Vector{UInt8},Dict}=nothing mime::MIME=MIME("text/plain") rootassignee::Union{Symbol,Nothing}=nothing "Time that the last output was created, used only on the frontend to rerender the output" last_run_timestamp::Float64=0 "Whether `this` inside `<script id=something>` should refer to the previously returned object in HTML output. This is used for fancy animations. true iff a cell runs as a reactive consequence." persist_js_state::Bool=false "Whether this cell uses @use_state or @use_effect" has_pluto_hook_features::Bool=false end struct CellDependencies{T} # T == Cell, but this has to be parametric to avoid a circular dependency of the structs downstream_cells_map::Dict{Symbol,Vector{T}} upstream_cells_map::Dict{Symbol,Vector{T}} precedence_heuristic::Int end "The building block of a `Notebook`. Contains code, output, reactivity data, mitochondria and ribosomes." Base.@kwdef mutable struct Cell <: PlutoDependencyExplorer.AbstractCell "Because Cells can be reordered, they get a UUID. The JavaScript frontend indexes cells using the UUID." cell_id::UUID=uuid1() code::String="" code_folded::Bool=false output::CellOutput=CellOutput() queued::Bool=false running::Bool=false published_objects::Dict{String,Any}=Dict{String,Any}() logs::Vector{Dict{String,Any}}=Vector{Dict{String,Any}}() errored::Bool=false runtime::Union{Nothing,UInt64}=nothing # note that this field might be moved somewhere else later. If you are interested in visualizing the cell dependencies, take a look at the cell_dependencies field in the frontend instead. cell_dependencies::CellDependencies{Cell}=CellDependencies{Cell}(Dict{Symbol,Vector{Cell}}(), Dict{Symbol,Vector{Cell}}(), 99) depends_on_disabled_cells::Bool=false depends_on_skipped_cells::Bool=false metadata::Dict{String,Any}=copy(DEFAULT_CELL_METADATA) end Cell(cell_id, code) = Cell(; cell_id, code) Cell(code) = Cell(uuid1(), code) cell_id(cell::Cell) = cell.cell_id function Base.convert(::Type{Cell}, cell::Dict) Cell( cell_id=UUID(cell["cell_id"]), code=cell["code"], code_folded=cell["code_folded"], metadata=cell["metadata"], ) end "Returns whether or not the cell is **explicitely** disabled." is_disabled(c::Cell) = get(c.metadata, METADATA_DISABLED_KEY, DEFAULT_CELL_METADATA[METADATA_DISABLED_KEY]) set_disabled(c::Cell, value::Bool) = if value == DEFAULT_CELL_METADATA[METADATA_DISABLED_KEY] delete!(c.metadata, METADATA_DISABLED_KEY) else c.metadata[METADATA_DISABLED_KEY] = value end can_show_logs(c::Cell) = get(c.metadata, METADATA_SHOW_LOGS_KEY, DEFAULT_CELL_METADATA[METADATA_SHOW_LOGS_KEY]) is_skipped_as_script(c::Cell) = get(c.metadata, METADATA_SKIP_AS_SCRIPT_KEY, DEFAULT_CELL_METADATA[METADATA_SKIP_AS_SCRIPT_KEY]) must_be_commented_in_file(c::Cell) = is_disabled(c) || is_skipped_as_script(c) || c.depends_on_disabled_cells || c.depends_on_skipped_cells import HTTP """Pluto Events interface Use this interface to hook up functionality into the Pluto world. Events are guaranteed to be run at least every time something interesting happens, but keep in mind that Pluto may be run the events multiple times "logically". For instance, the FileSaveEvent may be triggered whenever pluto wants to make sure the file is saved, which may be more often than the file is actually changed. Deduplicate on your own if you care about this. Define your function to handle the events using multiple dispatch: First assign a handler for all the types you will not use, using the supertype: ```julia-repl julia> function myfn(a::PlutoEvent) nothing end ``` And then create a special function for each event you want to handle specially ```julia-repl julia> function myfn(a::FileSaveEvent) HTTP.post("https://my.service.com/count_saves") end ``` Finally, pass the listener to Pluto's configurations with a keyword argument ```julia-repl julia> Pluto.run(; on_event = myfn) ``` """ abstract type PlutoEvent end function try_event_call(session, event::PlutoEvent) return try session.options.server.on_event(event) catch e # Do not print all the event; it's too big! @warn "Couldn't run event listener" event_type=typeof(event) exception=(e, catch_backtrace()) nothing end end # Triggered when the web server gets started struct ServerStartEvent <: PlutoEvent address::String port::UInt16 end # Triggered when a notebook is saved struct FileSaveEvent <: PlutoEvent notebook::Notebook file_contents::String path::String end FileSaveEvent(notebook::Notebook) = begin file_contents = sprint() do io save_notebook(io, notebook) end FileSaveEvent(notebook, file_contents, notebook.path) end # Triggered when the local code has changed (user typed something), # but the code hasn't run yet. # TODO: Remove me after 0.20 @deprecate struct FileEditEvent <: PlutoEvent notebook::Notebook file_contents::String path::String end FileEditEvent(notebook::Notebook) = begin FileEditEvent(notebook, "BROKEN", notebook.path) end """ Triggered when the notebook state changes (e.g. code changes, output changes, a log message appears, etc) """ struct StateChangeEvent <: PlutoEvent notebook::Notebook end # Triggered when we create a new notebook struct NewNotebookEvent <: PlutoEvent end # Triggered when we open any notebook struct OpenNotebookEvent <: PlutoEvent notebook::Notebook end # Triggered when Pluto completes an evaluation loop struct NotebookExecutionDoneEvent <: PlutoEvent notebook::Notebook user_requested_run::Bool end # This will be fired ONLY if URL params don't match anything else. # Useful if you want to create a file in a custom way, # before opening the notebook # Should return a redirect to /edit?id={notebook_id} # Note: use Pluto.open - try_launch_notebook_response will return a fitting response. struct CustomLaunchEvent <: PlutoEvent params::Dict{Any, Any} request::HTTP.Request try_launch_notebook_response::Function end # Triggered when a notebook has shut down. struct ShutdownNotebookEvent <: PlutoEvent notebook::Notebook end import Pkg using Base64 using HypertextLiteral import URIs const default_binder_url = "https://mybinder.org/v2/gh/fonsp/pluto-on-binder/v$(string(PLUTO_VERSION))" const cdn_version_override = nothing # const cdn_version_override = "2a48ae2" if cdn_version_override !== nothing @warn "Reminder to fonsi: Using a development version of Pluto for CDN assets. The binder button might not work. You should not see this on a released version of Pluto." cdn_version_override end cdnified_editor_html(; kwargs...) = cdnified_html("editor.html"; kwargs...) function cdnified_html(filename::AbstractString; version::Union{Nothing,VersionNumber,AbstractString}=nothing, pluto_cdn_root::Union{Nothing,AbstractString}=nothing, ) should_use_bundled_cdn = version  (nothing, PLUTO_VERSION) && pluto_cdn_root === nothing @something( if should_use_bundled_cdn try original = read(project_relative_path("frontend-dist", filename), String) cdn_root = "https://cdn.jsdelivr.net/gh/fonsp/Pluto.jl@$(string(PLUTO_VERSION))/frontend-dist/" @debug "Using CDN for Pluto assets:" cdn_root replace_with_cdn(original) do url # Because parcel creates filenames with a hash in them, we can check if the file exists locally to make sure that everything is in order. @assert isfile(project_relative_path("frontend-dist", url)) "Could not find the file $(project_relative_path("frontend-dist", url)) locally, that's a bad sign." URIs.resolvereference(cdn_root, url) |> string end catch e get(ENV, "JULIA_PLUTO_IGNORE_CDN_BUNDLE_WARNING", "false") == "true" || @warn "Could not use bundled CDN version of $(filename). You should only see this message if you are using a fork or development branch of Pluto." exception=(e,catch_backtrace()) maxlog=1 nothing end end, let original = read(project_relative_path("frontend", filename), String) cdn_root = something(pluto_cdn_root, "https://cdn.jsdelivr.net/gh/fonsp/Pluto.jl@$(something(cdn_version_override, string(something(version, PLUTO_VERSION))))/frontend/") @debug "Using CDN for Pluto assets:" cdn_root replace_with_cdn(original) do url URIs.resolvereference(cdn_root, url) |> string end end ) end const _insertion_meta = """<meta name="pluto-insertion-spot-meta">""" const _insertion_parameters = """<meta name="pluto-insertion-spot-parameters">""" const _insertion_preload = """<meta name="pluto-insertion-spot-preload">""" inserted_html(original_contents::AbstractString; meta::AbstractString="", parameters::AbstractString="", preload::AbstractString="", ) = replace_at_least_once( replace_at_least_once( replace_at_least_once(original_contents, _insertion_meta => """ $(meta) $(_insertion_meta) """ ), _insertion_parameters => """ $(parameters) $(_insertion_parameters) """ ), _insertion_preload => """ $(preload) $(_insertion_preload) """ ) function prefetch_statefile_html(statefile_js::AbstractString) if length(statefile_js) < 300 && startswith(statefile_js, '"') && endswith(statefile_js, '"') && !startswith(statefile_js, "\"data:") """\n<link rel="preload" as="fetch" href=$(statefile_js) crossorigin>\n""" else "" end end """ This function takes the `editor.html` file from Pluto's source code, and uses string replacements to insert custom data. By inserting a statefile (and more), you can create an HTML file that will display a notebook when opened: this is how the Static HTML export works. See [PlutoSliderServer.jl](https://github.com/JuliaPluto/PlutoSliderServer.jl) if you are interested in exporting notebooks programatically. """ function generate_html(; version::Union{Nothing,VersionNumber,AbstractString}=nothing, pluto_cdn_root::Union{Nothing,AbstractString}=nothing, notebookfile_js::AbstractString="undefined", statefile_js::AbstractString="undefined", slider_server_url_js::AbstractString="undefined", binder_url_js::AbstractString=repr(default_binder_url), recording_url_js::AbstractString="undefined", recording_audio_url_js::AbstractString="undefined", disable_ui::Bool=true, preamble_html_js::AbstractString="undefined", notebook_id_js::AbstractString="undefined", isolated_cell_ids_js::AbstractString="undefined", header_html::AbstractString="", )::String cdnified = cdnified_editor_html(; version, pluto_cdn_root) (length(statefile_js) > 32000000 || length(recording_url_js) > 32000000 || length(recording_audio_url_js) > 32000000) && @error "Statefile or recording URL embedded in HTML is very large. The file can be opened with Chrome and Safari, but probably not with Firefox. If you are using PlutoSliderServer to generate this file, then we recommend the setting `baked_statefile=false`. If you are not using PlutoSliderServer, then consider reducing the size of figures and output in the notebook." length(statefile_js) length(recording_url_js) length(recording_audio_url_js) parameters = """ <script data-pluto-file="launch-parameters"> window.pluto_notebook_id = $(notebook_id_js); window.pluto_isolated_cell_ids = $(isolated_cell_ids_js); window.pluto_notebookfile = $(notebookfile_js); window.pluto_disable_ui = $(disable_ui ? "true" : "false"); window.pluto_slider_server_url = $(slider_server_url_js); window.pluto_binder_url = $(binder_url_js); window.pluto_statefile = $(statefile_js); window.pluto_preamble_html = $(preamble_html_js); window.pluto_recording_url = $(recording_url_js); window.pluto_recording_audio_url = $(recording_audio_url_js); </script> """ preload = prefetch_statefile_html(statefile_js) inserted_html(cdnified; meta=header_html, parameters, preload) end function replace_at_least_once(s, pair) from, to = pair @assert occursin(from, s) replace(s, pair) end function generate_html(notebook; kwargs...)::String state = notebook_to_js(notebook) notebookfile_js = let notebookfile64 = base64encode() do io save_notebook(io, notebook) end "\"data:text/julia;charset=utf-8;base64,$(notebookfile64)\"" end statefile_js = let statefile64 = base64encode() do io pack(io, state) end "\"data:;base64,$(statefile64)\"" end fm = frontmatter(notebook) header_html = isempty(fm) ? "" : frontmatter_html(fm) # avoid loading HypertextLiteral if there is no frontmatter # We don't set `notebook_id_js` because this is generated by the server, the option is only there for funky setups. generate_html(; statefile_js, notebookfile_js, header_html, kwargs...) end const frontmatter_writers = ( ("title", x -> @htl(""" <title>$(x)</title> """)), ("description", x -> @htl(""" <meta name="description" content=$(x)> """)), ("tags", x -> x isa Vector ? @htl("$(( @htl(""" <meta property="og:article:tag" content=$(t)> """) for t in x ))") : nothing), ) const _og_properties = ("title", "type", "description", "image", "url", "audio", "video", "site_name", "locale", "locale:alternate", "determiner") const _default_frontmatter = Dict{String, Any}( "type" => "article", # Note: these defaults are skipped when there is no frontmatter at all. ) function frontmatter_html(frontmatter::Dict{String,Any}; default_frontmatter::Dict{String,Any}=_default_frontmatter)::String d = merge(default_frontmatter, frontmatter) repr(MIME"text/html"(), @htl("""$(( f(d[key]) for (key, f) in frontmatter_writers if haskey(d, key) ))$(( @htl("""<meta property=$("og:$(key)") content=$(val)> """) for (key, val) in d if key in _og_properties ))""")) end replace_substring(s::String, sub::SubString, newval::AbstractString) = *( SubString(s, 1, prevind(s, sub.offset + 1, 1)), newval, SubString(s, nextind(s, sub.offset + sub.ncodeunits)) ) const dont_cdnify = ("new","open","shutdown","move","notebooklist","notebookfile","statefile","notebookexport","notebookupload") const source_pattern = r"\s(?:src|href)=\"(.+?)\"" function replace_with_cdn(cdnify::Function, s::String, idx::Integer=1) next_match = match(source_pattern, s, idx) if next_match === nothing s else url = only(next_match.captures) if occursin("//", url) || url  dont_cdnify # skip this one replace_with_cdn(cdnify, s, nextind(s, next_match.offset)) else replace_with_cdn(cdnify, replace_substring( s, url, cdnify(url) )) end end end """ Generate a custom index.html that is designed to display a custom set of featured notebooks, without the file UI or Pluto logo. This is to be used by [PlutoSliderServer.jl](https://github.com/JuliaPluto/PlutoSliderServer.jl) to show a fancy index page. """ function generate_index_html(; version::Union{Nothing,VersionNumber,AbstractString}=nothing, pluto_cdn_root::Union{Nothing,AbstractString}=nothing, featured_direct_html_links::Bool=false, featured_sources_js::AbstractString="undefined", ) cdnified = cdnified_html("index.html"; version, pluto_cdn_root) meta = """ <style> section#open, section#mywork, section#title { display: none !important; } </style> """ parameters = """ <script data-pluto-file="launch-parameters"> window.pluto_featured_direct_html_links = $(featured_direct_html_links ? "true" : "false"); window.pluto_featured_sources = $(featured_sources_js); </script> """ preload = prefetch_statefile_html(featured_sources_js) inserted_html(cdnified; meta, parameters, preload) end # The `Notebook` struct! import UUIDs: UUID, uuid1 import .Configuration import .PkgCompat: PkgCompat, PkgContext import Pkg import .Status const DEFAULT_NOTEBOOK_METADATA = Dict{String, Any}() mutable struct BondValue value::Any end function Base.convert(::Type{BondValue}, dict::AbstractDict) BondValue(dict["value"]) end const ProcessStatus = ( ready="ready", starting="starting", no_process="no_process", waiting_to_restart="waiting_to_restart", waiting_for_permission="waiting_for_permission", ) """ A Pluto notebook, yay!  This mutable struct is a notebook session. It contains the information loaded from the `.jl` file, the cell outputs, package information, execution metadata and more. """ Base.@kwdef mutable struct Notebook "Cells are ordered in a `Notebook`, and this order can be changed by the user. Cells will always have a constant UUID." cells_dict::Dict{UUID,Cell} cell_order::Vector{UUID} path::String notebook_id::UUID=uuid1() topology::NotebookTopology _cached_topological_order::TopologicalOrder _cached_cell_dependencies::Dict{UUID,Dict{String,Any}}=Dict{UUID,Dict{String,Any}}() _cached_cell_dependencies_source::Union{Nothing,NotebookTopology}=nothing # buffer will contain all unfetched updates - must be big enough # We can keep 1024 updates pending. After this, any put! calls (i.e. calls that push an update to the notebook) will simply block, which is fine. # This does mean that the Notebook can't be used if nothing is clearing the update channel. pendingupdates::Channel=Channel(1024) executetoken::Token=Token() # per notebook compiler options # nothing means to use global session compiler options compiler_options::Union{Nothing,Configuration.CompilerOptions}=nothing nbpkg_ctx::Union{Nothing,PkgContext}=nothing # nbpkg_ctx::Union{Nothing,PkgContext}=PkgCompat.create_empty_ctx() nbpkg_ctx_instantiated::Bool=false nbpkg_restart_recommended_msg::Union{Nothing,String}=nothing nbpkg_restart_required_msg::Union{Nothing,String}=nothing nbpkg_terminal_outputs::Dict{String,String}=Dict{String,String}() nbpkg_install_time_ns::Union{Nothing,UInt64}=zero(UInt64) nbpkg_busy_packages::Vector{String}=String[] nbpkg_installed_versions_cache::Dict{String,String}=Dict{String,String}() process_status::String=ProcessStatus.starting status_tree::Status.Business=_initial_nb_status() wants_to_interrupt::Bool=false last_save_time::Float64=time() last_hot_reload_time::Float64=zero(time()) bonds::Dict{Symbol,BondValue}=Dict{Symbol,BondValue}() metadata::Dict{String, Any}=copy(DEFAULT_NOTEBOOK_METADATA) end function _initial_nb_status() b = Status.Business(name=:notebook, started_at=time()) Status.report_business_planned!(b, :workspace) Status.report_business_planned!(b, :pkg) Status.report_business_planned!(b, :run) return b end function _report_business_cells_planned!(notebook::Notebook) run_status = Status.report_business_planned!(notebook.status_tree, :run) Status.report_business_planned!(run_status, :resolve_topology) cell_status = Status.report_business_planned!(run_status, :evaluate) for (i,c) in enumerate(notebook.cells) c.running = false c.queued = true Status.report_business_planned!(cell_status, Symbol(i)) end end _collect_cells(cells_dict::Dict{UUID,Cell}, cells_order::Vector{UUID}) = map(i -> cells_dict[i], cells_order) _initial_topology(cells_dict::Dict{UUID,Cell}, cells_order::Vector{UUID}) = NotebookTopology{Cell}(; cell_order=ImmutableVector(_collect_cells(cells_dict, cells_order)), ) function Notebook(cells::Vector{Cell}, @nospecialize(path::AbstractString), notebook_id::UUID) cells_dict=Dict(map(cells) do cell (cell.cell_id, cell) end) cell_order=map(x -> x.cell_id, cells) topology = _initial_topology(cells_dict, cell_order) Notebook(; cells_dict, cell_order, topology, _cached_topological_order=topological_order(topology), path, notebook_id ) end Notebook(cells::Vector{Cell}, path::AbstractString=numbered_until_new(joinpath(new_notebooks_directory(), cutename()))) = Notebook(cells, path, uuid1()) function Base.getproperty(notebook::Notebook, property::Symbol) # This is so that you can do notebook.cells to get all cells as a vector. if property == :cells _collect_cells(notebook.cells_dict, notebook.cell_order) # This is for Firebasey I think elseif property == :cell_inputs notebook.cells_dict else getfield(notebook, property) end end # New method for this function with a `Notebook` as input. function PlutoDependencyExplorer.topological_order(notebook::Notebook) cached = notebook._cached_topological_order if cached === nothing || cached.input_topology !== notebook.topology notebook._cached_topological_order = topological_order(notebook.topology) else cached end end emptynotebook(args...) = Notebook([Cell()], args...) function sample_notebook(name::String) file = project_relative_path("sample", name * ".jl") nb = load_notebook_nobackup(file) nb.path = tempname() * ".jl" nb end create_cell_metadata(metadata::Dict{String,<:Any}) = merge(DEFAULT_CELL_METADATA, metadata) create_notebook_metadata(metadata::Dict{String,<:Any}) = merge(DEFAULT_NOTEBOOK_METADATA, metadata) get_metadata(cell::Cell)::Dict{String,Any} = cell.metadata get_metadata(notebook::Notebook)::Dict{String,Any} = notebook.metadata get_metadata_no_default(cell::Cell)::Dict{String,Any} = Dict{String,Any}(setdiff(pairs(cell.metadata), pairs(DEFAULT_CELL_METADATA))) get_metadata_no_default(notebook::Notebook)::Dict{String,Any} = Dict{String,Any}(setdiff(pairs(notebook.metadata), pairs(DEFAULT_NOTEBOOK_METADATA))) const FrontMatter = Dict{String,Any} """ frontmatter(nb::Notebook; raise::Bool=false)::Dict{String,Any} frontmatter(nb_path::String; raise::Bool=false)::Dict{String,Any} Extract frontmatter from a notebook, which is extra meta-information that the author attaches to the notebook, often including *title*, *description*, *tags*, *author*, and more. Search for *frontmatter* online to learn more. If `raise` is true, then parsing errors will be rethrown. If `false`, this function will always return a `Dict`. """ function frontmatter(nb::Notebook; raise::Bool=false) convert(FrontMatter, get(() -> FrontMatter(), get_metadata(nb), "frontmatter") ) end function frontmatter(abs_path::String; raise::Bool=false) try # this will load the notebook to analyse, it won't run it frontmatter(load_notebook_nobackup(abs_path; skip_nbpkg=true); raise) catch e if raise rethrow(e) else @error "Error reading notebook file." abs_path exception=(e,catch_backtrace()) FrontMatter() end end end """ ```julia set_frontmatter!(nb::Notebook, new_value::Dict) ``` Set the new frontmatter of the [`Notebook`](@ref). Use [`frontmatter(nb)`](@ref) to get the old dictionary. If you want to save the file, call [`save_notebook(nb)`](@ref) afterwards. `set_frontmatter!(nb, nothing)` will delete the frontmatter. """ function set_frontmatter!(nb::Notebook, ::Nothing) delete!(nb.metadata, "frontmatter") end function set_frontmatter!(nb::Notebook, new_value::Dict) nb.metadata["frontmatter"] = convert(FrontMatter, new_value) end import Base64: base64decode # from https://github.com/JuliaLang/julia/pull/36425 function detectwsl() Sys.islinux() && isfile("/proc/sys/kernel/osrelease") && occursin(r"Microsoft|WSL"i, read("/proc/sys/kernel/osrelease", String)) end """ maybe_convert_path_to_wsl(path) Return the WSL path if the system is using the Windows Subsystem for Linux (WSL) and return `path` otherwise. WSL mounts the windows drive to /mnt/ and provides a utility tool to convert windows paths into WSL paths. This function will try to use this tool to automagically convert paths pasted from windows (with the right click -> copy as path functionality) into paths Pluto can understand. Example: $(raw"C:\Users\pankg\OneDrive\Desktop\pluto\bakery_pnl_ready2.jl")  "/mnt/c/Users/pankg/OneDrive/Desktop/pluto/bakery_pnl_ready2.jl" but "/mnt/c/Users/pankg/OneDrive/Desktop/pluto/bakery_pnl_ready2.jl" stays the same """ function maybe_convert_path_to_wsl(path) try isfile(path) && return path if detectwsl() # wslpath utility prints path to stderr if it fails to convert # (it used to fail for WSL-valid paths) !isnothing(match(r"^/mnt/\w+/", path)) && return path return readchomp(pipeline(`wslpath -u $(path)`; stderr=devnull)) end catch e return path end return path end const adjectives = [ "groundbreaking" "revolutionary" "important" "novel" "fun" "interesting" "fascinating" "exciting" "surprising" "remarkable" "wonderful" "stunning" "mini" "small" "tiny" "cute" "friendly" "wild" ] const nouns = [ "discovery" "experiment" "story" "journal" "notebook" "revelation" "computation" "creation" "analysis" "invention" "blueprint" "report" "science" "magic" "program" "notes" "lecture" "theory" "proof" "conjecture" ] """ Generate a filename like `"Cute discovery"`. Does not end with `.jl`. """ function cutename() titlecase(rand(adjectives)) * " " * rand(nouns) end function new_notebooks_directory() try path = get( ENV, "JULIA_PLUTO_NEW_NOTEBOOKS_DIR", joinpath(first(DEPOT_PATH), "pluto_notebooks") ) if !isdir(path) mkdir(path) end path catch homedir() end end """ Standard Pluto file extensions, including `.jl` and `.pluto.jl`. Pluto can open files with any extension, but the default extensions are used when searching for notebooks, or when trying to create a nice filename for something else, like the backup file. """ const pluto_file_extensions = [ ".pluto.jl", ".Pluto.jl", ".nb.jl", ".jl", ".plutojl", ".pluto", ".nbjl", ".pljl", ".pluto.jl.txt", # MacOS can create these .txt files sometimes ".jl.txt", ] endswith_pluto_file_extension(s) = any(endswith(s, e) for e in pluto_file_extensions) """ Extract the Julia notebook file contents from a Pluto-exported HTML file. """ function embedded_notebookfile(html_contents::AbstractString)::String if !occursin("</html>", html_contents) throw(ArgumentError("Pass the contents of a Pluto-exported HTML file as argument.")) end m = match(r"pluto_notebookfile.*\"data\:.*base64\,(.*)\"", html_contents) if m === nothing throw(ArgumentError("Notebook does not have an embedded notebook file.")) else String(base64decode(m.captures[1])) end end """ Does the path end with a pluto file extension (like `.jl` or `.pluto.jl`) and does the first line say `### A Pluto.jl notebook ###`? """ is_pluto_notebook(path::String) = endswith_pluto_file_extension(path) && readline(path) == "### A Pluto.jl notebook ###" function without_pluto_file_extension(s) for e in pluto_file_extensions if endswith(s, e) return s[1:prevind(s, ncodeunits(s), ncodeunits(e))] end end s end """ Return `base` * `suffix` if the file does not exist yet. If it does, return `base * sep * string(n) * suffix`, where `n` is the smallest natural number such that the file is new. (no 0 is not a natural number you snake) """ function numbered_until_new(base::AbstractString; sep::AbstractString=" ", suffix::AbstractString=".jl", create_file::Bool=true, skip_original::Bool=false) chosen = base * suffix n = 1 while (n == 1 && skip_original) || isfile(chosen) chosen = base * sep * string(n) * suffix n += 1 end if create_file touch(chosen) end chosen end backup_filename(path) = numbered_until_new(without_pluto_file_extension(path); sep=" backup ", suffix=".jl", create_file=false, skip_original=true) "Like `cp` except we create the file manually (to fix permission issues). (It's not plagiarism if you use this function to copy homework.)" function readwrite(from::AbstractString, to::AbstractString) write(to, read(from, String)) end function tryexpanduser(path) try expanduser(path) catch ex path end end const tamepath = abspath  tryexpanduser "Block until reading the file two times in a row gave the same result." function wait_until_file_unchanged(filename::String, timeout::Real, last_contents::String="-=-=-=-")::Nothing new_contents = try read(filename, String) catch "" end @info "Waiting for file to stabilize..."# last_contents new_contents if last_contents == new_contents # yayyy return else sleep(timeout) wait_until_file_unchanged(filename, timeout, new_contents) end end import TOML import UUIDs: UUID const _notebook_header = "### A Pluto.jl notebook ###" const _notebook_metadata_prefix = "#> " # We use a creative delimiter to avoid accidental use in code # so don't get inspired to suddenly use these in your code! const _cell_id_delimiter = "#  " const _cell_metadata_prefix = "#  " const _order_delimiter = "# " const _order_delimiter_folded = "# " const _cell_suffix = "\n\n" const _disabled_prefix = "#=\n" const _disabled_suffix = "\n  =#" const _ptoml_cell_id = UUID(1) const _mtoml_cell_id = UUID(2) ### # SAVING ### """ Save the notebook to `io`, `file` or to `notebook.path`. In the produced file, cells are not saved in the notebook order. If `notebook.topology` is up-to-date, I will save cells in _topological order_. This guarantees that you can run the notebook file outside of Pluto, with `julia my_notebook.jl`. Have a look at our [JuliaCon 2020 presentation](https://youtu.be/IAF8DjrQSSk?t=1085) to learn more! """ function save_notebook(io::IO, notebook::Notebook) println(io, _notebook_header) println(io, "# ", PLUTO_VERSION_STR) # Notebook metadata let nb_metadata_toml = strip(sprint(TOML.print, get_metadata_no_default(notebook))) if !isempty(nb_metadata_toml) println(io) for line in split(nb_metadata_toml, "\n") println(io, _notebook_metadata_prefix, line) end end end # (Anything between the version string and the first UUID delimiter will be ignored by the notebook loader.) # We insert these two imports because they are also imported by default in the Pluto session. You might use these packages in your code, so we add the imports to the file, so the file can run as a script. println(io, "") println(io, "using Markdown") println(io, "using InteractiveUtils") # Super Advanced Code Analysis to add the @bind macro to the saved file if it's used somewhere. if any(!must_be_commented_in_file(c) && occursin("@bind", c.code) for c in notebook.cells) println(io, "") println(io, "# This Pluto notebook uses @bind for interactivity. When running this notebook outside of Pluto, the following 'mock version' of @bind gives bound variables a default value (instead of an error).") println(io, PlutoRunner.fake_bind) end println(io) cells_ordered = collect(topological_order(notebook)) # NOTE: the notebook topological is cached on every update_dependency! call # .... so it is possible that a cell was added/removed since this last update. # .... in this case, it will not contain that cell since it is build from its # .... store notebook topology. therefore, we compute an updated topological # .... order in this unlikely case. if length(cells_ordered) != length(notebook.cells_dict) cells = notebook.cells updated_topo = updated_topology(notebook.topology, notebook, cells) cells_ordered = collect(topological_order(updated_topo, cells)) end for c in cells_ordered println(io, _cell_id_delimiter, string(c.cell_id)) let metadata_toml = strip(sprint(TOML.print, get_metadata_no_default(c))) if metadata_toml != "" for line in split(metadata_toml, "\n") println(io, _cell_metadata_prefix, line) end end end # Do one little string replacement to make it impossible to use the Pluto cell delimiter inside of actual cell code. If this would happen, then the notebook file cannot load correctly. So we just remove it from your code (sorry!) current_code = replace(c.code, _cell_id_delimiter => "# ") if must_be_commented_in_file(c) print(io, _disabled_prefix) print(io, current_code) print(io, _disabled_suffix) print(io, _cell_suffix) else # write the cell code and prevent collisions with the cell delimiter print(io, current_code) print(io, _cell_suffix) end end using_plutopkg = notebook.nbpkg_ctx !== nothing write_package = if using_plutopkg ptoml_contents = PkgCompat.read_project_file(notebook) mtoml_contents = PkgCompat.read_manifest_file(notebook) !isempty(strip(ptoml_contents)) else false end if write_package println(io, _cell_id_delimiter, string(_ptoml_cell_id)) print(io, "PLUTO_PROJECT_TOML_CONTENTS = \"\"\"\n") write(io, ptoml_contents) print(io, "\"\"\"") print(io, _cell_suffix) println(io, _cell_id_delimiter, string(_mtoml_cell_id)) print(io, "PLUTO_MANIFEST_TOML_CONTENTS = \"\"\"\n") write(io, mtoml_contents) print(io, "\"\"\"") print(io, _cell_suffix) end println(io, _cell_id_delimiter, "Cell order:") for c in notebook.cells delim = c.code_folded ? _order_delimiter_folded : _order_delimiter println(io, delim, string(c.cell_id)) end if write_package println(io, _order_delimiter_folded, string(_ptoml_cell_id)) println(io, _order_delimiter_folded, string(_mtoml_cell_id)) end notebook end # UTILS function write_buffered(fn::Function, path) file_content = sprint(fn) write(path, file_content) end function save_notebook(notebook::Notebook, path::String) # @warn "Saving to file!!" exception=(ErrorException(""), backtrace()) notebook.last_save_time = time() Status.report_business!(notebook.status_tree, :saving) do write_buffered(path) do io save_notebook(io, notebook) end end end save_notebook(notebook::Notebook) = save_notebook(notebook, notebook.path) ### # LOADING ### function _read_notebook_metadata!(@nospecialize(io::IO)) firstline = String(readline(io))::String if firstline != _notebook_header error( if occursin("<!DOCTYPE", firstline) || occursin("<html", firstline) """File is an HTML file, not a notebook file. Open the file directly, and click the "Edit or run" button to get the notebook file.""" else "File is not a Pluto.jl notebook." end ) end file_VERSION_STR = readline(io)[3:end] if file_VERSION_STR != PLUTO_VERSION_STR # @info "Loading a notebook saved with Pluto $(file_VERSION_STR). This is Pluto $(PLUTO_VERSION_STR)." end # Read all remaining file contents before the first cell delimiter. header_content = readuntil(io, _cell_id_delimiter) header_lines = split(header_content, "\n") nb_prefix_length = ncodeunits(_notebook_metadata_prefix) nb_metadata_toml_lines = String[ line[begin+nb_prefix_length:end] for line in header_lines if startswith(line, _notebook_metadata_prefix) ] notebook_metadata = try create_notebook_metadata(TOML.parse(join(nb_metadata_toml_lines, "\n"))) catch e @error "Failed to parse embedded TOML content" exception=(e, catch_backtrace()) DEFAULT_NOTEBOOK_METADATA end return notebook_metadata end function _read_notebook_collected_cells!(@nospecialize(io::IO)) collected_cells = Dict{UUID,Cell}() while !eof(io) cell_id_str = String(readline(io)) if cell_id_str == "Cell order:" break else cell_id = UUID(cell_id_str) metadata_toml_lines = String[] initial_code_line = "" while !eof(io) line = String(readline(io)) if startswith(line, _cell_metadata_prefix) prefix_length = ncodeunits(_cell_metadata_prefix) push!(metadata_toml_lines, line[begin+prefix_length:end]) else initial_code_line = line break end end code_raw = initial_code_line * "\n" * String(readuntil(io, _cell_id_delimiter)) # change Windows line endings to Linux code_normalised = replace(code_raw, "\r\n" => "\n") # remove the disabled on startup comments for further processing in Julia code_normalised = replace(replace(code_normalised, _disabled_prefix => ""), _disabled_suffix => "") # remove the cell suffix code = code_normalised[1:prevind(code_normalised, end, length(_cell_suffix))] # parse metadata metadata = try create_cell_metadata(TOML.parse(join(metadata_toml_lines, "\n"))) catch @error "Failed to parse embedded TOML content" cell_id exception=(e, catch_backtrace()) DEFAULT_CELL_METADATA end read_cell = Cell(; cell_id, code, metadata) collected_cells[cell_id] = read_cell end end return collected_cells end function _read_notebook_cell_order!(@nospecialize(io::IO), collected_cells) cell_order = UUID[] while !eof(io) cell_id_str = String(readline(io)) if length(cell_id_str) >= 36 && (startswith(cell_id_str, _order_delimiter_folded) || startswith(cell_id_str, _order_delimiter)) cell_id = let UUID(cell_id_str[end - 35:end]) end next_cell = get(collected_cells, cell_id, nothing) if next_cell !== nothing next_cell.code_folded = startswith(cell_id_str, _order_delimiter_folded) end push!(cell_order, cell_id) else break end end return cell_order end function _read_notebook_nbpkg_ctx(cell_order::Vector{UUID}, collected_cells::Dict{Base.UUID, Cell}) read_package = _ptoml_cell_id  cell_order && _mtoml_cell_id  cell_order && haskey(collected_cells, _ptoml_cell_id) && haskey(collected_cells, _mtoml_cell_id) nbpkg_ctx = if read_package ptoml_code = string(collected_cells[_ptoml_cell_id].code)::String mtoml_code = string(collected_cells[_mtoml_cell_id].code)::String ptoml_contents = lstrip(split(ptoml_code, "\"\"\"")[2]) mtoml_contents = lstrip(split(mtoml_code, "\"\"\"")[2]) env_dir = mktempdir() write(joinpath(env_dir, "Project.toml"), ptoml_contents) write(joinpath(env_dir, "Manifest.toml"), mtoml_contents) try PkgCompat.load_ctx(env_dir) catch e @error "Failed to load notebook files: Project.toml+Manifest.toml parse error. Trying to recover Project.toml without Manifest.toml..." exception=(e,catch_backtrace()) try rm(joinpath(env_dir, "Manifest.toml")) PkgCompat.load_ctx(env_dir) catch e @error "Failed to load notebook files: Project.toml parse error." exception=(e,catch_backtrace()) PkgCompat.create_empty_ctx() end end else PkgCompat.create_empty_ctx() end return nbpkg_ctx end function _read_notebook_appeared_order!(cell_order::Vector{UUID}, collected_cells::Dict{Base.UUID, Cell}) setdiff!( union!( # don't include cells that only appear in the order, but no code was given intersect!(cell_order, keys(collected_cells)), # add cells that appeared in code, but not in the order. keys(collected_cells) ), # remove Pkg cells (_ptoml_cell_id, _mtoml_cell_id) ) end "Load a notebook without saving it or creating a backup; returns a `Notebook`. REMEMBER TO CHANGE THE NOTEBOOK PATH after loading it to prevent it from autosaving and overwriting the original file." function load_notebook_nobackup(@nospecialize(io::IO), @nospecialize(path::AbstractString); skip_nbpkg::Bool=false)::Notebook notebook_metadata = _read_notebook_metadata!(io) collected_cells = _read_notebook_collected_cells!(io) cell_order = _read_notebook_cell_order!(io, collected_cells) nbpkg_ctx = skip_nbpkg ? nothing : _read_notebook_nbpkg_ctx(cell_order, collected_cells) appeared_order = _read_notebook_appeared_order!(cell_order, collected_cells) appeared_cells_dict = filter(collected_cells) do (k, v) k  appeared_order end topology = _initial_topology(appeared_cells_dict, appeared_order) Notebook(; cells_dict=appeared_cells_dict, cell_order=appeared_order, topology, _cached_topological_order=topological_order(topology), path, nbpkg_ctx, nbpkg_installed_versions_cache=nbpkg_cache(nbpkg_ctx), metadata=notebook_metadata, ) end # UTILS function load_notebook_nobackup(path::String; kwargs...)::Notebook open(path, "r") do io load_notebook_nobackup(io, path; kwargs...) end end # BACKUPS "Create a backup of the given file, load the file as a .jl Pluto notebook, save the loaded notebook, compare the two files, and delete the backup of the newly saved file is mostly equal to the backup." function load_notebook(path::String; disable_writing_notebook_files::Bool=false)::Notebook backup_path = backup_filename(path) # local backup_num = 1 # backup_path = path # while isfile(backup_path) # backup_path = path * ".backup" * string(backup_num) # backup_num += 1 # end disable_writing_notebook_files || readwrite(path, backup_path) loaded = load_notebook_nobackup(path) # Analyze cells so that the initial save is in topological order loaded.topology = updated_topology(loaded.topology, loaded, loaded.cells) |> static_resolve_topology # We update cell dependency on skip_as_script and disabled to avoid removing block comments on the file. See https://github.com/fonsp/Pluto.jl/issues/2182 update_disabled_cells_dependency!(loaded) update_skipped_cells_dependency!(loaded) update_dependency_cache!(loaded) disable_writing_notebook_files || save_notebook(loaded) loaded.topology = NotebookTopology{Cell}(; cell_order=ImmutableVector(loaded.cells)) disable_writing_notebook_files || if only_versions_or_lineorder_differ(path, backup_path) rm(backup_path) else @warn "Old Pluto notebook might not have loaded correctly. Backup saved to: " backup_path end loaded end _after_first_cell(lines) = lines[something(findfirst(startswith(_cell_id_delimiter), lines), 1):end] """ Check if two savefiles are identical, up to their version numbers and a possible line shuffle. If a notebook has not yet had all of its cells analysed, we can't deduce the topological cell order. (but can we ever??) (no) """ function only_versions_or_lineorder_differ(pathA::AbstractString, pathB::AbstractString)::Bool Set(readlines(pathA) |> _after_first_cell) == Set(readlines(pathB) |> _after_first_cell) end function only_versions_differ(pathA::AbstractString, pathB::AbstractString)::Bool readlines(pathA) |> _after_first_cell == readlines(pathB) |> _after_first_cell end "Set `notebook.path` to the new value, save the notebook, verify file integrity, and if all OK, delete the old savefile. Normalizes the given path to make it absolute. Moving is always hard. " function move_notebook!(notebook::Notebook, newpath::String; disable_writing_notebook_files::Bool=false) # Will throw exception and return if anything goes wrong, so at least one file is guaranteed to exist. oldpath_tame = tamepath(notebook.path) newpath_tame = tamepath(newpath) if !disable_writing_notebook_files save_notebook(notebook, oldpath_tame) save_notebook(notebook, newpath_tame) # @assert that the new file looks alright @assert only_versions_differ(oldpath_tame, newpath_tame) notebook.path = newpath_tame if oldpath_tame != newpath_tame rm(oldpath_tame) end else notebook.path = newpath_tame end if isdir("$oldpath_tame.assets") mv("$oldpath_tame.assets", "$newpath_tame.assets") end notebook end ### A Pluto.jl notebook ### # v0.19.24 using Markdown using InteractiveUtils #  6f1cb799-21b7-459a-a7c5-6c42b1376b11 #  skip_as_script = true #= using BenchmarkTools: @benchmark  =# #  98ac036d-1a37-457c-b89d-8e954ab6f039 #  skip_as_script = true #= using PlutoUI: Slider  =# #  e665ae4e-bf4e-11ed-330a-f3670dd00a95 #  skip_as_script = true #= str = read(download("https://raw.githubusercontent.com/fonsp/disorganised-mess/main/ansidemo.txt"), String)  =# #  032a10cc-8b2b-4e85-bac4-b9623b91d016 #  disabled = true #  skip_as_script = true #= str = "\nInstantiating...\n\e[32m\e[1m No Changes\e[22m\e[39m to `/private/var/folders/v_/fhpj9jn151d4p9c2fdw2gv780000gn/T/jl_DivOhV/Project.toml`\n\e[32m\e[1m No Changes\e[22m\e[39m to `/private/var/folders/v_/fhpj9jn151d4p9c2fdw2gv780000gn/T/jl_DivOhV/Manifest.toml`\n\e[?25l\e[?25h\e[2K\nResolving...\n\e[32m\e[1m No Changes\e[22m\e[39m to `/private/var/folders/v_/fhpj9jn151d4p9c2fdw2gv780000gn/T/jl_DivOhV/Project.toml`\n\e[32m\e[1m No Changes\e[22m\e[39m to `/private/var/folders/v_/fhpj9jn151d4p9c2fdw2gv780000gn/T/jl_DivOhV/Manifest.toml`\n\e[?25l\e[?25h\e[2K"  =# #  209f221c-be89-4976-b712-2b9214c3291c #= Text(str)  =# #  dff16b61-7fa6-4702-95a6-a0d2913c3070 #= collect(str)  =# #  b6fe176d-cfbe-4aac-ad46-3345b0acbb74 #= length(str)  =# #  c952da01-ba67-4a49-99df-c61939ba81be # ESC[H moves cursor to home position (0, 0) # ESC[{line};{column}H # ESC[{line};{column}f moves cursor to line #, column # # ESC[#A moves cursor up # lines # ESC[#B moves cursor down # lines # ESC[#C moves cursor right # columns # ESC[#D moves cursor left # columns # ESC[#E moves cursor to beginning of next line, # lines down # ESC[#F moves cursor to beginning of previous line, # lines up # ESC[#G moves cursor to column # # ESC[6n request cursor position (reports as ESC[#;#R) # ESC M moves cursor one line up, scrolling if needed # ESC 7 save cursor position (DEC) # ESC 8 restores the cursor to the last saved position (DEC) # ESC[s save cursor position (SCO) # ESC[u restores the cursor to the last saved position (SCO) #  f661240f-4950-4be0-a2b2-98ed77f6e4a2 const stoppers = ('H', 'f', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'n', 's', 'u', 'J', 'K', 'm', 'l', 'h', 'S') #  af48f80f-345d-4d90-9cf1-b8be9004a53d Base.@kwdef mutable struct ANSITerminalState lines::Dict{Int,Vector{Char}}=Dict{Int,Vector{Char}}() col::Int=1 row::Int=1 end #  f54f7282-87fd-4f05-8f8a-220d83186e9a getline(state::ANSITerminalState) = get!(state.lines, state.row) do Char[] end #  0a4587f7-55b2-4621-89e8-a9492d32bc09 #= @benchmark ncodeunits(str)  =# #  84a457d4-1e3c-4797-aa94-a7ca9ce628e4 """ Update the `ANSITerminalState` with new data from a TTY stream. """ function consume!(state::ANSITerminalState, str::AbstractString) ind = 0 L = ncodeunits(str) while ind < L ind = nextind(str, ind) if ind > L break end c = str[ind] if c === '\n' state.col = 1 state.row += 1 elseif c === '\r' state.col = 1 elseif c === '\e' # ANSI control character # see https://en.wikipedia.org/wiki/ANSI_escape_code for descriptions of these. ind += 1 # ignoring this character, it should be a '[' "will contain the characters between ] and the stopper character" buffer_vec = Char[] # keep reading until we have the stopper character local stopper = '\0' while ind <= L - 1 ind += 1 next_c = str[ind] if next_c  stoppers stopper = next_c break end push!(buffer_vec, next_c) end # @info "Escape sequence read" stopper buffer if stopper === 'l' || stopper === 'h' # ignored elseif stopper === '\0' # this means that no stop was found, ignoring... elseif stopper === 'K'# && buffer == "2" # @assert buffer_vec == ['2'] line = getline(state) line .= ' ' elseif stopper === 'A' state.row = max(1, state.row - parse(Int, String(buffer_vec))) elseif stopper === 'G' state.col = parse(Int, String(buffer_vec)) elseif stopper === 'J'# && buffer == "0" # @assert buffer_vec == ['0'] # clear the remainder of this line resize!(getline(state), state.col - 1) # and clear all rows below for row in state.row+1 : maximum(keys(state.lines)) delete!(state.lines, row) end elseif stopper === 'm' # keep it in the output because we use ansiup to handle colors in the frontend line = getline(state) push!(line, '\e', '[') append!(line, buffer_vec) push!(line, stopper) state.col += 3 + length(buffer_vec) elseif stopper === 'S' diff = isempty(buffer_vec) ? 1 : parse(Int, String(buffer_vec)) state.row += diff else @warn "Unrecogized escape sequence" stopper String(buffer_vec) end else # no escape character, just a regular char line = getline(state) while length(line) < state.col push!(line, ' ') end line[state.col] = c state.col += 1 end end end #  2c02ca66-e3fe-479a-b3f1-13ea0850f3d7 function consume_safe!(state::ANSITerminalState, str::AbstractString) try consume!(state, str) catch e @debug "ANSI escape sequence glitch" exception=(e,catch_backtrace()) line = getline(state) append!(line, codeunits(str)) state.row += ncodeunits(str) end end #  6a8f71f5-7113-4074-8009-14eea11cc958 #= @bind L Slider(1:length(str))  =# #  f9232436-618c-401f-9d18-f9f8f1f04a77 function build_str(state::ANSITerminalState) d = state.lines join( (String(get(() -> Char[], d, i)) for i in 1:maximum(keys(d))), "\n" ) end #  a1ebfaf3-dadb-40dc-95c5-ae8a9d6de1ec #  skip_as_script = true #= function solve(str::AbstractString) state = ANSITerminalState() consume_safe!(state, str) build_str(state) end  =# #  6dc56641-7d0d-4bd7-a1ef-22e373908207 #= @benchmark solve($str) seconds=1  =# #  48c686f9-722f-4d84-ac9d-9b11c934370c #= solve(str) |> Text  =# #  ee9a24d4-7a9f-4024-8100-f7ce4ef436cb #= collect(str) => collect(solve(str))  =# #  2453d8e0-ca26-4033-a272-c5c51fc8d16d #= solve(SubString(str, 1, nextind(str, 0, L))) |> Text  =# #  535f471d-a049-4d74-aa52-9be86e2d4352 #= let state = ANSITerminalState() mid = nextind(str, 0, L  2) top = nextind(str, 0, L) consume!(state, SubString(str, 1, mid)) consume!(state, SubString(str, nextind(str, mid), top)) build_str(state) |> Text end  =# #  195a8e3c-427f-487f-9c7c-d31ce374de81 #  skip_as_script = true #= ANSITerminalState( lines=Dict(3 => Char['a', 't'], 1 => ['c']) ) |> build_str |> Text  =# #  00000000-0000-0000-0000-000000000001 PLUTO_PROJECT_TOML_CONTENTS = """ [deps] BenchmarkTools = "6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf" PlutoUI = "7f904dfe-b85e-4ff6-b463-dae2292396a8" [compat] BenchmarkTools = "~1.3.2" PlutoUI = "~0.7.50" """ #  00000000-0000-0000-0000-000000000002 PLUTO_MANIFEST_TOML_CONTENTS = """ # This file is machine-generated - editing it directly is not advised julia_version = "1.9.0-rc2" manifest_format = "2.0" project_hash = "ce886088241d89ce6a9168ebf44701e0c6f4421f" [[deps.AbstractPlutoDingetjes]] deps = ["Pkg"] git-tree-sha1 = "8eaf9f1b4921132a4cff3f36a1d9ba923b14a481" uuid = "6e696c72-6542-2067-7265-42206c756150" version = "1.1.4" [[deps.ArgTools]] uuid = "0dad84c5-d112-42e6-8d28-ef12dabb789f" version = "1.1.1" [[deps.Artifacts]] uuid = "56f22d72-fd6d-98f1-02f0-08ddc0907c33" [[deps.Base64]] uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f" [[deps.BenchmarkTools]] deps = ["JSON", "Logging", "Printf", "Profile", "Statistics", "UUIDs"] git-tree-sha1 = "d9a9701b899b30332bbcb3e1679c41cce81fb0e8" uuid = "6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf" version = "1.3.2" [[deps.ColorTypes]] deps = ["FixedPointNumbers", "Random"] git-tree-sha1 = "eb7f0f8307f71fac7c606984ea5fb2817275d6e4" uuid = "3da002f7-5984-5a60-b8a6-cbb66c0b333f" version = "0.11.4" [[deps.CompilerSupportLibraries_jll]] deps = ["Artifacts", "Libdl"] uuid = "e66e0078-7015-5450-92f7-15fbd957f2ae" version = "1.0.2+0" [[deps.Dates]] deps = ["Printf"] uuid = "ade2ca70-3891-5945-98fb-dc099432e06a" [[deps.Downloads]] deps = ["ArgTools", "FileWatching", "LibCURL", "NetworkOptions"] uuid = "f43a241f-c20a-4ad4-852c-f6b1247861c6" version = "1.6.0" [[deps.FileWatching]] uuid = "7b1f6079-737a-58dc-b8bc-7a2ca5c1b5ee" [[deps.FixedPointNumbers]] deps = ["Statistics"] git-tree-sha1 = "335bfdceacc84c5cdf16aadc768aa5ddfc5383cc" uuid = "53c48c17-4a7d-5ca2-90c5-79b7896eea93" version = "0.8.4" [[deps.Hyperscript]] deps = ["Test"] git-tree-sha1 = "8d511d5b81240fc8e6802386302675bdf47737b9" uuid = "47d2ed2b-36de-50cf-bf87-49c2cf4b8b91" version = "0.0.4" [[deps.HypertextLiteral]] deps = ["Tricks"] git-tree-sha1 = "c47c5fa4c5308f27ccaac35504858d8914e102f9" uuid = "ac1192a8-f4b3-4bfe-ba22-af5b92cd3ab2" version = "0.9.4" [[deps.IOCapture]] deps = ["Logging", "Random"] git-tree-sha1 = "f7be53659ab06ddc986428d3a9dcc95f6fa6705a" uuid = "b5f81e59-6552-4d32-b1f0-c071b021bf89" version = "0.2.2" [[deps.InteractiveUtils]] deps = ["Markdown"] uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240" [[deps.JSON]] deps = ["Dates", "Mmap", "Parsers", "Unicode"] git-tree-sha1 = "3c837543ddb02250ef42f4738347454f95079d4e" uuid = "682c06a0-de6a-54ab-a142-c8b1cf79cde6" version = "0.21.3" [[deps.LibCURL]] deps = ["LibCURL_jll", "MozillaCACerts_jll"] uuid = "b27032c2-a3e7-50c8-80cd-2d36dbcbfd21" version = "0.6.3" [[deps.LibCURL_jll]] deps = ["Artifacts", "LibSSH2_jll", "Libdl", "MbedTLS_jll", "Zlib_jll", "nghttp2_jll"] uuid = "deac9b47-8bc7-5906-a0fe-35ac56dc84c0" version = "7.84.0+0" [[deps.LibGit2]] deps = ["Base64", "NetworkOptions", "Printf", "SHA"] uuid = "76f85450-5226-5b5a-8eaa-529ad045b433" [[deps.LibSSH2_jll]] deps = ["Artifacts", "Libdl", "MbedTLS_jll"] uuid = "29816b5a-b9ab-546f-933c-edad1886dfa8" version = "1.10.2+0" [[deps.Libdl]] uuid = "8f399da3-3557-5675-b5ff-fb832c97cbdb" [[deps.LinearAlgebra]] deps = ["Libdl", "OpenBLAS_jll", "libblastrampoline_jll"] uuid = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e" [[deps.Logging]] uuid = "56ddb016-857b-54e1-b83d-db4d58db5568" [[deps.MIMEs]] git-tree-sha1 = "65f28ad4b594aebe22157d6fac869786a255b7eb" uuid = "6c6e2e6c-3030-632d-7369-2d6c69616d65" version = "0.1.4" [[deps.Markdown]] deps = ["Base64"] uuid = "d6f4376e-aef5-505a-96c1-9c027394607a" [[deps.MbedTLS_jll]] deps = ["Artifacts", "Libdl"] uuid = "c8ffd9c3-330d-5841-b78e-0817d7145fa1" version = "2.28.2+0" [[deps.Mmap]] uuid = "a63ad114-7e13-5084-954f-fe012c677804" [[deps.MozillaCACerts_jll]] uuid = "14a3606d-f60d-562e-9121-12d972cd8159" version = "2022.10.11" [[deps.NetworkOptions]] uuid = "ca575930-c2e3-43a9-ace4-1e988b2c1908" version = "1.2.0" [[deps.OpenBLAS_jll]] deps = ["Artifacts", "CompilerSupportLibraries_jll", "Libdl"] uuid = "4536629a-c528-5b80-bd46-f80d51c5b363" version = "0.3.21+4" [[deps.Parsers]] deps = ["Dates", "SnoopPrecompile"] git-tree-sha1 = "478ac6c952fddd4399e71d4779797c538d0ff2bf" uuid = "69de0a69-1ddd-5017-9359-2bf0b02dc9f0" version = "2.5.8" [[deps.Pkg]] deps = ["Artifacts", "Dates", "Downloads", "FileWatching", "LibGit2", "Libdl", "Logging", "Markdown", "Printf", "REPL", "Random", "SHA", "Serialization", "TOML", "Tar", "UUIDs", "p7zip_jll"] uuid = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f" version = "1.9.0" [[deps.PlutoUI]] deps = ["AbstractPlutoDingetjes", "Base64", "ColorTypes", "Dates", "FixedPointNumbers", "Hyperscript", "HypertextLiteral", "IOCapture", "InteractiveUtils", "JSON", "Logging", "MIMEs", "Markdown", "Random", "Reexport", "URIs", "UUIDs"] git-tree-sha1 = "5bb5129fdd62a2bbbe17c2756932259acf467386" uuid = "7f904dfe-b85e-4ff6-b463-dae2292396a8" version = "0.7.50" [[deps.Preferences]] deps = ["TOML"] git-tree-sha1 = "47e5f437cc0e7ef2ce8406ce1e7e24d44915f88d" uuid = "21216c6a-2e73-6563-6e65-726566657250" version = "1.3.0" [[deps.Printf]] deps = ["Unicode"] uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7" [[deps.Profile]] deps = ["Printf"] uuid = "9abbd945-dff8-562f-b5e8-e1ebf5ef1b79" [[deps.REPL]] deps = ["InteractiveUtils", "Markdown", "Sockets", "Unicode"] uuid = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb" [[deps.Random]] deps = ["SHA", "Serialization"] uuid = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c" [[deps.Reexport]] git-tree-sha1 = "45e428421666073eab6f2da5c9d310d99bb12f9b" uuid = "189a3867-3050-52da-a836-e630ba90ab69" version = "1.2.2" [[deps.SHA]] uuid = "ea8e919c-243c-51af-8825-aaa63cd721ce" version = "0.7.0" [[deps.Serialization]] uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b" [[deps.SnoopPrecompile]] deps = ["Preferences"] git-tree-sha1 = "e760a70afdcd461cf01a575947738d359234665c" uuid = "66db9d55-30c0-4569-8b51-7e840670fc0c" version = "1.0.3" [[deps.Sockets]] uuid = "6462fe0b-24de-5631-8697-dd941f90decc" [[deps.SparseArrays]] deps = ["Libdl", "LinearAlgebra", "Random", "Serialization", "SuiteSparse_jll"] uuid = "2f01184e-e22b-5df5-ae63-d93ebab69eaf" [[deps.Statistics]] deps = ["LinearAlgebra", "SparseArrays"] uuid = "10745b16-79ce-11e8-11f9-7d13ad32a3b2" version = "1.9.0" [[deps.SuiteSparse_jll]] deps = ["Artifacts", "Libdl", "Pkg", "libblastrampoline_jll"] uuid = "bea87d4a-7f5b-5778-9afe-8cc45184846c" version = "5.10.1+6" [[deps.TOML]] deps = ["Dates"] uuid = "fa267f1f-6049-4f14-aa54-33bafae1ed76" version = "1.0.3" [[deps.Tar]] deps = ["ArgTools", "SHA"] uuid = "a4e569a6-e804-4fa4-b0f3-eef7a1d5b13e" version = "1.10.0" [[deps.Test]] deps = ["InteractiveUtils", "Logging", "Random", "Serialization"] uuid = "8dfed614-e22c-5e08-85e1-65c5234f0b40" [[deps.Tricks]] git-tree-sha1 = "6bac775f2d42a611cdfcd1fb217ee719630c4175" uuid = "410a4b4d-49e4-4fbc-ab6d-cb71b17b3775" version = "0.1.6" [[deps.URIs]] git-tree-sha1 = "074f993b0ca030848b897beff716d93aca60f06a" uuid = "5c2747f8-b7ea-4ff2-ba2e-563bfd36b1d4" version = "1.4.2" [[deps.UUIDs]] deps = ["Random", "SHA"] uuid = "cf7118a7-6976-5b1a-9a39-7adc72f591a4" [[deps.Unicode]] uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5" [[deps.Zlib_jll]] deps = ["Libdl"] uuid = "83775a58-1f1d-513f-b197-d71354ab007a" version = "1.2.13+0" [[deps.libblastrampoline_jll]] deps = ["Artifacts", "Libdl"] uuid = "8e850b90-86db-534c-a0d3-1478176c7d93" version = "5.4.0+0" [[deps.nghttp2_jll]] deps = ["Artifacts", "Libdl"] uuid = "8e850ede-7688-5339-a07c-302acd2aaf8d" version = "1.48.0+0" [[deps.p7zip_jll]] deps = ["Artifacts", "Libdl"] uuid = "3f19e933-33d8-53b3-aaab-bd5110c3b7a0" version = "17.4.0+0" """ #  Cell order: # e665ae4e-bf4e-11ed-330a-f3670dd00a95 # 032a10cc-8b2b-4e85-bac4-b9623b91d016 # 209f221c-be89-4976-b712-2b9214c3291c # dff16b61-7fa6-4702-95a6-a0d2913c3070 # b6fe176d-cfbe-4aac-ad46-3345b0acbb74 # c952da01-ba67-4a49-99df-c61939ba81be # f661240f-4950-4be0-a2b2-98ed77f6e4a2 # af48f80f-345d-4d90-9cf1-b8be9004a53d # f54f7282-87fd-4f05-8f8a-220d83186e9a # 0a4587f7-55b2-4621-89e8-a9492d32bc09 # 2c02ca66-e3fe-479a-b3f1-13ea0850f3d7 # 84a457d4-1e3c-4797-aa94-a7ca9ce628e4 # a1ebfaf3-dadb-40dc-95c5-ae8a9d6de1ec # 6dc56641-7d0d-4bd7-a1ef-22e373908207 # 6f1cb799-21b7-459a-a7c5-6c42b1376b11 # 48c686f9-722f-4d84-ac9d-9b11c934370c # ee9a24d4-7a9f-4024-8100-f7ce4ef436cb # 2453d8e0-ca26-4033-a272-c5c51fc8d16d # 535f471d-a049-4d74-aa52-9be86e2d4352 # 98ac036d-1a37-457c-b89d-8e954ab6f039 # 6a8f71f5-7113-4074-8009-14eea11cc958 # f9232436-618c-401f-9d18-f9f8f1f04a77 # 195a8e3c-427f-487f-9c7c-d31ce374de81 # 00000000-0000-0000-0000-000000000001 # 00000000-0000-0000-0000-000000000002 module ANSIEmulation include("./ANSIEmulation.jl") end "A polling system to watch for writes to a `Base.BufferStream`. Up-to-date content will be passed as string to the `callback` function." Base.@kwdef struct IOListener callback::Function interval::Real=1.0/60 running::Ref{Bool}=Ref(false) buffer::Base.BufferStream=Base.BufferStream() ansi_state::ANSIEmulation.ANSITerminalState=ANSIEmulation.ANSITerminalState() end function trigger(listener::IOListener) if !eof(listener.buffer) && isreadable(listener.buffer) newdata = readavailable(listener.buffer) isempty(newdata) && return s = String(newdata) ANSIEmulation.consume_safe!( listener.ansi_state, s ) new_contents = ANSIEmulation.build_str(listener.ansi_state) listener.callback(new_contents) end end function startlistening(listener::IOListener) if !listener.running[] listener.running[] = true @async try while listener.running[] trigger(listener) sleep(listener.interval) end catch ex println(stderr, "IOListener loop error") showerror(stderr, ex, stacktrace(catch_backtrace())) rethrow(ex) end end end function stoplistening(listener::IOListener) if listener.running[] listener.running[] = false bytesavailable(listener.buffer) > 0 && trigger(listener) close(listener.buffer) end end freeze_loading_spinners(s::AbstractString) = replace(s, '' => '', '' => '', '' => '') phasemessage(iolistener, phase::String) = phasemessage(iolistener.buffer, phase) function phasemessage(io::IO, phase::String) ioc = IOContext(io, :color=>true) printstyled(ioc, "\n$phase...\n"; bold=true) printstyled(ioc, "===\n"; color=:light_black) end import ExpressionExplorer: external_package_names import .PkgCompat import .PkgCompat: select, is_stdlib import Logging import LoggingExtras import .Configuration: CompilerOptions, _merge_notebook_compiler_options, _convert_to_flags import GracefulPkg const tiers = unique(( Pkg.PRESERVE_ALL_INSTALLED, Pkg.PRESERVE_ALL, Pkg.PRESERVE_DIRECT, Pkg.PRESERVE_SEMVER, Pkg.PRESERVE_NONE, )) const pkg_token = Token() _default_cleanup() = nothing # This list appears multiple times in our codebase. Be sure to match edits everywhere. function use_plutopkg(topology::NotebookTopology) !any(values(topology.nodes)) do node Symbol("Pkg.activate")  node.references || Symbol("Pkg.API.activate")  node.references || Symbol("Pkg.develop")  node.references || Symbol("Pkg.API.develop")  node.references || Symbol("Pkg.add")  node.references || Symbol("Pkg.API.add")  node.references || Symbol("TestEnv.activate")  node.references || # https://juliadynamics.github.io/DrWatson.jl/dev/project/#DrWatson.quickactivate Symbol("quickactivate")  node.references || Symbol("@quickactivate")  node.references || Symbol("DrWatson.@quickactivate")  node.references || Symbol("DrWatson.quickactivate")  node.references end end PkgCompat.project_file(notebook::Notebook) = PkgCompat.project_file(PkgCompat.env_dir(notebook.nbpkg_ctx)) PkgCompat.manifest_file(notebook::Notebook) = PkgCompat.manifest_file(PkgCompat.env_dir(notebook.nbpkg_ctx)) """ ```julia sync_nbpkg_core(notebook::Notebook; on_terminal_output::Function=((args...) -> nothing)) ``` Update the notebook package environment to match the notebook's code. This will: - Add packages that should be added (because they are imported in a cell). - Remove packages that are no longer needed. - Make sure that the environment is instantiated. - Detect the use of `Pkg.activate` and enable/disabled nbpkg accordingly. """ function sync_nbpkg_core( notebook::Notebook, old_topology::NotebookTopology, new_topology::NotebookTopology; on_terminal_output::Function=((args...) -> nothing), cleanup_iolistener::Ref{Function}=Ref{Function}(_default_cleanup), lag::Real=0, compiler_options::CompilerOptions=CompilerOptions(), ) pkg_status = Status.report_business_started!(notebook.status_tree, :pkg) Status.report_business_started!(pkg_status, :analysis)  = false use_plutopkg_old = notebook.nbpkg_ctx !== nothing use_plutopkg_new = use_plutopkg(new_topology) if !use_plutopkg_old && use_plutopkg_new @debug "PlutoPkg: Started using PlutoPkg!! HELLO reproducibility!" notebook.path  = true notebook.nbpkg_ctx = PkgCompat.create_empty_ctx() notebook.nbpkg_install_time_ns = 0 end if use_plutopkg_old && !use_plutopkg_new @debug "PlutoPkg: Stopped using PlutoPkg " notebook.path no_packages_loaded_yet = ( notebook.nbpkg_restart_required_msg === nothing && notebook.nbpkg_restart_recommended_msg === nothing && all(PkgCompat.is_stdlib, keys(PkgCompat.project(notebook.nbpkg_ctx).dependencies)) )  = !no_packages_loaded_yet notebook.nbpkg_ctx = nothing notebook.nbpkg_install_time_ns = nothing end if !use_plutopkg_new notebook.nbpkg_install_time_ns = nothing notebook.nbpkg_ctx_instantiated = true end (lag > 0) && sleep(lag * (0.5 + rand())) # sleep(0) would yield to the process manager which we dont want if use_plutopkg_new @assert notebook.nbpkg_ctx !== nothing PkgCompat.mark_original!(notebook.nbpkg_ctx) old_packages = String.(keys(PkgCompat.project(notebook.nbpkg_ctx).dependencies)) new_packages = String.(external_package_names(new_topology)) # search all cells for imports and usings removed = setdiff(old_packages, new_packages) added = setdiff(new_packages, old_packages) can_skip = isempty(removed) && isempty(added) && notebook.nbpkg_ctx_instantiated iolistener = let busy_packages = notebook.nbpkg_ctx_instantiated ? union(added, removed) : new_packages report_to = ["nbpkg_sync", busy_packages...] IOListener(callback=(s -> on_terminal_output(report_to, freeze_loading_spinners(s)))) end cleanup_iolistener[] = () -> stoplistening(iolistener) Status.report_business_finished!(pkg_status, :analysis) # We remember which Pkg.Types.PreserveLevel was used. If it's too low, we will recommend/require a notebook restart later. local used_tier = first(tiers) if !can_skip # We have a global lock, `pkg_token`, on Pluto-managed Pkg operations, which is shared between all notebooks. If this lock is not ready right now then that means that we are going to wait at the `withtoken(pkg_token)` line below. # We want to report that we are waiting, with a best guess of why. wait_business = if !isready(pkg_token) reg = !PkgCompat._updated_registries_compat[] # Print something in the terminal logs phasemessage(iolistener, "Waiting for $(reg ? "the package registry to update" : "other notebooks to finish Pkg operations")") trigger(iolistener) # manual trigger because we did not start listening yet # Create a business item Status.report_business_started!(pkg_status, reg ? :registry_update : :waiting_for_others ) end return withtoken(pkg_token) do withlogcapture(iolistener) do let # Status stuff isnothing(wait_business) || Status.report_business_finished!(wait_business) if !notebook.nbpkg_ctx_instantiated Status.report_business_planned!(pkg_status, :instantiate1) Status.report_business_planned!(pkg_status, :resolve) Status.report_business_planned!(pkg_status, :precompile) end isempty(removed) || Status.report_business_planned!(pkg_status, :remove) isempty(added) || Status.report_business_planned!(pkg_status, :add) if !isempty(added) || !isempty(removed) Status.report_business_planned!(pkg_status, :instantiate2) Status.report_business_planned!(pkg_status, :precompile) end end should_precompile_later = false PkgCompat.refresh_registry_cache() PkgCompat.clear_stdlib_compat_entries!(notebook.nbpkg_ctx) should_instantiate_initially = !notebook.nbpkg_ctx_instantiated if should_instantiate_initially should_precompile_later = true # Resolve the package environment, using GracefulPkg.jl to solve any issues Status.report_business!(pkg_status, :resolve) do with_auto_fixes(notebook) do _resolve(notebook, iolistener) end end Status.report_business!(pkg_status, :instantiate1) do _instantiate(notebook, iolistener) end end to_add = filter(PkgCompat.package_exists, added) to_remove = filter(removed) do p haskey(PkgCompat.project(notebook.nbpkg_ctx).dependencies, p) end @debug "PlutoPkg:" notebook.path to_add to_remove if !isempty(to_remove) Status.report_business_started!(pkg_status, :remove) # See later comment mkeys() = Set(filter(!is_stdlib, [m.name for m in values(PkgCompat.dependencies(notebook.nbpkg_ctx))])) old_manifest_keys = mkeys() with_io_setup(notebook, iolistener) do Pkg.rm(notebook.nbpkg_ctx, [ Pkg.PackageSpec(name=p) for p in to_remove ]) end notebook.nbpkg_install_time_ns = nothing # we lose our estimate of install time # We record the manifest before and after, to prevent recommending a reboot when nothing got removed from the manifest (e.g. when removing GR, but leaving Plots), or when only stdlibs got removed. new_manifest_keys = mkeys() # TODO: we might want to upgrade other packages now that constraints have loosened? Does this happen automatically? Status.report_business_finished!(pkg_status, :remove) end # TODO: instead of Pkg.PRESERVE_ALL, we actually want: # "Pkg.PRESERVE_DIRECT, but preserve exact verisons of Base.loaded_modules" if !isempty(to_add) Status.report_business_started!(pkg_status, :add) start_time = time_ns() with_io_setup(notebook, iolistener) do phasemessage(iolistener, "Adding packages") # We temporarily clear the "semver-compatible" [deps] entries, because Pkg already respects semver, unless it doesn't, in which case we don't want to force it. PkgCompat.clear_auto_compat_entries!(notebook.nbpkg_ctx) try for tier in tiers used_tier = tier try withenv("JULIA_PKG_PRECOMPILE_AUTO" => 0) do Pkg.add(notebook.nbpkg_ctx, [ Pkg.PackageSpec(name=p) for p in to_add ]; preserve=used_tier) end break catch e if used_tier == Pkg.PRESERVE_NONE # give up rethrow(e) end end end finally PkgCompat.write_auto_compat_entries!(notebook.nbpkg_ctx) end # Now that Pkg is set up, the notebook process will call `using Package`, which can take some time. We write this message to the io, to notify the user. println(iolistener.buffer, "\e[32m\e[1mLoading\e[22m\e[39m packages...") end notebook.nbpkg_install_time_ns = notebook.nbpkg_install_time_ns === nothing ? nothing : (notebook.nbpkg_install_time_ns + (time_ns() - start_time)) Status.report_business_finished!(pkg_status, :add) @debug "PlutoPkg: done" notebook.path end should_instantiate_again = !notebook.nbpkg_ctx_instantiated || !isempty(to_add) || !isempty(to_remove) if should_instantiate_again should_precompile_later = true Status.report_business!(pkg_status, :instantiate2) do _instantiate(notebook, iolistener) end end if should_precompile_later Status.report_business!(pkg_status, :precompile) do _precompile(notebook, iolistener, compiler_options) end end stoplistening(iolistener) Status.report_business_finished!(pkg_status) return (; did_something= || ( should_instantiate_initially || should_instantiate_again || (use_plutopkg_old != use_plutopkg_new) ), used_tier=used_tier, # changed_versions=Dict{String,Pair}(), restart_recommended= || ( (!isempty(to_remove) && old_manifest_keys != new_manifest_keys) || used_tier  (Pkg.PRESERVE_ALL, Pkg.PRESERVE_ALL_INSTALLED) ), restart_required= || ( used_tier  (Pkg.PRESERVE_SEMVER, Pkg.PRESERVE_NONE) ), ) end end end end Status.report_business_finished!(pkg_status) return ( did_something= || (use_plutopkg_old != use_plutopkg_new), used_tier=Pkg.PRESERVE_ALL_INSTALLED, # changed_versions=Dict{String,Pair}(), restart_recommended= || false, restart_required= || false, ) end """ ```julia sync_nbpkg(session::ServerSession, notebook::Notebook; save::Bool=true) ``` In addition to the steps performed by [`sync_nbpkg_core`](@ref): - Capture terminal outputs and store them in the `notebook` - Update the clients connected to `notebook` - `try` `catch` and reset the package environment on failure. """ function sync_nbpkg(session, notebook, old_topology::NotebookTopology, new_topology::NotebookTopology; save::Bool=true, take_token::Bool=true) @assert will_run_pkg(notebook) cleanup_iolistener = Ref{Function}(_default_cleanup) try Status.report_business_started!(notebook.status_tree, :pkg) pkg_result = (take_token ? withtoken : (f, _) -> f())(notebook.executetoken) do function iocallback(pkgs, s) notebook.nbpkg_busy_packages = pkgs for p in pkgs notebook.nbpkg_terminal_outputs[p] = s end # incoming IO is a good sign that something might have changed, so we update the cache. update_nbpkg_cache!(notebook) # This is throttled "automatically": # If io is coming in very fast, then it won't build up a queue of updates for the client. That's because `send_notebook_changes!` is a blocking call, including the websocket transfer. So this `iocallback` function will only return when the last message has been sent. # The nice thing is that IOCallback is not actually implemented as a callback when IO comes in, but it is an async loop that sleeps, maybe calls iocallback with the latest buffer content, and repeats. So a blocking iocallback avoids overflowing the queue. send_notebook_changes!(ClientRequest(; session, notebook)) end sync_nbpkg_core( notebook, old_topology, new_topology; on_terminal_output=iocallback, cleanup_iolistener, lag=session.options.server.simulated_pkg_lag, compiler_options=_merge_notebook_compiler_options(notebook, session.options.compiler), ) end if pkg_result.did_something @debug "PlutoPkg: success!" notebook.path pkg_result if _has_executed_effectful_code(session, notebook) if pkg_result.restart_recommended notebook.nbpkg_restart_recommended_msg = "Yes, something changed during regular sync." @debug "PlutoPkg: Notebook restart recommended" notebook.path notebook.nbpkg_restart_recommended_msg end if pkg_result.restart_required notebook.nbpkg_restart_required_msg = "Yes, something changed during regular sync." @debug "PlutoPkg: Notebook restart REQUIRED" notebook.path notebook.nbpkg_restart_required_msg end end notebook.nbpkg_busy_packages = String[] update_nbpkg_cache!(notebook) send_notebook_changes!(ClientRequest(; session, notebook)) save && save_notebook(session, notebook) end catch e bt = catch_backtrace() Status.report_business_finished!(notebook.status_tree, :pkg, false) old_packages = try String.(keys(PkgCompat.project(notebook.nbpkg_ctx).dependencies)); catch; ["unknown"] end new_packages = try String.(external_package_names(new_topology)); catch; ["unknown"] end @warn """ PlutoPkg: Failed to add/remove packages! Resetting package environment... """ PLUTO_VERSION VERSION old_packages new_packages exception=(e, bt) # TODO: send to user showerror(stderr, e, bt) error_text = e isa PrecompilationFailedException ? e.msg : sprint(showerror, e, bt) for p in notebook.nbpkg_busy_packages old = get(notebook.nbpkg_terminal_outputs, p, "") notebook.nbpkg_terminal_outputs[p] = old * "\n\n\e[1mPkg error!\e[22m\n" * error_text end notebook.nbpkg_busy_packages = String[] update_nbpkg_cache!(notebook) send_notebook_changes!(ClientRequest(; session, notebook)) # Clear the embedded Project and Manifest and require a restart from the user. reset_nbpkg!(notebook, new_topology; keep_project=false, save=save) notebook.nbpkg_restart_required_msg = "Yes, because sync_nbpkg_core failed. \n\n$(error_text)" notebook.nbpkg_install_time_ns = nothing notebook.nbpkg_ctx_instantiated = false update_nbpkg_cache!(notebook) send_notebook_changes!(ClientRequest(; session, notebook)) save && save_notebook(session, notebook) finally cleanup_iolistener[]() Status.report_business_finished!(notebook.status_tree, :pkg) end end function _has_executed_effectful_code(session::ServerSession, notebook::Notebook) workspace = WorkspaceManager.get_workspace((session, notebook); allow_creation=false) workspace === nothing ? false : workspace.has_executed_effectful_code end function writebackup(notebook::Notebook) backup_path = backup_filename(notebook.path) Pluto.readwrite(notebook.path, backup_path) @info "Backup saved to" backup_path backup_path end function _instantiate(notebook::Notebook, iolistener::IOListener) start_time = time_ns() with_io_setup(notebook, iolistener) do phasemessage(iolistener, "Instantiating") @debug "PlutoPkg: Instantiating" notebook.path # update registries if this is the first time PkgCompat.update_registries(; force=false) # Pkg.instantiate assumes that the environment to be instantiated is active, so we will have to modify the LOAD_PATH of this Pluto server # We could also run the Pkg calls on the notebook process, but somehow I think that doing it on the server is more charming, though it requires this workaround. env_dir = PkgCompat.env_dir(notebook.nbpkg_ctx) pushfirst!(LOAD_PATH, env_dir) try # instantiate without forcing registry update PkgCompat.instantiate(notebook.nbpkg_ctx; update_registry=false, allow_autoprecomp=false) finally # reset the LOAD_PATH if LOAD_PATH[1] == env_dir popfirst!(LOAD_PATH) else @warn "LOAD_PATH modified during Pkg.instantiate... this is unexpected!" end end end notebook.nbpkg_install_time_ns = notebook.nbpkg_install_time_ns === nothing ? nothing : (notebook.nbpkg_install_time_ns + (time_ns() - start_time)) notebook.nbpkg_ctx_instantiated = true end function _precompile(notebook::Notebook, iolistener::IOListener, compiler_options::CompilerOptions) start_time = time_ns() with_io_setup(notebook, iolistener) do phasemessage(iolistener, "Precompiling") @debug "PlutoPkg: Precompiling" notebook.path env_dir = PkgCompat.env_dir(notebook.nbpkg_ctx) precompile_isolated(env_dir; io=iolistener.buffer, compiler_options, ) end notebook.nbpkg_install_time_ns = notebook.nbpkg_install_time_ns === nothing ? nothing : (notebook.nbpkg_install_time_ns + (time_ns() - start_time)) end function _resolve(notebook::Notebook, iolistener::IOListener) startlistening(iolistener) with_io_setup(notebook, iolistener) do phasemessage(iolistener, "Resolving") @debug "PlutoPkg: Resolving" notebook.path Pkg.resolve(notebook.nbpkg_ctx) end end const gracefulpkg_strats = filter!(collect(GracefulPkg.DEFAULT_STRATEGIES)) do strat !(strat isa GracefulPkg.StrategyRemoveProject) end """ Run `f` (e.g. `Pkg.instantiate`) on the notebook's package environment. Keep trying more and more invasive strategies to fix problems until the operation succeeds. """ function with_auto_fixes(f::Function, notebook::Notebook) env_dir = PkgCompat.env_dir(notebook.nbpkg_ctx) is_first = Ref(true) report = GracefulPkg.gracefully(; env_dir, throw=false, strategies=gracefulpkg_strats) do try if !is_first[] PkgCompat.load_ctx!(notebook.nbpkg_ctx, env_dir) end f() finally is_first[] = false end end steps = report.strategy_reports if any(x -> x.strategy isa GracefulPkg.StrategyLoosenCompat, steps) notebook.nbpkg_ctx_instantiated = false end if !GracefulPkg.is_success(report) throw(GracefulPkg.NothingWorked(report)) end end """ Reset the package environment of a notebook. This will remove the `Project.toml` and `Manifest.toml` files from the notebook's secret package environment folder, and if `save` is `true`, it will then save the notebook without embedded Project and Manifest. If `keep_project` is `true` (default `false`), the `Project.toml` file will be kept, but the `Manifest.toml` file will be removed. This function is useful when we are not able to resolve/activate/instantiate a notebook's environment after loading, which happens when e.g. the environment was created on a different OS or Julia version. """ function reset_nbpkg!(notebook::Notebook, topology::Union{NotebookTopology,Nothing}=nothing; keep_project::Bool=false, backup::Bool=true, save::Bool=true) backup && save && writebackup(notebook) if notebook.nbpkg_ctx !== nothing p = PkgCompat.project_file(notebook) m = PkgCompat.manifest_file(notebook) keep_project || (isfile(p) && rm(p)) isfile(m) && rm(m) PkgCompat.load_ctx!(notebook.nbpkg_ctx, PkgCompat.env_dir(notebook.nbpkg_ctx)) else notebook.nbpkg_ctx = if use_plutopkg(something(topology, notebook.topology)) PkgCompat.load_empty_ctx!(notebook.nbpkg_ctx) else nothing end end save && save_notebook(notebook) end function update_nbpkg_core( notebook::Notebook; level::Pkg.UpgradeLevel=Pkg.UPLEVEL_MAJOR, on_terminal_output::Function=((args...) -> nothing), cleanup_iolistener::Ref{Function}=Ref{Function}(default_cleanup), compiler_options::CompilerOptions=CompilerOptions(), ) if notebook.nbpkg_ctx !== nothing PkgCompat.mark_original!(notebook.nbpkg_ctx) old_packages = String.(keys(PkgCompat.project(notebook.nbpkg_ctx).dependencies)) iolistener = let # we don't know which packages will be updated, so we send terminal output to all installed packages report_to = ["nbpkg_update", old_packages...] IOListener(callback=(s -> on_terminal_output(report_to, freeze_loading_spinners(s)))) end cleanup_iolistener[] = () -> stoplistening(iolistener) if !isready(pkg_token) phasemessage(iolistener, "Waiting for other notebooks to finish Pkg operations") trigger(iolistener) end return withtoken(pkg_token) do withlogcapture(iolistener) do PkgCompat.refresh_registry_cache() PkgCompat.clear_stdlib_compat_entries!(notebook.nbpkg_ctx) if !notebook.nbpkg_ctx_instantiated with_auto_fixes(notebook) do _resolve(notebook, iolistener) end _instantiate(notebook, iolistener) end with_io_setup(notebook, iolistener) do phasemessage(iolistener, "Updating packages") # We temporarily clear the "semver-compatible" [deps] entries, because it is difficult to update them after the update . TODO PkgCompat.clear_auto_compat_entries!(notebook.nbpkg_ctx) try ### withenv("JULIA_PKG_PRECOMPILE_AUTO" => 0) do Pkg.update(notebook.nbpkg_ctx; level=level) end ### finally PkgCompat.write_auto_compat_entries!(notebook.nbpkg_ctx) end end PkgCompat.refresh_registry_cache()  = !PkgCompat.is_original(notebook.nbpkg_ctx) should_instantiate_again = !notebook.nbpkg_ctx_instantiated ||  if should_instantiate_again # Status.report_business!(pkg_status, :instantiate2) do _instantiate(notebook, iolistener) _precompile(notebook, iolistener, compiler_options) # end end stoplistening(iolistener) ( did_something=, restart_recommended=, restart_required=, ) end end end ( did_something=false, restart_recommended=false, restart_required=false, ) end function update_nbpkg(session, notebook::Notebook; level::Pkg.UpgradeLevel=Pkg.UPLEVEL_MAJOR, backup::Bool=true, save::Bool=true) @assert will_run_pkg(notebook) bp = if backup && save writebackup(notebook) end cleanup_iolistener = Ref{Function}(_default_cleanup) try pkg_result = withtoken(notebook.executetoken) do original_outputs = deepcopy(notebook.nbpkg_terminal_outputs) function iocallback(pkgs, s) notebook.nbpkg_busy_packages = pkgs for p in pkgs original = get(original_outputs, p, "") notebook.nbpkg_terminal_outputs[p] = original * "\n\n" * s end update_nbpkg_cache!(notebook) send_notebook_changes!(ClientRequest(; session, notebook)) end update_nbpkg_core( notebook; level, on_terminal_output=iocallback, cleanup_iolistener, compiler_options=_merge_notebook_compiler_options(notebook, session.options.compiler), ) end if pkg_result.did_something if pkg_result.restart_recommended notebook.nbpkg_restart_recommended_msg = "Yes, something changed during regular update_nbpkg." @debug "PlutoPkg: Notebook restart recommended" notebook.path notebook.nbpkg_restart_recommended_msg end if pkg_result.restart_required notebook.nbpkg_restart_required_msg = "Yes, something changed during regular update_nbpkg." @debug "PlutoPkg: Notebook restart REQUIRED" notebook.path notebook.nbpkg_restart_required_msg end else !isnothing(bp) && isfile(bp) && rm(bp) end finally cleanup_iolistener[]() notebook.nbpkg_busy_packages = String[] update_nbpkg_cache!(notebook) send_notebook_changes!(ClientRequest(; session, notebook)) save && save_notebook(session, notebook) end end nbpkg_cache(ctx::Union{Nothing,PkgContext}) = ctx === nothing ? Dict{String,String}() : Dict{String,String}( x => string(PkgCompat.get_manifest_version(ctx, x)) for x in keys(PkgCompat.project(ctx).dependencies) ) function update_nbpkg_cache!(notebook::Notebook) notebook.nbpkg_installed_versions_cache = nbpkg_cache(notebook.nbpkg_ctx) notebook end function is_nbpkg_equal(a::Union{Nothing,PkgContext}, b::Union{Nothing,PkgContext})::Bool if (a isa Nothing) != (b isa Nothing) the_other = something(a, b) ptoml_contents = PkgCompat.read_project_file(the_other) the_other_is_empty = isempty(strip(ptoml_contents)) if the_other_is_empty # then both are essentially 'empty' environments, i.e. equal true else # they are different false end elseif a isa Nothing true else ptoml_contents_a = strip(PkgCompat.read_project_file(a)) ptoml_contents_b = strip(PkgCompat.read_project_file(b)) if ptoml_contents_a == ptoml_contents_b == "" true else mtoml_contents_a = strip(PkgCompat.read_project_file(a)) mtoml_contents_b = strip(PkgCompat.read_project_file(b)) (ptoml_contents_a == ptoml_contents_b) && (mtoml_contents_a == mtoml_contents_b) end end end function with_io_setup(f::Function, notebook::Notebook, iolistener::IOListener) startlistening(iolistener) PkgCompat.withio(notebook.nbpkg_ctx, IOContext(iolistener.buffer, :color => true, :sneaky_enable_tty => true)) do withinteractive(false) do f() end end end withlogcapture(f::Function, iolistener::IOListener) = Logging.with_logger(f, LoggingExtras.TeeLogger( Logging.current_logger(), Logging.ConsoleLogger(IOContext(iolistener.buffer, :color => true), Logging.Info) )) const is_interactive_defined = isdefined(Base, :is_interactive) && !Base.isconst(Base, :is_interactive) function withinteractive(f::Function, value::Bool) old_value = isinteractive() @static if is_interactive_defined Core.eval(Base, :(is_interactive = $value)) end try f() finally @static if is_interactive_defined Core.eval(Base, :(is_interactive = $old_value)) end end end module PkgCompat export package_versions, registered_package_names import REPL import Pkg import Pkg.Types: VersionRange import RegistryInstances import ..Pluto import GracefulPkg @static if isdefined(Pkg,:REPLMode) && isdefined(Pkg.REPLMode, :complete_remote_package) const REPLMode = Pkg.REPLMode else const REPLMode = Base.get_extension(Pkg, :REPLExt) end # Should be in Base flatmap(args...) = vcat(map(args...)...) # Should be in Base function select(f::Function, xs) for x  xs if f(x) return x end end nothing end #= NOTE ABOUT PUBLIC/INTERNAL PKG API Pkg.jl exposes lots of API, but only some of it is "public": guaranteed to remain available. API is public if it is listed here: https://pkgdocs.julialang.org/v1/api/ In this file, I labeled functions by their status using , , etc. A status in brackets (like this) means that it is only called within this file, and the fallback might be in a caller function. --- I tried to only use public API, except: - I use the `Pkg.Types.Context` value as first argument for many functions, since the server process manages multiple notebook processes, each with their own package environment. We could get rid of this, by settings `Base.ACTIVE_PROJECT[]` before and after each Pkg call. (This is temporarily activating the notebook environment.) This does have a performance impact, since the project and manifest caches are regenerated every time. - https://github.com/JuliaLang/Pkg.jl/issues/2607 seems to be impossible with the current public API. - Some functions try to use internal API for optimization/better features. =# ### # CONTEXT ### const PkgContext = if isdefined(Pkg, :Types) && isdefined(Pkg.Types, :Context) Pkg.Types.Context elseif isdefined(Pkg, :API) && isdefined(Pkg.API, :Context) Pkg.API.Context else Pkg.Types.Context end function PkgContext!(ctx::PkgContext; kwargs...) for (k, v) in kwargs setfield!(ctx, k, v) end ctx end #  "Public API", but using PkgContext load_ctx(env_dir)::PkgContext = PkgContext(;env=Pkg.Types.EnvCache(joinpath(env_dir, "Project.toml"))) #  "Public API", but using PkgContext load_ctx!(ctx::PkgContext, env_dir)::PkgContext = PkgContext!(ctx; env=Pkg.Types.EnvCache(joinpath(env_dir, "Project.toml"))) #  "Public API", but using PkgContext load_empty_ctx!(ctx) = @static if :io  fieldnames(PkgContext) PkgContext!(create_empty_ctx(); io=ctx.io) else create_empty_ctx() end #  "Public API", but using PkgContext create_empty_ctx()::PkgContext = load_ctx!(PkgContext(), mktempdir()) #  Internal API with fallback function load_ctx!(original::PkgContext) original_project = deepcopy(original.env.original_project) original_manifest = deepcopy(original.env.original_manifest) new = load_ctx!(original, env_dir(original)) try new.env.original_project = original_project new.env.original_manifest = original_manifest catch e @warn "Pkg compat: failed to set original_project" exception=(e,catch_backtrace()) end new end #  Internal API with fallback function mark_original!(ctx::PkgContext) try ctx.env.original_project = deepcopy(ctx.env.project) ctx.env.original_manifest = deepcopy(ctx.env.manifest) catch e @warn "Pkg compat: failed to set original_project" exception=(e,catch_backtrace()) end end #  Internal API with fallback function is_original(ctx::PkgContext)::Bool try ctx.env.original_project == ctx.env.project && ctx.env.original_manifest == ctx.env.manifest catch e @warn "Pkg compat: failed to get original_project" exception=(e,catch_backtrace()) false end end #  "Public API", but using PkgContext env_dir(ctx::PkgContext) = dirname(ctx.env.project_file) project_file(x::AbstractString) = joinpath(x, "Project.toml") manifest_file(x::AbstractString) = joinpath(x, "Manifest.toml") project_file(ctx::PkgContext) = joinpath(env_dir(ctx), "Project.toml") manifest_file(ctx::PkgContext) = joinpath(env_dir(ctx), "Manifest.toml") function read_project_file(x)::String path = project_file(x) isfile(path) ? read(path, String) : "" end function read_manifest_file(x)::String path = manifest_file(x) isfile(path) ? read(path, String) : "" end #  Internal API with fallback function withio(f::Function, ctx::PkgContext, io::IO) @static if :io  fieldnames(PkgContext) old_io = ctx.io ctx.io = io result = try f() finally ctx.io = old_io nothing end result else f() end end # I'm a pirate harrr  @static if isdefined(Pkg, :can_fancyprint) Pkg.can_fancyprint(io::Union{IOContext{IOBuffer},IOContext{Base.BufferStream}}) = get(io, :sneaky_enable_tty, false) === true end @static if isdefined(Base, :Precompilation) && isdefined(Base.Precompilation, :can_fancyprint) Base.Precompilation.can_fancyprint(io::Union{IOContext{IOBuffer},IOContext{Base.BufferStream}}) = get(io, :sneaky_enable_tty, false) === true end ### # REGISTRIES ### # ( "Public" API using RegistryInstances) "Return all installed registries as `RegistryInstances.RegistryInstance` structs." _get_registries() = RegistryInstances.reachable_registries() # ( "Public" API using RegistryInstances) "The cached output value of `_get_registries`." const _parsed_registries = Ref(RegistryInstances.RegistryInstance[]) # ( "Public" API using RegistryInstances) "Re-parse the installed registries from disk." function refresh_registry_cache() _parsed_registries[] = _get_registries() end #  Internal API with fallback const _updated_registries_compat = @static if isdefined(Pkg, :UPDATED_REGISTRY_THIS_SESSION) && Pkg.UPDATED_REGISTRY_THIS_SESSION isa Ref{Bool} Pkg.UPDATED_REGISTRY_THIS_SESSION else Ref(false) end #  Public API function update_registries(; force::Bool=false) if force || !_updated_registries_compat[] try Pkg.Registry.update() catch # sometimes it just fails but we dont want Pluto to be too sensitive to that Pkg.Registry.update() end try refresh_registry_cache() catch end _updated_registries_compat[] = true end end #  Public API """ Check when the registries were last updated. If it is recent (max 7 days), then `Pkg.UPDATED_REGISTRY_THIS_SESSION[]` is set to `true`, which will prevent Pkg from doing an automatic registry update. Returns the new value of `Pkg.UPDATED_REGISTRY_THIS_SESSION[]`. """ function check_registry_age(max_age_ms = 1000.0 * 60 * 60 * 24 * 7)::Bool if get(ENV, "GITHUB_ACTIONS", "false") == "true" # don't do this optimization in CI return false end paths = [s.path for s in _get_registries()] isempty(paths) && return _updated_registries_compat[] mtimes = map(paths) do p try mtime(p) catch zero(time()) end end if all(mtimes .> time() - max_age_ms) _updated_registries_compat[] = true end _updated_registries_compat[] end ### # Instantiate ### #  Internal API function instantiate(ctx; update_registry::Bool, allow_autoprecomp::Bool) Pkg.instantiate(ctx; update_registry, allow_autoprecomp) # Not sure how to make a fallback: # - hasmethod cannot test for kwargs because instantiate takes kwargs... that are passed on somewhere else # - can't catch for a CallError because the error is weird end ### # Standard Libraries ### #  Public API is_stdlib(package_name::String) = package_name  GracefulPkg.stdlibs_past_present_future # Initial fill of registry cache function __init__() refresh_registry_cache() global global_ctx=PkgContext() end ### # Package names ### # ( "Public" API) """ Return names of all registered packages. """ function registered_package_names(;registries::Vector=_parsed_registries[])::Vector{String} flatmap(registries) do reg packages = values(reg.pkgs) union!(String[ d.name for d in packages ], GracefulPkg.stdlibs_past_present_future) end |> sort! end ### # Package versions ### # ( "Public" API) """ Return paths to all found registry entries of a given package name. # Example ```julia julia> Pluto.PkgCompat._registry_entries("Pluto") 1-element Vector{String}: "/Users/fons/.julia/registries/General/P/Pluto" ``` """ function _registry_entries(package_name::AbstractString, registries::Vector=_parsed_registries[])::Vector{String} flatmap(registries) do reg packages = values(reg.pkgs) String[ joinpath(reg.path, d.path) for d in packages if d.name == package_name ] end end #  "Public" API using RegistryInstances """ Return all registered versions of the given package. Returns `["stdlib"]` for standard libraries, a `Vector{VersionNumber}` for registered packages, or `["latest"]` if it crashed. """ function package_versions(package_name::AbstractString)::Vector if is_stdlib(package_name) ["stdlib"] else try flatmap(_parsed_registries[]) do reg uuids_with_name = RegistryInstances.uuids_from_name(reg, package_name) flatmap(uuids_with_name) do u pkg = get(reg, u, nothing) if pkg !== nothing info = RegistryInstances.registry_info(pkg) collect(keys(info.version_info)) else [] end end end |> sort! catch e @warn "Pkg compat: failed to get installable versions." exception=(e,catch_backtrace()) ["latest"] end end end #  "Public" API using RegistryInstances """ Return the URL of the package's documentation (if possible) or homepage. Returns `nothing` if the package was not found. """ function package_url(package_name::AbstractString)::Union{String,Nothing} if is_stdlib(package_name) "https://docs.julialang.org/en/v1/stdlib/$(package_name)/" else try for reg in _parsed_registries[] for u in RegistryInstances.uuids_from_name(reg, package_name) pkg = get(reg, u, nothing) if pkg !== nothing return RegistryInstances.registry_info(pkg).repo end end end catch e @warn "Pkg compat: failed to get package URL." exception=(e,catch_backtrace()) end end end #  Internal API with fallback "Does a package with this name exist in one of the installed registries?" package_exists(package_name::AbstractString)::Bool = package_versions(package_name) |> !isempty #  "Public API", but using PkgContext function dependencies(ctx) try # ctx.env.manifest @static if hasmethod(Pkg.dependencies, (PkgContext,)) Pkg.dependencies(ctx) else Pkg.dependencies(ctx.env) end catch e if !any(occursin(sprint(showerror, e)), ( "could not find source path for", # https://github.com/fonsp/Pluto.jl/issues/3176 r"expected.*exist.*manifest", r"no method.*project_rel_path.*Nothing\)", # https://github.com/JuliaLang/Pkg.jl/issues/3404 )) @error """ Pkg error: you might need to use Pluto.reset_notebook_environment(notebook_path) to reset this notebook's environment. Before doing so, consider sending your notebook file to https://github.com/fonsp/Pluto.jl/issues together with the following info: """ Pluto.PLUTO_VERSION VERSION exception=(e,catch_backtrace()) end Dict() end end function project(ctx::PkgContext) @static if hasmethod(Pkg.project, (PkgContext,)) Pkg.project(ctx) else Pkg.project(ctx.env) end end #  "Public API", but using PkgContext "Find a package in the manifest. Return `nothing` if not found." _get_manifest_entry(ctx::PkgContext, package_name::AbstractString) = select(e -> e.name == package_name, values(dependencies(ctx))) #  Internal API with fallback """ Find a package in the manifest given its name, and return its installed version. Return `"stdlib"` for a standard library, and `nothing` if not found. """ function get_manifest_version(ctx::PkgContext, package_name::AbstractString) if is_stdlib(package_name) "stdlib" else entry = _get_manifest_entry(ctx, package_name) entry === nothing ? nothing : entry.version end end ### # WRITING COMPAT ENTRIES ### const _project_key_order = ["name", "uuid", "keywords", "license", "desc", "deps", "weakdeps", "sources", "extensions", "compat"] project_key_order(key::String) = something(findfirst(x -> x == key, _project_key_order), length(_project_key_order) + 1) #  Public API function _modify_compat!(f!::Function, ctx::PkgContext)::PkgContext project_path = project_file(ctx) toml = if isfile(project_path) Pkg.TOML.parsefile(project_path) else Dict{String,Any}() end compat = get!(Dict{String,Any}, toml, "compat") f!(compat) isempty(compat) && delete!(toml, "compat") write(project_path, sprint() do io Pkg.TOML.print(io, toml; sorted=true, by=(key -> (project_key_order(key), key))) end) return _update_project_hash!(load_ctx!(ctx)) end #  Internal API with fallback "Update the project hash in the manifest file (https://github.com/JuliaLang/Pkg.jl/pull/2815)" function _update_project_hash!(ctx::PkgContext) isfile(manifest_file(ctx)) && try Pkg.Operations.record_project_hash(ctx.env) Pkg.Types.write_manifest(ctx.env) catch e @info "Failed to update project hash." exception=(e,catch_backtrace()) end ctx end #  Public API """ Add any missing [`compat`](https://pkgdocs.julialang.org/v1/compatibility/) entries to the `Project.toml` for all direct dependencies. This serves as a 'fallback' in case someone (with a different Julia version) opens your notebook without being able to load the `Manifest.toml`. Return the new `PkgContext`. The automatic compat entry is: `"~" * string(installed_version)`. """ function write_auto_compat_entries!(ctx::PkgContext)::PkgContext _modify_compat!(ctx) do compat for p in keys(project(ctx).dependencies) if !haskey(compat, p) m_version = get_manifest_version(ctx, p) if m_version !== nothing && !is_stdlib(p) compat[p] = "~" * string(VersionNumber(m_version.major, m_version.minor, m_version.patch)) # drop build number end end end end end #  Public API """ Remove any automatically-generated [`compat`](https://pkgdocs.julialang.org/v1/compatibility/) entries from the `Project.toml`. This will undo the effects of [`write_auto_compat_entries!`](@ref) but leave other (e.g. manual) compat entries intact. Return the new `PkgContext`. """ function clear_auto_compat_entries!(ctx::PkgContext)::PkgContext if isfile(project_file(ctx)) _modify_compat!(ctx) do compat for p in keys(compat) m_version = get_manifest_version(ctx, p) if m_version !== nothing && !is_stdlib(p) if compat[p] == "~" * string(m_version) delete!(compat, p) end end end end else ctx end end #  Public API """ Remove any [`compat`](https://pkgdocs.julialang.org/v1/compatibility/) entries from the `Project.toml` for standard libraries. These entries are created when an old version of Julia uses a package that later became a standard library, like https://github.com/JuliaPackaging/Artifacts.jl. Return the new `PkgContext`. """ function clear_stdlib_compat_entries!(ctx::PkgContext)::PkgContext if isfile(project_file(ctx)) _modify_compat!(ctx) do compat for p in keys(compat) if is_stdlib(p) @info "Removing compat entry for stdlib" p delete!(compat, p) end end end else ctx end end end module PkgUtils import FileWatching import Pkg import ..Pluto import ..Pluto: Notebook, save_notebook, load_notebook, load_notebook_nobackup, withtoken, Token, readwrite, PkgCompat import ..Pluto.PkgCompat: project_file, manifest_file using Markdown export activate_notebook ensure_has_nbpkg(notebook::Notebook) = if !will_use_pluto_pkg(notebook) # TODO: update_save the notebook to init packages and stuff? error(""" This notebook is not using Pluto's package manager. This means that the notebook contains Pkg.activate or Pkg.add call. Open the notebook using Pluto to see what's up. """) else for f in [notebook |> project_file, notebook |> manifest_file] isfile(f) || touch(f) end end function assert_has_manifest(dir::String) @assert isdir(dir) if !isfile(dir |> project_file) error("The given directory does not contain a Project.toml file -- it is not a package environment.") end if !isfile(dir |> manifest_file) error("The given directory does not contain a Manifest.toml file. Use `Pkg.resolve` to generate a Manifest.toml from a Project.toml.") end end function nb_and_dir_environments_equal(notebook::Notebook, dir::String) try ensure_has_nbpkg(notebook) assert_has_manifest(dir) true catch false end && let read(notebook |> project_file) == read(dir |> project_file) && read(notebook |> manifest_file) == read(dir |> manifest_file) end end function write_nb_to_dir(notebook::Notebook, dir::String) ensure_has_nbpkg(notebook) mkpath(dir) readwrite(notebook |> project_file, dir |> project_file) readwrite(notebook |> manifest_file, dir |> manifest_file) end function write_dir_to_nb(dir::String, notebook::Notebook) assert_has_manifest(dir) notebook.nbpkg_ctx = Pluto.PkgCompat.create_empty_ctx() readwrite(dir |> project_file, notebook |> project_file) readwrite(dir |> manifest_file, notebook |> manifest_file) save_notebook(notebook) end write_dir_to_nb(dir::String, notebook_path::String) = write_dir_to_nb(dir::String, load_notebook(notebook_path)) write_nb_to_dir(notebook_path::String, dir::String) = write_nb_to_dir(load_notebook(notebook_path), dir) nb_and_dir_environments_equal(notebook_path::String, dir::String) = nb_and_dir_environments_equal(load_notebook(notebook_path), dir) """ ```julia reset_notebook_environment(notebook_path::String; keep_project::Bool=false, backup::Bool=true) ``` Remove the embedded `Project.toml` and `Manifest.toml` from a notebook file, modifying the notebook file. If `keep_project` is true, only `Manifest.toml` will be deleted. A backup of the notebook file is created by default. """ function reset_notebook_environment(path::String; kwargs...) Pluto.reset_nbpkg!( load_notebook_nobackup(path); kwargs... ) end """ ```julia update_notebook_environment(notebook_path::String; backup::Bool=true, level::Pkg.UpgradeLevel=Pkg.UPLEVEL_MAJOR) ``` Call `Pkg.update` in the package environment embedded in a notebook file, modifying the notebook file. A [`Pkg.UpgradeLevel`](@ref) can be passed to the `level` keyword argument. A backup file is created by default. """ function update_notebook_environment(path::String; kwargs...) Pluto.update_nbpkg( Pluto.ServerSession(), load_notebook_nobackup(path); kwargs... ) end """ ```julia will_use_pluto_pkg(notebook_path::String)::Bool ``` Will this notebook use the Pluto package manager? `false` means that the notebook contains `Pkg.activate` or another deactivator. """ will_use_pluto_pkg(path::String) = will_use_pluto_pkg(load_notebook_nobackup(path)) function will_use_pluto_pkg(notebook::Notebook) ctx = notebook.nbpkg_ctx # if one of the two files is not empty: if ctx !== nothing && !isempty(PkgCompat.read_project_file(ctx)) || !isempty(PkgCompat.read_manifest_file(ctx)) return true end # otherwise, check for Pkg.activate: # when nbpkg_ctx is defined but the files are empty: check if the notebook would use one (i.e. that Pkg.activate is not used). topology = Pluto.updated_topology(notebook.topology, notebook, notebook.cells) return Pluto.use_plutopkg(topology) end """ ```julia activate_notebook_environment(notebook_path::String; show_help::Bool=true)::Nothing ``` Activate the package environment embedded in a notebook file, for interactive use. This will allow you to use the Pkg REPL and Pkg commands to modify the environment, and any changes you make will be automatically saved in the notebook file. More help will be displayed if `show_help` is `true`. Limitations: - Shut down the notebook before using this functionality. - Non-interactive use is limited, use the functional form instead, or insert `sleep` calls after modifying the environment. !!! info This functionality works using file watching. A dummy repository contains a copy of the embedded tomls and gets activated, and the notebook file is updated when the dummy repository changes. """ function activate_notebook_environment(path::String; show_help::Bool=true) notebook_ref = Ref(load_notebook_nobackup(path)) ensure_has_nbpkg(notebook_ref[]) ourpath = joinpath(mktempdir(), basename(path)) mkpath(ourpath) still_needed = Ref(true) save_token = Token() function maybe_update_nb() withtoken(save_token) do if still_needed[] if !nb_and_dir_environments_equal(notebook_ref[], ourpath) write_dir_to_nb(ourpath, notebook_ref[]) println() @info "Notebook file updated " println() end end end end function maybe_update_dir() withtoken(save_token) do if still_needed[] if !nb_and_dir_environments_equal(notebook_ref[], ourpath) write_nb_to_dir(notebook_ref[], ourpath) println() @info "REPL environment updated from notebook " println() end end end end maybe_update_dir() atexit(maybe_update_nb) Base.ACTIVE_PROJECT[] = ourpath # WATCH DIR PROJECT FILE Pluto.@asynclog begin while Base.ACTIVE_PROJECT[] == ourpath FileWatching.watch_file(ourpath |> project_file) # @warn "DIR PROJECT UPDATED" sleep(.2) maybe_update_nb() end still_needed[] = false end # WATCH DIR MANIFEST FILE Pluto.@asynclog begin while Base.ACTIVE_PROJECT[] == ourpath FileWatching.watch_file(ourpath |> manifest_file) # @warn "DIR MANIFEST UPDATED" sleep(.5) maybe_update_nb() end still_needed[] = false end # WATCH NOTEBOOK FILE Pluto.@asynclog begin while Base.ACTIVE_PROJECT[] == ourpath FileWatching.watch_file(path) # @warn "NOTEBOOK FILE UPDATED" # we update the dir after a longer delay, because changes to the environment in the directory (written to the notebook) take precedence. sleep(1) notebook_ref[] = load_notebook(path) maybe_update_dir() end still_needed[] = false end # CHECK EVERY 5 SECONDS # Pluto.@asynclog begin # while still_needed[] # sleep(5) # maybe_update_nb() # # we update the dir second, because changes to the environment in the directory (written to the notebook) take precedence. # maybe_update_dir() # end # end if show_help println() """ > Notebook environment activated! ## Step 1. _Press `]` to open the Pkg REPL._ The notebook environment is currently active. ## Step 2. The notebook file and your REPL environment are now synced. This means that: 1. Any changes you make in the REPL environment will be written to the notebook file. For example, you can `pkg> update` or `pkg> add SomePackage`, and the notebook file will update. 2. Whenever the notebook file changes, the REPL environment will be updated from the notebook file. ## Step 3. When you are done, you can exit the notebook environment by deactivating it: ``` pkg> activate ``` """ |> Markdown.parse |> display println() end nothing end """ ```julia activate_notebook_environment(f::Function, notebook_path::String) ``` Temporarily activate the package environment embedded in a notebook file, for use inside scripts. Inside your function `f`, you can use Pkg commands to modify the environment, and any changes you make will be automatically saved in the notebook file after your function finishes. Not thread-safe. This method is best for scripts that update notebook files. For interactive use, the method `activate_notebook_environment(notebook_path::String)` is recommended. # Example ```julia Pluto.activate_notebook_environment("notebook.jl") do Pkg.add("Example") end # Now the file "notebook.jl" was updated! ``` !!! warning This function uses the private method `Pkg.activate(f::Function, path::String)`. This API might not be available in future Julia versions.  """ function activate_notebook_environment(f::Function, path::String) notebook = load_notebook_nobackup(path) ensure_has_nbpkg(notebook) ourpath = joinpath(mktempdir(), basename(path)) mkpath(ourpath) write_nb_to_dir(notebook, ourpath) result = Pkg.activate(f, ourpath) if !nb_and_dir_environments_equal(notebook, ourpath) write_dir_to_nb(ourpath, notebook) end result end const activate_notebook = activate_notebook_environment function testnb(name="simple_stdlib_import.jl") t = tempname() readwrite(Pluto.project_relative_path("test", "packages", name), t) t end end function precompile_isolated( environment::String; compiler_options::Configuration.CompilerOptions=Configuration.CompilerOptions(), io::IO, ) flags = Configuration._convert_to_flags(compiler_options) code = """ # import Pkg with safe load path pushfirst!(LOAD_PATH, "@stdlib") import Pkg popfirst!(LOAD_PATH) out_stream = IOContext(stdout, :color => true) # I'm a pirate harrr  @static if isdefined(Pkg, :can_fancyprint) Pkg.can_fancyprint(io::IO) = true end @static if isdefined(Base, :Precompilation) && isdefined(Base.Precompilation, :can_fancyprint) Base.Precompilation.can_fancyprint(io::IO) = true end Pkg.activate($(repr(environment)); io=out_stream) Pkg.precompile(; already_instantiated=true, io=out_stream) """ cmd = `$(Base.julia_cmd()[1]) $(flags) -e $(code)` stderr_buffer = IOBuffer() stderr_capture = tee_io(stderr, stderr_buffer) # not to io because any stderr content will be shown eventually by the `error`. try Base.run(pipeline( cmd; stdout=io, stderr=stderr_capture.io, )) catch e if e isa ProcessFailedException throw(PrecompilationFailedException("Precompilation failed\n\n$(String(take!(stderr_buffer)))")) else rethrow(e) end finally stderr_capture.close() end # In the future we could allow interrupting the precompilation process (e.g. when the notebook is shut down) # by running this code using Malt.jl end struct PrecompilationFailedException <: Exception msg::String end # Create a new IO object that redirects all writes to the given capture IOs. It's like the `tee` linux command. Return a named tuple with the IO object and a function to close it which you should not forget to call. function tee_io(captures...) bs = Base.BufferStream() t = @async begin while !eof(bs) data = readavailable(bs) isempty(data) && continue for s in captures write(s, data) end end end function closeme() close(bs) wait(t) end return (io=bs, close=closeme) end # The goal of this file is to import PlutoRunner into Main, on the process of the notebook (created by Malt.jl). # # This is difficult because PlutoRunner uses standard libraries and packages that are not necessarily available in the standard environment. # # Our solution is to create a temporary environment just for loading PlutoRunner. This environment is stored in a scratchspace parameterized by the Pluto version and julia version, # and used by all notebook launches. Reusing the environment means extra speed. begin pushfirst!(LOAD_PATH, "@stdlib") import Pkg popfirst!(LOAD_PATH) local original_LP = copy(LOAD_PATH) local original_AP = Base.ACTIVE_PROJECT[] # Path to our notebook boot package environment which is set by WorkspaceManager # when spawning the process. local runner_env_dir = pluto_boot_environment_path local new_LP = ["@", "@stdlib"] local new_AP = runner_env_dir try # Activate the environment copy!(LOAD_PATH, new_LP) Base.ACTIVE_PROJECT[] = new_AP # Set up our notebook boot package environment by adding a single package: path = joinpath(@__DIR__, "PlutoRunner") try Pkg.develop([Pkg.PackageSpec(; path)]; io=devnull) catch @warn "Something went wrong while initializing the notebook boot environment... Trying again and showing you the output." Pkg.develop([Pkg.PackageSpec(; path)]) end # Resolve try Pkg.resolve(; io=devnull) # supress IO catch @warn "Something went wrong while initializing the notebook boot environment... Trying again and showing you the output." try Pkg.resolve() catch e @error "Failed to resolve notebook boot environment" exception = (e, catch_backtrace()) end end # Instantiate try # we don't suppress IO for this one because it can take very long, and that would be a frustrating experience without IO # precompilation switched off because of https://github.com/fonsp/Pluto.jl/issues/875 Pkg.instantiate(; update_registry=false, allow_autoprecomp=false) catch e @error "Failed to instantiate notebook boot environment" exception = (e, catch_backtrace()) end # Import PlutoRunner into Main import PlutoRunner finally # Reset the pkg environment copy!(LOAD_PATH, original_LP) Base.ACTIVE_PROJECT[] = original_AP end end name = "PlutoRunner" uuid = "dc6b355a-2368-4481-ae6d-ae0351418d79" authors = ["Michiel Dral <m.c.dral@gmail.com>", "Fons van der Plas <fons@plutojl.org>", "Paul Berg <paul@plutojl.org>"] version = "29.14.98" [deps] Base64 = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f" Dates = "ade2ca70-3891-5945-98fb-dc099432e06a" Distributed = "8ba89e20-285c-5b6f-9357-94700520ee1b" InteractiveUtils = "b77e0a4c-d291-57a0-90e8-8db25a27a240" Logging = "56ddb016-857b-54e1-b83d-db4d58db5568" Markdown = "d6f4376e-aef5-505a-96c1-9c027394607a" Pkg = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f" PrecompileTools = "aea7be01-6a6a-4083-8856-8a6e6704d82a" REPL = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb" Sockets = "6462fe0b-24de-5631-8697-dd941f90decc" UUIDs = "cf7118a7-6976-5b1a-9a39-7adc72f591a4" [compat] # these compat entries should match those for the same packages in Project.toml of Pluto.jl itself. PrecompileTools = "~1.2, ~1.3" This module will be evaluated _inside_ the workspace process. Pluto does most things on the server, but it uses worker processes to evaluate notebook code in. These processes don't import Pluto, they only import this module. Functions from this module are called by WorkspaceManager.jl via Malt. When reading this file, pretend that you are living in a worker process, and you are communicating with Pluto's server, who lives in the main process. The package environment that this file is loaded with is the NotebookProcessProject.toml file in this directory. # SOME EXTRA NOTES Restrict the communication between PlutoRunner and the Pluto server to only use *Base Julia types*, like `String`, `Dict`, `NamedTuple`, etc. # DEVELOPMENT TIP If you are editing PlutoRunner, you cannot use Revise unfortunately. However! You don't need to restart Pluto to test your changes! You just need to restart the notebook from the Pluto main menu, and the new PlutoRunner.jl will be loaded. """ !!! danger "Danger zone" Hello! Nice to see that you are interested in how Pluto works! Be careful when using `PlutoRunner` for your project, nothing in this module is public API, and things might suddenly break in a future Pluto version. Instead, try to use AbstractPlutoDingetjes.jl and PlutoHooks.jl to achieve your goal  this is our public API. """ module PlutoRunner export @bind # using these two for two reasons: # - something related to package loading (original text for 4 years ago: "so that they can be imported from Main on the worker process if it launches without the stdlibs in its LOAD_PATH") # - static macro expansion (`maybe_macroexpand_pluto`) happens in this module, and expects these to be using-ed. using Markdown using InteractiveUtils # shared things between files: using UUIDs const ObjectID = typeof(objectid("hello computer")) struct CachedMacroExpansion original_expr_hash::UInt64 expanded_expr::Expr expansion_duration::UInt64 has_pluto_hook_features::Bool did_mention_expansion_time::Bool expansion_logs::Vector{Any} end const supported_integration_features = Any[] abstract type SpecialPlutoExprValue end struct GiveMeCellID <: SpecialPlutoExprValue end struct GiveMeRerunCellFunction <: SpecialPlutoExprValue end struct GiveMeRegisterCleanupFunction <: SpecialPlutoExprValue end # TODO: clear key when a cell is deleted furever const cell_results = Dict{UUID,Any}() const cell_runtimes = Dict{UUID,Union{Nothing,UInt64}}() const cell_published_objects = Dict{UUID,Dict{String,Any}}() const cell_registered_bond_names = Dict{UUID,Set{Symbol}}() const cell_expanded_exprs = Dict{UUID,CachedMacroExpansion}() """ `PlutoRunner.notebook_id[]` gives you the notebook ID used to identify a session. """ const notebook_id = Ref{UUID}(UUID(0)) include("./evaluation/workspace.jl") include("./evaluation/return.jl") include("./evaluation/macro.jl") include("./evaluation/collect_soft_definitions.jl") include("./evaluation/run_expression.jl") include("./evaluation/deleting globals.jl") include("./display/LaTeX.jl") include("./display/HeaderID.jl") include("./display/format_output.jl") include("./display/IOContext.jl") include("./display/syntax error.jl") include("./display/Exception.jl") include("./display/mime dance.jl") include("./display/tree viewer.jl") include("./integrations.jl") include("./ide features/completions.jl") include("./ide features/docs.jl") include("./bonds.jl") include("./js/published_to_js.jl") include("./display/embed_display.jl") include("./display/DivElement.jl") include("./js/jslink.jl") include("./io/logging.jl") include("./io/stdout.jl") include("./precompile.jl") function __init__() original_stderr[] = stderr end end import Base64 import UUIDs: UUID const registered_bond_elements = Dict{Symbol, Any}() function transform_bond_value(s::Symbol, value_from_js) element = get(registered_bond_elements, s, nothing) return try transform_value_ref[](element, value_from_js) catch e @error " AbstractPlutoDingetjes: Bond value transformation errored." exception=(e, catch_backtrace()) (; message=Text(" AbstractPlutoDingetjes: Bond value transformation errored."), exception=Text( sprint(showerror, e, stacktrace(catch_backtrace())) ), value_from_js, ) end end function get_bond_names(cell_id) get(cell_registered_bond_names, cell_id, Set{Symbol}()) end function possible_bond_values(s::Symbol; get_length::Bool=false) element = registered_bond_elements[s] possible_values = possible_bond_values_ref[](element) if possible_values === :NotGiven # Short-circuit to avoid the checks below, which only work if AbstractPlutoDingetjes is loaded. :NotGiven elseif possible_values isa AbstractPlutoDingetjes.Bonds.InfinitePossibilities # error("Bond \"$s\" has an unlimited number of possible values, try changing the `@bind` to something with a finite number of possible values like `PlutoUI.CheckBox(...)` or `PlutoUI.Slider(...)` instead.") :InfinitePossibilities elseif (possible_values isa AbstractPlutoDingetjes.Bonds.NotGiven) # error("Bond \"$s\" did not specify its possible values with `AbstractPlutoDingetjes.Bond.possible_values()`. Try using PlutoUI for the `@bind` values.") # If you change this, change it everywhere in this file. :NotGiven else get_length ? try length(possible_values) catch length(make_serializable(possible_values)) end : make_serializable(possible_values) end end make_serializable(x::Any) = x make_serializable(x::Union{AbstractVector,AbstractSet,Base.Generator}) = collect(x) make_serializable(x::Union{Vector,Set,OrdinalRange}) = x """ _The name is Bond, James Bond._ Wraps around an `element` and not much else. When you `show` a `Bond` with the `text/html` MIME type, you will get: ```html <bond def="\$(bond.defines)"> \$(repr(MIME"text/html"(), bond.element)) </bond> ``` For example, `Bond(html"<input type=range>", :x)` becomes: ```html <bond def="x"> <input type=range> </bond> ``` The actual reactive-interactive functionality is not done in Julia - it is handled by the Pluto front-end (JavaScript), which searches cell output for `<bond>` elements, and attaches event listeners to them. Put on your slippers and have a look at the JS code to learn more. """ struct Bond element::Any defines::Symbol unique_id::String Bond(element, defines::Symbol) = showable(MIME"text/html"(), element) ? new(element, defines, Base64.base64encode(rand(UInt8,9))) : error("""Can only bind to html-showable objects, ie types T for which show(io, ::MIME"text/html", x::T) is defined.""") end function create_bond(element, defines::Symbol, cell_id::UUID) push!(cell_registered_bond_names[cell_id], defines) registered_bond_elements[defines] = element Bond(element, defines) end function Base.show(io::IO, m::MIME"text/html", bond::Bond) # The attribute unique_id has no direct purpose. The only purpose is to force a re-render when it changes. It changes when the @bind expression re-runs (not just on macroexpand). Markdown.withtag(io, :bond, :def => bond.defines, :unique_id => bond.unique_id) do show(io, m, bond.element) end end const initial_value_getter_ref = Ref{Function}(element -> missing) const transform_value_ref = Ref{Function}((element, x) -> x) const possible_bond_values_ref = Ref{Function}((_args...; _kwargs...) -> :NotGiven) """ ```julia @bind symbol element ``` Return the HTML `element`, and use its latest JavaScript value as the definition of `symbol`. # Example ```julia @bind x html"<input type=range>" ``` and in another cell: ```julia x^2 ``` The first cell will show a slider as the cell's output, ranging from 0 until 100. The second cell will show the square of `x`, and is updated in real-time as the slider is moved. """ macro bind(def, element) if def isa Symbol quote $(load_integrations_if_needed)() local el = $(esc(element)) global $(esc(def)) = Core.applicable(Base.get, el) ? Base.get(el) : $(initial_value_getter_ref)[](el) PlutoRunner.create_bond(el, $(Meta.quot(def)), $(GiveMeCellID())) end else :(throw(ArgumentError("""\nMacro example usage: \n\n\t@bind my_number html"<input type='range'>"\n\n"""))) end end """ Will be inserted in saved notebooks that use the @bind macro, make sure that they still contain legal syntax when executed as a vanilla Julia script. Overloading `Base.get` for custom UI objects gives bound variables a sensible value. Also turns off Runic and JuliaFormatter formatting to avoid issues with the formatter trying to change code that the user does not control. See https://domluna.github.io/JuliaFormatter.jl/stable/#Turn-off/on-formatting or https://github.com/fredrikekre/Runic.jl?tab=readme-ov-file#toggle-formatting """ const fake_bind = """ macro bind(def, element) #! format: off return quote local iv = try Base.loaded_modules[Base.PkgId(Base.UUID("6e696c72-6542-2067-7265-42206c756150"), "AbstractPlutoDingetjes")].Bonds.initial_value catch; b -> missing; end local el = \$(esc(element)) global \$(esc(def)) = Core.applicable(Base.get, el) ? Base.get(el) : iv(el) el end #! format: on end""" Base.@kwdef struct Integration id::Base.PkgId code::Expr loaded::Ref{Bool}=Ref(false) end # We have a super cool viewer for objects that are a Tables.jl table. To avoid version conflicts, we only load this code after the user (indirectly) loaded the package Tables.jl. # This is similar to how Requires.jl works, except we don't use a callback, we just check every time. const integrations = Integration[ Integration( id = Base.PkgId(Base.UUID(reinterpret(UInt128, codeunits("Paul Berg Berlin")) |> first), "AbstractPlutoDingetjes"), code = quote @assert v"1.0.0" <= AbstractPlutoDingetjes.MY_VERSION < v"2.0.0" supported!(xs...) = append!(supported_integration_features, xs) # don't need feature checks for these because they existed in every version of AbstractPlutoDingetjes: supported!( AbstractPlutoDingetjes, AbstractPlutoDingetjes.Bonds, AbstractPlutoDingetjes.Bonds.initial_value, AbstractPlutoDingetjes.Bonds.transform_value, AbstractPlutoDingetjes.Bonds.possible_values, ) initial_value_getter_ref[] = AbstractPlutoDingetjes.Bonds.initial_value transform_value_ref[] = AbstractPlutoDingetjes.Bonds.transform_value possible_bond_values_ref[] = AbstractPlutoDingetjes.Bonds.possible_values # feature checks because these were added in a later release of AbstractPlutoDingetjes if isdefined(AbstractPlutoDingetjes, :Display) supported!(AbstractPlutoDingetjes.Display) if isdefined(AbstractPlutoDingetjes.Display, :published_to_js) supported!(AbstractPlutoDingetjes.Display.published_to_js) end if isdefined(AbstractPlutoDingetjes.Display, :with_js_link) supported!(AbstractPlutoDingetjes.Display.with_js_link) end end end, ), Integration( id = Base.PkgId(UUID("0c5d862f-8b57-4792-8d23-62f2024744c7"), "Symbolics"), code = quote pluto_showable(::MIME"application/vnd.pluto.tree+object", ::Symbolics.Arr) = false end, ), Integration( id = Base.PkgId(UUID("0703355e-b756-11e9-17c0-8b28908087d0"), "DimensionalData"), code = quote pluto_showable(::MIME"application/vnd.pluto.tree+object", ::DimensionalData.DimArray) = false end, ), Integration( id = Base.PkgId(UUID("bd369af6-aec1-5ad0-b16a-f7cc5008161c"), "Tables"), code = quote function maptruncated(f::Function, xs, filler, limit; truncate=true) if truncate result = Any[ # not xs[1:limit] because of https://github.com/JuliaLang/julia/issues/38364 f(xs[i]) for i in Iterators.take(eachindex(xs), limit) ] push!(result, filler) result else Any[f(x) for x in xs] end end function table_data(x::Any, io::Context) rows = Tables.rows(x) my_row_limit = get_my_display_limit(x, 1, 0, io, table_row_display_limit, table_row_display_limit_increase) # TODO: the commented line adds support for lazy loading columns, but it uses the same extra_items counter as the rows. So clicking More Rows will also give more columns, and vice versa, which isn't ideal. To fix, maybe use (objectid,dimension) as index instead of (objectid)? my_column_limit = get_my_display_limit(x, 2, 0, io, table_column_display_limit, table_column_display_limit_increase) # my_column_limit = table_column_display_limit # additional 5 so that we don't cut off 1 or 2 itmes - that's silly truncate_rows = my_row_limit + 5 < length(rows) truncate_columns = if isempty(rows) false else my_column_limit + 5 < length(first(rows)) end row_data_for(row) = maptruncated(row, "more", my_column_limit; truncate=truncate_columns) do el format_output_default(el, io) end # ugliest code in Pluto: # not a map(row) because it needs to be a Vector # not enumerate(rows) because of some silliness # not rows[i] because `getindex` is not guaranteed to exist L = truncate_rows ? my_row_limit : length(rows) row_data = Vector{Any}(undef, L) for (i, row) in zip(1:L,rows) row_data[i] = (i, row_data_for(row)) end if truncate_rows push!(row_data, "more") # In some environments this fails. Not sure why. last_row = applicable(lastindex, rows) ? try last(rows) catch e nothing end : nothing if !isnothing(last_row) push!(row_data, (length(rows), row_data_for(last_row))) end end # TODO: render entire schema by default? schema = Tables.schema(rows) schema_data = schema === nothing ? nothing : Dict{Symbol,Any}( :names => maptruncated(string, schema.names, "more", my_column_limit; truncate=truncate_columns), :types => String.(maptruncated(trynameof, schema.types, "more", my_column_limit; truncate=truncate_columns)), ) Dict{Symbol,Any}( :objectid => objectid2str(x), :schema => schema_data, :rows => row_data, ) end #= If the object we're trying to fileview provides rowaccess, let's try to show it. This is guaranteed to be fast (while Table.rows() may be slow). If the object is a lazy iterator, the show method will probably crash and return text repr. That's good because we don't want the show method of lazy iterators (e.g. database cursors) to be changing the (external) iterator implicitly =# pluto_showable(::MIME"application/vnd.pluto.table+object", x::Any) = try Tables.rowaccess(x)::Bool catch; false end pluto_showable(::MIME"application/vnd.pluto.table+object", t::Type) = false pluto_showable(::MIME"application/vnd.pluto.table+object", t::AbstractVector{<:NamedTuple}) = false pluto_showable(::MIME"application/vnd.pluto.table+object", t::AbstractVector{<:Dict{Symbol,<:Any}}) = false pluto_showable(::MIME"application/vnd.pluto.table+object", t::AbstractVector{Union{}}) = false end, ), Integration( id = Base.PkgId(UUID("91a5bcdd-55d7-5caf-9e0b-520d859cae80"), "Plots"), code = quote approx_size(p::Plots.Plot) = try sum(p.series_list; init=0) do series length(something(get(series, :y, ()), ())) end catch e @warn "Failed to guesstimate plot size" exception=(e,catch_backtrace()) 0 end const max_plot_size = 8000 function pluto_showable(::MIME"image/svg+xml", p::Plots.Plot{Plots.GRBackend}) format = try p.attr[:html_output_format] catch :auto end format === :svg || ( format === :auto && approx_size(p) <= max_plot_size ) end pluto_showable(::MIME"text/html", p::Plots.Plot{Plots.GRBackend}) = false end, ), Integration( id = Base.PkgId(UUID("4e3cecfd-b093-5904-9786-8bbb286a6a31"), "ImageShow"), code = quote pluto_showable(::MIME"text/html", ::AbstractMatrix{<:ImageShow.Colorant}) = false end, ), ] function load_integration_if_needed(integration::Integration) if !integration.loaded[] && haskey(Base.loaded_modules, integration.id) load_integration(integration) end end load_integrations_if_needed() = load_integration_if_needed.(integrations) function load_integration(integration::Integration) integration.loaded[] = true try eval(quote const $(Symbol(integration.id.name)) = Base.loaded_modules[$(integration.id)] $(integration.code) end) true catch e @error "Failed to load integration with $(integration.id.name).jl" exception=(e, catch_backtrace()) false end end using PrecompileTools: PrecompileTools using UUIDs: uuid1 const __TEST_NOTEBOOK_ID = uuid1() const __precompile_test_workspace = VERSION < v"1.12.0-aaa" ? Module() : Main PrecompileTools.@compile_workload begin let channel = Channel{Any}(10) PlutoRunner.setup_plutologger( __TEST_NOTEBOOK_ID, channel, ) end expr = Expr(:toplevel, :(1 + 1)) cell_id = uuid1() PlutoRunner.run_expression(__precompile_test_workspace, expr, __TEST_NOTEBOOK_ID, cell_id, nothing); PlutoRunner.formatted_result_of(__TEST_NOTEBOOK_ID, cell_id, false, String[], nothing, __precompile_test_workspace; capture_stdout=true) foreach(("sq", "\\sq", "Base.a", "sqrt(", "sum(x; dim")) do s PlutoRunner.completion_fetcher(s, s, Main) end end Base.@kwdef struct DivElement children::Vector style::String="" class::Union{String,Nothing}=nothing end tree_data(@nospecialize(e::DivElement), context::Context) = Dict{Symbol, Any}( :style => e.style, :classname => e.class, :children => Any[ format_output_default(value, context) for value in e.children ], ) pluto_showable(::MIME"application/vnd.pluto.divelement+object", ::DivElement) = true function Base.show(io::IO, m::MIME"text/html", e::DivElement) Base.show(io, m, embed_display(e)) end "Downstream packages can set this to false to obtain unprettified stack traces." const PRETTY_STACKTRACES = Ref(true) function frame_is_from_plutorunner(frame::Base.StackTraces.StackFrame) if frame.linfo isa Core.MethodInstance frame.linfo.def.module === PlutoRunner else endswith(String(frame.file), "PlutoRunner.jl") end end frame_is_from_usercode(frame::Base.StackTraces.StackFrame) = occursin("#==#", String(frame.file)) function method_from_frame(frame::Base.StackTraces.StackFrame) if frame.linfo isa Core.MethodInstance frame.linfo.def elseif frame.linfo isa Method frame.linfo else nothing end end frame_url(m::Method) = Base.url(m) frame_url(::Any) = nothing function source_package(m::Union{Method,Module}) next = parentmodule(m) next === m ? m : source_package(next) end source_package(::Any) = nothing function format_output(val::CapturedException; context=default_iocontext) if val.ex isa PrettySyntaxError dict = convert_parse_error_to_dict(val.ex.ex.detail) return dict, MIME"application/vnd.pluto.parseerror+object"() end if PRETTY_STACKTRACES[] ## We hide the part of the stacktrace that belongs to Pluto's evalling of user code. stack = [s for (s, _) in val.processed_bt] # function_wrap_index = findfirst(f -> occursin("function_wrapped_cell", String(f.func)), stack) function_wrap_index = findlast(frame_is_from_usercode, stack) internal_index = findfirst(frame_is_from_plutorunner, stack) limit = if function_wrap_index !== nothing function_wrap_index elseif internal_index !== nothing internal_index - 1 else nothing end stack_relevant = stack[1:something(limit, end)] pretty = map(stack_relevant) do s func = s.func === nothing ? nothing : s.func isa Symbol ? String(s.func) : repr(s.func) method = method_from_frame(s) sp = source_package(method) pm = method isa Method ? parentmodule(method) : nothing call = replace(pretty_stackcall(s, s.linfo), r"Main\.var\"workspace#\d+\"\." => "") Dict( :call => call, :call_short => type_depth_limit(call, 0), :func => func, :inlined => s.inlined, :from_c => s.from_c, :file => basename(String(s.file)), :path => String(s.file), :line => s.line, :linfo_type => string(typeof(s.linfo)), :url => frame_url(method), :source_package => sp === nothing ? nothing : string(sp), :parent_module => pm === nothing ? nothing : string(pm), ) end ( Dict{Symbol,Any}( :msg => sprint(try_showerror, val.ex), :stacktrace => pretty, :plain_error => sprint() do io try_showerror(io, val.ex, val.processed_bt[1:something(limit, end)]; color=false) end, ), MIME"application/vnd.pluto.stacktrace+object"() ) else Dict{Symbol,Any}(:msg => sprint(try_showerror, val.ex), :stacktrace => val), MIME"application/vnd.pluto.stacktrace+object"() end end # from the Julia source code: function pretty_stackcall(frame::Base.StackFrame, linfo::Nothing)::String if frame.func isa Symbol if occursin("function_wrapped_cell", String(frame.func)) "top-level scope" else String(frame.func) end else repr(frame.func) end end function pretty_stackcall(frame::Base.StackFrame, linfo::Core.CodeInfo) "top-level scope" end function pretty_stackcall(frame::Base.StackFrame, linfo::Union{Core.MethodInstance, Core.CodeInstance}) if linfo.def isa Method @static if isdefined(Base.StackTraces, :show_spec_linfo) && hasmethod(Base.StackTraces.show_spec_linfo, Tuple{IO, Base.StackFrame}) sprint(Base.StackTraces.show_spec_linfo, frame; context=:backtrace => true) else split(string(frame), " at ") |> first end else sprint(Base.show, linfo) end end function pretty_stackcall(frame::Base.StackFrame, linfo::Method) sprint(Base.show_tuple_as_call, linfo.name, linfo.sig) end function pretty_stackcall(frame::Base.StackFrame, linfo::Module) sprint(Base.show, linfo) end function type_depth_limit(call::String, n::Int) !occursin("{" , call) && return call @static if isdefined(Base, :type_depth_limit) && hasmethod(Base.type_depth_limit, Tuple{String, Int}) Base.type_depth_limit(call, n) else call end end "Because even showerror can error... " function try_showerror(io::IO, e, args...; color::Bool=true) try showerror(IOContext(io, :color => color), e, args...) catch show_ex print(io, "\nFailed to show error:\n\n") try_showerror(io, show_ex, stacktrace(catch_backtrace())) end end import Markdown # We add a method for the Markdown -> HTML conversion that automatically adds an id to headers function Markdown.html(io::IOContext, header::Markdown.Header{l}) where l id = text_to_id(stripped(header.text)) Markdown.withtag(io, "h$l", :id => id) do Markdown.htmlinline(io, header.text) end end # We don't need to encode this, Markdown stdlib will do it for us. But we want to replace space with `-` text_to_id(text::String) = replace(text, " " => "-") stripped(x::Vector) = join(Iterators.map(stripped, x)) stripped(s::String) = s stripped(s::Union{Markdown.Italic,Markdown.Link,Markdown.Bold}) = stripped(s.text) stripped(s::Markdown.LaTeX) = stripped(s.formula) stripped(s::Markdown.Code) = stripped(s.code) stripped(s::Markdown.Image) = "" stripped(s) = "" # because i like that Base.IOContext(io::IOContext, ::Nothing) = io "The `IOContext` used for converting arbitrary objects to pretty strings." const default_iocontext = IOContext(devnull, :color => true, :limit => true, :displaysize => (18, 88), :is_pluto => true, :pluto_supported_integration_features => supported_integration_features, :pluto_published_to_js => (io, x) -> core_published_to_js(io, x), :pluto_with_js_link => (io, callback, on_cancellation) -> core_with_js_link(io, callback, on_cancellation), ) # `stdout` mimics a TTY, the only relevant property is :color const default_stdout_iocontext = IOContext(devnull, :color => true, :is_pluto => false, ) # `display` sees a richer context like in the REPL, see #2727 const default_display_iocontext = IOContext(devnull, :color => true, :limit => true, :displaysize => (18, 75), :is_pluto => false, ) import Markdown # We add a method for the Markdown -> HTML conversion that takes a LaTeX chunk from the Markdown tree and adds our custom span function Markdown.htmlinline(io::IO, x::Markdown.LaTeX) Markdown.withtag(io, :span, :class => "tex") do print(io, '$') Markdown.htmlesc(io, x.formula) print(io, '$') end end # this one for block equations: (double $$) function Markdown.html(io::IO, x::Markdown.LaTeX) Markdown.withtag(io, :p, :class => "tex") do print(io, '$', '$') Markdown.htmlesc(io, x.formula) print(io, '$', '$') end end const _EmbeddableDisplay_enable_html_shortcut = Ref{Bool}(true) struct EmbeddableDisplay x script_id::String end function Base.show(io::IO, m::MIME"text/html", e::EmbeddableDisplay) body, mime = format_output_default(e.x, io) to_write = if mime === m && _EmbeddableDisplay_enable_html_shortcut[] # In this case, we can just embed the HTML content directly. body else s = """<pluto-display></pluto-display><script id=$(e.script_id)> // see https://plutocon2021-demos.netlify.app/fonsp%20%E2%80%94%20javascript%20inside%20pluto to learn about the techniques used in this script const body = $(PublishedToJavascript(body)); const mime = "$(string(mime))"; const create_new = this == null || this._mime !== mime; const display = create_new ? currentScript.previousElementSibling : this; display.persist_js_state = true; display.sanitize_html = false; display.body = body; if(create_new) { // only set the mime if necessary, it triggers a second preact update display.mime = mime; // add it also as unwatched property to prevent interference from Preact display._mime = mime; } return display; </script>""" replace(replace(s, r"//.+" => ""), "\n" => "") end write(io, to_write) end export embed_display """ embed_display(x) A wrapper around any object that will display it using Pluto's interactive multimedia viewer (images, arrays, tables, etc.), the same system used to display cell output. The returned object can be **embedded in HTML output** (we recommend [HypertextLiteral.jl](https://github.com/MechanicalRabbit/HypertextLiteral.jl) or [HyperScript.jl](https://github.com/yurivish/Hyperscript.jl)), which means that you can use it to create things like _"table viewer left, plot right"_. # Example Markdown can interpolate HTML-showable objects, including the embedded display: ```julia md"\"" # Cool data \$(embed_display(rand(10))) Wow! "\"" ``` You can use HTML templating packages to create cool layouts, like two arrays side-by-side: ```julia using HypertextLiteral ``` ```julia @htl("\"" <div style="display: flex;"> \$(embed_display(rand(4))) \$(embed_display(rand(4))) </div> "\"") ``` """ embed_display(x) = EmbeddableDisplay(x, rand('a':'z',16) |> join) # if an embedded display is being rendered _directly by Pluto's viewer_, then rendered the embedded object directly. When interpolating an embedded display into HTML, the user code will render the embedded display to HTML using the HTML show method above, and this shortcut is not called. # We add this short-circuit to increase performance for UI that uses an embedded display when it is not necessary. format_output_default(@nospecialize(val::EmbeddableDisplay), @nospecialize(context=default_iocontext)) = format_output_default(val.x, context) # This is not a struct to make it easier to pass these objects between processes. const MimedOutput = Tuple{Union{String,Vector{UInt8},Dict{Symbol,Any}},MIME} const ObjectDimPair = Tuple{ObjectID,Int64} const tree_display_limit = 30 const tree_display_limit_increase = 40 const table_row_display_limit = 10 const table_row_display_limit_increase = 60 const table_column_display_limit = 8 const table_column_display_limit_increase = 30 const tree_display_extra_items = Dict{UUID,Dict{ObjectDimPair,Int64}}() # This is not a struct to make it easier to pass these objects between processes. const FormattedCellResult = NamedTuple{(:output_formatted, :errored, :interrupted, :process_exited, :runtime, :published_objects, :has_pluto_hook_features),Tuple{PlutoRunner.MimedOutput,Bool,Bool,Bool,Union{UInt64,Nothing},Dict{String,Any},Bool}} function formatted_result_of( notebook_id::UUID, cell_id::UUID, ends_with_semicolon::Bool, known_published_objects::Vector{String}=String[], showmore::Union{ObjectDimPair,Nothing}=nothing, workspace::Module=Main; capture_stdout::Bool=true, )::FormattedCellResult load_integrations_if_needed() currently_running_cell_id[] = cell_id extra_items = if showmore === nothing tree_display_extra_items[cell_id] = Dict{ObjectDimPair,Int64}() else old = get!(() -> Dict{ObjectDimPair,Int64}(), tree_display_extra_items, cell_id) old[showmore] = get(old, showmore, 0) + 1 old end has_pluto_hook_features = haskey(cell_expanded_exprs, cell_id) && cell_expanded_exprs[cell_id].has_pluto_hook_features ans = cell_results[cell_id] errored = ans isa CapturedException output_formatted = if (!ends_with_semicolon || errored) with_logger_and_io_to_logs(get_cell_logger(notebook_id, cell_id); capture_stdout) do format_output(ans; context=IOContext( default_iocontext, :extra_items=>extra_items, :module => workspace, :pluto_notebook_id => notebook_id, :pluto_cell_id => cell_id, )) end else ("", MIME"text/plain"()) end published_objects = get(cell_published_objects, cell_id, Dict{String,Any}()) for k in known_published_objects if haskey(published_objects, k) published_objects[k] = nothing end end return (; output_formatted, errored, interrupted = false, process_exited = false, runtime = get(cell_runtimes, cell_id, nothing), published_objects, has_pluto_hook_features, ) end """ Format `val` using the richest possible output, return formatted string and used MIME type. See [`allmimes`](@ref) for the ordered list of supported MIME types. """ function format_output_default(@nospecialize(val), @nospecialize(context=default_iocontext))::MimedOutput try io_sprinted, (value, mime) = show_richest_withreturned(context, val) if value === nothing if mime  imagemimes (io_sprinted, mime) else (String(io_sprinted)::String, mime) end else (value, mime) end catch ex title = ErrorException("Failed to show value: \n" * sprint(try_showerror, ex)) bt = stacktrace(catch_backtrace()) format_output(CapturedException(title, bt)) end end format_output(@nospecialize(x); context=default_iocontext) = format_output_default(x, context) format_output(::Nothing; context=default_iocontext) = ("", MIME"text/plain"()) function format_output(binding::Base.Docs.Binding; context=default_iocontext) try (""" <div class="pluto-docs-binding"> <span id="$(binding.var)">$(binding.var)</span> $(repr(MIME"text/html"(), Base.Docs.doc(binding))) </div> """, MIME"text/html"()) catch e @warn "Failed to pretty-print binding" exception=(e, catch_backtrace()) repr(binding, MIME"text/plain"()) end end import Markdown const imagemimes = MIME[MIME"image/svg+xml"(), MIME"image/png"(), MIME"image/jpg"(), MIME"image/jpeg"(), MIME"image/bmp"(), MIME"image/gif"()] # in descending order of coolness # text/plain always matches - almost always """ The MIMEs that Pluto supports, in order of how much I like them. `text/plain` should always match - the difference between `show(::IO, ::MIME"text/plain", x)` and `show(::IO, x)` is an unsolved mystery. """ const allmimes = MIME[MIME"application/vnd.pluto.table+object"(); MIME"application/vnd.pluto.divelement+object"(); MIME"text/html"(); imagemimes; MIME"application/vnd.pluto.tree+object"(); MIME"text/latex"(); MIME"text/plain"()] "Return a `(String, Any)` tuple containing function output as the second entry." function show_richest_withreturned(context::IOContext, @nospecialize(args)) buffer = IOBuffer(; sizehint=0) val = show_richest(IOContext(buffer, context), args) return (take!(buffer), val) end "Super important thing don't change." struct  end const struct_showmethod = which(show, (IO, )) const struct_showmethod_mime = which(show, (IO, MIME"text/plain", )) function use_tree_viewer_for_struct(@nospecialize(x::T))::Bool where T # types that have no specialized show methods (their fallback is text/plain) are displayed using Pluto's interactive tree viewer. # this is how we check whether this display method is appropriate: isstruct = try T isa DataType && # there are two ways to override the plaintext show method: which(show, (IO, MIME"text/plain", T)) === struct_showmethod_mime && which(show, (IO, T)) === struct_showmethod catch false end isstruct && let # from julia source code, dont know why nf = nfields(x) nb = sizeof(x) nf != 0 || nb == 0 end end """ is_mime_enabled(::MIME) -> Bool Return whether the argument's mimetype is enabled. This defaults to `true`, but additional dispatches can be set to `false` by downstream packages. """ is_mime_enabled(::MIME) = true "Return the first mimetype in `allmimes` which can show `x`." function mimetype(x) # ugly code to fix an ugly performance problem for m in allmimes if pluto_showable(m, x) && is_mime_enabled(m) return m end end end """ Like two-argument `Base.show`, except: 1. the richest MIME type available to Pluto will be used 2. the used MIME type is returned as second element 3. if the first returned element is `nothing`, then we wrote our data to `io`. If it is something else (a Dict), then that object will be the cell's output, instead of the buffered io stream. This allows us to output rich objects to the frontend that are not necessarily strings or byte streams """ function show_richest(io::IO, @nospecialize(x))::Tuple{<:Any,MIME} mime = mimetype(x) if mime isa MIME"text/plain" && is_mime_enabled(MIME"application/vnd.pluto.tree+object"()) && use_tree_viewer_for_struct(x) tree_data(x, io), MIME"application/vnd.pluto.tree+object"() elseif mime isa MIME"application/vnd.pluto.tree+object" try tree_data(x, IOContext(io, :compact => true)), mime catch show(io, MIME"text/plain"(), x) nothing, MIME"text/plain"() end elseif mime isa MIME"application/vnd.pluto.table+object" try table_data(x, IOContext(io, :compact => true)), mime catch show(io, MIME"text/plain"(), x) nothing, MIME"text/plain"() end elseif mime isa MIME"application/vnd.pluto.divelement+object" tree_data(x, io), mime elseif mime  imagemimes show(io, mime, x) nothing, mime elseif mime isa MIME"text/latex" # Some reprs include $ at the start and end. # We strip those, since Markdown.LaTeX should contain the math content. # (It will be rendered by MathJax, which is math-first, not text-first.) texed = repr(mime, x) Markdown.html(io, Markdown.LaTeX(strip(texed, ('$', '\n', ' ')))) nothing, MIME"text/html"() else # the classic: show(io, mime, x) nothing, mime end end # we write our own function instead of extending Base.showable with our new MIME because: # we need the method Base.showable(::MIME"asdfasdf", ::Any) = Tables.rowaccess(x) # but overload ::MIME{"asdf"}, ::Any will cause ambiguity errors in other packages that write a method like: # Base.showable(m::MIME, x::Plots.Plot) # because MIME is less specific than MIME"asdff", but Plots.PLot is more specific than Any. pluto_showable(m::MIME, @nospecialize(x))::Bool = Base.invokelatest(showable, m, x) # @codemirror/lint has only three levels function convert_julia_syntax_level(level) level == :error ? "error" : level == :warning ? "warning" : "info" end """ map_byte_range_to_utf16_codepoints(s::String, start_byte::Int, end_byte::Int)::Tuple{Int,Int} Taken from `Base.transcode(::Type{UInt16}, src::Vector{UInt8})` but without line constraints. It also does not support invalid UTF-8 encoding which `String` should never be anyway. This maps the given raw byte range `(start_byte, end_byte)` range to UTF-16 codepoints indices. The resulting range can then be used by code-mirror on the frontend, quoting from the code-mirror docs: > Character positions are counted from zero, and count each line break and UTF-16 code unit as one unit. Examples: ```julia 123 vv julia> map_byte_range_to_utf16_codepoints("abc", 2, 3) (2, 3) 1122 v v julia> map_byte_range_to_utf16_codepoints("", 1, 8) (1, 4) 11233 v v julia> map_byte_range_to_utf16_codepoints("c", 1, 5) (1, 3) ``` """ function map_byte_range_to_utf16_codepoints(s, start_byte, end_byte) invalid_utf8() = error("invalid UTF-8 string") codeunit(s) == UInt8 || invalid_utf8() i, n = 1, ncodeunits(s) u16 = 0 from, to = -1, -1 a = codeunit(s, 1) while true if i == start_byte from = u16 end if i == end_byte to = u16 break end if i < n && -64 <= a % Int8 <= -12 # multi-byte character i += 1 b = codeunit(s, i) if -64 <= (b % Int8) || a == 0xf4 && 0x8f < b # invalid UTF-8 (non-continuation of too-high code point) invalid_utf8() elseif a < 0xe0 # 2-byte UTF-8 if i == start_byte from = u16 end if i == end_byte to = u16 break end elseif i < n # 3/4-byte character i += 1 c = codeunit(s, i) if -64 <= (c % Int8) # invalid UTF-8 (non-continuation) invalid_utf8() elseif a < 0xf0 # 3-byte UTF-8 if i == start_byte from = u16 end if i == end_byte to = u16 break end elseif i < n i += 1 d = codeunit(s, i) if -64 <= (d % Int8) # invalid UTF-8 (non-continuation) invalid_utf8() elseif a == 0xf0 && b < 0x90 # overlong encoding invalid_utf8() else # 4-byte UTF-8 && 2 codeunits UTF-16 u16 += 1 if i == start_byte from = u16 end if i == end_byte to = u16 break end end else # too short invalid_utf8() end else # too short invalid_utf8() end else # ASCII or invalid UTF-8 (continuation byte or too-high code point) end u16 += 1 if i >= n break end i += 1 a = codeunit(s, i) end if from == -1 from = u16 end if to == -1 to = u16 end return (from, to) end function convert_diagnostic_to_dict(source, diag) code = source.code # JuliaSyntax uses `last_byte < first_byte` to signal an empty range. # https://github.com/JuliaLang/JuliaSyntax.jl/blob/97e2825c68e770a3f56f0ec247deda1a8588070c/src/diagnostics.jl#L67-L75 # it references the byte range as such: `source[first_byte:last_byte]` whereas codemirror # is non inclusive, therefore we move the `last_byte` to the next valid character in the string, # an empty range then becomes `from == to`, also JuliaSyntax is one based whereas code-mirror is zero-based # but this is handled in `map_byte_range_to_utf16_codepoints` with `u16 = 0` initially. first_byte = min(diag.first_byte, lastindex(code) + 1) last_byte = min(nextind(code, diag.last_byte), lastindex(code) + 1) from, to = map_byte_range_to_utf16_codepoints(code, first_byte, last_byte) Dict(:from => from, :to => to, :message => diag.message, :source => "JuliaSyntax.jl", :line => first(Base.JuliaSyntax.source_location(source, diag.first_byte)), :severity => convert_julia_syntax_level(diag.level)) end function convert_parse_error_to_dict(ex) Dict( :source => ex.source.code, :diagnostics => [ convert_diagnostic_to_dict(ex.source, diag) for diag in ex.diagnostics ] ) end """ *Internal* wrapper for syntax errors which have diagnostics. Thrown through PlutoRunner.throw_syntax_error """ struct PrettySyntaxError <: Exception ex::Any end function throw_syntax_error(@nospecialize(syntax_err)) syntax_err isa String && (syntax_err = "syntax: $syntax_err") syntax_err isa Exception || (syntax_err = ErrorException(syntax_err)) if syntax_err isa Base.Meta.ParseError && syntax_err.detail isa Base.JuliaSyntax.ParseError syntax_err = PrettySyntaxError(syntax_err) end throw(syntax_err) end # We invent our own MIME _because we can_ but don't use it somewhere else because it might change :) pluto_showable(::MIME"application/vnd.pluto.tree+object", x::AbstractVector{<:Any}) = try eltype(eachindex(x)) === Int; catch; false; end pluto_showable(::MIME"application/vnd.pluto.tree+object", ::AbstractSet{<:Any}) = true pluto_showable(::MIME"application/vnd.pluto.tree+object", ::AbstractDict{<:Any,<:Any}) = true pluto_showable(::MIME"application/vnd.pluto.tree+object", ::Tuple) = true pluto_showable(::MIME"application/vnd.pluto.tree+object", ::NamedTuple) = true pluto_showable(::MIME"application/vnd.pluto.tree+object", ::Pair) = true pluto_showable(::MIME"application/vnd.pluto.tree+object", ::AbstractRange) = false pluto_showable(::MIME"application/vnd.pluto.tree+object", ::Any) = false # in the next functions you see a `context` argument # this is really only used for the circular reference tracking const Context = IOContext{IOBuffer} function tree_data_array_elements(@nospecialize(x::AbstractVector{<:Any}), indices::AbstractVector{I}, context::Context) where {I<:Integer} Tuple{I,Any}[ if isassigned(x, i) i, format_output_default(x[i], context) else i, format_output_default(Text(Base.undef_ref_str), context) end for i in indices ] |> collect end precompile(tree_data_array_elements, (Vector{Any}, Vector{Int}, Context)) function array_prefix(@nospecialize(x::Vector{<:Any})) string(eltype(x))::String end function array_prefix(@nospecialize(x)) original = sprint(Base.showarg, x, false; context=:limit => true) string(lstrip(original, ':'), ": ")::String end function get_my_display_limit(@nospecialize(x), dim::Integer, depth::Integer, context::Context, a::Integer, b::Integer)::Int # needs to be system-dependent Int because it is used as array index let if depth < 3 a  (1 + 2 * depth) else 0 end end + let d = get(context, :extra_items, nothing) if d === nothing 0 else b * get(d, (objectid(x), dim), 0) end end end objectid2str(@nospecialize(x)) = string(objectid(x); base=16)::String function circular(@nospecialize(x)) return Dict{Symbol,Any}( :objectid => objectid2str(x), :type => :circular ) end function tree_data(@nospecialize(x::AbstractSet{<:Any}), context::Context) if Base.show_circular(context, x) return circular(x) else depth = get(context, :tree_viewer_depth, 0) recur_io = IOContext(context, Pair{Symbol,Any}(:SHOWN_SET, x), Pair{Symbol,Any}(:tree_viewer_depth, depth + 1)) my_limit = get_my_display_limit(x, 1, depth, context, tree_display_limit, tree_display_limit_increase) L = min(my_limit+1, length(x)) elements = Vector{Any}(undef, L) index = 1 for value in x if index <= my_limit elements[index] = (index, format_output_default(value, recur_io)) else elements[index] = "more" break end index += 1 end Dict{Symbol,Any}( :prefix => string(typeof(x)), :prefix_short => string(typeof(x) |> trynameof), :objectid => objectid2str(x), :type => :Set, :elements => elements ) end end function tree_data(@nospecialize(x::AbstractVector{<:Any}), context::Context) if Base.show_circular(context, x) return circular(x) else depth = get(context, :tree_viewer_depth, 0)::Int recur_io = IOContext(context, Pair{Symbol,Any}(:SHOWN_SET, x), Pair{Symbol,Any}(:tree_viewer_depth, depth + 1)) indices = eachindex(x) my_limit = get_my_display_limit(x, 1, depth, context, tree_display_limit, tree_display_limit_increase) # additional couple of elements so that we don't cut off 1 or 2 itmes - that's silly elements = if length(x) <= ((my_limit * 6)  5) tree_data_array_elements(x, indices, recur_io) else firsti = firstindex(x) from_end = my_limit > 20 ? 10 : my_limit > 1 ? 1 : 0 Any[ tree_data_array_elements(x, indices[firsti:firsti-1+my_limit-from_end], recur_io); "more"; tree_data_array_elements(x, indices[end+1-from_end:end], recur_io) ] end prefix = array_prefix(x) Dict{Symbol,Any}( :prefix => prefix, :prefix_short => x isa Vector ? "" : prefix, # if not abstract :objectid => objectid2str(x), :type => :Array, :elements => elements ) end end function tree_data(@nospecialize(x::Tuple), context::Context) depth = get(context, :tree_viewer_depth, 0) recur_io = IOContext(context, Pair{Symbol,Any}(:tree_viewer_depth, depth + 1)) elements = Tuple[] for val in x out = format_output_default(val, recur_io) push!(elements, out) end Dict{Symbol,Any}( :objectid => objectid2str(x), :type => :Tuple, :elements => collect(enumerate(elements)), ) end function tree_data(@nospecialize(x::AbstractDict{<:Any,<:Any}), context::Context) if Base.show_circular(context, x) return circular(x) else depth = get(context, :tree_viewer_depth, 0) recur_io = IOContext(context, Pair{Symbol,Any}(:SHOWN_SET, x), Pair{Symbol,Any}(:tree_viewer_depth, depth + 1)) elements = [] my_limit = get_my_display_limit(x, 1, depth, context, tree_display_limit, tree_display_limit_increase) row_index = 1 for pair in x k, v = pair if row_index <= my_limit push!(elements, (format_output_default(k, recur_io), format_output_default(v, recur_io))) else push!(elements, "more") break end row_index += 1 end Dict{Symbol,Any}( :prefix => string(typeof(x)), :prefix_short => string(typeof(x) |> trynameof), :objectid => objectid2str(x), :type => :Dict, :elements => elements ) end end function tree_data_nt_row(@nospecialize(pair::Tuple), context::Context) # this is an entry of a NamedTuple, the first element of the Tuple is a Symbol, which we want to print as `x` instead of `:x` k, element = pair string(k), format_output_default(element, context) end function tree_data(@nospecialize(x::NamedTuple), context::Context) depth = get(context, :tree_viewer_depth, 0) recur_io = IOContext(context, Pair{Symbol,Any}(:tree_viewer_depth, depth + 1)) elements = Tuple[] for key in eachindex(x) val = x[key] data = tree_data_nt_row((key, val), recur_io) push!(elements, data) end Dict{Symbol,Any}( :objectid => objectid2str(x), :type => :NamedTuple, :elements => elements ) end function tree_data(@nospecialize(x::Pair), context::Context) k, v = x Dict{Symbol,Any}( :objectid => objectid2str(x), :type => :Pair, :key_value => (format_output_default(k, context), format_output_default(v, context)), ) end # Based on Julia source code but without writing to IO function tree_data(@nospecialize(x::Any), context::Context) if Base.show_circular(context, x) return circular(x) else depth = get(context, :tree_viewer_depth, 0) recur_io = IOContext(context, Pair{Symbol,Any}(:SHOWN_SET, x), Pair{Symbol,Any}(:typeinfo, Any), Pair{Symbol,Any}(:tree_viewer_depth, depth + 1), ) t = typeof(x) nf = nfields(x) nb = sizeof(x) elements = Any[ let f = fieldname(t, i) if !isdefined(x, f) Base.undef_ref_str f, format_output_default(Text(Base.undef_ref_str), recur_io) else f, format_output_default(getfield(x, i), recur_io) end end for i in 1:nf ] Dict{Symbol,Any}( :prefix => repr(t; context), :prefix_short => string(trynameof(t)), :objectid => objectid2str(x), :type => :struct, :elements => elements, ) end end function trynameof(::Type{Union{T,Missing}}) where T name = trynameof(T) return name === Symbol() ? name : Symbol(name, "?") end trynameof(x::DataType) = nameof(x) trynameof(x::Any) = Symbol() function exported_names(mod::Module) @static if VERSION  v"1.11.0-DEV.469" filter!(Base.Fix1(Base.isexported, mod), names(mod; all=true)) else names(mod) end end function get_module_names(workspace_module, module_ex::Expr) try Core.eval(workspace_module, Expr(:call, exported_names, module_ex)) |> Set{Symbol} catch Set{Symbol}() end end function collect_soft_definitions(workspace_module, modules::Set{Expr}) mapreduce(module_ex -> get_module_names(workspace_module, module_ex), union!, modules; init=Set{Symbol}()) end # This function checks whether the symbol provided to it represents a name of a memoized_cache variable from Memoize.jl, see https://github.com/fonsp/Pluto.jl/issues/2305 for more details is_memoized_cache(s::Symbol) = startswith(string(s), "##") && endswith(string(s), "_memoized_cache") function do_reimports(workspace_name, module_imports_to_move::Set{Expr}) for expr in module_imports_to_move try Core.eval(workspace_name, expr) catch e end # TODO catch specificallly end end """ Move some of the globals over from one workspace to another. This is how Pluto "deletes" globals - it doesn't, it just executes your new code in a new module where those globals are not defined. Notebook code does run in `Main` - it runs in workspace modules. Every time that you run cells, a new module is created, called `Main.workspace123` with `123` an increasing number. The trick boils down to two things: 1. When we create a new workspace module, we move over some of the global from the old workspace. (But not the ones that we want to 'delete'!) 2. If a function used to be defined, but now we want to delete it, then we go through the method table of that function and snoop out all methods that were defined by us, and not by another package. This is how we reverse extending external functions. For example, if you run a cell with `Base.sqrt(s::String) = "the square root of" * s`, and then delete that cell, then you can still call `sqrt(1)` but `sqrt("one")` will err. Cool right! """ function move_vars( old_workspace_name::Symbol, new_workspace_name::Symbol, vars_to_delete::Set{Symbol}, methods_to_delete::Set{Tuple{UUID,Tuple{Vararg{Symbol}}}}, module_imports_to_move::Set{Expr}, cells_to_macro_invalidate::Set{UUID}, cells_to_js_link_invalidate::Set{UUID}, keep_registered::Set{Symbol}, ) old_workspace = getfield(Main, old_workspace_name) new_workspace = getfield(Main, new_workspace_name) for cell_id in cells_to_macro_invalidate delete!(cell_expanded_exprs, cell_id) end foreach(unregister_js_link, cells_to_js_link_invalidate) # TODO: delete Core.eval(new_workspace, :(import ..($(old_workspace_name)))) old_names = names(old_workspace, all=true, imported=true) funcs_with_no_methods_left = filter(methods_to_delete) do f !try_delete_toplevel_methods(old_workspace, f) end name_symbols_of_funcs_with_no_methods_left = last.(last.(funcs_with_no_methods_left)) for symbol in old_names if (symbol  vars_to_delete) || (symbol  name_symbols_of_funcs_with_no_methods_left) # var will be redefined - unreference the value so that GC can snoop it if haskey(registered_bond_elements, symbol) && symbol  keep_registered delete!(registered_bond_elements, symbol) end # free memory for other variables # & delete methods created in the old module: # for example, the old module might extend an imported function: # `import Base: show; show(io::IO, x::Flower) = print(io, "")` # when you delete/change this cell, you want this extension to disappear. if isdefined(old_workspace, symbol) # try_delete_toplevel_methods(old_workspace, symbol) try # We are clearing this variable from the notebook, so we need to find it's root # If its root is "controlled" by Pluto's workspace system (and is not a package module for example), # we are just clearing out the definition in the old_module, besides giving an error # (so that's what that `catch; end` is for) # will not actually free it from Julia, the older module will still have a reference. module_to_remove_from = which(old_workspace, symbol) if is_pluto_controlled(module_to_remove_from) && !isconst(module_to_remove_from, symbol) Core.eval(module_to_remove_from, :($(symbol) = nothing)) end catch; end # sometimes impossible, eg. when $symbol was constant end else # var will not be redefined in the new workspace, move it over if !(symbol == :eval || symbol == :include || (string(symbol)[1] == '#' && !is_memoized_cache(symbol)) || startswith(string(symbol), "workspace#")) try getfield(old_workspace, symbol) # Expose the variable in the scope of `new_workspace` Core.eval(new_workspace, :(import ..($(old_workspace_name)).$(symbol))) catch ex if !(ex isa UndefVarError) @warn "Failed to move variable $(symbol) to new workspace:" showerror(original_stderr[], ex, stacktrace(catch_backtrace())) end end end end end do_reimports(new_workspace, module_imports_to_move) revise_if_possible(new_workspace) end "Return whether the `method` was defined inside this notebook, and not in external code." isfromcell(method::Method, cell_id::UUID) = endswith(String(method.file), string(cell_id)) """ delete_method_doc(m::Method) Tries to delete the documentation for this method, this is used when methods are removed. """ function delete_method_doc(m::Method) binding = Docs.Binding(m.module, m.name) meta = Docs.meta(m.module) if haskey(meta, binding) method_sig = Tuple{m.sig.parameters[2:end]...} multidoc = meta[binding] filter!(multidoc.order) do msig if method_sig == msig pop!(multidoc.docs, msig) false else true end end end end """ Delete all methods of `f` that were defined in this notebook, and leave the ones defined in other packages, base, etc.  Return whether the function has any methods left after deletion. """ function delete_toplevel_methods(f::Function, cell_id::UUID)::Bool # we can delete methods of functions! # instead of deleting all methods, we only delete methods that were defined in this notebook. This is necessary when the notebook code extends a function from remote code methods_table = typeof(f).name.mt deleted_sigs = Set{Type}() Base.visit(methods_table) do method # iterates through all methods of `f`, including overridden ones if isfromcell(method, cell_id) && method.deleted_world == alive_world_val Base.delete_method(method) delete_method_doc(method) push!(deleted_sigs, method.sig) end end if VERSION < v"1.12.0-0" # not necessary in Julia after https://github.com/JuliaLang/julia/pull/53415  # if `f` is an extension to an external function, and we defined a method that overrides a method, for example, # we define `Base.isodd(n::Integer) = rand(Bool)`, which overrides the existing method `Base.isodd(n::Integer)` # calling `Base.delete_method` on this method won't bring back the old method, because our new method still exists in the method table, and it has a world age which is newer than the original. (our method has a deleted_world value set, which disables it) # # To solve this, we iterate again, and _re-enable any methods that were hidden in this way_, by adding them again to the method table with an even newer `primary_world`. if !isempty(deleted_sigs) to_insert = Method[] Base.visit(methods_table) do method if !isfromcell(method, cell_id) && method.sig  deleted_sigs push!(to_insert, method) end end # separate loop to avoid visiting the recently added method for method in Iterators.reverse(to_insert) if VERSION >= v"1.11.0-0" @atomic method.primary_world = one(typeof(alive_world_val)) # `1` will tell Julia to increment the world counter and set it as this function's world @atomic method.deleted_world = alive_world_val # set the `deleted_world` property back to the 'alive' value (for Julia v1.6 and up) else method.primary_world = one(typeof(alive_world_val)) method.deleted_world = alive_world_val end ccall(:jl_method_table_insert, Cvoid, (Any, Any, Ptr{Cvoid}), methods_table, method, C_NULL) # i dont like doing this either! end end end return !isempty(methods(f).ms) end # function try_delete_toplevel_methods(workspace::Module, name::Symbol) # try_delete_toplevel_methods(workspace, [name]) # end function try_delete_toplevel_methods(workspace::Module, (cell_id, name_parts)::Tuple{UUID,Tuple{Vararg{Symbol}}})::Bool try val = workspace for name in name_parts val = getfield(val, name) end try (val isa Function) && delete_toplevel_methods(val, cell_id) catch ex @warn "Failed to delete methods for $(name_parts)" showerror(original_stderr[], ex, stacktrace(catch_backtrace())) false end catch false end end const alive_world_val = methods(Base.sqrt).ms[1].deleted_world # typemax(UInt) in Julia v1.3, Int(-1) in Julia 1.0 function revise_if_possible(m::Module) # Revise.jl support if isdefined(m, :Revise) && isdefined(m.Revise, :revise) && m.Revise.revise isa Function && isdefined(m.Revise, :revision_queue) && m.Revise.revision_queue isa AbstractSet if !isempty(m.Revise.revision_queue) # to avoid the sleep(0.01) in revise() m.Revise.revise() end end end function wrap_dot(ref::GlobalRef) complete_mod_name = fullname(ref.mod) |> wrap_dot Expr(:(.), complete_mod_name, QuoteNode(ref.name)) end function wrap_dot(name) if length(name) == 1 name[1] else Expr(:(.), wrap_dot(name[1:end-1]), QuoteNode(name[end])) end end """ collect_and_eliminate_globalrefs!(ref::Union{GlobalRef,Expr}, mutable_ref_list::Vector{Pair{Symbol,Symbol}}=[]) Goes through an expression and removes all "global" references to workspace modules (e.g. Main.workspace#XXX). It collects the names that we replaced these references with, so that we can add assignments to these special names later. This is useful for us because when we macroexpand, the global refs will normally point to the module it was built in. We don't re-build the macro in every workspace, so we need to remove these refs manually in order to point to the new module instead. TODO? Don't remove the refs, but instead replace them with a new ref pointing to the new module? """ function collect_and_eliminate_globalrefs!(ref::GlobalRef, mutable_ref_list=[]) if is_pluto_workspace(ref.mod) new_name = gensym(ref.name) push!(mutable_ref_list, ref.name => new_name) new_name else ref end end function collect_and_eliminate_globalrefs!(expr::Expr, mutable_ref_list=[]) # Fix for .+ and .|> inside macros # https://github.com/fonsp/Pluto.jl/pull/1032#issuecomment-868819317 # I'm unsure if this was all necessary but  # I take the :call with a GlobalRef to `.|>` or `.+` as args[1], # and then I convert it into a `:.` expr, which is basically (|>).(args...) # which is consistent for us to handle. if expr.head == :call && expr.args[1] isa GlobalRef && startswith(string(expr.args[1].name), ".") old_globalref = expr.args[1] non_broadcast_name = string(old_globalref.name)[begin+1:end] new_globalref = GlobalRef(old_globalref.mod, Symbol(non_broadcast_name)) new_expr = Expr(:., new_globalref, Expr(:tuple, expr.args[begin+1:end]...)) result = collect_and_eliminate_globalrefs!(new_expr, mutable_ref_list) return result else Expr(expr.head, map(arg -> collect_and_eliminate_globalrefs!(arg, mutable_ref_list), expr.args)...) end end collect_and_eliminate_globalrefs!(other, mutable_ref_list=[]) = other function globalref_to_workspaceref(expr) mutable_ref_list = Pair{Symbol, Symbol}[] new_expr = collect_and_eliminate_globalrefs!(expr, mutable_ref_list) Expr(:block, # Create new lines to assign to the replaced names of the global refs. # This way the expression explorer doesn't care (it just sees references to variables outside of the workspace), # and the variables don't get overwriten by local assigments to the same name (because we have special names). (mutable_ref_list .|> ref -> :(local $(ref[2])))..., map(mutable_ref_list) do ref # I can just do Expr(:isdefined, ref[1]) here, but it feels better to macroexpand, # because it's more obvious what's going on, and when they ever change the ast, we're safe :D macroexpand(Main, quote if @isdefined($(ref[1])) $(ref[2]) = $(ref[1]) end end) end..., new_expr, ) end replace_pluto_properties_in_expr(::GiveMeCellID; cell_id, kwargs...) = cell_id replace_pluto_properties_in_expr(::GiveMeRerunCellFunction; rerun_cell_function, kwargs...) = rerun_cell_function replace_pluto_properties_in_expr(::GiveMeRegisterCleanupFunction; register_cleanup_function, kwargs...) = register_cleanup_function replace_pluto_properties_in_expr(expr::Expr; kwargs...) = Expr(expr.head, map(arg -> replace_pluto_properties_in_expr(arg; kwargs...), expr.args)...) replace_pluto_properties_in_expr(m::Module; kwargs...) = if is_pluto_workspace(m) PLUTO_INNER_MODULE_NAME else m end replace_pluto_properties_in_expr(other; kwargs...) = other function replace_pluto_properties_in_expr(ln::LineNumberNode; cell_id, kwargs...) # See https://github.com/fonsp/Pluto.jl/pull/2241 file = string(ln.file) out = if endswith(file, string(cell_id)) # We already have the correct cell_id in this LineNumberNode ln else # We append to the LineNumberNode file #@#==# + cell_id LineNumberNode(ln.line, Symbol(file * "#@#==#$(cell_id)")) end return out end "Similar to [`replace_pluto_properties_in_expr`](@ref), but just checks for existance and doesn't check for [`GiveMeCellID`](@ref)" has_hook_style_pluto_properties_in_expr(::GiveMeRerunCellFunction) = true has_hook_style_pluto_properties_in_expr(::GiveMeRegisterCleanupFunction) = true has_hook_style_pluto_properties_in_expr(expr::Expr)::Bool = any(has_hook_style_pluto_properties_in_expr, expr.args) has_hook_style_pluto_properties_in_expr(other) = false function sanitize_expr(ref::GlobalRef) wrap_dot(ref) end function sanitize_expr(expr::Expr) Expr(expr.head, sanitize_expr.(expr.args)...) end sanitize_expr(linenumbernode::LineNumberNode) = linenumbernode sanitize_expr(quoted::QuoteNode) = QuoteNode(sanitize_expr(quoted.value)) sanitize_expr(bool::Bool) = bool sanitize_expr(symbol::Symbol) = symbol sanitize_expr(number::Union{Int,Int8,Float32,Float64}) = number # In all cases of more complex objects, we just don't send it. # It's not like the expression explorer will look into them at all. sanitize_expr(other) = nothing function try_macroexpand(mod::Module, notebook_id::UUID, cell_id::UUID, expr; capture_stdout::Bool=true) # Remove the precvious cached expansion, so when we error somewhere before we update, # the old one won't linger around and get run accidentally. pop!(cell_expanded_exprs, cell_id, nothing) # Remove toplevel block, as that screws with the computer and everything expr_not_toplevel = if Meta.isexpr(expr, (:toplevel, :block)) Expr(:block, expr.args...) else @warn "try_macroexpand expression not :toplevel or :block" expr Expr(:block, expr) end capture_logger = CaptureLogger(nothing, get_cell_logger(notebook_id, cell_id), Dict[]) expanded_expr, elapsed_ns = with_logger_and_io_to_logs(capture_logger; capture_stdout) do elapsed_ns = time_ns() expanded_expr = macroexpand(mod, expr_not_toplevel)::Expr elapsed_ns = time_ns() - elapsed_ns expanded_expr, elapsed_ns end logs = capture_logger.logs # Removes baked in references to the module this was macroexpanded in. # Fix for https://github.com/fonsp/Pluto.jl/issues/1112 expr_without_return = CantReturnInPluto.replace_returns_with_error(expanded_expr)::Expr expr_without_globalrefs = globalref_to_workspaceref(expr_without_return) has_pluto_hook_features = has_hook_style_pluto_properties_in_expr(expr_without_globalrefs) expr_to_save = replace_pluto_properties_in_expr(expr_without_globalrefs; cell_id, rerun_cell_function=() -> rerun_cell_from_notebook(cell_id), register_cleanup_function=(fn) -> UseEffectCleanups.register_cleanup(fn, cell_id), ) did_mention_expansion_time = false cell_expanded_exprs[cell_id] = CachedMacroExpansion( expr_hash(expr), expr_to_save, elapsed_ns, has_pluto_hook_features, did_mention_expansion_time, logs, ) return (sanitize_expr(expr_to_save), expr_hash(expr_to_save)) end """ All code necessary for throwing errors when cells return. Right now it just throws an error from the position of the return, this is nice because you get to the line number of the return. However, now it is suddenly possibly to catch the return error... so we might want to actually return the error instead of throwing it, and then handle it in `run_expression` or something. """ module CantReturnInPluto struct CantReturnInPlutoException end function Base.showerror(io::IO, ::CantReturnInPlutoException) print(io, "Pluto: You can only use return inside a function.") end """ We do macro expansion now, so we can also check for `return` statements "statically". This method goes through an expression and replaces all `return` statements with `throw(CantReturnInPlutoException())` """ function replace_returns_with_error(expr::Expr)::Expr if expr.head == :return :(throw($(CantReturnInPlutoException()))) elseif expr.head == :quote Expr(:quote, replace_returns_with_error_in_interpolation(expr.args[1])) elseif Meta.isexpr(expr, :(=)) && expr.args[1] isa Expr && (expr.args[1].head == :call || expr.args[1].head == :where || (expr.args[1].head == :(::) && expr.args[1].args[1] isa Expr && expr.args[1].args[1].head == :call)) # f(x) = ... expr elseif expr.head == :function || expr.head == :macro || expr.head == :(->) expr else Expr(expr.head, map(arg -> replace_returns_with_error(arg), expr.args)...) end end replace_returns_with_error(other) = other "Go through a quoted expression and remove returns" function replace_returns_with_error_in_interpolation(expr::Expr) if expr.head == :$ Expr(:$, replace_returns_with_error_in_interpolation(expr.args[1])) else # We are still in a quote, so we do go deeper, but we keep ignoring everything except :$'s Expr(expr.head, map(arg -> replace_returns_with_error_in_interpolation(arg), expr.args)...) end end replace_returns_with_error_in_interpolation(ex) = ex end import Logging struct Computer f::Function expr_id::ObjectID input_globals::Vector{Symbol} output_globals::Vector{Symbol} end expr_hash(e::Expr) = objectid(e.head) + mapreduce(p -> objectid((p[1], expr_hash(p[2]))), +, enumerate(e.args); init=zero(ObjectID)) expr_hash(x) = objectid(x) const computers = Dict{UUID,Computer}() const computer_workspace = Main const cells_with_hook_functionality_active = Set{UUID}() "Registers a new computer for the cell, cleaning up the old one if there is one." function register_computer(expr::Expr, key::ObjectID, cell_id::UUID, input_globals::Vector{Symbol}, output_globals::Vector{Symbol}) @gensym result e = Expr(:function, Expr(:call, gensym(:function_wrapped_cell), input_globals...), Expr(:block, Expr(:(=), result, timed_expr(expr)), Expr(:tuple, result, Expr(:tuple, map(x -> :(@isdefined($(x)) ? $(x) : $(OutputNotDefined())), output_globals)...) ) )) f = Core.eval(computer_workspace, e) if haskey(computers, cell_id) delete_computer!(computers, cell_id) end computers[cell_id] = Computer(f, key, input_globals, output_globals) end function delete_computer!(computers::Dict{UUID,Computer}, cell_id::UUID) computer = pop!(computers, cell_id) UseEffectCleanups.trigger_cleanup(cell_id) Base.visit(Base.delete_method, methods(computer.f).mt) # Make the computer function uncallable end parse_cell_id(filename::Symbol) = filename |> string |> parse_cell_id parse_cell_id(filename::AbstractString) = match(r"#==#(.*)", filename).captures |> only |> UUID module UseEffectCleanups import UUIDs: UUID const cell_cleanup_functions = Dict{UUID,Set{Function}}() function register_cleanup(f::Function, cell_id::UUID) cleanup_functions = get!(cell_cleanup_functions, cell_id, Set{Function}()) push!(cleanup_functions, f) nothing end function trigger_cleanup(cell_id::UUID) for cleanup_func in get!(cell_cleanup_functions, cell_id, Set{Function}()) try cleanup_func() catch error @warn "Cleanup function gave an error" cell_id error stacktrace=stacktrace(catch_backtrace()) end end delete!(cell_cleanup_functions, cell_id) end end quote_if_needed(x) = x quote_if_needed(x::Union{Expr, Symbol, QuoteNode, LineNumberNode}) = QuoteNode(x) struct OutputNotDefined end function compute(m::Module, computer::Computer) # 1. get the referenced global variables # this might error if the global does not exist, which is exactly what we want input_global_values = Vector{Any}(undef, length(computer.input_globals)) for (i, s) in enumerate(computer.input_globals) input_global_values[i] = getfield(m, s) end # 2. run the function out = Base.invokelatest(computer.f, input_global_values...) result, output_global_values = out for (name, val) in zip(computer.output_globals, output_global_values) # Core.eval(m, Expr(:(=), name, quote_if_needed(val))) Core.eval(m, quote if $(quote_if_needed(val)) !== $(OutputNotDefined()) $(name) = $(quote_if_needed(val)) end end) end result end "Wrap `expr` inside a timing block." function timed_expr(expr::Expr)::Expr # @assert ExpressionExplorer.is_toplevel_expr(expr) @gensym result @gensym elapsed_ns # we don't use `quote ... end` here to avoid the LineNumberNodes that it adds (these would taint the stack trace). Expr(:block, :(local $elapsed_ns = time_ns()), :(local $result = $expr), :($elapsed_ns = time_ns() - $elapsed_ns), :(($result, $elapsed_ns)), ) end """ Run the expression or function inside a try ... catch block, and verify its "return proof". """ function run_inside_trycatch(m::Module, f::Union{Expr,Function})::Tuple{Any,Union{UInt64,Nothing}} return try if f isa Expr # We eval `f` in the global scope of the workspace module: Core.eval(m, f) else # f is a function f() end catch ex bt = stacktrace(catch_backtrace()) (CapturedException(ex, bt), nothing) end end add_runtimes(::Nothing, ::UInt64) = nothing add_runtimes(a::UInt64, b::UInt64) = a+b contains_macrocall(expr::Expr) = if expr.head == :macrocall if VERSION >= v"1.12.0-aaa" && length(expr.args) >= 1 && expr.args[1] == :(Base.var"@__doc__") # special case to support https://github.com/JuliaLang/julia/pull/53515 false else true end elseif expr.head == :module # Modules don't get expanded, sadly, so we don't touch it false else any(arg -> contains_macrocall(arg), expr.args) end contains_macrocall(other) = false """ Run the given expression in the current workspace module. If the third argument is `nothing`, then the expression will be `Core.eval`ed. The result and runtime are stored inside [`cell_results`](@ref) and [`cell_runtimes`](@ref). If the third argument is a `Tuple{Set{Symbol}, Set{Symbol}}` containing the referenced and assigned variables of the expression (computed by the ExpressionExplorer), then the expression will be **wrapped inside a function**, with the references as inputs, and the assignments as outputs. Instead of running the expression directly, Pluto will call this function, with the right globals as inputs. This function is memoized: running the same expression a second time will simply call the same generated function again. This is much faster than evaluating the expression, because the function only needs to be Julia-compiled once. See https://github.com/fonsp/Pluto.jl/pull/720 """ function run_expression( m::Module, expr::Any, notebook_id::UUID, cell_id::UUID, @nospecialize(function_wrapped_info::Union{Nothing,Tuple{Set{Symbol},Set{Symbol}}}=nothing), @nospecialize(forced_expr_id::Union{ObjectID,Nothing}=nothing); user_requested_run::Bool=true, capture_stdout::Bool=true, ) if user_requested_run # TODO Time elapsed? Possibly relays errors in cleanup function? UseEffectCleanups.trigger_cleanup(cell_id) # TODO Could also put explicit `try_macroexpand` here, to make clear that user_requested_run => fresh macro identity end old_currently_running_cell_id = currently_running_cell_id[] currently_running_cell_id[] = cell_id logger = get_cell_logger(notebook_id, cell_id) # reset published objects cell_published_objects[cell_id] = Dict{String,Any}() # reset registered bonds cell_registered_bond_names[cell_id] = Set{Symbol}() # reset JS links unregister_js_link(cell_id) # If the cell contains macro calls, we want those macro calls to preserve their identity, # so we macroexpand this earlier (during expression explorer stuff), and then we find it here. # NOTE Turns out sometimes there is no macroexpanded version even though the expression contains macro calls... # .... So I macroexpand when there is no cached version just to be sure  # NOTE Errors during try_macroexpand will cause no expanded version to be stored. # .... This is fine, because it allows us to try again here and throw the error... # .... But ideally we wouldn't re-macroexpand and store the error the first time (TODO-ish) if !haskey(cell_expanded_exprs, cell_id) || cell_expanded_exprs[cell_id].original_expr_hash != expr_hash(expr) try try_macroexpand(m, notebook_id, cell_id, expr; capture_stdout) catch e result = CapturedException(e, stacktrace(catch_backtrace())) cell_results[cell_id], cell_runtimes[cell_id] = (result, nothing) return (result, nothing) end end # We can be sure there is a cached expression now, yay expanded_cache = cell_expanded_exprs[cell_id] original_expr = expr expr = expanded_cache.expanded_expr # Re-play logs from expansion cache for log in expanded_cache.expansion_logs (level, msg, _module, group, id, file, line, kwargs) = log Logging.handle_message(logger, level, msg, _module, group, id, file, line; kwargs...) end empty!(expanded_cache.expansion_logs) # We add the time it took to macroexpand to the time for the first call, # but we make sure we don't mention it on subsequent calls expansion_runtime = if expanded_cache.did_mention_expansion_time === false did_mention_expansion_time = true cell_expanded_exprs[cell_id] = CachedMacroExpansion( expanded_cache.original_expr_hash, expanded_cache.expanded_expr, expanded_cache.expansion_duration, expanded_cache.has_pluto_hook_features, did_mention_expansion_time, expanded_cache.expansion_logs, ) expanded_cache.expansion_duration else zero(UInt64) end if contains_macrocall(expr) @error "Expression contains a macrocall" expr throw("Expression still contains macro calls!!") end result, runtime = with_logger_and_io_to_logs(logger; capture_stdout) do # about 200ns + 3ms overhead if function_wrapped_info === nothing toplevel_expr = Expr(:toplevel, expr) wrapped = timed_expr(toplevel_expr) ans, runtime = run_inside_trycatch(m, wrapped) (ans, add_runtimes(runtime, expansion_runtime)) else expr_id = forced_expr_id !== nothing ? forced_expr_id : expr_hash(expr) local computer = get(computers, cell_id, nothing) if computer === nothing || computer.expr_id !== expr_id try computer = register_computer(expr, expr_id, cell_id, collect.(function_wrapped_info)...) catch e # @error "Failed to generate computer function" expr exception=(e,stacktrace(catch_backtrace())) return run_expression(m, original_expr, notebook_id, cell_id, nothing; user_requested_run=user_requested_run) end end # This check solves the problem of a cell like `false && variable_that_does_not_exist`. This should run without error, but will fail in our function-wrapping-magic because we get the value of `variable_that_does_not_exist` before calling the generated function. # The fix is to detect this situation and run the expression in the classical way. ans, runtime = if any(name -> !isdefined(m, name), computer.input_globals) # Do run_expression but with function_wrapped_info=nothing so it doesn't go in a Computer() # @warn "Got variables that don't exist, running outside of computer" not_existing=filter(name -> !isdefined(m, name), computer.input_globals) run_expression(m, original_expr, notebook_id, cell_id; user_requested_run) else run_inside_trycatch(m, () -> compute(m, computer)) end ans, add_runtimes(runtime, expansion_runtime) end end currently_running_cell_id[] = old_currently_running_cell_id if (result isa CapturedException) && (result.ex isa InterruptException) throw(result.ex) end cell_results[cell_id], cell_runtimes[cell_id] = result, runtime end precompile(run_expression, (Module, Expr, UUID, UUID, Nothing, Nothing)) # Channel to trigger implicits run const run_channel = Channel{UUID}(10) function rerun_cell_from_notebook(cell_id::UUID) # make sure only one of this cell_id is in the run channel # by emptying it and filling it again new_uuids = UUID[] while isready(run_channel) uuid = take!(run_channel) if uuid != cell_id push!(new_uuids, uuid) end end for uuid in new_uuids put!(run_channel, uuid) end put!(run_channel, cell_id) end "These expressions get evaluated inside every newly create module inside a `Workspace`." const workspace_preamble = [ :(using Main.PlutoRunner, Main.PlutoRunner.Markdown, Main.PlutoRunner.InteractiveUtils), :(show, showable, showerror, repr, string, print, println), # https://github.com/JuliaLang/julia/issues/18181 ] const PLUTO_INNER_MODULE_NAME = Symbol("#___this_pluto_module_name") const moduleworkspace_count = Ref(0) function increment_current_module()::Symbol id = (moduleworkspace_count[] += 1) new_workspace_name = Symbol("workspace#", id) Core.eval(Main, :( module $(new_workspace_name) $(workspace_preamble...) const $(PLUTO_INNER_MODULE_NAME) = $(new_workspace_name) end )) new_workspace_name end import REPL: REPL, REPLCompletions import REPL.REPLCompletions: Completion, BslashCompletion, ModuleCompletion, PropertyCompletion, FieldCompletion, PathCompletion, DictCompletion function basic_completion_priority((s, description, exported, from_notebook)) c = first(s) if islowercase(c) 1 - 10exported elseif isuppercase(c) 2 - 10exported else 3 - 10exported end end completion_value_type_inner(x::Function) = :Function completion_value_type_inner(x::Number) = :Number completion_value_type_inner(x::AbstractString) = :String completion_value_type_inner(x::Module) = :Module completion_value_type_inner(x::AbstractArray) = :Array completion_value_type_inner(x::Any) = :Any completion_value_type(c::ModuleCompletion) = try completion_value_type_inner(getfield(c.parent, Symbol(c.mod)))::Symbol catch :unknown end completion_value_type(::Completion) = :unknown completion_special_symbol_value(::Completion) = nothing function completion_special_symbol_value(completion::BslashCompletion) s = @static hasfield(BslashCompletion, :bslash) ? completion.bslash : completion.completion symbol_dict = startswith(s, "\\:") ? REPLCompletions.emoji_symbols : REPLCompletions.latex_symbols get(symbol_dict, s, nothing) end function is_pluto_workspace(m::Module) isdefined(m, PLUTO_INNER_MODULE_NAME) && which(m, PLUTO_INNER_MODULE_NAME) == m end """ Returns wether the module is a pluto workspace or any of its ancestors is. For example, writing the following julia code in Pluto: ```julia import Plots module A end ``` will give the following module tree: ``` Main (not pluto controlled)  var"workspace#1" (pluto controlled)  A (pluto controlled)  var"workspace#2" (pluto controlled)  A (pluto controlled) Plots (not pluto controlled) ``` """ function is_pluto_controlled(m::Module) is_pluto_workspace(m) && return true parent = parentmodule(m) parent != m && is_pluto_controlled(parent) end function completions_exported(cs::Vector{<:Completion}) map(cs) do c if c isa ModuleCompletion sym = Symbol(c.mod) @static if isdefined(Base, :ispublic) Base.ispublic(c.parent, sym) else Base.isexported(c.parent, sym) end else true end end end completion_from_notebook(c::ModuleCompletion) = is_pluto_workspace(c.parent) && c.mod != "include" && c.mod != "eval" && !startswith(c.mod, "#") completion_from_notebook(c::Completion) = false completion_type(::REPLCompletions.PathCompletion) = :path completion_type(::REPLCompletions.DictCompletion) = :dict completion_type(::REPLCompletions.MethodCompletion) = :method completion_type(::REPLCompletions.ModuleCompletion) = :module completion_type(::REPLCompletions.BslashCompletion) = :bslash completion_type(::REPLCompletions.FieldCompletion) = :field completion_type(::REPLCompletions.KeywordArgumentCompletion) = :keyword_argument completion_type(::REPLCompletions.KeywordCompletion) = :keyword completion_type(::REPLCompletions.PropertyCompletion) = :property completion_type(::REPLCompletions.Text) = :text completion_type(::Completion) = :unknown function completion_contents(c::Completion) @static if isdefined(REPLCompletions, :named_completion) REPLCompletions.named_completion(c).completion else REPLCompletions.completion_text(c) end end is_method_completions_results(results) = length(results) >= 1 && results[1] isa REPLCompletions.MethodCompletion "You say Linear, I say Algebra!" function completion_fetcher(query::String, query_full::String, workspace::Module) results, loc, found = REPLCompletions.completions( query, lastindex(query), workspace ) ## METHODS & KWARGS if query != query_full && is_method_completions_results(results) # We are doing autocomplete inside a method call. # But because query != query_full, we know that we are actually typing something extra # Try if the full query gives a different result. results, loc, found = REPLCompletions.completions( query_full, lastindex(query_full), workspace ) # If they give a keywordargument completion, then we should use that. if !any(r -> r isa REPLCompletions.KeywordArgumentCompletion, results) # Otherwise, we are just completing a function argument. So let's just complete the empty string. results, loc, found = REPLCompletions.completions("", 0, workspace) # loc is wrong, because the empty string has a different offset. loc = (ncodeunits(query)+1):ncodeunits(query_full) end end ## TOO MANY RESULTS if length(results) > 2000 && query != query_full results, loc, found = REPLCompletions.completions( query_full, lastindex(query_full), workspace ) end if (too_long = length(results) > 2000) results = results[1:2000] end ## SPECIAL ENDINGS if endswith(query, '.') filter!(is_dot_completion, results) # we are autocompleting a module, and we want to see its fields alphabetically sort!(results; by=completion_contents) elseif endswith(query, '/') filter!(is_path_completion, results) sort!(results; by=completion_contents) elseif endswith(query, '[') filter!(is_dict_completion, results) sort!(results; by=completion_contents) else contains_slash = '/'  query if !contains_slash filter!(!is_path_completion, results) end end # Add this if you are seeing keyword completions twice in results # filter!(!is_keyword_completion, results) exported = completions_exported(results) smooshed_together = map(zip(results, exported)) do (result, rexported) ( completion_contents(result)::String, completion_value_type(result)::Symbol, rexported::Bool, completion_from_notebook(result)::Bool, completion_type(result)::Symbol, completion_special_symbol_value(result), ) end sort!(smooshed_together; alg=MergeSort, by=basic_completion_priority) (smooshed_together, loc, found, too_long) end is_dot_completion(::Union{ModuleCompletion,PropertyCompletion,FieldCompletion}) = true is_dot_completion(::Completion) = false is_path_completion(::PathCompletion) = true is_path_completion(::Completion) = false is_dict_completion(::DictCompletion) = true is_dict_completion(::Completion) = false is_kwarg_completion(::REPLCompletions.KeywordArgumentCompletion) = true is_kwarg_completion(::Completion) = false is_keyword_completion(::REPLCompletions.KeywordCompletion) = true is_keyword_completion(::Completion) = false import REPL """ is_pure_expression(expression::ReturnValue{Meta.parse}) Checks if an expression is approximately pure. Not sure if the type signature conveys it, but this take anything that is returned from `Meta.parse`. It obviously does not actually check if something is strictly pure, as `getproperty()` could be extended, and suddenly there can be side effects everywhere. This is just an approximation. """ function is_pure_expression(expr::Expr) if expr.head == :. || expr.head === :curly || expr.head === :ref all((is_pure_expression(x) for x in expr.args)) else false end end is_pure_expression(s::Symbol) = true is_pure_expression(q::QuoteNode) = true is_pure_expression(q::Number) = true is_pure_expression(q::String) = true is_pure_expression(x) = false # Better safe than sorry I guess # Based on /base/docs/bindings.jl from Julia source code function binding_from(x::Expr, workspace::Module) if x.head == :macrocall macro_name = x.args[1] if is_pure_expression(macro_name) Core.eval(workspace, macro_name) else error("Couldn't infer `$x` for Live Docs.") end elseif is_pure_expression(x) if x.head == :. # Simply calling Core.eval on `a.b` will retrieve the value instead of the binding m = Core.eval(workspace, x.args[1]) isa(m, Module) && return Docs.Binding(m, x.args[2].value) end Core.eval(workspace, x) else error("Couldn't infer `$x` for Live Docs.") end end binding_from(s::Symbol, workspace::Module) = Docs.Binding(workspace, s) binding_from(r::GlobalRef, workspace::Module) = Docs.Binding(r.mod, r.name) binding_from(other, workspace::Module) = error("Invalid @var syntax `$other`.") const DOC_SUGGESTION_LIMIT = 10 struct Suggestion match::String query::String end # inspired from REPL.printmatch() function Base.show(io::IO, ::MIME"text/html", suggestion::Suggestion) print(io, "<a href=\"@ref\"><code>") is, _ = REPL.bestmatch(suggestion.query, suggestion.match) for (i, char) in enumerate(suggestion.match) esc_c = get(Markdown._htmlescape_chars, char, char) if i in is print(io, "<b>", esc_c, "</b>") else print(io, esc_c) end end print(io, "</code></a>") end "You say doc_fetcher, I say You say doc_fetcher, I say You say doc_fetcher, I say You say doc_fetcher, I say ...!!!!" function doc_fetcher(query, workspace::Module) try parsed_query = Meta.parse(query; raise=false, depwarn=false) doc_md = if Meta.isexpr(parsed_query, (:incomplete, :error, :return)) && haskey(Docs.keywords, Symbol(query)) Docs.parsedoc(Docs.keywords[Symbol(query)]) else binding = binding_from(parsed_query, workspace) doc_md = Docs.doc(binding) if !showable(MIME("text/html"), doc_md) # PyPlot returns `Text{String}` objects from their docs... # which is a bit silly, but turns out it actually is markdown if you look hard enough. doc_md = Markdown.parse(repr(doc_md)) end improve_docs!(doc_md, parsed_query, binding) end # TODO: # completion_value_type_inner # typeof(x) |> string # parentmodule(x) |> string (repr(MIME("text/html"), doc_md), :) catch ex (nothing, :) end end function improve_docs!(doc_md::Markdown.MD, query::Symbol, binding::Docs.Binding) # Reverse latex search ("\scrH" -> "\srcH<tab>") symbol = string(query) latex = REPL.symbol_latex(symbol) if !isempty(latex) push!(doc_md.content, Markdown.HorizontalRule(), Markdown.Paragraph([ Markdown.Code(symbol), " can be typed by ", Markdown.Code(latex), Base.Docs.HTML("<kbd>&lt;tab&gt;</kbd>"), ".", ])) end # Add function signature if it's not there already # Add suggestions results if no docstring was found if !Docs.defined(binding) && haskey(doc_md.meta, :results) && isempty(doc_md.meta[:results]) suggestions = REPL.accessible(binding.mod) suggestions_scores = map(s -> REPL.fuzzyscore(symbol, s), suggestions) removed_indices = [i for (i, s) in enumerate(suggestions_scores) if s < 0] deleteat!(suggestions_scores, removed_indices) deleteat!(suggestions, removed_indices) perm = sortperm(suggestions_scores; rev=true) permute!(suggestions, perm) links = map(s -> Suggestion(string(s), symbol), Iterators.take(suggestions, DOC_SUGGESTION_LIMIT)) if length(links) > 0 push!(doc_md.content, Markdown.HorizontalRule(), Markdown.Paragraph(["Similar result$(length(links) > 1 ? "s" : ""):"]), Markdown.List(links)) end end doc_md end improve_docs!(other, _, _) = other import Logging const original_stderr = Ref{IO}() const old_logger = Ref{Union{Logging.AbstractLogger,Nothing}}(nothing) struct PlutoCellLogger <: Logging.AbstractLogger stream # some packages expect this field to exist... log_channel::Channel{Any} cell_id::UUID workspace_count::Int # Used to invalidate previous logs message_limits::Dict{Any,Int} end function PlutoCellLogger(notebook_id, cell_id) notebook_log_channel = pluto_log_channels[notebook_id] PlutoCellLogger(nothing, notebook_log_channel, cell_id, moduleworkspace_count[], Dict{Any,Int}()) end struct CaptureLogger <: Logging.AbstractLogger stream logger::PlutoCellLogger logs::Vector{Any} end Logging.shouldlog(cl::CaptureLogger, args...) = Logging.shouldlog(cl.logger, args...) Logging.min_enabled_level(cl::CaptureLogger) = Logging.min_enabled_level(cl.logger) Logging.catch_exceptions(cl::CaptureLogger) = Logging.catch_exceptions(cl.logger) function Logging.handle_message(cl::CaptureLogger, level, msg, _module, group, id, file, line; kwargs...) push!(cl.logs, (level, msg, _module, group, id, file, line, kwargs)) end const pluto_cell_loggers = Dict{UUID,PlutoCellLogger}() # One logger per cell const pluto_log_channels = Dict{UUID,Channel{Any}}() # One channel per notebook function get_cell_logger(notebook_id, cell_id) logger = get!(() -> PlutoCellLogger(notebook_id, cell_id), pluto_cell_loggers, cell_id) if logger.workspace_count < moduleworkspace_count[] logger = pluto_cell_loggers[cell_id] = PlutoCellLogger(notebook_id, cell_id) end logger end function Logging.shouldlog(logger::PlutoCellLogger, level, _module, _...) # Accept logs # - Only if the logger is the latest for this cell using the increasing workspace_count tied to each logger # - From the user's workspace module # - Info level and above for other modules # - LogLevel(-1) because that's what ProgressLogging.jl uses for its messages current_logger = pluto_cell_loggers[logger.cell_id] if current_logger.workspace_count > logger.workspace_count return false end level = convert(Logging.LogLevel, level) (_module isa Module && is_pluto_workspace(_module)) || level >= Logging.Info || level == progress_log_level || level == stdout_log_level end const BuiltinInts = @static isdefined(Core, :BuiltinInts) ? Core.BuiltinInts : Union{Bool, Int32, Int64, UInt32, UInt64, UInt8, Int128, Int16, Int8, UInt128, UInt16} Logging.min_enabled_level(::PlutoCellLogger) = min(Logging.Debug, stdout_log_level) Logging.catch_exceptions(::PlutoCellLogger) = false function Logging.handle_message(pl::PlutoCellLogger, level, msg, _module, group, id, file, line; kwargs...) # println("receiving msg from ", _module, " ", group, " ", id, " ", msg, " ", level, " ", line, " ", file) # println("with types: ", "_module: ", typeof(_module), ", ", "msg: ", typeof(msg), ", ", "group: ", typeof(group), ", ", "id: ", typeof(id), ", ", "file: ", typeof(file), ", ", "line: ", typeof(line), ", ", "kwargs: ", typeof(kwargs)) # thanks Copilot # https://github.com/JuliaLang/julia/blob/eb2e9687d0ac694d0aa25434b30396ee2cfa5cd3/stdlib/Logging/src/ConsoleLogger.jl#L110-L115 if get(kwargs, :maxlog, nothing) isa BuiltinInts maxlog = kwargs[:maxlog] remaining = get!(pl.message_limits, id, Int(maxlog)::Int) pl.message_limits[id] = remaining - 1 if remaining <= 0 return end end try yield() po() = get(cell_published_objects, pl.cell_id, Dict{String,Any}()) before_published_object_keys = collect(keys(po())) # Render the log arguments: msg_formatted = format_output_default(msg isa AbstractString ? Text(msg) : msg) kwargs_formatted = Tuple{String,Any}[(string(k), format_log_value(v)) for (k, v) in kwargs if k != :maxlog] after_published_object_keys = collect(keys(po())) new_published_object_keys = setdiff(after_published_object_keys, before_published_object_keys) # (Running `put!(pl.log_channel, x)` will send `x` to the pluto server. See `start_relaying_logs` for the receiving end.) put!(pl.log_channel, Dict{String,Any}( "level" => string(level), "msg" => msg_formatted, # This is a dictionary containing all published objects that were published during the rendering of the log arguments (we cannot track which objects were published during the execution of the log statement itself i think...) "new_published_objects" => Dict{String,Any}( key => po()[key] for key in new_published_object_keys ), "group" => string(group), "id" => string(id), "file" => string(file), "cell_id" => pl.cell_id, "line" => line isa Union{Int32,Int64} ? line : nothing, "kwargs" => kwargs_formatted, )) yield() catch e Logging.with_logger(Logging.ConsoleLogger(original_stderr[])) do @error "Failed to relay log from PlutoRunner" exception=(e, catch_backtrace()) end nothing end end format_log_value(v) = format_output_default(v) format_log_value(v::Tuple{<:Exception,Vector{<:Any}}) = format_output(CapturedException(v...)) function with_logger_and_io_to_logs(f, logger; capture_stdout=true, stdio_loglevel=stdout_log_level) Logging.with_logger(logger) do with_io_to_logs(f; enabled=capture_stdout, loglevel=stdio_loglevel) end end function setup_plutologger(notebook_id::UUID, log_channel::Channel{Any}) pluto_log_channels[notebook_id] = log_channel end function _send_stdio_output!(output, loglevel) output_str = String(take!(output)) if !isempty(output_str) Logging.@logmsg loglevel output_str end end const stdout_log_level = Logging.LogLevel(-555) # https://en.wikipedia.org/wiki/555_timer_IC const progress_log_level = Logging.LogLevel(-1) # https://github.com/JuliaLogging/ProgressLogging.jl/blob/0e7933005233722d6214b0debe3316c82b4d14a7/src/ProgressLogging.jl#L36 function with_io_to_logs(f::Function; enabled::Bool=true, loglevel::Logging.LogLevel=Logging.LogLevel(1)) if !enabled return f() end # Taken from https://github.com/JuliaDocs/IOCapture.jl/blob/master/src/IOCapture.jl with some modifications to make it log. # Original implementation from Documenter.jl (MIT license) # Save the default output streams. default_stdout = stdout default_stderr = stderr # Redirect both the `stdout` and `stderr` streams to a single `Pipe` object. pipe = Pipe() Base.link_pipe!(pipe; reader_supports_async = true, writer_supports_async = true) pe_stdout = IOContext(pipe.in, default_stdout_iocontext) pe_stderr = IOContext(pipe.in, default_stdout_iocontext) redirect_stdout(pe_stdout) redirect_stderr(pe_stderr) # Bytes written to the `pipe` are captured in `output` and eventually converted to a # `String`. We need to use an asynchronous task to continously tranfer bytes from the # pipe to `output` in order to avoid the buffer filling up and stalling write() calls in # user code. execution_done = Ref(false) output = IOBuffer() @async begin pipe_reader = Base.pipe_reader(pipe) try while !eof(pipe_reader) write(output, readavailable(pipe_reader)) # NOTE: we don't really have to wait for the end of execution to stream output logs # so maybe we should just enable it? if execution_done[] _send_stdio_output!(output, loglevel) end end _send_stdio_output!(output, loglevel) catch err @error "Failed to redirect stdout/stderr to logs" exception=(err,catch_backtrace()) if err isa InterruptException rethrow(err) end end end # To make the `display` function work. redirect_display = TextDisplay(IOContext(pe_stdout, default_display_iocontext)) pushdisplay(redirect_display) # Run the function `f`, capturing all output that it might have generated. # Success signals whether the function `f` did or did not throw an exception. result = try f() finally # Restore display try popdisplay(redirect_display) catch e # This happens when the user calls `popdisplay()`, fine. # @warn "Pluto's display was already removed?" e end execution_done[] = true # Restore the original output streams. redirect_stdout(default_stdout) redirect_stderr(default_stderr) close(pe_stdout) close(pe_stderr) end result end struct JSLink callback::Function on_cancellation::Union{Nothing,Function} cancelled_ref::Ref{Bool} end const cell_js_links = Dict{UUID,Dict{String,JSLink}}() function core_with_js_link(io, callback, on_cancellation) _cell_id = get(io, :pluto_cell_id, currently_running_cell_id[])::UUID link_id = String(rand('a':'z', 16)) links = get!(() -> Dict{String,JSLink}(), cell_js_links, _cell_id) links[link_id] = JSLink(callback, on_cancellation, Ref(false)) write(io, "/* See the documentation for AbstractPlutoDingetjes.Display.with_js_link */ _internal_getJSLinkResponse(\"$(_cell_id)\", \"$(link_id)\")") end function unregister_js_link(cell_id::UUID) # cancel old links old_links = get!(() -> Dict{String,JSLink}(), cell_js_links, cell_id) for (name, link) in old_links link.cancelled_ref[] = true end for (name, link) in old_links c = link.on_cancellation c === nothing || c() end # clear cell_js_links[cell_id] = Dict{String,JSLink}() end function evaluate_js_link(notebook_id::UUID, cell_id::UUID, link_id::String, input::Any) links = get(() -> Dict{String,JSLink}(), cell_js_links, cell_id) link = get(links, link_id, nothing) with_logger_and_io_to_logs(get_cell_logger(notebook_id, cell_id); capture_stdout=false) do if link === nothing @warn " AbstractPlutoDingetjes: JS link not found." link_id (false, "link not found") elseif link.cancelled_ref[] @warn " AbstractPlutoDingetjes: JS link has already been invalidated." link_id (false, "link has been invalidated") else try result = link.callback(input) assertpackable(result) (true, result) catch ex @error " AbstractPlutoDingetjes.Display.with_js_link: Exception while evaluating Julia callback." input exception=(ex, catch_backtrace()) (false, "exception in Julia callback:\n\n$(ex)") end end end end import Dates: DateTime using UUIDs """ **(Internal API.)** A `Ref` containing the id of the cell that is currently **running** or **displaying**. """ const currently_running_cell_id = Ref{UUID}(UUIDs.uuid4()) function core_published_to_js(io, x) assertpackable(x) id_start = objectid2str(x) _notebook_id = get(io, :pluto_notebook_id, notebook_id[])::UUID _cell_id = get(io, :pluto_cell_id, currently_running_cell_id[])::UUID # The unique identifier of this object id = "$_notebook_id/$id_start" d = get!(Dict{String,Any}, cell_published_objects, _cell_id) d[id] = x write(io, "/* See the documentation for AbstractPlutoDingetjes.Display.published_to_js */ getPublishedObject(\"$(id)\")") return nothing end # TODO: This is the deprecated old function. Remove me at some point. struct PublishedToJavascript published_object end function Base.show(io::IO, ::MIME"text/javascript", published::PublishedToJavascript) core_published_to_js(io, published.published_object) end Base.show(io::IO, ::MIME"text/plain", published::PublishedToJavascript) = show(io, MIME("text/javascript"), published) Base.show(io::IO, published::PublishedToJavascript) = show(io, MIME("text/javascript"), published) # TODO: This is the deprecated old function. Remove me at some point. function publish_to_js(x) @warn "Deprecated, use `AbstractPlutoDingetjes.Display.published_to_js(x)` instead of `PlutoRunner.publish_to_js(x)`." assertpackable(x) PublishedToJavascript(x) end const Packable = Union{Nothing,Missing,String,Symbol,Int64,Int32,Int16,Int8,UInt64,UInt32,UInt16,UInt8,Float32,Float64,Bool,MIME,UUID,DateTime} assertpackable(::Packable) = nothing assertpackable(t::Any) = throw(ArgumentError("Only simple objects can be shared with JS, like vectors and dictionaries. $(string(typeof(t))) is not compatible.")) assertpackable(::Vector{<:Packable}) = nothing assertpackable(::Dict{<:Packable,<:Packable}) = nothing assertpackable(x::Vector) = foreach(assertpackable, x) assertpackable(d::Dict) = let foreach(assertpackable, keys(d)) foreach(assertpackable, values(d)) end assertpackable(t::Tuple) = foreach(assertpackable, t) assertpackable(t::NamedTuple) = foreach(assertpackable, t) import .Throttled """ Return whether the `request` was authenticated in one of two ways: 1. the session's `secret` was included in the URL as a search parameter, or 2. the session's `secret` was included in a cookie. """ function is_authenticated(session::ServerSession, request::HTTP.Request) ( secret_in_url = try uri = HTTP.URI(request.target) query = HTTP.queryparams(uri) get(query, "secret", "") == session.secret catch e @warn "Failed to authenticate request using URL" exception = (e, catch_backtrace()) false end ) || ( secret_in_cookie = try cookies = HTTP.cookies(request) any(cookies) do cookie cookie.name == "secret" && cookie.value == session.secret end catch e @warn "Failed to authenticate request using cookies" exception = (e, catch_backtrace()) false end ) # that ) || ( kind of looks like Krabs from spongebob end # Function to log the url with secret on the Julia CLI when a request comes to the server without the secret. Executes at most once every 5 seconds const log_secret_throttled = Throttled.simple_leading_throttle(5) do session::ServerSession, request::HTTP.Request host = HTTP.header(request, "Host") target = request.target url = Text(string(HTTP.URI(HTTP.URI("http://$host/"); query=Dict("secret" => session.secret)))) @info("No longer authenticated? Visit this URL to continue:", url) end function add_set_secret_cookie!(session::ServerSession, response::HTTP.Response) HTTP.setheader(response, "Set-Cookie" => "secret=$(session.secret); SameSite=Strict; HttpOnly") response end # too many layers i know """ Generate a middleware (i.e. a function `HTTP.Handler -> HTTP.Handler`) that stores the `session` in every `request`'s context. """ function create_session_context_middleware(session::ServerSession) function session_context_middleware(handler::Function)::Function function(request::HTTP.Request) request.context[:pluto_session] = session handler(request) end end end session_from_context(request::HTTP.Request) = request.context[:pluto_session]::ServerSession function auth_required(session::ServerSession, request::HTTP.Request) path = HTTP.URI(request.target).path ext = splitext(path)[2] security = session.options.security if path  ("/ping", "/possible_binder_token_please") || ext  (".ico", ".js", ".css", ".png", ".gif", ".svg", ".ico", ".woff2", ".woff", ".ttf", ".eot", ".otf", ".json", ".map") false elseif path  ("", "/") # / does not need security.require_secret_for_open_links, because this is how we handle the configuration where: # require_secret_for_open_links == true # require_secret_for_access == false # # This means that access to all 'risky' endpoints is restricted to authenticated requests (to prevent CSRF), but we allow an unauthenticated request to visit the `/` page and acquire the cookie (see `add_set_secret_cookie!`). # # (By default, `require_secret_for_access` (and `require_secret_for_open_links`) is `true`.) security.require_secret_for_access else security.require_secret_for_access || security.require_secret_for_open_links end end """ auth_middleware(f::HTTP.Handler) -> HTTP.Handler Returns an `HTTP.Handler` (i.e. a function `HTTP.Request  HTTP.Response`) which does three things: 1. Check whether the request is authenticated (by calling `is_authenticated`), if not, return a 403 error. 2. Call your `f(request)` to create the response message. 3. Add a `Set-Cookie` header to the response with the session's `secret`. This is for HTTP requests, the authentication mechanism for WebSockets is separate. """ function auth_middleware(handler) return function (request::HTTP.Request) session = session_from_context(request) required = auth_required(session, request) if !required || is_authenticated(session, request) response = handler(request) if !required filter!(p -> p[1] != "Access-Control-Allow-Origin", response.headers) HTTP.setheader(response, "Access-Control-Allow-Origin" => "*") end if required || HTTP.URI(request.target).path  ("", "/") add_set_secret_cookie!(session, response) end response else log_secret_throttled(session, request) error_response(403, "Not yet authenticated", "<b>Open the link that was printed in the terminal where you launched Pluto.</b> It includes a <em>secret</em>, which is needed to access this server.<br><br>If you are running the server yourself and want to change this configuration, have a look at the keyword arguments to <em>Pluto.run</em>. <br><br>Please <a href='https://github.com/fonsp/Pluto.jl/issues'>report this error</a> if you did not expect it!") end end end import UUIDs: uuid1 import .PkgCompat import .Status "Will hold all 'response handlers': functions that respond to a WebSocket request from the client." const responses = Dict{Symbol,Function}() Base.@kwdef struct ClientRequest session::ServerSession notebook::Union{Nothing,Notebook} body::Any=nothing initiator::Union{Initiator,Nothing}=nothing end require_notebook(r::ClientRequest) = if r.notebook === nothing throw(ArgumentError("Notebook request called without a notebook ")) end ### # RESPONDING TO A NOTEBOOK STATE UPDATE ### """ ## State management in Pluto *Aka: how do the server and clients stay in sync?* A Pluto notebook session has *state*: with this, we mean: 1. The input and ouput of each cell, the cell order, and more metadata about the notebook and cells [^state] This state needs to be **synchronised between the server and all clients** (we support multiple synchronised clients), and note that: - Either side wants to update the state. Generally, a client will update cell inputs, the server will update cell outputs. - Both sides want to *react* to state updates - The server is in Julia, the clients are in JS - This is built on top of our websocket+msgpack connection, but that doesn't matter too much We do this by implementing something similar to how you use Google Firebase: there is **one shared state object, any party can mutate it, and it will synchronise to all others automatically**. The state object is a nested structure of mutable `Dict`s, with immutable ints, strings, bools, arrays, etc at the endpoints. Some cool things are: - Our system uses object diffing, so only *changes to the state* are actually tranferred over the network. But you can use it as if the entire state is sent around constantly. - In the frontend, the *shared state* is part of the *react state*, i.e. shared state updates automatically trigger visual updates. - Within the client, state changes take effect instantly, without waiting for a round trip to the server. This means that when you add a cell, it shows up instantly. Diffing is done using `immer.js` (frontend) and `src/webserver/Firebasey.jl` (server). We wrote Firebasey ourselves to match immer's functionality, and the cool thing is: **it is a Pluto notebook**! Since Pluto notebooks are `.jl` files, we can just `include` it in our module. The shared state object is generated by [`notebook_to_js`](@ref). Take a look! The Julia server orchestrates this firebasey stuff. For this, we keep a **copy** of the latest state of each client on the server (see [`current_state_for_clients`](@ref)). When anything changes to the Julia state (e.g. when a cell finished running), we call [`send_notebook_changes!`](@ref), which will call [`notebook_to_js`](@ref) to compute the new desired state object. For each client, we diff the new state to their last known state, and send them the difference. ### Responding to changes made by a client When a client updates the shared state object, we want the server to *react* to that change by taking an action. Which action to take depends on which field changes. For example, when `state["path"]` changes, we should rename the notebook file. When `state["cell_inputs"][a_cell_id]["code"]` changes, we should reparse and analyze that cel, etc. This location of the change, e.g. `"cell_inputs/<a_cell_id>/code"` is called the *path* of the change. [`effects_of_changed_state`](@ref) define these pattern-matchers. We use a `Wildcard()` to take the place of *any* key, see [`Wildcard`](@ref), and we use the change/update/patch inside the given function. ### Not everything uses the shared state (yet) Besides `:update_notebook`, you will find more functions in [`responses`](@ref) that respond to classic 'client requests', such as `:reshow_cell` and `:shutdown_notebook`. Some of these requests get a direct response, like the list of autocomplete options to a `:complete` request (in `src/webserver/REPLTools.jl`). On the javascript side, these direct responses can be `awaited`, because every message has a unique ID. [^state]: Two other meanings of _state_ could be: 2. The reactivity data: the parsed AST (`Expr`) of each cell, which variables are defined or referenced by which cells, in what order will cells run? 3. The state of the Julia process: i.e. which variables are defined, which packages are imported, etc. The first two (1 & 2) are stored in a [`Notebook`](@ref) struct, remembered by the server process (Julia). (In fact, (2) is entirely described by (1), but we store it for performance reasons.) I included (3) for completeness, but it is not stored by us, we hope to control and minimize (3) by keeping track of (1) and (2). """ module Firebasey include("./Firebasey.jl") end module FirebaseyUtils # I put Firebasey here manually THANKS JULIA import ..Firebasey include("./FirebaseyUtils.jl") end # All of the arrays in the notebook_to_js object are 'immutable' (we write code as if they are), so we can enable this optimization: Firebasey.use_triple_equals_for_arrays[] = true # the only possible Arrays are: # - cell_order # - cell_execution_order # - cell_result > * > output > body # - bonds > * > value > * # - cell_dependencies > * > downstream_cells_map > * > # - cell_dependencies > * > upstream_cells_map > * > function notebook_to_js(notebook::Notebook) Dict{String,Any}( "pluto_version" => PLUTO_VERSION_STR, "notebook_id" => notebook.notebook_id, "path" => notebook.path, "shortpath" => basename(notebook.path), "in_temp_dir" => startswith(notebook.path, new_notebooks_directory()), "process_status" => notebook.process_status, "last_save_time" => notebook.last_save_time, "last_hot_reload_time" => notebook.last_hot_reload_time, "cell_inputs" => Dict{UUID,Dict{String,Any}}( id => Dict{String,Any}( "cell_id" => cell.cell_id, "code" => cell.code, "code_folded" => cell.code_folded, "metadata" => cell.metadata, ) for (id, cell) in notebook.cells_dict), "cell_results" => Dict{UUID,Dict{String,Any}}( id => Dict{String,Any}( "cell_id" => cell.cell_id, "depends_on_disabled_cells" => cell.depends_on_disabled_cells, "output" => FirebaseyUtils.ImmutableMarker(cell.output), "published_object_keys" => collect(keys(cell.published_objects)), "queued" => cell.queued, "running" => cell.running, "errored" => cell.errored, "runtime" => cell.runtime, "logs" => FirebaseyUtils.AppendonlyMarker(cell.logs), "depends_on_skipped_cells" => cell.depends_on_skipped_cells, ) for (id, cell) in notebook.cells_dict), "cell_order" => notebook.cell_order, "published_objects" => merge!(Dict{String,Any}(), (c.published_objects for c in values(notebook.cells_dict))...), "bonds" => Dict{String,Dict{String,Any}}( String(key) => Dict{String,Any}( "value" => bondvalue.value, ) for (key, bondvalue) in notebook.bonds), "metadata" => notebook.metadata, "nbpkg" => let ctx = notebook.nbpkg_ctx Dict{String,Any}( "enabled" => ctx !== nothing, "waiting_for_permission" => notebook.process_status === ProcessStatus.waiting_for_permission, "waiting_for_permission_but_probably_disabled" => notebook.process_status === ProcessStatus.waiting_for_permission && !use_plutopkg(notebook.topology), "restart_recommended_msg" => notebook.nbpkg_restart_recommended_msg, "restart_required_msg" => notebook.nbpkg_restart_required_msg, "installed_versions" => ctx === nothing ? Dict{String,String}() : notebook.nbpkg_installed_versions_cache, "terminal_outputs" => notebook.nbpkg_terminal_outputs, "install_time_ns" => notebook.nbpkg_install_time_ns, "busy_packages" => notebook.nbpkg_busy_packages, "instantiated" => notebook.nbpkg_ctx_instantiated, ) end, "status_tree" => Status.tojs(notebook.status_tree), "cell_dependencies" => notebook._cached_cell_dependencies, "cell_execution_order" => cell_id.(collect(notebook._cached_topological_order)), ) end """ For each connected client, we keep a copy of their current state. This way we know exactly which updates to send when the server-side state changes. """ const current_state_for_clients = WeakKeyDict{ClientSession,Any}() const current_state_for_clients_lock = ReentrantLock() const update_counter_for_debugging = Ref(0) const update_counter_for_clients_1 = WeakKeyDict{ClientSession,Any}() const update_counter_for_clients_2 = WeakKeyDict{ClientSession,Any}() const update_counter_for_clients_3 = WeakKeyDict{ClientSession,Any}() """ Update the local state of all clients connected to this notebook. """ function send_notebook_changes!(::ClientRequest; commentary::Any=nothing) outbox = Set{Tuple{ClientSession,UpdateMessage}}() lock(current_state_for_clients_lock) do counter = update_counter_for_debugging[] += 1 if !isempty(.session.connected_clients) notebook_dict = notebook_to_js(.notebook) for (_, client) in .session.connected_clients if client.connected_notebook !== nothing && client.connected_notebook.notebook_id == .notebook.notebook_id current_dict = get(current_state_for_clients, client, :empty) patches = Firebasey.diff(current_dict, notebook_dict) patches_as_dicts = Firebasey._convert(Vector{Dict}, patches) current_state_for_clients[client] = deep_enough_copy(notebook_dict) # Make sure we do send a confirmation to the client who made the request, even without changes is_response = .initiator !== nothing && client == .initiator.client if !isempty(patches) || is_response response = Dict( :counter => counter, :patches => patches_as_dicts, :response => is_response ? commentary : nothing ) push!(outbox, (client, UpdateMessage(:notebook_diff, response, .notebook, nothing, .initiator))) end end end for (client, msg) in outbox putclientupdates!(client, msg) end end try_event_call(.session, StateChangeEvent(.notebook)) end end "Like `deepcopy`, but anything other than `Dict` gets a shallow (reference) copy." @generated function deep_enough_copy(d::Dict) quote out = $d() for (k,v) in d out[k] = deep_enough_copy(v) end out end end deep_enough_copy(d) = d """ A placeholder path. The path elements that it replaced will be given to the function as arguments. """ struct Wildcard end abstract type Changed end struct CodeChanged <: Changed end struct FileChanged <: Changed end struct BondChanged <: Changed bond_name::Symbol is_first_value::Bool end # to support push!(x, y...) # with y = [] Base.push!(x::Set{Changed}) = x const no_changes = Changed[] const effects_of_changed_state = Dict( "path" => function(; request::ClientRequest, patch::Firebasey.ReplacePatch) SessionActions.move(request.session, request.notebook, patch.value) return no_changes end, "process_status" => function(; request::ClientRequest, patch::Firebasey.ReplacePatch) newstatus = patch.value @info "Process status set by client" newstatus end, "in_temp_dir" => function(; _...) no_changes end, "cell_inputs" => Dict( Wildcard() => function(cell_id, rest...; request::ClientRequest, patch::Firebasey.JSONPatch) Firebasey.applypatch!(request.notebook, patch) if length(rest) == 0 [CodeChanged(), FileChanged()] elseif length(rest) == 1 && Symbol(rest[1]) == :code [CodeChanged(), FileChanged()] else [FileChanged()] end end, ), "cell_order" => function(; request::ClientRequest, patch::Firebasey.ReplacePatch) Firebasey.applypatch!(request.notebook, patch) [FileChanged()] end, "bonds" => Dict( Wildcard() => function(name; request::ClientRequest, patch::Firebasey.JSONPatch) name = Symbol(name) Firebasey.applypatch!(request.notebook, patch) [BondChanged(name, patch isa Firebasey.AddPatch)] end, ), "metadata" => Dict( Wildcard() => function(property; request::ClientRequest, patch::Firebasey.JSONPatch) Firebasey.applypatch!(request.notebook, patch) [FileChanged()] end ) ) responses[:update_notebook] = function response_update_notebook(::ClientRequest) require_notebook() try notebook = .notebook patches = (Base.convert(Firebasey.JSONPatch, update) for update in .body["updates"]) if length(patches) == 0 send_notebook_changes!() return nothing end if !haskey(current_state_for_clients, .initiator.client) throw(ErrorException("Updating without having a first version of the notebook??")) end # TODO Immutable ?? for patch in patches Firebasey.applypatch!(current_state_for_clients[.initiator.client], patch) end changes = Set{Changed}() for patch in patches (mutator, matches, rest) = trigger_resolver(effects_of_changed_state, patch.path) current_changes = if isempty(rest) && applicable(mutator, matches...) mutator(matches...; request=, patch) else mutator(matches..., rest...; request=, patch) end union!(changes, current_changes) end # We put a flag to check whether any patch changes the skip_as_script metadata. This is to eventually trigger a notebook updated if no reactive_run is part of this update skip_as_script_changed = any(patches) do patch path = patch.path metadata_idx = findfirst(isequal("metadata"), path) if metadata_idx === nothing false else isequal(path[metadata_idx+1], "skip_as_script") end end # If CodeChanged  changes, then the client will also send a request like run_multiple_cells, which will trigger a file save _before_ running the cells. # In the future, we should get rid of that request, and save the file here. For now, we don't save the file here, to prevent unnecessary file IO. # (You can put a log in save_notebook to track how often the file is saved) if FileChanged()  changes && CodeChanged()  changes if skip_as_script_changed # If skip_as_script has changed but no cell run is happening we want to update the notebook dependency here before saving the file update_skipped_cells_dependency!(notebook) end save_notebook(.session, notebook) end let bond_changes = filter(x -> x isa BondChanged, changes) bound_sym_names = Symbol[x.bond_name for x in bond_changes] is_first_values = Bool[x.is_first_value for x in bond_changes] set_bond_values_reactive(; session=.session, notebook=.notebook, bound_sym_names=bound_sym_names, is_first_values=is_first_values, run_async=true, initiator=.initiator, ) end send_notebook_changes!(; commentary=Dict(:update_went_well => :)) catch ex @error "Update notebook failed" .body["updates"] exception=(ex, stacktrace(catch_backtrace())) response = Dict( :update_went_well => :, :why_not => sprint(showerror, ex), :should_i_tell_the_user => ex isa SessionActions.UserError, ) send_notebook_changes!(; commentary=response) end end function trigger_resolver(anything, path, values=[]) (value=anything, matches=values, rest=path) end function trigger_resolver(resolvers::Dict, path, values=[]) if isempty(path) throw(BoundsError("resolver path ends at Dict with keys $(keys(resolvers))")) end segment, rest... = path if haskey(resolvers, segment) trigger_resolver(resolvers[segment], rest, values) elseif haskey(resolvers, Wildcard()) trigger_resolver(resolvers[Wildcard()], rest, (values..., segment)) else throw(BoundsError("failed to match path $(path), possible keys $(keys(resolvers))")) end end ### # MISC RESPONSES ### responses[:current_time] = function response_current_time(::ClientRequest) putclientupdates!(.session, .initiator, UpdateMessage(:current_time, Dict(:time => time()), nothing, nothing, .initiator)) end responses[:connect] = function response_connect(::ClientRequest) putclientupdates!(.session, .initiator, UpdateMessage(:, Dict( :notebook_exists => (.notebook !== nothing), :session_options => .session.options, :version_info => Dict( :pluto => PLUTO_VERSION_STR, :julia => JULIA_VERSION_STR, :dismiss_update_notification => .session.options.server.dismiss_update_notification, ), ), nothing, nothing, .initiator)) end responses[:ping] = function response_ping(::ClientRequest) putclientupdates!(.session, .initiator, UpdateMessage(:pong, Dict(), nothing, nothing, .initiator)) end responses[:reset_shared_state] = function response_reset_shared_state(::ClientRequest) delete!(current_state_for_clients, .initiator.client) if .notebook !== nothing send_notebook_changes!(; commentary=Dict(:from_reset => true)) end end """ This function updates current_state_for_clients for our client with `cell.queued = true`. We do the same on the client side, where we set the cell's result to `queued = true` immediately in that client's local state. Search for `` in the frontend code. This is also kinda related to https://github.com/fonsp/Pluto.jl/pull/1892 but not really, see https://github.com/fonsp/Pluto.jl/pull/2989. I actually think this does not make a differency anymore, see https://github.com/fonsp/Pluto.jl/pull/2999. """ function _set_cells_to_queued_in_local_state(client, notebook, cells) if haskey(current_state_for_clients, client) results = current_state_for_clients[client]["cell_results"] for cell in cells if haskey(results, cell.cell_id) old = results[cell.cell_id]["queued"] results[cell.cell_id]["queued"] = true end end end end responses[:run_multiple_cells] = function response_run_multiple_cells(::ClientRequest) require_notebook() uuids = UUID.(.body["cells"]) cells = map(uuids) do uuid .notebook.cells_dict[uuid] end if will_run_code(.notebook) foreach(cell -> cell.queued = true, cells) if .initiator !== nothing _set_cells_to_queued_in_local_state(.initiator.client, .notebook, cells) end end function on_auto_solve_multiple_defs(disabled_cells_dict) response = Dict{Symbol,Any}( :disabled_cells => Dict{UUID,Any}(cell_id(k) => v for (k,v) in disabled_cells_dict), ) putclientupdates!(.session, .initiator, UpdateMessage(:run_feedback, response, .notebook, nothing, .initiator)) end wfp = .notebook.process_status == ProcessStatus.waiting_for_permission update_save_run!(.session, .notebook, cells; run_async=true, save=true, auto_solve_multiple_defs=true, on_auto_solve_multiple_defs, # special case: just render md cells in "Safe preview" mode prerender_text=wfp, clear_not_prerenderable_cells=wfp, ) end responses[:get_all_notebooks] = function response_get_all_notebooks(::ClientRequest) putplutoupdates!(.session, clientupdate_notebook_list(.session.notebooks, initiator=.initiator)) end responses[:interrupt_all] = function response_interrupt_all(::ClientRequest) require_notebook() session_notebook = (.session, .notebook) workspace = WorkspaceManager.get_workspace(session_notebook; allow_creation=false) already_interrupting = .notebook.wants_to_interrupt anything_running = !isready(workspace.dowork_token) if !already_interrupting && anything_running .notebook.wants_to_interrupt = true WorkspaceManager.interrupt_workspace(session_notebook) end # TODO: notify user whether interrupt was successful end responses[:shutdown_notebook] = function response_shutdown_notebook(::ClientRequest) require_notebook() SessionActions.shutdown(.session, .notebook; keep_in_session=.body["keep_in_session"]) end without_initiator(::ClientRequest) = ClientRequest(session=.session, notebook=.notebook) responses[:restart_process] = function response_restart_process(::ClientRequest; run_async::Bool=true) require_notebook() if .notebook.process_status != ProcessStatus.waiting_to_restart .notebook.process_status = ProcessStatus.waiting_to_restart .session.options.evaluation.run_notebook_on_load && _report_business_cells_planned!(.notebook) send_notebook_changes!( |> without_initiator) # TODO skip necessary? SessionActions.shutdown(.session, .notebook; keep_in_session=true, async=true) .notebook.process_status = ProcessStatus.starting send_notebook_changes!( |> without_initiator) update_save_run!(.session, .notebook, .notebook.cells; run_async=run_async, save=true) end end responses[:reshow_cell] = function response_reshow_cell(::ClientRequest) require_notebook() @assert will_run_code(.notebook) cell = let cell_id = UUID(.body["cell_id"]) .notebook.cells_dict[cell_id] end run = WorkspaceManager.format_fetch_in_workspace( (.session, .notebook), cell.cell_id, ends_with_semicolon(cell.code), collect(keys(cell.published_objects)), (parse(PlutoRunner.ObjectID, .body["objectid"], base=16), convert(Int64, .body["dim"])), ) set_output!(cell, run, ExprAnalysisCache(.notebook.topology.codes[cell]); persist_js_state=true) # send to all clients, why not send_notebook_changes!( |> without_initiator) end responses[:request_js_link_response] = function response_request_js_link_response(::ClientRequest) require_notebook() @assert will_run_code(.notebook) Threads.@spawn try result = WorkspaceManager.eval_fetch_in_workspace( (.session, .notebook), quote PlutoRunner.evaluate_js_link( $(.notebook.notebook_id), $(UUID(.body["cell_id"])), $(.body["link_id"]), $(.body["input"]), ) end ) putclientupdates!(.session, .initiator, UpdateMessage(:, result, nothing, nothing, .initiator)) catch ex @error "Error in request_js_link_response" exception=(ex, stacktrace(catch_backtrace())) end end responses[:nbpkg_available_versions] = function response_nbpkg_available_versions(::ClientRequest) # require_notebook() all_versions = PkgCompat.package_versions(.body["package_name"]) url = PkgCompat.package_url(.body["package_name"]) putclientupdates!(.session, .initiator, UpdateMessage(:, Dict( :versions => string.(all_versions), :url => url, ), nothing, nothing, .initiator)) end responses[:all_registered_package_names] = function response_all_registered_package_names(::ClientRequest) results = PkgCompat.registered_package_names() putclientupdates!(.session, .initiator, UpdateMessage(:, Dict( :results => results, ), nothing, nothing, .initiator)) end responses[:pkg_update] = function response_pkg_update(::ClientRequest) require_notebook() update_nbpkg(.session, .notebook) putclientupdates!(.session, .initiator, UpdateMessage(:, Dict(), nothing, nothing, .initiator)) end ### A Pluto.jl notebook ### # v0.20.4 using Markdown using InteractiveUtils #  e748600a-2de1-11eb-24be-d5f0ecab8fa4 #  show_logs = false #  skip_as_script = true #= # Only define this in Pluto - assume we are `using Test` otherwise begin import Pkg Pkg.activate(mktempdir()) Pkg.add(Pkg.PackageSpec(name="PlutoTest")) using PlutoTest end  =# #  3e07f976-6cd0-4841-9762-d40337bb0645 #  skip_as_script = true #= using Markdown: @md_str  =# #  d948dc6e-2de1-11eb-19e7-cb3bb66353b6 #  skip_as_script = true #= md"# Diffing"  =# #  1a6e1853-6db1-4074-bce0-5f274351cece #  skip_as_script = true #= md""" We define a _diffing system_ for Julia `Dict`s, which is analogous to the diffing system of immer.js. This notebook is part of Pluto's source code (included in `src/webserver/Dynamic.jl`). """  =# #  49fc1f97-3b8f-4297-94e5-2e24c001d35c #  skip_as_script = true #= md""" ## Example Computing a diff: """  =# #  d8e73b90-24c5-4e50-830b-b1dbe6224c8e #  skip_as_script = true #= dict_1 = Dict{String,Any}( "a" => 1, "b" => Dict( "c" => [3,4], "d" => 99, ), "e" => "hello!" );  =# #  19646596-b35b-44fa-bfcf-891f9ffb748c #  skip_as_script = true #= dict_2 = Dict{String,Any}( "a" => 1, "b" => Dict( "c" => [3,4,5], "d" => 99, "" => "", ), );  =# #  9d2c07d9-16a9-4b9f-a375-2adb6e5b907a #  skip_as_script = true #= md""" Applying a set of patches: """  =# #  336bfd4f-8a8e-4a2d-be08-ee48d6a9f747 #  skip_as_script = true #= md""" ## JSONPatch objects """  =# #  db116c0a-2de1-11eb-2a56-872af797c547 abstract type JSONPatch end #  bd0d46bb-3e58-4522-bae0-83eb799196c4 const PatchPath = Vector #  db2d8a3e-2de1-11eb-02b8-9ffbfaeff61c struct AddPatch <: JSONPatch path::PatchPath value::Any end #  ffe9b3d9-8e35-4a31-bab2-8787a4140594 struct RemovePatch <: JSONPatch path::PatchPath end #  894de8a7-2757-4d7a-a2be-1069fa872911 struct ReplacePatch <: JSONPatch path::PatchPath value::Any end #  9a364714-edb1-4bca-9387-a8bbacccd10d struct CopyPatch <: JSONPatch path::PatchPath from::PatchPath end #  9321d3be-cb91-4406-9dc7-e5c38f7d377c struct MovePatch <: JSONPatch path::PatchPath from::PatchPath end #  73631aea-5e93-4da2-a32d-649029660d4e const Patches = Vector{JSONPatch} #  0fd3e910-abcc-4421-9d0b-5cfb90034338 const NoChanges = Patches() #  aad7ab32-eecf-4aad-883d-1c802cad6c0c #  skip_as_script = true #= md"### =="  =# #  732fd744-acdb-4507-b1de-6866ec5563dd Base.hash(a::AddPatch, h::UInt) = hash((AddPatch, a.value, a.path), h) #  17606cf6-2d0f-4245-89a3-746ad818a664 Base.hash(a::RemovePatch, h::UInt) = hash((RemovePatch, a.path), h) #  c7ac7d27-7bf9-4209-8f3c-e4d52c543e29 Base.hash(a::ReplacePatch, h::UInt) = hash((ReplacePatch, a.value, a.path), h) #  042f7788-e996-430e-886d-ffb4f70dea9e Base.hash(a::CopyPatch, h::UInt) = hash((CopyPatch, a.from, a.path), h) #  9d2dde5c-d404-4fbc-b8e0-5024303c8052 Base.hash(a::MovePatch, h::UInt) = hash((MovePatch, a.from, a.path), h) #  f649f67c-aab0-4d35-a799-f398e5f3ecc4 function Base.:(==)(a::AddPatch, b::AddPatch) a.value == b.value && a.path == b.path end #  63087738-d70c-46f5-b072-21cd8953df35 function Base.:(==)(a::RemovePatch, b::RemovePatch) a.path == b.path end #  aa81974a-7254-45e0-9bfe-840c4793147f function Base.:(==)(a::ReplacePatch, b::ReplacePatch) a.path == b.path && a.value == b.value end #  31188a03-76ba-40cf-a333-4d339ce37711 function Base.:(==)(a::CopyPatch, b::CopyPatch) a.path == b.path && a.from == b.from end #  7524a9e8-1a6d-4851-b50e-19415f25a84b function Base.:(==)(a::MovePatch, b::MovePatch) a.path == b.path && a.from == b.from end #  5ddfd616-db20-451b-bc1e-2ad52e0e2777 #= @test Base.hash(ReplacePatch(["asd"], Dict("a" => 2))) == Base.hash(ReplacePatch(["asd"], Dict("a" => 2)))  =# #  24e93923-eab9-4a7b-9bc7-8d8a1209a78f #= @test ReplacePatch(["asd"], Dict("a" => 2)) == ReplacePatch(["asd"], Dict("a" => 2))  =# #  09ddf4d9-5ccb-4530-bfab-d11b864e872a #= @test Base.hash(RemovePatch(["asd"])) == Base.hash(RemovePatch(["asd"]))  =# #  d9e764db-94fc-44f7-8c2e-3d63f4809617 #= @test RemovePatch(["asd"]) == RemovePatch(["asd"])  =# #  99df99ad-aad5-4275-97d4-d1ceeb2f8d15 #= @test Base.hash(RemovePatch(["aasd"])) != Base.hash(RemovePatch(["asd"]))  =# #  2d665639-7274-495a-ae9d-f358a8219bb7 #= @test Base.hash(ReplacePatch(["asd"], Dict("a" => 2))) != Base.hash(AddPatch(["asd"], Dict("a" => 2)))  =# #  f658a72d-871d-49b3-9b73-7efedafbd7a6 #  skip_as_script = true #= md"### convert(::Type{Dict}, ::JSONPatch)"  =# #  230bafe2-aaa7-48f0-9fd1-b53956281684 function Base.convert(::Type{Dict}, patch::AddPatch) Dict{String,Any}("op" => "add", "path" => patch.path, "value" => patch.value) end #  b48e2c08-a94a-4247-877d-949d92dde626 function Base.convert(::Type{Dict}, patch::RemovePatch) Dict{String,Any}("op" => "remove", "path" => patch.path) end #  921a130e-b028-4f91-b077-3bd79dcb6c6d function Base.convert(::Type{JSONPatch}, patch_dict::Dict) op = patch_dict["op"] if op == "add" AddPatch(patch_dict["path"], patch_dict["value"]) elseif op == "remove" RemovePatch(patch_dict["path"]) elseif op == "replace" ReplacePatch(patch_dict["path"], patch_dict["value"]) else throw(ArgumentError("Unknown operation :$(patch_dict["op"]) in Dict to JSONPatch conversion")) end end #  07eeb122-6706-4544-a007-1c8d6581eec8 #  skip_as_script = true #= Base.convert(Dict, AddPatch([:x, :y], 10))  =# #  c59b30b9-f702-41f1-bb2e-1736c8cd5ede #  skip_as_script = true #= Base.convert(Dict, RemovePatch([:x, :y]))  =# #  6d67f8a5-0e0c-4b6e-a267-96b34d580946 #  skip_as_script = true #= add_patch = AddPatch(["counter"], 10)  =# #  56b28842-4a67-44d7-95e7-55d457a44fb1 #  skip_as_script = true #= remove_patch = RemovePatch(["counter"])  =# #  f10e31c0-1d2c-4727-aba5-dd676a10041b #  skip_as_script = true #= replace_patch = ReplacePatch(["counter"], 10)  =# #  3a99e22d-42d6-4b2d-9381-022b41b0e852 #  skip_as_script = true #= md"### wrappath"  =# #  831d84a6-1c71-4e68-8c7c-27d9093a82c4 function wrappath(path::PatchPath, patches::Vector{JSONPatch}) map(patches) do patch wrappath(path, patch) end end #  2ad11c73-4691-4283-8f98-3d2a87926b99 function wrappath(path, patch::AddPatch) AddPatch([path..., patch.path...], patch.value) end #  5513ea3b-9498-426c-98cb-7dc23d32f72e function wrappath(path, patch::RemovePatch) RemovePatch([path..., patch.path...]) end #  0c2d6da1-cad3-4c9f-93e9-922457083945 function wrappath(path, patch::ReplacePatch) ReplacePatch([path..., patch.path...], patch.value) end #  84c87031-7733-4d1f-aa90-f8ab71506251 function wrappath(path, patch::CopyPatch) CopyPatch([path..., patch.path...], patch.from) end #  8f265a33-3a2d-4508-9477-ca62e8ce3c12 function wrappath(path, patch::MovePatch) MovePatch([path..., patch.path...], patch.from) end #  daf9ec12-2de1-11eb-3a8d-59d9c2753134 #  skip_as_script = true #= md"## Diff"  =# #  0b50f6b2-8e85-4565-9f04-f99c913b4592 const use_triple_equals_for_arrays = Ref(false) #  59e94cb2-c2f9-4f6c-9562-45e8c15931af function diff(old::T, new::T) where T <: AbstractArray if use_triple_equals_for_arrays[] ? ((old === new) || (old == new)) : (old == new) NoChanges else JSONPatch[ReplacePatch([], new)] end end #  c9d5d81c-b0b6-4d1a-b1de-96d3b3701700 function diff(old::T, new::T) where T if old == new NoChanges else JSONPatch[ReplacePatch([], new)] end end #  24389a0a-c3ac-4438-9dfe-1d14cd033d25 diff(::Missing, ::Missing) = NoChanges #  9cbaaec2-709c-4769-886c-ec92b12c18bc struct Deep{T} value::T end #  db75df12-2de1-11eb-0726-d1995cebd382 function diff(old::Deep{T}, new::Deep{T}) where T changes = JSONPatch[] for property in propertynames(old.value) for change in diff(getproperty(old.value, property), getproperty(new.value, property)) push!(changes, wrappath([property], change)) end end changes # changes = [] # for property in fieldnames(T) # for change in diff(getfield(old.value, property), getfield(new.value, property)) # push!(changes, wrappath([property], change)) # end # end # changes end #  dbc7f97a-2de1-11eb-362f-055a734d1a9e function diff(o1::AbstractDict, o2::AbstractDict) changes = JSONPatch[] # for key in keys(o1)  keys(o2) # for change in diff(get(o1, key, nothing), get(o2, key, nothing)) # push!(changes, wrappath([key], change)) # end # end # same as above but faster: for (key1, val1) in o1 for change in diff(val1, get(o2, key1, nothing)) push!(changes, wrappath([key1], change)) end end for (key2, val2) in o2 if !haskey(o1, key2) for change in diff(nothing, val2) push!(changes, wrappath([key2], change)) end end end changes end #  67ade214-2de3-11eb-291d-135a397d629b function diff(o1, o2) JSONPatch[ReplacePatch([], o2)] end #  b8c58aa4-c24d-48a3-b2a8-7c01d50a3349 function diff(o1::Nothing, o2) JSONPatch[AddPatch([], o2)] end #  5ab390f9-3b0c-4978-9e21-2aaa61db2ce4 function diff(o1, o2::Nothing) JSONPatch[RemovePatch([])] end #  09f53db0-21ae-490b-86b5-414eba403d57 function diff(o1::Nothing, o2::Nothing) NoChanges end #  7ca087b8-73ac-49ea-9c5a-2971f0da491f #= example_patches = diff(dict_1, dict_2)  =# #  59b46bfe-da74-43af-9c11-cb0bdb2c13a2 #  skip_as_script = true #= md""" ### Dict example """  =# #  200516da-8cfb-42fe-a6b9-cb4730168923 #  skip_as_script = true #= celldict1 = Dict(:x => 1, :y => 2, :z => 3)  =# #  76326e6c-b95a-4b2d-a78c-e283e5fadbe2 #  skip_as_script = true #= celldict2 = Dict(:x => 1, :y => 2, :z => 4)  =# #  664cd334-91c7-40dd-a2bf-0da720307cfc #  skip_as_script = true #= notebook1 = Dict( :x => 1, :y => 2, )  =# #  b7fa5625-6178-4da8-a889-cd4f014f43ba #  skip_as_script = true #= notebook2 = Dict( :y => 4, :z => 5 )  =# #  dbdd1df0-2de1-11eb-152f-8d1af1ad02fe #= notebook1_to_notebook2 = diff(notebook1, notebook2)  =# #  3924953f-787a-4912-b6ee-9c9d3030f0f0 #  skip_as_script = true #= md""" ### Large Dict example 1 """  =# #  80689881-1b7e-49b2-af97-9e3ab639d006 #  skip_as_script = true #= big_array = rand(UInt8, 1_000_000)  =# #  fd22b6af-5fd2-428a-8291-53e223ea692c #  skip_as_script = true #= big_string = repeat('a', 1_000_000);  =# #  bcd5059b-b0d2-49d8-a756-92349aa56aca #= large_dict_1 = Dict{String,Any}( "cell_$(i)" => Dict{String,Any}( "x" => 1, "y" => big_array, "z" => big_string, ) for i in 1:10 );  =# #  e7fd6bab-c114-4f3e-b9ad-1af2d1147770 #= begin large_dict_2 = Dict{String,Any}( "cell_$(i)" => Dict{String,Any}( "x" => 1, "y" => big_array, "z" => big_string, ) for i in 1:10 ) large_dict_2["cell_5"]["y"] = [2,20] delete!(large_dict_2, "cell_2") large_dict_2["hello"] = Dict("a" => 1, "b" => 2) large_dict_2 end;  =# #  43c36ab7-e9ac-450a-8abe-435412f2be1d #= diff(large_dict_1, large_dict_2)  =# #  1cf22fe6-4b58-4220-87a1-d7a18410b4e8 #  skip_as_script = true #= md""" With `===` comparison for arrays: """  =# #  ffb01ab4-e2e3-4fa4-8c0b-093d2899a536 #  skip_as_script = true #= md""" ### Large Dict example 2 """  =# #  8188de75-ae6e-48aa-9495-111fd27ffd26 #  skip_as_script = true #= many_items_1 = Dict{String,Any}( "cell_$(i)" => Dict{String,Any}( "x" => 1, "y" => [2,3], "z" => "four", ) for i in 1:100 )  =# #  fdc427f0-dfe8-4114-beca-48fc15434534 #= @test isempty(diff(many_items_1, many_items_1))  =# #  d807195e-ba27-4015-92a7-c9294d458d47 #= begin many_items_2 = deepcopy(many_items_1) many_items_2["cell_5"]["y"][2] = 20 delete!(many_items_2, "cell_2") many_items_2["hello"] = Dict("a" => 1, "b" => 2) many_items_2 end  =# #  2e91a1a2-469c-4123-a0d7-3dcc49715738 #= diff(many_items_1, many_items_2)  =# #  b8061c1b-dd03-4cd1-b275-90359ae2bb39 fairly_equal(a,b) = Set(a) == Set(b) #  2983f6d4-c1ca-4b66-a2d3-f858b0df2b4c #= @test fairly_equal(diff(large_dict_1, large_dict_2), [ ReplacePatch(["cell_5","y"], [2,20]), RemovePatch(["cell_2"]), AddPatch(["hello"], Dict("b" => 2, "a" => 1)), ])  =# #  61b81430-d26e-493c-96da-b6818e58c882 #= @test fairly_equal(diff(many_items_1, many_items_2), [ ReplacePatch(["cell_5","y"], [2,20]), RemovePatch(["cell_2"]), AddPatch(["hello"], Dict("b" => 2, "a" => 1)), ])  =# #  aeab3363-08ba-47c2-bd33-04a004ed72c4 #= diff(many_items_1, many_items_1)  =# #  62de3e79-4b4e-41df-8020-769c3c255c3e #= @test isempty(diff(many_items_1, many_items_1))  =# #  c7de406d-ccfe-41cf-8388-6bd2d7c42d64 #  skip_as_script = true #= md"### Struct example"  =# #  b9cc11ae-394b-44b9-bfbe-541d7720ead0 #  skip_as_script = true #= struct Cell id code folded end  =# #  c3c675be-9178-4176-afe0-30501786b72c #= deep_diff(old::Cell, new::Cell) = diff(Deep(old), Deep(new))  =# #  02585c72-1d92-4526-98c2-1ca07aad87a3 #= function direct_diff(old::Cell, new::Cell) changes = [] if old.id  new.id push!(changes, ReplacePatch([:id], new.id)) end if old.code  new.code push!(changes, ReplacePatch([:code], new.code)) end if old.folded  new.folded push!(changes, ReplacePatch([:folded], new.folded)) end changes end  =# #  2d084dd1-240d-4443-a8a2-82ae6e0b8900 #  skip_as_script = true #= cell1 = Cell(1, 2, 3)  =# #  3e05200f-071a-4ebe-b685-ff980f07cde7 #  skip_as_script = true #= cell2 = Cell(1, 2, 4)  =# #  dd312598-2de1-11eb-144c-f92ed6484f5d #  skip_as_script = true #= md"## Update"  =# #  d2af2a4b-8982-4e43-9fd7-0ecfdfb70511 const strict_applypatch = Ref(false) #  640663fc-06ba-491e-bd85-299514237651 begin function force_convert_key(::Dict{T,<:Any}, value::T) where T value end function force_convert_key(::Dict{T,<:Any}, value::Any) where T T(value) end end #  48a45941-2489-4666-b4e5-88d3f82e5145 function getpath(value, path) if length(path) == 0 return value end current, rest... = path if value isa AbstractDict key = force_convert_key(value, current) getpath(getindex(value, key), rest) else getpath(getproperty(value, Symbol(current)), rest) end end #  752b2da3-ff24-4758-8843-186368069888 function applypatch!(value, patches::Array{JSONPatch}) for patch in patches applypatch!(value, patch) end return value end #  3e285076-1d97-4728-87cf-f71b22569e57 #  skip_as_script = true #= md"### applypatch! AddPatch"  =# #  d7ea6052-9d9f-48e3-92fb-250afd69e417 begin _convert(::Type{Base.UUID}, s::String) = Base.UUID(s) _convert(::Type{T}, a::AbstractArray) where {T<:Array} = _convert.(eltype(T), a) _convert(x, y) = convert(x, y) function _convert(::Type{<:Dict}, patch::ReplacePatch) Dict{String,Any}("op" => "replace", "path" => patch.path, "value" => patch.value) end function _setproperty!(x, f::Symbol, v) type = fieldtype(typeof(x), f) return setfield!(x, f, _convert(type, v)) end end #  7feeee3a-3aec-47ce-b8d7-74a0d9b0b381 #  skip_as_script = true #= _convert(Dict, ReplacePatch([:x, :y], 10))  =# #  dd87ca7e-2de1-11eb-2ec3-d5721c32f192 function applypatch!(value, patch::AddPatch) if length(patch.path) == 0 throw("Impossible") else last = patch.path[end] rest = patch.path[begin:end - 1] subvalue = getpath(value, rest) if subvalue isa AbstractDict key = force_convert_key(subvalue, last) if strict_applypatch[] @assert get(subvalue, key, nothing) === nothing end subvalue[key] = patch.value else key = Symbol(last) if strict_applypatch[] @assert getproperty(subvalue, key) === nothing end _setproperty!(subvalue, key, patch.value) end end return value end #  a11e4082-4ff4-4c1b-9c74-c8fa7dcceaa6 #  skip_as_script = true #= md"*Should throw in strict mode:*"  =# #  be6b6fc4-e12a-4cef-81d8-d5115fda50b7 #  skip_as_script = true #= md"### applypatch! ReplacePatch"  =# #  6509d62e-77b6-499c-8dab-4a608e44720a function applypatch!(value, patch::ReplacePatch) if length(patch.path) == 0 throw("Impossible") else last = patch.path[end] rest = patch.path[begin:end - 1] subvalue = getpath(value, rest) if subvalue isa AbstractDict key = force_convert_key(subvalue, last) if strict_applypatch[] @assert get(subvalue, key, nothing) !== nothing end subvalue[key] = patch.value else key = Symbol(last) if strict_applypatch[] @assert getproperty(subvalue, key) !== nothing end _setproperty!(subvalue, key, patch.value) end end return value end #  f1dde1bd-3fa4-48b7-91ed-b2f98680fcc1 #  skip_as_script = true #= md"*Should throw in strict mode:*"  =# #  f3ef354b-b480-4b48-8358-46dbf37e1d95 #  skip_as_script = true #= md"### applypatch! RemovePatch"  =# #  ddaf5b66-2de1-11eb-3348-b905b94a984b function applypatch!(value, patch::RemovePatch) if length(patch.path) == 0 throw("Impossible") else last = patch.path[end] rest = patch.path[begin:end - 1] subvalue = getpath(value, rest) if subvalue isa AbstractDict key = force_convert_key(subvalue, last) if strict_applypatch[] @assert get(subvalue, key, nothing) !== nothing end delete!(subvalue, key) else key = Symbol(last) if strict_applypatch[] @assert getproperty(subvalue, key) !== nothing end _setproperty!(subvalue, key, nothing) end end return value end #  e65d483a-4c13-49ba-bff1-1d54de78f534 #= let dict_1_copy = deepcopy(dict_1) applypatch!(dict_1_copy, example_patches) end  =# #  595fdfd4-3960-4fbd-956c-509c4cf03473 #= @test applypatch!(deepcopy(notebook1), notebook1_to_notebook2) == notebook2  =# #  c3e4738f-4568-4910-a211-6a46a9d447ee #= @test applypatch!(Dict(:y => "x"), AddPatch([:x], "-")) == Dict(:y => "x", :x => "-")  =# #  0f094932-10e5-40f9-a3fc-db27a85b4999 #= @test applypatch!(Dict(:x => "x"), AddPatch([:x], "-")) == Dict(:x => "-")  =# #  a560fdca-ee12-469c-bda5-62d7203235b8 #= @test applypatch!(Dict(:x => "x"), ReplacePatch([:x], "-")) == Dict(:x => "-")  =# #  01e3417e-334e-4a8d-b086-4bddc42737b3 #= @test applypatch!(Dict(:y => "x"), ReplacePatch([:x], "-")) == Dict(:x => "-", :y => "x")  =# #  96a80a23-7c56-4c41-b489-15bc1c4e3700 #= @test applypatch!(Dict(:x => "x"), RemovePatch([:x])) == Dict()  =# #  df41caa7-f0fc-4b0d-ab3d-ebdab4804040 #  skip_as_script = true #= md"*Should throw in strict mode:*"  =# #  fac65755-2a2a-4a3c-b5a8-fc4f6d256754 #= @test applypatch!(Dict(:y => "x"), RemovePatch([:x])) == Dict(:y => "x")  =# #  e55d1cea-2de1-11eb-0d0e-c95009eedc34 #  skip_as_script = true #= md"## Testing"  =# #  b05fcb88-3781-45d0-9f24-e88c339a72e5 #  skip_as_script = true #= macro test2(expr) quote nothing end end  =# #  e7e8d076-2de1-11eb-0214-8160bb81370a #= @test notebook1 == deepcopy(notebook1)  =# #  ee70e282-36d5-4772-8585-f50b9a67ca54 #  skip_as_script = true #= md"## Track"  =# #  a3e8fe70-cbf5-4758-a0f2-d329d138728c #  skip_as_script = true #= function prettytime(time_ns::Number) suffices = ["ns", "s", "ms", "s"] current_amount = time_ns suffix = "" for current_suffix in suffices if current_amount >= 1000.0 current_amount = current_amount / 1000.0 else suffix = current_suffix break end end # const roundedtime = time_ns.toFixed(time_ns >= 100.0 ? 0 : 1) roundedtime = if current_amount >= 100.0 round(current_amount; digits=0) else round(current_amount; digits=1) end return "$(roundedtime) $(suffix)" end  =# #  0e1c6442-9040-49d9-b754-173583db7ba2 #  skip_as_script = true #= begin Base.@kwdef struct Tracked expr value time bytes times_ran = 1 which = nothing code_info = nothing end function Base.show(io::IO, mime::MIME"text/html", value::Tracked) times_ran = if value.times_ran === 1 "" else """<span style="opacity: 0.5"> ($(value.times_ran))</span>""" end # method = sprint(show, MIME("text/plain"), value.which) code_info = if value.code_info  nothing codelength = length(value.code_info.first.code) "$(codelength) frames in @code_typed" else "" end color = if value.time > 1 "red" elseif value.time > 0.001 "orange" elseif value.time > 0.0001 "blue" else "green" end show(io, mime, HTML(""" <div style=" display: flex; flex-direction: row; align-items: center; " > <div style=" width: 12px; height: 12px; border-radius: 50%; background-color: $(color); " ></div> <div style="width: 12px"></div> <div> <code class="language-julia" style=" background-color: transparent; filter: grayscale(1) brightness(0.8); " >$(value.expr)</code> <div style=" font-family: monospace; font-size: 12px; color: $(color); "> $(prettytime(value.time * 1e9 / value.times_ran)) $(times_ran) </div> <div style=" font-family: monospace; font-size: 12px; color: gray; ">$(code_info)</div> </div> </div> """)) end Tracked end  =# #  7618aef7-1884-4e32-992d-0fd988e1ab20 #  skip_as_script = true #= macro track(expr) times_ran_expr = :(1) expr_to_show = expr if expr.head == :for @assert expr.args[1].head == :(=) times_ran_expr = expr.args[1].args[2] expr_to_show = expr.args[2].args[2] end Tracked # reference so that baby Pluto understands quote local times_ran = length($(esc(times_ran_expr))) local value, time, bytes = @timed $(esc(expr)) local method = nothing local code_info = nothing try # Uhhh method = @which $(expr_to_show) code_info = @code_typed $(expr_to_show) catch nothing end Tracked( expr=$(QuoteNode(expr_to_show)), value=value, time=time, bytes=bytes, times_ran=times_ran, which=method, code_info=code_info ) end end  =# #  db99e5c9-d8cd-4314-90e2-99976e81283d #= @track example_patches[1] == example_patches[2]  =# #  2b342873-916c-4efb-b62c-b46a96bd6a2d #= @track example_patches[2] == example_patches[2]  =# #  7b8ab89b-bf56-4ddf-b220-b4881f4a2050 #= @track Base.convert(JSONPatch, convert(Dict, add_patch)) == add_patch  =# #  48ccd28a-060d-4214-9a39-f4c4e506d1aa #= @track Base.convert(JSONPatch, convert(Dict, remove_patch)) == remove_patch  =# #  34d86e02-dd34-4691-bb78-3023568a5d16 #= @track Base.convert(JSONPatch, _convert(Dict, replace_patch)) == replace_patch  =# #  95ff676d-73c8-44cb-ac35-af94418737e9 #= @track for _ in 1:100 diff(celldict1, celldict2) end  =# #  8c069015-d922-4c60-9340-8d65c80b1a06 #= @track for _ in 1:1000 diff(large_dict_1, large_dict_1) end  =# #  bc9a0822-1088-4ee7-8c79-98e06fd50f11 #= @track for _ in 1:1000 diff(large_dict_1, large_dict_2) end  =# #  ddf1090c-5239-41df-ae4d-70aeb3a75f2b #= let old = use_triple_equals_for_arrays[] use_triple_equals_for_arrays[] = true result = @track for _ in 1:1000 diff(large_dict_1, large_dict_1) end use_triple_equals_for_arrays[] = old result end  =# #  88009db3-f40e-4fd0-942a-c7f4a7eecb5a #= let old = use_triple_equals_for_arrays[] use_triple_equals_for_arrays[] = true result = @track for _ in 1:1000 diff(large_dict_1, large_dict_2) end use_triple_equals_for_arrays[] = old result end  =# #  c287009f-e864-45d2-a4d0-a525c988a6e0 #= @track for _ in 1:1000 diff(many_items_1, many_items_1) end  =# #  67a1ae27-f7df-4f84-8809-1cc6a9bcd1ce #= @track for _ in 1:1000 diff(many_items_1, many_items_2) end  =# #  fa959806-3264-4dd5-9f94-ba369697689b #= @track for _ in 1:1000 direct_diff(cell2, cell1) end  =# #  a9088341-647c-4fe1-ab85-d7da049513ae #= @track for _ in 1:1000 diff(Deep(cell1), Deep(cell2)) end  =# #  1a26eed8-670c-43bf-9726-2db84b1afdab #= @track sleep(0.1)  =# #  Cell order: # d948dc6e-2de1-11eb-19e7-cb3bb66353b6 # 1a6e1853-6db1-4074-bce0-5f274351cece # 49fc1f97-3b8f-4297-94e5-2e24c001d35c # d8e73b90-24c5-4e50-830b-b1dbe6224c8e # 19646596-b35b-44fa-bfcf-891f9ffb748c # 7ca087b8-73ac-49ea-9c5a-2971f0da491f # 9d2c07d9-16a9-4b9f-a375-2adb6e5b907a # e65d483a-4c13-49ba-bff1-1d54de78f534 # 336bfd4f-8a8e-4a2d-be08-ee48d6a9f747 # db116c0a-2de1-11eb-2a56-872af797c547 # bd0d46bb-3e58-4522-bae0-83eb799196c4 # db2d8a3e-2de1-11eb-02b8-9ffbfaeff61c # ffe9b3d9-8e35-4a31-bab2-8787a4140594 # 894de8a7-2757-4d7a-a2be-1069fa872911 # 9a364714-edb1-4bca-9387-a8bbacccd10d # 9321d3be-cb91-4406-9dc7-e5c38f7d377c # 73631aea-5e93-4da2-a32d-649029660d4e # 0fd3e910-abcc-4421-9d0b-5cfb90034338 # aad7ab32-eecf-4aad-883d-1c802cad6c0c # 732fd744-acdb-4507-b1de-6866ec5563dd # 17606cf6-2d0f-4245-89a3-746ad818a664 # c7ac7d27-7bf9-4209-8f3c-e4d52c543e29 # 042f7788-e996-430e-886d-ffb4f70dea9e # 9d2dde5c-d404-4fbc-b8e0-5024303c8052 # f649f67c-aab0-4d35-a799-f398e5f3ecc4 # 63087738-d70c-46f5-b072-21cd8953df35 # aa81974a-7254-45e0-9bfe-840c4793147f # 31188a03-76ba-40cf-a333-4d339ce37711 # 7524a9e8-1a6d-4851-b50e-19415f25a84b # db99e5c9-d8cd-4314-90e2-99976e81283d # 2b342873-916c-4efb-b62c-b46a96bd6a2d # 5ddfd616-db20-451b-bc1e-2ad52e0e2777 # 24e93923-eab9-4a7b-9bc7-8d8a1209a78f # 09ddf4d9-5ccb-4530-bfab-d11b864e872a # d9e764db-94fc-44f7-8c2e-3d63f4809617 # 99df99ad-aad5-4275-97d4-d1ceeb2f8d15 # 2d665639-7274-495a-ae9d-f358a8219bb7 # f658a72d-871d-49b3-9b73-7efedafbd7a6 # 230bafe2-aaa7-48f0-9fd1-b53956281684 # 07eeb122-6706-4544-a007-1c8d6581eec8 # b48e2c08-a94a-4247-877d-949d92dde626 # c59b30b9-f702-41f1-bb2e-1736c8cd5ede # 7feeee3a-3aec-47ce-b8d7-74a0d9b0b381 # 921a130e-b028-4f91-b077-3bd79dcb6c6d # 6d67f8a5-0e0c-4b6e-a267-96b34d580946 # 7b8ab89b-bf56-4ddf-b220-b4881f4a2050 # 56b28842-4a67-44d7-95e7-55d457a44fb1 # 48ccd28a-060d-4214-9a39-f4c4e506d1aa # f10e31c0-1d2c-4727-aba5-dd676a10041b # 34d86e02-dd34-4691-bb78-3023568a5d16 # 3a99e22d-42d6-4b2d-9381-022b41b0e852 # 831d84a6-1c71-4e68-8c7c-27d9093a82c4 # 2ad11c73-4691-4283-8f98-3d2a87926b99 # 5513ea3b-9498-426c-98cb-7dc23d32f72e # 0c2d6da1-cad3-4c9f-93e9-922457083945 # 84c87031-7733-4d1f-aa90-f8ab71506251 # 8f265a33-3a2d-4508-9477-ca62e8ce3c12 # daf9ec12-2de1-11eb-3a8d-59d9c2753134 # 0b50f6b2-8e85-4565-9f04-f99c913b4592 # 59e94cb2-c2f9-4f6c-9562-45e8c15931af # c9d5d81c-b0b6-4d1a-b1de-96d3b3701700 # 24389a0a-c3ac-4438-9dfe-1d14cd033d25 # 9cbaaec2-709c-4769-886c-ec92b12c18bc # db75df12-2de1-11eb-0726-d1995cebd382 # dbc7f97a-2de1-11eb-362f-055a734d1a9e # 67ade214-2de3-11eb-291d-135a397d629b # b8c58aa4-c24d-48a3-b2a8-7c01d50a3349 # 5ab390f9-3b0c-4978-9e21-2aaa61db2ce4 # 09f53db0-21ae-490b-86b5-414eba403d57 # 59b46bfe-da74-43af-9c11-cb0bdb2c13a2 # 200516da-8cfb-42fe-a6b9-cb4730168923 # 76326e6c-b95a-4b2d-a78c-e283e5fadbe2 # 95ff676d-73c8-44cb-ac35-af94418737e9 # 664cd334-91c7-40dd-a2bf-0da720307cfc # b7fa5625-6178-4da8-a889-cd4f014f43ba # dbdd1df0-2de1-11eb-152f-8d1af1ad02fe # 595fdfd4-3960-4fbd-956c-509c4cf03473 # 3924953f-787a-4912-b6ee-9c9d3030f0f0 # 80689881-1b7e-49b2-af97-9e3ab639d006 # fd22b6af-5fd2-428a-8291-53e223ea692c # bcd5059b-b0d2-49d8-a756-92349aa56aca # e7fd6bab-c114-4f3e-b9ad-1af2d1147770 # 43c36ab7-e9ac-450a-8abe-435412f2be1d # 2983f6d4-c1ca-4b66-a2d3-f858b0df2b4c # fdc427f0-dfe8-4114-beca-48fc15434534 # 8c069015-d922-4c60-9340-8d65c80b1a06 # bc9a0822-1088-4ee7-8c79-98e06fd50f11 # 1cf22fe6-4b58-4220-87a1-d7a18410b4e8 # ddf1090c-5239-41df-ae4d-70aeb3a75f2b # 88009db3-f40e-4fd0-942a-c7f4a7eecb5a # ffb01ab4-e2e3-4fa4-8c0b-093d2899a536 # 8188de75-ae6e-48aa-9495-111fd27ffd26 # d807195e-ba27-4015-92a7-c9294d458d47 # 2e91a1a2-469c-4123-a0d7-3dcc49715738 # 61b81430-d26e-493c-96da-b6818e58c882 # b8061c1b-dd03-4cd1-b275-90359ae2bb39 # aeab3363-08ba-47c2-bd33-04a004ed72c4 # 62de3e79-4b4e-41df-8020-769c3c255c3e # c287009f-e864-45d2-a4d0-a525c988a6e0 # 67a1ae27-f7df-4f84-8809-1cc6a9bcd1ce # c7de406d-ccfe-41cf-8388-6bd2d7c42d64 # b9cc11ae-394b-44b9-bfbe-541d7720ead0 # c3c675be-9178-4176-afe0-30501786b72c # 02585c72-1d92-4526-98c2-1ca07aad87a3 # 2d084dd1-240d-4443-a8a2-82ae6e0b8900 # 3e05200f-071a-4ebe-b685-ff980f07cde7 # fa959806-3264-4dd5-9f94-ba369697689b # a9088341-647c-4fe1-ab85-d7da049513ae # dd312598-2de1-11eb-144c-f92ed6484f5d # d2af2a4b-8982-4e43-9fd7-0ecfdfb70511 # 640663fc-06ba-491e-bd85-299514237651 # 48a45941-2489-4666-b4e5-88d3f82e5145 # 752b2da3-ff24-4758-8843-186368069888 # 3e285076-1d97-4728-87cf-f71b22569e57 # d7ea6052-9d9f-48e3-92fb-250afd69e417 # dd87ca7e-2de1-11eb-2ec3-d5721c32f192 # c3e4738f-4568-4910-a211-6a46a9d447ee # a11e4082-4ff4-4c1b-9c74-c8fa7dcceaa6 # 0f094932-10e5-40f9-a3fc-db27a85b4999 # be6b6fc4-e12a-4cef-81d8-d5115fda50b7 # 6509d62e-77b6-499c-8dab-4a608e44720a # a560fdca-ee12-469c-bda5-62d7203235b8 # f1dde1bd-3fa4-48b7-91ed-b2f98680fcc1 # 01e3417e-334e-4a8d-b086-4bddc42737b3 # f3ef354b-b480-4b48-8358-46dbf37e1d95 # ddaf5b66-2de1-11eb-3348-b905b94a984b # 96a80a23-7c56-4c41-b489-15bc1c4e3700 # df41caa7-f0fc-4b0d-ab3d-ebdab4804040 # fac65755-2a2a-4a3c-b5a8-fc4f6d256754 # e55d1cea-2de1-11eb-0d0e-c95009eedc34 # 3e07f976-6cd0-4841-9762-d40337bb0645 # e748600a-2de1-11eb-24be-d5f0ecab8fa4 # b05fcb88-3781-45d0-9f24-e88c339a72e5 # e7e8d076-2de1-11eb-0214-8160bb81370a # ee70e282-36d5-4772-8585-f50b9a67ca54 # 1a26eed8-670c-43bf-9726-2db84b1afdab # 0e1c6442-9040-49d9-b754-173583db7ba2 # 7618aef7-1884-4e32-992d-0fd988e1ab20 # a3e8fe70-cbf5-4758-a0f2-d329d138728c ### A Pluto.jl notebook ### # v0.19.9 using Markdown using InteractiveUtils #  092c4b11-8b75-446f-b3ad-01fa858daebb #  show_logs = false #  skip_as_script = true #= # Only define this in Pluto using skip_as_script = true begin import Pkg Pkg.activate(mktempdir()) Pkg.add(Pkg.PackageSpec(name="PlutoTest")) using PlutoTest end  =# #  058a3333-0567-43b7-ac5f-1f6688325a08 begin """ Mark an instance of a custom struct as immutable. The resulting object is also an `AbstractDict`, where the keys are the struct fields (converted to strings). """ struct ImmutableMarker{T} <: AbstractDict{String,Any} source::T end function Base.getindex(ldict::ImmutableMarker, key::String) Base.getfield(ldict.source, Symbol(key)) end # disabled because it's immutable! # Base.setindex!(ldict::ImmutableMarker, args...) = Base.setindex!(ldict.source, args...) # Base.delete!(ldict::ImmutableMarker, args...) = Base.delete!(ldict.source, args...) Base.keys(ldict::ImmutableMarker{T}) where T = String.(fieldnames(T)) # Base.values(ldict::ImmutableMarker) = Base.values(ldict.source) Base.length(ldict::ImmutableMarker) = nfields(ldict.source) Base.iterate(ldict::ImmutableMarker) = Base.iterate(ldict, 1) function Base.iterate(ldict::ImmutableMarker{T}, i) where T a = ldict.source if i <= nfields(a) name = fieldname(T, i) (String(name) => getfield(a, name), i + 1) end end function Base.show(io::IO, t::ImmutableMarker) print(io, typeof(t), "(") show(io, t.source) print(io, ")") end end #  55975e53-f70f-4b70-96d2-b144f74e7cde #  skip_as_script = true #= struct A x y z end  =# #  d7e0de85-5cb2-4036-a2e3-ca416ea83737 #= id1 = ImmutableMarker(A(1,"asdf",3))  =# #  08350326-526e-4c34-ab27-df9fbf69243e #= id2 = ImmutableMarker(A(1,"asdf",4))  =# #  aa6192e8-410f-4924-8250-4775e21b1590 #= id1d, id2d = Dict(id1), Dict(id2)  =# #  273c7c85-8178-44a7-99f0-581754aeb8c8 begin """ Mark a vector as being append-only: let Firebasey know that it can diff this array simply by comparing lengths, without looking at its contents. It was made specifically for logs: Logs are always appended, OR the whole log stream is reset. AppendonlyMarker is like SubArray (a view into another array) except we agree to only ever append to the source array. This way, firebase can just look at the index and diff based on that. """ struct AppendonlyMarker{T} <: AbstractVector{T} mutable_source::Vector{T} length_at_time_of_creation::Int64 end AppendonlyMarker(arr::Vector) = AppendonlyMarker(arr, length(arr)) # We use a view here to ensure that if the source array got appended after creation, those newer elements are not accessible. _contents(a::AppendonlyMarker) = view(a.mutable_source, 1:a.length_at_time_of_creation) # Poor mans vector-proxy # I think this is enough for Pluto to show, and for msgpack to pack Base.size(arr::AppendonlyMarker) = Base.size(_contents(arr)) Base.getindex(arr::AppendonlyMarker, index::Int) = Base.getindex(_contents(arr), index) Base.iterate(arr::AppendonlyMarker, args...) = Base.iterate(_contents(arr), args...) end #  ef7032d1-a666-48a6-a56e-df175f5ed832 #  skip_as_script = true #= md""" ## ImmutableMarker """  =# #  183cef1f-bfe9-42cd-8239-49e9ed00a7b6 #  skip_as_script = true #= md""" ## AppendonlyMarker(s) Example of how to solve performance problems with Firebasey: We make a new type with a specific diff function. It might be very specific per problem, but that's fine for performance problems (I think). It also keeps the performance solutions as separate modules/packages to whatever it is you're actually modeling. """  =# #  35d3bcd7-af51-466a-b4c4-cc055e74d01d #  skip_as_script = true #= appendonly_1, appendonly_2 = let array_1 = [1,2,3,4] appendonly_1 = AppendonlyMarker(array_1) push!(array_1, 5) appendonly_2 = AppendonlyMarker(array_1) appendonly_1, appendonly_2 end;  =# #  1017f6cc-58ac-4c7b-a6d0-a03f5e387f1b #  skip_as_script = true #= appendonly_1_large, appendonly_2_large = let large_array_1 = [ Dict{String,Any}( "x" => 1, "y" => [1,2,3,4], "z" => "hi", ) for i in 1:10000 ]; appendonly_1 = AppendonlyMarker(large_array_1) push!(large_array_1, Dict("x" => 5)) appendonly_2 = AppendonlyMarker(large_array_1) appendonly_1, appendonly_2 end;  =# #  06492e8d-4500-4efe-80ee-55bf1ee2348c #= @test length([AppendonlyMarker([1,2,3])...]) == 3  =# #  2284ae12-5b8c-4542-81fa-c4d34f2483e7 # @test length([AppendonlyMarker([1,2,3], 1)...]) == 1 #  dc5cd268-9cfb-49bf-87fb-5b7db4fa6e3c #  skip_as_script = true #= md"## Import Firebasey when running inside notebook"  =# #  0c2f23d8-8e98-47b7-9c4f-5daa70a6c7fb # OH how I wish I would put in the time to refactor with fromFile or SOEMTGHINLAS LDKJ JULIA WHY ARE YOU LIKE THIS GROW UP if !@isdefined(Firebasey) Firebasey = let wrapper_module = Module() Core.eval(wrapper_module, :(module Firebasey include("Firebasey.jl") end )) wrapper_module.Firebasey end end #  2903d17e-c6fd-4cea-8585-4db26a00b0e7 function Firebasey.diff(a::AppendonlyMarker, b::AppendonlyMarker) if a.mutable_source !== b.mutable_source [Firebasey.ReplacePatch([], b)] else if a.length_at_time_of_creation > b.length_at_time_of_creation throw(ErrorException("Not really supposed to diff AppendonlyMarker with the original being longer than the next version (you know, 'append only' and al)")) end map(a.length_at_time_of_creation+1:b.length_at_time_of_creation) do index Firebasey.AddPatch([index], b.mutable_source[index]) end end end #  129dee79-61c0-4524-9bef-388837f035bb function Firebasey.diff(a::ImmutableMarker, b::ImmutableMarker) if a.source !== b.source Firebasey.diff(Dict(a), Dict(b)) # Firebasey.JSONPatch[Firebasey.ReplacePatch([], b)] else Firebasey.JSONPatch[] end end #  138d2cc2-59ba-4f76-bf66-ecdb98cf4fd5 #= Firebasey.diff(id1, id2)  =# #  8537488d-2ff9-42b7-8bfc-72d43fca713f #= @test Firebasey.diff(appendonly_1, appendonly_2) == [Firebasey.AddPatch([5], 5)]  =# #  721e3c90-15ae-43f2-9234-57b38e3e6b69 #  skip_as_script = true #= md""" ## Track """  =# #  e830792c-c809-4fde-ae55-8ae01b4c04b9 #  skip_as_script = true #= function prettytime(time_ns::Number) suffices = ["ns", "s", "ms", "s"] current_amount = time_ns suffix = "" for current_suffix in suffices if current_amount >= 1000.0 current_amount = current_amount / 1000.0 else suffix = current_suffix break end end # const roundedtime = time_ns.toFixed(time_ns >= 100.0 ? 0 : 1) roundedtime = if current_amount >= 100.0 round(current_amount; digits=0) else round(current_amount; digits=1) end return "$(roundedtime) $(suffix)" end  =# #  16b03608-0f5f-421a-bab4-89365528b0b4 #  skip_as_script = true #= begin Base.@kwdef struct Tracked expr value time bytes times_ran = 1 which = nothing code_info = nothing end function Base.show(io::IO, mime::MIME"text/html", value::Tracked) times_ran = if value.times_ran === 1 "" else """<span style="opacity: 0.5"> ($(value.times_ran))</span>""" end # method = sprint(show, MIME("text/plain"), value.which) code_info = if value.code_info  nothing codelength = length(value.code_info.first.code) "$(codelength) frames in @code_typed" else "" end color = if value.time > 1 "red" elseif value.time > 0.001 "orange" elseif value.time > 0.0001 "blue" else "green" end show(io, mime, HTML(""" <div style=" display: flex; flex-direction: row; align-items: center; " > <div style=" width: 12px; height: 12px; border-radius: 50%; background-color: $(color); " ></div> <div style="width: 12px"></div> <div> <code class="language-julia" style=" background-color: transparent; filter: grayscale(1) brightness(0.8); " >$(value.expr)</code> <div style=" font-family: monospace; font-size: 12px; color: $(color); "> $(prettytime(value.time * 1e9 / value.times_ran)) $(times_ran) </div> <div style=" font-family: monospace; font-size: 12px; color: gray; ">$(code_info)</div> </div> </div> """)) end Tracked end  =# #  875fd249-37cc-49da-8a7d-381fe0e21063 #= macro track(expr) times_ran_expr = :(1) expr_to_show = expr if expr.head == :for @assert expr.args[1].head == :(=) times_ran_expr = expr.args[1].args[2] expr_to_show = expr.args[2].args[2] end Tracked # reference so that baby Pluto understands quote local times_ran = length($(esc(times_ran_expr))) local value, time, bytes = @timed $(esc(expr)) local method = nothing local code_info = nothing try # Uhhh method = @which $(expr_to_show) code_info = @code_typed $(expr_to_show) catch nothing end Tracked( expr=$(QuoteNode(expr_to_show)), value=value, time=time, bytes=bytes, times_ran=times_ran, which=method, code_info=code_info ) end end  =# #  a5f43f47-6189-413f-95a0-d98f927bb7ce #= @track for _ in 1:1000 Firebasey.diff(id1, id1) end  =# #  ab5089cc-fec8-43b9-9aa4-d6fa96e231e0 #= @track for _ in 1:1000 Firebasey.diff(id1d, id1d) end  =# #  a84dcdc3-e9ed-4bf5-9bec-c9cbfc267c17 #= @track for _ in 1:1000 Firebasey.diff(id1, id2) end  =# #  f696bb85-0bbd-43c9-99ea-533816bc8e0d #= @track for _ in 1:1000 Firebasey.diff(id1d, id2d) end  =# #  37fe8c10-09f0-4f72-8cfd-9ce044c78c13 #= @track for _ in 1:1000 Firebasey.diff(appendonly_1_large, appendonly_2_large) end  =# #  9862ee48-48a0-4178-8ec4-306792827e17 #= @track sleep(0.1)  =# #  Cell order: # ef7032d1-a666-48a6-a56e-df175f5ed832 # 058a3333-0567-43b7-ac5f-1f6688325a08 # 129dee79-61c0-4524-9bef-388837f035bb # 55975e53-f70f-4b70-96d2-b144f74e7cde # d7e0de85-5cb2-4036-a2e3-ca416ea83737 # 08350326-526e-4c34-ab27-df9fbf69243e # 138d2cc2-59ba-4f76-bf66-ecdb98cf4fd5 # aa6192e8-410f-4924-8250-4775e21b1590 # a5f43f47-6189-413f-95a0-d98f927bb7ce # ab5089cc-fec8-43b9-9aa4-d6fa96e231e0 # a84dcdc3-e9ed-4bf5-9bec-c9cbfc267c17 # f696bb85-0bbd-43c9-99ea-533816bc8e0d # 183cef1f-bfe9-42cd-8239-49e9ed00a7b6 # 273c7c85-8178-44a7-99f0-581754aeb8c8 # 2903d17e-c6fd-4cea-8585-4db26a00b0e7 # 35d3bcd7-af51-466a-b4c4-cc055e74d01d # 1017f6cc-58ac-4c7b-a6d0-a03f5e387f1b # 06492e8d-4500-4efe-80ee-55bf1ee2348c # 2284ae12-5b8c-4542-81fa-c4d34f2483e7 # 8537488d-2ff9-42b7-8bfc-72d43fca713f # 37fe8c10-09f0-4f72-8cfd-9ce044c78c13 # dc5cd268-9cfb-49bf-87fb-5b7db4fa6e3c # 0c2f23d8-8e98-47b7-9c4f-5daa70a6c7fb # 092c4b11-8b75-446f-b3ad-01fa858daebb # 721e3c90-15ae-43f2-9234-57b38e3e6b69 # 9862ee48-48a0-4178-8ec4-306792827e17 # 16b03608-0f5f-421a-bab4-89365528b0b4 # 875fd249-37cc-49da-8a7d-381fe0e21063 # e830792c-c809-4fde-ae55-8ae01b4c04b9 # this file is just a bunch of ugly code to make sure that the Julia Array{UInt8,1} becomes a JS Uint8Array() instead of a normal array. This improves performance of the client. # ignore it if you are not interested in that kind of stuff import Dates import UUIDs: UUID import MsgPack import .Configuration import Pkg # MsgPack.jl doesn't define a serialization method for MIME and UUID objects, so we write these ourselves: MsgPack.msgpack_type(::Type{<:MIME}) = MsgPack.StringType() MsgPack.msgpack_type(::Type{UUID}) = MsgPack.StringType() MsgPack.msgpack_type(::Type{VersionNumber}) = MsgPack.StringType() MsgPack.msgpack_type(::Type{Pkg.Types.VersionRange}) = MsgPack.StringType() MsgPack.to_msgpack(::MsgPack.StringType, m::MIME) = string(m) MsgPack.to_msgpack(::MsgPack.StringType, u::UUID) = string(u) MsgPack.to_msgpack(::MsgPack.StringType, v::VersionNumber) = string(v) MsgPack.to_msgpack(::MsgPack.StringType, v::Pkg.Types.VersionRange) = string(v) # Support for sending Dates MsgPack.msgpack_type(::Type{Dates.DateTime}) = MsgPack.ExtensionType() MsgPack.to_msgpack(::MsgPack.ExtensionType, d::Dates.DateTime) = let millisecs_since_1970_because_thats_how_computers_work = Dates.value(d - Dates.DateTime(1970)) MsgPack.Extension(0x0d, reinterpret(UInt8, [millisecs_since_1970_because_thats_how_computers_work])) end # Our Configuration types: MsgPack.msgpack_type(::Type{Configuration.Options}) = MsgPack.StructType() MsgPack.msgpack_type(::Type{Configuration.EvaluationOptions}) = MsgPack.StructType() MsgPack.msgpack_type(::Type{Configuration.CompilerOptions}) = MsgPack.StructType() MsgPack.msgpack_type(::Type{Configuration.ServerOptions}) = MsgPack.StructType() MsgPack.msgpack_type(::Type{Configuration.SecurityOptions}) = MsgPack.StructType() # Don't try to send callback functions which can't be serialized (see ServerOptions.event_listener) MsgPack.msgpack_type(::Type{Function}) = MsgPack.NilType() MsgPack.to_msgpack(::MsgPack.NilType, ::Function) = nothing # We want typed integer arrays to arrive as JS typed integer arrays: const JSTypedIntSupport = [Int8, UInt8, Int16, UInt16, Int32, UInt32, Float32, Float64] const JSTypedInt = Union{Int8,UInt8,Int16,UInt16,Int32,UInt32,Float32,Float64} MsgPack.msgpack_type(::Type{Vector{T}}) where T <: JSTypedInt = MsgPack.ExtensionType() function MsgPack.to_msgpack(::MsgPack.ExtensionType, x::Vector{T}) where T <: JSTypedInt type = findfirst(isequal(T), JSTypedIntSupport) + 0x10 MsgPack.Extension(type, reinterpret(UInt8, x)) end MsgPack.msgpack_type(::Type{Vector{Union{}}}) = MsgPack.ArrayType() # The other side does the same (/frontend/common/MsgPack.js), and we decode it here: function decode_extension_and_addbits(x::MsgPack.Extension) if x.type == 0x0d # the datetime type millisecs_since_1970_because_thats_how_computers_work = reinterpret(Int64, x.data)[1] Dates.DateTime(1970) + Dates.Millisecond(millisecs_since_1970_because_thats_how_computers_work) # TODO? Dates.unix2datetime does exactly this ?? - DRAL else # the array types julia_type = JSTypedIntSupport[x.type - 0x10] if eltype(x.data) == julia_type x.data else reinterpret(julia_type, x.data) end end end function decode_extension_and_addbits(x::Dict) # we mutate the dictionary, that's fine in our use case and saves memory? for (k, v) in x x[k] = decode_extension_and_addbits(v) end x end decode_extension_and_addbits(x::Array) = map(decode_extension_and_addbits, x) # We also convert everything (except the JS typed arrays) to 64 bit numbers, just to make it easier to work with. decode_extension_and_addbits(x::T) where T <: Union{Signed,Unsigned} = Int64(x) decode_extension_and_addbits(x::T) where T <: AbstractFloat = Float64(x) decode_extension_and_addbits(x::Any) = x function pack(args...) MsgPack.pack(args...) end function unpack(args...) MsgPack.unpack(args...) |> decode_extension_and_addbits end precompile(unpack, (Vector{UInt8},)) function serialize_message_to_stream(io::IO, message::UpdateMessage) to_send = Dict(:type => message.type, :message => message.message) if message.notebook !== nothing to_send[:notebook_id] = message.notebook.notebook_id end if message.cell !== nothing to_send[:cell_id] = message.cell.cell_id end if message.initiator !== nothing to_send[:initiator_id] = message.initiator.client_id to_send[:request_id] = message.initiator.request_id end pack(io, to_send) end function serialize_message(message::UpdateMessage) io = IOBuffer() serialize_message_to_stream(io, message) take!(io) end "Send `messages` to all clients connected to the `notebook`." function putnotebookupdates!(session::ServerSession, notebook::Notebook, messages::UpdateMessage...; flush::Bool=true) listeners = filter(collect(values(session.connected_clients))) do c c.connected_notebook !== nothing && c.connected_notebook.notebook_id == notebook.notebook_id end for next_to_send in messages, client in listeners put!(client.pendingupdates, next_to_send) end flush && flushallclients(session, listeners) listeners end "Send `messages` to all connected clients." function putplutoupdates!(session::ServerSession, messages::UpdateMessage...; flush::Bool=true) listeners = collect(values(session.connected_clients)) for next_to_send in messages, client in listeners put!(client.pendingupdates, next_to_send) end flush && flushallclients(session, listeners) listeners end "Send `messages` to a `client`." function putclientupdates!(client::ClientSession, messages::UpdateMessage...) for next_to_send in messages put!(client.pendingupdates, next_to_send) end flushclient(client) client end "Send `messages` to the `ClientSession` who initiated." function putclientupdates!(session::ServerSession, initiator::Initiator, messages::UpdateMessage...) # Prevent long, scary looking, error. if !haskey(session.connected_clients, initiator.client_id) @warn "Trying to send clientupdate to disconnected client." messages=map(x -> x.type, messages) return end putclientupdates!(session.connected_clients[initiator.client_id], messages...) end # https://github.com/JuliaWeb/HTTP.jl/issues/382 const flushtoken = Token() function send_message(stream::HTTP.WebSocket, msg) HTTP.send(stream, serialize_message(msg)) end function send_message(stream::IO, msg) write(stream, serialize_message(msg)) end function is_stream_open(stream::HTTP.WebSocket) !HTTP.WebSockets.isclosed(stream) end function is_stream_open(io::IO) isopen(io) end function flushclient(client::ClientSession) take!(flushtoken) while isready(client.pendingupdates) next_to_send = take!(client.pendingupdates) try if client.stream !== nothing if is_stream_open(client.stream) let lag = client.simulated_lag (lag > 0) && sleep(lag * (0.5 + rand())) # sleep(0) would yield to the process manager which we dont want end send_message(client.stream, next_to_send) else put!(flushtoken) return false end end catch ex bt = stacktrace(catch_backtrace()) if ex isa Base.IOError || (ex isa ArgumentError && occursin("closed", ex.msg)) # client socket closed, so we return false (about 5 lines after this one) else @warn "Failed to write to WebSocket of $(client.id) " exception = (ex, bt) end put!(flushtoken) return false end end put!(flushtoken) true end function flushallclients(session::ServerSession, subset::Union{Set{ClientSession},AbstractVector{ClientSession}}) disconnected = Set{Symbol}() for client in subset stillconnected = flushclient(client) if !stillconnected push!(disconnected, client.id) end end for to_delete_id in disconnected delete!(session.connected_clients, to_delete_id) end end function flushallclients(session::ServerSession) flushallclients(session, values(session.connected_clients)) end import Malt using Markdown import REPL ### # RESPONSES FOR AUTOCOMPLETE & DOCS ### function format_path_completion(completion) replace(REPL.REPLCompletions.completion_text(completion), "\\ " => " ", "\\\\" => "\\") end responses[:completepath] = function response_completepath(::ClientRequest) path = .body["query"] pos = lastindex(path) results, loc, found = REPL.REPLCompletions.complete_path(path, pos) ishidden(path_completion) = let p = path_completion.path startswith(basename(isdirpath(p) ? dirname(p) : p), ".") end filter!(!ishidden, results) start_utf8 = let # REPLCompletions takes into account that spaces need to be prefixed with `\` in the shell, so it subtracts the number of spaces in the filename from `start`: # https://github.com/JuliaLang/julia/blob/c54f80c785a3107ae411267427bbca05f5362b0b/stdlib/REPL/src/REPLCompletions.jl#L270 # we don't use prefixes, so we need to reverse this. # this is from the Julia source code: # https://github.com/JuliaLang/julia/blob/c54f80c785a3107ae411267427bbca05f5362b0b/stdlib/REPL/src/REPLCompletions.jl#L195-L204 if Base.Sys.isunix() && occursin(r"^~(?:/|$)", path) # if the path is just "~", don't consider the expanded username as a prefix if path == "~" dir, prefix = homedir(), "" else dir, prefix = splitdir(homedir() * path[2:end]) end else dir, prefix = splitdir(path) end loc.start + count(isequal(' '), prefix) end stop_utf8 = nextind(path, pos) # advance one unicode char, js uses exclusive upper bound formatted = format_path_completion.(results) msg = UpdateMessage(:completion_result, Dict( :start => start_utf8 - 1, # 1-based index (julia) to 0-based index (js) :stop => stop_utf8 - 1, # idem :results => formatted, ), .notebook, nothing, .initiator) putclientupdates!(.session, .initiator, msg) end responses[:complete] = function response_complete(::ClientRequest) try require_notebook() catch; return; end query = .body["query"] query_full = get(.body, "query_full", query) workspace = WorkspaceManager.get_workspace((.session, .notebook); allow_creation=false) results, loc, found, too_long = if will_run_code(.notebook) && workspace isa WorkspaceManager.Workspace && isready(workspace.dowork_token) # we don't use eval_format_fetch_in_workspace because we don't want the output to be string-formatted. # This works in this particular case, because the return object, a `Completion`, exists in this scope too. Malt.remote_eval_fetch(workspace.worker, quote PlutoRunner.completion_fetcher( $query, $query_full, getfield(Main, $(QuoteNode(workspace.module_name))), ) end) else # We can at least autocomplete general julia things: PlutoRunner.completion_fetcher(query, query_full, Main) end start_utf8 = loc.start stop_utf8 = nextind(query, lastindex(query)) # advance one unicode char, js uses exclusive upper bound msg = UpdateMessage(:completion_result, Dict( :start => start_utf8 - 1, # 1-based index (julia) to 0-based index (js) :stop => stop_utf8 - 1, # idem :results => results, :too_long => too_long ), .notebook, nothing, .initiator) putclientupdates!(.session, .initiator, msg) end responses[:complete_symbols] = function response_complete_symbols(::ClientRequest) msg = UpdateMessage(:completion_result, Dict( :latex => REPL.REPLCompletions.latex_symbols, :emoji => REPL.REPLCompletions.emoji_symbols, ), .notebook, nothing, .initiator) putclientupdates!(.session, .initiator, msg) end responses[:docs] = function response_docs(::ClientRequest) require_notebook() query = .body["query"] # Expand string macro calls to their macro form: # `html"` should yield `@html_str` and # `Markdown.md"` should yield `@Markdown.md_str`. (Ideally `Markdown.@md_str` but the former is easier) if endswith(query, '"') && query != "\"" query = string("@", SubString(query, firstindex(query), prevind(query, lastindex(query))), "_str") end workspace = WorkspaceManager.get_workspace((.session, .notebook); allow_creation=false) query_as_symbol = Symbol(query) base_binding = Docs.Binding(Base, query_as_symbol) doc_md = Docs.doc(base_binding) doc_html, status = if doc_md isa Markdown.MD && haskey(doc_md.meta, :results) && !isempty(doc_md.meta[:results]) # available in Base, no need to ask worker PlutoRunner.improve_docs!(doc_md, query_as_symbol, base_binding) (repr(MIME("text/html"), doc_md), :) else if will_run_code(.notebook) && workspace isa WorkspaceManager.Workspace && isready(workspace.dowork_token) Malt.remote_eval_fetch(workspace.worker, quote PlutoRunner.doc_fetcher( $query, getfield(Main, $(QuoteNode(workspace.module_name))), ) end) else (nothing, :) end end msg = UpdateMessage(:doc_result, Dict( :status => status, :doc => doc_html, ), .notebook, nothing, .initiator) putclientupdates!(.session, .initiator, msg) end function http_router_for(session::ServerSession) router = HTTP.Router(default_404_response) security = session.options.security function create_serve_onefile(path) return request::HTTP.Request -> asset_response(normpath(path)) end HTTP.register!(router, "GET", "/", create_serve_onefile(project_relative_path(frontend_directory(), "index.html"))) HTTP.register!(router, "GET", "/edit", create_serve_onefile(project_relative_path(frontend_directory(), "editor.html"))) HTTP.register!(router, "GET", "/ping", r -> HTTP.Response(200, "OK!")) HTTP.register!(router, "GET", "/possible_binder_token_please", r -> session.binder_token === nothing ? HTTP.Response(200,"") : HTTP.Response(200, session.binder_token)) function try_launch_notebook_response( action::Function, path_or_url::AbstractString; as_redirect=true, title="", advice="", home_url="./", action_kwargs... ) try nb = action(session, path_or_url; action_kwargs...) notebook_response(nb; home_url, as_redirect) catch e if e isa SessionActions.NotebookIsRunningException notebook_response(e.notebook; home_url, as_redirect) else error_response(500, title, advice, sprint(showerror, e, stacktrace(catch_backtrace()))) end end end function serve_newfile(request::HTTP.Request) notebook_response(SessionActions.new(session); as_redirect=(request.method == "GET")) end HTTP.register!(router, "GET", "/new", serve_newfile) HTTP.register!(router, "POST", "/new", serve_newfile) # This is not in Dynamic.jl because of bookmarks, how HTML works, # real loading bars and the rest; Same for CustomLaunchEvent function serve_openfile(request::HTTP.Request) try uri = HTTP.URI(request.target) query = HTTP.queryparams(uri) as_sample = haskey(query, "as_sample") execution_allowed = haskey(query, "execution_allowed") if haskey(query, "path") path = tamepath(maybe_convert_path_to_wsl(query["path"])) if isfile(path) return try_launch_notebook_response( SessionActions.open, path; execution_allowed, as_redirect=(request.method == "GET"), as_sample, risky_file_source=nothing, title="Failed to load notebook", advice="The file <code>$(htmlesc(path))</code> could not be loaded. Please <a href='https://github.com/fonsp/Pluto.jl/issues'>report this error</a>!", ) else return error_response(404, "Can't find a file here", "Please check whether <code>$(htmlesc(path))</code> exists.") end elseif haskey(query, "url") url = query["url"] return try_launch_notebook_response( SessionActions.open_url, url; execution_allowed, as_redirect=(request.method == "GET"), as_sample, risky_file_source=url, title="Failed to load notebook", advice="The notebook from <code>$(htmlesc(url))</code> could not be loaded. Please <a href='https://github.com/fonsp/Pluto.jl/issues'>report this error</a>!" ) else # You can ask Pluto to handle CustomLaunch events # and do some magic with how you open files. # You are responsible to keep this up to date. # See Events.jl for types and explanation # maybe_notebook_response = try_event_call(session, CustomLaunchEvent(query, request, try_launch_notebook_response)) isnothing(maybe_notebook_response) && return error("Empty request") return maybe_notebook_response end catch e return error_response(400, "Bad query", "Please <a href='https://github.com/fonsp/Pluto.jl/issues'>report this error</a>!", sprint(showerror, e, stacktrace(catch_backtrace()))) end end HTTP.register!(router, "GET", "/open", serve_openfile) HTTP.register!(router, "POST", "/open", serve_openfile) # normally shutdown is done through Dynamic.jl, with the exception of shutdowns made from the desktop app function serve_shutdown(request::HTTP.Request) notebook = notebook_from_uri(request) SessionActions.shutdown(session, notebook) return HTTP.Response(200) end HTTP.register!(router, "GET", "/shutdown", serve_shutdown) HTTP.register!(router, "POST", "/shutdown", serve_shutdown) # used in desktop app # looks like `/move?id=<notebook-id>&newpath=<new-notebook-path>`` function serve_move(request::HTTP.Request) uri = HTTP.URI(request.target) query = HTTP.queryparams(uri) notebook = notebook_from_uri(request) newpath = query["newpath"] try SessionActions.move(session, notebook, newpath) HTTP.Response(200, notebook.path) catch e error_response(400, "Bad query", "Please <a href='https://github.com/fonsp/Pluto.jl/issues'>report this error</a>!", sprint(showerror, e, stacktrace(catch_backtrace()))) end end HTTP.register!(router, "GET", "/move", serve_move) HTTP.register!(router, "POST", "/move", serve_move) function serve_notebooklist(request::HTTP.Request) return HTTP.Response(200, pack(Dict(k => v.path for (k, v) in session.notebooks))) end HTTP.register!(router, "GET", "/notebooklist", serve_notebooklist) function serve_sample(request::HTTP.Request) uri = HTTP.URI(request.target) sample_filename = split(HTTP.unescapeuri(uri.path), "sample/")[2] sample_path = project_relative_path("sample", sample_filename) try_launch_notebook_response( SessionActions.open, sample_path; as_redirect=(request.method == "GET"), home_url="../", as_sample=true, title="Failed to load sample", advice="Please <a href='https://github.com/fonsp/Pluto.jl/issues'>report this error</a>!" ) end HTTP.register!(router, "GET", "/sample/*", serve_sample) HTTP.register!(router, "POST","/sample/*", serve_sample) notebook_from_uri(request) = let uri = HTTP.URI(request.target) query = HTTP.queryparams(uri) id = UUID(query["id"]) session.notebooks[id] end function serve_notebookfile(request::HTTP.Request) try notebook = notebook_from_uri(request) response = HTTP.Response(200, sprint(save_notebook, notebook)) HTTP.setheader(response, "Content-Type" => "text/julia; charset=utf-8") HTTP.setheader(response, "Content-Disposition" => "inline; filename=\"$(basename(notebook.path))\"") response catch e return error_response(400, "Bad query", "Please <a href='https://github.com/fonsp/Pluto.jl/issues'>report this error</a>!", sprint(showerror, e, stacktrace(catch_backtrace()))) end end HTTP.register!(router, "GET", "/notebookfile", serve_notebookfile) function serve_statefile(request::HTTP.Request) try notebook = notebook_from_uri(request) response = HTTP.Response(200, Pluto.pack(Pluto.notebook_to_js(notebook))) HTTP.setheader(response, "Content-Type" => "application/octet-stream") HTTP.setheader(response, "Content-Disposition" => "attachment; filename=\"$(without_pluto_file_extension(basename(notebook.path))).plutostate\"") response catch e return error_response(400, "Bad query", "Please <a href='https://github.com/fonsp/Pluto.jl/issues'>report this error</a>!", sprint(showerror, e, stacktrace(catch_backtrace()))) end end HTTP.register!(router, "GET", "/statefile", serve_statefile) function serve_notebookexport(request::HTTP.Request) try notebook = notebook_from_uri(request) response = HTTP.Response(200, generate_html(notebook)) HTTP.setheader(response, "Content-Type" => "text/html; charset=utf-8") HTTP.setheader(response, "Content-Disposition" => "attachment; filename=\"$(without_pluto_file_extension(basename(notebook.path))).html\"") response catch e return error_response(400, "Bad query", "Please <a href='https://github.com/fonsp/Pluto.jl/issues'>report this error</a>!", sprint(showerror, e, stacktrace(catch_backtrace()))) end end HTTP.register!(router, "GET", "/notebookexport", serve_notebookexport) function serve_notebookupload(request::HTTP.Request) uri = HTTP.URI(request.target) query = HTTP.queryparams(uri) save_path = SessionActions.save_upload(request.body; filename_base=get(query, "name", nothing)) try_launch_notebook_response( SessionActions.open, save_path; as_redirect=false, as_sample=false, execution_allowed=haskey(query, "execution_allowed"), clear_frontmatter=haskey(query, "clear_frontmatter"), title="Failed to load notebook", advice="The contents could not be read as a Pluto notebook file. When copying contents from somewhere else, make sure that you copy the entire notebook file. You can also <a href='https://github.com/fonsp/Pluto.jl/issues'>report this error</a>!" ) end HTTP.register!(router, "POST", "/notebookupload", serve_notebookupload) function serve_asset(request::HTTP.Request) uri = HTTP.URI(request.target) filepath = project_relative_path(frontend_directory(), relpath(HTTP.unescapeuri(uri.path), "/")) asset_response(filepath; cacheable=should_cache(filepath)) end HTTP.register!(router, "GET", "/**", serve_asset) HTTP.register!(router, "GET", "/favicon.ico", create_serve_onefile(project_relative_path(frontend_directory(allow_bundled=false), "img", "favicon.ico"))) return scoped_router(session.options.server.base_url, router) end """ scoped_router(base_url::String, base_router::HTTP.Router)::HTTP.Router Returns a new `HTTP.Router` which delegates all requests to `base_router` but with requests trimmed so that they seem like they arrived at `/**` instead of `/\$base_url/**`. """ function scoped_router(base_url, base_router) base_url == "/" && return base_router @assert startswith(base_url, '/') "base_url \"$base_url\" should start with a '/'" @assert endswith(base_url, '/') "base_url \"$base_url\" should end with a '/'" @assert !occursin('*', base_url) "'*' not allowed in base_url \"$base_url\" " function handler(request) request.target = request.target[length(base_url):end] return base_router(request) end router = HTTP.Router(base_router._404, base_router._405) HTTP.register!(router, base_url * "**", handler) HTTP.register!(router, base_url, handler) return router end import UUIDs: UUID, uuid1 import .Configuration ### # CLIENT ### mutable struct ClientSession id::Symbol stream::Any connected_notebook::Union{Notebook,Nothing} pendingupdates::Channel simulated_lag::Float64 end ClientSession(id::Symbol, stream, simulated_lag=0.0) = let ClientSession(id, stream, nothing, Channel(1024), simulated_lag) end "A combination of _client ID_ and a _request ID_. The front-end generates a unqique ID for every request that it sends. The back-end (the stuff you are currently reading) can respond to a specific request. In that case, the response does not go through the normal message handlers in the front-end, but it flies directly to the place where the message was sent. (It resolves the promise returned by `send(...)`.)" struct Initiator client::ClientSession request_id::Symbol end function Base.getproperty(initiator::Initiator, property::Symbol) if property == :client_id getfield(initiator, :client).id else getfield(initiator, property) end end ### # SERVER ### """ The `ServerSession` keeps track of: - `connected_clients`: connected (web) clients - `notebooks`: running notebooks - `secret`: the web access token - `options`: global pluto configuration `Options` for this session. """ Base.@kwdef mutable struct ServerSession connected_clients::Dict{Symbol,ClientSession} = Dict{Symbol,ClientSession}() notebooks::Dict{UUID,Notebook} = Dict{UUID,Notebook}() secret::String = String(rand(('a':'z')  ('A':'Z')  ('0':'9'), 8)) binder_token::Union{String,Nothing} = nothing options::Configuration.Options = Configuration.Options() end function save_notebook(session::ServerSession, notebook::Notebook) # Notify event_listener from here try_event_call(session, FileSaveEvent(notebook)) if !session.options.server.disable_writing_notebook_files save_notebook(notebook, notebook.path) end end ### # UPDATE MESSAGE ### struct UpdateMessage type::Symbol message::Any notebook::Union{Notebook,Nothing} cell::Union{Cell,Nothing} initiator::Union{Initiator,Nothing} end UpdateMessage(type::Symbol, message::Any) = UpdateMessage(type, message, nothing, nothing, nothing) UpdateMessage(type::Symbol, message::Any, notebook::Notebook) = UpdateMessage(type, message, notebook, nothing, nothing) function clientupdate_notebook_list(notebooks; initiator::Union{Initiator,Nothing}=nothing) UpdateMessage(:notebook_list, Dict( :notebooks => [ Dict( :notebook_id => notebook.notebook_id, :path => notebook.path, :in_temp_dir => startswith(notebook.path, new_notebooks_directory()), :shortpath => basename(notebook.path), :process_status => notebook.process_status, ) for notebook in values(notebooks) ] ), nothing, nothing, initiator) end module SessionActions import ..Pluto: Pluto, Status, ServerSession, Notebook, Cell, emptynotebook, tamepath, new_notebooks_directory, without_pluto_file_extension, numbered_until_new, cutename, readwrite, update_save_run!, update_nbpkg_cache!, update_from_file, wait_until_file_unchanged, putnotebookupdates!, putplutoupdates!, load_notebook, clientupdate_notebook_list, WorkspaceManager, try_event_call, NewNotebookEvent, OpenNotebookEvent, ShutdownNotebookEvent, @asynclog, ProcessStatus, maybe_convert_path_to_wsl, move_notebook!, Throttled using FileWatching import ..Pluto.DownloadCool: download_cool import HTTP import UUIDs: UUID, uuid1 struct NotebookIsRunningException <: Exception notebook::Notebook end abstract type AbstractUserError <: Exception end struct UserError <: AbstractUserError msg::String end function Base.showerror(io::IO, e::UserError) print(io, e.msg) end function open_url(session::ServerSession, url::AbstractString; kwargs...) name_from_url = startswith(url, r"https?://") ? strip(HTTP.unescapeuri(splitext(basename(HTTP.URI(url).path))[1])) : "" new_name = isempty(name_from_url) ? cutename() : name_from_url random_notebook = emptynotebook() random_notebook.path = numbered_until_new( joinpath( new_notebooks_directory(), new_name ); suffix=".jl") path = download_cool(url, random_notebook.path) result = try_event_call(session, NewNotebookEvent()) notebook = if result isa UUID open(session, path; notebook_id=result, kwargs...) else open(session, path; kwargs...) end return notebook end "Open the notebook at `path` into `session::ServerSession` and run it. Returns the `Notebook`." function open(session::ServerSession, path::AbstractString; execution_allowed::Bool=true, run_async::Bool=true, compiler_options=nothing, as_sample::Bool=false, risky_file_source::Union{Nothing,String}=nothing, clear_frontmatter::Bool=false, notebook_id::UUID=uuid1() ) path = maybe_convert_path_to_wsl(path) if as_sample new_filename = "sample " * without_pluto_file_extension(basename(path)) new_path = numbered_until_new(joinpath(new_notebooks_directory(), new_filename); suffix=".jl") readwrite(path, new_path) path = new_path end for notebook in values(session.notebooks) if isfile(notebook.path) && realpath(notebook.path) == realpath(tamepath(path)) throw(NotebookIsRunningException(notebook)) end end notebook = load_notebook(tamepath(path); disable_writing_notebook_files=session.options.server.disable_writing_notebook_files) execution_allowed = execution_allowed && !haskey(notebook.metadata, "risky_file_source") notebook.notebook_id = notebook_id if !isnothing(risky_file_source) notebook.metadata["risky_file_source"] = risky_file_source end notebook.process_status = execution_allowed ? ProcessStatus.starting : ProcessStatus.waiting_for_permission # overwrites the notebook environment if specified if compiler_options !== nothing notebook.compiler_options = compiler_options end if clear_frontmatter Pluto.set_frontmatter!(notebook, nothing) end session.notebooks[notebook.notebook_id] = notebook if execution_allowed && session.options.evaluation.run_notebook_on_load Pluto._report_business_cells_planned!(notebook) end if !execution_allowed Status.delete_business!(notebook.status_tree, :run) Status.delete_business!(notebook.status_tree, :workspace) Status.delete_business!(notebook.status_tree, :pkg) end update_nbpkg_cache!(notebook) update_save_run!(session, notebook, notebook.cells; run_async, prerender_text=true) add(session, notebook; run_async) try_event_call(session, OpenNotebookEvent(notebook)) return notebook end function add(session::ServerSession, notebook::Notebook; run_async::Bool=true) session.notebooks[notebook.notebook_id] = notebook if run_async @asynclog putplutoupdates!(session, clientupdate_notebook_list(session.notebooks)) else putplutoupdates!(session, clientupdate_notebook_list(session.notebooks)) end update_from_file_throttled = let running = Ref(false) function() if !running[] running[] = true @info "Updating from file..." sleep(0.1) ## There seems to be a synchronization issue if your OS is VERYFAST wait_until_file_unchanged(notebook.path, .3) # call update_from_file. If it returns false, that means that the notebook file was corrupt, so we try again, a maximum of 10 times. for _ in 1:10 if update_from_file(session, notebook) break end end @info "Updating from file done!" running[] = false end end end in_session() = get(session.notebooks, notebook.notebook_id, nothing) === notebook session.options.server.auto_reload_from_file && @asynclog try while in_session() if !isfile(notebook.path) # notebook file deleted... let's ignore this, changing the notebook will cause it to save again. Fine for now sleep(2) else e = watch_file(notebook.path, 3) if e.timedout continue end # the above call is blocking until the file changes local modified_time = mtime(notebook.path) local _tries = 0 # mtime might return zero if the file is temporarily removed while modified_time == 0.0 && _tries < 10 modified_time = mtime(notebook.path) _tries += 1 sleep(.05) end # current_time = time() # @info "File changed" (current_time - notebook.last_save_time) (modified_time - notebook.last_save_time) (current_time - modified_time) if !in_session() break end # if current_time - notebook.last_save_time < 2.0 # @info "Notebook was saved by me very recently, not reloading from file." if modified_time == 0.0 # @warn "Failed to hot reload: file no longer exists." elseif modified_time - notebook.last_save_time < session.options.server.auto_reload_from_file_cooldown # @info "Modified time is very close to my last save time, not reloading from file." else update_from_file_throttled() end end end catch e if !(e isa InterruptException) rethrow(e) end end notebook.status_tree.update_listener_ref[] = Throttled.throttled(1.0 / 8; runtime_multiplier=4.0) do Pluto.send_notebook_changes!(Pluto.ClientRequest(; session, notebook)) end return notebook end """ Generate a non-existing new notebook filename, and write `contents` to that file. Return the generated filename. # Example ```julia save_upload(some_notebook_data; filename_base="hello") == "~/.julia/pluto_notebooks/hello 5.jl" ``` """ function save_upload(contents::Union{String,Vector{UInt8}}; filename_base::Union{Nothing,AbstractString}=nothing) save_path = numbered_until_new( joinpath( new_notebooks_directory(), something(filename_base, cutename()) ); suffix=".jl") write(save_path, contents) save_path end "Create a new empty notebook inside `session::ServerSession`. Returns the `Notebook`." function new(session::ServerSession; run_async=true, notebook_id::UUID=uuid1()) notebook = if session.options.compiler.sysimage === nothing emptynotebook() else Notebook([Cell("import Pkg"), Cell("# This cell disables Pluto's package manager and activates the global environment. Click on ? inside the bubble next to Pkg.activate to learn more.\n# (added automatically because a sysimage is used)\nPkg.activate()"), Cell()]) end # Run NewNotebookEvent handler before assigning ID isid = try_event_call(session, NewNotebookEvent()) notebook.notebook_id = isnothing(isid) ? notebook_id : isid update_save_run!(session, notebook, notebook.cells; run_async, prerender_text=true) add(session, notebook; run_async) try_event_call(session, OpenNotebookEvent(notebook)) return notebook end "Shut down `notebook` inside `session`. If `keep_in_session` is `false` (default), you will not be allowed to run a notebook with the same notebook_id again." function shutdown(session::ServerSession, notebook::Notebook; keep_in_session::Bool=false, async::Bool=false, verbose::Bool=true) notebook.nbpkg_restart_recommended_msg = nothing notebook.nbpkg_restart_required_msg = nothing if notebook.process_status  (ProcessStatus.ready, ProcessStatus.starting) notebook.process_status = ProcessStatus.no_process end if !keep_in_session listeners = putnotebookupdates!(session, notebook) # TODO: shutdown message delete!(session.notebooks, notebook.notebook_id) putplutoupdates!(session, clientupdate_notebook_list(session.notebooks)) for client in listeners @async close(client.stream) end end WorkspaceManager.unmake_workspace((session, notebook); async, verbose, allow_restart=keep_in_session) try_event_call(session, ShutdownNotebookEvent(notebook)) end function move(session::ServerSession, notebook::Notebook, newpath::String) newpath = tamepath(newpath) if isfile(newpath) error("File exists already - you need to delete the old file manually.") else move_notebook!(notebook, newpath; disable_writing_notebook_files=session.options.server.disable_writing_notebook_files) putplutoupdates!(session, clientupdate_notebook_list(session.notebooks)) let workspace = WorkspaceManager.get_workspace((session, notebook); allow_creation=false) isnothing(workspace) || WorkspaceManager.cd_workspace(workspace, newpath) end end end end import HTTP import Markdown: htmlesc import Pkg import MIMEs function frontend_directory(; allow_bundled::Bool=true) if allow_bundled && isdir(project_relative_path("frontend-dist")) && (get(ENV, "JULIA_PLUTO_FORCE_BUNDLED", "nein") == "ja" || !is_pluto_dev()) "frontend-dist" else "frontend" end end function should_cache(path::String) dir, filename = splitdir(path) endswith(dir, "frontend-dist") && occursin(r"\.[0-9a-f]{8}\.", filename) end const day = let second = 1 hour = 60second day = 24hour end function default_404_response(req = nothing) HTTP.Response(404, "Not found!") end function asset_response(path; cacheable::Bool=false) if !isfile(path) && !endswith(path, ".html") return asset_response(path * ".html"; cacheable) end if isfile(path) data = read(path) response = HTTP.Response(200, data) HTTP.setheader(response, "Content-Type" => MIMEs.contenttype_from_mime(MIMEs.mime_from_path(path, MIME"application/octet-stream"()))) HTTP.setheader(response, "Content-Length" => string(length(data))) HTTP.setheader(response, "Access-Control-Allow-Origin" => "*") cacheable && HTTP.setheader(response, "Cache-Control" => "public, max-age=$(30day), immutable") response else default_404_response() end end function error_response( status_code::Integer, title, advice, body="") template = read(project_relative_path(frontend_directory(), "error.jl.html"), String) body_title = body == "" ? "" : "Error message:" filled_in = replace(template, "\$STYLE" => """<style>$(read(project_relative_path("frontend", "error.css"), String))</style>""", "\$TITLE" => title, "\$ADVICE" => advice, "\$BODYTITLE" => body_title, "\$BODY" => htmlesc(body), ) response = HTTP.Response(status_code, filled_in) HTTP.setheader(response, "Content-Type" => MIMEs.contenttype_from_mime(MIME"text/html"())) response end function notebook_response(notebook; home_url="./", as_redirect=true) if as_redirect response = HTTP.Response(302, "") HTTP.setheader(response, "Location" => home_url * "edit?id=" * string(notebook.notebook_id)) return response else HTTP.Response(200, string(notebook.notebook_id)) end end const found_is_pluto_dev = Ref{Bool}() """ Is the Pluto package `dev`ed? Returns `false` for normal Pluto installation from the registry. """ function is_pluto_dev() if isassigned(found_is_pluto_dev) return found_is_pluto_dev[] end found_is_pluto_dev[] = try # is the package located in .julia/packages ? if startswith(pkgdir(@__MODULE__), joinpath(get(DEPOT_PATH, 1, "zzz"), "packages")) false else deps = Pkg.dependencies() p_index = findfirst(p -> p.name == "Pluto", deps) p = deps[p_index] p.is_tracking_path end catch e @debug "is_pluto_dev failed" e false end end """ This module contains the "Status" system from Pluto, which you can see in the bottom right in the Pluto editor. It's used to track what is currently happening, for how long. (E.g. "Notebook startup > Julia process starting".) The Status system is hierachical: a status item can have multiple subtasks. E.g. the "Package manager" status can have subtask "instantiate" and "precompile". In the UI, these are sections that you can fold out. !!! warning This module is not public API of Pluto. """ module Status _default_update_listener() = nothing Base.@kwdef mutable struct Business name::Symbol=:ignored success::Union{Nothing,Bool}=nothing started_at::Union{Nothing,Float64}=nothing finished_at::Union{Nothing,Float64}=nothing subtasks::Dict{Symbol,Business}=Dict{Symbol,Business}() update_listener_ref::Ref{Any}=Ref{Any}(_default_update_listener) lock::Threads.SpinLock=Threads.SpinLock() end tojs(b::Business) = Dict{String,Any}( "name" => b.name, "success" => b.success, "started_at" => b.started_at, "finished_at" => b.finished_at, "subtasks" => Dict{String,Any}( String(s) => tojs(r) for (s, r) in b.subtasks ), ) function report_business_started!(business::Business) lock(business.lock) do business.success = nothing business.started_at = time() business.finished_at = nothing empty!(business.subtasks) end business.update_listener_ref[]() return business end function report_business_finished!(business::Business, success::Bool=true) if business.success === nothing && business.started_at !== nothing && business.finished_at === nothing business.success = success end lock(business.lock) do # if it never started, then lets "start" it now business.started_at = something(business.started_at, time()) # if it already finished, then leave the old finish time. business.finished_at = something(business.finished_at, max(business.started_at, time())) end # also finish all subtasks (this can't be inside the same lock) for v in values(business.subtasks) report_business_finished!(v, success) end business.update_listener_ref[]() return business end create_for_child(parent::Business, name::Symbol) = function() Business(; name, update_listener_ref=parent.update_listener_ref, lock=parent.lock) end get_child(parent::Business, name::Symbol) = lock(parent.lock) do get!(create_for_child(parent, name), parent.subtasks, name) end report_business_finished!(parent::Business, name::Symbol, success::Bool=true) = report_business_finished!(get_child(parent, name), success) report_business_started!(parent::Business, name::Symbol) = get_child(parent, name) |> report_business_started! report_business_planned!(parent::Business, name::Symbol) = get_child(parent, name) function report_business!(f::Function, parent::Business, name::Symbol) local success = false try report_business_started!(parent, name) f() success = true finally report_business_finished!(parent, name, success) end end delete_business!(business::Business, name::Symbol) = lock(business.lock) do delete!(business.subtasks, name) end # GLOBAL # registry update ## once per process # waiting for other notebook packages # PER NOTEBOOK # notebook process starting # installing packages # updating packages # running cells end import MsgPack import UUIDs: UUID import HTTP import Sockets import .PkgCompat function open_in_default_browser(url::AbstractString)::Bool try if Sys.isapple() Base.run(`open $url`) true elseif Sys.iswindows() || detectwsl() Base.run(`powershell.exe Start "'$url'"`) true elseif Sys.islinux() Base.run(`xdg-open $url`, devnull, devnull, devnull) true else false end catch ex false end end function swallow_exception(f, exception_type::Type{T}) where {T} try f() catch e isa(e, T) || rethrow(e) end end """ Pluto.run() Start Pluto! ## Keyword arguments You can configure some of Pluto's more technical behaviour using keyword arguments, but this is mostly meant to support testing and strange setups like Docker. If you want to do something exciting with Pluto, you can probably write a creative notebook to do it! Pluto.run(; kwargs...) For the full list, see the [`Pluto.Configuration`](@ref) module. Some **common parameters**: - `launch_browser`: Optional. Whether to launch the system default browser. Disable this on SSH and such. - `host`: Optional. The default `host` is `"127.0.0.1"`. For wild setups like Docker and heroku, you might need to change this to `"0.0.0.0"`. - `port`: Optional. The default `port` is `1234`. - `auto_reload_from_file`: Reload when the `.jl` file is modified. The default is `false`. ## Technobabble This will start the static HTTP server and a WebSocket server. The server runs _synchronously_ (i.e. blocking call) on `http://[host]:[port]/`. Pluto notebooks can be started from the main menu in the web browser. """ function run(; kwargs...) options = Configuration.from_flat_kwargs(; kwargs...) run(options) end function run(options::Configuration.Options) session = ServerSession(; options) run(session) end # Deprecation errors function run(host::String, port::Union{Nothing,Integer} = nothing; kwargs...) @error """run(host, port) is deprecated in favor of: run(;host="$host", port=$port) """ end function run(port::Integer; kwargs...) @error "Oopsie! This is the old command to launch Pluto. The new command is: Pluto.run() without the port as argument - it will choose one automatically. If you need to specify the port, use: Pluto.run(port=$port) " end const is_first_run = Ref(true) "Return a port and serversocket to use while taking into account the `favourite_port`." function port_serversocket(hostIP::Sockets.IPAddr, favourite_port, port_hint) local port, serversocket if favourite_port === nothing port, serversocket = Sockets.listenany(hostIP, UInt16(port_hint)) else port = UInt16(favourite_port) try serversocket = Sockets.listen(hostIP, port) catch e error("Cannot listen on port $port. It may already be in use, or you may not have sufficient permissions. Use Pluto.run() to automatically select an available port.") end end return port, serversocket end struct RunningPlutoServer http_server initial_registry_update_task::Task end function Base.close(ssc::RunningPlutoServer) close(ssc.http_server) wait(ssc.http_server) wait(ssc.initial_registry_update_task) end function Base.wait(ssc::RunningPlutoServer) try # create blocking call and switch the scheduler back to the server task, so that interrupts land there while isopen(ssc.http_server) sleep(.1) end catch e println() println() Base.close(ssc) (e isa InterruptException) || rethrow(e) end nothing end """ run(session::ServerSession) Specifiy the [`Pluto.ServerSession`](@ref) to run the web server on, which includes the configuration. Passing a session as argument allows you to start the web server with some notebooks already running. See [`SessionActions`](@ref) to learn more about manipulating a `ServerSession`. """ function run(session::ServerSession) Base.wait(run!(session)) end function run!(session::ServerSession) if is_first_run[] is_first_run[] = false @info "Loading..." end warn_julia_compat() pluto_router = http_router_for(session) store_session_middleware = create_session_context_middleware(session) app = pluto_router |> auth_middleware |> store_session_middleware let n = session.options.server.notebook SessionActions.open.((session,), n === nothing ? [] : n isa AbstractString ? [n] : n; run_async=true, ) end host = session.options.server.host hostIP = parse(Sockets.IPAddr, host) favourite_port = session.options.server.port port_hint = session.options.server.port_hint local port, serversocket = port_serversocket(hostIP, favourite_port, port_hint) on_shutdown() = @sync begin # Triggered by HTTP.jl @info("\nClosing Pluto... Restart Julia for a fresh session. \n\nHave a nice day! \n\n") # TODO: put do_work tokens back @async swallow_exception(() -> close(serversocket), Base.IOError) for client in values(session.connected_clients) @async swallow_exception(() -> close(client.stream), Base.IOError) end empty!(session.connected_clients) for nb in values(session.notebooks) @asynclog SessionActions.shutdown(session, nb; keep_in_session=false, async=false, verbose=false) end end server = HTTP.listen!(hostIP, port; stream=true, server=serversocket, on_shutdown, verbose=-1) do http::HTTP.Stream # the if statement below asks if the current request is a "websocket upgrade" request: the start of a websocket connection. if HTTP.WebSockets.isupgrade(http.message) secret_required = let s = session.options.security s.require_secret_for_access || s.require_secret_for_open_links end finish() = try HTTP.setstatus(http, 403) HTTP.startwrite(http) write(http, "Forbidden") HTTP.closewrite(http) catch e if !(e isa Base.IOError) rethrow(e) end end if !secret_required || is_authenticated(session, http.message) try # "upgrade" means accept and start the websocket connection that the client requested HTTP.WebSockets.upgrade(http) do clientstream if HTTP.WebSockets.isclosed(clientstream) return end found_client_id_ref = Ref(Symbol(:none)) try # the loop below will keep running for this websocket connection, it iterates over all incoming websocket messages. for message in clientstream # This stream contains data received over the WebSocket. # It is formatted and MsgPack-encoded by send(...) in PlutoConnection.js local parentbody = nothing local did_read = false try parentbody = unpack(message) # for debug only let lag = session.options.server.simulated_lag (lag > 0) && sleep(lag * (0.5 + rand())) # sleep(0) would yield to the process manager which we dont want end did_read = true if found_client_id_ref[] === :none found_client_id_ref[] = Symbol(parentbody["client_id"]) end process_ws_message(session, parentbody, clientstream) catch ex if ex isa InterruptException || ex isa HTTP.WebSockets.WebSocketError || ex isa EOFError # that's fine! else bt = catch_backtrace() if did_read @warn "Processing message failed for unknown reason:" parentbody exception = (ex, bt) else @warn "Reading WebSocket client stream failed for unknown reason:" parentbody exception = (ex, bt) end end end end catch ex if ex isa InterruptException || ex isa HTTP.WebSockets.WebSocketError || ex isa EOFError || (ex isa Base.IOError && occursin("connection reset", ex.msg)) # that's fine! else bt = stacktrace(catch_backtrace()) @warn "Reading WebSocket client stream failed for unknown reason:" exception = (ex, bt) end finally if haskey(session.connected_clients, found_client_id_ref[]) @debug "Removing client $(found_client_id_ref[]) from connected_clients" delete!(session.connected_clients, found_client_id_ref[]) end end end catch ex if ex isa InterruptException # that's fine! elseif ex isa Base.IOError # that's fine! elseif ex isa ArgumentError && occursin("stream is closed", ex.msg) # that's fine! else bt = stacktrace(catch_backtrace()) @warn "HTTP upgrade failed for unknown reason" exception = (ex, bt) end finally # if we never wrote a response, then do it now if isopen(http) && !iswritable(http) finish() end end else finish() end else # then it's a regular HTTP request, not a WS upgrade request::HTTP.Request = http.message request.body = read(http) # HTTP.closeread(http) # If a "token" url parameter is passed in from binder, then we store it to add to every URL (so that you can share the URL to collaborate). let params = HTTP.queryparams(HTTP.URI(request.target)) if haskey(params, "token") && params["token"]  ("null", "undefined", "") && session.binder_token === nothing session.binder_token = params["token"] end end ### response_body = app(request) ### request.response::HTTP.Response = response_body request.response.request = request try HTTP.setheader(http, "Content-Length" => string(length(request.response.body))) # https://github.com/fonsp/Pluto.jl/pull/722 HTTP.setheader(http, "Referrer-Policy" => "same-origin") # https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#:~:text=is%202%20minutes.-,14.38%20Server HTTP.setheader(http, "Server" => "Pluto.jl/$(PLUTO_VERSION_STR[2:end]) Julia/$(JULIA_VERSION_STR[2:end])") HTTP.startwrite(http) write(http, request.response.body) catch e if isa(e, Base.IOError) || isa(e, ArgumentError) # @warn "Attempted to write to a closed stream at $(request.target)" else rethrow(e) end end end end server_running() = try HTTP.get("http://$(hostIP):$(port)$(session.options.server.base_url)ping"; status_exception=false, retry=false, connect_timeout=10, readtimeout=10).status == 200 catch false end # Wait for the server to start up before opening the browser. We have a 5 second grace period for allowing the connection, and then 10 seconds for the server to write data. WorkspaceManager.poll(server_running, 5.0, 1.0) address = pretty_address(session, hostIP, port) if session.options.server.launch_browser && open_in_default_browser(address) @info("\nOpening $address in your default browser... ~ have fun!") else @info("\nGo to $address in your browser to start writing ~ have fun!") end @info("\nPress Ctrl+C in this terminal to stop Pluto\n\n") # Trigger ServerStartEvent with server details try_event_call(session, ServerStartEvent(address, port)) if frontend_directory() == "frontend" @info("It looks like you are developing the Pluto package, using the unbundled frontend...") end # Start this in the background, so that the first notebook launch (which will trigger registry update) will be faster initial_registry_update_task = @asynclog withtoken(pkg_token) do will_update = !PkgCompat.check_registry_age() PkgCompat.update_registries(; force = false) will_update && println(" Updating registry done ") end return RunningPlutoServer(server, initial_registry_update_task) end precompile(run, (ServerSession, HTTP.Handlers.Router{Symbol("##001")})) function pretty_address(session::ServerSession, hostIP, port) root = if session.options.server.root_url !== nothing @assert endswith(session.options.server.root_url, "/") replace(session.options.server.root_url, "{PORT}" => string(Int(port))) elseif haskey(ENV, "JH_APP_URL") "$(ENV["JH_APP_URL"])proxy/$(Int(port))/" else host_str = string(hostIP) host_pretty = if isa(hostIP, Sockets.IPv6) if host_str == "::1" "localhost" else "[$(host_str)]" end elseif host_str == "127.0.0.1" # Assuming the other alternative is IPv4 "localhost" else host_str end port_pretty = Int(port) base_url = session.options.server.base_url "http://$(host_pretty):$(port_pretty)$(base_url)" end url_params = Dict{String,String}() if session.options.security.require_secret_for_access url_params["secret"] = session.secret end fav_notebook = let n = session.options.server.notebook n isa AbstractVector ? (isempty(n) ? nothing : first(n)) : n end new_root = if fav_notebook !== nothing # since this notebook already started running, this will get redicted to that session url_params["path"] = string(fav_notebook) root * "open" else root end string(HTTP.URI(HTTP.URI(new_root); query = url_params)) end """ All messages sent over the WebSocket from the client get decoded+deserialized and end up here. It calls one of the functions from the `responses` Dict, see the file Dynamic.jl. """ function process_ws_message(session::ServerSession, parentbody::Dict, clientstream) client_id = Symbol(parentbody["client_id"]) client = get!(session.connected_clients, client_id) do ClientSession(client_id, clientstream, session.options.server.simulated_lag) end client.stream = clientstream # it might change when the same client reconnects messagetype = Symbol(parentbody["type"]) request_id = Symbol(parentbody["request_id"]) notebook = if haskey(parentbody, "notebook_id") && parentbody["notebook_id"] !== nothing notebook = let notebook_id = UUID(parentbody["notebook_id"]) get(session.notebooks, notebook_id, nothing) end if messagetype === :connect if notebook === nothing messagetype === :connect || @warn "Remote notebook not found locally!" else client.connected_notebook = notebook end end notebook else nothing end body = parentbody["body"] if haskey(responses, messagetype) responsefunc = responses[messagetype] try responsefunc(ClientRequest(session, notebook, body, Initiator(client, request_id))) catch ex @warn "Response function to message of type $(repr(messagetype)) failed" rethrow(ex) end else @warn "Message of type $(messagetype) not recognised" end end ### A Pluto.jl notebook ### # v0.19.9 using Markdown using InteractiveUtils #  b987a8a2-6ab0-4e88-af3c-d7f2778af657 #  show_logs = false #  skip_as_script = true #= begin import Pkg # create a local environment for this notebook # used to install and load PlutoTest local_env = mktempdir() Pkg.activate(local_env) Pkg.add(name="PlutoTest", version="0.2") pushfirst!(LOAD_PATH, local_env) # activate Pluto's environment, used to load HTTP.jl Pkg.activate(Base.current_project(@__FILE__)) using PlutoTest end  =# #  cc180e7e-46c3-11ec-3fff-05e1b5c77986 #  skip_as_script = true #= md""" # Download Data URLs """  =# #  2385dd3b-15f8-4790-907f-e0576a56c4c0 #  skip_as_script = true #= random_data = rand(UInt8, 30)  =# #  d8ed6d44-33cd-4c9d-828b-d237d43769f5 # try # download("asdffads") # catch e # e # end |> typeof #  b3f685a3-b52d-4190-9196-6977a7e76aa1 begin import HTTP.URIs import Base64 import Downloads end #  a85c0c0b-47d0-4377-bc22-3c87239a67b3 """ ```julia download_cool(url::AbstractString, [path::AbstractString = tempname()]) -> path ``` The same as [`Base.download`](@ref), but also supports [Data URLs](https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/Data_URIs). """ function download_cool(url::AbstractString, path::AbstractString=tempname()) if startswith(url, "data:") comma_index = findfirst(',', url) @assert comma_index isa Int "Invalid data URL." metadata_str = url[length("data:")+1:comma_index-1] metadata_parts = split(metadata_str, ';'; limit=2) if length(metadata_parts) == 2 @assert metadata_parts[2] == "base64" "Invalid data URL." end mime = MIME(first(metadata_parts)) is_base64 = length(metadata_parts) == 2 data_str = SubString(url, comma_index+1) data = is_base64 ? Base64.base64decode(data_str) : URIs.unescapeuri(data_str) write(path, data) path else Downloads.download(url, path) end end #  6e1dd79c-a7bf-44d6-bfa6-ced75b45170a download_cool_string(args...) = read(download_cool(args...), String) #  6339496d-11be-40d0-b4e5-9247e5199367 #= @test download_cool_string("data:,Hello%2C%20World%21") == "Hello, World!"  =# #  bf7b4241-9cb0-4d90-9ded-b527bf220803 #= @test download_cool_string("data:text/plain,Hello%2C%20World%21") == "Hello, World!"  =# #  d6e01532-a8e4-4173-a270-eae37c8002c7 #= @test download_cool_string("data:text/plain;base64,SGVsbG8sIFdvcmxkIQ==") == "Hello, World!"  =# #  b0ba1add-f452-4a44-ab23-becbc610e2b9 #= @test download_cool_string("data:;base64,SGVsbG8sIFdvcmxkIQ==") == "Hello, World!"  =# #  e630e261-1c2d-4117-9c44-dd49199fa3de #= @test download_cool_string("data:,hello") == "hello"  =# #  4bb75573-09bd-4ce7-b76f-34c0249d7b88 #= @test download_cool_string("data:text/html,%3Ch1%3EHello%2C%20World%21%3C%2Fh1%3E") == "<h1>Hello, World!</h1>"  =# #  301eee81-7715-4d39-89aa-37bffde3557f #= @test download_cool_string("data:text/html,<script>alert('hi');</script>") == "<script>alert('hi');</script>"  =# #  525b2cb6-b7b9-436e-898e-a951e6a1f2f1 #= @test occursin("reactive", download_cool_string("https://raw.githubusercontent.com/fonsp/Pluto.jl/v0.17.1/README.md"))  =# #  3630b4bc-ff63-426d-b95d-ae4e4f9ccd88 download_cool_data(args...) = read(download_cool(args...)) #  40b48818-e191-4509-85ad-b9ff745cd0cb #= @test_throws Exception download_cool("data:xoxo;base10,asdfasdfasdf")  =# #  1f175fcd-8b94-4f13-a912-02a21c95f8ca #= @test_throws Exception download_cool("data:text/plain;base10,asdfasdfasdf")  =# #  a4f671e6-0e23-4753-9301-048b2ef505e3 #= @test_throws Exception download_cool("data:asdfasdfasdf")  =# #  ae296e09-08dd-4ee8-87ac-eb2bf24b28b9 #= random_data_url = "data:asf;base64,$( Base64.base64encode(random_data) )"  =# #  2eabfa58-2d8f-4479-9c00-a58b934638d9 #= @test download_cool_data(random_data_url) == random_data  =# #  Cell order: # cc180e7e-46c3-11ec-3fff-05e1b5c77986 # a85c0c0b-47d0-4377-bc22-3c87239a67b3 # 6339496d-11be-40d0-b4e5-9247e5199367 # bf7b4241-9cb0-4d90-9ded-b527bf220803 # d6e01532-a8e4-4173-a270-eae37c8002c7 # b0ba1add-f452-4a44-ab23-becbc610e2b9 # e630e261-1c2d-4117-9c44-dd49199fa3de # 4bb75573-09bd-4ce7-b76f-34c0249d7b88 # 301eee81-7715-4d39-89aa-37bffde3557f # 2385dd3b-15f8-4790-907f-e0576a56c4c0 # ae296e09-08dd-4ee8-87ac-eb2bf24b28b9 # 2eabfa58-2d8f-4479-9c00-a58b934638d9 # 525b2cb6-b7b9-436e-898e-a951e6a1f2f1 # 6e1dd79c-a7bf-44d6-bfa6-ced75b45170a # 3630b4bc-ff63-426d-b95d-ae4e4f9ccd88 # 40b48818-e191-4509-85ad-b9ff745cd0cb # 1f175fcd-8b94-4f13-a912-02a21c95f8ca # a4f671e6-0e23-4753-9301-048b2ef505e3 # d8ed6d44-33cd-4c9d-828b-d237d43769f5 # b3f685a3-b52d-4190-9196-6977a7e76aa1 # b987a8a2-6ab0-4e88-af3c-d7f2778af657 declare global { /** * This namespace is meant for [PlutoDesktop](https://github.com/JuliaPluto/PlutoDesktop) * related types and interfaces. */ namespace Desktop { /** * This enum has to be in sync with the enum "PlutoExport" * defined in PlutoDesktop/{branch:master}/types/enums.ts * * @note Unfortunately enums can't be exported from .d.ts files. * Inorder to use this, just map integers to the enum values * - PlutoExport.FILE -> **0** * - PlutoExport.HTML -> **1** * - PlutoExport.STATE -> **2** * - PlutoExport.PDF -> **3** */ enum PlutoExport { FILE, HTML, STATE, PDF, } /** * This type has to be in sync with the interface "Window" * defined in PlutoDesktop/{branch:master}/src/renderer/preload.d.ts */ type PlutoDesktop = { ipcRenderer: { sendMessage(channel: String, args: unknown[]): void on(channel: string, func: (...args: unknown[]) => void): (() => void) | undefined once(channel: string, func: (...args: unknown[]) => void): void } fileSystem: { /** * @param type [default = 'new'] whether you want to open a new notebook * open a notebook from a path or from a url * @param pathOrURL location to the file, not needed if opening a new file, * opens that notebook. If false and no path is there, opens the file selector. * If true, opens a new blank notebook. */ openNotebook(type?: "url" | "path" | "new", pathOrURL?: string): void shutdownNotebook(id?: string): void moveNotebook(id?: string): void exportNotebook(id: string, type: PlutoExport): void } } } interface Window { plutoDesktop?: Desktop.PlutoDesktop } } export {} @import url("./editor.css"); @import url("./binder.css"); @import url("./treeview.css"); @import url("./highlightjs.css"); /* ANSI color classes */ :root { --ansi-black: rgb(0, 0, 0); --ansi-red: rgb(187, 0, 0); --ansi-green: rgb(0, 187, 0); --ansi-yellow: rgb(187, 187, 0); --ansi-blue: rgb(0, 0, 187); --ansi-magenta: rgb(187, 0, 187); --ansi-cyan: rgb(0, 187, 187); --ansi-white: rgb(255, 255, 255); --ansi-bright-black: rgb(85, 85, 85); --ansi-bright-red: rgb(255, 85, 85); --ansi-bright-green: rgb(0, 255, 0); --ansi-bright-yellow: rgb(255, 255, 85); --ansi-bright-blue: rgb(85, 85, 255); --ansi-bright-magenta: rgb(255, 85, 255); --ansi-bright-cyan: rgb(85, 255, 255); --ansi-bright-white: rgb(255, 255, 255); } /* Foreground colors */ .ansi-black-fg { color: var(--ansi-black); } .ansi-red-fg { color: var(--ansi-red); } .ansi-green-fg { color: var(--ansi-green); } .ansi-yellow-fg { color: var(--ansi-yellow); } .ansi-blue-fg { color: var(--ansi-blue); } .ansi-magenta-fg { color: var(--ansi-magenta); } .ansi-cyan-fg { color: var(--ansi-cyan); } .ansi-white-fg { color: var(--ansi-white); } .ansi-bright-black-fg { color: var(--ansi-bright-black); } .ansi-bright-red-fg { color: var(--ansi-bright-red); } .ansi-bright-green-fg { color: var(--ansi-bright-green); } .ansi-bright-yellow-fg { color: var(--ansi-bright-yellow); } .ansi-bright-blue-fg { color: var(--ansi-bright-blue); } .ansi-bright-magenta-fg { color: var(--ansi-bright-magenta); } .ansi-bright-cyan-fg { color: var(--ansi-bright-cyan); } .ansi-bright-white-fg { color: var(--ansi-bright-white); } /* Background colors */ .ansi-black-bg { background-color: var(--ansi-black); } .ansi-red-bg { background-color: var(--ansi-red); } .ansi-green-bg { background-color: var(--ansi-green); } .ansi-yellow-bg { background-color: var(--ansi-yellow); } .ansi-blue-bg { background-color: var(--ansi-blue); } .ansi-magenta-bg { background-color: var(--ansi-magenta); } .ansi-cyan-bg { background-color: var(--ansi-cyan); } .ansi-white-bg { background-color: var(--ansi-white); } .ansi-bright-black-bg { background-color: var(--ansi-bright-black); } .ansi-bright-red-bg { background-color: var(--ansi-bright-red); } .ansi-bright-green-bg { background-color: var(--ansi-bright-green); } .ansi-bright-yellow-bg { background-color: var(--ansi-bright-yellow); } .ansi-bright-blue-bg { background-color: var(--ansi-bright-blue); } .ansi-bright-magenta-bg { background-color: var(--ansi-bright-magenta); } .ansi-bright-cyan-bg { background-color: var(--ansi-bright-cyan); } .ansi-bright-white-bg { background-color: var(--ansi-bright-white); } /* Some custom styles for stdout to improve contrast and readability */ pluto-log-dot.Stdout { --ansi-red: rgb(220, 0, 0); --ansi-magenta: rgb(200, 0, 200); --ansi-white: rgb(220, 220, 220); --ansi-bright-black: rgb(50, 50, 50); --ansi-bright-yellow: rgb(240, 240, 50); --ansi-bright-blue: rgb(110, 110, 255); --ansi-bright-white: rgb(240, 240, 240); --ansi-bright-black: rgb(183 183 183); } pluto-log-dot.Stdout .ansi-bright-black-fg:is(.ansi-white-bg, .ansi-bright-white-bg) { text-shadow: none; color: black; } #binder_spinners { width: 100%; height: 100%; display: block; position: absolute; overflow: hidden; opacity: 0; transition: opacity 1s ease-in-out; pointer-events: none; } binder-spinner#ring_1 { border-top-color: #f5a252; width: 300px; height: 300px; margin-left: calc(-0.5 * 300px); margin-top: calc(-0.5 * 300px); animation-duration: 2s; } binder-spinner#ring_2 { border-top-color: #579aca; width: 380px; height: 380px; margin-left: calc(-0.5 * 380px); margin-top: calc(-0.5 * 380px); animation-duration: 3s; } binder-spinner#ring_3 { border-top-color: #e56581; width: 460px; height: 460px; margin-left: calc(-0.5 * 460px); margin-top: calc(-0.5 * 460px); animation-duration: 4s; } binder-spinner { top: 117px; left: 80vw; position: absolute; border: 25px solid transparent; display: block; border-radius: 100%; animation: spin 4s linear infinite; } @keyframes spin { 0% { transform: rotate(0deg); } 100% { transform: rotate(360deg); } } .edit_or_run { /* position: absolute; */ position: fixed; z-index: 2000; top: 9px; right: 8px; /* width: 153.2px; */ /* height: 60px; */ } .binder_help_text button { padding: 7px 20px; background: #fffdf7; box-shadow: 0px 0px 20px 0px #ffffff; cursor: pointer; display: block; } body.wiggle_binder .edit_or_run > button { /* position: fixed; */ animation: wiggle-binder-button 0.3s ease-in-out 0s 1; } @keyframes wiggle-binder-button { 0% { transform: rotate(0deg); } 30% { transform: rotate(10deg); } 70% { transform: rotate(-10deg); } 100% { transform: rotate(0deg); } } .binder_help_text button img { margin: -8px; margin-left: 0px; font-style: normal; color: black; font-weight: 900; height: 2.2em; } .edit_or_run > button { width: 100%; display: block; text-align: center; z-index: 2000; box-shadow: none; cursor: pointer; background: unset; background-color: var(--overlay-button-bg); border: 3px solid hsl(236deg 28% 50% / 46%); font-size: 16px; /* font-style: italic; */ font-family: var(--lato-ui-font-stack); letter-spacing: 0.1px; color: var(--black); white-space: nowrap; padding: 8px 16px; border-radius: 30px; } .edit_or_run > button:hover { text-decoration: underline; } .binder_help_text { --width: min(85vw, 570px); position: fixed; max-height: calc(100vh - 4rem); overflow: auto; width: var(--width); padding: 16px; border-radius: 8px; background-color: white; color: black; color-scheme: light; box-shadow: 0px 0px 0px 100vmax #0000004a; font-family: var(--sans-serif-font-stack); border: 0; } .binder_help_text a { color: black; } @media (max-width: 500px) { .binder_help_text { top: 0; width: 100vw; left: 0; max-height: 100vh; } } .binder_help_text .close { position: absolute; --size: 32px; top: 5px; right: 5px; width: var(--size); height: var(--size); background-size: var(--size) var(--size); cursor: pointer; background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/close-outline.svg"); } .download_div, .copy_div { padding: 4px 8px; display: flex; justify-content: space-between; align-items: center; height: 40px; margin-bottom: 0.75rem; } .download_div, .binder_help_text button, .download_div, .copy_div { width: max(60%, 10rem); margin: 0px auto; border: 3px solid #3f448c5e; border-radius: 8px; overflow: hidden; } .download_div a, .copy_div input { width: calc(100% - 8px - 1rem); outline: none; border: none; font-size: 0.7rem; font-family: "Roboto Mono", monospace; line-height: 1.4; cursor: text; } .download_div, .download_div a { cursor: pointer; } .download_icon, .copy_icon { position: relative; cursor: pointer; height: 1.5rem; width: 1.5rem; background-size: 1rem 1rem; background-position: center; background-repeat: no-repeat; background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/copy-outline.svg"); box-shadow: 0px 0px 60px 60px white; } .download_icon { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/download-outline.svg"); } .copy_icon.success_copy::after { content: "Copied!"; position: absolute; background-color: rgb(220, 235, 245); border-radius: 0.5rem; line-height: 1.5rem; text-align: center; width: 4rem; font-size: 0.75rem; font-family: JuliaMono, monospace; font-weight: bold; /* left: calc(-2rem + 0.75rem); */ animation: fadeout 3s; } .copy_icon:not(.success_copy):hover::after { content: "Copy"; position: absolute; background-color: rgb(244, 245, 220); border-radius: 0.5rem; line-height: 1.5rem; text-align: center; width: 4rem; font-size: 0.75rem; font-family: JuliaMono, monospace; font-weight: bold; } .copy_icon::after { right: 1.5rem; } @keyframes fadeout { 0% { opacity: 1; } 20% { opacity: 1; } 40% { opacity: 0; } 100% { opacity: 0; } } .command { font-size: 1.2rem; font-weight: bold; margin-bottom: 0.75rem; } .edit_or_run li { margin-bottom: 2rem; } .edit_or_run li video, .edit_or_run li img { /* outline: 1px solid black; */ border: 5px solid rgb(212, 212, 212); border-radius: 5px; width: 100%; } .expected_runtime_box { padding: 0.6em 1em; border-radius: 0.6em; font-style: italic; display: block; background: linear-gradient(45deg, hsl(222deg 52% 87%), #e5f7ff); margin: 2em 0em -2em 0em; color: #323232; } .expected_runtime_box span { font-style: initial; font-weight: bold; } @charset "UTF-8"; @import url("https://cdn.jsdelivr.net/npm/dialog-polyfill@0.5.6/dist/dialog-polyfill.css"); /* @import url("https://fonts.googleapis.com/css?family=Roboto+Mono:400,400i,500,700&display=swap&subset=cyrillic,cyrillic-ext,greek,greek-ext,latin-ext"); */ @import url("https://cdn.jsdelivr.net/npm/@fontsource/roboto-mono@4.4.5/400.css"); @import url("https://cdn.jsdelivr.net/npm/@fontsource/roboto-mono@4.4.5/400-italic.css"); @import url("https://cdn.jsdelivr.net/npm/@fontsource/roboto-mono@4.4.5/500.css"); @import url("https://cdn.jsdelivr.net/npm/@fontsource/roboto-mono@4.4.5/700.css"); /* @import url("https://fonts.googleapis.com/css?family=Alegreya+Sans:400,400i,500,500i,700,700i&display=swap&subset=cyrillic,cyrillic-ext,greek,greek-ext,latin-ext,vietnamese"); */ /* @import url("https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/400.css"); @import url("https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/500.css"); @import url("https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/700.css"); @import url("https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/400-italic.css"); @import url("https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/500-italic.css"); @import url("https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/700-italic.css"); */ @import url("./fonts/alegreya.css"); @import url("./fonts/juliamono.css"); @import url("./fonts/vollkorn.css"); /* @import url("https://fonts.googleapis.com/css2?family=Lato&display=swap"); */ @import url("https://cdn.jsdelivr.net/npm/@fontsource/lato@4.4.5/400.css"); @import url("https://cdn.jsdelivr.net/npm/@fontsource/lato@4.4.5/400-italic.css"); @import url("./themes/light.css"); @import url("./themes/dark.css"); @import url("./ansi-colors.css"); @import url("./featured-card.css"); @import url("./hide-ui.css"); /* VARIABLES */ :root { --pluto-cell-spacing: 17px; /* use the value "contextual" to enable contextual ligatures `document.documentElement.style.setProperty('--pluto-operator-ligatures', 'contextual');` for julia mono see here: https://cormullion.github.io/pages/2020-07-26-JuliaMono/#contextual_and_stylistic_alternates_and_ligatures */ --pluto-operator-ligatures: none; --julia-mono-font-stack: JuliaMono, Menlo, "Roboto Mono", "Lucida Sans Typewriter", "Source Code Pro", monospace; --sans-serif-font-stack: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Oxygen, Ubuntu, Cantarell, "Open Sans", "Helvetica Neue", sans-serif; --lato-ui-font-stack: "Lato", -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Oxygen-Sans, Cantarell, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", system-ui, sans-serif; --roboto-mono-font-stack: "Roboto Mono", "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", monospace; --system-ui-font-stack: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Oxygen-Sans, Cantarell, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", system-ui, sans-serif; color-scheme: light dark; } /* GENERAL PAGE LAYOUT */ html { font-size: 16px; } * { box-sizing: border-box; } body { margin: 0px; overflow-anchor: none; overflow-x: hidden; position: relative; min-height: 100vh; background-color: var(--main-bg-color); } pluto-editor { flex: 1 1 auto; display: flex; flex-direction: column; align-items: center; min-width: 0; } main { flex: 1; max-width: calc(700px + 25px + 6px); /* 700px + both paddings */ padding-top: 0px; padding-bottom: 4rem; padding-left: 25px; padding-right: 6px; width: 100%; } pluto-editor:not(.disable_ui) > main { padding-bottom: 16rem; } body:has(pluto-editor:not(.disable_ui)) { overscroll-behavior: contain; } /* | main=25px+700px+6px=731px | pluto-helpbox=350px - 500px | */ /* min-width: 731px+ */ pluto-editor main { align-self: flex-end; margin-right: max( /* First part: center it */ max(0px, (100% - (700px + 25px + 6px)) / 2), /* Second part: push away from the right to take up all free space */ min((100% - (700px + 25px + 6px)), /* but don't do this more than */ 500px) ); position: relative; } pluto-notebook { display: block; background: var(--main-bg-color); } /* STANDARD HTML ELEMENTS */ /* (can be overriden by custom style) */ pluto-output { font-family: "Alegreya Sans", "Trebuchet MS", sans-serif; font-size: 14.5px; font-weight: 400; color: var(--pluto-output-color); } pluto-output h1, pluto-output h2, pluto-output h3, pluto-output h4, pluto-output h5, pluto-output h6 { font-family: "Vollkorn", Palatino, Georgia, serif; font-feature-settings: "lnum", "pnum"; font-weight: 600; color: var(--pluto-output-h-color); margin-block-start: 1rem; margin-block-end: 0rem; line-height: 1.25em; } pluto-output h1, pluto-output h2 { font-weight: 700; margin-block-start: 2rem; } pluto-output h1:first-child, pluto-output h2:first-child { margin-block-start: calc(2rem - var(--pluto-cell-spacing)); } pluto-output h1 { font-size: 2.2rem; border-bottom: 3px solid var(--rule-color); margin-bottom: 0.5rem; } pluto-output h2 { font-size: 1.8rem; border-bottom: 2px dotted var(--rule-color); margin-bottom: 0.5rem; } pluto-output h1:empty, pluto-output h2:empty { border-bottom: none; } pluto-output h3 { font-size: 1.6rem; /* border-bottom: 2px dotted rgba(0,0,0,.15); */ } pluto-output h4 { font-size: 1.4rem; } pluto-output h5 { font-size: 1.2rem; } pluto-output h6 { font-size: 1rem; } pluto-output h3:first-child, pluto-output h4:first-child, pluto-output h5:first-child, pluto-output h6:first-child { margin-block-start: 0px; } pluto-output br, pluto-output p { margin-block-start: 0px; margin-block-end: var(--pluto-cell-spacing); word-spacing: 0.053em; line-height: 1.6em; } /* This allows a linebreak in Markdown using backslash with smaller spacing compared to paragraph in firefox */ pluto-output br { margin-block-end: 0; } pluto-output p:first-child { margin-block-start: 0px; } /* https://github.com/necolas/normalize.css/blob/fc091cce1534909334c1911709a39c22d406977b/normalize.css#L96 */ b, strong { font-weight: bolder; } /* I actually just want to get rid of the margin-block-start on the p tag, but css doesn't allow that. So I have to move the list up, instead of not moving it up */ pluto-output li p + ul, pluto-output li p + ol { margin-block-start: calc(var(--pluto-cell-spacing) * -1); } pluto-output p:last-child { margin-block-end: 0px; } pluto-output img, pluto-output video { max-width: 100%; } a { color: var(--black); /* font-weight: bold; */ text-decoration-thickness: 2px; text-decoration-color: var(--a-underline); /* text-shadow: 1px -1px 0px #ffffff33; */ } a:hover { text-decoration-color: var(--black); } .cm-cursor { border-left: 1.2px solid var(--cursor-color) !important; } pluto-output code { font-family: var(--julia-mono-font-stack); font-size: 0.9em; /* not rem */ font-variant-ligatures: none; } pluto-output code .cm-editor .cm-line { font-family: var(--julia-mono-font-stack); } pluto-output pre > code { font-size: inherit; } pluto-output.rich_output code { padding: 0.18em; border-radius: 8px; background-color: var(--pluto-output-bg-color); } pluto-output.rich_output pre > code { padding: 0px; background-color: transparent; } pluto-log-dot pre, pluto-output pre { display: inline-block; margin: 0px; white-space: pre-wrap; word-break: break-all; tab-size: 4; -moz-tab-size: 4; /* https://bugzilla.mozilla.org/show_bug.cgi?id=737785 */ font-family: var(--julia-mono-font-stack); font-size: 0.8rem; font-variant-ligatures: none; } pluto-display pre { white-space: pre; word-break: normal; } pluto-output hr { border: none; border-top: 3px solid var(--rule-color); margin-left: 0; margin-right: 0; } pluto-output blockquote { margin-left: 1rem; margin-right: 1rem; } pluto-output.rich_output pre:not(.no-block), pluto-output blockquote { margin-block-start: 0px; margin-block-end: var(--pluto-cell-spacing); display: block; padding: 15px; border-radius: 15px; background-color: var(--blockquote-bg); color: var(--blockquote-color); } pluto-output.rich_output pre:not(.no-block):last-child, pluto-output blockquote:last-child { margin-block-end: 0px; } pluto-output div.admonition { border-radius: 8px; margin-block-start: 1em; margin-block-end: 1em; } pluto-output div.admonition .admonition-title { font-family: "Vollkorn", Palatino, sans-serif; font-feature-settings: "lnum", "pnum"; color: var(--admonition-title-color); font-weight: 600; margin-block-end: 0px; padding-left: 0.3em; font-size: 1.3em; } pluto-output div.admonition .admonition-title ~ * { margin-block-end: 0.5em; margin-block-start: 0.5em; transition: filter linear 0.1s; } pluto-output div.admonition { padding-left: 0.5rem; padding-right: 0.5rem; background: var(--jl-message-color); border: 5px solid var(--jl-message-accent-color); } pluto-output div.admonition .admonition-title { background: var(--jl-message-accent-color); margin: -1px; /* Fixes a rendering glitch in Chrome */ margin-left: -0.55rem; margin-right: -0.55rem; } pluto-output div.admonition.note, pluto-output div.admonition.info, pluto-output div.admonition.hint { background: var(--jl-info-color); border: 5px solid var(--jl-info-accent-color); } pluto-output div.admonition.note > .admonition-title, pluto-output div.admonition.info > .admonition-title, pluto-output div.admonition.hint > .admonition-title { background: var(--jl-info-accent-color); } pluto-output div.admonition.warning { background: var(--jl-warn-color); border: 5px solid var(--jl-warn-accent-color); } pluto-output div.admonition.warning > .admonition-title { background: var(--jl-warn-accent-color); } pluto-output div.admonition.danger { background: var(--jl-danger-color); border: 5px solid var(--jl-danger-accent-color); } pluto-output div.admonition.danger > .admonition-title { background: var(--jl-danger-accent-color); } pluto-output div.admonition.hint > .admonition-title ~ * { filter: blur(0.25em); } pluto-output div.admonition.hint:hover > .admonition-title ~ *, pluto-output div.admonition.hint:focus-within > .admonition-title ~ * { filter: blur(0em); } pluto-output div.footnote { margin-block-start: 1em; margin-block-end: 1em; } pluto-output div.footnote p.footnote-title::before { content: "["; } pluto-output div.footnote p.footnote-title::after { content: "]: "; } pluto-output a.footnote, pluto-output div.footnote p.footnote-title { font-family: var(--roboto-mono-font-stack); font-size: 0.75rem; font-weight: 700; margin-block-end: 0px; letter-spacing: -0.05em; /* font-variant: small-caps; */ } pluto-output div.footnote p.footnote-title ~ * { margin-left: 0.1em; padding-left: 0.4em; border-left: 5px solid var(--footnote-border-color); padding-bottom: var(--pluto-cell-spacing); margin-block-end: 0px; } pluto-output div.footnote p:last-child { padding-bottom: 0px; } pluto-output.scroll_y { max-height: 80vh; max-height: 502px; overflow: auto; } pluto-output table { border-collapse: collapse; border: 2px solid var(--table-border-color); border-left: none; border-right: none; margin: 0 auto; margin-block-start: var(--pluto-cell-spacing); margin-block-end: var(--pluto-cell-spacing); } pluto-output table > thead { border-bottom: 1px solid var(--table-border-color); } pluto-output table > tbody td { font-family: var(--julia-mono-font-stack); font-size: 0.75rem; font-variant-ligatures: none; } pluto-output table > tbody td code { font-size: 0.75rem; } pluto-output table td, pluto-output table th { padding: 0.2rem 0.5rem; } pluto-output table > tbody tr:hover { background-color: var(--table-bg-hover-color); } pluto-output table pre { white-space: pre; } pluto-output kbd, kbd { font-family: "Space Mono", monospace; font-size: 0.7rem; letter-spacing: -0.7px; border: 1px solid var(--kbd-border-color); padding: 0px 5px; border-radius: 3px; } /* Avoid scrollbar in in-line latex with markdown, see https://github.com/fonsp/Pluto.jl/issues/1309 */ pluto-output mjx-assistive-mml { height: 1px; } /* Avoid scrollbar in cells that contain just a h3 element, like the h3 headers in https://computationalthinking.mit.edu/Fall22/images_abstractions/transformations_and_autodiff/ */ .raw-html-wrapper > div.markdown { overflow: hidden; } pluto-output details { border: 1px solid var(--rule-color); border-radius: 4px; padding: 0.5em 0.5em 0; margin-block-start: 0; margin-block-end: var(--pluto-cell-spacing); } pluto-output details:first-child { margin-block-start: 0; } pluto-output details:last-child { margin-block-end: 0; } pluto-output details summary { cursor: pointer; font-family: var(--system-ui-font-stack); font-weight: bold; border-radius: 3px; padding: 0.5em; margin: -0.5em -0.5em 0; transition: color 0.25s ease-in-out, background-color 0.25s ease-in-out; /* Border may have transparency, let's not change the border with a background-color */ background-clip: padding-box; } pluto-output details summary:hover { color: var(--blockquote-color); background-color: var(--blockquote-bg); } pluto-output details[open] { padding: 0.5em; } pluto-output details[open] summary { border-bottom: 1px solid var(--rule-color); border-bottom-right-radius: 0; border-bottom-left-radius: 0; margin-bottom: 0.5em; } /* HEADER */ header#pluto-nav { /* position: absolute; top: 0px; */ width: 100%; min-height: 60px; z-index: 60; background-color: var(--header-bg-color); transform: translateY(0px); transition: background-color 0.5s ease-in-out, transform 0.25s cubic-bezier(0.18, 0.89, 0.49, 1.13); border-bottom: solid 1px var(--header-border-color); font-family: var(--roboto-mono-font-stack); font-weight: 400; font-size: 0.8rem; } header#pluto-nav.show_export { position: sticky; top: 0; transform: translateY(130px); } dialog#export { position: absolute; display: block; top: 0; width: 100%; height: 130px; background: var(--export-bg-color); color: var(--export-color); transform: translateY(calc(-100% - 1px)); /* overlay: none !important; */ overflow: visible; margin: 0; padding: 0; max-width: none; max-height: none; border: none; } dialog#export::before { content: ""; position: absolute; bottom: 100%; left: 0; right: 0; height: 100px; background: inherit; } dialog.pride#export { background: linear-gradient( 80deg, hsl(0deg 43.55% 26.63%), hsl(30deg 50.05% 37.86%), hsl(55deg 49.41% 34.19%), hsl(140deg 12.48% 45.98%), hsl(220deg 35.66% 38.34%), hsl(280deg 30.8% 32.11%) ); } dialog#export::after { transform: translateY(1px); content: ""; position: absolute; bottom: 0; left: 0; height: 8px; width: 100%; pointer-events: none; } dialog.pride#export::after { --c1: hsl(51.39deg 89.27% 68.71%); --c2: hsl(281.12deg 68.59% 80.75%); --c3: var(--c1); --c4: hsl(0deg 0% 84.61%); --c5: hsl(334.67deg 58.4% 75.81%); --c6: hsl(204.67deg 41.19% 68.46%); --c7: hsl(41deg 70.19% 37.26%); --c8: hsl(0deg 0% 26.02%); background: repeating-linear-gradient( to right, var(--c1) 0%, var(--c1) 12.5%, var(--c2) 12.5%, var(--c2) 25%, var(--c3) 25%, var(--c3) 37.5%, var(--c4) 37.5%, var(--c4) 50%, var(--c5) 50%, var(--c5) 62.5%, var(--c6) 62.5%, var(--c6) 75%, var(--c7) 75%, /* black */ var(--c7) 87.5%, var(--c8) 87.5%, /* white */ var(--c8) 100% ); } dialog#export div#container { flex-direction: row; display: flex; max-width: 1000px; padding-right: 10em; margin: 0 auto; position: relative; } header dialog#export div#container { /* to prevent the div from taking up horizontal page when the export pane is closed. On small screen this causes overscroll on the right. */ overflow-x: hidden; } header.show_export dialog#export div#container { overflow-x: auto; } a.export_card { margin: 20px 5px; flex: 0 0 auto; width: 169px; height: 90px; border: 5px solid transparent; background: var(--export-card-bg-color); border-radius: 8px; color: var(--export-card-title-color); box-shadow: 0px 2px 10px var(--export-card-shadow-color); text-decoration: none; } div.export_title { height: 90px; flex: 0 0 auto; border-radius: 8px; text-orientation: sideways-right; /* Not supported by Chrome: */ /* writing-mode: sideways-lr; */ writing-mode: vertical-lr; transform: rotate(180deg); margin-top: 10px; font-weight: 700; font-size: 1rem; } a.export_card header { margin-block: 0px; font-family: "Vollkorn", Palatino, sans-serif; /* font-family: system-ui; */ font-feature-settings: "lnum", "pnum"; font-size: 17px; } a.export_card section { color: var(--export-card-text-color); font-weight: 500; /* font-family: system-ui; */ /* font-size: 14px; */ padding: 3px; } dialog#export .export_small_btns { display: flex; flex-direction: row; padding: 0.9em; border-radius: 0.9em; position: absolute; right: 0.8em; top: 0em; background: var(--export-bg-color); } dialog#export .pride_message { flex: 1 0 auto; align-self: center; margin-left: 32ch; } dialog#export .pride_message p { margin: 0; padding: 0.2ch 1ch; background: black; transform: rotate(3deg); } pluto-editor.static_preview button.toggle_export { display: none; } nav#at_the_top h1 { color: var(--nav-h1-text-color); letter-spacing: 2px; } nav#at_the_top { margin: 0 auto; max-width: 1000px; padding-left: 20px; padding-right: 20px; flex-wrap: wrap; min-height: 60px; display: flex; flex-direction: row; z-index: 100; } nav#at_the_top > * { flex: 0 0 auto; align-self: center; margin: 10px 0px; } nav#at_the_top > .flex_grow_1 { flex-grow: 1; } nav#at_the_top > .flex_grow_2 { flex-grow: 2; } nav#at_the_top > a { /* margin-right: 5rem; */ } nav#at_the_top h1 { font-weight: 700; font-size: 1.8rem; display: inline; border-bottom: none; } nav#at_the_top img#logo-small, nav#at_the_top img#logo-big { height: 39px; margin-bottom: -8px; filter: var(--image-filters); } nav#at_the_top img#logo-small { display: none; } @media (max-width: 800px) { nav#at_the_top img#logo-small { display: unset; } nav#at_the_top img#logo-big { display: none; } nav#at_the_top > a { /* margin-right: 1rem; */ } } nav#at_the_top > pluto-filepicker, nav#at_the_top > div.desktop_picker { width: 210px; flex-grow: 1; display: flex; flex-direction: row; } nav#at_the_top > pluto-filepicker .cm-editor, nav#at_the_top > div.desktop_picker span { height: calc(1rem + 4px + 4px + 4px); display: inline-block; min-width: 10rem; font-style: normal; font-weight: 500; font-family: inherit; font-size: 0.8rem; letter-spacing: 1px; background: none; color: var(--nav-filepicker-color); border: 2px solid var(--footer-input-border-color); border-radius: 3px; border-right: none; border-top-right-radius: 0; border-bottom-right-radius: 0; } pluto-filepicker .cm-scroller { scrollbar-width: none; /* Firefox */ } pluto-filepicker .cm-scroller::-webkit-scrollbar { display: none; /* Safari and Chrome */ } pluto-filepicker button, div.desktop_picker button { cursor: pointer; height: auto; } pluto-filepicker button:disabled { cursor: not-allowed; } /* .desktop_picker button { border-top-left-radius: 0; border-bottom-left-radius: 0; } */ div.desktop_picker span { white-space: nowrap; overflow-x: hidden; text-overflow: ellipsis; vertical-align: middle; padding: 0 5px; line-height: 1.8; cursor: pointer; } button.start_stop_recording, button.toggle_export, .export_small_btns button { border: none; background: none; cursor: pointer; opacity: 0.5; } button.start_stop_recording span, button.toggle_export span, .export_small_btns button span { display: block; content: " " !important; background-size: 25px 25px; height: 25px; width: 25px; } nav#at_the_top button.start_stop_recording span { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/radio-button-on-outline.svg"); } nav#at_the_top button.start_stop_recording.stop span { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/stop-circle-outline.svg"); } nav#at_the_top button.toggle_export span { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/share-outline.svg"); filter: var(--image-filters); } dialog#export button.toggle_export span { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/close-outline.svg"); filter: invert(1); } dialog#export button.toggle_frontmatter_edit span { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/newspaper-outline.svg"); filter: invert(1); } dialog#export button.toggle_presentation span { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/easel-outline.svg"); filter: invert(1); } nav#at_the_top:after { margin-left: auto; align-self: center; } .cm-tooltip-autocomplete li.file.new:before { content: ""; color: black; } .cm-tooltip-autocomplete li.file:before { content: ""; color: black; } .cm-tooltip-autocomplete li.dir:before { content: ""; color: black; } @media (any-pointer: fine) { nav#at_the_top > pluto-filepicker .cm-editor, nav#at_the_top > div.desktop_picker span { border: 2px solid transparent; border-right: none; transition: border 0.15s ease-in-out; } nav#at_the_top > pluto-filepicker button, nav#at_the_top > div.desktop_picker button { opacity: 0; transition: opacity 0.15s ease-in-out; } header:hover > nav#at_the_top > pluto-filepicker .cm-editor, header:focus-within > nav#at_the_top > pluto-filepicker .cm-editor, header:hover > nav#at_the_top > div.desktop_picker span, header:focus-within > nav#at_the_top > div.desktop_picker span { border: 2px solid var(--footer-input-border-color); border-right: none; } header:hover > nav#at_the_top > pluto-filepicker button, header:focus-within > nav#at_the_top > pluto-filepicker button, header:hover > nav#at_the_top > div.desktop_picker button, header:focus-within > nav#at_the_top > div.desktop_picker button { opacity: 1; } } pluto-editor.binder > header#pluto-nav > nav#at_the_top > pluto-filepicker > * { display: none; } pluto-editor.binder > header#pluto-nav > nav#at_the_top > pluto-filepicker > a { display: block; font-size: 16px; font-family: var(--julia-mono-font-stack); opacity: 0.8; text-decoration: none; } pluto-editor.nbpkg_restart_recommended > header#pluto-nav, pluto-editor.nbpkg_restart_required > header#pluto-nav, pluto-editor.binder.loading > header#pluto-nav, pluto-editor.process_waiting_for_permission > header#pluto-nav, pluto-editor.process_dead > header#pluto-nav, pluto-editor.disconnected > header#pluto-nav { position: sticky; top: 0; backdrop-filter: blur(10px); -webkit-backdrop-filter: blur(10px); } pluto-editor.nbpkg_restart_recommended > header#pluto-nav { background-color: var(--restart-recc-header-color); } pluto-editor.nbpkg_restart_required > header#pluto-nav { background-color: var(--restart-req-header-color); } pluto-editor.process_dead > header#pluto-nav { background-color: var(--dead-process-header-color); } pluto-editor.process_waiting_for_permission > header#pluto-nav { background-color: var(--restart-recc-header-color); } pluto-editor.loading > header#pluto-nav { background-color: var(--loading-header-color); } pluto-editor.disconnected > header#pluto-nav { background-color: var(--disconnected-header-color); } pluto-editor.binder.loading > header#pluto-nav { background-color: var(--binder-loading-header-color); } nav#at_the_top > #process_status { font-size: 1rem; font-weight: 500; padding: 5px 10px; margin: 5px; background: var(--nav-process-status-bg-color); color: var(--nav-process-status-color); border-radius: 10px; z-index: 10; margin-left: 1em; } nav#at_the_top > #process_status:empty { display: none; } pluto-editor.fullscreen .statefile-fetch-progress { /* width: 200px; height: 27px; background: white; border: 5px solid #d1d9e4; border-radius: 6px; */ --w: min(80vw, 300px); position: absolute; left: calc(50% - 0.5 * var(--w)); top: 0; z-index: 300000; /* box-sizing: content-box; */ /* height: 2rem; */ width: var(--w); } loading-bar { height: 6px; width: 10vw; /* background-color: #b1c9dd; */ background: linear-gradient(90deg, var(--loading-grad-color-1), var(--loading-grad-color-2), var(--loading-grad-color-1)); background-size: 100vw 100%; position: fixed; top: 0px; left: 0px; display: block; transition: width cubic-bezier(0.14, 0.71, 0, 0.99) 2s, opacity linear 0.2s; opacity: 0; z-index: 12000; animation: move-background 2s ease-in-out infinite; } loading-bar.slow { transition: width cubic-bezier(0.14, 0.71, 0, 0.99) 10s, opacity linear 0.5s; } @media (prefers-reduced-motion) { loading-bar { transition: none; } } pluto-editor.binder.loading #binder_spinners { opacity: 0.25; } @keyframes move-background { 0% { background-position-x: 0vw; } 100% { background-position-x: 100vw; } } .outline-frame { z-index: 1500; pointer-events: none; position: fixed; top: 0px; left: 0px; width: 100vw; height: 100vh; /* outline: red inset 15px; */ box-sizing: border-box; } pluto-editor.process_waiting_for_permission > .outline-frame.safe-preview { border-bottom: 12px solid var(--restart-recc-header-color); } pluto-editor.recording_waiting_to_start > .outline-frame.recording { border: 12px solid #be6f6fba; } pluto-editor.is_recording > .outline-frame.recording { border: 12px solid #645e5eba; } .outline-frame.playback { opacity: 1; position: absolute; transition: top 0.3s ease-in-out, opacity 0.3s ease-in-out; border: 12px solid #357ddcba; /* background: #7b77ff12; */ box-shadow: inset 0px 0px 20px 20px #919bff2b; } pluto-editor.recording_waiting_to_start > header#pluto-nav, pluto-editor.is_recording > header#pluto-nav { display: none; } .outline-frame-actions-container { position: fixed; top: 3px; z-index: 1501; display: flex; flex-direction: row; flex-wrap: wrap; } .outline-frame-actions-container.safe-preview { top: auto; bottom: 4px; } .outline-frame-actions-container > .overlay-button { /* background: pink; */ border-color: #e86f6c; margin: 0 3px; } .outline-frame-actions-container > .overlay-button.record-no-audio { border-color: #dcc6c6; } .outline-frame-actions-container > .overlay-button.playback { border-color: #c6c6dc; } span.pluto-icon.stop-recording-icon::after { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/stop-circle-outline.svg"); } span.pluto-icon.microphone-icon::after { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/mic-outline.svg"); } span.pluto-icon.info-icon::after { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/information-circle-outline.svg"); } span.pluto-icon.offline-icon::after { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/cloud-offline-outline.svg"); } span.pluto-icon.mute-icon::after { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/mic-off-outline.svg"); } span.pluto-icon.follow-recording-icon::after { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/arrow-undo-outline.svg"); } div.recording-playback { width: min(500px, 90vw); position: fixed; bottom: 16px; z-index: 1501; } div.recording-playback audio { width: 100%; } .safe-preview-info { color: var(--black); font-family: var(--system-ui-font-stack); font-weight: 700; background: var(--white); border: 3px solid var(--restart-recc-accent-color); padding: 0.3em 0.8em; border-radius: 0.8em; } .safe-preview-info > span { display: flex; /* flex-direction: row; */ } .safe-preview-info button { background: none; border: none; cursor: pointer; } .safe-preview-output { color: var(--helpbox-header-color); font-family: var(--system-ui-font-stack); font-weight: 700; opacity: 0.5; font-size: 0.8rem; padding: 0.2em 0.4em; background: var(--restart-recc-header-color); border-radius: 0.4em; display: inline-flex; margin: 0.7em 0; gap: 0.3em; align-items: baseline; } /* PREAMBLE */ .raw-html-wrapper.preamble { width: 100%; } main > preamble { display: flex; height: 20px; position: sticky; top: 5px; margin-top: 5px; padding-right: 5px; z-index: 200; pointer-events: none; } .overlay-button { background: var(--overlay-button-bg); color: var(--overlay-button-color); padding: 5px 8px; border: 3px solid var(--overlay-button-border); border-radius: 12px; height: 35px; font-family: var(--roboto-mono-font-stack); font-size: 0.75rem; pointer-events: all; /* break-inside: avoid; */ white-space: nowrap; } .overlay-button button:not(.asdfdsf) { color: var(--overlay-button-color); } main > preamble #saveall-container { margin-left: auto; } pluto-editor.fullscreen main > preamble #saveall-container { transform: translateX(max(0px, 100vw - 700px - 25px)); /* position: relative; */ } @media screen and (min-width: calc(700px + 25px + 6px + 500px)) { pluto-editor.fullscreen main > preamble #saveall-container { transform: translateX(500px); } } main > preamble #saveall-container.ask_to_save { border-color: var(--overlay-button-border-save); } main > preamble #saveall-container.saving, main > preamble #saveall-container.saved { border-color: transparent; } main > preamble #saveall-container.saving > span, main > preamble #saveall-container.saved > span { opacity: 0.5; } span.pluto-icon::after { content: ""; display: inline-block; padding-right: 1.5em; height: 1.3em; margin-bottom: -0.3em; transform: translateY(-0.1em); background-size: 1.3em; background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/sync-circle-outline.svg"); background-repeat: no-repeat; background-position-x: right; background-position-y: 1px; filter: var(--image-filters); } main > preamble span.saved-icon::after, .overlay-button span.saved-icon::after { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/checkmark-outline.svg"); filter: var(--image-filters); } #saveall-container .only-on-hover { display: none; } #saveall-container:hover .only-on-hover { display: inline; } /* CELL */ pluto-cell { display: block; /* CodeMirror line height (defined by their JS somehow) + 1px border top + 1px border bottom */ min-height: calc(23px + 1px + 1px); margin-top: var(--pluto-cell-spacing); position: relative; } pluto-output { display: block; padding-left: 10px; padding-right: 10px; align-items: baseline; overflow-x: auto; background-color: var(--pluto-output-bg-color); } .scroll_y { overflow-y: auto; max-height: 80vh; } pluto-output:focus { outline: none; } pluto-output:not(.rich_output) { display: flex; flex-wrap: wrap; padding-top: 3px; padding-bottom: 3px; } pluto-output > assignee { font-family: var(--julia-mono-font-stack); font-size: 0.75rem; font-variant-ligatures: none; } pluto-output > assignee::after { content: "\a0=\a0"; opacity: 0.6; } pluto-output > assignee:empty { display: none; } .errored pluto-output > assignee { display: none; } pluto-output > div { flex-shrink: 0; overflow-y: hidden; } pluto-output div.raw-html-wrapper { display: contents; } pluto-output:not(.rich_output) > div > pre { display: flex; } .bonds_disabled:where(.offer_binder, .offer_local) bond { opacity: 0.6; filter: grayscale(1); } @keyframes fadeintext { from { color: transparent; } to { color: inherit; } } /* as embedded display: */ pluto-display { display: contents; } pluto-display > div { display: contents; } /* isolated cell view */ .isolated-cell > pluto-output { padding: 0; } .hidden-cell { display: none; } /* DISABLED CELLS */ pluto-cell.depends_on_disabled_cells > pluto-output, pluto-cell.running_disabled > pluto-output, pluto-cell.depends_on_disabled_cells > pluto-trafficlight, pluto-cell.running_disabled > pluto-trafficlight, pluto-cell.depends_on_disabled_cells > pluto-input .cm-editor, pluto-cell.running_disabled > pluto-input .cm-editor, pluto-cell.depends_on_disabled_cells > pluto-logs-container, pluto-cell.running_disabled > pluto-logs-container { opacity: 0.3; } pluto-cell.running_disabled > pluto-input .cm-editor, pluto-cell.running_disabled > pluto-output { background-color: var(--disabled-cell-bg-color); } /* SKIP AS SCRIPT CELLS */ pluto-cell.skip_as_script .skip_as_script_marker, pluto-cell.depends_on_skipped_cells .depends_on_skipped_marker { display: block; position: absolute; top: 0; bottom: 0; cursor: help; z-index: 20; right: -3px; width: 4px; border-radius: 0px 4px 4px 0px; background-color: var(--skip-as-script-background-color); } pluto-cell.depends_on_skipped_cells .depends_on_skipped_marker { background-color: var(--depends-on-skip-as-script-background-color); } pluto-cell.skip_as_script pluto-input .cm-editor, pluto-cell.depends_on_skipped_cells pluto-input .cm-editor { border-bottom-right-radius: 0px; } /* SELECTION */ pluto-cell.selected { background: var(--selected-cell-bg-color); border-radius: 0 3px 3px 0; } pluto-cell.selected > pluto-input > div.cm-editor, pluto-cell.selected > pluto-output { opacity: 0.7; } main { cursor: vertical-text; } pluto-cell { cursor: auto; } /* SCROLLBAR FIREFOX */ pluto-output > div { scrollbar-width: thin; scrollbar-color: transparent transparent; } pluto-cell:hover > pluto-output > div { scrollbar-color: var(--hover-scrollbar-color-1) var(--hover-scrollbar-color-2); } /* SCROLLBAR CHROME */ pluto-output > div::-webkit-scrollbar { height: 6px; background: transparent; } pluto-output > div::-webkit-scrollbar-thumb { background: transparent; } pluto-cell:hover > pluto-output > div::-webkit-scrollbar { background: var(--hover-scrollbar-color-2); } pluto-cell:hover > pluto-output > div::-webkit-scrollbar-thumb { background: var(--hover-scrollbar-color-1); } /* CELL INPUT */ pluto-input .cm-editor { z-index: 20; border-bottom-right-radius: 4px; border: 1px solid var(--normal-cell-color); border-left: none; /* Make sure that scrolling an editor into view gives some breathing room */ scroll-margin-block: 20vh; min-height: 25px; } pluto-input:focus-within .cm-editor { /* z-index increased by 1 to make sure that the autocomplete window shows above all other editors, see https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Positioning/Understanding_z_index/The_stacking_context */ z-index: 21; } pluto-input .cm-editor .cm-line { transition: background-color 0.15s ease-in-out; } pluto-input .cm-editor span.cm-highlighted-range, pluto-input .cm-editor .cm-line.cm-highlighted-line { background-color: var(--cm-highlighted); border-radius: 3px; } pluto-cell:not(.show_input) > pluto-input { display: none; } pluto-cell.code_folded.show_input > pluto-input:not(:focus-within) { opacity: 0.4; } pluto-cell.code_differs > pluto-input > .cm-editor { border: 1px solid var(--code-differs-cell-color); border-left: none; } /* UI */ button.floating_back_button, .overlay-button button, pluto-cell > button, pluto-input > button, pluto-runarea > button, pluto-shoulder > button, nav#slide_controls > button { position: absolute; margin: 0px; padding: 1px; opacity: 50%; border: none; background: none; cursor: pointer; /* color: hsl(204, 86%, 35%); */ color: var(--ui-button-color); font-family: var(--roboto-mono-font-stack); font-size: 0.75rem; z-index: 30; /* CodeMirror is 2 */ } .overlay-button button { position: relative; } /* CELL SHOULDER */ pluto-shoulder { position: absolute; /* top: 0px; */ /* bottom: 0px; */ --invisible-border: calc(0.5 * var(--pluto-cell-spacing)); --shoulder-width: calc(28px + var(--invisible-border)); --border-radius: calc(5px + var(--invisible-border)); left: calc(0px - var(--shoulder-width)); width: var(--shoulder-width); border-radius: var(--border-radius) 0px 0px var(--border-radius); cursor: move; display: flex; flex-direction: row; justify-content: flex-end; align-items: flex-start; /* Add an invisible border around the shoulder, to make it easier to click on. (The area between two cells is divided in two, each half belongs to the closest pluto-cell.) */ top: calc(0px - var(--invisible-border)); bottom: calc(0px - var(--invisible-border)); border: var(--invisible-border) solid rgba(0, 0, 0, 0); border-right: none; } pluto-editor.fullscreen pluto-shoulder { --shoulder-width: 2000px; } pluto-shoulder:hover { background: var(--shoulder-hover-bg-color); background-clip: padding-box; } pluto-shoulder > button { flex: 0 0 auto; position: sticky; top: 0; padding: 4px 5px 4px 10px; } pluto-cell:focus-within > pluto-shoulder > button { /* we use padding instead of 4px extra margin to move the eye to the left so that the hitbox becomes grows - you want to be able to double click the button */ padding-right: 9px; } /* pluto-cell.code_folded.inline-output > pluto-shoulder > button { margin-top: 3px; } */ pluto-shoulder > button > span::after { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/eye-outline.svg"); filter: var(--image-filters); } pluto-cell.code_folded > pluto-shoulder > button > span::after { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/eye-off-outline.svg"); filter: var(--image-filters); } /* TRAFFIC LIGHT */ pluto-trafficlight { /* --patternHeight is derived from the pattern length (16px) div sin(45deg) = 16px * sqrt(2) */ --patternHeight: 22.62741699797px; box-sizing: content-box; margin-right: -1px; /* fix a visual glitch of imperfect alignment */ width: 4px; position: absolute; left: -4px; top: 0px; bottom: 0px; pointer-events: none; border-top-left-radius: 4px; border-bottom-left-radius: 4px; border-left-color: var(--normal-cell-color); background: var(--normal-cell-color); overflow: hidden; } pluto-trafficlight::after { content: ""; /* Calc needs units everywhere to work */ top: calc(0px - 10 * var(--patternHeight)); left: 0; width: 100%; height: calc(100% + 10 * var(--patternHeight)); position: absolute; opacity: 0; } /* * This class will toggle animation. * Other classes will make animation visible with opacity (which is CHEAP?) * */ pluto-cell.activate_animation pluto-trafficlight::after { animation: 10s linear 0s infinite running scrollbackground; } /* in ascending order of severity: */ /* we need the :not(.___) to fix our CSS selector specificity when `pluto-editor.disable_ui` */ pluto-editor:not(.___) pluto-cell.code_folded > pluto-trafficlight { background: none; } @media screen and (any-pointer: fine) { pluto-editor:not(.disable_ui) pluto-cell:hover > pluto-trafficlight { background: var(--normal-cell-color); } } pluto-editor:not(.___) pluto-cell:focus-within > pluto-trafficlight { border-left-width: 4px; border-left-style: solid; margin-left: -4px; background-clip: padding-box; background-color: var(--normal-cell-color); } pluto-editor:not(.___) pluto-cell.selected > pluto-trafficlight { background: var(--selected-cell-color); border-left-color: var(--selected-cell-color); background-clip: padding-box; } pluto-editor:not(.___) pluto-cell.code_differs > pluto-trafficlight { background: var(--code-differs-cell-color); border-left-color: var(--code-differs-cell-color); background-clip: padding-box; } pluto-editor:not(.___) pluto-cell.errored > pluto-trafficlight { background: var(--error-cell-color); border-left-color: var(--error-cell-color); background-clip: padding-box; } pluto-editor:not(.___) pluto-cell.queued > pluto-trafficlight::after { background: repeating-linear-gradient(-45deg, rgba(0, 0, 0, 0), rgba(0, 0, 0, 0) 8px, var(--normal-cell-color) 8px, var(--normal-cell-color) 16px); background-clip: padding-box; /* [1] Queued, Running, Errored are really fast changing props. We make sure to on-off the gradients with opacity and not toggle the animation (Which is running all this time) Make sure the browser won't skip creating a stacking context https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Positioning/Understanding_z_index/The_stacking_context https://www.w3.org/TR/css-color-3/#transparency */ opacity: 0.99; background-size: 4px var(--patternHeight); animation-duration: 20s; } pluto-editor:not(.___) pluto-cell.running > pluto-trafficlight::after { background: repeating-linear-gradient( -45deg, var(--normal-cell-color), var(--normal-cell-color) 8px, var(--dark-normal-cell-color) 8px, var(--dark-normal-cell-color) 16px ); background-clip: content-box; opacity: 0.99; /* [1] */ background-size: 4px var(--patternHeight); } pluto-editor:not(.___) pluto-cell.queued.errored > pluto-trafficlight::after { background: repeating-linear-gradient( -45deg, var(--light-error-cell-color), var(--light-error-cell-color) 8px, var(--bright-error-cell-color) 8px, var(--bright-error-cell-color) 16px ); background-clip: content-box; opacity: 0.99; /* [1] */ background-size: 4px var(--patternHeight); /* 16 * sqrt(2) */ } pluto-editor:not(.___) pluto-cell.running.errored > pluto-trafficlight::after { background: repeating-linear-gradient( -45deg, var(--light-error-cell-color), var(--light-error-cell-color) 8px, var(--bright-error-cell-color) 8px, var(--bright-error-cell-color) 16px ); background-clip: content-box; opacity: 0.99; /* [1] */ background-size: 4px var(--patternHeight); /* 16 * sqrt(2) */ } /* Define --patternHeight for this keyframes animation to work! */ @keyframes scrollbackground { 0% { transform: translate(0px, 0px); } 100% { transform: translate(0, calc(10 * var(--patternHeight))); } } /* BUTTONS */ pluto-cell > button > span, pluto-input > button > span { pointer-events: none; } @media screen and (any-pointer: fine) { pluto-cell > button, pluto-input > button, pluto-runarea > button, pluto-shoulder > button, pluto-cell > pluto-runarea { opacity: 0; /* to make it feel smooth: */ transition: opacity 0.25s ease-in-out; } .export_small_btns button, button.toggle_export, button.start_stop_recording, pluto-cell:hover > button, pluto-cell:focus-within > button, pluto-cell:hover > pluto-input > button, pluto-cell:focus-within > pluto-input > button, pluto-cell > pluto-runarea > button, pluto-cell:hover > pluto-shoulder > button, pluto-cell:focus-within > pluto-shoulder > button { opacity: 0.6; transition: opacity 0.25s ease-in-out; } .export_small_btns button:hover, button.toggle_export:hover, button.start_stop_recording:hover, .overlay-button button:hover, pluto-cell > button:hover, pluto-cell > pluto-input > button:hover, pluto-cell > pluto-runarea > button:hover, pluto-cell > pluto-shoulder > button:hover, pluto-cell:hover > pluto-runarea { opacity: 1; /* to make it feel snappy: */ transition: opacity 0.05s ease-in-out; } } @media screen and (pointer: coarse) { pluto-cell > button.add_cell, pluto-input > button, pluto-shoulder > button { opacity: 0.25; transition: opacity 0.25s ease-in-out; } pluto-cell:not(:first-of-type, :last-of-type) > button.add_cell { /* because there are two overlapping buttons */ opacity: 0.125; } /* pluto-cell:first-of-type > button.add_cell.before, pluto-cell:last-of-type > button.add_cell.after { opacity: 0.25; } */ pluto-cell:focus-within > button.add_cell, pluto-cell:focus-within > pluto-input > button, pluto-cell:focus-within > pluto-runarea, pluto-cell:focus-within > pluto-shoulder > button { opacity: 0.6; transition: opacity 0.25s ease-in-out; } pluto-cell > pluto-input > button:focus-within, pluto-cell > button:focus-within, pluto-cell > pluto-input > button:focus-within pluto-cell > pluto-runarea > button:focus-within, pluto-cell > pluto-shoulder > button:focus-within, pluto-cell > pluto-runarea { opacity: 1; /* to make it feel snappy: */ transition: opacity 0.05s ease-in-out; } } pluto-cell > button > span::after, pluto-input > button > span::after, pluto-runarea > button > span::after, pluto-shoulder > button > span::after { display: block; content: " " !important; background-size: 17px 17px; height: 17px; width: 17px; } pluto-cell > button.add_cell { left: -12px; --hit-box-extend: 20px; margin-left: calc(-1 * var(--hit-box-extend)); margin-right: calc(-1 * var(--hit-box-extend)); padding-left: var(--hit-box-extend); padding-right: var(--hit-box-extend); } pluto-cell > button.add_cell.before { margin-top: calc(-19px - 0.5 * (var(--pluto-cell-spacing) - 19px)) !important; } pluto-cell > button.add_cell.after { bottom: 1px; margin-bottom: calc(-20px - 0.5 * (var(--pluto-cell-spacing) - 19px)); } pluto-cell > button.add_cell > span::after { /* background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/add-circle-outline.svg"); */ background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/add-outline.svg"); filter: var(--image-filters); } pluto-input > .input_context_menu ul { margin: 0; padding: 0px; border-radius: 6px; display: grid; grid-template-columns: max-content; border: 1px solid var(--input-context-menu-border-color); background-color: var(--input-context-menu-bg-color); } pluto-input { position: relative; display: block; } pluto-input > div.input_context_menu { left: 100%; top: -8px; position: absolute; z-index: 1400; } @media screen and (min-width: 921px) { pluto-input > div.input_context_menu { left: calc(100% - 3px); } } @media screen and (max-width: 920px) { pluto-input > div.input_context_menu { right: 0px; left: unset; z-index: 1401; } } pluto-input > .input_context_menu li { list-style: none; margin-block-end: 0px; display: flex; align-items: stretch; flex-direction: column; } pluto-input > .input_context_menu li button { font-family: var(--system-ui-font-stack); font-size: 0.9rem; margin-block-end: 0px; color: var(--input-context-menu-li-color); position: relative; padding: 8px; height: 32px; display: flex; align-items: center; justify-content: flex-start; border-radius: 2px; cursor: pointer; background: none; border: none; } .input_context_menu li:last-child { border-bottom-left-radius: 6px; border-bottom-right-radius: 6px; } .input_context_menu li:first-child { border-top-left-radius: 6px; border-top-right-radius: 6px; } .input_context_menu li.coming_soon { color: var(--input-context-menu-soon-color); } .input_context_menu li.coming_soon:hover { cursor: not-allowed; background-color: var(--input-context-menu-hover-bg-color); } .input_context_menu li:hover { transition-property: background-color; transition-duration: 200ms; background-color: var(--input-context-menu-hover-bg-color); } .ctx_icon, .icon { --size: 17px; width: var(--size); height: var(--size); margin-top: 1px; background-size: var(--size) var(--size); background-position: center; margin-right: calc(var(--size) / 3); } .ctx_icon { filter: var(--image-filters); } .ctx_icon.show_logs { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/document-text-outline.svg"); } .ctx_icon.hide_logs { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/document-text-outline.svg"); } .ctx_icon.enable_cell { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/caret-forward-circle-outline.svg"); } .ctx_icon.disable_cell { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/ban-outline.svg"); background-size: 15px; background-repeat: no-repeat; } .ctx_icon.delete { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/close-circle-outline.svg"); } .ctx_icon.run_as_script { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/document-lock-outline.svg"); } .ctx_icon.skip_as_script { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/document-text-outline.svg"); } .ctx_icon.copy_output { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/copy-outline.svg"); } .ctx_icon.ask_ai { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/chatbubble-ellipses-outline.svg"); } pluto-input > button.input_context_menu { /* top: 3px; */ /* left: -26px; */ right: 0px; padding: 5px; } pluto-input > .input_context_menu.open { opacity: 1; } pluto-input > .input_context_menu span.icon::after { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/ellipsis-horizontal-circle-outline.svg"); filter: var(--image-filters); } pluto-input > .preview_hidden_code_info { display: none; position: absolute; bottom: -1.1rem; left: 0; right: 0; margin-right: auto; margin-left: auto; height: 1.4rem; width: 19ch; font-style: italic; text-align: center; background: var(--jl-info-color); border-radius: 0.4rem; font-family: var(--system-ui-font-stack); font-size: 0.9rem; z-index: 22; pointer-events: none; } pluto-editor:not(.process_waiting_for_permission) pluto-cell.code_folded > pluto-input > .preview_hidden_code_info, pluto-cell.code_folded:focus-within > pluto-input > .preview_hidden_code_info { display: block; } /* AI Context */ pluto-popup.ai-context > div { background: var(--ai-gradient-bg); box-shadow: 0 0 3rem #00000061; } .ai-context-container { padding: 0.2rem; } .ai-context-container h2 { margin: 0 0 0.5rem 0; font-size: 1.2rem; color: var(--pluto-output-h-color); } .ai-context-intro { margin: 0.5rem 0; font-size: 0.9rem; color: var(--pluto-output-color); } .ai-context-question-input { width: 100%; padding: 8px 12px; margin: 8px 0; border: 3px solid transparent; border-radius: 4px; font-size: 0.95em; background: var(--main-bg-color); transition: border-color 0.2s; } .ai-context-question-input:focus, .ai-context-question-input:not(:placeholder-shown) { outline: none; border-color: var(--overlay-button-border); } .ai-context-question-input::placeholder { color: var(--pluto-input-color); opacity: 0.6; } .ai-context-prompt-container { position: relative; margin-top: 1rem; border-radius: 6px; box-shadow: 0 2px 11px #0000001f; background: var(--ai-prompt-bg); } .ai-context-prompt { max-height: 15ch; overflow-y: auto; padding: 0.5rem; line-height: 1.5; } .ai-context-prompt pre { margin: 0; white-space: pre-wrap; word-wrap: break-word; font-family: inherit; font-size: 0.5rem; } .ai-context-container .copy-button { position: absolute; bottom: 0.5rem; right: 0.5rem; padding: 0.5rem 1rem; background: var(--white); border: 3px solid var(--overlay-button-border); border-radius: 0.7rem; color: var(--black); font-size: 0.9rem; font-weight: bold; cursor: pointer; transition: all 0.2s ease; /* box-shadow: 0 1px 10px #0000001a; */ } .ai-context-container .copy-button.copied { /* color: var(--pluto-output-bg); */ } /* PKG UI */ pkg-status-mark { width: 1em; height: 1em; margin: 0px 0.6em 0px 0.2em; display: inline-block; } pluto-editor.nbpkg_disabled pkg-status-mark:not(.disable_pkg) { display: none; } pkg-status-mark > button { margin: 0px; padding: 0px; border: none; background: none; cursor: context-menu; position: relative; top: -0.2em; /* background: rgba(127, 176, 76, 0.24); border: 3px solid rgba(84, 182, 237, 0); border-radius: 5px; */ } pkg-status-mark > button > span::after { display: inline-block; content: " " !important; background-size: 1.5em; height: 1.5em; width: 1.5em; background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/time-outline.svg"); opacity: 0.3; filter: var(--image-filters); } pkg-status-mark.installed > button > span::after { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/checkmark-outline.svg"); /* background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/cloud-done-outline.svg"); */ filter: var(--image-filters); } pkg-status-mark.busy > button > span::after { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/sync-outline.svg"); animation: loadspin 3s ease-in-out infinite; filter: var(--image-filters); } pkg-status-mark.not_found > button > span::after { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/cloud-offline-outline.svg"); opacity: 0.6; filter: var(--image-filters); } pkg-status-mark.will_be_installed > button > span::after { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/cloud-download-outline.svg"); opacity: 0.6; filter: var(--image-filters); } pkg-status-mark.disable_pkg > button > span::after { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/chatbox-ellipses-outline.svg"); opacity: 0.6; filter: var(--image-filters); } pluto-popup { display: block; position: absolute; z-index: 1800; /* left: 1.5em; */ /* top: calc((1.5em - 200px) * 0.5); */ --max-size: 251px; width: min(90vw, var(--max-size)); /* min-height: 80px; */ margin-left: 0.4rem; margin-top: 0px; margin-top: -1rem; /* margin-top: calc(0.5 * (1rem - 80px)); */ overflow-wrap: break-word; font-family: var(--system-ui-font-stack); opacity: 0; transform: scale(0.2); transform-origin: left; transition: transform 0.5s ease-in-out, opacity 0.1s ease-in-out; pointer-events: none; } pluto-popup.visible { opacity: 1; transform: scale(1); transition: transform 0.2s ease-in-out, opacity 0.2s ease-in-out; pointer-events: initial; } pluto-popup.big { --max-size: 25em; } pluto-popup > * { display: block; background: var(--overlay-button-bg); border: 3px solid var(--overlay-button-border); color: var(--black); border-radius: 10px; padding: 8px; /* Slightly changes the layout of the three pkg buttons... in just the way that we want it! */ position: absolute; max-width: 100%; max-height: 80vh; overflow-y: auto; } pluto-popup > div > *:first-child { margin-block-start: 0; } pluto-popup h1 { font-size: 1.6em; } pluto-popup.warn > * { background: var(--pluto-logs-warn-color); border-color: var(--pluto-logs-warn-accent-color); } pluto-popup code.auto_disabled_variable { font-family: var(--julia-mono-font-stack); font-size: 0.8rem; font-variant-ligatures: none; } pluto-popup > pkg-popup { background: var(--pkg-popup-bg); border: 3px solid var(--pkg-popup-border-color); } pkg-popup.busy { border: 3px solid hsl(282deg 31% 62%); } pkg-version { font-family: "Space Mono", monospace; font-size: 0.75rem; opacity: 0.5; } pkg-popup .pkg-buttons { float: right; /* position: absolute; */ /* right: 0px; */ /* bottom: 0px; */ display: inline-flex; height: 1em; flex-direction: row; } .ionicon { filter: var(--image-filters); } .package-name .ionicon { margin-bottom: -0.1ch; } a.stdout-info img, pkg-popup .pkg-buttons img { filter: var(--image-filters); width: 17px; } a.stdout-info, pkg-popup .pkg-buttons > * { display: block; height: 17px; padding: 4px; box-sizing: content-box; background: var(--pkg-popup-buttons-bg-color); border-radius: 10px; z-index: 52; margin-left: -4px; } pkg-popup .toggle-terminal { right: 20px; } .pkg-time-estimate { font-size: 0.8em; margin: 0.5em 0em; padding: 0.5em 0.5em; background: var(--pluto-logs-warn-color); border-radius: 0.5em; } pkg-terminal { display: block; /* width: 20rem; */ cursor: text; margin-top: 6px; background: var(--pkg-terminal-bg-color); color: white; border-radius: 6px; border: 3px solid var(--pkg-terminal-border-color); padding: 3px; } pkg-terminal > .scroller { max-height: 10rem; overflow-y: auto; padding: 4px; width: 100%; } body pkg-terminal:not(.asdf) pre:not(.asdf) { white-space: pre-wrap; word-break: break-all; font-size: 0.6rem; font-family: "Space Mono", monospace; font-variant-ligatures: none; margin: 0; color: inherit; background: none; } pkg-terminal .make-me-spin { display: inline-block; animation: identifier-spin 1s linear infinite; transform-origin: 50% 59%; animation-delay: var(--animation-delay); } pkg-popup pkg-terminal { display: none; } pkg-popup.showterminal pkg-terminal { display: block; } @keyframes loadspin { 0% { transform: rotate(0deg); } 25% { transform: rotate(180deg); } 50% { transform: rotate(180deg); } 75% { transform: rotate(360deg); } 100% { transform: rotate(360deg); } } /* RUNAREA */ pluto-runarea { margin-right: 3px; display: block; height: 17px; position: absolute; right: 0px; min-width: 75px; background-color: var(--pluto-runarea-bg-color); /* border: 2px solid hsla(0, 0%, 0%, 0.1); */ border-bottom-left-radius: 5px; border-bottom-right-radius: 5px; border-top: none; /* One less than z-index for pluto-input .cm-editor. Otherwise it gets on top of the tooltips */ z-index: 19; } pluto-runarea > span { display: inline-block; position: absolute; top: 1px; left: 22px; width: 45px; font-family: var(--roboto-mono-font-stack); font-size: 0.6em; font-style: italic; color: var(--pluto-runarea-span-color); text-align: center; } pluto-runarea > button.runcell { top: -1px; left: 1px; } pluto-runarea > button.runcell > span::after { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/caret-forward-circle-outline.svg"); filter: var(--image-filters); } pluto-runarea.interrupt > button.runcell > span::after { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/stop-circle-outline.svg"); filter: var(--image-filters); } pluto-runarea.jump > button.runcell > span::after { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/arrow-redo-circle-outline.svg"); filter: var(--image-filters); } pluto-runarea.save > button.runcell > span::after { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/chevron-forward-circle-outline.svg"); filter: var(--image-filters); } pluto-cell:not(.show_input) > pluto-runarea { display: none; } pluto-cell.code_folded.show_input > pluto-input:not(:focus-within) { opacity: 0.4; } pluto-cell:focus-within > pluto-runarea, pluto-cell.code_differs > pluto-runarea { opacity: 100%; } pluto-cell.code_differs > pluto-runarea > button { animation-name: ; animation-duration: 4s; animation-iteration-count: infinite; } @keyframes  { 0%, 80%, 100% { transform: scale(1); opacity: 50%; } 90% { transform: scale(1.2); opacity: 100%; } } /* DRAG DROP */ dropruler { position: absolute; display: none; left: 0px; right: 0px; height: 4px; margin-top: calc(-2px - 0.5 * var(--pluto-cell-spacing)); background: var(--dropruler-bg-color); transition: top 0.05s; } /* LIVE DOCS */ #helpbox-wrapper { display: none; position: sticky; bottom: 0px; height: 0px; width: 100%; z-index: 50; } @media (min-width: 500px) { #helpbox-wrapper { display: block; } } pluto-helpbox { bottom: 0px; right: 20px; position: absolute; display: flex; flex-direction: column; width: calc(100vw - 50px - (700px + 25px + 6px)); /* compat */ width: clamp(300px, calc(100vw - 50px - (700px + 25px + 6px)), 450px); height: 95vh; /* compat */ height: min(70vh, 900px); /* overflow: hidden; */ background-color: var(--helpbox-bg-color); color: var(--helpbox-text-color); /* border: 2px solid darkgray; */ border-right: none; border-bottom: none; border-top-left-radius: 9px; border-top-right-radius: 9px; box-shadow: 0 0 11px 0px var(--helpbox-box-shadow-color); } pluto-helpbox > section { height: 100%; overflow: auto; padding: 10px; display: flex; flex-direction: column; } pluto-helpbox > header { display: flex; padding: 0.6em; background-color: var(--helpbox-header-bg-color); color: var(--helpbox-header-color); font-family: var(--system-ui-font-stack); font-variant-numeric: tabular-nums; font-size: 0.9rem; --border-radius: 0.4em; font-weight: 500; border-top-left-radius: var(--border-radius); border-top-right-radius: var(--border-radius); /* border-top: 4px solid #8a8a8a; */ gap: 0.5em; } pluto-helpbox > header > button.helpbox-tab-key > .tabicon { /* font-size: 1em; */ --size: 1.1em; width: var(--size); height: var(--size); background-size: var(--size); margin-bottom: calc(-0.15 * var(--size)); filter: var(--image-filters); margin-right: 0.6em; display: inline-block; } pluto-helpbox > header > button.helpbox-docs > .tabicon { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/search.svg"); /* content: " "; */ } pluto-helpbox > header > button.helpbox-docs.active > .tabicon { /* content: " "; */ } pluto-helpbox > header > button.helpbox-process > .tabicon { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/terminal.svg"); background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/pulse.svg"); /* content: " "; */ } pluto-helpbox > header > button.helpbox-tab-key:disabled > .tabicon { /* display: none; */ opacity: 0.5; } /* button.helpbox-process.helpbox-tab-key::before { content: ""; display: block; position: absolute; --offset: -5px; top: var(--offset); right: var(--offset); left: var(--offset); bottom: var(--offset); background: var(--process-busy); border-radius: var(--border-radius); background: linear-gradient(447deg, var(--process-finished), var(--process-finished) 50%, var(--process-busy) 50%, var(--process-busy)); background: linear-gradient( 447deg, var(--process-busy) 0%, var(--process-busy) 25%, var(--process-finished) 25%, var(--process-finished) 50%, var(--process-busy) 50%, var(--process-busy) 75%, var(--process-finished) 75%, var(--process-finished) 100% ); animation: move-bg 5s linear infinite; background-size: 100px auto; z-index: -1; opacity: 0.6; } */ /* @keyframes rotate-bg { 0% { background: linear-gradient( 0deg, var(--process-busy) 0%, var(--process-busy) 20%, var(--process-finished) 20%, var(--process-finished) 40%, var(--process-busy) 40%, var(--process-busy) 60%, var(--process-finished) 60%, var(--process-finished) 80%, var(--process-busy) 80%, var(--process-busy) ); } 100% { background: linear-gradient( 70deg, var(--process-busy) 0%, var(--process-busy) 20%, var(--process-finished) 20%, var(--process-finished) 40%, var(--process-busy) 40%, var(--process-busy) 60%, var(--process-finished) 60%, var(--process-finished) 80%, var(--process-busy) 80%, var(--process-busy) ); } } */ /* @keyframes move-bg { 0% { background-position-x: 0; } 100% { background-position-x: 100px; } } button.helpbox-process.helpbox-tab-key::after { content: ""; display: block; position: absolute; top: 0; right: 0; left: 0; bottom: 0; border-radius: var(--border-radius); border: none; background: var(--helpbox-header-tab-bg-color); z-index: -1; } */ pluto-helpbox .live-docs-searchbox { display: flex; margin: 1em; } pluto-helpbox .live-docs-searchbox input { flex-grow: 1; background-color: inherit; color: inherit; border: none; padding: 0.5em; margin: auto; border: 3px solid var(--helpbox-search-border-color); border-radius: 0.3em; background: var(--helpbox-search-bg-color); font-family: var(--julia-mono-font-stack); font-size: 0.9rem; } pluto-helpbox .live-docs-searchbox.notfound input { color: var(--helpbox-notfound-search-color); } pluto-helpbox .live-docs-searchbox input:focus { outline: none; } button.helpbox-tab-key { padding: 0.5em 0.6em; cursor: pointer; font-family: inherit; font-weight: inherit; font-style: inherit; font-size: inherit; font-variant: inherit; border-radius: var(--border-radius); border: none; background: var(--helpbox-header-tab-bg-color); overflow: hidden; text-overflow: ellipsis; white-space: nowrap; /* position: relative; */ /* flex: 1 1 auto; */ /* z-index: 1; */ /* overflow: hidden; */ } button.helpbox-tab-key.helpbox-process { margin-right: auto; } button.helpbox-process.busy { outline: 6px solid var(--process-busy); } @media (prefers-reduced-motion: no-preference) { button.helpbox-process.busy { animation: outline-heartbeat 0.8s ease-in infinite; animation-direction: alternate; } } @keyframes outline-heartbeat { 0% { outline-offset: -1px; outline-width: 3px; } 100% { outline-offset: 0px; outline-width: 6px; } } button.active.helpbox-tab-key { outline: 3px solid #99afb9; animation: none; } pluto-helpbox > header > button:is(.helpbox-close, .helpbox-popout) { border: none; background: none; cursor: pointer; /* for bigger hitbox */ margin: -15px; border: 15px solid transparent; } pluto-helpbox > header > button:is(.helpbox-close, .helpbox-popout) > span { display: block; content: " " !important; background-size: 1em 1em; height: 1em; width: 1em; background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/chevron-down-outline.svg"); filter: var(--image-filters); } pluto-helpbox > header > button.helpbox-popout > span { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/open-outline.svg"); } pluto-helpbox.hidden { height: initial; width: auto; } pluto-helpbox.hidden > section { display: none; } /* pop out */ body > pluto-helpbox { position: static; width: auto; height: 100vmax; } body > pluto-helpbox > header > button:is(.helpbox-close, .helpbox-popout) { display: none; } /* NOTE */ /* We try to match the visual style of Documenter.jl, so we have copied over some rules from */ /* https://docs.julialang.org/en/v1/assets/themes/documenter-light.css */ /* see https://github.com/JuliaDocs/Documenter.jl for author information */ .helpbox-docs { font-family: var(--lato-ui-font-stack); line-height: 1.5; /* font-size: 0.9rem; */ } .helpbox-docs pre, .helpbox-docs code, .helpbox-docs .cm-line { /* from https://docs.julialang.org/en/v1/assets/themes/documenter-light.css */ font-family: var(--julia-mono-font-stack); font-variant-ligatures: none; font-size: 0.95em; line-height: initial; } .helpbox-docs pre code { font-size: 1em; } .helpbox-docs pre code.hljs { padding: 0; } .helpbox-docs code .cm-editor .cm-content { padding: 0px; } .helpbox-docs img { max-width: 100%; } .helpbox-docs > section h1, .helpbox-docs > section h2, .helpbox-docs > section h3, .helpbox-docs > section h4, .helpbox-docs > section h5, .helpbox-docs > section h6 { font-family: inherit; border-bottom: none; font-size: 1rem; } .helpbox-docs > section h1 { font-size: 1.3rem; overflow-wrap: anywhere; } .helpbox-docs > section pre { padding: 0.7rem 0.5rem; -webkit-overflow-scrolling: touch; overflow-x: auto; background: var(--code-section-bg-color); border: 1px solid var(--code-section-border-color); border-radius: 4px; white-space: pre; word-wrap: normal; } /* .helpbox-docs > section code { background-color: whitesmoke; padding: 0.1em; } */ .helpbox-docs > section hr { border: none; border-top: 3px solid var(--rule-color); } .pluto-docs-binding { margin: 0.5em; padding: 1em; border-radius: 1em; background: var(--docs-binding-bg); color: var(--pluto-output-color); } .pluto-docs-binding > span { display: inline-block; transform: translate(-1.2em, -73%); font-family: var(--julia-mono-font-stack); font-size: 0.9rem; font-weight: 700; margin-top: -1em; padding: 0.235rem; border-radius: 0.4rem; background: var(--pluto-output-bg-color); color: var(--black); } .pluto-docs-binding h1 { font-size: 1.4em; } .pluto-docs-binding h2 { font-size: 1.3em; } .pluto-docs-binding h3, .pluto-docs-binding h4, .pluto-docs-binding h5, .pluto-docs-binding h6 { font-size: 1.1em; } /* PROCESS TAB */ pl-status { --status-color: var(--process-undefined); font-family: var(--system-ui-font-stack); font-size: 0.9rem; display: flex; flex-direction: column; border-radius: 0.2em; --indent: 0.7rem; margin-left: var(--indent); border-left: 3px solid transparent; margin-top: 0.4em; overflow: hidden; flex: 0 0 auto; } pl-status::before { flex: 1 2 auto; /* content: ""; */ display: inline-block; left: 0; right: 0px; width: 3px; height: 10px; bottom: 3px; top: 3px; background: pink; } pl-status.busy { --status-color: var(--process-busy); } pl-status.finished { --status-color: var(--process-finished); } pl-status.failed { --status-color: var(--process-failed); } pl-status.can_open { cursor: auto; border-color: #98989854; } pl-status.can_open > div { border-top-left-radius: 0; border-bottom-left-radius: 0; cursor: pointer; } pl-status.can_open.is_open { border-color: var(--status-color); } pl-status[data-depth="0"], pl-status[data-depth="1"] { margin-left: 0; } pl-status > div { display: flex; flex-direction: row; align-items: center; /* margin: 0em 0em 0.4em 0; */ padding: 0.2em; background: var(--process-item-bg); border-radius: 0.4em; /* flex: 1 0 auto; */ } pl-status > div > .status-icon { content: ""; display: inline-block; width: 1em; height: 1em; border-radius: 50%; background-color: var(--status-color); /* border: 3px solid green; */ margin: 0em 0.5em; flex: 0 0 auto; } pl-status.busy > div > .status-icon { border: 3px solid transparent; border-right-color: hsl(126deg 30% 60%); border-bottom-color: hsl(126deg 30% 60%); /* border-left-color: hsl(126deg 30% 60%); */ animation: identifier-spin 1.7s linear infinite; } @keyframes identifier-spin { 0% { transform: rotate(0deg); } 100% { transform: rotate(360deg); } } .subprogress-counter { opacity: 0.5; /* font-variant-numeric: tabular-nums; */ font-size: 0.8em; } pl-status .status-time { margin-left: auto; /* align-self: end; */ padding-right: 0.5em; padding-left: 0.5em; opacity: 0.6; font-size: 0.7rem; font-variant-numeric: tabular-nums; } .discrete-progress-bar { display: flex; flex-direction: row; background: var(--process-item-bg); padding: 3px; border-radius: 4px; gap: 2px; contain: strict; height: 1em; align-items: stretch; } .discrete-progress-bar > div { background: var(--process-undefined); flex: 1 1 auto; border-radius: 2px; /* margin: 0.4px; */ } .discrete-progress-bar > div.done { background: var(--process-finished); } .discrete-progress-bar > div.busy { background: var(--process-busy); } .discrete-progress-bar > div.failed { background: var(--process-failed); } .discrete-progress-bar.mid { gap: 1px; } .discrete-progress-bar.big { gap: 0px; } pl-status pkg-terminal { margin-left: var(--indent); } pluto-helpbox.helpbox-process > section { padding-bottom: 3.6rem; /* to make space for the notification toggle */ } .notify-when-done { position: absolute; bottom: 0.3em; right: 0; left: 0; font-family: var(--system-ui-font-stack); font-weight: bold; font-size: 0.8rem; transition: opacity 0.2s; display: flex; justify-content: center; opacity: 0; user-select: none; } .notify-when-done.visible { opacity: 1; } .notify-when-done.visible label { cursor: pointer; } .notify-when-done label { display: flex; align-items: center; background: var(--process-notify-bg); padding: 0.3em 0.6em; border-radius: 1000px; box-shadow: 0px 3px 5px #0000003b; } .notify-when-done label::before { content: ""; display: inline-block; width: 1em; height: 1em; background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/notifications-outline.svg"); background-size: contain; filter: var(--image-filters); margin-bottom: -0.2em; margin-right: 0.3em; } /* FOOTER */ footer { width: 100%; min-height: 3.5rem; font-family: var(--roboto-mono-font-stack); font-size: 0.75rem; background-color: var(--footer-bg-color); color: var(--footer-color); z-index: 70; } footer form { min-height: 1.5rem; opacity: 1; transition: opacity 5s; display: flex; flex-wrap: wrap; } footer form > * { flex: 0 0 auto; } footer form > a, footer form > label { align-self: center; } footer form > label { margin-right: 1em; } footer form > a { margin-right: 1em; } footer a { color: var(--footer-atag-color); opacity: 0.6; font-weight: 700; } @media (max-width: 650px) { footer form > label { display: none; } } footer input { margin: 0px; border: 2px solid var(--footer-input-border-color); background: var(--white); font-family: inherit; font-size: inherit; border-radius: 3px 0 0 3px; padding: 3px; border-right: none; } header#pluto-nav pluto-filepicker button, footer button { margin: 0px; background: var(--footer-filepicker-focus-color); border-radius: 0 3px 3px 0; border: 3px solid var(--footer-filepicker-focus-color); color: var(--footer-filepicker-button-color); /* border: none; */ font-family: inherit; font-weight: 600; height: auto; font-size: 0.75rem; } footer #info { max-width: 9400px; margin: 0 auto; padding: 1rem; text-align: right; /* height: 1.5rem; */ display: flex; flex-direction: row; justify-content: flex-end; align-items: center; } /* UNDO DELETED CELL */ nav#undo_delete { z-index: 2000; display: block; position: fixed; bottom: 0px; left: 0px; margin: 0.75rem; padding: 0.5rem; font-family: var(--system-ui-font-stack); font-size: 0.8rem; background-color: var(--helpbox-header-tab-bg-color); border-radius: 8px; color: var(--black); box-shadow: 00px 00px 10px 2px var(--undo-delete-box-shadow-color); opacity: 1; transition: 0.2s linear box-shadow, 0.2s ease-out margin-bottom, 0.05s linear opacity; } nav#undo_delete.hidden { margin-bottom: 0px; box-shadow: 00px 00px 10px -5px var(--undo-delete-box-shadow-color); opacity: 0; pointer-events: none; } @keyframes shadow-fadeout { 0% { box-shadow: 00px 00px 10px 2px var(--undo-delete-box-shadow-color); opacity: 1; } 80% { opacity: 1; } 100% { opacity: 0; } } /* LOGS */ @media screen and (max-width: calc(700px + 25px + 6px + 100px)) { pluto-cell.shrunk { /* max-width: calc(100% - 100px); */ } } pluto-logs-container { display: block; /* Show logs up to the RunArea */ z-index: 25; overflow-x: hidden; overflow-y: auto; max-height: 50vh; margin-right: 1.3rem; } pluto-logs-container:not(:empty) { background: var(--pluto-logs-bg-color); padding: 6px; } pluto-logs-container > header { font-family: var(--roboto-mono-font-stack); font-size: 1.3rem; padding: 0.2em; padding-bottom: 0; opacity: 0.6; font-weight: 700; /* background: #494949; */ } pluto-logs-container pluto-progress-bar-container { overflow: hidden; outline: 3px solid var(--pluto-logs-progress-border); outline-offset: -2px; border-radius: 6px; background: var(--pluto-logs-progress-bg); font-size: 0.7rem; flex: 0 1 200px; } pluto-logs-container pluto-progress-name { white-space: pre-wrap; font-family: var(--julia-mono-font-stack); font-size: 0.8rem; font-variant-ligatures: none; padding: 0 0.4rem 0 0.1rem; } pluto-logs-container pluto-progress-name:empty { padding: 0; } pluto-logs-container pluto-progress-bar { --c: var(--pluto-logs-progress-fill); padding: 0.3em 0.6em; background: linear-gradient(90deg, var(--c), var(--c)); background-repeat: no-repeat; transition: background-size cubic-bezier(0.14, 0.71, 0, 0.99) 0.5s, opacity linear 0.2s; display: grid; align-items: center; } pluto-logs-container pluto-progress-bar.collapsed { height: 0; } pluto-logs { display: flex; flex-direction: column; } pluto-logs:not(:first-child):not(:empty) { margin-top: 10px; } pluto-log-dot { /* part 2 */ /* box-shadow: -2px 0px 1px #00000014; */ font-family: var(--roboto-mono-font-stack); font-size: 0.6rem; position: relative; display: flex; flex-grow: 1; flex-direction: column; padding: 1px 3px; min-width: 18px; min-height: 18px; /* border-radius: 7px; */ padding: 0.6em 0.9em 0.6em 0.3em; } pluto-log-dot-positioner { /* border-bottom: 1px solid #71717140; */ display: flex; flex-direction: row; --bg-color: var(--pluto-logs-info-color); --accent-color: var(--pluto-logs-info-accent-color); --icon-image: unset; background: var(--bg-color); margin: 2px; border-radius: 6px; /* border: 2px solid var(--accent-color); */ /* border: 2px solid #0000001c; */ /* box-shadow: 0px 0px 6px #00000036; */ background: linear-gradient(148deg, var(--bg-color), transparent); background-size: 200% 100%; } pluto-log-dot > pre { color: var(--accent-color); } pluto-log-truncated { display: grid; place-items: center; font-family: var(--system-ui-font-stack); opacity: 0.7; padding: 0.7em; font-style: italic; } pluto-log-icon::before { content: ""; width: 1em; height: 1em; background-image: var(--icon-image); background-size: 1em; filter: var(--image-filters); display: inline-flex; margin: 0.3em; } pluto-log-dot-positioner.Info { --icon-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/information-circle-outline.svg"); } pluto-log-dot-positioner.Info pluto-log-icon::before { opacity: 0.4; } pluto-log-dot-positioner.Warn { --bg-color: var(--pluto-logs-warn-color); --accent-color: var(--pluto-logs-warn-accent-color); --icon-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/warning-outline.svg"); } pluto-log-dot-positioner.Error { --bg-color: var(--pluto-logs-danger-color); --accent-color: var(--pluto-logs-danger-accent-color); --icon-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/close-circle-outline.svg"); } pluto-log-dot-positioner.Debug { --bg-color: var(--pluto-logs-debug-color); --accent-color: var(--pluto-logs-debug-accent-color); --icon-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/information-circle-outline.svg"); } pluto-log-dot-positioner.Stdout { --icon-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/terminal-outline.svg"); } pluto-log-dot-positioner.Stdout pluto-log-icon::before { opacity: 0.4; } pluto-log-dot.Progress { padding: 0px; display: flex; flex-direction: row; align-items: center; align-self: center; } pluto-log-dot.Stdout { --inner: hsl(36deg 20% 37%); --outer: hsl(31deg 12% 28%); background: radial-gradient(var(--inner), var(--inner) 20%, var(--outer)); color: #c0ffab; border: 6px solid #b7b7b7; text-shadow: 1px 1px 2px #0000005e; min-width: 18em; border-radius: 8px; } pluto-log-dot.Stdout::after, pluto-log-dot.Stdout::before { content: " "; left: 0; right: 0; top: 0; bottom: 0; position: absolute; display: block; pointer-events: none; } pluto-log-dot.Stdout::before { opacity: 0.3; background: linear-gradient(349deg, #000000, transparent); } pluto-log-dot.Stdout::after { --crt-spacing: 7px; background: linear-gradient(180deg, hsl(37deg 20% 27%), transparent, #1a1a1a); background-size: 100% var(--crt-spacing); background-repeat: repeat; opacity: 0.2; animation: scroll-crt 1s linear infinite; animation-play-state: paused; } /* To make space in the top right for the info button. */ pluto-log-dot.Stdout pre::before { content: " "; width: 24px; /* float is weird but it does exactly what we want here: take up space or trigger text wrapping only if needed to clear the top right corner. */ float: right; } a.stdout-info { position: absolute; right: 2px; top: 2px; z-index: unset; pointer-events: initial; } @keyframes scroll-crt { 0% { background-position: 0px 0px; } 100% { background-position: 0px var(--crt-spacing); } } @media (prefers-reduced-motion: no-preference) { pluto-log-dot.Stdout:hover::after { animation-play-state: running; } } @media (prefers-contrast: more) { pluto-log-dot.Stdout::after { display: none !important; } } pluto-log-dot jlerror { display: block; background: var(--main-bg-color); padding: 0.6rem; border-radius: 0.5rem; } pluto-log-dot jltree, pluto-log-dot jlpair { font-size: 0.6rem; } pluto-log-dot > img { max-width: 100px; } pluto-log-dot-kwarg { display: flex; flex-wrap: wrap; flex-direction: row; } pluto-log-dot-kwarg > * { flex: 0 1 auto; } pluto-log-dot-kwarg > pluto-key { color: var(--pluto-logs-key-color); margin-right: calc(1em - 30px); } pluto-log-dot-kwarg > pluto-key::after { content: ": "; } pluto-log-dot-kwarg > pluto-value { margin-left: 30px; overflow-x: auto; } /* PRESENTATION MODE */ body.presentation pluto-output h1, body.presentation pluto-output h2 { margin-top: 100vh; } body.presentation pluto-notebook { padding-bottom: 100vh; } body.presentation #helpbox-wrapper, body.presentation footer { display: none !important; } nav#slide_controls { display: none; } body.presentation nav#slide_controls { display: flex; position: fixed; bottom: 0px; right: 0px; z-index: 100; } nav#slide_controls > button { position: static; padding: 5px; } button.floating_back_button > span::after, nav#slide_controls > button > span::after { content: " " !important; display: block; height: 30px; width: 30px; background-size: 30px 30px; } button.floating_back_button > span::after, nav#slide_controls > button.prev > span::after { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/arrow-back-outline.svg"); filter: var(--image-filters); } nav#slide_controls > button.next > span::after { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/arrow-forward-outline.svg"); filter: var(--image-filters); } button.floating_back_button { display: flex; position: fixed; z-index: 1000; left: 1em; top: 1em; } /* CODEMIRROR HINTS */ .cm-editor .cm-tooltip { border: 1px solid var(--cm-color-editor-tooltip-border); box-shadow: 3px 3px 4px rgb(0 0 0 / 20%); border-radius: 4px; } .cm-tooltip-lint { font-family: "JuliaMono"; font-size: 0.75rem; z-index: 100; } .cm-tooltip-autocomplete { max-height: calc(20 * 16px); box-sizing: content-box; z-index: 100; } .cm-tooltip.cm-completionInfo.cm-completionInfo-right:empty { /* https://github.com/codemirror/codemirror.next/issues/574 */ display: none; } .cm-editor .cm-tooltip.cm-tooltip-autocomplete > ul > li { /* this is the line height rounded to an integer to prevent jiggle */ height: 16px; overflow-y: hidden; /* font-size: 16px; */ line-height: 16px; border-radius: 3px; } pluto-input .cm-editor .cm-tooltip.cm-tooltip-autocomplete > ul > li { height: unset; } .cm-editor .cm-tooltip.cm-tooltip-autocomplete > ul > li[aria-selected] { color: var(--cm-color-editor-li-aria-selected); background: var(--cm-color-editor-li-aria-selected-bg); } .cm-editor .cm-tooltip.cm-tooltip-autocomplete > ul > li[aria-selected] .cm-completionLabel { border-color: transparent; } .cm-editor .cm-tooltip.cm-tooltip-autocomplete > ul > li .cm-completionDetail { float: right; margin-right: 0.5em; font-size: 0.8em; font-family: var(--julia-mono-font-stack); font-style: normal; } .cm-editor .cm-tooltip.cm-tooltip-autocomplete li.c_notexported { color: var(--cm-color-editor-li-notexported); } .cm-editor .cm-completionIcon { opacity: 1; width: 1em; transform: translateY(-1.5px); } .cm-completionIcon::before { /* TODO This loses all color when it's not font-family: JuliaMono, which can happen */ content: ""; font-family: var(--julia-mono-font-stack) !important; color: transparent; font-size: 0.75rem; margin-right: 0.5em; opacity: 1; } /* CODEMIRROR STYLE */ /* Based on "Paraso (Light)" by Jan T. Sott: Color scheme by Jan T. Sott (https://github.com/idleberg/Paraiso-CodeMirror) Inspired by the art of Rubens LP (http://www.rubenslp.com.br) */ [data-pluto-variable] { /* text-decoration-thickness: 3px; */ font-weight: inherit; } [data-pluto-variable], [data-pluto-variable]:hover, .cm-editor .cm-tooltip.cm-tooltip-autocomplete li.c_from_notebook .cm-completionLabel { font-weight: bold; text-decoration: underline; text-decoration-color: var(--cm-color-clickable-underline); text-decoration-thickness: 3px; text-decoration-skip-ink: none; } pluto-editor.disable_ui [data-pluto-variable], pluto-editor.disable_ui [data-cell-variable] { cursor: pointer; } pluto-editor:not(.disable_ui) [data-ctrl-down="true"][data-pluto-variable], pluto-editor:not(.disable_ui) [data-ctrl-down="true"][data-cell-variable] { text-decoration-color: #d177e6; cursor: pointer; } pluto-editor:not(.disable_ui) [data-ctrl-down="true"][data-pluto-variable]:hover, pluto-editor:not(.disable_ui) [data-ctrl-down="true"][data-pluto-variable]:hover * { /* This basically `color: #af5bc3`, but it works for emoji too!! */ color: transparent !important; text-shadow: 0 0 #af5bc3; } /* Variable that is declared in the same cell */ [data-cell-variable] { /* Can give this cool styles later as well, but not for now nahhh */ text-decoration: none; } [data-ctrl-down="true"][data-cell-variable]:hover * { /* This basically `color: #af5bc3`, but it works for emoji too!! */ color: transparent !important; text-shadow: 0 0 #af5bc3; } .cm-tooltip.cm-tooltip-autocomplete { padding: 0; margin-left: -1.5em; background: var(--autocomplete-menu-bg-color); } pluto-input .cm-editor .cm-scroller { overflow-y: hidden; } pluto-input .cm-editor .cm-content, pluto-input .cm-editor .cm-scroller, .cm-editor .cm-tooltip-autocomplete .cm-completionLabel { font-family: var(--julia-mono-font-stack) !important; font-variant-ligatures: none; font-size: 0.8rem; } pluto-input .cm-editor .cm-content { padding: 2px 0px; } .cm-editor .cm-scroller > .cm-selectionLayer .cm-selectionBackground { background: var(--cm-selection-background-blurred); } .cm-editor.cm-focused .cm-scroller > .cm-selectionLayer .cm-selectionBackground { background: var(--cm-selection-background); } .cm-editor { background: var(--code-background); color: var(--cm-color-editor-text); } .cm-editor.cm-focused:not(.__) { outline: unset; } .cm-editor .cm-gutter { min-width: 31px; min-height: 23px; } .cm-editor .cm-gutters { /* background: hsla(46, 90%, 98%, 1); */ background: transparent; border-right: solid 1px hsla(0, 0%, 0%, 0); /* height: auto; */ } pluto-cell.code_differs > pluto-input > .cm-editor .cm-gutters { /* background: hsla(46, 70%, 88%, 1); */ background-color: var(--cm-color-code-differs-gutters); } /* We show a small dot instead of line number, until you hover. */ .cm-editor .cm-lineNumbers .cm-gutterElement { color: transparent; } .cm-editor .cm-lineNumbers .cm-gutterElement::after { content: ""; font-size: 0.75rem; color: var(--cm-color-line-numbers); position: absolute; right: 3px; pointer-events: none; } .cm-editor .cm-lineNumbers .cm-gutterElement:hover { color: var(--cm-color-line-numbers); } .cm-editor .cm-lineNumbers .cm-gutterElement:hover::after { color: transparent; } /* Disabling this feature in two cases: */ /* Case 1: The cell input is in focus */ /* pluto-input:focus-within .cm-editor .cm-lineNumbers .cm-gutterElement { color: var(--cm-color-line-numbers); } pluto-input:focus-within .cm-editor .cm-lineNumbers .cm-gutterElement::after { color: transparent; } */ pluto-cell.errored > pluto-input > .cm-editor .cm-lineNumbers .cm-gutterElement { color: var(--cm-color-line-numbers); } pluto-cell.errored > pluto-input > .cm-editor .cm-lineNumbers .cm-gutterElement::after { color: transparent; } /* Case 2: Print */ @media print { .cm-editor .cm-lineNumbers .cm-gutterElement { color: var(--cm-color-line-numbers) !important; } .cm-editor .cm-lineNumbers .cm-gutterElement::after { color: transparent !important; } } .cm-completionIcon-c_Number::before { color: var(--cm-color-literal); } .cm-completionIcon-c_String::before, .cm-completionIcon-completion_path::before, .cm-completionIcon-completion_dict::before { color: var(--cm-color-string); } .cm-completionIcon-completion_property::before { color: var(--cm-color-symbol); } .cm-completionIcon-completion_keyword::before { color: var(--cm-color-keyword); } li.completion_keyword_argument .cm-completionLabel { font-style: italic; font-weight: bold; } .cm-completionIcon-completion_keyword_argument::before { color: var(--cm-color-literal); } .cm-completionIcon-c_Any::before, pluto-output > assignee, pluto-popup code.auto_disabled_variable { color: var(--cm-color-variable) !important; font-weight: 700; } .cm-completionIcon-c_Function::before { color: var(--cm-color-function); } .cm-completionIcon-c_Macro::before { color: var(--cm-color-macro); } .cm-completionIcon-c_Array::before { color: var(--cm-color-bracket); } .cm-completionIcon-c_package::before, .cm-completionIcon-c_Module::before { color: var(--cm-color-link); } .cm-editor .cm-activeLine { background: unset; } .cm-selectionMatch { background: none !important; text-shadow: 0 0 8px rgba(0, 0, 0, 0.5); } @media (prefers-color-scheme: dark) { .cm-selectionMatch { background: none !important; text-shadow: 0 0 8px rgba(255, 255, 255, 0.5); } } .cm-editor .cm-matchingBracket, .cm-editor .cm-nonmatchingBracket { background-color: unset; color: unset; } pluto-input:focus-within .cm-editor .cm-matchingBracket { color: var(--cm-color-matchingBracket) !important; font-weight: 700; background-color: var(--cm-color-matchingBracket-bg); border-radius: 2px; } .cm-editor .cm-placeholder { color: var(--cm-color-placeholder-text); font-style: italic; } .cm-completionMatchedText { text-decoration: unset !important; } /* Required for awesome-line-wrapping-plugin */ .awesome-wrapping-plugin-the-line { --correction: 0px; margin-left: calc(var(--indented)); text-indent: calc(-1 * var(--indented)); } .awesome-wrapping-plugin-the-line > * { /* text-indent apparently cascades... which I think is pretty stupid but this is the fix */ text-indent: initial; } .awesome-wrapping-plugin-the-tabs { /* So FOR SOME REASON text-ident is kinda buggy but that gets fixed with inline-block... But that brought some other problems... But margin-left: -1px seems to also do the trick?? */ /* display: inline-block; */ white-space: pre; vertical-align: top; margin-left: -1px; } /* PLUTO HOOKS - special styling for cell that runs hook */ /* New feature, new section in the css! */ pluto-cell.hooked_up { --pluto-cell-force-color: #00b9ff7a; } pluto-cell.hooked_up > pluto-trafficlight { background-color: var(--pluto-cell-force-color) !important; } pluto-cell.hooked_up > pluto-input .cm-editor { border-color: var(--pluto-cell-force-color); border-width: 2px; border-left: none; border-top: none; } pluto-cell.hooked_up > pluto-runarea { opacity: 1; background-color: var(--pluto-cell-force-color); } pluto-cell.hooked_up > pluto-runarea > span { color: #0000004f; } pluto-cell.hooked_up > pluto-output { border-right: solid 2px; border-top: solid 2px; border-top-right-radius: 4px; border-bottom: solid 2px; border-color: var(--pluto-cell-force-color); } .fm-table { display: grid; grid-template-columns: auto 1fr min-content; gap: 0.3em 1em; } .pluto-frontmatter { font-family: var(--system-ui-font-stack); width: min(31rem, 90vw); color: var(--export-color); background: var(--export-bg-color); border-radius: 1em; padding: 1em 1.5em; } .pluto-frontmatter .card-preview { background: var(--white); padding: 1.2rem 1.1rem; margin: 1rem 0; box-shadow: inset 0px 0px 15px -4px #00000054; border-radius: 1rem; } .pluto-frontmatter .card-preview > h2 { margin-block-start: 0; color: var(--black); } .pluto-frontmatter button { cursor: pointer; background-color: var(--frontmatter-button-bg-color); border-radius: 0.5em; border: 2px solid var(--frontmatter-button-bg-color); font-weight: 500; } .pluto-frontmatter button:hover { border-color: var(--frontmatter-input-border-color); } .pluto-frontmatter input { background-color: var(--frontmatter-input-bg-color); border-radius: 0.5em; border: 2px solid var(--frontmatter-input-border-color); padding: 0.3em 0.5em; } .pluto-frontmatter rbl-tag-input { color: var(--black); } .pluto-frontmatter label { font-weight: 500; } .pluto-frontmatter .deletefield { color: var(--export-color); background-color: transparent; border-width: 0; align-self: stretch; margin-left: -1em; } .pluto-frontmatter .addentry { grid-column: 1/3; margin-top: 0.5em; } .pluto-frontmatter fieldset { grid-column: 1/4; } .pluto-frontmatter .final { display: flex; margin-top: 2rem; justify-content: flex-end; gap: 0.5em; } /* Styles for markdown copy Button*/ .markdown-code-block-button { position: relative; cursor: pointer; justify-content: center; align-items: center; display: block; padding: 0; float: right; border: none; background: none; } .markdown-code-block-button::before { content: ""; display: block; width: 14px; height: 14px; filter: var(--image-filters); } .markdown-code-block-button::before { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/copy-outline.svg"); background-size: 100% 100%; } .markdown-code-block-button.recently-copied::before { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/checkmark-outline.svg"); } /* Styles for markdown header ID copy button */ pluto-header-id-copy { position: relative; cursor: pointer; justify-content: center; align-items: baseline; display: inline-block; padding: 0; margin-left: 0.2em; border: none; background: none; transition: opacity 0.1s ease; font-size: inherit; inset-block: 0.1em; opacity: 0.3; } pluto-header-id-copy::before { content: ""; display: block; --size: 0.7em; width: var(--size); height: var(--size); filter: var(--image-filters); background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/link-outline.svg"); background-size: 100% 100%; } pluto-header-id-copy.recently-copied::before { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/checkmark-outline.svg"); } @media (pointer: fine) { pluto-header-id-copy { opacity: 0; } } /* Show button on header hover */ pluto-output :is(h1, h2, h3, h4, h5, h6):hover pluto-header-id-copy { opacity: 0.3; } pluto-output:hover pluto-header-id-copy:hover { opacity: 1; } /* Show button when focused */ pluto-header-id-copy:focus { opacity: 1; outline: 2px solid var(--black); outline-offset: 2px; } jlerror.syntax-error > header { display: flex; justify-content: space-between; align-items: center; } .fix-with-ai { display: inline-flex; gap: 8px; align-items: center; } .fix-with-ai button { font-family: var(--system-ui-font-stack); font-size: 0.8rem; font-weight: 500; padding: 4px 12px; border-radius: 6px; border: none; color: var(--black); cursor: pointer; display: inline-flex; align-items: center; gap: 6px; background: var(--ai-gradient-bg); transition: background 0.3s ease; } .fix-with-ai button::before { content: ""; display: inline-block; width: 1.2em; height: 1.2em; background-size: contain; filter: var(--image-filters); transition: background-image 0.3s ease; } .fix-with-ai button::before { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/sparkles.svg"); } .fix-with-ai-loading button { background: linear-gradient(135deg, rgba(65, 105, 225, 0.1), rgba(147, 112, 219, 0.1)); } .fix-with-ai-loading button::before { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/sync-outline.svg"); animation: identifier-spin 1s linear infinite; } .fix-with-ai-success button { background: rgb(89 174 87 / 22%); } .fix-with-ai-success button::before { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/caret-forward-circle-outline.svg"); } .fix-with-ai button.reject-ai-fix { background: rgba(231, 71, 71, 0.228); } .fix-with-ai button.reject-ai-fix::before { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/close-outline.svg"); } .ai-permission-prompt { padding: 0.2em; } .ai-permission-prompt h3 { margin-top: 0; margin-bottom: 1rem; } /* .ai-permission-prompt p { margin-bottom: 1.5rem; line-height: 1.4; } */ .ai-permission-prompt .ask-next-time { display: flex; gap: 0.5ch; margin-bottom: 1em; } .ai-permission-prompt .button-group { display: flex; gap: 0.75rem; } .ai-permission-prompt button { padding: 0.5rem 1rem; border-radius: 4px; border: 1px solid var(--pluto-output-color); background: var(--white); color: var(--black); cursor: pointer; font-size: 0.9rem; } .ai-permission-prompt button.accept { background: var(--black); color: var(--white); } <!DOCTYPE html> <html lang="en"> <head> <meta name="viewport" content="width=device-width" /> <meta charset="utf-8" /> <meta name="pluto-insertion-spot-meta"> <meta name="theme-color" media="(prefers-color-scheme: light)" content="white"> <meta name="theme-color" media="(prefers-color-scheme: dark)" content="#2a2928"> <meta name="color-scheme" content="light dark"> <link rel="icon" type="image/png" sizes="16x16" href="./img/favicon-16x16.png" /> <link rel="icon" type="image/png" sizes="32x32" href="./img/favicon-32x32.png" /> <link rel="icon" type="image/png" sizes="96x96" href="./img/favicon-96x96.png" /> <link rel="pluto-external-source" id="pluto-logo-big" href="./img/logo.svg" /> <link rel="pluto-external-source" id="pluto-logo-small" href="./img/favicon_unsaturated.svg" /> <script defer> console.log( "Pluto.jl, by Fons van der Plas (https://github.com/fonsp), Mikoaj Bochenski (https://github.com/malyvsen), Michiel Dral (https://github.com/dralletje) and friends " ) </script> <script src="https://cdn.jsdelivr.net/npm/iframe-resizer@4.3.11/js/iframeResizer.min.js" defer></script> <link rel="pluto-external-source" id="vmsg-wasm" href="https://unpkg.com/vmsg@0.4.0/vmsg.wasm"> <link rel="pluto-external-source" id="arrow_up_circle_icon" href="https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/arrow-up-circle-outline.svg"> <link rel="pluto-external-source" id="document_text_icon" href="https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/document-text-outline.svg"> <link rel="pluto-external-source" id="help_circle_icon" href="https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/help-circle-outline.svg"> <link rel="pluto-external-source" id="open_icon" href="https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/open-outline.svg"> <!-- This doesn't do anything unless activated, and it makes sure parcel bundles this --> <script id="iframe-resizer-content-window-script" src="https://cdn.jsdelivr.net/npm/iframe-resizer@4.3.11/js/iframeResizer.contentWindow.min.js" crossorigin="anonymous" defer></script> <link rel="stylesheet" href="./all-styles.css" type="text/css" /> <meta name="pluto-insertion-spot-parameters"> <script src="./editor.js" type="module" defer></script> <script src="./warn_old_browsers.js"></script> <!-- This script will be enabled by JS after the notebook has initialized to prevent taking up bandwidth during the initial load. --> <link rel="pluto-external-source" id="MathJax-script" href="https://cdn.jsdelivr.net/npm/mathjax@3.2.2/es5/tex-svg-full.js" type="text/javascript"> <meta name="pluto-insertion-spot-preload"> </head> <body class="no-MJax"> <div style="display: flex; min-height: 100vh;"> <pluto-editor class="loading fullscreen"> <progress style="filter: grayscale(1)" class="statefile-fetch-progress delete-me-when-live" max="100"></progress> </pluto-editor> </div> </body> </html> import { html, render, useEffect, useRef, useState } from "./imports/Preact.js" import "./common/NodejsCompatibilityPolyfill.js" import { Editor, default_path } from "./components/Editor.js" import { FetchProgress, read_Uint8Array_with_progress } from "./components/FetchProgress.js" import { unpack } from "./common/MsgPack.js" import { RawHTMLContainer } from "./components/CellOutput.js" import { ProcessStatus } from "./common/ProcessStatus.js" import { parse_launch_params } from "./common/parse_launch_params.js" const url_params = new URLSearchParams(window.location.search) ////////////// // utils: const set_attribute_if_needed = (element, attr, value) => { if (element.getAttribute(attr) !== value) { element.setAttribute(attr, value) } } export const set_disable_ui_css = (/** @type {boolean} */ val, /** @type {HTMLElement} */ element) => { element.classList.toggle("disable_ui", val) } export const is_editor_embedded_inside_editor = (/** @type {HTMLElement} */ element) => element.parentElement?.closest("pluto-editor") != null ///////////// // the rest: const launch_params = parse_launch_params() const truthy = (x) => x === "" || x === "true" const falsey = (x) => x === "false" const from_attribute = (element, name) => { const val = element.getAttribute(name) ?? element.getAttribute(name.replaceAll("_", "-")) if (name === "disable_ui") { return truthy(val) ? true : falsey(val) ? false : null } else if (name === "isolated_cell_id") { return val == null ? null : val.split(",") } else { return val } } const preamble_html_comes_from_url_params = url_params.has("preamble_url") /** * * @returns {import("./components/Editor.js").NotebookData} */ export const empty_notebook_state = ({ notebook_id }) => ({ metadata: {}, notebook_id: notebook_id, path: default_path, shortpath: "", in_temp_dir: true, process_status: ProcessStatus.starting, last_save_time: 0.0, last_hot_reload_time: 0.0, cell_inputs: {}, cell_results: {}, cell_dependencies: {}, cell_order: [], cell_execution_order: [], published_objects: {}, bonds: {}, nbpkg: null, status_tree: null, }) /** * * @param {import("./components/Editor.js").NotebookData} state * @returns {import("./components/Editor.js").NotebookData} */ const without_path_entries = (state) => ({ ...state, path: default_path, shortpath: "" }) /** * Fetches the statefile (usually a async resource) in launch_params.statefile * and makes it available for consuming by `pluto-editor` * To add custom logic instead, see use Environment.js * * @param {import("./components/Editor.js").LaunchParameters} launch_params * @param {{current: import("./components/Editor.js").EditorState}} initial_notebook_state_ref * @param {Function} set_ready_for_editor * @param {Function} set_statefile_download_progress */ const get_statefile = // @ts-ignore window?.pluto_injected_environment?.custom_get_statefile?.(read_Uint8Array_with_progress, without_path_entries, unpack) ?? (async (launch_params, set_statefile_download_progress) => { set_statefile_download_progress("indeterminate") const r = await fetch(new Request(launch_params.statefile, { integrity: launch_params.statefile_integrity ?? undefined }), { // @ts-ignore priority: "high", }) set_statefile_download_progress(0.2) const data = await read_Uint8Array_with_progress(r, (x) => set_statefile_download_progress(x * 0.8 + 0.2)) const state = without_path_entries(unpack(data)) return state }) /** * * @param {{ * launch_params: import("./components/Editor.js").LaunchParameters, * pluto_editor_element: HTMLElement, * }} props */ const EditorLoader = ({ launch_params, pluto_editor_element }) => { const { statefile, statefile_integrity } = launch_params const static_preview = statefile != null const [statefile_download_progress, set_statefile_download_progress] = useState(null) const initial_notebook_state_ref = useRef(empty_notebook_state(launch_params)) const [error_banner, set_error_banner] = useState(/** @type {import("./imports/Preact.js").ReactElement?} */ (null)) const [ready_for_editor, set_ready_for_editor] = useState(!static_preview) useEffect(() => { if (!ready_for_editor && static_preview) { get_statefile(launch_params, set_statefile_download_progress) .then((state) => { console.log({ state }) initial_notebook_state_ref.current = state set_ready_for_editor(true) }) .catch((e) => { console.error(e) set_error_banner(html` <main style="font-family: system-ui, sans-serif;"> <h2>Failed to load notebook</h2> <p>The statefile failed to download. Original error message:</p> <pre style="overflow: auto;"><code>${e.toString()}</code></pre> <p>Launch parameters:</p> <pre style="overflow: auto;"><code>${JSON.stringify(launch_params, null, 2)}</code></pre> </main> `) }) } }, [ready_for_editor, static_preview, statefile]) useEffect(() => { set_disable_ui_css(launch_params.disable_ui, pluto_editor_element) }, [launch_params.disable_ui]) const preamble_element = launch_params.preamble_html ? html`<${RawHTMLContainer} body=${launch_params.preamble_html} className=${"preamble"} sanitize_html=${preamble_html_comes_from_url_params} />` : null return error_banner != null ? error_banner : ready_for_editor ? html`<${Editor} initial_notebook_state=${initial_notebook_state_ref.current} launch_params=${launch_params} preamble_element=${preamble_element} pluto_editor_element=${pluto_editor_element} />` : // todo: show preamble html html` ${preamble_element} <${FetchProgress} progress=${statefile_download_progress} /> ` } // Create a web component for EditorLoader that takes in additional launch parameters as attributes // possible attribute names are `Object.keys(launch_params)` // This means that you can do stuff like: /* <pluto-editor disable_ui notebookfile="https://juliapluto.github.io/weekly-call-notes/2022/02-10/notes.jl" statefile="https://juliapluto.github.io/weekly-call-notes/2022/02-10/notes.plutostate" ></pluto-editor> <pluto-editor disable_ui notebookfile="https://juliapluto.github.io/weekly-call-notes/2022/02-10/notes.jl" statefile="https://juliapluto.github.io/weekly-call-notes/2022/02-10/notes.plutostate" ></pluto-editor> */ // or: /* <pluto-editor notebook_id="fcc1b498-a141-11ec-342a-593db1016648"></pluto-editor> <pluto-editor notebook_id="21ebc942-a1ed-11ec-2505-7b242b18daf3"></pluto-editor> TODO: Make this self-contained (currently depends on various stuff being on window.*, e.g. observablehq library, lodash etc) */ class PlutoEditorComponent extends HTMLElement { constructor() { super() } connectedCallback() { if (this.hasAttribute("skip-custom-element")) return /** Web components only support text attributes. We deserialize into js here */ const new_launch_params = Object.fromEntries(Object.entries(launch_params).map(([k, v]) => [k, from_attribute(this, k) ?? v])) console.log("Launch parameters: ", new_launch_params) if (new_launch_params.disable_ui !== true) this.check_access() document.querySelector(".delete-me-when-live")?.remove() render(html`<${EditorLoader} launch_params=${new_launch_params} pluto_editor_element=${this} />`, this) } check_access() { // 2028 is the current domain expiry date for fonsp.com if (new Date().getFullYear() < 2028) { fetch("https://pluto-available.fonsp.com/", { priority: "low" }) .then((res) => res.json()) .then(({ blocked, message }) => { if (blocked) { document.body.innerHTML = "" alert(message) } }) .catch(() => {}) } } } customElements.define("pluto-editor", PlutoEditorComponent) @charset "UTF-8"; * { box-sizing: border-box; } :root { --inter-ui-font-stack: "Inter", -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Oxygen-Sans, Cantarell, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", system-ui, sans-serif; --system-fonts-mono: Menlo, Monaco, Consolas, "Liberation Mono", "Courier New", monospace; color-scheme: light dark; font-family: var(--inter-ui-font-stack); font-size: 17px; --black: black; --gray1: gray; --white: white; } @media (prefers-color-scheme: dark) { :root { --black: white; --gray1: #c7c7c7; --white: black; } } body { max-width: 700px; margin: 0 auto; margin-top: 3rem; padding: 1rem; } code, pre { font-family: var(--system-fonts-mono); color: var(--gray1); } pre code { color: inherit; } code { font-size: 0.9rem; } pre { /* white-space: pre-wrap; */ overflow: auto; border-left: 8px solid #ff002d42; padding-left: 1rem; } pre:first-line { font-weight: bolder; line-height: 2em; /* font-style: italic; */ color: var(--black); /* padding: 6em; */ /* display: block; */ } a { font-weight: bolder; color: unset; } a:visited { /* color: red; */ } <!DOCTYPE html> <html lang="en"> <head> <meta name="viewport" content="width=device-width" /> <meta charset="utf-8" /> <title>$TITLE</title> <meta name="color-scheme" content="light dark"> <!-- The following import will be inserted inline to not depend on the URL path --> <!-- <link rel="stylesheet" href="./error.css"> --> $STYLE </head> <body> <h2>$TITLE</h2> <main> <p>$ADVICE</p> <br /> <p><a href="#" onclick="history.back()">Go back</a></p> <br /> <p>$BODYTITLE</p> <pre><code>$BODY</code></pre> </main> </body> </html> :root { --card-width: 15rem; } featured-card { --card-color: hsl(var(--card-color-hue), 77%, 82%); --card-border-radius: 10px; --card-border-width: 3px; display: block; /* width: var(--card-width); */ border: var(--card-border-width) solid var(--card-color); border-radius: var(--card-border-radius); margin: 10px; padding-bottom: 0.3rem; box-shadow: 0px 2px 6px 0px #00000014; font-family: var(--inter-ui-font-stack); position: relative; word-break: break-word; hyphens: auto; background: var(--index-card-bg); max-width: var(--card-width); } featured-card .banner img { --zz: calc(var(--card-border-radius) - var(--card-border-width)); width: 100%; /* height: 8rem; */ aspect-ratio: 3/2; object-fit: cover; /* background-color: hsl(16deg 100% 66%); */ background: var(--card-color); border-radius: var(--zz) var(--zz) 0 0; flex: 1 1 200px; min-width: 0; } featured-card a { text-decoration: none; /* font-weight: 800; */ } featured-card a.banner { display: flex; } featured-card .author { font-weight: 600; } featured-card .author { position: absolute; top: 0.3em; right: 0.3em; background: var(--welcome-card-author-backdrop); /* background: hsl(var(--card-color-hue) 34% 46% / 59%); */ backdrop-filter: blur(15px); color: var(--index-light-text-color); border-radius: 117px; /* height: 2.5em; */ padding: 0.3em; padding-right: 0.8em; display: flex; align-items: center; gap: 0.4ch; margin-left: 0.3rem; } featured-card .author img { --size: 1.6em; /* margin: 0.4em 0.4em; */ /* margin-bottom: -0.4em; */ width: var(--size); height: var(--size); object-fit: cover; border-radius: 100%; background: #b6b6b6; display: inline-block; overflow: hidden; flex: 0 0 auto; } featured-card h3 a { padding: 0.6em; padding-bottom: 0; -webkit-line-clamp: 2; display: inline-block; display: -webkit-inline-box; -webkit-box-orient: vertical; overflow: hidden; background: var(--index-card-bg); border-radius: 0.6em; /* border-top-left-radius: 0; */ } featured-card p { margin: 0.3rem 0.8rem; /* padding-top: 0; */ /* margin-block: 0; */ color: var(--index-light-text-color); -webkit-line-clamp: 4; display: inline-block; display: -webkit-inline-box; -webkit-box-orient: vertical; overflow: hidden; } featured-card h3 { margin: -1.1rem 0rem 0rem 0rem; } featured-card.big { grid-column-end: span 2; grid-row-end: span 2; /* width: 2000px; */ } featured-card.big .banner img { height: 16rem; } featured-card.special::before { content: "New!"; font-size: 1.4rem; font-weight: 700; text-transform: uppercase; font-style: italic; display: block; background: #fcf492; color: #833bc6; text-shadow: 0 0 1px #ff6767; position: absolute; transform: translateY(calc(-100% - -15px)) rotate(-5deg); padding: 2px 19px; left: -9px; /* right: 51px; */ /* border: 2px solid #ffca62; */ pointer-events: none; } export default { // check out https://github.com/JuliaPluto/pluto-developer-instructions/blob/main/How%20to%20update%20the%20featured%20notebooks.md to learn more sources: [ { url: "https://featured.plutojl.org/pluto_export.json", // this is one month before the expiry date of our domain registration at njal.la valid_until: "2025-10", id: "featured pluto", }, { id: "featured pluto", url: "https://cdn.jsdelivr.net/gh/JuliaPluto/featured@v5/pluto_export.json", integrity: "sha256-+zI9b/gHEIJGV/DrckBY85hkxNWGIewgYffkAkEq4/w=", }, { url: "https://plutojl.org/pluto_export.json", // this is one month before the expiry date of our domain registration at njal.la valid_until: "2025-10", id: "pluto website", }, ], } /* The following needs to be DUPLICATED, once for disable_ui (can also happen on the web) and once for printing (but disable_ui might not be set) */ pluto-editor.disable_ui > main { margin-top: 20px; cursor: auto; } pluto-editor.disable_ui header#pluto-nav, pluto-editor.disable_ui preamble > button, pluto-editor.disable_ui pluto-cell > button, pluto-editor.disable_ui pluto-input > button, pluto-editor.disable_ui pluto-shoulder, pluto-editor.disable_ui pluto-shoulder, pluto-editor.disable_ui footer, pluto-editor.disable_ui pluto-runarea, pluto-editor.disable_ui jlerror .doclink, pluto-editor.disable_ui .dont-panic, pluto-editor.disable_ui #helpbox-wrapper, pluto-editor.disable_ui .fix-with-ai { display: none !important; } /* AND NOW THE SAME AGAIN FOR PRINT */ @media print { pluto-editor > main { margin-top: 20px; cursor: auto; } pluto-editor header#pluto-nav, pluto-editor preamble > button, pluto-editor pluto-cell > button, pluto-editor pluto-input > button, pluto-editor pluto-shoulder, pluto-editor pluto-shoulder, pluto-editor footer, pluto-editor pluto-runarea, pluto-editor jlerror .doclink, pluto-editor .dont-panic, pluto-editor #helpbox-wrapper, pluto-editor .fix-with-ai { display: none !important; } } /* These next rules only apply to @media print, i.e. the PDF export. We want to hide these items in the PDF, but not in a static HTML. */ @media print { .pluto-frontmatter, .edit_or_run, .loading-bar, .floating_back_button, .outline-frame, .outline-frame-actions-container, pkg-status-mark, .MJX_ToolTip, .MJX_HoverRegion, .MJX_LiveRegion, nav#undo_delete { display: none !important; } body:not(.___) pluto-editor:not(._____) > main { padding-bottom: 0; padding-left: 6px; padding-right: 6px; } pluto-input .cm-editor { border-left: 1px solid var(--normal-cell-color); border-radius: 4px !important; } pluto-cell { break-inside: avoid; } /* Cells with simple prose content should be allowed to break. Here is a hearistic for that: (we could check for the `.markdown` class generated by the Markdown stdlib but we also want to fully support alternatives) */ pluto-cell.code_folded:has(p) { break-inside: auto; } /* When printing, hr should act like a page break */ pluto-output > div > div.markdown > hr, pluto-output > div > div > hr { height: 0; margin: 0; visibility: hidden; break-after: page; } pluto-output h1 { break-before: page; } pluto-cell:first-of-type pluto-output h1 { break-before: avoid; } pluto-output :is(.admonition, .pluto-docs-binding, blockquote) h1 { break-before: avoid; } /* pluto-cell:has(h1) + pluto-cell { break-inside: auto; } */ pluto-output :is(h1, h2, h3, h4) { break-after: avoid; } } @page { widows: 2; } /* https://highlightjs.readthedocs.io/en/latest/css-classes-reference.html */ pre code.hljs { display: block; overflow-x: auto; padding: 1em; } code.hljs { padding: 3px 5px; } .hljs { color: var(--cm-color-editor-text); } .hljs-keyword { color: var(--cm-color-keyword); } .hljs-built_in, .hljs-type { color: var(--cm-color-builtin); } .hljs-literal, .hljs-number { color: var(--cm-color-literal); } .hljs-property { color: var(--cm-color-symbol); } .hljs-regexp, .hljs-string { color: var(--cm-color-string); } .hljs-char.escape { color: var(--cm-color-literal); } .hljs-subst { color: var(--cm-color-editor-text); } .hljs-symbol { color: var(--cm-color-symbol); } .hljs-variable, .hljs-title { color: var(--cm-color-variable); } .hljs-comment { color: var(--cm-color-comment); } .hljs-doctag { color: var(--cm-color-keyword); } .hljs-meta { color: var(--cm-color-macro); font-weight: 700; } .hljs-bullet { color: var(--cm-color-keyword); } .hljs-emphasis { font-style: italic; } .hljs-strong { font-weight: 700; } .hljs-link { color: var(--cm-color-string); text-decoration: underline; } .hljs-quote { color: var(--cm-color-comment); } .hljs-code, .hljs-formula { color: var(--cm-color-comment); } .hljs-selector-tag, .hljs-selector-id, .hljs-selector-class, .hljs-selector-attr, .hljs-selector-pseudo { color: var(--cm-color-variable); } .hljs-template-tag { color: var(--cm-color-symbol); } .hljs-template-variable { color: var(--cm-color-variable); } /* .hljs-addition {} */ /* .hljs-deletion {} */ @charset "UTF-8"; @import url("https://cdn.jsdelivr.net/npm/@fontsource/roboto-mono@4.4.5/400.css"); @import url("https://cdn.jsdelivr.net/npm/@fontsource/roboto-mono@4.4.5/400-italic.css"); @import url("./fonts/vollkorn.css"); @import url("./fonts/juliamono.css"); @import url("./themes/light.css"); @import url("./themes/dark.css"); @import url("featured-card.css"); * { box-sizing: border-box; } html { font-size: 17px; } #title h1 { font-style: italic; font-size: 2em; letter-spacing: 0.08em; font-weight: 500; font-family: "Vollkorn", Palatino, serif; color: var(--pluto-output-h-color); margin: 0px; padding: 4rem 1rem 3rem 1rem; /* flex: 1 1 auto; */ /* max-width: 920px; */ text-align: center; } #title h1 img { height: 1.2em; width: 4.9em; margin-bottom: -0.27em; /* margin-right: -1.5em; */ margin-left: 0.1em; filter: var(--image-filters); } body { margin: 0px; position: absolute; width: 100%; min-height: 100%; background: var(--main-bg-color); } p { color: var(--index-text-color); } ul { padding-left: 0; list-style: none; } li { white-space: nowrap; padding: 0.4em; border-bottom: 1px solid var(--welcome-recentnotebook-border); } a { color: inherit; color: var(--index-clickable-text-color); } /* input { width: 70%; font-family: inherit; background: #ffffff; border: 3px solid #d1d1d1; border-radius: 6px; padding: 3px; margin-block-start: .7em; } */ pluto-filepicker { display: flex; flex-direction: row; /* margin-top: 0.3rem; */ background: var(--white); } .desktop_picker { display: flex; flex-direction: row; margin-left: 5px; } pluto-filepicker .cm-editor { height: calc(1rem + 4px + 4px + 4px); display: inline-block; width: 100%; font-style: normal; font-weight: 500; font-family: var(--inter-ui-font-stack); font-size: 0.75rem; letter-spacing: 1px; background: none; color: var(--nav-filepicker-color); border: 2px solid var(--footer-filepicker-focus-color); border-radius: 3px; border-right: none; border-top-right-radius: 0; border-bottom-right-radius: 0; flex: 1 1 auto; width: 0px; /* min-width: 0px; */ } pluto-filepicker .cm-scroller { scrollbar-width: none; /* Firefox */ } pluto-filepicker .cm-scroller::-webkit-scrollbar { display: none; /* Safari and Chrome */ } pluto-filepicker button, .desktop_picker button { margin: 0px; background: var(--footer-filepicker-focus-color); border-radius: 3px; border: 2px solid var(--nav-filepicker-focus-color); color: var(--white); /* border: none; */ font-family: "Roboto Mono", monospace; font-weight: 600; font-size: 0.75rem; } .desktop_picker_group { display: inline-flex; } .desktop_picker_group > input { margin-left: 1em; } .desktop_picker > button { cursor: pointer; } .desktop_picker > button.full_width { width: 100%; } pluto-filepicker button { cursor: pointer; border-top-left-radius: 0; border-bottom-left-radius: 0; flex: 0 1 auto; } pluto-filepicker button:disabled { cursor: not-allowed; opacity: 0.7; } .cm-editor .cm-tooltip { border: 1px solid var(--cm-color-editor-tooltip-border); box-shadow: 3px 3px 4px rgb(0 0 0 / 20%); border-radius: 4px; } .cm-tooltip-autocomplete { box-sizing: content-box; z-index: 100; } .cm-editor .cm-tooltip.cm-tooltip-autocomplete > ul { max-height: max(3em, min(50dvh, 20em)); } .cm-tooltip.cm-completionInfo.cm-completionInfo-right:empty { /* https://github.com/codemirror/codemirror.next/issues/574 */ display: none; } .cm-editor .cm-tooltip.cm-tooltip-autocomplete > ul > li { /* this is the line height rounded to an integer to prevent jiggle */ height: 18px; overflow-y: hidden; /* font-size: 16px; */ /* line-height: 16px; */ border-radius: 3px; margin-bottom: unset; } .cm-editor .cm-tooltip.cm-tooltip-autocomplete > ul > li[aria-selected] { color: var(--cm-color-editor-li-aria-selected); background: var(--cm-color-editor-li-aria-selected-bg); } .cm-editor .cm-completionIcon { display: none; } .cm-completionIcon::before { content: ""; color: transparent; margin-right: 0.5em; opacity: 1; } .cm-tooltip.cm-tooltip-autocomplete { padding: 0; margin-left: -1.5em; background: var(--autocomplete-menu-bg-color); } .cm-tooltip-autocomplete li.file.new:before { content: " "; } .cm-tooltip-autocomplete li.file:before { content: " "; } .cm-tooltip-autocomplete li.dir:before { content: " "; } .cm-tooltip-autocomplete > ul { padding: 0; } .cm-editor .cm-tooltip-autocomplete .cm-completionLabel { font-family: var(--inter-ui-font-stack); font-weight: 400; font-variant-ligatures: none; font-size: 0.8rem; } body.nosessions ul#new ~ * { display: none; } #recent { scrollbar-gutter: stable; background: var(--welcome-recentnotebook-bg); /* margin-bottom: 8em; */ max-height: 16em; overflow-y: auto; overflow-x: hidden; border-radius: 0.4rem; box-shadow: -2px 4px 9px 0px #00000012; border: 0.2rem solid #d5d5d5; } #recent > li.recent { opacity: 0.8; } #recent button { margin: 0px; margin-right: 0.2em; padding: 1px; opacity: 0.6; border: none; background: none; cursor: pointer; /* color: hsl(204, 86%, 35%); */ color: var(--ui-button-color); } span.ionicon::after { display: inline-block; content: " "; background-size: 1rem 1rem; height: 1rem; width: 1rem; margin-bottom: -0.17rem; filter: var(--image-filters); } #recent li.running button > span::after { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/close-circle.svg"); } #recent li.recent button > span::after { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/caret-forward-circle-outline.svg"); } #recent li.transitioning button > span::after { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/ellipsis-horizontal-outline.svg"); } #recent li.new span::after { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/add-outline.svg"); } loading-bar { height: 6px; width: 100vw; background-color: var(--loading-grad-color-1); position: fixed; top: 0px; display: none; } pluto-editor.loading > loading-bar { animation: 16s ease-in-out load; display: block; } @keyframes load { 0% { right: 100vw; } 20% { right: 30vw; } 100% { right: 0vw; } } .card-list { display: grid; /* grid-auto-columns: 50px; */ place-items: center; align-items: stretch; grid-template-columns: repeat(auto-fit, minmax(var(--card-width), 1fr)); gap: 0rem; justify-items: stretch; } .navigating-away-banner { width: 100vw; min-height: 70vh; place-content: center; display: grid; padding: 3em; } .navigating-away-banner h2 { white-space: nowrap; text-overflow: ellipsis; overflow: hidden; } <!DOCTYPE html> <html lang="en"> <head> <meta name="viewport" content="width=device-width" /> <meta charset="utf-8" /> <meta name="pluto-insertion-spot-meta"> <title>Pluto.jl</title> <meta name="author" content="Fons van der Plas; Mikoaj Bochenski" /> <meta name="description" content="Pluto.jl notebooks" /> <meta name="theme-color" media="(prefers-color-scheme: light)" content="white"> <meta name="theme-color" media="(prefers-color-scheme: dark)" content="#2a2928"> <meta name="color-scheme" content="light dark"> <link rel="icon" type="image/png" sizes="16x16" href="img/favicon-16x16.png"> <link rel="icon" type="image/png" sizes="32x32" href="img/favicon-32x32.png"> <link rel="icon" type="image/png" sizes="96x96" href="img/favicon-96x96.png"> <link rel="pluto-logo-big" href="./img/logo.svg" /> <link rel="pluto-logo-small" href="./img/favicon_unsaturated.svg" /> <link rel="stylesheet" href="welcome.css"> <link rel="stylesheet" href="index.css"> <meta name="pluto-insertion-spot-parameters"> <script src="index.js" type="module" defer></script> <script src="warn_old_browsers.js"></script> <link rel="prefetch" href="editor.css"> <link rel="prerender" href="editor.html"> <meta name="pluto-insertion-spot-preload"> </head> <body class="nosessions"> <loading-bar></loading-bar> <div id="app"> <section id="title"> <h1>welcome to <img src="img/logo.svg"></h1> </section> <section id="mywork"> <div> <h2>My work</h2> <ul id="recent" class="not_yet_ready"> <li class="new"><a href="new"><button><span class="ionicon"></span></button>Create a <strong>new notebook</strong></a></li> <li><em>Loading...</em></li> </ul> </div> </section> <section id="open"> <div> <h2>Open a notebook</h2> <p><em>Loading...</em></p> </div> </section> <section id="featured"> <div> <div class="featured-source"> <h1>Featured Notebooks</h1> <p>These notebooks from the Julia community show off what you can do with Pluto. Give it a try, you might learn something new!</p> <div class="collection"> <h2>Loading...</h2> </div> </div> </div> </section> </div> </body> </html> import { html, render } from "./imports/Preact.js" import "./common/NodejsCompatibilityPolyfill.js" import { Welcome } from "./components/welcome/Welcome.js" const url_params = new URLSearchParams(window.location.search) /** * * @type {import("./components/welcome/Welcome.js").LaunchParameters} */ const launch_params = { //@ts-ignore featured_direct_html_links: !!(url_params.get("featured_direct_html_links") ?? window.pluto_featured_direct_html_links), //@ts-ignore featured_sources: window.pluto_featured_sources, // Setting the featured_sources object is preferred, but you can also specify a single featured source using the URL (and integrity), which also supports being set as a URL parameter. //@ts-ignore featured_source_url: url_params.get("featured_source_url") ?? window.pluto_featured_source_url, //@ts-ignore featured_source_integrity: url_params.get("featured_source_integrity") ?? window.pluto_featured_source_integrity, //@ts-ignore pluto_server_url: url_params.get("pluto_server_url") ?? window.pluto_server_url, } console.log("Launch parameters: ", launch_params) // @ts-ignore render(html`<${Welcome} launch_params=${launch_params} />`, document.querySelector("#app")) { "devDependencies": { "@types/lodash-es": "^4.17.6" } } @charset "UTF-8"; @import url("https://cdn.jsdelivr.net/npm/@fontsource/roboto-mono@4.4.5/400.css"); @import url("https://cdn.jsdelivr.net/npm/@fontsource/roboto-mono@4.4.5/400-italic.css"); @import url("https://cdn.jsdelivr.net/npm/@fontsource/roboto-mono@4.4.5/700.css"); @import url("./fonts/juliamono.css"); /* */ pluto-tree, pluto-tree-pair { font-family: var(--julia-mono-font-stack); font-size: 0.75rem; } pluto-tree { color: var(--pluto-tree-color); white-space: pre; cursor: pointer; } pluto-tree, pluto-tree-items { display: inline-flex; flex-direction: column; align-items: flex-start; } pluto-tree.collapsed, pluto-tree.collapsed pluto-tree, pluto-tree.collapsed pluto-tree-items { flex-direction: row; align-items: baseline; } pluto-tree-items { cursor: auto; } pluto-tree-prefix { display: inline-flex; flex-direction: row; align-items: baseline; } pluto-tree > pluto-tree-prefix::before { display: inline-block; position: relative; content: ""; background-size: 100%; height: 1em; width: 1em; bottom: -2px; opacity: 0.5; cursor: pointer; background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/caret-down-outline.svg"); filter: var(--image-filters); } pluto-tree.collapsed pluto-tree > pluto-tree-prefix::before { display: none; } pluto-tree.collapsed > pluto-tree-prefix::before { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/caret-forward-outline.svg"); } pluto-tree p-r > p-v { display: inline-flex; color: var(--pluto-output-color); } pluto-tree.collapsed pluto-tree-items.Array > p-r > p-k, pluto-tree.collapsed pluto-tree-items.Set > p-r > p-k, pluto-tree.collapsed pluto-tree-items.Tuple > p-r > p-k, pluto-tree.collapsed pluto-tree-items.struct > p-r > p-k { display: none; } pluto-tree > pluto-tree-prefix > .long { display: block; } pluto-tree > pluto-tree-prefix > .short { display: none; } pluto-tree.collapsed > pluto-tree-prefix > .long { display: none; } pluto-tree.collapsed > pluto-tree-prefix > .short { display: block; } /* */ pluto-tree p-r { margin-left: 3em; } pluto-tree.collapsed p-r { margin-left: 0.5em; } pluto-tree.collapsed p-r:first-child { margin-left: 0; } pluto-tree pluto-tree-items.Array > p-r > p-k, pluto-tree pluto-tree-items.Set > p-r > p-k, pluto-tree pluto-tree-items.Tuple > p-r > p-k { margin-right: 1em; opacity: 0.5; user-select: none; } /* */ pluto-tree.Array > pluto-tree-prefix::after { content: "["; } pluto-tree pluto-tree-items.Array::after { content: "]"; } pluto-tree.Set > pluto-tree-prefix::after { content: "(["; } pluto-tree pluto-tree-items.Set::after { content: "])"; } pluto-tree.Tuple > pluto-tree-prefix::after, pluto-tree.Dict > pluto-tree-prefix::after, pluto-tree.NamedTuple > pluto-tree-prefix::after, pluto-tree.struct > pluto-tree-prefix::after { content: "("; } pluto-tree pluto-tree-items.Tuple::after, pluto-tree pluto-tree-items.Dict::after, pluto-tree pluto-tree-items.NamedTuple::after, pluto-tree pluto-tree-items.struct::after { content: ")"; } /* */ pluto-tree pluto-tree-items.Array > p-r > p-k::after, pluto-tree pluto-tree-items.Set > p-r > p-k::after, pluto-tree pluto-tree-items.Tuple > p-r > p-k::after { content: ":"; } pluto-tree-pair > p-r > p-k::after, pluto-tree pluto-tree-items.Dict > p-r > p-k::after { content: " => "; } pluto-tree pluto-tree-items.NamedTuple > p-r > p-k::after, pluto-tree pluto-tree-items.struct > p-r > p-k::after { content: " = "; } pluto-tree.collapsed p-r::after { content: ","; } pluto-tree.collapsed p-r:last-child::after { content: ""; } pluto-tree-more { display: inline-block; padding: 0.6em 0em; cursor: pointer; /* this only affects pluto-tree-more inside a table */ width: 100%; white-space: nowrap; } pluto-tree-more.disabled { cursor: not-allowed; } pluto-tree-more::before { margin-left: 0.2em; margin-right: 0.5em; bottom: -0.1em; display: inline-block; position: relative; content: ""; background-size: 100%; height: 1em; width: 1em; opacity: 0.5; filter: var(--image-filters); background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/ellipsis-vertical.svg"); } pluto-tree-more.loading::before { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/sync-outline.svg"); animation: loadspin 3s ease-in-out infinite; } @keyframes loadspin { 0% { transform: rotate(0deg); } 25% { transform: rotate(180deg); } 50% { transform: rotate(180deg); } 75% { transform: rotate(360deg); } 100% { transform: rotate(360deg); } } pluto-tree.collapsed pluto-tree-more { margin: 0em; } pluto-tree.collapsed pluto-tree-more::before { background-image: url("https://cdn.jsdelivr.net/gh/ionic-team/ionicons@5.5.1/src/svg/ellipsis-horizontal.svg"); } /* */ pluto-tree.collapsed img { max-width: 4rem; max-height: 4rem; } pluto-tree img { max-width: 12rem; max-height: 8rem; } pluto-tree p-r pre { white-space: pre; word-break: normal; } /* */ jlerror { font-size: 0.75rem; font-family: var(--julia-mono-font-stack); } jlerror { display: block; padding: 1em; background-color: var(--code-section-bg-color); border: 3px solid var(--pkg-terminal-border-color); border-radius: 0.6em; margin: 1em 0; overflow-wrap: break-word; } jlerror > header { color: var(--jlerror-header-color); border-left: 3px solid var(--jlerror-header-color); padding: 0.7rem; background: var(--white); border-radius: 3px; } jlerror > header > p { margin-block-end: 0.2em; white-space: pre-wrap; } jlerror > header > p:first-child { font-weight: bold; margin-block-start: 0; } jlerror .stacktrace-header, jlerror .error-header { font-family: var(--system-ui-font-stack); } jlerror .error-header { margin-block-end: 1em; } jlerror secret-h1 { font-size: 1.9rem; font-weight: 700; color: var(--pluto-output-h-color); } jlerror > section { border-block-start: 3px dashed var(--pkg-terminal-border-color); margin-block-start: 1rem; padding-block-start: 1rem; } jlerror .stacktrace-waiting-to-view { display: flex; justify-content: flex-start; align-items: center; padding: 1rem 0 0 0; } jlerror .stacktrace-waiting-to-view button { background: var(--white); border: 3px solid var(--pkg-terminal-border-color); border: none; border-radius: 6px; padding: 0.5em 1em; color: var(--pluto-output-color); font-family: var(--system-ui-font-stack); cursor: pointer; font-weight: 700; } jlerror > section > ol { line-height: 1.6; /* transform: perspective(29rem) rotateX(-12.7deg); */ /* transform-origin: top; */ /* perspective-origin: top; */ } jlerror > section > ol > li { margin-block-end: 1em; } jlerror > section > ol > li:not(.important):not(:hover) { opacity: 0.5; } jlerror > section > ol > li:not(.important)::marker { font-weight: 100; } jlerror > section > ol > li.from_this_notebook { --bg: var(--jl-info-acccolor); background: var(--bg); outline: 3px solid var(--bg); padding: 0.4em 0em; border-radius: 0.6em; } jlerror > section .classical-frame > mark { background: var(--jlerror-mark-bg-color); border-radius: 6px; color: var(--jlerror-mark-color); font-family: var(--julia-mono-font-stack); font-variant-ligatures: none; } jlerror > section .classical-frame > mark > strong { color: var(--black); } jlerror > section .classical-frame s-span { /* color: var(--cm-color-type); */ } jlerror > section .classical-frame s-span .argument_name { color: var(--jlerror-mark-color); color: var(--cm-color-variable); color: var(--cm-color-type); } jlerror > section .frame-source { display: flex; flex-direction: row; align-items: baseline; /* justify-content: flex-end; */ } jlerror > section .frame-source > a { background: var(--jlerror-a-bg-color); border-radius: 4px; padding: 1px 7px; text-decoration: none; border-left: 3px solid var(--jlerror-a-border-left-color); /* font-family: var(--system-ui-font-stack); */ } jlerror > section .frame-source > a:not([href]) { filter: grayscale(1); } jlerror > section .frame-source > a[href].remote-url { filter: hue-rotate(160deg); } jlerror > section li.from_this_notebook:not(.from_this_cell) .frame-source > a[href] { filter: hue-rotate(50deg); } jlerror > section .frame-source > span { opacity: 0.4; padding: 0px 0.2em; } jlerror > section .doclink { user-select: none; } jlerror li::marker { background: red; border: 3px solid red; /* font-size: 0.7rem; */ font-weight: 900; color: var(--pluto-logs-key-color); } jlerror li.from_this_notebook .classical-frame { /* opacity: 0.4; */ } jlerror li a.frame-line-preview { display: block; text-decoration: none; border: 3px solid var(--cm-color-clickable-underline); --br: 0.6em; border-radius: var(--br); --crop: -0.5em; } jlerror li .frame-line-preview pre:not(.asdfdsaf) { background-color: var(--code-background); padding: 0; border-radius: var(--br); overflow: hidden; position: relative; display: block; } jlerror li:not(.from_this_cell) .frame-line-preview pre::after { content: "cell preview"; display: block; position: absolute; bottom: 0; right: 1ch; font-weight: 900; opacity: 0.6; } jlerror li .frame-line-preview pre > code { padding: 0; } jlerror li .frame-line-preview pre > code:not(:only-child).frame-line { background: var(--cm-highlighted); } jlerror li .frame-line-preview pre > code:not(.frame-line) { opacity: 0.7; } jlerror li .frame-line-preview pre > code::before { content: var(--before-content); color: var(--cm-color-line-numbers); margin-right: 0.7em; width: 2ch; display: inline-block; text-align: right; } jlerror li .frame-line-preview pre > code:first-of-type:not(.frame-line) { margin-top: var(--crop); } jlerror li .frame-line-preview pre > code:last-of-type:not(.frame-line) { margin-bottom: var(--crop); } jlerror .dont-panic { position: absolute; top: 0; right: 0; padding: 0.5em; background: var(--pluto-logs-debug-color); color: var(--black); border-radius: 0.2em; font-family: var(--system-ui-font-stack); font-size: 1.2rem; font-weight: 700; transform: rotate(6deg); } pluto-editor.disable_ui jlerror .dont-panic { display: none; } pluto-logs jlerror .dont-panic { display: none; } table.pluto-table { table-layout: fixed; } table.pluto-table td { max-width: 300px; overflow: auto; } @supports (-moz-appearance: none) { table.pluto-table td { max-width: unset; overflow: visible; } table.pluto-table td > div { max-width: 300px; overflow: auto; } } table.pluto-table .schema-types { color: var(--pluto-schema-types-color); font-family: var(--julia-mono-font-stack); font-size: 0.75rem; opacity: 0; } table.pluto-table .schema-types th { border-bottom: 1px solid var(--pluto-schema-types-border-color); background-color: var(--main-bg-color); height: 2rem; } table.pluto-table thead:hover .schema-types { opacity: 1; } table.pluto-table .schema-names th { transform: translate(0, 0.5em); transition: transform 0.1s ease-in-out; } table.pluto-table .schema-names th:first-child, table.pluto-table .schema-types th:first-child { z-index: 2; left: -10px; } table.pluto-table .schema-names th, table.pluto-table .schema-types th:first-child { background-color: var(--main-bg-color); position: sticky; top: calc(0.25rem - var(--pluto-cell-spacing)); height: 2rem; z-index: 1; } table.pluto-table thead:hover .schema-names th { transform: translate(0, 0); } table.pluto-table tbody th:first-child { background-color: var(--main-bg-color); position: sticky; left: -10px; /* padding-left of pluto-output*/ white-space: nowrap; } table.pluto-table .pluto-tree-more-td { text-align: left; overflow: unset; } table.pluto-table .pluto-tree-more-td pluto-tree-more { overflow: unset; position: sticky; left: 0; top: 2rem; max-width: 650px; } table.pluto-table tr.empty div { display: flex; justify-content: center; align-items: center; width: 100%; font-size: 1.5rem; flex-flow: column nowrap; } table.pluto-table tr.empty small { font-size: 0.5rem; } pluto-tree.collapsed p-v > pre { max-height: 2em; overflow-y: hidden; } function ismodern() { try { // See: https://kangax.github.io/compat-table/es2016plus/ // 2020 check: // return eval("let {a, ...r} = {a:1,b:1}; r?.a != r.b; 1 ?? 2") // 2021 check: // return eval("let {a, ...r} = {a:1,b:1}; r?.a != r.b; 1 ?? 2; a ||= false") // 2021 check (Chrome 85+, Firefox 77+, Safari 13.1+) // Please check with macs return Boolean(String.prototype.replaceAll) } catch (ex) { return false } } window.addEventListener("DOMContentLoaded", function () { if (!ismodern()) { document.body.innerHTML = "<div style='width: 100%; height: 100%; font-family: sans-serif;'><div style='top: 0;right: 0;left: 0;bottom: 50%;width: 300px;height: 300px;margin: auto;position: absolute;background: white; z-index: 100;'><h1>You need a shiny new browser to use Pluto!</h1><p>The latest versions of Firefox and Chrome will work best.</p></div></div>" } }) @charset "UTF-8"; @import url("https://cdn.jsdelivr.net/npm/inter-ui@3.19.3/inter-latin.css"); :root { --pluto-cell-spacing: 17px; /* use the value "contextual" to enable contextual ligatures `document.documentElement.style.setProperty('--pluto-operator-ligatures', 'contextual');` for julia mono see here: https://cormullion.github.io/pages/2020-07-26-JuliaMono/#contextual_and_stylistic_alternates_and_ligatures */ --pluto-operator-ligatures: none; --julia-mono-font-stack: JuliaMono, Menlo, "Roboto Mono", "Lucida Sans Typewriter", "Source Code Pro", monospace; --sans-serif-font-stack: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Oxygen, Ubuntu, Cantarell, "Open Sans", "Helvetica Neue", sans-serif; --lato-ui-font-stack: "Lato", -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Oxygen-Sans, Cantarell, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", system-ui, sans-serif; --inter-ui-font-stack: "Inter", -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Oxygen-Sans, Cantarell, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", system-ui, sans-serif; color-scheme: light dark; } html { font-family: var(--inter-ui-font-stack); font-size: 17px; } main { display: block; max-width: 1200px; padding: 1rem; margin: 0 auto; } /* font-size: 1.7em; */ header { background-color: #f5efd2; /* background-image: linear-gradient(to top, white, #fff0), var(--noise-4); */ /* filter: var(--noise-filter-1); */ display: flex; background-size: cover; justify-content: center; padding: 1.3rem; } header h1 { font-weight: 500; /* font-style: italic; */ text-align: center; /* color: white; */ } section#mywork, section#open { /* --c1: rgb(255 255 255 / 20%); */ /* --c2: rgb(255 255 255 / 7%); */ /* background-color: #92add9; */ /* --grad-stops: transparent 9%, var(--c1) 10%, var(--c1) 12%, transparent 13%, transparent 29%, var(--c2) 30%, var(--c2) 31%, transparent 32%, transparent 49%, var(--c2) 50%, var(--c2) 51%, transparent 52%, transparent 69%, var(--c2) 70%, var(--c2) 71%, transparent 72%, transparent 89%, var(--c2) 90%, var(--c2) 91%, transparent 92%, transparent; */ /* background-size: 40px 40px; */ /* background-position: 20px 20px; */ /* background-image: linear-gradient(360deg, var(--grad-stops)), linear-gradient(90deg, var(--grad-stops)); */ /* background: #f5f5f6; */ /* position: relative; */ /* background: url("https://computationalthinking.mit.edu/Spring21/homepage/bg.svg"); */ /* background-size: cover; */ /* background-position: 0% 70%; */ background: var(--welcome-mywork-bg); /* background: var(--header-bg-color); */ position: relative; } .pluto-logo { font-style: normal; font-weight: 800; color: inherit; /* padding: 0.3em; */ display: flex; flex-direction: row; padding: 0.5em; align-items: center; gap: 0.5ch; font-family: var(--inter-ui-font-stack); transform: translateY(0.23em); } .pluto-logo img { height: 1.2em; width: 1.2em; } #new { background: var(--welcome-open-bg); box-shadow: -2px 4px 9px 0px #00000012; padding: 1.3rem; border-radius: 0.6rem; margin: 1rem; /* border: 0.3rem solid #d6e0d8; */ } #new.desktop_opener { display: flex; flex-direction: row; align-content: center; justify-content: space-around; box-shadow: none; position: relative; } #new.desktop_opener .desktop_picker { width: 100%; } section { display: flex; /* overflow: hidden; */ /* place-items: center; */ /* margin: 0rem 0rem; */ flex-direction: row; justify-content: center; } section > div { margin: 1rem 1rem; max-width: 614px; /* margin: auto; */ flex: 1 1 auto; min-width: 0; } .pluto-logo { background: white; border-radius: 0.4em; display: flex; flex: 0 1 auto; transform: none; font-size: 1.6rem; } section#open { /* background: #f5f5f6; */ /* box-shadow: inset 1px 1px 20px red; */ position: relative; } section#featured > div { max-width: 900px; } header > div { max-width: 62rem; /* margin: 0 auto; */ flex: 1 1 auto; display: flex; z-index: 1; } section#mywork::before, section#open::after { --c: hsl(196deg 20% 26% / 6%); content: ""; height: 50px; top: 0px; left: 0; right: 0; position: absolute; display: block; background: linear-gradient(0deg, transparent, var(--c)); pointer-events: none; z-index: 0; } :where(#mywork, #open) h2 { /* color: black; */ --off: 4px; --offm: -4px; --oc: #ffffff; /* text-shadow: var(--off) 0 var(--oc), var(--off) var(--off) var(--oc), 0 var(--off) var(--oc), var(--offm) var(--off) var(--oc), var(--offm) 0 var(--oc), var(--offm) var(--offm) var(--oc), 0 var(--offm) var(--oc), var(--off) var(--offm) var(--oc); */ display: inline-block; /* background: #fffffffc; */ /* padding: 0.4em; */ border-radius: 0.4em; /* color: white; */ /* text-transform: uppercase; */ margin: 2rem 0rem 0rem 0; } section#open::after { top: unset; bottom: 0; background: linear-gradient(0deg, var(--c), transparent); } div#app { /* background: url(https://computationalthinking.mit.edu/Spring21/homepage/bg.svg); */ background-size: cover; background-position: 0% 77%; } section#featured { /* background: white; */ } .new a { text-decoration: none; /* font-weight: 700; */ font-weight: 500; font-style: italic; } li.new { position: sticky; background: var(--welcome-newnotebook-bg); top: 0; z-index: 2; } h1 { font-size: 2.8rem; margin-block-end: 0em; } .collection { margin: 6em 0; } .collection h2 { font-size: 2.5rem; font-weight: 600; margin: 0; } #featured p { max-width: 54ch; } #github img { aspect-ratio: 1; filter: var(--image-filters); width: 2rem; } a#github { display: block; position: absolute; top: 0.5rem; right: 0.5rem; } .show_scrollbar::-webkit-scrollbar { width: 10px; opacity: 0.1; } .show_scrollbar::-webkit-scrollbar-track { } .show_scrollbar::-webkit-scrollbar-thumb { /* height: 11px; */ background-color: var(--black); opacity: 0.6; border-radius: 1000px; } .show_scrollbar::-webkit-scrollbar-thumb:hover { opacity: 1; } // @ts-ignore import vmsg from "https://cdn.jsdelivr.net/npm/vmsg@0.4.0/vmsg.js" // when modifying, also modify the version number in all other files. import { get_included_external_source } from "./external_source.js" const create_recorder_mp3 = async () => { const wasmURL = get_included_external_source("vmsg-wasm")?.href if (!wasmURL) throw new Error("wasmURL not found") const recorder = new vmsg.Recorder({ wasmURL }) return { start: async () => { await recorder.initAudio() await recorder.initWorker() recorder.startRecording() }, stop: async () => { const blob = await recorder.stopRecording() return window.URL.createObjectURL(blob) }, } } export const create_recorder = () => { try { return create_recorder_mp3() } catch (e) { console.error("Failed to create mp3 recorder", e) } return create_recorder_native() } // really nice but it can only record to audio/ogg or sometihng, nothing that works across all browsers const create_recorder_native = async () => { const stream = await navigator.mediaDevices.getUserMedia({ audio: true }) let chunks = [] const mediaRecorder = new MediaRecorder(stream, {}) mediaRecorder.ondataavailable = function (e) { chunks.push(e.data) } let start_return_promise = new Promise((r) => { mediaRecorder.onstart = r }) let stop_return_promise = new Promise((r) => { mediaRecorder.onstop = function (e) { const blob = new Blob(chunks, { type: "audio/ogg; codecs=opus" }) chunks = [] const audioURL = window.URL.createObjectURL(blob) r(audioURL) } }) return { start: () => { mediaRecorder.start() start_return_promise }, stop: () => { mediaRecorder.stop() return stop_return_promise }, } } // taken from https://github.com/edoudou/create-silent-audio // original license from https://github.com/edoudou/create-silent-audio/blob/master/LICENSE /* MIT License Copyright (c) 2019 Edouard Short Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. */ export function createSilentAudio(time, freq = 44100) { const length = time * freq // @ts-ignore const AudioContext = window.AudioContext || window.webkitAudioContext || window.mozAudioContext if (!AudioContext) { console.log("No Audio Context") } const context = new AudioContext() const audioFile = context.createBuffer(1, length, freq) return URL.createObjectURL(bufferToWave(audioFile, length)) } function bufferToWave(abuffer, len) { let numOfChan = abuffer.numberOfChannels, length = len * numOfChan * 2 + 44, buffer = new ArrayBuffer(length), view = new DataView(buffer), channels = [], i, sample, offset = 0, pos = 0 // write WAVE header setUint32(0x46464952) setUint32(length - 8) setUint32(0x45564157) setUint32(0x20746d66) setUint32(16) setUint16(1) setUint16(numOfChan) setUint32(abuffer.sampleRate) setUint32(abuffer.sampleRate * 2 * numOfChan) setUint16(numOfChan * 2) setUint16(16) setUint32(0x61746164) setUint32(length - pos - 4) // write interleaved data for (i = 0; i < abuffer.numberOfChannels; i++) channels.push(abuffer.getChannelData(i)) while (pos < length) { for (i = 0; i < numOfChan; i++) { // interleave channels sample = Math.max(-1, Math.min(1, channels[i][offset])) // clamp sample = (0.5 + sample < 0 ? sample * 32768 : sample * 32767) | 0 // scale to 16-bit signed int view.setInt16(pos, sample, true) // write 16-bit sample pos += 2 } offset++ // next source sample } // create Blob return new Blob([buffer], { type: "audio/wav" }) function setUint16(data) { view.setUint16(pos, data, true) pos += 2 } function setUint32(data) { view.setUint32(pos, data, true) pos += 4 } } import immer from "../imports/immer.js" import { timeout_promise, ws_address_from_base } from "./PlutoConnection.js" import { with_query_params } from "./URLTools.js" export const BackendLaunchPhase = { wait_for_user: 0, requesting: 0.4, created: 0.6, responded: 0.7, notebook_running: 0.9, ready: 1.0, } // The following function is based on the wonderful https://github.com/executablebooks/thebe which has the following license: /* LICENSE Copyright Executable Books Project Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ export const trailingslash = (s) => (s.endsWith("/") ? s : s + "/") export const request_binder = (build_url, { on_log }) => new Promise((resolve, reject) => { console.log("Starting binder connection to", build_url) try { let es = new EventSource(build_url) es.onerror = (err) => { console.error("Binder error: Lost connection to " + build_url, err) es.close() reject(err) } let phase = null let logs = `` let report_log = (msg) => { console.log("Binder: ", msg, ` at ${new Date().toLocaleTimeString()}`) logs = `${logs}${msg}\n` on_log(logs) } es.onmessage = (evt) => { let msg = JSON.parse(evt.data) if (msg.phase && msg.phase !== phase) { phase = msg.phase.toLowerCase() report_log(`\n\n Binder subphase: ${phase}\n`) } if (msg.message) { report_log(msg.message.replace(`] `, `]\n`)) } switch (msg.phase) { case "failed": console.error("Binder error: Failed to build", build_url, msg) es.close() reject(new Error(msg)) break case "ready": es.close() resolve({ binder_session_url: trailingslash(msg.url) + "pluto/", binder_session_token: msg.token, }) break } } } catch (err) { console.error(err) reject("Failed to open event source the mybinder.org. This probably means that the URL is invalid.") } }) // view stats on https://stats.plutojl.org/ export const count_stat = (page) => fetch(`https://stats.plutojl.org/count?p=/${page}&s=${screen.width},${screen.height},${devicePixelRatio}#skip_sw`, { cache: "no-cache" }).catch(() => {}) /** * Start a 'headless' binder session, open our notebook in it, and connect to it. */ export const start_binder = async ({ setStatePromise, connect, launch_params }) => { try { // view stats on https://stats.plutojl.org/ count_stat(`binder-start`) await setStatePromise( immer((/** @type {import("../components/Editor.js").EditorState} */ state) => { state.backend_launch_phase = BackendLaunchPhase.requesting state.disable_ui = false // Clear the Status of the process that generated the HTML state.notebook.status_tree = null }) ) /// PART 1: Creating a binder session.. const { binder_session_url, binder_session_token } = await request_binder(launch_params.binder_url.replace("mybinder.org/v2/", "mybinder.org/build/"), { on_log: (logs) => setStatePromise( immer((/** @type {import("../components/Editor.js").EditorState} */ state) => { state.backend_launch_logs = logs }) ), }) const with_token = (u) => with_query_params(u, { token: binder_session_token }) console.log("Binder URL:", with_token(binder_session_url)) //@ts-ignore window.shutdown_binder = () => { fetch(with_token(new URL("../api/shutdown", binder_session_url)), { method: "POST" }) } await setStatePromise( immer((/** @type {import("../components/Editor.js").EditorState} */ state) => { state.backend_launch_phase = BackendLaunchPhase.created state.binder_session_url = binder_session_url state.binder_session_token = binder_session_token }) ) // fetch index to say hello to the pluto server. this ensures that the pluto server is running and it triggers JIT compiling some of the HTTP code. await fetch(with_token(binder_session_url)) await setStatePromise( immer((/** @type {import("../components/Editor.js").EditorState} */ state) => { state.backend_launch_phase = BackendLaunchPhase.responded }) ) /// PART 2: Using Pluto's REST API to open the notebook file. We either upload the notebook with a POST request, or we let the server open by giving it the filename/URL. let download_locally_and_upload = async () => { const upload_url = with_token( with_query_params(new URL("notebookupload", binder_session_url), { name: new URLSearchParams(window.location.search).get("name"), execution_allowed: "true", }) ) console.log(`downloading locally and uploading `, upload_url, launch_params.notebookfile) return fetch(upload_url, { method: "POST", body: await (await fetch(new Request(launch_params.notebookfile, { integrity: launch_params.notebookfile_integrity }))).arrayBuffer(), }) } let open_remotely = async (p1, p2) => { const open_url = with_query_params(new URL("open", binder_session_url), { [p1]: p2, execution_allowed: "true", }) console.log(`open ${p1}:`, open_url) return fetch(with_token(open_url), { method: "POST", }) } let open_remotely_fn = (p1, p2) => () => open_remotely(p1, p2) let methods_to_try = launch_params.notebookfile.startsWith("data:") ? [download_locally_and_upload] : [ // open_remotely_fn("path", launch_params.notebookfile), // open_remotely_fn("url", new URL(launch_params.notebookfile, window.location.href).href), // download_locally_and_upload, ] let open_response = new Response() for (let method of methods_to_try) { open_response = await method() if (open_response.ok) { break } } if (!open_response.ok) { let b = await open_response.blob() window.location.href = URL.createObjectURL(b) return } // Opening a notebook gives us the notebook ID, which means that we have a running session! Time to connect. const new_notebook_id = await open_response.text() const edit_url = with_token(with_query_params(new URL("edit", binder_session_url), { id: new_notebook_id })) console.info("notebook_id:", new_notebook_id) await setStatePromise( immer((/** @type {import("../components/Editor.js").EditorState} */ state) => { state.notebook.notebook_id = new_notebook_id state.backend_launch_phase = BackendLaunchPhase.notebook_running state.refresh_target = edit_url }) ) /// PART 3: We open the WebSocket connection to the Pluto server, connecting to the notebook ID that was created for us. If this fails, or after a 20 second timeout, we give up on hot-swapping a live backend into this static view, and instead we just redirect to the binder URL. console.log("Connecting WebSocket") const connect_promise = connect(with_token(new URL("channels", ws_address_from_base(binder_session_url)))) await timeout_promise(connect_promise, 20_000).catch((e) => { console.error("Failed to establish connection within 20 seconds. Navigating to the edit URL directly.", e) window.parent.location.href = edit_url }) } catch (err) { console.error("Failed to initialize binder!", err) alert("Something went wrong! \n\nWe failed to initialize the binder connection. Please try again with a different browser, or come back later.") } } // import Generators_input from "https://unpkg.com/@observablehq/stdlib@3.3.1/src/generators/input.js" // import Generators_input from "https://unpkg.com/@observablehq/stdlib@3.3.1/src/generators/input.js" import { open_pluto_popup } from "../common/open_pluto_popup.js" import _ from "../imports/lodash.js" import { html } from "../imports/Preact.js" import observablehq from "./SetupCellEnvironment.js" /** * Copied from the observable stdlib source, but we need it to be faster than Generator.input because Generator.input is async by nature, so will lag behind that one tick that is breaking the code. * https://github.com/observablehq/stdlib/blob/170f137ac266b397446320e959c36dd21888357b/src/generators/input.js#L13 * @param {Element} input * @returns {any} */ export function get_input_value(input) { if (input instanceof HTMLInputElement) { switch (input.type) { case "range": case "number": return input.valueAsNumber case "date": // "time" uses .value, which is a string. This matches observable. return input.valueAsDate case "checkbox": return input.checked case "file": return input.multiple ? input.files : input.files?.[0] default: return input.value } } else if (input instanceof HTMLSelectElement && input.multiple) { return Array.from(input.selectedOptions, (o) => o.value) } else { //@ts-ignore return input.value } } /** * Copied from the observable stdlib source (https://github.com/observablehq/stdlib/blob/170f137ac266b397446320e959c36dd21888357b/src/generators/input.js) without modifications. * @param {Element} input * @returns {string} */ export function eventof(input) { //@ts-ignore switch (input.type) { case "button": case "submit": case "checkbox": return "click" case "file": return "change" default: return "input" } } /** * Copied from the observable stdlib source (https://github.com/observablehq/stdlib/blob/170f137ac266b397446320e959c36dd21888357b/src/generators/input.js) but using our own `get_input_value` for consistency. * @param {Element} input * @returns */ function input_generator(input) { return observablehq.Generators.observe(function (change) { var event = eventof(input), value = get_input_value(input) function inputted() { change(get_input_value(input)) } input.addEventListener(event, inputted) if (value !== undefined) change(value) return function () { input.removeEventListener(event, inputted) } }) } /** * @param {Element} input * @param {any} new_value */ export const set_input_value = (input, new_value) => { if (input instanceof HTMLInputElement && input.type === "file") { return } if (new_value == null) { //@ts-ignore input.value = new_value return } if (input instanceof HTMLInputElement) { switch (input.type) { case "range": case "number": { if (input.valueAsNumber !== new_value) { input.valueAsNumber = new_value } return } case "date": { if (input.valueAsDate == null || Number(input.valueAsDate) !== Number(new_value)) { input.valueAsDate = new_value } return } case "checkbox": { if (input.checked !== new_value) { input.checked = new_value } return } case "file": { // Can't set files :( return } } } else if (input instanceof HTMLSelectElement && input.multiple) { for (let option of Array.from(input.options)) { option.selected = new_value.includes(option.value) } return } //@ts-ignore if (input.value !== new_value) { //@ts-ignore input.value = new_value } } /** * @param {NodeListOf<Element>} bond_nodes * @param {import("../components/Editor.js").BondValuesDict} bond_values */ export const set_bound_elements_to_their_value = (bond_nodes, bond_values) => { bond_nodes.forEach((bond_node) => { let bond_name = bond_node.getAttribute("def") if (bond_name != null && bond_node.firstElementChild != null && bond_values[bond_name] != null) { let val = bond_values[bond_name].value try { set_input_value(bond_node.firstElementChild, val) } catch (error) { console.error(`Error while setting input value`, bond_node.firstElementChild, `to value`, val, `: `, error) } } }) } /** * @param {NodeListOf<Element>} bond_nodes * @param {Promise<void>} invalidation */ export const add_bonds_disabled_message_handler = (bond_nodes, invalidation) => { bond_nodes.forEach((bond_node) => { const listener = (e) => { if (e.target.closest(".bonds_disabled:where(.offer_binder, .offer_local)")) { open_pluto_popup({ type: "info", source_element: e.target, body: html`${`You are viewing a static document. `} <a href="#" onClick=${(e) => { //@ts-ignore window.open_edit_or_run_popup() e.preventDefault() window.dispatchEvent(new CustomEvent("close pluto popup")) }} >Run this notebook</a > ${` to enable interactivity.`}`, }) } } bond_node.addEventListener("click", listener) invalidation.then(() => { bond_node.removeEventListener("click", listener) }) }) } /** * @param {NodeListOf<Element>} bond_nodes * @param {(name: string, value: any) => Promise} on_bond_change * @param {import("../components/Editor.js").BondValuesDict} known_values Object of variable names that already have a value in the state, which we may not want to send the initial bond value for. When reloading the page, bonds are set to their values from the state, and we don't want to trigger a change event for those. * @param {Promise<void>} invalidation */ export const add_bonds_listener = (bond_nodes, on_bond_change, known_values, invalidation) => { // the <bond> node will be invalidated when the cell re-evaluates. when this happens, we need to stop processing input events let node_is_invalidated = false invalidation.then(() => { node_is_invalidated = true }) bond_nodes.forEach(async (bond_node) => { const name = bond_node.getAttribute("def") const bound_element_node = bond_node.firstElementChild if (name != null && bound_element_node != null) { const initial_value = get_input_value(bound_element_node) let skip_initialize = Object.keys(known_values).includes(name) && _.isEqual(known_values[name]?.value, initial_value) // Initialize the bond. This will send the data to the backend for the first time. If it's already there, and the value is the same, cells won't rerun. const init_promise = skip_initialize ? null : on_bond_change(name, initial_value).catch(console.error) // see the docs on Generators.input from observablehq/stdlib let skippped_first = false for (let val of input_generator(bound_element_node)) { if (node_is_invalidated) break if (skippped_first === false) { skippped_first = true continue } // wait for a new input value. If a value is ready, then this promise resolves immediately const to_send = await transformed_val(await val) // send to the Pluto back-end (have a look at set_bond in Editor.js) // await the setter to avoid collisions //TODO : get this from state await init_promise await on_bond_change(name, to_send).catch(console.error) } } }) } /** * The identity function in most cases, loads file contents when appropriate * @type {((val: FileList) => Promise<Array<File>>) * & ((val: File) => Promise<{ name: string, type: string, data: Uint8Array }>) * & ((val: any) => Promise<any>) * } */ const transformed_val = async (val) => { if (val instanceof FileList) { return Promise.all(Array.from(val).map((file) => transformed_val(file))) } else if (val instanceof File) { return await new Promise((res) => { const reader = new FileReader() // @ts-ignore reader.onload = () => res({ name: val.name, type: val.type, data: new Uint8Array(reader.result) }) reader.onerror = () => res({ name: val.name, type: val.type, data: null }) reader.readAsArrayBuffer(val) }) } else { return val } } // EXAMPLE: /* cl({a: true, b: false, c: true}) == "a c " */ export const cl = (classTable) => { if(!classTable){ return null } return Object.entries(classTable).reduce((allClasses, [nextClass, enable]) => (enable ? nextClass + " " + allClasses : allClasses), "") } function environment({ client, editor, imports: { preact: { html, useEffect, useState, useMemo }, }, }) { const noop = () => false return { custom_editor_header_component: noop, custom_welcome: noop, custom_recent: noop, custom_filepicker: noop, } } export const get_environment = async (client) => { let environment // Draal this is for you // @ts-ignore if (!window.pluto_injected_environment) { const { default: env } = await import(client.session_options.server.injected_javascript_data_url) environment = env } else { // @ts-ignore environment = window.pluto_injected_environment.environment } return environment } export default environment import { timeout_promise } from "./PlutoConnection.js" // Sorry Fons, even this part of the code is now unnessarily overengineered. // But at least, I overengineered this on purpose. - DRAL let async = async (async) => async() let firebase_load_promise = null const init_firebase = async () => { if (firebase_load_promise == null) { firebase_load_promise = async(async () => { let [{ initializeApp }, firestore_module] = await Promise.all([ // @ts-ignore import("https://www.gstatic.com/firebasejs/10.8.0/firebase-app.js"), // @ts-ignore import("https://www.gstatic.com/firebasejs/10.8.0/firebase-firestore.js"), ]) let { getFirestore, addDoc, doc, collection } = firestore_module // @ts-ignore let app = initializeApp({ apiKey: "AIzaSyC0DqEcaM8AZ6cvApXuNcNU2RgZZOj7F68", authDomain: "localhost", projectId: "pluto-feedback", }) let db = getFirestore(app) let feedback_db = collection(db, "feedback") let add_feedback = async (feedback) => { let docref = await addDoc(feedback_db, feedback) console.debug("Firestore doc created ", docref.id, docref) } console.log("base loaded", { initializeApp, firestore_module, app, db, feedback_db, add_feedback }) // @ts-ignore return add_feedback }) } return await firebase_load_promise } export const init_feedback = async () => { try { // Only load firebase when the feedback form is touched const feedbackform = document.querySelector("form#feedback") if (feedbackform == null) return feedbackform.addEventListener("submit", (e) => { const email = prompt("Would you like us to contact you?\n\nEmail: (leave blank to stay anonymous )") e.preventDefault() async(async () => { try { const feedback = String(new FormData(e.target).get("opinion")) if (feedback.length < 4) return let add_feedback = await init_firebase() await timeout_promise( add_feedback({ feedback, timestamp: Date.now(), email: email ? email : "", }), 5000 ) let message = "Submitted. Thank you for your feedback! " console.log(message) alert(message) // @ts-ignore feedbackform.querySelector("#opinion").value = "" } catch (error) { let message = "Whoops, failed to send feedback \nWe would really like to hear from you! Please got to https://github.com/fonsp/Pluto.jl/issues to report this failure:\n\n" console.error(message) console.error(error) alert(message + error) } }) }) feedbackform.addEventListener("focusin", () => { // Start loading firebase when someone interacts with the form init_firebase() }) } catch (error) { console.error("Something went wrong loading the feedback form:", error) // @ts-ignore document.querySelector("form#feedback").style.opacity = 0 for (let char of "Oh noooooooooooooooooo...") { // @ts-ignore document.querySelector("form#feedback input").value += char await new Promise((resolve) => setTimeout(resolve, 200)) } } } import { useEffect, useState } from "../imports/Preact.js" import _ from "../imports/lodash.js" const loading_times_url = `https://julia-loading-times-test.netlify.app/pkg_load_times.csv` const package_list_url = `https://julia-loading-times-test.netlify.app/top_packages_sorted_with_deps.txt` /** @typedef {{ install: Number, precompile: Number, load: Number }} LoadingTime */ /** * @typedef PackageTimingData * @type {{ * times: Map<String,LoadingTime>, * packages: Map<String,String[]>, * }} */ /** @type {{ current: Promise<PackageTimingData>? }} */ const data_promise_ref = { current: null } export const get_data = () => { if (data_promise_ref.current != null) { return data_promise_ref.current } else { const times_p = fetch(loading_times_url) .then((res) => res.text()) .then((text) => { const lines = text.split("\n") const header = lines[0].split(",") return new Map( lines.slice(1).map((line) => { let [pkg, ...times] = line.split(",") return [pkg, { install: Number(times[0]), precompile: Number(times[1]), load: Number(times[2]) }] }) ) }) const packages_p = fetch(package_list_url) .then((res) => res.text()) .then( (text) => new Map( text.split("\n").map((line) => { let [pkg, ...deps] = line.split(",") return [pkg, deps] }) ) ) data_promise_ref.current = Promise.all([times_p, packages_p]).then(([times, packages]) => ({ times, packages })) return data_promise_ref.current } } export const usePackageTimingData = () => { const [data, set_data] = useState(/** @type {PackageTimingData?} */ (null)) useEffect(() => { get_data().then(set_data) }, []) return data } const recursive_deps = (/** @type {PackageTimingData} */ data, /** @type {string} */ pkg, found = []) => { const deps = data.packages.get(pkg) if (deps == null) { return [] } else { const newfound = _.union(found, deps) return [...deps, ..._.difference(deps, found).flatMap((dep) => recursive_deps(data, dep, newfound))] } } export const time_estimate = (/** @type {PackageTimingData} */ data, /** @type {string[]} */ packages) => { let deps = packages.flatMap((pkg) => recursive_deps(data, pkg)) let times = _.uniq([...packages, ...deps]) .map((pkg) => data.times.get(pkg)) .filter((x) => x != null) let sum = (xs) => xs.reduce((acc, x) => acc + (x == null || isNaN(x) ? 0 : x), 0) return { install: sum(times.map(_.property("install"))) * timing_weights.install, precompile: sum(times.map(_.property("precompile"))) * timing_weights.precompile, load: sum(times.map(_.property("load"))) * timing_weights.load, } } const timing_weights = { // Because the GitHub Action runner has superfast internet install: 2, // Because the GitHub Action runner has average compute speed load: 1, // Because precompilation happens in parallel precompile: 0.3, } // @ts-ignore export let is_mac_keyboard = /Mac/i.test(navigator.userAgentData?.platform ?? navigator.platform) export let control_name = is_mac_keyboard ? "" : "Ctrl" export let ctrl_or_cmd_name = is_mac_keyboard ? "" : "Ctrl" export let alt_or_options_name = is_mac_keyboard ? "" : "Alt" export let and = is_mac_keyboard ? " " : "+" export let has_ctrl_or_cmd_pressed = (event) => event.ctrlKey || (is_mac_keyboard && event.metaKey) export let map_cmd_to_ctrl_on_mac = (keymap) => { if (!is_mac_keyboard) { return keymap } let keymap_with_cmd = { ...keymap } for (let [key, handler] of Object.entries(keymap)) { keymap_with_cmd[key.replace(/Ctrl/g, "Cmd")] = handler // remove Ctrl-D from Pluto.jl keybind for MacOS if (key == "Ctrl-D") { delete keymap_with_cmd[key] } } return keymap_with_cmd } export let in_textarea_or_input = () => { const in_footer = document.activeElement?.closest("footer") != null const in_header = document.activeElement?.closest("header") != null const in_cm = document.activeElement?.closest(".cm-editor") != null const { tagName } = document.activeElement ?? {} return tagName === "INPUT" || tagName === "TEXTAREA" || in_footer || in_header || in_cm } const log_functions = { Info: console.info, Error: console.error, Warn: console.warn, Debug: console.debug, } export const handle_log = ({ level, msg, file, line, kwargs }, filename) => { try { const f = log_functions[level] || console.log const args = [`%c ${level}:\n`, `font-weight: bold`, msg] if (Object.keys(kwargs).length !== 0) { args.push(kwargs) } if (file.startsWith(filename)) { const cell_id = file.substring(file.indexOf("#==#") + 4) const cell_node = document.getElementById(cell_id) args.push(`\n\nfrom`, cell_node) } f(...args) } catch (err) {} // TODO } // ES6 import for msgpack-lite, we use the fonsp/msgpack-lite fork to make it ES6-importable (without nodejs) import msgpack from "../imports/msgpack-lite.js" // based on https://github.com/kawanet/msgpack-lite/blob/5b71d82cad4b96289a466a6403d2faaa3e254167/lib/ext-packer.js const codec = msgpack.createCodec() const packTypedArray = (x) => new Uint8Array(x.buffer, x.byteOffset, x.byteLength) codec.addExtPacker(0x11, Int8Array, packTypedArray) codec.addExtPacker(0x12, Uint8Array, packTypedArray) codec.addExtPacker(0x13, Int16Array, packTypedArray) codec.addExtPacker(0x14, Uint16Array, packTypedArray) codec.addExtPacker(0x15, Int32Array, packTypedArray) codec.addExtPacker(0x16, Uint32Array, packTypedArray) codec.addExtPacker(0x17, Float32Array, packTypedArray) codec.addExtPacker(0x18, Float64Array, packTypedArray) codec.addExtPacker(0x12, Uint8ClampedArray, packTypedArray) codec.addExtPacker(0x12, ArrayBuffer, (x) => new Uint8Array(x)) codec.addExtPacker(0x12, DataView, packTypedArray) // Pack and unpack dates. However, encoding a date does throw on Safari because it doesn't have BigInt64Array. // This isn't too much a problem, as Safari doesn't even support <input type=date /> yet... // But it does throw when I create a custom @bind that has a Date value... // For decoding I now also use a "Invalid Date", but the code in https://stackoverflow.com/a/55338384/2681964 did work in Safari. // Also there is no way now to send an "Invalid Date", so it just does nothing codec.addExtPacker(0x0d, Date, (d) => new BigInt64Array([BigInt(+d)])) codec.addExtUnpacker(0x0d, (uintarray) => { if ("getBigInt64" in DataView.prototype) { let dataview = new DataView(uintarray.buffer, uintarray.byteOffset, uintarray.byteLength) let bigint = dataview.getBigInt64(0, true) // true here is "littleEndianes", not sure if this only Works On My Machine if (bigint > Number.MAX_SAFE_INTEGER) { throw new Error(`Can't read too big number as date (how far in the future is this?!)`) } return new Date(Number(bigint)) } else { return new Date(NaN) } }) codec.addExtUnpacker(0x11, (x) => new Int8Array(x.buffer)) codec.addExtUnpacker(0x12, (x) => new Uint8Array(x.buffer)) codec.addExtUnpacker(0x13, (x) => new Int16Array(x.buffer)) codec.addExtUnpacker(0x14, (x) => new Uint16Array(x.buffer)) codec.addExtUnpacker(0x15, (x) => new Int32Array(x.buffer)) codec.addExtUnpacker(0x16, (x) => new Uint32Array(x.buffer)) codec.addExtUnpacker(0x17, (x) => new Float32Array(x.buffer)) codec.addExtUnpacker(0x18, (x) => new Float64Array(x.buffer)) /** @param {any} x */ export const pack = (x) => { return msgpack.encode(x, { codec: codec }) } /** @param {Uint8Array} x */ export const unpack = (x) => { return msgpack.decode(x, { codec: codec }) } export const new_update_message = (client) => fetch_pluto_releases() .then((releases) => { const local = client.version_info.pluto const latest = releases[releases.length - 1].tag_name console.log(`Pluto version ${local}`) const local_index = releases.findIndex((r) => r.tag_name === local) if (local_index !== -1) { const updates = releases.slice(local_index + 1) const recommended_updates = updates.filter((r) => r.body.toLowerCase().includes("recommended update")) if (recommended_updates.length > 0) { console.log(`Newer version ${latest} is available`) if (!client.version_info.dismiss_update_notification) { alert( "A new version of Pluto.jl is available! \n\n You have " + local + ", the latest is " + latest + '.\n\nYou can update Pluto.jl using the julia package manager:\n import Pkg; Pkg.update("Pluto")\nAfterwards, exit Pluto.jl and restart julia.' ) } } } }) .catch(() => { // Having this as a uncaught promise broke the frontend tests for me // so I'm just swallowing the error explicitly - DRAL }) const fetch_pluto_releases = async () => { let response = await fetch("https://api.github.com/repos/fonsp/Pluto.jl/releases", { method: "GET", mode: "cors", cache: "no-cache", headers: { "Content-Type": "application/json", }, redirect: "follow", referrerPolicy: "no-referrer", }) return (await response.json()).reverse() } // @ts-nocheck /* Some packages look for `process.env`, so we give it to them. */ /* Why not just `window.process = { env: NODE_ENV }`? I once had an extension that was broken and exported its `window.process` to all pages. I'm not saying we should support that, but this code made it work. The extension itself is now fixed, but this might just work when an extension does something similar in the future  */ try { if (window.process == null) { window.process = {} } if (window.process.env == null) { window.process.env = {} } window.process.env.NODE_ENV = "production" } catch (err) { console.warn(`Couldn't set window.process.env, this might break some things`) } // should strip characters similar to how github converts filenames into the #file-... URL hash. // test on: https://gist.github.com/fonsp/f7d230da4f067a11ad18de15bff80470 const gist_normalizer = (str) => str .toLowerCase() .normalize("NFD") .replace(/[^a-z1-9]/g, "") export const guess_notebook_location = async (path_or_url) => { try { const u = new URL(path_or_url) if (!["http:", "https:", "ftp:", "ftps:"].includes(u.protocol)) { throw "Not a web URL" } if (u.host === "gist.github.com") { console.log("Gist URL detected") const parts = u.pathname.substring(1).split("/") const gist_id = parts[1] const gist = await ( await fetch(`https://api.github.com/gists/${gist_id}`, { headers: { Accept: "application/vnd.github.v3+json" }, }).then((r) => (r.ok ? r : Promise.reject(r))) ).json() console.log(gist) const files = Object.values(gist.files) const selected = files.find((f) => gist_normalizer("#file-" + f.filename) === gist_normalizer(u.hash)) if (selected != null) { return { type: "url", path_or_url: selected.raw_url, } } return { type: "url", path_or_url: files[0].raw_url, } } else if (u.host === "github.com") { u.searchParams.set("raw", "true") } return { type: "url", path_or_url: u.href, } } catch (ex) { /* Remove eventual single/double quotes from the path if they surround it, see https://github.com/fonsp/Pluto.jl/issues/1639 */ if (path_or_url[path_or_url.length - 1] === '"' && path_or_url[0] === '"') { path_or_url = path_or_url.slice(1, -1) /* Remove first and last character */ } return { type: "path", path_or_url: path_or_url, } } } import { Promises } from "../common/SetupCellEnvironment.js" import { pack, unpack } from "./MsgPack.js" import "./Polyfill.js" import { Stack } from "./Stack.js" import { with_query_params } from "./URLTools.js" const reconnect_after_close_delay = 500 const retry_after_connect_failure_delay = 5000 /** * Return a promise that resolves to: * - the resolved value of `promise` * - an error after `time_ms` milliseconds * whichever comes first. * @template T * @param {Promise<T>} promise * @param {number} time_ms * @returns {Promise<T>} */ export const timeout_promise = (promise, time_ms) => Promise.race([ promise, new Promise((resolve, reject) => { setTimeout(() => { reject(new Error("Promise timed out.")) }, time_ms) }), ]) /** * Keep calling @see f until it resolves, with a delay before each try. * @param {Function} f Function that returns a promise * @param {Number} time_ms Timeout for each call to @see f */ const retry_until_resolved = (f, time_ms) => timeout_promise(f(), time_ms).catch((e) => { console.error(e) console.error("godverdomme") return retry_until_resolved(f, time_ms) }) /** * @template T * @returns {{current: Promise<T>, resolve: (value: T) => void, reject: (error: any) => void }} */ export const resolvable_promise = () => { let resolve = () => {} let reject = () => {} const p = new Promise((_resolve, _reject) => { //@ts-ignore resolve = _resolve reject = _reject }) return { current: p, resolve: resolve, reject: reject, } } /** * @returns {string} */ const get_unique_short_id = () => crypto.getRandomValues(new Uint32Array(1))[0].toString(36) const socket_is_alright = (socket) => socket.readyState == WebSocket.OPEN || socket.readyState == WebSocket.CONNECTING const socket_is_alright_with_grace_period = (socket) => new Promise((res) => { if (socket_is_alright(socket)) { res(true) } else { setTimeout(() => { res(socket_is_alright(socket)) }, 1000) } }) const try_close_socket_connection = (/** @type {WebSocket} */ socket) => { socket.onopen = () => { try_close_socket_connection(socket) } socket.onmessage = socket.onclose = socket.onerror = null try { socket.close(1000, "byebye") } catch (ex) {} } /** * Open a 'raw' websocket connection to an API with MessagePack serialization. The method is asynchonous, and resolves to a @see WebsocketConnection when the connection is established. * @typedef {{socket: WebSocket, send: Function}} WebsocketConnection * @param {string} address The WebSocket URL * @param {{on_message: Function, on_socket_close:Function}} callbacks * @param {number} timeout_s Timeout for creating the websocket connection (seconds) * @returns {Promise<WebsocketConnection>} */ const create_ws_connection = (address, { on_message, on_socket_close }, timeout_s = 30) => { return new Promise((resolve, reject) => { const socket = new WebSocket(address) let has_been_open = false const timeout_handle = setTimeout(() => { console.warn("Creating websocket timed out", new Date().toLocaleTimeString()) try_close_socket_connection(socket) reject("Socket timeout") }, timeout_s * 1000) const send_encoded = (message) => { const encoded = pack(message) if (socket.readyState === WebSocket.CLOSED || socket.readyState === WebSocket.CLOSING) throw new Error("Socket is closed") socket.send(encoded) } let last_task = Promise.resolve() socket.onmessage = (event) => { // we read and deserialize the incoming messages asynchronously // they arrive in order (WS guarantees this), i.e. this socket.onmessage event gets fired with the message events in the right order // but some message are read and deserialized much faster than others, because of varying sizes, so _after_ async read & deserialization, messages are no longer guaranteed to be in order // // the solution is a task queue, where each task includes the deserialization and the update handler last_task = last_task.then(async () => { try { const buffer = await event.data.arrayBuffer() const message = unpack(new Uint8Array(buffer)) try { on_message(message) } catch (process_err) { console.error("Failed to process message from websocket", process_err, { message }) // prettier-ignore alert(`Something went wrong! You might need to refresh the page.\n\nPlease open an issue on https://github.com/fonsp/Pluto.jl with this info:\n\nFailed to process update\n${process_err.message}\n\n${JSON.stringify(event)}`) } } catch (unpack_err) { console.error("Failed to unpack message from websocket", unpack_err, { event }) // prettier-ignore alert(`Something went wrong! You might need to refresh the page.\n\nPlease open an issue on https://github.com/fonsp/Pluto.jl with this info:\n\nFailed to unpack message\n${unpack_err}\n\n${JSON.stringify(event)}`) } }) } socket.onerror = async (e) => { console.error(`Socket did an oopsie - ${e.type}`, new Date().toLocaleTimeString(), "was open:", has_been_open, e) if (await socket_is_alright_with_grace_period(socket)) { console.log("The socket somehow recovered from an error?! Onbegrijpelijk") console.log(socket) console.log(socket.readyState) } else { if (has_been_open) { on_socket_close() try_close_socket_connection(socket) } else { reject(e) } } } socket.onclose = async (e) => { console.warn(`Socket did an oopsie - ${e.type}`, new Date().toLocaleTimeString(), "was open:", has_been_open, e) if (has_been_open) { on_socket_close() try_close_socket_connection(socket) } else { reject(e) } } socket.onopen = () => { console.log("Socket opened", new Date().toLocaleTimeString()) clearInterval(timeout_handle) has_been_open = true resolve({ socket: socket, send: send_encoded, }) } console.log("Waiting for socket to open...", new Date().toLocaleTimeString()) }) } let next_tick_promise = () => { return new Promise((resolve) => setTimeout(resolve, 0)) } /** * batched_updates(send) creates a wrapper around the real send, and understands the update_notebook messages. * Whenever those are sent, it will wait for a "tick" (basically the end of the code running now) * and then send all updates from this tick at once. We use this to fix https://github.com/fonsp/Pluto.jl/issues/928 * * I need to put it here so other code, * like running cells, will also wait for the updates to complete. * I SHALL MAKE IT MORE COMPLEX! (https://www.youtube.com/watch?v=aO3JgPUJ6iQ&t=195s) * @param {import("./PlutoConnectionSendFn").SendFn} send * @returns {import("./PlutoConnectionSendFn").SendFn} */ const batched_updates = (send) => { let current_combined_updates_promise = null let current_combined_updates = [] let current_combined_notebook_id = null let batched = async (message_type, body, metadata, no_broadcast) => { if (message_type === "update_notebook") { if (current_combined_notebook_id != null && current_combined_notebook_id != metadata.notebook_id) { // prettier-ignore throw new Error("Switched notebook inbetween same-tick updates??? WHAT?!?!") } current_combined_updates = [...current_combined_updates, ...body.updates] current_combined_notebook_id = metadata.notebook_id if (current_combined_updates_promise == null) { current_combined_updates_promise = next_tick_promise().then(async () => { let sending_current_combined_updates = current_combined_updates current_combined_updates_promise = null current_combined_updates = [] current_combined_notebook_id = null return await send(message_type, { updates: sending_current_combined_updates }, metadata, no_broadcast) }) } return await current_combined_updates_promise } else { return await send(message_type, body, metadata, no_broadcast) } } return batched } export const ws_address_from_base = (/** @type {string | URL} */ base_url) => { const ws_url = new URL("./", base_url) ws_url.protocol = ws_url.protocol.replace("http", "ws") // if the original URL had a secret in the URL, we can also add it here: const ws_url_with_secret = with_query_params(ws_url, { secret: new URL(base_url).searchParams.get("secret") }) return ws_url_with_secret } const default_ws_address = () => ws_address_from_base(window.location.href) /** * @typedef PlutoConnection * @type {{ * session_options: Record<string,any>, * send: import("./PlutoConnectionSendFn").SendFn, * kill: () => void, * version_info: { * julia: string, * pluto: string, * dismiss_update_notification: boolean, * }, * notebook_exists: boolean, * message_log: import("./Stack.js").Stack<any>, * }} */ /** * @typedef PlutoMessage * @type {any} */ /** * Open a connection with Pluto, that supports a question-response mechanism. The method is asynchonous, and resolves to a @see PlutoConnection when the connection is established. * * The server can also send messages to all clients, without being requested by them. These end up in the @see on_unrequested_update callback. * * @param {{ * on_unrequested_update: (message: PlutoMessage, by_me: boolean) => void, * on_reconnect: () => Promise<boolean>, * on_connection_status: (connection_status: boolean, hopeless: boolean) => void, * connect_metadata?: Object, * ws_address?: String, * }} options * @return {Promise<PlutoConnection>} */ export const create_pluto_connection = async ({ on_unrequested_update, on_reconnect, on_connection_status, connect_metadata = {}, ws_address = default_ws_address(), }) => { let ws_connection = /** @type {WebsocketConnection?} */ (null) // will be defined later i promise const message_log = new Stack(100) // @ts-ignore window.pluto_get_message_log = () => message_log.get() const client = { // send: null, // session_options: null, version_info: { julia: "unknown", pluto: "unknown", dismiss_update_notification: false, }, notebook_exists: true, // kill: null, message_log, } // same const client_id = get_unique_short_id() const sent_requests = new Map() /** @type {import("./PlutoConnectionSendFn").SendFn} */ const send = async (message_type, body = {}, metadata = {}, no_broadcast = true) => { if (ws_connection == null) { throw new Error("No connection established yet") } const request_id = get_unique_short_id() // This data will be sent: const message = { type: message_type, client_id: client_id, request_id: request_id, body: body, ...metadata, } let p = resolvable_promise() sent_requests.set(request_id, (response_message) => { p.resolve(response_message) if (no_broadcast === false) { on_unrequested_update(response_message, true) } }) ws_connection.send(message) return await p.current } client.send = batched_updates(send) const connect = async () => { let update_url_with_binder_token = async () => { try { const on_a_binder_server = window.location.href.includes("binder") if (!on_a_binder_server) return const url = new URL(window.location.href) const response = await fetch("possible_binder_token_please") if (!response.ok) { return } const possible_binder_token = await response.text() if (possible_binder_token !== "" && url.searchParams.get("token") !== possible_binder_token) { url.searchParams.set("token", possible_binder_token) history.replaceState({}, "", url.toString()) } } catch (error) { console.warn("Error while setting binder url:", error) } } update_url_with_binder_token() try { ws_connection = await create_ws_connection(String(ws_address), { on_message: (update) => { message_log.push(update) const by_me = update.initiator_id == client_id const request_id = update.request_id if (by_me && request_id) { const request = sent_requests.get(request_id) if (request) { request(update) sent_requests.delete(request_id) return } } on_unrequested_update(update, by_me) }, on_socket_close: async () => { on_connection_status(false, false) console.log(`Starting new websocket`, new Date().toLocaleTimeString()) await Promises.delay(reconnect_after_close_delay) await connect() // reconnect! console.log(`Starting state sync`, new Date().toLocaleTimeString()) const accept = await on_reconnect() console.log(`State sync ${accept ? "" : "not "}successful`, new Date().toLocaleTimeString()) on_connection_status(accept, false) if (!accept) { alert("Connection out of sync \n\nRefresh the page to continue") } }, }) // let's say hello console.log("Hello?") const u = await send("connect", {}, connect_metadata) console.log("Hello!") client.kill = () => { if (ws_connection) ws_connection.socket.close() } client.session_options = u.message.session_options client.version_info = u.message.version_info client.notebook_exists = u.message.notebook_exists console.log("Client object: ", client) if (connect_metadata.notebook_id != null && !u.message.notebook_exists) { on_connection_status(false, true) return {} } on_connection_status(true, false) const ping = () => { send("ping", {}, {}) .then(() => { // Ping faster than timeout? setTimeout(ping, 28 * 1000) }) .catch(() => undefined) } ping() return u.message } catch (ex) { console.error("connect() failed", ex) await Promises.delay(retry_after_connect_failure_delay) return await connect() } } await connect() return /** @type {PlutoConnection} */ (client) } /** Send a message to the Pluto backend, and return a promise that resolves when the backend sends a response. Not all messages receive a response. */ export function SendFn ( message_type: string, body?: Object, metadata?: {notebook_id?: string, cell_id?: string}, /** * If false, then the server response will go through the `on_unrequested_update` callback of this client. (This is useful for requests that will cause a state change, like "Run cell". We want the response (state patch) to be handled by the `on_unrequested_update` logic.) * * If true (default), then the response will *only* go to you. This is useful for isolated requests, like "Please autocomplete `draw_rectang`". * @default true */ skip_onupdate_callback?: boolean, ): Promise<Record<string,any>> import { createContext } from "../imports/Preact.js" export let PlutoActionsContext = createContext() // export let PlutoActionsContext = createContext(/** @type {Record<string,Function?>?} */ (null)) export let PlutoBondsContext = createContext(/** @type {import("../components/Editor.js").BondValuesDict?} */ (null)) export let PlutoJSInitializingContext = createContext(/** @type {SetWithEmptyCallback<HTMLElement>?} */ (null)) // Hey copilot, create a class like the built-in `Set` class, but with a callback that is fired when the set becomes empty. /** * A class like the built-in `Set` class, but with a callback that is fired when the set becomes empty. * @template T * @extends {Set<T>} */ export class SetWithEmptyCallback extends Set { /** * @param {() => void} callback */ constructor(callback) { super() this.callback = callback } /** * @param {T} value */ delete(value) { let result = super.delete(value) if (result && this.size === 0) { this.callback() } return result } } // THANKS COPILOT  //@ts-ignore import { sha256 } from "https://cdn.jsdelivr.net/gh/JuliaPluto/js-sha256@v0.9.0-es6/src/sha256.mjs" export const base64_arraybuffer = async (/** @type {BufferSource} */ data) => { /** @type {string} */ const base64url = await new Promise((r) => { const reader = new FileReader() // @ts-ignore reader.onload = () => r(reader.result) reader.readAsDataURL(new Blob([data])) }) return base64url.substring(base64url.indexOf(',')+1) } /** Encode a buffer using the `base64url` encoding, which uses URL-safe special characters, see https://en.wikipedia.org/wiki/Base64#Variants_summary_table */ export const base64url_arraybuffer = async (/** @type {BufferSource} */ data) => { // This is roughly 0.5 as fast as `base64_arraybuffer`. See https://gist.github.com/fonsp/d2b84265012942dc40d0082b1fd405ba for benchmark and even slower alternatives. let original = await base64_arraybuffer(data) return base64_to_base64url(original) } /** Turn a base64-encoded string into a base64url-encoded string containing the same data. Do not apply on a `data://` URL. */ export const base64_to_base64url = (/** @type {string} */ original) => { return original.replaceAll(/[\+\/\=]/g, (s) => { const c = s.charCodeAt(0) return c === 43 ? "-" : c === 47 ? "_" : "" }) } /** Turn a base64url-encoded string into a base64-encoded string containing the same data. Do not apply on a `data://` URL. */ export const base64url_to_base64 = (/** @type {string} */ original) => { const result_before_padding = original.replaceAll(/[-_]/g, (s) => { const c = s.charCodeAt(0) return c === 45 ? "+" : c === 95 ? "/" : "" }) return result_before_padding + "=".repeat((4 - (result_before_padding.length % 4)) % 4) } const t1 = "AAA/AAA+ZMg=" const t2 = "AAA_AAA-ZMg" console.assert(base64_to_base64url(t1) === t2) console.assert(base64url_to_base64(t2) === t1) base64_arraybuffer(new Uint8Array([0, 0, 63, 0, 0, 62, 100, 200])).then((r) => console.assert(r === t1, r)) base64url_arraybuffer(new Uint8Array([0, 0, 63, 0, 0, 62, 100, 200])).then((r) => console.assert(r === t2, r)) export const plutohash_arraybuffer = async (/** @type {BufferSource} */ data) => { const hash = sha256.create() hash.update(data) const hashed_buffer = hash.arrayBuffer() // const hashed_buffer = await window.crypto.subtle.digest("SHA-256", data) return await base64url_arraybuffer(hashed_buffer) } export const plutohash_str = async (/** @type {string} */ s) => { const data = new TextEncoder().encode(s) return await plutohash_arraybuffer(data) } // hash_str("Hannes").then((r) => console.assert(r === "OI48wVWerxEEnz5lIj6CPPRB8NOwwba+LkFYTDp4aUU=", r)) plutohash_str("Hannes").then((r) => console.assert(r === "OI48wVWerxEEnz5lIj6CPPRB8NOwwba-LkFYTDp4aUU", r)) export const debounced_promises = (async_function) => { let currently_running = false let rerun_when_done = false return async () => { if (currently_running) { rerun_when_done = true } else { currently_running = true rerun_when_done = true while (rerun_when_done) { rerun_when_done = false await async_function() } currently_running = false } } } /** @returns {Promise<string>} */ export const blob_url_to_data_url = async (/** @type {string} */ blob_url) => { const blob = await (await fetch(blob_url)).blob() return await new Promise((r) => { const reader = new FileReader() // @ts-ignore reader.onload = () => r(reader.result) reader.readAsDataURL(blob) }) } // Polyfill for Blob::text when there is none (safari) // This is not "officialy" supported if (Blob.prototype.text == null) { Blob.prototype.text = function () { const reader = new FileReader() const promise = new Promise((resolve, reject) => { // on read success reader.onload = () => { resolve(reader.result) } // on failure reader.onerror = (e) => { reader.abort() reject(e) } }) reader.readAsText(this) return promise } } // Polyfill for Blob::arrayBuffer when there is none (safari) // This is not "officialy" supported if (Blob.prototype.arrayBuffer == null) { Blob.prototype.arrayBuffer = function () { return new Response(this).arrayBuffer() } } //@ts-ignore import { polyfill as scroll_polyfill } from "https://esm.sh/seamless-scroll-polyfill@2.1.8/lib/polyfill.js?pin=v113&target=es2020" scroll_polyfill() export const ProcessStatus = { ready: "ready", starting: "starting", no_process: "no_process", waiting_to_restart: "waiting_to_restart", waiting_for_permission: "waiting_for_permission", } import immer from "../imports/immer.js" import { BackendLaunchPhase } from "./Binder.js" import { timeout_promise } from "./PlutoConnection.js" import { with_query_params } from "./URLTools.js" // This file is very similar to `start_binder` in Binder.js /** * * @param {{ * launch_params: import("../components/Editor.js").LaunchParameters, * setStatePromise: any, * connect: () => Promise<void>, * }} props */ export const start_local = async ({ setStatePromise, connect, launch_params }) => { try { if (launch_params.pluto_server_url == null || launch_params.notebookfile == null) throw Error("Invalid launch parameters for starting locally.") await setStatePromise( immer((/** @type {import("../components/Editor.js").EditorState} */ state) => { state.backend_launch_phase = BackendLaunchPhase.responded state.disable_ui = false // Clear the Status of the process that generated the HTML state.notebook.status_tree = null }) ) const with_token = (x) => String(x) const binder_session_url = new URL(launch_params.pluto_server_url, window.location.href) let open_response // We download the notebook file contents, and then upload them to the Pluto server. const notebook_contents = await ( await fetch(new Request(launch_params.notebookfile, { integrity: launch_params.notebookfile_integrity ?? undefined })) ).arrayBuffer() open_response = await fetch( with_token( with_query_params(new URL("notebookupload", binder_session_url), { name: new URLSearchParams(window.location.search).get("name"), clear_frontmatter: "yesplease", execution_allowed: "yepperz", }) ), { method: "POST", body: notebook_contents, } ) if (!open_response.ok) { let b = await open_response.blob() window.location.href = URL.createObjectURL(b) return } const new_notebook_id = await open_response.text() const edit_url = with_query_params(new URL("edit", binder_session_url), { id: new_notebook_id }) console.info("notebook_id:", new_notebook_id) window.history.replaceState({}, "", edit_url) await setStatePromise( immer((/** @type {import("../components/Editor.js").EditorState} */ state) => { state.notebook.notebook_id = new_notebook_id state.backend_launch_phase = BackendLaunchPhase.notebook_running }) ) console.log("Connecting WebSocket") const connect_promise = connect() await timeout_promise(connect_promise, 20_000).catch((e) => { console.error("Failed to establish connection within 20 seconds. Navigating to the edit URL directly.", e) window.parent.location.href = with_token(edit_url) }) } catch (err) { console.error("Failed to initialize binder!", err) alert("Something went wrong! \n\nWe failed to open this notebook. Please try again with a different browser, or come back later.") } } /** * Serialize an array of cells into a string form (similar to the .jl file). * * Used for implementing clipboard functionality. This isn't in topological * order, so you won't necessarily be able to run it directly. * * @param {Array<import("../components/Editor.js").CellInputData>} cells * @return {String} */ export function serialize_cells(cells) { return cells.map((cell) => `#  ${cell.cell_id}\n` + cell.code + "\n").join("\n") } /** * Deserialize a Julia program or output from `serialize_cells`. * * If a Julia program, it will return a single String containing it. Otherwise, * it will split the string into cells based on the special delimiter. * * @param {String} serialized_cells * @return {Array<String>} */ export function deserialize_cells(serialized_cells) { const segments = serialized_cells.replace(/\r\n/g, "\n").split(/#  \S+\n/) return segments.map((s) => s.trim()).filter((s) => s !== "") } const JULIA_REPL_PROMPT = "julia> " /** * Deserialize a Julia REPL session. * * It will split the string into cells based on the Julia prompt. Multiple * lines are detected based on indentation. * * @param {String} repl_session * @return {Array<String>} */ export function deserialize_repl(repl_session) { const segments = repl_session.replace(/\r\n/g, "\n").split(JULIA_REPL_PROMPT) const indent = " ".repeat(prompt.length) return segments .map(function (s) { return (indent + s) .split("\n") .filter((line) => line.startsWith(indent)) .map((s) => s.replace(indent, "")) .join("\n") }) .map((s) => s.trim()) .filter((s) => s !== "") } export const detect_deserializer = (/** @type {string} */ topaste) => topaste.trim().startsWith(JULIA_REPL_PROMPT) ? deserialize_repl : topaste.match(/#  ........-....-....-....-............/g)?.length ? deserialize_cells : null // @ts-ignore import { Library } from "https://cdn.jsdelivr.net/npm/@observablehq/stdlib@3.3.1/+esm" export const make_library = () => { // @ts-ignore const library = new Library() return { DOM: library.DOM, Files: library.Files, Generators: library.Generators, Promises: library.Promises, now: library.now, svg: library.svg(), html: library.html(), require: library.require(), } // TODO // observablehq.md and observablehq.tex will call d3-require, which will create a conflict if something else is using d3-require // in particular, plotly.js will break // observablehq.md(observablehq.require()).then((md) => (observablehq_exports.md = md)) // observablehq.tex().then(tex => observablehq_exports.tex = tex) } // We use two different observable stdlib instances: one for ourselves and one for the JS code in cell outputs const observablehq_for_myself = make_library() export const observablehq_for_cells = make_library() export { observablehq_for_myself as default } export const DOM = observablehq_for_myself.DOM export const Files = observablehq_for_myself.Files export const Generators = observablehq_for_myself.Generators export const Promises = observablehq_for_myself.Promises export const now = observablehq_for_myself.now export const svg = observablehq_for_myself.svg export const html = observablehq_for_myself.html export const require = observablehq_for_myself.require import "https://cdn.jsdelivr.net/npm/requestidlecallback-polyfill@1.0.2/index.js" import { get_included_external_source } from "./external_source.js" let setup_done = false export const setup_mathjax = () => { if (setup_done) { return } setup_done = true const deprecated = () => console.error( "Pluto.jl: Pluto loads MathJax 3 globally, but a MathJax 2 function was called. The two version can not be used together on the same web page." ) const twowasloaded = () => console.error( "Pluto.jl: MathJax 2 is already loaded in this page, but Pluto wants to load MathJax 3. Packages that import MathJax 2 in their html display will break Pluto's ability to render latex." ) // @ts-ignore window.MathJax = { options: { ignoreHtmlClass: "no-MJax", processHtmlClass: "tex", }, startup: { typeset: true, // because we load MathJax asynchronously ready: () => { // @ts-ignore window.MathJax.startup.defaultReady() // plotly uses MathJax 2, so we have this shim to make it work kindof // @ts-ignore window.MathJax.Hub = { Queue: function () { for (var i = 0, m = arguments.length; i < m; i++) { // @ts-ignore var fn = window.MathJax.Callback(arguments[i]) // @ts-ignore window.MathJax.startup.promise = window.MathJax.startup.promise.then(fn) } // @ts-ignore return window.MathJax.startup.promise }, Typeset: function (elements, callback) { // @ts-ignore var promise = window.MathJax.typesetPromise(elements) if (callback) { promise = promise.then(callback) } return promise }, Register: { MessageHook: deprecated, StartupHook: deprecated, LoadHook: deprecated, }, Config: deprecated, Configured: deprecated, setRenderer: deprecated, } }, }, tex: { inlineMath: [ ["$", "$"], ["\\(", "\\)"], ], }, svg: { fontCache: "global", }, } requestIdleCallback( () => { console.log("Loading mathjax!!") const src = get_included_external_source("MathJax-script") if (!src) throw new Error("Could not find mathjax source") const script = document.createElement("script") script.addEventListener("load", () => { console.log("MathJax loaded!") if (window["MathJax"]?.version !== "3.2.2") { twowasloaded() } }) script.crossOrigin = src.crossOrigin script.integrity = src.integrity script.src = src.href document.head.append(script) }, { timeout: 2000 } ) } import { trailingslash } from "./Binder.js" import { plutohash_arraybuffer, debounced_promises, base64url_arraybuffer } from "./PlutoHash.js" import { pack, unpack } from "./MsgPack.js" import immer from "../imports/immer.js" import _ from "../imports/lodash.js" const assert_response_ok = (/** @type {Response} */ r) => (r.ok ? r : Promise.reject(r)) const actions_to_keep = ["get_published_object", "get_notebook"] const where_referenced = (/** @type {import("../components/Editor.js").CellDependencyGraph} */ graph, /** @type {Set<string> | string[]} */ vars) => { const all_cells = Object.keys(graph) return all_cells.filter((cell_id) => _.some([...vars], (v) => Object.keys(graph[cell_id].upstream_cells_map).includes(v))) } const where_assigned = (/** @type {import("../components/Editor.js").CellDependencyGraph} */ graph, /** @type {Set<string> | string[]} */ vars) => { const all_cells = Object.keys(graph) return all_cells.filter((cell_id) => _.some([...vars], (v) => Object.keys(graph[cell_id].downstream_cells_map).includes(v))) } export const downstream_recursive = (/** @type {import("../components/Editor.js").CellDependencyGraph} */ graph, starts, { recursive = true } = {}) => { /** @type {Set<string>} */ const deps = new Set() const ends = [...starts] while (ends.length > 0) { const node = ends.splice(0, 1)[0] _.flatten(Object.values(graph[node].downstream_cells_map)).forEach((next_cellid) => { if (!deps.has(next_cellid)) { if (recursive) ends.push(next_cellid) deps.add(next_cellid) } }) } return deps } export const upstream_recursive = (/** @type {import("../components/Editor.js").CellDependencyGraph} */ graph, starts, { recursive = true } = {}) => { /** @type {Set<string>} */ const deps = new Set() const ends = [...starts] while (ends.length > 0) { const node = ends.splice(0, 1)[0] _.flatten(Object.values(graph[node].upstream_cells_map)).forEach((next_cellid) => { if (!deps.has(next_cellid)) { if (recursive) ends.push(next_cellid) deps.add(next_cellid) } }) } return deps } const disjoint = (a, b) => !_.some([...a], (x) => b.has(x)) export const is_noop_action = (action) => action?.__is_noop_action === true const create_noop_action = (name) => { const fn = (...args) => { console.info("Ignoring action", name, { args }) } fn.__is_noop_action = true return fn } export const nothing_actions = ({ actions }) => Object.fromEntries( Object.entries(actions).map(([k, v]) => [ k, actions_to_keep.includes(k) ? // the original action v : // a no-op action create_noop_action(k), ]) ) export const slider_server_actions = ({ setStatePromise, launch_params, actions, get_original_state, get_current_state, apply_notebook_patches }) => { setStatePromise( immer((state) => { state.slider_server.connecting = true }) ) const notebookfile_hash = fetch(new Request(launch_params.notebookfile, { integrity: launch_params.notebookfile_integrity })) .then(assert_response_ok) .then((r) => r.arrayBuffer()) .then(plutohash_arraybuffer) notebookfile_hash.then((x) => console.log("Notebook file hash:", x)) const bond_connections = notebookfile_hash .then((hash) => fetch(trailingslash(launch_params.slider_server_url) + "bondconnections/" + hash)) .then(assert_response_ok) .then((r) => r.arrayBuffer()) .then((b) => unpack(new Uint8Array(b))) bond_connections.then((x) => { console.log("Bond connections:", x) setStatePromise( immer((state) => { state.slider_server.connecting = false state.slider_server.interactive = Object.keys(x).length > 0 }) ) }) bond_connections.catch((x) => { setStatePromise( immer((state) => { state.slider_server.connecting = false state.slider_server.interactive = false }) ) }) const mybonds = {} const bonds_to_set = { current: new Set(), } const request_bond_response = debounced_promises(async () => { const base = trailingslash(launch_params.slider_server_url) const hash = await notebookfile_hash const graph = await bond_connections const explicit_bond_names = bonds_to_set.current bonds_to_set.current = new Set() /// // PART 1: Compute dependencies /// const dep_graph = get_current_state().cell_dependencies /** Cells that define an explicit bond */ const starts = where_assigned(dep_graph, explicit_bond_names) const first_layer = where_referenced(dep_graph, explicit_bond_names) const next_layers = [...downstream_recursive(dep_graph, first_layer)] const cells_depending_on_explicits = _.uniq([...first_layer, ...next_layers]) const to_send = new Set(explicit_bond_names) explicit_bond_names.forEach((varname) => (graph[varname] ?? []).forEach((x) => to_send.add(x))) // Take only the bonds we need, and sort the based on variable name const mybonds_filtered = Object.fromEntries( _.sortBy( Object.entries(mybonds).filter(([k, v]) => to_send.has(k)), ([k, v]) => k ) ) const need_to_send_explicits = (() => { const _to_send_starts = where_assigned(dep_graph, to_send) const _depends_on_to_send = downstream_recursive(dep_graph, _to_send_starts) return !disjoint(_to_send_starts, _depends_on_to_send) })() /// // PART: Update visual cell running status /// const update_cells_running = async (running) => await setStatePromise( immer((state) => { cells_depending_on_explicits.forEach((cell_id) => (state.notebook.cell_results[cell_id]["queued"] = running)) starts.forEach((cell_id) => (state.notebook.cell_results[cell_id]["running"] = running)) }) ) await update_cells_running(true) /// // PART: Make the request to PSS /// if (explicit_bond_names.size > 0) { console.debug("Requesting bonds", { explicit_bond_names, to_send, mybonds_filtered, need_to_send_explicits }) const packed = pack(mybonds_filtered) const packed_explicits = pack(Array.from(explicit_bond_names)) let unpacked = null try { const url = base + "staterequest/" + hash + "/" // https://github.com/fonsp/Pluto.jl/pull/3158 let add_explicits = async (url) => { let u = new URL(url, window.location.href) // We can skip this if all bonds are explicit: if (need_to_send_explicits) if (!_.isEqual(explicit_bond_names, to_send)) u.searchParams.set("explicit", await base64url_arraybuffer(packed_explicits)) return u } const force_post = get_current_state().metadata["sliderserver_force_post"] ?? false const use_get = !force_post && url.length + (packed.length * 4 + packed_explicits.length * 4) / 3 + 20 + 12 < 8000 const response = use_get ? await fetch(await add_explicits(url + (await base64url_arraybuffer(packed))), { method: "GET", }).then(assert_response_ok) : await fetch(await add_explicits(url), { method: "POST", body: packed, }).then(assert_response_ok) unpacked = unpack(new Uint8Array(await response.arrayBuffer())) console.debug("Received state", unpacked) const { patches } = unpacked await apply_notebook_patches( patches, // We can just apply the patches as-is, but for complete correctness we have to take into account that these patches are not generated: // NOT: diff(current_state_of_this_browser, what_it_should_be) // but // YES: diff(original_statefile_state, what_it_should_be) // // And because of previous bond interactions, our current state will have drifted from the original_statefile_state. // // Luckily immer lets us deal with this perfectly by letting us provide a custom "old" state. // For the old state, we will use: // the current state of this browser (we dont want to change too much) // but all cells that will be affected by this run: // the statefile state // // Crazy!! immer((state) => { const original = get_original_state() cells_depending_on_explicits.forEach((id) => { state.cell_results[id] = original.cell_results[id] }) })(get_current_state()) ) } catch (e) { console.error(unpacked, e) } finally { // reset cell running state regardless of request outcome await update_cells_running(false) } } }) return { ...nothing_actions({ actions }), set_bond: async (symbol, value) => { setStatePromise( immer((state) => { state.notebook.bonds[symbol] = { value: value } }) ) if (mybonds[symbol] == null || !_.isEqual(mybonds[symbol].value, value)) { mybonds[symbol] = { value: _.cloneDeep(value) } bonds_to_set.current.add(symbol) await request_bond_response() } }, } } /** * @template T * @type {Stack<T>} */ export class Stack { /** * @param {number} max_size */ constructor(max_size) { this.max_size = max_size this.arr = [] } /** * @param {T} item * @returns {void} */ push(item) { this.arr.push(item) if (this.arr.length > this.max_size) { this.arr.shift() } } /** * @returns {T[]} */ get() { return this.arr } } export const with_query_params = (/** @type {String | URL} */ url_str, /** @type {Record<string,string | null | undefined>} */ params) => { const fake_base = "http://delete-me.com/" const url = new URL(url_str, fake_base) Object.entries(params).forEach(([key, val]) => { if (val != null) url.searchParams.append(key, val) }) return url.toString().replace(fake_base, "") } console.assert(with_query_params("https://example.com/", { a: "b c" }) === "https://example.com/?a=b+c") console.assert(with_query_params(new URL("https://example.com/"), { a: "b c" }) === "https://example.com/?a=b+c") console.assert(with_query_params(new URL("https://example.com/"), { a: "b c", asdf: null, xx: "123" }) === "https://example.com/?a=b+c&xx=123") console.assert(with_query_params("index.html", { a: "b c" }) === "index.html?a=b+c") console.assert(with_query_params("index.html?x=123", { a: "b c" }) === "index.html?x=123&a=b+c") console.assert(with_query_params("index.html?x=123#asdf", { a: "b c" }) === "index.html?x=123&a=b+c#asdf") const te = new TextEncoder() const td = new TextDecoder() export const length_utf8 = (str, startindex_utf16 = 0, endindex_utf16 = undefined) => te.encode(str.substring(startindex_utf16, endindex_utf16)).length export const utf8index_to_ut16index = (str, index_utf8) => td.decode(te.encode(str).slice(0, index_utf8)).length export const splice_utf8 = (original, startindex_utf8, endindex_utf8, replacement) => { // JS uses UTF-16 for internal representation of strings, e.g. // "e".length == 1, "".length == 1, "".length == 2 // Julia uses UTF-8, e.g. // ncodeunits("e") == 1, ncodeunits("") == 2, ncodeunits("") == 4 // length("e") == 1, length("") == 1, length("") == 1 // Completion results from julia will give the 'splice indices': "where should the completed keyword be inserted?" // we need to splice into javascript string, so we convert to a UTF-8 byte array, then splice, then back to the string. const original_enc = te.encode(original) const replacement_enc = te.encode(replacement) const result_enc = new Uint8Array(original_enc.length + replacement_enc.length - (endindex_utf8 - startindex_utf8)) result_enc.set(original_enc.slice(0, startindex_utf8), 0) result_enc.set(replacement_enc, startindex_utf8) result_enc.set(original_enc.slice(endindex_utf8), startindex_utf8 + replacement_enc.length) return td.decode(result_enc) } export const slice_utf8 = (original, startindex_utf8, endindex_utf8) => { // JS uses UTF-16 for internal representation of strings, e.g. // "e".length == 1, "".length == 1, "".length == 2 // Julia uses UTF-8, e.g. // ncodeunits("e") == 1, ncodeunits("") == 2, ncodeunits("") == 4 // length("e") == 1, length("") == 1, length("") == 1 const original_enc = te.encode(original) return td.decode(original_enc.slice(startindex_utf8, endindex_utf8)) } console.assert(splice_utf8("e   is a dog", 5, 9, "hannes ") === "e  hannes  is a dog") console.assert(slice_utf8("e   is a dog", 5, 9) === "") import { useContext, useState, useEffect } from "../imports/Preact.js" import { PlutoActionsContext } from "./PlutoContext.js" /** Request the current time from the server, compare with the local time, and return the current best estimate of our time difference. Updates regularly. * @param {{connected: boolean}} props */ export const useMyClockIsAheadBy = ({ connected }) => { let pluto_actions = useContext(PlutoActionsContext) const [my_clock_is_ahead_by, set_my_clock_is_ahead_by] = useState(0) useEffect(() => { if (connected) { let f = async () => { let getserver = () => pluto_actions.send("current_time").then((m) => m.message.time) let getlocal = () => Date.now() / 1000 // once to precompile and to make sure that the server is not busy with other tasks // console.log("getting server time warmup") for (let i = 0; i < 16; i++) await getserver() // console.log("getting server time warmup done") let t1 = await getlocal() let s1 = await getserver() let s2 = await getserver() let t2 = await getlocal() // console.log("getting server time done") let mytime = (t1 + t2) / 2 let servertime = (s1 + s2) / 2 let diff = mytime - servertime // console.info("My clock is ahead by ", diff, " s") if (!isNaN(diff)) set_my_clock_is_ahead_by(diff) } f() let handle = setInterval(f, 60 * 1000) return () => clearInterval(handle) } }, [connected]) return my_clock_is_ahead_by } /** * Get a `<link rel="pluto-external-source">` element from editor.html. * @param {String} id * @returns {HTMLLinkElement?} */ export const get_included_external_source = (id) => document.head.querySelector(`link[rel='pluto-external-source'][id='${id}']`) export const open_pluto_popup = (/** @type{import("../components/Popup").PkgPopupDetails | import("../components/Popup").MiscPopupDetails} */ detail) => { window.dispatchEvent( new CustomEvent("open pluto popup", { detail, }) ) } /** * * @return {import("../components/Editor.js").LaunchParameters} */ export const parse_launch_params = () => { const url_params = new URLSearchParams(window.location.search) return { //@ts-ignore notebook_id: url_params.get("id") ?? window.pluto_notebook_id, //@ts-ignore statefile: url_params.get("statefile") ?? window.pluto_statefile, //@ts-ignore statefile_integrity: url_params.get("statefile_integrity") ?? window.pluto_statefile_integrity, //@ts-ignore notebookfile: url_params.get("notebookfile") ?? window.pluto_notebookfile, //@ts-ignore notebookfile_integrity: url_params.get("notebookfile_integrity") ?? window.pluto_notebookfile_integrity, //@ts-ignore disable_ui: !!(url_params.get("disable_ui") ?? window.pluto_disable_ui), //@ts-ignore preamble_html: url_params.get("preamble_html") ?? window.pluto_preamble_html, //@ts-ignore isolated_cell_ids: url_params.has("isolated_cell_id") ? url_params.getAll("isolated_cell_id") : window.pluto_isolated_cell_ids, //@ts-ignore binder_url: url_params.get("binder_url") ?? window.pluto_binder_url, //@ts-ignore pluto_server_url: url_params.get("pluto_server_url") ?? window.pluto_pluto_server_url, //@ts-ignore slider_server_url: url_params.get("slider_server_url") ?? window.pluto_slider_server_url, //@ts-ignore recording_url: url_params.get("recording_url") ?? window.pluto_recording_url, //@ts-ignore recording_url_integrity: url_params.get("recording_url_integrity") ?? window.pluto_recording_url_integrity, //@ts-ignore recording_audio_url: url_params.get("recording_audio_url") ?? window.pluto_recording_audio_url, } } //@ts-ignore import dialogPolyfill from "https://cdn.jsdelivr.net/npm/dialog-polyfill@0.5.6/dist/dialog-polyfill.esm.min.js" import { useLayoutEffect, useMemo, useRef } from "../imports/Preact.js" /** * @returns {[import("../imports/Preact.js").Ref<HTMLDialogElement?>, () => void, () => void, () => void]} */ export const useDialog = () => { const dialog_ref = useRef(/** @type {HTMLDialogElement?} */ (null)) useLayoutEffect(() => { if (dialog_ref.current != null && typeof HTMLDialogElement !== "function") dialogPolyfill.registerDialog(dialog_ref.current) }, [dialog_ref.current]) return useMemo(() => { const open = () => { if (!dialog_ref.current?.open) dialog_ref.current?.showModal() } const close = () => { if (dialog_ref.current?.open === true) dialog_ref.current?.close?.() } const toggle = () => (dialog_ref.current?.open === true ? dialog_ref.current?.close?.() : dialog_ref.current?.showModal?.()) return [dialog_ref, open, close, toggle] }, [dialog_ref]) } import { useCallback, useEffect } from "../imports/Preact.js" /** * @typedef EventListenerAddable * @type Document | HTMLElement | Window | EventSource | MediaQueryList | null */ export const useEventListener = ( /** @type {EventListenerAddable | import("../imports/Preact.js").Ref<EventListenerAddable>} */ element, /** @type {string} */ event_name, /** @type {EventListenerOrEventListenerObject} */ handler, /** @type {any[] | undefined} */ deps ) => { let handler_cached = useCallback(handler, deps) useEffect(() => { const e = element const useme = e == null || e instanceof Document || e instanceof HTMLElement || e instanceof Window || e instanceof EventSource || e instanceof MediaQueryList ? /** @type {EventListenerAddable} */ (e) : e.current if (useme == null) return useme.addEventListener(event_name, handler_cached) return () => useme.removeEventListener(event_name, handler_cached) }, [element, event_name, handler_cached]) } import _ from "../imports/lodash.js" import { html, useState, useRef, useContext } from "../imports/Preact.js" import { cl } from "../common/ClassTable.js" import { PlutoActionsContext } from "../common/PlutoContext.js" import { useEventListener } from "../common/useEventListener.js" import { upstream_recursive } from "../common/SliderServerClient.js" const format_code = (s) => s == null ? "" : `<julia-code-block> ${s} </julia-code-block>` const format_cell_output = (/** @type {import("./Editor.js").CellResultData?} */ cell_result, /** @type {number} */ truncate_limit) => { const text = cell_output_to_plaintext(cell_result, truncate_limit) return text == null ? "" : `<pluto-ai-context-cell-output errored="${cell_result?.errored ?? "false"}"> ${text === "" || text == null ? "nothing" : text} </pluto-ai-context-cell-output>` } const packages_context = (/** @type {import("./Editor.js").NotebookData} */ notebook) => { const has_nbpkg = notebook.nbpkg?.enabled === true const installed = Object.keys(notebook.nbpkg?.installed_versions ?? {}) return !has_nbpkg ? "" : ` <pluto-ai-context-packages> The following packages are currently installed in this notebook: ${installed.join(", ")}. </pluto-ai-context-packages>` } export const AIContext = ({ cell_id, current_code }) => { const pluto_actions = useContext(PlutoActionsContext) const [copied, setCopied] = useState(false) const notebook = /** @type{import("./Editor.js").NotebookData} */ (pluto_actions.get_notebook()) const default_question = notebook.cell_results[cell_id]?.errored === true ? "Why does this cell error?" : "" const [userQuestion, setUserQuestion] = useState(default_question) const recursive = true const prompt_args = { userQuestion, recursive, notebook, cell_id, current_code, } let prompt = generate_prompt(prompt_args) let prompt_tokens = count_openai_tokens(prompt) if (prompt_tokens > 3000) { console.log("Prompt is too long, truncating...", prompt, prompt_tokens) prompt = generate_prompt({ ...prompt_args, recursive: false, truncate_limit_current_cell: 1000, }) prompt_tokens = count_openai_tokens(prompt) } const copyToClipboard = async () => { try { await navigator.clipboard.writeText(prompt) setCopied(true) setTimeout(() => setCopied(false), 2000) } catch (err) { console.error("Failed to copy text:", err) } } const formRef = useRef(null) useEventListener(formRef, "submit", (e) => { e.preventDefault() copyToClipboard() console.log("submitted") }) return html` <form class="ai-context-container" ref=${formRef}> <h2>AI Prompt Generator <em style="font-size: 0.8em; opacity: .7;">(beta)</em></h2> <p class="ai-context-intro">You can copy this text into an AI chat to give it context from your notebook.</p> <input type="text" name="pluto-ai-context-question" class="ai-context-question-input" placeholder="Type your question here..." autocomplete="off" value=${userQuestion} onInput=${(e) => setUserQuestion(e.target.value)} /> <div class="ai-context-prompt-container"> <button class=${cl({ "copy-button": true, "copied": copied, })} type="submit" title="Copy to clipboard" > ${copied ? "Copied!" : "Copy"} </button> <div class=${cl({ "ai-context-prompt": true, "ai-context-prompt-with-question": userQuestion.length > 0, })} > <pre>${prompt.trim()}</pre> </div> </div> </form> ` } /** * @param {{ * userQuestion: string, * recursive: boolean, * notebook: import("./Editor.js").NotebookData, * cell_id: string, * current_code: string, * truncate_limit_current_cell?: number, * }} props * @returns {string} */ const generate_prompt = ({ userQuestion, recursive, notebook, cell_id, current_code, truncate_limit_current_cell = 800 }) => { const current_cell = ` <pluto-ai-context-current-cell> The current cell has the following code: ${format_code(current_code)} ${format_cell_output(notebook.cell_results[cell_id], truncate_limit_current_cell)} </pluto-ai-context-current-cell> ` const graph = notebook.cell_dependencies // Get the list of upstream cells, in the order that they appear in the notebook. const upstream_cellids = upstream_recursive(graph, [cell_id], { recursive }) const upstream_cells = notebook.cell_order.filter((cid) => upstream_cellids.has(cid)) // Get the variables that are used in the current cell, which are defined in other cells. const upstream_direct = notebook.cell_dependencies[cell_id]?.upstream_cells_map ?? {} const variables_used_from_upstream = Object.entries(upstream_direct) .filter(([_var, upstream_cell_ids]) => upstream_cell_ids.length > 0) .map(([variable]) => variable) const variable_context = ` <pluto-ai-context-variables> The current cell uses the following variables from other cells: ${variables_used_from_upstream.join(", ")}. These variables are defined in the following cells: ${upstream_cells.map((cid) => format_code(notebook.cell_inputs[cid].code)).join("\n\n")} </pluto-ai-context-variables> ` const prompt = `${userQuestion} <pluto-ai-context> To help me answer my question, here is some auto-generated context. The code is from a Pluto Julia notebook. We are concerned with one specific cell in the notebook, called "the current cell". And you will get additional context. When suggesting new code, give each cell its own code block, and keep global variables names as they are. ${current_cell} ${upstream_cells.length > 0 ? variable_context : ""} ${packages_context(notebook)} </pluto-ai-context> ` return prompt } const cell_output_to_plaintext = (/** @type {import("./Editor.js").CellResultData?} */ cell_result, /** @type {number} */ truncate_limit) => { if (cell_result == null) return null const cell_output = cell_result.output if (cell_output.mime === "text/plain") { return cell_output.body } if (cell_output.mime === "application/vnd.pluto.stacktrace+object") { return cell_output.body.plain_error } if (cell_output.mime.includes("image")) return "<!-- Image -->" if (cell_output.mime === "text/html") { try { return JSON.stringify(cell_result, (key, value) => { if (typeof value === "string" && value.length > truncate_limit) { return ( value.substring(0, truncate_limit / 2) + `... <!-- ${value.length - truncate_limit / 2 - 20} CHARACTERS TRUNCATED --> ... ` + value.substring(value.length - 20) ) } return value }) } catch (e) { return "<!-- HTML content that couldn't be stringified -->" } } return JSON.stringify(cell_output) } /** Rough heuristic for counting tokens in a string. */ const count_openai_tokens = (text) => { const num_seps = text.match(/[^\p{L}]+/gmu)?.length ?? 0 const val1 = num_seps * 2.3 const val2 = text.length * 0.29 // Average return (val1 + val2) / 2 } import _ from "../imports/lodash.js" import { createSilentAudio } from "../common/AudioRecording.js" import { html, useEffect, useState, useRef, useLayoutEffect } from "../imports/Preact.js" let run = (x) => x() export const AudioPlayer = ({ onPlay, src, loaded_recording, audio_element_ref }) => { useLayoutEffect(() => { run(async () => { if (src == null) { // We create a silent audio track to play. The duration is the last timestamp of the loaded recording state. let last_timestamp = (things) => _.last([[0, null], ...things])[0] let fake_duration = Math.max(last_timestamp((await loaded_recording).scrolls), last_timestamp((await loaded_recording).steps)) fake_duration = Math.ceil(fake_duration + 0.1) console.log({ fake_duration }) let fake_source = createSilentAudio(fake_duration) audio_element_ref.current.src = fake_source } else { audio_element_ref.current.src = src } }) }, []) return html`<div class="recording-playback"><audio ref=${audio_element_ref} onPlay=${onPlay} controls></audio></div>` } import { html, useState, useRef, useEffect, useMemo } from "../imports/Preact.js" import { cl } from "../common/ClassTable.js" import { LiveDocsTab } from "./LiveDocsTab.js" import { is_finished, StatusTab, total_done, total_tasks, useStatusItem } from "./StatusTab.js" import { useMyClockIsAheadBy } from "../common/clock sync.js" import { BackendLaunchPhase } from "../common/Binder.js" import { useEventListener } from "../common/useEventListener.js" /** * @typedef PanelTabName * @type {"docs" | "process" | null} */ export const open_bottom_right_panel = (/** @type {PanelTabName} */ tab) => window.dispatchEvent(new CustomEvent("open_bottom_right_panel", { detail: tab })) /** * @param {{ * notebook: import("./Editor.js").NotebookData, * desired_doc_query: string?, * on_update_doc_query: (query: string?) => void, * connected: boolean, * backend_launch_phase: number?, * backend_launch_logs: string?, * sanitize_html?: boolean, * }} props */ export let BottomRightPanel = ({ desired_doc_query, on_update_doc_query, notebook, connected, backend_launch_phase, backend_launch_logs, sanitize_html = true, }) => { let container_ref = useRef() const focus_docs_on_open_ref = useRef(false) const [open_tab, set_open_tab] = useState(/** @type { PanelTabName} */ (null)) const hidden = open_tab == null // Open panel when "open_bottom_right_panel" event is triggered useEventListener( window, "open_bottom_right_panel", (/** @type {CustomEvent} */ e) => { console.log(e.detail) // https://github.com/fonsp/Pluto.jl/issues/321 focus_docs_on_open_ref.current = false set_open_tab(e.detail) if (window.getComputedStyle(container_ref.current).display === "none") { alert("This browser window is too small to show docs.\n\nMake the window bigger, or try zooming out.") } }, [set_open_tab] ) const status = useWithBackendStatus(notebook, backend_launch_phase) const [status_total, status_done] = useMemo( () => status == null ? [0, 0] : [ // total_tasks minus 1, to exclude the notebook task itself total_tasks(status) - 1, // the notebook task should never be done, but lets be sure and subtract 1 if it is: total_done(status) - (is_finished(status) ? 1 : 0), ], [status] ) const busy = status_done < status_total const show_business_outline = useDelayedTruth(busy, 700) const show_business_counter = useDelayedTruth(busy, 3000) const my_clock_is_ahead_by = useMyClockIsAheadBy({ connected }) const on_popout_click = async () => { // Open a Picture-in-Picture window, see https://developer.chrome.com/docs/web-platform/document-picture-in-picture/ // @ts-ignore const pip_window = await documentPictureInPicture.requestWindow() // Copy style sheets ;[...document.styleSheets].forEach((styleSheet) => { try { const style = document.createElement("style") style.textContent = [...styleSheet.cssRules].map((rule) => rule.cssText).join("") pip_window.document.head.appendChild(style) } catch (e) { const link = document.createElement("link") link.rel = "stylesheet" link.type = styleSheet.type // @ts-ignore link.media = styleSheet.media // @ts-ignore link.href = styleSheet.href pip_window.document.head.appendChild(link) } }) pip_window.document.body.append(container_ref.current.firstElementChild) pip_window.addEventListener("pagehide", (event) => { const pipPlayer = event.target.querySelector("pluto-helpbox") container_ref.current.append(pipPlayer) }) } return html` <aside id="helpbox-wrapper" ref=${container_ref}> <pluto-helpbox class=${cl({ hidden, [`helpbox-${open_tab ?? hidden}`]: true })}> <header translate=${false}> <button title="Live Docs: Search for Julia documentation, and get live documentation of everything you type." class=${cl({ "helpbox-tab-key": true, "helpbox-docs": true, "active": open_tab === "docs", })} onClick=${() => { focus_docs_on_open_ref.current = true set_open_tab(open_tab === "docs" ? null : "docs") // TODO: focus the docs input }} > <span class="tabicon"></span> <span class="tabname">Live Docs</span> </button> <button title=${"Process status"} class=${cl({ "helpbox-tab-key": true, "helpbox-process": true, "active": open_tab === "process", "busy": show_business_outline, "something_is_happening": busy || !connected, })} id="process-status-tab-button" onClick=${() => { set_open_tab(open_tab === "process" ? null : "process") }} > <span class="tabicon"></span> <span class="tabname" >${open_tab === "process" || !show_business_counter ? "Status" : html`Status${" "}<span class="subprogress-counter">(${status_done}/${status_total})</span>`}</span > </button> ${hidden ? null : html` ${"documentPictureInPicture" in window ? html`<button class="helpbox-popout" title="Pop out panel" onClick=${on_popout_click}> <span></span> </button>` : null} <button class="helpbox-close" title="Close panel" onClick=${() => { set_open_tab(null) }} > <span></span> </button>`} </header> ${open_tab === "docs" ? html`<${LiveDocsTab} focus_on_open=${focus_docs_on_open_ref.current} desired_doc_query=${desired_doc_query} on_update_doc_query=${on_update_doc_query} notebook=${notebook} sanitize_html=${sanitize_html} />` : open_tab === "process" ? html`<${StatusTab} notebook=${notebook} backend_launch_logs=${backend_launch_logs} my_clock_is_ahead_by=${my_clock_is_ahead_by} status=${status} />` : null} </pluto-helpbox> </aside> ` } export const useDelayedTruth = (/** @type {boolean} */ x, /** @type {number} */ timeout) => { const [output, set_output] = useState(false) useEffect(() => { if (x) { let handle = setTimeout(() => { set_output(true) }, timeout) return () => clearTimeout(handle) } else { set_output(false) } }, [x]) return output } /** * * @param {import("./Editor.js").NotebookData} notebook * @param {number?} backend_launch_phase * @returns {import("./Editor.js").StatusEntryData?} */ const useWithBackendStatus = (notebook, backend_launch_phase) => { const backend_launch = useBackendStatus(backend_launch_phase) return backend_launch_phase == null ? notebook.status_tree : { name: "notebook", started_at: 0, finished_at: null, subtasks: { ...notebook.status_tree?.subtasks, backend_launch, }, } } const useBackendStatus = (/** @type {number | null} */ backend_launch_phase) => { let x = backend_launch_phase ?? -1 const subtasks = Object.fromEntries( ["requesting", "created", "responded", "notebook_running"].map((key) => { let val = BackendLaunchPhase[key] let name = `backend_${key}` return [name, useStatusItem(name, x >= val, x > val)] }) ) return useStatusItem( "backend_launch", backend_launch_phase != null && backend_launch_phase > BackendLaunchPhase.wait_for_user, backend_launch_phase === BackendLaunchPhase.ready, subtasks ) } import _ from "../imports/lodash.js" import { html, useState, useEffect, useMemo, useRef, useContext, useLayoutEffect, useErrorBoundary, useCallback } from "../imports/Preact.js" import { CellOutput } from "./CellOutput.js" import { CellInput } from "./CellInput.js" import { Logs } from "./Logs.js" import { RunArea, useDebouncedTruth } from "./RunArea.js" import { cl } from "../common/ClassTable.js" import { PlutoActionsContext } from "../common/PlutoContext.js" import { open_pluto_popup } from "../common/open_pluto_popup.js" import { SafePreviewOutput } from "./SafePreviewUI.js" import { useEventListener } from "../common/useEventListener.js" const useCellApi = (node_ref, published_object_keys, pluto_actions) => { const [cell_api_ready, set_cell_api_ready] = useState(false) const published_object_keys_ref = useRef(published_object_keys) published_object_keys_ref.current = published_object_keys useLayoutEffect(() => { Object.assign(node_ref.current, { getPublishedObject: (id) => { if (!published_object_keys_ref.current.includes(id)) throw `getPublishedObject: ${id} not found` return pluto_actions.get_published_object(id) }, _internal_pluto_actions: pluto_actions, }) set_cell_api_ready(true) }) return cell_api_ready } /** * @param {String} a_cell_id * @param {import("./Editor.js").NotebookData} notebook * @returns {Array<String>} */ const upstream_of = (a_cell_id, notebook) => Object.values(notebook?.cell_dependencies?.[a_cell_id]?.upstream_cells_map || {}).flatMap((x) => x) /** * @param {String} a_cell_id * @param {import("./Editor.js").NotebookData} notebook * @param {Function} predicate * @param {Set<String>} explored * @returns {String | null} */ const find_upstream_of = (a_cell_id, notebook, predicate, explored = new Set([])) => { if (explored.has(a_cell_id)) return null explored.add(a_cell_id) if (predicate(a_cell_id)) { return a_cell_id } for (let upstream of upstream_of(a_cell_id, notebook)) { const upstream_val = find_upstream_of(upstream, notebook, predicate, explored) if (upstream_val !== null) { return upstream_val } } return null } /** * @param {String} flag_name * @returns {Function} */ const hasTargetBarrier = (flag_name) => { return (a_cell_id, notebook) => { return notebook?.cell_inputs?.[a_cell_id].metadata[flag_name] } } const on_jump = (hasBarrier, pluto_actions, cell_id) => () => { const notebook = pluto_actions.get_notebook() || {} const barrier_cell_id = find_upstream_of(cell_id, notebook, (c) => hasBarrier(c, notebook)) if (barrier_cell_id !== null) { window.dispatchEvent( new CustomEvent("cell_focus", { detail: { cell_id: barrier_cell_id, line: 0, // 1-based to 0-based index }, }) ) } } /** * @param {{ * cell_result: import("./Editor.js").CellResultData, * cell_input: import("./Editor.js").CellInputData, * cell_input_local: { code: String }, * cell_dependencies: import("./Editor.js").CellDependencyData * nbpkg: import("./Editor.js").NotebookPkgData?, * selected: boolean, * force_hide_input: boolean, * focus_after_creation: boolean, * process_waiting_for_permission: boolean, * sanitize_html: boolean, * [key: string]: any, * }} props * */ export const Cell = ({ cell_input: { cell_id, code, code_folded, metadata }, cell_result: { queued, running, runtime, errored, output, logs, published_object_keys, depends_on_disabled_cells, depends_on_skipped_cells }, cell_dependencies, cell_input_local, notebook_id, selected, force_hide_input, focus_after_creation, is_process_ready, disable_input, process_waiting_for_permission, sanitize_html = true, nbpkg, global_definition_locations, is_first_cell, }) => { const { show_logs, disabled: running_disabled, skip_as_script } = metadata let pluto_actions = useContext(PlutoActionsContext) // useCallback because pluto_actions.set_doc_query can change value when you go from viewing a static document to connecting (to binder) const on_update_doc_query = useCallback((...args) => pluto_actions.set_doc_query(...args), [pluto_actions]) const on_focus_neighbor = useCallback((...args) => pluto_actions.focus_on_neighbor(...args), [pluto_actions]) const on_change = useCallback((val) => pluto_actions.set_local_cell(cell_id, val), [cell_id, pluto_actions]) const variables = useMemo(() => Object.keys(cell_dependencies?.downstream_cells_map ?? {}), [cell_dependencies]) // We need to unmount & remount when a destructive error occurs. // For that reason, we will use a simple react key and increment it on error const [key, setKey] = useState(0) const cell_key = useMemo(() => cell_id + key, [cell_id, key]) const [, resetError] = useErrorBoundary((error) => { console.log(`An error occured in the CodeMirror code, resetting CellInput component. See error below:\n\n${error}\n\n -------------- `) setKey(key + 1) resetError() }) const remount = useMemo(() => () => setKey(key + 1)) // cm_forced_focus is null, except when a line needs to be highlighted because it is part of a stack trace const [cm_forced_focus, set_cm_forced_focus] = useState(/** @type {any} */ (null)) const [cm_highlighted_range, set_cm_highlighted_range] = useState(/** @type {{from, to}?} */ (null)) const [cm_highlighted_line, set_cm_highlighted_line] = useState(null) const [cm_diagnostics, set_cm_diagnostics] = useState([]) useEventListener( window, "cell_diagnostics", (e) => { if (e.detail.cell_id === cell_id) { set_cm_diagnostics(e.detail.diagnostics) } }, [cell_id, set_cm_diagnostics] ) useEventListener( window, "cell_highlight_range", (e) => { if (e.detail.cell_id == cell_id && e.detail.from != null && e.detail.to != null) { set_cm_highlighted_range({ from: e.detail.from, to: e.detail.to }) } else { set_cm_highlighted_range(null) } }, [cell_id] ) useEventListener( window, "cell_focus", useCallback((e) => { if (e.detail.cell_id === cell_id) { if (e.detail.line != null) { const ch = e.detail.ch if (ch == null) { set_cm_forced_focus([ { line: e.detail.line, ch: 0 }, { line: e.detail.line, ch: Infinity }, { scroll: true, definition_of: e.detail.definition_of }, ]) } else { set_cm_forced_focus([ { line: e.detail.line, ch: ch }, { line: e.detail.line, ch: ch }, { scroll: true, definition_of: e.detail.definition_of }, ]) } } } }, []) ) // When you click to run a cell, we use `waiting_to_run` to immediately set the cell's traffic light to 'queued', while waiting for the backend to catch up. const [waiting_to_run, set_waiting_to_run] = useState(false) useEffect(() => { set_waiting_to_run(false) }, [queued, running, output?.last_run_timestamp, depends_on_disabled_cells, running_disabled]) // We activate animations instantly BUT deactivate them NSeconds later. // We then toggle animation visibility using opacity. This saves a bunch of repaints. const activate_animation = useDebouncedTruth(running || queued || waiting_to_run) const class_code_differs = code !== (cell_input_local?.code ?? code) const no_output_yet = (output?.last_run_timestamp ?? 0) === 0 const code_not_trusted_yet = process_waiting_for_permission && no_output_yet // during the initial page load, force_hide_input === true, so that cell outputs render fast, and codemirrors are loaded after let show_input = !force_hide_input && (code_not_trusted_yet || errored || class_code_differs || cm_forced_focus != null || !code_folded) const [line_heights, set_line_heights] = useState([15]) const node_ref = useRef(/** @type {HTMLElement?} */ (null)) const disable_input_ref = useRef(disable_input) disable_input_ref.current = disable_input const should_set_waiting_to_run_ref = useRef(true) should_set_waiting_to_run_ref.current = !running_disabled && !depends_on_disabled_cells useEventListener( window, "set_waiting_to_run_smart", (e) => { if (e.detail.cell_ids.includes(cell_id)) set_waiting_to_run(should_set_waiting_to_run_ref.current) }, [cell_id, set_waiting_to_run] ) const cell_api_ready = useCellApi(node_ref, published_object_keys, pluto_actions) const on_delete = useCallback(() => { pluto_actions.confirm_delete_multiple("Delete", pluto_actions.get_selected_cells(cell_id, selected)) }, [pluto_actions, selected, cell_id]) const on_submit = useCallback(() => { if (!disable_input_ref.current) { pluto_actions.set_and_run_multiple([cell_id]) } }, [pluto_actions, cell_id]) const on_change_cell_input = useCallback( (new_code) => { if (!disable_input_ref.current) { if (code_folded && cm_forced_focus != null) { pluto_actions.fold_remote_cells([cell_id], false) } on_change(new_code) } }, [code_folded, cm_forced_focus, pluto_actions, on_change] ) const on_add_after = useCallback(() => { pluto_actions.add_remote_cell(cell_id, "after") }, [pluto_actions, cell_id, selected]) const on_code_fold = useCallback(() => { pluto_actions.fold_remote_cells(pluto_actions.get_selected_cells(cell_id, selected), !code_folded) }, [pluto_actions, cell_id, selected, code_folded]) const on_run = useCallback(() => { pluto_actions.set_and_run_multiple(pluto_actions.get_selected_cells(cell_id, selected)) }, [pluto_actions, cell_id, selected]) const set_show_logs = useCallback( (show_logs) => pluto_actions.update_notebook((notebook) => { notebook.cell_inputs[cell_id].metadata.show_logs = show_logs }), [pluto_actions, cell_id] ) const set_cell_disabled = useCallback( async (new_val) => { await pluto_actions.update_notebook((notebook) => { notebook.cell_inputs[cell_id].metadata["disabled"] = new_val }) // we also 'run' the cell if it is disabled, this will make the backend propage the disabled state to dependent cells await on_submit() }, [pluto_actions, cell_id, on_submit] ) const any_logs = useMemo(() => !_.isEmpty(logs), [logs]) const skip_as_script_jump = useCallback(on_jump(hasTargetBarrier("skip_as_script"), pluto_actions, cell_id), [pluto_actions, cell_id]) const disabled_jump = useCallback(on_jump(hasTargetBarrier("disabled"), pluto_actions, cell_id), [pluto_actions, cell_id]) return html` <pluto-cell key=${cell_key} ref=${node_ref} class=${cl({ queued: queued || (waiting_to_run && is_process_ready), internal_test_queued: !is_process_ready && (queued || waiting_to_run), running, activate_animation, errored, selected, code_differs: class_code_differs, code_folded, skip_as_script, running_disabled, depends_on_disabled_cells, depends_on_skipped_cells, show_input, shrunk: Object.values(logs).length > 0, hooked_up: output?.has_pluto_hook_features ?? false, no_output_yet, })} id=${cell_id} > ${variables.map((name) => html`<span id=${encodeURI(name)} />`)} <button onClick=${() => { pluto_actions.add_remote_cell(cell_id, "before") }} class="add_cell before" title="Add cell (Ctrl + Enter)" tabindex=${is_first_cell ? undefined : "-1"} > <span></span> </button> <pluto-shoulder draggable="true" title="Drag to move cell"> <button onClick=${on_code_fold} class="foldcode" title="Show/hide code"> <span></span> </button> </pluto-shoulder> <pluto-trafficlight></pluto-trafficlight> ${code_not_trusted_yet ? html`<${SafePreviewOutput} />` : cell_api_ready ? html`<${CellOutput} errored=${errored} ...${output} sanitize_html=${sanitize_html} cell_id=${cell_id} />` : html``} <${CellInput} local_code=${cell_input_local?.code ?? code} remote_code=${code} global_definition_locations=${global_definition_locations} disable_input=${disable_input} focus_after_creation=${focus_after_creation} cm_forced_focus=${cm_forced_focus} set_cm_forced_focus=${set_cm_forced_focus} show_input=${show_input} skip_static_fake=${is_first_cell} on_submit=${on_submit} on_delete=${on_delete} on_add_after=${on_add_after} on_change=${on_change_cell_input} on_update_doc_query=${on_update_doc_query} on_focus_neighbor=${on_focus_neighbor} on_line_heights=${set_line_heights} nbpkg=${nbpkg} cell_id=${cell_id} notebook_id=${notebook_id} metadata=${metadata} any_logs=${any_logs} show_logs=${show_logs} set_show_logs=${set_show_logs} set_cell_disabled=${set_cell_disabled} cm_highlighted_line=${cm_highlighted_line} cm_highlighted_range=${cm_highlighted_range} cm_diagnostics=${cm_diagnostics} onerror=${remount} /> ${show_logs && cell_api_ready ? html`<${Logs} logs=${Object.values(logs)} line_heights=${line_heights} set_cm_highlighted_line=${set_cm_highlighted_line} sanitize_html=${sanitize_html} />` : null} <${RunArea} cell_id=${cell_id} running_disabled=${running_disabled} depends_on_disabled_cells=${depends_on_disabled_cells} on_run=${on_run} on_interrupt=${() => { pluto_actions.interrupt_remote(cell_id) }} set_cell_disabled=${set_cell_disabled} runtime=${runtime} running=${running} code_differs=${class_code_differs} queued=${queued} on_jump=${disabled_jump} /> <button onClick=${() => { pluto_actions.add_remote_cell(cell_id, "after") }} class="add_cell after" title="Add cell (Ctrl + Enter)" > <span></span> </button> ${skip_as_script ? html`<div class="skip_as_script_marker" title=${`This cell is directly flagged as disabled in file. Click to know more!`} onClick=${(e) => { open_pluto_popup({ type: "info", source_element: e.target, body: html`This cell is currently stored in the notebook file as a Julia <em>comment</em>, instead of <em>code</em>.<br /> This way, it will not run when the notebook runs as a script outside of Pluto.<br /> Use the context menu to enable it again`, }) }} ></div>` : depends_on_skipped_cells ? html`<div class="depends_on_skipped_marker" title=${`This cell is indirectly flagged as disabled in file. Click to know more!`} onClick=${(e) => { open_pluto_popup({ type: "info", source_element: e.target, body: html`This cell is currently stored in the notebook file as a Julia <em>comment</em>, instead of <em>code</em>.<br /> This way, it will not run when the notebook runs as a script outside of Pluto.<br /> An upstream cell is <b> indirectly</b> <em>disabling in file</em> this one; enable <span onClick=${skip_as_script_jump} style="cursor: pointer; text-decoration: underline"> the upstream one</span> to affect this cell.`, }) }} ></div>` : null} </pluto-cell> ` } /** * @param {{ * cell_result: import("./Editor.js").CellResultData, * cell_input: import("./Editor.js").CellInputData, * [key: string]: any, * }} props * */ export const IsolatedCell = ({ cell_input: { cell_id, metadata }, cell_result: { logs, output, published_object_keys }, hidden, sanitize_html = true }) => { const node_ref = useRef(/** @type {HTMLElement?} */ (null)) let pluto_actions = useContext(PlutoActionsContext) const cell_api_ready = useCellApi(node_ref, published_object_keys, pluto_actions) const { show_logs } = metadata return html` <pluto-cell ref=${node_ref} id=${cell_id} class=${hidden ? "hidden-cell" : "isolated-cell"}> ${cell_api_ready ? html`<${CellOutput} ...${output} sanitize_html=${sanitize_html} cell_id=${cell_id} />` : html``} ${show_logs ? html`<${Logs} logs=${Object.values(logs)} line_heights=${[15]} set_cm_highlighted_line=${() => {}} />` : null} </pluto-cell> ` } import { html, useState, useEffect, useLayoutEffect, useRef, useContext, useMemo } from "../imports/Preact.js" import _ from "../imports/lodash.js" import { utf8index_to_ut16index } from "../common/UnicodeTools.js" import { PlutoActionsContext } from "../common/PlutoContext.js" import { get_selected_doc_from_state } from "./CellInput/LiveDocsFromCursor.js" import { go_to_definition_plugin, GlobalDefinitionsFacet } from "./CellInput/go_to_definition_plugin.js" // import { debug_syntax_plugin } from "./CellInput/debug_syntax_plugin.js" import { EditorState, EditorSelection, Compartment, EditorView, placeholder, keymap, history, historyKeymap, defaultKeymap, indentMore, indentLess, tags, HighlightStyle, lineNumbers, highlightSpecialChars, drawSelection, indentOnInput, closeBrackets, rectangularSelection, highlightSelectionMatches, closeBracketsKeymap, foldKeymap, indentUnit, autocomplete, htmlLanguage, markdownLanguage, javascriptLanguage, pythonLanguage, syntaxHighlighting, cssLanguage, setDiagnostics, moveLineUp, Facet, } from "../imports/CodemirrorPlutoSetup.js" import { markdown, html as htmlLang, javascript, sqlLang, python, julia_mixed } from "./CellInput/mixedParsers.js" import { julia } from "../imports/CodemirrorPlutoSetup.js" import { pluto_autocomplete } from "./CellInput/pluto_autocomplete.js" import { NotebookpackagesFacet, pkgBubblePlugin } from "./CellInput/pkg_bubble_plugin.js" import { awesome_line_wrapping, get_start_tabs } from "./CellInput/awesome_line_wrapping.js" import { cell_movement_plugin, prevent_holding_a_key_from_doing_things_across_cells } from "./CellInput/cell_movement_plugin.js" import { pluto_paste_plugin } from "./CellInput/pluto_paste_plugin.js" import { bracketMatching } from "./CellInput/block_matcher_plugin.js" import { cl } from "../common/ClassTable.js" import { HighlightLineFacet, HighlightRangeFacet, highlightLinePlugin, highlightRangePlugin } from "./CellInput/highlight_line.js" import { commentKeymap } from "./CellInput/comment_mixed_parsers.js" import { ScopeStateField } from "./CellInput/scopestate_statefield.js" import { mod_d_command } from "./CellInput/mod_d_command.js" import { open_bottom_right_panel } from "./BottomRightPanel.js" import { timeout_promise } from "../common/PlutoConnection.js" import { LastFocusWasForcedEffect, tab_help_plugin } from "./CellInput/tab_help_plugin.js" import { useEventListener } from "../common/useEventListener.js" import { moveLineDown } from "../imports/CodemirrorPlutoSetup.js" import { open_pluto_popup } from "../common/open_pluto_popup.js" import { AIContext } from "./AIContext.js" import { AiSuggestionPlugin } from "./CellInput/ai_suggestion.js" export const ENABLE_CM_MIXED_PARSER = window.localStorage.getItem("ENABLE_CM_MIXED_PARSER") === "true" export const ENABLE_CM_SPELLCHECK = window.localStorage.getItem("ENABLE_CM_SPELLCHECK") === "true" export const ENABLE_CM_AUTOCOMPLETE_ON_TYPE = (window.localStorage.getItem("ENABLE_CM_AUTOCOMPLETE_ON_TYPE") ?? (/Mac/.test(navigator.platform) ? "true" : "false")) === "true" if (ENABLE_CM_MIXED_PARSER) { console.log(`YOU ENABLED THE CODEMIRROR MIXED LANGUAGE PARSER Thanks! Awesome! Please let us know if you find any bugs... If enough people do this, we can make it the default parser. `) } // Added this so we can have people test the mixed parser, because I LIKE IT SO MUCH - DRAL // @ts-ignore window.PLUTO_TOGGLE_CM_MIXED_PARSER = (val = !ENABLE_CM_MIXED_PARSER) => { window.localStorage.setItem("ENABLE_CM_MIXED_PARSER", String(val)) window.location.reload() } // @ts-ignore window.PLUTO_TOGGLE_CM_SPELLCHECK = (val = !ENABLE_CM_SPELLCHECK) => { window.localStorage.setItem("ENABLE_CM_SPELLCHECK", String(val)) window.location.reload() } // @ts-ignore window.PLUTO_TOGGLE_CM_AUTOCOMPLETE_ON_TYPE = (val = !ENABLE_CM_AUTOCOMPLETE_ON_TYPE) => { window.localStorage.setItem("ENABLE_CM_AUTOCOMPLETE_ON_TYPE", String(val)) window.location.reload() } const common_style_tags = [ { tag: tags.comment, color: "var(--cm-color-comment)", fontStyle: "italic", filter: "none" }, { tag: tags.variableName, color: "var(--cm-color-variable)", fontWeight: 700 }, { tag: tags.propertyName, color: "var(--cm-color-symbol)", fontWeight: 700 }, { tag: tags.macroName, color: "var(--cm-color-macro)", fontWeight: 700 }, { tag: tags.typeName, filter: "var(--cm-filter-type)", fontWeight: "lighter" }, { tag: tags.atom, color: "var(--cm-color-symbol)" }, { tag: tags.string, color: "var(--cm-color-string)" }, { tag: tags.special(tags.string), color: "var(--cm-color-command)" }, { tag: tags.character, color: "var(--cm-color-literal)" }, { tag: tags.literal, color: "var(--cm-color-literal)" }, { tag: tags.keyword, color: "var(--cm-color-keyword)" }, // TODO: normal operators { tag: tags.definitionOperator, color: "var(--cm-color-keyword)" }, { tag: tags.logicOperator, color: "var(--cm-color-keyword)" }, { tag: tags.controlOperator, color: "var(--cm-color-keyword)" }, { tag: tags.bracket, color: "var(--cm-color-bracket)" }, // TODO: tags.self, tags.null ] export const pluto_syntax_colors_julia = HighlightStyle.define(common_style_tags, { all: { color: `var(--cm-color-editor-text)` }, scope: julia().language, }) export const pluto_syntax_colors_javascript = HighlightStyle.define(common_style_tags, { all: { color: `var(--cm-color-editor-text)`, filter: `contrast(0.5)` }, scope: javascriptLanguage, }) export const pluto_syntax_colors_python = HighlightStyle.define(common_style_tags, { all: { color: `var(--cm-color-editor-text)`, filter: `contrast(0.5)` }, scope: pythonLanguage, }) export const pluto_syntax_colors_css = HighlightStyle.define( [ { tag: tags.comment, color: "var(--cm-color-comment)", fontStyle: "italic" }, { tag: tags.variableName, color: "var(--cm-color-css-accent)", fontWeight: 700 }, { tag: tags.propertyName, color: "var(--cm-color-css-accent)", fontWeight: 700 }, { tag: tags.tagName, color: "var(--cm-color-css)", fontWeight: 700 }, //{ tag: tags.className, color: "var(--cm-css-why-doesnt-codemirror-highlight-all-the-text-aaa)" }, //{ tag: tags.constant(tags.className), color: "var(--cm-css-why-doesnt-codemirror-highlight-all-the-text-aaa)" }, { tag: tags.definitionOperator, color: "var(--cm-color-css)" }, { tag: tags.keyword, color: "var(--cm-color-css)" }, { tag: tags.modifier, color: "var(--cm-color-css-accent)" }, { tag: tags.literal, color: "var(--cm-color-css)" }, // { tag: tags.unit, color: "var(--cm-color-css-accent)" }, { tag: tags.punctuation, opacity: 0.5 }, ], { scope: cssLanguage, all: { color: "var(--cm-color-css)" }, } ) export const pluto_syntax_colors_html = HighlightStyle.define( [ { tag: tags.comment, color: "var(--cm-color-comment)", fontStyle: "italic" }, { tag: tags.content, color: "var(--cm-color-html)", fontWeight: 400 }, { tag: tags.tagName, color: "var(--cm-color-html-accent)", fontWeight: 600 }, { tag: tags.documentMeta, color: "var(--cm-color-html-accent)" }, { tag: tags.attributeName, color: "var(--cm-color-html-accent)", fontWeight: 600 }, { tag: tags.attributeValue, color: "var(--cm-color-html-accent)" }, { tag: tags.angleBracket, color: "var(--cm-color-html-accent)", fontWeight: 600, opacity: 0.7 }, ], { all: { color: "var(--cm-color-html)" }, scope: htmlLanguage, } ) // https://github.com/lezer-parser/markdown/blob/d4de2b03180ced4610bad9cef0ad3a805c43b63a/src/markdown.ts#L1890 export const pluto_syntax_colors_markdown = HighlightStyle.define( [ { tag: tags.comment, color: "var(--cm-color-comment)", fontStyle: "italic" }, { tag: tags.content, color: "var(--cm-color-md)" }, { tag: tags.heading, color: "var(--cm-color-md)", fontWeight: 700 }, // TODO? tags.list { tag: tags.quote, color: "var(--cm-color-md)" }, { tag: tags.emphasis, fontStyle: "italic" }, { tag: tags.strong, fontWeight: "bolder" }, { tag: tags.link, textDecoration: "underline" }, { tag: tags.url, color: "var(--cm-color-md)", textDecoration: "none" }, { tag: tags.monospace, color: "var(--cm-color-md-accent)" }, // Marks: `-` for lists, `#` for headers, etc. { tag: tags.processingInstruction, color: "var(--cm-color-md-accent) !important", opacity: "0.5" }, ], { all: { color: "var(--cm-color-md)" }, scope: markdownLanguage, } ) const getValue6 = (/** @type {EditorView} */ cm) => cm.state.doc.toString() const setValue6 = (/** @type {EditorView} */ cm, value) => cm.dispatch({ changes: { from: 0, to: cm.state.doc.length, insert: value }, }) const replaceRange6 = (/** @type {EditorView} */ cm, text, from, to) => cm.dispatch({ changes: { from, to, insert: text }, }) // Compartments: https://codemirror.net/6/examples/config/ let useCompartment = (/** @type {import("../imports/Preact.js").Ref<EditorView?>} */ codemirror_ref, value) => { const compartment = useRef(new Compartment()) const initial_value = useRef(compartment.current.of(value)) useLayoutEffect(() => { codemirror_ref.current?.dispatch?.({ effects: compartment.current.reconfigure(value), }) }, [value]) return initial_value.current } export const LastRemoteCodeSetTimeFacet = Facet.define({ combine: (values) => values[0], compare: _.isEqual, }) let line_and_ch_to_cm6_position = (/** @type {import("../imports/CodemirrorPlutoSetup.js").Text} */ doc, { line, ch }) => { let line_object = doc.line(_.clamp(line + 1, 1, doc.lines)) let ch_clamped = _.clamp(ch, 0, line_object.length) return line_object.from + ch_clamped } /** * @param {{ * local_code: string, * remote_code: string, * scroll_into_view_after_creation: boolean, * nbpkg: import("./Editor.js").NotebookPkgData?, * global_definition_locations: { [variable_name: string]: string }, * [key: string]: any, * }} props */ export const CellInput = ({ local_code, remote_code, disable_input, focus_after_creation, cm_forced_focus, set_cm_forced_focus, show_input, skip_static_fake = false, on_submit, on_delete, on_add_after, on_change, on_update_doc_query, on_focus_neighbor, on_line_heights, nbpkg, cell_id, notebook_id, any_logs, show_logs, set_show_logs, set_cell_disabled, cm_highlighted_line, cm_highlighted_range, metadata, global_definition_locations, cm_diagnostics, }) => { let pluto_actions = useContext(PlutoActionsContext) const { disabled: running_disabled, skip_as_script } = metadata let [error, set_error] = useState(null) if (error) { const to_throw = error set_error(null) throw to_throw } const notebook_id_ref = useRef(notebook_id) notebook_id_ref.current = notebook_id const newcm_ref = useRef(/** @type {EditorView?} */ (null)) const dom_node_ref = useRef(/** @type {HTMLElement?} */ (null)) const remote_code_ref = useRef(/** @type {string?} */ (null)) let nbpkg_compartment = useCompartment(newcm_ref, NotebookpackagesFacet.of(nbpkg)) let global_definitions_compartment = useCompartment(newcm_ref, GlobalDefinitionsFacet.of(global_definition_locations)) let highlighted_line_compartment = useCompartment(newcm_ref, HighlightLineFacet.of(cm_highlighted_line)) let highlighted_range_compartment = useCompartment(newcm_ref, HighlightRangeFacet.of(cm_highlighted_range)) let editable_compartment = useCompartment(newcm_ref, EditorState.readOnly.of(disable_input)) let last_remote_code_set_time_compartment = useCompartment( newcm_ref, useMemo(() => LastRemoteCodeSetTimeFacet.of(Date.now()), [remote_code]) ) let on_change_compartment = useCompartment( newcm_ref, // Functions are hard to compare, so I useMemo manually useMemo(() => { return EditorView.updateListener.of((update) => { if (update.docChanged) { on_change(update.state.doc.toString()) } }) }, [on_change]) ) const [show_static_fake_state, set_show_static_fake] = useState(!skip_static_fake) const show_static_fake_excuses_ref = useRef(false) show_static_fake_excuses_ref.current ||= navigator.userAgent.includes("Firefox") || focus_after_creation || cm_forced_focus != null || skip_static_fake const show_static_fake = show_static_fake_excuses_ref.current ? false : show_static_fake_state useLayoutEffect(() => { if (!show_static_fake) return let node = dom_node_ref.current if (node == null) return let observer const show = () => { set_show_static_fake(false) observer.disconnect() window.removeEventListener("beforeprint", show) } observer = new IntersectionObserver((e) => { if (e.some((e) => e.isIntersecting)) { show() } }) observer.observe(node) window.addEventListener("beforeprint", show) return () => { observer.disconnect() window.removeEventListener("beforeprint", show) } }, []) useLayoutEffect(() => { if (show_static_fake) return if (dom_node_ref.current == null) return const keyMapSubmit = (/** @type {EditorView} */ cm) => { autocomplete.closeCompletion(cm) on_submit() return true } let run = async (fn) => await fn() const keyMapRun = (/** @type {EditorView} */ cm) => { autocomplete.closeCompletion(cm) run(async () => { // we await to prevent an out-of-sync issue await on_add_after() const new_value = cm.state.doc.toString() if (new_value !== remote_code_ref.current) { on_submit() } }) return true } let select_autocomplete_command = autocomplete.completionKeymap.find((keybinding) => keybinding.key === "Enter") let keyMapTab = (/** @type {EditorView} */ cm) => { // I think this only gets called when we are not in an autocomplete situation, otherwise `tab_completion_command` is called. I think it only happens when you have a selection. if (cm.state.readOnly) { return false } // This will return true if the autocomplete select popup is open if (select_autocomplete_command?.run?.(cm)) { return true } const anySelect = cm.state.selection.ranges.some((r) => !r.empty) if (anySelect) { return indentMore(cm) } else { cm.dispatch( cm.state.changeByRange((selection) => ({ range: EditorSelection.cursor(selection.from + 1), changes: { from: selection.from, to: selection.to, insert: "\t" }, })) ) return true } } const keyMapMD = () => { const cm = /** @type{EditorView} */ (newcm_ref.current) const value = getValue6(cm) const trimmed = value.trim() const offset = value.length - value.trimStart().length console.table({ value, trimmed, offset }) if (trimmed.startsWith('md"') && trimmed.endsWith('"')) { // Markdown cell, change to code let start, end if (trimmed.startsWith('md"""') && trimmed.endsWith('"""')) { // Block markdown start = 5 end = trimmed.length - 3 } else { // Inline markdown start = 3 end = trimmed.length - 1 } if (start >= end || trimmed.substring(start, end).trim() == "") { // Corner case: block is empty after removing markdown setValue6(cm, "") } else { while (/\s/.test(trimmed[start])) { ++start } while (/\s/.test(trimmed[end - 1])) { --end } // Keep the selection from [start, end) while maintaining cursor position replaceRange6(cm, "", end + offset, cm.state.doc.length) // cm.replaceRange("", cm.posFromIndex(end + offset), { line: cm.lineCount() }) replaceRange6(cm, "", 0, start + offset) // cm.replaceRange("", { line: 0, ch: 0 }, cm.posFromIndex(start + offset)) } } else { // Replacing ranges will maintain both the focus, the selections and the cursor let prefix = `md"""\n` let suffix = `\n"""` // TODO Multicursor? let selection = cm.state.selection.main cm.dispatch({ changes: [ { from: 0, to: 0, insert: prefix }, { from: cm.state.doc.length, to: cm.state.doc.length, insert: suffix, }, ], selection: selection.from === 0 ? { anchor: selection.from + prefix.length, head: selection.to + prefix.length, } : undefined, }) } return true } const keyMapDelete = (/** @type {EditorView} */ cm) => { if (cm.state.facet(EditorState.readOnly)) { return false } if (cm.state.doc.length === 0) { on_focus_neighbor(cell_id, +1) on_delete() return true } return false } const keyMapBackspace = (/** @type {EditorView} */ cm) => { if (cm.state.facet(EditorState.readOnly)) { return false } // Previously this was a very elaborate timed implementation...... // But I found out that keyboard events have a `.repeated` property which is perfect for what we want... // So now this is just the cell deleting logic (and the repeated stuff is in a separate plugin) if (cm.state.doc.length === 0) { // `Infinity, Infinity` means: last line, last character on_focus_neighbor(cell_id, -1, Infinity, Infinity) on_delete() return true } return false } const keyMapMoveLine = (/** @type {EditorView} */ cm, direction) => { if (cm.state.facet(EditorState.readOnly)) { return false } const selection = cm.state.selection.main const all_is_selected = selection.anchor === 0 && selection.head === cm.state.doc.length if (all_is_selected || cm.state.doc.lines === 1) { pluto_actions.move_remote_cells([cell_id], pluto_actions.get_notebook().cell_order.indexOf(cell_id) + (direction === -1 ? -1 : 2)) // workaround for https://github.com/preactjs/preact/issues/4235 // but the scrollIntoView behaviour is nice, also when the preact issue is fixed. requestIdleCallback(() => { cm.dispatch({ // TODO: remove me after fix selection: { anchor: 0, head: cm.state.doc.length, }, // TODO: keep me after fix scrollIntoView: true, }) // TODO: remove me after fix cm.focus() }) return true } else { return direction === 1 ? moveLineDown(cm) : moveLineUp(cm) } } const keyMapFold = (/** @type {EditorView} */ cm, new_value) => { set_cm_forced_focus(true) pluto_actions.fold_remote_cells([cell_id], new_value) return true } const plutoKeyMaps = [ { key: "Shift-Enter", run: keyMapSubmit }, { key: "Ctrl-Enter", mac: "Cmd-Enter", run: keyMapRun }, { key: "Ctrl-Enter", run: keyMapRun }, { key: "Tab", run: keyMapTab, shift: indentLess }, { key: "Ctrl-m", mac: "Cmd-m", run: keyMapMD }, { key: "Ctrl-m", run: keyMapMD }, // Codemirror6 doesn't like capslock { key: "Ctrl-M", run: keyMapMD }, // TODO Move Delete and backspace to cell movement plugin { key: "Delete", run: keyMapDelete }, { key: "Ctrl-Delete", run: keyMapDelete }, { key: "Backspace", run: keyMapBackspace }, { key: "Ctrl-Backspace", run: keyMapBackspace }, { key: "Alt-ArrowUp", run: (x) => keyMapMoveLine(x, -1) }, { key: "Alt-ArrowDown", run: (x) => keyMapMoveLine(x, 1) }, { key: "Ctrl-Shift-[", mac: "Cmd-Alt-[", run: (x) => keyMapFold(x, true) }, { key: "Ctrl-Shift-]", mac: "Cmd-Alt-]", run: (x) => keyMapFold(x, false) }, mod_d_command, ] let DOCS_UPDATER_VERBOSE = false const docs_updater = EditorView.updateListener.of((update) => { if (!update.view.hasFocus) { return } if (update.docChanged || update.selectionSet) { let state = update.state DOCS_UPDATER_VERBOSE && console.groupCollapsed("Live docs updater") try { let result = get_selected_doc_from_state(state, DOCS_UPDATER_VERBOSE) if (result != null) { on_update_doc_query(result) } } finally { DOCS_UPDATER_VERBOSE && console.groupEnd() } } }) const unsubmitted_globals_updater = EditorView.updateListener.of((update) => { if (update.docChanged) { const before = [...update.startState.field(ScopeStateField).definitions.keys()] const after = [...update.state.field(ScopeStateField).definitions.keys()] if (!_.isEqual(before, after)) { pluto_actions.set_unsubmitted_global_definitions(cell_id, after) } } }) const usesDarkTheme = window.matchMedia("(prefers-color-scheme: dark)").matches const newcm = (newcm_ref.current = new EditorView({ state: EditorState.create({ doc: local_code, extensions: [ EditorView.theme({}, { dark: usesDarkTheme }), // Compartments coming from react state/props nbpkg_compartment, highlighted_line_compartment, highlighted_range_compartment, global_definitions_compartment, editable_compartment, last_remote_code_set_time_compartment, highlightLinePlugin(), highlightRangePlugin(), // This is waaaay in front of the keys it is supposed to override, // Which is necessary because it needs to run before *any* keymap, // as the first keymap will activate the keymap extension which will attach the // keymap handlers at that point, which is likely before this extension. // TODO Use https://codemirror.net/6/docs/ref/#state.Prec when added to pluto-codemirror-setup prevent_holding_a_key_from_doing_things_across_cells, pkgBubblePlugin({ pluto_actions, notebook_id_ref }), ScopeStateField, syntaxHighlighting(pluto_syntax_colors_julia), syntaxHighlighting(pluto_syntax_colors_html), syntaxHighlighting(pluto_syntax_colors_markdown), syntaxHighlighting(pluto_syntax_colors_javascript), syntaxHighlighting(pluto_syntax_colors_python), syntaxHighlighting(pluto_syntax_colors_css), lineNumbers(), highlightSpecialChars(), history(), drawSelection(), EditorState.allowMultipleSelections.of(true), // Multiple cursors with `alt` instead of the default `ctrl` (which we use for go to definition) EditorView.clickAddsSelectionRange.of((event) => event.altKey && !event.shiftKey), indentOnInput(), // Experimental: Also add closing brackets for tripple string // TODO also add closing string when typing a string macro EditorState.languageData.of((state, pos, side) => { return [{ closeBrackets: { brackets: ["(", "[", "{"] } }] }), closeBrackets(), rectangularSelection({ eventFilter: (e) => e.altKey && e.shiftKey && e.button == 0, }), highlightSelectionMatches({ minSelectionLength: 2, wholeWords: true }), bracketMatching(), docs_updater, unsubmitted_globals_updater, tab_help_plugin, // Remove selection on blur EditorView.domEventHandlers({ blur: (event, view) => { // it turns out that this condition is true *exactly* if and only if the blur event was triggered by blurring the window let caused_by_window_blur = document.activeElement === view.contentDOM if (!caused_by_window_blur) { // then it's caused by focusing something other than this cell in the editor. // in this case, we want to collapse the selection into a single point, for aesthetic reasons. setTimeout(() => { view.dispatch({ selection: { anchor: view.state.selection.main.head, }, scrollIntoView: false, }) // and blur the DOM again (because the previous transaction might have re-focused it) view.contentDOM.blur() }, 0) set_cm_forced_focus(null) } }, }), pluto_paste_plugin({ pluto_actions, cell_id, }), // Update live docs when in a cell that starts with `?` EditorView.updateListener.of((update) => { if (!update.docChanged) return if (update.state.doc.length > 0 && update.state.sliceDoc(0, 1) === "?") { open_bottom_right_panel("docs") } }), EditorState.tabSize.of(4), indentUnit.of("\t"), ...(ENABLE_CM_MIXED_PARSER ? [ julia_mixed(), markdown({ defaultCodeLanguage: julia_mixed(), }), htmlLang(), //Provides tag closing!, javascript(), python(), sqlLang, ] : [ // julia(), ]), go_to_definition_plugin, AiSuggestionPlugin(), pluto_autocomplete({ request_autocomplete: async ({ query, query_full }) => { let response = await timeout_promise( pluto_actions.send("complete", { query, query_full }, { notebook_id: notebook_id_ref.current }), 5000 ).catch(console.warn) if (!response) return null let { message } = response return { start: utf8index_to_ut16index(query_full ?? query, message.start), stop: utf8index_to_ut16index(query_full ?? query, message.stop), results: message.results, too_long: message.too_long, } }, request_packages: () => pluto_actions.send("all_registered_package_names").then(({ message }) => message.results), request_special_symbols: () => pluto_actions.send("complete_symbols").then(({ message }) => message), on_update_doc_query: on_update_doc_query, request_unsubmitted_global_definitions: () => pluto_actions.get_unsubmitted_global_definitions(), cell_id, }), // I put plutoKeyMaps separately because I want make sure we have // higher priority  keymap.of(plutoKeyMaps), keymap.of(commentKeymap), // Before default keymaps (because we override some of them) // but after the autocomplete plugin, because we don't want to move cell when scrolling through autocomplete cell_movement_plugin({ focus_on_neighbor: ({ cell_delta, line, character }) => on_focus_neighbor(cell_id, cell_delta, line, character), }), keymap.of([...closeBracketsKeymap, ...defaultKeymap, ...historyKeymap, ...foldKeymap]), placeholder("Enter cell code..."), EditorView.contentAttributes.of({ spellcheck: String(ENABLE_CM_SPELLCHECK) }), EditorView.lineWrapping, awesome_line_wrapping, // Reset diagnostics on change EditorView.updateListener.of((update) => { if (!update.docChanged) return update.view.dispatch(setDiagnostics(update.state, [])) }), on_change_compartment, // This is my weird-ass extension that checks the AST and shows you where // there're missing nodes.. I'm not sure if it's a good idea to have it // show_missing_syntax_plugin(), // Enable this plugin if you want to see the lezer tree, // and possible lezer errors and maybe more debug info in the console: // debug_syntax_plugin, // Handle errors hopefully? EditorView.exceptionSink.of((exception) => { set_error(exception) console.error("EditorView exception!", exception) // alert( // `We ran into an issue! We have lost your cursor \n If this appears again, please press F12, then click the "Console" tab, eport an issue at https://github.com/fonsp/Pluto.jl/issues` // ) }), ], }), parent: dom_node_ref.current, })) // For use from useDropHandler // @ts-ignore newcm.dom.CodeMirror = { getValue: () => getValue6(newcm), setValue: (x) => setValue6(newcm, x), } if (focus_after_creation) { setTimeout(() => { let view = newcm_ref.current if (view == null) return view.dom.scrollIntoView({ behavior: "smooth", block: "nearest", }) view.dispatch({ selection: { anchor: view.state.doc.length, head: view.state.doc.length, }, effects: [LastFocusWasForcedEffect.of(true)], }) view.focus() }) } // @ts-ignore const lines_wrapper_dom_node = dom_node_ref.current.querySelector("div.cm-content") if (lines_wrapper_dom_node) { const lines_wrapper_resize_observer = new ResizeObserver(() => { const line_nodes = lines_wrapper_dom_node.children const tops = _.map(line_nodes, (c) => /** @type{HTMLElement} */ (c).offsetTop) const diffs = tops.slice(1).map((y, i) => y - tops[i]) const heights = [...diffs, 15] on_line_heights(heights) }) lines_wrapper_resize_observer.observe(lines_wrapper_dom_node) return () => { lines_wrapper_resize_observer.unobserve(lines_wrapper_dom_node) } } }, [show_static_fake]) useEffect(() => { if (newcm_ref.current == null) return const cm = newcm_ref.current const diagnostics = cm_diagnostics cm.dispatch(setDiagnostics(cm.state, diagnostics)) }, [cm_diagnostics]) // Effect to apply "remote_code" to the cell when it changes... // ideally this won't be necessary as we'll have actual multiplayer, // or something to tell the user that the cell is out of sync. useEffect(() => { if (newcm_ref.current == null) return // Not sure when and why this gave an error, but now it doesn't const current_value = getValue6(newcm_ref.current) ?? "" if (remote_code_ref.current == null && remote_code === "" && current_value !== "") { // this cell is being initialized with empty code, but it already has local code set. // this happens when pasting or dropping cells return } remote_code_ref.current = remote_code if (current_value !== remote_code) { setValue6(newcm_ref.current, remote_code) } }, [remote_code]) useEffect(() => { const cm = newcm_ref.current if (cm == null) return if (cm_forced_focus == null) { cm.dispatch({ selection: { anchor: cm.state.selection.main.head, head: cm.state.selection.main.head, }, }) } else if (cm_forced_focus === true) { } else { let new_selection = { anchor: line_and_ch_to_cm6_position(cm.state.doc, cm_forced_focus[0]), head: line_and_ch_to_cm6_position(cm.state.doc, cm_forced_focus[1]), } if (cm_forced_focus[2]?.definition_of) { let scopestate = cm.state.field(ScopeStateField) let definition = scopestate?.definitions.get(cm_forced_focus[2]?.definition_of) if (definition) { new_selection = { anchor: definition.from, head: definition.to, } } } let dom = /** @type {HTMLElement} */ (cm.dom) dom.scrollIntoView({ behavior: "smooth", block: "nearest", // UNCOMMENT THIS AND SEE, this feels amazing but I feel like people will not like it // block: "center", }) cm.focus() cm.dispatch({ scrollIntoView: true, selection: new_selection, effects: [ EditorView.scrollIntoView(EditorSelection.range(new_selection.anchor, new_selection.head), { yMargin: 80, }), LastFocusWasForcedEffect.of(true), ], }) } }, [cm_forced_focus]) return html` <pluto-input ref=${dom_node_ref} class="CodeMirror" translate=${false}> ${show_static_fake ? (show_input ? html`<${StaticCodeMirrorFaker} value=${remote_code} />` : null) : null} <${InputContextMenu} on_delete=${on_delete} cell_id=${cell_id} run_cell=${on_submit} skip_as_script=${skip_as_script} running_disabled=${running_disabled} any_logs=${any_logs} show_logs=${show_logs} set_show_logs=${set_show_logs} set_cell_disabled=${set_cell_disabled} get_current_code=${() => { let cm = newcm_ref.current return cm == null ? "" : getValue6(cm) }} /> ${PreviewHiddenCode} </pluto-input> ` } const PreviewHiddenCode = html`<div class="preview_hidden_code_info"> Reading hidden code</div>` const InputContextMenu = ({ on_delete, cell_id, run_cell, skip_as_script, running_disabled, any_logs, show_logs, set_show_logs, set_cell_disabled, get_current_code, }) => { const timeout = useRef(null) let pluto_actions = useContext(PlutoActionsContext) const [open, setOpenState] = useState(false) const button_ref = useRef(/** @type {HTMLButtonElement?} */ (null)) const list_ref = useRef(/** @type {HTMLButtonElement?} */ (null)) const prevously_focused_element_ref = useRef(/** @type {Element?} */ (null)) const setOpen = (val) => { if (val) { prevously_focused_element_ref.current = document.activeElement } setOpenState(val) } useLayoutEffect(() => { if (open) { list_ref.current?.querySelector("button")?.focus() } else { let e = prevously_focused_element_ref.current if (e instanceof HTMLElement) e.focus() } }, [open]) const mouseenter = () => { if (timeout.current) clearTimeout(timeout.current) } const toggle_skip_as_script = async (e) => { const new_val = !skip_as_script e.preventDefault() // e.stopPropagation() await pluto_actions.update_notebook((notebook) => { notebook.cell_inputs[cell_id].metadata["skip_as_script"] = new_val }) } const toggle_running_disabled = async (e) => { const new_val = !running_disabled await set_cell_disabled(new_val) } const toggle_logs = () => set_show_logs(!show_logs) const is_copy_output_supported = () => { let notebook = /** @type{import("./Editor.js").NotebookData?} */ (pluto_actions.get_notebook()) let cell_result = notebook?.cell_results?.[cell_id] if (cell_result == null) return false return ( (!cell_result.errored && cell_result.output.mime === "text/plain" && cell_result.output.body != null) || (cell_result.errored && cell_result.output.mime === "application/vnd.pluto.stacktrace+object") ) } const copy_output = () => { let notebook = /** @type{import("./Editor.js").NotebookData?} */ (pluto_actions.get_notebook()) let cell_result = notebook?.cell_results?.[cell_id] if (cell_result == null) return let cell_output = cell_result.output.mime === "text/plain" ? cell_result.output.body : // @ts-ignore cell_result.output.body.plain_error if (cell_output != null) navigator.clipboard.writeText(cell_output).catch(() => { alert(`Error copying cell output`) }) } const ask_ai = () => { open_pluto_popup({ type: "info", big: true, css_class: "ai-context", should_focus: true, // source_element: button_ref.current, body: html`<${AIContext} cell_id=${cell_id} current_code=${get_current_code()} />`, }) } useEventListener( window, "keydown", (/** @type {KeyboardEvent} */ e) => { if (e.key === "Escape") { setOpen(false) } }, [] ) return html` <button onClick=${(e) => { setOpen(!open) }} class=${cl({ input_context_menu: true, open, })} title="Actions" ref=${button_ref} > <span class="icon"></span> </button> <div class=${cl({ input_context_menu: true, open, })} ref=${list_ref} onfocusout=${(e) => { const li_focused = list_ref.current?.matches(":focus-within") || list_ref.current?.contains(e.relatedTarget) if ( !li_focused || // or the focus is on the list itself e.relatedTarget === list_ref.current ) setOpen(false) }} > ${open ? html`<ul onMouseenter=${mouseenter}> <${InputContextMenuItem} tag="delete" contents="Delete cell" title="Delete cell" onClick=${on_delete} setOpen=${setOpen} /> <${InputContextMenuItem} title=${running_disabled ? "Enable and run the cell" : "Disable this cell, and all cells that depend on it"} tag=${running_disabled ? "enable_cell" : "disable_cell"} contents=${running_disabled ? html`<b>Enable cell</b>` : html`Disable cell`} onClick=${toggle_running_disabled} setOpen=${setOpen} /> ${any_logs ? html`<${InputContextMenuItem} title=${show_logs ? "Show cell logs" : "Hide cell logs"} tag=${show_logs ? "hide_logs" : "show_logs"} contents=${show_logs ? "Hide logs" : "Show logs"} onClick=${toggle_logs} setOpen=${setOpen} />` : null} ${is_copy_output_supported() ? html`<${InputContextMenuItem} tag="copy_output" contents="Copy output" title="Copy the output of this cell to the clipboard." onClick=${copy_output} setOpen=${setOpen} />` : null} <${InputContextMenuItem} title=${skip_as_script ? "This cell is currently stored in the notebook file as a Julia comment. Click here to disable." : "Store this code in the notebook file as a Julia comment. This way, it will not run when the notebook runs as a script outside of Pluto."} tag=${skip_as_script ? "run_as_script" : "skip_as_script"} contents=${skip_as_script ? html`<b>Enable in file</b>` : html`Disable in file`} onClick=${toggle_skip_as_script} setOpen=${setOpen} /> ${pluto_actions.get_session_options?.()?.server?.enable_ai_editor_features !== false ? html`<${InputContextMenuItem} tag="ask_ai" contents="Ask AI" title="Ask AI about this cell" onClick=${ask_ai} setOpen=${setOpen} />` : null} </ul>` : html``} </div> ` } const InputContextMenuItem = ({ contents, title, onClick, setOpen, tag }) => html`<li> <button tabindex="0" title=${title} onClick=${(e) => { setOpen(false) onClick(e) }} class=${tag} > <span class=${`${tag} ctx_icon`} />${contents} </button> </li>` const StaticCodeMirrorFaker = ({ value }) => { const lines = value.split("\n").map((line, i) => { const start_tabs = get_start_tabs(line) const tabbed_line = start_tabs.length == 0 ? line : html`<span class="awesome-wrapping-plugin-the-tabs"><span class="o">${start_tabs}</span></span >${line.substring(start_tabs.length)}` return html`<div class="awesome-wrapping-plugin-the-line cm-line" style="--indented: ${4 * start_tabs.length}ch;"> ${line.length === 0 ? html`<br />` : tabbed_line} </div>` }) return html` <div class="cm-editor 1 2 4 4z cm-ssr-fake"> <div tabindex="-1" class="cm-scroller"> <div class="cm-gutters" aria-hidden="true"> <div class="cm-gutter cm-lineNumbers"></div> </div> <div spellcheck="false" autocorrect="off" autocapitalize="off" translate="no" contenteditable="false" style="tab-size: 4;" class="cm-content cm-lineWrapping" role="textbox" aria-multiline="true" aria-autocomplete="list" > ${lines} </div> </div> </div> ` } import { html, Component, useRef, useLayoutEffect, useContext } from "../imports/Preact.js" import DOMPurify from "../imports/DOMPurify.js" import { ansi_to_html } from "../imports/AnsiUp.js" import { ErrorMessage, ParseError } from "./ErrorMessage.js" import { TreeView, TableView, DivElement } from "./TreeView.js" import { add_bonds_disabled_message_handler, add_bonds_listener, set_bound_elements_to_their_value, get_input_value, set_input_value, eventof, } from "../common/Bond.js" import { cl } from "../common/ClassTable.js" import { observablehq_for_cells } from "../common/SetupCellEnvironment.js" import { PlutoBondsContext, PlutoActionsContext, PlutoJSInitializingContext } from "../common/PlutoContext.js" import register from "../imports/PreactCustomElement.js" import { EditorState, EditorView, defaultHighlightStyle, syntaxHighlighting } from "../imports/CodemirrorPlutoSetup.js" import { pluto_syntax_colors_julia, ENABLE_CM_MIXED_PARSER } from "./CellInput.js" import hljs from "../imports/highlightjs.js" import { julia_mixed } from "./CellInput/mixedParsers.js" import { julia } from "../imports/CodemirrorPlutoSetup.js" import { SafePreviewSanitizeMessage } from "./SafePreviewUI.js" import lodashLibrary from "../imports/lodash.js" const prettyAssignee = (assignee) => assignee && assignee.startsWith("const ") ? html`<span style="color: var(--cm-color-keyword)">const</span> ${assignee.slice(6)}` : assignee export class CellOutput extends Component { constructor() { super() this.state = { output_changed_once: false, } this.old_height = 0 // @ts-ignore Is there a way to use the latest DOM spec? this.resize_observer = new ResizeObserver((entries) => { const new_height = this.base.offsetHeight // Scroll the page to compensate for change in page height: if (document.body.querySelector("pluto-cell:focus-within")) { const cell_outputs_after_focused = document.body.querySelectorAll("pluto-cell:focus-within ~ pluto-cell > pluto-output") // CSS wizardry  if ( !(document.activeElement?.tagName === "SUMMARY") && (cell_outputs_after_focused.length === 0 || !Array.from(cell_outputs_after_focused).includes(this.base)) ) { window.scrollBy(0, new_height - this.old_height) } } this.old_height = new_height }) } shouldComponentUpdate({ last_run_timestamp, sanitize_html }) { return last_run_timestamp !== this.props.last_run_timestamp || sanitize_html !== this.props.sanitize_html } componentDidUpdate(old_props) { if (this.props.last_run_timestamp !== old_props.last_run_timestamp) { this.setState({ output_changed_once: true }) } } componentDidMount() { this.resize_observer.observe(this.base) } componentWillUnmount() { this.resize_observer.unobserve(this.base) } render() { const rich_output = this.props.errored || !this.props.body || (this.props.mime !== "application/vnd.pluto.tree+object" && this.props.mime !== "application/vnd.pluto.table+object" && this.props.mime !== "text/plain") const allow_translate = !this.props.errored && rich_output return html` <pluto-output class=${cl({ rich_output, scroll_y: this.props.mime === "application/vnd.pluto.table+object" || this.props.mime === "text/plain", })} translate=${allow_translate} mime=${this.props.mime} aria-live=${this.state.output_changed_once ? "polite" : "off"} aria-atomic="true" aria-relevant="all" aria-label=${this.props.rootassignee == null ? "Result of unlabeled cell:" : `Result of variable ${this.props.rootassignee}:`} > <assignee aria-hidden="true" translate=${false}>${prettyAssignee(this.props.rootassignee)}</assignee> <${OutputBody} ...${this.props} /> </pluto-output> ` } } export let PlutoImage = ({ body, mime }) => { // I know I know, this looks stupid. // BUT it is necessary to make sure the object url is only created when we are actually attaching to the DOM, // and is removed when we are detatching from the DOM let imgref = useRef() useLayoutEffect(() => { let url = URL.createObjectURL(new Blob([body], { type: mime })) imgref.current.onload = imgref.current.onerror = () => { if (imgref.current) { imgref.current.style.display = null } } if (imgref.current.src === "") { // an <img> that is loading takes up 21 vertical pixels, which causes a 1-frame scroll flicker // the solution is to make the <img> invisible until the image is loaded imgref.current.style.display = "none" } imgref.current.type = mime imgref.current.src = url return () => URL.revokeObjectURL(url) }, [body, mime]) return html`<img ref=${imgref} type=${mime} src=${""} />` } /** * @param {{ * mime: string, * body: any, * cell_id: string, * persist_js_state: boolean | string, * last_run_timestamp: number?, * sanitize_html?: boolean | string, * }} args */ export const OutputBody = ({ mime, body, cell_id, persist_js_state = false, last_run_timestamp, sanitize_html = true }) => { // These two arguments might have been passed as strings if OutputBody was used as the custom HTML element <pluto-display>, with string attributes as arguments. sanitize_html = sanitize_html !== "false" && sanitize_html !== false persist_js_state = persist_js_state === "true" || persist_js_state === true switch (mime) { case "image/png": case "image/jpg": case "image/jpeg": case "image/gif": case "image/bmp": case "image/svg+xml": return html`<div><${PlutoImage} mime=${mime} body=${body} /></div>` break case "text/html": // Snippets starting with <!DOCTYPE or <html are considered "full pages" that get their own iframe. // Not entirely sure if this works the best, or if this slows down notebooks with many plots. // AFAIK JSServe and Plotly both trigger this code. // NOTE: Jupyter doesn't do this, jupyter renders everything directly in pages DOM. // -DRAL if (body.startsWith("<!DOCTYPE") || body.startsWith("<html")) { return sanitize_html ? null : html`<${IframeContainer} body=${body} />` } else { return html`<${RawHTMLContainer} cell_id=${cell_id} body=${body} persist_js_state=${persist_js_state} last_run_timestamp=${last_run_timestamp} sanitize_html=${sanitize_html} />` } break case "application/vnd.pluto.tree+object": return html`<div> <${TreeView} cell_id=${cell_id} body=${body} persist_js_state=${persist_js_state} sanitize_html=${sanitize_html} /> </div>` break case "application/vnd.pluto.table+object": return html`<${TableView} cell_id=${cell_id} body=${body} persist_js_state=${persist_js_state} sanitize_html=${sanitize_html} />` break case "application/vnd.pluto.parseerror+object": return html`<div><${ParseError} cell_id=${cell_id} last_run_timestamp=${last_run_timestamp} ...${body} /></div>` break case "application/vnd.pluto.stacktrace+object": return html`<div><${ErrorMessage} cell_id=${cell_id} ...${body} /></div>` break case "application/vnd.pluto.divelement+object": return DivElement({ cell_id, ...body, persist_js_state, sanitize_html }) break case "text/plain": if (body) { return html`<div><${ANSITextOutput} body=${body} /></div>` } else { return html`<div></div>` } break case null: case undefined: case "": return html`` break default: return html`<pre title="Something went wrong displaying this object"></pre>` break } } register(OutputBody, "pluto-display", ["mime", "body", "cell_id", "persist_js_state", "last_run_timestamp", "sanitize_html"]) let IframeContainer = ({ body }) => { let iframeref = useRef() useLayoutEffect(() => { let url = URL.createObjectURL(new Blob([body], { type: "text/html" })) iframeref.current.src = url run(async () => { await new Promise((resolve) => iframeref.current.addEventListener("load", () => resolve(null))) /** @type {Document} */ let iframeDocument = iframeref.current.contentWindow.document /** Grab the <script> tag for the iframe content window resizer */ let original_script_element = /** @type {HTMLScriptElement} */ (document.querySelector("#iframe-resizer-content-window-script")) // Insert iframe resizer inside the iframe let iframe_resizer_content_script = iframeDocument.createElement("script") iframe_resizer_content_script.src = original_script_element.src iframe_resizer_content_script.crossOrigin = "anonymous" iframeDocument.head.appendChild(iframe_resizer_content_script) // Apply iframe resizer from the host side new Promise((resolve) => iframe_resizer_content_script.addEventListener("load", () => resolve(null))) // @ts-ignore window.iFrameResize({ checkOrigin: false }, iframeref.current) }) return () => URL.revokeObjectURL(url) }, [body]) return html`<iframe style=${{ width: "100%", border: "none" }} src="" ref=${iframeref} frameborder="0" allow="accelerometer; ambient-light-sensor; autoplay; battery; camera; display-capture; document-domain; encrypted-media; execution-while-not-rendered; execution-while-out-of-viewport; fullscreen; geolocation; gyroscope; layout-animations; legacy-image-formats; magnetometer; microphone; midi; navigation-override; oversized-images; payment; picture-in-picture; publickey-credentials-get; sync-xhr; usb; wake-lock; screen-wake-lock; vr; web-share; xr-spatial-tracking" allowfullscreen ></iframe>` } /** * Call a block of code with with environment inserted as local bindings (even this) * * @param {{ code: string, environment: { [name: string]: any } }} options */ let execute_dynamic_function = async ({ environment, code }) => { // single line so that we don't affect line numbers in the stack trace const wrapped_code = `"use strict"; return (async () => {${code}})()` let { ["this"]: this_value, ...args } = environment let arg_names = Object.keys(args) let arg_values = Object.values(args) const result = await Function(...arg_names, wrapped_code).bind(this_value)(...arg_values) return result } /** * It is possible for `execute_scripttags` to run during the execution of `execute_scripttags`, and this variable counts the depth of this nesting. * * One case where nesting occurs is when using PlutoRunner.embed_display. In its HTML render, it outputs a `<script>`, which will render a `<pluto-display>` element with content. If that content contains a `<script>` tag, then it will be executed during the execution of the original script, etc. * * See https://github.com/fonsp/Pluto.jl/pull/2329 */ let nested_script_execution_level = 0 /** * Runs the code `fn` with `document.currentScript` being set to a new script_element thats * is placed on the page where `script_element` was. * * Why? So we can run the javascript code with extra cool Pluto variables and return value, * but still have a script at the same position as `document.currentScript`. * This way you can do `document.currentScript.insertBefore()` and have it work! * * This will remove the passed in `script_element` from the DOM! * * @param {HTMLOrSVGScriptElement} script_element * @param {() => any} fn */ let execute_inside_script_tag_that_replaces = async (script_element, fn) => { // Mimick as much as possible from the original script (only attributes but sure) let new_script_tag = document.createElement("script") for (let attr of script_element.attributes) { //@ts-ignore because of https://github.com/microsoft/TypeScript-DOM-lib-generator/issues/1260 new_script_tag.attributes.setNamedItem(attr.cloneNode(true)) } const container_name = `____FUNCTION_TO_RUN_INSIDE_SCRIPT_${nested_script_execution_level}` new_script_tag.textContent = `{ window.${container_name}.result = window.${container_name}.function_to_run(window.${container_name}.currentScript) }` // @ts-ignore // I use this long variable name to pass the function and result to and from the script we created window[container_name] = { function_to_run: fn, currentScript: new_script_tag, result: null } // Put the script in the DOM, this will run the script const parent = script_element.parentNode if (parent == null) { throw "Failed to execute script it has no parent in DOM." } parent.replaceChild(new_script_tag, script_element) // @ts-ignore - Get the result back let result = await window[container_name].result // @ts-ignore - Reset the global variable "just in case" window[container_name] = { function_to_run: fn, result: null } return { node: new_script_tag, result: result } } const is_displayable = (result) => result instanceof Element && result.nodeType === Node.ELEMENT_NODE /** * @typedef {HTMLScriptElement} PlutoScript * @property {boolean?} pluto_is_loading_me */ /** * * @param {{ * root_node: HTMLElement, * script_nodes: Array<PlutoScript>, * previous_results_map: Map, * invalidation: Promise<void>, * pluto_actions: any, * }} param0 * @returns */ const execute_scripttags = async ({ root_node, script_nodes, previous_results_map, invalidation, pluto_actions }) => { let results_map = new Map() // Reattach DOM results from old scripts, you might want to skip reading this for (let node of script_nodes) { if (node.src != null && node.src !== "") { } else { let script_id = node.id let old_result = script_id ? previous_results_map.get(script_id) : null if (is_displayable(old_result)) { node.parentElement?.insertBefore(old_result, node) } } } // Run scripts sequentially for (let node of script_nodes) { nested_script_execution_level += 1 if (node.src != null && node.src !== "") { // If it has a remote src="", de-dupe and copy the script to head let script_el = Array.from(document.head.querySelectorAll("script")).find((s) => s.src === node.src) if (script_el == undefined) { script_el = document.createElement("script") script_el.referrerPolicy = node.referrerPolicy script_el.crossOrigin = node.crossOrigin script_el.integrity = node.integrity script_el.noModule = node.noModule script_el.nonce = node.nonce script_el.type = node.type script_el.src = node.src // Not copying defer or async because this script is not included in the initial HTML document, so it has no effect. // @ts-ignore script_el.pluto_is_loading_me = true } let script_el_really = script_el // for typescript // @ts-ignore const need_to_await = script_el_really.pluto_is_loading_me != null if (need_to_await) { await new Promise((resolve) => { script_el_really.addEventListener("load", resolve) script_el_really.addEventListener("error", resolve) document.head.appendChild(script_el_really) }) // @ts-ignore script_el_really.pluto_is_loading_me = undefined } } else { // If there is no src="", we take the content and run it in an observablehq-like environment try { let code = node.innerText let script_id = node.id let old_result = script_id ? previous_results_map.get(script_id) : null if (node.type === "module") { console.warn("We don't (yet) fully support <script type=module> (loading modules with <script type=module src=...> is fine).") } if (node.type === "" || node.type === "text/javascript" || node.type === "module") { if (is_displayable(old_result)) { node.parentElement?.insertBefore(old_result, node) } const cell = root_node.closest("pluto-cell") let { node: new_node, result } = await execute_inside_script_tag_that_replaces(node, async (currentScript) => { return await execute_dynamic_function({ environment: { this: script_id ? old_result : window, currentScript: currentScript, invalidation: invalidation, // @ts-ignore getPublishedObject: (id) => cell.getPublishedObject(id), _internal_getJSLinkResponse: (cell_id, link_id) => (input) => pluto_actions.request_js_link_response(cell_id, link_id, input).then(([success, result]) => { if (success) return result throw result }), getBoundElementValueLikePluto: get_input_value, setBoundElementValueLikePluto: set_input_value, getBoundElementEventNameLikePluto: eventof, getNotebookMetadataExperimental: (key) => pluto_actions.get_notebook()?.metadata?.[key], setNotebookMetadataExperimental: (key, value) => pluto_actions.update_notebook((notebook) => { notebook.metadata[key] = value }), deleteNotebookMetadataExperimental: (key) => pluto_actions.update_notebook((notebook) => { delete notebook.metadata[key] }), ...(cell == null ? {} : { getCellMetadataExperimental: (key, { cell_id = null } = {}) => pluto_actions.get_notebook()?.cell_inputs?.[cell_id ?? cell.id]?.metadata?.[key], setCellMetadataExperimental: (key, value, { cell_id = null } = {}) => pluto_actions.update_notebook((notebook) => { notebook.cell_inputs[cell_id ?? cell.id].metadata[key] = value }), deleteCellMetadataExperimental: (key, { cell_id = null } = {}) => pluto_actions.update_notebook((notebook) => { delete notebook.cell_inputs[cell_id ?? cell.id].metadata[key] }), }), ...observablehq_for_cells, _: lodashLibrary, }, code, }) }) // Save result for next run if (script_id != null) { results_map.set(script_id, result) } // Insert returned element if (result !== old_result) { if (is_displayable(old_result)) { old_result.remove() } if (is_displayable(result)) { new_node.parentElement?.insertBefore(result, new_node) } } } } catch (err) { console.error("Couldn't execute script:", node) // needs to be in its own console.error so that the stack trace is printed console.error(err) // TODO: relay to user } } nested_script_execution_level -= 1 } return results_map } let run = (f) => f() /** * Support declarative shadowroot  * https://web.dev/declarative-shadow-dom/ * The polyfill they mention on the page is nice and all, but we need more. * For one, we need the polyfill anyway as we're adding html using innerHTML (just like we need to run the scripts ourselves) * Also, we want to run the scripts inside the shadow roots, ideally in the same order that a browser would. * And we want nested shadowroots, which their polyfill doesn't provide (and I hope the spec does) * * @param {HTMLTemplateElement} template */ let declarative_shadow_dom_polyfill = (template) => { try { const mode = template.getAttribute("shadowroot") // @ts-ignore const shadowRoot = template.parentElement.attachShadow({ mode }) // @ts-ignore shadowRoot.appendChild(template.content) template.remove() // To mimick as much as possible the browser behavior, I const scripts_or_shadowroots = Array.from(shadowRoot.querySelectorAll("script, template[shadowroot]")) return scripts_or_shadowroots.flatMap((script_or_shadowroot) => { if (script_or_shadowroot.nodeName === "SCRIPT") { return [script_or_shadowroot] } else if (script_or_shadowroot.nodeName === "TEMPLATE") { // @ts-ignore return declarative_shadow_dom_polyfill(script_or_shadowroot) } }) } catch (error) { console.error(`Couldn't attach declarative shadow dom to`, template, `because of`, error) return [] } } export let RawHTMLContainer = ({ body, className = "", persist_js_state = false, last_run_timestamp, sanitize_html = true, sanitize_html_message = true }) => { let pluto_actions = useContext(PlutoActionsContext) let pluto_bonds = useContext(PlutoBondsContext) let js_init_set = useContext(PlutoJSInitializingContext) let previous_results_map = useRef(new Map()) let invalidate_scripts = useRef(() => {}) let container_ref = useRef(/** @type {HTMLElement?} */ (null)) useLayoutEffect(() => { if (container_ref.current && pluto_bonds) set_bound_elements_to_their_value(container_ref.current.querySelectorAll("bond"), pluto_bonds) }, [body, persist_js_state, pluto_actions, pluto_bonds, sanitize_html]) useLayoutEffect(() => { const container = container_ref.current if (container == null) return // Invalidate current scripts and create a new invalidation token immediately let invalidation = new Promise((resolve) => { invalidate_scripts.current = () => { resolve(null) } }) const dump = document.createElement("p-dumpster") // @ts-ignore dump.append(...container.childNodes) let html_content_to_set = sanitize_html ? DOMPurify.sanitize(body, { FORBID_TAGS: ["style"], ADD_ATTR: ["target"], }) : body // Actually "load" the html container.innerHTML = html_content_to_set if (sanitize_html_message && html_content_to_set !== body) { // DOMPurify also resolves HTML entities, which can give a false positive. To fix this, we use DOMParser to parse both strings, and we compare the innerHTML of the resulting documents. const parser = new DOMParser() const p1 = parser.parseFromString(body, "text/html") const p2 = parser.parseFromString(html_content_to_set, "text/html") if (p2.documentElement.innerHTML !== p1.documentElement.innerHTML) { console.info("HTML sanitized", { body, html_content_to_set }) let info_element = document.createElement("div") info_element.innerHTML = SafePreviewSanitizeMessage container.prepend(info_element) } } if (sanitize_html) return let scripts_in_shadowroots = Array.from(container.querySelectorAll("template[shadowroot]")).flatMap((template) => { // @ts-ignore return declarative_shadow_dom_polyfill(template) }) // do this synchronously after loading HTML const new_scripts = [...scripts_in_shadowroots, ...Array.from(container.querySelectorAll("script"))] run(async () => { try { js_init_set?.add(container) previous_results_map.current = await execute_scripttags({ root_node: container, script_nodes: new_scripts, invalidation, previous_results_map: persist_js_state ? previous_results_map.current : new Map(), pluto_actions, }) if (pluto_actions != null) { const on_bond_value = (name, value) => pluto_actions?.set_bond?.(name, value) ?? Promise.resolve() const bond_nodes = container.querySelectorAll("bond") set_bound_elements_to_their_value(bond_nodes, pluto_bonds ?? {}) add_bonds_listener(bond_nodes, on_bond_value, pluto_bonds ?? {}, invalidation) add_bonds_disabled_message_handler(bond_nodes, invalidation) } // Convert LaTeX to svg // @ts-ignore if (window.MathJax?.typeset != undefined) { try { // @ts-ignore window.MathJax.typeset(container.querySelectorAll(".tex")) } catch (err) { console.info("Failed to typeset TeX:") console.info(err) } } // Apply syntax highlighting try { container.querySelectorAll("code").forEach((code_element) => { code_element.classList.forEach((className) => { if (className.startsWith("language-") && !className.endsWith("undefined")) { // Remove "language-" let language = className.substring(9) highlight(code_element, language) } }) }) } catch (err) { console.warn("Highlighting failed", err) } // Find code blocks and add a copy button: try { if (container.firstElementChild?.matches("div.markdown")) { container.querySelectorAll("pre > code").forEach((code_element) => { const pre = code_element.parentElement generateCopyCodeButton(pre) }) container.querySelectorAll("h1, h2, h3, h4, h5, h6").forEach((header_element) => { if (header_element.closest("table, pluto-display, bond")) return generateCopyHeaderIdButton(/** @type {HTMLHeadingElement} */ (header_element), pluto_actions) }) } } catch (err) { console.warn("Adding markdown code copy button failed", err) } } finally { js_init_set?.delete(container) } }) return () => { js_init_set?.delete(container) invalidate_scripts.current?.() } }, [body, last_run_timestamp, pluto_actions, sanitize_html]) return html`<div class="raw-html-wrapper ${className}" ref=${container_ref}></div>` } // https://github.com/fonsp/Pluto.jl/issues/1692 const ENABLE_CM_HIGHLIGHTING = false /** @param {HTMLElement} code_element */ export let highlight = (code_element, language) => { language = language.toLowerCase() language = language === "jl" ? "julia" : language if (code_element.children.length === 0) { if ( ENABLE_CM_HIGHLIGHTING && language === "julia" && // CodeMirror does not want to render inside a `<details>`... // I tried to debug this, it does not happen on a clean webpage with the same CM versions: // https://glitch.com/edit/#!/wobbly-sweet-fibre?path=script.js%3A51%3A76 code_element.closest("details") == null ) { const editorview = new EditorView({ state: EditorState.create({ // Remove references to `Main.workspace#xx.` in the docs since // its shows up as a comment and can be confusing doc: code_element.innerText .trim() .replace(/Main.var\"workspace#\d+\"\./, "") .replace(/Main.workspace#\d+\./, "") .replace(/Main.workspace#(\d+)/, 'Main.var"workspace#$1"'), extensions: [ syntaxHighlighting(pluto_syntax_colors_julia), syntaxHighlighting(defaultHighlightStyle, { fallback: true }), EditorState.tabSize.of(4), // TODO Other languages possibly? ...(language === "julia" ? [ENABLE_CM_MIXED_PARSER ? julia_mixed() : julia()] : []), EditorView.lineWrapping, EditorView.editable.of(false), ].filter((x) => x != null), }), }) code_element.replaceChildren(editorview.dom) // Weird hack to make it work inline  // Probably should be using [HighlightTree](https://codemirror.net/6/docs/ref/#highlight.highlightTree) editorview.dom.style.setProperty("display", "inline-flex", "important") editorview.dom.style.setProperty("background-color", "transparent", "important") } else { if (language === "htmlmixed") { code_element.classList.remove("language-htmlmixed") code_element.classList.add("language-html") } hljs.highlightElement(code_element) } } } /** * Generates a copy button for Markdown code blocks. */ export const generateCopyCodeButton = (/** @type {HTMLElement?} */ pre) => { if (!pre) return // create copy button const button = document.createElement("button") button.title = "Copy to clipboard" button.className = "markdown-code-block-button" button.addEventListener("click", (e) => { const txt = pre.textContent ?? "" navigator.clipboard.writeText(txt) button.classList.add("recently-copied") setTimeout(() => { button.classList.remove("recently-copied") }, 1300) }) // Append copy button to the code block element pre.prepend(button) } /** * Generates a copy button for Markdown header elements, to copy the URL to this header using the `id`. */ export const generateCopyHeaderIdButton = (/** @type {HTMLHeadingElement} */ header, /** @type {any} */ pluto_actions) => { const id = header.id if (!id) return const button = document.createElement("pluto-header-id-copy") button.title = "Click to copy URL to this header" button.ariaLabel = "Copy URL to this header" button.role = "button" button.tabIndex = 0 const listener = (_e) => { const id = header.id if (!id) return let url_to_copy = `#${id}` const launch_params = /** @type {import("./Editor.js").LaunchParameters?} */ (pluto_actions?.get_launch_params?.()) if (!launch_params) return if (launch_params.isolated_cell_ids != null) return const root = new URL(window.location.href) root.hash = "" const is_localhost_hostname = (hostname) => hostname === "localhost" || hostname === "127.0.0.1" || hostname === "0.0.0.0" if (launch_params.disable_ui && launch_params.notebook_id == null && launch_params.pluto_server_url == null && !is_localhost_hostname(root.hostname)) { url_to_copy = `${root.href}${url_to_copy}` } navigator.clipboard.writeText(url_to_copy) button.classList.add("recently-copied") setTimeout(() => { button.classList.remove("recently-copied") }, 1300) } button.addEventListener("click", listener) button.addEventListener("keydown", (e) => { if (e.key !== "Enter" && e.key !== " ") return listener(e) e.preventDefault() }) header.append(button) } export const ANSITextOutput = ({ body }) => { const has_ansi = /\x1b\[\d+m/.test(body) if (has_ansi) { return html`<${ANSIUpContents} body=${body} />` } else { return html`<pre class="no-block"><code>${body}</code></pre>` } } const ANSIUpContents = ({ body }) => { const node_ref = useRef(/** @type {HTMLElement?} */ (null)) useLayoutEffect(() => { if (!node_ref.current) return node_ref.current.innerHTML = ansi_to_html(body) }, [body]) return html`<pre class="no-block"><code ref=${node_ref}></code></pre>` } import { cl } from "../common/ClassTable.js" import { html, useEffect, useRef, useState } from "../imports/Preact.js" export const DiscreteProgressBar = ({ onClick, total, done, busy, failed_indices }) => { total = Math.max(1, total) return html` <div class=${cl({ "discrete-progress-bar": true, "small": total < 8, "mid": total >= 8 && total < 48, "big": total >= 48, })} data-total=${total} onClick=${onClick} > ${[...Array(total)].map((_, i) => { return html`<div class=${cl({ done: i < done, failed: failed_indices.includes(i), busy: i >= done && i < done + busy, })} ></div>` })} </div> ` } export const DiscreteProgressBarTest = () => { const [done_total, set_done_total] = useState([0, 0, 0]) const done_total_ref = useRef(done_total) done_total_ref.current = done_total useEffect(() => { let handle = setInterval(() => { const [done, busy, total] = done_total_ref.current if (Math.random() < 0.3) { if (done < total) { if (Math.random() < 0.1) { set_done_total([done, 1, total + 5]) } else { set_done_total([done + 1, 1, total]) } } else { set_done_total([0, 1, Math.ceil(Math.random() * Math.random() * 100)]) } } }, 100) return () => clearInterval(handle) }, []) return html`<${DiscreteProgressBar} total=${done_total[2]} busy=${done_total[1]} done=${done_total[0]} />` } import { html, Component } from "../imports/Preact.js" import _ from "../imports/lodash.js" /** * @typedef DropRulerProps * @type {{ * actions: any, * selected_cells: string[], * set_scroller: (enabled: any) => void * serialize_selected: (id: string) => string | undefined, * pluto_editor_element: HTMLElement, * }} */ /** * @augments Component<DropRulerProps,any> */ export class DropRuler extends Component { constructor(/** @type {DropRulerProps} */ props) { super(props) this.dropee = null this.dropped = null this.cell_edges = [] this.pointer_position = { pageX: 0, pageY: 0 } this.precompute_cell_edges = () => { /** @type {Array<HTMLElement>} */ const cell_nodes = Array.from(this.props.pluto_editor_element.querySelectorAll(":scope > main > pluto-notebook > pluto-cell")) this.cell_edges = cell_nodes.map((el) => el.offsetTop) this.cell_edges.push(last(cell_nodes).offsetTop + last(cell_nodes).scrollHeight) } this.getDropIndexOf = ({ pageX, pageY }) => { const editorY = pageY - ((this.props.pluto_editor_element.querySelector("main") ?? this.props.pluto_editor_element).getBoundingClientRect().top + document.documentElement.scrollTop) const distances = this.cell_edges.map((p) => Math.abs(p - editorY - 8)) // 8 is the magic computer number: https://en.wikipedia.org/wiki/8 return argmin(distances) } this.state = { drag_start: false, drag_target: false, drop_index: 0, } } componentDidMount() { const event_not_for_me = (/** @type {MouseEvent} */ e) => { return (e.target instanceof Element ? e.target.closest("pluto-editor") : null) !== this.props.pluto_editor_element } document.addEventListener("dragstart", (e) => { if (event_not_for_me(e)) return if (!e.dataTransfer) return let target = /** @type {Element} */ (e.target) let pe = target.parentElement if (target.matches("pluto-shoulder") && pe != null) { this.dropee = pe let data = this.props.serialize_selected(pe.id) if (data) e.dataTransfer.setData("text/pluto-cell", data) this.dropped = false this.precompute_cell_edges() this.setState({ drag_start: true, drop_index: this.getDropIndexOf(e), }) this.props.set_scroller({ up: true, down: true }) } else { this.setState({ drag_start: false, drag_target: false, }) this.props.set_scroller({ up: false, down: false }) this.dropee = null } }) document.addEventListener("dragenter", (e) => { if (event_not_for_me(e)) return if (!e.dataTransfer) return if (e.dataTransfer.types[0] !== "text/pluto-cell") return if (!this.state.drag_target) this.precompute_cell_edges() this.lastenter = e.target this.setState({ drag_target: true }) e.preventDefault() }) document.addEventListener("dragleave", (e) => { if (event_not_for_me(e)) return if (!e.dataTransfer) return if (e.dataTransfer.types[0] !== "text/pluto-cell") return if (e.target === this.lastenter) { this.setState({ drag_target: false }) } }) const precompute_cell_edges_throttled = _.throttle(this.precompute_cell_edges, 4000, { leading: false, trailing: true }) const update_drop_index_throttled = _.throttle( () => { this.setState({ drop_index: this.getDropIndexOf(this.pointer_position), }) }, 100, { leading: false, trailing: true } ) document.addEventListener("dragover", (e) => { if (event_not_for_me(e)) return if (!e.dataTransfer) return // Called continuously during drag if (e.dataTransfer.types[0] !== "text/pluto-cell") return this.pointer_position = e precompute_cell_edges_throttled() update_drop_index_throttled() if (this.state.drag_start) { // Then we're dragging a cell from within the notebook. Use a move icon: e.dataTransfer.dropEffect = "move" } e.preventDefault() }) document.addEventListener("dragend", (e) => { if (event_not_for_me(e)) return // Called after drag, also when dropped outside of the browser or when ESC is pressed update_drop_index_throttled.flush() this.setState({ drag_start: false, drag_target: false, }) this.props.set_scroller({ up: false, down: false }) }) document.addEventListener("drop", (e) => { if (event_not_for_me(e)) return if (!e.dataTransfer) return // Guaranteed to fire before the 'dragend' event // Ignore files if (e.dataTransfer.types[0] !== "text/pluto-cell") { return } this.setState({ drag_target: false, }) this.dropped = true if (this.dropee && this.state.drag_start) { // Called when drag-dropped somewhere on the page const drop_index = this.getDropIndexOf(e) const friend_ids = this.props.selected_cells.includes(this.dropee.id) ? this.props.selected_cells : [this.dropee.id] this.props.actions.move_remote_cells(friend_ids, drop_index) } else { // Called when cell(s) from another window are dragged onto the page const drop_index = this.getDropIndexOf(e) const data = e.dataTransfer.getData("text/pluto-cell") this.props.actions.add_deserialized_cells(data, drop_index) } }) } render() { const styles = this.state.drag_target ? { display: "block", top: this.cell_edges[this.state.drop_index] + "px", } : {} return html`<dropruler style=${styles}></dropruler>` } } const argmin = (x) => { let best_val = Infinity let best_index = -1 let val for (let i = 0; i < x.length; i++) { val = x[i] if (val < best_val) { best_index = i best_val = val } } return best_index } const last = (x) => x[x.length - 1] import _ from "../imports/lodash.js" import { BackendLaunchPhase } from "../common/Binder.js" import { html, useEffect, useState, useRef, useLayoutEffect } from "../imports/Preact.js" import { has_ctrl_or_cmd_pressed } from "../common/KeyboardShortcuts.js" import { useDialog } from "../common/useDialog.js" export const RunLocalButton = ({ show, start_local }) => { //@ts-ignore window.open_edit_or_run_popup = () => { start_local() } return html`<div class="edit_or_run"> <button onClick=${(e) => { e.stopPropagation() e.preventDefault() start_local() }} > <b>Edit</b> or <b>run</b> this notebook </button> </div>` } /** * @param {{ * notebook: import("./Editor.js").NotebookData, * notebookfile: string?, * start_binder: () => Promise<void>, * offer_binder: boolean, * }} props * */ export const BinderButton = ({ offer_binder, start_binder, notebookfile, notebook }) => { const [dialog_ref, openModal, closeModal, toggleModal] = useDialog() const [showCopyPopup, setShowCopyPopup] = useState(false) const notebookfile_ref = useRef("") notebookfile_ref.current = notebookfile ?? "" //@ts-ignore window.open_edit_or_run_popup = openModal useEffect(() => { //@ts-ignore // allow user-written JS to start the binder window.start_binder = offer_binder ? start_binder : null return () => { //@ts-ignore window.start_binder = null } }, [start_binder, offer_binder]) const recommend_download = notebookfile_ref.current.startsWith("data:") const runtime_str = expected_runtime_str(notebook) return html`<div class="edit_or_run"> <button onClick=${(e) => { toggleModal() e.stopPropagation() e.preventDefault() }} > <b>Edit</b> or <b>run</b> this notebook </button> <dialog ref=${dialog_ref} class="binder_help_text"> <span onClick=${closeModal} class="close"></span> ${offer_binder ? html` <p style="text-align: center;"> ${`To be able to edit code and run cells, you need to run the notebook yourself. `} <b>Where would you like to run the notebook?</b> </p> ${runtime_str == null ? null : html` <div class="expected_runtime_box">${`This notebook takes about `}<span>${runtime_str}</span>${` to run.`}</div>`} <h2 style="margin-top: 3em;">In the cloud <em>(experimental)</em></h2> <div style="padding: 0 2rem;"> <button onClick=${start_binder}> <img src="https://cdn.jsdelivr.net/gh/jupyterhub/binderhub@0.2.0/binderhub/static/logo.svg" height="30" alt="binder" /> </button> </div> <p style="opacity: .5; margin: 20px 10px;"> <a target="_blank" href="https://mybinder.org/">Binder</a> is a free, open source service that runs scientific notebooks in the cloud! It will take a while, usually 2-7 minutes to get a session. </p> <h2 style="margin-top: 4em;">On your computer</h2> <p style="opacity: .5;">(Recommended if you want to store your changes.)</p> ` : null} <ol style="padding: 0 2rem;"> <li> <div> ${recommend_download ? html` <div class="command">Download the notebook:</div> <div onClick=${(e) => { e.target.tagName === "A" || e.target.closest("div").firstElementChild.click() }} class="download_div" > <a href=${notebookfile_ref.current} target="_blank" download="notebook.jl">notebook.jl</a> <span class="download_icon"></span> </div> ` : html` <div class="command">Copy the notebook URL:</div> <div class="copy_div"> <input onClick=${(e) => e.target.select()} value=${notebookfile_ref.current} readonly /> <span class=${`copy_icon ${showCopyPopup ? "success_copy" : ""}`} onClick=${async () => { await navigator.clipboard.writeText(notebookfile_ref.current) setShowCopyPopup(true) setTimeout(() => setShowCopyPopup(false), 3000) }} /> </div> `} </div> </li> <li> <div class="command">Run Pluto</div> <p> ${"(Also see: "} <a target="_blank" href="https://plutojl.org/#install">How to install Julia and Pluto</a>) </p> <img src="https://user-images.githubusercontent.com/6933510/107865594-60864b00-6e68-11eb-9625-2d11fd608e7b.png" /> </li> <li> ${recommend_download ? html` <div class="command">Open the notebook file</div> <p>Type the saved filename in the <em>open</em> box.</p> <img src="https://user-images.githubusercontent.com/6933510/119374043-65556900-bcb9-11eb-9026-149c1ba2d05b.png" /> ` : html` <div class="command">Paste URL in the <em>Open</em> box</div> <video playsinline autoplay loop src="https://i.imgur.com/wf60p5c.mp4" /> `} </li> </ol> </dialog> </div>` } const expected_runtime = (/** @type {import("./Editor.js").NotebookData} */ notebook) => { return ((notebook.nbpkg?.install_time_ns ?? NaN) + _.sum(Object.values(notebook.cell_results).map((c) => c.runtime ?? 0))) / 1e9 } const runtime_overhead = 15 // seconds const runtime_multiplier = 1.5 const expected_runtime_str = (/** @type {import("./Editor.js").NotebookData} */ notebook) => { const ex = expected_runtime(notebook) if (isNaN(ex)) { return null } const sec = _.round(runtime_overhead + ex * runtime_multiplier, -1) return pretty_long_time(sec) } export const pretty_long_time = (/** @type {number} */ sec) => { const min = sec / 60 const sec_r = Math.ceil(sec) const min_r = Math.round(min) if (sec < 60) { return `${sec_r} second${sec_r > 1 ? "s" : ""}` } else { return `${min_r} minute${min_r > 1 ? "s" : ""}` } } import { html, Component } from "../imports/Preact.js" import * as preact from "../imports/Preact.js" import immer, { applyPatches, produceWithPatches } from "../imports/immer.js" import _ from "../imports/lodash.js" import { empty_notebook_state, is_editor_embedded_inside_editor, set_disable_ui_css } from "../editor.js" import { create_pluto_connection, ws_address_from_base } from "../common/PlutoConnection.js" import { init_feedback } from "../common/Feedback.js" import { serialize_cells, deserialize_cells, detect_deserializer } from "../common/Serialization.js" import { FilePicker } from "./FilePicker.js" import { Preamble } from "./Preamble.js" import { Notebook } from "./Notebook.js" import { BottomRightPanel } from "./BottomRightPanel.js" import { DropRuler } from "./DropRuler.js" import { SelectionArea } from "./SelectionArea.js" import { RecentlyDisabledInfo, UndoDelete } from "./UndoDelete.js" import { SlideControls } from "./SlideControls.js" import { Scroller } from "./Scroller.js" import { ExportBanner } from "./ExportBanner.js" import { Popup } from "./Popup.js" import { slice_utf8, length_utf8 } from "../common/UnicodeTools.js" import { has_ctrl_or_cmd_pressed, ctrl_or_cmd_name, is_mac_keyboard, in_textarea_or_input, and, control_name, alt_or_options_name, } from "../common/KeyboardShortcuts.js" import { PlutoActionsContext, PlutoBondsContext, PlutoJSInitializingContext, SetWithEmptyCallback } from "../common/PlutoContext.js" import { BackendLaunchPhase, count_stat } from "../common/Binder.js" import { setup_mathjax } from "../common/SetupMathJax.js" import { slider_server_actions, nothing_actions } from "../common/SliderServerClient.js" import { ProgressBar } from "./ProgressBar.js" import { NonCellOutput } from "./NonCellOutput.js" import { IsolatedCell } from "./Cell.js" import { RecordingPlaybackUI, RecordingUI } from "./RecordingUI.js" import { HijackExternalLinksToOpenInNewTab } from "./HackySideStuff/HijackExternalLinksToOpenInNewTab.js" import { FrontMatterInput } from "./FrontmatterInput.js" import { EditorLaunchBackendButton } from "./Editor/LaunchBackendButton.js" import { get_environment } from "../common/Environment.js" import { ProcessStatus } from "../common/ProcessStatus.js" import { SafePreviewUI } from "./SafePreviewUI.js" import { open_pluto_popup } from "../common/open_pluto_popup.js" import { get_included_external_source } from "../common/external_source.js" // This is imported asynchronously - uncomment for development // import environment from "../common/Environment.js" export const default_path = "" const DEBUG_DIFFING = false // Be sure to keep this in sync with DEFAULT_CELL_METADATA in Cell.jl /** @type {CellMetaData} */ const DEFAULT_CELL_METADATA = { disabled: false, show_logs: true, skip_as_script: false, } // from our friends at https://stackoverflow.com/a/2117523 // i checked it and it generates Julia-legal UUIDs and that's all we need -SNOF const uuidv4 = () => //@ts-ignore "10000000-1000-4000-8000-100000000000".replace(/[018]/g, (c) => (c ^ (crypto.getRandomValues(new Uint8Array(1))[0] & (15 >> (c / 4)))).toString(16)) /** * @typedef {import('../imports/immer').Patch} Patch * */ const Main = ({ children }) => { return html`<main>${children}</main>` } /** * Map of status => Bool. In order of decreasing priority. */ const statusmap = (/** @type {EditorState} */ state, /** @type {LaunchParameters} */ launch_params) => ({ disconnected: !(state.connected || state.initializing || state.static_preview), loading: (state.backend_launch_phase != null && BackendLaunchPhase.wait_for_user < state.backend_launch_phase && state.backend_launch_phase < BackendLaunchPhase.ready) || state.initializing || state.moving_file, process_waiting_for_permission: state.notebook.process_status === ProcessStatus.waiting_for_permission && !state.initializing, process_restarting: state.notebook.process_status === ProcessStatus.waiting_to_restart, process_dead: state.notebook.process_status === ProcessStatus.no_process || state.notebook.process_status === ProcessStatus.waiting_to_restart, nbpkg_restart_required: state.notebook.nbpkg?.restart_required_msg != null, nbpkg_restart_recommended: state.notebook.nbpkg?.restart_recommended_msg != null, nbpkg_disabled: state.notebook.nbpkg?.enabled === false || state.notebook.nbpkg?.waiting_for_permission_but_probably_disabled === true, static_preview: state.static_preview, bonds_disabled: !( state.initializing || // connected to regular pluto server state.connected || // connected to slider server (launch_params.slider_server_url != null && (state.slider_server?.connecting || state.slider_server?.interactive)) ), offer_binder: state.backend_launch_phase === BackendLaunchPhase.wait_for_user && launch_params.binder_url != null, offer_local: state.backend_launch_phase === BackendLaunchPhase.wait_for_user && launch_params.pluto_server_url != null, binder: launch_params.binder_url != null && state.backend_launch_phase != null, code_differs: state.notebook.cell_order.some( (cell_id) => state.cell_inputs_local[cell_id] != null && state.notebook.cell_inputs[cell_id].code !== state.cell_inputs_local[cell_id].code ), recording_waiting_to_start: state.recording_waiting_to_start, is_recording: state.is_recording, isolated_cell_view: launch_params.isolated_cell_ids != null && launch_params.isolated_cell_ids.length > 0, sanitize_html: state.notebook.process_status === ProcessStatus.waiting_for_permission, }) const first_true_key = (obj) => { for (let [k, v] of Object.entries(obj)) { if (v) { return k } } } /** * @typedef CellMetaData * @type {{ * disabled: boolean, * show_logs: boolean, * skip_as_script: boolean * }} * * @typedef CellInputData * @type {{ * cell_id: string, * code: string, * code_folded: boolean, * metadata: CellMetaData, * }} */ /** * @typedef LogEntryData * @type {{ * level: number, * msg: string, * file: string, * line: number, * kwargs: Object, * }} */ /** * @typedef StatusEntryData * @type {{ * name: string, * success?: boolean, * started_at: number?, * finished_at: number?, * timing?: "remote" | "local", * subtasks: Record<string,StatusEntryData>, * }} */ /** * @typedef CellResultData * @type {{ * cell_id: string, * queued: boolean, * running: boolean, * errored: boolean, * runtime: number?, * downstream_cells_map: { [variable: string]: [string]}, * upstream_cells_map: { [variable: string]: [string]}, * precedence_heuristic: number?, * depends_on_disabled_cells: boolean, * depends_on_skipped_cells: boolean, * output: { * body: string | Object, * persist_js_state: boolean, * last_run_timestamp: number, * mime: string, * rootassignee: string?, * has_pluto_hook_features: boolean, * }, * logs: Array<LogEntryData>, * published_object_keys: [string], * }} */ /** * @typedef CellDependencyData * @property {string} cell_id * @property {Record<string, Array<string>>} downstream_cells_map A map where the keys are the variables *defined* by this cell, and a value is the list of cell IDs that reference a variable. * @property {Record<string, Array<string>>} upstream_cells_map A map where the keys are the variables *referenced* by this cell, and a value is the list of cell IDs that define a variable. * @property {number} precedence_heuristic */ /** * @typedef CellDependencyGraph * @type {{ [uuid: string]: CellDependencyData }} */ /** * @typedef NotebookPkgData * @type {{ * enabled: boolean, * waiting_for_permission: boolean?, * waiting_for_permission_but_probably_disabled: boolean?, * restart_recommended_msg: string?, * restart_required_msg: string?, * installed_versions: { [pkg_name: string]: string }, * terminal_outputs: { [pkg_name: string]: string }, * install_time_ns: number?, * busy_packages: string[], * instantiated: boolean, * }} */ /** * @typedef LaunchParameters * @type {{ * notebook_id: string?, * statefile: string?, * statefile_integrity: string?, * notebookfile: string?, * notebookfile_integrity: string?, * disable_ui: boolean, * preamble_html: string?, * isolated_cell_ids: string[]?, * binder_url: string?, * pluto_server_url: string?, * slider_server_url: string?, * recording_url: string?, * recording_url_integrity: string?, * recording_audio_url: string?, * }} */ /** * @typedef BondValueContainer * @type {{ value: any }} */ /** * @typedef BondValuesDict * @type {{ [name: string]: BondValueContainer }} */ /** * @typedef NotebookData * @type {{ * pluto_version?: string, * notebook_id: string, * path: string, * shortpath: string, * in_temp_dir: boolean, * process_status: string, * last_save_time: number, * last_hot_reload_time: number, * cell_inputs: { [uuid: string]: CellInputData }, * cell_results: { [uuid: string]: CellResultData }, * cell_dependencies: CellDependencyGraph, * cell_order: Array<string>, * cell_execution_order: Array<string>, * published_objects: { [objectid: string]: any}, * bonds: BondValuesDict, * nbpkg: NotebookPkgData?, * metadata: object, * status_tree: StatusEntryData?, * }} */ const url_logo_big = get_included_external_source("pluto-logo-big")?.href export const url_logo_small = get_included_external_source("pluto-logo-small")?.href /** * @typedef EditorProps * @type {{ * launch_params: LaunchParameters, * initial_notebook_state: NotebookData, * preamble_element: preact.ReactElement?, * pluto_editor_element: HTMLElement, * }} */ /** * @typedef EditorState * @type {{ * notebook: NotebookData, * cell_inputs_local: { [uuid: string]: { code: String } }, * unsumbitted_global_definitions: { [uuid: string]: String[] } * desired_doc_query: ?String, * recently_deleted: ?Array<{ index: number, cell: CellInputData }>, * recently_auto_disabled_cells: Record<string,[string,string]>, * last_update_time: number, * disable_ui: boolean, * static_preview: boolean, * backend_launch_phase: ?number, * backend_launch_logs: ?string, * binder_session_url: ?string, * binder_session_token: ?string, * refresh_target: ?string, * connected: boolean, * initializing: boolean, * moving_file: boolean, * scroller: { * up: boolean, * down: boolean, * }, * export_menu_open: boolean, * last_created_cell: ?string, * selected_cells: Array<string>, * extended_components: any, * is_recording: boolean, * recording_waiting_to_start: boolean, * slider_server: { connecting: boolean, interactive: boolean }, * }} */ /** * @augments Component<EditorProps,EditorState> */ export class Editor extends Component { constructor(/** @type {EditorProps} */ props) { super(props) const { launch_params, initial_notebook_state } = this.props /** @type {EditorState} */ this.state = { notebook: initial_notebook_state, cell_inputs_local: {}, unsumbitted_global_definitions: {}, desired_doc_query: null, recently_deleted: [], recently_auto_disabled_cells: {}, last_update_time: 0, disable_ui: launch_params.disable_ui, static_preview: launch_params.statefile != null, backend_launch_phase: launch_params.notebookfile != null && (launch_params.binder_url != null || launch_params.pluto_server_url != null) ? BackendLaunchPhase.wait_for_user : null, backend_launch_logs: null, binder_session_url: null, binder_session_token: null, refresh_target: null, connected: false, initializing: true, moving_file: false, scroller: { up: false, down: false, }, export_menu_open: false, last_created_cell: null, selected_cells: [], extended_components: { CustomHeader: null, }, is_recording: false, recording_waiting_to_start: false, slider_server: { connecting: false, interactive: false, }, } this.setStatePromise = (fn) => new Promise((r) => this.setState(fn, r)) // these are things that can be done to the local notebook this.real_actions = { get_notebook: () => this?.state?.notebook ?? {}, get_session_options: () => this.client.session_options, get_launch_params: () => this.props.launch_params, send: (message_type, ...args) => this.client.send(message_type, ...args), get_published_object: (objectid) => this.state.notebook.published_objects[objectid], //@ts-ignore update_notebook: (...args) => this.update_notebook(...args), set_doc_query: (query) => this.setState({ desired_doc_query: query }), set_local_cell: (cell_id, new_val) => { return this.setStatePromise( immer((/** @type {EditorState} */ state) => { state.cell_inputs_local[cell_id] = { code: new_val, } state.selected_cells = [] }) ) }, set_unsubmitted_global_definitions: (cell_id, new_val) => { return this.setStatePromise( immer((/** @type {EditorState} */ state) => { state.unsumbitted_global_definitions[cell_id] = new_val }) ) }, get_unsubmitted_global_definitions: () => _.pick(this.state.unsumbitted_global_definitions, this.state.notebook.cell_order), focus_on_neighbor: (cell_id, delta, line = delta === -1 ? Infinity : -1, ch = 0) => { const i = this.state.notebook.cell_order.indexOf(cell_id) const new_i = i + delta if (new_i >= 0 && new_i < this.state.notebook.cell_order.length) { window.dispatchEvent( new CustomEvent("cell_focus", { detail: { cell_id: this.state.notebook.cell_order[new_i], line: line, ch: ch, }, }) ) } }, add_deserialized_cells: async (data, index_or_id, deserializer = deserialize_cells) => { let new_codes = deserializer(data) /** @type {Array<CellInputData>} Create copies of the cells with fresh ids */ let new_cells = new_codes.map((code) => ({ cell_id: uuidv4(), code: code, code_folded: false, metadata: { ...DEFAULT_CELL_METADATA, }, })) let index if (typeof index_or_id === "number") { index = index_or_id } else { /* if the input is not an integer, try interpreting it as a cell id */ index = this.state.notebook.cell_order.indexOf(index_or_id) if (index !== -1) { /* Make sure that the cells are pasted after the current cell */ index += 1 } } if (index === -1) { index = this.state.notebook.cell_order.length } /** Update local_code. Local code doesn't force CM to update it's state * (the usual flow is keyboard event -> cm -> local_code and not the opposite ) * See ** 1 ** */ this.setState( immer((/** @type {EditorState} */ state) => { // Deselect everything first, to clean things up state.selected_cells = [] for (let cell of new_cells) { state.cell_inputs_local[cell.cell_id] = cell } state.last_created_cell = new_cells[0]?.cell_id }) ) /** * Create an empty cell in the julia-side. * Code will differ, until the user clicks 'run' on the new code */ await update_notebook((notebook) => { for (const cell of new_cells) { notebook.cell_inputs[cell.cell_id] = { ...cell, // Fill the cell with empty code remotely, so it doesn't run unsafe code code: "", metadata: { ...DEFAULT_CELL_METADATA, }, } } notebook.cell_order = [ ...notebook.cell_order.slice(0, index), ...new_cells.map((x) => x.cell_id), ...notebook.cell_order.slice(index, Infinity), ] }) }, wrap_remote_cell: async (cell_id, block_start = "begin", block_end = "end") => { const cell = this.state.notebook.cell_inputs[cell_id] const new_code = `${block_start}\n\t${cell.code.replace(/\n/g, "\n\t")}\n${block_end}` await this.setStatePromise( immer((/** @type {EditorState} */ state) => { state.cell_inputs_local[cell_id] = { code: new_code, } }) ) await this.actions.set_and_run_multiple([cell_id]) }, split_remote_cell: async (cell_id, boundaries, submit = false) => { const cell = this.state.notebook.cell_inputs[cell_id] const old_code = cell.code const padded_boundaries = [0, ...boundaries] /** @type {Array<String>} */ const parts = boundaries.map((b, i) => slice_utf8(old_code, padded_boundaries[i], b).trim()).filter((x) => x !== "") /** @type {Array<CellInputData>} */ const cells_to_add = parts.map((code) => { return { cell_id: uuidv4(), code: code, code_folded: false, metadata: { ...DEFAULT_CELL_METADATA, }, } }) this.setState( immer((/** @type {EditorState} */ state) => { for (let cell of cells_to_add) { state.cell_inputs_local[cell.cell_id] = cell } }) ) await update_notebook((notebook) => { // delete the old cell delete notebook.cell_inputs[cell_id] // add the new ones for (let cell of cells_to_add) { notebook.cell_inputs[cell.cell_id] = cell } notebook.cell_order = notebook.cell_order.flatMap((c) => { if (cell_id === c) { return cells_to_add.map((x) => x.cell_id) } else { return [c] } }) }) if (submit) { await this.actions.set_and_run_multiple(cells_to_add.map((x) => x.cell_id)) } }, interrupt_remote: (cell_id) => { // TODO Make this cooler // set_notebook_state((prevstate) => { // return { // cells: prevstate.cells.map((c) => { // return { ...c, errored: c.errored || c.running || c.queued } // }), // } // }) this.client.send("interrupt_all", {}, { notebook_id: this.state.notebook.notebook_id }, false) }, move_remote_cells: (cell_ids, new_index) => { return update_notebook((notebook) => { new_index = Math.max(0, new_index) let before = notebook.cell_order.slice(0, new_index).filter((x) => !cell_ids.includes(x)) let after = notebook.cell_order.slice(new_index, Infinity).filter((x) => !cell_ids.includes(x)) notebook.cell_order = [...before, ...cell_ids, ...after] }) }, add_remote_cell_at: async (index, code = "") => { let id = uuidv4() this.setState({ last_created_cell: id }) await update_notebook((notebook) => { notebook.cell_inputs[id] = { cell_id: id, code, code_folded: false, metadata: { ...DEFAULT_CELL_METADATA }, } notebook.cell_order = [...notebook.cell_order.slice(0, index), id, ...notebook.cell_order.slice(index, Infinity)] }) await this.client.send("run_multiple_cells", { cells: [id] }, { notebook_id: this.state.notebook.notebook_id }) return id }, add_remote_cell: async (cell_id, before_or_after, code) => { const index = this.state.notebook.cell_order.indexOf(cell_id) const delta = before_or_after == "before" ? 0 : 1 return await this.actions.add_remote_cell_at(index + delta, code) }, confirm_delete_multiple: async (verb, cell_ids) => { if (cell_ids.length <= 1 || confirm(`${verb} ${cell_ids.length} cells?`)) { if (cell_ids.some((cell_id) => this.state.notebook.cell_results[cell_id].running || this.state.notebook.cell_results[cell_id].queued)) { if (confirm("This cell is still running - would you like to interrupt the notebook?")) { this.actions.interrupt_remote(cell_ids[0]) } } else { this.setState( immer((/** @type {EditorState} */ state) => { state.recently_deleted = cell_ids.map((cell_id) => { return { index: this.state.notebook.cell_order.indexOf(cell_id), cell: this.state.notebook.cell_inputs[cell_id], } }) state.selected_cells = [] for (let c of cell_ids) { delete state.unsumbitted_global_definitions[c] } }) ) await update_notebook((notebook) => { for (let cell_id of cell_ids) { delete notebook.cell_inputs[cell_id] } notebook.cell_order = notebook.cell_order.filter((cell_id) => !cell_ids.includes(cell_id)) }) await this.client.send("run_multiple_cells", { cells: [] }, { notebook_id: this.state.notebook.notebook_id }) } } }, fold_remote_cells: async (cell_ids, new_value) => { await update_notebook((notebook) => { for (let cell_id of cell_ids) { notebook.cell_inputs[cell_id].code_folded = new_value ?? !notebook.cell_inputs[cell_id].code_folded } }) }, set_and_run_all_changed_remote_cells: () => { const changed = this.state.notebook.cell_order.filter( (cell_id) => this.state.cell_inputs_local[cell_id] != null && this.state.notebook.cell_inputs[cell_id].code !== this.state.cell_inputs_local[cell_id]?.code ) this.actions.set_and_run_multiple(changed) return changed.length > 0 }, set_and_run_multiple: async (cell_ids) => { // TODO: this function is called with an empty list sometimes, where? if (cell_ids.length > 0) { window.dispatchEvent( new CustomEvent("set_waiting_to_run_smart", { detail: { cell_ids, }, }) ) await update_notebook((notebook) => { for (let cell_id of cell_ids) { if (this.state.cell_inputs_local[cell_id]) { notebook.cell_inputs[cell_id].code = this.state.cell_inputs_local[cell_id].code } } }) await this.setStatePromise( immer((/** @type {EditorState} */ state) => { for (let cell_id of cell_ids) { delete state.unsumbitted_global_definitions[cell_id] // This is a "dirty" trick, as this should actually be stored in some shared request_status => status state // But for now... this is fine  if (state.notebook.cell_results[cell_id] != null) { state.notebook.cell_results[cell_id].queued = this.is_process_ready() } else { // nothing } } }) ) const result = await this.client.send("run_multiple_cells", { cells: cell_ids }, { notebook_id: this.state.notebook.notebook_id }) const { disabled_cells } = result.message if (Object.entries(disabled_cells).length > 0) { await this.setStatePromise({ recently_auto_disabled_cells: disabled_cells, }) } } }, /** * * @param {string} name name of bound variable * @param {*} value value (not in wrapper object) */ set_bond: async (name, value) => { await update_notebook((notebook) => { // Wrap the bond value in an object so immer assumes it is changed let new_bond = { value: value } notebook.bonds[name] = new_bond }) }, reshow_cell: (cell_id, objectid, dim) => this.client.send( "reshow_cell", { objectid, dim, cell_id, }, { notebook_id: this.state.notebook.notebook_id }, false ), request_js_link_response: (cell_id, link_id, input) => { return this.client .send( "request_js_link_response", { cell_id, link_id, input, }, { notebook_id: this.state.notebook.notebook_id } ) .then((r) => r.message) }, /** This actions avoids pushing selected cells all the way down, which is too heavy to handle! */ get_selected_cells: (cell_id, /** @type {boolean} */ allow_other_selected_cells) => allow_other_selected_cells ? this.state.selected_cells : [cell_id], get_avaible_versions: async ({ package_name, notebook_id }) => { const { message } = await this.client.send("nbpkg_available_versions", { package_name: package_name }, { notebook_id: notebook_id }) return message }, } this.actions = { ...this.real_actions } const apply_notebook_patches = (patches, /** @type {NotebookData?} */ old_state = null, get_reverse_patches = false) => new Promise((resolve) => { if (patches.length !== 0) { const should_ignore_patch_error = (/** @type {string} */ failing_path) => failing_path.startsWith("status_tree") let _copy_of_patches, reverse_of_patches = [] this.setState( immer((/** @type {EditorState} */ state) => { let new_notebook try { // To test this, uncomment the lines below: // if (Math.random() < 0.25) { // throw new Error(`Error: [Immer] minified error nr: 15 '${patches?.[0]?.path?.join("/")}' .`) // } if (get_reverse_patches) { ;[new_notebook, _copy_of_patches, reverse_of_patches] = produceWithPatches(old_state ?? state.notebook, (state) => { applyPatches(state, patches) }) // TODO: why was `new_notebook` not updated? // this is why the line below is also called when `get_reverse_patches === true` } new_notebook = applyPatches(old_state ?? state.notebook, patches) } catch (exception) { /** @type {String} Example: `"a.b[2].c"` */ const failing_path = String(exception).match(".*'(.*)'.*")?.[1].replace(/\//gi, ".") ?? exception const path_value = _.get(this.state.notebook, failing_path, "Not Found") console.log(String(exception).match(".*'(.*)'.*")?.[1].replace(/\//gi, ".") ?? exception, failing_path, typeof failing_path) const ignore = should_ignore_patch_error(failing_path) ;(ignore ? console.log : console.error)( `#######################**************************######################## PlutoError: StateOutOfSync: Failed to apply patches. Please report this: https://github.com/fonsp/Pluto.jl/issues adding the info below: failing path: ${failing_path} notebook previous value: ${path_value} patch: ${JSON.stringify( patches?.find(({ path }) => path.join("") === failing_path), null, 1 )} all patches: ${JSON.stringify(patches, null, 1)} #######################**************************########################`, exception ) let parts = failing_path.split(".") for (let i = 0; i < parts.length; i++) { let path = parts.slice(0, i).join(".") console.log(path, _.get(this.state.notebook, path, "Not Found")) } if (ignore) { console.info("Safe to ignore this patch failure...") } else if (this.state.connected) { console.error("Trying to recover: Refetching notebook...") this.client.send( "reset_shared_state", {}, { notebook_id: this.state.notebook.notebook_id, }, false ) } else if (this.state.static_preview && launch_params.slider_server_url != null) { open_pluto_popup({ type: "warn", body: html`Something went wrong while updating the notebook state. Please refresh the page to try again.`, }) } else { console.error("Trying to recover: reloading...") window.parent.location.href = this.state.refresh_target ?? window.location.href } return } if (DEBUG_DIFFING) { console.group("Update!") for (let patch of patches) { console.group(`Patch :${patch.op}`) console.log(patch.path) console.log(patch.value) console.groupEnd() } console.groupEnd() } let cells_stuck_in_limbo = new_notebook.cell_order.filter((cell_id) => new_notebook.cell_inputs[cell_id] == null) if (cells_stuck_in_limbo.length !== 0) { console.warn(`cells_stuck_in_limbo:`, cells_stuck_in_limbo) new_notebook.cell_order = new_notebook.cell_order.filter((cell_id) => new_notebook.cell_inputs[cell_id] != null) } this.on_patches_hook(patches) state.notebook = new_notebook }), () => resolve(reverse_of_patches) ) } else { resolve([]) } }) this.apply_notebook_patches = apply_notebook_patches // these are update message that are _not_ a response to a `send(*, *, {create_promise: true})` this.last_update_counter = -1 const check_update_counter = (new_val) => { if (new_val <= this.last_update_counter) { console.error("State update out of order", new_val, this.last_update_counter) alert("Oopsie!! please refresh your browser and everything will be alright!") } this.last_update_counter = new_val } const on_update = (update, by_me) => { if (this.state.notebook.notebook_id === update.notebook_id) { const show_debugs = launch_params.binder_url != null if (show_debugs) console.debug("on_update", update, by_me) const message = update.message switch (update.type) { case "notebook_diff": check_update_counter(message?.counter) let apply_promise = Promise.resolve() if (message?.response?.from_reset) { console.log("Trying to reset state after failure") apply_promise = apply_notebook_patches( message.patches, empty_notebook_state({ notebook_id: this.state.notebook.notebook_id }) ).catch((e) => { alert("Oopsie!! please refresh your browser and everything will be alright!") throw e }) } else if (message.patches.length !== 0) { apply_promise = apply_notebook_patches(message.patches) } const set_waiting = () => { let from_update = message?.response?.update_went_well != null let is_just_acknowledgement = from_update && message.patches.length === 0 let is_relevant_for_bonds = message.patches.some(({ path }) => path.length === 0 || path[0] !== "status_tree") // console.debug("Received patches!", is_just_acknowledgement, is_relevant_for_bonds, message.patches, message.response) if (!is_just_acknowledgement && is_relevant_for_bonds) { this.waiting_for_bond_to_trigger_execution = false } } apply_promise.finally(set_waiting).then(() => { this.maybe_send_queued_bond_changes() }) break default: console.error("Received unknown update type!", update) // alert("Something went wrong \n Try clearing your browser cache and refreshing the page") break } if (show_debugs) console.debug("on_update done") } else { // Update for a different notebook, TODO maybe log this as it shouldn't happen } } const on_establish_connection = async (client) => { // nasty Object.assign(this.client, client) try { const environment = await get_environment(client) const { custom_editor_header_component, custom_non_cell_output } = environment({ client, editor: this, imports: { preact } }) this.setState({ extended_components: { ...this.state.extended_components, CustomHeader: custom_editor_header_component, NonCellOutputComponents: custom_non_cell_output, }, }) } catch (e) {} // @ts-ignore window.version_info = this.client.version_info // for debugging // @ts-ignore window.kill_socket = this.client.kill // for debugging if (!client.notebook_exists) { console.error("Notebook does not exist. Not connecting.") return } console.debug("Sending update_notebook request...") await this.client.send("update_notebook", { updates: [] }, { notebook_id: this.state.notebook.notebook_id }, false) console.debug("Received update_notebook request") this.setState({ initializing: false, static_preview: false, backend_launch_phase: this.state.backend_launch_phase == null ? null : BackendLaunchPhase.ready, }) this.client.send("complete", { query: "sq" }, { notebook_id: this.state.notebook.notebook_id }) this.client.send("complete", { query: "\\sq" }, { notebook_id: this.state.notebook.notebook_id }) setTimeout(init_feedback, 2 * 1000) // 2 seconds - load feedback a little later for snappier UI } const on_connection_status = (val, hopeless) => { this.setState({ connected: val }) if (hopeless) { // https://github.com/fonsp/Pluto.jl/issues/55 // https://github.com/fonsp/Pluto.jl/issues/2398 open_pluto_popup({ type: "warn", body: html`<p>A new server was started - this notebook session is no longer running.</p> <p>Would you like to go back to the main menu?</p> <br /> <a href="./">Go back</a> <br /> <a href="#" onClick=${(e) => { e.preventDefault() window.dispatchEvent(new CustomEvent("close pluto popup")) }} >Stay here</a >`, should_focus: false, }) } } const on_reconnect = async () => { console.warn("Reconnected! Checking states") await this.client.send( "reset_shared_state", {}, { notebook_id: this.state.notebook.notebook_id, }, false ) return true } this.export_url = (/** @type {string} */ u) => this.state.binder_session_url == null ? `./${u}?id=${this.state.notebook.notebook_id}` : `${this.state.binder_session_url}${u}?id=${this.state.notebook.notebook_id}&token=${this.state.binder_session_token}` /** @type {import('../common/PlutoConnection').PlutoConnection} */ this.client = /** @type {import('../common/PlutoConnection').PlutoConnection} */ ({}) this.connect = (/** @type {string | undefined} */ ws_address = undefined) => create_pluto_connection({ ws_address: ws_address, on_unrequested_update: on_update, on_connection_status: on_connection_status, on_reconnect: on_reconnect, connect_metadata: { notebook_id: this.state.notebook.notebook_id }, }).then(on_establish_connection) this.on_disable_ui = () => { set_disable_ui_css(this.state.disable_ui, props.pluto_editor_element) // Pluto has three modes of operation: // 1. (normal) Connected to a Pluto notebook. // 2. Static HTML with PlutoSliderServer. All edits are ignored, but bond changes are processes by the PlutoSliderServer. // 3. Static HTML without PlutoSliderServer. All interactions are ignored. // // To easily support all three with minimal changes to the source code, we sneakily swap out the `this.actions` object (`pluto_actions` in other source files) with a different one: Object.assign( this.actions, // if we have no pluto server... this.state.disable_ui || (launch_params.slider_server_url != null && !this.state.connected) ? // then use a modified set of actions launch_params.slider_server_url != null ? slider_server_actions({ setStatePromise: this.setStatePromise, actions: this.actions, launch_params: launch_params, apply_notebook_patches, get_original_state: () => this.props.initial_notebook_state, get_current_state: () => this.state.notebook, }) : nothing_actions({ actions: this.actions, }) : // otherwise, use the real actions this.real_actions ) } this.on_disable_ui() setInterval(() => { if (!this.state.static_preview && document.visibilityState === "visible") { // view stats on https://stats.plutojl.org/ //@ts-ignore count_stat(`editing/${window?.version_info?.pluto ?? this.state.notebook.pluto_version ?? "unknown"}${window.plutoDesktop ? "-desktop" : ""}`) } }, 1000 * 15 * 60) setInterval(() => { if (!this.state.static_preview && document.visibilityState === "visible") { update_stored_recent_notebooks(this.state.notebook.path) } }, 1000 * 5) // Not completely happy with this yet, but it will do for now - DRAL /** Patches that are being delayed until all cells have finished running. */ this.bond_changes_to_apply_when_done = [] this.maybe_send_queued_bond_changes = () => { if (this.notebook_is_idle() && this.bond_changes_to_apply_when_done.length !== 0) { // console.log("Applying queued bond changes!", this.bond_changes_to_apply_when_done) let bonds_patches = this.bond_changes_to_apply_when_done this.bond_changes_to_apply_when_done = [] this.update_notebook((notebook) => { applyPatches(notebook, bonds_patches) }) } } /** This tracks whether we just set a bond value which will trigger a cell to run, but we are still waiting for the server to process the bond value (and run the cell). During this time, we won't send new bond values. See https://github.com/fonsp/Pluto.jl/issues/1891 for more info. */ this.waiting_for_bond_to_trigger_execution = false /** Number of local updates that have not yet been applied to the server's state. */ this.pending_local_updates = 0 /** * User scripts that are currently running (possibly async). * @type {SetWithEmptyCallback<HTMLElement>} */ this.js_init_set = new SetWithEmptyCallback(() => { // console.info("All scripts finished!") this.maybe_send_queued_bond_changes() }) // @ts-ignore This is for tests document.body._js_init_set = this.js_init_set /** Is the notebook ready to execute code right now? (i.e. are no cells queued or running?) */ this.notebook_is_idle = () => { return !( this.waiting_for_bond_to_trigger_execution || this.pending_local_updates > 0 || // a cell is running: Object.values(this.state.notebook.cell_results).some((cell) => cell.running || cell.queued) || // a cell is initializing JS: !_.isEmpty(this.js_init_set) || !this.is_process_ready() ) } this.is_process_ready = () => this.state.notebook.process_status === ProcessStatus.starting || this.state.notebook.process_status === ProcessStatus.ready const bond_will_trigger_evaluation = (/** @type {string|PropertyKey} */ sym) => Object.entries(this.state.notebook.cell_dependencies).some(([cell_id, deps]) => { // if the other cell depends on the variable `sym`... if (deps.upstream_cells_map.hasOwnProperty(sym)) { // and the cell is not disabled const running_disabled = this.state.notebook.cell_inputs[cell_id].metadata.disabled // or indirectly disabled const indirectly_disabled = this.state.notebook.cell_results[cell_id].depends_on_disabled_cells return !(running_disabled || indirectly_disabled) } }) /** * We set `waiting_for_bond_to_trigger_execution` to `true` if it is *guaranteed* that this bond change will trigger something to happen (i.e. a cell to run). See https://github.com/fonsp/Pluto.jl/pull/1892 for more info about why. * * This is guaranteed if there is a cell in the notebook that references the bound variable. We use our copy of the notebook's toplogy to check this. * * # Gotchas: * 1. We (the frontend) might have an out-of-date copy of the notebook's topology: this bond might have dependents *right now*, but the backend might already be processing a code change that removes that dependency. * * However, this change in topology will result in a patch, which will set `waiting_for_bond_to_trigger_execution` back to `false`. * * 2. The backend has a "first value" mechanism: if bond values are being set for the first time *and* this value is already set on the backend, then the value will be skipped. See https://github.com/fonsp/Pluto.jl/issues/275. If all bond values are skipped, then we might get zero patches back (because no cells will run). * * A bond value is considered a "first value" if it is sent using an `"add"` patch. This is why we require `x.op === "replace"`. */ const bond_patch_will_trigger_evaluation = (/** @type {Patch} */ x) => x.op === "replace" && x.path.length >= 1 && bond_will_trigger_evaluation(x.path[1]) let last_update_notebook_task = Promise.resolve() /** @param {(notebook: NotebookData) => void} mutate_fn */ let update_notebook = (mutate_fn) => { const new_task = last_update_notebook_task.then(async () => { // if (this.state.initializing) { // console.error("Update notebook done during initializing, strange") // return // } let [new_notebook, changes, inverseChanges] = produceWithPatches(this.state.notebook, (notebook) => { mutate_fn(notebook) }) // If "notebook is not idle" we seperate and store the bonds updates, // to send when the notebook is idle. This delays the updating of the bond for performance, // but when the server can discard bond updates itself (now it executes them one by one, even if there is a newer update ready) // this will no longer be necessary let is_idle = this.notebook_is_idle() let changes_involving_bonds = changes.filter((x) => x.path[0] === "bonds") if (!is_idle) { this.bond_changes_to_apply_when_done = [...this.bond_changes_to_apply_when_done, ...changes_involving_bonds] changes = changes.filter((x) => x.path[0] !== "bonds") } if (DEBUG_DIFFING) { try { let previous_function_name = new Error().stack?.split("\n")[2].trim().split(" ")[1] console.log(`Changes to send to server from "${previous_function_name}":`, changes) } catch (error) {} } for (let change of changes) { if (change.path.some((x) => typeof x === "number")) { throw new Error("This sounds like it is editing an array...") } } if (changes.length === 0) { return } if (is_idle) { this.waiting_for_bond_to_trigger_execution = this.waiting_for_bond_to_trigger_execution || changes_involving_bonds.some(bond_patch_will_trigger_evaluation) } this.pending_local_updates++ this.on_patches_hook(changes) try { // console.log("Sending changes to server:", changes) await Promise.all([ this.client.send("update_notebook", { updates: changes }, { notebook_id: this.state.notebook.notebook_id }, false).then((response) => { if (response.message?.response?.update_went_well === "") { // We only throw an error for functions that are waiting for this // Notebook state will already have the changes reversed throw new Error(`Pluto update_notebook error: (from Julia: ${response.message.response.why_not})`) } }), this.setStatePromise({ notebook: new_notebook, last_update_time: Date.now(), }), ]) } finally { this.pending_local_updates-- // this property is used to tell our frontend tests that the updates are done //@ts-ignore document.body._update_is_ongoing = this.pending_local_updates > 0 } }) last_update_notebook_task = new_task.catch(console.error) return new_task } this.update_notebook = update_notebook //@ts-ignore window.shutdownNotebook = this.close = () => { this.client.send( "shutdown_notebook", { keep_in_session: false, }, { notebook_id: this.state.notebook.notebook_id, }, false ) } this.submit_file_change = async (new_path, reset_cm_value) => { const old_path = this.state.notebook.path if (old_path === new_path) { return } if (!this.state.notebook.in_temp_dir) { if (!confirm("Are you sure? Will move from\n\n" + old_path + "\n\nto\n\n" + new_path)) { throw new Error("Declined by user") } } this.setState({ moving_file: true }) try { await update_notebook((notebook) => { notebook.in_temp_dir = false notebook.path = new_path }) // @ts-ignore document.activeElement?.blur() } catch (error) { alert("Failed to move file:\n\n" + error.message) } finally { this.setState({ moving_file: false }) } } this.desktop_submit_file_change = async () => { this.setState({ moving_file: true }) /** * `window.plutoDesktop?.ipcRenderer` is basically what allows the * frontend to communicate with the electron side. It is an IPC * bridge between render process and main process. More info * [here](https://www.electronjs.org/docs/latest/api/ipc-renderer). * * "PLUTO-MOVE-NOTEBOOK" is an event triggered in the main process * once the move is complete, we listen to it using `once`. * More info [here](https://www.electronjs.org/docs/latest/api/ipc-renderer#ipcrendereroncechannel-listener) */ window.plutoDesktop?.ipcRenderer.once("PLUTO-MOVE-NOTEBOOK", async (/** @type {string?} */ loc) => { if (!!loc) await this.setStatePromise( immer((/** @type {EditorState} */ state) => { state.notebook.in_temp_dir = false state.notebook.path = loc }) ) this.setState({ moving_file: false }) // @ts-ignore document.activeElement?.blur() }) // ask the electron backend to start moving the notebook. The event above will be fired once it is done. window.plutoDesktop?.fileSystem.moveNotebook() } this.delete_selected = (verb) => { if (this.state.selected_cells.length > 0) { this.actions.confirm_delete_multiple(verb, this.state.selected_cells) return true } } this.run_selected = () => { return this.actions.set_and_run_multiple(this.state.selected_cells) } this.fold_selected = (new_val) => { if (_.isEmpty(this.state.selected_cells)) return return this.actions.fold_remote_cells(this.state.selected_cells, new_val) } this.move_selected = (/** @type {KeyboardEvent} */ e, /** @type {1|-1} */ delta) => { if (this.state.selected_cells.length > 0) { const current_indices = this.state.selected_cells.map((id) => this.state.notebook.cell_order.indexOf(id)) const new_index = (delta > 0 ? Math.max : Math.min)(...current_indices) + (delta === -1 ? -1 : 2) e.preventDefault() return this.actions.move_remote_cells(this.state.selected_cells, new_index).then( // scroll into view () => { document.getElementById((delta > 0 ? _.last : _.first)(this.state.selected_cells) ?? "")?.scrollIntoView({ block: "nearest" }) } ) } } this.serialize_selected = (/** @type {string?} */ cell_id = null) => { const cells_to_serialize = cell_id == null || this.state.selected_cells.includes(cell_id) ? this.state.selected_cells : [cell_id] if (cells_to_serialize.length) { return serialize_cells(cells_to_serialize.map((id) => this.state.notebook.cell_inputs[id])) } } this.patch_listeners = [] this.on_patches_hook = (patches) => { this.patch_listeners.forEach((f) => f(patches)) } let ctrl_down_last_val = { current: false } const set_ctrl_down = (value) => { if (value !== ctrl_down_last_val.current) { ctrl_down_last_val.current = value document.body.querySelectorAll("[data-pluto-variable], [data-cell-variable]").forEach((el) => { el.setAttribute("data-ctrl-down", value ? "true" : "false") }) } } document.addEventListener("keyup", (e) => { set_ctrl_down(has_ctrl_or_cmd_pressed(e)) }) document.addEventListener("visibilitychange", (e) => { set_ctrl_down(false) setTimeout(() => { set_ctrl_down(false) }, 100) }) document.addEventListener("keydown", (e) => { set_ctrl_down(has_ctrl_or_cmd_pressed(e)) // if (e.defaultPrevented) { // return // } if (e.key?.toLowerCase() === "q" && has_ctrl_or_cmd_pressed(e)) { // This one can't be done as cmd+q on mac, because that closes chrome - Dral if (Object.values(this.state.notebook.cell_results).some((c) => c.running || c.queued)) { this.actions.interrupt_remote() } e.preventDefault() } else if (e.key?.toLowerCase() === "s" && has_ctrl_or_cmd_pressed(e)) { const some_cells_ran = this.actions.set_and_run_all_changed_remote_cells() if (!some_cells_ran) { // all cells were in sync allready // TODO: let user know that the notebook autosaves } e.preventDefault() } else if (["BracketLeft", "BracketRight"].includes(e.code) && (is_mac_keyboard ? e.altKey && e.metaKey : e.ctrlKey && e.shiftKey)) { this.fold_selected(e.code === "BracketLeft") } else if (e.key === "Backspace" || e.key === "Delete") { if (this.delete_selected("Delete")) { e.preventDefault() } } else if (e.key === "Enter" && e.shiftKey) { this.run_selected() } else if (e.key === "ArrowUp" && e.altKey) { this.move_selected(e, -1) } else if (e.key === "ArrowDown" && e.altKey) { this.move_selected(e, 1) } else if ((e.key === "?" && has_ctrl_or_cmd_pressed(e)) || e.key === "F1") { // On mac "cmd+shift+?" is used by chrome, so that is why this needs to be ctrl as well on mac // Also pressing "ctrl+shift" on mac causes the key to show up as "/", this madness // I hope we can find a better solution for this later - Dral const fold_prefix = is_mac_keyboard ? `${and}` : `Ctrl${and}Shift` alert( ` ${and}Enter: run cell ${ctrl_or_cmd_name}${and}Enter: run cell and add cell below ${ctrl_or_cmd_name}${and}S: submit all changes Delete or Backspace: delete empty cell PageUp or fn${and}: jump to cell above PageDown or fn${and}: jump to cell below ${control_name}${and}click: jump to definition ${alt_or_options_name}${and}: move line/cell up ${alt_or_options_name}${and}: move line/cell down ${control_name}${and}/: toggle comment ${control_name}${and}M: toggle markdown ${fold_prefix}${and}[: hide cell code ${fold_prefix}${and}]: show cell code ${ctrl_or_cmd_name}${and}Q: interrupt notebook Select multiple cells by dragging a selection box from the space between cells. ${ctrl_or_cmd_name}${and}C: copy selected cells ${ctrl_or_cmd_name}${and}X: cut selected cells ${ctrl_or_cmd_name}${and}V: paste selected cells The notebook file saves every time you run a cell.` ) e.preventDefault() } else if (e.key === "Escape") { this.setState({ recording_waiting_to_start: false, selected_cells: [], export_menu_open: false, }) } if (this.state.disable_ui && this.state.backend_launch_phase === BackendLaunchPhase.wait_for_user) { // const code = e.key?.charCodeAt(0) if (e.key === "Enter" || e.key?.length === 1) { if (!document.body.classList.contains("wiggle_binder")) { document.body.classList.add("wiggle_binder") setTimeout(() => { document.body.classList.remove("wiggle_binder") }, 1000) } } } }) document.addEventListener("copy", (e) => { if (!in_textarea_or_input()) { const serialized = this.serialize_selected() if (serialized) { e.preventDefault() // wait one frame to get transient user activation requestAnimationFrame(() => navigator.clipboard.writeText(serialized).catch((err) => { console.error("Error copying cells", e, err, navigator.userActivation) alert(`Error copying cells: ${err?.message ?? err}`) }) ) } } }) document.addEventListener("cut", (e) => { // Disabled because we don't want to accidentally delete cells // or we can enable it with a prompt // Even better would be excel style: grey out until you paste it. If you paste within the same notebook, then it is just a move. // if (!in_textarea_or_input()) { // const serialized = this.serialize_selected() // if (serialized) { // navigator.clipboard // .writeText(serialized) // .then(() => this.delete_selected("Cut")) // .catch((err) => { // alert(`Error cutting cells: ${e}`) // }) // } // } }) document.addEventListener("paste", async (e) => { const topaste = e.clipboardData?.getData("text/plain") if (topaste) { const deserializer = detect_deserializer(topaste) if (deserializer != null) { this.actions.add_deserialized_cells(topaste, -1, deserializer) e.preventDefault() } } }) window.addEventListener("beforeunload", (event) => { const unsaved_cells = this.state.notebook.cell_order.filter( (id) => this.state.cell_inputs_local[id] && this.state.notebook.cell_inputs[id].code !== this.state.cell_inputs_local[id].code ) const first_unsaved = unsaved_cells[0] if (first_unsaved != null) { window.dispatchEvent(new CustomEvent("cell_focus", { detail: { cell_id: first_unsaved } })) // } else if (this.state.notebook.in_temp_dir) { // window.scrollTo(0, 0) // // TODO: focus file picker console.log("Preventing unload") event.stopImmediatePropagation() event.preventDefault() event.returnValue = "" } else { console.warn("unloading  disconnecting websocket") //@ts-ignore if (window.shutdown_binder != null) { // hmmmm that would also shut down the binder if you refreshed, or if you navigate to the binder session main menu by clicking the pluto logo. // Let's keep it disabled for now and let the timeout take care of shutting down the binder // window.shutdown_binder() } // and don't prevent the unload } }) } componentDidMount() { const lp = this.props.launch_params if (this.state.static_preview) { this.setState({ initializing: false, }) // view stats on https://stats.plutojl.org/ count_stat( lp.pluto_server_url != null ? // record which featured notebook was viewed, e.g. basic/Markdown.jl `featured-view${lp.notebookfile != null ? new URL(lp.notebookfile).pathname : ""}` : // @ts-ignore `article-view/${window?.version_info?.pluto ?? this.state.notebook.pluto_version ?? "unknown"}` ) } else { this.connect(lp.pluto_server_url ? ws_address_from_base(lp.pluto_server_url) : undefined) } } componentDidUpdate(/** @type {EditorProps} */ old_props, /** @type {EditorState} */ old_state) { //@ts-ignore window.editor_state = this.state //@ts-ignore window.editor_state_set = this.setStatePromise const new_state = this.state if (old_state?.notebook?.path !== new_state.notebook.path) { update_stored_recent_notebooks(new_state.notebook.path, old_state?.notebook?.path) } if (old_state?.notebook?.shortpath !== new_state.notebook.shortpath) { if (!is_editor_embedded_inside_editor(old_props.pluto_editor_element)) document.title = " " + new_state.notebook.shortpath + "  Pluto.jl" } this.maybe_send_queued_bond_changes() if (old_state.backend_launch_phase !== this.state.backend_launch_phase && this.state.backend_launch_phase != null) { const phase = Object.entries(BackendLaunchPhase).find(([k, v]) => v == this.state.backend_launch_phase)?.[0] console.info(`Binder phase: ${phase} at ${new Date().toLocaleTimeString()}`) } if (old_state.disable_ui !== this.state.disable_ui || old_state.connected !== this.state.connected) { this.on_disable_ui() } if (!this.state.initializing) { setup_mathjax() } if (old_state.notebook.nbpkg?.restart_recommended_msg !== new_state.notebook.nbpkg?.restart_recommended_msg) { console.warn(`New restart recommended message: ${new_state.notebook.nbpkg?.restart_recommended_msg}`) } if (old_state.notebook.nbpkg?.restart_required_msg !== new_state.notebook.nbpkg?.restart_required_msg) { console.warn(`New restart required message: ${new_state.notebook.nbpkg?.restart_required_msg}`) } } componentWillUpdate(new_props, new_state) { this.cached_status = statusmap(new_state, this.props.launch_params) Object.entries(this.cached_status).forEach(([k, v]) => { new_props.pluto_editor_element.classList.toggle(k, v === true) }) } render() { const { launch_params } = this.props let { export_menu_open, notebook } = this.state const status = this.cached_status ?? statusmap(this.state, launch_params) const statusval = first_true_key(status) if (status.isolated_cell_view) { return html` <${PlutoActionsContext.Provider} value=${this.actions}> <${PlutoBondsContext.Provider} value=${this.state.notebook.bonds}> <${PlutoJSInitializingContext.Provider} value=${this.js_init_set}> <${ProgressBar} notebook=${this.state.notebook} backend_launch_phase=${this.state.backend_launch_phase} status=${status}/> <div style="width: 100%"> ${this.state.notebook.cell_order.map( (cell_id, i) => html` <${IsolatedCell} cell_input=${notebook.cell_inputs[cell_id]} cell_result=${this.state.notebook.cell_results[cell_id]} hidden=${!launch_params.isolated_cell_ids?.includes(cell_id)} sanitize_html=${status.sanitize_html} /> ` )} </div> </${PlutoJSInitializingContext.Provider}> </${PlutoBondsContext.Provider}> </${PlutoActionsContext.Provider}> ` } const warn_about_untrusted_code = this.client.session_options?.security?.warn_about_untrusted_code ?? true const restart = async (maybe_confirm = false) => { let source = notebook.metadata?.risky_file_source if ( !warn_about_untrusted_code || !maybe_confirm || source == null || confirm(` Danger! Are you sure that you trust this file? \n\n${source}\n\nA malicious notebook can steal passwords and data.`) ) { await this.actions.update_notebook((notebook) => { delete notebook.metadata.risky_file_source }) await this.client.send( "restart_process", {}, { notebook_id: notebook.notebook_id, } ) } } const restart_button = (text, maybe_confirm = false) => html`<a href="#" id="restart-process-button" onClick=${() => restart(maybe_confirm)}>${text}</a>` return html` ${this.state.disable_ui === false && html`<${HijackExternalLinksToOpenInNewTab} />`} <${PlutoActionsContext.Provider} value=${this.actions}> <${PlutoBondsContext.Provider} value=${this.state.notebook.bonds}> <${PlutoJSInitializingContext.Provider} value=${this.js_init_set}> ${ status.static_preview && status.offer_local ? html`<button title="Go back" onClick=${() => { history.back() }} class="floating_back_button" > <span></span> </button>` : null } <${Scroller} active=${this.state.scroller} /> <${ProgressBar} notebook=${this.state.notebook} backend_launch_phase=${this.state.backend_launch_phase} status=${status}/> <header id="pluto-nav" className=${export_menu_open ? "show_export" : ""}> <${ExportBanner} notebook_id=${this.state.notebook.notebook_id} print_title=${ this.state.notebook.metadata?.frontmatter?.title ?? new URLSearchParams(window.location.search).get("name") ?? this.state.notebook.shortpath } notebookfile_url=${this.export_url("notebookfile")} notebookexport_url=${this.export_url("notebookexport")} open=${export_menu_open} onClose=${() => this.setState({ export_menu_open: false })} start_recording=${() => this.setState({ recording_waiting_to_start: true })} /> ${ status.binder ? html`<div id="binder_spinners"> <binder-spinner id="ring_1"></binder-spinner> <binder-spinner id="ring_2"></binder-spinner> <binder-spinner id="ring_3"></binder-spinner> </div>` : null } <nav id="at_the_top"> <a href=${ this.state.binder_session_url != null ? `${this.state.binder_session_url}?token=${this.state.binder_session_token}` : "./" }> <h1><img id="logo-big" src=${url_logo_big} alt="Pluto.jl" /><img id="logo-small" src=${url_logo_small} /></h1> </a> ${ this.state.extended_components.CustomHeader && html`<${this.state.extended_components.CustomHeader} notebook_id=${this.state.notebook.notebook_id} />` } <div class="flex_grow_1"></div> ${ this.state.extended_components.CustomHeader == null && (status.binder ? html`<pluto-filepicker><a href=${this.export_url("notebookfile")} target="_blank">Save notebook...</a></pluto-filepicker>` : html`<${FilePicker} client=${this.client} value=${notebook.in_temp_dir ? "" : notebook.path} on_submit=${this.submit_file_change} on_desktop_submit=${this.desktop_submit_file_change} clear_on_blur=${true} suggest_new_file=${{ base: this.client.session_options?.server?.notebook_path_suggestion ?? "", }} placeholder="Save notebook..." button_label=${notebook.in_temp_dir ? "Choose" : "Move"} />`) } <div class="flex_grow_2"></div> <div id="process_status">${ status.binder && status.loading ? "Loading binder..." : statusval === "disconnected" ? "Reconnecting..." : statusval === "loading" ? "Loading..." : statusval === "nbpkg_restart_required" ? html`${restart_button("Restart notebook")}${" (required)"}` : statusval === "nbpkg_restart_recommended" ? html`${restart_button("Restart notebook")}${" (recommended)"}` : statusval === "process_restarting" ? "Process exited  restarting..." : statusval === "process_dead" ? html`${"Process exited  "}${restart_button("restart")}` : statusval === "process_waiting_for_permission" ? html`${restart_button("Run notebook code", true)}` : null }</div> <button class="toggle_export" title="Export..." onClick=${() => { this.setState({ export_menu_open: !export_menu_open }) }}><span></span></button> </nav> </header> <${SafePreviewUI} process_waiting_for_permission=${status.process_waiting_for_permission} risky_file_source=${notebook.metadata?.risky_file_source} restart=${restart} warn_about_untrusted_code=${warn_about_untrusted_code} /> <${RecordingUI} notebook_name=${notebook.shortpath} recording_waiting_to_start=${this.state.recording_waiting_to_start} set_recording_states=${({ is_recording, recording_waiting_to_start }) => this.setState({ is_recording, recording_waiting_to_start })} is_recording=${this.state.is_recording} patch_listeners=${this.patch_listeners} export_url=${this.export_url} /> <${RecordingPlaybackUI} launch_params=${launch_params} initializing=${this.state.initializing} apply_notebook_patches=${this.apply_notebook_patches} reset_notebook_state=${() => this.setStatePromise( immer((/** @type {EditorState} */ state) => { state.notebook = this.props.initial_notebook_state }) )} /> <${EditorLaunchBackendButton} editor=${this} launch_params=${launch_params} status=${status} /> <${FrontMatterInput} filename=${notebook.shortpath} remote_frontmatter=${notebook.metadata?.frontmatter} set_remote_frontmatter=${(newval) => this.actions.update_notebook((nb) => { nb.metadata["frontmatter"] = newval })} /> ${this.props.preamble_element} <${Main}> <${Preamble} last_update_time=${this.state.last_update_time} any_code_differs=${status.code_differs} last_hot_reload_time=${notebook.last_hot_reload_time} connected=${this.state.connected} /> <${Notebook} notebook=${notebook} cell_inputs_local=${this.state.cell_inputs_local} disable_input=${this.state.disable_ui || !this.state.connected /* && this.state.backend_launch_phase == null*/} last_created_cell=${this.state.last_created_cell} selected_cells=${this.state.selected_cells} is_initializing=${this.state.initializing} is_process_ready=${this.is_process_ready()} process_waiting_for_permission=${status.process_waiting_for_permission} sanitize_html=${status.sanitize_html} /> <${DropRuler} actions=${this.actions} selected_cells=${this.state.selected_cells} set_scroller=${(enabled) => this.setState({ scroller: enabled })} serialize_selected=${this.serialize_selected} pluto_editor_element=${this.props.pluto_editor_element} /> <${NonCellOutput} notebook_id=${this.state.notebook.notebook_id} environment_component=${this.state.extended_components.NonCellOutputComponents} /> </${Main}> ${ this.state.disable_ui || html`<${SelectionArea} cell_order=${this.state.notebook.cell_order} set_scroller=${(enabled) => { this.setState({ scroller: enabled }) }} on_selection=${(selected_cell_ids) => { // @ts-ignore if ( selected_cell_ids.length !== this.state.selected_cells.length || _.difference(selected_cell_ids, this.state.selected_cells).length !== 0 ) { this.setState({ selected_cells: selected_cell_ids, }) } }} />` } <${BottomRightPanel} desired_doc_query=${this.state.desired_doc_query} on_update_doc_query=${this.actions.set_doc_query} connected=${this.state.connected} backend_launch_phase=${this.state.backend_launch_phase} backend_launch_logs=${this.state.backend_launch_logs} notebook=${this.state.notebook} sanitize_html=${status.sanitize_html} /> <${Popup} notebook=${this.state.notebook} disable_input=${this.state.disable_ui || !this.state.connected /* && this.state.backend_launch_phase == null*/} /> <${RecentlyDisabledInfo} recently_auto_disabled_cells=${this.state.recently_auto_disabled_cells} notebook=${this.state.notebook} /> <${UndoDelete} recently_deleted=${this.state.recently_deleted} on_click=${() => { const rd = this.state.recently_deleted if (rd == null) return this.update_notebook((notebook) => { for (let { index, cell } of rd) { notebook.cell_inputs[cell.cell_id] = cell notebook.cell_order = [...notebook.cell_order.slice(0, index), cell.cell_id, ...notebook.cell_order.slice(index, Infinity)] } }).then(() => { this.actions.set_and_run_multiple(rd.map(({ cell }) => cell.cell_id)) }) }} /> <${SlideControls} /> <footer> <div id="info"> <a href="https://github.com/fonsp/Pluto.jl/wiki" target="_blank">FAQ</a> <span style="flex: 1"></span> <form id="feedback" action="#" method="post"> <label for="opinion"> How can we make <a href="https://plutojl.org/" target="_blank">Pluto.jl</a> better?</label> <input type="text" name="opinion" id="opinion" autocomplete="off" placeholder="Instant feedback..." /> <button>Send</button> </form> </div> </footer> </${PlutoJSInitializingContext.Provider}> </${PlutoBondsContext.Provider}> </${PlutoActionsContext.Provider}> ` } } /* LOCALSTORAGE NOTEBOOKS LIST */ // TODO This is now stored locally, lets store it somewhere central  export const update_stored_recent_notebooks = (recent_path, /** @type {string | undefined} */ also_delete = undefined) => { if (recent_path != null && recent_path !== default_path) { const stored_string = localStorage.getItem("recent notebooks") const stored_list = stored_string != null ? JSON.parse(stored_string) : [] const oldpaths = stored_list const newpaths = [recent_path, ...oldpaths.filter((path) => path !== recent_path && path !== also_delete)] if (!_.isEqual(oldpaths, newpaths)) { localStorage.setItem("recent notebooks", JSON.stringify(newpaths.slice(0, 50))) } } } import { cl } from "../common/ClassTable.js" import { html, useContext, useEffect, useLayoutEffect, useMemo, useRef, useState } from "../imports/Preact.js" import { PlutoActionsContext } from "../common/PlutoContext.js" import { highlight } from "./CellOutput.js" import { PkgTerminalView } from "./PkgTerminalView.js" import _ from "../imports/lodash.js" import { open_bottom_right_panel } from "./BottomRightPanel.js" import { ansi_to_html } from "../imports/AnsiUp.js" import { FixWithAIButton } from "./FixWithAIButton.js" const nbsp = "\u00A0" const extract_cell_id = (/** @type {string} */ file) => { if (file.includes("#@#==#")) return null const sep = "#==#" const sep_index = file.indexOf(sep) if (sep_index != -1) { return file.substring(sep_index + sep.length, sep_index + sep.length + 36) } else { return null } } const focus_line = (cell_id, line) => window.dispatchEvent( new CustomEvent("cell_focus", { detail: { cell_id: cell_id, line: line, }, }) ) const DocLink = ({ frame }) => { let pluto_actions = useContext(PlutoActionsContext) if (extract_cell_id(frame.file)) return null if (frame.parent_module == null) return null if (ignore_funccall(frame)) return null const funcname = frame.func if (funcname === "") return null const nb = pluto_actions.get_notebook() const pkg_name = frame.source_package const builtin = ["Main", "Core", "Base"].includes(pkg_name) const installed = nb?.nbpkg?.installed_versions?.[frame.source_package] != null if (!builtin && nb?.nbpkg != null && !installed) return null return html` ${nbsp}<span ><a href="#" class="doclink" onClick=${(e) => { e.preventDefault() open_bottom_right_panel("docs") pluto_actions.set_doc_query(`${frame.parent_module}.${funcname}`) }} >docs</a ></span >` } const noline = (line) => line == null || line < 1 const StackFrameFilename = ({ frame, cell_id }) => { if (ignore_location(frame)) return null const frame_cell_id = extract_cell_id(frame.file) const line = frame.line if (frame_cell_id != null) { return html`<a internal-file=${frame.file} href=${`#${frame_cell_id}`} onclick=${(e) => { focus_line(frame_cell_id, noline(line) ? null : line - 1) e.preventDefault() }} > ${frame_cell_id == cell_id ? "This\xa0cell" : "Other\xa0cell"}${noline(line) ? null : html`:${nbsp}<em>line${nbsp}${line}</em>`} </a>` } else { const sp = frame.source_package const origin = ["Main", "Core", "Base"].includes(sp) ? "julia" : sp const file_line = html`<em>${frame.file.replace(/#@#==#.*/, "")}${noline(frame.line) ? null : `:${frame.line}`}</em>` const text = sp != null ? html`<strong>${origin}</strong>${nbsp}${nbsp}${file_line}` : file_line const href = frame?.url?.startsWith?.("https") ? frame.url : null return html`<a title=${frame.path} class="remote-url" href=${href}>${text}</a>` } } const at = html`<span> from${nbsp}</span>` const ignore_funccall = (frame) => frame.call === "top-level scope" const ignore_location = (frame) => frame.file === "none" const funcname_args = (call) => { const anon_match = call.indexOf(")(") if (anon_match != -1) { return [call.substring(0, anon_match + 1), call.substring(anon_match + 1)] } else { const bracket_index = call.indexOf("(") if (bracket_index != -1) { return [call.substring(0, bracket_index), call.substring(bracket_index)] } else { return [call, ""] } } } const Funccall = ({ frame }) => { let [expanded_state, set_expanded] = useState(false) useEffect(() => { set_expanded(false) }, [frame]) const silly_to_hide = (frame.call_short.match(//g) ?? "").length <= 1 && frame.call.length < frame.call_short.length + 7 const expanded = expanded_state || (frame.call === frame.call_short && frame.func === funcname_args(frame.call)[0]) || silly_to_hide if (ignore_funccall(frame)) return null const call = expanded ? frame.call : frame.call_short const call_funcname_args = funcname_args(call) const funcname = expanded ? call_funcname_args[0] : frame.func // if function name is #12 or #15#16 then it is an anonymous function const funcname_display = funcname.match(/^#\d+(#\d+)?$/) ? html`<abbr title="A (mini-)function that is defined without the 'function' keyword, but using -> or 'do'.">anonymous function</abbr>` : funcname let inner = html`<strong>${funcname_display}</strong><${HighlightCallArgumentNames} code=${call_funcname_args[1]} />` const id = useMemo(() => Math.random().toString(36).substring(7), [frame]) return html`<mark id=${id}>${inner}</mark> ${!expanded ? html`<a aria-expanded=${expanded} aria-controls=${id} title="Display the complete type information of this function call" role="button" href="#" onClick=${(e) => { e.preventDefault() set_expanded(true) }} >...show types...</a >` : null}` } const LinePreview = ({ frame, num_context_lines = 2 }) => { let pluto_actions = useContext(PlutoActionsContext) let cell_id = extract_cell_id(frame.file) if (cell_id) { let code = /** @type{import("./Editor.js").NotebookData?} */ (pluto_actions.get_notebook())?.cell_inputs[cell_id]?.code if (code) { const lines = code.split("\n") return html`<a onclick=${(e) => { focus_line(cell_id, frame.line - 1) e.preventDefault() }} href=${`#${cell_id}`} class="frame-line-preview" ><div> <pre> ${lines.map((line, i) => frame.line - 1 - num_context_lines <= i && i <= frame.line - 1 + num_context_lines ? html`<${JuliaHighlightedLine} code=${line} i=${i} frameLine=${i === frame.line - 1} />` : null )}</pre > </div></a >` } } } const JuliaHighlightedLine = ({ code, frameLine, i }) => { const code_ref = useRef(/** @type {HTMLPreElement?} */ (null)) useLayoutEffect(() => { if (code_ref.current) { code_ref.current.innerText = code delete code_ref.current.dataset.highlighted highlight(code_ref.current, "julia") } }, [code_ref.current, code]) return html`<code ref=${code_ref} style=${`--before-content: "${i + 1}";`} class=${cl({ "language-julia": true, "frame-line": frameLine, })} ></code>` } const HighlightCallArgumentNames = ({ code }) => { const code_ref = useRef(/** @type {HTMLPreElement?} */ (null)) useLayoutEffect(() => { if (code_ref.current) { const html = code.replaceAll(/([^():{},; ]*)::/g, "<span class='argument_name'>$1</span>::") code_ref.current.innerHTML = html } }, [code_ref.current, code]) return html`<s-span ref=${code_ref} class="language-julia"></s-span>` } const insert_commas_and_and = (/** @type {any[]} */ xs) => xs.flatMap((x, i) => (i === xs.length - 1 ? [x] : i === xs.length - 2 ? [x, " and "] : [x, ", "])) export const ParseError = ({ cell_id, diagnostics, last_run_timestamp }) => { useEffect(() => { window.dispatchEvent( new CustomEvent("cell_diagnostics", { detail: { cell_id, diagnostics, }, }) ) return () => window.dispatchEvent(new CustomEvent("cell_diagnostics", { detail: { cell_id, diagnostics: [] } })) }, [diagnostics]) return html` <jlerror class="syntax-error"> <header> <p>Syntax error</p> <${FixWithAIButton} cell_id=${cell_id} diagnostics=${diagnostics} last_run_timestamp=${last_run_timestamp} /> </header> <section> <div class="stacktrace-header"> <secret-h1>Syntax errors</secret-h1> </div> <ol> ${diagnostics.map( ({ message, from, to, line }) => html`<li class="from_this_notebook from_this_cell important" onmouseenter=${() => cell_is_unedited(cell_id) ? window.dispatchEvent(new CustomEvent("cell_highlight_range", { detail: { cell_id, from, to } })) : null} onmouseleave=${() => window.dispatchEvent(new CustomEvent("cell_highlight_range", { detail: { cell_id, from: null, to: null } }))} > <div class="classical-frame"> ${message} <div class="frame-source">${at}<${StackFrameFilename} frame=${{ file: "#==#" + cell_id, line }} cell_id=${cell_id} /></div> </div> </li>` )} </ol> </section> </jlerror> ` } const cell_is_unedited = (cell_id) => document.querySelector(`pluto-cell[id="${cell_id}"].code_differs`) == null const frame_is_important_heuristic = (frame, frame_index, limited_stacktrace, frame_cell_id) => { if (frame_cell_id != null) return true const [funcname, params] = funcname_args(frame.call) if (["_collect", "collect_similar", "iterate", "error", "macro expansion"].includes(funcname)) { return false } if (funcname.includes("throw")) return false // too sciency if (frame.inlined) return false // makes no sense anyways if (frame.line < 1) return false if (params == null) { // no type signature... must be some function call that got optimized away or something special // probably not directly relevant return false } if ((funcname.match(/#/g) ?? "").length >= 2) { // anonymous function: #plot#142 return false } return true } const AnsiUpLine = (/** @type {{value: string}} */ { value }) => { const node_ref = useRef(/** @type {HTMLElement?} */ (null)) const did_ansi_up = useRef(false) useLayoutEffect(() => { if (!node_ref.current) return node_ref.current.innerHTML = ansi_to_html(value) did_ansi_up.current = true }, [node_ref.current, value]) // placeholder while waiting for AnsiUp to render, to prevent layout flash const without_ansi_chars = value.replace(/\u001b\[[0-9;]*m/g, "") return value === "" ? html`<p><br /></p>` : html`<p ref=${node_ref}>${did_ansi_up.current ? null : without_ansi_chars}</p>` } export const ErrorMessage = ({ msg, stacktrace, plain_error, cell_id }) => { let pluto_actions = useContext(PlutoActionsContext) const default_rewriter = { pattern: /.?/, display: (/** @type{string} */ x) => _.dropRightWhile(x.split("\n"), (s) => s === "").map((line) => html`<${AnsiUpLine} value=${line} />`), } const rewriters = [ { pattern: /syntax: extra token after end of expression/, display: (/** @type{string} */ x) => { const begin_hint = html`<a href="#" onClick=${(e) => { e.preventDefault() pluto_actions.wrap_remote_cell(cell_id, "begin") }} >Wrap all code in a <em>begin ... end</em> block.</a >` if (x.includes("\n\nBoundaries: ")) { const boundaries = JSON.parse(x.split("\n\nBoundaries: ")[1]).map((x) => x - 1) // Julia to JS index const split_hint = html`<p> <a href="#" onClick=${(e) => { e.preventDefault() pluto_actions.split_remote_cell(cell_id, boundaries, true) }} >Split this cell into ${boundaries.length} cells</a >, or </p>` return html`<p>Multiple expressions in one cell.</p> <p>How would you like to fix it?</p> <ul> <li>${split_hint}</li> <li>${begin_hint}</li> </ul>` } else { return html`<p>Multiple expressions in one cell.</p> <p>${begin_hint}</p>` } }, show_stacktrace: () => false, }, { pattern: /LoadError: cannot assign a value to variable workspace#\d+\..+ from module workspace#\d+/, display: () => html`<p>Tried to reevaluate an <code>include</code> call, this is not supported. You might need to restart this notebook from the main menu.</p> <p> For a workaround, use the alternative version of <code>include</code> described here: <a target="_blank" href="https://github.com/fonsp/Pluto.jl/issues/115#issuecomment-661722426">GH issue 115</a> </p> <p>In the future, <code>include</code> will be deprecated, and this will be the default.</p>`, }, { pattern: /MethodError: no method matching .*\nClosest candidates are:/, display: (/** @type{string} */ x) => x.split("\n").map((line) => html`<p style="white-space: nowrap;">${line}</p>`), }, { pattern: /Cyclic references among (.*)\./, display: (/** @type{string} */ x) => x.split("\n").map((line) => { const match = line.match(/Cyclic references among (.*)\./) if (match) { let syms_string = match[1] let syms = syms_string.split(/, | and /) let symbol_links = syms.map((what) => html`<a href="#${encodeURI(what)}">${what}</a>`) return html`<p>Cyclic references among${" "}${insert_commas_and_and(symbol_links)}.</p>` } else { return html`<p>${line}</p>` } }), }, { pattern: /Multiple definitions for (.*)/, display: (/** @type{string} */ x) => x.split("\n").map((line) => { const match = line.match(/Multiple definitions for (.*)/) if (match) { // replace: remove final dot let syms_string = match[1].replace(/\.$/, "") let syms = syms_string.split(/, | and /) let symbol_links = syms.map((what) => { const onclick = (ev) => { const where = document.querySelector(`pluto-cell:not([id='${cell_id}']) span[id='${encodeURI(what)}']`) ev.preventDefault() where?.scrollIntoView() } return html`<a href="#" onclick=${onclick}>${what}</a>` }) return html`<p>Multiple definitions for${" "}${insert_commas_and_and(symbol_links)}.</p>` } else { return html`<p>${line}</p>` } }), }, { pattern: /^syntax: (.*)$/, display: default_rewriter.display, show_stacktrace: () => false, }, { pattern: /^\s*$/, display: () => default_rewriter.display("Error"), }, { pattern: /^UndefVarError: (.*) not defined/, display: (/** @type{string} */ x) => { const notebook = /** @type{import("./Editor.js").NotebookData?} */ (pluto_actions.get_notebook()) const erred_upstreams = get_erred_upstreams(notebook, cell_id) // Verify that the UndefVarError is indeed about a variable from an upstream cell. const match = x.match(/UndefVarError: (.*) not defined/) let sym = (match?.[1] ?? "").replaceAll("`", "") const undefvar_is_from_upstream = Object.values(notebook?.cell_dependencies ?? {}).some((map) => Object.keys(map.downstream_cells_map).includes(sym) ) if (Object.keys(erred_upstreams).length === 0 || !undefvar_is_from_upstream) { return html`<p>${x}</p>` } const symbol_links = Object.keys(erred_upstreams).map((key) => { const onclick = (ev) => { ev.preventDefault() const where = document.querySelector(`pluto-cell[id='${erred_upstreams[key]}']`) where?.scrollIntoView() } return html`<a href="#" onclick=${onclick}>${key}</a>` }) // const plural = symbol_links.length > 1 return html`<p><em>Another cell defining ${insert_commas_and_and(symbol_links)} contains errors.</em></p>` }, show_stacktrace: () => { const erred_upstreams = get_erred_upstreams(pluto_actions.get_notebook(), cell_id) return Object.keys(erred_upstreams).length === 0 }, }, { pattern: /^ArgumentError: Package (.*) not found in current path/, display: (/** @type{string} */ x) => { if (pluto_actions.get_notebook().nbpkg?.enabled === false) return default_rewriter.display(x) const match = x.match(/^ArgumentError: Package (.*) not found in current path/) const package_name = (match?.[1] ?? "").replaceAll("`", "") const pkg_terminal_value = pluto_actions.get_notebook()?.nbpkg?.terminal_outputs?.[package_name] return html`<p>The package <strong>${package_name}.jl</strong> could not load because it failed to initialize.</p> <p>That's not nice! Things you could try:</p> <ul> <li>Restart the notebook.</li> <li>Try a different Julia version.</li> <li>Contact the developers of ${package_name}.jl about this error.</li> </ul> ${pkg_terminal_value == null ? null : html` <p>You might find useful information in the package installation log:</p> <${PkgTerminalView} value=${pkg_terminal_value} />`} ` }, show_stacktrace: () => false, }, default_rewriter, ] const matched_rewriter = rewriters.find(({ pattern }) => pattern.test(msg)) ?? default_rewriter const [show_more, set_show_more] = useState(false) useEffect(() => { set_show_more(false) }, [msg, stacktrace, cell_id]) const first_stack_from_here = stacktrace.findIndex((frame) => extract_cell_id(frame.file) != null) const limited = !show_more && first_stack_from_here != -1 && first_stack_from_here < stacktrace.length - 1 const limited_stacktrace = (limited ? stacktrace.slice(0, first_stack_from_here + 1) : stacktrace).filter( (frame) => !(ignore_location(frame) && ignore_funccall(frame)) ) const first_package = get_first_package(limited_stacktrace) const [stacktrace_waiting_to_view, set_stacktrace_waiting_to_view] = useState(true) useEffect(() => { set_stacktrace_waiting_to_view(true) }, [msg, stacktrace, cell_id]) return html`<jlerror> <div class="error-header"> <secret-h1>Error message${first_package == null ? null : ` from ${first_package}`}</secret-h1> <!-- <p>This message was included with the error:</p> --> </div> <header>${matched_rewriter.display(msg)}</header> ${stacktrace.length == 0 || !(matched_rewriter.show_stacktrace?.() ?? true) ? null : stacktrace_waiting_to_view ? html`<section class="stacktrace-waiting-to-view"> <button onClick=${() => set_stacktrace_waiting_to_view(false)}>Show stack trace...</button> </section>` : html`<section> <div class="stacktrace-header"> <secret-h1>Stack trace</secret-h1> <p>Here is what happened, the most recent locations are first:</p> </div> <ol> ${limited_stacktrace.map((frame, frame_index) => { const frame_cell_id = extract_cell_id(frame.file) const from_this_notebook = frame_cell_id != null const from_this_cell = cell_id === frame_cell_id const important = frame_is_important_heuristic(frame, frame_index, limited_stacktrace, frame_cell_id) return html`<li class=${cl({ from_this_notebook, from_this_cell, important })}> <div class="classical-frame"> <${Funccall} frame=${frame} /> <div class="frame-source"> ${at}<${StackFrameFilename} frame=${frame} cell_id=${cell_id} /> <${DocLink} frame=${frame} /> </div> </div> ${from_this_notebook ? html`<${LinePreview} frame=${frame} num_context_lines=${from_this_cell ? 1 : 2} />` : null} </li>` })} ${limited ? html`<li class="important"> <a href="#" onClick=${(e) => { set_show_more(true) e.preventDefault() }} >Show more...</a > </li>` : null} </ol> </section>`} ${pluto_actions.get_session_options?.()?.server?.dismiss_motivational_quotes !== true ? html`<${Motivation} stacktrace=${stacktrace} />` : null} </jlerror>` } const get_first_package = (limited_stacktrace) => { for (let [i, frame] of limited_stacktrace.entries()) { const frame_cell_id = extract_cell_id(frame.file) if (frame_cell_id) return undefined const important = frame_is_important_heuristic(frame, i, limited_stacktrace, frame_cell_id) if (!important) continue if (frame.source_package) return frame.source_package } } const motivational_word_probability = 0.1 const motivational_words = [ // "Don't panic!", "Keep calm, you got this!", "You got this!", "Goofy computer!", "This one is on the computer!", "beep boop CRASH ", "computer bad, you GREAT!", "Probably not your fault!", "Try asking on Julia Discourse!", "uhmmmmmm??!", "Maybe time for a break? ", "Everything is going to be okay!", "Computers are hard!", "C'est la vie !", "\\_()_/", "Oh no! ", "oopsie ", "Be patient :)", ] const Motivation = ({ stacktrace }) => { const msg = useMemo(() => { return Math.random() < motivational_word_probability ? motivational_words[Math.floor(Math.random() * motivational_words.length)] : null }, [stacktrace]) return msg == null ? null : html`<div class="dont-panic">${msg}</div>` } const get_erred_upstreams = ( /** @type {import("./Editor.js").NotebookData?} */ notebook, /** @type {string} */ cell_id, /** @type {string[]} */ visited_edges = [] ) => { let erred_upstreams = {} if (notebook != null && notebook?.cell_results?.[cell_id]?.errored) { const referenced_variables = Object.keys(notebook.cell_dependencies[cell_id]?.upstream_cells_map) referenced_variables.forEach((key) => { if (!visited_edges.includes(key)) { visited_edges.push(key) const cells_that_define_this_variable = notebook.cell_dependencies[cell_id]?.upstream_cells_map[key] cells_that_define_this_variable.forEach((upstream_cell_id) => { let upstream_errored_cells = get_erred_upstreams(notebook, upstream_cell_id, visited_edges) ?? {} erred_upstreams = { ...erred_upstreams, ...upstream_errored_cells } // if upstream got no errors and current cell is errored // then current cell is responsible for errors if (Object.keys(upstream_errored_cells).length === 0 && notebook.cell_results[upstream_cell_id].errored && upstream_cell_id !== cell_id) { erred_upstreams[key] = upstream_cell_id } }) } }) } return erred_upstreams } //@ts-ignore import dialogPolyfill from "https://cdn.jsdelivr.net/npm/dialog-polyfill@0.5.6/dist/dialog-polyfill.esm.min.js" import { useEventListener } from "../common/useEventListener.js" import { html, useLayoutEffect, useRef } from "../imports/Preact.js" const Circle = ({ fill }) => html` <svg width="48" height="48" viewBox="0 0 48 48" style=" height: .7em; width: .7em; margin-left: .3em; margin-right: .2em; " > <circle cx="24" cy="24" r="24" fill=${fill}></circle> </svg> ` const Triangle = ({ fill }) => html` <svg width="48" height="48" viewBox="0 0 48 48" style="height: .7em; width: .7em; margin-left: .3em; margin-right: .2em; margin-bottom: -.1em;"> <polygon points="24,0 48,40 0,40" fill=${fill} stroke="none" /> </svg> ` const Square = ({ fill }) => html` <svg width="48" height="48" viewBox="0 0 48 48" style="height: .7em; width: .7em; margin-left: .3em; margin-right: .2em; margin-bottom: -.1em;"> <polygon points="0,0 0,40 40,40 40,0" fill=${fill} stroke="none" /> </svg> ` export const WarnForVisisblePasswords = () => { if ( Array.from(document.querySelectorAll("bond")).some((bond_el) => Array.from(bond_el.querySelectorAll(`input[type="password"]`)).some((input) => { // @ts-ignore if (input?.value !== "") { input.scrollIntoView() return true } }) ) ) { alert( "Warning: this notebook includes a password input with something typed in it. The contents of this password field will be included in the exported file in an unsafe way. \n\nClear the password field and export again to avoid this problem." ) } } export const ExportBanner = ({ notebook_id, print_title, open, onClose, notebookfile_url, notebookexport_url, start_recording }) => { // @ts-ignore const isDesktop = !!window.plutoDesktop const exportNotebook = (/** @type {{ preventDefault: () => void; }} */ e, /** @type {Desktop.PlutoExport} */ type) => { if (isDesktop) { e.preventDefault() window.plutoDesktop?.fileSystem.exportNotebook(notebook_id, type) } } // let print_old_title_ref = useRef("") useEventListener( window, "beforeprint", (e) => { if (!e.detail?.fake) { print_old_title_ref.current = document.title document.title = print_title.replace(/\.jl$/, "").replace(/\.plutojl$/, "") } }, [print_title] ) useEventListener( window, "afterprint", () => { document.title = print_old_title_ref.current }, [print_title] ) const element_ref = useRef(/** @type {HTMLDialogElement?} */ (null)) useLayoutEffect(() => { if (element_ref.current != null && typeof HTMLDialogElement !== "function") dialogPolyfill.registerDialog(element_ref.current) }, [element_ref.current]) useLayoutEffect(() => { // Closing doesn't play well if the browser is old and the dialog not open // https://github.com/GoogleChrome/dialog-polyfill/issues/149 if (open) { element_ref.current?.open === false && element_ref.current?.show?.() } else { element_ref.current?.open && element_ref.current?.close?.() } }, [open, element_ref.current]) const onCloseRef = useRef(onClose) onCloseRef.current = onClose useEventListener( element_ref.current, "focusout", () => { if (!element_ref.current?.matches(":focus-within")) onCloseRef.current() }, [] ) const pride = true const prideMonth = new Date().getMonth() === 5 return html` <dialog id="export" inert=${!open} open=${open} ref=${element_ref} class=${prideMonth ? "pride" : ""}> <div id="container"> <div class="export_title">export</div> <!-- no "download" attribute here: we want the jl contents to be shown in a new tab --> <a href=${notebookfile_url} target="_blank" class="export_card" onClick=${(e) => exportNotebook(e, 0)}> <header role="none"><${Triangle} fill="#a270ba" /> Notebook file</header> <section>Download a copy of the <b>.jl</b> script.</section> </a> <a href=${notebookexport_url} target="_blank" class="export_card" download="" onClick=${(e) => { WarnForVisisblePasswords() exportNotebook(e, 1) }} > <header role="none"><${Square} fill="#E86F51" /> Static HTML</header> <section>An <b>.html</b> file for your web page, or to share online.</section> </a> <a href="#" class="export_card" onClick=${() => window.print()}> <header role="none"><${Square} fill="#619b3d" /> PDF</header> <section>A static <b>.pdf</b> file for print or email.</section> </a> ${html` <div class="export_title">record</div> <a href="#" onClick=${(e) => { WarnForVisisblePasswords() start_recording() onClose() e.preventDefault() }} class="export_card" > <header role="none"><${Circle} fill="#E86F51" /> Record <em>(preview)</em></header> <section>Capture the entire notebook, and any changes you make.</section> </a> `} ${prideMonth ? html`<div class="pride_message"> <p>The future is <strong>queer</strong>!</p> </div>` : null} </div> <div class="export_small_btns"> <button title="Edit frontmatter" class="toggle_frontmatter_edit" onClick=${() => { onClose() window.dispatchEvent(new CustomEvent("open pluto frontmatter")) }} > <span></span> </button> <button title="Start presentation" class="toggle_presentation" onClick=${() => { onClose() // @ts-ignore window.present() }} > <span></span> </button> <button title="Close" class="toggle_export" onClick=${() => onClose()}> <span></span> </button> </div> </dialog> ` } import { html } from "../imports/Preact.js" export const read_Uint8Array_with_progress = async (/** @type {Response} */ response, on_progress) => { if (response.body != null) { const length_str = response.headers.get("Content-Length") const length = length_str == null ? null : Number(length_str) const reader = response.body.getReader() let receivedLength = 0 let chunks = [] while (true) { const { done, value } = await reader.read() if (done) { break } chunks.push(value) receivedLength += value.length if (length != null) { on_progress(Math.min(1, receivedLength / length)) } else { // fake progress: 50% at 1MB, 67% at 2MB, 75% at 3MB, etc. const z = 1e6 on_progress(1.0 - z / (receivedLength - z)) } console.log({ receivedLength }) } on_progress(1) const buffer = new Uint8Array(receivedLength) let position = 0 for (let chunk of chunks) { buffer.set(chunk, position) position += chunk.length } return buffer } else { return new Uint8Array(await response.arrayBuffer()) } } export const FetchProgress = ({ progress }) => progress == null || progress === 1 ? null : html`<progress class="statefile-fetch-progress" max="100" value=${progress === "indeterminate" ? undefined : Math.round(progress * 100)}> ${progress === "indeterminate" ? null : Math.round(progress * 100)}% </progress>` import { html, Component, useState, useRef, useEffect, useLayoutEffect } from "../imports/Preact.js" import { utf8index_to_ut16index } from "../common/UnicodeTools.js" import { map_cmd_to_ctrl_on_mac } from "../common/KeyboardShortcuts.js" import { EditorState, EditorSelection, EditorView, placeholder as Placeholder, keymap, history, autocomplete, drawSelection, Compartment, StateEffect, } from "../imports/CodemirrorPlutoSetup.js" import { guess_notebook_location } from "../common/NotebookLocationFromURL.js" import { tab_help_plugin } from "./CellInput/tab_help_plugin.js" import _ from "../imports/lodash.js" let { autocompletion, completionKeymap } = autocomplete let start_autocomplete_command = completionKeymap.find((keybinding) => keybinding.key === "Ctrl-Space") let accept_autocomplete_command = completionKeymap.find((keybinding) => keybinding.key === "Enter") let close_autocomplete_command = completionKeymap.find((keybinding) => keybinding.key === "Escape") const assert_not_null = (x) => { if (x == null) { throw new Error("Unexpected null value") } else { return x } } const set_cm_value = (/** @type{EditorView} */ cm, /** @type {string} */ value, scroll = true) => { cm.dispatch({ changes: { from: 0, to: cm.state.doc.length, insert: value }, selection: EditorSelection.cursor(value.length), // a long path like /Users/fons/Documents/article-test-1/asdfasdfasdfsadf.jl does not fit in the little box, so we scroll it to the left so that you can see the filename easily. scrollIntoView: scroll, }) } const is_desktop = !!window.plutoDesktop if (is_desktop) { console.log("Running in Desktop Environment! Found following properties/methods:", window.plutoDesktop) } /** * @param {{ * value: String, * suggest_new_file: {base: String}, * button_label: String, * placeholder: String, * on_submit: (new_path: String) => Promise<void>, * on_desktop_submit?: (loc?: string) => Promise<void>, * client: import("../common/PlutoConnection.js").PlutoConnection, * clear_on_blur: Boolean, * }} props */ export const FilePicker = ({ value, suggest_new_file, button_label, placeholder, on_submit, on_desktop_submit, client, clear_on_blur }) => { const [is_button_disabled, set_is_button_disabled] = useState(true) const [url_value, set_url_value] = useState("") const forced_value = useRef("") /** @type {import("../imports/Preact.js").Ref<HTMLElement>} */ const base = useRef(/** @type {any} */ (null)) const cm = useRef(/** @type {EditorView?} */ (null)) const suggest_not_tmp = () => { const current_cm = cm.current if (current_cm == null) return if (suggest_new_file != null && current_cm.state.doc.length === 0) { // current_cm.focus() set_cm_value(current_cm, suggest_new_file.base, false) request_path_completions() } window.dispatchEvent(new CustomEvent("collapse_cell_selection", {})) } let run = async (fn) => await fn() const onSubmit = () => { const current_cm = cm.current if (current_cm == null) return if (!is_desktop) { const my_val = current_cm.state.doc.toString() if (my_val === forced_value.current) { suggest_not_tmp() return true } } run(async () => { try { if (is_desktop && on_desktop_submit) { await on_desktop_submit((await guess_notebook_location(url_value)).path_or_url) } else { await on_submit(current_cm.state.doc.toString()) } current_cm.dom.blur() } catch (error) { set_cm_value(current_cm, forced_value.current, true) current_cm.dom.blur() } }) return true } const onBlur = (e) => { const still_in_focus = base.current?.matches(":focus-within") || base.current?.contains(e.relatedTarget) if (still_in_focus) return const current_cm = cm.current if (current_cm == null) return if (clear_on_blur) requestAnimationFrame(() => { if (!current_cm.hasFocus) { set_cm_value(current_cm, forced_value.current, true) } }) } const request_path_completions = () => { const current_cm = cm.current if (current_cm == null) return let selection = current_cm.state.selection.main if (selection.from !== selection.to) return if (current_cm.state.doc.length !== selection.to) return return assert_not_null(start_autocomplete_command).run(current_cm) } useLayoutEffect(() => { const usesDarkTheme = window.matchMedia("(prefers-color-scheme: dark)").matches const keyMapSubmit = () => { onSubmit() return true } cm.current = new EditorView({ state: EditorState.create({ doc: "", extensions: [ drawSelection(), EditorView.domEventHandlers({ focus: (event, cm) => { setTimeout(() => { if (suggest_new_file) { suggest_not_tmp() } else { request_path_completions() } }, 0) return true }, }), EditorView.updateListener.of((update) => { if (update.docChanged) { set_is_button_disabled(update.state.doc.length === 0) } }), EditorView.theme( { "&": { fontSize: "inherit", }, ".cm-scroller": { fontFamily: "inherit", overflowY: "hidden", overflowX: "auto", }, }, { dark: usesDarkTheme } ), // EditorView.updateListener.of(onCM6Update), history(), autocompletion({ activateOnTyping: true, override: [ pathhints({ suggest_new_file: suggest_new_file, client: client, }), ], defaultKeymap: false, // We add these manually later, so we can override them if necessary maxRenderedOptions: 512, // fons's magic number optionClass: (c) => c.type ?? "", }), // When a completion is picked, immediately start autocompleting again EditorView.updateListener.of((update) => { update.transactions.forEach((transaction) => { const completion = transaction.annotation(autocomplete.pickedCompletion) if (completion != null) { update.view.dispatch({ effects: EditorView.scrollIntoView(update.state.doc.length), selection: EditorSelection.cursor(update.state.doc.length), }) request_path_completions() } }) }), keymap.of([ { key: "Enter", run: (cm) => { // If there is autocomplete open, accept that. It will return `true` return assert_not_null(accept_autocomplete_command).run(cm) }, }, { key: "Enter", run: keyMapSubmit, }, { key: "Ctrl-Enter", mac: "Cmd-Enter", run: keyMapSubmit, }, { key: "Ctrl-Shift-Enter", mac: "Cmd-Shift-Enter", run: keyMapSubmit, }, { key: "Tab", run: (cm) => { // If there is autocomplete open, accept that if (assert_not_null(accept_autocomplete_command).run(cm)) { // and request the next ones request_path_completions() return true } // Else, activate it (possibly) return request_path_completions() }, }, ]), keymap.of(completionKeymap), Placeholder(placeholder), tab_help_plugin, ], }), }) const current_cm = cm.current if (!is_desktop) base.current.insertBefore(current_cm.dom, base.current.firstElementChild) // window.addEventListener("resize", () => { // if (!cm.current.hasFocus()) { // deselect(cm.current) // } // }) }, []) useLayoutEffect(() => { if (forced_value.current != value) { if (cm.current == null) return set_cm_value(cm.current, value, true) forced_value.current = value } }) return is_desktop ? html`<div class="desktop_picker_group" ref=${base}> <input value=${url_value} placeholder="Enter notebook URL..." onChange=${(v) => { set_url_value(v.target.value) }} /> <div onClick=${onSubmit} class="desktop_picker"> <button>${button_label}</button> </div> </div>` : html` <pluto-filepicker ref=${base} onfocusout=${onBlur}> <button onClick=${onSubmit} disabled=${is_button_disabled}>${button_label}</button> </pluto-filepicker> ` } const dirname = (/** @type {string} */ str) => { // using regex /\/|\\/ const idx = [...str.matchAll(/[\/\\]/g)].map((r) => r.index) return idx.length > 0 ? str.slice(0, _.last(idx) + 1) : str } const basename = (/** @type {string} */ str) => (str.split("/").pop() ?? "").split("\\").pop() ?? "" const pathhints = ({ client, suggest_new_file }) => /** @type {autocomplete.CompletionSource} */ (ctx) => { const query_full = /** @type {String} */ (ctx.state.sliceDoc(0, ctx.pos)) const query = dirname(query_full) return client .send("completepath", { query, }) .then((update) => { const queryFileName = basename(query_full) const results = update.message.results const from = utf8index_to_ut16index(query, update.message.start) // if the typed text matches one of the paths exactly, stop autocomplete immediately. if (results.includes(queryFileName)) { return null } let styledResults = results.map((r) => { let dir = r.endsWith("/") || r.endsWith("\\") return { label: r, type: dir ? "dir" : "file", boost: dir ? 1 : 0, } }) if (suggest_new_file != null) { for (let initLength = 3; initLength >= 0; initLength--) { const init = ".jl".substring(0, initLength) if (queryFileName.endsWith(init)) { let suggestedFileName = queryFileName + ".jl".substring(initLength) if (suggestedFileName == ".jl") { suggestedFileName = "notebook.jl" } if (initLength == 3) { return null } if (!results.includes(suggestedFileName)) { styledResults.push({ label: suggestedFileName + " (new)", apply: suggestedFileName, type: "file new", boost: -99, }) } break } } } const validFor = (/** @type {string} */ text) => { return ( /[\p{L}\p{Nl}\p{Sc}\d_!-\.]*$/u.test(text) && // if the typed text matches one of the paths exactly, stop autocomplete immediately. !results.includes(basename(text)) ) } return { options: styledResults, from: from, validFor, } }) } import { html, useContext, useRef, useState, useEffect } from "../imports/Preact.js" import { PlutoActionsContext } from "../common/PlutoContext.js" import { cl } from "../common/ClassTable.js" import { open_pluto_popup } from "../common/open_pluto_popup.js" import { start_ai_suggestion } from "./CellInput/ai_suggestion.js" const ai_server_url = "https://pluto-simple-llm-features.deno.dev/" const endpoint_url = `${ai_server_url}fix-syntax-error-v1` const pluto_premium_llm_key = localStorage.getItem("pluto_premium_llm_key") // Server availability state management let serverAvailabilityPromise = null const checkServerAvailability = async () => { if (serverAvailabilityPromise === null) { serverAvailabilityPromise = Promise.all([ // Check our AI endpoint fetch(endpoint_url, { method: "GET", }) .then((response) => response.ok) .catch(() => { console.warn("AI features disabled: Unable to access Pluto AI server. This may be due to network restrictions.") return false }), // Check if ChatGPT domain is accessible. If not, then the uni has blocked the domain (probably) and we want to disable AI features. fetch("https://chat.openai.com/favicon.ico", { method: "HEAD", mode: "no-cors", }) .then(() => true) .catch(() => { console.warn("AI features disabled: Unable to access ChatGPT domain. This may be due to network restrictions.") return false }), ]).then(([endpointAvailable, chatGPTAvailable]) => endpointAvailable && chatGPTAvailable) } return serverAvailabilityPromise } const AIPermissionPrompt = ({ onAccept, onDecline }) => { const [dontAskAgain, setDontAskAgain] = useState(false) const handleAccept = () => { if (dontAskAgain) { localStorage.setItem("pluto_ai_permission_syntax_v1", "granted") } onAccept() } const handleDecline = () => { onDecline() } return html` <div class="ai-permission-prompt"> <h3>Use AI to fix syntax errors?</h3> <p>Pluto will send code from this cell to a commericial LLM service to fix syntax errors. Updated code will not run without confirmation.</p> <p>Submitted code can be used (anonymously) by Pluto developers to improve the AI service.</p> <label class="ask-next-time"> <input type="checkbox" checked=${dontAskAgain} onChange=${(e) => setDontAskAgain(e.target.checked)} /> Don't ask again </label> <div class="button-group" role="group"> <button onClick=${handleDecline} class="decline" title="Decline AI syntax fix and close">No</button> <button onClick=${handleAccept} class="accept" title="Accept AI syntax fix and close">Yes</button> </div> </div> ` } export const FixWithAIButton = ({ cell_id, diagnostics, last_run_timestamp }) => { const pluto_actions = useContext(PlutoActionsContext) if (pluto_actions.get_session_options?.()?.server?.enable_ai_editor_features === false) return null const node_ref = useRef(/** @type {HTMLElement?} */ (null)) const [buttonState, setButtonState] = useState("initial") // "initial" | "loading" | "success" const [showButton, setShowButton] = useState(false) // Reset whenever a prop changes useEffect(() => { setButtonState("initial") }, [cell_id, diagnostics, last_run_timestamp]) // Check server availability when component mounts useEffect(() => { checkServerAvailability().then((available) => { setShowButton(available) }) }, []) // Don't render anything if server is not available if (!showButton) { return null } const handleFixWithAI = async () => { // Check if we have permission stored const storedPermission = localStorage.getItem("pluto_ai_permission_syntax_v1") if (storedPermission !== "granted") { // Show permission prompt open_pluto_popup({ type: "info", source_element: node_ref.current, body: html`<${AIPermissionPrompt} onAccept=${async () => { window.dispatchEvent(new CustomEvent("close pluto popup")) await performFix() }} onDecline=${() => { window.dispatchEvent(new CustomEvent("close pluto popup")) }} />`, }) return } await performFix() } const original_code_ref = useRef("") const performFix = async () => { try { setButtonState("loading") // Get the current cell's code const notebook = pluto_actions.get_notebook() const code = notebook?.cell_inputs[cell_id]?.code original_code_ref.current = code if (!code) { throw new Error("Could not find cell code") } // Combine all diagnostic messages into a single error message const error_message = diagnostics.map((d) => d.message).join("\n") const response = await fetch(endpoint_url, { method: "POST", headers: { "Content-Type": "application/json", ...(pluto_premium_llm_key ? { "X-Pluto-Premium-LLM-Key": pluto_premium_llm_key } : {}), }, body: JSON.stringify({ code, error_message, }), }) if (!response.ok) { const error = await response.json() throw new Error(error.error || "Failed to fix syntax error") } const { fixed_code } = await response.json() console.debug("fixed_code", fixed_code) // Update the cell's local code without running it if (fixed_code.trim() == "missing") throw new Error("Failed to fix syntax error") await start_ai_suggestion(node_ref.current, { code: fixed_code }) setButtonState("success") } catch (error) { console.error("Error fixing syntax:", error) setButtonState("initial") // Show error to user in UI open_pluto_popup({ type: "warn", source_element: node_ref.current, body: html`<p>Failed to fix syntax error: ${error.message}</p>`, }) } } const handleRejectAI = async () => { await start_ai_suggestion(node_ref.current, { code: original_code_ref.current, reject: true }) setButtonState("initial") } const handleRunCell = async () => { await pluto_actions.set_and_run_multiple([cell_id]) } return html`<div class=${cl({ "fix-with-ai": true, [`fix-with-ai-${buttonState}`]: true, })} > <button ref=${node_ref} onClick=${buttonState === "success" ? handleRunCell : handleFixWithAI} title=${buttonState === "success" ? "Run the fixed cell" : "Attempt to fix this syntax error using an LLM service"} aria-busy=${buttonState === "loading"} aria-live="polite" disabled=${buttonState === "loading"} > ${buttonState === "success" ? "Accept & Run" : buttonState === "loading" ? "Loading..." : "Fix syntax with AI"} </button> ${buttonState === "success" ? html`<button onClick=${handleRejectAI} class="reject-ai-fix" title="Reject the AI fix and revert to original code">Reject</button>` : null} </div>` } import { html, useRef, useLayoutEffect, useState, useEffect, useCallback } from "../imports/Preact.js" import { has_ctrl_or_cmd_pressed } from "../common/KeyboardShortcuts.js" import _ from "../imports/lodash.js" import "https://cdn.jsdelivr.net/gh/fonsp/rebel-tag-input@1.0.6/lib/rebel-tag-input.mjs" //@ts-ignore import immer from "../imports/immer.js" import { useDialog } from "../common/useDialog.js" import { FeaturedCard } from "./welcome/FeaturedCard.js" import { useEventListener } from "../common/useEventListener.js" /** * @param {{ * filename: String, * remote_frontmatter: Record<String,any>?, * set_remote_frontmatter: (newval: Record<String,any>) => Promise<void>, * }} props * */ export const FrontMatterInput = ({ filename, remote_frontmatter, set_remote_frontmatter }) => { const [frontmatter, set_frontmatter] = useState(remote_frontmatter ?? {}) useEffect(() => { set_frontmatter(remote_frontmatter ?? {}) }, [remote_frontmatter]) const fm_setter = (key) => (value) => set_frontmatter( immer((fm) => { _.set(fm, key, value) }) ) const [dialog_ref, open, close, _toggle] = useDialog() const cancel = () => { set_frontmatter(remote_frontmatter ?? {}) close() } const set_remote_frontmatter_ref = useRef(set_remote_frontmatter) set_remote_frontmatter_ref.current = set_remote_frontmatter const submit = useCallback(() => { set_remote_frontmatter_ref .current(clean_data(frontmatter) ?? {}) .then(() => alert("Frontmatter synchronized \n\nThese parameters will be used in future exports.")) close() }, [clean_data, frontmatter, close]) useEventListener(window, "open pluto frontmatter", open) useEventListener( window, "keydown", (e) => { if (dialog_ref.current != null) if (dialog_ref.current.contains(e.target)) if (e.key === "Enter" && has_ctrl_or_cmd_pressed(e)) submit() }, [submit] ) const frontmatter_with_defaults = { title: null, description: null, date: null, tags: [], author: [{}], ...frontmatter, } const show_entry = ([key, value]) => !((_.isArray(value) && field_type(key) !== "tags") || _.isPlainObject(value)) const entries_input = (data, base_path) => { return html` ${Object.entries(data) .filter(show_entry) .map(([key, value]) => { let path = `${base_path}${key}` let id = `fm-${path}` return html` <label for=${id}>${key}</label> <${Input} type=${field_type(key)} id=${id} value=${value} on_value=${fm_setter(path)} /> <button class="deletefield" title="Delete field" aria-label="Delete field" onClick=${() => { // TODO set_frontmatter( immer((fm) => { _.unset(fm, path) }) ) }} >  </button> ` })} <button class="addentry" onClick=${() => { const fieldname = prompt("Field name:") if (fieldname) { set_frontmatter( immer((fm) => { _.set(fm, `${base_path}${fieldname}`, null) }) ) } }} > Add entry + </button> ` } return html`<dialog ref=${dialog_ref} class="pluto-frontmatter"> <h1>Frontmatter</h1> <p> If you are publishing this notebook on the web, you can set the parameters below to provide HTML metadata. This is useful for search engines and social media. </p> <div class="card-preview" aria-hidden="true"> <h2>Preview</h2> <${FeaturedCard} entry=${ /** @type {import("./welcome/Featured.js").SourceManifestNotebookEntry} */ ({ id: filename.replace(/\.jl$/, ""), hash: "xx", frontmatter: clean_data(frontmatter) ?? {}, }) } image_loading=${"lazy"} disable_links=${true} /> </div> <div class="fm-table"> ${entries_input(frontmatter_with_defaults, ``)} ${!_.isArray(frontmatter_with_defaults.author) ? null : frontmatter_with_defaults.author.map((author, i) => { let author_with_defaults = { name: null, url: null, ...author, } return html` <fieldset class="fm-table"> <legend>Author ${i + 1}</legend> ${entries_input(author_with_defaults, `author[${i}].`)} </fieldset> ` })} ${!_.isArray(frontmatter_with_defaults.author) ? null : html`<button class="addentry" onClick=${() => { set_frontmatter((fm) => ({ ...fm, author: [...(fm?.author ?? []), {}] })) }} > Add author + </button>`} </div> <div class="final"><button onClick=${cancel}>Cancel</button><button onClick=${submit}>Save</button></div> </dialog>` } const clean_data = (obj) => { let a = _.isPlainObject(obj) ? Object.fromEntries( Object.entries(obj) .map(([key, val]) => [key, clean_data(val)]) .filter(([key, val]) => val != null) ) : _.isArray(obj) ? obj.map(clean_data).filter((x) => x != null) : obj return !_.isNumber(a) && _.isEmpty(a) ? null : a } let test = clean_data({ a: 1, b: "", c: null, d: [], e: [1, "", null, 2], f: {}, g: [{}], h: [{ z: "asdf" }] }) console.assert( _.isEqual(test, { a: 1, e: [1, 2], h: [{ z: "asdf" }], }), test ) const special_field_names = ["tags", "date", "license", "url", "color"] const field_type = (name) => { for (const t of special_field_names) { if (name === t || name.endsWith(`_${t}`)) { return t } } return "text" } const Input = ({ value, on_value, type, id }) => { const input_ref = useRef(/** @type {HTMLInputElement?} */ (null)) useLayoutEffect(() => { if (!input_ref.current) return input_ref.current.value = value }, [input_ref.current, value]) useLayoutEffect(() => { if (!input_ref.current) return const listener = (e) => { if (!input_ref.current) return on_value(input_ref.current.value) } input_ref.current.addEventListener("input", listener) return () => { input_ref.current?.removeEventListener("input", listener) } }, [input_ref.current]) const placeholder = type === "url" ? "https://..." : undefined return type === "tags" ? html`<rbl-tag-input id=${id} ref=${input_ref} />` : type === "license" ? LicenseInput({ ref: input_ref, id }) : html`<input type=${type} id=${id} ref=${input_ref} placeholder=${placeholder} />` } // https://choosealicense.com/licenses/ // and check https://github.com/sindresorhus/spdx-license-list/blob/HEAD/spdx-simple.json const code_licenses = ["AGPL-3.0", "GPL-3.0", "LGPL-3.0", "MPL-2.0", "Apache-2.0", "MIT", "BSL-1.0", "Unlicense"] // https://creativecommons.org/about/cclicenses/ // and check https://github.com/sindresorhus/spdx-license-list/blob/HEAD/spdx-simple.json const creative_licenses = ["CC-BY-4.0", "CC-BY-SA-4.0", "CC-BY-NC-4.0", "CC-BY-NC-SA-4.0", "CC-BY-ND-4.0", "CC-BY-NC-ND-4.0", "CC0-1.0"] const licenses = [...code_licenses, ...creative_licenses] const LicenseInput = ({ ref, id }) => { return html` <input ref=${ref} id=${id} type="text" list="oss-licenses" /> <datalist id="oss-licenses">${licenses.map((name) => html`<option>${name}</option>`)}</datalist> ` } import { html, useState, useRef, useLayoutEffect, useEffect, useMemo, useContext } from "../imports/Preact.js" import immer from "../imports/immer.js" import observablehq from "../common/SetupCellEnvironment.js" import { RawHTMLContainer, highlight } from "./CellOutput.js" import { PlutoActionsContext } from "../common/PlutoContext.js" import { cl } from "../common/ClassTable.js" /** * @param {{ * focus_on_open: boolean, * desired_doc_query: string?, * on_update_doc_query: (query: string) => void, * notebook: import("./Editor.js").NotebookData, * sanitize_html?: boolean, * }} props */ export let LiveDocsTab = ({ focus_on_open, desired_doc_query, on_update_doc_query, notebook, sanitize_html = true }) => { let pluto_actions = useContext(PlutoActionsContext) let live_doc_search_ref = useRef(/** @type {HTMLInputElement?} */ (null)) // This is all in a single state object so that we can update multiple field simultaneously let [state, set_state] = useState({ shown_query: null, searched_query: null, body: `<p>Welcome to the <b>Live docs</b>! Keep this little window open while you work on the notebook, and you will get documentation of everything you type!</p><p>You can also type a query above.</p><hr><p><em>Still stuck? Here are <a target="_blank" href="https://julialang.org/about/help/">some tips</a>.</em></p>`, loading: false, }) let update_state = (mutation) => set_state(immer((state) => mutation(state))) useEffect(() => { if (state.loading) { return } if (desired_doc_query != null && !/[^\s]/.test(desired_doc_query)) { // only whitespace return } if (state.searched_query !== desired_doc_query) { fetch_docs(desired_doc_query) } }, [desired_doc_query, state.loading, state.searched_query]) useLayoutEffect(() => { if (focus_on_open && live_doc_search_ref.current) { live_doc_search_ref.current.focus({ preventScroll: true }) live_doc_search_ref.current.select() } }, [focus_on_open]) let fetch_docs = (new_query) => { update_state((state) => { state.loading = true state.searched_query = new_query }) Promise.race([ observablehq.Promises.delay(2000, false), pluto_actions.send("docs", { query: new_query.replace(/^\?/, "") }, { notebook_id: notebook.notebook_id }).then((u) => { if (u.message.status === "") { return false } if (u.message.status === "") { update_state((state) => { state.shown_query = new_query state.body = u.message.doc }) return true } }), ]).then(() => { update_state((state) => { state.loading = false }) }) } let docs_element = useMemo( () => html`<${RawHTMLContainer} body=${without_workspace_stuff(state.body)} sanitize_html=${sanitize_html} sanitize_html_message=${false} />`, [state.body, sanitize_html] ) let no_docs_found = state.loading === false && state.searched_query !== "" && state.searched_query !== state.shown_query return html` <div class=${cl({ "live-docs-searchbox": true, "loading": state.loading, "notfound": no_docs_found, })} translate=${false} > <input title=${no_docs_found ? `"${state.searched_query}" not found` : ""} id="live-docs-search" placeholder="Search docs..." ref=${live_doc_search_ref} onInput=${(e) => on_update_doc_query(e.target.value)} value=${desired_doc_query} type="search" ></input> </div> <section ref=${(ref) => ref != null && post_process_doc_node(ref, on_update_doc_query)}> <h1><code>${state.shown_query}</code></h1> ${docs_element} </section> ` } const post_process_doc_node = (node, on_update_doc_query) => { // Apply syntax highlighting to code blocks: // In the standard HTML container we already do this for code.language-julia blocks, // but in the docs it's safe to extend to to all highlighting I think // Actually, showing the jldoctest stuff wasn't as pretty... should make a mode for that sometimes // for (let code_element of container_ref.current.querySelectorAll("code.language-jldoctest")) { // highlight(code_element, "julia") // } for (let code_element of node.querySelectorAll("code:not([class])")) { highlight(code_element, "julia") } // Resolve @doc reference links: for (let anchor of node.querySelectorAll("a")) { const href = anchor.getAttribute("href") if (href != null && href.startsWith("@ref")) { const query = href.length > 4 ? href.substr(5) : anchor.textContent anchor.onclick = (e) => { on_update_doc_query(query) e.preventDefault() } } } } const without_workspace_stuff = (str) => str .replace(/Main\.var&quot;workspace\#\d+&quot;\./g, "") // remove workspace modules from variable names .replace(/Main\.workspace\#\d+\./g, "") // remove workspace modules from variable names .replace(/ in Main\.var&quot;workspace\#\d+&quot;/g, "") // remove workspace modules from method lists .replace(/ in Main\.workspace\#\d+/g, "") // remove workspace modules from method lists .replace(/#&#61;&#61;#[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}\:\d+/g, "") // remove UUIDs from filenames import _ from "../imports/lodash.js" import { cl } from "../common/ClassTable.js" import { html, useState, useEffect, useLayoutEffect, useRef, useMemo } from "../imports/Preact.js" import { SimpleOutputBody } from "./TreeView.js" import { help_circle_icon } from "./Popup.js" import { ansi_to_html } from "../imports/AnsiUp.js" import { open_pluto_popup } from "../common/open_pluto_popup.js" const LOGS_VISIBLE_START = 60 const LOGS_VISIBLE_END = 20 const PROGRESS_LOG_LEVEL = "LogLevel(-1)" const STDOUT_LOG_LEVEL = "LogLevel(-555)" // const RESIZE_THROTTLE = 60 const is_progress_log = (log) => { return log.level == PROGRESS_LOG_LEVEL && log.kwargs.find((kwarg) => kwarg[0] === "progress") !== undefined } const is_stdout_log = (log) => { return log.level == STDOUT_LOG_LEVEL } export const Logs = ({ logs, line_heights, set_cm_highlighted_line, sanitize_html }) => { const progress_logs = logs.filter(is_progress_log) const latest_progress_logs = progress_logs.reduce((progress_logs, log) => ({ ...progress_logs, [log.id]: log }), {}) const stdout_log = logs.reduce((stdout_log, log) => { if (!is_stdout_log(log)) { return stdout_log } if (stdout_log === null) { return log } return { ...stdout_log, msg: [stdout_log.msg[0] + log.msg[0]], // Append to the previous stdout } }, null) const [_, __, grouped_progress_and_logs] = logs.reduce( ([seen_progress, seen_stdout, final_logs], log) => { const ipl = is_progress_log(log) if (ipl && !seen_progress.has(log.id)) { seen_progress.add(log.id) return [seen_progress, seen_stdout, [...final_logs, latest_progress_logs[log.id]]] } else if (!ipl) { if (is_stdout_log(log) && !seen_stdout) { return [seen_progress, true, [...final_logs, stdout_log]] } else if (!is_stdout_log(log)) { return [seen_progress, seen_stdout, [...final_logs, log]] } } return [seen_progress, seen_stdout, final_logs] }, [new Set(), false, []] ) const is_hidden_input = line_heights[0] === 0 if (logs.length === 0) { return null } const dot = (log, i) => html`<${Dot} set_cm_highlighted_line=${set_cm_highlighted_line} level=${log.level} msg=${log.msg} kwargs=${log.kwargs} sanitize_html=${sanitize_html} key=${i} y=${is_hidden_input ? 0 : log.line - 1} /> ` return html` <pluto-logs-container> <pluto-logs> ${grouped_progress_and_logs.length <= LOGS_VISIBLE_END + LOGS_VISIBLE_START ? grouped_progress_and_logs.map(dot) : [ ...grouped_progress_and_logs.slice(0, LOGS_VISIBLE_START).map(dot), html`<pluto-log-truncated> ${grouped_progress_and_logs.length - LOGS_VISIBLE_START - LOGS_VISIBLE_END} logs not shown... </pluto-log-truncated>`, ...grouped_progress_and_logs .slice(-LOGS_VISIBLE_END) .map((log, i) => dot(log, i + grouped_progress_and_logs.length - LOGS_VISIBLE_END)), ]} </pluto-logs> </pluto-logs-container> ` } const Progress = ({ name, progress }) => { return html`<pluto-progress-name>${name}</pluto-progress-name> <pluto-progress-bar-container><${ProgressBar} progress=${progress} /></pluto-progress-bar-container>` } const ProgressBar = ({ progress }) => { const bar_ref = useRef(/** @type {HTMLElement?} */ (null)) useLayoutEffect(() => { if (!bar_ref.current) return bar_ref.current.style.backgroundSize = `${progress * 100}% 100%` }, [bar_ref.current, progress]) return html`<pluto-progress-bar ref=${bar_ref}>${Math.ceil(100 * progress)}%</pluto-progress-bar>` } const Dot = ({ set_cm_highlighted_line, msg, kwargs, y, level, sanitize_html }) => { const is_progress = is_progress_log({ level, kwargs }) const is_stdout = level === STDOUT_LOG_LEVEL let progress = null if (is_progress) { progress = kwargs.find((p) => p[0] === "progress")[1][0] if (progress === "nothing") { progress = 0 } else if (progress === '"done"') { progress = 1 } else { progress = parseFloat(progress) } level = "Progress" } if (is_stdout) { level = "Stdout" } const mimepair_output = (pair) => html`<${SimpleOutputBody} cell_id=${"cell_id_not_known"} mime=${pair[1]} body=${pair[0]} persist_js_state=${false} sanitize_html=${sanitize_html} />` useEffect(() => { return () => set_cm_highlighted_line(null) }, []) return html`<pluto-log-dot-positioner class=${cl({ [level]: true })} onMouseenter=${() => is_progress || set_cm_highlighted_line(y + 1)} onMouseleave=${() => { set_cm_highlighted_line(null) }} > <pluto-log-icon></pluto-log-icon> <pluto-log-dot class=${level} >${is_progress ? html`<${Progress} name="${msg[0]}" progress=${progress} />` : is_stdout ? html`<${MoreInfo} body=${html`${"This text was written to the "} <a href="https://en.wikipedia.org/wiki/Standard_streams" target="_blank">terminal stream</a>${" while running the cell. "}<span style="opacity: .5" >${"(It is not the "}<em>return value</em>${" of the cell.)"}</span >`} /> <${LogViewAnsiUp} value=${msg[0]} />` : html`${mimepair_output(msg)}${kwargs.map( ([k, v]) => html`<pluto-log-dot-kwarg><pluto-key>${k}</pluto-key><pluto-value>${mimepair_output(v)}</pluto-value></pluto-log-dot-kwarg>` )}`}</pluto-log-dot > </pluto-log-dot-positioner>` } const MoreInfo = (/** @type{{body: import("../imports/Preact.js").ReactElement}} */ { body }) => { return html`<a class="stdout-info" target="_blank" title="Click for more info" href="#" onClick=${(/** @type{Event} */ e) => { open_pluto_popup({ type: "info", source_element: /** @type {HTMLElement?} */ (e.currentTarget), body, }) e.preventDefault() }} ><img alt="" src=${help_circle_icon} /></a>` } const LogViewAnsiUp = (/** @type {{value: string}} */ { value }) => { const node_ref = useRef(/** @type {HTMLElement?} */ (null)) useEffect(() => { if (!node_ref.current) return node_ref.current.innerHTML = ansi_to_html(value) }, [node_ref.current, value]) return html`<pre ref=${node_ref}></pre>` } import { useEventListener } from "../common/useEventListener.js" import { html, useEffect, useRef, useState } from "../imports/Preact.js" /** * Sometimes, we want to render HTML outside of the Cell Output, * for to add toolboxes like the Table of Contents or something similar. * * Additionally, the environment may want to inject some non cell/non editor * specific HTML to be rendered in the page. This component acts as a sink for * rendering these usecases. * * That way, the Cell Output can change with a different lifecycle than * the Non-Cell output and environments can inject UI. * * This component listens to events like the one below and updates * document.dispatchEvent( * new CustomEvent("experimental_add_node_non_cell_output", { * detail: { order: 1, node: html`<div>...</div>`, name: "Name of toolbox" } * })) */ export const NonCellOutput = ({ environment_component, notebook_id }) => { const surely_the_latest_updated_set = useRef() const [component_set, update_component_set] = useState({}) useEventListener( document, "eexperimental_add_node_non_cell_output", (e) => { try { const { name, node, order } = e.detail surely_the_latest_updated_set.current = { ...surely_the_latest_updated_set.current, [name]: { node, order } } update_component_set(surely_the_latest_updated_set.current) } catch (e) {} }, [surely_the_latest_updated_set, update_component_set] ) let components = Object.values(component_set) components.sort(({ order: o1 }, { order: o2 }) => o1 - o2) components = components.map(({ node }) => node) return html`<div class="non-cell-output"> ${environment_component ? html`<${environment_component} notebook_id=${notebook_id} />` : null} ${components} </div>` } import { PlutoActionsContext } from "../common/PlutoContext.js" import { html, useContext, useEffect, useLayoutEffect, useMemo, useRef, useState } from "../imports/Preact.js" import { Cell } from "./Cell.js" import { nbpkg_fingerprint } from "./PkgStatusMark.js" /** Like `useMemo`, but explain to the console what invalidated the memo. */ export const useMemoDebug = (fn, args) => { const last_values = useRef(args) return useMemo(() => { const new_values = args console.group("useMemoDebug: something changed!") if (last_values.current.length !== new_values.length) { console.log("Length changed. ", " old ", last_values.current, " new ", new_values) } else { for (let i = 0; i < last_values.current.length; i++) { if (last_values.current[i] !== new_values[i]) { console.log("Element changed. Index: ", i, " old ", last_values.current[i], " new ", new_values[i]) } } } console.groupEnd() return fn() }, args) } const CellMemo = ({ cell_result, cell_input, cell_input_local, notebook_id, cell_dependencies, selected, focus_after_creation, force_hide_input, is_process_ready, disable_input, sanitize_html = true, process_waiting_for_permission, show_logs, set_show_logs, nbpkg, global_definition_locations, is_first_cell, }) => { const { body, last_run_timestamp, mime, persist_js_state, rootassignee } = cell_result?.output || {} const { queued, running, runtime, errored, depends_on_disabled_cells, logs, depends_on_skipped_cells } = cell_result || {} const { cell_id, code, code_folded, metadata } = cell_input || {} return useMemo(() => { return html` <${Cell} cell_result=${cell_result} cell_dependencies=${cell_dependencies} cell_input=${cell_input} cell_input_local=${cell_input_local} notebook_id=${notebook_id} selected=${selected} force_hide_input=${force_hide_input} focus_after_creation=${focus_after_creation} is_process_ready=${is_process_ready} disable_input=${disable_input} process_waiting_for_permission=${process_waiting_for_permission} sanitize_html=${sanitize_html} nbpkg=${nbpkg} global_definition_locations=${global_definition_locations} is_first_cell=${is_first_cell} /> ` }, [ // Object references may invalidate this faster than the optimal. To avoid this, spread out objects to primitives! cell_id, ...Object.keys(metadata), ...Object.values(metadata), depends_on_disabled_cells, depends_on_skipped_cells, queued, running, runtime, errored, body, last_run_timestamp, mime, persist_js_state, rootassignee, logs, code, code_folded, cell_input_local, notebook_id, cell_dependencies, selected, force_hide_input, focus_after_creation, is_process_ready, disable_input, process_waiting_for_permission, sanitize_html, ...nbpkg_fingerprint(nbpkg), global_definition_locations, is_first_cell, ]) } /** * Rendering cell outputs can slow down the initial page load, so we delay rendering them using this heuristic function to determine the length of the delay (as a function of the number of cells in the notebook). Since using CodeMirror 6, cell inputs do not cause a slowdown when out-of-viewport, rendering is delayed until they come into view. * @param {Number} num_cells */ const render_cell_outputs_delay = (num_cells) => (num_cells > 20 ? 100 : 0) /** * The first <x> cells will bypass the {@link render_cell_outputs_delay} heuristic and render directly. */ const render_cell_outputs_minimum = 20 /** * @param {{ * notebook: import("./Editor.js").NotebookData, * cell_inputs_local: { [uuid: string]: { code: String } }, * on_update_doc_query: any, * on_cell_input: any, * on_focus_neighbor: any, * last_created_cell: string, * selected_cells: Array<string>, * is_initializing: boolean, * is_process_ready: boolean, * disable_input: boolean, * process_waiting_for_permission: boolean, * sanitize_html: boolean, * }} props * */ export const Notebook = ({ notebook, cell_inputs_local, last_created_cell, selected_cells, is_initializing, is_process_ready, disable_input, process_waiting_for_permission, sanitize_html = true, }) => { let pluto_actions = useContext(PlutoActionsContext) // Add new cell when the last cell gets deleted useEffect(() => { // This might look kinda silly... // and it is... but it covers all the cases... - DRAL if (notebook.cell_order.length === 0 && !is_initializing) { pluto_actions.add_remote_cell_at(0) } }, [is_initializing, notebook.cell_order.length]) // Only render the notebook partially during the first few seconds const [cell_outputs_delayed, set_cell_outputs_delayed] = useState(true) useEffect(() => { if (cell_outputs_delayed && notebook.cell_order.length > 0) { setTimeout(() => { set_cell_outputs_delayed(false) }, render_cell_outputs_delay(notebook.cell_order.length)) } }, [cell_outputs_delayed, notebook.cell_order.length]) let global_definition_locations = useMemo( () => Object.fromEntries( Object.values(notebook?.cell_dependencies ?? {}).flatMap((x) => Object.keys(x.downstream_cells_map) .filter((variable) => !variable.includes(".")) .map((variable) => [variable, x.cell_id]) ) ), [notebook?.cell_dependencies] ) useLayoutEffect(() => { let oldhash = window.location.hash if (oldhash.length > 1) { let go = () => { window.location.hash = "#" window.location.hash = oldhash } go() // Scrolling there might trigger some codemirrors to render and change height, so let's do it again. requestIdleCallback(go) } }, [cell_outputs_delayed]) return html` <pluto-notebook id=${notebook.notebook_id}> ${notebook.cell_order .filter((_, i) => !(cell_outputs_delayed && i > render_cell_outputs_minimum)) .map( (cell_id, i) => html`<${CellMemo} key=${cell_id} cell_result=${notebook.cell_results[cell_id] ?? { cell_id: cell_id, queued: true, running: false, errored: false, runtime: null, output: null, logs: [], }} cell_input=${notebook.cell_inputs[cell_id]} cell_dependencies=${notebook?.cell_dependencies?.[cell_id] ?? {}} cell_input_local=${cell_inputs_local[cell_id]} notebook_id=${notebook.notebook_id} selected=${selected_cells.includes(cell_id)} focus_after_creation=${last_created_cell === cell_id} force_hide_input=${false} is_process_ready=${is_process_ready} disable_input=${disable_input} process_waiting_for_permission=${process_waiting_for_permission} sanitize_html=${sanitize_html} nbpkg=${notebook.nbpkg} global_definition_locations=${global_definition_locations} is_first_cell=${i === 0} />` )} ${ // Waiting for the last deleted cell to be recovered... notebook.cell_order.length === 0 || // Waiting for all cells to be displayed... (cell_outputs_delayed && notebook.cell_order.length >= render_cell_outputs_minimum) ? html`<div style=" font-family: system-ui; font-style: italic; padding: 0.3rem 1rem; margin: 1rem 0rem; border-radius: .3rem; background: var(--blockquote-bg); opacity: 0.6; animation: fadeintext .2s 1.5s linear; animation-fill-mode: both; margin-bottom: ${Math.max(0, (notebook.cell_order.length - render_cell_outputs_minimum) * 10)}rem;" > Loading cells... </div>` : null } </pluto-notebook> ` } import { html, useEffect, useState } from "../imports/Preact.js" import { cl } from "../common/ClassTable.js" import { is_finished, total_done } from "./StatusTab.js" import { useDelayedTruth } from "./BottomRightPanel.js" import { url_logo_small } from "./Editor.js" import { open_pluto_popup } from "../common/open_pluto_popup.js" /** * @param {{ * status: import("./Editor.js").StatusEntryData, * }} props */ export let NotifyWhenDone = ({ status }) => { const all_done = Object.values(status.subtasks).every(is_finished) const [enabled, setEnabled] = useState(false) useEffect(() => { if (enabled && all_done) { console.log("all done") /** @type {Notification?} */ let notification = null let timeouthandler = setTimeout(() => { setEnabled(false) let count = total_done(status) notification = new Notification("Pluto: notebook ready", { tag: "notebook ready", body: ` All ${count} steps completed`, lang: "en-US", dir: "ltr", icon: url_logo_small, }) notification.onclick = (e) => { parent.focus() window.focus() notification?.close() } }, 3000) const vishandler = () => { if (document.visibilityState === "visible") { notification?.close() } } document.addEventListener("visibilitychange", vishandler) document.body.addEventListener("click", vishandler) return () => { notification?.close() clearTimeout(timeouthandler) document.removeEventListener("visibilitychange", vishandler) document.body.removeEventListener("click", vishandler) } } }, [all_done]) const visible = useDelayedTruth(!all_done, 2500) || enabled return html` <div class=${cl({ visible, "notify-when-done": true })} inert=${!visible}> <label >${"Notify when done"} <input type="checkbox" checked=${enabled} disabled=${!visible} onInput=${(e) => { if (e.target.checked) { Notification.requestPermission().then((r) => { console.log(r) const granted = r === "granted" setEnabled(granted) e.target.checked = granted if (!granted) open_pluto_popup({ type: "warn", body: html` Pluto needs permission to show notifications. <strong>Enable notifications</strong> in your browser settings to use this feature. `, }) }) } else { setEnabled(false) } }} /></label> </div> ` } import { useEventListener } from "../common/useEventListener.js" import { html, useEffect, useRef } from "../imports/Preact.js" import { link_edit } from "./welcome/Open.js" const detectNotebook = (inputtext) => { // Add a newline in the end for the case user didn't copy it // That helps if the user copied up to the last line of the cell order const text = `${inputtext}\n`.replace("\r\n", "\n") const from = text.indexOf("### A Pluto.jl notebook ###") const cellsfound = text.match(/# ... ........-....-....-....-............/g) const cellscount = cellsfound?.length ?? 0 const cellsorder = text.indexOf("#  Cell order:") + "#  Cell order:".length + 1 let to = cellsorder for (let i = 1; i <= cellscount; i++) { to = text.indexOf("\n", to + 1) + 1 } return text.slice(from, to) } const readMovedText = (movedDataTransferItem) => new Promise((resolve, reject) => { try { movedDataTransferItem.getAsString((text) => { console.log(text) resolve(text) }) } catch (ex) { reject(ex) } }) const readFile = (file) => new Promise((resolve, reject) => { const { name, type } = file const fr = new FileReader() fr.onerror = () => reject("Failed to read file!") fr.onloadstart = () => {} fr.onprogress = ({ loaded, total }) => {} fr.onload = () => {} fr.onloadend = () => resolve({ file: fr.result, name, type }) fr.readAsText(file) }) export const PasteHandler = ({ on_start_navigation }) => { const processFile = async (ev) => { let notebook console.log(ev) // Don't do anything if paste on CodeMirror if ((ev?.path ?? ev?.composedPath()).filter((node) => node?.classList?.contains(".cm-editor"))?.length > 0) { return } switch (ev.type) { case "paste": notebook = detectNotebook(ev.clipboardData.getData("text/plain")) break case "dragstart": { ev.dataTransfer.dropEffect = "move" return } case "dragover": { ev.preventDefault() return } case "drop": { ev.preventDefault() notebook = ev.dataTransfer.types.includes("Files") ? await readFile(ev.dataTransfer.files[0]).then(({ file }) => file) : detectNotebook(await readMovedText(ev.dataTransfer.items[0])) break } } if (!notebook) { // Notebook not found! Doing nothing :) return } on_start_navigation("notebook from clipboard", false) document.body.classList.add("loading") const response = await fetch("./notebookupload", { method: "POST", body: notebook, }) if (response.ok) { window.location.href = link_edit(await response.text()) } else { let b = await response.blob() window.location.href = URL.createObjectURL(b) } } useEventListener(document, "paste", processFile, [processFile]) useEventListener(document, "drop", processFile, [processFile]) useEventListener(document, "dragstart", processFile, [processFile]) useEventListener(document, "dragover", processFile, [processFile]) return html`<span />` } import { open_pluto_popup } from "../common/open_pluto_popup.js" import _ from "../imports/lodash.js" import { html, useEffect, useState } from "../imports/Preact.js" import { open_icon } from "./Popup.js" export const nbpkg_fingerprint = (nbpkg) => (nbpkg == null ? [null] : Object.entries(nbpkg).flat()) export const nbpkg_fingerprint_without_terminal = (nbpkg) => nbpkg == null ? [null] : Object.entries(nbpkg).flatMap(([k, v]) => (k === "terminal_outputs" ? [] : [v])) const can_update = (installed, available) => { if (installed === "stdlib" || !_.isArray(available)) { return false } else { // return true return _.last(available) !== installed } } /** * @typedef PackageStatus * @property {string} status * @property {import("../imports/Preact.js").ReactElement} hint * @property {string} hint_raw * @property {string[]?} available_versions * @property {string?} chosen_version * @property {string?} package_url * @property {boolean} busy * @property {boolean} offer_update */ /** * @param {{ * package_name: string, * package_url?: string, * is_disable_pkg: boolean, * available_versions?: string[], * nbpkg: import("./Editor.js").NotebookPkgData?, * }} props * @returns {PackageStatus} */ export const package_status = ({ nbpkg, package_name, available_versions, is_disable_pkg, package_url }) => { let status = "error" let hint_raw = "error" let hint = html`error` let offer_update = false package_url = package_url ?? `https://juliahub.com/ui/Packages/General/${package_name}` const chosen_version = nbpkg?.installed_versions[package_name] ?? null const nbpkg_waiting_for_permission = nbpkg?.waiting_for_permission ?? false const busy = !nbpkg_waiting_for_permission && ((nbpkg?.busy_packages ?? []).includes(package_name) || !(nbpkg?.instantiated ?? true)) const package_name_pretty = html`<a class="package-name" href=${package_url}><b>${package_name}</b></a> ` if (is_disable_pkg) { const f_name = package_name status = "disable_pkg" hint_raw = `${f_name} disables Pluto's built-in package manager.` hint = html`<b>${f_name}</b> disables Pluto's built-in package manager.` } else if (chosen_version != null || _.isEqual(available_versions, ["stdlib"])) { if (chosen_version == null || chosen_version === "stdlib") { status = "installed" hint_raw = `${package_name} is part of Julia's pre-installed 'standard library'.` hint = html`${package_name_pretty} is part of Julia's pre-installed <em>standard library</em>.` } else { if (nbpkg_waiting_for_permission) { status = "will_be_installed" hint_raw = `${package_name} (v${_.last(available_versions)}) will be installed when you run this notebook.` hint = html`<header>${package_name_pretty} <pkg-version>v${_.last(available_versions)}</pkg-version></header> will be installed when you run this notebook.` } else if (busy) { status = "busy" hint_raw = `${package_name} (v${chosen_version}) is installing...` hint = html`<header>${package_name_pretty} <pkg-version>v${chosen_version}</pkg-version></header> is installing...` } else { status = "installed" hint_raw = `${package_name} (v${chosen_version}) is installed in the notebook.` hint = html`<header> ${package_name_pretty} <pkg-version>v${chosen_version}</pkg-version> </header> is installed in the notebook.` offer_update = can_update(chosen_version, available_versions) } } } else { if (available_versions != null && _.isArray(available_versions)) { if (available_versions.length === 0) { status = "not_found" hint_raw = `The package "${package_name}" could not be found in the registry. Did you make a typo?` hint = html`The package <em>"${package_name}"</em> could not be found in the registry. <section><em>Did you make a typo?</em></section>` } else { status = "will_be_installed" hint_raw = `${package_name} (v${_.last(available_versions)}) will be installed in the notebook when you run this cell.` hint = html`<header>${package_name_pretty} <pkg-version>v${_.last(available_versions)}</pkg-version></header> will be installed in the notebook when you run this cell.` } } } return { status, hint, hint_raw, available_versions: available_versions ?? null, chosen_version, busy, offer_update, package_url } } /** * The little icon that appears inline next to a package import in code (e.g. `using PlutoUI `) * @param {{ * package_name: string, * pluto_actions: any, * notebook_id: string, * nbpkg: import("./Editor.js").NotebookPkgData?, * }} props */ export const PkgStatusMark = ({ package_name, pluto_actions, notebook_id, nbpkg }) => { const [available_versions_msg, set_available_versions_msg] = useState(/** @type {{ versions?: string[], package_url?: string }?} */ (null)) const [package_url, set_package_url] = useState(/** @type {string[]?} */ (null)) useEffect(() => { let available_version_promise = pluto_actions.get_avaible_versions({ package_name, notebook_id }) ?? Promise.resolve([]) available_version_promise.then(set_available_versions_msg) }, [package_name]) const { status, hint_raw } = package_status({ nbpkg: nbpkg, package_name: package_name, is_disable_pkg: false, available_versions: available_versions_msg?.versions, package_url: available_versions_msg?.package_url, }) return html` <pkg-status-mark title=${hint_raw} className=${status === "busy" ? "busy" : status === "installed" ? "installed" : status === "not_found" ? "not_found" : status === "will_be_installed" ? "will_be_installed" : ""} > <button onClick=${(event) => { open_pluto_popup({ type: "nbpkg", source_element: event.currentTarget.parentElement, package_name: package_name, is_disable_pkg: false, should_focus: true, }) }} > <span></span> </button> </pkg-status-mark> ` } export const PkgActivateMark = ({ package_name }) => { const { hint_raw } = package_status({ nbpkg: null, package_name: package_name, is_disable_pkg: true, }) return html` <pkg-status-mark title=${hint_raw} class="disable_pkg"> <button onClick=${(event) => { open_pluto_popup({ type: "nbpkg", source_element: event.currentTarget.parentElement, package_name: package_name, is_disable_pkg: true, should_focus: true, }) }} > <span></span> </button> </pkg-status-mark> ` } import { ansi_to_html } from "../imports/AnsiUp.js" import { html, Component, useState, useEffect, useRef, useLayoutEffect } from "../imports/Preact.js" const make_spinner_spin = (original_html) => original_html.replaceAll("", `<span class="make-me-spin"></span>`) const TerminalViewAnsiUp = ({ value }) => { const node_ref = useRef(/** @type {HTMLElement?} */ (null)) const start_time = useRef(Date.now()) useEffect(() => { if (!node_ref.current) return node_ref.current.style.cssText = `--animation-delay: -${(Date.now() - start_time.current) % 1000}ms` node_ref.current.innerHTML = make_spinner_spin(ansi_to_html(value)) const parent = node_ref.current.parentElement if (parent) parent.scrollTop = 1e5 }, [node_ref.current, value]) return !!value ? html`<pkg-terminal ><div class="scroller" tabindex="0"><pre ref=${node_ref} class="pkg-terminal"></pre></div ></pkg-terminal>` : null } export const PkgTerminalView = TerminalViewAnsiUp import { html, useState, useRef, useEffect, useContext, useCallback, useLayoutEffect } from "../imports/Preact.js" import { cl } from "../common/ClassTable.js" import { PlutoActionsContext } from "../common/PlutoContext.js" import { package_status, nbpkg_fingerprint_without_terminal } from "./PkgStatusMark.js" import { PkgTerminalView } from "./PkgTerminalView.js" import { useDebouncedTruth } from "./RunArea.js" import { time_estimate, usePackageTimingData } from "../common/InstallTimeEstimate.js" import { pretty_long_time } from "./EditOrRunButton.js" import { useEventListener } from "../common/useEventListener.js" import { get_included_external_source } from "../common/external_source.js" export const arrow_up_circle_icon = get_included_external_source("arrow_up_circle_icon")?.href export const document_text_icon = get_included_external_source("document_text_icon")?.href export const help_circle_icon = get_included_external_source("help_circle_icon")?.href export const open_icon = get_included_external_source("open_icon")?.href /** * @typedef PkgPopupDetails * @property {"nbpkg"} type * @property {HTMLElement} [source_element] * @property {Boolean} [big] * @property {string} [css_class] * @property {Boolean} [should_focus] Should the popup receive keyboard focus after opening? Rule of thumb: yes if the popup opens on a click, no if it opens spontaneously. * @property {string} package_name * @property {boolean} is_disable_pkg */ /** * @typedef MiscPopupDetails * @property {"info" | "warn"} type * @property {import("../imports/Preact.js").ReactElement} body * @property {HTMLElement?} [source_element] * @property {string} [css_class] * @property {Boolean} [big] * @property {Boolean} [should_focus] Should the popup receive keyboard focus after opening? Rule of thumb: yes if the popup opens on a click, no if it opens spontaneously. */ export const Popup = ({ notebook, disable_input }) => { const [recent_event, set_recent_event] = useState(/** @type{(PkgPopupDetails | MiscPopupDetails)?} */ (null)) const recent_event_ref = useRef(/** @type{(PkgPopupDetails | MiscPopupDetails)?} */ (null)) recent_event_ref.current = recent_event const recent_source_element_ref = useRef(/** @type{HTMLElement?} */ (null)) const pos_ref = useRef("") const open = useCallback( (/** @type {CustomEvent} */ e) => { const el = e.detail.source_element recent_source_element_ref.current = el if (el == null) { pos_ref.current = `top: 20%; left: 50%; transform: translate(-50%, -50%); position: fixed;` } else { const elb = el.getBoundingClientRect() const bodyb = document.body.getBoundingClientRect() pos_ref.current = `top: ${0.5 * (elb.top + elb.bottom) - bodyb.top}px; left: min(max(0px,100vw - 251px - 30px), ${elb.right - bodyb.left}px);` } set_recent_event(e.detail) }, [set_recent_event] ) const close = useCallback(() => { set_recent_event(null) }, [set_recent_event]) useEventListener(window, "open pluto popup", open, [open]) useEventListener(window, "close pluto popup", close, [close]) useEventListener( window, "pointerdown", (e) => { if (recent_event_ref.current == null) return if (e.target == null) return if (e.target.closest("pluto-popup") != null) return if (recent_source_element_ref.current != null && recent_source_element_ref.current.contains(e.target)) return close() }, [close] ) useEventListener( window, "keydown", (e) => { if (e.key === "Escape") close() }, [close] ) // focus the popup when it opens const element_focused_before_popup = useRef(/** @type {any} */ (null)) useLayoutEffect(() => { if (recent_event != null) { if (recent_event.should_focus === true) { requestAnimationFrame(() => { element_focused_before_popup.current = document.activeElement /** @type {HTMLElement?} */ const el = element_ref.current?.querySelector("a, input, button") ?? element_ref.current // console.debug("restoring focus to", el) el?.focus?.() }) } else { element_focused_before_popup.current = null } } }, [recent_event != null]) const element_ref = useRef(/** @type {HTMLElement?} */ (null)) // if the popup was focused on opening: // when the popup loses focus (and the focus did not move to the source element): // 1. close the popup // 2. return focus to the element that was focused before the popup opened useEventListener( element_ref.current, "focusout", (e) => { if (recent_event_ref.current != null && recent_event_ref.current.should_focus === true) { if (element_ref.current?.matches(":focus-within")) return if (element_ref.current?.contains(e.relatedTarget)) return if ( recent_source_element_ref.current != null && (recent_source_element_ref.current.contains(e.relatedTarget) || recent_source_element_ref.current.matches(":focus-within")) ) return close() e.preventDefault() element_focused_before_popup.current?.focus?.() } }, [close] ) const type = recent_event?.type return html`<pluto-popup class=${cl({ visible: recent_event != null, [type ?? ""]: type != null, big: recent_event?.big === true, [recent_event?.css_class ?? ""]: recent_event?.css_class != null, })} style="${pos_ref.current}" ref=${element_ref} tabindex=${ "0" /* this makes the popup itself focusable (not just its buttons), just like a <dialog> element. It also makes the `.matches(":focus-within")` trick work. */ } > ${type === "nbpkg" ? html`<${PkgPopup} notebook=${notebook} disable_input=${disable_input} recent_event=${recent_event} clear_recent_event=${() => set_recent_event(null)} />` : type === "info" || type === "warn" ? html`<div>${recent_event?.body}</div>` : null} </pluto-popup> <div tabindex="0"> <!-- We need this dummy tabindexable element here so that the element_focused_before_popup mechanism works on static exports. When tabbing out of the popup, focus would otherwise leave the page altogether because it's the last focusable element in DOM. --> </div>` } /** * @param {{ * notebook: import("./Editor.js").NotebookData, * recent_event: PkgPopupDetails, * clear_recent_event: () => void, * disable_input: boolean, * }} props */ const PkgPopup = ({ notebook, recent_event, clear_recent_event, disable_input }) => { let pluto_actions = useContext(PlutoActionsContext) const [pkg_status, set_pkg_status] = useState(/** @type{import("./PkgStatusMark.js").PackageStatus?} */ (null)) useEffect(() => { let still_valid = true if (recent_event == null) { set_pkg_status(null) } else if (recent_event?.type === "nbpkg") { ;(pluto_actions.get_avaible_versions({ package_name: recent_event.package_name, notebook_id: notebook.notebook_id }) ?? Promise.resolve([])).then( ({ versions, url }) => { if (still_valid) { set_pkg_status( package_status({ nbpkg: notebook.nbpkg, package_name: recent_event.package_name, is_disable_pkg: recent_event.is_disable_pkg, available_versions: versions, package_url: url, }) ) } } ) } return () => { still_valid = false } }, [recent_event, ...nbpkg_fingerprint_without_terminal(notebook.nbpkg)]) // hide popup when nbpkg is switched on/off const valid = recent_event.is_disable_pkg || (notebook.nbpkg?.enabled ?? true) useEffect(() => { if (!valid) { clear_recent_event() } }, [valid]) const [showterminal, set_showterminal] = useState(false) const needs_first_instatiation = notebook.nbpkg?.restart_required_msg == null && !(notebook.nbpkg?.instantiated ?? true) const busy = recent_event != null && ((notebook.nbpkg?.busy_packages ?? []).includes(recent_event.package_name) || needs_first_instatiation) const debounced_busy = useDebouncedTruth(busy, 2) useEffect(() => { set_showterminal(debounced_busy) }, [debounced_busy]) const terminal_value = notebook.nbpkg?.terminal_outputs == null ? "Loading..." : notebook.nbpkg?.terminal_outputs[recent_event?.package_name] ?? "" const showupdate = pkg_status?.offer_update ?? false const timingdata = usePackageTimingData() const estimate = timingdata == null || recent_event?.package_name == null ? null : time_estimate(timingdata, [recent_event?.package_name]) const total_time = estimate == null ? 0 : estimate.install + estimate.load + estimate.precompile const total_second_time = estimate == null ? 0 : estimate.load // <header>${recent_event?.package_name}</header> return html`<pkg-popup class=${cl({ busy, showterminal, showupdate, })} > ${pkg_status?.hint ?? "Loading..."} ${(pkg_status?.status === "will_be_installed" || pkg_status?.status === "busy") && total_time > 10 ? html`<div class="pkg-time-estimate"> Installation can take <strong>${pretty_long_time(total_time)}</strong>${`. `}<br />${`Afterwards, it loads in `} <strong>${pretty_long_time(total_second_time)}</strong>. </div>` : null} <div class="pkg-buttons"> ${recent_event?.is_disable_pkg || disable_input || notebook.nbpkg?.waiting_for_permission ? null : html`<a class="pkg-update" target="_blank" title="Update packages" style=${!!showupdate ? "" : "opacity: .4;"} href="#" onClick=${(e) => { if (busy) { alert("Pkg is currently busy with other packages... come back later!") } else { if (confirm("Would you like to check for updates and install them? A backup of the notebook file will be created.")) { console.warn("Pkg.updating!") pluto_actions.send("pkg_update", {}, { notebook_id: notebook.notebook_id }) } } e.preventDefault() }} ><img alt="" src=${arrow_up_circle_icon} width="17" /></a>`} <a class="toggle-terminal" target="_blank" title="Show/hide Pkg terminal output" style=${!!terminal_value ? "" : "display: none;"} href="#" onClick=${(e) => { set_showterminal(!showterminal) e.preventDefault() }} ><img alt="" src=${document_text_icon} width="17" /></a> <a class="help" target="_blank" title="Go to help page" href="https://plutojl.org/pkg/"><img alt="" src=${help_circle_icon} width="17" /></a> </div> <${PkgTerminalView} value=${terminal_value ?? "Loading..."} /> </pkg-popup>` } import { html, useEffect, useState, useContext, useRef, useMemo } from "../imports/Preact.js" import { cl } from "../common/ClassTable.js" import { PlutoActionsContext } from "../common/PlutoContext.js" import { is_mac_keyboard } from "../common/KeyboardShortcuts.js" const await_focus = () => document.visibilityState === "visible" ? Promise.resolve() : new Promise((res) => { const h = () => { await_focus().then(res) document.removeEventListener("visibilitychange", h) } document.addEventListener("visibilitychange", h) }) export const Preamble = ({ any_code_differs, last_update_time, last_hot_reload_time, connected }) => { let pluto_actions = useContext(PlutoActionsContext) const [state, set_state] = useState("") const [reload_state, set_reload_state] = useState("") const timeout_ref = useRef(null) const reload_timeout_ref = useRef(null) useEffect(() => { // console.log("code differs", any_code_differs) clearTimeout(timeout_ref?.current) if (any_code_differs) { set_state("ask_to_save") } else { if (Date.now() - last_update_time < 1000) { set_state("saved") timeout_ref.current = setTimeout(() => { set_state("") }, 1000) } else { set_state("") } } return () => clearTimeout(timeout_ref?.current) }, [any_code_differs]) // silly bits to not show "Reloaded from file" immediately const [old_enough, set_old_enough] = useState(false) useEffect(() => { if (connected) { setTimeout(() => set_old_enough(true), 1000) } }, [connected]) useEffect(() => { console.log("Hottt", last_hot_reload_time, old_enough) if (old_enough) { set_reload_state("reloaded_from_file") console.log("set state") await_focus().then(() => { reload_timeout_ref.current = setTimeout(() => { set_reload_state("") console.log("reset state") }, 8000) }) return () => clearTimeout(reload_timeout_ref?.current) } }, [last_hot_reload_time]) return html`<preamble> ${state === "ask_to_save" ? html` <div id="saveall-container" class="overlay-button ${state}"> <button onClick=${() => { set_state("saving") pluto_actions.set_and_run_all_changed_remote_cells() }} class=${cl({ runallchanged: true })} title="Save and run all changed cells" > <span class="only-on-hover"><b>Save all changes</b> </span>${is_mac_keyboard ? html`<kbd> S</kbd>` : html`<kbd>Ctrl</kbd>+<kbd>S</kbd>`} </button> </div> ` : // : state === "saving" // ? html` <div id="saveall-container" class="overlay-button ${state}">Saving... <span class="saving-icon"></span></div> ` state === "saved" || state === "saving" ? html` <div id="saveall-container" class="overlay-button ${state}"> <span><span class="only-on-hover">Saved </span><span class="saved-icon pluto-icon"></span></span> </div> ` : reload_state === "reloaded_from_file" ? html` <div id="saveall-container" class="overlay-button ${state}"> <span>File change detected, <b>notebook updated </b><span class="saved-icon pluto-icon"></span></span> </div> ` : null} </preamble>` } import _ from "../imports/lodash.js" import { html, useContext, useEffect, useMemo, useState } from "../imports/Preact.js" import { useDelayedTruth } from "./BottomRightPanel.js" import { scroll_cell_into_view } from "./Scroller.js" /** * @param {{ * notebook: import("./Editor.js").NotebookData, * backend_launch_phase: number?, * status: Record<string,any>, * }} props */ export const ProgressBar = ({ notebook, backend_launch_phase, status }) => { const [recently_running, set_recently_running] = useState(/** @type {string[]} */ ([])) const [currently_running, set_currently_running] = useState(/** @type {string[]} */ ([])) useEffect( () => { const currently = Object.values(notebook.cell_results) .filter((c) => c.running || c.queued) .map((c) => c.cell_id) set_currently_running(currently) if (currently.length === 0) { // all cells completed set_recently_running([]) } else { // add any new running cells to our pile set_recently_running(_.union(currently, recently_running)) } }, Object.values(notebook.cell_results).map((c) => c.running || c.queued) ) let cell_progress = recently_running.length === 0 ? 0 : 1 - Math.max(0, currently_running.length - 0.3) / recently_running.length let binder_loading = status.loading && status.binder let progress = binder_loading ? backend_launch_phase ?? 0 : cell_progress const anything = (binder_loading || recently_running.length !== 0) && progress !== 1 // Double inversion with ! to short-circuit the true, not the false const anything_for_a_short_while = !useDelayedTruth(!anything, 500) const anything_for_a_long_while = !useDelayedTruth(!anything, 2000) if (!(anything || anything_for_a_short_while || anything_for_a_long_while)) { return null } // set to 1 when all cells completed, instead of moving the progress bar to the start if (anything_for_a_short_while && !(binder_loading || recently_running.length !== 0)) { progress = 1 } const title = binder_loading ? "Loading binder..." : `Running cells... (${recently_running.length - currently_running.length}/${recently_running.length} done)` return html`<loading-bar class=${binder_loading ? "slow" : "fast"} style=${` width: ${100 * progress}vw; opacity: ${anything && anything_for_a_short_while ? 1 : 0}; ${anything || anything_for_a_short_while ? "" : "transition: none;"} pointer-events: ${anything ? "auto" : "none"}; cursor: ${!binder_loading && anything ? "pointer" : "auto"}; `} onClick=${(e) => { if (!binder_loading) { scroll_to_busy_cell(notebook) } }} aria-hidden="true" title=${title} ></loading-bar>` } export const scroll_to_busy_cell = (notebook) => { const running_cell_id = notebook == null ? (document.querySelector("pluto-cell.running") ?? document.querySelector("pluto-cell.queued"))?.id : (Object.values(notebook.cell_results).find((c) => c.running) ?? Object.values(notebook.cell_results).find((c) => c.queued))?.cell_id if (running_cell_id) { scroll_cell_into_view(running_cell_id) } } import _ from "../imports/lodash.js" import { createSilentAudio, create_recorder } from "../common/AudioRecording.js" import { html, useEffect, useState, useRef, useCallback, useLayoutEffect, useMemo } from "../imports/Preact.js" import { AudioPlayer } from "./AudioPlayer.js" import immer from "../imports/immer.js" import { base64_arraybuffer, blob_url_to_data_url } from "../common/PlutoHash.js" import { pack, unpack } from "../common/MsgPack.js" const assert_response_ok = (/** @type {Response} */ r) => (r.ok ? r : Promise.reject(r)) let run = (x) => x() /** * @typedef {[number, Array?]} PatchStep */ /** * @typedef {Object} RecordingData * @property {Array<PatchStep>} steps * @property {Array<[number, {cell_id: string, relative_distance: number}]>} scrolls */ /** * @typedef {RecordingData & { * initial_html: string, * scroll_handler: (x: number) => void, * audio_recorder: {start: () => void, stop: () => Promise<string>}? * }} RecordingState */ export const RecordingUI = ({ notebook_name, is_recording, recording_waiting_to_start, set_recording_states, patch_listeners, export_url }) => { let current_recording_ref = useRef(/** @type{RecordingState?} */ (null)) let recording_start_time_ref = useRef(0) useEffect(() => { let listener = (patches) => { if (current_recording_ref.current != null) { current_recording_ref.current.steps = [ ...current_recording_ref.current.steps, [(Date.now() - recording_start_time_ref.current) / 1000, patches], ] } } patch_listeners.push(listener) return () => { patch_listeners.splice(patch_listeners.indexOf(listener), 1) } }, []) const start_recording = async ({ want_audio }) => { let audio_recorder = null, audio_record_start_promise let abort = async (e) => { alert( `We were unable to activate your microphone. Make sure that it is connected, and that this site (${ window.location.protocol + "//" + window.location.host }) has permission to use the microphone.` ) console.warn("Failed to create audio recorder asdfasdf ", e) await stop_recording() } if (want_audio) { try { audio_recorder = await create_recorder() audio_record_start_promise = audio_recorder.start() } catch (e) { await abort(e) return } } let initial_html = await (await fetch(export_url("notebookexport")).then(assert_response_ok)).text() initial_html = initial_html.replaceAll( "https://cdn.jsdelivr.net/gh/fonsp/Pluto.jl@0.17.3/frontend/", "https://cdn.jsdelivr.net/gh/fonsp/Pluto.jl@8d243df/frontend/" ) // initial_html = initial_html.replaceAll("https://cdn.jsdelivr.net/gh/fonsp/Pluto.jl@0.17.3/frontend/", "http://localhost:1234/") const scroll_handler_direct = () => { if (current_recording_ref.current == null) return let y = window.scrollY + window.innerHeight / 2 /** @type {Array<HTMLElement>} */ const cell_nodes = Array.from(document.querySelectorAll("pluto-notebook > pluto-cell")) let best_index = "" let relative_distance = 0 cell_nodes.forEach((el, i) => { let cy = el.offsetTop if (cy <= y) { best_index = el.id relative_distance = (y - cy) / el.offsetHeight } }) current_recording_ref.current.scrolls = [ ...current_recording_ref.current.scrolls, [ (Date.now() - recording_start_time_ref.current) / 1000, { cell_id: best_index, relative_distance, }, ], ] } const scroll_handler = _.debounce(scroll_handler_direct, 500) try { await audio_record_start_promise } catch (e) { await abort(e) return } current_recording_ref.current = { initial_html, steps: [], scrolls: [], scroll_handler, audio_recorder, } recording_start_time_ref.current = Date.now() set_recording_states({ is_recording: true, recording_waiting_to_start: false }) // call it once to record the start scroll position scroll_handler_direct() window.addEventListener("scroll", scroll_handler, { passive: true }) } let notebook_name_ref = useRef(notebook_name) notebook_name_ref.current = _.last(notebook_name.split("/")) .replace(/\.jl$/, "") .replace(/\.plutojl$/, "") const stop_recording = async () => { if (current_recording_ref.current != null) { const { audio_recorder, initial_html, steps, scrolls, scroll_handler } = current_recording_ref.current // @ts-ignore window.removeEventListener("scroll", scroll_handler, { passive: true }) const audio_blob_url = await audio_recorder?.stop() const audio_data_url = audio_blob_url == null ? null : await blob_url_to_data_url(audio_blob_url) const magic_tag = `<meta name="pluto-insertion-spot-parameters">` const output_html = initial_html.replace( magic_tag, ` <script> window.pluto_recording_url = "data:;base64,${await base64_arraybuffer(pack({ steps: steps, scrolls: scrolls }))}"; window.pluto_recording_audio_url = ${audio_data_url == null ? null : `"${audio_data_url}"`}; </script> ${magic_tag}` ) console.log(current_recording_ref.current) let element = document.createElement("a") element.setAttribute("href", "data:text/html;charset=utf-8," + encodeURIComponent(output_html)) element.setAttribute("download", `${notebook_name_ref.current} recording.html`) element.style.display = "none" document.body.appendChild(element) element.click() document.body.removeChild(element) } recording_start_time_ref.current = 0 current_recording_ref.current = null set_recording_states({ is_recording: false, recording_waiting_to_start: false }) } return html` <div class="outline-frame recording"></div> ${recording_waiting_to_start ? html`<div class="outline-frame-actions-container"> <div class="overlay-button"> <button onclick=${() => { start_recording({ want_audio: true }) }} > <span><b>Start recording</b><span class="microphone-icon pluto-icon"></span></span> </button> </div> <div class="overlay-button record-no-audio"> <button onclick=${() => { start_recording({ want_audio: false }) }} > <span><b>Start recording</b> (no audio)<span class="mute-icon pluto-icon"></span></span> </button> </div> </div>` : is_recording ? html`<div class="outline-frame-actions-container"> <div class="overlay-button"> <button onclick=${() => { stop_recording() }} > <span><b>Stop recording</b><span class="stop-recording-icon pluto-icon"></span></span> </button> </div> </div>` : null} ` } let get_scroll_top = ({ cell_id, relative_distance }) => { let cell = document.getElementById(cell_id) if (cell) return cell.offsetTop + relative_distance * cell.offsetHeight - window.innerHeight / 2 } /** * * @param {{ * launch_params: import("./Editor.js").LaunchParameters, * initializing: boolean, * [key: string]: any, * }} props * @returns */ export const RecordingPlaybackUI = ({ launch_params, initializing, apply_notebook_patches, reset_notebook_state }) => { const { recording_url, recording_url_integrity, recording_audio_url } = launch_params let loaded_recording = useMemo( () => Promise.resolve().then(async () => { if (recording_url) { return unpack( new Uint8Array( await ( await fetch(new Request(recording_url, { integrity: recording_url_integrity ?? undefined })).then(assert_response_ok) ).arrayBuffer() ) ) } else { return null } }), [recording_url] ) let computed_reverse_patches_ref = useRef(/** @type{Array<PatchStep>?} */ (null)) useEffect(() => { loaded_recording.then(console.log) }, [loaded_recording]) let audio_element_ref = useRef(/** @type {HTMLAudioElement?} */ (null)) let match_state_to_playback_running_ref = useRef(false) let current_state_timestamp_ref = useRef(0) let [current_scrollY, set_current_scrollY] = useState(/** @type {number?} */ (null)) let [following_scroll, set_following_scroll] = useState(true) let following_scroll_ref = useRef(following_scroll) following_scroll_ref.current = following_scroll let was_playing_before_scrollout_ref = useRef(false) let last_manual_window_scroll_time_ref = useRef(0) let last_manual_window_smoothscroll_time_ref = useRef(0) const scroll_window = (scrollY, smooth = true) => { last_manual_window_scroll_time_ref.current = Date.now() last_manual_window_smoothscroll_time_ref.current = Date.now() window.scrollTo({ top: scrollY, behavior: smooth ? "smooth" : "auto", }) } let on_scroll = ({ cell_id, relative_distance }, smooth = true) => { let scrollY = get_scroll_top({ cell_id, relative_distance }) if (scrollY == null) return set_current_scrollY(scrollY) if (following_scroll_ref.current) { scroll_window(scrollY, smooth) } } const match_state_to_playback_ref = useRef(() => {}) match_state_to_playback_ref.current = async () => { match_state_to_playback_running_ref.current = true const deserialized = /** @type {RecordingData} */ (await loaded_recording) computed_reverse_patches_ref.current = computed_reverse_patches_ref.current ?? deserialized.steps.map(([t, s]) => [t, undefined]) const audio = audio_element_ref.current if (audio == null) return let new_timestamp = audio.currentTime let forward = new_timestamp >= current_state_timestamp_ref.current let directed = forward ? _.identity : _.reverse let lower = Math.min(current_state_timestamp_ref.current, new_timestamp) let upper = Math.max(current_state_timestamp_ref.current, new_timestamp) let scrolls_in_time_window = deserialized.scrolls.filter(([t, s]) => lower < t && t <= upper) if (scrolls_in_time_window.length > 0) { let scroll_state = _.last(directed(scrolls_in_time_window))?.[1] if (scroll_state) on_scroll(scroll_state) } let steps_in_current_direction = forward ? deserialized.steps : computed_reverse_patches_ref.current let steps_and_indices = steps_in_current_direction.map((x, i) => /** @type{[PatchStep, number]} */ ([x, i])) let steps_and_indices_in_time_window = steps_and_indices.filter(([[t, s], i]) => lower < t && t <= upper) let reverse_patches = [] for (let [[t, patches], i] of directed(steps_and_indices_in_time_window)) { reverse_patches = await apply_notebook_patches(patches, undefined, forward) if (forward) { computed_reverse_patches_ref.current[i] = [t, reverse_patches] } } // if (!_.isEmpty(steps_and_indices_in_time_window)) console.log(computed_reverse_patches_ref.current) current_state_timestamp_ref.current = new_timestamp if (audio.paused) { match_state_to_playback_running_ref.current = false } else { requestAnimationFrame(() => match_state_to_playback_ref.current()) } } let on_audio_playback_change = useCallback( (e) => { // console.log(e) if (!match_state_to_playback_running_ref.current) { match_state_to_playback_ref.current() } }, [match_state_to_playback_running_ref, match_state_to_playback_ref] ) const event_names = ["seeked", "suspend", "play", "pause", "ended", "waiting"] useLayoutEffect(() => { const audio_el = audio_element_ref.current if (audio_el) { event_names.forEach((en) => { audio_el.addEventListener(en, on_audio_playback_change) }) return () => { event_names.forEach((en) => { audio_el.removeEventListener(en, on_audio_playback_change) }) } } }, [audio_element_ref.current, on_audio_playback_change]) useEffect(() => { if (!initializing && recording_url != null) { // if we are playing a recording, fix the initial scroll position loaded_recording.then((x) => { let first_scroll = _.first(x?.scrolls) if (first_scroll) { let obs = new ResizeObserver(() => { console.log("Scrolling back to first recorded scroll position...") on_scroll(first_scroll[1], false) }) let old_value = history.scrollRestoration history.scrollRestoration = "manual" obs.observe(document.body) setTimeout(() => { history.scrollRestoration = old_value obs.disconnect() }, 3000) on_scroll(first_scroll[1], false) } document.fonts.ready.then(() => { console.info("Fonts loaded") on_scroll(first_scroll[1], false) }) }) } }, [initializing]) useEffect(() => { if (!initializing) { // TODO fons wat was je plan hier? } }, [initializing]) useEffect(() => { if (!initializing && recording_url != null) { let on_scroll = (e) => { let now = Date.now() let dt = (now - last_manual_window_scroll_time_ref.current) / 1000 let smooth_dt = (now - last_manual_window_smoothscroll_time_ref.current) / 1000 let is_first_smooth_scroll = smooth_dt === dt let ignore = dt < 1 && (is_first_smooth_scroll || smooth_dt < 0.2) if (ignore) { // then this must have been a browser-initiated smooth scroll event last_manual_window_smoothscroll_time_ref.current = now // console.log("ignoring scroll", { ignore, dt, smooth_dt, e }) } if (!ignore) { if (following_scroll_ref.current) { console.warn("Manual scroll detected, no longer following playback scroll", { dt, smooth_dt, e }) if (audio_element_ref.current != null) { was_playing_before_scrollout_ref.current = !audio_element_ref.current.paused audio_element_ref.current.pause() } set_following_scroll(false) } } } document.fonts.ready.then(() => { window.addEventListener("scroll", on_scroll, { passive: true }) }) return () => { // @ts-ignore window.removeEventListener("scroll", on_scroll, { passive: true }) } } }, [initializing, recording_url]) let frame = html`<div style=${{ opacity: following_scroll ? 0.0 : 1, top: `${current_scrollY ?? 0}px`, }} class="outline-frame playback" ></div>` return html` ${recording_url ? html`${!following_scroll ? html` <div class="outline-frame-actions-container"> <div class="overlay-button playback"> <button onclick=${() => { scroll_window(current_scrollY, true) set_following_scroll(true) if (was_playing_before_scrollout_ref.current && audio_element_ref.current) audio_element_ref.current.play() }} > <span>Back to <b>recording</b> <span class="follow-recording-icon pluto-icon"></span></span> </button> </div> </div>` : null} ${frame} <${AudioPlayer} audio_element_ref=${audio_element_ref} src=${recording_audio_url} loaded_recording=${loaded_recording} />` : null} ` } import _ from "../imports/lodash.js" import { html, useContext, useEffect, useMemo, useState } from "../imports/Preact.js" import { in_textarea_or_input } from "../common/KeyboardShortcuts.js" import { PlutoActionsContext } from "../common/PlutoContext.js" import { open_pluto_popup } from "../common/open_pluto_popup.js" export const RunArea = ({ runtime, running, queued, code_differs, on_run, on_interrupt, set_cell_disabled, depends_on_disabled_cells, running_disabled, on_jump, }) => { const on_save = on_run /* because disabled cells save without running */ const local_time_running_ms = useMillisSinceTruthy(running) const local_time_running_ns = local_time_running_ms == null ? null : 1e6 * local_time_running_ms const pluto_actions = useContext(PlutoActionsContext) const action = running || queued ? "interrupt" : running_disabled ? "save" : depends_on_disabled_cells && !code_differs ? "jump" : "run" const fmap = { on_interrupt, on_save, on_jump, on_run, } const titlemap = { interrupt: "Interrupt (Ctrl + Q)", save: "Save code without running", jump: "This cell depends on a disabled cell", run: "Run cell (Shift + Enter)", } const on_double_click = (/** @type {MouseEvent} */ e) => { console.log(running_disabled) if (running_disabled) open_pluto_popup({ type: "info", source_element: /** @type {HTMLElement?} */ (e.target), body: html`${`This cell is disabled. `} <a href="#" onClick=${(e) => { //@ts-ignore set_cell_disabled(false) e.preventDefault() window.dispatchEvent(new CustomEvent("close pluto popup")) }} >Enable this cell</a > ${` to run the code.`}`, }) } return html` <pluto-runarea class=${action}> <button onDblClick=${on_double_click} onClick=${fmap[`on_${action}`]} class="runcell" title=${titlemap[action]}> <span></span> </button> <span class="runtime">${prettytime(running ? local_time_running_ns ?? runtime : runtime)}</span> </pluto-runarea> ` } export const prettytime = (time_ns) => { if (time_ns == null) { return "---" } let result = time_ns const prefices = ["n", "", "m", ""] let i = 0 while (i < prefices.length - 1 && result >= 1000.0) { i += 1 result /= 1000 } const roundedtime = result.toFixed(time_ns < 100 || result >= 100.0 ? 0 : 1) return roundedtime + "\xa0" + prefices[i] + "s" } const update_interval = 50 /** * Returns the milliseconds passed since the argument became truthy. * If argument is falsy, returns undefined. * * @param {boolean} truthy */ export const useMillisSinceTruthy = (truthy) => { const [now, setNow] = useState(0) const [startRunning, setStartRunning] = useState(0) useEffect(() => { let interval if (truthy) { const now = +new Date() setStartRunning(now) setNow(now) interval = setInterval(() => setNow(+new Date()), update_interval) } return () => { interval && clearInterval(interval) } }, [truthy]) return truthy ? now - startRunning : undefined } export const useDebouncedTruth = (truthy, delay = 5) => { const [mytruth, setMyTruth] = useState(truthy) const setMyTruthAfterNSeconds = useMemo(() => _.debounce(setMyTruth, delay * 1000), [setMyTruth]) useEffect(() => { if (truthy) { setMyTruth(true) setMyTruthAfterNSeconds.cancel() } else { setMyTruthAfterNSeconds(false) } return () => {} }, [truthy]) return mytruth } import { open_pluto_popup } from "../common/open_pluto_popup.js" import _ from "../imports/lodash.js" import { html } from "../imports/Preact.js" export const SafePreviewUI = ({ process_waiting_for_permission, risky_file_source, restart, warn_about_untrusted_code }) => { return html` <div class="outline-frame safe-preview"></div> ${process_waiting_for_permission ? html`<div class="outline-frame-actions-container safe-preview"> <div class="safe-preview-info"> <span >Safe preview <button onclick=${(e) => { open_pluto_popup({ type: "info", big: true, should_focus: true, body: html` <h1>Safe preview</h1> <p>You are reading and editing this file without running Julia code.</p> <p> ${`When you are ready, you can `}<a href="#" onClick=${(e) => { e.preventDefault() restart(true) window.dispatchEvent(new CustomEvent("close pluto popup")) }} >run this notebook</a >. </p> ${warn_about_untrusted_code ? html` <pluto-output class="rich_output" ><div class="markdown"> <div class="admonition warning"> <p class="admonition-title">Warning</p> <p>Are you sure that you trust this file?</p> ${risky_file_source == null ? null : html`<p><code>${risky_file_source}</code></pre>`} <p>A malicious notebook can steal passwords and data.</p> </div> </div></pluto-output > ` : null} `, }) }} > <span><span class="info-icon pluto-icon"></span></span> </button> </span> </div> </div>` : null} ` } export const SafePreviewOutput = () => { return html`<pluto-output class="rich_output" ><div class="safe-preview-output"> <span class="offline-icon pluto-icon"></span><span>${`Code not executed in `}<em>Safe preview</em></span> </div></pluto-output >` } export const SafePreviewSanitizeMessage = `<div class="safe-preview-output"> <span class="offline-icon pluto-icon"></span><span>${`Scripts and styles not rendered in `}<em>Safe preview</em></span> </div>` import { useEventListener } from "../common/useEventListener.js" import { useEffect, useRef, useState } from "../imports/Preact.js" /** * Utility component that scrolls the page automatically, when the pointer is * moved to the upper or lower 30%. * * Useful for things like selections and drag-and-drop. */ export const Scroller = ({ active }) => { const pointer = useRef() const onmove = (e) => { pointer.current = { x: e.clientX, y: e.clientY } } useEventListener(window, "pointermove", onmove, []) useEventListener(window, "dragover", onmove, []) useEffect(() => { if (active.up || active.down) { let prev_time = null let current = true const scroll_update = (timestamp) => { if (current) { if (prev_time == null) { prev_time = timestamp } const dt = timestamp - prev_time prev_time = timestamp if (pointer.current) { const y_ratio = pointer.current.y / window.innerHeight if (active.up && y_ratio < 0.3) { window.scrollBy(0, (((-1200 * (0.3 - y_ratio)) / 0.3) * dt) / 1000) } else if (active.down && y_ratio > 0.7) { window.scrollBy(0, (((1200 * (y_ratio - 0.7)) / 0.3) * dt) / 1000) } } window.requestAnimationFrame(scroll_update) } } window.requestAnimationFrame(scroll_update) return () => (current = false) } }, [active.up, active.down]) return null } export const scroll_cell_into_view = (/** @type {string} */ cell_id) => { document.getElementById(cell_id)?.scrollIntoView({ block: "center", behavior: "smooth", }) } import { html, Component, useState, useEffect, useRef } from "../imports/Preact.js" import { has_ctrl_or_cmd_pressed } from "../common/KeyboardShortcuts.js" const get_element_position_in_document = (element) => { let top = 0 let left = 0 do { top += element.offsetTop || 0 left += element.offsetLeft || 0 element = element.offsetParent } while (element) return { top: top, left: left, } } const in_request_animation_frame = (fn) => { let last_known_arguments = null let ticking = false return (...args) => { last_known_arguments = args if (!ticking) { window.requestAnimationFrame(() => { fn(...last_known_arguments) ticking = false }) ticking = true } } } /** * * @typedef Coordinate2D * @property {number} x * @property {number} y */ export const SelectionArea = ({ on_selection, set_scroller, cell_order }) => { const mouse_position_ref = useRef() const is_selecting_ref = useRef(false) const element_ref = useRef(/** @type {HTMLElement?} */ (null)) const [selection, set_selection] = useState(/** @type {{start: Coordinate2D, end: Coordinate2D}?} */ (null)) useEffect(() => { const event_target_inside_this_notebook = (/** @type {MouseEvent} */ e) => { if (e.target == null) { return false } // this should also work for notebooks inside notebooks! let closest_editor = /** @type {HTMLElement} */ (e.target).closest("pluto-editor") let my_editor = element_ref.current?.closest("pluto-editor") return closest_editor === my_editor } const onmousedown = (/** @type {MouseEvent} */ e) => { // @ts-ignore const t = e.target?.tagName // TODO: also allow starting the selection in one codemirror and stretching it to another cell if ( e.button === 0 && event_target_inside_this_notebook(e) && (t === "PLUTO-EDITOR" || t === "MAIN" || t === "PLUTO-NOTEBOOK" || t === "PREAMBLE") ) { on_selection([]) set_selection({ start: { x: e.pageX, y: e.pageY }, end: { x: e.pageX, y: e.pageY } }) is_selecting_ref.current = true } } const onmouseup = (/** @type {MouseEvent} */ e) => { if (is_selecting_ref.current) { set_selection(null) set_scroller({ up: false, down: false }) is_selecting_ref.current = false } else { // if you didn't click on a UI element... if ( !e.composedPath().some((e) => { // @ts-ignore const tag = e.tagName if (e instanceof HTMLElement) return e.matches("pluto-shoulder, button.input_context_menu, button.foldcode") || e.closest(".input_context_menu") }) ) { // ...clear the selection on_selection([]) } } } let update_selection = in_request_animation_frame(({ pageX, pageY }) => { if (!is_selecting_ref.current || selection == null) return let new_selection_end = { x: pageX, y: pageY } const cell_nodes = Array.from(document.querySelectorAll("pluto-notebook > pluto-cell")) let A = { start_left: Math.min(selection.start.x, new_selection_end.x), start_top: Math.min(selection.start.y, new_selection_end.y), end_left: Math.max(selection.start.x, new_selection_end.x), end_top: Math.max(selection.start.y, new_selection_end.y), } let in_selection = cell_nodes.filter((cell) => { let cell_position = get_element_position_in_document(cell) let cell_size = cell.getBoundingClientRect() let B = { start_left: cell_position.left, start_top: cell_position.top, end_left: cell_position.left + cell_size.width, end_top: cell_position.top + cell_size.height, } return A.start_left < B.end_left && A.end_left > B.start_left && A.start_top < B.end_top && A.end_top > B.start_top }) set_scroller({ up: true, down: true }) on_selection(in_selection.map((x) => x.id)) set_selection({ start: selection.start, end: new_selection_end }) }) const onscroll = (e) => { if (is_selecting_ref.current) { update_selection({ pageX: mouse_position_ref.current.clientX, pageY: mouse_position_ref.current.clientY + document.documentElement.scrollTop }) } } const onmousemove = (e) => { mouse_position_ref.current = e if (is_selecting_ref.current) { update_selection({ pageX: e.pageX, pageY: e.pageY }) e.preventDefault() } } const onselectstart = (e) => { if (is_selecting_ref.current) { e.preventDefault() } } // Ctrl+A to select all cells const onkeydown = (e) => { if (e.key?.toLowerCase() === "a" && has_ctrl_or_cmd_pressed(e)) { // if you are not writing text somewhere else if (document.activeElement === document.body && (window.getSelection()?.isCollapsed ?? true)) { on_selection(cell_order) e.preventDefault() } } } document.addEventListener("mousedown", onmousedown) document.addEventListener("mouseup", onmouseup) document.addEventListener("mousemove", onmousemove) document.addEventListener("selectstart", onselectstart) document.addEventListener("keydown", onkeydown) document.addEventListener("scroll", onscroll, { passive: true }) return () => { document.removeEventListener("mousedown", onmousedown) document.removeEventListener("mouseup", onmouseup) document.removeEventListener("mousemove", onmousemove) document.removeEventListener("selectstart", onselectstart) document.removeEventListener("keydown", onkeydown) // @ts-ignore document.removeEventListener("scroll", onscroll, { passive: true }) } }, [selection]) // let translateY = `translateY(${Math.min(selection_start.y, selection_end.y)}px)` // let translateX = `translateX(${Math.min(selection_start.x, selection_end.x)}px)` // let scaleX = `scaleX(${Math.abs(selection_start.x - selection_end.x)})` // let scaleY = `scaleY(${Math.abs(selection_start.y - selection_end.y)})` if (selection == null) { return html`<span ref=${element_ref}></span>` } return html` <pl-select-area ref=${element_ref} style=${{ position: "absolute", background: "rgba(40, 78, 189, 0.24)", zIndex: 1000000, // Yes, really top: Math.min(selection.start.y, selection.end.y), left: Math.min(selection.start.x, selection.end.x), width: Math.abs(selection.start.x - selection.end.x), height: Math.abs(selection.start.y - selection.end.y), // Transform could be faster // top: 0, // left: 0, // width: 1, // height: 1, // transformOrigin: "top left", // transform: `${translateX} ${translateY} ${scaleX} ${scaleY}`, }} ></pl-select-area> ` } import { html, useRef, useState, useLayoutEffect, useEffect } from "../imports/Preact.js" export const SlideControls = () => { const button_prev_ref = useRef(/** @type {HTMLButtonElement?} */ (null)) const button_next_ref = useRef(/** @type {HTMLButtonElement?} */ (null)) const [presenting, set_presenting] = useState(false) const move_slides_with_keyboard = (/** @type {KeyboardEvent} */ e) => { const activeElement = document.activeElement if ( activeElement != null && activeElement !== document.body && activeElement !== button_prev_ref.current && activeElement !== button_next_ref.current ) { // We do not move slides with arrow if we have an active element return } if (e.key === "ArrowLeft" || e.key === "PageUp") { button_prev_ref.current?.click() } else if (e.key === "ArrowRight" || e.key === " " || e.key === "PageDown") { button_next_ref.current?.click() } else if (e.key === "Escape") { set_presenting(false) } else { return } e.preventDefault() } const calculate_slide_positions = (/** @type {Event} */ e) => { const notebook_node = /** @type {HTMLElement?} */ (e.target)?.closest("pluto-editor")?.querySelector("pluto-notebook") if (!notebook_node) return [] const height = window.innerHeight const headers = Array.from(notebook_node.querySelectorAll("pluto-output h1, pluto-output h2")) const pos = headers.map((el) => el.getBoundingClientRect()) const edges = pos.map((rect) => rect.top + window.scrollY) edges.push(notebook_node.getBoundingClientRect().bottom + window.scrollY) const scrollPositions = headers.map((el, i) => { if (el.tagName == "H1") { // center vertically const slideHeight = edges[i + 1] - edges[i] - height return edges[i] - Math.max(0, (height - slideHeight) / 2) } else { // align to top return edges[i] - 20 } }) return scrollPositions } const go_previous_slide = (/** @type {Event} */ e) => { const positions = calculate_slide_positions(e) const pos = positions.reverse().find((y) => y < window.scrollY - 10) if (pos) window.scrollTo(window.scrollX, pos) } const go_next_slide = (/** @type {Event} */ e) => { const positions = calculate_slide_positions(e) const pos = positions.find((y) => y - 10 > window.scrollY) if (pos) window.scrollTo(window.scrollX, pos) } const presenting_ref = useRef(false) presenting_ref.current = presenting // @ts-ignore window.present = () => { set_presenting(!presenting_ref.current) } useLayoutEffect(() => { document.body.classList.toggle("presentation", presenting) if (!presenting) return // We do not add listeners if not presenting window.addEventListener("keydown", move_slides_with_keyboard) return () => { window.removeEventListener("keydown", move_slides_with_keyboard) } }, [presenting]) return html` <nav id="slide_controls" inert=${!presenting}> <button ref=${button_prev_ref} class="changeslide prev" title="Previous slide" onClick=${go_previous_slide}><span></span></button> <button ref=${button_next_ref} class="changeslide next" title="Next slide" onClick=${go_next_slide}><span></span></button> </nav> ` } import { html, useEffect, useMemo, useRef, useState } from "../imports/Preact.js" import { cl } from "../common/ClassTable.js" import { prettytime, useMillisSinceTruthy } from "./RunArea.js" import { DiscreteProgressBar } from "./DiscreteProgressBar.js" import { PkgTerminalView } from "./PkgTerminalView.js" import { NotifyWhenDone } from "./NotifyWhenDone.js" import { scroll_to_busy_cell } from "./ProgressBar.js" /** * @param {{ * status: import("./Editor.js").StatusEntryData, * notebook: import("./Editor.js").NotebookData, * backend_launch_logs: string?, * my_clock_is_ahead_by: number, * }} props */ export const StatusTab = ({ status, notebook, backend_launch_logs, my_clock_is_ahead_by }) => { return html` <section> <${StatusItem} status_tree=${status} path=${[]} my_clock_is_ahead_by=${my_clock_is_ahead_by} nbpkg=${notebook.nbpkg} backend_launch_logs=${backend_launch_logs} /> <${NotifyWhenDone} status=${status} /> </section> ` } /** * Status items are sorted in the same order as they appear in list. Unspecified items are sorted to the end. */ const global_order = ` workspace create_process init_process pkg analysis waiting_for_others resolve remove add instantiate instantiate1 instantiate2 instantiate3 precompile run saving ` .split("\n") .map((x) => x.trim()) .filter((x) => x.length > 0) const blocklist = ["saving"] /** @type {Record<string,string>} */ const descriptions = { workspace: "Workspace setup", create_process: "Start Julia", init_process: "Initialize", pkg: "Package management", instantiate1: "instantiate", instantiate2: "instantiate", instantiate3: "instantiate", run: "Evaluating cells", evaluate: "Running code", registry_update: "Updating package registry", waiting_for_others: "Waiting for other notebooks to finish package operations", backend_launch: "Connecting to backend", backend_requesting: "Requesting a worker", backend_created: "Starting Pluto server", backend_responded: "Opening notebook file", backend_notebook_running: "Switching to live editing", } export const friendly_name = (/** @type {string} */ task_name) => { const descr = descriptions[task_name] return descr != null ? descr : isnumber(task_name) ? `Step ${task_name}` : task_name } const to_ns = (x) => x * 1e9 /** * @param {{ * status_tree: import("./Editor.js").StatusEntryData?, * path: string[], * my_clock_is_ahead_by: number, * nbpkg: import("./Editor.js").NotebookPkgData?, * backend_launch_logs: string?, * }} props */ const StatusItem = ({ status_tree, path, my_clock_is_ahead_by, nbpkg, backend_launch_logs }) => { if (status_tree == null) return null const mystatus = path.reduce((entry, key) => entry.subtasks[key], status_tree) if (!mystatus) return null const [is_open, set_is_open] = useState(path.length < 1) const started = path.length > 0 && is_started(mystatus) const finished = started && is_finished(mystatus) const busy = started && !finished const start = mystatus.started_at ?? 0 const end = mystatus.finished_at ?? 0 const local_busy_time = (useMillisSinceTruthy(busy) ?? 0) / 1000 const mytime = Date.now() / 1000 const busy_time = Math.max(local_busy_time, mytime - start - (mystatus.timing === "local" ? 0 : my_clock_is_ahead_by)) useEffect(() => { if (busy || mystatus.success === false) { let handle = setTimeout(() => { set_is_open(true) }, Math.max(100, 500 - path.length * 200)) return () => clearTimeout(handle) } }, [busy || mystatus.success === false]) useEffectWithPrevious( ([old_finished]) => { if (!old_finished && finished) { // let audio = new Audio("https://proxy.notificationsounds.com/message-tones/succeeded-message-tone/download/file-sounds-1210-succeeded.mp3") // audio.play() let handle = setTimeout(() => { set_is_open(false) }, 1800 - path.length * 200) return () => clearTimeout(handle) } }, [finished] ) const render_child_tasks = () => Object.entries(mystatus.subtasks) .sort((a, b) => sort_on(a[1], b[1])) .map(([key, _subtask]) => blocklist.includes(key) ? null : html`<${StatusItem} key=${key} status_tree=${status_tree} my_clock_is_ahead_by=${my_clock_is_ahead_by} path=${[...path, key]} nbpkg=${nbpkg} backend_launch_logs=${backend_launch_logs} />` ) const render_child_progress = () => { let kids = Object.values(mystatus.subtasks) let done = kids.reduce((acc, x) => acc + (is_finished(x) ? 1 : 0), 0) let busy = kids.reduce((acc, x) => acc + (is_busy(x) ? 1 : 0), 0) let total = kids.length let failed_indices = kids.reduce((acc, x, i) => (x.success === false ? [...acc, i] : acc), []) const onClick = mystatus.name === "evaluate" ? () => scroll_to_busy_cell() : undefined return html`<${DiscreteProgressBar} busy=${busy} done=${done} total=${total} failed_indices=${failed_indices} onClick=${onClick} />` } const inner = is_open ? // are all kids a numbered task? Object.values(mystatus.subtasks).every((x) => isnumber(x.name)) && Object.values(mystatus.subtasks).length > 0 ? render_child_progress() : render_child_tasks() : null let inner_progress = null if (started) { let t = total_tasks(mystatus) let d = total_done(mystatus) if (t > 1) { inner_progress = html`<span class="subprogress-counter">${" "}(${d}/${t})</span>` } } const can_open = Object.values(mystatus.subtasks).length > 0 return path.length === 0 ? inner : html`<pl-status data-depth=${path.length} class=${cl({ started, failed: mystatus.success === false, finished, busy, is_open, can_open, })} aria-expanded=${can_open ? is_open : undefined} > <div onClick=${(e) => { set_is_open(!is_open) }} > <span class="status-icon"></span> <span class="status-name">${friendly_name(mystatus.name)}${inner_progress}</span> <span class="status-time">${finished ? prettytime(to_ns(end - start)) : busy ? prettytime(to_ns(busy_time)) : null}</span> </div> ${inner} ${is_open && mystatus.name === "pkg" ? html`<${PkgTerminalView} value=${nbpkg?.terminal_outputs?.nbpkg_sync} />` : is_open && mystatus.name === "backend_launch" ? html`<${PkgTerminalView} value=${backend_launch_logs} />` : undefined} </pl-status>` } const isnumber = (str) => /^\d+$/.test(str) /** * @param {import("./Editor.js").StatusEntryData} a * @param {import("./Editor.js").StatusEntryData} b */ const sort_on = (a, b) => { const a_order = global_order.indexOf(a.name) const b_order = global_order.indexOf(b.name) if (a_order === -1 && b_order === -1) { if (a.started_at != null || b.started_at != null) { return (a.started_at ?? Infinity) - (b.started_at ?? Infinity) } else if (isnumber(a.name) && isnumber(b.name)) { return parseInt(a.name) - parseInt(b.name) } else { return a.name.localeCompare(b.name) } } else { let m = (x) => (x === -1 ? Infinity : x) return m(a_order) - m(b_order) } } /** * @param {import("./Editor.js").StatusEntryData} status */ export const is_finished = (status) => status.finished_at != null /** * @param {import("./Editor.js").StatusEntryData} status */ export const is_started = (status) => status.started_at != null /** * @param {import("./Editor.js").StatusEntryData} status */ export const is_busy = (status) => is_started(status) && !is_finished(status) /** * @param {import("./Editor.js").StatusEntryData} status * @returns {number} */ export const total_done = (status) => Object.values(status.subtasks).reduce((total, status) => total + total_done(status), is_finished(status) ? 1 : 0) /** * @param {import("./Editor.js").StatusEntryData} status * @returns {number} */ export const total_tasks = (status) => Object.values(status.subtasks).reduce((total, status) => total + total_tasks(status), 1) /** * @param {import("./Editor.js").StatusEntryData} status * @returns {string[]} */ export const path_to_first_busy_business = (status) => { for (let [name, child_status] of Object.entries(status.subtasks).sort((a, b) => sort_on(a[1], b[1]))) { if (is_busy(child_status)) { return [name, ...path_to_first_busy_business(child_status)] } } return [] } /** @returns {import("./Editor.js").StatusEntryData} */ export const useStatusItem = (/** @type {string} */ name, /** @type {boolean} */ started, /** @type {boolean} */ finished, subtasks = {}) => ({ name, subtasks, timing: "local", started_at: useMemo(() => (started || finished ? Date.now() / 1000 : null), [started || finished]), finished_at: useMemo(() => (finished ? Date.now() / 1000 : null), [finished]), }) /** Like `useEffect`, but the handler function gets the previous deps value as argument. */ const useEffectWithPrevious = (fn, deps) => { const ref = useRef(deps) useEffect(() => { let result = fn(ref.current) ref.current = deps return result }, deps) } import { html, useRef, useState, useContext, useEffect, useLayoutEffect } from "../imports/Preact.js" import { ANSITextOutput, OutputBody, PlutoImage } from "./CellOutput.js" import { PlutoActionsContext } from "../common/PlutoContext.js" import { useEventListener } from "../common/useEventListener.js" import { is_noop_action } from "../common/SliderServerClient.js" // this is different from OutputBody because: // it does not wrap in <div>. We want to do that in OutputBody for reasons that I forgot (feel free to try and remove it), but we dont want it here // i think this is because i wrote those css classes with the assumption that pluto cell output is wrapped in a div, and tree viewer contents are not // whatever // // We use a `<pre>${body}` instead of `<pre><code>${body}`, also for some CSS reasons that I forgot // // TODO: remove this, use OutputBody instead (maybe add a `wrap_in_div` option), and fix the CSS classes so that i all looks nice again export const SimpleOutputBody = ({ mime, body, cell_id, persist_js_state, sanitize_html = true }) => { switch (mime) { case "image/png": case "image/jpg": case "image/jpeg": case "image/gif": case "image/bmp": case "image/svg+xml": return html`<${PlutoImage} mime=${mime} body=${body} />` break case "text/plain": // Check if the content contains ANSI escape codes return html`<${ANSITextOutput} body=${body} />` case "application/vnd.pluto.tree+object": return html`<${TreeView} cell_id=${cell_id} body=${body} persist_js_state=${persist_js_state} sanitize_html=${sanitize_html} />` break default: return OutputBody({ mime, body, cell_id, persist_js_state, sanitize_html, last_run_timestamp: null }) break } } const More = ({ on_click_more, disable }) => { const [loading, set_loading] = useState(false) const element_ref = useRef(/** @type {HTMLElement?} */ (null)) useKeyboardClickable(element_ref) return html`<pluto-tree-more ref=${element_ref} tabindex=${disable ? "-1" : "0"} role="button" aria-disabled=${disable ? "true" : "false"} disable=${disable} class=${loading ? "loading" : disable ? "disabled" : ""} onclick=${(e) => { if (!loading && !disable) { if (on_click_more() !== false) { set_loading(true) } } }} >more</pluto-tree-more >` } const useKeyboardClickable = (element_ref) => { useEventListener( element_ref, "keydown", (e) => { if (e.key === " ") { e.preventDefault() } if (e.key === "Enter") { e.preventDefault() element_ref.current.click() } }, [] ) useEventListener( element_ref, "keyup", (e) => { if (e.key === " ") { e.preventDefault() element_ref.current.click() } }, [] ) } const prefix = ({ prefix, prefix_short }) => { const element_ref = useRef(/** @type {HTMLElement?} */ (null)) useKeyboardClickable(element_ref) return html`<pluto-tree-prefix role="button" tabindex="0" ref=${element_ref} ><span class="long">${prefix}</span><span class="short">${prefix_short}</span></pluto-tree-prefix >` } const actions_show_more = ({ pluto_actions, cell_id, node_ref, objectid, dim }) => { const actions = pluto_actions ?? node_ref.current.closest("pluto-cell")._internal_pluto_actions return actions.reshow_cell(cell_id ?? node_ref.current.closest("pluto-cell").id, objectid, dim) } export const TreeView = ({ mime, body, cell_id, persist_js_state, sanitize_html = true }) => { let pluto_actions = useContext(PlutoActionsContext) const node_ref = useRef(/** @type {HTMLElement?} */ (null)) const onclick = (e) => { // TODO: this could be reactified but no rush let self = node_ref.current if (!self) return let clicked = e.target.closest("pluto-tree-prefix") != null ? e.target.closest("pluto-tree-prefix").parentElement : e.target if (clicked !== self && !self.classList.contains("collapsed")) { return } const parent_tree = self.parentElement?.closest("pluto-tree") if (parent_tree != null && parent_tree.classList.contains("collapsed")) { return // and bubble upwards } self.classList.toggle("collapsed") } const on_click_more = () => { if (node_ref.current == null || node_ref.current.closest("pluto-tree.collapsed") != null) { return false } return actions_show_more({ pluto_actions, cell_id, node_ref, objectid: body.objectid, dim: 1, }) } const more_is_noop_action = is_noop_action(pluto_actions?.reshow_cell) const mimepair_output = (pair) => html`<${SimpleOutputBody} cell_id=${cell_id} mime=${pair[1]} body=${pair[0]} persist_js_state=${persist_js_state} sanitize_html=${sanitize_html} />` const more = html`<p-r><${More} disable=${more_is_noop_action || cell_id === "cell_id_not_known"} on_click_more=${on_click_more} /></p-r>` let inner = null switch (body.type) { case "Pair": const r = body.key_value return html`<pluto-tree-pair class=${body.type} ><p-r><p-k>${mimepair_output(r[0])}</p-k><p-v>${mimepair_output(r[1])}</p-v></p-r></pluto-tree-pair >` case "circular": return html`<em>circular reference</em>` case "Array": case "Set": case "Tuple": inner = html`<${prefix} prefix=${body.prefix} prefix_short=${body.prefix_short} /><pluto-tree-items class=${body.type} >${body.elements.map((r) => r === "more" ? more : html`<p-r>${body.type === "Set" ? "" : html`<p-k>${r[0]}</p-k>`}<p-v>${mimepair_output(r[1])}</p-v></p-r>` )}</pluto-tree-items >` break case "Dict": inner = html`<${prefix} prefix=${body.prefix} prefix_short=${body.prefix_short} /><pluto-tree-items class=${body.type} >${body.elements.map((r) => r === "more" ? more : html`<p-r><p-k>${mimepair_output(r[0])}</p-k><p-v>${mimepair_output(r[1])}</p-v></p-r>` )}</pluto-tree-items >` break case "NamedTuple": inner = html`<${prefix} prefix=${body.prefix} prefix_short=${body.prefix_short} /><pluto-tree-items class=${body.type} >${body.elements.map((r) => r === "more" ? more : html`<p-r><p-k>${r[0]}</p-k><p-v>${mimepair_output(r[1])}</p-v></p-r>` )}</pluto-tree-items >` break case "struct": inner = html`<${prefix} prefix=${body.prefix} prefix_short=${body.prefix_short} /><pluto-tree-items class=${body.type} >${body.elements.map((r) => html`<p-r><p-k>${r[0]}</p-k><p-v>${mimepair_output(r[1])}</p-v></p-r>`)}</pluto-tree-items >` break } return html`<pluto-tree class="collapsed ${body.type}" onclick=${onclick} ref=${node_ref}>${inner}</pluto-tree>` } const EmptyCols = ({ colspan = 999 }) => html`<thead> <tr class="empty"> <td colspan=${colspan}> <div> <small>(This table has no columns)</small></div> </td> </tr> </thead>` const EmptyRows = ({ colspan = 999 }) => html`<tr class="empty"> <td colspan=${colspan}> <div> <div></div> <small>(This table has no rows)</small> </div> </td> </tr>` export const TableView = ({ mime, body, cell_id, persist_js_state, sanitize_html }) => { let pluto_actions = useContext(PlutoActionsContext) const node_ref = useRef(null) const mimepair_output = (pair) => html`<${SimpleOutputBody} cell_id=${cell_id} mime=${pair[1]} body=${pair[0]} persist_js_state=${persist_js_state} sanitize_html=${sanitize_html} />` const more = (dim) => html`<${More} on_click_more=${() => actions_show_more({ pluto_actions, cell_id, node_ref, objectid: body.objectid, dim, })} />` // More than the columns, not big enough to break Firefox (https://bugzilla.mozilla.org/show_bug.cgi?id=675417) const maxcolspan = 3 + (body?.schema?.names?.length ?? 1) const thead = (body?.schema?.names?.length ?? 0) === 0 ? html`<${EmptyCols} colspan=${maxcolspan} />` : html`<thead> <tr class="schema-names"> ${["", ...body.schema.names].map((x) => html`<th>${x === "more" ? more(2) : x}</th>`)} </tr> <tr class="schema-types"> ${["", ...body.schema.types].map((x) => html`<th>${x === "more" ? null : x}</th>`)} </tr> </thead>` const tbody = html`<tbody> ${(body.rows?.length ?? 0) !== 0 ? body.rows.map( (row) => html`<tr> ${row === "more" ? html`<td class="pluto-tree-more-td" colspan=${maxcolspan}>${more(1)}</td>` : html`<th>${row[0]}</th> ${row[1].map((x) => html`<td><div>${x === "more" ? null : mimepair_output(x)}</div></td>`)}`} </tr>` ) : html`<${EmptyRows} colspan=${maxcolspan} />`} </tbody>` return html`<table class="pluto-table" ref=${node_ref}> ${thead}${tbody} </table>` } export let DivElement = ({ cell_id, style, classname, children, persist_js_state = false, sanitize_html = true }) => { const mimepair_output = (pair) => html`<${SimpleOutputBody} cell_id=${cell_id} mime=${pair[1]} body=${pair[0]} persist_js_state=${persist_js_state} sanitize_html=${sanitize_html} />` return html`<div style=${style} class=${classname}>${children.map(mimepair_output)}</div>` } import { html, useState, useEffect } from "../imports/Preact.js" import { cl } from "../common/ClassTable.js" import { scroll_cell_into_view } from "./Scroller.js" import { open_pluto_popup } from "../common/open_pluto_popup.js" export const UndoDelete = ({ recently_deleted, on_click }) => { const [hidden, set_hidden] = useState(true) useEffect(() => { if (recently_deleted != null && recently_deleted.length > 0) { set_hidden(false) const interval = setTimeout(() => { set_hidden(true) }, 8000 * Math.pow(recently_deleted.length, 1 / 3)) return () => { clearTimeout(interval) } } }, [recently_deleted]) let text = recently_deleted == null ? "" : recently_deleted.length === 1 ? "Cell deleted" : `${recently_deleted.length} cells deleted` return html` <nav id="undo_delete" inert=${hidden} class=${cl({ hidden })}> ${text} (<a href="#" onClick=${(e) => { e.preventDefault() set_hidden(true) on_click() }} ><strong>UNDO</strong></a >) </nav> ` } /** * @param {{ * notebook: import("./Editor.js").NotebookData, * recently_auto_disabled_cells: Record<string,[string,string]>, * }} props * */ export const RecentlyDisabledInfo = ({ notebook, recently_auto_disabled_cells }) => { useEffect(() => { Object.entries(recently_auto_disabled_cells).forEach(([cell_id, reason]) => { open_pluto_popup({ type: "info", source_element: document.getElementById(reason[0]), body: html`<a href=${`#${cell_id}`} onClick=${(e) => { scroll_cell_into_view(cell_id) e.preventDefault() e.stopPropagation() }} >Another cell</a >${` has been disabled because it also defined `}<code class="auto_disabled_variable">${reason[1]}</code>.`, }) }) }, [recently_auto_disabled_cells]) return null } import { EditorState, syntaxTree } from "../../imports/CodemirrorPlutoSetup.js" import { ScopeStateField } from "./scopestate_statefield.js" let get_root_variable_from_expression = (cursor) => { if (cursor.name === "IndexExpression") { cursor.firstChild() return get_root_variable_from_expression(cursor) } if (cursor.name === "FieldExpression") { cursor.firstChild() return get_root_variable_from_expression(cursor) } if (cursor.name === "Identifier") { cursor.firstChild() return cursor.node } return null } let VALID_DOCS_TYPES = [ "Identifier", "Field", "FieldExpression", "IndexExpression", "MacroFieldExpression", "MacroIdentifier", "Operator", "TypeHead", "Signature", "ParametrizedExpression", ] let keywords_that_have_docs_and_are_cool = [ "import", "export", "try", "catch", "finally", "quote", "do", "struct", "mutable", "module", "baremodule", "if", "let", ".", ] let is_docs_searchable = (/** @type {import("../../imports/CodemirrorPlutoSetup.js").TreeCursor} */ cursor) => { if (keywords_that_have_docs_and_are_cool.includes(cursor.name)) { return true } else if (VALID_DOCS_TYPES.includes(cursor.name)) { if (cursor.firstChild()) { do { // Numbers themselves can't be docs searched, but using numbers inside IndexExpression can be. if (cursor.name === "IntegerLiteral" || cursor.name === "FloatLiteral") { continue } // This is for the VERY specific case like `Vector{Int}(1,2,3,4) which I want to yield `Vector{Int}` if (cursor.name === "BraceExpression") { continue } if (cursor.name === "FieldName" || cursor.name === "MacroName" || cursor.name === "MacroFieldName") { continue } if (!is_docs_searchable(cursor)) { return false } } while (cursor.nextSibling()) cursor.parent() return true } else { return true } } else { return false } } export let get_selected_doc_from_state = (/** @type {EditorState} */ state, verbose = false) => { let selection = state.selection.main let scopestate = state.field(ScopeStateField) if (selection.empty) { // If the cell starts with a questionmark, we interpret it as a // docs query, so I'm gonna spit out exactly what the user typed. let current_line = state.doc.lineAt(selection.from).text if (current_line[0] === "?") { return current_line.slice(1) } let tree = syntaxTree(state) let cursor = tree.cursor() verbose && console.log(`Full tree:`, cursor.toString()) cursor.moveTo(selection.to, -1) let iterations = 0 do { verbose && console.group(`Iteration #${iterations}`) try { verbose && console.log("cursor", cursor.toString()) // Just to make sure we don't accidentally end up in an infinite loop if (iterations > 100) { console.group("Infinite loop while checking docs") console.log("Selection:", selection, state.doc.sliceString(selection.from, selection.to).trim()) console.log("Current node:", cursor.name, state.doc.sliceString(cursor.from, cursor.to).trim()) console.groupEnd() break } iterations = iterations + 1 // Collect parents in a list so I can compare them easily let parent_cursor = cursor.node.cursor() let parents = [] while (parent_cursor.parent()) { parents.push(parent_cursor.name) } // Also just have the first parent as a node let parent = cursor.node.parent if (parent == null) { break } verbose && console.log(`parents:`, parents) let index_of_struct_in_parents = parents.indexOf("StructDefinition") if (index_of_struct_in_parents !== -1) { verbose && console.log(`in a struct?`) // If we're in a struct, we basically barely want to search the docs: // - Struct name is useless: you are looking at the definition // - Properties are just named, not in the workspace or anything // Only thing we do want, are types and the right hand side of `=`'s. if (parents.includes("binding") && parents.indexOf("binding") < index_of_struct_in_parents) { // We're inside a `... = ...` inside the struct } else if (parents.includes("TypedExpression") && parents.indexOf("TypedExpression") < index_of_struct_in_parents) { // We're inside a `x::X` inside the struct } else if (parents.includes("SubtypedExpression") && parents.indexOf("SubtypedExpression") < index_of_struct_in_parents) { // We're inside `Real` in `struct MyNumber<:Real` while (parent?.name !== "SubtypedExpression") { parent = parent.parent } const type_node = parent.lastChild if (type_node.from <= cursor.from && type_node.to >= cursor.to) { return state.doc.sliceString(type_node.from, type_node.to) } } else if (cursor.name === "struct" || cursor.name === "mutable") { cursor.parent() cursor.firstChild() if (cursor.name === "struct") return "struct" if (cursor.name === "mutable") { cursor.nextSibling() // @ts-ignore if (cursor.name === "struct") return "mutable struct" } return undefined } else { return undefined } } if (cursor.name === "AbstractDefinition") { return "abstract type" } // `callee(...)` should yield "callee" // (Only if it is on the `(` or `)`, or in like a space, // not on arguments (those are handle later)) if (cursor.name === "CallExpression") { cursor.firstChild() // Move to callee return is_docs_searchable(cursor) ? state.doc.sliceString(cursor.from, cursor.to) : undefined } // `Base.:%` should yield... "Base.:%" if ( (cursor.name === "Operator" || cursor.name === "" || cursor.name === "Identifier") && parent.name === "QuoteExpression" && parent.parent?.name === "FieldExpression" ) { verbose && console.log("Quirky symbol in a quote expression") // TODO Needs a fix added to is_docs_searchable, but this works fine for now return state.sliceDoc(parent.parent.from, parent.parent.to) } if (cursor.name === "ParameterizedIdentifier") { cursor.firstChild() // Move to callee return is_docs_searchable(cursor) ? state.doc.sliceString(cursor.from, cursor.to) : undefined } // `html"asd"` should yield "html" if (cursor.name === "Identifier" && parent.name === "Prefix") { continue } if (cursor.name === "PrefixedString") { cursor.firstChild() // Move to callee let name = state.doc.sliceString(cursor.from, cursor.to) return `${name}"` } // For identifiers in typed expressions e.g. `a::Number` always show the type if (cursor.name === "Identifier" && parent.name === "TypedExpression") { cursor.parent() // Move to TypedExpression cursor.lastChild() // Move to type Identifier return is_docs_searchable(cursor) ? state.doc.sliceString(cursor.from, cursor.to) : undefined } // For the :: inside a typed expression, show the type if (cursor.name === "TypedExpression") { cursor.lastChild() // Move to callee return is_docs_searchable(cursor) ? state.doc.sliceString(cursor.from, cursor.to) : undefined } // Docs for spread operator when you're in a SpreadExpression if (cursor.name === "SpreadExpression") { return "..." } // For Identifiers, we expand them in the hopes of finding preceding (left side) parts. // So we make sure we don't move to the left (`to` stays the same) and then possibly expand if (parent.to === cursor.to) { if (VALID_DOCS_TYPES.includes(cursor.name) && VALID_DOCS_TYPES.includes(parent.name)) { verbose && console.log("Expanding identifier") continue } } // If we are an identifier inside a NamedField, we want to show whatever we are a named part of // EXEPT, when we are in the last part (the value) of a NamedField, because then we can show // the value. if (cursor.name === "Identifier" && parent.name === "NamedField") { if (parent.lastChild.from != cursor.from && parent.lastChild.to != cursor.to) { continue } } // `a = 1` would yield `=`, `a += 1` would yield `+=` if (cursor.name === "binding") { let end_of_first = cursor.node.firstChild.to let beginning_of_last = cursor.node.lastChild.from return state.doc.sliceString(end_of_first, beginning_of_last).trim() } // If we happen to be in an argumentslist, we should go to the parent if (cursor.name === "ArgumentList") { continue } // If we are on an identifiers inside the argumentslist of a function *declaration*, // we should go to the parent. if ( cursor.name === "Identifier" && parent.name === "ArgumentList" && (parent.parent.parent.name === "FunctionAssignmentExpression" || parent.parent.name === "FunctionDefinition") ) { continue } // Identifier that's actually a symbol? Not useful at all! if (cursor.name === "Identifier" && parent.name === "Symbol") { continue } // If we happen to be anywhere else in a function declaration, we want the function name // `function X() ... end` should yield `X` if (cursor.name === "FunctionDefinition") { cursor.firstChild() // "function" cursor.nextSibling() // Identifier return is_docs_searchable(cursor) ? state.doc.sliceString(cursor.from, cursor.to) : undefined } // `X() = ...` should yield `X` if (cursor.name === "FunctionAssignmentExpression") { cursor.firstChild() // Identifier return is_docs_searchable(cursor) ? state.doc.sliceString(cursor.from, cursor.to) : undefined } if (cursor.name === "Identifier" && parent.name === "MacroIdentifier") { continue } // `@X` should yield `X` if (cursor.name === "MacroExpression") { cursor.firstChild() return state.doc.sliceString(cursor.from, cursor.to) } // `1 + 1` should yield `+` // A bit odd, but we don't get the span of the actual operator in a binary expression, // so we infer it from the end of the left side and start of the right side. if (cursor.name === "BinaryExpression") { let end_of_first = cursor.node.firstChild.to let beginning_of_last = cursor.node.lastChild.from return state.doc.sliceString(end_of_first, beginning_of_last).trim() } // Putting in a special case for ternary expressions (a ? b : c) because I think // these might be confusing to new users so I want to extra show them docs about it. // Sad thing is, our current docs think that `?:` means "give me info about :", so // TODO Make Pluto treat `?` prefixes (or specifically `?:`) as normal identifiers if (cursor.name === "TernaryExpression") { return "??:" } if (VALID_DOCS_TYPES.includes(cursor.name) || keywords_that_have_docs_and_are_cool.includes(cursor.name)) { if (!is_docs_searchable(cursor)) { verbose && console.log("Not searchable aaa") return undefined } // When we can already see that a variable is local, we don't want to show docs for it // because we won't be able to load it in anyway. let root_variable_node = get_root_variable_from_expression(cursor.node.cursor) if (root_variable_node == null) { return state.doc.sliceString(cursor.from, cursor.to) } // We have do find the current usage of the variable, and make sure it has no definition inside this cell let usage = scopestate.usages.find((x) => x.usage.from === root_variable_node.from && x.usage.to === root_variable_node.to) // If we can't find the usage... we just assume it can be docs showed I guess if (usage?.definition == null) { return state.doc.sliceString(cursor.from, cursor.to) } } // If you get here (so you have no cool other matches) and your parent is a FunctionDefinition, // I don't want to show you the function name, so imma head out. if (parent.name === "FunctionDefinition") { return undefined } // If we are expanding to an AssigmentExpression, we DONT want to show `=` if (parent.name === "binding") { return undefined } } finally { verbose && console.groupEnd() } } while (cursor.parent()) } else { return state.doc.sliceString(selection.from, selection.to).trim() } } import { render } from "../../imports/Preact.js" import { WidgetType } from "../../imports/CodemirrorPlutoSetup.js" /** * Use this Widget to render (P)react components as codemirror widgets. */ export class ReactWidget extends WidgetType { /** @param {import("../../imports/Preact.js").ReactElement} element */ constructor(element) { super() this.element = element } eq(other) { return false } toDOM() { let span = document.createElement("span") render(this.element, span) return span } updateDOM(dom) { render(this.element, dom) return true } } import { Compartment, merge, StateField, ViewPlugin, StateEffect, EditorState, EditorView, Annotation, invertedEffects, } from "../../imports/CodemirrorPlutoSetup.js" import { LastRemoteCodeSetTimeFacet } from "../CellInput.js" /** * Suggest AI-generated code as the new input of a cell. * @param {HTMLElement?} start_node Any node that is a child of a cell. AI suggestion will happen in the parent cell. * @param {{code: string, reject?: boolean}} detail `reject` means reject the AI suggestion. * @returns {Promise<void>} */ export const start_ai_suggestion = (start_node, detail) => new Promise(async (resolve, reject) => { const get_cm = () => start_node?.closest("pluto-cell")?.querySelector("pluto-input > .cm-editor .cm-content") const cm = get_cm() if (cm) { const get_live_cm = () => { const cm = get_cm() if (cm?.hasAttribute("data-currently-live")) { return cm } return null } let live_cm = get_live_cm() if (!live_cm) { cm.scrollIntoView({ behavior: "smooth", block: "nearest", inline: "nearest" }) } while (!live_cm) { await new Promise((resolve) => setTimeout(resolve, 50)) live_cm = get_live_cm() } live_cm.dispatchEvent(new CustomEvent("ai-suggestion", { detail })) resolve() } else { reject(new Error("Could not find an editor that belongs to this element")) } }) export const AiSuggestionPlugin = () => { const compartment = new Compartment() const start_ai_suggestion = (/** @type {EditorView} */ view, suggested_code) => { const state = view.state return view.dispatch({ effects: [ AISuggestionTimeEffect.of(Date.now()), compartment.reconfigure([ // merge.unifiedMergeView({ original: state.doc, gutter: false, allowInlineDiffs: true || suggested_code.split("\n").length === 1, }), AllAccepted, DisableMergeWhenAllAccepted(compartment), DontDiffNewChanges, DontDiffNewChangesInverter, ]), ], changes: { from: 0, to: state.doc.length, insert: suggested_code, }, }) } const disabled_extension = [] const reject_ai_suggestion = (/** @type {EditorView} */ view) => { const state = view.state // @ts-ignore const is_active = compartment.get(state)?.length !== disabled_extension.length if (!is_active) return const { chunks } = merge.getChunks(state) ?? {} if (!chunks) return if (chunks.length === 0) return const original_doc = merge.getOriginalDoc(state) view.dispatch({ changes: chunks.map((chunk) => ({ from: chunk.fromB, to: Math.min(state.doc.length, chunk.toB), insert: original_doc.slice(chunk.fromA, chunk.toA), })), effects: [compartment.reconfigure([])], }) } const ai_event_listener = EditorView.domEventHandlers({ "ai-suggestion": (event, view) => { const { code, reject, on_ } = event.detail if (reject) { reject_ai_suggestion(view) } else { start_ai_suggestion(view, code) } return true }, }) return [AISuggestionTime, ai_event_listener, hello_im_available, compartment.of(disabled_extension)] } const hello_im_available = ViewPlugin.define((view) => { view.contentDOM.setAttribute("data-currently-live", "true") return {} }) const AllAccepted = StateField.define({ create: () => false, update: (all_accepted, tr) => { if (!tr.docChanged) all_accepted return merge.getOriginalDoc(tr.state).eq(tr.newDoc) }, }) /** * @type {any} */ const AISuggestionTimeEffect = StateEffect.define() const AISuggestionTime = StateField.define({ create: () => 0, update(value, tr) { for (let effect of tr.effects) { if (effect.is(AISuggestionTimeEffect)) return effect.value } return value }, }) const DisableMergeWhenAllAccepted = (/** @type {Compartment} */ compartment) => EditorState.transactionExtender.of((tr) => { const code_was_submitted_after_ai_suggestion = tr.startState.field(AISuggestionTime) < tr.startState.facet(LastRemoteCodeSetTimeFacet) if (code_was_submitted_after_ai_suggestion || tr.startState.field(AllAccepted)) { console.log("auto-disabling merge") return { effects: [compartment.reconfigure([])], } } return null }) const EditWasMadeByDontDiffNewChanges = Annotation.define() /** * An extension to add to the unified merge view. With this extension, when you make edits that are outside one of the existing chunks, no new chunk will be created. */ const DontDiffNewChanges = EditorState.transactionExtender.of((tr) => { if (!tr.docChanged) return null if (!tr.isUserEvent) return null const original_doc = merge.getOriginalDoc(tr.startState) const gc = merge.getChunks(tr.startState) if (!gc) return null const { chunks } = gc if (chunks.length === 0) return null // Go from a position in the editable doc to the position in the original doc. const map_pos_to_original = (pos) => { let out = pos for (const chunk of chunks) { if (chunk.fromB <= pos) { out = Math.max(chunk.fromA, pos + chunk.toA - chunk.toB) } } return out } const changes = [] tr.changes.iterChanges((fromA, toA, fromB, toB, inserted) => { for (let chunk of chunks) { // If the change is completely contained in a chunk, don't modify the original  just let the user edit the chunk. if (chunk.fromB <= fromA && toA <= chunk.toB && fromA < chunk.toB) return } // Otherwise, this is the matching change in the original doc. changes.push({ from: map_pos_to_original(fromA), to: map_pos_to_original(toA), insert: inserted, }) }) if (changes.length === 0) return null const changes_mapped_to_original_doc = EditorState.create({ doc: original_doc }).changes(changes) return { effects: merge.originalDocChangeEffect(tr.startState, changes_mapped_to_original_doc), annotations: EditWasMadeByDontDiffNewChanges.of(changes_mapped_to_original_doc.invert(original_doc)), } }) /** Ensure that the effects from DontDiffNewChanges are undone when you Ctrl+Z. */ const DontDiffNewChangesInverter = invertedEffects.of((tr) => { const an = tr.annotation(EditWasMadeByDontDiffNewChanges) return an ? [merge.originalDocChangeEffect(tr.state, an)] : [] }) import _ from "../../imports/lodash.js" import { StateField, EditorView, Decoration } from "../../imports/CodemirrorPlutoSetup.js" import { ReactWidget } from "./ReactWidget.js" import { html } from "../../imports/Preact.js" const ARBITRARY_INDENT_LINE_WRAP_LIMIT = 12 export const get_start_tabs = (line) => /^\t*/.exec(line)?.[0] ?? "" const get_decorations = (/** @type {import("../../imports/CodemirrorPlutoSetup.js").EditorState} */ state) => { let decorations = [] // TODO? Don't create new decorations when a line hasn't changed? for (let i of _.range(0, state.doc.lines)) { let line = state.doc.line(i + 1) const num_tabs = get_start_tabs(line.text).length if (num_tabs === 0) continue const how_much_to_indent = Math.min(num_tabs, ARBITRARY_INDENT_LINE_WRAP_LIMIT) const offset = how_much_to_indent * state.tabSize const linerwapper = Decoration.line({ attributes: { style: `--indented: ${offset}ch;`, class: "awesome-wrapping-plugin-the-line", }, }) // Need to push before the tabs one else codemirror gets madddd decorations.push(linerwapper.range(line.from, line.from)) if (how_much_to_indent > 0) { decorations.push( Decoration.mark({ class: "awesome-wrapping-plugin-the-tabs", }).range(line.from, line.from + how_much_to_indent) ) } if (num_tabs > how_much_to_indent) { for (let i of _.range(how_much_to_indent, num_tabs)) { decorations.push( Decoration.replace({ widget: new ReactWidget(html`<span style=${{ opacity: 0.2 }}> </span>`), block: false, }).range(line.from + i, line.from + i + 1) ) } } } return Decoration.set(decorations) } /** * Plugin that makes line wrapping in the editor respect the identation of the line. * It does this by adding a line decoration that adds padding-left (as much as there is indentation), * and adds the same amount as negative "text-indent". The nice thing about text-indent is that it * applies to the initial line of a wrapped line. */ export const awesome_line_wrapping = StateField.define({ create(state) { return get_decorations(state) }, update(deco, tr) { if (!tr.docChanged) return deco return get_decorations(tr.state) }, provide: (f) => EditorView.decorations.from(f), }) import { combineConfig, Facet, StateField } from "../../imports/CodemirrorPlutoSetup.js" import { syntaxTree } from "../../imports/CodemirrorPlutoSetup.js" import { EditorView } from "../../imports/CodemirrorPlutoSetup.js" import { Decoration } from "../../imports/CodemirrorPlutoSetup.js" /** * ADAPTED MATCH BRACKETS PLUGIN FROM CODEMIRROR * Original: https://github.com/codemirror/matchbrackets/blob/99bf6b7e6891c09987269e370dc45ab1d588b875/src/matchbrackets.ts * * I changed it to ignore the closedBy and openBy properties provided by lezer, because these * are all wrong for julia... Also, this supports returning multiple matches, like `if ... elseif ... end`, etc. * On top of that I added `match_block` to match the block brackets, like `begin ... end` and all of those. * Also it doesn't do non-matching now, there is just matching or nothing. */ function match_try_node(node) { let try_node = node.parent.firstChild let possibly_end = node.parent.lastChild let did_match = possibly_end.name === "end" if (!did_match) return null let catch_node = node.parent.getChild("CatchClause")?.firstChild let else_node = node.parent.getChild("TryElseClause")?.firstChild let finally_node = node.parent.getChild("FinallyClause")?.firstChild return [ { from: try_node.from, to: try_node.to }, catch_node && { from: catch_node.from, to: catch_node.to }, else_node && { from: else_node.from, to: else_node.to }, finally_node && { from: finally_node.from, to: finally_node.to }, { from: possibly_end.from, to: possibly_end.to }, ].filter((x) => x != null) } function match_block(node) { if (node.name === "end") { if (node.parent.name === "IfStatement") { // Try moving to the "if" part because // the rest of the code is looking for that node = node.parent?.firstChild?.firstChild } else { node = node.parent.firstChild } } if (node == null) { return [] } // if (node.name === "StructDefinition") node = node.firstChild if (node.name === "mutable" || node.name === "struct") { if (node.name === "struct") node = node.parent.firstChild let struct_node = node.parent.getChild("struct") let possibly_end = node.parent.lastChild let did_match = possibly_end.name === "end" if (!did_match || !struct_node) return null return [ { from: node.from, to: struct_node.to }, { from: possibly_end.from, to: possibly_end.to }, ] } if (node.name === "struct") { let possibly_end = node.parent.lastChild let did_match = possibly_end.name === "end" if (!did_match) return null return [ { from: node.from, to: node.to }, { from: possibly_end.from, to: possibly_end.to }, ] } if (node.name === "quote") { let possibly_end = node.parent.lastChild let did_match = possibly_end.name === "end" if (!did_match) return null return [ { from: node.from, to: node.to }, { from: possibly_end.from, to: possibly_end.to }, ] } if (node.name === "begin") { let possibly_end = node.parent.lastChild let did_match = possibly_end.name === "end" if (!did_match) return null return [ { from: node.from, to: node.to }, { from: possibly_end.from, to: possibly_end.to }, ] } if (node.name === "do") { let possibly_end = node.parent.lastChild let did_match = possibly_end.name === "end" if (!did_match) return null return [ { from: node.from, to: node.to }, { from: possibly_end.from, to: possibly_end.to }, ] } if (node.name === "for") { let possibly_end = node.parent.lastChild let did_match = possibly_end.name === "end" if (!did_match) return null return [ { from: node.from, to: node.to }, { from: possibly_end.from, to: possibly_end.to }, ] } if (node.name === "let") { let possibly_end = node.parent.lastChild let did_match = possibly_end.name === "end" if (!did_match) return null return [ { from: node.from, to: node.to }, { from: possibly_end.from, to: possibly_end.to }, ] } if (node.name === "macro") { let possibly_end = node.parent.lastChild let did_match = possibly_end.name === "end" if (!did_match) return null return [ { from: node.from, to: node.to }, { from: possibly_end.from, to: possibly_end.to }, ] } if (node.name === "function") { let possibly_end = node.parent.lastChild let did_match = possibly_end.name === "end" if (!did_match) return null return [ { from: node.from, to: node.to }, { from: possibly_end.from, to: possibly_end.to }, ] } if (node.name === "while") { let possibly_end = node.parent.lastChild let did_match = possibly_end.name === "end" if (!did_match) return null return [ { from: node.from, to: node.to }, { from: possibly_end.from, to: possibly_end.to }, ] } if (node.name === "type") node = node.parent.firstChild if (node.name === "abstract" || node.name === "primitive") { let possibly_end = node.parent.lastChild let did_match = possibly_end.name === "end" let struct_node = node.parent.getChild("type") if (!did_match || !struct_node) return null return [ { from: node.from, to: struct_node.to }, { from: possibly_end.from, to: possibly_end.to }, ] } if (node.name === "if" || node.name === "else" || node.name === "elseif") { if (node.name === "if") node = node.parent let iselse = false if (node.name === "else") { node = node.parent iselse = true } if (node.name === "elseif") node = node.parent.parent let try_node = node.parent.firstChild let possibly_end = node.parent.lastChild let did_match = possibly_end.name === "end" if (!did_match) return null if (iselse && try_node.name === "try") { return match_try_node(node) // try catch else finally end } let decorations = [] decorations.push({ from: try_node.from, to: try_node.to }) for (let elseif_clause_node of node.parent.getChildren("ElseifClause")) { let elseif_node = elseif_clause_node.firstChild decorations.push({ from: elseif_node.from, to: elseif_node.to }) } for (let else_clause_node of node.parent.getChildren("ElseClause")) { let else_node = else_clause_node.firstChild decorations.push({ from: else_node.from, to: else_node.to }) } decorations.push({ from: possibly_end.from, to: possibly_end.to }) return decorations } if (node.name === "try" || node.name === "catch" || node.name === "finally" || node.name === "else") { if (node.name === "catch") node = node.parent if (node.name === "finally") node = node.parent if (node.name === "else") node = node.parent let possibly_end = node.parent.lastChild let did_match = possibly_end.name === "end" if (!did_match) return null return match_try_node(node) } if (node.name === "module" || node.name === "baremodule") { let possibly_end = node.parent.lastChild let did_match = possibly_end.name === "end" if (!did_match) return null return [ { from: node.from, to: node.to }, { from: possibly_end.from, to: possibly_end.to }, ] } return null } const baseTheme = EditorView.baseTheme({ ".cm-matchingBracket": { backgroundColor: "#328c8252" }, ".cm-nonmatchingBracket": { backgroundColor: "#bb555544" }, }) const DefaultScanDist = 10000, DefaultBrackets = "()[]{}" const bracketMatchingConfig = Facet.define({ combine(configs) { return combineConfig(configs, { afterCursor: true, brackets: DefaultBrackets, maxScanDistance: DefaultScanDist, }) }, }) const matchingMark = Decoration.mark({ class: "cm-matchingBracket" }), nonmatchingMark = Decoration.mark({ class: "cm-nonmatchingBracket" }) const bracketMatchingState = StateField.define({ create() { return Decoration.none }, update(deco, tr) { if (!tr.docChanged && !tr.selection) return deco let decorations = [] let config = tr.state.facet(bracketMatchingConfig) for (let range of tr.state.selection.ranges) { if (!range.empty) continue let match = matchBrackets(tr.state, range.head, -1, config) || (range.head > 0 && matchBrackets(tr.state, range.head - 1, 1, config)) || (config.afterCursor && (matchBrackets(tr.state, range.head, 1, config) || (range.head < tr.state.doc.length && matchBrackets(tr.state, range.head + 1, -1, config)))) if (!match) continue let mark = matchingMark for (let pos of match) { decorations.push(mark.range(pos.from, pos.to)) } } return Decoration.set(decorations, true) }, provide: (f) => EditorView.decorations.from(f), }) const bracketMatchingUnique = [bracketMatchingState, baseTheme] /// Create an extension that enables bracket matching. Whenever the /// cursor is next to a bracket, that bracket and the one it matches /// are highlighted. Or, when no matching bracket is found, another /// highlighting style is used to indicate this. export function bracketMatching(config = {}) { return [bracketMatchingConfig.of(config), bracketMatchingUnique] } /// Find the matching bracket for the token at `pos`, scanning /// direction `dir`. Only the `brackets` and `maxScanDistance` /// properties are used from `config`, if given. Returns null if no /// bracket was found at `pos`, or a match result otherwise. export function matchBrackets(state, pos, dir, config = {}) { let maxScanDistance = config.maxScanDistance || DefaultScanDist, brackets = config.brackets || DefaultBrackets let tree = syntaxTree(state), node = tree.resolveInner(pos, dir) let result = match_block(node) return result || matchPlainBrackets(state, pos, dir, tree, bracket_node_name_normalizer(node.name), maxScanDistance, brackets) } function matchPlainBrackets(state, pos, dir, tree, tokenType, maxScanDistance, brackets) { let startCh = dir < 0 ? state.sliceDoc(pos - 1, pos) : state.sliceDoc(pos, pos + 1) let bracket = brackets.indexOf(startCh) if (bracket < 0 || (bracket % 2 == 0) != dir > 0) return null let startToken = { from: dir < 0 ? pos - 1 : pos, to: dir > 0 ? pos + 1 : pos } let iter = state.doc.iterRange(pos, dir > 0 ? state.doc.length : 0), depth = 0 for (let distance = 0; !iter.next().done && distance <= maxScanDistance; ) { let text = iter.value if (dir < 0) distance += text.length let basePos = pos + distance * dir for (let pos = dir > 0 ? 0 : text.length - 1, end = dir > 0 ? text.length : -1; pos != end; pos += dir) { let found = brackets.indexOf(text[pos]) if (found < 0 || bracket_node_name_normalizer(tree.resolve(basePos + pos, 1).name) != tokenType) continue if ((found % 2 == 0) == dir > 0) { depth++ } else if (depth == 1) { // Closing if (found >> 1 == bracket >> 1) { return [startToken, { from: basePos + pos, to: basePos + pos + 1 }] } else { return null } } else { depth-- } } if (dir > 0) distance += text.length } return iter.done ? [startToken] : null } /** * Little modification to the original matchPlainBrackets function: in our Julia language, the node that opens a bracket is called "(". In e.g. markdown it's called LinkMark or something (the same name for opening and closing). We don't have this so we make them equal. */ const bracket_node_name_normalizer = (/** @type {String} */ node_name) => { switch (node_name) { case "(": case ")": return "()" case "[": case "]": return "[]" case "{": case "}": return "{}" default: return node_name } } import { EditorView, autocomplete, EditorState, keymap } from "../../imports/CodemirrorPlutoSetup.js" /** * Cell movement plugin! * * Two goals: * - Make movement and operations on the edges of cells work with their neighbors * - Prevent holding a button down to continue operations on neighboring cells * * I lean a lot on `view.moveByChar` and `view.moveVertically` from codemirror. * They will give you the position of the cursor after moving, and comparing that * to the current selection will tell you if the cursor would have moved normally. * If it would have moved normally, we don't do anything. Else, it's our time * * We use that in the keysmaps and the prevention of holding a button down. * * TODO Move the cursor to the same column in the new cell when moving vertically * TODO Put delete and backspace and such here too, but is harder because they * .... need to also modify this/the neighbor cell. */ /** * @typedef FocusOnNeighborFunction * @type {(options: { cell_delta: number, line: number, character: number }) => void} */ /** * @param {object} options * @param {FocusOnNeighborFunction} options.focus_on_neighbor */ let cell_movement_keys = ({ focus_on_neighbor }) => { // All arrows do basically the same now: // - Check if the cursor would have moved normally // - If it would have moved normally, don't do anything so codemirror can move the cursor // - Else move the cursor to the neighbor cell // TODO for verticals: const CellArrowLeft = (/** @type {EditorView} */ view) => { let selection = view.state.selection.main if (!selection.empty) return false if (!view.moveByChar(selection, false).eq(selection)) return false focus_on_neighbor({ cell_delta: -1, line: Infinity, character: Infinity, }) return true } const CellArrowRight = (/** @type {EditorView} */ view) => { let selection = view.state.selection.main if (!selection.empty) return false if (!view.moveByChar(selection, true).eq(selection)) return false focus_on_neighbor({ cell_delta: 1, line: 0, character: 0, }) return true } const CellArrowUp = (/** @type {EditorView} */ view) => { let selection = view.state.selection.main if (!selection.empty) return false if (!view.moveVertically(selection, false).eq(selection)) return false focus_on_neighbor({ cell_delta: -1, line: Infinity, character: Infinity, }) return true } const CellArrowDown = (/** @type {EditorView} */ view) => { let selection = view.state.selection.main if (!selection.empty) return false if (!view.moveVertically(selection, true).eq(selection)) return false focus_on_neighbor({ cell_delta: 1, line: 0, character: 0, }) return true } const CellPageUp = () => { focus_on_neighbor({ cell_delta: -1, line: 0, character: 0, }) return true } const CellPageDown = () => { focus_on_neighbor({ cell_delta: +1, line: 0, character: 0, }) return true } return keymap.of([ { key: "PageUp", run: CellPageUp }, { key: "PageDown", run: CellPageDown }, { key: "ArrowLeft", run: CellArrowLeft }, { key: "ArrowUp", run: CellArrowUp }, { key: "ArrowRight", run: CellArrowRight }, { key: "ArrowDown", run: CellArrowDown }, ]) } // Don't-accidentally-remove-cells-plugin // Because we need some extra info about the key, namely if it is on repeat or not, // we can't use a keymap (keymaps don't give us the event with `repeat` property), // so we use a custom keydown event handler. export let prevent_holding_a_key_from_doing_things_across_cells = EditorView.domEventHandlers({ keydown: (event, view) => { // TODO We could also require a re-press after a force focus, because // .... currently if you delete another cell, but keep holding down the backspace (or delete), // .... you'll still be deleting characters (because view.state.doc.length will be > 0) // Screw multicursor support on these things let selection = view.state.selection.main // Also only cursors and not selections if (!selection.empty) return false // Kinda the whole thing of this plugin, no? if (!event.repeat) return false if (event.key === "Backspace") { if (view.state.doc.length === 0) { // Only if this would be a cell-deleting backspace, we jump in return true } } if (event.key === "Delete") { if (view.state.doc.length === 0) { // Only if this would be a cell-deleting backspace, we jump in return true } } // Because of the "hacky" way this works, we need to check if autocompletion is open... // else we'll block the ability to press ArrowDown for autocomplete.... let autocompletion_open = autocomplete.completionStatus(view.state) === "active" // If we have a cursor instead of a multicharacter selection: if (event.key === "ArrowUp" && !autocompletion_open) { if (!view.moveVertically(view.state.selection.main, false).eq(selection)) return false return true } if (event.key === "ArrowDown" && !autocompletion_open) { if (!view.moveVertically(view.state.selection.main, true).eq(selection)) return false return true } if (event.key === "ArrowLeft" && event.repeat) { if (!view.moveByChar(selection, false).eq(selection)) return false return true } if (event.key === "ArrowRight") { if (!view.moveByChar(selection, true).eq(selection)) return false return true } }, }) /** * @param {object} options * @param {FocusOnNeighborFunction} options.focus_on_neighbor */ export let cell_movement_plugin = ({ focus_on_neighbor }) => cell_movement_keys({ focus_on_neighbor }) // This is "just" https://github.com/codemirror/comment/blob/da336b8660dedab23e06ced2b430f2ac56ef202d/src/comment.ts // (@codemirror/comment) // But with a change that helps mixed parser environments: // If the comment-style (`#`, `//`, `/* */`, etc) at the begin and end of the selection is the same, // we don't use the comment-style per line, but use the begin/end style for every line. import { EditorSelection, EditorState } from "../../imports/CodemirrorPlutoSetup.js" /// Comment or uncomment the current selection. Will use line comments /// if available, otherwise falling back to block comments. export const toggleComment = (target) => { let config = getConfig(target.state) return config.line ? toggleLineComment(target) : config.block ? toggleBlockComment(target) : false } function command(f, option) { return ({ state, dispatch }) => { if (state.readOnly) return false let tr = f(option, state.selection.ranges, state) if (!tr) return false dispatch(state.update(tr)) return true } } /// Comment or uncomment the current selection using line comments. /// The line comment syntax is taken from the /// [`commentTokens`](#comment.CommentTokens) [language /// data](#state.EditorState.languageDataAt). export const toggleLineComment = command(changeLineComment, 0 /* Toggle */) /// Comment the current selection using line comments. export const lineComment = command(changeLineComment, 1 /* Comment */) /// Uncomment the current selection using line comments. export const lineUncomment = command(changeLineComment, 2 /* Uncomment */) /// Comment or uncomment the current selection using block comments. /// The block comment syntax is taken from the /// [`commentTokens`](#comment.CommentTokens) [language /// data](#state.EditorState.languageDataAt). export const toggleBlockComment = command(changeBlockComment, 0 /* Toggle */) /// Comment the current selection using block comments. export const blockComment = command(changeBlockComment, 1 /* Comment */) /// Uncomment the current selection using block comments. export const blockUncomment = command(changeBlockComment, 2 /* Uncomment */) /// Default key bindings for this package. /// /// - Ctrl-/ (Cmd-/ on macOS): [`toggleComment`](#comment.toggleComment). /// - Shift-Alt-a: [`toggleBlockComment`](#comment.toggleBlockComment). export const commentKeymap = [ { key: "Mod-/", run: toggleComment }, { key: "Alt-A", run: toggleBlockComment }, ] function getConfig(state, pos = state.selection.main.head) { let data = state.languageDataAt("commentTokens", pos) return data.length ? data[0] : {} } const SearchMargin = 50 /// Determines if the given range is block-commented in the given /// state. function findBlockComment(state, { open, close }, from, to) { let textBefore = state.sliceDoc(from - SearchMargin, from) let textAfter = state.sliceDoc(to, to + SearchMargin) let spaceBefore = /\s*$/.exec(textBefore)[0].length, spaceAfter = /^\s*/.exec(textAfter)[0].length let beforeOff = textBefore.length - spaceBefore if (textBefore.slice(beforeOff - open.length, beforeOff) == open && textAfter.slice(spaceAfter, spaceAfter + close.length) == close) { return { open: { pos: from - spaceBefore, margin: spaceBefore && 1 }, close: { pos: to + spaceAfter, margin: spaceAfter && 1 } } } let startText, endText if (to - from <= 2 * SearchMargin) { startText = endText = state.sliceDoc(from, to) } else { startText = state.sliceDoc(from, from + SearchMargin) endText = state.sliceDoc(to - SearchMargin, to) } let startSpace = /^\s*/.exec(startText)[0].length, endSpace = /\s*$/.exec(endText)[0].length let endOff = endText.length - endSpace - close.length if (startText.slice(startSpace, startSpace + open.length) == open && endText.slice(endOff, endOff + close.length) == close) { return { open: { pos: from + startSpace + open.length, margin: /\s/.test(startText.charAt(startSpace + open.length)) ? 1 : 0 }, close: { pos: to - endSpace - close.length, margin: /\s/.test(endText.charAt(endOff - 1)) ? 1 : 0 }, } } return null } // Performs toggle, comment and uncomment of block comments in // languages that support them. function changeBlockComment(option, ranges, state) { let tokens = ranges.map((r) => getConfig(state, r.from).block) if (!tokens.every((c) => c)) return null let comments = ranges.map((r, i) => findBlockComment(state, tokens[i], r.from, r.to)) if (option != 2 /* Uncomment */ && !comments.every((c) => c)) { let index = 0 return state.changeByRange((range) => { let { open, close } = tokens[index++] if (comments[index]) return { range } let shift = open.length + 1 return { changes: [ { from: range.from, insert: open + " " }, { from: range.to, insert: " " + close }, ], range: EditorSelection.range(range.anchor + shift, range.head + shift), } }) } else if (option != 1 /* Comment */ && comments.some((c) => c)) { let changes = [] for (let i = 0, comment; i < comments.length; i++) if ((comment = comments[i])) { let token = tokens[i], { open, close } = comment changes.push( { from: open.pos - token.open.length, to: open.pos + open.margin }, { from: close.pos - close.margin, to: close.pos + token.close.length } ) } return { changes } } return null } // Performs toggle, comment and uncomment of line comments. function changeLineComment(option, ranges, state) { let lines = [] let prevLine = -1 for (let { from, to } of ranges) { // DRAL EDIIIIIITS // If the comment tokens at the begin and end are the same, // I assume we want these for the whole range! let comment_token_from = getConfig(state, from).line let comment_token_to = getConfig(state, to).line let overwrite_token = comment_token_from === comment_token_to ? comment_token_from : null let startI = lines.length, minIndent = 1e9 for (let pos = from; pos <= to; ) { let line = state.doc.lineAt(pos) if (line.from > prevLine && (from == to || to > line.from)) { prevLine = line.from // DRAL EDIIIIIIIITS let token = overwrite_token ?? getConfig(state, pos).line if (!token) continue let indent = /^\s*/.exec(line.text)[0].length let empty = indent == line.length let comment = line.text.slice(indent, indent + token.length) == token ? indent : -1 if (indent < line.text.length && indent < minIndent) minIndent = indent lines.push({ line, comment, token, indent, empty, single: false }) } pos = line.to + 1 } if (minIndent < 1e9) for (let i = startI; i < lines.length; i++) if (lines[i].indent < lines[i].line.text.length) lines[i].indent = minIndent if (lines.length == startI + 1) lines[startI].single = true } if (option != 2 /* Uncomment */ && lines.some((l) => l.comment < 0 && (!l.empty || l.single))) { let changes = [] for (let { line, token, indent, empty, single } of lines) if (single || !empty) changes.push({ from: line.from + indent, insert: token + " " }) let changeSet = state.changes(changes) return { changes: changeSet, selection: state.selection.map(changeSet, 1) } } else if (option != 1 /* Comment */ && lines.some((l) => l.comment >= 0)) { let changes = [] for (let { line, comment, token } of lines) if (comment >= 0) { let from = line.from + comment, to = from + token.length if (line.text[to - line.from] == " ") to++ changes.push({ from, to }) } return { changes } } return null } import { EditorView, syntaxTree, syntaxTreeAvailable, Text } from "../../imports/CodemirrorPlutoSetup.js" import { iterate_with_cursor } from "./lezer_template.js" /** * @param {Text} doc * @param {ReturnType<typeof syntaxTree>} tree */ let find_error_nodes = (doc, tree) => { iterate_with_cursor({ tree: tree, enter: (cursor) => { if (cursor.type.isError) { console.group(`Found error node in ${cursor.node.parent?.name}`) try { let text_before_cursor = doc.sliceString(cursor.from - 10, cursor.from) let text = doc.sliceString(cursor.from, cursor.to) let text_after_cursor = doc.sliceString(cursor.to, cursor.to + 10) if (text === "") { console.log(`${text_before_cursor}${text_after_cursor}`) console.log(`${" ".repeat(text_before_cursor.length)}^$${" ".repeat(text_after_cursor.length)}`) } else { console.log(`${text_before_cursor}${text}${text_after_cursor}`) console.log(`${" ".repeat(text_before_cursor.length)}${"^".repeat(text.length)}$${" ".repeat(text_after_cursor.length)}`) } } finally { console.groupEnd() } return false } }, leave: () => {}, }) } export const debug_syntax_plugin = EditorView.updateListener.of((update) => { if (update.docChanged || update.selectionSet || syntaxTree(update.state) !== syntaxTree(update.startState)) { if (syntaxTreeAvailable(update.state)) { let state = update.state console.group("Selection") try { console.groupCollapsed("Lezer tree") try { console.log(syntaxTree(state).toString()) } finally { console.groupEnd() } console.groupCollapsed("Document text") try { console.log(update.state.doc.sliceString(0, update.state.doc.length)) } finally { console.groupEnd() } console.group("Lezer errors") try { find_error_nodes(update.state.doc, syntaxTree(state)) } finally { console.groupEnd() } } finally { console.groupEnd() } } else { console.log(" Full syntax tree not available") } } }) import { Facet, ViewPlugin, Decoration, EditorView } from "../../imports/CodemirrorPlutoSetup.js" import { ctrl_or_cmd_name, has_ctrl_or_cmd_pressed } from "../../common/KeyboardShortcuts.js" import _ from "../../imports/lodash.js" import { ScopeStateField } from "./scopestate_statefield.js" /** * @param {any} state * @param {{ * scopestate: import("./scopestate_statefield.js").ScopeState, * global_definitions: { [key: string]: string } * }} context */ let get_variable_marks = (state, { scopestate, global_definitions }) => { return Decoration.set( filter_non_null( scopestate.usages.map(({ definition, usage, name }) => { if (definition == null) { // TODO variables_with_origin_cell should be notebook wide, not just in the current cell // .... Because now it will only show variables after it has run once if (global_definitions[name]) { return Decoration.mark({ tagName: "a", attributes: { "title": `${ctrl_or_cmd_name}-Click to jump to the definition of ${name}.`, "data-pluto-variable": name, "href": `#${name}`, }, }).range(usage.from, usage.to) } else { // This could be used to trigger @edit when clicked, to open // in whatever editor the person wants to use. // return Decoration.mark({ // tagName: "a", // attributes: { // "title": `${ctrl_or_cmd_name}-Click to jump to the definition of ${text}.`, // "data-external-variable": text, // "href": `#`, // }, // }).range(usage.from, usage.to) return null } } else { // Could be used to select the definition of a variable inside the current cell return Decoration.mark({ tagName: "a", attributes: { "title": `${ctrl_or_cmd_name}-Click to jump to the definition of ${name}.`, "data-cell-variable": name, "data-cell-variable-from": `${definition.from}`, "data-cell-variable-to": `${definition.to}`, "href": `#`, }, }).range(usage.from, usage.to) } }) ), true ) } /** * * @argument {Array<T?>} xs * @template T * @return {Array<T>} */ const filter_non_null = (xs) => /** @type {Array<T>} */ (xs.filter((x) => x != null)) /** * Key: variable name, value: cell id. * @type {Facet<{ [variable_name: string]: string }, { [variable_name: string]: string }>} */ export const GlobalDefinitionsFacet = Facet.define({ combine: (values) => values[0], compare: _.isEqual, }) export const go_to_definition_plugin = ViewPlugin.fromClass( class { /** * @param {EditorView} view */ constructor(view) { let global_definitions = view.state.facet(GlobalDefinitionsFacet) this.decorations = get_variable_marks(view.state, { scopestate: view.state.field(ScopeStateField), global_definitions, }) } update(update) { // My best take on getting this to update when GlobalDefinitionsFacet does  let global_definitions = update.state.facet(GlobalDefinitionsFacet) if (update.docChanged || update.viewportChanged || global_definitions !== update.startState.facet(GlobalDefinitionsFacet)) { this.decorations = get_variable_marks(update.state, { scopestate: update.state.field(ScopeStateField), global_definitions, }) } } }, { decorations: (v) => v.decorations, eventHandlers: { click: (event, view) => { if (event.target instanceof Element) { let pluto_variable = event.target.closest("[data-pluto-variable]") if (pluto_variable) { let variable = pluto_variable.getAttribute("data-pluto-variable") if (variable == null) { return false } if (!(has_ctrl_or_cmd_pressed(event) || view.state.readOnly)) { return false } event.preventDefault() let scrollto_selector = `[id='${encodeURI(variable)}']` document.querySelector(scrollto_selector)?.scrollIntoView({ behavior: "smooth", block: "center", }) // TODO Something fancy where it counts going to definition as a page in history, // .... so pressing/swiping back will go back to where you clicked on the definition. // window.history.replaceState({ scrollTop: document.documentElement.scrollTop }, null) // window.history.pushState({ scrollTo: scrollto_selector }, null) let global_definitions = view.state.facet(GlobalDefinitionsFacet) // TODO Something fancy where we actually emit the identifier we are looking for, // .... and the cell then selects exactly that definition (using lezer and cool stuff) if (global_definitions[variable]) { window.dispatchEvent( new CustomEvent("cell_focus", { detail: { cell_id: global_definitions[variable], line: 0, // 1-based to 0-based index definition_of: variable, }, }) ) return true } } let cell_variable = event.target.closest("[data-cell-variable]") if (cell_variable) { let variable_name = cell_variable.getAttribute("data-cell-variable") let variable_from = Number(cell_variable.getAttribute("data-cell-variable-from")) let variable_to = Number(cell_variable.getAttribute("data-cell-variable-to")) if (variable_name == null || variable_from == null || variable_to == null) { return false } if (!(has_ctrl_or_cmd_pressed(event) || view.state.readOnly)) { return false } event.preventDefault() view.dispatch({ scrollIntoView: true, selection: { anchor: variable_from, head: variable_to }, }) view.focus() return true } } }, }, } ) import { Decoration, ViewPlugin, EditorView, Facet, ViewUpdate } from "../../imports/CodemirrorPlutoSetup.js" const highlighted_line = Decoration.line({ attributes: { class: "cm-highlighted-line" }, }) const highlighted_range = Decoration.mark({ attributes: { class: "cm-highlighted-range" }, }) /** * @param {EditorView} view */ function create_line_decorations(view) { let line_number = view.state.facet(HighlightLineFacet) if (line_number == null || line_number == undefined || line_number < 0 || line_number > view.state.doc.lines) { return Decoration.set([]) } let line = view.state.doc.line(line_number) return Decoration.set([highlighted_line.range(line.from, line.from)]) } /** * @param {EditorView} view */ function create_range_decorations(view) { let range = view.state.facet(HighlightRangeFacet) if (range == null) { return Decoration.set([]) } let { from, to } = range if (from < 0 || from == to) { return Decoration.set([]) } // Check if range is within document bounds const docLength = view.state.doc.length if (from > docLength || to > docLength) { return Decoration.set([]) } return Decoration.set([highlighted_range.range(from, to)]) } /** * @type Facet<number?, number?> */ export const HighlightLineFacet = Facet.define({ combine: (values) => values[0], compare: (a, b) => a === b, }) /** * @type Facet<{from: number, to: number}?, {from: number, to: number}?> */ export const HighlightRangeFacet = Facet.define({ combine: (values) => values[0], compare: (a, b) => a === b, }) export const highlightLinePlugin = () => ViewPlugin.fromClass( class { updateDecos(view) { this.decorations = create_line_decorations(view) } /** * @param {EditorView} view */ constructor(view) { this.decorations = Decoration.set([]) this.updateDecos(view) } /** * @param {ViewUpdate} update */ update(update) { if (update.docChanged || update.state.facet(HighlightLineFacet) !== update.startState.facet(HighlightLineFacet)) { this.updateDecos(update.view) } } }, { decorations: (v) => v.decorations, } ) export const highlightRangePlugin = () => ViewPlugin.fromClass( class { updateDecos(view) { this.decorations = create_range_decorations(view) } /** * @param {EditorView} view */ constructor(view) { this.decorations = Decoration.set([]) this.updateDecos(view) } /** * @param {ViewUpdate} update */ update(update) { if (update.docChanged || update.state.facet(HighlightRangeFacet) !== update.startState.facet(HighlightRangeFacet)) { this.updateDecos(update.view) } } }, { decorations: (v) => v.decorations, } ) /** * Like Lezers `iterate`, but instead of `{ from, to, getNode() }` * this will give `enter()` and `leave()` the `cursor` (which can be effeciently matches with lezer template) * * @param {{ * tree: any, * enter: (cursor: import("../../imports/CodemirrorPlutoSetup.js").TreeCursor) => (void | boolean), * leave?: (cursor: import("../../imports/CodemirrorPlutoSetup.js").TreeCursor) => (void | boolean), * from?: number, * to?: number, * }} options */ export function iterate_with_cursor({ tree, enter, leave, from = 0, to = tree.length }) { let cursor = tree.cursor() while (true) { let mustLeave = false if (cursor.from <= to && cursor.to >= from && (cursor.type.isAnonymous || enter(cursor) !== false)) { if (cursor.firstChild()) continue if (!cursor.type.isAnonymous) mustLeave = true } while (true) { if (mustLeave && leave) leave(cursor) mustLeave = cursor.type.isAnonymous if (cursor.nextSibling()) break if (!cursor.parent()) return mustLeave = true } } } import _ from "../../imports/lodash.js" import { html, htmlLanguage, javascriptLanguage, markdownLanguage, markdown, parseMixed, PostgreSQL, pythonLanguage, sql, javascript, python, julia, parseCode, } from "../../imports/CodemirrorPlutoSetup.js" const htmlParser = htmlLanguage.parser // @ts-ignore const mdParserExt = markdownLanguage.parser.configure(parseCode({ htmlParser })) const postgresParser = PostgreSQL.language.parser const sqlLang = sql({ dialect: PostgreSQL }) const pythonParser = pythonLanguage.parser /** * Markdown tags list; we create both `md""` and `@md("")` instances. */ const MD_TAGS = ["md", "mermaid", "cm", "markdown", "mdx", "mdl", "markdownliteral"].flatMap((x) => [x, `@${x}`]) /** * Julia strings are do not represent the exact code that is going to run * for example the following julia string: * * ```julia * """ * const test = "five" * const five = \${test} * """ * ``` * * is going to be executed as javascript, after escaping the \$ to $ * * ```javascript * """ * const test = "five" * const five = ${test} * """ * ``` * * The overlays already remove the string interpolation parts of the julia string. * This hack additionally removes the `\` from the overlay for common interpolations, so the underlaying parser * will get the javascript version of the string, and not the julia version of the string (which is invalid js) * */ const overlayHack = (overlay, input) => { return overlay.flatMap(({ from, to }) => { const text = input.read(from, to) // const newlines = [...text.matchAll(/\\n/g)].map(({ index }) => ({ from: from + index, to: from + index + 2 })) // const escdollars = [...text.matchAll(/\\\$/g)].map(({ index }) => ({ from: from + index, to: from + index + 1 })) // const escjuliadollars = [...text.matchAll(/[^\\]\$/g)].map(({ index }) => ({ from: from + index, to: from + index + 2 })) // const extraOverlaysNegatives = _.sortBy([...newlines, ...escdollars, ...escjuliadollars], "from") // For simplicity I removed the newlines stuff and just removed the \$ from the overlays // Curious to see edge cases that this misses - DRAL const result = [] let last_content_start = from for (let { index: relative_escape_start } of text.matchAll(/\\\$/g)) { let next_escape_start = from + relative_escape_start if (last_content_start !== next_escape_start) { result.push({ from: last_content_start, to: next_escape_start }) } last_content_start = next_escape_start + 1 } if (last_content_start !== to) { result.push({ from: last_content_start, to: to }) } return result }) } export const STRING_NODE_NAMES = new Set(["StringLiteral", "CommandLiteral", "NsStringLiteral", "NsCommandLiteral"]) const juliaWrapper = parseMixed((cursor, input) => { if (cursor.name !== "NsStringLiteral" && cursor.name !== "StringLiteral") { return null } const node = cursor.node const first_string_delim = node.getChild('"""') ?? node.getChild('"') if (first_string_delim == null) return null const last_string_delim = node.lastChild if (last_string_delim == null) return null // const offset = first_string_delim.to - first_string_delim.from // console.log({ first_string_delim, last_string_delim, offset }) const string_content_from = first_string_delim.to const string_content_to = Math.min(last_string_delim.from, input.length) if (string_content_from >= string_content_to) { return null } let tagNode if (cursor.name === "NsStringLiteral") { tagNode = node.firstChild // if (tagNode) tag = input.read(tagNode.from, tagNode.to) } else { // must be a string, let's search for the parent `@htl`. const start = node const p1 = start.parent if (p1 != null && p1.name === "Arguments") { const p2 = p1.parent if (p2 != null && p2.name === "MacrocallExpression") { tagNode = p2.getChild("MacroIdentifier") } } } if (tagNode == null) return null const is_macro = tagNode.name === "MacroIdentifier" const tag = input.read(tagNode.from, tagNode.to) let parser = null if (tag === "@htl" || tag === "html") { parser = htmlParser } else if (MD_TAGS.includes(tag)) { parser = mdParserExt } else if (tag === "@javascript" || tag === "@js" || tag === "js" || tag === "javascript") { parser = javascriptLanguage.parser } else if (tag === "py" || tag === "pyr" || tag === "python" || tag === "@python") { parser = pythonParser } else if (tag === "sql") { parser = postgresParser } else { return null } let overlay = [] if (node.firstChild != null) { let last_content_start = string_content_from let child = node.firstChild.cursor() do { if (last_content_start < child.from) { overlay.push({ from: last_content_start, to: child.from }) } last_content_start = child.to } while (child.nextSibling()) if (last_content_start < string_content_to) { overlay.push({ from: last_content_start, to: string_content_to }) } } else { overlay = [{ from: string_content_from, to: string_content_to }] } // If it is a macro, thus supports interpolations (prefixed strings only have faux-interpolations) but not raw strings (`\n` will be a newline, for the character `\n` you need to do `\\n`) // we need to remove `\$` (which should just be `$` in the javascript) if (is_macro) { overlay = overlayHack(overlay, input) } // No overlays for markdown yet // (They sometimes work, but sometimes also randomly crash when adding an interpolation // I guess this has something to do with the fact that markdown isn't parsed with lezer, // but has some custom made thing that emulates lezer.) if ([...MD_TAGS].includes(tag)) { return { parser, overlay: [{ from: string_content_from, to: string_content_to }] } } return { parser, overlay } }) const julia_mixed = (config) => { const julia_simple = julia(config) // @ts-ignore julia_simple.language.parser = julia_simple.language.parser.configure({ wrap: juliaWrapper }) return julia_simple } export { julia_mixed, sqlLang, pythonLanguage, javascript, htmlLanguage, javascriptLanguage, python, markdown, html } import { EditorView, EditorSelection, selectNextOccurrence, syntaxTree } from "../../imports/CodemirrorPlutoSetup.js" let array_at = (array, pos) => { return array.slice(pos, pos + 1)[0] } export let mod_d_command = { key: "Mod-d", /** @param {EditorView} view */ run: ({ state, dispatch }) => { if (state.selection.main.empty) { let nodes_that_i_like = ["Identifier", "FieldName"] // Expand to closest Identifier let cursor_left = syntaxTree(state).cursorAt(state.selection.main.from, -1) let cursor_right = syntaxTree(state).cursorAt(state.selection.main.from, 1) for (let node_i_like of nodes_that_i_like) { let matching_node = cursor_left.name === node_i_like ? cursor_left : cursor_right.name === node_i_like ? cursor_right : null if (matching_node) { dispatch({ selection: { anchor: matching_node.from, head: matching_node.to }, }) return true } } // If there is no cool syntax thing (say we are in a string), then just select the word. let line = state.doc.lineAt(state.selection.main.from) let position_relative_to_line = state.selection.main.from - line.from let before_cursor = line.text.slice(0, position_relative_to_line) let after_cursor = line.text.slice(position_relative_to_line) let word_before_cursor = before_cursor.match(/(\w+)$/)?.[0] ?? "" let word_after_cursor = after_cursor.match(/^(\w+)/)?.[0] ?? "" dispatch({ selection: { anchor: state.selection.main.from - word_before_cursor.length, head: state.selection.main.from + word_after_cursor.length }, }) } else { selectNextOccurrence({ state, dispatch }) } return false }, /** @param {EditorView} view */ shift: ({ state, dispatch }) => { if (state.selection.ranges.length === 1) return false // So funny thing, the index "before" (might wrap around) the mainIndex is the one you just selected // @ts-ignore let just_selected = state.selection.ranges.at(state.selection.mainIndex - 1) let new_ranges = state.selection.ranges.filter((x) => x !== just_selected) let new_main_index = new_ranges.indexOf(state.selection.main) let previous_selected = array_at(new_ranges, state.selection.mainIndex - 1) dispatch({ selection: EditorSelection.create(new_ranges, new_main_index), effects: previous_selected == null ? [] : EditorView.scrollIntoView(previous_selected.from), }) return true }, preventDefault: true, } import _ from "../../imports/lodash.js" import { EditorView, syntaxTree, Decoration, ViewUpdate, ViewPlugin, Facet, EditorState } from "../../imports/CodemirrorPlutoSetup.js" import { PkgStatusMark, PkgActivateMark } from "../PkgStatusMark.js" import { html } from "../../imports/Preact.js" import { ReactWidget } from "./ReactWidget.js" import { iterate_with_cursor } from "./lezer_template.js" /** * @typedef PkgstatusmarkWidgetProps * @type {{ nbpkg: import("../Editor.js").NotebookPkgData, pluto_actions: any, notebook_id: string }} */ // This list appears multiple times in our codebase. Be sure to match edits everywhere. export const pkg_disablers = [ "Pkg.activate", "Pkg.API.activate", "Pkg.develop", "Pkg.API.develop", "Pkg.add", "Pkg.API.add", "TestEnv.activate", // https://juliadynamics.github.io/DrWatson.jl/dev/project/#DrWatson.quickactivate "quickactivate", "@quickactivate", ] /** * @param {object} a * @param {EditorState} a.state * @param {Number} a.from * @param {Number} a.to */ function find_import_statements({ state, from, to }) { const doc = state.doc const tree = syntaxTree(state) let things_to_return = [] let currently_using_or_import = "import" let currently_selected_import = false iterate_with_cursor({ tree, from, to, enter: (node) => { let go_to_parent_afterwards = null if (node.name === "QuoteExpression" || node.name === "FunctionDefinition") return false if (node.name === "import") currently_using_or_import = "import" if (node.name === "using") currently_using_or_import = "using" // console.group("exploring", node.name, doc.sliceString(node.from, node.to), node) if (node.name === "CallExpression" || node.name === "MacrocallExpression") { let callee = node.node.firstChild if (callee) { let callee_name = doc.sliceString(callee.from, callee.to) if (pkg_disablers.includes(callee_name)) { things_to_return.push({ type: "package_disabler", name: callee_name, from: node.from, to: node.to, }) } } return false } if (node.name === "ImportStatement") { currently_selected_import = false } if (node.name === "SelectedImport") { currently_selected_import = true node.firstChild() go_to_parent_afterwards = true } if (node.name === "ImportPath") { const package_name = doc.sliceString(node.from, node.to).split(".")[0] if (package_name === "") return false const item = { type: "package", name: package_name, from: node.from, to: node.to, } things_to_return.push(item) // This is just for show... might delete it later if (currently_using_or_import === "using" && !currently_selected_import) things_to_return.push({ ...item, type: "implicit_using" }) } if (go_to_parent_afterwards) { node.parent() return false } }, }) return things_to_return } /** * @param {EditorView} view * @param {PkgstatusmarkWidgetProps} props */ function pkg_decorations(view, { pluto_actions, notebook_id, nbpkg }) { let seen_packages = new Set() let widgets = view.visibleRanges .flatMap(({ from, to }) => { let things_to_mark = find_import_statements({ state: view.state, from: from, to: to, }) return things_to_mark.map((thing) => { if (thing.type === "package") { let { name: package_name } = thing if (package_name !== "Base" && package_name !== "Core" && !seen_packages.has(package_name)) { seen_packages.add(package_name) let deco = Decoration.widget({ widget: new ReactWidget(html` <${PkgStatusMark} key=${package_name} package_name=${package_name} pluto_actions=${pluto_actions} notebook_id=${notebook_id} nbpkg=${nbpkg} /> `), side: 1, }) return deco.range(thing.to) } } else if (thing.type === "package_disabler") { let deco = Decoration.widget({ widget: new ReactWidget(html` <${PkgActivateMark} package_name=${thing.name} /> `), side: 1, }) return deco.range(thing.to) } else if (thing.type === "implicit_using") { if (thing.name === "HypertextLiteral") { let deco = Decoration.widget({ widget: new ReactWidget(html`<span style=${{ position: "relative" }}> <div style=${{ position: `absolute`, display: `inline`, left: 0, whiteSpace: `nowrap`, opacity: 0.3, pointerEvents: `none`, }} > : @htl, @htl_str </div> </span>`), side: 1, }) return deco.range(thing.to) } } }) }) .filter((x) => x != null) return Decoration.set(widgets, true) } /** * @type {Facet<import("../Editor.js").NotebookPkgData?, import("../Editor.js").NotebookPkgData?>} */ export const NotebookpackagesFacet = Facet.define({ combine: (values) => values[0], compare: _.isEqual, }) export const pkgBubblePlugin = ({ pluto_actions, notebook_id_ref }) => ViewPlugin.fromClass( class { update_decos(view) { const ds = pkg_decorations(view, { pluto_actions, notebook_id: notebook_id_ref.current, nbpkg: view.state.facet(NotebookpackagesFacet) }) this.decorations = ds } /** * @param {EditorView} view */ constructor(view) { this.update_decos(view) } /** * @param {ViewUpdate} update */ update(update) { if ( update.docChanged || update.viewportChanged || update.state.facet(NotebookpackagesFacet) !== update.startState.facet(NotebookpackagesFacet) ) { this.update_decos(update.view) return } } }, { // @ts-ignore decorations: (v) => v.decorations, } ) import _ from "../../imports/lodash.js" import { EditorView, EditorState, keymap, autocomplete, syntaxTree, StateField, StateEffect, Transaction } from "../../imports/CodemirrorPlutoSetup.js" import { get_selected_doc_from_state } from "./LiveDocsFromCursor.js" import { cl } from "../../common/ClassTable.js" import { ScopeStateField } from "./scopestate_statefield.js" import { open_bottom_right_panel } from "../BottomRightPanel.js" import { ENABLE_CM_AUTOCOMPLETE_ON_TYPE } from "../CellInput.js" import { GlobalDefinitionsFacet } from "./go_to_definition_plugin.js" import { STRING_NODE_NAMES } from "./mixedParsers.js" let { autocompletion, completionKeymap, completionStatus, acceptCompletion, selectedCompletion } = autocomplete // These should be imported from @codemirror/autocomplete, but they are not exported. const completionState = autocompletion()[1] /** @param {EditorView} cm */ const tab_completion_command = (cm) => { // This will return true if the autocomplete select popup is open // To test the exception sink, uncomment these lines: // if (Math.random() > 0.7) { // throw "LETS CRASH THIS" // } if (acceptCompletion(cm)) { return true } if (cm.state.readOnly) { return false } let selection = cm.state.selection.main if (!selection.empty) return false let last_char = cm.state.sliceDoc(selection.from - 1, selection.from) let last_line = cm.state.sliceDoc(cm.state.doc.lineAt(selection.from).from, selection.from) // Some exceptions for when to trigger tab autocomplete if ("\t \n=".includes(last_char)) return false // ?([1,2], 3)<TAB> should trigger autocomplete if (last_char === ")" && !last_line.includes("?")) return false return autocomplete.startCompletion(cm) } // Remove this if we find that people actually need the `?` in their queries, but I very much doubt it. // (Also because the ternary operator does require a space before the ?, thanks Julia!) let open_docs_if_autocomplete_is_open_command = (cm) => { if (autocomplete.completionStatus(cm.state) != null) { open_bottom_right_panel("docs") return true } return false } const pluto_autocomplete_keymap = [ { key: "Tab", run: tab_completion_command }, { key: "?", run: open_docs_if_autocomplete_is_open_command }, ] /** * @param {(query: string) => void} on_update_doc_query */ let update_docs_from_autocomplete_selection = (on_update_doc_query) => { let last_query = null return EditorView.updateListener.of((update) => { // But we can use `selectedCompletion` to better check if the autocomplete is open // (for some reason `autocompletion_state?.open != null` isn't enough anymore?) // Sadly we still need `update.state.field(completionState, false)` as well because we can't // apply the result from `selectedCompletion()` yet (has no .from and .to, for example) if (selectedCompletion(update.state) == null) return let autocompletion_state = update.state.field(completionState, false) let open_autocomplete = autocompletion_state?.open if (open_autocomplete == null) return let selected_option = open_autocomplete.options[open_autocomplete.selected] let text_to_apply = selected_option.completion.apply ?? selected_option.completion.label if (typeof text_to_apply !== "string") return // Option.source is now the source, we find to find the corresponding ActiveResult (internal type) const active_result = update.view.state.field(completionState).active.find((a) => a.source == selected_option.source) if (active_result?.hasResult?.() !== true) return // not an ActiveResult instance const from = active_result.from, to = Math.min(active_result.to, update.state.doc.length) // Apply completion to state, which will yield us a `Transaction`. // The nice thing about this is that we can use the resulting state from the transaction, // without updating the actual state of the editor. // NOTE This could bite someone who isn't familiar with this, but there isn't an easy way to fix it without a lot of console spam: // .... THIS UPDATE WILL DO CONSOLE.LOG'S LIKE ANY UPDATE WOULD DO // .... Which means you sometimes get double logs from codemirror extensions... // .... Very disorienting  let result_transaction = update.state.update({ changes: { from, to, insert: text_to_apply, }, }) // So we can use `get_selected_doc_from_state` on our virtual state let docs_string = get_selected_doc_from_state(result_transaction.state) if (docs_string != null) { if (last_query != docs_string) { last_query = docs_string on_update_doc_query(docs_string) } } }) } /** Are we matching something like `\lambd...`? */ const match_latex_symbol_complete = (/** @type {autocomplete.CompletionContext} */ ctx) => ctx.matchBefore(/\\[\d\w\!\(\)\+\-\/\:\=\^\_]*/) /** Are we matching something like `Base.:writing_a_symbo...`? */ const match_operator_symbol_complete = (/** @type {autocomplete.CompletionContext} */ ctx) => ctx.matchBefore(/\.\:[^\s"'`()\[\]\{\}\.\,=]*/) /** Are we matching inside a string at given pos? * @param {EditorState} state * @param {number} pos * @returns {boolean} **/ function match_string_complete(state, pos) { const tree = syntaxTree(state) const node = tree.resolve(pos) if (node == null || !STRING_NODE_NAMES.has(node.name)) { return false } return true } let override_text_to_apply_in_field_expression = (text) => { return !/^[@\p{L}\p{Sc}\d_][\p{L}\p{Nl}\p{Sc}\d_!]*"?$/u.test(text) ? (text === ":" ? `:(${text})` : `:${text}`) : null } const section_regular = { name: "Suggestions", header: () => document.createElement("div"), rank: 0, } const section_operators = { name: "Operators", rank: 1, } const field_rank_heuristic = (text, is_exported) => is_exported * 3 + (/^\p{Ll}/u.test(text) ? 2 : /^\p{Lu}/u.test(text) ? 1 : 0) const julia_commit_characters = (/** @type {autocomplete.CompletionContext} */ ctx) => { return ["."] } const validFor = (/** @type {string} */ text) => { let expected_char = /[\p{L}\p{Nl}\p{Sc}\d_!]*$/u.test(text) return expected_char && !endswith_keyword_regex.test(text) } const not_explicit_and_too_boring = (/** @type {autocomplete.CompletionContext} */ ctx, allow_strings = false) => { if (ctx.explicit) return false if (ctx.matchBefore(/[ =)+-/,*:]$/)) return true if (ctx.tokenBefore(["IntegerLiteral", "FloatLiteral", "LineComment", "BlockComment", "Symbol"]) != null) return true if (!allow_strings) { if (ctx.tokenBefore([...STRING_NODE_NAMES]) != null) { // don't complete inside a string, unless the user is doing string interpolation. if (ctx.matchBefore(/\$[(\p{L}\p{Nl}\p{Sc}\d_!]$/u) == null) { return true } } } return false } /** Use the completion results from the Julia server to create CM completion objects. */ const julia_code_completions_to_cm = (/** @type {PlutoRequestAutocomplete} */ request_autocomplete) => /** @returns {Promise<autocomplete.CompletionResult?>} */ async (/** @type {autocomplete.CompletionContext} */ ctx) => { if (match_latex_symbol_complete(ctx)) return null if (!ctx.explicit && writing_variable_name_or_keyword(ctx)) return null if (not_explicit_and_too_boring(ctx)) return null let to_complete_full = /** @type {String} */ (ctx.state.sliceDoc(0, ctx.pos)) let to_complete = to_complete_full // Another rough hack... If it detects a `.:`, we want to cut out the `:` so we get all results from julia, // but then codemirror will put the `:` back in filtering let is_symbol_completion = match_operator_symbol_complete(ctx) if (is_symbol_completion) { to_complete = to_complete.slice(0, is_symbol_completion.from + 1) + to_complete.slice(is_symbol_completion.from + 2) } else { // Generalized logic: send up to and including the last non-variable character // (not matching /[\p{L}\p{Nl}\p{Sc}\d_!]/u) const match = to_complete.match(/[\p{L}\p{Nl}\p{Sc}\d_!]*$/u) if (match && match[0].length < to_complete.length) { to_complete = to_complete.slice(0, to_complete.length - match[0].length) } else { to_complete = "" } } const globals = ctx.state.facet(GlobalDefinitionsFacet) const is_already_a_global = (text) => { const val = text != null && Object.keys(globals).includes(text) // console.log("is_already_a_global", text, val) return val } let found = await request_autocomplete({ query: to_complete, query_full: to_complete_full }) // console.log("received autocomplete results", { query: to_complete, query_full: to_complete_full }, found) if (!found) return null let { start, stop, results, too_long } = found if (is_symbol_completion) { // If this is a symbol completion thing, we need to add the `:` back in by moving the end a bit furher stop = stop + 1 } const to_complete_onto = to_complete.slice(0, start) const is_field_expression = to_complete_onto.endsWith(".") // skip autocomplete's filter if we are completing a ~ path (userexpand) const skip_filter = ctx.matchBefore(/\~[^\s\"]*/) != null const result = { from: start, to: ctx.pos, // This tells codemirror to not query this function again as long as the string matches the regex. // If the number of results was too long, then typing more should re-query (to be able to find results that were cut off) validFor: too_long ? undefined : validFor, commitCharacters: julia_commit_characters(ctx), filter: !skip_filter, options: [ ...results .filter( ([text, _1, _2, is_from_notebook, completion_type]) => (ctx.explicit || completion_type != "path") && (ctx.explicit || completion_type != "method") && !is_already_a_global(text) ) .map(([text, value_type, is_exported, is_from_notebook, completion_type, _ignored], i) => { // (quick) fix for identifiers that need to be escaped // Ideally this is done with Meta.isoperator on the julia side let text_to_apply = completion_type === "method" ? to_complete : is_field_expression ? override_text_to_apply_in_field_expression(text) ?? text : text value_type = value_type === "Function" && text.startsWith("@") ? "Macro" : value_type return { label: text, apply: text_to_apply, type: cl({ c_notexported: !is_exported, [`c_${value_type}`]: true, [`completion_${completion_type}`]: true, c_from_notebook: is_from_notebook, }) ?? undefined, section: section_regular, // detail: completion_type, boost: completion_type === "keyword_argument" ? 7 : is_field_expression ? field_rank_heuristic(text_to_apply, is_exported) : undefined, // boost: 50 - i / results.length, commitCharacters: completion_type === "keyword_argument" || value_type === "Macro" ? [] : undefined, } }), // This is a small thing that I really want: // You want to see what fancy symbols a module has? Pluto will show these at the very end of the list, // for Base there is no way you're going to find them! With this you can type `.:` and see all the fancy symbols. // TODO This whole block shouldn't use `override_text_to_apply_in_field_expression` but the same // `Meta.isoperator` thing mentioned above ...results .filter(([text]) => is_field_expression && override_text_to_apply_in_field_expression(text) != null) .map(([text, value_type, is_exported], i) => { let text_to_apply = override_text_to_apply_in_field_expression(text) ?? "" return { label: text_to_apply, apply: text_to_apply, type: (is_exported ? "" : "c_notexported ") + (value_type == null ? "" : "c_" + value_type), // boost: -99 - i / results.length, // Display below all normal results section: section_operators, // Non-standard is_not_exported: !is_exported, } }), ], } // console.log("cm completion result", result) return result } const complete_anyword = async (/** @type {autocomplete.CompletionContext} */ ctx) => { if (match_latex_symbol_complete(ctx)) return null if (!ctx.explicit && writing_variable_name_or_keyword(ctx)) return null if (not_explicit_and_too_boring(ctx)) return null const results_from_cm = await autocomplete.completeAnyWord(ctx) if (results_from_cm === null) return null return { from: results_from_cm.from, commitCharacters: julia_commit_characters(ctx), options: results_from_cm.options.map(({ label }, i) => ({ // See https://github.com/codemirror/codemirror.next/issues/788 about `type: null` label, apply: label, type: undefined, section: section_regular, // boost: 0 - i, })), } } const from_notebook_type = "c_from_notebook completion_module c_Any" /** * Are we currently writing a variable name? In that case we don't want autocomplete. * * E.g. `const hel<TAB>` should not autocomplete. */ const writing_variable_name_or_keyword = (/** @type {autocomplete.CompletionContext} */ ctx) => { let just_finished_a_keyword = ctx.matchBefore(endswith_keyword_regex) if (just_finished_a_keyword) return true // Regex explaination: // 1. a keyword that could be followed by a variable name like `catch ex` where `ex` is a variable name that should not get completed // 2. a space // 3. a sequence of either: // 3a. a variable name character `@\p{L}\p{Nl}\p{Sc}\d_!`. Also allowed is a bracket or a comma, this is to handle multiple vars `const (a,b)`. // 3b. a `, ` comma-space, to treat `const a, b` but not `for a in // 4. a `$` to match the end of the line let after_keyword = ctx.matchBefore(/(catch|local|module|abstract type|struct|macro|const|for|function|let|do) ([@\p{L}\p{Nl}\p{Sc}\d_!,\(\)]|, )*$/u) if (after_keyword) return true let inside_do_argument_expression = ctx.matchBefore(/do [\(\), \p{L}\p{Nl}\p{Sc}\d_!]*$/u) if (inside_do_argument_expression) return true let node = syntaxTree(ctx.state).resolve(ctx.pos, -1) let npn = node?.parent?.name if (node?.name === "Identifier" && npn === "StructDefinition") return true if (node?.name === "Identifier" && npn === "KeywordArguments") return true let node2 = npn === "OpenTuple" || npn === "TupleExpression" ? node?.parent : node let n2pn = node2?.parent?.name let inside_assigment_lhs = node?.name === "Identifier" && (n2pn === "Assignment" || n2pn === "KwArg") && node2?.nextSibling != null if (inside_assigment_lhs) return true return false } const global_variables_completion = (/** @type {() => { [uuid: String]: String[]}} */ request_unsubmitted_global_definitions, cell_id) => /** @returns {Promise<autocomplete.CompletionResult?>} */ async (/** @type {autocomplete.CompletionContext} */ ctx) => { if (ctx.matchBefore(/[(\p{L}\p{Nl}\p{Sc}\d_!]$/u) == null) return null if (match_latex_symbol_complete(ctx)) return null if (!ctx.explicit && writing_variable_name_or_keyword(ctx)) return null if (not_explicit_and_too_boring(ctx)) return null // see `is_wc_cat_id_start` in Julia's source for a complete list const there_is_a_dot_before = ctx.matchBefore(/\.[\p{L}\p{Nl}\p{Sc}\d_!]*$/u) if (there_is_a_dot_before) return null const globals = ctx.state.facet(GlobalDefinitionsFacet) const local_globals = request_unsubmitted_global_definitions() const possibles = _.union( // Globals that are not redefined locally Object.entries(globals) .filter(([_, cell_id]) => local_globals[cell_id] == null) .map(([name]) => name), // Globals that are redefined locally in other cells ...Object.values(_.omit(local_globals, cell_id)) ) return await make_it_julian( autocomplete.completeFromList( possibles.map((label) => { return { label, apply: label, type: from_notebook_type, section: section_regular, boost: 1, } }) ) )(ctx) } /** @returns {autocomplete.CompletionSource} */ const make_it_julian = (/** @type {autocomplete.CompletionSource} */ source) => (/** @type {autocomplete.CompletionContext} */ ctx) => { const c = source(ctx) return c == null ? null : { ...c, validFor, commitCharacters: julia_commit_characters(ctx), } } // Get this list with // import REPL; REPL.REPLCompletions.sorted_keywords  REPL.REPLCompletions.sorted_keyvals |> repr |> clipboard const sorted_keywords = [ "abstract type", "baremodule", "begin", "break", "catch", "ccall", "const", "continue", "do", "else", "elseif", "end", "export", "finally", "for", "function", "global", "if", "import", "let", "local", "macro", "module", "mutable struct", "primitive type", "quote", "return", "struct", "try", "using", "while", "false", "true", ] // Get this list with // join(map(d -> split(d, " ")[end], REPL.REPLCompletions.sorted_keywords  REPL.REPLCompletions.sorted_keyvals) |> unique |> sort, "|") const endswith_keyword_regex = /^(.*\s)?(baremodule|begin|break|catch|ccall|const|continue|do|else|elseif|end|export|false|finally|for|function|global|if|import|let|local|macro|module|quote|return|struct|true|try|type|using|while)$/ const keyword_completions = sorted_keywords.map((label) => ({ label, apply: label, type: "completion_keyword", section: section_regular, })) const keyword_completions_generator = make_it_julian(autocomplete.completeFromList(keyword_completions)) const complete_keyword = async (/** @type {autocomplete.CompletionContext} */ ctx) => { if (ctx.matchBefore(/[a-z]$/) == null) return null if (match_latex_symbol_complete(ctx)) return null if (!ctx.explicit && writing_variable_name_or_keyword(ctx)) return null if (not_explicit_and_too_boring(ctx)) return null return await keyword_completions_generator(ctx) } const complete_package_name = (/** @type {() => Promise<string[]>} */ request_packages) => { let found = null const get_packages = async () => { if (found == null) { const data = await request_packages().catch((e) => { console.warn("Failed to fetch packages", e) return null }) if (data == null) return null found = data.map((name, i) => ({ label: name, apply: name, type: "c_package", })) } return found } return async (/** @type {autocomplete.CompletionContext} */ ctx) => { // space before the package name to only find remote packages if (ctx.matchBefore(/[ ,][a-zA-Z0-9]+$/) == null) return null if (ctx.tokenBefore(["Identifier"]) == null) return null const tree = syntaxTree(ctx.state) const node = tree.resolve(ctx.pos, -1) if (!(node.matchContext(["UsingStatement", "ImportPath"]) || node.matchContext(["ImportStatement", "ImportPath"]))) return null const packages = await get_packages() return await make_it_julian(autocomplete.completeFromList(packages))(ctx) } } const local_variables_completion = async (/** @type {autocomplete.CompletionContext} */ ctx) => { let scopestate = ctx.state.field(ScopeStateField) let identifier = ctx.tokenBefore(["Identifier"]) if (identifier == null) return null let { from, to } = identifier const possibles = scopestate.locals .filter(({ validity }) => from > validity.from && to <= validity.to) .map(({ name }, i) => ({ // See https://github.com/codemirror/codemirror.next/issues/788 about `type: null` label: name, apply: name, type: undefined, boost: 99 - i, })) return await make_it_julian(autocomplete.completeFromList(possibles))(ctx) } const special_latex_examples = ["\\sqrt", "\\pi", "\\approx"] const special_emoji_examples = ["", "", "", "", "", "", "", "", ""] /** Apply completion to detail when completion is equal to detail * https://codemirror.net/docs/ref/#autocomplete.Completion.apply * Example: * For latex completions, if inside string only complete to label unless label is already fully typed. * \lamb<tab> ->  * "\lamb<tab>" -> "\lambda" * "\lambda<tab>" -> "" * For emojis, we always complete to detail: * \:cat:<tab> ->  * "\:ca" ->  * @param {EditorView} view * @param {autocomplete.Completion} completion * @param {number} from * @param {number} to * */ const apply_completion = (view, completion, from, to) => { const currentComp = view.state.sliceDoc(from, to) let insert = completion.detail ?? completion.label const is_emoji = completion.label.startsWith("\\:") if (!is_emoji && currentComp !== completion.label) { const is_inside_string = match_string_complete(view.state, to) if (is_inside_string) { insert = completion.label } } view.dispatch({ ...autocomplete.insertCompletionText(view.state, insert, from, to), annotations: autocomplete.pickedCompletion.of(completion), }) } const special_symbols_completion = (/** @type {() => Promise<SpecialSymbols?>} */ request_special_symbols) => { let found = null const get_special_symbols = async () => { if (found == null) { const data = await request_special_symbols().catch((e) => { console.warn("Failed to fetch special symbols", e) return null }) if (data != null) { const { latex, emoji } = data found = [emoji, latex].flatMap((map) => Object.entries(map).map(([label, value]) => { return { label, apply: apply_completion, detail: value ?? undefined, type: "c_special_symbol", boost: label === "\\in" ? 3 : special_latex_examples.includes(label) ? 2 : special_emoji_examples.includes(value) ? 1 : 0, } }) ) } } return found } return async (/** @type {autocomplete.CompletionContext} */ ctx) => { if (!match_latex_symbol_complete(ctx)) return null if (!ctx.explicit && writing_variable_name_or_keyword(ctx)) return null if (not_explicit_and_too_boring(ctx, true)) return null const result = await get_special_symbols() return await autocomplete.completeFromList(result ?? [])(ctx) } } /** * * @typedef PlutoAutocompleteResult * @type {[ * text: string, * value_type: string, * is_exported: boolean, * is_from_notebook: boolean, * completion_type: string, * special_symbol: string | null, * ]} * * @typedef PlutoAutocompleteResults * @type {{ start: number, stop: number, results: Array<PlutoAutocompleteResult>, too_long: boolean }} * * @typedef PlutoRequestAutocomplete * @type {(options: { query: string, query_full?: string }) => Promise<PlutoAutocompleteResults?>} * * @typedef SpecialSymbols * @type {{emoji: Record<string, string>, latex: Record<string, string>}} */ /** * @param {object} props * @param {PlutoRequestAutocomplete} props.request_autocomplete * @param {() => Promise<SpecialSymbols?>} props.request_special_symbols * @param {() => Promise<string[]>} props.request_packages * @param {(query: string) => void} props.on_update_doc_query * @param {() => { [uuid: string] : String[]}} props.request_unsubmitted_global_definitions * @param {string} props.cell_id */ export let pluto_autocomplete = ({ request_autocomplete, request_special_symbols, request_packages, on_update_doc_query, request_unsubmitted_global_definitions, cell_id, }) => { let last_query = null let last_result = null /** * To make stuff a bit easier, we let all the generators fetch all the time and run their logic, but just do one request. * Previously I had checks to make sure when `unicode_hint_generator` matches it wouldn't fetch in `julia_code_completions_to_cm`.. * but that became cumbersome with `expanduser` autocomplete.. also because THERE MIGHT be a case where * `~/` actually needs a different completion? Idk, I decided to put this "memoize last" thing here deal with it. * @type {PlutoRequestAutocomplete} **/ let memoize_last_request_autocomplete = async (options) => { if (_.isEqual(options, last_query)) { let result = await last_result if (result != null) return result } last_query = options last_result = request_autocomplete(options) return await last_result } return [ autocompletion({ activateOnTyping: ENABLE_CM_AUTOCOMPLETE_ON_TYPE, override: [ global_variables_completion(request_unsubmitted_global_definitions, cell_id), special_symbols_completion(request_special_symbols), julia_code_completions_to_cm(memoize_last_request_autocomplete), complete_keyword, complete_package_name(request_packages), // complete_anyword, // TODO: Disabled because of performance problems, see https://github.com/fonsp/Pluto.jl/pull/1925. Remove `complete_anyword` once fixed. See https://github.com/fonsp/Pluto.jl/pull/2013 local_variables_completion, ], defaultKeymap: false, // We add these manually later, so we can override them if necessary maxRenderedOptions: 512, // fons's magic number optionClass: (c) => c.type ?? "", }), update_docs_from_autocomplete_selection(on_update_doc_query), keymap.of(pluto_autocomplete_keymap), keymap.of(completionKeymap), ] } import { detect_deserializer } from "../../common/Serialization.js" import { EditorView } from "../../imports/CodemirrorPlutoSetup.js" export let pluto_paste_plugin = ({ pluto_actions, cell_id }) => { return EditorView.domEventHandlers({ paste: (event, view) => { if (!view.hasFocus) { // Tell codemirror it doesn't have to handle this when it doesn't have focus console.log("CodeMirror, why are you registring this paste? You aren't focused!") return true } // Prevent this event from reaching the Editor-level paste handler event.stopPropagation() const topaste = event.clipboardData.getData("text/plain") const deserializer = detect_deserializer(topaste) if (deserializer == null) { return false } // If we have the whole cell selected, the user doesn't want their current code to survive... // So we paste the cells, but then remove the original cell! (Ideally I want to keep that cell and fill it with the first deserialized one) // (This also applies to pasting in an empty cell) if (view.state.selection.main.from === 0 && view.state.selection.main.to === view.state.doc.length) { pluto_actions.add_deserialized_cells(topaste, cell_id, deserializer) pluto_actions.confirm_delete_multiple("This Should Never Be Visible", [cell_id]) return true } // End of cell, add new cells below if (view.state.selection.main.to === view.state.doc.length) { pluto_actions.add_deserialized_cells(topaste, cell_id, deserializer) return true } // Start of cell, ideally we'd add new cells above, but we don't have that yet if (view.state.selection.main.from === 0) { pluto_actions.add_deserialized_cells(topaste, cell_id, deserializer) return true } return false }, }) } import { syntaxTree, StateField, NodeWeakMap, Text } from "../../imports/CodemirrorPlutoSetup.js" import _ from "../../imports/lodash.js" const VERBOSE = false /** * @typedef TreeCursor * @type {import("../../imports/CodemirrorPlutoSetup.js").TreeCursor} */ /** * @typedef SyntaxNode * @type {TreeCursor["node"]} */ /** * @typedef Range * @property {number} from * @property {number} to * * @typedef {Range & {valid_from: number}} Definition * * @typedef ScopeState * @property {Array<{ * usage: Range, * definition: Range | null, * name: string, * }>} usages * @property {Map<String, Definition>} definitions * @property {Array<{ definition: Range, validity: Range, name: string }>} locals */ const r = (cursor) => ({ from: cursor.from, to: cursor.to }) const find_local_definition = (locals, name, cursor) => { for (let lo of locals) { if (lo.name === name && cursor.from >= lo.validity.from && cursor.to <= lo.validity.to) { return lo } } } const HardScopeNames = new Set([ "WhileStatement", "ForStatement", "TryStatement", "LetStatement", "FunctionDefinition", "MacroDefinition", "DoClause", "Generator", ]) const does_this_create_scope = (/** @type {TreeCursor} */ cursor) => { if (HardScopeNames.has(cursor.name)) return true if (cursor.name === "Assignment") { const reset = cursor.firstChild() try { // f(x) = x // @ts-ignore if (cursor.name === "CallExpression") return true } finally { if (reset) cursor.parent() } } return false } /** * Look into the left-hand side of an Assigment expression and find all ranges where variables are defined. * E.g. `a, (b,c) = something` will return ranges for a, b, c. * @param {TreeCursor} root_cursor * @returns {Range[]} */ const explore_assignment_lhs = (root_cursor) => { const a = cursor_not_moved_checker(root_cursor) let found = [] root_cursor.iterate((cursor) => { if (cursor.name === "Identifier" || cursor.name === "MacroIdentifier" || cursor.name === "Operator") { found.push(r(cursor)) } if (cursor.name === "IndexExpression" || cursor.name === "FieldExpression") { // not defining a variable but modifying an object return false } }) a() return found } /** * Remember the position where this is called, and return a function that will move into parents until we are are back at that position. * * You can use this before exploring children of a cursor, and then go back when you are done. */ const back_to_parent_resetter = (/** @type {TreeCursor} */ cursor) => { const map = new NodeWeakMap() map.cursorSet(cursor, "here") return () => { while (map.cursorGet(cursor) !== "here") { if (!cursor.parent()) throw new Error("Could not find my back to the original parent!") } } } const cursor_not_moved_checker = (cursor) => { const map = new NodeWeakMap() map.cursorSet(cursor, "yay") const debug = (cursor) => `${cursor.name}(${cursor.from},${cursor.to})` const debug_before = debug(cursor) return () => { if (map.cursorGet(cursor) !== "yay") { throw new Error(`Cursor changed position when forbidden! Before: ${debug_before}, after: ${debug(cursor)}`) } } } const i_am_nth_child = (cursor) => { const map = new NodeWeakMap() map.cursorSet(cursor, "here") if (!cursor.parent()) throw new Error("Cannot be toplevel") cursor.firstChild() let i = 0 while (map.cursorGet(cursor) !== "here") { i++ if (!cursor.nextSibling()) { throw new Error("Could not find my way back") } } return i } /** * @param {TreeCursor} cursor * @returns {Range[]} */ const explore_funcdef_arguments = (cursor, { enter, leave }) => { VERBOSE && console.assert(cursor.name === "TupleExpression" || cursor.name === "Arguments", cursor.name) let found = [] const position_validation = cursor_not_moved_checker(cursor) const position_resetter = back_to_parent_resetter(cursor) if (!cursor.firstChild()) throw new Error(`Expected to go into function definition argument expression, stuck at ${cursor.name}`) // should be in the TupleExpression now cursor.firstChild() const explore_argument = () => { if (cursor.name === "Identifier" || cursor.name === "Operator") { found.push(r(cursor)) } else if (cursor.name === "KwArg") { let went_in = cursor.firstChild() explore_argument() cursor.nextSibling() // find stuff used here cursor.iterate(enter, leave) if (went_in) cursor.parent() } else if (cursor.name === "BinaryExpression") { let went_in = cursor.firstChild() explore_argument() cursor.nextSibling() cursor.nextSibling() // find stuff used here cursor.iterate(enter, leave) if (went_in) cursor.parent() } } do { if (cursor.name === "KeywordArguments") { cursor.firstChild() // go into kwarg arguments } explore_argument() } while (cursor.nextSibling()) position_resetter() position_validation() VERBOSE && console.log({ found }) return found } /** * @param {TreeCursor | SyntaxNode} tree * @param {Text} doc * @param {any} _scopestate * @param {boolean} [verbose] * @returns {ScopeState} */ export let explore_variable_usage = (tree, doc, _scopestate, verbose = VERBOSE) => { if ("cursor" in tree) { console.trace("`explore_variable_usage()` called with a SyntaxNode, not a TreeCursor") tree = tree.cursor() } const scopestate = { usages: [], definitions: new Map(), locals: [], } let local_scope_stack = /** @type {Range[]} */ ([]) const definitions = /** @type {Map<string, Definition>} */ new Map() const locals = /** @type {Array<{ definition: Range, validity: Range, name: string }>} */ ([]) const usages = /** @type {Array<{ usage: Range, definition: Range | null, name: string }>} */ ([]) const return_false_immediately = new NodeWeakMap() let enter, leave enter = (/** @type {TreeCursor} */ cursor) => { if (verbose) { console.group(`Explorer: ${cursor.name}`) console.groupCollapsed("Details") try { console.log(`Full tree: ${cursor.toString()}`) console.log("Full text:", doc.sliceString(cursor.from, cursor.to)) console.log(`scopestate:`, scopestate) } finally { console.groupEnd() } } if ( return_false_immediately.cursorGet(cursor) || cursor.name === "ModuleDefinition" || cursor.name === "QuoteStatement" || cursor.name === "QuoteExpression" || cursor.name === "MacroIdentifier" || cursor.name === "ImportStatement" || cursor.name === "UsingStatement" ) { if (verbose) console.groupEnd() return false } const register_variable = (range) => { const name = doc.sliceString(range.from, range.to) if (local_scope_stack.length === 0) definitions.set(name, { ...range, valid_from: range.from, }) else locals.push({ name, validity: _.last(local_scope_stack), definition: range }) } if (does_this_create_scope(cursor)) { local_scope_stack.push(r(cursor)) } if (cursor.name === "Identifier" || cursor.name === "MacroIdentifier" || cursor.name === "Operator") { const name = doc.sliceString(cursor.from, cursor.to) usages.push({ name: name, usage: { from: cursor.from, to: cursor.to, }, definition: find_local_definition(locals, name, cursor) ?? null, }) } else if (cursor.name === "Assignment" || cursor.name === "KwArg" || cursor.name === "ForBinding" || cursor.name === "CatchClause") { if (cursor.firstChild()) { // @ts-ignore if (cursor.name === "catch") cursor.nextSibling() // CallExpression means function definition `f(x) = x`, this is handled elsewhere // @ts-ignore if (cursor.name !== "CallExpression") { explore_assignment_lhs(cursor).forEach(register_variable) // mark this one as finished return_false_immediately.cursorSet(cursor, true) } cursor.parent() } } else if (cursor.name === "Parameters") { explore_assignment_lhs(cursor).forEach(register_variable) if (verbose) console.groupEnd() return false } else if (cursor.name === "Field") { if (verbose) console.groupEnd() return false } else if (cursor.name === "CallExpression") { if (cursor.matchContext(["FunctionDefinition", "Signature"]) || (cursor.matchContext(["Assignment"]) && i_am_nth_child(cursor) === 0)) { const pos_resetter = back_to_parent_resetter(cursor) cursor.firstChild() // Now we should have the function name // @ts-ignore if (cursor.name === "Identifier" || cursor.name === "Operator" || cursor.name === "FieldExpression") { if (verbose) console.log("found function name", doc.sliceString(cursor.from, cursor.to), cursor.name) const last_scoper = local_scope_stack.pop() register_variable(r(cursor)) if (last_scoper) local_scope_stack.push(last_scoper) cursor.nextSibling() } if (verbose) console.log("expl funcdef ", doc.sliceString(cursor.from, cursor.to)) explore_funcdef_arguments(cursor, { enter, leave }).forEach(register_variable) if (verbose) console.log("expl funcdef ", doc.sliceString(cursor.from, cursor.to)) pos_resetter() if (verbose) console.log("end of FunctionDefinition, currently at ", cursor.node) if (verbose) console.groupEnd() return false } } else if (cursor.name === "Generator") { // This is: (f(x) for x in xs) or [f(x) for x in xs] const savior = back_to_parent_resetter(cursor) // We do a Generator in two steps: // First we explore all the ForBindings (where locals get defined), and then we go into the first child (where those locals are used). // 1. The for bindings `x in xs` if (cursor.firstChild()) { // Note that we skip the first child here, which is what we want! That's the iterated expression that we leave for the end. while (cursor.nextSibling()) { cursor.iterate(enter, leave) } savior() } // 2. The iterated expression `f(x)` if (cursor.firstChild()) { cursor.iterate(enter, leave) savior() } // k thx byeee leave(cursor) return false } } leave = (/** @type {TreeCursor} */ cursor) => { if (verbose) { console.groupEnd() } if (does_this_create_scope(cursor)) { local_scope_stack.pop() } } const debugged_enter = (cursor) => { const a = cursor_not_moved_checker(cursor) const result = enter(cursor) a() return result } tree.iterate(verbose ? debugged_enter : enter, leave) if (local_scope_stack.length > 0) throw new Error(`Some scopes were not leaved... ${JSON.stringify(local_scope_stack)}`) const output = { usages, definitions, locals } if (verbose) console.log(output) return output } /** * @type {StateField<ScopeState>} */ export let ScopeStateField = StateField.define({ create(state) { try { let cursor = syntaxTree(state).cursor() let scopestate = explore_variable_usage(cursor, state.doc, undefined) return scopestate } catch (error) { console.error("Something went wrong while parsing variables...", error) return { usages: [], definitions: new Map(), locals: [], } } }, update(value, tr) { try { if (syntaxTree(tr.state) != syntaxTree(tr.startState)) { let cursor = syntaxTree(tr.state).cursor() let scopestate = explore_variable_usage(cursor, tr.state.doc, null) return scopestate } else { return value } } catch (error) { console.error("Something went wrong while parsing variables...", error) return { usages: [], definitions: new Map(), locals: [], } } }, }) import { open_pluto_popup } from "../../common/open_pluto_popup.js" import { ViewPlugin, StateEffect, StateField } from "../../imports/CodemirrorPlutoSetup.js" import _ from "../../imports/lodash.js" import { html } from "../../imports/Preact.js" /** @type {any} */ const TabHelpEffect = StateEffect.define() const TabHelp = StateField.define({ create() { return false }, update(value, tr) { for (let effect of tr.effects) { if (effect.is(TabHelpEffect)) return effect.value } return value }, }) /** @type {any} */ export const LastFocusWasForcedEffect = StateEffect.define() const LastFocusWasForced = StateField.define({ create() { return false }, update(value, tr) { for (let effect of tr.effects) { if (effect.is(LastFocusWasForcedEffect)) return effect.value } return value }, }) export const tab_help_plugin = ViewPlugin.define( (view) => ({ setready: (x) => requestIdleCallback(() => { view.dispatch({ effects: [TabHelpEffect.of(x)], }) }), }), { provide: (p) => [TabHelp, LastFocusWasForced], eventObservers: { focus: function (event, view) { // The next key should trigger the popup this.setready(true) }, blur: function (event, view) { this.setready(false) requestIdleCallback(() => { view.dispatch({ effects: [LastFocusWasForcedEffect.of(false)], }) }) }, click: function (event, view) { // This means you are not doing keyboard navigation :) this.setready(false) }, keydown: function (event, view) { if (event.key == "Tab") { if (view.state.field(TabHelp) && !view.state.field(LastFocusWasForced) && !view.state.readOnly) { open_pluto_popup({ type: "info", source_element: view.dom, body: html`Press <kbd>Esc</kbd> and then <kbd>Tab</kbd> to continue navigation. <em style="font-size: .6em;">skkrt!</em>`, }) this.setready(false) } } else { this.setready(false) } }, }, } ) import { html, useMemo, useEffect } from "../../imports/Preact.js" import * as preact from "../../imports/Preact.js" import { RunLocalButton, BinderButton } from "../EditOrRunButton.js" import { start_local } from "../../common/RunLocal.js" import { BackendLaunchPhase, start_binder } from "../../common/Binder.js" import immer, { applyPatches, produceWithPatches } from "../../imports/immer.js" export const EditorLaunchBackendButton = ({ editor, launch_params, status }) => { try { const EnvRun = useMemo( // @ts-ignore () => window?.pluto_injected_environment?.environment?.({ client: editor.client, editor, imports: { immer, preact } })?.custom_run_or_edit, [editor.client, editor] ) // @ts-ignore if (window?.pluto_injected_environment?.provides_backend) { // @ts-ignore return html`<${EnvRun} editor=${editor} backend_phases=${BackendLaunchPhase} launch_params=${launch_params} />` // Don't allow a misconfigured environment to stop offering other backends } } catch (e) {} if (status == null) return null if (status.offer_local) return html`<${RunLocalButton} start_local=${() => start_local({ setStatePromise: editor.setStatePromise, connect: editor.connect, launch_params: launch_params, })} />` if (status.offer_binder) return html`<${BinderButton} offer_binder=${status.offer_binder} start_binder=${() => start_binder({ setStatePromise: editor.setStatePromise, connect: editor.connect, launch_params: launch_params, })} notebookfile=${launch_params.notebookfile == null ? null : new URL(launch_params.notebookfile, window.location.href).href} notebook=${editor.state.notebook} />` return null } import { useEventListener } from "../../common/useEventListener.js" import { useEffect } from "../../imports/Preact.js" /** * Time flies when you're building Pluto... * At one moment you self-assignee to issue number #1, next moment we're approaching issue #2000... * * We can't just put `<base target="_blank">` in the `<head>`, because this also opens hash links * like `#id` in a new tab... * * This components takes every click event on an <a> that points to another origin (i.e. not `#id`) * and sneakily puts in a `target="_blank"` attribute so it opens in a new tab. * * Fixes https://github.com/fonsp/Pluto.jl/issues/1 * Based on https://stackoverflow.com/a/12552017/2681964 */ export let HijackExternalLinksToOpenInNewTab = () => { useEventListener( document, "click", (event) => { if (event.defaultPrevented) return const origin = event.target.closest(`a`) if (origin && !origin.hasAttribute("target")) { let as_url = new URL(origin.href) if (as_url.origin !== window.location.origin) { origin.target = "_blank" } } }, [] ) return null } import _ from "../../imports/lodash.js" import { html, useEffect, useState } from "../../imports/Preact.js" import register from "../../imports/PreactCustomElement.js" import { FeaturedCard } from "./FeaturedCard.js" /** * @typedef SourceManifestNotebookEntry * @type {{ * id: String, * hash: String, * html_path?: String, * statefile_path?: String, * notebookfile_path?: String, * frontmatter?: Record<string,any>, * }} */ /** * @typedef SourceManifestCollectionEntry * @type {{ * title?: String, * description?: String, * tags?: Array<String> | "everything", * }} */ /** * @typedef SourceManifest * @type {{ * notebooks: Record<string,SourceManifestNotebookEntry>, * collections?: Array<SourceManifestCollectionEntry>, * pluto_version?: String, * julia_version?: String, * format_version?: String, * source_url?: String, * title?: String, * description?: String, * binder_url?: String, * slider_server_url?: String, * }} */ /** * This data is used as placeholder while the real data is loading from the network. * @type {SourceManifest[]} */ const placeholder_data = [ { title: "Featured Notebooks", description: "These notebooks from the Julia community show off what you can do with Pluto. Give it a try, you might learn something new!", collections: [ { title: "Loading...", tags: [], }, ], notebooks: {}, }, ] /** This HTML is shown instead of the featured notebooks if the user is offline. */ const offline_html = html` <div class="featured-source"> <h1>${placeholder_data[0].title}</h1> <p>Here are a couple of notebooks to get started with Pluto.jl:</p> <ul> <li>1. <a href="sample/Getting%20started.jl">Getting started</a></li> <li>2. <a href="sample/Markdown.jl">Markdown</a></li> <li>3. <a href="sample/Basic%20mathematics.jl">Basic mathematics</a></li> <li>4. <a href="sample/Interactivity.jl">Interactivity</a></li> <li>5. <a href="sample/PlutoUI.jl.jl">PlutoUI.jl</a></li> <li>6. <a href="sample/Plots.jl.jl">Plots.jl</a></li> <li>7. <a href="sample/Tower%20of%20Hanoi.jl">Tower of Hanoi</a></li> <li>8. <a href="sample/JavaScript.jl">JavaScript</a></li> </ul> <br /> <br /> <br /> <br /> <br /> <br /> <p>Tip: <em>Visit this page again when you are connected to the internet to read our online collection of featured notebooks.</em></p> </div> ` /** * If no collections are defined, then this special collection will just show all notebooks under the "Notebooks" category. * No collections are defined if no `pluto_export_configuration.json` file was provided to PlutoSliderServer.jl. * @type {SourceManifestCollectionEntry[]} */ const fallback_collections = [ { title: "Notebooks", tags: "everything", }, ] /** * @typedef FeaturedSource * @type {{ * url: String, * id?: String, * integrity?: String, * valid_until?: String * }} */ const get_id = (/** @type {FeaturedSource} */ source) => source?.id ?? source.url /** * @param {{ * sources: FeaturedSource[]?, * direct_html_links: boolean, * }} props */ export const Featured = ({ sources, direct_html_links }) => { // source_data will be a mapping from [source URL] => [data from that source] const [source_data, set_source_data] = useState(/** @type {Record<String,SourceManifest>} */ ({})) useEffect(() => { if (sources != null) { set_waited_too_long(false) set_source_data({}) const ids = Array.from(new Set(sources.map(get_id))) const promises = ids.map((id) => { const sources_for_id = sources.filter((source) => get_id(source) === id) let result = promise_any_with_priority( sources_for_id.map(async (source) => { const { url, integrity, valid_until } = source if (valid_until != null && new Date(valid_until) < new Date()) { throw new Error(`Source ${url} is expired with valid_until ${valid_until}`) } const data = await (await fetch(new Request(url, { integrity: integrity ?? undefined }))).json() if (data.format_version !== "1") { throw new Error(`Invalid format version: ${data.format_version}`) } return [data, id, url] }) ) return result.then(([data, id, url]) => { set_source_data((old) => ({ ...old, [id]: { ...data, source_url: url, }, })) }) }) Promise.any(promises).catch((e) => { console.error("All featured sources failed to load: ", e) ;(e?.errors ?? []).forEach((e) => console.error(e)) set_waited_too_long(true) }) } }, [sources]) useEffect(() => { if (Object.entries(source_data).length > 0) { console.log("Sources:", source_data) } }, [source_data]) const [waited_too_long, set_waited_too_long] = useState(false) useEffect(() => { setTimeout(() => { set_waited_too_long(true) }, 8 * 1000) }, []) const no_data = Object.entries(source_data).length === 0 const ids = Array.from(new Set(sources?.map(get_id) ?? [])) const sorted_on_source_order = ids.map((id) => source_data[id]).filter((d) => d != null) return no_data && waited_too_long ? offline_html : html` ${(no_data ? placeholder_data : sorted_on_source_order).map((/** @type {SourceManifest} */ data) => { let collections = data?.collections ?? fallback_collections return html` <div class="featured-source"> <h1>${data.title}</h1> <p>${data.description}</p> ${collections.map((coll) => { return html` <div class="collection"> <h2>${coll.title}</h2> <p>${coll.description}</p> <div class="card-list"> ${collection(Object.values(data.notebooks), coll.tags ?? []).map( (entry) => html`<${FeaturedCard} entry=${entry} source_manifest=${data} direct_html_links=${direct_html_links} />` )} </div> </div> ` })} </div> ` })} ` } register(Featured, "pluto-featured", ["sources", "direct_html_links"]) /** Return all notebook entries that have at least one of the given `tags`. Notebooks are sorted on `notebook.frontmatter.order` or `notebook.id`. */ const collection = (/** @type {SourceManifestNotebookEntry[]} */ notebooks, /** @type {String[] | "everything"} */ tags) => { const nbs = tags === "everything" ? notebooks : notebooks.filter((notebook) => tags.some((t) => (notebook.frontmatter?.tags ?? []).includes(t))) let n = (s) => (isNaN(s) ? s : Number(s)) return /** @type {SourceManifestNotebookEntry[]} */ (_.sortBy(nbs, [(nb) => n(nb?.frontmatter?.order), "id"])) } /** * Given a list promises, return promise[0].catch(() => promise[1].catch(() => promise[2]... etc)) * @param {Promise[]} promises * @returns {Promise} */ const promise_any_with_priority = (/** @type {Promise[]} */ promises, /** @type {Promise[]} */ already_rejected = []) => { if (promises.length <= 1) { return Promise.any([...promises, ...already_rejected]) } else { return promises[0].catch(() => promise_any_with_priority(promises.slice(1), [...already_rejected, promises[0]])) } } import { base64url_to_base64 } from "../../common/PlutoHash.js" import { with_query_params } from "../../common/URLTools.js" import _ from "../../imports/lodash.js" import { html, useEffect, useState, useMemo } from "../../imports/Preact.js" const transparent_svg = "data:image/svg+xml;charset=utf8,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%3E%3C/svg%3E" const str_to_degree = (s) => ([...s].reduce((a, b) => a + b.charCodeAt(0), 0) * 79) % 360 /** * @param {{ * source_manifest?: import("./Featured.js").SourceManifest, * entry: import("./Featured.js").SourceManifestNotebookEntry, * direct_html_links: boolean, * disable_links: boolean, * image_loading?: string, * }} props */ export const FeaturedCard = ({ entry, source_manifest, direct_html_links, disable_links, image_loading }) => { const title = entry.frontmatter?.title const { source_url } = source_manifest ?? {} const u = (/** @type {string | null | undefined} */ x) => source_url == null ? _.isEmpty(x) ? null : x : x == null ? null : // URLs are relative to the source URL... new URL( x, // ...and the source URL is relative to the current location new URL(source_url, window.location.href) ).href // `direct_html_links` means that we will navigate you directly to the exported HTML file. Otherwise, we use our local editor, with the exported state as parameters. This lets users run the featured notebooks locally. const href = disable_links ? "#" : direct_html_links ? u(entry.html_path) : with_query_params(`edit`, { statefile: u(entry.statefile_path), notebookfile: u(entry.notebookfile_path), notebookfile_integrity: entry.hash == null ? null : `sha256-${base64url_to_base64(entry.hash)}`, disable_ui: `true`, name: title == null ? null : `sample ${title}`, pluto_server_url: `.`, // Little monkey patch because we don't want to use the slider server when for the CDN source, only for the featured.plutojl.org source. But both sources have the same pluto_export.json so this is easiest. slider_server_url: source_url?.includes("cdn.jsdelivr.net/gh/JuliaPluto/featured") ? null : u(source_manifest?.slider_server_url), }) const author = author_info(entry.frontmatter) return html` <featured-card style=${`--card-color-hue: ${str_to_degree(entry.id)}deg;`}> <a class="banner" href=${href}><img src=${u(entry?.frontmatter?.image) ?? transparent_svg} loading=${image_loading} /></a> ${author?.name == null ? null : html` <div class="author"> <img src=${author.image ?? transparent_svg} loading=${image_loading} /> <span> <a href=${author.url}>${author.name}</a> ${author.has_coauthors ? html` and others` : null} </span> </div> `} <h3><a href=${href} title=${entry?.frontmatter?.title}>${entry?.frontmatter?.title ?? entry.id}</a></h3> <p title=${entry?.frontmatter?.description}>${entry?.frontmatter?.description}</p> </featured-card> ` } /** * @typedef AuthorInfo * @type {{ * name: string?, * url: string?, * image: string?, * has_coauthors?: boolean, * }} */ /** * @returns {AuthorInfo?} */ const author_info = (frontmatter) => author_info_item(frontmatter.author) ?? author_info_item({ name: frontmatter.author_name, url: frontmatter.author_url, image: frontmatter.author_image, }) /** * @returns {AuthorInfo?} */ const author_info_item = (x) => { if (x instanceof Array) { const first = author_info_item(x[0]) if (first?.name) { const has_coauthors = x.length > 1 return { ...first, has_coauthors } } } else if (typeof x === "string") { return { name: x, url: null, image: null, } } else if (x instanceof Object) { let { name, image, url } = x if (image == null && !_.isEmpty(url)) { image = url + ".png?size=48" } return { name, url, image, } } return null } import _ from "../../imports/lodash.js" import { html } from "../../imports/Preact.js" import { FilePicker } from "../FilePicker.js" import { PasteHandler } from "../PasteHandler.js" import { guess_notebook_location } from "../../common/NotebookLocationFromURL.js" /** * @param {{ * client: import("../../common/PlutoConnection.js").PlutoConnection?, * connected: Boolean, * show_samples: Boolean, * CustomPicker: {text: String, placeholder: String}?, * on_start_navigation: (string) => void, * }} props */ export const Open = ({ client, connected, CustomPicker, show_samples, on_start_navigation }) => { const on_open_path = async (new_path) => { const processed = await guess_notebook_location(new_path) on_start_navigation(processed.path_or_url) window.location.href = (processed.type === "path" ? link_open_path : link_open_url)(processed.path_or_url) } const desktop_on_open_path = async (_p) => { window.plutoDesktop?.fileSystem.openNotebook("path") } const desktop_on_open_url = async (url) => { window.plutoDesktop?.fileSystem.openNotebook("url", url) } const picker = CustomPicker ?? { text: "Open a notebook", placeholder: "Enter path or URL...", } return html`<${PasteHandler} on_start_navigation=${on_start_navigation} /> <h2>${picker.text}</h2> <div id="new" class=${!!window.plutoDesktop ? "desktop_opener" : ""}> <${FilePicker} key=${picker.placeholder} client=${client} value="" on_submit=${on_open_path} on_desktop_submit=${desktop_on_open_path} clear_on_blur=${false} button_label=${window.plutoDesktop ? "Open File" : "Open"} placeholder=${picker.placeholder} /> ${window.plutoDesktop != null ? html`<${FilePicker} key=${picker.placeholder} client=${client} value="" on_desktop_submit=${desktop_on_open_url} button_label="Open from URL" placeholder=${picker.placeholder} />` : null} </div>` } export const link_open_path = (path, execution_allowed = false) => "open?" + new URLSearchParams({ path: path }).toString() export const link_open_url = (url) => "open?" + new URLSearchParams({ url: url }).toString() export const link_edit = (notebook_id) => "edit?id=" + notebook_id import _ from "../../imports/lodash.js" import { html, useEffect, useState, useRef } from "../../imports/Preact.js" import * as preact from "../../imports/Preact.js" import { cl } from "../../common/ClassTable.js" import { link_edit, link_open_path } from "./Open.js" import { ProcessStatus } from "../../common/ProcessStatus.js" /** * @typedef CombinedNotebook * @type {{ * path: string, * transitioning: Boolean, * entry?: import("./Welcome.js").NotebookListEntry, * }} */ /** * @param {string} path * @returns {CombinedNotebook} */ const entry_notrunning = (path) => { return { transitioning: false, // between running and being shut down entry: undefined, // undefined means that it is not running path: path, } } /** * @param {import("./Welcome.js").NotebookListEntry} entry * @returns {CombinedNotebook} */ const entry_running = (entry) => { return { transitioning: false, // between running and being shut down entry, path: entry.path, } } const split_at_level = (path, level) => path.split(/\/|\\/).slice(-level).join("/") const shortest_path = (path, allpaths) => { let level = 1 for (const otherpath of allpaths) { if (otherpath !== path) { while (split_at_level(path, level) === split_at_level(otherpath, level)) { level++ } } } return split_at_level(path, level) } /** * @param {{ * client: import("../../common/PlutoConnection.js").PlutoConnection?, * connected: Boolean, * remote_notebooks: Array<import("./Welcome.js").NotebookListEntry>, * CustomRecent: preact.ReactElement?, * on_start_navigation: (string) => void, * }} props */ export const Recent = ({ client, connected, remote_notebooks, CustomRecent, on_start_navigation }) => { const [combined_notebooks, set_combined_notebooks] = useState(/** @type {Array<CombinedNotebook>?} */ (null)) const combined_notebooks_ref = useRef(combined_notebooks) combined_notebooks_ref.current = combined_notebooks const set_notebook_state = (path, new_state_props) => { set_combined_notebooks( (prevstate) => prevstate?.map((nb) => { return nb.path == path ? { ...nb, ...new_state_props } : nb }) ?? null ) } useEffect(() => { if (client != null && connected) { client.send("get_all_notebooks", {}, {}).then(({ message }) => { const running = /** @type {Array<import("./Welcome.js").NotebookListEntry>} */ (message.notebooks).map((nb) => entry_running(nb)) const recent_notebooks = get_stored_recent_notebooks() // show running notebooks first, in the order defined by the recent notebooks, then recent notebooks const combined_notebooks = [ ..._.sortBy(running, [(nb) => _.findIndex([...recent_notebooks, ...running], (r) => r.path === nb.path)]), ..._.differenceBy(recent_notebooks, running, (nb) => nb.path), ] set_combined_notebooks(combined_notebooks) document.body.classList.remove("loading") }) } }, [client != null && connected]) useEffect(() => { const new_running = remote_notebooks if (combined_notebooks_ref.current != null) { // a notebook list updates happened while the welcome screen is open, because a notebook started running for example // the list has already been generated and rendered to the page. We try to maintain order as much as possible, to prevent the list order "jumping around" while you are interacting with it. // You can always get a neatly sorted list by refreshing the page. // already rendered notebooks will be added to this list: const rendered_and_running = [] const new_combined_notebooks = combined_notebooks_ref.current.map((nb) => { // try to find a matching notebook in the remote list let running_version = null if (nb.entry != null) { // match notebook_ids to handle a path change running_version = new_running.find((rnb) => rnb.notebook_id === nb.entry?.notebook_id) } else { // match paths to handle a notebook bootup running_version = new_running.find((rnb) => rnb.path === nb.path) } if (running_version == null) { return entry_notrunning(nb.path) } else { const new_notebook = entry_running(running_version) rendered_and_running.push(running_version) return new_notebook } }) const not_rendered_but_running = new_running.filter((rnb) => !rendered_and_running.includes(rnb)).map(entry_running) set_combined_notebooks([...not_rendered_but_running, ...new_combined_notebooks]) } }, [remote_notebooks]) const on_session_click = (/** @type {CombinedNotebook} */ nb) => { if (nb.transitioning) { return } const running = nb.entry != null if (running) { if (client == null) return if (confirm(nb.entry?.process_status === ProcessStatus.waiting_for_permission ? "Close notebook session?" : "Shut down notebook process?")) { set_notebook_state(nb.path, { running: false, transitioning: true, }) client.send("shutdown_notebook", { keep_in_session: false }, { notebook_id: nb.entry?.notebook_id }, false) } } else { set_notebook_state(nb.path, { transitioning: true, }) fetch(link_open_path(nb.path) + "&execution_allowed=true", { method: "GET", }) .then((r) => { if (!r.redirected) { throw new Error("file not found maybe? try opening the notebook directly") } }) .catch((e) => { console.error("Failed to start notebook in background") console.error(e) set_notebook_state(nb.path, { transitioning: false, notebook_id: null, }) }) } } useEffect(() => { document.body.classList.toggle("nosessions", !(combined_notebooks == null || combined_notebooks.length > 0)) }, [combined_notebooks]) /// RENDER const all_paths = combined_notebooks?.map((nb) => nb.path) let recents = combined_notebooks == null ? html`<li class="not_yet_ready"><em>Loading...</em></li>` : combined_notebooks.map((nb) => { const running = nb.entry != null return html`<li key=${nb.path} class=${cl({ running: running, recent: !running, transitioning: nb.transitioning, })} > <button onclick=${() => on_session_click(nb)} title=${running ? nb.entry?.process_status === ProcessStatus.waiting_for_permission ? "Stop session" : "Shut down notebook" : "Start notebook in background"} > <span class="ionicon"></span> </button> <a href=${running ? link_edit(nb.entry?.notebook_id) : link_open_path(nb.path)} title=${nb.path} onClick=${(e) => { if (!running) { on_start_navigation(shortest_path(nb.path, all_paths)) set_notebook_state(nb.path, { transitioning: true, }) } }} >${shortest_path(nb.path, all_paths)}</a > </li>` }) if (CustomRecent == null) { return html` <h2>My work</h2> <ul id="recent" class="show_scrollbar"> <li class="new"> <a href="new" onClick=${(e) => { on_start_navigation("new notebook") }} ><button><span class="ionicon"></span></button>Create a <strong>new notebook</strong></a > </li> ${recents} </ul> ` } else { return html`<${CustomRecent} cl=${cl} combined=${combined_notebooks} client=${client} recents=${recents} />` } } const get_stored_recent_notebooks = () => { const storedString = localStorage.getItem("recent notebooks") const storedData = storedString != null ? JSON.parse(storedString) : [] const storedList = storedData instanceof Array ? storedData : [] return storedList.map(entry_notrunning) } import _ from "../../imports/lodash.js" import { html, useEffect, useState, useRef } from "../../imports/Preact.js" import * as preact from "../../imports/Preact.js" import { create_pluto_connection, ws_address_from_base } from "../../common/PlutoConnection.js" import { new_update_message } from "../../common/NewUpdateMessage.js" import { Open } from "./Open.js" import { Recent } from "./Recent.js" import { Featured } from "./Featured.js" import { get_environment } from "../../common/Environment.js" import default_featured_sources from "../../featured_sources.js" // This is imported asynchronously - uncomment for development // import environment from "../../common/Environment.js" /** * @typedef NotebookListEntry * @type {{ * notebook_id: string, * path: string, * in_temp_dir: boolean, * shortpath: string, * process_status: string, * }} */ /** * @typedef LaunchParameters * @type {{ * pluto_server_url: string?, * featured_direct_html_links: boolean, * featured_sources: import("./Featured.js").FeaturedSource[]?, * featured_source_url?: string, * featured_source_integrity?: string, * }} */ // We use a link from the head instead of directing linking "img/logo.svg" because parcel does not bundle preact files const url_logo_big = document.head.querySelector("link[rel='pluto-logo-big']")?.getAttribute("href") ?? "" /** * @param {{ * launch_params: LaunchParameters, * }} props */ export const Welcome = ({ launch_params }) => { const [remote_notebooks, set_remote_notebooks] = useState(/** @type {Array<NotebookListEntry>} */ ([])) const [connected, set_connected] = useState(false) const [extended_components, set_extended_components] = useState({ show_samples: true, CustomPicker: null, CustomRecent: null, }) const client_ref = useRef(/** @type {import('../../common/PlutoConnection').PlutoConnection} */ ({})) useEffect(() => { const on_update = ({ message, type }) => { if (type === "notebook_list") { // a notebook list updates happened while the welcome screen is open, because a notebook started running for example set_remote_notebooks(message.notebooks) } } const on_connection_status = set_connected const client_promise = create_pluto_connection({ on_unrequested_update: on_update, on_connection_status: on_connection_status, on_reconnect: async () => true, ws_address: launch_params.pluto_server_url ? ws_address_from_base(launch_params.pluto_server_url) : undefined, }) client_promise.then(async (client) => { Object.assign(client_ref.current, client) set_connected(true) try { const environment = await get_environment(client) const { custom_recent, custom_filepicker, show_samples = true } = environment({ client, editor: this, imports: { preact } }) set_extended_components((old) => ({ ...old, CustomRecent: custom_recent, CustomPicker: custom_filepicker, show_samples, })) } catch (e) {} new_update_message(client) // to start JIT'ting client.send("current_time") client.send("completepath", { query: "" }, {}) }) }, []) const { show_samples, CustomRecent, CustomPicker } = extended_components // When block_screen_with_this_text is null (default), all is fine. When it is a string, we show a big banner with that text, and disable all other UI. https://github.com/fonsp/Pluto.jl/pull/2292 const [block_screen_with_this_text, set_block_screen_with_this_text] = useState(/** @type {string?} */ (null)) const on_start_navigation = (value, expect_navigation = true) => { if (expect_navigation) { // Instead of calling set_block_screen_with_this_text(value) directly, we wait for the beforeunload to happen, and then we do it. If this event does not happen within 1 second, then that means that the user right-clicked, or Ctrl+Clicked (to open in a new tab), and we don't want to clear the main menu. https://github.com/fonsp/Pluto.jl/issues/2301 const handler = (e) => { set_block_screen_with_this_text(value) } window.addEventListener("beforeunload", handler) setTimeout(() => window.removeEventListener("beforeunload", handler), 1000) } else { set_block_screen_with_this_text(value) } } /** * These are the sources from which we will download the featured notebook titles and metadata. * @type {import("./Featured.js").FeaturedSource[]} */ const featured_sources = preact.useMemo( () => // Option 1: configured directly launch_params.featured_sources ?? // Option 2: configured through url and integrity strings (launch_params.featured_source_url ? [{ url: launch_params.featured_source_url, integrity: launch_params.featured_source_integrity }] : // Option 3: default default_featured_sources.sources), [launch_params] ) if (block_screen_with_this_text != null) { return html` <div class="navigating-away-banner"> <h2>Loading ${block_screen_with_this_text}...</h2> </div> ` } return html` <section id="title"> <h1>welcome to <img src=${url_logo_big} /></h1> </section> <section id="mywork"> <div> <${Recent} client=${client_ref.current} connected=${connected} remote_notebooks=${remote_notebooks} CustomRecent=${CustomRecent} on_start_navigation=${on_start_navigation} /> </div> </section> <section id="open"> <div> <${Open} client=${client_ref.current} connected=${connected} CustomPicker=${CustomPicker} show_samples=${show_samples} on_start_navigation=${on_start_navigation} /> </div> </section> <section id="featured"> <div> <${Featured} sources=${featured_sources} direct_html_links=${launch_params.featured_direct_html_links} /> </div> </section> ` } /* alegreya-sans-cyrillic-ext-400-normal */ @font-face { font-family: "Alegreya Sans"; font-style: normal; font-display: swap; size-adjust: 119%; font-weight: 400; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-cyrillic-ext-400-normal.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-cyrillic-ext-400-normal.woff) format("woff"); unicode-range: U+0460-052F, U+1C80-1C88, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F; } /* alegreya-sans-cyrillic-400-normal */ @font-face { font-family: "Alegreya Sans"; font-style: normal; font-display: swap; size-adjust: 119%; font-weight: 400; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-cyrillic-400-normal.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-cyrillic-400-normal.woff) format("woff"); unicode-range: U+0301, U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116; } /* alegreya-sans-greek-ext-400-normal */ @font-face { font-family: "Alegreya Sans"; font-style: normal; font-display: swap; size-adjust: 119%; font-weight: 400; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-greek-ext-400-normal.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-greek-ext-400-normal.woff) format("woff"); unicode-range: U+1F00-1FFF; } /* alegreya-sans-greek-400-normal */ @font-face { font-family: "Alegreya Sans"; font-style: normal; font-display: swap; size-adjust: 119%; font-weight: 400; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-greek-400-normal.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-greek-400-normal.woff) format("woff"); unicode-range: U+0370-0377, U+037A-037F, U+0384-038A, U+038C, U+038E-03A1, U+03A3-03FF; } /* alegreya-sans-vietnamese-400-normal */ @font-face { font-family: "Alegreya Sans"; font-style: normal; font-display: swap; size-adjust: 119%; font-weight: 400; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-vietnamese-400-normal.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-vietnamese-400-normal.woff) format("woff"); unicode-range: U+0102-0103, U+0110-0111, U+0128-0129, U+0168-0169, U+01A0-01A1, U+01AF-01B0, U+0300-0301, U+0303-0304, U+0308-0309, U+0323, U+0329, U+1EA0-1EF9, U+20AB; } /* alegreya-sans-latin-ext-400-normal */ @font-face { font-family: "Alegreya Sans"; font-style: normal; font-display: swap; size-adjust: 119%; font-weight: 400; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-latin-ext-400-normal.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-latin-ext-400-normal.woff) format("woff"); unicode-range: U+0100-02AF, U+0304, U+0308, U+0329, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C0, U+2113, U+2C60-2C7F, U+A720-A7FF; } /* alegreya-sans-latin-400-normal */ @font-face { font-family: "Alegreya Sans"; font-style: normal; font-display: swap; size-adjust: 119%; font-weight: 400; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-latin-400-normal.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-latin-400-normal.woff) format("woff"); unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+2074, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD; } /* alegreya-sans-cyrillic-ext-500-normal */ @font-face { font-family: "Alegreya Sans"; font-style: normal; font-display: swap; size-adjust: 119%; font-weight: 500; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-cyrillic-ext-500-normal.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-cyrillic-ext-500-normal.woff) format("woff"); unicode-range: U+0460-052F, U+1C80-1C88, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F; } /* alegreya-sans-cyrillic-500-normal */ @font-face { font-family: "Alegreya Sans"; font-style: normal; font-display: swap; size-adjust: 119%; font-weight: 500; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-cyrillic-500-normal.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-cyrillic-500-normal.woff) format("woff"); unicode-range: U+0301, U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116; } /* alegreya-sans-greek-ext-500-normal */ @font-face { font-family: "Alegreya Sans"; font-style: normal; font-display: swap; size-adjust: 119%; font-weight: 500; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-greek-ext-500-normal.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-greek-ext-500-normal.woff) format("woff"); unicode-range: U+1F00-1FFF; } /* alegreya-sans-greek-500-normal */ @font-face { font-family: "Alegreya Sans"; font-style: normal; font-display: swap; size-adjust: 119%; font-weight: 500; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-greek-500-normal.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-greek-500-normal.woff) format("woff"); unicode-range: U+0370-0377, U+037A-037F, U+0384-038A, U+038C, U+038E-03A1, U+03A3-03FF; } /* alegreya-sans-vietnamese-500-normal */ @font-face { font-family: "Alegreya Sans"; font-style: normal; font-display: swap; size-adjust: 119%; font-weight: 500; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-vietnamese-500-normal.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-vietnamese-500-normal.woff) format("woff"); unicode-range: U+0102-0103, U+0110-0111, U+0128-0129, U+0168-0169, U+01A0-01A1, U+01AF-01B0, U+0300-0301, U+0303-0304, U+0308-0309, U+0323, U+0329, U+1EA0-1EF9, U+20AB; } /* alegreya-sans-latin-ext-500-normal */ @font-face { font-family: "Alegreya Sans"; font-style: normal; font-display: swap; size-adjust: 119%; font-weight: 500; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-latin-ext-500-normal.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-latin-ext-500-normal.woff) format("woff"); unicode-range: U+0100-02AF, U+0304, U+0308, U+0329, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C0, U+2113, U+2C60-2C7F, U+A720-A7FF; } /* alegreya-sans-latin-500-normal */ @font-face { font-family: "Alegreya Sans"; font-style: normal; font-display: swap; size-adjust: 119%; font-weight: 500; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-latin-500-normal.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-latin-500-normal.woff) format("woff"); unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+2074, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD; } /* alegreya-sans-cyrillic-ext-700-normal */ @font-face { font-family: "Alegreya Sans"; font-style: normal; font-display: swap; size-adjust: 119%; font-weight: 700; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-cyrillic-ext-700-normal.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-cyrillic-ext-700-normal.woff) format("woff"); unicode-range: U+0460-052F, U+1C80-1C88, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F; } /* alegreya-sans-cyrillic-700-normal */ @font-face { font-family: "Alegreya Sans"; font-style: normal; font-display: swap; size-adjust: 119%; font-weight: 700; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-cyrillic-700-normal.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-cyrillic-700-normal.woff) format("woff"); unicode-range: U+0301, U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116; } /* alegreya-sans-greek-ext-700-normal */ @font-face { font-family: "Alegreya Sans"; font-style: normal; font-display: swap; size-adjust: 119%; font-weight: 700; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-greek-ext-700-normal.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-greek-ext-700-normal.woff) format("woff"); unicode-range: U+1F00-1FFF; } /* alegreya-sans-greek-700-normal */ @font-face { font-family: "Alegreya Sans"; font-style: normal; font-display: swap; size-adjust: 119%; font-weight: 700; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-greek-700-normal.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-greek-700-normal.woff) format("woff"); unicode-range: U+0370-0377, U+037A-037F, U+0384-038A, U+038C, U+038E-03A1, U+03A3-03FF; } /* alegreya-sans-vietnamese-700-normal */ @font-face { font-family: "Alegreya Sans"; font-style: normal; font-display: swap; size-adjust: 119%; font-weight: 700; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-vietnamese-700-normal.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-vietnamese-700-normal.woff) format("woff"); unicode-range: U+0102-0103, U+0110-0111, U+0128-0129, U+0168-0169, U+01A0-01A1, U+01AF-01B0, U+0300-0301, U+0303-0304, U+0308-0309, U+0323, U+0329, U+1EA0-1EF9, U+20AB; } /* alegreya-sans-latin-ext-700-normal */ @font-face { font-family: "Alegreya Sans"; font-style: normal; font-display: swap; size-adjust: 119%; font-weight: 700; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-latin-ext-700-normal.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-latin-ext-700-normal.woff) format("woff"); unicode-range: U+0100-02AF, U+0304, U+0308, U+0329, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C0, U+2113, U+2C60-2C7F, U+A720-A7FF; } /* alegreya-sans-latin-700-normal */ @font-face { font-family: "Alegreya Sans"; font-style: normal; font-display: swap; size-adjust: 119%; font-weight: 700; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-latin-700-normal.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-latin-700-normal.woff) format("woff"); unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+2074, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD; } /* alegreya-sans-cyrillic-ext-400-italic */ @font-face { font-family: "Alegreya Sans"; font-style: italic; font-display: swap; size-adjust: 119%; font-weight: 400; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-cyrillic-ext-400-italic.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-cyrillic-ext-400-italic.woff) format("woff"); unicode-range: U+0460-052F, U+1C80-1C88, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F; } /* alegreya-sans-cyrillic-400-italic */ @font-face { font-family: "Alegreya Sans"; font-style: italic; font-display: swap; size-adjust: 119%; font-weight: 400; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-cyrillic-400-italic.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-cyrillic-400-italic.woff) format("woff"); unicode-range: U+0301, U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116; } /* alegreya-sans-greek-ext-400-italic */ @font-face { font-family: "Alegreya Sans"; font-style: italic; font-display: swap; size-adjust: 119%; font-weight: 400; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-greek-ext-400-italic.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-greek-ext-400-italic.woff) format("woff"); unicode-range: U+1F00-1FFF; } /* alegreya-sans-greek-400-italic */ @font-face { font-family: "Alegreya Sans"; font-style: italic; font-display: swap; size-adjust: 119%; font-weight: 400; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-greek-400-italic.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-greek-400-italic.woff) format("woff"); unicode-range: U+0370-0377, U+037A-037F, U+0384-038A, U+038C, U+038E-03A1, U+03A3-03FF; } /* alegreya-sans-vietnamese-400-italic */ @font-face { font-family: "Alegreya Sans"; font-style: italic; font-display: swap; size-adjust: 119%; font-weight: 400; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-vietnamese-400-italic.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-vietnamese-400-italic.woff) format("woff"); unicode-range: U+0102-0103, U+0110-0111, U+0128-0129, U+0168-0169, U+01A0-01A1, U+01AF-01B0, U+0300-0301, U+0303-0304, U+0308-0309, U+0323, U+0329, U+1EA0-1EF9, U+20AB; } /* alegreya-sans-latin-ext-400-italic */ @font-face { font-family: "Alegreya Sans"; font-style: italic; font-display: swap; size-adjust: 119%; font-weight: 400; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-latin-ext-400-italic.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-latin-ext-400-italic.woff) format("woff"); unicode-range: U+0100-02AF, U+0304, U+0308, U+0329, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C0, U+2113, U+2C60-2C7F, U+A720-A7FF; } /* alegreya-sans-latin-400-italic */ @font-face { font-family: "Alegreya Sans"; font-style: italic; font-display: swap; size-adjust: 119%; font-weight: 400; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-latin-400-italic.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-latin-400-italic.woff) format("woff"); unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+2074, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD; } /* alegreya-sans-cyrillic-ext-500-italic */ @font-face { font-family: "Alegreya Sans"; font-style: italic; font-display: swap; size-adjust: 119%; font-weight: 500; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-cyrillic-ext-500-italic.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-cyrillic-ext-500-italic.woff) format("woff"); unicode-range: U+0460-052F, U+1C80-1C88, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F; } /* alegreya-sans-cyrillic-500-italic */ @font-face { font-family: "Alegreya Sans"; font-style: italic; font-display: swap; size-adjust: 119%; font-weight: 500; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-cyrillic-500-italic.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-cyrillic-500-italic.woff) format("woff"); unicode-range: U+0301, U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116; } /* alegreya-sans-greek-ext-500-italic */ @font-face { font-family: "Alegreya Sans"; font-style: italic; font-display: swap; size-adjust: 119%; font-weight: 500; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-greek-ext-500-italic.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-greek-ext-500-italic.woff) format("woff"); unicode-range: U+1F00-1FFF; } /* alegreya-sans-greek-500-italic */ @font-face { font-family: "Alegreya Sans"; font-style: italic; font-display: swap; size-adjust: 119%; font-weight: 500; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-greek-500-italic.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-greek-500-italic.woff) format("woff"); unicode-range: U+0370-0377, U+037A-037F, U+0384-038A, U+038C, U+038E-03A1, U+03A3-03FF; } /* alegreya-sans-vietnamese-500-italic */ @font-face { font-family: "Alegreya Sans"; font-style: italic; font-display: swap; size-adjust: 119%; font-weight: 500; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-vietnamese-500-italic.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-vietnamese-500-italic.woff) format("woff"); unicode-range: U+0102-0103, U+0110-0111, U+0128-0129, U+0168-0169, U+01A0-01A1, U+01AF-01B0, U+0300-0301, U+0303-0304, U+0308-0309, U+0323, U+0329, U+1EA0-1EF9, U+20AB; } /* alegreya-sans-latin-ext-500-italic */ @font-face { font-family: "Alegreya Sans"; font-style: italic; font-display: swap; size-adjust: 119%; font-weight: 500; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-latin-ext-500-italic.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-latin-ext-500-italic.woff) format("woff"); unicode-range: U+0100-02AF, U+0304, U+0308, U+0329, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C0, U+2113, U+2C60-2C7F, U+A720-A7FF; } /* alegreya-sans-latin-500-italic */ @font-face { font-family: "Alegreya Sans"; font-style: italic; font-display: swap; size-adjust: 119%; font-weight: 500; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-latin-500-italic.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-latin-500-italic.woff) format("woff"); unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+2074, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD; } /* alegreya-sans-cyrillic-ext-700-italic */ @font-face { font-family: "Alegreya Sans"; font-style: italic; font-display: swap; size-adjust: 119%; font-weight: 700; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-cyrillic-ext-700-italic.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-cyrillic-ext-700-italic.woff) format("woff"); unicode-range: U+0460-052F, U+1C80-1C88, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F; } /* alegreya-sans-cyrillic-700-italic */ @font-face { font-family: "Alegreya Sans"; font-style: italic; font-display: swap; size-adjust: 119%; font-weight: 700; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-cyrillic-700-italic.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-cyrillic-700-italic.woff) format("woff"); unicode-range: U+0301, U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116; } /* alegreya-sans-greek-ext-700-italic */ @font-face { font-family: "Alegreya Sans"; font-style: italic; font-display: swap; size-adjust: 119%; font-weight: 700; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-greek-ext-700-italic.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-greek-ext-700-italic.woff) format("woff"); unicode-range: U+1F00-1FFF; } /* alegreya-sans-greek-700-italic */ @font-face { font-family: "Alegreya Sans"; font-style: italic; font-display: swap; size-adjust: 119%; font-weight: 700; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-greek-700-italic.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-greek-700-italic.woff) format("woff"); unicode-range: U+0370-0377, U+037A-037F, U+0384-038A, U+038C, U+038E-03A1, U+03A3-03FF; } /* alegreya-sans-vietnamese-700-italic */ @font-face { font-family: "Alegreya Sans"; font-style: italic; font-display: swap; size-adjust: 119%; font-weight: 700; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-vietnamese-700-italic.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-vietnamese-700-italic.woff) format("woff"); unicode-range: U+0102-0103, U+0110-0111, U+0128-0129, U+0168-0169, U+01A0-01A1, U+01AF-01B0, U+0300-0301, U+0303-0304, U+0308-0309, U+0323, U+0329, U+1EA0-1EF9, U+20AB; } /* alegreya-sans-latin-ext-700-italic */ @font-face { font-family: "Alegreya Sans"; font-style: italic; font-display: swap; size-adjust: 119%; font-weight: 700; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-latin-ext-700-italic.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-latin-ext-700-italic.woff) format("woff"); unicode-range: U+0100-02AF, U+0304, U+0308, U+0329, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C0, U+2113, U+2C60-2C7F, U+A720-A7FF; } /* alegreya-sans-latin-700-italic */ @font-face { font-family: "Alegreya Sans"; font-style: italic; font-display: swap; size-adjust: 119%; font-weight: 700; src: url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-latin-700-italic.woff2) format("woff2"), url(https://cdn.jsdelivr.net/npm/@fontsource/alegreya-sans@5.0.12/files/alegreya-sans-latin-700-italic.woff) format("woff"); unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+2074, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD; } @font-face { font-family: JuliaMono; src: url("https://cdn.jsdelivr.net/gh/cormullion/juliamono@0.060/webfonts/JuliaMono-RegularLatin.woff2") format("woff2"); font-display: swap; font-weight: 400; unicode-range: U+00-7F; /* Basic Latin characters */ } @font-face { font-family: JuliaMono; src: url("https://cdn.jsdelivr.net/gh/cormullion/juliamono@0.060/webfonts/JuliaMono-BoldLatin.woff2") format("woff2"); font-display: swap; font-weight: 700; unicode-range: U+00-7F; /* Basic Latin characters */ } @font-face { font-family: JuliaMono; src: url("https://cdn.jsdelivr.net/gh/cormullion/juliamono@0.060/webfonts/JuliaMono-Regular.woff2") format("woff2"); font-display: swap; font-weight: 400; } @font-face { font-family: JuliaMono; src: url("https://cdn.jsdelivr.net/gh/cormullion/juliamono@0.060/webfonts/JuliaMono-Bold.woff2") format("woff2"); font-display: swap; font-weight: 700; } @font-face { font-family: JuliaMono; src: url("https://cdn.jsdelivr.net/gh/cormullion/juliamono@0.060/webfonts/JuliaMono-RegularItalic.woff2") format("woff2"); font-display: swap; font-weight: 400; font-style: italic; } @font-face { font-family: Vollkorn; src: url("https://cdn.jsdelivr.net/gh/fonsp/Vollkorn-Typeface@1/fonts/woff2/Vollkorn-SemiBold.woff2") format("woff2"); font-display: swap; font-weight: 600; } @font-face { font-family: Vollkorn; font-style: italic; src: url("https://cdn.jsdelivr.net/gh/fonsp/Vollkorn-Typeface@1/fonts/woff2/Vollkorn-SemiBoldItalic.woff2") format("woff2"); font-display: swap; font-weight: 600; } @font-face { font-family: Vollkorn; src: url("https://cdn.jsdelivr.net/gh/fonsp/Vollkorn-Typeface@1/fonts/woff2/Vollkorn-Bold.woff2") format("woff2"); font-display: swap; font-weight: 700; } @font-face { font-family: Vollkorn; font-style: italic; src: url("https://cdn.jsdelivr.net/gh/fonsp/Vollkorn-Typeface@1/fonts/woff2/Vollkorn-BoldItalic.woff2") format("woff2"); font-display: swap; font-weight: 700; } @font-face { font-family: Vollkorn; src: url("https://cdn.jsdelivr.net/gh/fonsp/Vollkorn-Typeface@1/fonts/woff2/Vollkorn-Black.woff2") format("woff2"); font-display: swap; font-weight: 900; } @font-face { font-family: Vollkorn; font-style: italic; src: url("https://cdn.jsdelivr.net/gh/fonsp/Vollkorn-Typeface@1/fonts/woff2/Vollkorn-BlackItalic.woff2") format("woff2"); font-display: swap; font-weight: 900; } @media (prefers-color-scheme: dark) { :root { --image-filters: invert(1) hue-rotate(180deg) contrast(0.8); --out-of-focus-opacity: 0.5; /* Color scheme */ --main-bg-color: hsl(0deg 0% 12%); --rule-color: rgba(255, 255, 255, 0.15); --kbd-border-color: #222222; --header-bg-color: hsl(30deg 3% 16%); --header-border-color: transparent; --ui-button-color: rgb(255, 255, 255); --cursor-color: white; /* Cells */ --normal-cell: 100, 100, 100; /* --code-differs */ --error-color: 255, 125, 125; --normal-cell-color: rgba(var(--normal-cell), 0.2); --dark-normal-cell-color: rgba(var(--normal-cell), 0.4); --selected-cell-color: rgb(40 147 189 / 65%); --code-differs-cell-color: #9b906c; --error-cell-color: rgba(var(--error-color), 0.6); --bright-error-cell-color: rgba(var(--error-color), 0.9); --light-error-cell-color: rgba(var(--error-color), 0); /*Export styling*/ --export-bg-color: hsl(225deg 17% 18%); --export-color: rgb(255 255 255 / 84%); --export-card-bg-color: rgb(73 73 73); --export-card-title-color: rgba(255, 255, 255, 0.85); --export-card-text-color: rgb(255 255 255 / 70%); --export-card-shadow-color: #0000001c; /*Frontmatter styling*/ --frontmatter-button-bg-color: #554e4e; --frontmatter-outline-color: rgb(255 248 235); --frontmatter-input-bg-color: #2c2c2c; --frontmatter-input-border-color: #757575; /*Pluto output styling */ --pluto-schema-types-color: rgba(255, 255, 255, 0.6); --pluto-schema-types-border-color: rgba(255, 255, 255, 0.2); --pluto-dim-output-color: hsl(0, 0, 70%); --pluto-output-color: hsl(0deg 0% 77%); --pluto-output-h-color: hsl(0, 0%, 90%); --pluto-output-bg-color: var(--main-bg-color); --a-underline: #ffffff69; --blockquote-color: inherit; --blockquote-bg: #2e2e2e; --admonition-title-color: black; --jl-message-color: rgb(38 90 32); --jl-message-accent-color: rgb(131 191 138); --jl-info-color: rgb(42 73 115); --jl-info-accent-color: rgb(92 140 205); --jl-warn-color: rgb(96 90 34); --jl-warn-accent-color: rgb(221 212 100); --jl-danger-color: rgb(100 47 39); --jl-danger-accent-color: rgb(255, 117, 98); --jl-debug-color: hsl(288deg 33% 27%); --jl-debug-accent-color: hsl(283deg 59% 69%); /* --footnote-border-color */ --table-border-color: rgba(255, 255, 255, 0.2); --table-bg-hover-color: rgba(193, 192, 235, 0.15); --pluto-tree-color: rgb(209 207 207 / 61%); /*pluto cell styling*/ --disabled-cell-bg-color: rgba(139, 139, 139, 0.25); --selected-cell-bg-color: rgb(42 115 205 / 78%); --hover-scrollbar-color-1: rgba(0, 0, 0, 0.15); --hover-scrollbar-color-2: rgba(0, 0, 0, 0.05); --skip-as-script-background-color: #888; --depends-on-skip-as-script-background-color: #666; /* Pluto shoulders */ --shoulder-hover-bg-color: rgba(255, 255, 255, 0.05); /* Logs */ --pluto-logs-bg-color: hsl(0deg 0% 17%); --pluto-logs-key-color: rgba(255, 255, 255, 0.6); --pluto-logs-progress-fill: #5f7f5b; --pluto-logs-progress-bg: #3d3d3d; --pluto-logs-progress-border: hsl(210deg 35% 72%); --pluto-logs-info-color: #484848; --pluto-logs-info-accent-color: inherit; --pluto-logs-warn-color: rgb(80 76 38); --pluto-logs-warn-accent-color: rgb(239 231 135); --pluto-logs-danger-color: rgb(100 47 39); --pluto-logs-danger-accent-color: rgb(255 147 132); --pluto-logs-debug-color: hsl(288deg 19% 25%); --pluto-logs-debug-accent-color: hsl(283deg 56% 79%); /*Top navbar styling*/ --nav-h1-text-color: white; --nav-filepicker-color: #b6b6b6; --nav-filepicker-border-color: #c7c7c7; --nav-process-status-bg-color: rgb(82, 82, 82); --nav-process-status-color: var(--pluto-output-h-color); /*header*/ --restart-recc-header-color: rgb(44 106 157 / 56%); --restart-recc-accent-color: rgb(44 106 157); --restart-req-header-color: rgb(145 66 60 / 56%); --dead-process-header-color: rgba(250, 75, 21, 0.473); --loading-header-color: hsl(0deg 0% 20% / 50%); --disconnected-header-color: rgba(255, 169, 114, 0.56); --binder-loading-header-color: hsl(51deg 64% 90% / 50%); /*loading bar*/ --loading-grad-color-1: #a9d4f1; --loading-grad-color-2: #d0d4d7; /*saveall container*/ --overlay-button-bg: #2c2c2c; --overlay-button-border: #9e9e9e70; --overlay-button-border-save: #c7a74670; --overlay-button-color: white; /*input_context_menu*/ --input-context-menu-border-color: rgba(255, 255, 255, 0.1); --input-context-menu-bg-color: rgb(39, 40, 47); --input-context-menu-soon-color: #b1b1b144; --input-context-menu-hover-bg-color: rgba(255, 255, 255, 0.1); --input-context-menu-li-color: #c7c7c7; /* ai features */ --ai-gradient-bg: linear-gradient(20deg, #371d43, #613c35); --ai-prompt-bg: #3a3a3a; /*Pkg status*/ --pkg-popup-bg: #3d2f44; --pkg-popup-border-color: #574f56; --pkg-popup-buttons-bg-color: var(--input-context-menu-bg-color); --black: white; --white: black; --pkg-terminal-bg-color: #252627; --pkg-terminal-border-color: #c3c3c388; /* run area*/ --pluto-runarea-bg-color: rgb(43, 43, 43); --pluto-runarea-span-color: hsl(353, 5%, 64%); /*drop ruler*/ --dropruler-bg-color: rgba(255, 255, 255, 0.1); /* jlerror */ --jlerror-header-color: #d9baba; --jlerror-mark-bg-color: rgb(0 0 0 / 18%); --jlerror-a-bg-color: hsl(65.82deg 17.14% 27.45%); --jlerror-a-border-left-color: hsl(66 27% 35% / 1); --jlerror-mark-color: #b1a9a9; /* helpbox */ --helpbox-bg-color: rgb(30 34 31); --helpbox-box-shadow-color: #00000017; --helpbox-header-bg-color: #2c3e36; --helpbox-header-tab-bg-color: #554e4e; --helpbox-header-color: rgb(255 248 235); --helpbox-search-bg-color: #2c2c2c; --helpbox-search-border-color: #757575; --helpbox-notfound-search-color: rgb(139, 139, 139); --helpbox-text-color: rgb(230, 230, 230); --code-section-bg-color: rgb(44, 44, 44); --code-section-border-color: #555a64; --process-item-bg: #443d44; --process-busy: #ffcd70; --process-finished: hsl(126deg 30% 60%); --process-undefined: rgb(151, 151, 151); --process-failed: hsl(4, 30%, 60%); --process-notify-bg: hsl(0 0% 21%); /*footer*/ --footer-color: #cacaca; --footer-bg-color: var(--header-bg-color); --footer-atag-color: rgb(114, 161, 223); --footer-input-border-color: #6c6c6c; --footer-filepicker-button-color: black; --footer-filepicker-focus-color: #c1c1c1; --footnote-border-color: rgba(114, 225, 231, 0.15); /* undo delete cell*/ --undo-delete-box-shadow-color: rgb(0 0 0 / 6%); /*codemirror hints*/ --cm-color-editor-tooltip-border: rgba(0, 0, 0, 0.2); --cm-color-editor-li-aria-selected-bg: #3271e7; --cm-color-editor-li-aria-selected: white; --cm-color-editor-li-notexported: rgba(255, 255, 255, 0.5); --code-background: hsl(222deg 16% 19%); --cm-color-code-differs-gutters: rgb(235 213 28 / 11%); --cm-color-line-numbers: #8d86875e; --cm-selection-background: hsl(215deg 64% 59% / 48%); --cm-selection-background-blurred: hsl(215deg 0% 59% / 48%); --cm-highlighted: #cbceb629; /* code highlighting */ --cm-color-editor-text: oklch(90% 3% 0deg); --cm-color-comment: oklch(80% 30% 0deg); --cm-color-keyword: oklch(70% 40% 30deg); --cm-color-symbol: oklch(80% 30% 60deg); --cm-color-macro: oklch(80% 30% 150deg); --cm-color-command: oklch(80% 10% 120deg); --cm-color-string: oklch(80% 10% 180deg); --cm-color-variable: oklch(80% 10% 270deg); --cm-color-literal: oklch(80% 30% 330deg); --cm-filter-type: brightness(80%) saturate(80%); --cm-color-builtin: #5e7ad3; --cm-color-function: #f99b15; --cm-color-link: #815ba4; --cm-color-bracket: #b8ab87; --cm-color-matchingBracket: white; --cm-color-matchingBracket-bg: #c58c237a; --cm-color-placeholder-text: rgb(255 255 255 / 20%); --cm-color-clickable-underline: #5d5f70; /* Mixed parsers */ --cm-color-html: #00ab85; --cm-color-html-accent: #00e7b4; --cm-color-css: #ebd073; --cm-color-css-accent: #fffed2; --cm-css-why-doesnt-codemirror-highlight-all-the-text-aaa: #ffffea; --cm-color-md: #a2c9d5; --cm-color-md-accent: #00a9d1; /*autocomplete menu*/ --autocomplete-menu-bg-color: var(--input-context-menu-bg-color); /* Landing colors */ --index-text-color: rgb(199, 199, 199); --index-light-text-color: rgb(199, 199, 199); --index-clickable-text-color: rgb(235, 235, 235); --docs-binding-bg: #323431; --index-card-bg: #313131; --welcome-mywork-bg: var(--header-bg-color); --welcome-newnotebook-bg: rgb(68 72 102); --welcome-recentnotebook-bg: #3b3b3b; --welcome-recentnotebook-border: #6e6e6e; --welcome-open-bg: hsl(233deg 20% 33%); --welcome-card-author-backdrop: #0000006b; } @media (prefers-contrast: more) { :root { --cm-color-line-numbers: #b3b3b3; } } } @media (prefers-color-scheme: light) { :root { --image-filters: none; --out-of-focus-opacity: 0.25; /* Color scheme */ --main-bg-color: white; --rule-color: rgba(0, 0, 0, 0.15); --kbd-border-color: #dfdfdf; --header-bg-color: white; --header-border-color: rgba(0, 0, 0, 0.1); --ui-button-color: #2a2a2b; --cursor-color: black; /* Cells */ --normal-cell: 0, 0, 0; --code-differs: 160, 130, 28; --error-color: 240, 168, 168; --normal-cell-color: rgba(var(--normal-cell), 0.1); --dark-normal-cell-color: rgba(var(--normal-cell), 0.2); --selected-cell-color: rgba(40, 78, 189, 0.4); --code-differs-cell-color: rgba(var(--code-differs), 0.68); --error-cell-color: rgba(var(--error-color), 0.7); --bright-error-cell-color: rgb(var(--error-color)); --light-error-cell-color: rgba(var(--error-color), 0.05); /*Export styling*/ --export-bg-color: rgb(60, 67, 101); --export-color: rgb(228, 228, 228); --export-card-bg-color: rgba(255, 255, 255, 0.8); --export-card-title-color: rgba(0, 0, 0, 0.7); --export-card-text-color: rgba(0, 0, 0, 0.5); --export-card-shadow-color: #00000029; /*Frontmatter styling*/ --frontmatter-button-bg-color: white; --frontmatter-outline-color: hsl(230, 14%, 11%); --frontmatter-input-bg-color: #fbfbfb; --frontmatter-input-border-color: hsl(207deg 24% 87%); /*Pluto output styling */ --pluto-schema-types-color: rgba(0, 0, 0, 0.4); --pluto-schema-types-border-color: rgba(0, 0, 0, 0.2); /* --pluto-dim-output-color */ --pluto-output-color: hsl(0, 0%, 25%); --pluto-output-h-color: hsl(0, 0%, 12%); --pluto-output-bg-color: white; --a-underline: #00000059; --blockquote-color: #555; --blockquote-bg: #f2f2f2; --admonition-title-color: white; --jl-message-color: rgb(219 233 212); --jl-message-accent-color: rgb(158, 200, 137); --jl-info-color: rgb(214 227 244); --jl-info-accent-color: rgb(148, 182, 226); --jl-warn-color: rgb(236 234 213); --jl-warn-accent-color: rgb(207, 199, 138); --jl-danger-color: rgb(245 218 215); --jl-danger-accent-color: rgb(226, 157, 148); --jl-debug-color: rgb(245 218 215); --jl-debug-accent-color: rgb(226, 157, 148); --footnote-border-color: rgba(23, 115, 119, 0.15); --table-border-color: rgba(0, 0, 0, 0.2); --table-bg-hover-color: rgba(159, 158, 224, 0.15); --pluto-tree-color: rgb(0 0 0 / 38%); /*pluto cell styling*/ --disabled-cell-bg-color: rgba(139, 139, 139, 0.25); --selected-cell-bg-color: rgba(40, 78, 189, 0.24); --hover-scrollbar-color-1: rgba(0, 0, 0, 0.15); --hover-scrollbar-color-2: rgba(0, 0, 0, 0.05); --skip-as-script-background-color: #ccc; --depends-on-skip-as-script-background-color: #eee; /* Pluto shoulders */ --shoulder-hover-bg-color: rgba(0, 0, 0, 0.05); /* Logs */ --pluto-logs-bg-color: hsl(0deg 0% 98%); --pluto-logs-key-color: rgb(0 0 0 / 51%); --pluto-logs-progress-fill: #ffffff; --pluto-logs-progress-bg: #e7e7e7; --pluto-logs-progress-border: hsl(210deg 16% 74%); --pluto-logs-info-color: white; --pluto-logs-info-accent-color: inherit; --pluto-logs-warn-color: rgb(236 234 213); --pluto-logs-warn-accent-color: #665f26; --pluto-logs-danger-color: rgb(245 218 215); --pluto-logs-danger-accent-color: rgb(172 66 52); --pluto-logs-debug-color: rgb(236 223 247); --pluto-logs-debug-accent-color: rgb(100 50 179); /*Top navbar styling*/ --nav-h1-text-color: black; --nav-filepicker-color: #6f6f6f; --nav-filepicker-border-color: #b2b2b2; --nav-process-status-bg-color: white; --nav-process-status-color: var(--pluto-output-h-color); /*header*/ --restart-recc-header-color: rgba(114, 192, 255, 0.56); --restart-recc-accent-color: rgba(114, 192, 255); --restart-req-header-color: rgba(170, 41, 32, 0.56); --dead-process-header-color: rgb(230 88 46 / 38%); --loading-header-color: hsla(290, 10%, 80%, 0.5); --disconnected-header-color: rgba(255, 169, 114, 0.56); --binder-loading-header-color: hsl(51deg 64% 90% / 50%); /*loading bar*/ --loading-grad-color-1: #f1dba9; --loading-grad-color-2: #d7d7d0; /*saveall container*/ --overlay-button-bg: #ffffff; --overlay-button-border: hsl(0 4% 91% / 1); --overlay-button-border-save: #f3f2f2; /* --overlay-button-color */ /*input_context_menu*/ --input-context-menu-border-color: rgba(0, 0, 0, 0.1); --input-context-menu-bg-color: white; --input-context-menu-soon-color: #55555544; --input-context-menu-hover-bg-color: rgba(0, 0, 0, 0.1); --input-context-menu-li-color: #1e1e1e; /* ai features */ --ai-gradient-bg: linear-gradient(20deg, #f9ecff, #fff4f2); --ai-prompt-bg: #fbffff; /*Pkg status*/ --pkg-popup-bg: white; --pkg-popup-border-color: #f0e4ee; --pkg-popup-buttons-bg-color: white; --black: black; --white: white; --pkg-terminal-bg-color: #232433; --pkg-terminal-border-color: #c3c3c3; /* run area*/ --pluto-runarea-bg-color: hsl(0, 0, 97%); --pluto-runarea-span-color: hsl(353, 5%, 64%); /*drop ruler*/ --dropruler-bg-color: rgba(0, 0, 0, 0.5); /* jlerror */ --jlerror-header-color: #4f1616; --jlerror-mark-bg-color: rgb(243 243 243); --jlerror-a-bg-color: #f5efd9; --jlerror-a-border-left-color: #704141; --jlerror-mark-color: black; /* helpbox */ --helpbox-bg-color: white; --helpbox-box-shadow-color: #00000010; --helpbox-header-bg-color: hsl(0deg 0% 92%); --helpbox-header-tab-bg-color: white; --helpbox-header-color: hsl(230, 14%, 11%); --helpbox-search-bg-color: #fbfbfb; --helpbox-search-border-color: hsl(207deg 24% 87%); --helpbox-notfound-search-color: rgb(139, 139, 139); --helpbox-text-color: black; --code-section-bg-color: whitesmoke; --code-section-bg-color: #f3f3f3; /* TODO: Should this be `-border-color`? */ --process-item-bg: #f2f2f2; --process-busy: #ffcd70; --process-finished: hsl(126deg 30% 60%); --process-undefined: rgb(151, 151, 151); --process-failed: hsl(0deg 72.62% 64.6%); --process-notify-bg: hsl(44.86deg 50% 94%); /*footer*/ --footer-color: #333333; --footer-bg-color: #d7dcd3; --footer-atag-color: black; --footer-input-border-color: #818181; --footer-filepicker-button-color: white; --footer-filepicker-focus-color: #896c6c; --footnote-border-color: rgba(23, 115, 119, 0.15); /* undo delete cell*/ --undo-delete-box-shadow-color: #0000000f; /*codemirror hints*/ --cm-color-editor-tooltip-border: rgba(0, 0, 0, 0.2); --cm-color-editor-li-aria-selected-bg: #16659d; --cm-color-editor-li-aria-selected: white; --cm-color-editor-li-notexported: rgba(0, 0, 0, 0.5); --code-background: hsla(46, 90%, 98%, 1); --cm-color-code-differs-gutters: rgba(214, 172, 35, 0.2); --cm-color-line-numbers: #8d86875e; --cm-selection-background: hsl(214deg 100% 73% / 48%); --cm-selection-background-blurred: hsl(214deg 0% 73% / 48%); --cm-highlighted: #cbceb668; /* code highlighting */ --cm-color-editor-text: oklch(30% 0% 0deg); --cm-color-comment: oklch(45% 60% 0deg); --cm-color-keyword: oklch(45% 80% 30deg); --cm-color-symbol: oklch(45% 80% 90deg); --cm-color-command: oklch(35% 100% 120deg); --cm-color-macro: oklch(45% 80% 150deg); --cm-color-string: oklch(35% 100% 180deg); --cm-color-variable: oklch(45% 40% 270deg); --cm-color-literal: oklch(45% 60% 330deg); --cm-filter-type: brightness(150%) saturate(50%); --cm-color-builtin: #5e7ad3; --cm-color-function: #cc80ac; --cm-color-link: #815ba4; --cm-color-bracket: #41323f; --cm-color-matchingBracket: black; --cm-color-matchingBracket-bg: #1b4bbb21; --cm-color-placeholder-text: rgba(0, 0, 0, 0.2); --cm-color-clickable-underline: #ced2ef; /* Mixed parsers */ --cm-color-html: #48b685; --cm-color-html-accent: #00ab85; --cm-color-css: #876800; --cm-color-css-accent: #696200; --cm-css-why-doesnt-codemirror-highlight-all-the-text-aaa: #3b3700; --cm-color-md: #005a9b; --cm-color-md-accent: #00a9d1; /*autocomplete menu*/ --autocomplete-menu-bg-color: white; /* Landing colors */ --index-text-color: hsl(0, 0, 60); --index-light-text-color: #838383; --index-clickable-text-color: black; --docs-binding-bg: #8383830a; --index-card-bg: white; --welcome-mywork-bg: hsl(35deg 66% 93%); --welcome-newnotebook-bg: whitesmoke; --welcome-recentnotebook-bg: white; --welcome-recentnotebook-border: #dfdfdf; --welcome-open-bg: #fbfbfb; --welcome-card-author-backdrop: #ffffffb0; } } using Test import Pluto import Pluto: update_run!, update_save_run!, WorkspaceManager, ClientSession, ServerSession, Notebook, Cell import Malt @testset "Bonds" begin  = ServerSession() .options.evaluation.workspace_use_distributed = false @testset "Don't write to file" begin notebook = Notebook([ Cell(""" @bind x html"<input>" """), Cell("x"), ]) update_save_run!(, notebook, notebook.cells) old_mtime = mtime(notebook.path) setcode!(notebook.cells[2], "x #asdf") update_save_run!(, notebook, notebook.cells[2]) @test old_mtime != mtime(notebook.path) old_mtime = mtime(notebook.path) function set_bond_value(name, value, is_first_value=false) notebook.bonds[name] = Dict("value" => value) Pluto.set_bond_values_reactive(; session=, notebook, bound_sym_names=[name], is_first_values=[is_first_value], run_async=false, ) end set_bond_value(:x, 1, true) @test old_mtime == mtime(notebook.path) set_bond_value(:x, 2, false) @test old_mtime == mtime(notebook.path) end @testset "AbstractPlutoDingetjes.jl" begin .options.evaluation.workspace_use_distributed = true # because we use AbstractPlutoDingetjes notebook = Notebook([ # 1 Cell(""" begin import AbstractPlutoDingetjes as APD import AbstractPlutoDingetjes.Bonds end """), # 2 Cell(""" begin struct HTMLShower f::Function end Base.show(io::IO, m::MIME"text/html", hs::HTMLShower) = hs.f(io) end """), Cell(""" APD.is_inside_pluto() """), # 4 Cell(""" HTMLShower() do io write(io, string(APD.is_inside_pluto(io))) end """), Cell(""" HTMLShower() do io write(io, string(APD.is_supported_by_display(io, Bonds.initial_value))) end """), Cell(""" HTMLShower() do io write(io, string(APD.is_supported_by_display(io, Bonds.transform_value))) end """), Cell(""" HTMLShower() do io write(io, string(APD.is_supported_by_display(io, Bonds.validate_value))) end """), Cell(""" HTMLShower() do io write(io, string(APD.is_supported_by_display(io, sqrt))) end """), # 9 Cell(""" @bind x_simple html"<input type=range>" """), Cell(""" x_simple """), # 11 Cell(""" begin struct OldSlider end Base.show(io::IO, m::MIME"text/html", os::OldSlider) = write(io, "<input type=range value=1>") Base.get(os::OldSlider) = 1 end """), Cell(""" @bind x_old OldSlider() """), Cell(""" x_old """), # 14 Cell(""" begin struct NewSlider end Base.show(io::IO, m::MIME"text/html", os::NewSlider) = write(io, "<input type=range value=1>") Bonds.initial_value(os::NewSlider) = 1 Bonds.possible_values(s::NewSlider) = [1,2,3] end """), Cell(""" @bind x_new NewSlider() """), Cell(""" x_new """), # 17 Cell(""" begin struct TransformSlider end Base.show(io::IO, m::MIME"text/html", os::TransformSlider) = write(io, "<input type=range value=1>") Bonds.initial_value(os::TransformSlider) = "x" Bonds.possible_values(os::TransformSlider) = 1:10 Bonds.transform_value(os::TransformSlider, from_js) = repeat("x", from_js) end """), Cell(""" @bind x_transform TransformSlider() """), Cell(""" x_transform """), # 20 Cell(""" begin struct BadTransformSlider end Base.show(io::IO, m::MIME"text/html", os::BadTransformSlider) = write(io, "<input type=range value=1>") Bonds.initial_value(os::BadTransformSlider) = "x" Bonds.possible_values(os::BadTransformSlider) = 1:10 Bonds.transform_value(os::BadTransformSlider, from_js) = error("bad") end """), Cell(""" @bind x_badtransform BadTransformSlider() """), Cell(""" x_badtransform """), # 23 Cell(""" count = Ref(0) """), Cell(""" @bind x_counter NewSlider() #or OldSlider(), same idea """), Cell(""" let x_counter count[] += 1 end """), # 26 Cell(""" @assert x_old == 1 """), Cell(""" @assert x_new == 1 """), Cell(""" @assert x_transform == "x" """), # 29 Cell(""" begin struct PossibleValuesTest possible_values::Any end Base.show(io::IO, m::MIME"text/html", ::PossibleValuesTest) = write(io, "hello") Bonds.possible_values(pvt::PossibleValuesTest) = pvt.possible_values end """), Cell("@bind pv1 PossibleValuesTest(Bonds.NotGiven())"), Cell("@bind pv2 PossibleValuesTest(Bonds.InfinitePossibilities())"), Cell("@bind pv3 PossibleValuesTest([1,2,3])"), Cell("@bind pv4 PossibleValuesTest((x+1 for x in 1:10))"), # 34 Cell("@bind pv5 PossibleValuesTest(1:10)"), # 35 - https://github.com/fonsp/Pluto.jl/issues/2465 Cell(""), Cell("@bind ts2465 TransformSlider()"), Cell("ts2465"), ]) function set_bond_value(name, value, is_first_value=false) notebook.bonds[name] = Dict("value" => value) Pluto.set_bond_values_reactive(; session=, notebook, bound_sym_names=[name], is_first_values=[is_first_value], run_async=false, ) end # before loading AbstractPlutoDingetjes, test the default behaviour: update_run!(, notebook, notebook.cells[9:10]) @test noerror(notebook.cells[9]) @test noerror(notebook.cells[10]) @test Pluto.possible_bond_values(, notebook, :x_simple) == :NotGiven @test notebook.cells[10].output.body == "missing" set_bond_value(:x_simple, 1, true) @test notebook.cells[10].output.body == "1" update_run!(, notebook, notebook.cells) @test noerror(notebook.cells[1]) @test noerror(notebook.cells[2]) @test noerror(notebook.cells[3]) @test noerror(notebook.cells[4]) @test noerror(notebook.cells[5]) @test noerror(notebook.cells[6]) @test noerror(notebook.cells[7]) @test noerror(notebook.cells[8]) @test notebook.cells[3].output.body == "true" @test notebook.cells[4].output.body == "true" @test notebook.cells[5].output.body == "true" @test notebook.cells[6].output.body == "true" @test notebook.cells[7].output.body == "false" @test notebook.cells[8].output.body == "false" @test noerror(notebook.cells[9]) @test noerror(notebook.cells[10]) @test noerror(notebook.cells[11]) @test noerror(notebook.cells[12]) @test noerror(notebook.cells[13]) @test noerror(notebook.cells[14]) @test noerror(notebook.cells[15]) @test noerror(notebook.cells[16]) @test noerror(notebook.cells[17]) @test noerror(notebook.cells[18]) @test noerror(notebook.cells[19]) @test noerror(notebook.cells[20]) @test noerror(notebook.cells[21]) @test noerror(notebook.cells[22]) @test noerror(notebook.cells[23]) @test noerror(notebook.cells[24]) @test noerror(notebook.cells[25]) @test noerror(notebook.cells[26]) @test noerror(notebook.cells[27]) @test noerror(notebook.cells[28]) @test noerror(notebook.cells[29]) @test noerror(notebook.cells[30]) @test noerror(notebook.cells[31]) @test noerror(notebook.cells[32]) @test noerror(notebook.cells[33]) @test noerror(notebook.cells[34]) @test length(notebook.cells) == 37 @test Pluto.possible_bond_values(, notebook, :x_new) == [1,2,3] @test_throws Exception Pluto.possible_bond_values(, notebook, :asdfasdfx_new) @test Pluto.possible_bond_values(, notebook, :pv1) == :NotGiven @test Pluto.possible_bond_values(, notebook, :pv2) == :InfinitePossibilities @test Pluto.possible_bond_values(, notebook, :pv3) == [1,2,3] @test Pluto.possible_bond_values(, notebook, :pv4) == 2:11 @test Pluto.possible_bond_values(, notebook, :pv5) === 1:10 @test Pluto.possible_bond_values_length(, notebook, :pv1) == :NotGiven @test Pluto.possible_bond_values_length(, notebook, :pv2) == :InfinitePossibilities @test Pluto.possible_bond_values_length(, notebook, :pv3) == 3 @test Pluto.possible_bond_values_length(, notebook, :pv4) == 10 @test Pluto.possible_bond_values_length(, notebook, :pv5) == 10 @test notebook.cells[10].output.body == "missing" set_bond_value(:x_simple, 1, true) @test notebook.cells[10].output.body == "1" @test notebook.cells[13].output.body == "1" set_bond_value(:x_old, 1, true) @test notebook.cells[13].output.body == "1" set_bond_value(:x_old, 99, false) @test notebook.cells[13].output.body == "99" @test notebook.cells[16].output.body == "1" set_bond_value(:x_new, 1, true) @test notebook.cells[16].output.body == "1" set_bond_value(:x_new, 99, false) @test notebook.cells[16].output.body == "99" @test notebook.cells[19].output.body == "\"x\"" set_bond_value(:x_transform, 1, true) @test notebook.cells[19].output.body == "\"x\"" set_bond_value(:x_transform, 3, false) @test notebook.cells[19].output.body == "\"xxx\"" @test notebook.cells[22].output.body != "missing" @info "The following error is expected:" set_bond_value(:x_badtransform, 1, true) @test notebook.cells[22].output.body != "missing" @test notebook.cells[25].output.body == "1" set_bond_value(:x_counter, 1, true) @test notebook.cells[25].output.body == "1" set_bond_value(:x_counter, 7, false) @test notebook.cells[25].output.body == "2" # https://github.com/fonsp/Pluto.jl/issues/2465 update_run!(, notebook, notebook.cells[35:37]) @test noerror(notebook.cells[35]) @test noerror(notebook.cells[36]) @test noerror(notebook.cells[37]) @test notebook.cells[37].output.body == "\"x\"" @test isempty(notebook.cells[35].code) # this should not deregister the TransformSlider setcode!(notebook.cells[35], notebook.cells[36].code) setcode!(notebook.cells[36], "") update_run!(, notebook, notebook.cells[35:36]) @test noerror(notebook.cells[35]) @test noerror(notebook.cells[36]) @test notebook.cells[37].output.body == "\"x\"" set_bond_value(:ts2465, 2, false) @test noerror(notebook.cells[35]) @test noerror(notebook.cells[36]) @test notebook.cells[37].output.body == "\"xx\"" cleanup(, notebook) .options.evaluation.workspace_use_distributed = false # test that the notebook file is runnable: test_proc = Malt.Worker() Malt.remote_eval_wait(test_proc, quote import Pkg try Pkg.UPDATED_REGISTRY_THIS_SESSION[] = true catch; end Pkg.activate(mktempdir()) Pkg.add("AbstractPlutoDingetjes") end) @test Malt.remote_eval_fetch(test_proc, quote include($(notebook.path)) true end) Malt.stop(test_proc) end @testset "Dependent Bound Variables" begin  = ServerSession() .options.evaluation.workspace_use_distributed = true notebook = Notebook([ Cell(raw"""@bind x HTML("<input type=range min=1 max=10>")"""), Cell(raw"""@bind y HTML("<input type=range min=1 max=$(x)>")"""), Cell(raw"""x"""), #3 Cell(raw"""y"""), #4 Cell(raw""" begin struct TransformSlider range::AbstractRange end Base.show(io::IO, m::MIME"text/html", os::TransformSlider) = write(io, "<input type=range value=$(minimum(os.range)) min=$(minimum(os.range)) max=$(maximum(os.range))>") Bonds.initial_value(os::TransformSlider) = Bonds.transform_value(os, minimum(os.range)) Bonds.possible_values(os::TransformSlider) = os.range Bonds.transform_value(os::TransformSlider, from_js) = from_js * 2 end """), Cell(raw"""begin hello1 = 123 @bind a TransformSlider(1:10) end"""), Cell(raw"""begin hello2 = 234 @bind b TransformSlider(1:a) end"""), Cell(raw"""a"""), #8 Cell(raw"""b"""), #9 Cell(raw"""hello1"""), #10 Cell(raw"""hello2"""), #11 Cell(raw"""using AbstractPlutoDingetjes"""), ]) update_run!(, notebook, notebook.cells) # Test the get_bond_names function @test Pluto.get_bond_names(, notebook) == Set([:a, :b, :x, :y]) function set_bond_values!(notebook:: Notebook, bonds:: Dict; is_first_value=false) for (name, value) in bonds notebook.bonds[name] = Dict("value" => value) end Pluto.set_bond_values_reactive(; session=, notebook, bound_sym_names=collect(keys(bonds)), run_async=false, is_first_values=fill(is_first_value, length(bonds))) end @test notebook.cells[3].output.body == "missing" @test notebook.cells[4].output.body == "missing" # no initial value defined for simple html slider (in contrast to TransformSlider) @test notebook.cells[8].output.body == "2" @test notebook.cells[9].output.body == "2" @test notebook.cells[10].output.body == "123" @test notebook.cells[11].output.body == "234" set_bond_values!(notebook, Dict(:x => 1, :a => 1); is_first_value=true) @test notebook.cells[3].output.body == "1" @test notebook.cells[4].output.body == "missing" # no initial value defined for simple html slider (in contrast to TransformSlider) @test notebook.cells[8].output.body == "2" # TransformSlider scales values *2 @test notebook.cells[9].output.body == "2" @test notebook.cells[10].output.body == "123" @test notebook.cells[11].output.body == "234" set_bond_values!(notebook, Dict(:y => 1, :b => 1); is_first_value=true) @test notebook.cells[3].output.body == "1" @test notebook.cells[4].output.body == "1" @test notebook.cells[8].output.body == "2" @test notebook.cells[9].output.body == "2" @test notebook.cells[10].output.body == "123" @test notebook.cells[11].output.body == "234" set_bond_values!(notebook, Dict(:x => 5)) @test notebook.cells[3].output.body == "5" @test notebook.cells[4].output.body == "missing" # the slider object is re-defined, therefore its value is the default one set_bond_values!(notebook, Dict(:y => 3)) @test notebook.cells[3].output.body == "5" @test notebook.cells[4].output.body == "3" set_bond_values!(notebook, Dict(:x => 10, :y => 5)) @test notebook.cells[3].output.body == "10" @test notebook.cells[4].output.body == "5" # this would fail without PR #2014 - previously `y` was reset to the default value `missing` set_bond_values!(notebook, Dict(:b => 2)) @test notebook.cells[8].output.body == "2" @test notebook.cells[9].output.body == "4" @test notebook.cells[10].output.body == "123" @test notebook.cells[11].output.body == "234" set_bond_values!(notebook, Dict(:a => 8, :b => 12)) @test notebook.cells[8].output.body == "16" @test notebook.cells[9].output.body == "24" # this would fail without PR #2014 @test notebook.cells[10].output.body == "123" @test notebook.cells[11].output.body == "234" set_bond_values!(notebook, Dict(:a => 1, :b => 1)) setcode!(notebook.cells[10], "a + hello1") setcode!(notebook.cells[11], "b + hello2") update_run!(, notebook, notebook.cells[10:11]) @test notebook.cells[10].output.body == "125" @test notebook.cells[11].output.body == "236" set_bond_values!(notebook, Dict(:a => 2, :b => 2)) @test notebook.cells[10].output.body == "127" @test notebook.cells[11].output.body == "238" set_bond_values!(notebook, Dict(:b => 3)) @test notebook.cells[10].output.body == "127" @test notebook.cells[11].output.body == "240" set_bond_values!(notebook, Dict(:a => 1)) @test notebook.cells[10].output.body == "125" @test notebook.cells[11].output.body == "236" # changing a will reset b cleanup(, notebook) end end using HTTP using Test using Pluto using Pluto: ServerSession, ClientSession, SessionActions using Pluto.Configuration using Pluto.Configuration: notebook_path_suggestion, from_flat_kwargs, _convert_to_flags using Pluto.WorkspaceManager: poll import URIs @testset "Configurations" begin cd(Pluto.project_relative_path("test")) do @test notebook_path_suggestion() == joinpath(pwd(), "") end @testset "from_flat_kwargs" begin opt = from_flat_kwargs(; compile="min", launch_browser=false) @test opt.compiler.compile == "min" @test opt.server.launch_browser == false @test_throws MethodError from_flat_kwargs(; asdfasdf="test") structs_kwargs = let structs = [ Pluto.Configuration.ServerOptions, Pluto.Configuration.SecurityOptions, Pluto.Configuration.EvaluationOptions, Pluto.Configuration.CompilerOptions ] sets = [collect(fieldnames(s)) for s in structs] vcat(sets...)::Vector{Symbol} end from_flat_kwargs_kwargs = let method = only(methods(Pluto.Configuration.from_flat_kwargs)) syms = method.slot_syms names = split(syms, "\0")[2:end-1] Symbol.(names)::Vector{Symbol} end # Verify that all struct fields can be set via `from_flat_kwargs`. # Also verifies ordering to improve code readability. @test structs_kwargs == from_flat_kwargs_kwargs end @testset "flag conversion" begin reference_flags = ["--startup-file=no", "--history-file=no", "--threads=123"] @test _convert_to_flags(Configuration.CompilerOptions(threads="123")) == reference_flags @test _convert_to_flags(Configuration.CompilerOptions(threads=123)) == reference_flags @test _convert_to_flags(Configuration.CompilerOptions())  ["--startup-file=no", "--history-file=no"] @test _convert_to_flags(Configuration.CompilerOptions(compile="min"))  ["--compile=min", "--startup-file=no", "--history-file=no"] end @testset "Authentication" begin basic_nb_path = Pluto.project_relative_path("sample", "Basic.jl") port = 23832 options = Pluto.Configuration.from_flat_kwargs(; port, launch_browser=false, workspace_use_distributed=false)  = Pluto.ServerSession(; options) host = .options.server.host secret = .secret println("Launching test server...") server = Pluto.run!() local_url(suffix) = "http://$host:$port/$suffix" withsecret(url) = occursin('?', url) ? "$url&secret=$secret" : "$url?secret=$secret" function request(url, method; kwargs...) HTTP.request(method, url, nothing, method == "POST" ? read(basic_nb_path) : UInt8[]; status_exception=false, redirect=false, cookies=false, kwargs...) end function shares_secret(response) any(occursin(secret, y) for (x,y) in response.headers) end public_routes = [ ("favicon.ico", "GET"), ("possible_binder_token_please", "GET"), ("index.css", "GET"), ("index.js", "GET"), ("img/favicon-32x32.png", "GET"), ] broken_routes = [ ("../tsconfig.json", "GET"), ("/img/", "GET"), ("open.png?url=$(URIs.escapeuri("https://raw.githubusercontent.com/fonsp/Pluto.jl/v0.14.5/sample/Basic.jl"))", "GET"), ] for (suffix, method) in public_routes url = local_url(suffix) r = request(url, method) @test r.status == 200 @test !shares_secret(r) end for (suffix, method) in broken_routes url = local_url(suffix) r = request(url, method) @test r.status  400:499 @test !shares_secret(r) end notebook = SessionActions.open(, basic_nb_path; as_sample=true) simple_routes = [ ("", "GET"), ("edit?id=$(notebook.notebook_id)", "GET"), ("editor.html", "GET"), ("notebookfile?id=$(notebook.notebook_id)", "GET"), ("notebookexport?id=$(notebook.notebook_id)", "GET"), ("statefile?id=$(notebook.notebook_id)", "GET"), ] function tempcopy(x) p = tempname() Pluto.readwrite(x, p) p end @assert isfile(basic_nb_path) effect_routes = [ ("new", "GET"), ("new", "POST"), ("open?url=$(URIs.escapeuri("https://raw.githubusercontent.com/fonsp/Pluto.jl/v0.14.5/sample/Basic.jl"))", "GET"), ("open?url=$(URIs.escapeuri("https://raw.githubusercontent.com/fonsp/Pluto.jl/v0.14.5/sample/Basic.jl"))&execution_allowed=asdf", "GET"), ("open?url=$(URIs.escapeuri("https://raw.githubusercontent.com/fonsp/Pluto.jl/v0.14.5/sample/Basic.jl"))", "POST"), ("open?path=$(URIs.escapeuri(basic_nb_path |> tempcopy))", "GET"), ("open?path=$(URIs.escapeuri(basic_nb_path |> tempcopy))", "POST"), ("sample/Basic.jl", "GET"), ("sample/Basic.jl", "POST"), ("notebookupload", "POST"), ("notebookupload?execution_allowed=asdf", "POST"), ] @testset "simple & effect w/o auth $suffix $method" for (suffix, method) in simple_routes  effect_routes url = local_url(suffix) r = request(url, method) @test r.status == 403 @test !shares_secret(r) end # no notebooks were opened @test length(.notebooks) == 1 @testset "require secret only for open links" begin @test !shares_secret(request(local_url(""), "GET")) jar = HTTP.Cookies.CookieJar() # Let's test the config # require_secret_for_access = false # require_secret_for_open_links = true .options.security.require_secret_for_access = false # Effectful paths should not work without a secret. @testset "simple & effect w/o auth 1 $suffix $method" for (suffix, method) in effect_routes url = local_url(suffix) r = request(url, method; cookies=true, jar) @test r.status == 403 @test !shares_secret(r) end # With this config, the / path should work and share the secret, even when requested without a secret. r = request(local_url(""), "GET"; cookies=true, jar) @test r.status == 200 @test shares_secret(r) # Now, the other effectful paths should work bc of the secret. @testset "simple w/o auth 2 $suffix $method" for (suffix, method) in simple_routes url = local_url(suffix) r = request(url, method; cookies=true, jar) @test r.status  200:299 # 2xx is OK @test shares_secret(r) end .options.security.require_secret_for_access = true end jar = HTTP.Cookies.CookieJar() @test shares_secret(request(local_url("") |> withsecret, "GET"; cookies=true, jar)) @testset "simple w/ auth $suffix $method" for (suffix, method) in simple_routes # should work because of cookie url = local_url(suffix) r = request(url, method; cookies=true, jar) @test r.status  200:299 # 2xx is OK @test shares_secret(r) # see reasoning in of https://github.com/fonsp/Pluto.jl/commit/20515dd46678a49ca90e042fcfa3eab1e5c8e162 # Without cookies, but with secret in URL r = request(url |> withsecret, method) @test r.status  200:299 # 2xx is OK @test shares_secret(r) end @testset "effect w/ auth $suffix $method" for (suffix, method) in effect_routes old_ids = collect(keys(.notebooks)) url = local_url(suffix) |> withsecret r = request(url, method) @test r.status  200:399 # 3xx are redirects @test shares_secret(r) # see reasoning in of https://github.com/fonsp/Pluto.jl/commit/20515dd46678a49ca90e042fcfa3eab1e5c8e162 new_ids = collect(keys(.notebooks)) notebook = .notebooks[only(setdiff(new_ids, old_ids))] if any(x -> occursin(x, suffix), ["new", "execution_allowed", "sample/Basic.jl"]) @test Pluto.will_run_code(notebook) @test Pluto.will_run_pkg(notebook) else @test !Pluto.will_run_code(notebook) @test !Pluto.will_run_pkg(notebook) @test notebook.process_status === Pluto.ProcessStatus.waiting_for_permission end end close(server) end @testset "disable mimetype via workspace_custom_startup_expr" begin  = ServerSession() .options.evaluation.workspace_use_distributed = true .options.evaluation.workspace_custom_startup_expr = """ 1 + 1 PlutoRunner.is_mime_enabled(m::MIME"application/vnd.pluto.tree+object") = false """ notebook = Pluto.Notebook([ Pluto.Cell("x = [1, 2]") Pluto.Cell("struct Foo; x; end") Pluto.Cell("Foo(x)") ]) Pluto.update_run!(, notebook, notebook.cells) @test notebook.cells[1].output.body == repr(MIME"text/plain"(), [1,2]) @test notebook.cells[1].output.mime isa MIME"text/plain" @test notebook.cells[3].output.mime isa MIME"text/plain" cleanup(, notebook) end end using Test using Pluto using Pluto: update_run!, ServerSession, ClientSession, Cell, Notebook @testset "CellDepencencyVisualization" begin  = ServerSession() .options.evaluation.workspace_use_distributed = false notebook = Notebook([ Cell("x = 1"), # prerequisite of test cell Cell("f(x) = x + y"), # depends on test cell Cell("f(3)"), Cell("""begin g(a) = x g(a,b) = y end"""), # depends on test cell Cell("y = x"), # test cell below Cell("g(6) + g(6,6)"), Cell("using Dates"), ]) update_run!(, notebook, notebook.cells) state = Pluto.notebook_to_js(notebook) id(i) = notebook.cells[i].cell_id order_of(i) = findfirst(isequal(id(i)), state["cell_execution_order"]) @test order_of(7) < order_of(1) < order_of(5) < order_of(2) < order_of(3) deps = state["cell_dependencies"] @test deps[id(5)]["downstream_cells_map"] |> keys == Set(["y"]) @test deps[id(5)]["downstream_cells_map"]["y"] == [id(2), id(4)] @test deps[id(5)]["upstream_cells_map"] |> keys == Set(["x"]) @test deps[id(5)]["upstream_cells_map"]["x"] == [id(1)] # test if this also works for function definitions @test deps[id(2)]["downstream_cells_map"] |> keys == Set(["f"]) @test deps[id(2)]["downstream_cells_map"]["f"] == [id(3)] @test deps[id(2)]["upstream_cells_map"] |> keys == Set(["y", "+"]) @test deps[id(2)]["upstream_cells_map"]["y"] == [id(5)] @test deps[id(2)]["upstream_cells_map"]["+"] == [] # + function is not defined / extended in the notebook @test deps[id(1)]["precedence_heuristic"] > deps[id(7)]["precedence_heuristic"] end using Test import Pluto import Pluto: update_save_run!, WorkspaceManager, ClientSession, ServerSession, Notebook, Cell import UUIDs: UUID, uuid1 function get_unique_short_id() string(uuid1())[1:8] end function stringify_keys(d::Dict) Dict(string(k) => stringify_keys(v) for (k, v) in d) end stringify_keys(x::Any) = x import Pluto.Firebasey function await_with_timeout(check::Function, timeout::Real=60.0, interval::Real=.05) starttime = time() while !check() sleep(interval) if time() - starttime >= timeout error("Timeout after $(timeout) seconds") end end end @testset "Communication protocol" begin @testset "Functionality sweep" begin u = [uuid1() for _ in 1:100] buffer = IOBuffer() client = ClientSession(:buffery, buffer)  = ServerSession() notebook = Notebook([ Cell( u[1], "" ), ]) .notebooks[notebook.notebook_id] = notebook update_save_run!(, notebook, notebook.cells) client.connected_notebook = notebook read(buffer) local_state = Pluto.notebook_to_js(notebook) function send(type, body, metadata = Dict(:notebook_id => string(notebook.notebook_id))) request_id = get_unique_short_id() Pluto.process_ws_message(, Dict( "type" => string(type), "client_id" => string(client.id), "request_id" => request_id, "body" => body, metadata... ) |> stringify_keys, client.stream) end function send_new_state(new_state) patches::Array{Dict} = Firebasey.diff(new_state, local_state) # @info "patches" patches send(:update_notebook, Dict("updates" => patches)) new_state end # function update_local_notebook(mutate_fn::Function) # mutable_notebook = deepcopy(notebook) # mutate_fn(mutable_notebook) # new_state = Pluto.notebook_to_js(mutable_notebook) # send_new_state(new_state) # end last_position = position(buffer) function wait_for_updates(process=true) # @info "Waiting for updates" await_with_timeout() do position(buffer) != last_position end seek(buffer, last_position) response = Pluto.unpack(buffer) last_position = position(buffer) if process if response["type"] == "notebook_diff" message = response["message"] patches = [Base.convert(Firebasey.JSONPatch, update) for update in message["patches"]] # @show patches for patch in patches Firebasey.applypatch!(local_state, patch) end end end # @info "Response received" response local_state end # function patch_and_check(mutate_fn::Function) # desired = update_local_notebook(mutate_fn) # result = wait_for_updates() # desired == result # end @test_nowarn send(:connect, Dict()) wait_for_updates() @test_nowarn send_new_state(local_state) wait_for_updates(false) read(buffer) #= We would also like to test: - add cell - set code and run - fold cell - move cell - delete cell - run multiple cells - move cells - set bond - move notebook file - search for docs - show more items of an array =# @testset "Extension of response" begin Pluto.responses[:custom_response] = function (::Pluto.ClientRequest) return "Yes this can be extended!" end @test send(:custom_response, Dict()) == "Yes this can be extended!" end @test_nowarn send(:shutdown_notebook, Dict("keep_in_session" => false)) @test_nowarn await_with_timeout() do !haskey(.notebooks, notebook.notebook_id) end end @testset "Docs" begin @test occursin("square root", Pluto.PlutoRunner.doc_fetcher("sqrt", Main)[1]) @test occursin("square root", Pluto.PlutoRunner.doc_fetcher("Base.sqrt", Main)[1]) @test occursin("Functions are defined", Pluto.PlutoRunner.doc_fetcher("function", Main)[1]) @test occursin("Within a module", Pluto.PlutoRunner.doc_fetcher("module", Main)[1]) @test occursin("No documentation found", Pluto.PlutoRunner.doc_fetcher("Base.findmeta", Main)[1]) let doc_output = Pluto.PlutoRunner.doc_fetcher("sor", Main)[1] @test occursin("Similar results:", doc_output) @test occursin("<b>s</b><b>o</b><b>r</b>t", doc_output) end @test occursin("\\div", Pluto.PlutoRunner.doc_fetcher("", Main)[1]) @test occursin("\\gamma", Pluto.PlutoRunner.doc_fetcher("", Main)[1]) let # the expression is not valid, so this doc fetch fails doc_output, result = Pluto.PlutoRunner.doc_fetcher("\"", Main) @test isnothing(doc_output) @test result == : end # Issue #1128 # Ref https://docs.julialang.org/en/v1/manual/documentation/#Dynamic-documentation m = Module() Core.eval(m, :( module DocTest "Normal docstring" struct MyType value::String end Docs.getdoc(t::MyType) = "Documentation for MyType with value $(t.value)" const x = MyType("x") "A global variable" global y end )) @test occursin("Normal docstring", Pluto.PlutoRunner.doc_fetcher("MyType", m.DocTest)[1]) @test occursin("Normal docstring", Pluto.PlutoRunner.doc_fetcher("DocTest.MyType", m)[1]) @test occursin("Documentation for MyType with value", Pluto.PlutoRunner.doc_fetcher("x", m.DocTest)[1]) @test occursin("Documentation for MyType with value", Pluto.PlutoRunner.doc_fetcher("DocTest.x", m)[1]) @test occursin("A global variable", Pluto.PlutoRunner.doc_fetcher("y", m.DocTest)[1]) @test occursin("A global variable", Pluto.PlutoRunner.doc_fetcher("DocTest.y", m)[1]) end @testset "PlutoRunner API" begin  = ServerSession() # .options.evaluation.workspace_use_distributed = true cid = uuid1() notebook = Notebook([ Cell("PlutoRunner.notebook_id[] |> Text"), # These cells tests `core_published_to_js`, which is the function used by the official API (`AbtractPlutoDingetjes.Display.published_to_js`). Cell(cid, """ begin a = Dict( "hello" => "world", "xx" => UInt8[6,7,8], ) b = "cool" struct ZZZ x y end function Base.show(io::IO, ::MIME"text/html", z::ZZZ) write(io, "<script>\n") PlutoRunner.core_published_to_js(io, z.x) PlutoRunner.core_published_to_js(io, z.y) write(io, "\n</script>") end ZZZ(a, b) end """), Cell(""" begin struct ABC x end ZZZ( 123, Dict("a" => 234, "b" => ABC(4)), ) end """), # This is the deprecated API: Cell("PlutoRunner.publish_to_js(Ref(4))"), Cell("PlutoRunner.publish_to_js((ref=5,))"), Cell("x = Dict(:a => 6)"), Cell("PlutoRunner.publish_to_js(x)"), ]) update_save_run!(, notebook, notebook.cells) @test notebook.cells[1].output.body == notebook.notebook_id |> string @test notebook.cells[2] |> noerror @test notebook.cells[2].output.mime isa MIME"text/html" ab1, ab2 = keys(notebook.cells[2].published_objects) @test occursin(ab1, notebook.cells[2].output.body) @test occursin(ab2, notebook.cells[2].output.body) ab() = sort(collect(keys(notebook.cells[2].published_objects)); by=(s -> findfirst(s, notebook.cells[2].output.body) |> first)) a, b = ab() p = notebook.cells[2].published_objects @test p[a] == Dict( "hello" => "world", "xx" => UInt8[6,7,8], ) @test p[b] == "cool" old_pa = p[a] old_pb = p[b] update_save_run!(, notebook, notebook.cells) p = notebook.cells[2].published_objects a, b = ab() @test p[a] == old_pa @test p[b] == old_pb @test !isempty(notebook.cells[2].published_objects) # display should have failed @test only(values(notebook.cells[3].published_objects)) == 123 msg = notebook.cells[3].output.body[:msg] @test occursin("Failed to show value", msg) @test occursin("ABC is not compatible", msg) setcode!(notebook.cells[2], "2") update_save_run!(, notebook, notebook.cells) @test isempty(notebook.cells[2].published_objects) @test isempty(notebook.cells[2].published_objects) @test notebook.cells[4].errored @test notebook.cells[5] |> noerror @test !isempty(notebook.cells[5].published_objects) p = notebook.cells[7].published_objects @test length(p) == 1 old_x = values(p) |> first @test old_x == Dict(:a => 6) update_save_run!(, notebook, notebook.cells[7]) p = notebook.cells[7].published_objects new_x = values(p) |> first @test new_x == old_x @test new_x === old_x # did not change, because we don't resync the same object update_save_run!(, notebook, notebook.cells[6]) p = notebook.cells[7].published_objects new_x = values(p) |> first @test new_x == old_x @test new_x !== old_x # changed, because a new (mutable) Dict was created @test isempty(notebook.cells[2].published_objects) @test !isempty(notebook.cells[5].published_objects) cleanup(, notebook) end end using Test import Pluto: Notebook, ServerSession, ClientSession, Cell, load_notebook, load_notebook_nobackup, save_notebook, WorkspaceManager, cutename, numbered_until_new, readwrite, without_pluto_file_extension, PlutoEvent, update_run! import Random import Pkg import UUIDs: UUID @testset "Private API stability for extended Pluto deployments" begin events = [] function test_listener(a::PlutoEvent) # @info "this run!" push!(events, typeof(a)) end  = ServerSession() .options.server.on_event = test_listener .options.evaluation.workspace_use_distributed = false notebook = Notebook([ Cell("[1,1,[1]]"), Cell("Dict(:a => [:b, :c])"), ]) update_run!(, notebook, notebook.cells) WorkspaceManager.unmake_workspace((, notebook); verbose=false) @test_broken events[1:3] == ["NewNotebookEvent", "OpenNotebookEvent" , "FileSaveEvent"] # Pluto.CustomLaunchEvent: Gets fired # Pluto.NewNotebookEvent: Gets fired # Pluto.OpenNotebookEvent: Gets fired # Pluto.FileSaveEvent: Gets fired # Pluto.responses[:juliahub_initiate] = function (::Pluto.ClientRequest) EXTEND end # Pluto.SessionActions.open(session, string(jhnb_path); notebook_id = UUID(jhub_params[:id]),) # Pluto.cutename(): returns string # Pluto.save_notebook(io::IOBuffer, notebook): saves notebook to IO # Pluto.ServerSession(;options, event_listener) end const ObjectID = typeof(objectid("hello computer")) function Base.show(io::IO, s::SymbolsState) print(io, "SymbolsState([") join(io, s.references, ", ") print(io, "], [") join(io, s.assignments, ", ") print(io, "], [") join(io, s.funccalls, ", ") print(io, "], [") if isempty(s.funcdefs) print(io, "]") else println(io) for (k, v) in s.funcdefs print(io, " ", k, ": ", v) println(io) end print(io, "]") end if !isempty(s.macrocalls) print(io, "], [") print(io, s.macrocalls) print(io, "])") else print(io, ")") end end "Calls `ExpressionExplorer.compute_symbolreferences` on the given `expr` and test the found SymbolsState against a given one, with convient syntax. # Example ```jldoctest julia> @test testee(:( begin a = b + 1 f(x) = x / z end), [:b, :+], # 1st: expected references [:a, :f], # 2nd: expected definitions [:+], # 3rd: expected function calls [ :f => ([:z, :/], [], [:/], []) ]) # 4th: expected function definitions, with inner symstate using the same syntax true ``` " function testee(expr::Any, expected_references, expected_definitions, expected_funccalls, expected_funcdefs, expected_macrocalls = []; verbose::Bool=true, transformer::Function=identify) expected = easy_symstate(expected_references, expected_definitions, expected_funccalls, expected_funcdefs, expected_macrocalls) expr_transformed = transformer(expr) original_hash = expr_hash(expr_transformed) result = ExpressionExplorer.compute_symbolreferences(expr_transformed) # should not throw: ReactiveNode(result) new_hash = expr_hash(expr_transformed) if original_hash != new_hash error("\n== The expression explorer modified the expression. Don't do that! ==\n") end # Anonymous function are given a random name, which looks like __ExprExpl_anon__67387237861123 # To make testing easier, we rename all such functions to anon new_name(fn::FunctionName) = FunctionName(map(new_name, fn.parts)...) new_name(sym::Symbol) = startswith(string(sym), "__ExprExpl_anon__") ? :anon : sym result.assignments = Set(new_name.(result.assignments)) result.funcdefs = let newfuncdefs = Dict{FunctionNameSignaturePair,SymbolsState}() for (k, v) in result.funcdefs union!(newfuncdefs, Dict(FunctionNameSignaturePair(new_name(k.name), hash("hello")) => v)) end newfuncdefs end if verbose && expected != result println() println("FAILED TEST") println(expr) println() dump(expr, maxdepth=20) println() dump(expr_transformed, maxdepth=20) println() @show expected resulted = result @show resulted println() end return expected == result end expr_hash(e::Expr) = objectid(e.head) + mapreduce(p -> objectid((p[1], expr_hash(p[2]))), +, enumerate(e.args); init=zero(ObjectID)) expr_hash(x) = objectid(x) function easy_symstate(expected_references, expected_definitions, expected_funccalls, expected_funcdefs, expected_macrocalls = []) array_to_set(array) = map(array) do k new_k = FunctionName(k) return new_k end |> Set new_expected_funccalls = array_to_set(expected_funccalls) new_expected_funcdefs = map(expected_funcdefs) do (k, v) new_k = FunctionName(k) new_v = v isa SymbolsState ? v : easy_symstate(v...) return FunctionNameSignaturePair(new_k, hash("hello")) => new_v end |> Dict new_expected_macrocalls = array_to_set(expected_macrocalls) SymbolsState(Set(expected_references), Set(expected_definitions), new_expected_funccalls, new_expected_funcdefs, new_expected_macrocalls) end t(args...; kwargs...) = testee(args...; transformer=Pluto.ExpressionExplorerExtras.pretransform_pluto, kwargs...) """ Like `t` but actually a convenient syntax """ function test_expression_explorer(; expr, references=[], definitions=[], funccalls=[], funcdefs=[], macrocalls=[], kwargs...) t(expr, references, definitions, funccalls, funcdefs, macrocalls; kwargs...) end @testset "Macros w/ Pluto 1" begin # Macros tests are not just in ExpressionExplorer now @test t(:(@time a = 2), [], [], [], [], [Symbol("@time")]) @test t(:(@f(x; y=z)), [], [], [], [], [Symbol("@f")]) @test t(:(@f(x, y = z)), [], [], [], [], [Symbol("@f")]) # https://github.com/fonsp/Pluto.jl/issues/252 @test t(:(Base.@time a = 2), [], [], [], [], [[:Base, Symbol("@time")]]) # @test_nowarn t(:(@enum a b = d c), [:d], [:a, :b, :c], [Symbol("@enum")], []) # @enum is tested in test/React.jl instead @test t(:(@gensym a b c), [], [:a, :b, :c], [:gensym], [], [Symbol("@gensym")]) @test t(:(Base.@gensym a b c), [], [:a, :b, :c], [:gensym], [], [[:Base, Symbol("@gensym")]]) @test t(:(Base.@kwdef struct A; x = 1; y::Int = two; z end), [], [], [], [], [[:Base, Symbol("@kwdef")]]) # @test t(quote "asdf" f(x) = x end, [], [], [], [], [Symbol("@doc")]) # @test t(:(@bind a b), [], [], [], [], [Symbol("@bind")]) # @test t(:(PlutoRunner.@bind a b), [], [], [], [], [[:PlutoRunner, Symbol("@bind")]]) # @test_broken t(:(Main.PlutoRunner.@bind a b), [:b], [:a], [[:Base, :get], [:Core, :applicable], [:PlutoRunner, :create_bond], [:PlutoRunner, Symbol("@bind")]], [], verbose=false) # @test t(:(let @bind a b end), [], [], [], [], [Symbol("@bind")]) @test t(:(`hey $(a = 1) $(b)`), [:b], [], [:cmd_gen], [], [Symbol("@cmd")]) # @test t(:(md"hey $(@bind a b) $(a)"), [:a], [], [[:getindex]], [], [Symbol("@md_str"), Symbol("@bind")]) # @test t(:(md"hey $(a) $(@bind a b)"), [:a], [], [[:getindex]], [], [Symbol("@md_str"), Symbol("@bind")]) @test t(:(@asdf a = x1 b = x2 c = x3), [], [], [], [], [Symbol("@asdf")]) # https://github.com/fonsp/Pluto.jl/issues/670 @test t(:(@aa @bb xxx), [], [], [], [], [Symbol("@aa"), Symbol("@bb")]) @test t(:(@aa @bb(xxx) @cc(yyy)), [], [], [], [], [Symbol("@aa"), Symbol("@bb"), Symbol("@cc")]) @test t(:(Pkg.activate()), [:Pkg], [], [[:Pkg,:activate]], [], []) @test t(:(@aa(Pkg.activate())), [:Pkg], [], [[:Pkg,:activate]], [], [Symbol("@aa")]) @test t(:(@aa @bb(Pkg.activate())), [:Pkg], [], [[:Pkg,:activate]], [], [Symbol("@aa"), Symbol("@bb")]) @test t(:(@aa @assert @bb(Pkg.activate())), [:Pkg], [], [[:Pkg,:activate], [:throw], [:AssertionError]], [], [Symbol("@aa"), Symbol("@assert"), Symbol("@bb")]) @test t(:(@aa @bb(Xxx.xxxxxxxx())), [], [], [], [], [Symbol("@aa"), Symbol("@bb")]) @test t(:(include()), [], [], [[:include]], [], []) @test t(:(:(include())), [], [], [], [], []) @test t(:(:($(include()))), [], [], [[:include]], [], []) @test t(:(@xx include()), [], [], [[:include]], [], [Symbol("@xx")]) @test t(quote module A include() Pkg.activate() @xoxo asdf end end, [], [:A], [], [], []) @test t(:(@aa @bb(using Zozo)), [], [:Zozo], [], [], [Symbol("@aa"), Symbol("@bb")]) @test t(:(@aa(using Zozo)), [], [:Zozo], [], [], [Symbol("@aa")]) @test t(:(using Zozo), [], [:Zozo], [], [], []) e = :(using Zozo) @test ExpressionExplorer.compute_usings_imports( e ).usings == [e] @test ExpressionExplorer.compute_usings_imports( :(@aa @bb($e)) ).usings == [e] @test t(:(@einsum a[i,j] := x[i]*y[j]), [], [], [], [], [Symbol("@einsum")]) @test t(:(@tullio a := f(x)[i+2j, k[j]] init=z), [], [], [], [], [Symbol("@tullio")]) @test t(:(Pack.@asdf a[1,k[j]] := log(x[i]/y[j])), [], [], [], [], [[:Pack, Symbol("@asdf")]]) @test t(:(html"a $(b = c)"), [], [], [], [], [Symbol("@html_str")]) @test t(:(md"a $(b = c) $(b)"), [:c], [:b], [:getindex], [], [Symbol("@md_str")]) @test t(:(md"\* $r"), [:r], [], [:getindex], [], [Symbol("@md_str")]) @test t(:(md"a \$(b = c)"), [], [], [:getindex], [], [Symbol("@md_str")]) @test t(:(macro a() end), [], [], [], [ Symbol("@a") => ([], [], [], []) ]) @test t(:(macro a(b::Int); b end), [], [], [], [ Symbol("@a") => ([:Int], [], [], []) ]) @test t(:(macro a(b::Int=c) end), [], [], [], [ Symbol("@a") => ([:Int, :c], [], [], []) ]) @test t(:(macro a(); b = c; return b end), [], [], [], [ Symbol("@a") => ([:c], [], [], []) ]) @test test_expression_explorer(; expr=:(@parent @child 10), macrocalls=[Symbol("@parent"), Symbol("@child")], ) @test test_expression_explorer(; expr=:(@parent begin @child 1 + @grandchild 10 end), macrocalls=[Symbol("@parent"), Symbol("@child"), Symbol("@grandchild")], ) @test t(macroexpand(Main, :(@noinline f(x) = x)), [], [], [], [ Symbol("f") => ([], [], [], []) ]) end @testset "Macros w/ Pluto 2" begin @test t(:(@bind a b), [:b, :PlutoRunner, :Base, :Core], [:a], [[:PlutoRunner, :create_bond], [:Core, :applicable], [:Base, :get]], [], [Symbol("@bind")]) @test t(:(PlutoRunner.@bind a b), [:b, :PlutoRunner, :Base, :Core], [:a], [[:PlutoRunner, :create_bond], [:Core, :applicable], [:Base, :get]], [], [[:PlutoRunner, Symbol("@bind")]]) @test_broken t(:(Main.PlutoRunner.@bind a b), [:b, :PlutoRunner, :Base, :Core], [:a], [[:Base, :get], [:Core, :applicable], [:PlutoRunner, :create_bond], [:PlutoRunner, Symbol("@bind")]], [], verbose=false) @test t(:(let @bind a b end), [:b, :PlutoRunner, :Base, :Core], [:a], [[:PlutoRunner, :create_bond], [:Core, :applicable], [:Base, :get]], [], [Symbol("@bind")]) @test t(:(`hey $(a = 1) $(b)`), [:b], [], [:cmd_gen], [], [Symbol("@cmd")]) @test t(:(md"hey $(@bind a b) $(a)"), [:b, :PlutoRunner, :Base, :Core], [:a], [[:PlutoRunner, :create_bond], [:Core, :applicable], [:Base, :get], :getindex], [], [Symbol("@md_str"), Symbol("@bind")]) @test t(:(md"hey $(a) $(@bind a b)"), [:a, :b, :PlutoRunner, :Base, :Core], [:a], [[:PlutoRunner, :create_bond], [:Core, :applicable], [:Base, :get], :getindex], [], [Symbol("@md_str"), Symbol("@bind")]) end using Test import Pluto: ServerSession, update_run!, WorkspaceManager @testset "Test Firebasey" begin  = ServerSession() file = tempname() write(file, read(normpath(Pluto.project_relative_path("src", "webserver", "Firebasey.jl")))) notebook = Pluto.load_notebook_nobackup(file) update_run!(, notebook, notebook.cells) # Test that the resulting file is runnable @test jl_is_runnable(file) # and also that Pluto can figure out the execution order on its own @test all(noerror, notebook.cells) cleanup(, notebook) end using Test import UUIDs import Pluto: PlutoRunner, Notebook, WorkspaceManager, Cell, ServerSession, ClientSession, update_run! using Pluto.WorkspaceManager: poll @testset "Logging" begin  = ServerSession() .options.evaluation.workspace_use_distributed = true notebook = Notebook(Cell.([ "println(123)", "println(stdout, 123)", "println(stderr, 123)", "display(123)", "show(123)", "popdisplay()", "println(123)", "pushdisplay(TextDisplay(devnull))", "print(12); print(3)", # 9 """ for i in 1:10 @info "logging" i maxlog=2 end """, # 10 """ for i in 1:10 @info "logging" i maxlog=2 @info "logging more" maxlog = 4 @info "even more logging" end """, # 11 "t1 = @async sleep(3)", # 12 "!istaskfailed(t1) && !istaskdone(t1)", # 13 "t2 = @async run(`sleep 3`)", # 14 "!istaskfailed(t2) && !istaskdone(t2)", # 15 """ macro hello() a = rand() @info a nothing end """, # 16 "@hello", # 17 "123", # 18 "struct StructWithCustomShowThatLogs end", # 19 """ # 20 function Base.show(io::IO, ::StructWithCustomShowThatLogs) println("stdio log") @info "showing StructWithCustomShowThatLogs" show(io, "hello") end """, "StructWithCustomShowThatLogs()", # 21 """ printstyled(stdout, "hello", color=:red) """, # 22 "show(collect(1:500))", # 23 "show(stdout, collect(1:500))", # 24 "show(stdout, \"text/plain\", collect(1:500))", # 25 "display(collect(1:500))", # 26 "struct StructThatErrorsOnPrinting end", # 27 """ Base.print(::IO, ::StructThatErrorsOnPrinting) = error("Can't print this") """, # 28 """ @info "" _id=StructThatErrorsOnPrinting() """, # 29 ])) @testset "Stdout" begin idx_123 = [1,2,3,4,5,7,9] update_run!(, notebook, notebook.cells[1:9]) @test notebook.cells[1] |> noerror @test notebook.cells[2] |> noerror @test notebook.cells[3] |> noerror @test notebook.cells[4] |> noerror @test notebook.cells[5] |> noerror @test notebook.cells[6] |> noerror @test notebook.cells[7] |> noerror @test notebook.cells[8] |> noerror @test notebook.cells[9] |> noerror @test poll(5, 1/60) do all(notebook.cells[idx_123]) do c length(c.logs) == 1 end end @testset "123 - $(i)" for i in idx_123 log = only(notebook.cells[i].logs) @test log["level"] == "LogLevel(-555)" @test strip(log["msg"][1]) == "123" @test log["msg"][2] == MIME"text/plain"() end update_run!(, notebook, notebook.cells[12:15]) update_run!(, notebook, notebook.cells[[12,14]]) @test notebook.cells[13].output.body == "true" Sys.iswindows() || @test notebook.cells[15].output.body == "true" update_run!(, notebook, notebook.cells[16:18]) @test isempty(notebook.cells[16].logs) @test length(notebook.cells[17].logs) == 1 @test isempty(notebook.cells[18].logs) update_run!(, notebook, notebook.cells[18]) update_run!(, notebook, notebook.cells[17]) @test isempty(notebook.cells[16].logs) @test length(notebook.cells[17].logs) == 1 @test isempty(notebook.cells[18].logs) update_run!(, notebook, notebook.cells[16]) @test isempty(notebook.cells[16].logs) @test length(notebook.cells[17].logs) == 1 @test isempty(notebook.cells[18].logs) update_run!(, notebook, notebook.cells[19:21]) @test isempty(notebook.cells[19].logs) @test isempty(notebook.cells[20].logs) @test poll(5, 1/60) do length(notebook.cells[21].logs) == 2 end end @testset "ANSI Color Output" begin update_run!(, notebook, notebook.cells[22]) msg = only(notebook.cells[22].logs)["msg"][1] @test startswith(msg, Base.text_colors[:red]) @test endswith(msg, Base.text_colors[:default]) end @testset "show(...) and display(...) behavior" begin update_run!(, notebook, notebook.cells[23:25]) msgs_show = [only(cell.logs)["msg"][1] for cell in notebook.cells[23:25]] # `show` should show a middle element of the big array for msg in msgs_show @test contains(msg, "1") && contains(msg, "500") @test contains(msg, "250") end update_run!(, notebook, notebook.cells[26]) msg_display = only(notebook.cells[26].logs)["msg"][1] # `display` should not display the middle element of the big array @test contains(msg_display, "1") && contains(msg_display, "500") @test !contains(msg_display, "250") end @testset "Logging respects maxlog" begin @testset "Single log" begin update_run!(, notebook, notebook.cells[10]) @test notebook.cells[10] |> noerror @test poll(5, 1/60) do length(notebook.cells[10].logs) == 2 end # Check that maxlog doesn't occur in the message @test all(notebook.cells[10].logs) do log all(log["kwargs"]) do kwarg kwarg[1] != "maxlog" end end end @testset "Multiple log" begin update_run!(, notebook, notebook.cells[11]) @test notebook.cells[11] |> noerror # Wait until all 16 logs are in @test poll(5, 1/60) do length(notebook.cells[11].logs) == 16 end # Get the ids of the three logs and their counts. We are # assuming that the logs are ordered same as in the loop. ids = unique(getindex.(notebook.cells[11].logs, "id")) counts = [count(log -> log["id"] == id, notebook.cells[11].logs) for id in ids] @test counts == [2, 4, 10] # Check that maxlog doesn't occur in the messages @test all(notebook.cells[11].logs) do log all(log["kwargs"]) do kwarg kwarg[1] != "maxlog" end end end end cleanup(, notebook) @testset "Logging error fallback" begin # This testset needs to use a local worker to capture the worker stderr (which is # different from the notebook stderr)  = ServerSession() .options.evaluation.workspace_use_distributed = false io = IOBuffer() old_stderr = PlutoRunner.original_stderr[] PlutoRunner.original_stderr[] = io update_run!(, notebook, notebook.cells[27:29]) msg = String(take!(io)) close(io) PlutoRunner.original_stderr[] = old_stderr @test notebook.cells[27] |> noerror @test notebook.cells[28] |> noerror @test notebook.cells[29] |> noerror @test occursin("Failed to relay log from PlutoRunner", msg) cleanup(, notebook) end end using Test import UUIDs import Pluto: PlutoRunner, Notebook, WorkspaceManager, Cell, ServerSession, ClientSession, update_run! import Memoize: @memoize @testset "Macro analysis" begin  = ServerSession() .options.evaluation.workspace_use_distributed = false @testset "Base macro call" begin notebook = Notebook([ Cell("@enum Fruit  "), Cell("my_fruit = "), Cell("jam(fruit::Fruit) = cook(fruit)"), ]) cell(idx) = notebook.cells[idx] update_run!(, notebook, notebook.cells) @test cell(1) |> noerror @test [:, :]  notebook.topology.nodes[cell(1)].definitions @test :Fruit  notebook.topology.nodes[cell(1)].funcdefs_without_signatures @test Symbol("@enum")  notebook.topology.nodes[cell(1)].references @test cell(2) |> noerror @test :  notebook.topology.nodes[cell(2)].references @test cell(3) |> noerror @test :Fruit  notebook.topology.nodes[cell(3)].references cleanup(, notebook) end @testset "User defined macro 1" begin notebook = Notebook([ Cell("""macro my_assign(sym, val) :(\$(esc(sym)) = \$(val)) end"""), Cell("@my_assign x 1+1"), ]) cell(idx) = notebook.cells[idx] update_run!(, notebook, notebook.cells) @test :x  notebook.topology.nodes[cell(2)].definitions @test Symbol("@my_assign")  notebook.topology.nodes[cell(2)].references update_run!(, notebook, notebook.cells) # Works on second time because of old workspace @test :x  notebook.topology.nodes[cell(2)].definitions @test Symbol("@my_assign")  notebook.topology.nodes[cell(2)].references cleanup(, notebook) end @testset "User defined macro 2" begin notebook = Notebook([ Cell("@my_identity(f(123))"), Cell(""), Cell(""), ]) cell(idx) = notebook.cells[idx] update_run!(, notebook, notebook.cells) setcode!(cell(2), """macro my_identity(expr) esc(expr) end""") update_run!(, notebook, cell(2)) setcode!(cell(3), "f(x) = x") update_run!(, notebook, cell(3)) @test cell(1) |> noerror @test cell(2) |> noerror @test cell(3) |> noerror @test cell(1).output.body == "123" update_run!(, notebook, cell(1)) @test cell(1) |> noerror @test cell(2) |> noerror @test cell(3) |> noerror cleanup(, notebook) end @testset "User defined macro 3" begin notebook = Notebook([ Cell(""" macro mymap() quote [1, 2, 3] .|> sqrt end end """), Cell("@mymap") ]) cell(idx) = notebook.cells[idx] update_run!(, notebook, notebook.cells) @test cell(1) |> noerror @test cell(2) |> noerror update_run!(, notebook, cell(1)) @test cell(2) |> noerror cleanup(, notebook) end @testset "User defined macro 4" begin notebook = Notebook([ Cell("""macro my_assign(ex) esc(ex) end"""), Cell("@macroexpand @my_assign 1+1"), ]) cell(idx) = notebook.cells[idx] update_run!(, notebook, notebook.cells) @test Symbol("@my_assign")  notebook.topology.nodes[cell(2)].references cleanup(, notebook) end @testset "User defined macro 5" begin notebook = Notebook([ Cell("""macro dynamic_values(ex) [:a, :b, :c] end"""), Cell("myarray = @dynamic_values()"), ]) references(idx) = notebook.topology.nodes[notebook.cells[idx]].references update_run!(, notebook, notebook.cells) @test :a  references(2) @test :b  references(2) @test :c  references(2) cleanup(, notebook) end @testset "User defined macro 6" begin notebook = Notebook([ Cell("""macro my_macro() esc(:(y + x)) end"""), Cell("""function my_function() @my_macro() end"""), Cell("my_function()"), Cell("x = 1"), Cell("y = 2"), ]) cell(idx) = notebook.cells[idx] update_run!(, notebook, notebook.cells) @test [Symbol("@my_macro"), :x, :y]  notebook.topology.nodes[cell(2)].references @test cell(3).output.body == "3" cleanup(, notebook) end @testset "Function docs" begin notebook = Notebook([ Cell(""" "my function doc" f(x) = 2x """), Cell("f"), ]) cell(idx) = notebook.cells[idx] temp_topology = Pluto.updated_topology(notebook.topology, notebook, notebook.cells) |> Pluto.static_resolve_topology # @test :f  temp_topology.nodes[cell(1)].funcdefs_without_signatures update_run!(, notebook, notebook.cells) @test :f  notebook.topology.nodes[cell(1)].funcdefs_without_signatures @test :f  notebook.topology.nodes[cell(2)].references cleanup(, notebook) end @testset "Expr sanitization" begin struct A; end f(x) = x unserializable_expr = :($(f)(A(), A[A(), A(), A()], PlutoRunner, PlutoRunner.sanitize_expr)) get_expr_types(other) = typeof(other) get_expr_types(ex::Expr) = get_expr_types.(ex.args) flatten(x, acc=[]) = push!(acc, x) function flatten(arr::AbstractVector, acc=[]) foreach(x -> flatten(x, acc), arr); acc end sanitized_expr = PlutoRunner.sanitize_expr(unserializable_expr) types = sanitized_expr |> get_expr_types |> flatten |> Set # Checks that no fancy type is part of the serialized expression @test Set([Nothing, Symbol, QuoteNode]) == types end @testset "Macrodef cells not root of run" begin notebook = Notebook([ Cell(""), Cell(""), Cell(""), ]) cell(idx) = notebook.cells[idx] update_run!(, notebook, notebook.cells) @test all(noerror, notebook.cells) setcode!(cell(1), raw""" macro test(sym) esc(:($sym = true)) end """) update_run!(, notebook, cell(1)) setcode!(cell(2), "x") setcode!(cell(3), "@test x") update_run!(, notebook, notebook.cells[2:3]) @test cell(2).output.body == "true" @test all(noerror, notebook.cells) cleanup(, notebook) end @testset "Reverse order" begin notebook = Notebook([Cell() for _ in 1:3]) cell(idx) = notebook.cells[idx] update_run!(, notebook, notebook.cells) setcode!(cell(1), "x") update_run!(, notebook, cell(1)) @test cell(1).errored == true setcode!(cell(2), "@bar x") update_run!(, notebook, cell(2)) @test cell(1).errored == true @test cell(2).errored == true setcode!(cell(3), raw"""macro bar(sym) esc(:($sym = "yay")) end""") update_run!(, notebook, cell(3)) @test cell(1) |> noerror @test cell(2) |> noerror @test cell(3) |> noerror @test cell(1).output.body == "\"yay\"" cleanup(, notebook) end @testset "@a defines @b" begin notebook = Notebook([Cell() for _ in 1:4]) cell(idx) = notebook.cells[idx] update_run!(, notebook, notebook.cells) setcode!(cell(1), "x") update_run!(, notebook, cell(1)) @test cell(1).errored == true setcode!(cell(3), "@a()") setcode!(cell(2), raw"""macro a() quote macro b(sym) esc(:($sym = 42)) end end |> esc end""") update_run!(, notebook, notebook.cells[2:3]) @test cell(1).errored == true @test cell(2) |> noerror @test cell(3) |> noerror setcode!(cell(4), "@b x") update_run!(, notebook, cell(4)) @test cell(1) |> noerror @test cell(2) |> noerror @test cell(3) |> noerror @test cell(4) |> noerror @test cell(1).output.body == "42" cleanup(, notebook) end @testset "Removing macros undefvar errors dependent cells" begin notebook = Notebook(Cell.([ """macro m() :(1 + 1) end""", "@m()", ])) update_run!(, notebook, notebook.cells) @test all(noerror, notebook.cells) setcode!(notebook.cells[begin], "") # remove definition of m update_run!(, notebook, notebook.cells[begin]) @test notebook.cells[begin] |> noerror @test notebook.cells[end].errored @test expecterror(UndefVarError(Symbol("@m")), notebook.cells[end]; strict=true) cleanup(, notebook) end @testset "Redefines macro with new SymbolsState" begin notebook = Notebook(Cell.([ "@b x", raw"""macro b(sym) esc(:($sym = 42)) end""", "x", "y", ])) cell(idx) = notebook.cells[idx] update_run!(, notebook, notebook.cells) @test cell(3).output.body == "42" @test cell(4).errored == true setcode!(cell(2), """macro b(_) esc(:(y = 42)) end""") update_run!(, notebook, cell(2)) @test cell(4).output.body == "42" @test cell(3).errored == true notebook = Notebook(Cell.([ "@b x", raw"""macro b(sym) esc(:($sym = 42)) end""", "x", "y", ])) update_run!(, notebook, notebook.cells) @test cell(3).output.body == "42" @test cell(4).errored == true setcode!(cell(2), """macro b(_) esc(:(y = 42)) end""") update_run!(, notebook, [cell(1), cell(2)]) # Cell 4 is executed even because cell(1) is in the root # of the reactive run because the expansion is done with the new version # of the macro in the new workspace because of the current_roots parameter of `resolve_topology`. # See Run.jl#resolve_topology. @test cell(4).output.body == "42" @test cell(3).errored == true cleanup(, notebook) end @testset "Reactive macro update does not invalidate the macro calls" begin notebook = Notebook(Cell.([ raw"""macro b(sym) if z > 40 esc(:($sym = $z)) else esc(:(y = $z)) end end""", "z = 42", "@b(x)", "x", "y", ])) cell(idx) = notebook.cells[idx] update_run!(, notebook, notebook.cells) @test cell(1) |> noerror @test cell(2) |> noerror @test cell(3) |> noerror @test cell(4) |> noerror @test cell(5).errored == true setcode!(cell(2), "z = 39") # running only 2, running all cells here works however update_run!(, notebook, cell(2)) @test cell(1) |> noerror @test cell(2) |> noerror @test cell(3) |> noerror @test cell(4).output.body != "42" @test cell(4).errored == true @test cell(5) |> noerror cleanup(, notebook) end @testset "Explicitely running macrocalls updates the reactive node" begin notebook = Notebook(Cell.([ "@b()", "ref = Ref{Int}(0)", raw"""macro b() ex = if iseven(ref[]) :(x = 10) else :(y = 10) end |> esc ref[] += 1 ex end""", "x", "y", ])) cell(i) = notebook.cells[i] update_run!(, notebook, notebook.cells) @test cell(1) |> noerror @test cell(2) |> noerror @test cell(3) |> noerror @test cell(4) |> noerror @test cell(5).errored == true update_run!(, notebook, cell(1)) @test cell(4).errored == true @test cell(5) |> noerror cleanup(, notebook) end @testset "Implicitely running macrocalls updates the reactive node" begin notebook = Notebook(Cell.([ "updater; @b()", "ref = Ref{Int}(0)", raw"""macro b() ex = if iseven(ref[]) :(x = 10) else :(y = 10) end |> esc ref[] += 1 ex end""", "x", "y", "updater = 1", ])) cell(i) = notebook.cells[i] update_run!(, notebook, notebook.cells) @test cell(1) |> noerror @test cell(2) |> noerror @test cell(3) |> noerror @test cell(4) |> noerror output_1 = cell(4).output.body @test cell(5).errored == true @test cell(6) |> noerror setcode!(cell(6), "updater = 2") update_run!(, notebook, cell(6)) # the output of cell 4 has not changed since the underlying computer # has not been regenerated. To update the reactive node and macrocall # an explicit run of @b() must be done. @test cell(4).output.body == output_1 @test cell(5).errored == true cleanup(, notebook) end @testset "Weird behavior" begin # https://github.com/fonsp/Pluto.jl/issues/1591 notebook = Notebook(Cell.([ "macro huh(_) throw(\"Fail!\") end", "huh(e) = e", "@huh(z)", "z = 101010", ])) cell(idx) = notebook.cells[idx] update_run!(, notebook, notebook.cells) @test cell(3).errored == true setcode!(cell(3), "huh(z)") update_run!(, notebook, cell(3)) @test cell(3) |> noerror @test cell(3).output.body == "101010" setcode!(cell(4), "z = 1234") update_run!(, notebook, cell(4)) @test cell(3) |> noerror @test cell(3).output.body == "1234" cleanup(, notebook) end @testset "Cell failing first not re-run?" begin notebook = Notebook(Cell.([ "x", "@b x", raw"macro b(sym) esc(:($sym = 42)) end", ])) update_run!(, notebook, notebook.cells) # CELL 1 "x" was run first and failed because the definition # of x was not yet found. However, it was not run re-run when the definition of # x ("@b(x)") was run. Should it? Maybe set a higher precedence to cells that define # macros inside the notebook. @test_broken noerror(notebook.cells[1]; verbose=false) cleanup(, notebook) end @testset "@a defines @b initial loading" begin notebook = Notebook(Cell.([ "x", "@b x", "@a", raw"""macro a() quote macro b(sym) esc(:($sym = 42)) end end |> esc end""" ])) cell(idx) = notebook.cells[idx] update_run!(, notebook, notebook.cells) @test cell(1) |> noerror @test cell(2) |> noerror @test cell(3) |> noerror @test cell(4) |> noerror @test cell(1).output.body == "42" cleanup(, notebook) end @testset "Macro with long compile time gets function wrapped" begin ms = 1e-3 ns = 1e-9 sleep_time = 40ms notebook = Notebook(Cell.([ "updater; @b()", """macro b() x = rand() sleep($sleep_time) :(1+\$x) end""", "updater = :slow", ])) cell(idx) = notebook.cells[idx] update_run!(, notebook, notebook.cells) @test noerror(cell(1)) runtime = cell(1).runtime*ns output_1 = cell(1).output.body @test sleep_time <= runtime setcode!(cell(3), "updater = :fast") update_run!(, notebook, cell(3)) @test noerror(cell(1)) runtime = cell(1).runtime*ns @test runtime < sleep_time # no recompilation! # output is the same because no new compilation happens @test output_1 == cell(1).output.body # force recompilation by explicitely running the cell update_run!(, notebook, cell(1)) @test cell(1) |> noerror @test output_1 != cell(1).output.body output_3 = cell(1).output.body setcode!(cell(1), "@b()") # changing code generates a new  update_run!(, notebook, cell(1)) @test cell(1) |> noerror @test output_3 != cell(1).output.body cleanup(, notebook) end @testset "Macro Prefix" begin .options.evaluation.workspace_use_distributed = true notebook = Notebook(Cell.([ "@sprintf \"answer = %d\" x", "x = y+1", raw""" macro def(sym) esc(:($sym=41)) end """, "@def y", "import Printf: @sprintf", ])) cell(idx) = notebook.cells[idx] update_run!(, notebook, cell(1)) @test cell(1).errored == true update_run!(, notebook, cell(5)) @test expecterror(UndefVarError(:x), cell(1)) update_run!(, notebook, cell(3)) update_run!(, notebook, cell(2)) update_run!(, notebook, cell(4)) @test cell(1) |> noerror cleanup(, notebook) .options.evaluation.workspace_use_distributed = false end @testset "Package macro 1" begin notebook = Notebook([ Cell("using Dates"), Cell("df = dateformat\"Y-m-d\""), ]) cell(idx) = notebook.cells[idx] update_run!(, notebook, cell(2)) @test cell(2).errored == true @test expecterror(UndefVarError(Symbol("@dateformat_str")), cell(2); strict=true) update_run!(, notebook, notebook.cells) @test cell(1) |> noerror @test cell(2) |> noerror notebook = Notebook([ Cell("using Dates"), Cell("df = dateformat\"Y-m-d\""), ]) update_run!(, notebook, notebook.cells) @test cell(1) |> noerror @test cell(2) |> noerror cleanup(, notebook) end @testset "Package macro 2" begin .options.evaluation.workspace_use_distributed = true notebook = Notebook([ Cell("z = x^2 + y"), Cell("@variables x y"), Cell(""" begin import Pkg Pkg.activate(mktempdir()) Pkg.add([ Pkg.PackageSpec(name="Symbolics", version="5.5.1"), # to avoid https://github.com/JuliaObjects/ConstructionBase.jl/issues/92 Pkg.PackageSpec(name="ConstructionBase", version="1.5.6"), ]) import Symbolics: @variables end """), ]) cell(idx) = notebook.cells[idx] update_run!(, notebook, notebook.cells[1:2]) @test cell(1).errored == true @test cell(2).errored == true update_run!(, notebook, cell(3)) @test cell(1) |> noerror @test cell(2) |> noerror @test cell(3) |> noerror update_run!(, notebook, notebook.cells) @test cell(1) |> noerror @test cell(2) |> noerror @test cell(3) |> noerror update_run!(, notebook, cell(2)) @test cell(1) |> noerror @test cell(2) |> noerror setcode!(cell(2), "@variables  y") update_run!(, notebook, cell(2)) @test cell(1).errored @test cell(2) |> noerror setcode!(cell(1), "z = ^2 + y") update_run!(, notebook, cell(1)) @test cell(1) |> noerror @test cell(2) |> noerror cleanup(, notebook) .options.evaluation.workspace_use_distributed = false end @testset "Previous workspace for unknowns" begin notebook = Notebook([ Cell("""macro my_identity(expr) expr end"""), Cell("(@__MODULE__, (@my_identity 1 + 1))"), Cell("@__MODULE__"), ]) cell(idx) = notebook.cells[idx] update_run!(, notebook, cell(1)) update_run!(, notebook, notebook.cells[2:end]) @test cell(1) |> noerror @test cell(2) |> noerror @test cell(3) |> noerror module_from_cell2 = cell(2).output.body[:elements][1][2][1] module_from_cell3 = cell(3).output.body @test module_from_cell2 == module_from_cell3 cleanup(, notebook) end @testset "Definitions" begin notebook = Notebook([ Cell("""macro my_assign(sym, val) :(\$(esc(sym)) = \$(val)) end"""), Cell("c = :hello"), Cell("@my_assign b c"), Cell("b"), ]) cell(idx) = notebook.cells[idx] update_run!(, notebook, notebook.cells) update_run!(, notebook, notebook.cells) @test ":hello" == cell(3).output.body @test ":hello" == cell(4).output.body @test :b  notebook.topology.nodes[cell(3)].definitions @test [:c, Symbol("@my_assign")]  notebook.topology.nodes[cell(3)].references setcode!(notebook.cells[2], "c = :world") update_run!(, notebook, cell(2)) @test ":world" == cell(3).output.body @test ":world" == cell(4).output.body cleanup(, notebook) end @testset "Is just text macros" begin notebook = Notebook(Cell.([ """ md"# Hello world!" """, """ "no julia value here" """, ])) update_run!(, notebook, notebook.cells) @test isempty(notebook.topology.unresolved_cells) cleanup(, notebook) end @testset "Macros using import" begin notebook = Notebook(Cell.([ """ @option "option_a" struct OptionA option_value::option_type end """, "option_type = String", "import Configurations: @option", ])) cell(idx) = notebook.cells[idx] update_run!(, notebook, notebook.cells) @test :option_type  notebook.topology.nodes[cell(1)].references @test cell(1) |> noerror cleanup(, notebook) end @testset "GlobalRefs in macros should be respected" begin notebook = Notebook(Cell.([ """ macro identity(expr) expr end """, """ x = 20 """, """ let x = 10 @identity(x) end """, ])) cell(idx) = notebook.cells[idx] update_run!(, notebook, notebook.cells) @test all(cell.([1,2,3]) .|> noerror) @test cell(3).output.body == "20" cleanup(, notebook) end @testset "GlobalRefs shouldn't break unreached undefined references" begin notebook = Notebook(Cell.([ """ macro get_x_but_actually_not() quote if false x else :this_should_be_returned end end end """, """ @get_x_but_actually_not() """, ])) cell(idx) = notebook.cells[idx] update_run!(, notebook, notebook.cells) @test all(cell.([1,2]) .|> noerror) @test cell(2).output.body == ":this_should_be_returned" cleanup(, notebook) end @testset "Doc strings" begin notebook = Notebook(Cell.([ "x = 1", raw""" "::Bool" f(::Bool) = x """, raw""" "::Int" f(::Int) = 1 """, ])) trigger, bool, int = notebook.cells workspace = WorkspaceManager.get_workspace((, notebook)) workspace_module = getproperty(Main, workspace.module_name) # Propose suggestions when no binding is found doc_content, status = PlutoRunner.doc_fetcher("filer", workspace_module) @test status == : @test occursin("Similar results:", doc_content) @test occursin("<b>f</b><b>i</b><b>l</b>t<b>e</b><b>r</b>", doc_content) update_run!(, notebook, notebook.cells) @test all(noerror, notebook.cells) @test occursin("::Bool", bool.output.body) @test !occursin("::Int", bool.output.body) @test occursin("::Bool", int.output.body) @test occursin("::Int", int.output.body) setcode!(int, raw""" "::Int new docstring" f(::Int) = 1 """) update_run!(, notebook, int) @test occursin("::Bool", int.output.body) @test occursin("::Int new docstring", int.output.body) update_run!(, notebook, trigger) @test occursin("::Bool", bool.output.body) @test occursin("::Int new docstring", bool.output.body) @test length(eachmatch(r"Bool", bool.output.body) |> collect) == 1 @test length(eachmatch(r"Int", bool.output.body) |> collect) == 1 update_run!(, notebook, trigger) @test length(eachmatch(r"Bool", bool.output.body) |> collect) == 1 setcode!(int, "") update_run!(, notebook, [bool, int]) @test !occursin("::Int", bool.output.body) setcode!(bool, """ "An empty conjugate" Base.conj() = x """) update_run!(, notebook, bool) @test noerror(bool) @test noerror(trigger) @test occursin("An empty conjugate", bool.output.body) @test occursin("complex conjugate", bool.output.body) setcode!(bool, "Docs.doc(conj)") update_run!(, notebook, bool) @test !occursin("An empty conjugate", bool.output.body) @test occursin("complex conjugate", bool.output.body) cleanup(, notebook) end @testset "Delete methods from macros" begin  = ServerSession() .options.evaluation.workspace_use_distributed = false notebook = Notebook([ Cell("using Memoize"), Cell(""" macro user_defined() quote struct ASD end custom_func(::ASD) = "ASD" end |> esc end """), Cell("@user_defined"), Cell("methods(custom_func)"), Cell(""" @memoize function memoized_func(a) println("Running") 2a end """), Cell("methods(memoized_func)"), ]) cell(idx) = notebook.cells[idx] update_run!(, notebook, notebook.cells) @test :custom_func  notebook.topology.nodes[cell(3)].funcdefs_without_signatures @test cell(4) |> noerror @test :memoized_func  notebook.topology.nodes[cell(5)].funcdefs_without_signatures @test cell(6) |> noerror cell(3).code = "#=$(cell(3).code)=#" cell(5).code = "#=$(cell(5).code)=#" update_run!(, notebook, notebook.cells) @test :custom_func  notebook.topology.nodes[cell(3)].funcdefs_without_signatures @test expecterror(UndefVarError(:custom_func), cell(4)) @test :memoized_func  notebook.topology.nodes[cell(5)].funcdefs_without_signatures @test expecterror(UndefVarError(:memoized_func), cell(6)) cleanup(, notebook) end end using Test import Pluto.ExpressionExplorer: compute_symbolreferences @testset "Method signatures" begin disjoint(x,y) = isempty(x  y) function mutually_disjoint(x, xs...) all(xs) do y disjoint(x,y) end && mutually_disjoint(xs...) end mutually_disjoint(x) = true function methods_can_coexist(defs::Expr...) symstates = compute_symbolreferences.(defs) funcnamesigs = [keys(syms.funcdefs) for syms in symstates] mutually_disjoint(funcnamesigs...) end @testset "Different method signatures across cells" begin @test methods_can_coexist( :(f(x, y) = 1), :(f(x) = 2), ) @test methods_can_coexist( :(f(x::A) = 1), :(f(x::B) = 2), ) @test methods_can_coexist( :(f(x, y=1) = 1), :(f(x, y) = 2), ) @test methods_can_coexist( :(f(x::e(f{g})=3) = 1), :(f(x::h where i) = 2), ) @test methods_can_coexist( :(f(x::Tuple{X,T} where T) = 1), :(f(x::Tuple{X,T}) = 2), ) @test methods_can_coexist( :(f(x::A) where A = 1), :(f(x::A) = 2), ) @test methods_can_coexist( :(f(x::String, y) = 1), :(f(x, y::String) = 2), ) @test methods_can_coexist( :(f(x::A) = 1), :(f(x::B) = 2), ) @test methods_can_coexist( :(f(x, y...) = 1), :(f(x) = 2), ) @test methods_can_coexist( :(f(x, y::Z...) = 1), :(f(x, y::X...) = 2), ) # what is this called again? @test methods_can_coexist( :(f(x::T) where T = 1), :(f(x) = 2), ) end @testset "Identical method signatures across cells" begin @test !methods_can_coexist( :(f(x) = 3), :(f(y) = 3), ) @test !methods_can_coexist( :(f(x) = 3), :(f(y; z) = 3), ) @test !methods_can_coexist( :(f(x) = 3), :(f(y::Any) = 3), ) # function using built in type synonyms # like Int and Int64 @assert string(Int) == "Int64" || string(Int) == "Int32" @test_broken !methods_can_coexist( :(f(x::Int) = 3), Meta.parse("f(x::$(string(Int))) = 4"), ) # multiple methods per cell @test !methods_can_coexist( :(f(x::Int) = 1; f(x::String) = 2), :(f(x::String) = 3; f(x::Vector) = 4), ) # methods only differing in key word arguments @test !methods_can_coexist( :(f() = 1), :(f(; x) = 3), ) end end import Pluto: Pluto, Cell, ExpressionExplorerExtras import Pluto.MoreAnalysis using Test @testset "MoreAnalysis" begin file = joinpath(@__DIR__, "parallelpaths4.jl") newpath = tempname() Pluto.readwrite(file, newpath) notebook = Pluto.load_notebook(newpath) # Run pluto's analysis. This is like opening the notebook, without actually running it # s = Pluto.ServerSession() # Pluto.update_save_run!(s, notebook, notebook.cells; run) notebook.topology = Pluto.updated_topology(notebook.topology, notebook, notebook.cells) ind(c::Cell) = findfirst(isequal(c), notebook.cells) cell_indices(cs) = sort(ind.(cs)) @testset "Recursive stuff" begin @test cell_indices(MoreAnalysis.downstream_recursive(notebook, notebook.topology, notebook.cells[1:1])) == 3:7 @test cell_indices(MoreAnalysis.upstream_recursive(notebook, notebook.topology, notebook.cells[1:1])) == [] @test cell_indices(MoreAnalysis.downstream_recursive(notebook, notebook.topology, notebook.cells[6:6])) == [7] @test cell_indices(MoreAnalysis.upstream_recursive(notebook, notebook.topology, notebook.cells[6:6])) == [1,2,3,4] @test cell_indices(MoreAnalysis.downstream_recursive(notebook, notebook.topology, notebook.cells[7:7])) == [] @test cell_indices(MoreAnalysis.upstream_recursive(notebook, notebook.topology, notebook.cells[7:7])) == 1:6 @test cell_indices(MoreAnalysis.downstream_recursive(notebook, notebook.topology, notebook.cells[4:5])) == 6:7 @test cell_indices(MoreAnalysis.upstream_recursive(notebook, notebook.topology, notebook.cells[4:5])) == 1:3 @test cell_indices(MoreAnalysis.downstream_recursive(notebook, notebook.topology, notebook.cells[1:2])) == 3:7 @test cell_indices(MoreAnalysis.upstream_recursive(notebook, notebook.topology, notebook.cells[1:2])) == [] @test cell_indices(MoreAnalysis.downstream_recursive(notebook, notebook.topology, notebook.cells[1:7])) == 3:7 @test cell_indices(MoreAnalysis.upstream_recursive(notebook, notebook.topology, notebook.cells[1:7])) == 1:6 @test cell_indices(MoreAnalysis.downstream_recursive(notebook, notebook.topology, notebook.cells[[1,3]])) == 3:7 @test cell_indices(MoreAnalysis.upstream_recursive(notebook, notebook.topology, notebook.cells[[1,3]])) == [1] @test MoreAnalysis.upstream_recursive(notebook, notebook.topology, Cell[]) == Set{Cell}() @test MoreAnalysis.downstream_recursive(notebook, notebook.topology, Cell[]) == Set{Cell}() end @testset "Bond connections" begin # bound_variables = (map(notebook.cells) do cell # MoreAnalysis.find_bound_variables(cell.parsedcode) # end) connections = MoreAnalysis.bound_variable_connections_graph(notebook) @test !isempty(connections) wanted_connections = Dict( :x => [:y, :x], :y => [:y, :x], :show_dogs => [:show_dogs], :b => [:b], :c => [:c], :five1 => [:five1], :five2 => [:five2], :six1 => [:six2, :six1], :six2 => [:six3, :six2, :six1], :six3 => [:six3, :six2], :cool1 => [:cool1, :cool2], :cool2 => [:cool1, :cool2], :world => [:world], :boring => [:boring], ) transform(d) = Dict(k => sort(v) for (k, v) in d) @test transform(connections) == transform(wanted_connections) end @testset "can_be_function_wrapped" begin c = ExpressionExplorerExtras.can_be_function_wrapped @test c(quote a = b + C if d for i = 1:10 while Y end end end end) @test c(quote map(1:10) do i i + 1 end end) @test !c(quote function x(x) X end end) @test !c(quote if false using Asdf end end) end end using Test import Pluto: Notebook, ServerSession, ClientSession, Cell, load_notebook, load_notebook_nobackup, save_notebook, WorkspaceManager, cutename, numbered_until_new, readwrite, without_pluto_file_extension, update_run!, get_metadata_no_default, is_disabled, create_cell_metadata, update_skipped_cells_dependency! import Pluto.WorkspaceManager import Random import Pkg import UUIDs: UUID # We define some notebooks explicitly, and not as a .jl notebook file, to avoid circular reasoning  function basic_notebook() Notebook([ Cell("100*a + b"), Cell("a = 1"), Cell(" = :"), # ends with 4-byte character Cell("b = let\n\tx = a + a\n\tx*x\nend"), Cell("html\"<h1>Hoi!</h1>\n<p>My name is <em>kiki</em></p>\""), # test included Markdown import Cell("""md"# Cze! My name is **baba** and I like \$\\LaTeX\$ _support!_ \$\$\\begin{align} \\varphi &= \\sum_{i=1}^{\\infty} \\frac{\\left(\\sin{x_i}^2 + \\cos{x_i}^2\\right)}{i^2} \\\\ b &= \\frac{1}{2}\\,\\log \\exp{\\varphi} \\end{align}\$\$ ### The spectacle before us was indeed sublime. Apparently we had reached a great height in the atmosphere, for the sky was a dead black, and the stars had ceased to twinkle. By the same illusion which lifts the horizon of the sea to the level of the spectator on a hillside, the sable cloud beneath was dished out, and the car seemed to float in the middle of an immense dark sphere, whose upper half was strewn with silver. Looking down into the dark gulf below, I could see a ruddy light streaming through a rift in the clouds." """), # test included InteractiveUtils import Cell("subtypes(Number)"), ]) |> init_packages! end function cell_metadata_notebook() Notebook([ Cell( code="100*a + b", metadata=Dict( "a metadata tag" => Dict( "boolean" => true, "string" => "String", "number" => 10000, ), "disabled" => true, ) |> create_cell_metadata, ), ]) |> init_packages! end function ingredients(path::String) # this is from the Julia source code (evalfile in base/loading.jl) # but with the modification that it returns the module instead of the last object name = Symbol(basename(path)) m = Module(name) Core.eval(m, Expr(:toplevel, :(eval(x) = $(Expr(:core, :eval))($name, x)), :(include(x) = $(Expr(:top, :include))($name, x)), :(include(mapexpr::Function, x) = $(Expr(:top, :include))(mapexpr, $name, x)), :(include($path)))) m end function skip_as_script_notebook() Notebook([ Cell( code="skipped_var = 10", metadata=Dict( "skip_as_script" => true, ) |> create_cell_metadata, ), Cell( code="non_skipped_var = 15", ), Cell( code="dependent_var = skipped_var + 1", ), ]) |> init_packages! end function notebook_metadata_notebook() nb = Notebook([ Cell(code="n * (n + 1) / 2"), ]) |> init_packages! nb.metadata = Dict( "boolean" => true, "string" => "String", "number" => 10000, "ozymandias" => Dict( "l1" => "And on the pedestal, these words appear:", "l2" => "My name is Ozymandias, King of Kings;", "l3" => "Look on my Works, ye Mighty, and despair!", ), ) nb end function shuffled_notebook() Notebook([ Cell("z = y"), Cell("v = u"), Cell("y = x"), Cell("x = w"), Cell("using Dates"), Cell("t = 1"), Cell("w = v"), Cell("u = t"), ]) |> init_packages! end function shuffled_with_imports_notebook() Notebook([ Cell("c = uuid1()"), Cell("a = (b, today())"), Cell("y = 2"), Cell("using UUIDs"), Cell("y"), Cell("x = y"), Cell("b = base64encode"), Cell(""" begin using Dates using Base64 end"""), Cell("BasicREPL"), Cell(""" begin x using REPL end"""), ]) |> init_packages! end function init_packages!(nb::Notebook) nb.topology = Pluto.updated_topology(nb.topology, nb, nb.cells) Pluto.sync_nbpkg_core(nb, nb.topology, nb.topology) return nb end function bad_code_notebook() Notebook([ Cell("z = y"), Cell("y = z"), Cell(""";lka;fd;jasdf;;;\n\n\n\n\nasdfasdf [[["""), Cell("using Aasdfdsf"), ]) |> init_packages! end function bonds_notebook() Notebook([ Cell("y = x"), Cell("@bind x html\"<input type='range'>\""), Cell("@assert y === missing"), Cell("""struct Wow x end"""), Cell("Base.get(w::Wow) = w.x"), Cell("Base.show(io::IO, ::MIME\"text/html\", w::Wow) = nothing"), Cell("w = Wow(10)"), Cell("@bind z w"), Cell("@assert z == 10"), ]) |> init_packages! end function project_notebook() Notebook([ Cell("using Dates"), Cell("using Example"), ]) |> init_packages! end @testset "Notebook Files" begin nbs = [String(nameof(f)) => f() for f in [basic_notebook, shuffled_notebook, shuffled_with_imports_notebook, bad_code_notebook, bonds_notebook, project_notebook]] @testset "Sample notebooks " begin # Also adds them to the `nbs` list for file in ["Basic.jl", "Tower of Hanoi.jl", "Interactivity.jl"] path = normpath(Pluto.project_relative_path("sample", file)) @testset "$(file)" begin nb = @test_nowarn load_notebook_nobackup(path) @test length(nb.cells) > 0 push!(nbs, "sample " * file => nb) end end end  = ServerSession() for (name, nb) in nbs nb.path = tempname() * ".jl" end @testset "I/O basic" begin @testset "$(name)" for (name, nb) in nbs save_notebook(nb) # @info "File" name Text(read(nb.path,String)) result = load_notebook_nobackup(nb.path) @test_notebook_inputs_equal(nb, result) end end @testset "Cell Metadata" begin  = ServerSession() .options.evaluation.workspace_use_distributed = false @testset "Disabling & Metadata" begin nb = cell_metadata_notebook() update_run!(, nb, nb.cells) cell = first(values(nb.cells_dict)) @test get_metadata_no_default(cell) == Dict( "a metadata tag" => Dict( "boolean" => true, "string" => "String", "number" => 10000, ), "disabled" => true, # enhanced metadata because cell is disabled ) save_notebook(nb) result = load_notebook_nobackup(nb.path) @test_notebook_inputs_equal(nb, result) cell = first(nb.cells) @test is_disabled(cell) @test get_metadata_no_default(cell) == Dict( "a metadata tag" => Dict( "boolean" => true, "string" => "String", "number" => 10000, ), "disabled" => true, ) WorkspaceManager.unmake_workspace((, nb); verbose=false) end end @testset "Notebook Metadata" begin  = ServerSession() .options.evaluation.workspace_use_distributed = false nb = notebook_metadata_notebook() update_run!(, nb, nb.cells) @test nb.metadata == Dict( "boolean" => true, "string" => "String", "number" => 10000, "ozymandias" => Dict( "l1" => "And on the pedestal, these words appear:", "l2" => "My name is Ozymandias, King of Kings;", "l3" => "Look on my Works, ye Mighty, and despair!", ), ) save_notebook(nb) nb_loaded = load_notebook_nobackup(nb.path) @test nb.metadata == nb_loaded.metadata WorkspaceManager.unmake_workspace((, nb); verbose=false) end @testset "Skip as script" begin  = ServerSession() .options.evaluation.workspace_use_distributed = false nb = skip_as_script_notebook() update_run!(, nb, nb.cells) save_notebook(nb) m = ingredients(nb.path) @test !isdefined(m, :skipped_var) @test !isdefined(m, :dependent_var) @test m.non_skipped_var == 15 # Test that `load_notebook` doesn't break commented out cells load_notebook(nb.path) m = ingredients(nb.path) @test !isdefined(m, :skipped_var) @test !isdefined(m, :dependent_var) @test m.non_skipped_var == 15 nb.cells[1].metadata["skip_as_script"] = false update_skipped_cells_dependency!(nb) save_notebook(nb) m = ingredients(nb.path) @test m.skipped_var == 10 @test m.non_skipped_var == 15 @test m.dependent_var == 11 WorkspaceManager.unmake_workspace((, nb); verbose=false) end @testset "More Metadata" begin test_file_contents = """ ### A Pluto.jl notebook ### # v0.19.4 @hello from the future where we might put extra stuff here #> [hello] #> world = [1, 2, 3] #> [frontmatter] #> title = "cool stuff" using Markdown using SecretThings # asdfasdf #  a86be878-d616-11ec-05a3-c902726cee5f #  disabled = true #  fonsi = 123 #= 1 + 1  =# #  Cell order: # a86be878-d616-11ec-05a3-c902726cee5f # ok thx byeeeee """ test_filename = tempname() write(test_filename, test_file_contents) nb = load_notebook_nobackup(test_filename) @test nb.metadata == Dict( "hello" => Dict( "world" => [1,2,3], ), "frontmatter" => Dict( "title" => "cool stuff", ), ) @test get_metadata_no_default(only(nb.cells)) == Dict( "disabled" => true, "fonsi" => 123, ) @test Pluto.frontmatter(nb) == Dict( "title" => "cool stuff", ) Pluto.set_frontmatter!(nb, Dict("a" => "b")) @test Pluto.frontmatter(nb) == Dict("a" => "b") Pluto.set_frontmatter!(nb, nothing) @test Pluto.frontmatter(nb) == Dict() Pluto.set_frontmatter!(nb, nothing) @test Pluto.frontmatter(nb) == Dict() end @testset "I/O overloaded" begin @testset "$(name)" for (name, nb) in nbs let tasks = [] for i in 1:16 push!(tasks, @async save_notebook(nb)) if i <= 8 sleep(0.01) end end wait.(tasks) result = load_notebook_nobackup(nb.path) @test_notebook_inputs_equal(nb, result) end end end @testset "Bijection test" begin @testset "$(name)" for (name, nb) in nbs new_path = tempname() @assert !isfile(new_path) readwrite(nb.path, new_path) # load_notebook also does parsing and analysis - this is needed to save the notebook with cells in their correct order # load_notebook is how they are normally loaded, load_notebook_nobackup new_nb = load_notebook(new_path) before_contents = read(new_path, String) after_path = tempname() write(after_path, before_contents) after = load_notebook(after_path) after_contents = read(after_path, String) if name != String(nameof(bad_code_notebook)) @test Text(before_contents) == Text(after_contents) end end end @testset "Recover from bad cell order" begin contents = """ ### A Pluto.jl notebook ### # v0.17.3 using Markdown using InteractiveUtils #  cdd40e28-61be-11ec-28fd-111111111111 x = 1 #  cdd40e28-61be-11ec-28fd-222222222222 y = 2 #  cdd40e28-61be-11ec-28fd-333333333333 z = 3 #  Cell order: # cdd40e28-61be-11ec-28fd-111111111111 # cdd40e28-61be-11ec-28fd-333333333333 # cdd40e28-61be-11ec-28fd-444444444444 """ path = tempname() write(path, contents) nb = load_notebook(path) @test nb.cell_order == UUID.([ "cdd40e28-61be-11ec-28fd-111111111111", "cdd40e28-61be-11ec-28fd-333333333333", "cdd40e28-61be-11ec-28fd-222222222222", ]) @test keys(nb.cells_dict) == Set(nb.cell_order) end # Some notebooks are designed to error (inside/outside Pluto) expect_error = [String(nameof(bad_code_notebook)), String(nameof(project_notebook)), "sample Interactivity.jl"] @testset "Runnable without Pluto" begin @testset "$(name)" for (name, nb) in nbs new_path = tempname() @assert !isfile(new_path) cp(nb.path, new_path) new_nb = load_notebook(new_path) # println(read(new_nb.path, String)) if name  expect_error @test jl_is_runnable(new_nb.path; only_undefvar=false) end end end @testset "Runnable with Pluto" begin @testset "$(name)" for (name, nb) in nbs if name  expect_error @test nb_is_runnable(, nb) cleanup(, nb) end end end @testset "Line endings" begin @testset "$(name)" for (name, nb) in nbs file_contents = sprint(save_notebook, nb) let result = sread(load_notebook_nobackup, file_contents, nb.path) @test_notebook_inputs_equal(nb, result) end let file_contents_windowsed = replace(file_contents, "\n" => "\r\n") result_windowsed = sread(load_notebook_nobackup, file_contents_windowsed, nb.path) @test_notebook_inputs_equal(nb, result_windowsed) end end end @testset "Backups" begin @testset "$(name)" for (name, nb) in nbs save_notebook(nb) new_dir = mktempdir() new_path = joinpath(new_dir, "nb.jl") cp(nb.path, new_path) @test_nowarn load_notebook(new_path) @test num_backups_in(new_dir) == 0 # Delete last line # cp(nb.path, new_path, force=true) # to_write = readlines(new_path)[1:end - 1] # write(new_path, join(to_write, '\n')) # @test_logs (:warn, r"Backup saved to") load_notebook(new_path) # @test num_backups_in(new_dir) == 1 # Duplicate first line cp(nb.path, new_path, force=true) to_write = let old_lines = readlines(new_path) [old_lines[1], old_lines...] end write(new_path, join(to_write, '\n')) @test_nowarn load_notebook(new_path) @test num_backups_in(new_dir) == 0 @test readdir(new_dir) == ["nb.jl"] # Extra stuff in preamble cp(nb.path, new_path, force=true) to_write = let old_content = read(new_path, String) replace(old_content, "using Markdown" => "using Markdown\n1 + 1") end write(new_path, to_write) @test_nowarn load_notebook(new_path) @test num_backups_in(new_dir) == 0 @test readdir(new_dir) == ["nb.jl"] # Extra stuff at the end of the file cp(nb.path, new_path, force=true) to_write = let old_lines = readlines(new_path) [old_lines..., "", "1 + 1", Pluto._cell_id_delimiter * "heyyyy", "# coolio"] end write(new_path, join(to_write, '\n')) @test_logs (:warn, r"Backup saved to") load_notebook(new_path) @test num_backups_in(new_dir) == 1 @test readdir(new_dir) == ["nb backup 1.jl", "nb.jl"] # AGAIN write(new_path, join(to_write, '\n')) @test_logs (:warn, r"Backup saved to") load_notebook(new_path) @test num_backups_in(new_dir) == 2 @test Set(readdir(new_dir)) == Set(["nb backup 2.jl", "nb backup 1.jl", "nb.jl"] ) end end @testset "Export HTML" begin nb = basic_notebook() nb.metadata["frontmatter"] = Dict{String,Any}( "title" => "My<Title", "tags" => ["aaa", "bbb"], "description" => "ccc", ) export_html = replace(Pluto.generate_html(nb), "'" => "\"") @test occursin("<pluto-editor", export_html) @test occursin("<title>My&lt;Title</title>", export_html) @test occursin("""<meta name="description" content="ccc">""", export_html) @test occursin("""<meta property="og:description" content="ccc">""", export_html) @test occursin("""<meta property="og:article:tag" content="aaa">""", export_html) @test occursin("""<meta property="og:article:tag" content="bbb">""", export_html) @test occursin("""<meta property="og:type" content="article">""", export_html) embedded_jl = Pluto.embedded_notebookfile(export_html) jl_path = tempname() write(jl_path, embedded_jl) result = load_notebook_nobackup(jl_path) @test_notebook_inputs_equal(nb, result, false) filename = "\"howdy.jl\"" export_html = Pluto.generate_html(nb; notebookfile_js=filename) @test occursin(filename, export_html) @test_throws ArgumentError Pluto.embedded_notebookfile(export_html) filename = "\"some where/thing.plutostate\"" export_html = Pluto.generate_html(nb; statefile_js=filename) @test occursin("""pluto_statefile = "some where/""", export_html) @test occursin("""<link rel="preload" as="fetch" href="some where/""", export_html) export_html = Pluto.generate_index_html() @test occursin("</html>", export_html) @test !occursin("<pluto-editor", export_html) export_html = Pluto.generate_index_html(; featured_direct_html_links=true, featured_sources_js="[{url:`./zozozo.json`}]") @test occursin("zozozo", export_html) end @testset "Utilities" begin @testset "Cute file names" begin trash = mktempdir() for i in 1:200 numbered_until_new(joinpath(trash, cutename()); create_file=true) end @test all(!isfile, [numbered_until_new(joinpath(trash, cutename()); create_file=false) for _ in 1:200]) end end @test without_pluto_file_extension("julia.jl") == "julia" @test without_pluto_file_extension("julia.jl") == "julia.jl" @test without_pluto_file_extension("asdf.pluto.jl") == "asdf" @test without_pluto_file_extension("asdf.jl") == "asdf" # TODO: test bad dirs, filenames, permissions end using Test import Pluto: Configuration, Notebook, ServerSession, ClientSession, update_run!, Cell, WorkspaceManager import Pluto.Configuration: Options, EvaluationOptions ### MORE TESTS ARE IN PLUTODEPENDENCYEXPLORER.jL # The tests on the Pluto side are tests that rely more heavily on what Pluto implements on top of PlutoDependencyExplorer. # The tests in PlutoDependencyExplorer are focus in *reactive ordering*. @testset "Reactivity" begin  = ServerSession() .options.evaluation.workspace_use_distributed = false @testset "Basic $workertype" for workertype in [:Malt, :Distributed, :InProcess] .options.evaluation.workspace_use_distributed = workertype !== :InProcess .options.evaluation.workspace_use_distributed_stdlib = workertype === :Distributed notebook = Notebook([ Cell("x = 1"), Cell("y = x"), Cell("f(x) = x + y"), Cell("f(4)"), Cell("""begin g(a) = x g(a,b) = y end"""), Cell("g(6) + g(6,6)"), Cell(""" begin pushfirst!(LOAD_PATH, "@stdlib") import Distributed popfirst!(LOAD_PATH) end """), Cell("Distributed.myid()"), ]) @test !haskey(WorkspaceManager.active_workspaces, notebook.notebook_id) update_run!(, notebook, notebook.cells[1:2]) @test notebook.cells[1].output.body == notebook.cells[2].output.body @test notebook.cells[1].output.rootassignee == :x @test notebook.cells[1].runtime !== nothing setcode!(notebook.cells[1], "x = 12") update_run!(, notebook, notebook.cells[1]) @test notebook.cells[1].output.body == notebook.cells[2].output.body @test notebook.cells[2].runtime !== nothing update_run!(, notebook, notebook.cells[3]) @test notebook.cells[3] |> noerror @test notebook.cells[3].output.rootassignee === nothing update_run!(, notebook, notebook.cells[4]) @test notebook.cells[4].output.body == "16" @test notebook.cells[4] |> noerror @test notebook.cells[4].output.rootassignee === nothing setcode!(notebook.cells[1], "x = 912") update_run!(, notebook, notebook.cells[1]) @test notebook.cells[4].output.body == "916" setcode!(notebook.cells[3], "f(x) = x") update_run!(, notebook, notebook.cells[3]) @test notebook.cells[4].output.body == "4" setcode!(notebook.cells[1], "x = 1") setcode!(notebook.cells[2], "y = 2") update_run!(, notebook, notebook.cells[1:2]) update_run!(, notebook, notebook.cells[5:6]) @test notebook.cells[5] |> noerror @test notebook.cells[6].output.body == "3" setcode!(notebook.cells[2], "y = 1") update_run!(, notebook, notebook.cells[2]) @test notebook.cells[6].output.body == "2" setcode!(notebook.cells[1], "x = 2") update_run!(, notebook, notebook.cells[1]) @test notebook.cells[6].output.body == "3" update_run!(, notebook, notebook.cells[7:8]) if workertype === :Distributed @test notebook.cells[8].output.body  ("1", string(Distributed.myid())) elseif workertype === :Malt @test notebook.cells[8].output.body == "1" elseif workertype === :InProcess @test notebook.cells[8].output.body == string(Distributed.myid()) else error() end WorkspaceManager.unmake_workspace((, notebook); verbose=false) end .options.evaluation.workspace_use_distributed = false @testset "Simple insert cell" begin notebook = Notebook(Cell[]) update_run!(, notebook, notebook.cells) insert_cell!(notebook, Cell("a = 1")) update_run!(, notebook, notebook.cells[begin]) insert_cell!(notebook, Cell("b = 2")) update_run!(, notebook, notebook.cells[begin+1]) insert_cell!(notebook, Cell("c = 3")) update_run!(, notebook, notebook.cells[begin+2]) insert_cell!(notebook, Cell("a + b + c")) update_run!(, notebook, notebook.cells[begin+3]) @test notebook.cells[begin+3].output.body == "6" setcode!(notebook.cells[begin+1], "b = 10") update_run!(, notebook, notebook.cells[begin+1]) @test notebook.cells[begin+3].output.body == "14" end @testset "Simple delete cell" begin notebook = Notebook(Cell.([ "x = 42", "x", ])) update_run!(, notebook, notebook.cells) @test all(noerror, notebook.cells) delete_cell!(, notebook, notebook.cells[begin]) @test length(notebook.cells) == 1 @test expecterror(UndefVarError(:x), notebook.cells[begin]) end @testset ".. as an identifier" begin notebook = Notebook(Cell.([ ".. = 1", "..", ])) update_run!(, notebook, notebook.cells) @test all(noerror, notebook.cells) @test notebook.cells[end].output.body == "1" end @testset "Cleanup of workspace variable" begin notebook = Notebook([ Cell("x = 10000"), ]) update_run!(, notebook, notebook.cells[1:1]) @test haskey(WorkspaceManager.active_workspaces, notebook.notebook_id) w = fetch(WorkspaceManager.active_workspaces[notebook.notebook_id]) oldmod = getproperty(Main, w.module_name) setcode!(notebook.cells[begin], "") update_run!(, notebook, notebook.cells[1:1]) @test isdefined(oldmod, :x) @test isnothing(getproperty(oldmod, :x)) newmod = getproperty(Main, w.module_name) @test !isdefined(newmod, :x) end @testset "Cleanup only Pluto controlled modules" begin notebook = Notebook([ Cell("""Core.eval(Main, :( module var\"Pluto#2443\" x = 1000 end ))"""), Cell("import .Main.var\"Pluto#2443\": x"), Cell("x"), ]) update_run!(, notebook, notebook.cells) @test noerror(notebook.cells[1]) @test noerror(notebook.cells[2]) @test noerror(notebook.cells[3]) @test haskey(WorkspaceManager.active_workspaces, notebook.notebook_id) w = fetch(WorkspaceManager.active_workspaces[notebook.notebook_id]) oldmod = getproperty(Main, w.module_name) setcode!(notebook.cells[2], "") setcode!(notebook.cells[3], "") update_run!(, notebook, notebook.cells) @test isdefined(oldmod, :x) @test which(oldmod, :x) != oldmod @test !isnothing(getproperty(oldmod, :x)) newmod = getproperty(Main, w.module_name) @test !isdefined(newmod, :x) @test !isnothing(Main.var"Pluto#2443".x) end @testset "Reactive usings" begin notebook = Notebook([ Cell("June"), Cell("using Dates"), Cell("July"), ]) update_run!(, notebook, notebook.cells[1:1]) @test notebook.cells[1].errored == true # this cell is before the using Dates and will error @test notebook.cells[3] |> noerror # using the position in the notebook this cell will not error update_run!(, notebook, notebook.cells[2:2]) @test notebook.cells[1] |> noerror @test notebook.cells[3] |> noerror end @testset "Reactive usings 2" begin notebook = Notebook([ Cell("October"), Cell("using Dates"), Cell("December"), Cell(""), ]) update_run!(, notebook, notebook.cells) @test notebook.cells[1] |> noerror @test notebook.cells[3] |> noerror setcode!(notebook.cells[2], "") update_run!(, notebook, notebook.cells[2:2]) @test notebook.cells[1].errored == true @test notebook.cells[3].errored == true setcode!(notebook.cells[4], "December = 13") update_run!(, notebook, notebook.cells[4:4]) @test notebook.cells[1].errored == true @test notebook.cells[3] |> noerror setcode!(notebook.cells[2], "using Dates") update_run!(, notebook, notebook.cells[2:2]) @test notebook.cells[1] |> noerror @test notebook.cells[3] |> noerror @test notebook.cells[3].output.body == "13" end @testset "Reactive usings 3" begin notebook = Notebook([ Cell("archive_artifact"), Cell("using Unknown.Package"), ]) update_run!(, notebook, notebook.cells) @test notebook.cells[1].errored == true @test notebook.cells[2].errored == true setcode!(notebook.cells[2], "using Pkg.Artifacts") update_run!(, notebook, notebook.cells) @test notebook.cells[1] |> noerror @test notebook.cells[2] |> noerror end @testset "Reactive usings 4" begin .options.evaluation.workspace_use_distributed = true notebook = Notebook([ Cell("@sprintf \"double_december = %d\" double_december"), Cell("double_december = 2December"), Cell("archive_artifact"), Cell(""), ]) update_run!(, notebook, notebook.cells) @test notebook.cells[1].errored == true @test notebook.cells[2].errored == true @test notebook.cells[3].errored == true setcode!(notebook.cells[4], "import Pkg; using Dates, Printf, Pkg.Artifacts") update_run!(, notebook, notebook.cells[4:4]) @test notebook.cells[1] |> noerror @test notebook.cells[2] |> noerror @test notebook.cells[3] |> noerror @test notebook.cells[4] |> noerror @test notebook.cells[1].output.body == "\"double_december = 24\"" cleanup(, notebook) .options.evaluation.workspace_use_distributed = false end @testset "Reactive usings 5" begin notebook = Notebook(Cell.([ "", "x = ones(December * 2)", "December = 3", ])) update_run!(, notebook, notebook.cells) @test all(noerror, notebook.cells) setcode!(notebook.cells[begin], raw""" begin @eval(module Hello December = 12 export December end) using .Hello end """) update_run!(, notebook, notebook.cells[begin]) @test all(noerror, notebook.cells) WorkspaceManager.unmake_workspace((, notebook); verbose=false) end @testset "More challenging reactivity of extended function" begin notebook = Notebook(Cell.([ "Base.inv(s::String) = s", """ struct MyStruct x MyStruct(s::String) = new(inv(s)) end """, "Base.inv(ms::MyStruct) = inv(ms.x)", "MyStruct(\"hoho\")", "a = MyStruct(\"blahblah\")", "inv(a)", ])) cell(idx) = notebook.cells[idx] update_run!(, notebook, notebook.cells) @test all(noerror, notebook.cells) @test notebook.cells[end].output.body == "\"blahblah\"" setcode!(cell(1), "Base.inv(s::String) = s * \"suffix\"") update_run!(, notebook, cell(1)) @test all(noerror, notebook.cells) @test notebook.cells[end].output.body == "\"blahblahsuffixsuffix\"" # 2 invs, 1 in constructor, 1 in inv(::MyStruct) setcode!(cell(3), "Base.inv(ms::MyStruct) = ms.x") # remove inv in inv(::MyStruct) update_run!(, notebook, cell(3)) @test all(noerror, notebook.cells) @test notebook.cells[end].output.body == "\"blahblahsuffix\"" # only one inv # Empty and run cells to remove the Base overloads that we created, just to be sure setcode!.(notebook.cells, [""]) update_run!(, notebook, notebook.cells) WorkspaceManager.unmake_workspace((, notebook); verbose=false) end @testset "Reactive methods definitions" begin notebook = Notebook(Cell.([ raw""" Base.sqrt(s::String) = "sqrt($s)" """, """ string((sqrt(""), rand())) """, "", ])) cell(idx) = notebook.cells[idx] update_run!(, notebook, notebook.cells) output_21 = cell(2).output.body @test contains(output_21, "sqrt()") setcode!(cell(3), """ Base.sqrt(x::Int) = sqrt(Float64(x)^2) """) update_run!(, notebook, cell(3)) output_22 = cell(2).output.body @test cell(3) |> noerror @test cell(2) |> noerror @test cell(1) |> noerror @test output_21 != output_22 # cell2 re-run @test contains(output_22, "sqrt()") setcode!.(notebook.cells, [""]) update_run!(, notebook, notebook.cells) WorkspaceManager.unmake_workspace((, notebook); verbose=false) end @testset "Don't lose basic generic types with macros" begin notebook = Notebook(Cell.([ "f(::Val{1}) = @info x", "f(::Val{2}) = @info x", ])) update_run!(, notebook, notebook.cells) @test notebook.cells[1] |> noerror @test notebook.cells[2] |> noerror end @testset "Variable deletion" begin notebook = Notebook([ Cell("x = 1"), Cell("y = x"), Cell("struct a; x end"), Cell("a") ]) update_run!(, notebook, notebook.cells[1:2]) @test notebook.cells[1].output.body == notebook.cells[2].output.body setcode!(notebook.cells[1], "") update_run!(, notebook, notebook.cells[1]) @test notebook.cells[1] |> noerror @test expecterror(UndefVarError(:x), notebook.cells[2]) update_run!(, notebook, notebook.cells[4]) update_run!(, notebook, notebook.cells[3]) @test notebook.cells[3] |> noerror @test notebook.cells[4] |> noerror update_run!(, notebook, notebook.cells[3]) @test notebook.cells[3] |> noerror @test notebook.cells[4] |> noerror setcode!(notebook.cells[3], "struct a; x; y end") update_run!(, notebook, notebook.cells[3]) @test notebook.cells[3] |> noerror @test notebook.cells[4] |> noerror setcode!(notebook.cells[3], "") update_run!(, notebook, notebook.cells[3]) @test notebook.cells[3] |> noerror @test notebook.cells[4].errored == true WorkspaceManager.unmake_workspace((, notebook); verbose=false) end @testset "Variable cannot reference its previous value" begin notebook = Notebook([ Cell("x = 3") ]) update_run!(, notebook, notebook.cells[1]) setcode!(notebook.cells[1], "x = x + 1") update_run!(, notebook, notebook.cells[1]) @test occursinerror("UndefVarError", notebook.cells[1]) WorkspaceManager.unmake_workspace((, notebook); verbose=false) end notebook = Notebook([ Cell("y = 1"), Cell("f(x) = x + y"), Cell("f(3)"), Cell("g(a,b) = a+b"), Cell("g(5,6)"), Cell("h(x::Int) = x"), Cell("h(7)"), Cell("h(8.0)"), Cell("p(x) = 9"), Cell("p isa Function"), Cell("module Something export a a(x::String) = \"\" end"), Cell("using .Something"), Cell("a(x::Int) = x"), Cell("a(\"i am a \")"), Cell("a(15)"), Cell("module Different export b b(x::String) = \"\" end"), Cell("import .Different: b"), Cell("b(x::Int) = x"), Cell("b(\"i am a \")"), Cell("b(20)"), Cell("module Wow export c c(x::String) = \"\" end"), Cell("begin import .Wow: c c(x::Int) = x end"), Cell("c(\"i am a \")"), Cell("c(24)"), Cell("Ref((25,:fish))"), Cell("begin import Base: show show(io::IO, x::Ref{Tuple{Int,Symbol}}) = write(io, \"\") end"), Cell("Base.isodd(n::Integer) = \"\""), Cell("Base.isodd(28)"), Cell("isodd(29)"), Cell("using Dates"), Cell("year(DateTime(31))"), Cell("Base.tan(::Missing) = 32"), Cell("Base.tan(missing)"), Cell("tan(missing)"), ]) @testset "Changing functions" begin update_run!(, notebook, notebook.cells[2]) @test notebook.cells[2] |> noerror update_run!(, notebook, notebook.cells[1]) update_run!(, notebook, notebook.cells[3]) @test notebook.cells[3].output.body == "4" setcode!(notebook.cells[1], "y = 2") update_run!(, notebook, notebook.cells[1]) @test notebook.cells[3].output.body == "5" @test notebook.cells[2] |> noerror setcode!(notebook.cells[1], "y") update_run!(, notebook, notebook.cells[1]) @test occursinerror("UndefVarError", notebook.cells[1]) @test notebook.cells[2] |> noerror @test occursinerror("UndefVarError", notebook.cells[3]) update_run!(, notebook, notebook.cells[4]) update_run!(, notebook, notebook.cells[5]) @test notebook.cells[5].output.body == "11" setcode!(notebook.cells[4], "g(a) = a+a") update_run!(, notebook, notebook.cells[4]) @test notebook.cells[4] |> noerror @test notebook.cells[5].errored == true setcode!(notebook.cells[5], "g(5)") update_run!(, notebook, notebook.cells[5]) @test notebook.cells[5].output.body == "10" update_run!(, notebook, notebook.cells[6]) update_run!(, notebook, notebook.cells[7]) update_run!(, notebook, notebook.cells[8]) @test notebook.cells[6] |> noerror @test notebook.cells[7] |> noerror @test notebook.cells[8].errored == true setcode!(notebook.cells[6], "h(x::Float64) = 2.0 * x") update_run!(, notebook, notebook.cells[6]) @test notebook.cells[6] |> noerror @test notebook.cells[7].errored == true @test notebook.cells[8] |> noerror update_run!(, notebook, notebook.cells[9:10]) @test notebook.cells[9] |> noerror @test notebook.cells[10].output.body == "true" setcode!(notebook.cells[9], "p = p") update_run!(, notebook, notebook.cells[9]) @test occursinerror("UndefVarError", notebook.cells[9]) setcode!(notebook.cells[9], "p = 9") update_run!(, notebook, notebook.cells[9]) @test notebook.cells[9] |> noerror @test notebook.cells[10].output.body == "false" setcode!(notebook.cells[9], "p(x) = 9") update_run!(, notebook, notebook.cells[9]) @test notebook.cells[9] |> noerror @test notebook.cells[10].output.body == "true" end @testset "Extending imported functions" begin update_run!(, notebook, notebook.cells[11:15]) @test_broken notebook.cells[11] |> noerror @test_broken notebook.cells[12] |> noerror # multiple definitions for `Something` should be okay? == false @test notebook.cells[13] |> noerror @test notebook.cells[14].errored == true # the definition for a was created before `a` was used, so it hides the `a` from `Something` @test notebook.cells[15].output.body == "15" @test_nowarn update_run!(, notebook, notebook.cells[13:15]) @test notebook.cells[13] |> noerror @test notebook.cells[14].errored == true # the definition for a was created before `a` was used, so it hides the `a` from `Something` @test notebook.cells[15].output.body == "15" @test_nowarn update_run!(, notebook, notebook.cells[16:20]) @test notebook.cells[16] |> noerror @test occursinerror("Multiple", notebook.cells[17]) @test occursinerror("Multiple", notebook.cells[18]) @test occursinerror("UndefVarError", notebook.cells[19]) @test occursinerror("UndefVarError", notebook.cells[20]) @test_nowarn update_run!(, notebook, notebook.cells[21:24]) @test notebook.cells[21] |> noerror @test notebook.cells[22] |> noerror @test notebook.cells[23] |> noerror @test notebook.cells[23].output.body == "\"\"" @test notebook.cells[24].output.body == "24" setcode!(notebook.cells[22], "import .Wow: c") @test_nowarn update_run!(, notebook, notebook.cells[22]) @test notebook.cells[22] |> noerror @test notebook.cells[23].output.body == "\"\"" @test notebook.cells[23] |> noerror @test notebook.cells[24].errored == true # the extension should no longer exist # https://github.com/fonsp/Pluto.jl/issues/59 original_repr = Pluto.PlutoRunner.format_output(Ref((25, :fish)))[1] @test_nowarn update_run!(, notebook, notebook.cells[25]) @test notebook.cells[25].output.body isa Dict @test_nowarn update_run!(, notebook, notebook.cells[26]) @test_broken notebook.cells[25].output.body == "" # cell' don't automatically call `show` again when a new overload is defined - that' a minor issue @test_nowarn update_run!(, notebook, notebook.cells[25]) @test notebook.cells[25].output.body == "" setcode!(notebook.cells[26], "") @test_nowarn update_run!(, notebook, notebook.cells[26]) @test_nowarn update_run!(, notebook, notebook.cells[25]) @test notebook.cells[25].output.body isa Dict @test_nowarn update_run!(, notebook, notebook.cells[28:29]) @test notebook.cells[28].output.body == "false" @test notebook.cells[29].output.body == "true" @test_nowarn update_run!(, notebook, notebook.cells[27]) @test notebook.cells[28].output.body == "\"\"" @test notebook.cells[29].output.body == "\"\"" # adding the overload doesn't trigger automatic re-eval because `isodd` doesn't match `Base.isodd` @test_nowarn update_run!(, notebook, notebook.cells[28:29]) @test notebook.cells[28].output.body == "\"\"" @test notebook.cells[29].output.body == "\"\"" setcode!(notebook.cells[27], "") update_run!(, notebook, notebook.cells[27]) @test notebook.cells[28].output.body == "false" @test notebook.cells[29].output.body == "true" # removing the overload doesn't trigger automatic re-eval because `isodd` doesn't match `Base.isodd` update_run!(, notebook, notebook.cells[28:29]) @test notebook.cells[28].output.body == "false" @test notebook.cells[29].output.body == "true" end @testset "Using external libraries" begin update_run!(, notebook, notebook.cells[30:31]) @test notebook.cells[30] |> noerror @test notebook.cells[31].output.body == "31" update_run!(, notebook, notebook.cells[31]) @test notebook.cells[31].output.body == "31" setcode!(notebook.cells[30], "") update_run!(, notebook, notebook.cells[30:31]) @test occursinerror("UndefVarError", notebook.cells[31]) update_run!(, notebook, notebook.cells[32:34]) @test notebook.cells[32] |> noerror @test notebook.cells[33] |> noerror @test notebook.cells[34] |> noerror @test notebook.cells[33].output.body == "32" @test notebook.cells[34].output.body == "32" setcode!(notebook.cells[32], "") update_run!(, notebook, notebook.cells[32]) @test notebook.cells[32] |> noerror @test notebook.cells[33] |> noerror @test notebook.cells[34] |> noerror @test notebook.cells[33].output.body == "missing" @test notebook.cells[34].output.body == "missing" end WorkspaceManager.unmake_workspace((, notebook); verbose=false) @testset "Global assignments inside functions" begin # We currently have a slightly relaxed version of immutable globals: # globals can only be mutated/assigned _in a single cell_. notebook = Notebook([ Cell("x = 1"), Cell("x = 2"), Cell("y = -3; y = 3"), Cell("z = 4"), Cell("let global z = 5 end"), Cell("wowow"), Cell("function floep(x) global wowow = x end"), Cell("floep(8)"), Cell("v"), Cell("function g(x) global v = x end; g(10)"), Cell("g(11)"), Cell("let local r = 0 function f() r = 12 end f() r end"), Cell("apple"), Cell("map(14:14) do i; global apple = orange; end"), Cell("orange = 15"), ]) update_run!(, notebook, notebook.cells[1]) update_run!(, notebook, notebook.cells[2]) @test occursinerror("Multiple definitions for x", notebook.cells[1]) @test occursinerror("Multiple definitions for x", notebook.cells[1]) setcode!(notebook.cells[2], "x + 1") update_run!(, notebook, notebook.cells[2]) @test notebook.cells[1].output.body == "1" @test notebook.cells[2].output.body == "2" update_run!(, notebook, notebook.cells[3]) @test notebook.cells[3].output.body == "3" update_run!(, notebook, notebook.cells[4]) update_run!(, notebook, notebook.cells[5]) @test occursinerror("Multiple definitions for z", notebook.cells[4]) @test occursinerror("Multiple definitions for z", notebook.cells[5]) update_run!(, notebook, notebook.cells[6:7]) @test occursinerror("UndefVarError", notebook.cells[6]) # @test_broken occursinerror("assigns to global", notebook.cells[7]) # @test_broken occursinerror("wowow", notebook.cells[7]) # @test_broken occursinerror("floep", notebook.cells[7]) update_run!(, notebook, notebook.cells[8]) @test_broken !occursinerror("UndefVarError", notebook.cells[6]) update_run!(, notebook, notebook.cells[9:10]) @test !occursinerror("UndefVarError", notebook.cells[9]) @test notebook.cells[10] |> noerror update_run!(, notebook, notebook.cells[11]) @test_broken notebook.cells[9].errored == true @test_broken notebook.cells[10].errored == true @test_broken notebook.cells[11].errored == true update_run!(, notebook, notebook.cells[12]) @test notebook.cells[12].output.body == "12" update_run!(, notebook, notebook.cells[13:15]) @test notebook.cells[13].output.body == "15" @test notebook.cells[14] |> noerror setcode!(notebook.cells[15], "orange = 10005") update_run!(, notebook, notebook.cells[15]) @test notebook.cells[13].output.body == "10005" WorkspaceManager.unmake_workspace((, notebook); verbose=false) end @testset "No top level return" begin notebook = Notebook([ Cell("return 10"), Cell("return (0, 0)"), Cell("return (0, 0)"), Cell("return (0, 0, 0)"), Cell("begin return \"a string\" end"), Cell(""" let return [] end """), Cell("""filter(1:3) do x return true end"""), # create struct to disable the function-generating optimization Cell("struct A1 end; return 10"), Cell("struct A2 end; return (0, 0)"), Cell("struct A3 end; return (0, 0)"), Cell("struct A4 end; return (0, 0, 0)"), Cell("struct A5 end; begin return \"a string\" end"), Cell(""" struct A6 end; let return [] end """), Cell("""struct A7 end; filter(1:3) do x return true end"""), # Function assignments Cell("""f(x) = if x == 1 return false else return true end"""), Cell("""g(x::T) where {T} = if x == 1 return false else return true end"""), Cell("(h(x::T)::MyType) where {T} = return(x)"), Cell("i(x)::MyType = return(x)"), ]) update_run!(, notebook, notebook.cells) @test occursinerror("You can only use return inside a function.", notebook.cells[1]) @test occursinerror("You can only use return inside a function.", notebook.cells[2]) @test occursinerror("You can only use return inside a function.", notebook.cells[3]) @test occursinerror("You can only use return inside a function.", notebook.cells[4]) @test occursinerror("You can only use return inside a function.", notebook.cells[5]) @test occursinerror("You can only use return inside a function.", notebook.cells[6]) @test notebook.cells[7] |> noerror @test occursinerror("You can only use return inside a function.", notebook.cells[8]) @test occursinerror("You can only use return inside a function.", notebook.cells[9]) @test occursinerror("You can only use return inside a function.", notebook.cells[10]) @test occursinerror("You can only use return inside a function.", notebook.cells[11]) @test occursinerror("You can only use return inside a function.", notebook.cells[12]) @test occursinerror("You can only use return inside a function.", notebook.cells[13]) @test notebook.cells[14] |> noerror # Function assignments @test notebook.cells[15] |> noerror @test notebook.cells[16] |> noerror @test notebook.cells[17] |> noerror @test notebook.cells[18] |> noerror WorkspaceManager.unmake_workspace((, notebook); verbose=false) end @testset "Using package from module" begin notebook = Notebook([ Cell("""module A using Dates end"""), Cell(""), Cell("December"), ]) update_run!(, notebook, notebook.cells) @test notebook.cells[begin] |> noerror @test occursinerror("UndefVarError", notebook.cells[end]) setcode!(notebook.cells[2], "using Dates") update_run!(, notebook, [notebook.cells[2]]) @test notebook.cells[1] |> noerror @test notebook.cells[2] |> noerror @test notebook.cells[3] |> noerror WorkspaceManager.unmake_workspace((, notebook); verbose=false) end @testset "Function wrapping" begin notebook = Notebook([ Cell("false && jlaksdfjalskdfj"), Cell("fonsi = 2"), Cell(""" filter(1:fonsi) do x x = sum(1 for z in 1:x) x = sum(1 for z in 1:x) x = sum(1 for z in 1:x) x = sum(1 for z in 1:x) x = sum(1 for z in 1:x) x = sum(1 for z in 1:x) false end |> length """), Cell("4"), Cell("[5]"), Cell("6 / 66"), Cell("false && (seven = 7)"), Cell("seven"), Cell("nine = :identity"), Cell("nine"), Cell("@__FILE__; nine"), Cell("@__FILE__; twelve = :identity"), Cell("@__FILE__; twelve"), Cell("twelve"), Cell("fifteen = :(1 + 1)"), Cell("fifteen"), Cell("@__FILE__; fifteen"), Cell("@__FILE__; eighteen = :(1 + 1)"), Cell("@__FILE__; eighteen"), Cell("eighteen"), Cell("qb = quote value end"), Cell("typeof(qb)"), Cell("qn0 = QuoteNode(:value)"), Cell("qn1 = :(:value)"), Cell("qn0"), Cell("qn1"), Cell(""" named_tuple(obj::T) where {T} = NamedTuple{fieldnames(T),Tuple{fieldtypes(T)...}}(ntuple(i -> getfield(obj, i), fieldcount(T))) """), Cell("named_tuple"), Cell("ln = LineNumberNode(29, \"asdf\")"), Cell("@assert ln isa LineNumberNode"), ]) update_run!(, notebook, notebook.cells) @test notebook.cells[1] |> noerror @test notebook.cells[1].output.body == "false" @test notebook.cells[22].output.body == "Expr" @test notebook.cells[25].output.body == ":(:value)" @test notebook.cells[26].output.body == ":(:value)" function benchmark(fonsi) filter(1:fonsi) do x x = sum(1 for z in 1:x) x = sum(1 for z in 1:x) x = sum(1 for z in 1:x) x = sum(1 for z in 1:x) x = sum(1 for z in 1:x) x = sum(1 for z in 1:x) false end |> length end bad = @elapsed benchmark(2) good = 0.01 * @elapsed for i in 1:100 benchmark(2) end update_run!(, notebook, notebook.cells) @test 0.1 * good < notebook.cells[3].runtime / 1.0e9 < 0.5 * bad old = notebook.cells[4].output.body setcode!(notebook.cells[4], "4.0") update_run!(, notebook, notebook.cells[4]) @test old != notebook.cells[4].output.body old = notebook.cells[5].output.body setcode!(notebook.cells[5], "[5.0]") update_run!(, notebook, notebook.cells[5]) @test old != notebook.cells[5].output.body old = notebook.cells[6].output.body setcode!(notebook.cells[6], "66 / 6") update_run!(, notebook, notebook.cells[6]) @test old != notebook.cells[6].output.body @test notebook.cells[7] |> noerror @test notebook.cells[7].output.body == "false" @test occursinerror("UndefVarError", notebook.cells[8]) @test notebook.cells[9].output.body == ":identity" @test notebook.cells[10].output.body == ":identity" @test notebook.cells[11].output.body == ":identity" @test notebook.cells[12].output.body == ":identity" @test notebook.cells[13].output.body == ":identity" @test notebook.cells[14].output.body == ":identity" @test notebook.cells[15].output.body == ":(1 + 1)" @test notebook.cells[16].output.body == ":(1 + 1)" @test notebook.cells[17].output.body == ":(1 + 1)" @test notebook.cells[18].output.body == ":(1 + 1)" @test notebook.cells[19].output.body == ":(1 + 1)" @test notebook.cells[20].output.body == ":(1 + 1)" @test notebook.cells[27] |> noerror @test notebook.topology.codes[notebook.cells[27]].function_wrapped == false @test notebook.cells[28] |> noerror update_run!(, notebook, notebook.cells[29:30]) @test notebook.cells[29] |> noerror @test notebook.cells[30] |> noerror WorkspaceManager.unmake_workspace((, notebook); verbose=false) @testset "Expression hash" begin same(a,b) = Pluto.PlutoRunner.expr_hash(a) == Pluto.PlutoRunner.expr_hash(b) @test same(:(1), :(1)) @test !same(:(1), :(1.0)) @test same(:(x + 1), :(x + 1)) @test !same(:(x + 1), :(x + 1.0)) @test same(:(1 |> a |> a |> a), :(1 |> a |> a |> a)) @test same(:(a(b(1,2))), :(a(b(1,2)))) @test !same(:(a(b(1,2))), :(a(b(1,3)))) @test !same(:(a(b(1,2))), :(a(b(1,1)))) @test !same(:(a(b(1,2))), :(a(b(2,1)))) end end @testset "Broadcast bug - Issue #2211" begin notebook = Notebook(Cell.([ "abstract type AbstractFoo{T} <: AbstractMatrix{T} end", "struct X{T} <: AbstractFoo{T} end", "convert(::Type{AbstractArray{T}}, S::AbstractFoo) where {T<:Number} = convert(AbstractFoo{T}, S)", "Base.convert(::Type{AbstractArray{T}}, ::AbstractFoo) where {T} = nothing", "Base.size(::AbstractFoo) = (2,2)", "Base.getindex(::AbstractFoo{T}, args...) where {T} = one(T)", "x = X{Float64}()", "y = zeros(2,)", "x, y", ])) update_run!(, notebook, notebook.cells) @test all(noerror, notebook.cells) end @testset "ParseError messages" begin notebook = Notebook(Cell.([ "begin", "\n\nend", "throw(Meta.parse(\"begin\"; raise=false).args[end])", ])) update_run!(, notebook, notebook.cells) @test Pluto.is_just_text(notebook.topology, notebook.cells[1]) @test Pluto.is_just_text(notebook.topology, notebook.cells[2]) @test notebook.cells[1].errored @test notebook.cells[2].errored @test notebook.cells[3].errored @test haskey(notebook.cells[1].output.body, :source) @test haskey(notebook.cells[1].output.body, :diagnostics) @test haskey(notebook.cells[2].output.body, :source) @test haskey(notebook.cells[2].output.body, :diagnostics) # not literal syntax error @test haskey(notebook.cells[3].output.body, :msg) @test !haskey(notebook.cells[3].output.body, :source) @test !haskey(notebook.cells[3].output.body, :diagnostics) end @testset "using .LocalModule" begin notebook = Notebook(Cell.([ """ begin @eval module LocalModule const x = :exported export x end using .LocalModule end """, "x" ])) update_run!(, notebook, notebook.cells) @test notebook.cells[1] |> noerror @test notebook.cells[2] |> noerror output_2 = notebook.cells[2].output.body @test contains(output_2, "exported") setcode!( notebook.cells[1], """ begin @eval module LocalModule const x = :not_exported end using .LocalModule end """, ) update_run!(, notebook, [notebook.cells[1]]) @test expecterror(UndefVarError(:x), notebook.cells[end]) end end using Test import Pluto: Configuration, Notebook, ServerSession, ClientSession, update_run!, Cell, WorkspaceManager, SessionActions, save_notebook import Pluto.Configuration: Options, EvaluationOptions using Pluto.WorkspaceManager: poll import Pkg function retry(f::Function, n) try f() catch e if n > 0 retry(f, n - 1) else rethrow(e) end end end @testset "Reload from file" begin retry(3) do  = ServerSession() .options.evaluation.workspace_use_distributed = false .options.server.auto_reload_from_file = true timeout_between_tests = .options.server.auto_reload_from_file_cooldown * 1.5 notebook = SessionActions.new(; run_async=false) ### sleep(timeout_between_tests) nb1 = Notebook([ Cell("x = 123"), Cell("x + x"), Cell("rand()"), ]) file1 = sprint(Pluto.save_notebook, nb1) write(notebook.path, file1) @assert poll(30) do length(notebook.cells) == 3 end @assert poll(5) do notebook.cells[1].output.body == "123" end @assert poll(5) do all(c -> !c.running, notebook.cells) end @assert notebook.cells[2].output.body == "246" @assert notebook.cells[3] |> noerror original_rand_output = notebook.cells[3].output.body ### sleep(timeout_between_tests) nb2 = Notebook(reverse(nb1.cells)) file2 = sprint(Pluto.save_notebook, nb2) write(notebook.path, file2) @assert poll(10) do notebook.cells[3].output.body == "123" end # notebook order reversed, but cell should not re-run @assert original_rand_output == notebook.cells[1].output.body ### sleep(timeout_between_tests) file3 = replace(file1, "123" => "6") write(notebook.path, file3) @assert poll(10) do notebook.cells[1].output.body == "6" end @assert poll(5) do all(c -> !c.running, notebook.cells) end @assert notebook.cells[2].output.body == "12" # notebook order reversed again, but cell should not re-run @assert original_rand_output == notebook.cells[3].output.body ### sleep(timeout_between_tests) file4 = read(notebook.path, String) last_hot_reload_time4 = notebook.last_hot_reload_time notebook.cells[3].code_folded = true save_notebook(notebook) sleep(timeout_between_tests) file5 = read(notebook.path, String) @test file4 != file5 @test notebook.cells[3].code_folded write(notebook.path, file4) @assert poll(10) do notebook.cells[3].code_folded == false end # cell folded, but cell should not re-run @assert original_rand_output == notebook.cells[3].output.body @assert poll(10) do last_hot_reload_time5 = notebook.last_hot_reload_time last_hot_reload_time5 != last_hot_reload_time4 end ### sleep(timeout_between_tests) file6 = read(notebook.path, String) Pluto.set_disabled(notebook.cells[3], true) save_notebook(notebook) sleep(timeout_between_tests) file7 = read(notebook.path, String) @assert file6 != file7 @assert Pluto.is_disabled(notebook.cells[3]) write(notebook.path, file6) @assert poll(10) do !Pluto.is_disabled(notebook.cells[3]) end # cell disabled and re-enabled, so it should re-run @assert poll(10) do original_rand_output != notebook.cells[3].output.body end ### sleep(timeout_between_tests) file8 = read(joinpath(@__DIR__, "packages", "simple_stdlib_import.jl"), String) write(notebook.path, file8) @assert poll(10) do notebook.cells[2].output.body == "false" end @assert length(notebook.cells) == 2 @assert notebook.nbpkg_restart_required_msg !== nothing end @test true end using Test import Pluto import Pluto: update_run!, WorkspaceManager, ClientSession, ServerSession, Notebook, Cell function withref(f::Function, ref::Ref, x) oldval = ref[] try ref[] = x f() finally ref[] = oldval end end @testset "Rich output" begin  = ServerSession() .options.evaluation.workspace_use_distributed = false @testset "Tree viewer" begin @testset "Basics" begin notebook = Notebook([ Cell("[1,1,[1]]"), Cell("Dict(:a => [:b, :c])"), Cell("[3, Dict()]"), Cell("[4,[3, Dict()]]"), Cell("[5, missing, 5]"), Cell("[]"), Cell("(7,7)"), Cell("(a=8,b=[8])"), Cell("Ref(9)"), Cell("Vector(undef, 10)"), Cell("struct Eleven x end"), Cell("Eleven(12)"), Cell("struct Amazing{T} x::T end"), Cell("Amazing(14)"), Cell("1:15"), Cell("""begin mutable struct A x; y end a = A(16, 0) a.y = (2, Dict(6 => a)) a end"""), Cell("Set([17:20,\"Wonderful\"])"), Cell("Set(0 : 0.1 : 18)"), Cell("rand(50,50)"), Cell("rand(500,500)"), Cell("[ rand(50,50) ]"), Cell("[ rand(500,500) ]"), ]) update_run!(, notebook, notebook.cells) @test notebook.cells[1].output.mime isa MIME"application/vnd.pluto.tree+object" @test notebook.cells[2].output.mime isa MIME"application/vnd.pluto.tree+object" @test notebook.cells[3].output.mime isa MIME"application/vnd.pluto.tree+object" @test notebook.cells[4].output.mime isa MIME"application/vnd.pluto.tree+object" @test notebook.cells[5].output.mime isa MIME"application/vnd.pluto.tree+object" @test notebook.cells[6].output.mime isa MIME"application/vnd.pluto.tree+object" @test notebook.cells[7].output.mime isa MIME"application/vnd.pluto.tree+object" @test notebook.cells[8].output.mime isa MIME"application/vnd.pluto.tree+object" @test notebook.cells[9].output.mime isa MIME"application/vnd.pluto.tree+object" @test notebook.cells[10].output.mime isa MIME"application/vnd.pluto.tree+object" @test notebook.cells[1].output.body isa Dict @test notebook.cells[2].output.body isa Dict @test notebook.cells[3].output.body isa Dict @test notebook.cells[4].output.body isa Dict @test notebook.cells[5].output.body isa Dict @test notebook.cells[6].output.body isa Dict @test notebook.cells[7].output.body isa Dict @test notebook.cells[8].output.body isa Dict @test notebook.cells[9].output.body isa Dict @test notebook.cells[10].output.body isa Dict @test notebook.cells[12].output.mime isa MIME"application/vnd.pluto.tree+object" @test notebook.cells[12].output.body isa Dict @test notebook.cells[14].output.mime isa MIME"application/vnd.pluto.tree+object" @test notebook.cells[14].output.body isa Dict @test notebook.cells[15].output.mime isa MIME"text/plain" @test notebook.cells[16].output.mime isa MIME"application/vnd.pluto.tree+object" @test notebook.cells[16].output.body isa Dict @test occursin("circular", notebook.cells[16].output.body |> string) @test notebook.cells[17].output.body isa Dict @test length(notebook.cells[17].output.body[:elements]) == 2 @test notebook.cells[17].output.body[:prefix] == "Set{Any}" @test notebook.cells[17].output.mime isa MIME"application/vnd.pluto.tree+object" @test occursin("Set", notebook.cells[17].output.body |> string) @test notebook.cells[18].output.body isa Dict @test length(notebook.cells[18].output.body[:elements]) < 180 @test notebook.cells[18].output.body[:prefix] == "Set{Float64}" @test notebook.cells[18].output.mime isa MIME"application/vnd.pluto.tree+object" sizes = [length(string(notebook.cells[i].output.body)) for i in 19:22] # without truncation, we would have sizes[2]  sizes[1] * 10 * 10 # with truncation, their displayed sizes should be similar @test sizes[2] < sizes[1] * 1.5 @test sizes[4] < sizes[3] * 1.5 WorkspaceManager.unmake_workspace((, notebook); verbose=false) end @testset "Overloaded Base.show" begin notebook = Notebook([ Cell("""begin struct A x end Base.show(io::IO, ::MIME"image/png", x::Vector{A}) = print(io, "1") [A(1), A(1)] end"""), Cell("""begin struct B x end Base.show(io::IO, ::MIME"text/html", x::B) = print(io, "2") B(2) end"""), Cell("""begin struct C x end Base.show(io::IO, ::MIME"text/plain", x::C) = print(io, "3") C(3) end"""), ]) update_run!(, notebook, notebook.cells) @test notebook.cells[1].output.mime isa MIME"image/png" @test notebook.cells[1].output.body == codeunits("1") @test notebook.cells[2].output.mime isa MIME"text/html" @test notebook.cells[2].output.body == "2" @test notebook.cells[3].output.mime isa MIME"text/plain" @test notebook.cells[3].output.body == "3" WorkspaceManager.unmake_workspace((, notebook); verbose=false) end @testset "Special arrays" begin .options.evaluation.workspace_use_distributed = true notebook = Notebook([ Cell("using OffsetArrays"), Cell("OffsetArray(zeros(3), 20:22)"), Cell(""" begin struct BadImplementation <: AbstractVector{Int64} end function Base.show(io::IO, ::MIME"text/plain", b::BadImplementation) write(io, "fallback") end end """), Cell(""" begin struct OneTwoThree <: AbstractVector{Int64} end function Base.show(io::IO, ::MIME"text/plain", b::OneTwoThree) write(io, "fallback") end Base.size(::OneTwoThree) = (3,) Base.getindex(::OneTwoThree, i) = 100 + i end """), Cell("BadImplementation()"), Cell("OneTwoThree()"), ]) update_run!(, notebook, notebook.cells) @test notebook.cells[2].output.mime isa MIME"application/vnd.pluto.tree+object" s = string(notebook.cells[2].output.body) @test occursin("OffsetArray", s) @test occursin("21", s) # once in the prefix, once as index @test count("22", s) >= 2 @test notebook.cells[5].output.mime isa MIME"text/plain" @test notebook.cells[5].output.body == "fallback" @test notebook.cells[6].output.mime isa MIME"application/vnd.pluto.tree+object" s = string(notebook.cells[6].output.body) @test occursin("OneTwoThree", s) @test occursin("101", s) @test occursin("102", s) @test occursin("103", s) cleanup(, notebook) .options.evaluation.workspace_use_distributed = false end @testset "Circular references" begin notebook = Notebook([ Cell("""let x = Any[1,2,3] push!(x,x) push!(x,[x]) push!(x,(a=x,)) push!(x,:b=>x) end"""), Cell("""let x = Set(Any[1,2,3]) push!(x,x) end"""), Cell("""let x = Dict{Any,Any}(1 => 2, 3 => 4) x[5] = (123, x) end"""), Cell("""let x = Ref{Any}(123) x[] = x end"""), Cell("""let x = Ref{Any}(123) x[] = (1,x) end"""), ]) update_run!(, notebook, notebook.cells) @test notebook.cells[1] |> noerror @test notebook.cells[2] |> noerror @test notebook.cells[3] |> noerror @test notebook.cells[4] |> noerror end end @testset "Markdown" begin notebook = Notebook([ Cell("md\"# Why we need more \""), ]) update_run!(, notebook, notebook.cells) @test notebook.cells[1] |> noerror @test notebook.cells[1].output.mime isa MIME"text/html" r = notebook.cells[1].output.body @test occursin("id=\"Why-we-need-more-\"", r) @test occursin("Why we need more </h1>", r) end @testset "embed_display" begin .options.evaluation.workspace_use_distributed = false notebook = Notebook([ Cell("x = randn(10)"), Cell(raw"md\"x = $(embed_display(x))\"") ]) update_run!(, notebook, notebook.cells) @test notebook.cells[1] |> noerror @test notebook.cells[2] |> noerror @test notebook.cells[2].output.body isa String @test occursin("getPublishedObject", notebook.cells[2].output.body) end @testset "Table viewer" begin .options.evaluation.workspace_use_distributed = true notebook = Notebook([ Cell("using DataFrames, Tables"), Cell("DataFrame()"), Cell("DataFrame(:a => [])"), Cell("DataFrame(:a => [1,2,3], :b => [999, 5, 6])"), Cell("DataFrame(rand(20,20), :auto)"), Cell("DataFrame(rand(2000,20), :auto)"), Cell("DataFrame(rand(20,2000), :auto)"), Cell("@view DataFrame(rand(100,3), :auto)[:, 2:2]"), Cell("@view DataFrame(rand(3,100), :auto)[2:2, :]"), Cell("DataFrame"), Cell("Tables.table(rand(11,11))"), Cell("Tables.table(rand(120,120))"), Cell("""DataFrame(:a => ["missing", missing])"""), # the next three are technically "tables" according to `Tables.istable`, but I don't want to use the Table viewer for them. Cell("""[Dict(Symbol("x\$i") => i for i in 1:140)]"""), Cell("""Dict( :a => [15,15], :b => [15,15] )"""), Cell("""[ (a=16, b=16,) (a=16, b=16,) ]"""), Cell("Union{}[]"), ]) update_run!(, notebook, notebook.cells) # @test notebook.cells[2].output.mime isa MIME"application/vnd.pluto.table+object" # @test notebook.cells[3].output.mime isa MIME"application/vnd.pluto.table+object" @test notebook.cells[4].output.mime isa MIME"application/vnd.pluto.table+object" @test notebook.cells[5].output.mime isa MIME"application/vnd.pluto.table+object" @test notebook.cells[6].output.mime isa MIME"application/vnd.pluto.table+object" @test notebook.cells[7].output.mime isa MIME"application/vnd.pluto.table+object" @test notebook.cells[8].output.mime isa MIME"application/vnd.pluto.table+object" @test notebook.cells[9].output.mime isa MIME"application/vnd.pluto.table+object" @test notebook.cells[11].output.mime isa MIME"application/vnd.pluto.table+object" @test notebook.cells[12].output.mime isa MIME"application/vnd.pluto.table+object" @test notebook.cells[14].output.mime isa MIME"application/vnd.pluto.tree+object" @test notebook.cells[15].output.mime isa MIME"application/vnd.pluto.tree+object" @test notebook.cells[16].output.mime isa MIME"application/vnd.pluto.tree+object" @test notebook.cells[17].output.mime isa MIME"application/vnd.pluto.tree+object" # @test notebook.cells[2].output.body isa Dict # @test notebook.cells[3].output.body isa Dict @test notebook.cells[4].output.body isa Dict @test notebook.cells[5].output.body isa Dict @test notebook.cells[6].output.body isa Dict @test notebook.cells[7].output.body isa Dict @test notebook.cells[8].output.body isa Dict @test notebook.cells[9].output.body isa Dict @test notebook.cells[11].output.body isa Dict @test notebook.cells[12].output.body isa Dict @test notebook.cells[14].output.body isa Dict @test notebook.cells[15].output.body isa Dict @test notebook.cells[16].output.body isa Dict @test notebook.cells[17].output.body isa Dict @test occursin("String?", string(notebook.cells[13].output.body)) # Issue 1490. @test notebook.cells[10].output.mime isa MIME"text/plain" @test notebook.cells[10] |> noerror @test notebook.cells[17] |> noerror # Issue 1815 # to see if we truncated correctly, we convert the output to string and check how big it is # because we don't want to test too specifically roughsize(x) = length(string(x)) smallsize = roughsize(notebook.cells[5]) manyrowssize = roughsize(notebook.cells[6]) manycolssize = roughsize(notebook.cells[7]) @test manyrowssize < 50 * smallsize @test manycolssize < 50 * smallsize # TODO: test lazy loading more rows/cols cleanup(, notebook) .options.evaluation.workspace_use_distributed = false end begin escape_me = "16 \\ \" ' / \b \f \n \r \t  \x10 \$" notebook = Notebook([ Cell("a\\"), Cell("1 = 2"), Cell("b = 3.0\nb = 3"), Cell("\n# uhm\n\nc = 4\n\n# wowie \n\n"), Cell("d = 5;"), Cell("e = 6; f = 6"), Cell("g = 7; h = 7;"), Cell("\n\n0 + 8; 0 + 8;\n\n\n"), Cell("0 + 9; 9;\n\n\n"), Cell("0 + 10;\n10;"), Cell("0 + 11;\n11"), ]) @testset "Strange code" begin update_run!(, notebook, notebook.cells[1]) update_run!(, notebook, notebook.cells[2]) @test notebook.cells[1].errored == true @test notebook.cells[2].errored == true end @testset "Mutliple expressions & semicolon" begin update_run!(, notebook, notebook.cells[3:end]) @test occursinerror("syntax: extra token after", notebook.cells[3]) @test notebook.cells[4] |> noerror @test notebook.cells[4].output.body == "4" @test notebook.cells[4].output.rootassignee == :c @test notebook.cells[5] |> noerror @test notebook.cells[5].output.body == "" @test notebook.cells[5].output.rootassignee === nothing @test notebook.cells[6] |> noerror @test notebook.cells[6].output.body == "6" @test notebook.cells[6].output.rootassignee === nothing @test notebook.cells[7] |> noerror @test notebook.cells[7].output.body == "" @test notebook.cells[7].output.rootassignee === nothing @test notebook.cells[8] |> noerror @test notebook.cells[8].output.body == "" @test notebook.cells[9] |> noerror @test notebook.cells[9].output.body == "" @test occursinerror("syntax: extra token after", notebook.cells[10]) @test occursinerror("syntax: extra token after", notebook.cells[11]) end WorkspaceManager.unmake_workspace((, notebook); verbose=false) end @testset "Stack traces" begin escape_me = "16 \\ \" ' / \b \f \n \r \t  \x10 \$" codes = [ "sqrt(-1)", "let\n\nsqrt(-2)\nend", "\"Something very exciting!\"\nfunction w(x)\n\tsqrt(x)\nend", "w(-4)", "error(" * sprint(Base.print_quoted, escape_me) * ")", "6", ] notebook1 = Notebook([ Cell(code) for (i, code) in enumerate(codes) ]) # create struct to disable the function-generating optimization notebook2 = Notebook([ Cell("struct S$(i) end; $code") for (i, code) in enumerate(codes) ]) @testset "$(wrapped ? "With" : "Without") function wrapping" for wrapped in [false, true] notebook = wrapped ? notebook1 : notebook2 @test_nowarn update_run!(, notebook, notebook.cells[1:5]) @test occursinerror("DomainError", notebook.cells[1]) @test occursin("DomainError", notebook.cells[1].output.body[:plain_error]) let st = notebook.cells[1].output.body @test length(st[:stacktrace]) == 4 # check in REPL if Pluto.can_insert_filename @test st[:stacktrace][4][:line] == 1 @test occursin(notebook.cells[1].cell_id |> string, st[:stacktrace][4][:file]) @test occursin(notebook.path |> basename, st[:stacktrace][4][:file]) else @test_broken false end end @test occursinerror("DomainError", notebook.cells[2]) let st = notebook.cells[2].output.body @test length(st[:stacktrace]) == 4 if Pluto.can_insert_filename @test st[:stacktrace][4][:line] == 3 @test occursin(notebook.cells[2].cell_id |> string, st[:stacktrace][4][:file]) @test occursin(notebook.path |> basename, st[:stacktrace][4][:file]) else @test_broken false end end @test occursinerror("DomainError", notebook.cells[4]) let st = notebook.cells[4].output.body @test length(st[:stacktrace]) == 5 if Pluto.can_insert_filename @test st[:stacktrace][4][:line] == 3 @test occursin(notebook.cells[3].cell_id |> string, st[:stacktrace][4][:file]) @test occursin(notebook.path |> basename, st[:stacktrace][4][:file]) @test st[:stacktrace][5][:line] == 1 @test occursin(notebook.cells[4].cell_id |> string, st[:stacktrace][5][:file]) @test occursin(notebook.path |> basename, st[:stacktrace][5][:file]) else @test_broken false end end let st = notebook.cells[5].output.body @test occursin(escape_me, st[:msg]) end @testset "PlutoStaticHTML API" begin @testset "before" begin @test notebook.cells[6] |> noerror st = notebook.cells[1].output.body @test occursin(r"domain"i, st[:msg]) @test st[:stacktrace] isa Vector @test st[:stacktrace][1] isa Dict end @test .options.evaluation.workspace_use_distributed == false withref(PlutoRunner.PRETTY_STACKTRACES, false) do update_run!(, notebook, notebook.cells[1]) @testset "after" begin @test notebook.cells[6] |> noerror st = notebook.cells[1].output.body @test occursin(r"domain"i, st[:msg]) @test st[:stacktrace] isa Base.CapturedException end end end WorkspaceManager.unmake_workspace((, notebook); verbose=false) end end end import Pluto: Throttled using Pluto.WorkspaceManager: poll @testset "Throttled" begin x = Ref(0) function f() x[] += 1 end f() # f was not throttled @test x[] == 1 dt = 8 / 100 tf = Throttled.throttled(f, dt) for x in 1:10 tf() end # we have an initial cooldown period in which f should not fire... # ...so x is still 1... @test x[] == 1 sleep(1.5dt) # ...but after a delay, the call should go through. @test x[] == 2 # Let's wait for the cooldown period to end sleep(3dt) # nothing should have changed @test x[] == 2 # sleep(0) ## ASYNC MAGIC :( # at this point, the *initial* cooldown period is over # and the cooldown period for the first throttled calls is over for x in 1:10 tf() end # we want to send plots to the user as soon as they are available, # so no leading timeout @test x[] == 3 # the 2nd until 10th calls were still queued sleep(3dt) @test x[] == 4 for x in 1:5 tf() sleep(1.5dt) end @test x[] == 9 sleep(3dt) @test x[] == 9 ### # "call 1" tf() # no leading timeout, immediately set to 10 @test x[] == 10 sleep(.5dt) # "call 2" tf() # throttled @test x[] == 10 sleep(.7dt) # we waited 1.2dt > dt seconds since "call 1", which should have started the dt cooldown. "call 2" landed during that calldown, and should have triggered by now @test x[] == 11 sleep(3dt) @test x[] == 11 ### tf() tf() @test x[] == 12 flush(tf) @test x[] == 13 sleep(3dt) @test x[] == 13 #### tf() @test poll(3dt, dt/60) do x[] == 14 end # immediately fire again, right after the last fire tf() tf() # this should not do anything, because we are still in the cooldown period @test x[] == 14 # not even after a little while sleep(0.1dt) @test x[] == 14 # but eventually, our call should get queued sleep(3dt) @test x[] == 15 #### x[] = 0 @test tf.iscoolnow[] @test !tf.run_later[] Throttled.force_throttle_without_run(tf) @test !tf.iscoolnow[] @test !tf.run_later[] @test x[] == 0 tf() @test !tf.iscoolnow[] @test tf.run_later[] @test x[] == 0 sleep(.1dt) @test x[] == 0 sleep(3dt) @test x[] == 1 @test tf.iscoolnow[] @test !tf.run_later[] tf() @test x[] == 2 sleep(.1dt) tf() Throttled.force_throttle_without_run(tf) @test x[] == 2 sleep(3dt) @test x[] == 2 tf() @test x[] == 3 sleep(3dt) #### end using Test using Pluto.Configuration: CompilerOptions using Pluto.WorkspaceManager: _merge_notebook_compiler_options import Pluto: update_save_run!, update_run!, WorkspaceManager, ClientSession, ServerSession, Notebook, Cell, project_relative_path import Malt @testset "Workspace manager" begin # basic functionality is already tested by the reactivity tests @testset "Multiple notebooks" begin  = ServerSession() .options.evaluation.workspace_use_distributed = true notebookA = Notebook([ Cell("x = 3") ]) notebookB = Notebook([ Cell("x") ]) @test notebookA.path != notebookB.path Sys.iswindows() && sleep(.5) # workaround for https://github.com/JuliaLang/julia/issues/39270 update_save_run!(, notebookA, notebookA.cells[1]) Sys.iswindows() && sleep(.5) # workaround for https://github.com/JuliaLang/julia/issues/39270 update_save_run!(, notebookB, notebookB.cells[1]) @test notebookB.cells[1].errored == true Sys.iswindows() && sleep(.5) # workaround for https://github.com/JuliaLang/julia/issues/39270 WorkspaceManager.unmake_workspace((, notebookA)) Sys.iswindows() && sleep(.5) # workaround for https://github.com/JuliaLang/julia/issues/39270 WorkspaceManager.unmake_workspace((, notebookB)) end @testset "Variables with secret names" begin  = ServerSession() .options.evaluation.workspace_use_distributed = false notebook = Notebook([ Cell("result = 1"), Cell("result"), Cell("elapsed_ns = 3"), Cell("elapsed_ns"), ]) update_save_run!(, notebook, notebook.cells[1:4]) @test notebook.cells[1].output.body == "1" @test notebook.cells[2].output.body == "1" @test notebook.cells[3].output.body == "3" @test notebook.cells[4].output.body == "3" WorkspaceManager.unmake_workspace((, notebook); verbose=false) end Sys.iswindows() || @testset "Pluto inside Pluto" begin  = ServerSession() .options.evaluation.capture_stdout = false .options.evaluation.workspace_use_distributed_stdlib = false notebook = Notebook([ Cell("""begin import Pkg Pkg.activate() empty!(LOAD_PATH) push!(LOAD_PATH, $(repr(Base.load_path()))...) import Pluto end"""), Cell(""" begin s = Pluto.ServerSession() s.options.evaluation.workspace_use_distributed_stdlib = false end """), Cell(""" nb = Pluto.SessionActions.open(s, Pluto.project_relative_path("sample", "Tower of Hanoi.jl"); run_async=false, as_sample=true) """), Cell("length(nb.cells)"), Cell(""), ]) update_run!(, notebook, notebook.cells) @test notebook.cells[1] |> noerror @test notebook.cells[2] |> noerror @test notebook.cells[3] |> noerror @test notebook.cells[4] |> noerror @test notebook.cells[5] |> noerror setcode!(notebook.cells[5], "length(nb.cells)") update_run!(, notebook, notebook.cells[5]) @test notebook.cells[5] |> noerror setcode!(notebook.cells[5], "Pluto.SessionActions.shutdown(s, nb)") update_run!(, notebook, notebook.cells[5]) @test noerror(notebook.cells[5]) cleanup(, notebook) end end using Test using Pluto using Pluto: update_run!, ServerSession, ClientSession, Cell, Notebook, set_disabled, is_disabled, WorkspaceManager @testset "Cell Disabling" begin  = ServerSession() .options.evaluation.workspace_use_distributed = false notebook = Notebook([ Cell("const a = 1") Cell("const b = 2") Cell("const c = 3") Cell("const d = 4") Cell("const x = a") # 5 # these cells will be uncommented later Cell("# const x = b") # 6 Cell("# const x = c") # 7 Cell("const z = x") # 8 Cell("# const z = d") # 9 Cell("const y = z") # 10 Cell("things = []") # 11 Cell("""begin cool = 1 push!(things, 1) end""") # 12 Cell("""begin # cool = 2 # push!(things, 2) end""") # 13 Cell("cool; length(things)") # 14 ]) update_run!(, notebook, notebook.cells) # helper functions id(i) = notebook.cells[i].cell_id c(i) = notebook.cells[i] get_indirectly_disabled_cells(notebook) = [i for (i, c) in pairs(notebook.cells) if c.depends_on_disabled_cells] @test !any(is_disabled, notebook.cells) @test get_indirectly_disabled_cells(notebook) == [] @test all(noerror, notebook.cells) ### setcode!(c(6), "const x = b") update_run!(, notebook, c(6)) @test c(5).errored @test c(6).errored @test c(8).errored @test c(10).errored @test get_indirectly_disabled_cells(notebook) == [] ### set_disabled(c(1), true) update_run!(, notebook, c(1)) @test noerror(c(1)) @test noerror(c(6)) @test noerror(c(8)) @test noerror(c(10)) @test get_indirectly_disabled_cells(notebook) == [1, 5] update_run!(, notebook, c(5:6)) @test noerror(c(1)) @test noerror(c(6)) @test noerror(c(8)) @test noerror(c(10)) @test get_indirectly_disabled_cells(notebook) == [1, 5] ### set_disabled(c(1), false) update_run!(, notebook, c(1)) @test noerror(c(1)) @test c(5).errored @test c(6).errored @test c(8).errored @test c(10).errored @test get_indirectly_disabled_cells(notebook) == [] ### set_disabled(c(5), true) update_run!(, notebook, c(5)) @test noerror(c(1)) @test noerror(c(6)) @test noerror(c(8)) @test noerror(c(10)) @test get_indirectly_disabled_cells(notebook) == [5] ### set_disabled(c(1), true) update_run!(, notebook, c(1)) @test noerror(c(1)) @test noerror(c(6)) @test noerror(c(8)) @test noerror(c(10)) @test get_indirectly_disabled_cells(notebook) == [1, 5] ### set_disabled(c(5), false) setcode!(c(7), "const x = c") update_run!(, notebook, c([5,7])) @test c(5).errored @test c(6).errored @test c(7).errored @test c(8).errored @test c(10).errored @test get_indirectly_disabled_cells(notebook) == [1, 5] ### set_disabled(c(2), true) update_run!(, notebook, c(2)) @test noerror(c(3)) @test noerror(c(7)) @test noerror(c(8)) @test noerror(c(10)) @test get_indirectly_disabled_cells(notebook) == [1, 2, 5, 6] ### setcode!(c(9), "const z = d") update_run!(, notebook, c([9])) @test noerror(c(7)) @test c(8).errored @test c(9).errored @test c(10).errored @test get_indirectly_disabled_cells(notebook) == [1, 2, 5, 6] ### set_disabled(c(4), true) update_run!(, notebook, c(4)) @test noerror(c(3)) @test noerror(c(4)) @test noerror(c(7)) @test noerror(c(8)) @test noerror(c(10)) @test get_indirectly_disabled_cells(notebook) == [1, 2, 4, 5, 6, 9] ### set_disabled(c(1), true) set_disabled(c(2), false) set_disabled(c(3), true) set_disabled(c(4), false) set_disabled(c(5), true) set_disabled(c(6), true) set_disabled(c(7), false) set_disabled(c(8), false) set_disabled(c(9), true) setcode!(c(10), "const x = 123123") set_disabled(c(10), false) update_run!(, notebook, c(1:10)) @test noerror(c(1)) @test noerror(c(2)) @test noerror(c(3)) @test noerror(c(4)) @test noerror(c(8)) @test noerror(c(10)) @test get_indirectly_disabled_cells(notebook) == [1, 3, 5, 6, 7, 9] ### set_disabled(c(3), false) update_run!(, notebook, c(3)) @test get_indirectly_disabled_cells(notebook) == [1, 5, 6, 9] @test c(7).errored @test c(8).errored @test c(10).errored ### set_disabled(c(10), true) update_run!(, notebook, c(10)) @test get_indirectly_disabled_cells(notebook) == [1, 5, 6, 9, 10] @test noerror(c(7)) @test noerror(c(8)) ### set_disabled(c(7), true) set_disabled(c(10), false) update_run!(, notebook, c([7,10])) @test get_indirectly_disabled_cells(notebook) == [1, 5, 6, 7, 9] @test noerror(c(7)) @test noerror(c(8)) @test noerror(c(10)) ### check that they really don't run when disabled @test c(14).output.body == "1" setcode!(c(13), replace(c(13).code, "#" => "")) update_run!(, notebook, c([11,13])) @test c(12).errored @test c(13).errored @test c(14).errored set_disabled(c(13), true) update_run!(, notebook, c([13])) @test noerror(c(12)) @test noerror(c(14)) @test c(14).output.body == "1" update_run!(, notebook, c([11])) @test c(14).output.body == "1" update_run!(, notebook, c([12])) update_run!(, notebook, c([12])) @test c(14).output.body == "3" cleanup(, notebook) end @testset "Cell Disabling 1" begin  = ServerSession() .options.evaluation.workspace_use_distributed = false notebook = Notebook([ Cell("""y = begin 1 + x end"""), Cell("x = 2"), Cell("z = sqrt(y)"), Cell("a = 4x"), Cell("w = z^5"), Cell(""), ]) update_run!(, notebook, notebook.cells) # helper functions id(i) = notebook.cells[i].cell_id get_disabled_cells(notebook) = [i for (i, c) in pairs(notebook.cells) if c.depends_on_disabled_cells] @test !any(get(c.metadata, "disabled", false) for c in notebook.cells) @test !any(c.depends_on_disabled_cells for c in notebook.cells) # disable first cell notebook.cells[1].metadata["disabled"] = true update_run!(, notebook, notebook.cells[1]) should_be_disabled = [1, 3, 5] @test get_disabled_cells(notebook) == should_be_disabled @test notebook.cells[1].metadata["disabled"] == true # metadatum will exists in memory, but not in the serialized form @test all(haskey(notebook.cells[i].metadata , "disabled") for i=1:5) # change x, this change should not propagate through y original_y_output = notebook.cells[1].output.body original_z_output = notebook.cells[3].output.body original_a_output = notebook.cells[4].output.body original_w_output = notebook.cells[5].output.body setcode!(notebook.cells[2], "x = 123123") update_run!(, notebook, notebook.cells[2]) @test notebook.cells[1].output.body == original_y_output @test notebook.cells[3].output.body == original_z_output @test notebook.cells[4].output.body != original_a_output @test notebook.cells[5].output.body == original_w_output setcode!(notebook.cells[2], "x = 2") update_run!(, notebook, notebook.cells[2]) @test notebook.cells[1].output.body == original_y_output @test notebook.cells[3].output.body == original_z_output @test notebook.cells[4].output.body == original_a_output @test notebook.cells[5].output.body == original_w_output # disable root cell notebook.cells[2].metadata["disabled"] = true update_run!(, notebook, notebook.cells) @test get_disabled_cells(notebook) == collect(1:5) original_6_output = notebook.cells[6].output.body setcode!(notebook.cells[6], "x + 6") update_run!(, notebook, notebook.cells[6]) @test notebook.cells[6].depends_on_disabled_cells @test notebook.cells[6].errored === false @test notebook.cells[6].output.body == original_6_output # reactivate first cell - still all cells should be running_disabled notebook.cells[1].metadata["disabled"] = false update_run!(, notebook, notebook.cells) @test get_disabled_cells(notebook) == collect(1:6) # the x cell is disabled, so changing it should have no effect setcode!(notebook.cells[2], "x = 123123") update_run!(, notebook, notebook.cells[2]) @test notebook.cells[1].output.body == original_y_output @test notebook.cells[3].output.body == original_z_output @test notebook.cells[4].output.body == original_a_output @test notebook.cells[5].output.body == original_w_output # reactivate root cell notebook.cells[2].metadata["disabled"] = false update_run!(, notebook, notebook.cells) @test get_disabled_cells(notebook) == [] @test notebook.cells[1].output.body != original_y_output @test notebook.cells[3].output.body != original_z_output @test notebook.cells[4].output.body != original_a_output @test notebook.cells[5].output.body != original_w_output # disable first cell again notebook.cells[1].metadata["disabled"] = true update_run!(, notebook, notebook.cells) should_be_disabled = [1, 3, 5] @test get_disabled_cells(notebook) == should_be_disabled # and reactivate it notebook.cells[1].metadata["disabled"] = false update_run!(, notebook, notebook.cells) @test get_disabled_cells(notebook) == [] cleanup(, notebook) end @testset "Disabled cells should stay in the topology (#2676)" begin  = ServerSession() .options.evaluation.workspace_use_distributed = false notebook = Notebook(Cell.([ "using Dates", "b = 2; December", "b", ])) disabled_cell = notebook.cells[end] Pluto.set_disabled(disabled_cell, true) @test is_disabled(disabled_cell) old_topo = notebook.topology @test count(Pluto.is_disabled, notebook.cells) == 1 order = update_run!(, notebook, notebook.cells) # Disabled @test length(order.input_topology.disabled_cells) == 1 @test disabled_cell  order.input_topology.disabled_cells runned_cells = collect(order) @test length(runned_cells) == 2 @test disabled_cell  runned_cells topo = notebook.topology @test old_topo !== topo # topology was updated order = Pluto.topological_order(notebook) @test length(order.input_topology.disabled_cells) == 1 @test disabled_cell  order.input_topology.disabled_cells saved_cells = collect(order) @test length(saved_cells) == length(notebook.cells) @test issetequal(saved_cells, notebook.cells) io = IOBuffer() Pluto.save_notebook(io, notebook) seekstart(io) notebook2 = Pluto.load_notebook_nobackup(io, "mynotebook.jl") @test length(notebook2.cells) == length(notebook.cells) cleanup(, notebook) end @testset "Disabled cell definitions should be removed (#3089)" begin  = ServerSession() .options.evaluation.workspace_use_distributed = false notebook = Notebook(Cell.([ "x = 1", "y = 9", "x + y", "f() = 4", "f()", ])) update_run!(, notebook, notebook.cells) # start from the end so deletions do not mess-up indexing set_disabled(notebook.cells[4], true) # disable f definition update_run!(, notebook, notebook.cells[4]) delete_cell!(, notebook, notebook.cells[4]) # remove f definition # after the update, cell 4 is removed from the notebook # "f()" (cell 4 now) should error since f is gone @test notebook.cells[4].errored set_disabled(notebook.cells[1], true) update_run!(, notebook, notebook.cells[1]) delete_cell!(, notebook, notebook.cells[1]) # after the update, cell 1 is removed from the notebook # "x + y" (cell 2 now) should error since x is gone @test notebook.cells[2].errored cleanup(, notebook) end # Collect Time To First X (TTFX) # # A few notes about these compile times benchmarks. # 1. These benchmarks are meant to show where the biggest problems are and to be able to trace back where some regression was introduced. # 2. The benchmarks use `@eval` to avoid missing compile time, see the `@time` docstring for more info. # 3. Only add benchmarks for methods which take more than 1 seconds on the first run to reduce noise. # 4. Note that some benchmarks depend on disk and network speeds too, so focus on the number of allocations since those are more robust. module Foo end using UUIDs # setup required for run_exporession: const test_notebook_id = uuid1() let channel = Channel{Any}(10) Pluto.PlutoRunner.setup_plutologger( test_notebook_id, channel, ) end @timeit TOUT "PlutoRunner.run_expression" @eval Pluto.PlutoRunner.run_expression(Foo, Expr(:toplevel, :(1 + 1)), test_notebook_id, uuid1(), nothing); function wait_for_ready(notebook::Pluto.Notebook) while notebook.process_status != Pluto.ProcessStatus.ready sleep(0.1) end end  = Pluto.ServerSession() .options.server.disable_writing_notebook_files = true .options.evaluation.workspace_use_distributed = false path = joinpath(pkgdir(Pluto), "sample", "Basic.jl") @timeit TOUT "SessionActions.open" nb = @eval Pluto.SessionActions.open(, path; run_async=false) wait_for_ready(nb) Pluto.SessionActions.shutdown(, nb; async=false) # Compile HTTP get. Use no encoding since there seem to be an issue with Accept-Encoding: gzip HTTP.get("http://github.com") @timeit TOUT "Pluto.run" server_task = @eval let port = 13435 options = Pluto.Configuration.from_flat_kwargs(; port, launch_browser=false, workspace_use_distributed=false, require_secret_for_access=false, require_secret_for_open_links=false)  = Pluto.ServerSession(; options) server_task = @async Pluto.run() # Give the async task time to start. sleep(1) HTTP.get("http://localhost:$port/edit").status == 200 server_task end # Collect timing and allocations information; this is printed later. using TimerOutputs: TimerOutput, @timeit TOUT = TimerOutput() macro timeit_include(path::AbstractString) :(@timeit TOUT $path include($path)) end function print_timeroutput() # Sleep to avoid old logs getting tangled up in the output. sleep(6) println() show(TOUT; compact=true, sortby=:firstexec) println() end @timeit TOUT "import Pluto" import Pluto using ExpressionExplorer using Sockets using Test using HTTP import Pkg import Malt import Malt.Distributed function insert_cell!(notebook, cell) notebook.cells_dict[cell.cell_id] = cell push!(notebook.cell_order, cell.cell_id) end function delete_cell!(, notebook, cell) # this matches exactly what the frontend does when you delete a cell, check out `confirm_delete_multiple` in `Editor.js deleteat!(notebook.cell_order, findfirst(==(cell.cell_id), notebook.cell_order)) delete!(notebook.cells_dict, cell.cell_id) Pluto.update_run!(, notebook, Pluto.Cell[]) end function setcode!(cell, newcode) cell.code = newcode end function noerror(cell; verbose=true) if cell.errored && verbose @show cell.output.body cell.logs end !cell.errored end function occursinerror(needle, haystack::Pluto.Cell) haystack.errored && occursin(needle, haystack.output.body[:msg]) end function expecterror(err, cell; strict=true) cell.errored || return false msg = sprint(showerror, err) # UndefVarError(:x, #undef) if err isa UndefVarError && !isdefined(err, :scope) && VERSION >= v"1.11" strict = false msg = first(split(msg, '\n'; limit=2)) end if strict return cell.output.body[:msg] == msg else return occursin(msg, cell.output.body[:msg]) end end "Test notebook equality, ignoring cell UUIDs and such." macro test_notebook_inputs_equal(nbA, nbB, check_paths_equality::Bool=true) quote nbA = $(esc(nbA)) nbB = $(esc(nbB)) if $(check_paths_equality) @test normpath(nbA.path) == normpath(nbB.path) end @test length(nbA.cells) == length(nbB.cells) @test getproperty.(nbA.cells, :cell_id) == getproperty.(nbB.cells, :cell_id) @test getproperty.(nbA.cells, :code_folded) == getproperty.(nbB.cells, :code_folded) @test getproperty.(nbA.cells, :code) == getproperty.(nbB.cells, :code) @test get_metadata_no_default.(nbA.cells) == get_metadata_no_default.(nbB.cells) end |> Base.remove_linenums! end "Whether the given .jl file can be run without any errors. While notebooks cells can be in arbitrary order, their order in the save file must be topological. If `only_undefvar` is `true`, all errors other than an `UndefVarError` will be ignored." function jl_is_runnable(path; only_undefvar=false)  = Symbol("lab", time_ns())  = Core.eval(Main, :(module $() end)) try Core.eval(, :(include($path))) true catch ex if (!only_undefvar) || ex isa UndefVarError || (ex isa LoadError && ex.error isa UndefVarError) println(stderr, "\n$(path) failed to run. File contents:") println(stderr, "\n\n\n") println.(enumerate(readlines(path; keep=true))) println(stderr, "\n\n\n") showerror(stderr, ex, stacktrace(catch_backtrace())) println(stderr) false else true end end end "Whether the `notebook` runs without errors." function nb_is_runnable(session::Pluto.ServerSession, notebook::Pluto.Notebook) Pluto.update_run!(session, notebook, notebook.cells) errored = filter(c -> c.errored, notebook.cells) if !isempty(errored) @show errored end isempty(errored) end "The converse of Julia's `Base.sprint`." function sread(f::Function, input::String, args...) io = IOBuffer(input) output = f(io, args...) close(io) return output end function num_backups_in(dir::AbstractString) count(readdir(dir)) do fn occursin("backup", fn) end end has_embedded_pkgfiles(contents::AbstractString) = occursin("PROJECT", contents) && occursin("MANIFEST", contents) has_embedded_pkgfiles(nb::Pluto.Notebook) = read(nb.path, String) |> has_embedded_pkgfiles """ Log an error message if there are any running processes created by Distrubted, that were not shut down. """ function verify_no_running_processes() if length(Distributed.procs()) != 1 || !isempty(Malt.__iNtErNaL_get_running_procs()) @error "Not all notebook processes were closed during tests!" Distributed.procs() Malt.__iNtErNaL_get_running_procs() end end # We have our own registry for these test! Take a look at https://github.com/JuliaPluto/PlutoPkgTestRegistry#readme for more info about the test packages and their dependencies. pluto_test_registry_spec = Pkg.RegistrySpec(; url="https://github.com/JuliaPluto/PlutoPkgTestRegistry", uuid=Base.UUID("96d04d5f-8721-475f-89c4-5ee455d3eda0"), name="PlutoPkgTestRegistry", ) snapshots_dir = joinpath(@__DIR__, "snapshots") isdir(snapshots_dir) && rm(snapshots_dir; force=true, recursive=true) mkdir(snapshots_dir) function cleanup(session, notebook) testset_stack = get(task_local_storage(), :__BASETESTNEXT__, Test.AbstractTestSet[]) name = replace(join((t.description for t in testset_stack), "  "), r"[\:\?\r\n<>\|\*]" => "-") path = Pluto.numbered_until_new(joinpath(snapshots_dir, name); suffix=".html", create_file=true) write(path, Pluto.generate_html(notebook)) WorkspaceManager.unmake_workspace((session, notebook)) end using Test import Pluto: Notebook, Cell, updated_topology, static_resolve_topology, is_just_text, NotebookTopology @testset "is_just_text" begin notebook = Notebook([ Cell(""), Cell("md\"a\""), Cell("html\"a\""), Cell("md\"a \$b\$\""), Cell("md\"a ``b``\""), Cell(""" let x = md"a" md"r \$x" end """), Cell("html\"a 7 \$b\""), Cell("md\"a 8 \$b\""), Cell("@a md\"asdf 9\""), Cell("x()"), Cell("x() = y()"), Cell("12 + 12"), Cell("import Dates"), Cell("import Dates"), Cell("while false end"), Cell("for i in [16]; end"), Cell("[i for i in [17]]"), Cell("module x18 end"), Cell(""" module x19 exit() end """), Cell("""quote end"""), Cell("""quote x = 21 end"""), Cell("""quote \$(x = 22) end"""), Cell("""asdf"23" """), Cell("""@asdf("24") """), Cell("""@x"""), Cell("""@y z 26"""), Cell("""f(g"27")"""), ]) old = notebook.topology new = notebook.topology = updated_topology(old, notebook, notebook.cells) @test is_just_text(new, notebook.cells[1]) @test is_just_text(new, notebook.cells[2]) @test is_just_text(new, notebook.cells[3]) @test is_just_text(new, notebook.cells[4]) @test is_just_text(new, notebook.cells[5]) @test is_just_text(new, notebook.cells[6]) @test is_just_text(new, notebook.cells[7]) @test !is_just_text(new, notebook.cells[8]) @test !is_just_text(new, notebook.cells[9]) @test !is_just_text(new, notebook.cells[10]) @test !is_just_text(new, notebook.cells[11]) @test !is_just_text(new, notebook.cells[12]) @test !is_just_text(new, notebook.cells[13]) @test !is_just_text(new, notebook.cells[14]) @test !is_just_text(new, notebook.cells[15]) @test !is_just_text(new, notebook.cells[16]) @test !is_just_text(new, notebook.cells[17]) @test !is_just_text(new, notebook.cells[18]) @test !is_just_text(new, notebook.cells[19]) @test !is_just_text(new, notebook.cells[20]) @test !is_just_text(new, notebook.cells[21]) @test !is_just_text(new, notebook.cells[22]) @test !is_just_text(new, notebook.cells[23]) @test !is_just_text(new, notebook.cells[24]) @test !is_just_text(new, notebook.cells[25]) @test !is_just_text(new, notebook.cells[26]) @test !is_just_text(new, notebook.cells[27]) end @testset "Misc API" begin @testset "is_single_expression" begin @test Pluto.is_single_expression("") @test Pluto.is_single_expression("a") @test Pluto.is_single_expression("a + 1") @test Pluto.is_single_expression("a; a + 1") @test !Pluto.is_single_expression(""" a = 1 a + 1 """) @test Pluto.is_single_expression(""" "yooo" function f(x) X C \\ c end """) @test Pluto.is_single_expression(""" # asdf "yooo" function f(x) X C \\ c end; # aasasdf """) @test Pluto.is_single_expression(""" a a a a a // / // / 123 1 21 1313 """) end end ### A Pluto.jl notebook ### # v0.17.1 using Markdown using InteractiveUtils # This Pluto notebook uses @bind for interactivity. When running this notebook outside of Pluto, the following 'mock version' of @bind gives bound variables a default value (instead of an error). macro bind(def, element) quote local iv = try Base.loaded_modules[Base.PkgId(Base.UUID("6e696c72-6542-2067-7265-42206c756150"), "AbstractPlutoDingetjes")].Bonds.initial_value catch; b -> missing; end local el = $(esc(element)) global $(esc(def)) = Core.applicable(Base.get, el) ? Base.get(el) : iv(el) el end end #  e36eee87-a354-4eab-a156-734dee28b71f A = 123 #  de01d226-7fbb-4d6b-a044-7252415564ea B = C = 33 #  5696c569-7377-4faf-9f09-63fc01c55a00 D = A #  ea8654e0-4a25-4c2c-97a9-330a2b89c419 E = D + A #  b085e622-f4ec-49e1-9de1-5553f8ddce1c F = B - A #  9dac8ef1-fd6c-4eae-b468-b92411a66313 G = B + E #  b182e0b3-be07-4d29-81cc-af5e6ba1d6ea F + G #  8aac8df3-1551-4c9f-a8bd-a62751a29b2a md""" ## Path 1 """ #  03307e43-cb61-4321-95ac-7bbb16e0cfc6 @bind x html"<input type=range max=10000>" #  692746e0-7c96-47ac-b1ee-5ff34ee66751 @bind y html"<input type=range max=10000>" #  b18c2329-18d7-4041-962c-0ef98f8aa591 (x,y) #  399ef117-2085-4c9d-9d8d-d03a03baf5ef md""" ## Path 2 """ #  a3a04d5f-b0f0-4740-9b37-92570864f142 high_res = true #  a822ac82-5691-4f8e-a60a-1a4582cf59e7 dog_file = if high_res download("https://upload.wikimedia.org/wikipedia/commons/e/ef/Pluto_in_True_Color_-_High-Res.jpg") else download("https://upload.wikimedia.org/wikipedia/commons/thumb/e/ef/Pluto_in_True_Color_-_High-Res.jpg/240px-Pluto_in_True_Color_-_High-Res.jpg") end #  627461ce-9e80-4707-b0ba-ddc6bb9b4269 begin struct Dog end function Base.show(io::IO, ::MIME"image/jpg", ::Dog) write(io, read(dog_file)) end end #  5ce8ebc6-b509-42f0-acd5-8008673b04ab md"Downloaded image is $(filesize(dog_file) / 1000) kB" #  1f48fe19-3ee8-44ac-a591-7b4df2d2f93a md""" This cell will have very large Uint8Arrays in the output body """ #  5539db10-b0d2-48b6-8985-ef437b8ae0b5 @bind show_dogs html"<input type=checkbox>" #  74329553-ab9b-4b6c-a77b-9c24ac48490b show_dogs === true && Dog() #  8811b8bd-7d44-4f4a-b52d-0948dad39a51 md""" ## Path 3 """ #  f0575125-d7dd-4cf5-bfd3-3275d6bdd0ce a = 100 #  802be3da-6f24-4693-b48e-0479aec4a02c begin a @bind b html"<input>" end #  997f714f-1df4-4b2c-ab0a-50dd52cb4a82 md""" ## Path 4 """ #  a504bc18-fff4-4cd7-b74c-173abcf1ef68 begin a @bind c html"<input>" end #  00a0169e-fd93-4dfe-b45b-f088312e24ab md""" ## Path 5 """ #  e7b5f5ec-d673-4397-ac0c-7a4bfc364fde f(x) = x #  0a61c092-e61a-4219-a57f-172a8c8c4117 begin f(1) @bind five1 html"<input>" end #  31695eb5-7d77-4d6b-a32a-ff31cf1fd8e9 md""" ## Path 6 """ #  25774e63-94fb-4b03-a43a-72a62f08f0c5 begin f(1) @bind five2 html"<input>" end #  a04672a7-3d3f-4b0b-8c8f-f8b73f208de4 md""" ## Path 7 """ #  2d0f4354-20e0-47a0-8a86-99cd50bca80f @bind six1 html"<input>" #  fee1f75a-5f3e-4213-8ba3-872871cf7f68 @bind six2 html"<input>" #  20ebc38b-24c1-4e39-97b0-dcd53a9a5bf7 @bind six3 html"<input>" #  5545ac33-82e9-4994-be49-5776d512e2c1 (six1, six2) #  b776eba5-60bc-4d0f-8e93-e2baf2f695bc md""" ## Path 8 """ #  a56b8b24-bf38-49bf-b19e-d8feadd55db3 (six2, six3) #  6784f0e5-5108-48ca-99ac-9d154b1d3c55 md""" ## Path 9 """ #  aacc0632-797f-49d6-b9ce-3e728fade6c3 begin @bind cool1 let @bind cool2 html"nothing" html"<input type=range>" end end #  6357cfaf-e6d6-49f2-b3bb-5a27b28cf0fa cool1 #  4fd43c26-dd05-4520-93e9-901760ef49b4 cool2 #  357762fc-52fe-4727-b0a8-af55eea466b7 md""" ## Path 10 """ #  0024288a-9ee3-42b5-82c6-d996f45be9ed let md""" Hello $(@bind world html"<input value=world>") """ end #  ef732ddf-b034-42cf-815f-1c8b53da6401 world #  c143b2de-78c5-46ad-852f-3c2a9115cb72 md""" ## Path 11 """ #  cf628a57-933b-4984-a317-63360c345534 @bind boring html"<input>" #  22659c85-700f-4dad-a22a-7aafa71225c0 # boring is never referenced #  Cell order: # e36eee87-a354-4eab-a156-734dee28b71f # de01d226-7fbb-4d6b-a044-7252415564ea # 5696c569-7377-4faf-9f09-63fc01c55a00 # ea8654e0-4a25-4c2c-97a9-330a2b89c419 # b085e622-f4ec-49e1-9de1-5553f8ddce1c # 9dac8ef1-fd6c-4eae-b468-b92411a66313 # b182e0b3-be07-4d29-81cc-af5e6ba1d6ea # 8aac8df3-1551-4c9f-a8bd-a62751a29b2a # 03307e43-cb61-4321-95ac-7bbb16e0cfc6 # 692746e0-7c96-47ac-b1ee-5ff34ee66751 # b18c2329-18d7-4041-962c-0ef98f8aa591 # 399ef117-2085-4c9d-9d8d-d03a03baf5ef # 627461ce-9e80-4707-b0ba-ddc6bb9b4269 # a3a04d5f-b0f0-4740-9b37-92570864f142 # a822ac82-5691-4f8e-a60a-1a4582cf59e7 # 5ce8ebc6-b509-42f0-acd5-8008673b04ab # 1f48fe19-3ee8-44ac-a591-7b4df2d2f93a # 5539db10-b0d2-48b6-8985-ef437b8ae0b5 # 74329553-ab9b-4b6c-a77b-9c24ac48490b # 8811b8bd-7d44-4f4a-b52d-0948dad39a51 # f0575125-d7dd-4cf5-bfd3-3275d6bdd0ce # 802be3da-6f24-4693-b48e-0479aec4a02c # 997f714f-1df4-4b2c-ab0a-50dd52cb4a82 # a504bc18-fff4-4cd7-b74c-173abcf1ef68 # 00a0169e-fd93-4dfe-b45b-f088312e24ab # e7b5f5ec-d673-4397-ac0c-7a4bfc364fde # 0a61c092-e61a-4219-a57f-172a8c8c4117 # 31695eb5-7d77-4d6b-a32a-ff31cf1fd8e9 # 25774e63-94fb-4b03-a43a-72a62f08f0c5 # a04672a7-3d3f-4b0b-8c8f-f8b73f208de4 # 2d0f4354-20e0-47a0-8a86-99cd50bca80f # fee1f75a-5f3e-4213-8ba3-872871cf7f68 # 20ebc38b-24c1-4e39-97b0-dcd53a9a5bf7 # 5545ac33-82e9-4994-be49-5776d512e2c1 # b776eba5-60bc-4d0f-8e93-e2baf2f695bc # a56b8b24-bf38-49bf-b19e-d8feadd55db3 # 6784f0e5-5108-48ca-99ac-9d154b1d3c55 # aacc0632-797f-49d6-b9ce-3e728fade6c3 # 6357cfaf-e6d6-49f2-b3bb-5a27b28cf0fa # 4fd43c26-dd05-4520-93e9-901760ef49b4 # 357762fc-52fe-4727-b0a8-af55eea466b7 # 0024288a-9ee3-42b5-82c6-d996f45be9ed # ef732ddf-b034-42cf-815f-1c8b53da6401 # c143b2de-78c5-46ad-852f-3c2a9115cb72 # cf628a57-933b-4984-a317-63360c345534 # 22659c85-700f-4dad-a22a-7aafa71225c0 import Pkg Pkg.activate(mktempdir()) Pkg.develop(Pkg.PackageSpec(path=ARGS[1])) import Pluto s = Pluto.ServerSession() urls = [ "https://raw.githubusercontent.com/fonsp/Pluto.jl/v0.12.16/sample/Basic.jl", "https://raw.githubusercontent.com/fonsp/Pluto.jl/v0.12.16/sample/Basic.jl", "https://gist.githubusercontent.com/fonsp/4e164a262a60fc4bdd638e124e629d64/raw/8ffe93c680e539056068456a62dea7bf6b8eb622/basic_pkg_notebook.jl", ] success = all(urls) do url @show url nb = Pluto.SessionActions.open_url(s, url; run_async=false) @show [c.output.body for c in nb.cells] nb_success = !any(c.errored for c in nb.cells) @info "Done" url nb_success nb_success end exit(success ? 0 : 1) # Tip: dont run all tests # Comment all lines in this file, except: # - helpers.jl # - the testfile.jl that you want to run # (Paul has a better solution here based on terminal arguments) # Tip: you can use Revise: # Run this in the REPL #= using Revise, TestEnv, Pluto; (startswith(Base.active_project(),tempdir()) || TestEnv.activate("Pluto"; allow_reresolve=false)); include(joinpath(pkgdir(Pluto), "test", "runtests.jl")); =# include("helpers.jl") # tests that start new processes: @timeit_include("compiletimes.jl") verify_no_running_processes() if get(ENV, "PLUTO_TEST_ONLY_COMPILETIMES", nothing) == "true" print_timeroutput() exit(0) end @timeit_include("Events.jl") verify_no_running_processes() @timeit_include("Configuration.jl") verify_no_running_processes() @timeit_include("React.jl") verify_no_running_processes() @timeit_include("Bonds.jl") verify_no_running_processes() @timeit_include("RichOutput.jl") verify_no_running_processes() @timeit_include("packages/Basic.jl") verify_no_running_processes() @timeit_include("Dynamic.jl") verify_no_running_processes() @timeit_include("MacroAnalysis.jl") verify_no_running_processes() @timeit_include("Logging.jl") verify_no_running_processes() @timeit_include("webserver.jl") verify_no_running_processes() @timeit_include("Firebasey.jl") verify_no_running_processes() @timeit_include("Notebook.jl") verify_no_running_processes() @timeit_include("WorkspaceManager.jl") verify_no_running_processes() # tests that don't start new processes: @timeit_include("ReloadFromFile.jl") @timeit_include("packages/PkgCompat.jl") @timeit_include("packages/PkgUtils.jl") @timeit_include("MethodSignatures.jl") @timeit_include("MoreAnalysis.jl") @timeit_include("is_just_text.jl") @timeit_include("webserver_utils.jl") @timeit_include("DependencyCache.jl") @timeit_include("Throttled.jl") @timeit_include("cell_disabling.jl") @timeit_include("misc API.jl") verify_no_running_processes() print_timeroutput() @timeit_include("ExpressionExplorer.jl") using HTTP using Test using Pluto using Pluto: ServerSession, ClientSession, SessionActions, WorkspaceManager using Pluto.Configuration using Pluto.Configuration: notebook_path_suggestion, from_flat_kwargs, _convert_to_flags using Pluto.WorkspaceManager: WorkspaceManager, poll @testset "Web server" begin @testset "base_url" begin port = 13433 host = "localhost" n_components = rand(2:6) base_url = "/" for _ in 1:n_components base_url *= String(rand(collect('a':'z')  collect('0':'9'), rand(5:10))) * "/" end local_url(suffix) = "http://$host:$port$base_url$suffix" @show local_url("favicon.ico") server_running() = HTTP.get(local_url("favicon.ico")).status == 200 && HTTP.get(local_url("edit")).status == 200 # without notebook at startup options = Pluto.Configuration.from_flat_kwargs(; port, launch_browser=false, workspace_use_distributed=true, require_secret_for_access=false, require_secret_for_open_links=false, base_url, )  = Pluto.ServerSession(; options) server = Pluto.run!() @test server_running() sleep(3) @test poll(20) do # should not exist because of the base url setting HTTP.get("http://$host:$port/edit"; status_exception=false).status == 404 end for notebook in values(.notebooks) SessionActions.shutdown(, notebook; keep_in_session=false) end close(server) end @testset "UTF-8 to Codemirror UTF-16 byte mapping" begin # range ends are non inclusives tests = [ (" aaaa", (2, 4), (1, 3)), # cm is zero based (" ", (2, 6), (1, 3)), # a  is two UTF16 codeunits (" ", (6, 10), (3, 5)), # a  is two UTF16 codeunits ] for (s, (start_byte, end_byte), (from, to)) in tests @test PlutoRunner.map_byte_range_to_utf16_codepoints(s, start_byte, end_byte) == (from, to) end end @testset "Exports" begin port, socket = @inferred Pluto.port_serversocket(Sockets.ip"0.0.0.0", nothing, 5543) close(socket) @test 5543 <= port < 5600 port = 13432 host = "localhost" local_url(suffix) = "http://$host:$port/$suffix" server_running() = HTTP.get(local_url("favicon.ico")).status == 200 && HTTP.get(local_url("edit")).status == 200 # without notebook at startup options = Pluto.Configuration.from_flat_kwargs(; port, launch_browser=false, workspace_use_distributed=true, require_secret_for_access=false, require_secret_for_open_links=false )  = Pluto.ServerSession(; options) server = Pluto.run!() @test server_running() @test isempty(.notebooks) HTTP.get(local_url("sample/JavaScript.jl"); retry=false) # wait for the notebook to be added to the session @test poll(10) do length(.notebooks) == 1 end notebook = only(values(.notebooks)) # right now, the notebook was only added to the session and assigned an ID. Let's wait for it to get a process: @test poll(60) do haskey(WorkspaceManager.active_workspaces, notebook.notebook_id) end sleep(2) # Note that the notebook is running async right now! It's not finished yet. But we can already run these tests: fileA = try download(local_url("notebookfile?id=$(notebook.notebook_id)")) catch # try again :) sleep(1) download(local_url("notebookfile?id=$(notebook.notebook_id)")) end fileB = tempname() write(fileB, sprint(Pluto.save_notebook, notebook)) @test Pluto.only_versions_or_lineorder_differ(fileA, fileB) export_contents = read(download(local_url("notebookexport?id=$(notebook.notebook_id)")), String) @test occursin(string(Pluto.PLUTO_VERSION), export_contents) @test occursin("</html>", export_contents) for notebook in values(.notebooks) SessionActions.shutdown(, notebook; keep_in_session=false) end close(server) end end # testset module FirebaseyTestPlace end @testset "Firebasey" begin Core.eval(FirebaseyTestPlace, quote using Test include("../src/webserver/Firebasey.jl") end) end module DataUrlTestPlace end @testset "DataUrl" begin Core.eval(DataUrlTestPlace, quote using Test include("../src/webserver/data_url.jl") end) end # Pluto end-to-end tests All commands here are executed in this folder (`Pluto.jl/test/frontend`). ## Install packages `npm install` ## Run Pluto.jl server ``` PLUTO_PORT=2345; julia --project=/path/to/PlutoDev -e "import Pluto; Pluto.run(port=$PLUTO_PORT, require_secret_for_access=false, launch_browser=false)" ``` or if Pluto is dev'ed in your global environment: ``` PLUTO_PORT=2345; julia -e "import Pluto; Pluto.run(port=$PLUTO_PORT, require_secret_for_access=false, launch_browser=false)" ``` ## Run tests `PLUTO_PORT=2345 npm run test` ## View the browser in action Add `HEADLESS=false` when running the test command. `clear && HEADLESS=false PLUTO_PORT=1234 npm run test` ## Run a particular suite of tests Add `-- -t=name of the suite` to the end of the test command. `clear && HEADLESS=false PLUTO_PORT=1234 npm run test -- -t=PlutoAutocomplete` ## To make a test fail on a case that does not crash Pluto Use `console.error("PlutoError ...")`. This suite will fail if a console command has PlutoError in the text. Do that when a bad situation is handled but the underlying cause exists. module.exports = { presets: [ [ '@babel/preset-env', { targets: { node: 'current', }, }, ], ], } module.exports = { testTimeout: 300000, slowTestThreshold: 30, } { "name": "e2e", "version": "1.0.0", "description": "", "devDependencies": { "@babel/core": "^7.13.8", "@babel/preset-env": "^7.13.9", "@types/jest": "^26.0.20", "babel-jest": "^29.7.0", "jest": "^29.7.0", "lodash": "^4.17.21", "puppeteer": "^14.1.0" }, "scripts": { "test": "jest --verbose --runInBand" }, "author": "", "license": "MIT", "dependencies": { "mkdirp": "^1.0.4" } } { "compilerOptions": { "target": "ES2020", "module": "ES2020", "lib": ["ES2020", "DOM"], "types": ["node", "jest"], "allowJs": true, "checkJs": true, "noEmit": true, "strict": false, "noImplicitThis": true, "alwaysStrict": true, "esModuleInterop": true, "moduleResolution": "Node" } // "include": ["__tests__/**/*.js", "helpers/**/*.js"] } import puppeteer from "puppeteer" import { lastElement, saveScreenshot, createPage, waitForContentToBecome } from "../helpers/common" import { getCellIds, importNotebook, waitForCellOutput, getPlutoUrl, writeSingleLineInPlutoInput, shutdownCurrentNotebook, setupPlutoBrowser, gotoPlutoMainMenu, } from "../helpers/pluto" describe("PlutoAutocomplete", () => { /** * Launch a shared browser instance for all tests. * I don't use jest-puppeteer because it takes away a lot of control and works buggy for me, * so I need to manually create the shared browser. * @type {puppeteer.Browser} */ let browser = null /** @type {puppeteer.Page} */ let page = null beforeAll(async () => { browser = await setupPlutoBrowser() }) beforeEach(async () => { page = await createPage(browser) await gotoPlutoMainMenu(page) }) afterEach(async () => { await saveScreenshot(page) await shutdownCurrentNotebook(page) await page.close() page = null }) afterAll(async () => { await browser.close() browser = null }) it("should get the correct autocomplete suggestions", async () => { await importNotebook(page, "autocomplete_notebook.jl") const importedCellIds = await getCellIds(page) await Promise.all(importedCellIds.map((cellId) => waitForCellOutput(page, cellId))) // Add a new cell let lastPlutoCellId = lastElement(importedCellIds) await page.click(`pluto-cell[id="${lastPlutoCellId}"] .add_cell.after`) await page.waitForTimeout(500) // Type the partial input lastPlutoCellId = lastElement(await getCellIds(page)) await writeSingleLineInPlutoInput(page, `pluto-cell[id="${lastPlutoCellId}"] pluto-input`, "my_su") await page.waitForTimeout(500) // Trigger autocomplete suggestions await page.keyboard.press("Tab") await page.waitForSelector(".cm-tooltip-autocomplete") // Get suggestions const suggestions = await page.evaluate(() => Array.from(document.querySelectorAll(".cm-tooltip-autocomplete li")).map((suggestion) => suggestion.textContent) ) suggestions.sort() expect(suggestions).toEqual(["my_subtract", "my_sum1", "my_sum2"]) }) // Skipping because this doesn't work with FuzzyCompletions anymore it.skip("should automatically autocomplete if there is only one possible suggestion", async () => { await importNotebook(page, "autocomplete_notebook.jl") const importedCellIds = await getCellIds(page) await Promise.all(importedCellIds.map((cellId) => waitForCellOutput(page, cellId))) // Add a new cell let lastPlutoCellId = lastElement(importedCellIds) await page.click(`pluto-cell[id="${lastPlutoCellId}"] .add_cell.after`) await page.waitForTimeout(500) // Type the partial input lastPlutoCellId = lastElement(await getCellIds(page)) await writeSingleLineInPlutoInput(page, `pluto-cell[id="${lastPlutoCellId}"] pluto-input`, "my_sub") // Trigger autocomplete await page.keyboard.press("Tab") expect(await waitForContentToBecome(page, `pluto-cell[id="${lastPlutoCellId}"] pluto-input .CodeMirror-line`, "my_subtract")).toBe("my_subtract") }) }) import puppeteer from "puppeteer" import { saveScreenshot, createPage, paste, waitForContentToBecome, waitForContent } from "../helpers/common" import { createNewNotebook, getPlutoUrl, gotoPlutoMainMenu, importNotebook, runAllChanged, setupPlutoBrowser, shutdownCurrentNotebook, waitForCellOutput, waitForPlutoToCalmDown, } from "../helpers/pluto" // https://github.com/fonsp/Pluto.jl/issues/928 describe("@bind", () => { /** * Launch a shared browser instance for all tests. * I don't use jest-puppeteer because it takes away a lot of control and works buggy for me, * so I need to manually create the shared browser. * @type {puppeteer.Browser} */ let browser = null /** @type {puppeteer.Page} */ let page = null beforeAll(async () => { browser = await setupPlutoBrowser() }) beforeEach(async () => { page = await createPage(browser) await gotoPlutoMainMenu(page) }) afterEach(async () => { await saveScreenshot(page) await shutdownCurrentNotebook(page) await page.close() page = null }) afterAll(async () => { await browser.close() browser = null }) it("should not rerun bond values when refreshing page", async () => { await createNewNotebook(page) await paste( page, ` #  1e9cb0de-7f8f-11eb-2e49-37ac9451e455 @bind x html"<input type=range>" #  1a96fda9-73fa-4bd0-b80a-4db3593fd7d8 @bind y html"<input type=range>" #  1a96fda9-73fa-4bd0-b80a-4db3593fd7d8 @bind z html"<input type=range>" ` ) await runAllChanged(page) await paste( page, ` #  15f65099-1deb-4c73-b1cd-1bae1eec12e9 let x; y; z; numberoftimes[] += 1 end #  15f65099-1deb-4c73-b1cd-1bae1eec12e9 numberoftimes = Ref(0) ` ) await runAllChanged(page) await page.waitForFunction(() => Boolean(document.querySelector("pluto-cell:nth-of-type(5) pluto-output")?.textContent)) await waitForPlutoToCalmDown(page) let output_after_running_bonds = await page.evaluate(() => { return document.querySelector("pluto-cell:nth-of-type(5) pluto-output")?.textContent }) expect(output_after_running_bonds).not.toBe("") // Let's refresh and see await page.reload({ waitUntil: ["networkidle0", "domcontentloaded"] }) await page.waitForFunction(() => Boolean(document.querySelector("pluto-cell:nth-of-type(5) pluto-output")?.textContent)) await waitForPlutoToCalmDown(page) let output_after_reload = await page.evaluate(() => { return document.querySelector("pluto-cell:nth-of-type(5) pluto-output")?.textContent }) expect(output_after_reload).toBe(output_after_running_bonds) }) it("should ignore intermediate bond values while the notebook is running", async () => { const wait = (ms) => new Promise((resolve) => setTimeout(resolve, ms)) const chill = async () => { await wait(300) await waitForPlutoToCalmDown(page) await wait(1500) await waitForPlutoToCalmDown(page) } await importNotebook(page, "test_bind_dynamics.jl") await chill() await chill() const id = `029e1d1c-bf42-4e2c-a141-1e2eecc0800d` const output_selector = `pluto-cell[id="${id}"] pluto-output` //page.click is stupid const click = async (sel) => { await page.waitForSelector(sel) await page.evaluate((sel) => document.querySelector(sel).click(), sel) } const reset = async () => { await click(`#reset_xs_button`) await wait(300) await waitForPlutoToCalmDown(page) await waitForContentToBecome(page, output_selector, "") await wait(300) await waitForPlutoToCalmDown(page) await waitForContentToBecome(page, output_selector, "") await wait(300) } const start = async () => { await click(`#add_x_button`) await chill() return await waitForContent(page, output_selector) } await reset() await start() await chill() await reset() const val = await start() expect(val).toBe("1,done") }) }) import puppeteer from "puppeteer" import { lastElement, saveScreenshot, createPage } from "../helpers/common" import { getCellIds, waitForCellOutput, importNotebook, getPlutoUrl, prewarmPluto, writeSingleLineInPlutoInput, shutdownCurrentNotebook, setupPlutoBrowser, runAllChanged, gotoPlutoMainMenu, } from "../helpers/pluto" describe("PlutoImportNotebook", () => { /** * Launch a shared browser instance for all tests. * I don't use jest-puppeteer because it takes away a lot of control and works buggy for me, * so I need to manually create the shared browser. * @type {puppeteer.Browser} */ let browser = null /** @type {puppeteer.Page} */ let page = null beforeAll(async () => { browser = await setupPlutoBrowser() }) beforeEach(async () => { page = await createPage(browser) await gotoPlutoMainMenu(page) }) afterEach(async () => { await saveScreenshot(page) await shutdownCurrentNotebook(page) await page.close() page = null }) afterAll(async () => { await browser.close() browser = null }) test.each([ ["function_sum_notebook.jl", "3"], ["simple_sum_notebook.jl", "6"], ])("should import notebook %s with last cell output %s", async (notebookName, expectedLastCellOutput) => { await importNotebook(page, notebookName) const cellIds = await getCellIds(page) const outputs = await Promise.all(cellIds.map((cellId) => waitForCellOutput(page, cellId))) expect(lastElement(outputs)).toBe(expectedLastCellOutput) }) it("should add a new cell and re-evaluate the notebook", async () => { await importNotebook(page, "function_sum_notebook.jl") // Add a new cell let lastPlutoCellId = lastElement(await getCellIds(page)) await waitForCellOutput(page, lastPlutoCellId) await page.click(`pluto-cell[id="${lastPlutoCellId}"] .add_cell.after`) await page.waitForTimeout(500) // Use the previously defined sum function in the new cell lastPlutoCellId = lastElement(await getCellIds(page)) await writeSingleLineInPlutoInput(page, `pluto-cell[id="${lastPlutoCellId}"] pluto-input`, "sum(2, 3)") await runAllChanged(page) const lastCellContent = await waitForCellOutput(page, lastPlutoCellId) expect(lastCellContent).toBe("5") }) }) import puppeteer from "puppeteer" import { saveScreenshot, waitForContentToBecome, createPage, paste } from "../helpers/common" import { createNewNotebook, waitForNoUpdateOngoing, getPlutoUrl, shutdownCurrentNotebook, setupPlutoBrowser, waitForPlutoToCalmDown, runAllChanged, gotoPlutoMainMenu, } from "../helpers/pluto" describe("JavaScript API", () => { /** * Launch a shared browser instance for all tests. * I don't use jest-puppeteer because it takes away a lot of control and works buggy for me, * so I need to manually create the shared browser. * @type {puppeteer.Browser} */ let browser = null /** @type {puppeteer.Page} */ let page = null beforeAll(async () => { browser = await setupPlutoBrowser() }) beforeEach(async () => { page = await createPage(browser) await gotoPlutoMainMenu(page) await createNewNotebook(page) }) afterEach(async () => { await saveScreenshot(page) await shutdownCurrentNotebook(page) await page.close() page = null }) afterAll(async () => { await browser.close() browser = null }) it(" If you return an HTML node, it will be displayed.", async () => { const expected = "Success" await paste( page, `#  90cfa9a0-114d-49bf-8dea-e97d58fa2442 html"""<script> const div = document.createElement("find-me") div.innerHTML = "${expected}" return div; </script>""" ` ) await runAllChanged(page) await waitForPlutoToCalmDown(page, { polling: 100 }) const initialLastCellContent = await waitForContentToBecome(page, `pluto-cell:last-child pluto-output find-me`, expected) expect(initialLastCellContent).toBe(expected) }) it(" The observablehq/stdlib library is pre-imported, you can use DOM, html, Promises, etc.", async () => { const expected = "Success" await paste( page, `#  90cfa9a0-114d-49bf-8dea-e97d58fa2442 html"""<script> return html\`<span>${expected}\</span>\`; </script>""" ` ) await runAllChanged(page) await waitForPlutoToCalmDown(page, { polling: 100 }) let initialLastCellContent = await waitForContentToBecome(page, `pluto-cell:last-child pluto-output span`, expected) expect(initialLastCellContent).toBe(expected) await paste( page, `#  90cfa9a0-114d-49bf-8dea-e97d58fa2442 html"""<script> const span = DOM.element("span"); span.innerHTML = "${expected}" return span </script>""" ` ) await runAllChanged(page) await waitForPlutoToCalmDown(page, { polling: 100 }) initialLastCellContent = await waitForContentToBecome(page, `pluto-cell:last-child pluto-output span`, expected) expect(initialLastCellContent).toBe(expected) }) it(" When a cell re-runs reactively, this will be set to the previous output", async () => { await paste( page, ` #  90cfa9a0-114d-49bf-8dea-e97d58fa2442 @bind v html"""<span id="emit-from-here">emitter</span>""" #  cdb22342-4b79-4efe-bc2e-9edc61a0fef8 begin v html"""<script id="test-id"> const output = this ?? html\`<span id="test-id-2">span node that will be reused</span>\`; output._results = output._results || []; output._results.push(this); return output </script>""" end #  cdb22342-4b79-4efe-bc2e-9edc61a0fef9 v ` ) await runAllChanged(page) await waitForPlutoToCalmDown(page, { polling: 100 }) await waitForContentToBecome(page, `pluto-cell:nth-child(2) pluto-output`, "emitter") page.waitForTimeout(2000) // Send a custom event to increment value // Due to various optimizations this will take its time const incrementT = async () => await page.evaluate(() => { const span = document.querySelector(`#emit-from-here`) span.value = (span.value || 0) + 1 span.dispatchEvent(new CustomEvent("input")) return span.value }) // Wait until you see the value. // Only then you know reactivity reacted (and did not defer!) const waitV = async (t) => await waitForContentToBecome(page, `pluto-cell:last-child pluto-output`, `${t}`) let t = await incrementT() await waitV(t) t = await incrementT() await waitV(t) t = await incrementT() await waitV(t) await waitForNoUpdateOngoing(page, { polling: 100 }) await waitForContentToBecome(page, `pluto-cell:nth-child(2) pluto-output`, "emitter") const result = await page.evaluate(() => { // The script tag won't be in the DOM. The return'ed span will const results = document.querySelector("#test-id-2")._results return results[0] == null && results[2] === results[1] && results[1] === results[3] }) expect(result).toBe(true) // else will timout }) // TODO // it("The variable invalidation is a Promise that will get resolved when the cell output is changed or removed.", async () => { // expect("this").toBe("implemented") // }) }) import puppeteer from "puppeteer" import { waitForContent, lastElement, saveScreenshot, waitForContentToBecome, createPage } from "../helpers/common" import { createNewNotebook, getCellIds, getPlutoUrl, waitForCellOutputToChange, keyboardPressInPlutoInput, writeSingleLineInPlutoInput, shutdownCurrentNotebook, setupPlutoBrowser, waitForPlutoToCalmDown, manuallyEnterCells, runAllChanged, clearPlutoInput, gotoPlutoMainMenu, } from "../helpers/pluto" describe("PlutoNewNotebook", () => { /** * Launch a shared browser instance for all tests. * I don't use jest-puppeteer because it takes away a lot of control and works buggy for me, * so I need to manually create the shared browser. * @type {puppeteer.Browser} */ let browser = null /** @type {puppeteer.Page} */ let page = null beforeAll(async () => { browser = await setupPlutoBrowser() }) beforeEach(async () => { page = await createPage(browser) await gotoPlutoMainMenu(page) await createNewNotebook(page) }) afterEach(async () => { await saveScreenshot(page) await shutdownCurrentNotebook(page) await page.close() page = null }) afterAll(async () => { await browser.close() browser = null }) it("should run a single cell", async () => { const cellInputSelector = "pluto-input .cm-content" await page.waitForSelector(cellInputSelector) await writeSingleLineInPlutoInput(page, "pluto-input", "1+1") const runSelector = ".runcell" await page.waitForSelector(runSelector, { visible: true }) await page.click(runSelector) const content = await waitForContent(page, "pluto-output") expect(content).toBe("2") await clearPlutoInput(page, "pluto-input") const cells = ["a = 1", "b = 2", "c = 3", "a + b + c"] const plutoCellIds = await manuallyEnterCells(page, cells) await runAllChanged(page) const initialLastCellContent = await waitForContentToBecome(page, `pluto-cell[id="${plutoCellIds[3]}"] pluto-output`, "6") expect(initialLastCellContent).toBe("6") // Change second cell const secondCellInputSelector = `pluto-cell[id="${plutoCellIds[1]}"] pluto-input` // Delete 2 await keyboardPressInPlutoInput(page, secondCellInputSelector, "Backspace") // Enter 10 await writeSingleLineInPlutoInput(page, secondCellInputSelector, "10") await page.click(`pluto-cell[id="${plutoCellIds[1]}"] .runcell`) const reactiveLastCellContent = await waitForCellOutputToChange(page, lastElement(plutoCellIds), "6") expect(reactiveLastCellContent).toBe("14") }) }) import { waitForContent, lastElement, dismissBeforeUnloadDialogs, saveScreenshot, getTestScreenshotPath, waitForContentToBecome, dismissVersionDialogs, setupPage, paste, countCells, } from "../helpers/common" import { createNewNotebook, getCellIds, waitForCellOutput, waitForPlutoToCalmDown, getPlutoUrl, prewarmPluto, waitForCellOutputToChange, keyboardPressInPlutoInput, writeSingleLineInPlutoInput, manuallyEnterCells, shutdownCurrentNotebook, } from "../helpers/pluto" describe("Paste Functionality", () => { beforeAll(async () => { setupPage(page) await prewarmPluto(page) }) beforeEach(async () => { await gotoPlutoMainMenu(page) await createNewNotebook(page) await page.waitForSelector("pluto-input", { visible: true }) }) afterEach(async () => { await saveScreenshot(page) await shutdownCurrentNotebook(page) }) it("should *not* create new cell when you paste code into cell", async () => { const cells = ["a = 1", "b = 2", "c = 3", "a + b + c"] const plutoCellIds = await manuallyEnterCells(page, cells) await runAllChanged(page) const initialLastCellContent = await waitForContentToBecome(page, `pluto-cell[id="${plutoCellIds[3]}"] pluto-output`, "6") expect(initialLastCellContent).toBe("6") // Change second cell const secondCellInputSelector = `pluto-cell[id="${plutoCellIds[1]}"] pluto-input` // Delete 2 await keyboardPressInPlutoInput(page, secondCellInputSelector, "Backspace") // Enter 10 await writeSingleLineInPlutoInput(page, secondCellInputSelector, "10") await page.click(`pluto-cell[id="${plutoCellIds[1]}"] .runcell`) const reactiveLastCellContent = await waitForCellOutputToChange(page, lastElement(plutoCellIds), "6") await page.click(`pluto-cell[id="${plutoCellIds[1]}"] .runcell`) // Pasting "some code" into codemirror should *not* add new cell await paste(page, "some code", `pluto-cell[id="${plutoCellIds[1]}"] pluto-input .CodeMirror .CodeMirror-line`) await page.waitForTimeout(500) expect(await countCells(page)).toBe(5) }) it("should create new cell when you paste cell into page", async () => { const cells = ["a = 1", "b = 2", "c = 3", "a + b + c"] const plutoCellIds = await manuallyEnterCells(page, cells) await runAllChanged(page) const initialLastCellContent = await waitForContentToBecome(page, `pluto-cell[id="${plutoCellIds[3]}"] pluto-output`, "6") expect(initialLastCellContent).toBe("6") // Change second cell const secondCellInputSelector = `pluto-cell[id="${plutoCellIds[1]}"] pluto-input` // Delete 2 await keyboardPressInPlutoInput(page, secondCellInputSelector, "Backspace") // Enter 10 await writeSingleLineInPlutoInput(page, secondCellInputSelector, "10") await page.click(`pluto-cell[id="${plutoCellIds[1]}"] .runcell`) const reactiveLastCellContent = await waitForCellOutputToChange(page, lastElement(plutoCellIds), "6") await page.click(`pluto-cell[id="${plutoCellIds[1]}"] .runcell`) // Pasting "some code" into codemirror should *not* add new cell await paste(page, "some code", `pluto-cell[id="${plutoCellIds[1]}"] pluto-input .CodeMirror .CodeMirror-line`) await page.waitForTimeout(500) expect(await countCells(page)).toBe(5) // Pasting a cell into page should add a cell await paste( page, `#  0cacae2a-7e8f-11eb-2747-e3d010c9e054 1+1 ` ) await page.waitForTimeout(500) expect(await countCells(page)).toBe(6) }) it("should create new cell when you paste cell into cell", async () => { const cells = ["a = 1", "b = 2", "c = 3", "a + b + c"] const plutoCellIds = await manuallyEnterCells(page, cells) await runAllChanged(page) const initialLastCellContent = await waitForContentToBecome(page, `pluto-cell[id="${plutoCellIds[3]}"] pluto-output`, "6") expect(initialLastCellContent).toBe("6") // Change second cell const secondCellInputSelector = `pluto-cell[id="${plutoCellIds[1]}"] pluto-input` // Delete 2 await keyboardPressInPlutoInput(page, secondCellInputSelector, "Backspace") // Enter 10 await writeSingleLineInPlutoInput(page, secondCellInputSelector, "10") await page.click(`pluto-cell[id="${plutoCellIds[1]}"] .runcell`) const reactiveLastCellContent = await waitForCellOutputToChange(page, lastElement(plutoCellIds), "6") await page.click(`pluto-cell[id="${plutoCellIds[1]}"] .runcell`) // Pasting "some code" into codemirror should *not* add new cell await paste(page, "some code", `pluto-cell[id="${plutoCellIds[1]}"] pluto-input .CodeMirror .CodeMirror-line`) await page.waitForTimeout(500) expect(await countCells(page)).toBe(5) // Pasting a cell into page should add a cell await paste( page, `#  0cacae2a-7e8f-11eb-2747-e3d010c9e054 1+1 ` ) await page.waitForTimeout(500) expect(await countCells(page)).toBe(6) // Paste a cell into Codemirror should add a cell await paste( page, `#  0cacae2a-7e8f-11eb-2747-e3d010c9e054 1+1 `, `pluto-cell:nth-child(6) pluto-input .CodeMirror .CodeMirror-line` ) await page.waitForTimeout(500) expect(await countCells(page)).toBe(7) await page.waitForTimeout(500) expect(reactiveLastCellContent).toBe("14") }) }) import puppeteer from "puppeteer" import { saveScreenshot, createPage } from "../helpers/common" import { importNotebook, getPlutoUrl, shutdownCurrentNotebook, setupPlutoBrowser, gotoPlutoMainMenu } from "../helpers/pluto" describe("published_to_js", () => { /** * Launch a shared browser instance for all tests. * I don't use jest-puppeteer because it takes away a lot of control and works buggy for me, * so I need to manually create the shared browser. * @type {puppeteer.Browser} */ let browser = null /** @type {puppeteer.Page} */ let page = null beforeAll(async () => { browser = await setupPlutoBrowser() }) beforeEach(async () => { page = await createPage(browser) await gotoPlutoMainMenu(page) }) afterEach(async () => { await saveScreenshot(page) await shutdownCurrentNotebook(page) await page.close() page = null }) afterAll(async () => { await browser.close() browser = null }) it("Should correctly show published_to_js in cell output, and in logs", async () => { await importNotebook(page, "published_to_js.jl", { timeout: 120 * 1000 }) let output_of_published = await page.evaluate(() => { return document.querySelector("#to_cell_output")?.textContent }) expect(output_of_published).toBe("[1,2,3] MAGIC!") // The log content is not shown, so #to_cell_log does not exist let log_of_published = await page.evaluate(() => { return document.querySelector("#to_cell_log")?.textContent }) // This test is currently broken, due to https://github.com/fonsp/Pluto.jl/issues/2092 expect(log_of_published).toBe("[4,5,6] MAGIC!") }) }) import puppeteer from "puppeteer" import { saveScreenshot, createPage, paste, clickAndWaitForNavigation } from "../helpers/common" import { importNotebook, getPlutoUrl, shutdownCurrentNotebook, setupPlutoBrowser, waitForPlutoToCalmDown, restartProcess, getCellIds, clearPlutoInput, writeSingleLineInPlutoInput, runAllChanged, openPathOrURLNotebook, getAllCellOutputs, gotoPlutoMainMenu, } from "../helpers/pluto" describe("safe_preview", () => { /** * Launch a shared browser instance for all tests. * I don't use jest-puppeteer because it takes away a lot of control and works buggy for me, * so I need to manually create the shared browser. * @type {puppeteer.Browser} */ let browser = null /** @type {puppeteer.Page} */ let page = null beforeAll(async () => { browser = await setupPlutoBrowser() }) beforeEach(async () => { page = await createPage(browser) await gotoPlutoMainMenu(page) }) afterEach(async () => { await saveScreenshot(page) await shutdownCurrentNotebook(page) await page.close() page = null }) afterAll(async () => { await browser.close() browser = null }) const expect_safe_preview = async (/** @type {puppeteer.Page} */ page) => { await waitForPlutoToCalmDown(page) expect(await page.evaluate(() => window.I_DID_SOMETHING_DANGEROUS)).toBeUndefined() await page.waitForSelector("pluto-editor.process_waiting_for_permission") expect(await page.evaluate(() => [...document.querySelector("pluto-editor").classList])).toContain("process_waiting_for_permission") expect(await page.evaluate(() => document.querySelector("a#restart-process-button"))).not.toBeNull() expect(await page.evaluate(() => document.querySelector(".safe-preview-info"))).not.toBeNull() } it("Pasting notebook contents should open in safe preview", async () => { await Promise.all([ page.waitForNavigation(), paste( page, `### A Pluto.jl notebook ### # v0.14.0 using Markdown using InteractiveUtils #  b2d786ec-7f73-11ea-1a0c-f38d7b6bbc1e md""" Hello """ #  b2d79330-7f73-11ea-0d1c-a9aad1efaae1 1 + 2 #  Cell order: # b2d786ec-7f73-11ea-1a0c-f38d7b6bbc1e # b2d79330-7f73-11ea-0d1c-a9aad1efaae1 ` ), ]) await expect_safe_preview(page) }) it("Notebook from URL source", async () => { const url = "https://raw.githubusercontent.com/fonsp/Pluto.jl/v0.14.5/sample/Basic.jl" await openPathOrURLNotebook(page, url, { permissionToRunCode: false }) await expect_safe_preview(page) let expectWarningMessage = async () => { await page.waitForSelector(`a#restart-process-button`) const [dmsg, _] = await Promise.all([ new Promise((res) => { page.once("dialog", async (dialog) => { let msg = dialog.message() await dialog.dismiss() res(msg) }) }), page.click(`a#restart-process-button`), ]) expect(dmsg).toContain(url) expect(dmsg.toLowerCase()).toContain("danger") expect(dmsg.toLowerCase()).toContain("are you sure") await page.waitForTimeout(1000) await waitForPlutoToCalmDown(page) await expect_safe_preview(page) } await expectWarningMessage() // Make some edits expect((await getAllCellOutputs(page))[0]).toContain("Basel problem") let sel = `pluto-cell[id="${(await getCellIds(page))[0]}"]` await page.click(`${sel} .foldcode`) await clearPlutoInput(page, sel) await writeSingleLineInPlutoInput(page, sel, "1 + 1") await runAllChanged(page) expect((await getAllCellOutputs(page))[0]).toBe("Code not executed in Safe preview") await expect_safe_preview(page) ////////////////////////// // Let's shut it down // @ts-ignore let path = await page.evaluate(() => window.editor_state.notebook.path.replaceAll("\\", "\\\\")) let shutdown = async () => { await shutdownCurrentNotebook(page) await gotoPlutoMainMenu(page) // Wait for it to be shut down await page.waitForSelector(`li.recent a[title="${path}"]`) } await shutdown() // Run it again await clickAndWaitForNavigation(page, `a[title="${path}"]`) await page.waitForTimeout(1000) await waitForPlutoToCalmDown(page) await expect_safe_preview(page) await expectWarningMessage() //////////////////// await shutdown() // Now let's try to run the notebook in the background. This should start it in safe mode because of the risky source await page.evaluate((path) => { let a = document.querySelector(`a[title="${path}"]`) let btn = a.previousElementSibling btn.click() }, path) await page.waitForSelector(`li.running a[title="${path}"]`) await clickAndWaitForNavigation(page, `a[title="${path}"]`) await expect_safe_preview(page) await expectWarningMessage() // Let's run it await Promise.all([ new Promise((res) => { page.once("dialog", (dialog) => { res(dialog.accept()) }) }), page.click(`a#restart-process-button`), ]) await page.waitForTimeout(1000) await waitForPlutoToCalmDown(page) // Nice expect((await getAllCellOutputs(page))[0]).toBe("2") //////////////////// await shutdown() await clickAndWaitForNavigation(page, `a[title="${path}"]`) await expect_safe_preview(page) expect((await getAllCellOutputs(page))[0]).toBe("Code not executed in Safe preview") // Since we ran the notebook once, there should be no warning message: await page.waitForSelector(`a#restart-process-button`) await page.click(`a#restart-process-button`) // If there was a dialog, we would stall right now and the test would fail. await page.waitForTimeout(1000) await waitForPlutoToCalmDown(page) expect((await getAllCellOutputs(page))[0]).toBe("2") }) it("Importing notebook should open in safe preview", async () => { await importNotebook(page, "safe_preview.jl", { permissionToRunCode: false }) await expect_safe_preview(page) await waitForPlutoToCalmDown(page) let cell_contents = await getAllCellOutputs(page) expect(cell_contents[0]).toBe("one") expect(cell_contents[1]).toBe("Scripts and styles not rendered in Safe preview\n\ni should not be red\n\n\n\ntwo\n\n\nsafe") expect(cell_contents[2]).toBe("three") expect(cell_contents[3]).toBe("Code not executed in Safe preview") expect(cell_contents[4]).toBe("Code not executed in Safe preview") expect(cell_contents[5]).toContain("yntax") expect(cell_contents[6]).toBe("") expect(await page.evaluate(() => getComputedStyle(document.querySelector(`.zo`)).color)).not.toBe("rgb(255, 0, 0)") // Modifying should not execute code const cellids = await getCellIds(page) let sel = `pluto-cell[id="${cellids[0]}"] pluto-input` let expectNewOutput = async (contents) => { await clearPlutoInput(page, sel) await writeSingleLineInPlutoInput(page, sel, contents) await runAllChanged(page) return expect((await getAllCellOutputs(page))[0]) } ;(await expectNewOutput(`md"een"`)).toBe("een") ;(await expectNewOutput(`un`)).toBe("Code not executed in Safe preview") ;(await expectNewOutput(`md"one"`)).toBe("one") ;(await expectNewOutput(`a b c function`)).toContain("yntax") ;(await expectNewOutput(`md"one"`)).toBe("one") ;(await expectNewOutput(``)).toBe("") ;(await expectNewOutput(`md"one"`)).toBe("one") await restartProcess(page) await waitForPlutoToCalmDown(page) cell_contents = await getAllCellOutputs(page) expect(cell_contents[0]).toBe("one") expect(cell_contents[1]).toBe("\ni should not be red\n\ntwo\nsafe\nDANGER") expect(cell_contents[2]).toBe("three") expect(cell_contents[3]).toBe("123") expect(cell_contents[4]).toBe("") expect(cell_contents[5]).toContain("yntax") expect(cell_contents[6]).toBe("") expect(await page.evaluate(() => document.querySelector(`pluto-log-dot`).innerText)).toBe("four\nDANGER") expect(await page.evaluate(() => getComputedStyle(document.querySelector(`.zo`)).color)).toBe("rgb(255, 0, 0)") }) }) import puppeteer from "puppeteer" import { saveScreenshot, createPage, waitForContent } from "../helpers/common" import { createNewNotebook, getCellIds, getPlutoUrl, gotoPlutoMainMenu, importNotebook, manuallyEnterCells, runAllChanged, setupPlutoBrowser, shutdownCurrentNotebook, waitForPlutoToCalmDown, } from "../helpers/pluto" describe("slideControls", () => { /** * Launch a shared browser instance for all tests. * I don't use jest-puppeteer because it takes away a lot of control and works buggy for me, * so I need to manually create the shared browser. * @type {puppeteer.Browser} */ let browser = null /** @type {puppeteer.Page} */ let page = null beforeAll(async () => { browser = await setupPlutoBrowser() }) beforeEach(async () => { page = await createPage(browser) await gotoPlutoMainMenu(page) }) afterEach(async () => { await saveScreenshot(page) await shutdownCurrentNotebook(page) await page.close() page = null }) afterAll(async () => { await browser.close() browser = null }) it("should create titles", async () => { await importNotebook(page, "slides.jl", { permissionToRunCode: false, timeout: 120 * 1000 }) const plutoCellIds = await getCellIds(page) const content = await waitForContent(page, `pluto-cell[id="${plutoCellIds[1]}"] pluto-output`) expect(content).toBe("Slide 2\n") const slide_1_title = await page.$(`pluto-cell[id="${plutoCellIds[0]}"] pluto-output h1`) const slide_2_title = await page.$(`pluto-cell[id="${plutoCellIds[1]}"] pluto-output h1`) expect(await slide_2_title.isIntersectingViewport()).toBe(true) expect(await slide_1_title.isIntersectingViewport()).toBe(true) await page.click(`.toggle_export[title="Export..."]`) await page.waitForTimeout(500) await page.waitForSelector(".toggle_presentation", { visible: true }) await page.click(".toggle_presentation") await page.click(".changeslide.next") expect(await slide_1_title.isIntersectingViewport()).toBe(true) expect(await slide_2_title.isIntersectingViewport()).toBe(false) await page.click(".changeslide.next") expect(await slide_1_title.isIntersectingViewport()).toBe(false) expect(await slide_2_title.isIntersectingViewport()).toBe(true) await page.click(".changeslide.prev") expect(await slide_1_title.isIntersectingViewport()).toBe(true) expect(await slide_2_title.isIntersectingViewport()).toBe(false) }) }) import puppeteer from "puppeteer" import { saveScreenshot, waitForContentToBecome, createPage, paste } from "../helpers/common" import { createNewNotebook, waitForNoUpdateOngoing, getPlutoUrl, shutdownCurrentNotebook, setupPlutoBrowser, waitForPlutoToCalmDown, runAllChanged, importNotebook, gotoPlutoMainMenu, } from "../helpers/pluto" describe("wind_directions", () => { /** * Launch a shared browser instance for all tests. * I don't use jest-puppeteer because it takes away a lot of control and works buggy for me, * so I need to manually create the shared browser. * @type {puppeteer.Browser} */ let browser = null /** @type {puppeteer.Page} */ let page = null beforeAll(async () => { browser = await setupPlutoBrowser() page = await createPage(browser) await gotoPlutoMainMenu(page) await importNotebook(page, "wind_directions.jl", { permissionToRunCode: true, timeout: 180 * 1000 }) await page.waitForTimeout(1000) await waitForPlutoToCalmDown(page) }) beforeEach(async () => {}) afterEach(async () => { await saveScreenshot(page) }) afterAll(async () => { await shutdownCurrentNotebook(page) await page.close() page = null await browser.close() browser = null }) const get_cell_id_that_defines = async (page, variable_name) => { return await page.evaluate((variable_name) => { return document.querySelector(`pluto-cell > #${variable_name}`).parentElement.id }, variable_name) } let button_selector = (variable_name, value) => `pluto-cell[id="${variable_name}"] button[data-value="${value}"]` let slide_selector = (variable_name, value) => `pluto-cell[id="${variable_name}"] .carousel-slide:nth-child(${value})` it(" You can move the carousel", async () => { const xoxob = await get_cell_id_that_defines(page, "xoxob") const xoxob_again = await get_cell_id_that_defines(page, "xoxob_again") expect(await page.evaluate((sel) => document.querySelector(sel).disabled, button_selector(xoxob, -1))).toBe(true) expect(await page.evaluate((sel) => document.querySelector(sel).disabled, button_selector(xoxob, 1))).toBe(false) expect(await page.evaluate((sel) => document.querySelector(sel).disabled, button_selector(xoxob_again, -1))).toBe(true) expect(await page.evaluate((sel) => document.querySelector(sel).disabled, button_selector(xoxob_again, 1))).toBe(false) await page.click(button_selector(xoxob, 1)) await waitForPlutoToCalmDown(page) expect(await page.evaluate((sel) => document.querySelector(sel).disabled, button_selector(xoxob, -1))).toBe(false) expect(await page.evaluate((sel) => document.querySelector(sel).disabled, button_selector(xoxob, 1))).toBe(false) expect(await page.evaluate((sel) => document.querySelector(sel).disabled, button_selector(xoxob_again, -1))).toBe(false) expect(await page.evaluate((sel) => document.querySelector(sel).disabled, button_selector(xoxob_again, 1))).toBe(false) await page.click(button_selector(xoxob_again, 1)) await page.click(button_selector(xoxob_again, 1)) await waitForPlutoToCalmDown(page) expect(await page.evaluate((sel) => document.querySelector(sel).disabled, button_selector(xoxob, -1))).toBe(false) expect(await page.evaluate((sel) => document.querySelector(sel).disabled, button_selector(xoxob, 1))).toBe(true) expect(await page.evaluate((sel) => document.querySelector(sel).disabled, button_selector(xoxob_again, -1))).toBe(false) expect(await page.evaluate((sel) => document.querySelector(sel).disabled, button_selector(xoxob_again, 1))).toBe(true) }) it(" Wind directions UI", async () => { const big_input = await get_cell_id_that_defines(page, "big_input") expect(await page.evaluate((sel) => document.querySelector(sel).disabled, button_selector(big_input, -1))).toBe(true) expect(await page.evaluate((sel) => document.querySelector(sel).disabled, button_selector(big_input, 1))).toBe(false) let checkbox_selector = (i) => `${slide_selector(big_input, 1)} div:nth-child(${i + 1}) > input` await page.click(checkbox_selector(0)) await waitForPlutoToCalmDown(page) let expect_chosen_directions = async (expected) => { expect( await waitForContentToBecome(page, `pluto-cell[id="${await get_cell_id_that_defines(page, "chosen_directions_copy")}"] pluto-output`, expected) ).toBe(expected) } await expect_chosen_directions('chosen_directions_copyString1"North"') expect(await page.evaluate((sel) => document.querySelector(sel).checked, checkbox_selector(0))).toBe(true) await page.click(checkbox_selector(2)) await waitForPlutoToCalmDown(page) await expect_chosen_directions('chosen_directions_copyString1"North"2"South"') expect(await page.evaluate((sel) => document.querySelector(sel).checked, checkbox_selector(0))).toBe(true) expect(await page.evaluate((sel) => document.querySelector(sel).checked, checkbox_selector(1))).toBe(false) expect(await page.evaluate((sel) => document.querySelector(sel).checked, checkbox_selector(2))).toBe(true) }) }) import puppeteer from "puppeteer" import { saveScreenshot, createPage, waitForContentToBecome, getTextContent } from "../helpers/common" import { importNotebook, getPlutoUrl, shutdownCurrentNotebook, setupPlutoBrowser, getLogs, getLogSelector, writeSingleLineInPlutoInput, runAllChanged, waitForPlutoToCalmDown, gotoPlutoMainMenu, } from "../helpers/pluto" describe("with_js_link", () => { /** * Launch a shared browser instance for all tests. * I don't use jest-puppeteer because it takes away a lot of control and works buggy for me, * so I need to manually create the shared browser. * @type {puppeteer.Browser} */ let browser = null /** @type {puppeteer.Page} */ let page = null beforeAll(async () => { browser = await setupPlutoBrowser() page = await createPage(browser) await gotoPlutoMainMenu(page) await importNotebook(page, "with_js_link.jl", { timeout: 120 * 1000 }) }) beforeEach(async () => {}) afterEach(async () => { await saveScreenshot(page) }) afterAll(async () => { await shutdownCurrentNotebook(page) await page.close() page = null await browser.close() browser = null }) const submit_ev_input = (id, value) => page.evaluate( (id, value) => { document.querySelector(`.function_evaluator#${id} input`).value = value document.querySelector(`.function_evaluator#${id} input[type="submit"]`).click() }, id, value ) const ev_output_sel = (id) => `.function_evaluator#${id} textarea` const expect_ev_output = async (id, expected) => { expect(await waitForContentToBecome(page, ev_output_sel(id), expected)).toBe(expected) } it("basic", async () => { ////// BASIC await expect_ev_output("sqrt", "30") await submit_ev_input("sqrt", "25") await expect_ev_output("sqrt", "5") }) // TODO test refresh // TODO RERUN cELL // TODO invalidation it("LOGS AND ERRORS", async () => { ////// let log_id = "33a2293c-6202-47ca-80d1-4a9e261cae7f" const logs1 = await getLogs(page, log_id) expect(logs1).toEqual([{ class: "Info", description: "you should see this log 4", kwargs: {} }]) await submit_ev_input("logs1", "90") // TODO await page.waitForFunction( (sel) => { return document.querySelector(sel).textContent.includes("90") }, { polling: 100 }, getLogSelector(log_id) ) const logs2 = await getLogs(page, log_id) expect(logs2).toEqual([ { class: "Info", description: "you should see this log 4", kwargs: {} }, { class: "Info", description: "you should see this log 90", kwargs: {} }, ]) }) it("LOGS AND ERRORS 2", async () => { const logs3 = await getLogs(page, "480aea45-da00-4e89-b43a-38e4d1827ec2") expect(logs3.length).toEqual(2) expect(logs3[0]).toEqual({ class: "Warn", description: "You should see the following error:", kwargs: {} }) expect(logs3[1].class).toEqual("Error") expect(logs3[1].description).toContain("with_js_link") expect(logs3[1].kwargs.input).toEqual('"coOL"') expect(logs3[1].kwargs.exception).toContain("You should see this error COOL") }) it("LOGS AND ERRORS 3: assertpackable", async () => { const logs = await getLogs(page, "b310dd30-dddd-4b75-81d2-aaf35c9dd1d3") expect(logs.length).toEqual(2) expect(logs[0]).toEqual({ class: "Warn", description: "You should see the assertpackable fail after this log", kwargs: {} }) expect(logs[1].class).toEqual("Error") expect(logs[1].description).toContain("with_js_link") expect(logs[1].kwargs.input).toEqual('"4"') expect(logs[1].kwargs.exception).toContain("Only simple objects can be shared with JS") }) it("globals", async () => { await expect_ev_output("globals", "54") }) it("multiple in one cell", async () => { await expect_ev_output("uppercase", "") await expect_ev_output("lowercase", "") await submit_ev_input("uppercase", "wOw") await expect_ev_output("uppercase", "WOW") await expect_ev_output("lowercase", "") await submit_ev_input("lowercase", "drOEF") await expect_ev_output("uppercase", "WOW") await expect_ev_output("lowercase", "droef") }) it("repeated", async () => { await expect_ev_output(`length[cellid="40031867-ee3c-4aa9-884f-b76b5a9c4dec"]`, "7") await expect_ev_output(`length[cellid="7f6ada79-8e3b-40b7-b477-ce05ae79a668"]`, "7") await submit_ev_input(`length[cellid="40031867-ee3c-4aa9-884f-b76b5a9c4dec"]`, "yay") await expect_ev_output(`length[cellid="40031867-ee3c-4aa9-884f-b76b5a9c4dec"]`, "3") await expect_ev_output(`length[cellid="7f6ada79-8e3b-40b7-b477-ce05ae79a668"]`, "7") }) it("concurrency", async () => { await expect_ev_output("c1", "C1") await expect_ev_output("c2", "C2") await submit_ev_input("c1", "cc1") await submit_ev_input("c2", "cc2") await page.waitForTimeout(4000) // NOT // they dont run in parallel so right now only cc1 should be finished // expect(await page.evaluate((s) => document.querySelector(s).textContent, ev_output_sel("c1"))).toBe("CC1") // expect(await page.evaluate((s) => document.querySelector(s).textContent, ev_output_sel("c2"))).toBe("C2") // await expect_ev_output("c1", "CC1") // await expect_ev_output("c2", "CC2") // they should run in parallel: after 4 seconds both should be finished expect(await page.evaluate((s) => document.querySelector(s).textContent, ev_output_sel("c1"))).toBe("CC1") expect(await page.evaluate((s) => document.querySelector(s).textContent, ev_output_sel("c2"))).toBe("CC2") }) const expect_jslog = async (expected) => { expect(await waitForContentToBecome(page, "#checkme", expected)).toBe(expected) } it("js errors", async () => { await waitForPlutoToCalmDown(page) await page.waitForTimeout(100) await expect_jslog("hello!") await page.click("#jslogbtn") await page.waitForTimeout(500) await page.click("#jslogbtn") await page.waitForTimeout(100) // We clicked twice, but sometimes it only registers one click for some reason. I don't care, so let's check for either. let prefix = await Promise.race([ waitForContentToBecome(page, "#checkme", "hello!clickyay FRIETJE"), waitForContentToBecome(page, "#checkme", "hello!clickclickyay FRIETJEyay FRIETJE"), ]) const yolotriggerid = "8782cc14-eb1a-48a8-a114-2f71f77be275" await page.click(`pluto-cell[id="${yolotriggerid}"] pluto-output input[type="button"]`) await expect_jslog(`${prefix}hello!`) await page.click("#jslogbtn") await expect_jslog(`${prefix}hello!clicknee exception in Julia callback:ErrorException("bad")`) await page.click("#jslogbtn") await page.waitForTimeout(500) await page.click(`pluto-cell[id="${yolotriggerid}"] .runcell`) await expect_jslog(`${prefix}hello!clicknee exception in Julia callback:ErrorException("bad")clickhello!nee link not found`) }) }) ### A Pluto.jl notebook ### # v0.11.14 using Markdown using InteractiveUtils #  0e0ff076-f409-11ea-3774-250375ce2cc0 function my_sum1(a::Int, b::Int) a + b end #  20ac3636-f409-11ea-319f-05c98f069ef1 function my_sum2(a::Int, b::Int) a + b end #  1a89f222-f40a-11ea-1d2c-67a8f61101b2 function my_subtract(a::Int, b::Int) a - b end #  Cell order: # 0e0ff076-f409-11ea-3774-250375ce2cc0 # 20ac3636-f409-11ea-319f-05c98f069ef1 # 1a89f222-f40a-11ea-1d2c-67a8f61101b2 ### A Pluto.jl notebook ### # v0.11.14 using Markdown using InteractiveUtils #  d08a411c-f2c2-11ea-21a6-ad13627dfed8 function sum(a::Int, b::Int) a + b end #  e20da1b8-f2c2-11ea-2add-59f09ccd472b sum(1, 2) #  Cell order: # d08a411c-f2c2-11ea-21a6-ad13627dfed8 # e20da1b8-f2c2-11ea-2add-59f09ccd472b ### A Pluto.jl notebook ### # v0.19.27 using Markdown using InteractiveUtils #  2d69377e-23f8-11ee-116b-fb6a8f328528 begin using Pkg Pkg.activate(temp=true) # the latest versions of these packages: Pkg.add(url="https://github.com/JuliaPluto/AbstractPlutoDingetjes.jl", rev="main") Pkg.add("HypertextLiteral") end #  2ea26a4b-2d1e-4bcb-8b7b-cace79f7926a begin using AbstractPlutoDingetjes.Display: published_to_js using HypertextLiteral end #  043829fc-af3a-40b9-bb4f-f848ab50eb25 a = [1,2,3]; #  2f4609fd-7361-4048-985a-2cc74bb25606 @htl """ <script> const a = JSON.stringify($(published_to_js(a))) + " MAGIC!" return html`<div id='to_cell_output'>\${a}</div>` </script> """ #  28eba9fd-0416-49b8-966e-03a381c19ca7 b = [4,5,6]; #  0a4e8a19-6d43-4161-bb8c-1ebf8f8f68ba @info @htl """ <script> const a = JSON.stringify($(published_to_js(b))) + " MAGIC!" return html`<div id='to_cell_log'>\${a}</div>` </script> """ #  Cell order: # 2d69377e-23f8-11ee-116b-fb6a8f328528 # 2ea26a4b-2d1e-4bcb-8b7b-cace79f7926a # 043829fc-af3a-40b9-bb4f-f848ab50eb25 # 2f4609fd-7361-4048-985a-2cc74bb25606 # 28eba9fd-0416-49b8-966e-03a381c19ca7 # 0a4e8a19-6d43-4161-bb8c-1ebf8f8f68ba ### A Pluto.jl notebook ### # v0.19.29 using Markdown using InteractiveUtils #  e28131d9-9877-4b44-8213-9e6c041b5da5 md""" one """ #  ef63b97e-700d-11ee-2997-7bf929019c2d [[html""" <div class="zo"> i should not be red </div> <x>two</x> <div style="color: green;">safe</div> <script> window.I_DID_SOMETHING_DANGEROUS = true return html`<div style="color: red;">DANGER</div>` </script> <style> .zo { color: red; } </style> """]] #  99e2bfea-4e5d-4d94-bd96-77be7b04811d html"three" #  76e68adf-16ab-4e88-a601-3177f34db6ec 122 + 1 #  873d58c2-8590-4bb3-bf9c-596b1cdbe402 let stuff = html""" four <script> return html`<div style="color: red;">DANGER</div>` </script> """ @info stuff end #  55c74b79-41a6-461e-99c4-a61994673824 modify me to make me safe #  f5209e95-761d-4861-a00d-b7e33a1b3d69 #  Cell order: # e28131d9-9877-4b44-8213-9e6c041b5da5 # ef63b97e-700d-11ee-2997-7bf929019c2d # 99e2bfea-4e5d-4d94-bd96-77be7b04811d # 76e68adf-16ab-4e88-a601-3177f34db6ec # 873d58c2-8590-4bb3-bf9c-596b1cdbe402 # 55c74b79-41a6-461e-99c4-a61994673824 # f5209e95-761d-4861-a00d-b7e33a1b3d69 ### A Pluto.jl notebook ### # v0.11.14 using Markdown using InteractiveUtils #  cbcf36de-f360-11ea-0c7f-719e93324b27 a = 1 #  d71c5ee2-f360-11ea-2753-a132fa41871a b = 2 #  d8f5a4f6-f360-11ea-043d-47667f6a7e76 c = 3 #  dcd9ebb8-f360-11ea-2050-fd2e11d27c6d a + b + c #  Cell order: # cbcf36de-f360-11ea-0c7f-719e93324b27 # d71c5ee2-f360-11ea-2753-a132fa41871a # d8f5a4f6-f360-11ea-043d-47667f6a7e76 # dcd9ebb8-f360-11ea-2050-fd2e11d27c6d ### A Pluto.jl notebook ### # v0.19.40 using Markdown using InteractiveUtils #  cbcf36de-f360-11ea-0c7f-719e93324b27 md"# Slide 1" #  d71c5ee2-f360-11ea-2753-a132fa41871a md"# Slide 2" #  d8f5a4f6-f360-11ea-043d-47667f6a7e76 #  dcd9ebb8-f360-11ea-2050-fd2e11d27c6d #  Cell order: # cbcf36de-f360-11ea-0c7f-719e93324b27 # d71c5ee2-f360-11ea-2753-a132fa41871a # d8f5a4f6-f360-11ea-043d-47667f6a7e76 # dcd9ebb8-f360-11ea-2050-fd2e11d27c6d ### A Pluto.jl notebook ### # v0.19.45 using Markdown using InteractiveUtils # This Pluto notebook uses @bind for interactivity. When running this notebook outside of Pluto, the following 'mock version' of @bind gives bound variables a default value (instead of an error). macro bind(def, element) quote local iv = try Base.loaded_modules[Base.PkgId(Base.UUID("6e696c72-6542-2067-7265-42206c756150"), "AbstractPlutoDingetjes")].Bonds.initial_value catch; b -> missing; end local el = $(esc(element)) global $(esc(def)) = Core.applicable(Base.get, el) ? Base.get(el) : iv(el) el end end #  a0fe4e4d-eee5-4420-bd58-3f12749a9ed1 @bind reset_xs html"<input id=reset_xs_button type=button value=reset_xs>" #  58db6bd4-58a6-11ef-3795-fd6e57eceb68 @bind x html"""<div> <script> const div = currentScript.parentElement const btn = div.querySelector("button") let start_time = 0 let max_duration = 1000 const set = (x) => { div.value = x div.dispatchEvent(new CustomEvent("input")) } function go() { if(Date.now() - start_time < max_duration) { set(div.value + 1) requestAnimationFrame(go) } else { set("done") } } btn.onclick = () => { div.value = 0 start_time = Date.now() go() } </script> <button id=add_x_button>Start</button> </div>""" #  8568c646-0233-4a95-8332-2351e9c56027 @bind withsleep html"<input id=withsleep type=checkbox checked>" #  8a20fa4a-ac02-4a37-a54e-e4224628db66 x #  29f1d840-574e-463c-87d3-4b938e123493 begin reset_xs xs = [] end #  3155b6e0-8e19-4583-b2ab-4ab2db1f10b9 md""" Click **reset_xs**. Click **Start**. The cell below should give: `1,done`. Not: `1,2,done` or something like that. That means that an intermediate bond value (`2`) found its way through: [https://github.com/fonsp/Pluto.jl/issues/1891](https://github.com/fonsp/Pluto.jl/issues/1891) """ #  029e1d1c-bf42-4e2c-a141-1e2eecc0800d begin withsleep && sleep(1.5) push!(xs, x) xs_done = true join(xs[2:end], ",") |> Text end #  Cell order: # a0fe4e4d-eee5-4420-bd58-3f12749a9ed1 # 58db6bd4-58a6-11ef-3795-fd6e57eceb68 # 8568c646-0233-4a95-8332-2351e9c56027 # 8a20fa4a-ac02-4a37-a54e-e4224628db66 # 3155b6e0-8e19-4583-b2ab-4ab2db1f10b9 # 029e1d1c-bf42-4e2c-a141-1e2eecc0800d # 29f1d840-574e-463c-87d3-4b938e123493 ### A Pluto.jl notebook ### # v0.19.40 using Markdown using InteractiveUtils # This Pluto notebook uses @bind for interactivity. When running this notebook outside of Pluto, the following 'mock version' of @bind gives bound variables a default value (instead of an error). macro bind(def, element) quote local iv = try Base.loaded_modules[Base.PkgId(Base.UUID("6e696c72-6542-2067-7265-42206c756150"), "AbstractPlutoDingetjes")].Bonds.initial_value catch; b -> missing; end local el = $(esc(element)) global $(esc(def)) = Core.applicable(Base.get, el) ? Base.get(el) : iv(el) el end end #  2e54b8fc-7852-11ec-27d7-df0bfe7f344a using PlutoUI #  257ee9b0-d955-43d2-9c94-245716708a2d using HypertextLiteral #  82c316c7-a279-4728-b16a-921d7fc52886 #  0b19e53d-eb7a-42b6-a7db-d95bc8c63eae import MarkdownLiteral: @mdx #  0c0bab41-a020-41a0-83ad-0c57b4699ffa const Layout = PlutoUI.ExperimentalLayout #  c097b477-e154-47eb-b7d9-a4d2981dcf0e padded(x) = Layout.Div([x]; style=Dict("padding" => "0em 1em")) #  ddccf592-0d0f-475c-81ae-067c37ba3f7e const all_directions = ["North", "East", "South", "West"] #  d441b495-a00c-4de3-a232-7c75f55fc95b function Carousel2( elementsList; wraparound::Bool=false, peek::Bool=true, ) @assert peek carouselHTML = map(elementsList) do element Layout.Div([element]; class="carousel-slide") end h = Layout.Div([ @htl(""" <style> .carousel-box{ width: 100%; overflow: hidden; } .carousel-container{ top: 0; left: 0; display: flex; width: 100%; flex-flow: row nowrap; transform: translate(10%, 0px); transition: transform 200ms ease-in-out; } .carousel-controls{ display: flex; justify-content: center; align-items: center; } .carousel-controls button{ margin: 8px; width: 6em; } .carousel-slide { min-width: 80%; overflow-x: auto; } </style> """), Layout.Div([ Layout.Div(carouselHTML; class="carousel-container") ]; class="carousel-box"), @htl(""" <div class="carousel-controls"> <button data-value="-1">Previous</button> <button data-value="1">Next</button> </div> """), @htl(""" <script> // Here is a little trick! // We include the number of elements inside the code, which will make this script re-run whenever it changes. Pluto only re-renders HTML when it changed. const max = $(length(elementsList)) let div = currentScript.closest(".carousel-wrapper") let bound_element = div.parentElement.tagName === "PLUTO-DISPLAY" ? div.parentElement : div bound_element.value = 1 let count = 0 let buttons = div.querySelectorAll("button") const update_ui = () => { buttons[0].disabled = !$(wraparound) && count === 0 buttons[1].disabled = !$(wraparound) && count === max - 1 div.querySelector(".carousel-container").style = `transform: translate(\${10-count*80}%, 0px)`; } Object.defineProperty(bound_element, "value", { get: () => count + 1, set: (new_value) => { count = new_value - 1 update_ui() } }) const mod = (n, m) => ((n % m) + m) % m const clamp = (x, a, b) => Math.max(Math.min(x, b), a) const onclick = (e) => { const new_count = count + parseInt(e.target.dataset.value) if($(wraparound)){ count = mod(new_count, max) } else { count = clamp(new_count, 0, max - 1) } bound_element.dispatchEvent(new CustomEvent("input")) update_ui() e.preventDefault() } // This code is in a requestIdleCallback because we need the buttons to be rendered before we can select them. requestIdleCallback(() => { buttons = div.querySelectorAll("button") buttons.forEach(button => button.addEventListener("click", onclick)) update_ui() }) </script> """), ]; class="carousel-wrapper") # BondDefault(h,1) h end #  fa0b6647-6911-4c27-a1a6-240d215331d1 function Carousel( elementsList; wraparound::Bool=false, peek::Bool=true, ) @assert peek carouselHTML = map(elementsList) do element @htl("""<div class="carousel-slide"> $(element) </div>""") end h = @htl(""" <div> <style> .carousel-box{ width: 100%; overflow: hidden; } .carousel-container{ top: 0; left: 0; display: flex; width: 100%; flex-flow: row nowrap; transform: translate(10%, 0px); transition: transform 200ms ease-in-out; } .carousel-controls{ display: flex; justify-content: center; align-items: center; } .carousel-controls button{ margin: 8px; width: 6em; } .carousel-slide { min-width: 80%; } </style> <script> const div = currentScript.parentElement const buttons = div.querySelectorAll("button") const max = $(length(elementsList)) let count = 0 const mod = (n, m) => ((n % m) + m) % m const clamp = (x, a, b) => Math.max(Math.min(x, b), a) const update_ui = (count) => { buttons[0].disabled = !$(wraparound) && count === 0 buttons[1].disabled = !$(wraparound) && count === max - 1 div.querySelector(".carousel-container").style = `transform: translate(\${10-count*80}%, 0px)`; } const onclick = (e) => { const new_count = count + parseInt(e.target.dataset.value) if($(wraparound)){ count = mod(new_count, max) } else { count = clamp(new_count, 0, max - 1) } div.value = count + 1 div.dispatchEvent(new CustomEvent("input")) update_ui(div.value - 1) e.preventDefault() } buttons.forEach(button => button.addEventListener("click", onclick)) div.value = count + 1 update_ui(div.value - 1) </script> <div class="carousel-box"> <div class="carousel-container"> $(carouselHTML) </div> </div> <div class="carousel-controls"> <button data-value="-1">Previous</button> <button data-value="1">Next</button> </div> </div> """) # BondDefault(h,1) h end #  6c84a84f-9ead-4091-819e-0de088e2dd4d function wind_speeds(directions) PlutoUI.combine() do Child @htl(""" <h6>Wind speeds</h6> <ul> $([ @htl("<li>$(name): $(Child(name, Slider(1:100)))</li>") for name in directions ]) </ul> """) end end #  e866282e-7c63-4364-b344-46f4c6ad165c dogscats() = PlutoUI.combine() do Child md""" # Hi there! I have $( Child(Slider(1:10)) ) dogs and $( Child(Slider(5:100)) ) cats. Would you like to see them? $(Child(CheckBox(true))) """ end #  cd3b9ad1-8efc-4f92-96d0-b9b038d8cfae md""" ## MultiCheckBox copy This is a version of MultiCheckBox from PlutoUI that did not support synchronizing multiple bonds, i.e. it doesn't have `Object.defineProperty(wrapper, "input", {get, set})`. This means that this won't work be synced: ```julia bond = @bind value MultiCheckbox([1,2]) ``` ```julia bond ``` We need this for the test to be extra sensitive. """ #  79b6ac0f-4d0b-485f-8fb0-9849932dc34e import AbstractPlutoDingetjes.Bonds #  eaad4fed-ea22-4132-a84a-429f486ddce2 subarrays(x) = ( x[collect(I)] for I in Iterators.product(Iterators.repeated([true,false],length(x))...) |> collect |> vec ) #  146474b5-9aa6-4000-867d-ba91e4061d9b begin local result = begin """ ```julia MultiCheckBox(options::Vector; [default::Vector], [orientation  [:row, :column]], [select_all::Bool]) ``` A group of checkboxes - the user can choose which of the `options` to return. The value returned via `@bind` is a list containing the currently checked items. See also: [`MultiSelect`](@ref). `options` can also be an array of pairs `key::Any => value::String`. The `key` is returned via `@bind`; the `value` is shown. # Keyword arguments - `defaults` specifies which options should be checked initally. - `orientation` specifies whether the options should be arranged in `:row`'s `:column`'s. - `select_all` specifies whether or not to include a "Select All" checkbox. # Examples ```julia @bind snacks MultiCheckBox(["", "", ""])) if ""  snacks "Yum yum!" end ``` ```julia @bind functions MultiCheckBox([sin, cos, tan]) [f(0.5) for f in functions] ``` ```julia @bind snacks MultiCheckBox(["" => "", "" => "", "" => ""]; default=["", ""]) ``` ```julia @bind animals MultiCheckBox(["", "" , "", "", "", "" , "", ""]; orientation=:column, select_all=true) ``` """ struct MultiCheckBox{BT,DT} options::AbstractVector{Pair{BT,DT}} default::Union{Missing,AbstractVector{BT}} orientation::Symbol select_all::Bool end end MultiCheckBox(options::AbstractVector{<:Pair{BT,DT}}; default=missing, orientation=:row, select_all=false) where {BT,DT} = MultiCheckBox(options, default, orientation, select_all) MultiCheckBox(options::AbstractVector{BT}; default=missing, orientation=:row, select_all=false) where BT = MultiCheckBox{BT,BT}(Pair{BT,BT}[o => o for o in options], default, orientation, select_all) function Base.show(io::IO, m::MIME"text/html", mc::MultiCheckBox) @assert mc.orientation == :column || mc.orientation == :row "Invalid orientation $(mc.orientation). Orientation should be :row or :column" defaults = coalesce(mc.default, []) # Old: # checked = [k in defaults for (k,v) in mc.options] # # More complicated to fix https://github.com/JuliaPluto/PlutoUI.jl/issues/106 defaults_copy = copy(defaults) checked = [ let i = findfirst(isequal(k), defaults_copy) if i === nothing false else deleteat!(defaults_copy, i) true end end for (k,v) in mc.options] show(io, m, @htl(""" <plj-multi-checkbox style="flex-direction: $(mc.orientation);"></plj-multi-checkbox> <script type="text/javascript"> const labels = $([string(v) for (k,v) in mc.options]); const values = $(1:length(mc.options)); const checked = $(checked); const includeSelectAll = $(mc.select_all); const container = (currentScript ? currentScript : this.currentScript).previousElementSibling const my_id = crypto.getRandomValues(new Uint32Array(1))[0].toString(36) // Add checkboxes const inputEls = [] for (let i = 0; i < labels.length; i++) { const boxId = `\${my_id}-box-\${i}` const item = document.createElement('div') const checkbox = document.createElement('input') checkbox.type = 'checkbox' checkbox.id = boxId checkbox.name = labels[i] checkbox.value = values[i] checkbox.checked = checked[i] inputEls.push(checkbox) item.appendChild(checkbox) const label = document.createElement('label') label.htmlFor = boxId label.innerText = labels[i] item.appendChild(label) container.appendChild(item) } function setValue() { container.value = inputEls.filter((o) => o.checked).map((o) => o.value) } // Add listeners function sendEvent() { setValue() container.dispatchEvent(new CustomEvent('input')) } function updateSelectAll() {} if (includeSelectAll) { // Add select-all checkbox. const selectAllItem = document.createElement('div') selectAllItem.classList.add(`select-all`) const selectID = `\${my_id}-select-all` const selectAllInput = document.createElement('input') selectAllInput.type = 'checkbox' selectAllInput.id = selectID selectAllItem.appendChild(selectAllInput) const selectAllLabel = document.createElement('label') selectAllLabel.htmlFor = selectID selectAllLabel.innerText = 'Select All' selectAllItem.appendChild(selectAllLabel) container.prepend(selectAllItem) function onSelectAllClick(event) { event.stopPropagation() inputEls.forEach((o) => (o.checked = this.checked)) sendEvent() } selectAllInput.addEventListener('click', onSelectAllClick) selectAllInput.addEventListener('input', e => e.stopPropagation()) /// Taken from: https://stackoverflow.com/questions/10099158/how-to-deal-with-browser-differences-with-indeterminate-checkbox /// Determine the checked state to give to a checkbox /// with indeterminate state, so that it becomes checked /// on click on IE, Chrome and Firefox 5+ function getCheckedStateForIndeterminate() { // Create a unchecked checkbox with indeterminate state const test = document.createElement('input') test.type = 'checkbox' test.checked = false test.indeterminate = true // Try to click the checkbox const body = document.body body.appendChild(test) // Required to work on FF test.click() body.removeChild(test) // Required to work on FF // Check if the checkbox is now checked and cache the result if (test.checked) { getCheckedStateForIndeterminate = function () { return false } return false } else { getCheckedStateForIndeterminate = function () { return true } return true } } updateSelectAll = function () { const checked = inputEls.map((o) => o.checked) if (checked.every((x) => x)) { selectAllInput.checked = true selectAllInput.indeterminate = false } else if (checked.some((x) => x)) { selectAllInput.checked = getCheckedStateForIndeterminate() selectAllInput.indeterminate = true } else { selectAllInput.checked = false selectAllInput.indeterminate = false } } // Call once at the beginning to initialize. updateSelectAll() } function onItemClick(event) { event.stopPropagation() updateSelectAll() sendEvent() } setValue() inputEls.forEach((el) => el.addEventListener('click', onItemClick)) inputEls.forEach((el) => el.addEventListener('input', e => e.stopPropagation())) </script> <style type="text/css"> plj-multi-checkbox { display: flex; flex-wrap: wrap; /* max-height: 8em; */ } plj-multi-checkbox * { display: flex; } plj-multi-checkbox > div { margin: 0.1em 0.3em; align-items: center; } plj-multi-checkbox label, plj-multi-checkbox input { cursor: pointer; } plj-multi-checkbox .select-all { font-style: italic; color: hsl(0, 0%, 25%, 0.7); } </style> """)) end Base.get(select::MultiCheckBox) = Bonds.initial_value(select) Bonds.initial_value(select::MultiCheckBox{BT,DT}) where {BT,DT} = ismissing(select.default) ? BT[] : select.default Bonds.possible_values(select::MultiCheckBox) = subarrays(map(string, 1:length(select.options))) function Bonds.transform_value(select::MultiCheckBox{BT,DT}, val_from_js) where {BT,DT} # val_from_js will be a vector of Strings, but let's allow Integers as well, there's no harm in that @assert val_from_js isa Vector val_nums = ( v isa Integer ? v : tryparse(Int64, v) for v in val_from_js ) BT[select.options[v].first for v in val_nums] end function Bonds.validate_value(select::MultiCheckBox, val) val isa Vector && all(val_from_js) do v val_num = v isa Integer ? v : tryparse(Int64, v) 1  val_num  length(select.options) end end result end #  67e2cb97-e224-47ca-96ba-2e89d94959e7 ppp = @bind opop Slider(1:10); #  b1c0d12c-f383-44fb-bcfe-4157a2801b9a Layout.Div([ ppp, @htl("""$(opop)""") ]) #  a060f034-b540-4b1e-a87f-7e6185e15646 directions_bond = @bind chosen_directions MultiCheckBox(all_directions); #  466bf852-144c-47df-98e1-89935754f5f1 chosen_directions_copy = chosen_directions #  6e26a930-1b49-4ff5-8704-9149d3cab7e9 speeds_bond = @bind speeds wind_speeds(chosen_directions); #  ede20024-1aea-4d80-a19a-8a5ec88a00ac data = map(speeds) do s rand(50) .+ s end #  acb08d1f-30f9-4e01-8216-3440c82714c7 pairs(speeds) |> collect #  fffd6402-d508-48a6-abc6-3de333497787 big_input = Carousel2([ md""" ## Step 1: *directions* $(directions_bond) """ |> identity, md""" ## Step 2: *speeds* $(speeds_bond) """ |> identity, md""" ## Step 3:  $(embed_display( data )) """ |> identity, # md""" # ## Step 4:  # $(embed_display( # let # p = plot() # for (n,v) in pairs(data) # plot!(p, v; label=string(n)) # end # p # end # )) # """ |> padded, ]) #  0cb7b599-cec5-4391-9485-4e1c63cd9ff2 speeds_copy = speeds #  ab108b97-4dd5-49e4-845c-2f0fab131f8a Layout.vbox([ directions_bond, speeds_bond, speeds ]) #  596dbead-63ce-432a-8a0b-b3ea361e279e xoxob = @bind xoxo Carousel2([md"# a",md"# b",3,rand(4)]) #  8c96934c-3e23-45ed-b945-dce344bfb6eb xoxob_again = xoxob #  af1ad32b-af53-4865-9807-ba8d0fba2a8c xoxo #  00000000-0000-0000-0000-000000000001 PLUTO_PROJECT_TOML_CONTENTS = """ [deps] AbstractPlutoDingetjes = "6e696c72-6542-2067-7265-42206c756150" HypertextLiteral = "ac1192a8-f4b3-4bfe-ba22-af5b92cd3ab2" MarkdownLiteral = "736d6165-7244-6769-4267-6b50796e6954" PlutoUI = "7f904dfe-b85e-4ff6-b463-dae2292396a8" [compat] AbstractPlutoDingetjes = "~1.2.0" HypertextLiteral = "~0.9.4" MarkdownLiteral = "~0.1.1" PlutoUI = "~0.7.52" """ #  00000000-0000-0000-0000-000000000002 PLUTO_MANIFEST_TOML_CONTENTS = """ # This file is machine-generated - editing it directly is not advised [[AbstractPlutoDingetjes]] deps = ["Pkg"] git-tree-sha1 = "91bd53c39b9cbfb5ef4b015e8b582d344532bd0a" uuid = "6e696c72-6542-2067-7265-42206c756150" version = "1.2.0" [[ArgTools]] uuid = "0dad84c5-d112-42e6-8d28-ef12dabb789f" version = "1.1.1" [[Artifacts]] uuid = "56f22d72-fd6d-98f1-02f0-08ddc0907c33" [[Base64]] uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f" [[ColorTypes]] deps = ["FixedPointNumbers", "Random"] git-tree-sha1 = "eb7f0f8307f71fac7c606984ea5fb2817275d6e4" uuid = "3da002f7-5984-5a60-b8a6-cbb66c0b333f" version = "0.11.4" [[CommonMark]] deps = ["Crayons", "JSON", "PrecompileTools", "URIs"] git-tree-sha1 = "532c4185d3c9037c0237546d817858b23cf9e071" uuid = "a80b9123-70ca-4bc0-993e-6e3bcb318db6" version = "0.8.12" [[CompilerSupportLibraries_jll]] deps = ["Artifacts", "Libdl"] uuid = "e66e0078-7015-5450-92f7-15fbd957f2ae" version = "1.0.5+1" [[Crayons]] git-tree-sha1 = "249fe38abf76d48563e2f4556bebd215aa317e15" uuid = "a8cc5b0e-0ffa-5ad4-8c14-923d3ee1735f" version = "4.1.1" [[Dates]] deps = ["Printf"] uuid = "ade2ca70-3891-5945-98fb-dc099432e06a" [[Downloads]] deps = ["ArgTools", "FileWatching", "LibCURL", "NetworkOptions"] uuid = "f43a241f-c20a-4ad4-852c-f6b1247861c6" version = "1.6.0" [[FileWatching]] uuid = "7b1f6079-737a-58dc-b8bc-7a2ca5c1b5ee" [[FixedPointNumbers]] deps = ["Statistics"] git-tree-sha1 = "335bfdceacc84c5cdf16aadc768aa5ddfc5383cc" uuid = "53c48c17-4a7d-5ca2-90c5-79b7896eea93" version = "0.8.4" [[Hyperscript]] deps = ["Test"] git-tree-sha1 = "8d511d5b81240fc8e6802386302675bdf47737b9" uuid = "47d2ed2b-36de-50cf-bf87-49c2cf4b8b91" version = "0.0.4" [[HypertextLiteral]] deps = ["Tricks"] git-tree-sha1 = "c47c5fa4c5308f27ccaac35504858d8914e102f9" uuid = "ac1192a8-f4b3-4bfe-ba22-af5b92cd3ab2" version = "0.9.4" [[IOCapture]] deps = ["Logging", "Random"] git-tree-sha1 = "d75853a0bdbfb1ac815478bacd89cd27b550ace6" uuid = "b5f81e59-6552-4d32-b1f0-c071b021bf89" version = "0.2.3" [[InteractiveUtils]] deps = ["Markdown"] uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240" [[JSON]] deps = ["Dates", "Mmap", "Parsers", "Unicode"] git-tree-sha1 = "31e996f0a15c7b280ba9f76636b3ff9e2ae58c9a" uuid = "682c06a0-de6a-54ab-a142-c8b1cf79cde6" version = "0.21.4" [[LibCURL]] deps = ["LibCURL_jll", "MozillaCACerts_jll"] uuid = "b27032c2-a3e7-50c8-80cd-2d36dbcbfd21" version = "0.6.4" [[LibCURL_jll]] deps = ["Artifacts", "LibSSH2_jll", "Libdl", "MbedTLS_jll", "Zlib_jll", "nghttp2_jll"] uuid = "deac9b47-8bc7-5906-a0fe-35ac56dc84c0" version = "8.4.0+0" [[LibGit2]] deps = ["Base64", "LibGit2_jll", "NetworkOptions", "Printf", "SHA"] uuid = "76f85450-5226-5b5a-8eaa-529ad045b433" [[LibGit2_jll]] deps = ["Artifacts", "LibSSH2_jll", "Libdl", "MbedTLS_jll"] uuid = "e37daf67-58a4-590a-8e99-b0245dd2ffc5" version = "1.6.4+0" [[LibSSH2_jll]] deps = ["Artifacts", "Libdl", "MbedTLS_jll"] uuid = "29816b5a-b9ab-546f-933c-edad1886dfa8" version = "1.11.0+1" [[Libdl]] uuid = "8f399da3-3557-5675-b5ff-fb832c97cbdb" [[LinearAlgebra]] deps = ["Libdl", "OpenBLAS_jll", "libblastrampoline_jll"] uuid = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e" [[Logging]] uuid = "56ddb016-857b-54e1-b83d-db4d58db5568" [[MIMEs]] git-tree-sha1 = "65f28ad4b594aebe22157d6fac869786a255b7eb" uuid = "6c6e2e6c-3030-632d-7369-2d6c69616d65" version = "0.1.4" [[Markdown]] deps = ["Base64"] uuid = "d6f4376e-aef5-505a-96c1-9c027394607a" [[MarkdownLiteral]] deps = ["CommonMark", "HypertextLiteral"] git-tree-sha1 = "0d3fa2dd374934b62ee16a4721fe68c418b92899" uuid = "736d6165-7244-6769-4267-6b50796e6954" version = "0.1.1" [[MbedTLS_jll]] deps = ["Artifacts", "Libdl"] uuid = "c8ffd9c3-330d-5841-b78e-0817d7145fa1" version = "2.28.2+1" [[Mmap]] uuid = "a63ad114-7e13-5084-954f-fe012c677804" [[MozillaCACerts_jll]] uuid = "14a3606d-f60d-562e-9121-12d972cd8159" version = "2023.1.10" [[NetworkOptions]] uuid = "ca575930-c2e3-43a9-ace4-1e988b2c1908" version = "1.2.0" [[OpenBLAS_jll]] deps = ["Artifacts", "CompilerSupportLibraries_jll", "Libdl"] uuid = "4536629a-c528-5b80-bd46-f80d51c5b363" version = "0.3.23+2" [[Parsers]] deps = ["Dates", "PrecompileTools", "UUIDs"] git-tree-sha1 = "716e24b21538abc91f6205fd1d8363f39b442851" uuid = "69de0a69-1ddd-5017-9359-2bf0b02dc9f0" version = "2.7.2" [[Pkg]] deps = ["Artifacts", "Dates", "Downloads", "FileWatching", "LibGit2", "Libdl", "Logging", "Markdown", "Printf", "REPL", "Random", "SHA", "Serialization", "TOML", "Tar", "UUIDs", "p7zip_jll"] uuid = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f" version = "1.10.0" [[PlutoUI]] deps = ["AbstractPlutoDingetjes", "Base64", "ColorTypes", "Dates", "FixedPointNumbers", "Hyperscript", "HypertextLiteral", "IOCapture", "InteractiveUtils", "JSON", "Logging", "MIMEs", "Markdown", "Random", "Reexport", "URIs", "UUIDs"] git-tree-sha1 = "e47cd150dbe0443c3a3651bc5b9cbd5576ab75b7" uuid = "7f904dfe-b85e-4ff6-b463-dae2292396a8" version = "0.7.52" [[PrecompileTools]] deps = ["Preferences"] git-tree-sha1 = "03b4c25b43cb84cee5c90aa9b5ea0a78fd848d2f" uuid = "aea7be01-6a6a-4083-8856-8a6e6704d82a" version = "1.2.0" [[Preferences]] deps = ["TOML"] git-tree-sha1 = "00805cd429dcb4870060ff49ef443486c262e38e" uuid = "21216c6a-2e73-6563-6e65-726566657250" version = "1.4.1" [[Printf]] deps = ["Unicode"] uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7" [[REPL]] deps = ["InteractiveUtils", "Markdown", "Sockets", "Unicode"] uuid = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb" [[Random]] deps = ["SHA"] uuid = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c" [[Reexport]] git-tree-sha1 = "45e428421666073eab6f2da5c9d310d99bb12f9b" uuid = "189a3867-3050-52da-a836-e630ba90ab69" version = "1.2.2" [[SHA]] uuid = "ea8e919c-243c-51af-8825-aaa63cd721ce" version = "0.7.0" [[Serialization]] uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b" [[Sockets]] uuid = "6462fe0b-24de-5631-8697-dd941f90decc" [[SparseArrays]] deps = ["Libdl", "LinearAlgebra", "Random", "Serialization", "SuiteSparse_jll"] uuid = "2f01184e-e22b-5df5-ae63-d93ebab69eaf" version = "1.10.0" [[Statistics]] deps = ["LinearAlgebra", "SparseArrays"] uuid = "10745b16-79ce-11e8-11f9-7d13ad32a3b2" version = "1.10.0" [[SuiteSparse_jll]] deps = ["Artifacts", "Libdl", "libblastrampoline_jll"] uuid = "bea87d4a-7f5b-5778-9afe-8cc45184846c" version = "7.2.1+1" [[TOML]] deps = ["Dates"] uuid = "fa267f1f-6049-4f14-aa54-33bafae1ed76" version = "1.0.3" [[Tar]] deps = ["ArgTools", "SHA"] uuid = "a4e569a6-e804-4fa4-b0f3-eef7a1d5b13e" version = "1.10.0" [[Test]] deps = ["InteractiveUtils", "Logging", "Random", "Serialization"] uuid = "8dfed614-e22c-5e08-85e1-65c5234f0b40" [[Tricks]] git-tree-sha1 = "eae1bb484cd63b36999ee58be2de6c178105112f" uuid = "410a4b4d-49e4-4fbc-ab6d-cb71b17b3775" version = "0.1.8" [[URIs]] git-tree-sha1 = "67db6cc7b3821e19ebe75791a9dd19c9b1188f2b" uuid = "5c2747f8-b7ea-4ff2-ba2e-563bfd36b1d4" version = "1.5.1" [[UUIDs]] deps = ["Random", "SHA"] uuid = "cf7118a7-6976-5b1a-9a39-7adc72f591a4" [[Unicode]] uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5" [[Zlib_jll]] deps = ["Libdl"] uuid = "83775a58-1f1d-513f-b197-d71354ab007a" version = "1.2.13+1" [[libblastrampoline_jll]] deps = ["Artifacts", "Libdl"] uuid = "8e850b90-86db-534c-a0d3-1478176c7d93" version = "5.8.0+1" [[nghttp2_jll]] deps = ["Artifacts", "Libdl"] uuid = "8e850ede-7688-5339-a07c-302acd2aaf8d" version = "1.52.0+1" [[p7zip_jll]] deps = ["Artifacts", "Libdl"] uuid = "3f19e933-33d8-53b3-aaab-bd5110c3b7a0" version = "17.4.0+2" """ #  Cell order: # 67e2cb97-e224-47ca-96ba-2e89d94959e7 # b1c0d12c-f383-44fb-bcfe-4157a2801b9a # 82c316c7-a279-4728-b16a-921d7fc52886 # ede20024-1aea-4d80-a19a-8a5ec88a00ac # acb08d1f-30f9-4e01-8216-3440c82714c7 # c097b477-e154-47eb-b7d9-a4d2981dcf0e # 0b19e53d-eb7a-42b6-a7db-d95bc8c63eae # 2e54b8fc-7852-11ec-27d7-df0bfe7f344a # 0c0bab41-a020-41a0-83ad-0c57b4699ffa # 257ee9b0-d955-43d2-9c94-245716708a2d # ddccf592-0d0f-475c-81ae-067c37ba3f7e # fffd6402-d508-48a6-abc6-3de333497787 # a060f034-b540-4b1e-a87f-7e6185e15646 # 466bf852-144c-47df-98e1-89935754f5f1 # 6e26a930-1b49-4ff5-8704-9149d3cab7e9 # 0cb7b599-cec5-4391-9485-4e1c63cd9ff2 # ab108b97-4dd5-49e4-845c-2f0fab131f8a # 596dbead-63ce-432a-8a0b-b3ea361e279e # 8c96934c-3e23-45ed-b945-dce344bfb6eb # af1ad32b-af53-4865-9807-ba8d0fba2a8c # d441b495-a00c-4de3-a232-7c75f55fc95b # fa0b6647-6911-4c27-a1a6-240d215331d1 # 6c84a84f-9ead-4091-819e-0de088e2dd4d # e866282e-7c63-4364-b344-46f4c6ad165c # cd3b9ad1-8efc-4f92-96d0-b9b038d8cfae # 79b6ac0f-4d0b-485f-8fb0-9849932dc34e # eaad4fed-ea22-4132-a84a-429f486ddce2 # 146474b5-9aa6-4000-867d-ba91e4061d9b # 00000000-0000-0000-0000-000000000001 # 00000000-0000-0000-0000-000000000002 ### A Pluto.jl notebook ### # v0.19.40 using Markdown using InteractiveUtils # This Pluto notebook uses @bind for interactivity. When running this notebook outside of Pluto, the following 'mock version' of @bind gives bound variables a default value (instead of an error). macro bind(def, element) quote local iv = try Base.loaded_modules[Base.PkgId(Base.UUID("6e696c72-6542-2067-7265-42206c756150"), "AbstractPlutoDingetjes")].Bonds.initial_value catch; b -> missing; end local el = $(esc(element)) global $(esc(def)) = Core.applicable(Base.get, el) ? Base.get(el) : iv(el) el end end #  b0f2a778-885f-11ee-3d28-939ca4069ee8 begin import Pkg Pkg.activate(temp=true) Pkg.add([ Pkg.PackageSpec(name="AbstractPlutoDingetjes") Pkg.PackageSpec(name="HypertextLiteral") Pkg.PackageSpec(name="PlutoUI") ]) using AbstractPlutoDingetjes using PlutoUI using HypertextLiteral end #  5e42ea32-a1ce-49db-b55f-5e252c8c3f57 using Dates #  37aacc7f-61fd-4c4b-b24d-42361d508e8d @htl(""" <script> const sqrt_from_julia = $(AbstractPlutoDingetjes.Display.with_js_link(sqrt)) // I can now call sqrt_from_julia like a JavaScript function. It returns a Promise: const result = await sqrt_from_julia(9.0) console.log(result) </script> """) #  30d7c350-f792-47e9-873a-01adf909bc84 md""" If you change `String` to `AbstractString` here then you get some back logs: """ #  75752f77-1e3f-4997-869b-8bee2c12a2cb function cool(x::String) uppercase(x) end #  3098e16a-4730-4564-a484-02a6b0278930 # function cool() # end #  37fc039e-7a4d-4d2d-80f3-d409a9ee096d #  disabled = true #= # let # function f(x) # cool(x) # end # @htl(""" # <script> # const sqrt_from_julia = $(AbstractPlutoDingetjes.Display.with_js_link(f)) # let id = setInterval(async () => { # console.log(await sqrt_from_julia("hello")) # }, 500) # invalidation.then(() => setTimeout(() => { # clearInterval(id) # }, 1000)) # </script> # """) # end  =# #  977c59f7-9f3a-40ae-981d-2a8a48e08349 #  b3186d7b-8fd7-4575-bccf-8e89ce611010 md""" # Benchmark We call the `next` function from JavaScript in a loop until `max` is reached, to calculate the time of each round trip. """ #  82c7a083-c84d-4924-bad2-776d3cdad797 next(x) = x + 1; #  e8abaff9-f629-47c6-8009-066bcdf67693 max = 250; #  bf9861e0-be91-4041-aa61-8ac2ef6cb719 @htl(""" <div> <p><input type="submit" value="start"></p> <p>Current value: </p><input disabled type="number" value="0"></p> <p>Past values: <input disabled style="width: 100%; font-size: .4em;"></p> <p>Time per round trip: <input disabled ></p> <script> const f_from_julia = $(AbstractPlutoDingetjes.Display.with_js_link(next)) const div = currentScript.closest("div") const [submit,output,log,timer_log] = div.querySelectorAll("input") const max = $max submit.addEventListener("click", () => { output.value = 0 log.value = "0" timer_log.value = "running..." let start_time = performance.now() let next = async () => { let val = await f_from_julia(output.valueAsNumber) output.valueAsNumber = val log.value += `,\${val}` if(output.valueAsNumber < max) { await next() } else { let end_time = performance.now() let dt = (end_time - start_time) / max timer_log.value = `\${_.round(dt, 4)} ms` } } next() }) </script> </div> """) #  ebf79ee4-2590-4b5a-a957-213ed03a5921 md""" # Concurrency """ #  60444c4c-5705-4b92-8eac-2c102f14f395 #  07c832c1-fd8f-44de-bdfa-389048c1e4e9 md""" ## With a function in the closure """ #  10d80b00-f7ab-4bd7-9ea7-cca98c089e9c coolthing(x) = x #  bf7a885e-4d0a-408d-b6d5-d3289d794240 try sqrt(-1) catch e sprint(showerror, e) end #  0eff37d6-9cd5-42bb-b274-de364ca7ed53 #  663e5a70-4d07-4d6a-8725-dc9a2b26b65d md""" # Tests """ #  1d32fd55-9ca0-45c8-97f5-23cb29eaa8b3 md""" Test a closure """ #  5f3c590e-07f2-4dea-b6d1-e9d90f501fda some_other_global = rand(100) #  3d836ff3-995e-4353-807e-bf2cd78920e2 some_global = 51:81 #  2461d75e-81dc-4e00-99e3-bbc44000579f AbstractPlutoDingetjes.Display.with_js_link(x -> x) #  12e64b86-3866-4e21-9af5-0e546452b4e1 function function_evaluator(f::Function, default=""; id=string(f)) @htl(""" <div class="function_evaluator" id=$(id)> <p>Input:<br> <input>&nbsp;<input type="submit"></p> <p>Output:<br> <textarea cols=40 rows=5></textarea> <script> let sqrt_with_julia = $(AbstractPlutoDingetjes.Display.with_js_link(f)) let wrapper = currentScript.closest("div") wrapper.setAttribute("cellid", currentScript.closest("pluto-cell").id) let input = wrapper.querySelector("input") let submit = wrapper.querySelector("input[type='submit']") let output = wrapper.querySelector("textarea") submit.addEventListener("click", async () => { let result = await sqrt_with_julia(input.value) console.log({result}) output.innerText = result }) input.value = $(AbstractPlutoDingetjes.Display.published_to_js(default)) submit.click() </script> </div> """) end #  4b80dda0-74b6-4a0e-a50e-61c5380111a4 function_evaluator(900; id="sqrt") do input num = parse(Float64, input) sqrt(num) end #  a399cb12-39d4-43c0-a0a7-05cb683dffbd function_evaluator("c1"; id="c1") do input @info "start" Dates.now() sleep(3) # peakflops(3000) @warn "end" Dates.now() uppercase(input) end #  2bff3975-5918-40fe-9761-eb7b47f16df2 function_evaluator("c2"; id="c2") do input @info "start" Dates.now() sleep(3) # peakflops(3000) @warn "end" Dates.now() uppercase(input) end #  53e60352-3a56-4b5c-9568-1ac58b758497 function_evaluator("hello") do str sleep(5) result = coolthing(str) @info result result end #  2b5cc4b1-ca57-4cb6-a42a-dcb331ed2c26 let thing = function_evaluator("1") do str some_other_global[1:parse(Int,str)] end if false fff() = 123 end thing end #  85e9daf1-d8e3-4fc0-8acd-10d863e724d0 let x = rand(100) function_evaluator("1") do str x[parse(Int,str)] end end #  abb24301-357c-40f0-832e-86f26404d3d9 function_evaluator("THIS IN LOWERCASE") do input "you should see $(lowercase(input))" end #  33a2293c-6202-47ca-80d1-4a9e261cae7f function_evaluator(4; id="logs1") do input @info "you should see this log $(input)" println("(not currently supported) you should see this print $(input)") rand(parse(Int, input)) end #  480aea45-da00-4e89-b43a-38e4d1827ec2 function_evaluator("coOL") do input @warn("You should see the following error:") error("You should see this error $(uppercase(input))") end #  b310dd30-dddd-4b75-81d2-aaf35c9dd1d3 function_evaluator(4) do input @warn("You should see the assertpackable fail after this log") :(@heyyy cant msgpack me) end #  58999fba-6631-4482-a811-12bf2412d65e function_evaluator(4; id="globals") do input some_global[parse(Int, input)] end #  9e5c0f8d-6ac1-4aee-a00d-938f17eec146 md""" You should be able to use `with_js_link` multiple times within one cell, and they should work independently of eachother: """ #  306d03da-cd50-4b0c-a5dd-7ec1a278cde1 @htl(""" <div style="display: flex; flex-direction: row;"> $(function_evaluator(uppercase, "")) $(function_evaluator(lowercase, "")) </div> """) #  2cf033a7-bcd7-434d-9faf-ea761897fb64 md""" You should be able to set up a `with_js_link` in one cell, and use it in another. This example is a bit trivial though... """ #  40031867-ee3c-4aa9-884f-b76b5a9c4dec fe = function_evaluator(length, "Alberto") #  7f6ada79-8e3b-40b7-b477-ce05ae79a668 fe #  f344c4cb-8226-4145-ab92-a37542f697dd md""" You should see a warning message when `with_js_link` is not used inside an HTML renderer that supports it: """ #  8bbd32f8-56f7-4f29-aea8-6906416f6cfd let html_repr = repr(MIME"text/html"(), fe) HTML(html_repr) end #  8782cc14-eb1a-48a8-a114-2f71f77be275 @bind yolotrigger CounterButton() #  e5df2451-f4b9-4511-b25f-1a5e463f3eb2 name = yolotrigger > 0 ? "friet" : "frietje" #  3c5c1325-ad3e-4c54-8d29-c17939bb8529 function useme(x) length(x) > 5 ? uppercase(x) : error("bad") end #  6c5f79b9-598d-41ad-800d-0a9ff63d6f6c @htl(""" <input type=submit id=jslogbtn> <script id="yolo"> const btn = currentScript.parentElement.querySelector("input") const pre = this ?? document.createElement("pre") pre.id = "checkme" let log = (t) => { pre.innerText = pre.innerText + "\\n" + t } let logyay = x => log("yay " + x) let lognee = x => log("nee " + x) const f = $(AbstractPlutoDingetjes.Display.with_js_link(useme)) btn.addEventListener("click", () => { log("click") setTimeout(async () => { f($name).then(logyay).catch(lognee) }, 2000) }) log("hello!") return pre </script> """) #  Cell order: # b0f2a778-885f-11ee-3d28-939ca4069ee8 # 4b80dda0-74b6-4a0e-a50e-61c5380111a4 # 37aacc7f-61fd-4c4b-b24d-42361d508e8d # 30d7c350-f792-47e9-873a-01adf909bc84 # 75752f77-1e3f-4997-869b-8bee2c12a2cb # 3098e16a-4730-4564-a484-02a6b0278930 # 37fc039e-7a4d-4d2d-80f3-d409a9ee096d # 977c59f7-9f3a-40ae-981d-2a8a48e08349 # b3186d7b-8fd7-4575-bccf-8e89ce611010 # 82c7a083-c84d-4924-bad2-776d3cdad797 # e8abaff9-f629-47c6-8009-066bcdf67693 # bf9861e0-be91-4041-aa61-8ac2ef6cb719 # ebf79ee4-2590-4b5a-a957-213ed03a5921 # a399cb12-39d4-43c0-a0a7-05cb683dffbd # 5e42ea32-a1ce-49db-b55f-5e252c8c3f57 # 60444c4c-5705-4b92-8eac-2c102f14f395 # 2bff3975-5918-40fe-9761-eb7b47f16df2 # 07c832c1-fd8f-44de-bdfa-389048c1e4e9 # 10d80b00-f7ab-4bd7-9ea7-cca98c089e9c # 53e60352-3a56-4b5c-9568-1ac58b758497 # bf7a885e-4d0a-408d-b6d5-d3289d794240 # 0eff37d6-9cd5-42bb-b274-de364ca7ed53 # 663e5a70-4d07-4d6a-8725-dc9a2b26b65d # 1d32fd55-9ca0-45c8-97f5-23cb29eaa8b3 # 5f3c590e-07f2-4dea-b6d1-e9d90f501fda # 2b5cc4b1-ca57-4cb6-a42a-dcb331ed2c26 # 85e9daf1-d8e3-4fc0-8acd-10d863e724d0 # abb24301-357c-40f0-832e-86f26404d3d9 # 33a2293c-6202-47ca-80d1-4a9e261cae7f # 480aea45-da00-4e89-b43a-38e4d1827ec2 # b310dd30-dddd-4b75-81d2-aaf35c9dd1d3 # 3d836ff3-995e-4353-807e-bf2cd78920e2 # 58999fba-6631-4482-a811-12bf2412d65e # 2461d75e-81dc-4e00-99e3-bbc44000579f # 12e64b86-3866-4e21-9af5-0e546452b4e1 # 9e5c0f8d-6ac1-4aee-a00d-938f17eec146 # 306d03da-cd50-4b0c-a5dd-7ec1a278cde1 # 2cf033a7-bcd7-434d-9faf-ea761897fb64 # 40031867-ee3c-4aa9-884f-b76b5a9c4dec # 7f6ada79-8e3b-40b7-b477-ce05ae79a668 # f344c4cb-8226-4145-ab92-a37542f697dd # 8bbd32f8-56f7-4f29-aea8-6906416f6cfd # 8782cc14-eb1a-48a8-a114-2f71f77be275 # e5df2451-f4b9-4511-b25f-1a5e463f3eb2 # 3c5c1325-ad3e-4c54-8d29-c17939bb8529 # 6c5f79b9-598d-41ad-800d-0a9ff63d6f6c import puppeteer from "puppeteer" import path from "path"; import mkdirp from "mkdirp"; import * as process from "process"; // from https://github.com/puppeteer/puppeteer/issues/1908#issuecomment-380308269 class InflightRequests { constructor(page) { this._page = page; this._requests = new Map(); this._history = []; this._onStarted = this._onStarted.bind(this); this._onFinished = this._onFinished.bind(this); this._page.on('request', this._onStarted); this._page.on('requestfinished', this._onFinished); this._page.on('requestfailed', this._onFinished); } _onStarted(request) { // if(request.url().includes("data")) { // console.log('Start', request.url()) // }; this._history.push(["started", request.url()]); this._requests.set( request.url(), 1 + (this._requests.get(request.url()) ?? 0) ); } _onFinished(request) { // if(request.url().includes("data")) { // console.log('Finish', request.url()) // }; this._history.push(["finished", request.url()]); this._requests.set( request.url(), -1 + /* Multiple requests starts can have a single finish event. */ Math.min(1, this._requests.get(request.url()) ?? 0) ); } inflightRequests() { return Array.from([...this._requests.entries()].flatMap(([k,v]) => v > 0 ? [k] : [])); } dispose() { this._page.removeListener('request', this._onStarted); this._page.removeListener('requestfinished', this._onFinished); this._page.removeListener('requestfailed', this._onFinished); } } const with_connections_debug = (page, action) => { const tracker = new InflightRequests(page); return action().finally(() => { tracker.dispose(); const inflight = tracker.inflightRequests(); if(inflight.length > 0) { console.warn("Open connections: ", inflight, tracker._history.filter(([n,u]) => inflight.includes(u))); // console.warn([...tracker._requests.entries()]) } }).catch(e => { throw e }) } export const getTextContent = (page, selector) => { // https://developer.mozilla.org/en-US/docs/Web/API/Node/textContent#differences_from_innertext return page.evaluate( (selector) => document.querySelector(selector)?.textContent, selector ); }; export const countCells = async (page) => await page.evaluate(() => { const a = Array.from(document.querySelectorAll("pluto-cell")); return a?.length; }); export const paste = async (page, code, selector = "body") => { const ret = await page.evaluate( (code, selector) => { var clipboardEvent = new Event("paste", { bubbles: true, cancelable: true, composed: true, }); clipboardEvent["clipboardData"] = { getData: () => code, }; document.querySelector(selector).dispatchEvent(clipboardEvent); }, code, selector ); return ret; }; export const waitForContent = async (page, selector) => { await page.waitForSelector(selector, { visible: true }); await page.waitForFunction( (selector) => { const element = document.querySelector(selector); return element !== null && element.textContent.length > 0; }, { polling: 100 }, selector ); return getTextContent(page, selector); }; export const waitForContentToChange = async ( page, selector, currentContent ) => { await page.waitForSelector(selector, { visible: true }); await page.waitForFunction( (selector, currentContent) => { const element = document.querySelector(selector); console.log(`element:`, element); return element !== null && element.textContent !== currentContent; }, { polling: 100 }, selector, currentContent ); return getTextContent(page, selector); }; export const waitForContentToBecome = async (/** @type {puppeteer.Page} */ page, /** @type {string} */ selector, /** @type {string} */ targetContent) => { await page.waitForSelector(selector, { visible: true }); try{ await page.waitForFunction( (selector, targetContent) => { const element = document.querySelector(selector); // https://developer.mozilla.org/en-US/docs/Web/API/Node/textContent#differences_from_innertext return element !== null && element.textContent === targetContent; }, { polling: 100 }, selector, targetContent ); } catch(e) { console.error("Failed! Current content: ", JSON.stringify(await getTextContent(page, selector)), "Expected content: ", JSON.stringify(targetContent)) throw(e) } return getTextContent(page, selector); }; export const clickAndWaitForNavigation = async (page, selector) => { let t = with_connections_debug(page, () => page.waitForNavigation({ waitUntil: "networkidle0" })).catch(e => { console.warn("Network idle never happened after navigation... weird!", e) }) await page.click(selector) await t } const dismissBeforeUnloadDialogs = (page) => { page.on("dialog", async (dialog) => { if (dialog.type() === "beforeunload") { await dialog.accept(); } }); }; const dismissVersionDialogs = (page) => { page.on("dialog", async (dialog) => { if ( dialog.message().includes("A new version of Pluto.jl is available! ") ) { console.info( "Ignoring version warning for now (but do remember to update Project.toml!)." ); await dialog.accept(); } }); }; const failOnError = (page) => { page.on("console", async (msg) => { if (msg.type() === "error" && msg.text().includes("PlutoError")) { console.error(`Bad PlutoError - Failing\n${msg.text()}`); throw new Error("PlutoError encountered. Let's fix this!"); } }); }; let should_be_offline_input = process.env["PLUTO_TEST_OFFLINE"]?.toLowerCase() ?? "false" let should_be_offline = [true, 1, "true", "1"].includes(should_be_offline_input) console.log(`Offline mode enabled: ${should_be_offline}`) const blocked_domains = ["cdn.jsdelivr.net", "unpkg.com", "cdn.skypack.dev", "esm.sh", "firebase.google.com"] const hide_warning = url => url.includes("mathjax") export const createPage = async (browser) => { /** @type {puppeteer.Page} */ const page = await browser.newPage() failOnError(page); dismissBeforeUnloadDialogs(page); dismissVersionDialogs(page); if(should_be_offline) { page.setRequestInterception(true); page.on("request", (request) => { if(blocked_domains.some(domain => request.url().includes(domain))) { if(!hide_warning(request.url())) console.info(`Blocking request to ${request.url()}`) request.abort(); } else { request.continue(); } }); } return page }; let testname = () => expect.getState()?.currentTestName?.replace(/[ \:]/g, "_") ?? "unnkown"; export const lastElement = (arr) => arr[arr.length - 1]; const getFixturesDir = () => path.join(__dirname, "..", "fixtures"); export const getArtifactsDir = () => path.join(__dirname, "..", "artifacts"); export const getFixtureNotebookPath = (name) => path.join(getFixturesDir(), name); export const getTemporaryNotebookPath = () => path.join( getArtifactsDir(), `temporary_notebook_${testname()}_${Date.now()}.jl` ); export const getTestScreenshotPath = () => { return path.join( getArtifactsDir(), `screenshot_${testname()}_${Date.now()}.png` ); }; export const saveScreenshot = async (page, screenshot_path=getTestScreenshotPath()) => { let dirname = path.dirname(screenshot_path); await mkdirp(dirname); // Because some of our tests contain /'s  await page.screenshot({ path: screenshot_path }); }; import puppeteer from "puppeteer" import fs, { existsSync, writeFile, writeFileSync } from "fs" import { platform } from "process" import { Browser, Page } from "puppeteer" import { clickAndWaitForNavigation, getFixtureNotebookPath, getTemporaryNotebookPath, waitForContent, waitForContentToChange, getTextContent, lastElement, createPage, getArtifactsDir, waitForContentToBecome, } from "./common" import path from "path" // if (!process.env.PLUTO_PORT) { // throw new Error("You didn't set the PLUTO_PORT environment variable") // } export const getPlutoUrl = () => `http://localhost:${process.env.PLUTO_PORT}` // we use a file to keep track of whether we already prewarmed pluto or not const pluto_is_warm_path = path.join(getArtifactsDir(), `pluto_is_warm.txt`) /** * @param {Browser} browser */ export const prewarmPluto = async (browser) => { if (existsSync(pluto_is_warm_path)) { return } writeFileSync(pluto_is_warm_path, "yes") let page = await createPage(browser) await browser.defaultBrowserContext().overridePermissions(getPlutoUrl(), ["clipboard-read", "clipboard-write"]) await gotoPlutoMainMenu(page) await createNewNotebook(page) const cellInputSelector = "pluto-input .cm-content" await page.waitForSelector(cellInputSelector, { visible: true }) await writeSingleLineInPlutoInput(page, "pluto-input", "21*2") const runSelector = ".runcell" await page.waitForSelector(runSelector, { visible: true }) await page.click(runSelector) await waitForContent(page, "pluto-output") await shutdownCurrentNotebook(page) await page.close() } /** * @param {Page} page */ export const shutdownCurrentNotebook = async (page) => { await page.evaluate( //@ts-ignore () => window.shutdownNotebook?.() ) } export const setupPlutoBrowser = async () => { const browser = await puppeteer.launch({ headless: process.env.HEADLESS !== "false", args: ["--no-sandbox"], devtools: false, }) await prewarmPluto(browser) return browser } /** * @param {Page} page */ export const gotoPlutoMainMenu = async (page) => { await page.goto(getPlutoUrl(), { waitUntil: "domcontentloaded" }) await page.waitForFunction(() => document.querySelector(`.not_yet_ready`) == null) } /** * @param {Page} page */ export const createNewNotebook = async (page) => { const newNotebookSelector = 'a[href="new"]' await page.waitForSelector(newNotebookSelector) await clickAndWaitForNavigation(page, newNotebookSelector) await page.waitForTimeout(1000) await waitForPlutoToCalmDown(page) await page.waitForSelector("pluto-input", { visible: true }) } /** * @param {Page} page * @param {string} notebookName` */ export const importNotebook = async (page, notebookName, { permissionToRunCode = true, timeout = 60000 } = {}) => { // Copy notebook before using it, so we don't mess it up with test changes const notebookPath = getFixtureNotebookPath(notebookName) const artifactsPath = getTemporaryNotebookPath() fs.copyFileSync(notebookPath, artifactsPath) await openPathOrURLNotebook(page, artifactsPath, { permissionToRunCode, timeout }) } /** * @param {Page} page * @param {string} path_or_url */ export const openPathOrURLNotebook = async (page, path_or_url, { permissionToRunCode = true, timeout = 60000 } = {}) => { await page.waitForFunction(() => document.querySelector(`.not_yet_ready`) == null) const openFileInputSelector = "pluto-filepicker" await writeSingleLineInPlutoInput(page, openFileInputSelector, path_or_url) // await writeSingleLineInPlutoInput(page, openFileInputSelector, notebookPath) const openFileButton = "pluto-filepicker button" await clickAndWaitForNavigation(page, openFileButton) // Give permission to run code in this notebook if (permissionToRunCode) await restartProcess(page) await page.waitForTimeout(1000) await waitForPlutoToCalmDown(page, { polling: "raf", timeout }) } /** * @param {Page} page */ export const getCellIds = (page) => page.evaluate(() => Array.from(document.querySelectorAll("pluto-cell")).map((cell) => cell.id)) /** * @param {Page} page */ export const restartProcess = async (page) => { await page.waitForSelector(`a#restart-process-button`) await page.click(`a#restart-process-button`) // page.once("dialog", async (dialog) => { // await dialog.accept() // }) await page.waitForFunction(() => document?.querySelector(`a#restart-process-button`) == null) await page.waitForSelector(`#process-status-tab-button.something_is_happening`) } /** * @param {Page} page * @param {boolean} iWantBusiness */ const waitForPlutoBusy = async (page, iWantBusiness, options) => { await page.waitForTimeout(1) try { await page.waitForFunction( (iWantBusiness) => { const quiet_vals = [ // @ts-ignore document?.body?._update_is_ongoing, // @ts-ignore document?.body?._js_init_set?.size, document?.body?.classList?.contains("loading"), document?.querySelector(`#process-status-tab-button.something_is_happening`)?.id, document?.querySelector(`pluto-cell.running, pluto-cell.queued, pluto-cell.internal_test_queued`)?.id, ] let quiet = (quiet_vals[0] ?? false) === false && (quiet_vals[1] ?? 0) === 0 && quiet_vals[2] === false && quiet_vals[3] == null && quiet_vals[4] == null window["quiet_vals"] = quiet_vals return iWantBusiness ? !quiet : quiet }, { timeout: 60000, ...options, }, iWantBusiness ) } catch (e) { console.error( "waitForPlutoBusy failed\n", JSON.parse( await page.evaluate(() => { return JSON.stringify(window["quiet_vals"]) }) ) ) throw e } await page.waitForTimeout(1) } export const waitForPlutoToCalmDown = async (/** @type {puppeteer.Page} */ page, /** @type {{ polling: string | number; timeout?: number; }} */ options) => { await waitForPlutoBusy(page, false, options) } /** * @param {Page} page * @param {string} cellId */ export const waitForCellOutput = (page, cellId) => { const cellOutputSelector = `pluto-cell[id="${cellId}"] pluto-output` return waitForContent(page, cellOutputSelector) } /** * @param {Page} page */ export const getAllCellOutputs = (page) => page.evaluate(() => Array.from(document.querySelectorAll(`pluto-cell > pluto-output`)).map((c) => c.innerText)) /** * @param {Page} page * @param {string} cellId * @param {string} currentOutput */ export const waitForCellOutputToChange = (page, cellId, currentOutput) => { const cellOutputSelector = `pluto-cell[id="${cellId}"] pluto-output` return waitForContentToChange(page, cellOutputSelector, currentOutput) } export const waitForNoUpdateOngoing = async (page, options = {}) => { await page.waitForTimeout(1000) return await page.waitForFunction( () => //@ts-ignore (document.body?._update_is_ongoing ?? false) === false, options ) } export const getLogSelector = (cellId) => `pluto-cell[id="${cellId}"] pluto-logs` export const getLogs = async (page, cellid) => { return await page.evaluate((sel) => { const logs = document.querySelector(sel) return Array.from(logs.children).map((el) => ({ class: el.className.trim(), description: el.querySelector("pluto-log-dot > pre").textContent, kwargs: Object.fromEntries( Array.from(el.querySelectorAll("pluto-log-dot-kwarg")).map((x) => [ x.querySelector("pluto-key").textContent, x.querySelector("pluto-value").textContent, ]) ), })) }, getLogSelector(cellid)) } /** * @param {Page} page */ export const runAllChanged = async (page) => { await page.waitForSelector(`.runallchanged`, { visible: true, }) await page.click(`.runallchanged`) await waitForPlutoBusy(page, true) await waitForPlutoBusy(page, false) } /** * @param {Page} page * @param {string} plutoInputSelector * @param {string} text */ export const writeSingleLineInPlutoInput = async (page, plutoInputSelector, text) => { await page.waitForSelector(`${plutoInputSelector} .cm-editor:not(.cm-ssr-fake)`) await page.type(`${plutoInputSelector} .cm-content`, text) // Wait for CodeMirror to process the input and display the text return await page.waitForFunction( (plutoInputSelector, text) => { const codeMirrorLine = document.querySelector(`${plutoInputSelector} .cm-line`) return codeMirrorLine?.textContent?.endsWith?.(text) ?? false }, { polling: 100 }, plutoInputSelector, text ) } /** * @param {Page} page * @param {string} plutoInputSelector * @param {import("puppeteer").KeyInput} key */ export const keyboardPressInPlutoInput = async (page, plutoInputSelector, key) => { const currentLineText = await getTextContent(page, `${plutoInputSelector} .cm-line`) await page.focus(`${plutoInputSelector} .cm-content`) await page.waitForTimeout(500) // Move to end of the input await page.keyboard.down(platform === "darwin" ? "Meta" : "Control") await page.keyboard.press("ArrowDown") await page.keyboard.up(platform === "darwin" ? "Meta" : "Control") // Press the key we care about await page.keyboard.press(key) await page.waitForTimeout(500) // Wait for CodeMirror to process the input and display the text return waitForContentToChange(page, `${plutoInputSelector} .cm-line`, currentLineText) } /** * @param {Page} page * @param {string} plutoInputSelector */ export const clearPlutoInput = async (page, plutoInputSelector) => { await page.waitForSelector(`${plutoInputSelector} .cm-editor:not(.cm-ssr-fake)`) if ((await page.$(`${plutoInputSelector} .cm-placeholder`)) == null) { await page.focus(`${plutoInputSelector} .cm-content`) await page.waitForTimeout(500) // Move to end of the input await page.keyboard.down(platform === "darwin" ? "Meta" : "Control") await page.keyboard.press("KeyA") await page.keyboard.up(platform === "darwin" ? "Meta" : "Control") // Press the key we care about await page.keyboard.press("Delete") // Wait for CodeMirror to process the input and display the text await page.waitForSelector(`${plutoInputSelector} .cm-placeholder`) } } /** * @param {Page} page * @param {string[]} cells */ export const manuallyEnterCells = async (page, cells) => { const plutoCellIds = [] for (const cell of cells) { const plutoCellId = lastElement(await getCellIds(page)) plutoCellIds.push(plutoCellId) await page.waitForSelector(`pluto-cell[id="${plutoCellId}"] pluto-input .cm-editor:not(.cm-ssr-fake) .cm-content`) await writeSingleLineInPlutoInput(page, `pluto-cell[id="${plutoCellId}"] pluto-input`, cell) await page.click(`pluto-cell[id="${plutoCellId}"] .add_cell.after`) await page.waitForFunction((nCells) => document.querySelectorAll("pluto-cell").length === nCells, {}, plutoCellIds.length + 1) } return plutoCellIds } # using LibGit2 import Pkg using Test using Pluto.Configuration: CompilerOptions import Pluto: update_save_run!, update_run!, WorkspaceManager, ClientSession, ServerSession, Notebook, Cell, project_relative_path, SessionActions, load_notebook import Pluto.PkgUtils import Pluto.PkgCompat import Malt @testset "Built-in Pkg" begin # We have our own registry for these test! Take a look at https://github.com/JuliaPluto/PlutoPkgTestRegistry#readme for more info about the test packages and their dependencies. Pkg.Registry.add(pluto_test_registry_spec) @testset "Basic $(use_distributed_stdlib ? "Distributed" : "Malt")" for use_distributed_stdlib in (false, true)  = ServerSession() .options.evaluation.workspace_use_distributed_stdlib = use_distributed_stdlib # See https://github.com/JuliaPluto/PlutoPkgTestRegistry notebook = Notebook([ Cell("import PlutoPkgTestA"), # cell 1 Cell("PlutoPkgTestA.MY_VERSION |> Text"), Cell("import PlutoPkgTestB"), # cell 3 Cell("PlutoPkgTestB.MY_VERSION |> Text"), Cell("import PlutoPkgTestC"), # cell 5 Cell("PlutoPkgTestC.MY_VERSION |> Text"), Cell("import PlutoPkgTestD"), # cell 7 Cell("PlutoPkgTestD.MY_VERSION |> Text"), Cell("import Dates"), # eval to hide the import from Pluto's analysis Cell("eval(:(import DataFrames))"), Cell("import HelloWorldC_jll"), ]) @test !notebook.nbpkg_ctx_instantiated update_save_run!(, notebook, notebook.cells[[1, 2, 7, 8]]) # import A and D @test noerror(notebook.cells[1]) @test noerror(notebook.cells[2]) @test noerror(notebook.cells[7]) @test noerror(notebook.cells[8]) @test notebook.nbpkg_ctx !== nothing @test notebook.nbpkg_restart_recommended_msg === nothing @test notebook.nbpkg_restart_required_msg === nothing @test notebook.nbpkg_ctx_instantiated @test notebook.nbpkg_install_time_ns > 0 @test notebook.nbpkg_busy_packages == [] last_install_time = notebook.nbpkg_install_time_ns terminals = notebook.nbpkg_terminal_outputs @test haskey(terminals, "PlutoPkgTestA") @test haskey(terminals, "PlutoPkgTestD") # they were installed in one batch, so their terminal outputs should be the same @test terminals["PlutoPkgTestA"] == terminals["PlutoPkgTestD"] # " [9e88b42a] PackageName" should be present in terminal output @test !isnothing(match(r"\[........\] ", terminals["PlutoPkgTestA"])) @test notebook.cells[2].output.body == "0.3.1" # A @test notebook.cells[8].output.body == "0.1.0" # D @test PkgCompat.get_manifest_version(notebook.nbpkg_ctx, "PlutoPkgTestA") == v"0.3.1" @test PkgCompat.get_manifest_version(notebook.nbpkg_ctx, "PlutoPkgTestD") == v"0.1.0" old_A_terminal = deepcopy(terminals["PlutoPkgTestA"]) # @show old_A_terminal update_save_run!(, notebook, notebook.cells[[3, 4]]) # import B @test noerror(notebook.cells[3]) @test noerror(notebook.cells[4]) @test notebook.nbpkg_ctx !== nothing @test notebook.nbpkg_restart_recommended_msg === nothing @test notebook.nbpkg_restart_required_msg === nothing @test notebook.nbpkg_ctx_instantiated @test notebook.nbpkg_install_time_ns > last_install_time @test notebook.nbpkg_terminal_outputs["nbpkg_sync"] != "" @test notebook.nbpkg_terminal_outputs["PlutoPkgTestB"] != "" @test occursin("+ PlutoPkgTestB", notebook.nbpkg_terminal_outputs["PlutoPkgTestB"]) @test notebook.nbpkg_busy_packages == [] last_install_time = notebook.nbpkg_install_time_ns @test haskey(terminals, "PlutoPkgTestB") @test terminals["PlutoPkgTestA"] == terminals["PlutoPkgTestD"] == old_A_terminal @test terminals["PlutoPkgTestA"] != terminals["PlutoPkgTestB"] @test notebook.cells[4].output.body == "1.0.0" # B # running the 5th cell will import PlutoPkgTestC, putting a 0.2 compatibility bound on PlutoPkgTestA. This means that a notebook restart is required, since PlutoPkgTestA was already loaded at version 0.3.1. update_save_run!(, notebook, notebook.cells[[5, 6]]) @test noerror(notebook.cells[5]) @test noerror(notebook.cells[6]) @test notebook.nbpkg_ctx !== nothing @test ( notebook.nbpkg_restart_recommended_msg !== nothing || notebook.nbpkg_restart_required_msg !== nothing ) @test notebook.nbpkg_restart_required_msg !== nothing @test notebook.nbpkg_install_time_ns > last_install_time # running cells again should persist the restart message update_save_run!(, notebook, notebook.cells[1:8]) @test notebook.nbpkg_restart_required_msg !== nothing Pluto.response_restart_process(Pluto.ClientRequest( session=, notebook=notebook, ); run_async=false) # @test_nowarn SessionActions.shutdown(, notebook; keep_in_session=true, async=true) # @test_nowarn update_save_run!(, notebook, notebook.cells[1:8]; , save=true) @test noerror(notebook.cells[1]) @test noerror(notebook.cells[2]) @test noerror(notebook.cells[3]) @test noerror(notebook.cells[4]) @test noerror(notebook.cells[5]) @test noerror(notebook.cells[6]) @test noerror(notebook.cells[7]) @test noerror(notebook.cells[8]) @test noerror(notebook.cells[11]) @test notebook.nbpkg_ctx !== nothing @test notebook.nbpkg_restart_recommended_msg === nothing @test notebook.nbpkg_restart_required_msg === nothing @test notebook.cells[2].output.body == "0.2.2" @test notebook.cells[4].output.body == "1.0.0" @test notebook.cells[6].output.body == "1.0.0" @test notebook.cells[8].output.body == "0.1.0" update_save_run!(, notebook, notebook.cells[9]) @test noerror(notebook.cells[9]) @test notebook.nbpkg_ctx !== nothing @test notebook.nbpkg_restart_recommended_msg === nothing @test notebook.nbpkg_restart_required_msg === nothing # we should have an isolated environment, so importing DataFrames should not work, even though it is available in the parent process. update_save_run!(, notebook, notebook.cells[10]) @test notebook.cells[10].errored == true ptoml_contents() = PkgCompat.read_project_file(notebook) mtoml_contents() = PkgCompat.read_manifest_file(notebook) nb_contents() = read(notebook.path, String) @testset "Project & Manifest stored in notebook" begin @test occursin(ptoml_contents(), nb_contents()) @test occursin(mtoml_contents(), nb_contents()) @test occursin("PlutoPkgTestA", mtoml_contents()) @test occursin("PlutoPkgTestB", mtoml_contents()) @test occursin("PlutoPkgTestC", mtoml_contents()) @test occursin("PlutoPkgTestD", mtoml_contents()) @test occursin("Dates", mtoml_contents()) @test count("PlutoPkgTestA", ptoml_contents()) == 2 # once in [deps], once in [compat] @test count("PlutoPkgTestB", ptoml_contents()) == 2 @test count("PlutoPkgTestC", ptoml_contents()) == 2 @test count("PlutoPkgTestD", ptoml_contents()) == 2 @test count("Dates", ptoml_contents()) == 1 # once in [deps], but not in [compat] because it is a stdlib ptoml = Pkg.TOML.parse(ptoml_contents()) @test haskey(ptoml["compat"], "PlutoPkgTestA") @test haskey(ptoml["compat"], "PlutoPkgTestB") @test haskey(ptoml["compat"], "PlutoPkgTestC") @test haskey(ptoml["compat"], "PlutoPkgTestD") @test !haskey(ptoml["compat"], "Dates") end ## remove `import Dates` setcode!(notebook.cells[9], "") update_save_run!(, notebook, notebook.cells[9]) # removing a stdlib does not require a restart @test noerror(notebook.cells[9]) @test notebook.nbpkg_ctx !== nothing @test notebook.nbpkg_restart_recommended_msg === nothing @test notebook.nbpkg_restart_required_msg === nothing @test notebook.nbpkg_busy_packages == [] @test notebook.nbpkg_terminal_outputs["nbpkg_sync"] != "" @test notebook.nbpkg_terminal_outputs["Dates"] != "" @test occursin("- Dates", notebook.nbpkg_terminal_outputs["Dates"]) @test occursin("- Dates", notebook.nbpkg_terminal_outputs["nbpkg_sync"]) @test count("Dates", ptoml_contents()) == 0 ## remove `import PlutoPkgTestD` setcode!(notebook.cells[7], "") update_save_run!(, notebook, notebook.cells[7]) @test noerror(notebook.cells[7]) @test notebook.nbpkg_ctx !== nothing @test notebook.nbpkg_restart_recommended_msg !== nothing # recommend restart @test notebook.nbpkg_restart_required_msg === nothing @test notebook.nbpkg_install_time_ns === nothing # removing a package means that we lose our estimate @test notebook.nbpkg_busy_packages == [] @test notebook.nbpkg_terminal_outputs["nbpkg_sync"] != "" @test notebook.nbpkg_terminal_outputs["PlutoPkgTestD"] != "" @test occursin("- PlutoPkgTestD", notebook.nbpkg_terminal_outputs["PlutoPkgTestD"]) @test occursin("- PlutoPkgTestD", notebook.nbpkg_terminal_outputs["nbpkg_sync"]) @test count("PlutoPkgTestD", ptoml_contents()) == 0 cleanup(, notebook) end simple_import_path = joinpath(@__DIR__, "simple_import.jl") simple_import_notebook = read(simple_import_path, String) @testset "Manifest loading" begin  = ServerSession() dir = mktempdir() path = joinpath(dir, "hello.jl") write(path, simple_import_notebook) notebook = SessionActions.open(, path; run_async=false) @test num_backups_in(dir) == 0 @test notebook.nbpkg_ctx !== nothing @test notebook.nbpkg_restart_recommended_msg === nothing @test notebook.nbpkg_restart_required_msg === nothing @test noerror(notebook.cells[1]) @test noerror(notebook.cells[2]) @test notebook.cells[2].output.body == "0.2.2" cleanup(, notebook) end @testset "Package added by url" begin url_notebook = read(joinpath(@__DIR__, "url_import.jl"), String)  = ServerSession() dir = mktempdir() path = joinpath(dir, "hello.jl") write(path, url_notebook) notebook = SessionActions.open(, path; run_async=false) @test num_backups_in(dir) == 0 @test notebook.nbpkg_ctx !== nothing @test notebook.nbpkg_restart_recommended_msg === nothing @test notebook.nbpkg_restart_required_msg === nothing @test noerror(notebook.cells[1]) @test noerror(notebook.cells[2]) @test notebook.cells[2].output.body == "1.0.0" cleanup(, notebook) end future_notebook = read(joinpath(@__DIR__, "future_nonexisting_version.jl"), String) @testset "Recovery from unavailable versions" begin  = ServerSession() dir = mktempdir() path = joinpath(dir, "hello.jl") write(path, future_notebook) notebook = SessionActions.open(, path; run_async=false) @test num_backups_in(dir) == 0 @test notebook.nbpkg_ctx !== nothing @test notebook.nbpkg_restart_recommended_msg === nothing @test notebook.nbpkg_restart_required_msg === nothing @test noerror(notebook.cells[1]) @test noerror(notebook.cells[2]) @test notebook.cells[2].output.body == "0.3.1" cleanup(, notebook) end @testset "Pkg cell -- dynamically added" begin  = ServerSession() notebook = Notebook([ Cell("1"), Cell("2"), Cell("3"), Cell("4"), Cell("5"), Cell("6"), ]) update_save_run!(, notebook, notebook.cells) # not necessary since there are no packages: # @test has_embedded_pkgfiles(notebook) setcode!(notebook.cells[1], "import Pkg") update_save_run!(, notebook, notebook.cells[1]) setcode!(notebook.cells[2], "Pkg.activate(mktempdir())") update_save_run!(, notebook, notebook.cells[2]) @test noerror(notebook.cells[1]) @test noerror(notebook.cells[2]) @test notebook.nbpkg_ctx === nothing @test notebook.nbpkg_restart_recommended_msg === nothing @test notebook.nbpkg_restart_required_msg === nothing @test !has_embedded_pkgfiles(notebook) setcode!(notebook.cells[3], "Pkg.add(\"JSON\")") update_save_run!(, notebook, notebook.cells[3]) setcode!(notebook.cells[4], "using JSON") update_save_run!(, notebook, notebook.cells[4]) setcode!(notebook.cells[5], "using Dates") update_save_run!(, notebook, notebook.cells[5]) @test noerror(notebook.cells[3]) @test noerror(notebook.cells[4]) @test notebook.cells[5] |> noerror @test !has_embedded_pkgfiles(notebook) setcode!(notebook.cells[2], "2") setcode!(notebook.cells[3], "3") update_save_run!(, notebook, notebook.cells[2:3]) @test notebook.nbpkg_ctx !== nothing @test notebook.nbpkg_restart_required_msg !== nothing @test has_embedded_pkgfiles(notebook) cleanup(, notebook) end pkg_cell_notebook = read(joinpath(@__DIR__, "pkg_cell.jl"), String) @testset "Pkg cell -- loaded from file" begin  = ServerSession() dir = mktempdir() for n in ["Project.toml", "Manifest.toml"] cp(joinpath(@__DIR__, "pkg_cell_env", n), joinpath(dir, n)) end path = joinpath(dir, "hello.jl") write(path, pkg_cell_notebook) @test length(readdir(dir)) == 3 @test num_backups_in(dir) == 0 notebook = SessionActions.open(, path; run_async=false) nb_contents() = read(notebook.path, String) @test num_backups_in(dir) == 0 # @test num_backups_in(dir) == 1 @test noerror(notebook.cells[1]) @test noerror(notebook.cells[2]) @test noerror(notebook.cells[3]) @test noerror(notebook.cells[4]) @test noerror(notebook.cells[5]) @test noerror(notebook.cells[6]) @test noerror(notebook.cells[7]) @test noerror(notebook.cells[8]) @test noerror(notebook.cells[9]) @test noerror(notebook.cells[10]) @test notebook.cells[3].output.body == "0.2.0" file_after_loading = read(path, String) # test that no pkg cells got added @test !has_embedded_pkgfiles(notebook) # we can remove this test in the future if our file format changes same_num_chars = -10 < length(replace(file_after_loading, '\r' => "")) - length(replace(pkg_cell_notebook, '\r' => "")) < 10 if !same_num_chars @show file_after_loading pkg_cell_notebook end @test same_num_chars @test notebook.nbpkg_ctx === nothing @test notebook.nbpkg_restart_recommended_msg === nothing @test notebook.nbpkg_restart_required_msg === nothing cleanup(, notebook) end @testset "DrWatson cell" begin  = ServerSession() notebook = Notebook([ Cell("using Plots"), Cell("@quickactivate"), Cell("using DrWatson"), ]) notebook.topology = Pluto.updated_topology(Pluto.NotebookTopology{Cell}(cell_order=Pluto.ImmutableVector(notebook.cells)), notebook, notebook.cells) |> Pluto.static_resolve_topology @test !Pluto.use_plutopkg(notebook.topology) order = collect(Pluto.topological_order(notebook)) index_order = map(order) do order_cell findfirst(==(order_cell.cell_id), notebook.cell_order) end @test index_order == [3, 2, 1] end @testset "File format -- Backwards compat" begin  = ServerSession() pre_pkg_notebook = read(joinpath(@__DIR__, "old_import.jl"), String) dir = mktempdir() path = joinpath(dir, "hello.jl") write(path, pre_pkg_notebook) @test num_backups_in(dir) == 0 notebook = SessionActions.open(, path; run_async=false) nb_contents() = read(notebook.path, String) @test num_backups_in(dir) == 0 # @test num_backups_in(dir) == 1 post_pkg_notebook = read(path, String) # test that pkg cells got added @test length(post_pkg_notebook) > length(pre_pkg_notebook) + 50 @test has_embedded_pkgfiles(notebook) @test notebook.nbpkg_ctx !== nothing @test notebook.nbpkg_restart_recommended_msg === nothing @test notebook.nbpkg_restart_required_msg === nothing cleanup(, notebook) end @testset "PkgUtils -- reset" begin dir = mktempdir() f = joinpath(dir, "hello.jl") write(f, simple_import_notebook) @test num_backups_in(dir) == 0 Pluto.reset_notebook_environment(f) @test num_backups_in(dir) == 1 @test !has_embedded_pkgfiles(read(f, String)) end @testset "PkgUtils -- update" begin dir = mktempdir() f = joinpath(dir, "hello.jl") write(f, simple_import_notebook) @test !occursin("0.3.1", read(f, String)) @test num_backups_in(dir) == 0 Pluto.update_notebook_environment(f) @test num_backups_in(dir) == 1 @test has_embedded_pkgfiles(read(f, String)) @test !Pluto.only_versions_differ(f, simple_import_path) @test occursin("0.3.1", read(f, String)) Pluto.update_notebook_environment(f) @test_skip num_backups_in(dir) == 1 end @testset "Bad files" begin @testset "$(name)" for name in ["corrupted_manifest", "unregistered_import"] original_path = joinpath(@__DIR__, "$(name).jl") original_contents = read(original_path, String)  = ServerSession() dir = mktempdir() path = joinpath(dir, "hello.jl") write(path, original_contents) @test num_backups_in(dir) == 0 notebook = SessionActions.open(, path; run_async=false) nb_contents() = read(notebook.path, String) should_restart = ( notebook.nbpkg_restart_recommended_msg !== nothing || notebook.nbpkg_restart_required_msg !== nothing ) # if name == "corrupted_manifest" # @test !should_restart # end # this breaks julia for somee reason: # # we don't want to recommend restart right after launch, but it's easier for us # @test_broken !should_restart # end if should_restart Pluto.response_restart_process(Pluto.ClientRequest( session=, notebook=notebook, ); run_async=false) end if name != "unregistered_import" @test noerror(notebook.cells[1]) @test noerror(notebook.cells[2]) @test notebook.cells[2].output.body == "0.2.2" # the Project.toml remained, so we did not lose our compat bound. @test has_embedded_pkgfiles(notebook) end @test !Pluto.only_versions_differ(notebook.path, original_path) @test notebook.nbpkg_ctx !== nothing @test notebook.nbpkg_restart_recommended_msg === nothing @test notebook.nbpkg_restart_required_msg === nothing setcode!(notebook.cells[2], "1 + 1") update_save_run!(, notebook, notebook.cells[2]) @test notebook.cells[2].output.body == "2" setcode!(notebook.cells[2], """ begin import PlutoPkgTestD PlutoPkgTestD.MY_VERSION |> Text end """) update_save_run!(, notebook, notebook.cells[2]) @test notebook.cells[2].output.body == "0.1.0" @test has_embedded_pkgfiles(notebook) cleanup(, notebook) end end @testset "Race conditions" begin  = ServerSession() lag = 0.2 .options.server.simulated_pkg_lag = lag # See https://github.com/JuliaPluto/PlutoPkgTestRegistry notebook = Notebook([ Cell("import PlutoPkgTestA"), # cell 1 Cell("PlutoPkgTestA.MY_VERSION |> Text"), Cell("import PlutoPkgTestB"), # cell 3 Cell("PlutoPkgTestB.MY_VERSION |> Text"), Cell("import PlutoPkgTestC"), # cell 5 Cell("PlutoPkgTestC.MY_VERSION |> Text"), Cell("import PlutoPkgTestD"), # cell 7 Cell("PlutoPkgTestD.MY_VERSION |> Text"), Cell("import PlutoPkgTestE"), # cell 9 Cell("PlutoPkgTestE.MY_VERSION |> Text"), ]) @test !notebook.nbpkg_ctx_instantiated running_tasks = Task[] remember(t) = push!(running_tasks, t) update_save_run!(, notebook, notebook.cells[[7, 8]]; run_async=false) # import D (not async) update_save_run!(, notebook, notebook.cells[[1, 2]]; run_async=true) |> remember # import A for _ in 1:5 sleep(lag / 2) setcode!(notebook.cells[9], "import PlutoPkgTestE") update_save_run!(, notebook, notebook.cells[[9]]; run_async=true) |> remember # import E sleep(lag / 2) setcode!(notebook.cells[9], "") update_save_run!(, notebook, notebook.cells[[9]]; run_async=true) |> remember # don't import E end while !all(istaskdone, running_tasks) @test all(noerror, notebook.cells) sleep(lag / 3) end @test all(istaskdone, running_tasks) wait.(running_tasks) empty!(running_tasks) cleanup(, notebook) end @testset "PlutoRunner Syntax Error" begin  = ServerSession() notebook = Notebook([ Cell("1 +"), Cell("PlutoRunner.throw_syntax_error"), Cell("PlutoRunner.throw_syntax_error(1)"), ]) update_run!(, notebook, notebook.cells) @test notebook.cells[1].errored @test noerror(notebook.cells[2]) @test notebook.cells[3].errored @test Pluto.is_just_text(notebook.topology, notebook.cells[1]) @test !Pluto.is_just_text(notebook.topology, notebook.cells[2]) # Not a syntax error form @test Pluto.is_just_text(notebook.topology, notebook.cells[3]) cleanup(, notebook) end @testset "Precompilation" begin compilation_dir = joinpath(DEPOT_PATH[1], "compiled", "v$(VERSION.major).$(VERSION.minor)") @assert isdir(compilation_dir) compilation_dir_testA = joinpath(compilation_dir, "PlutoPkgTestA") precomp_entries() = isdir(compilation_dir_testA) ? readdir(compilation_dir_testA) : String[] @testset "Match compiler options: $(match)" for match in [true, false] let # clear compilation cache Sys.iswindows() && sleep(3) # workaround for https://github.com/JuliaLang/julia/issues/34700 isdir(compilation_dir_testA) && rm(compilation_dir_testA; force=true, recursive=true) end before_sync = precomp_entries() @test before_sync == []  = ServerSession() # make compiler settings of the worker match or not match the server settings let # you can find out which settings are relevant for cache validation by looking at the field names of `Base.CacheFlags`. flip = !match @test Base.JLOptions().use_pkgimages in 0:2 @test Base.JLOptions().check_bounds in 0:1 @test Base.JLOptions().opt_level in 0:3 @test Base.JLOptions().can_inline in 0:1 arg_value_from_int(x::Integer) = ("no", "yes", "existing")[x + 1] # this is how the command line argument values map to the JLOptions Integer values dontmatch(current_setting::Integer) = current_setting == 0 ? arg_value_from_int(1) : arg_value_from_int(0) .options.compiler.pkgimages = (match ? arg_value_from_int : dontmatch)(Base.JLOptions().use_pkgimages) .options.compiler.check_bounds = (flip  Base.JLOptions().check_bounds == 1) ? "yes" : "no" .options.compiler.optimize = match ? Base.JLOptions().opt_level : 3 - Base.JLOptions().opt_level if VERSION < v"1.12.0-aaa" # https://github.com/JuliaLang/julia/issues/58229 .options.compiler.inline = (flip  Base.JLOptions().can_inline == 1) ? "yes" : "no" end # cant set the debug level but whatevs end notebook = Notebook([ # An import for Pluto to recognize, but don't actually run it. When you run an import, Julia will precompile the package if necessary, which would skew our results. Cell("false && import PlutoPkgTestA"), ]) @test !notebook.nbpkg_ctx_instantiated update_save_run!(, notebook, notebook.cells) @test notebook.nbpkg_ctx_instantiated after_sync = precomp_entries() # syncing should have called Pkg.precompile(), which should have generated new precompile caches. # If `match == false`, then this is the second run, and the precompile caches should be different. # These new caches use the same filename (weird...), EXCEPT when the pkgimages flag changed, then you get a new filename. @test before_sync != after_sync @test length(before_sync) < length(after_sync) # Now actually run the import. setcode!(notebook.cells[1], """begin ENV["JULIA_DEBUG"] = "loading" PlutoRunner.Logging.shouldlog(logger::PlutoRunner.PlutoCellLogger, level, _module, _...) = true # https://github.com/fonsp/Pluto.jl/issues/2487 import PlutoPkgTestA end""") update_save_run!(, notebook, notebook.cells[1]) @test noerror(notebook.cells[1]) after_run = precomp_entries() full_logs = join([log["msg"][1] for log in notebook.cells[1].logs], "\n") is_broken_idk_why = Sys.iswindows() && v"1.11.0-aaa" <= VERSION < v"1.12.0-aaa" && !match is_broken_idk_why |= VERSION >= v"1.12.0-aaa" && !match if !is_broken_idk_why # There should be a log message about loading the cache. @test occursin(r"Loading.*cache"i, full_logs) # There should NOT be a log message about rejecting the cache. @test !occursin(r"reject.*cache"i, full_logs) end # Running the import should not have triggered additional precompilation, everything should have been precompiled during Pkg.precompile() (in sync_nbpkg). @test after_sync == after_run cleanup(, notebook) end end @testset "Inherit load path" begin notebook = Notebook([ Cell("import Pkg; Pkg.activate()"), Cell("LOAD_PATH[begin]"), Cell("LOAD_PATH[end]"), ])  = ServerSession() update_run!(, notebook, notebook.cells) @test isnothing(notebook.nbpkg_ctx) @test notebook.cells[2].output.body == sprint(Base.show, LOAD_PATH[begin]) @test notebook.cells[3].output.body == sprint(Base.show, LOAD_PATH[end]) cleanup(, notebook) end Pkg.Registry.rm(pluto_test_registry_spec) # Pkg.Registry.add("General") end # reg_path = mktempdir() # repo = LibGit2.clone("https://github.com/JuliaRegistries/General.git", reg_path) # LibGit2.checkout!(repo, "aef26d37e1d0e8f8387c011ccb7c4a38398a18f6") import Pluto.PkgCompat import Pluto import Pluto: update_save_run!, update_run!, WorkspaceManager, ClientSession, ServerSession, Notebook, Cell, project_relative_path, SessionActions, load_notebook using Test import Pkg @testset "PkgCompat" begin PkgCompat.refresh_registry_cache() @testset "Available versions" begin vs = PkgCompat.package_versions("HTTP") @test v"0.9.0"  vs @test v"0.9.1"  vs @test "stdlib"  vs @test PkgCompat.package_exists("HTTP") vs = PkgCompat.package_versions("Dates") @test vs == ["stdlib"] @test PkgCompat.package_exists("Dates") @test PkgCompat.is_stdlib("Dates") @test PkgCompat.is_stdlib("Markdown") @test PkgCompat.is_stdlib("Sockets") @test PkgCompat.is_stdlib("MbedTLS_jll") @test PkgCompat.is_stdlib("Test") @test PkgCompat.is_stdlib("Pkg") @test PkgCompat.is_stdlib("Random") @test PkgCompat.is_stdlib("FileWatching") @test PkgCompat.is_stdlib("Distributed") # upgradable stdlibs: @test PkgCompat.is_stdlib("Statistics") @test PkgCompat.is_stdlib("DelimitedFiles") @test !PkgCompat.is_stdlib("PlutoUI") vs = PkgCompat.package_versions("Dateskjashdfkjahsdfkjh") @test isempty(vs) @test !PkgCompat.package_exists("Dateskjashdfkjahsdfkjh") end @testset "URL" begin @test PkgCompat.package_url("HTTP") == "https://github.com/JuliaWeb/HTTP.jl.git" @test PkgCompat.package_url("HefefTTP") === nothing @test PkgCompat.package_url("Downloads") == "https://docs.julialang.org/en/v1/stdlib/Downloads/" end @testset "Registry queries" begin Pkg.Registry.add(pluto_test_registry_spec) PkgCompat.refresh_registry_cache() es = PkgCompat._registry_entries("PlutoPkgTestA") @test length(es) == 1 @test occursin("P/PlutoPkgTestA", only(es)) @test occursin("PlutoPkgTestRegistry", only(es)) es = PkgCompat._registry_entries("Pluto") @test length(es) == 1 @test occursin("P/Pluto", only(es)) @test occursin("General", only(es)) es = PkgCompat._registry_entries("HelloWorldC_jll") @test length(es) == 1 @test occursin("H/HelloWorldC_jll", only(es)) @test occursin("General", only(es)) Pkg.Registry.rm(pluto_test_registry_spec) end @testset "Installed versions" begin # we are querying the package environment that is currently active for testing ctx = Pkg.Types.Context() ctx = PkgCompat.create_empty_ctx() Pkg.add(ctx, [Pkg.PackageSpec("HTTP"), Pkg.PackageSpec("UUIDs"), ]) @test PkgCompat.get_manifest_version(ctx, "HTTP") > v"0.8.0" @test PkgCompat.get_manifest_version(ctx, "UUIDs") == "stdlib" end @testset "Completions" begin cs = PkgCompat.registered_package_names() @test "HypertextLiteral"  cs @test "Hyperscript"  cs @test "Dates"  cs end @testset "Compat manipulation" begin old_path = joinpath(@__DIR__, "old_artifacts_import.jl") old_contents = read(old_path, String) dir = mktempdir() path = joinpath(dir, "hello.jl") write(path, old_contents) notebook = load_notebook(path) ptoml_contents() = PkgCompat.read_project_file(notebook) mtoml_contents() = PkgCompat.read_manifest_file(notebook) @test num_backups_in(dir) == 0 @test Pluto.only_versions_or_lineorder_differ(old_path, path) ptoml = Pkg.TOML.parse(ptoml_contents()) @test haskey(ptoml["deps"], "PlutoPkgTestA") @test haskey(ptoml["deps"], "Artifacts") @test haskey(ptoml["compat"], "PlutoPkgTestA") @test haskey(ptoml["compat"], "Artifacts") PkgCompat.clear_stdlib_compat_entries!(notebook.nbpkg_ctx) ptoml = Pkg.TOML.parse(ptoml_contents()) @test haskey(ptoml["deps"], "PlutoPkgTestA") @test haskey(ptoml["deps"], "Artifacts") @test haskey(ptoml["compat"], "PlutoPkgTestA") if PkgCompat.is_stdlib("Artifacts") @test !haskey(ptoml["compat"], "Artifacts") end old_a_compat_entry = ptoml["compat"]["PlutoPkgTestA"] PkgCompat.clear_auto_compat_entries!(notebook.nbpkg_ctx) ptoml = Pkg.TOML.parse(ptoml_contents()) @test haskey(ptoml["deps"], "PlutoPkgTestA") @test haskey(ptoml["deps"], "Artifacts") @test !haskey(ptoml, "compat") compat = get(ptoml, "compat", Dict()) @test !haskey(compat, "PlutoPkgTestA") @test !haskey(compat, "Artifacts") PkgCompat.write_auto_compat_entries!(notebook.nbpkg_ctx) ptoml = Pkg.TOML.parse(ptoml_contents()) @test haskey(ptoml["deps"], "PlutoPkgTestA") @test haskey(ptoml["deps"], "Artifacts") @test haskey(ptoml["compat"], "PlutoPkgTestA") if PkgCompat.is_stdlib("Artifacts") @test !haskey(ptoml["compat"], "Artifacts") end end @testset "Misc" begin PkgCompat.create_empty_ctx() end end using Pluto.WorkspaceManager: WorkspaceManager, poll using Pluto without_pluto_version(s) = replace(s, r"# v.*" => "") @testset "PkgUtils" begin @testset "activate_notebook_environment" begin file = Pluto.PkgUtils.testnb() before = without_pluto_version(read(file, String)) @assert :activate_notebook_environment in names(Pluto) @test !occursin("Artifacts", Pluto.PkgCompat.read_project_file(Pluto.load_notebook_nobackup(file))) ap_before = Base.ACTIVE_PROJECT[] ### Pluto.activate_notebook_environment(file; show_help=false) @test Base.ACTIVE_PROJECT[] != ap_before @test sort(collect(keys(Pkg.project().dependencies))) == ["Dates"] Pkg.add("Artifacts") @test sort(collect(keys(Pkg.project().dependencies))) == ["Artifacts", "Dates"] ### EXIT, activate another env Base.ACTIVE_PROJECT[] = ap_before ### # get embedded project.toml from notebook: # (poll to wait for our previous changes to get picked up) @test poll(60, 1/4) do occursin("Artifacts", Pluto.PkgCompat.read_project_file(Pluto.load_notebook_nobackup(file))) end after = without_pluto_version(read(file, String)) @test before != after @test occursin("Artifacts", after) end @testset "activate_notebook_environment, functional 1" begin file = Pluto.PkgUtils.testnb() before = read(file, String) @assert :activate_notebook_environment in names(Pluto) ### projs = Pluto.activate_notebook_environment(file) do Pkg.project() end after = read(file, String) @test projs !== nothing @test sort(collect(keys(projs.dependencies))) == ["Dates"] @test before == after end @testset "activate_notebook_environment, functional 2" begin file = Pluto.PkgUtils.testnb() before = without_pluto_version(read(file, String)) @assert :activate_notebook_environment in names(Pluto) @test !occursin("Artifacts", Pluto.PkgCompat.read_project_file(Pluto.load_notebook_nobackup(file))) @test !occursin("Artifacts", before) ### projs = Pluto.activate_notebook_environment(file) do Pkg.add("Artifacts") end after = without_pluto_version(read(file, String)) @test occursin("Artifacts", Pluto.PkgCompat.read_project_file(Pluto.load_notebook_nobackup(file))) @test before != after @test occursin("Artifacts", after) end @testset "reset_notebook_environment" begin file = Pluto.PkgUtils.testnb() before = without_pluto_version(read(file, String)) @assert :reset_notebook_environment in names(Pluto) # project.toml fake cell id @test occursin("00001", before) @test occursin("[deps]", before) ### Pluto.reset_notebook_environment(file; backup=false) after = without_pluto_version(read(file, String)) @test before != after # project.toml fake cell id @test !occursin("00001", after) @test !occursin("[deps]", after) end # TODO: too lazy to get a notebook with updatable package so just running the function and checking for errors @testset "update_notebook_environment" begin file = Pluto.PkgUtils.testnb() before = without_pluto_version(read(file, String)) @assert :update_notebook_environment in names(Pluto) ### Pluto.update_notebook_environment(file) # whatever after = without_pluto_version(read(file, String)) @test occursin("[deps]", after) end @testset "will_use_pluto_pkg" begin file = Pluto.PkgUtils.testnb() before = read(file, String) @assert :will_use_pluto_pkg in names(Pluto) ### @test Pluto.will_use_pluto_pkg(file) after = read(file, String) @test before == after file2 = Pluto.PkgUtils.testnb("pkg_cell.jl") @test !Pluto.will_use_pluto_pkg(file2) end end ### A Pluto.jl notebook ### # v0.15.0 using Markdown using InteractiveUtils #  1bb0c024-c7a9-11eb-216f-839261af9684 import PlutoPkgTestA #  12fc51ea-0989-4bb2-a3a1-f1cc3bf2992d PlutoPkgTestA.MY_VERSION |> Text #  00000000-0000-0000-0000-000000000001 PLUTO_PROJECT_TOML_CONTENTS = """ [deps] PlutoPkgTestA = "419c6f8d-b8cd-4309-abdc-cee491252f94" [compat] PlutoPkgTestA = "~0.2.2" """ #  00000000-0000-0000-0000-000000000002 PLUTO_MANIFEST_TOML_CONTENTS = """ # This file is FONSI-generated - editing it directly is AKSLD>FJKLAWEM [afsd[ asdf ((((( [ 123 123lkj 123lkj asdfkj asldkfj [[ArgTools]]241f-c20a-4ad4-852c-f6b1247861c6" [[InteractiveUtil33d8-53b3-aaab-bd5110c3b7a0" """ #  Cell order: # 1bb0c024-c7a9-11eb-216f-839261af9684 # 12fc51ea-0989-4bb2-a3a1-f1cc3bf2992d # 00000000-0000-0000-0000-000000000001 # 00000000-0000-0000-0000-000000000002 ### A Pluto.jl notebook ### # v0.15.0 using Markdown using InteractiveUtils #  c581d17a-c965-11eb-1607-bbeb44933d25 # This file imports a future version of PlutoPkgTestA: 99.99.99, which does not actually exist. # It is generated by modifying the simple_import.jl file by hand. import PlutoPkgTestA #  aef57966-ea36-478f-8724-e71430f10be9 PlutoPkgTestA.MY_VERSION |> Text #  00000000-0000-0000-0000-000000000001 PLUTO_PROJECT_TOML_CONTENTS = """ [deps] PlutoPkgTestA = "419c6f8d-b8cd-4309-abdc-cee491252f94" [compat] PlutoPkgTestA = "~99.99.99" """ #  00000000-0000-0000-0000-000000000002 PLUTO_MANIFEST_TOML_CONTENTS = """ # This file is machine-generated - editing it directly is not advised [[Base64]] uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f" [[Dates]] deps = ["Printf"] uuid = "ade2ca70-3891-5945-98fb-dc099432e06a" [[InteractiveUtils]] deps = ["Markdown"] uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240" [[LibGit2]] deps = ["Printf"] uuid = "76f85450-5226-5b5a-8eaa-529ad045b433" [[Libdl]] uuid = "8f399da3-3557-5675-b5ff-fb832c97cbdb" [[Logging]] uuid = "56ddb016-857b-54e1-b83d-db4d58db5568" [[Markdown]] deps = ["Base64"] uuid = "d6f4376e-aef5-505a-96c1-9c027394607a" [[Pkg]] deps = ["Dates", "LibGit2", "Libdl", "Logging", "Markdown", "Printf", "REPL", "Random", "SHA", "UUIDs"] uuid = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f" [[PlutoPkgTestA]] deps = ["Pkg"] git-tree-sha1 = "6c9aa67135641123c559d59ba88e8cb938999999" uuid = "419c6f8d-b8cd-4309-abdc-cee491252f94" version = "99.99.99" [[Printf]] deps = ["Unicode"] uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7" [[REPL]] deps = ["InteractiveUtils", "Markdown", "Sockets"] uuid = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb" [[Random]] deps = ["Serialization"] uuid = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c" [[SHA]] uuid = "ea8e919c-243c-51af-8825-aaa63cd721ce" [[Serialization]] uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b" [[Sockets]] uuid = "6462fe0b-24de-5631-8697-dd941f90decc" [[UUIDs]] deps = ["Random", "SHA"] uuid = "cf7118a7-6976-5b1a-9a39-7adc72f591a4" [[Unicode]] uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5" """ #  Cell order: # c581d17a-c965-11eb-1607-bbeb44933d25 # aef57966-ea36-478f-8724-e71430f10be9 # 00000000-0000-0000-0000-000000000001 # 00000000-0000-0000-0000-000000000002 ### A Pluto.jl notebook ### # v0.15.0 using Markdown using InteractiveUtils #  c581d17a-c965-11eb-1607-bbeb44933d25 # This file imports an outdated version of PlutoPkgTestA: 0.2.1 (which is stored in the embedded Manifest file) and Artifacts, which is now a standard library (as of Julia 1.6), but it used to be a registered package (https://github.com/JuliaPackaging/Artifacts.jl). This notebook was generated on Julia 1.5, so the Manifest will be very very confusing for Julia 1.6 and up. # It is generated on Julia 1.5 (our oldest supported Julia version (at the time of writing), Manifest.toml is not backwards-compatible): # 1. add our test registry: # pkg> registry add https://github.com/JuliaPluto/PlutoPkgTestRegistry # 2. using Pluto, open the simple_import.jl notebook # 3. add the `import Artifacts` cell import PlutoPkgTestA #  aef57966-ea36-478f-8724-e71430f10be9 PlutoPkgTestA.MY_VERSION |> Text #  f9bdbb35-4326-4786-b308-88b6894923df import Artifacts #  00000000-0000-0000-0000-000000000001 PLUTO_PROJECT_TOML_CONTENTS = """ [deps] Artifacts = "56f22d72-fd6d-98f1-02f0-08ddc0907c33" PlutoPkgTestA = "419c6f8d-b8cd-4309-abdc-cee491252f94" [compat] Artifacts = "~1.3.0" PlutoPkgTestA = "~0.2.2" """ #  00000000-0000-0000-0000-000000000002 PLUTO_MANIFEST_TOML_CONTENTS = """ # This file is machine-generated - editing it directly is not advised [[Artifacts]] deps = ["Pkg"] git-tree-sha1 = "c30985d8821e0cd73870b17b0ed0ce6dc44cb744" uuid = "56f22d72-fd6d-98f1-02f0-08ddc0907c33" version = "1.3.0" [[Base64]] uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f" [[Dates]] deps = ["Printf"] uuid = "ade2ca70-3891-5945-98fb-dc099432e06a" [[InteractiveUtils]] deps = ["Markdown"] uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240" [[LibGit2]] deps = ["Printf"] uuid = "76f85450-5226-5b5a-8eaa-529ad045b433" [[Libdl]] uuid = "8f399da3-3557-5675-b5ff-fb832c97cbdb" [[Logging]] uuid = "56ddb016-857b-54e1-b83d-db4d58db5568" [[Markdown]] deps = ["Base64"] uuid = "d6f4376e-aef5-505a-96c1-9c027394607a" [[Pkg]] deps = ["Dates", "LibGit2", "Libdl", "Logging", "Markdown", "Printf", "REPL", "Random", "SHA", "UUIDs"] uuid = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f" [[PlutoPkgTestA]] deps = ["Pkg"] git-tree-sha1 = "6c9aa67135641123c559d59ba88e8cb93841773a" uuid = "419c6f8d-b8cd-4309-abdc-cee491252f94" version = "0.2.2" [[Printf]] deps = ["Unicode"] uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7" [[REPL]] deps = ["InteractiveUtils", "Markdown", "Sockets"] uuid = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb" [[Random]] deps = ["Serialization"] uuid = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c" [[SHA]] uuid = "ea8e919c-243c-51af-8825-aaa63cd721ce" [[Serialization]] uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b" [[Sockets]] uuid = "6462fe0b-24de-5631-8697-dd941f90decc" [[UUIDs]] deps = ["Random", "SHA"] uuid = "cf7118a7-6976-5b1a-9a39-7adc72f591a4" [[Unicode]] uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5" """ #  Cell order: # c581d17a-c965-11eb-1607-bbeb44933d25 # aef57966-ea36-478f-8724-e71430f10be9 # f9bdbb35-4326-4786-b308-88b6894923df # 00000000-0000-0000-0000-000000000001 # 00000000-0000-0000-0000-000000000002 ### A Pluto.jl notebook ### # v0.14.7 using Markdown using InteractiveUtils #  22364cc8-c792-11eb-3458-75afd80f5a03 using PlutoPkgTestA #  ca0765b8-ce3f-4869-bd65-855905d49a2d using Dates #  5cbe4ac1-1bc5-4ef1-95ce-e09749343088 domath(20) #  Cell order: # 22364cc8-c792-11eb-3458-75afd80f5a03 # ca0765b8-ce3f-4869-bd65-855905d49a2d # 5cbe4ac1-1bc5-4ef1-95ce-e09749343088 ### A Pluto.jl notebook ### # v0.16.1 using Markdown using InteractiveUtils #  c02664e7-2046-4103-8a59-dca4998638df begin import Pkg Pkg.activate(joinpath(@__DIR__)) Pkg.resolve() Pkg.instantiate() # Pkg.status() end #  8a90d8a0-eb33-417c-8ac3-440822ae99f3 LOAD_PATH #  3103370e-488a-4cac-9540-1ef4bec5503b using PlutoPkgTestA #  82d46919-ccb0-4ad2-bb3d-77adfafebc4d PlutoPkgTestA.MY_VERSION |> Text #  c44e23b8-3101-11ec-2112-df6ef8652469 @__DIR__ #  a6cee179-f82c-4ef5-8091-cf0be115ec92 pwd() #  26283153-f597-44b9-8a17-8018ca7ca34c Base.active_project() #  3b96bb61-08f0-4ba8-90d2-2a2be9902c2d Base.current_project() #  8cc87597-f0c5-4902-9f3e-4ed8c6798ee9 Base.current_project(@__DIR__) #  8559b034-f7d2-4eea-a080-557f22ed98d9 Base.current_project(joinpath(@__DIR__, "..")) |> normpath #  Cell order: # c02664e7-2046-4103-8a59-dca4998638df # 8a90d8a0-eb33-417c-8ac3-440822ae99f3 # 82d46919-ccb0-4ad2-bb3d-77adfafebc4d # 3103370e-488a-4cac-9540-1ef4bec5503b # c44e23b8-3101-11ec-2112-df6ef8652469 # a6cee179-f82c-4ef5-8091-cf0be115ec92 # 26283153-f597-44b9-8a17-8018ca7ca34c # 3b96bb61-08f0-4ba8-90d2-2a2be9902c2d # 8cc87597-f0c5-4902-9f3e-4ed8c6798ee9 # 8559b034-f7d2-4eea-a080-557f22ed98d9 ### A Pluto.jl notebook ### # v0.15.0 using Markdown using InteractiveUtils #  c581d17a-c965-11eb-1607-bbeb44933d25 # This file imports an outdated version of PlutoPkgTestA: 0.2.2 (which is stored in the embedded Manifest file). # It is generated on Julia 1.5 (our oldest supported Julia version (at the time of writing), Manifest.toml is not backwards-compatible): # 1. add our test registry: # pkg> registry add https://github.com/JuliaPluto/PlutoPkgTestRegistry # 2. using Pluto, create this notebook # 3. use `Pluto.activate_notebook_environment` to change the version of PlutoPkgTestA to 0.2.2 # 4. open the notebook in Pluto again. Add a second package (PlutoPkgTestD) and then remove it again. This adds the auto-generated compat entry for PlutoPkgTestA. import PlutoPkgTestA #  aef57966-ea36-478f-8724-e71430f10be9 PlutoPkgTestA.MY_VERSION |> Text #  00000000-0000-0000-0000-000000000001 PLUTO_PROJECT_TOML_CONTENTS = """ [deps] PlutoPkgTestA = "419c6f8d-b8cd-4309-abdc-cee491252f94" [compat] PlutoPkgTestA = "~0.2.2" """ #  00000000-0000-0000-0000-000000000002 PLUTO_MANIFEST_TOML_CONTENTS = """ # This file is machine-generated - editing it directly is not advised [[Base64]] uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f" [[Dates]] deps = ["Printf"] uuid = "ade2ca70-3891-5945-98fb-dc099432e06a" [[InteractiveUtils]] deps = ["Markdown"] uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240" [[LibGit2]] deps = ["Printf"] uuid = "76f85450-5226-5b5a-8eaa-529ad045b433" [[Libdl]] uuid = "8f399da3-3557-5675-b5ff-fb832c97cbdb" [[Logging]] uuid = "56ddb016-857b-54e1-b83d-db4d58db5568" [[Markdown]] deps = ["Base64"] uuid = "d6f4376e-aef5-505a-96c1-9c027394607a" [[Pkg]] deps = ["Dates", "LibGit2", "Libdl", "Logging", "Markdown", "Printf", "REPL", "Random", "SHA", "UUIDs"] uuid = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f" [[PlutoPkgTestA]] deps = ["Pkg"] git-tree-sha1 = "6c9aa67135641123c559d59ba88e8cb93841773a" uuid = "419c6f8d-b8cd-4309-abdc-cee491252f94" version = "0.2.2" [[Printf]] deps = ["Unicode"] uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7" [[REPL]] deps = ["InteractiveUtils", "Markdown", "Sockets"] uuid = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb" [[Random]] deps = ["Serialization"] uuid = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c" [[SHA]] uuid = "ea8e919c-243c-51af-8825-aaa63cd721ce" [[Serialization]] uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b" [[Sockets]] uuid = "6462fe0b-24de-5631-8697-dd941f90decc" [[UUIDs]] deps = ["Random", "SHA"] uuid = "cf7118a7-6976-5b1a-9a39-7adc72f591a4" [[Unicode]] uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5" """ #  Cell order: # c581d17a-c965-11eb-1607-bbeb44933d25 # aef57966-ea36-478f-8724-e71430f10be9 # 00000000-0000-0000-0000-000000000001 # 00000000-0000-0000-0000-000000000002 ### A Pluto.jl notebook ### # v0.16.1 using Markdown using InteractiveUtils #  912825ec-1e5f-11ec-13b6-f7222876a7d5 using Dates #  b7a923d4-f637-4f9e-8576-5371e9d72c88 isleapyear(1970) #  00000000-0000-0000-0000-000000000001 PLUTO_PROJECT_TOML_CONTENTS = """ [deps] Dates = "ade2ca70-3891-5945-98fb-dc099432e06a" """ #  00000000-0000-0000-0000-000000000002 PLUTO_MANIFEST_TOML_CONTENTS = """ # This file is machine-generated - editing it directly is not advised [[Dates]] deps = ["Printf"] uuid = "ade2ca70-3891-5945-98fb-dc099432e06a" [[Printf]] deps = ["Unicode"] uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7" [[Unicode]] uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5" """ #  Cell order: # 912825ec-1e5f-11ec-13b6-f7222876a7d5 # b7a923d4-f637-4f9e-8576-5371e9d72c88 # 00000000-0000-0000-0000-000000000001 # 00000000-0000-0000-0000-000000000002 ### A Pluto.jl notebook ### # v0.15.0 using Markdown using InteractiveUtils #  c581d17a-c965-11eb-1607-bbeb44933d25 # This file imports a package that is not registered: PlutoPkgTestZZZZ, and yet it is included in the embedded Manifest. This can happen when creating a notebook in a future version of Julia, or using a custom registry. # This file was created by editing `simple_import.jl` by hand. import PlutoPkgTestZZZZ #  aef57966-ea36-478f-8724-e71430f10be9 PlutoPkgTestZZZZ.MY_VERSION |> Text #  00000000-0000-0000-0000-000000000001 PLUTO_PROJECT_TOML_CONTENTS = """ [deps] PlutoPkgTestZZZZ = "99999999-b8cd-4309-abdc-cee491252f94"[deps] [compat] PlutoPkgTestZZZZ = "~0.2.2" """ #  00000000-0000-0000-0000-000000000002 PLUTO_MANIFEST_TOML_CONTENTS = """ # This file is machine-generated - editing it directly is not advised [[Base64]] uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f" [[Dates]] deps = ["Printf"] uuid = "ade2ca70-3891-5945-98fb-dc099432e06a" [[InteractiveUtils]] deps = ["Markdown"] uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240" [[LibGit2]] deps = ["Printf"] uuid = "76f85450-5226-5b5a-8eaa-529ad045b433" [[Libdl]] uuid = "8f399da3-3557-5675-b5ff-fb832c97cbdb" [[Logging]] uuid = "56ddb016-857b-54e1-b83d-db4d58db5568" [[Markdown]] deps = ["Base64"] uuid = "d6f4376e-aef5-505a-96c1-9c027394607a" [[Pkg]] deps = ["Dates", "LibGit2", "Libdl", "Logging", "Markdown", "Printf", "REPL", "Random", "SHA", "UUIDs"] uuid = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f" [[PlutoPkgTestZZZZ]] deps = ["Pkg"] git-tree-sha1 = "6c9aa67135641123c559d59ba88e8cb93841773a" uuid = "99999999-b8cd-4309-abdc-cee491252f94" version = "0.2.2" [[Printf]] deps = ["Unicode"] uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7" [[REPL]] deps = ["InteractiveUtils", "Markdown", "Sockets"] uuid = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb" [[Random]] deps = ["Serialization"] uuid = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c" [[SHA]] uuid = "ea8e919c-243c-51af-8825-aaa63cd721ce" [[Serialization]] uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b" [[Sockets]] uuid = "6462fe0b-24de-5631-8697-dd941f90decc" [[UUIDs]] deps = ["Random", "SHA"] uuid = "cf7118a7-6976-5b1a-9a39-7adc72f591a4" [[Unicode]] uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5" """ #  Cell order: # c581d17a-c965-11eb-1607-bbeb44933d25 # aef57966-ea36-478f-8724-e71430f10be9 # 00000000-0000-0000-0000-000000000001 # 00000000-0000-0000-0000-000000000002 ### A Pluto.jl notebook ### # v0.19.18 using Markdown using InteractiveUtils #  3717ac9c-821f-11ed-14bc-d3b0f9fd1efe import PlutoPkgTestF #  f6d8e48e-f400-4e27-8a83-f7bf2e72e992 PlutoPkgTestF.MY_VERSION |> Text #  00000000-0000-0000-0000-000000000001 PLUTO_PROJECT_TOML_CONTENTS = """ [deps] PlutoPkgTestF = "7007ab80-a928-4ddc-1332-8dd20f5a112f" """ #  00000000-0000-0000-0000-000000000002 PLUTO_MANIFEST_TOML_CONTENTS = """ # This file is machine-generated - editing it directly is not advised [[ArgTools]] uuid = "0dad84c5-d112-42e6-8d28-ef12dabb789f" [[Artifacts]] uuid = "56f22d72-fd6d-98f1-02f0-08ddc0907c33" [[Base64]] uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f" [[Dates]] deps = ["Printf"] uuid = "ade2ca70-3891-5945-98fb-dc099432e06a" [[Downloads]] deps = ["ArgTools", "LibCURL", "NetworkOptions"] uuid = "f43a241f-c20a-4ad4-852c-f6b1247861c6" [[InteractiveUtils]] deps = ["Markdown"] uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240" [[LibCURL]] deps = ["LibCURL_jll", "MozillaCACerts_jll"] uuid = "b27032c2-a3e7-50c8-80cd-2d36dbcbfd21" [[LibCURL_jll]] deps = ["Artifacts", "LibSSH2_jll", "Libdl", "MbedTLS_jll", "Zlib_jll", "nghttp2_jll"] uuid = "deac9b47-8bc7-5906-a0fe-35ac56dc84c0" [[LibGit2]] deps = ["Base64", "NetworkOptions", "Printf", "SHA"] uuid = "76f85450-5226-5b5a-8eaa-529ad045b433" [[LibSSH2_jll]] deps = ["Artifacts", "Libdl", "MbedTLS_jll"] uuid = "29816b5a-b9ab-546f-933c-edad1886dfa8" [[Libdl]] uuid = "8f399da3-3557-5675-b5ff-fb832c97cbdb" [[Logging]] uuid = "56ddb016-857b-54e1-b83d-db4d58db5568" [[Markdown]] deps = ["Base64"] uuid = "d6f4376e-aef5-505a-96c1-9c027394607a" [[MbedTLS_jll]] deps = ["Artifacts", "Libdl"] uuid = "c8ffd9c3-330d-5841-b78e-0817d7145fa1" [[MozillaCACerts_jll]] uuid = "14a3606d-f60d-562e-9121-12d972cd8159" [[NetworkOptions]] uuid = "ca575930-c2e3-43a9-ace4-1e988b2c1908" [[Pkg]] deps = ["Artifacts", "Dates", "Downloads", "LibGit2", "Libdl", "Logging", "Markdown", "Printf", "REPL", "Random", "SHA", "Serialization", "TOML", "Tar", "UUIDs", "p7zip_jll"] uuid = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f" [[PlutoPkgTestA]] deps = ["Pkg"] git-tree-sha1 = "9615c7f69a1780f70fe32890ac370728afbcb2f6" uuid = "419c6f8d-b8cd-4309-abdc-cee491252f94" version = "0.3.1" [[PlutoPkgTestF]] deps = ["Pkg", "PlutoPkgTestA"] git-tree-sha1 = "f486ebc1188475df900413cbd570ff7fabed738f" repo-rev = "main" repo-url = "https://github.com/JuliaPluto/PlutoPkgTestF.jl" uuid = "7007ab80-a928-4ddc-1332-8dd20f5a112f" version = "1.0.0" [[Printf]] deps = ["Unicode"] uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7" [[REPL]] deps = ["InteractiveUtils", "Markdown", "Sockets", "Unicode"] uuid = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb" [[Random]] deps = ["Serialization"] uuid = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c" [[SHA]] uuid = "ea8e919c-243c-51af-8825-aaa63cd721ce" [[Serialization]] uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b" [[Sockets]] uuid = "6462fe0b-24de-5631-8697-dd941f90decc" [[TOML]] deps = ["Dates"] uuid = "fa267f1f-6049-4f14-aa54-33bafae1ed76" [[Tar]] deps = ["ArgTools", "SHA"] uuid = "a4e569a6-e804-4fa4-b0f3-eef7a1d5b13e" [[UUIDs]] deps = ["Random", "SHA"] uuid = "cf7118a7-6976-5b1a-9a39-7adc72f591a4" [[Unicode]] uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5" [[Zlib_jll]] deps = ["Libdl"] uuid = "83775a58-1f1d-513f-b197-d71354ab007a" [[nghttp2_jll]] deps = ["Artifacts", "Libdl"] uuid = "8e850ede-7688-5339-a07c-302acd2aaf8d" [[p7zip_jll]] deps = ["Artifacts", "Libdl"] uuid = "3f19e933-33d8-53b3-aaab-bd5110c3b7a0" """ #  Cell order: # 3717ac9c-821f-11ed-14bc-d3b0f9fd1efe # f6d8e48e-f400-4e27-8a83-f7bf2e72e992 # 00000000-0000-0000-0000-000000000001 # 00000000-0000-0000-0000-000000000002 !Manifest.toml [deps] PlutoPkgTestA = "419c6f8d-b8cd-4309-abdc-cee491252f94"