Modern AI agents face a fundamental limitation: they cannot interact with terminal applications. Consider the tools developers use daily: debuggers like gdb that step through breakpoints interactively, SSH sessions that require a TUI interface, or git’s interactive rebase that launches an editor. These applications render their interfaces to terminal screens that agents cannot perceive or control.
To solve this problem, I built a Nushell plugin that exposes terminal applications as typed commands. The plugin spawns processes in pseudoterminals, reads their visual output as structured data, and accepts input using vim key notation.
let t = tui spawn vimThis command spawns a vim process and returns a record containing the session metadata:
╭────────────┬──────────────────────────────────────╮
│ id │ 11600a04-6a47-466b-b54b-ca31c7e4432e │
│ command │ vim │
│ args │ [list 0 items] │
│ rows │ 24 │
│ cols │ 80 │
│ spawned_at │ now │
╰────────────┴──────────────────────────────────────╯To read what the terminal displays, pipe the session to tui buffer:
$t | tui bufferThe command returns the vim welcome screen as a structured table:
╭────┬──────────────────────────────────────────────────────────────────────────────────╮
│ 0 │ │
│ 1 │ ~ │
│ 2 │ ~ │
│ 3 │ ~ │
│ 4 │ ~ │
│ 5 │ ~ │
│ 6 │ ~ VIM - Vi IMproved │
│ 7 │ ~ │
│ 8 │ ~ version 9.1.1833 │
│ 9 │ ~ by Bram Moolenaar et al. │
│ 10 │ ~ Vim is open source and freely distributable │
│ 11 │ ~ │
│ 12 │ ~ Become a registered Vim user! │
│ 13 │ ~ type :help register<Enter> for information │
│ 14 │ ~ │
│ 15 │ ~ type :q<Enter> to exit │
│ 16 │ ~ type :help<Enter> or <F1> for on-line help │
│ 17 │ ~ type :help version9<Enter> for version info │
│ 18 │ ~ │
│ 19 │ ~ │
│ 20 │ ~ │
│ 21 │ ~ │
│ 22 │ ~ │
│ 23 │ 0,0-1 All │
╰────┴──────────────────────────────────────────────────────────────────────────────────╯You can send input to the terminal and then read its updated state:
$t | tui send ':help<Enter>'
$t | tui buffer | find "VIM - main help file"The plugin enables several powerful workflows that were previously impossible for AI agents.
Test terminal user interfaces programmatically:
let editor = tui spawn vim
$editor | tui send ':help<Enter>'
assert ($editor | tui buffer | any { $in like "VIM - main help file" })Control interactive debuggers programmatically:
let debugger = tui spawn gdb program
$debugger | tui send 'break main<Enter>run<Enter>step<Enter>'
$debugger | tui buffer | find "Breakpoint"Execute commands on remote servers through SSH:
let session = tui spawn ssh user@server
$session | tui send 'tail -f /var/log/app.log<Enter>'
$session | tui bufferAutomate git interactive operations:
let rebase = tui spawn git rebase -i HEAD~3
$rebase | tui send 'jpick<Escape>:wq<Enter>'This plugin is built as an open-source Nushell extension for Superglide. You can find the source code on GitHub or join the Discord community.