About this Book

I've started using mdBook as a way to get better at writing and force myself to have a better understanding of what I'm working on. Hopefully this will help my future self when I need a reference, or potentially someone else.

In terms of mdBook itself, the best resource I've found is the guide book built with mdBook itself found at https://rust-lang.github.io/mdBook.

Quickstart

Create a book with

cargo install mdbook
mkdir book
cd book
mdbook init

The structure is found in src/SUMMARY.md and supports limited markdown syntax:

# This title is ignored
[Non-numered top-level section](path.md)
# Unclickable top-level title
- [Numered chapter title](path2.md)
    - [Nested sub-chapters](path3.md)
[Non-numered top-level section](path.md)

The contents files support normal markdown.

Deploy to GitHub Pages

Add a .github/workflows/gh-pages.yml file with the contents:

name: github pages

on:
  push:
    branches:
      - master

jobs:
  deploy:
    runs-on: ubuntu-18.04
    steps:
      - uses: actions/checkout@v2

      - name: Setup mdBook
        uses: peaceiris/actions-mdbook@v1
        with:
          mdbook-version: 'latest'

      - run: mdbook build

      - name: Deploy
        uses: peaceiris/actions-gh-pages@v3
        with:
          github_token: ${{ secrets.GITHUB_TOKEN }}
          publish_dir: ./book

Go to https://github.com/{username}/{repo}/settings/pages and ensure the source branch is gh-pages.

I had to wait some time for it to deploy.

A small ZSH script to replace autojump

Add this to a ~/.zshrc and ensure you have fzy installed.

When you type jd (jd followed by a space without pressing enter), fzy will pop open and let you change your $PWD to a previously visited directory. For details, just read the comments in the code snippet.

Note: To change the keybinding, modify the $JUMPDIR_KEYBIND variable.

JUMPDIR_KEYBIND='jd '
# Setup some data a data file to store visited directories
mkdir -p "$XDG_DATA_HOME/zshrc"
JD_DATA_DIR="$XDG_DATA_HOME/zshrc/chpwd.txt"
touch $JD_DATA_DIR
local tmp=$(mktemp)
cat $JD_DATA_DIR | while read dir ; do [[ -d $dir ]] && echo $dir ; done > $tmp
cat $tmp > $JD_DATA_DIR
# Track visited directories
chpwd_functions+=(on_chpwd)
function on_chpwd {
    local tmp=$(mktemp)
    { echo $PWD ; cat $JD_DATA_DIR } | sort | uniq 1> $tmp
    cat $tmp > $JD_DATA_DIR
}
# zle widget function
function fzy_jd {
    # check if `jd ` was triggered in the middle of another command
    # e.g. $ aaaaaaajd 
    # If so, we manually input the `jd `
    if [[ ! -z $BUFFER ]]; then
        # Append `jd ` to the prompt
        BUFFER=$BUFFER$JUMPDIR_KEYBIND
        # move the cursor to the end of the line
        zle end-of-line
        return 0
    fi
    # ask the user to select a directory to jump to
    local dir=$({ echo $HOME ; cat $JD_DATA_DIR } | fzy)
    if [[ -z $dir ]]; then
        # no directory was selected, reset the prompt to what it was before
        zle reset-prompt
        return 0
    fi
    # Setup the command to change the directory
    BUFFER="cd $dir"
    # Accepts the cd we setup above
    zle accept-line
    local ret=$?
    # force the prompt to redraw to mimic what would occur with a normal cd
    zle reset-prompt
    return $ret
}
# define the new widget function
zle -N fzy_jd
# bind the widget function to `jd `
bindkey $JUMPDIR_KEYBIND fzy_jd
# a nicety so that executing just jd will mimic the behaviour of just executing
# cd, that is, change the pwd to $HOME
eval "alias $(echo $JUMPDIR_KEYBIND|xargs)=cd"

Keeping Neovim and Kitty Terminal Colorschemes Consistent and Persistent

Whether it's to give my editor a new coat of paint or to make it easier on the eyes for 3am programming, I change its colours all the time. In editors such as Visual Studio Code or Atom, this is a simple matter of choosing a theme from a fuzzy matched drop-down menu. However, in Neovim we have to edit the init.lua and restart the editor (or source the init.lua). On top of this, we then need to update our terminal's colours to match (or at least I like to keep them consistent). This is tedious and nowhere near as clean as VSCode.

This tutorial will create a similar experience to VSCode where you can select a colorscheme using a dropdown fuzzy finder, and this colorscheme will remain consistent in Neovim and Kitty even if you close them and reopen.

Note: We will be using base16 colours but you can swap these out for something else.

We will use a file, $XDG_CONFIG_HOME/.base16_theme, as the single source of truth for our current colours. It will contain a single line with the name of our current base16 colours. For example,

base16-gruvbox-dark-hard

Kitty (Terminal) Setup

Note: These steps only work in kitty.

  1. Download base16-kitty to $HOME
  2. In a shell startup script (e.g. ~/.zshrc), add eval "kitty @ set-colors -c $HOME/base16-kitty/colors/$(cat $XDG_CONFIG_HOME/.base16_theme).conf". This will set the colours for the current terminal window and for newly created terminal windows.

Neovim Setup

  1. Install nvim-base16 plugin. You can use a different collection of colorschemes but this will match the themes in base16-kitty we used above.

  2. Create a function to update the colours for our current terminal window and current instance of Neovim. We'll also need to update our $XDG_CONFIG_HOME/.base16_theme.

-- this is our single source of truth created above
local base16_theme_fname = vim.fn.expand(vim.env.XDG_CONFIG_HOME..'/.base16_theme')
-- this function is the only way we should be setting our colorscheme
local function set_colorscheme(name)
    -- write our colorscheme back to our single source of truth
    vim.fn.writefile({name}, base16_theme_fname)
    -- set Neovim's colorscheme
    vim.cmd('colorscheme '..name)
    -- execute `kitty @ set-colors -c <color>` to change terminal window's
    -- colors and newly created terminal windows colors
    vim.loop.spawn('kitty', {
        args = {
            '@',
            'set-colors',
            '-c',
            string.format(vim.env.HOME..'/base16-kitty/colors/%s.conf', name)
        }
    }, nil)
end
  1. Read $XDG_CONFIG_HOME/.base16_theme to determine our current colours to use for Neovim.
set_colorscheme(vim.fn.readfile(base16_theme_fname)[1])

  1. Install telescope.nvim. Other fuzzy pickers will work, but this code snippet is for this plugin.

  2. Override the colorschemes picker in telescope.nvim to update our terminal colours too

nvim.nnoremap('<leader>c', function()
    -- get our base16 colorschemes
    local colors = vim.fn.getcompletion('base16', 'color')
    -- we're trying to mimic VSCode so we'll use dropdown theme
    local theme = require('telescope.themes').get_dropdown()
    -- create our picker
    require('telescope.pickers').new(theme, {
        prompt = 'Change Base16 Colorscheme',
        finder = require('telescope.finders').new_table {
            results = colors
        },
        sorter = require('telescope.config').values.generic_sorter(theme),
        attach_mappings = function(bufnr)
            -- change the colors upon selection
            telescope_actions.select_default:replace(function()
                set_colorscheme(action_state.get_selected_entry().value)
                telescope_actions.close(bufnr)
            end)
            telescope_action_set.shift_selection:enhance({
                -- change the colors upon scrolling
                post = function()
                    set_colorscheme(action_state.get_selected_entry().value)
                end
            })
            return true
        end
    }):find()
end)

Try it out

Open up Neovim, and hit <leader>c (or whatever mapping you used) and search for a colorscheme. As you scroll through the available colorschemes Neovim should update and Kitty should remain consistant. If you close then open Neovim, the colorscheme will persist with manually changing anything else. Not only that, but you can close and open Kitty and both Kitty and Neovim will maintain the previously selected colorscheme.

Write Your Own Plugin Manager With A Focus On Ergonomics

Note: This is written with Neovim in mind but the same concepts can be applied to Vim

I've been pretty dissatisfied with the state of plugin managers in Neovim. Almost all of them follow a similar flow of editing your init.lua/init.vim, running an install command, and restarting Neovim to have the plugin loaded and ready to use. If we compare that with VSCode, we click a button to install the extension, and it usually just works with the occasional need to restart the editor. It's a lot more streamlined in VSCode and not having to restart the editor or manually edit config files is much smoother. What's worse is this is all functionality that can be mimicked in Neovim with a clever use of :h packages.

At the end of this article we'll have a small plugin manager (~130 lines of Lua) that will let you install the plugin with a simple :PackAdd https://github.com/RRethy/vim-illuminate which will make it instantly available without the need for a restart and you won't need to touch your dotfiles. Removing the plugin will be as easy as deleting a line in the manifest (we'll get to this later).

Note: This code will not be put into it's own repo but you can still add it to your own dotfiles for use (I currently use it). I'm hoping that this style of plugin manager pushes new plugin managers to create a more ergonomic experience.

High-Level Overview

As with most plugin managers nowadays, we will build on top of the :help packages feature. However, unlike other managers, we will not use the start/ directory to store plugins since those are all sourced at startup, instead we will download all plugins into opt/ and execute a :packadd <plugin> to load it dynamically without restarting Neovim. We will also have a separate file that we'll call a pack manifest which is where our plugins will be listed, this manifest will be read at startup and each plugin will be :packadd! so it gets sourced, this has the added benefit that deleting a plugin is just deleting a line in the manifest.

Note: Not all plugins can be dynamically loaded, just most of them.

Picking a Name

TLDR: backpack.lua

Picking a name is always tough, should we aim for descriptiveness (like nvim-treesitter-textobjects) or attempt to confuse the user (like wildfire.vim or vim-hexokinase), that is the question. Personally, I prefer names with personalities which are at least somewhat indicative of their functionality. A tried a true method for this would be to take a word that is directly related (like pack since we're using :h packages) and search for superstrings of that word which also make sense (https://www.thefreedictionary.com/words-containing-pack). We'll use backpack.lua since it sounds good enough.

Entry Point

Let's create a file lua/backpack.lua (in our dotfiles) which will hold all of our code.

The directory for our plugins will be vim.fn.stdpath('data')..'/site/pack/backpack/opt/' which should be recognized by :h 'packpath' but if it isn't then add a vim.opt.packpath:append('~/.local/share/nvim/site') to your init.lua.

As mentioned above we'll have a manifest to specify what plugins we are using, we'll place this at vim.fn.stdpath('config')..'/packmanifest.lua' inside our dotfiles. Our manifest is a Lua file that we will dofile() and looks like this:

use { 'RRethy/nvim-treesitter-textsubjects' }
use {
    'RRethy/vim-hexokinase',
    post_update = { 'make' }
}
use { 'tpope/vim-fugitive' }

use will be a global we define specifically for use in the manifest to declare a plugin so it gets loaded and we track it in case it should be updated later on.

We'll also want to declare 3 commands for later use: :PackAdd to add a plugin, :PackUpdate to update our plugins, :PackEdit to view/edit the manifest.

Putting this into code looks like the following:

local M = {}

local opt = vim.fn.stdpath('data')..'/site/pack/backpack/opt/'
local manifest = vim.fn.stdpath('config')..'/packmanifest.lua'

local GITHUB_USERNAME = '<your github username>'

local function to_git_url(author, plugin)
    if author == GITHUB_USERNAME then
        return string.format('git@github.com:%s/%s.git', author, plugin)
    else
        return string.format('https://github.com:%s/%s.git', author, plugin)
    end
end

function M.setup()
    vim.fn.mkdir(opt, 'p') -- make sure opt exists

    M.plugins = {}
    -- 'use' will be define only for use in the manifest
    _G.use = function(opts)
        local _, _, author, plugin = string.find(opts[1], '^([^ /]+)/([^ /]+)$')
        -- track the plugin so it can be updated later with :PackUpdate
        table.insert(M.plugins, {
            plugin = plugin,
            author = author,
            post_update = opts.post_update,
        })
        -- adds the plugin to the end of :help 'runtimepath'
        -- this will be what makes the plugin sourced
        -- NOTE: :packadd! is not the same as :packadd
        if vim.fn.isdirectory(opt..'/'..plugin) ~= 0 then
            vim.cmd('packadd! '..plugin)
        else
            git_clone(plugin, to_git_url(author, plugin), function()
                vim.cmd('packadd! '..plugin)
            end)
        end
    end
    if vim.fn.filereadable(manifest) ~= 0 then
        dofile(manifest)
    end
    _G.use = nil

    -- these functions will be defined later, you can add a --bar too if you want to chain command usage
    vim.cmd [[ command! -nargs=1 PackAdd lua require('rrethy.backpack').pack_add(<f-args>) ]]
    vim.cmd [[ command! PackUpdate lua require('rrethy.backpack').pack_update() ]]
    vim.cmd [[ command! PackEdit lua require('rrethy.backpack').pack_edit() ]]
end

Then our init.lua will have:

require('backpack').setup()

:PackAdd

Before we add code to install a plugin, we'll need a few helpers to pull Git repos, clone Git repos, and parse a GitHub URL.

We'll start with parsing a GitHub URL. I'm a big fan of just copy pasting the URL (<cmd>l<cmd>c on OSX) after a :PackAdd , it's quite ergonomic compared to looking for installation instructions or copying the exact part of the URL.

local function parse_url(url)
    -- regex capture the username and plugin name from the url
    local username, plugin = string.match(url, '^https://github.com/([^/]+)/([^/]+)$')
    if not username or not plugin then
        -- failed to parse, we can spit an error out here
        return
    end

    local git_url
    if username == GITHUB_USERNAME then
        -- a nicety for plugins that you wrote, prefer ssh over https
        git_url = string.format('git@github.com:%s/%s.git', username, plugin)
    else
        git_url = string.format('https://github.com/%s/%s.git', username, plugin)
    end

    return git_url, username, plugin
end

Now we'll want functions to clone and pull Git repos which will be used by :PackAdd and :PackUpdate

local function git_pull(name, on_success)
    local dir = opt..name
    -- get the branch name, there might be a better way to do this
    local branch = vim.fn.system("git -C "..dir.." branch --show-current | tr -d '\n'")
    -- use Luv to execute an async `git pull` with a shallow fetch
    vim.loop.spawn('git', {
        args = { 'pull', 'origin', branch, '--update-shallow', '--ff-only', '--progress', '--rebase=false' },
        cwd = dir,
    }, vim.schedule_wrap(function(code)
            if code == 0 then
                on_success(name)
            else
                echoerr(name..' pulled unsuccessfully')
            end
        end))
end

local function git_clone(name, git_url, on_success)
    -- use Luv to execute an async `git clone` with a shallow clone
    vim.loop.spawn('git', {
        args = { 'clone', '--depth=1', git_url },
        cwd = opt,
    }, vim.schedule_wrap(function(code)
            if code == 0 then
                on_success(name)
            else
                echoerr(name..' cloned unsuccessfully')
            end
        end))
end

Now we can write our actual :PackAdd functionality:

function M.pack_add(url)
    local git_url, author, plugin = parse_url(url)
    if not git_url then
        -- failed to parse url
        return
    end

    -- track the plugin in case of a :PackUpdate later
    table.insert(M.plugins, {
        plugin = plugin,
        author = author,
    })
    local on_success = function()
        -- if successful, try loading the plugin dynamically without restarting Neovim
        vim.cmd('packadd '..plugin)
    end
    if vim.fn.isdirectory(opt..plugin) ~= 0 then
        git_pull(plugin, on_success)
    else
        git_clone(plugin, git_url, on_success)
    end

    -- automatically add the plugin data to our manifest
    vim.fn.system(string.format('echo "use { \'%s/%s\' }" >> %s', author, plugin, manifest))
end

:PackUpdate

Since we were tracking our plugins in M.plugins, we can now just clone/pull each of them.

function M.pack_update()
    for _, data in ipairs(M.plugins) do
        local on_success = function(plugin)
            vim.cmd('packadd '..plugin)
        end
        if vim.fn.isdirectory(opt..data.plugin) ~= 0 then
            git_pull(data.plugin, on_success)
        else
            git_clone(data.plugin, git_clone, on_success)
        end
    end
end

:PackEdit

Since the manifest is just a Lua file, we can just open it up in a new tab. Closing the manifest is as simple as :bw. While this may seem trivial, IMO it's enough.

function M.pack_edit()
    vim.cmd('tabnew')
    vim.cmd('edit '..manifest)
end

Post-Update Hooks

We were also tracking post-update hooks which is the only additional feature we'll be adding (see below for other feature that were intentionally omitted). This can take the form of a lua function callback or a table representing shell commands to run in the root of the plugin.

We'll add this in M.pack_update after we dynamically reload the plugin in the on_success callback.

if data.post_update then
    -- plugin root directory
    local dir = opt..'/'..plugin
    if type(data.post_update) == 'function' then
        -- execute the function and pass it the plugin dir
        data.post_update(dir)
    elseif type(data.post_update) == 'table' then
        -- use Luv to run the shell command in the plugin dir
        vim.loop.spawn(data.post_update[1], { args = data.post_update.args, cwd = dir },
            vim.schedule_wrap(function(code)
                if code ~= 0 then
                    vim.api.nvim_err_writeln(string.format('Failed to run %s', vim.inspect(data.post_update)))
                end
            end))
    end
end

What We Left Out (And Why)

  1. Lazy loading on filetype
    • This frustrates me to no end, this feature is redundant!!! I've seen far too many Plug 'vim-ruby/vim-ruby', { 'for': 'ruby' }, just look at the source code for the plugin, it already lazily loads for ruby or eruby filetypes. In fact, most plugins lazy load most of their code until it's actually used. The few plugins which don't are typically quite old or if the plugin author doesn't know about ftplugin/ or autoload/ (probably should look for another plugin in that case). On top of this, I don't think people know that this doesn't restrict the plugin to that filetype, once it gets loaded, it's there for all filetypes.
  2. Lazy loading on command
    • Same reason as above, most plugins are already lazy loaded. Before you lazy load on :Foo, take a look at it's definition to see if it's just calling out to an autoloaded function which hasn't been sourced, if so, then it's already lazy loaded. If a plugin is doing massive amounts of work at startup then it might be time to look for a better written plugin.
  3. Lua Rocks support
    • This would be nice to have but I hope this becomes part of Neovim rather than forcing plugin managers to add support for it.
  4. Rigorous Error Handling
    • I trimmed it off to reduce the complexity and size, simply writing errors to a log file is an easy way to add it back.
local echoerr = vim.api.nvim_err_writeln
local GITHUB_USERNAME = '<your username here>'

local M = {}

local opt = vim.fn.stdpath('data')..'/site/pack/backpack/opt/'
local manifest = vim.fn.stdpath('config')..'/packmanifest.lua'

local function to_git_url(author, plugin)
    if author == GITHUB_USERNAME then
        return string.format('git@github.com:%s/%s.git', author, plugin)
    else
        return string.format('https://github.com:%s/%s.git', author, plugin)
    end
end

local function parse_url(url)
    local username, plugin = string.match(url, '^https://github.com/([^/]+)/([^/]+)$')
    if not username or not plugin then
        return
    end

    local git_url
    if username == GITHUB_USERNAME then
        git_url = string.format('git@github.com:%s/%s.git', username, plugin)
    else
        git_url = string.format('https://github.com/%s/%s.git', username, plugin)
    end

    return git_url, username, plugin
end

local function git_pull(name, on_success)
    local dir = opt..name
    local branch = vim.fn.system("git -C "..dir.." branch --show-current | tr -d '\n'")
    vim.loop.spawn('git', {
        args = { 'pull', 'origin', branch, '--update-shallow', '--ff-only', '--progress', '--rebase=false' },
        cwd = dir,
    }, vim.schedule_wrap(function(code)
            if code == 0 then
                on_success(name)
            else
                echoerr(name..' pulled unsuccessfully')
            end
        end))
end

local function git_clone(name, git_url, on_success)
    vim.loop.spawn('git', {
        args = { 'clone', '--depth=1', git_url },
        cwd = opt,
    }, vim.schedule_wrap(function(code)
            if code == 0 then
                on_success(name)
            else
                echoerr(name..' cloned unsuccessfully')
            end
        end))
end

function M.setup()
    vim.fn.mkdir(opt, 'p')

    M.plugins = {}
    _G.use = function(opts)
        local _, _, author, plugin = string.find(opts[1], '^([^ /]+)/([^ /]+)$')
        table.insert(M.plugins, {
            plugin = plugin,
            author = author,
            post_update = opts.post_update,
        })
        if vim.fn.isdirectory(opt..'/'..plugin) ~= 0 then
            vim.cmd('packadd! '..plugin)
        else
            git_clone(plugin, to_git_url(author, plugin), function()
                vim.cmd('packadd! '..plugin)
            end)
        end
    end
    if vim.fn.filereadable(manifest) ~= 0 then
        dofile(manifest)
    end
    _G.use = nil

    vim.cmd [[ command! -nargs=1 PackAdd lua require('rrethy.backpack').pack_add(<f-args>) ]]
    vim.cmd [[ command! PackUpdate lua require('rrethy.backpack').pack_update() ]]
    vim.cmd [[ command! PackEdit lua require('rrethy.backpack').pack_edit() ]]
end

function M.pack_add(url)
    local git_url, author, plugin = parse_url(url)
    if not git_url then
        return
    end

    table.insert(M.plugins, {
        plugin = plugin,
        author = author,
    })
    local on_success = function()
        vim.cmd('packadd '..plugin)
        vim.cmd('helptags ALL')
    end
    if vim.fn.isdirectory(opt..plugin) ~= 0 then
        git_pull(plugin, on_success)
    else
        git_clone(plugin, git_url, on_success)
    end

    vim.fn.system(string.format('echo "use { \'%s/%s\' }" >> %s', author, plugin, manifest))
end

function M.pack_update()
    for _, data in ipairs(M.plugins) do
        local on_success = function(plugin)
            vim.cmd('packadd '..plugin)
            if data.post_update then
                local dir = opt..'/'..plugin
                if type(data.post_update) == 'function' then
                    data.post_update(dir)
                elseif type(data.post_update) == 'table' then
                    vim.loop.spawn(data.post_update[1], { args = data.post_update.args, cwd = dir },
                        vim.schedule_wrap(function(code)
                            if code ~= 0 then
                                vim.api.nvim_err_writeln(string.format('Failed to run %s', vim.inspect(data.post_update)))
                            end
                        end))
                end
            end
        end
        if vim.fn.isdirectory(opt..data.plugin) ~= 0 then
            git_pull(data.plugin, on_success)
        else
            git_clone(data.plugin, git_clone, on_success)
        end
    end
end

function M.pack_edit()
    vim.cmd('tabnew')
    vim.cmd('edit '..manifest)
end

return M