Syncing Dotfiles
May 9, 2022
For a while I kept some configuration files in Dropbox so I could get to them outside my home computer. I wrote a simple bash script that would move them from their normal locations to a directory in my local Dropbox, and then set up a cronjob to run that script every day. That was ok, but they weren't easily accessible publicly. Or at least not in the way many people share their dotfiles, which is to just have a repo for them.
So I decided to move them from Dropbox to GitHub Codeberg, which presented a small challenge — how to do the commit and push once I collected all the files into a git repository? Here's the simplified bit of bash for that:
git_status=
if ; then
&& &&
fi
If the stdout of running git status doesn't contain "nothing to commit", then it adds all files in the repo, commits with the message "Update", and pushes it. That's not a very meaningful commit message — especially not as the only message in the history after the initial set up — but I'm not particularly concerned with that and more with having the files always up-to-date and accessible.
Another small challenge was with cron. I didn't want to run the script repeatedly all day, but if I just ran it once a day there was a chance my computer wouldn't be on at the time and so the cronjob wouldn't run. Anacron to the rescue! Anacron will run jobs on a regular basis like cron, except that it is aware of the last time jobs ran and will run them again if they haven't run within the specified interval. Anacron isn't installed on Linux distos by default (or at least not Debian and its derivatives), but it's a simple sudo apt install anacron to install it. By default, anacron's configuration file is location at /etc/anacrontab and it tracks jobs run at /var/spool/anacron. I wanted these to be in my user space, so I created those directories/files under ~/.anacron. Here is the part of the config file (~/.anacron/etc/anacrontab) related to this project:
There are two other pieces to this. The first is including this in my ~.profile file, so that anacron runs on startup:
And the second is a regular cronjob that will run anacron every hour (which causes anacron to check if any jobs need to be run, and run them if so):
0 * * * * anacron -t "$HOME/.anacron/etc/anacrontab" -S "$HOME/.anacron/var/spool/anacron"
That's pretty much it. Here's the link to the repo, which includes the full manage_dotfiles bash script.
Braces
October 2, 2021
daybook, rust, bash | permalink
Rust and Bash have a very similar construct for iterating over ranges: x..y. Bash uses it like for i in {1..9} while Rust doesn't use the braces (though seems like parentheses may be used - but not required).
I spent some time reworking my bash note on parameter expansion (actually calling the value of a parameter by using the $ symbol), and think I have further strengthened my understanding of it, using doubles quotes in conjunction with it, and using braces in conjunction with it.
One important reason to use braces around a parameter's name (but after the $, like this: "${somevar}"), is that you can make a default value, either for immediate expansion or to assign the default value to the parameter. Immediate expansion (won't be used again if you reuse the parameter): ${somevar:-this}". Assignment (assigned here, will be used as value of the parameter in subsequent uses): "${somevar:=this}".
Various items
September 14, 2021
daybook, bash, database, python, ansible | permalink
Bash: I think I already knew this, or at least had a general feeling that it was true, but now I read explicitly (in Miell's Learn Bash the Hard Way) that wildcards are not the same as regular expressions.
Web apps/postgres: connection pools could have a significant effect on performance when there's a fair amount of traffic.
Python: you can include on requirements file inside another. Useful for a base/production file and then a separate dev reqs file.
Ansible: set_fact allows you to create variables at runtime (most variables are created at beginning of play).
Various items
July 12, 2021
daybook, bash, python, ansible | permalink
Bash: easy way to test out conditions is to do it right on the command line, and then echo out the result of the last command, e.g.:
That will result in "1" being printed to the terminal, since == does string evaluation.
Python: pypandoc (wrapper around program pandoc) is super easy to use to convert strings and files from one format to another.
Also, not something I learned but a good practice I've developed: anytime I want to install some new program, I do it via the Ansible playbook I set up for my computer. This way, I get more experience with ansible, and I can reproduce my computer if it dies unexpectedly or I get a new one or even if I just switch distros.
A Zettelkasten with Vim and Bash
February 7, 2021
coding, docs, vim, bash | permalink
A few years ago I decided to take coding from an on-and-off hobby to a professional endeavor and I've been keeping notes on the various things I learn since that time. This started off with notes on Python, I think in a libreoffice doc, in the form of dated entries. That soon became difficult to find information in, and so I moved to a markdown file, structured by topics (and subtopics, and then subtopics of subtopics, etc.). Meanwhile, I also started similar files on other things - git, contributing to open source, flask, javascript, css, etc. I was also bookmarking blog posts, talks, and other useful items, along with sometimes extensive notes on them. If I took an online course, I would also create a file for taking notes on the content. Sometimes I would piece it back into the topic files, but generally it was kind of a pain and so I didn't. Between the different files and the bookmarks, it was sometimes difficult to find something I swore I had taken notes on. And if I couldn't find it, back to Google or Stack Overflow.
More recently, I switched from using VSCode to Vim as my editor of choice, in part because of frequently needing to ssh into remote systems and write or modify some code on them. I quickly fell in love with its editing style and haven't looked back. I generally prefer to use as much vanilla Vim as possible and have about ten plugins installed, but I still tweak things now and again and I'm interested in how others have their Vim set up and how they use it. It was while following that interest that I stumbled across the concept of the "Zettelkasten" (German for "slip-box") in a blog post about note-taking in Vim.
I've now being using the zettelkasten (zk from here on out) system for a few months and can say that it's made what was previously a mess of notes much more useful, discoverable, and enjoyable to maintain. I started by taking better, more "atomic" notes, and then as I touch upon different subjects, I pull in other parts of my previous spaghetti-code-like notes and link to them or from them as necessary. Overall, I feel like the zk has made it much easier to both explore subjects and remind myself how I did something. I also use it for some non-coding things, and having it all in one place makes my professional and intellectual life more structured and organized. But I'm not here to write another blog post evangelizing the zk system or what to name different types of notes or how to connect them. I think there's probably been enough of that done, and if there were one resource I'd recommend, it would be Sönke Ahrens's How to Take Smart Notes.
What I am here to write about is my particular zk structure and the minimal tooling for adding to, managing, and exploring my zk. As the title of this post suggests, it basically comes down to Vim and Bash, through a handful of functions, commands, and shortcuts.
Structure
Everything is contained within plaintext files, in the following layout:
zk/
index.md
doc/
notes/
refs/
The doc/ folder is a holdover from my old system of long, topic documents. There's still about ten files in it, but I'm slowly working my way through them and breaking them up into smaller pieces. The refs/ contains references. These just have the author, title, date, and, if the reference is available online, a link to it. The filename is in the format author-yyyy.md. The actual content of the reference is stored in Zotero (in case the page ever goes down) or Calibre (if an ebook). The notes/ folder contains all the actual notes.
The index.md file is the main entry point of the zk. In addition to containing links to other notes, the index also includes a list of tags I've used, some notes on how to navigate about, and brief descriptions of the shortcuts and functions I've created.
I have a shortcut in my .bashrc file to the zk directory so that I can cd into it quickly from anywhere, with just cd $zk:
zk="/Dropbox/zk/"
As you can see, I have the zk in my Dropbox folder, so it's always synced and backed up. I also back it up to an external hard drive, under a timestamped directory, every three months.
Saving and Linking Notes
I'm often already in Vim when I want to create a new note, so I use :enew to start a new, unnamed buffer. If I'm not already in Vim, it's a quick Ctrl+Alt+t to open a terminal and then type vim into the prompt. After typing up the note, I use the :Zk command I created to save the file in the notes/ folder, with a timestamp for the filename. (And no title because I may actually tweak the title - the first line of the file - over time.) Here's the function and command for that, which I have in my .vimrc:
function! SaveWithTS()
if !expand('%:t')
let l:filename = escape( strftime("%Y-%m-%d-%H%M"), ' ' )
execute "write " . "~/Dropbox/zk/notes/" . l:filename . ".md"
else
echo "File already saved and named."
endif
endfunction
command! Zk call SaveWithTS()
Note that if the file has already been saved, running the command will just return a comment stating that.
I'm new to Vimscript so if that's not very good code, please let me know.
After the new note is saved, it needs to be linked to some other note. The following Vim shortcut helps with that. It will get the file's relative path, add double brackets around it, yank the first line of the file (its title), and then put that altogether into the unnamed register.
noremap <Leader>l :let @y = "[[" . expand("%") . "]] " <bar> :1,1y z <bar> let @" = @y . @z<CR>
So, for me, typing \l in normal mode will do this, as I'm using the default leader. Then it's just a matter of using p to put it where I want, and it will look like this, for instance:
[[notes/2021-01-20-0351.md]] Wagtail
To make these links further stand out, I created the file ~/.vim/after/syntax/markdown.vim and added the following:
syntax match markdownRef "\v\[\[[-a-z0-9\.\/\:]+\]\]"
highlight link markdownRef Type
In the colorscheme I use (solarized8), the brackets and the path between them appear in a mustard yellow color, setting it off nicely from the rest of the text.
To follow this link in Vim, just use the standard gf shortcut when the cursor is on it. The relative path works, because I always start my zk from the zk/ directory. Someday I may want to move the zk out of my Dropbox folder, and if I were to use full paths, all the links would then be broken.
Exploring the Zk
In addition to the index, where high-level notes can be easily followed through to all of the connected notes, I also have a couple of other ways to access notes. These are aided by the use of tags, though tags aren't strictly necessary. I make sure that every note (except reference notes) have at least one tag with them. I put tags on the second line of each file, just below the title, and preface them with an "@" symbol. So, for instance, I have a @django tag and a @bash tag and about 20 others so far (and even an @orphan tag for notes that haven't been connected to anything else yet). I'm trying to limit the amount of tags and keep them relatively broad, so it doesn't become too much of a mess and so I don't have to spend too much time thinking about how to tag something.
The tags are highlighted, in a dark lime green color, via the following in the markdown.vim file mentioned above:
syntax match markdownTag "\v\@[a-z0-9]+"
highlight link markdownTag Statement
I've also created two bash commands that will allow easy searching of the zk from a terminal, whether that be by a tag name or any other text. They are both in my ~/bin folder, and rely on the $zk variable in my .bashrc. Here is the first, named zkgrep:
#!/bin/bash
||
|
Calling this command followed by a pattern I'm looking for (e.g. zkgrep @bash) will use grep to search through all files in the notes/ and doc/ folders for that (case-insensitive) pattern and pipe it to less to display them. It will colorize the searched-for pattern (--color=always on the grep side and -R on the less side) in the output, include one line above the line where the pattern was found (-B 1) and two lines below it (-A 2), and precede each line returned with the filename and line number (-n). The -M +Gg options provide a more verbose prompt in less: current lines displayed (-M), plus total lines and % of total (+Gg), in order to provide an idea of how long the results are. The reason for getting the prior line and the two lines after the line that the search pattern appears on is for context. This is particularly true when I search for tags: because tags are on the second line of the file and the first line of the file is the title of the note, this returns the title, tags, and the next two lines.
I made a shortcut (gz for "go zkgrep") in Vim to this command, though it's slightly more limited as it can only search for the one word under the cursor. It works well for tags:
noremap gz :!zkgrep <cWORD><CR>
The second is very similar to the first, except rather than include four lines from each file, it outputs full files. Its name is zkgrepfull:
#!/bin/bash
||
|
Finally, the following command - zkrand - will open a random note from my zk as well as the index file. I use it every other day or so, just to take a peek at some note that I may not have otherwise seen recently. The idea is that doing so can help refresh my memory of ideas I've previously had or solutions or libraries I've used in coding, because maybe I've forgotten about them. Or, perhaps there has been a more recent note I wrote that is related to this random one and I didn't realize that at first, and I can make links between the two.
#!/bin/bash
||
That's everything I have, at least so far. I don't expect that I'll make any major changes to this setup, though maybe there will be some refinement. I hope this helps someone with their own zk. If you have any questions or comments, hit me up on twitter Mastodon.