Recent Content

Making docker-compose easier with wdocker

Published on:

Introduction

wdocker is a little utility written by a friend and former colleague of mine. It allows you to define commands for it in a Dockerfile. He wrote it because he used a lot of composite commands when writing docker images like:

docker stop CONTAINER && docker rm CONTAINER && docker rmi IMAGE && \
    docker build -t IMAGE && docker run --name CONTAINER IMAGE

By using wdocker to define a command he can greatly simplify his own workflow. Let's call it rebuild:

#wd# container = CONTAINER
#wd# image = IMAGE
#wd# stop = docker stop {container}
#wd# rm = docker rm {container}
#wd# rmi = docker rmi {container}
#wd# build = docker build -t {image}
#wd# run = docker run --name {container} {image}

#wd# rebuild: {stop} && {rm} && {rmi} && {build} && {run}

FROM ubuntu

# ...

Now he can use the following command instead of the list presented before:

wdocker rebuild

Syntax

wdocker has very simple syntax. You can define variables and commands:

#wd# variable = value
#wd# command: program

Variables can be used by putting them in braces, including in other variables, as you've seen in the first example.

#wd# variable = -l
#wd# list: ls {variable}

This would run ls -l when the command wdocker list is called.

As you can see you're not limited to using docker in your wdocker commands. This property is what allows me to use wdocker in my workflow.

Combining with docker-compose

I started using docker not too long ago at work to develop our projects in. This is nice because it allows me to completely isolate my development environments. Since we have a few processes running together a single docker image isn't a great option, so I use docker-compose to define and combine the containers I need.

As a side-effect this requires me to write long commands to do something like run rspec tests:

docker-compose run --rm -e RACK_ENV=test -e RAILS_ENV=test \
    container bundle exec rspec

The alternative is defining a specialized test container with a bogus entry command (such as true) and use that, which would still make the command:

docker-compose run --rm test-container bundle exec rspec

Instead I can define a wdocker command in the Dockerfile used to build the containers used:

#wd# rspec: docker-compose run --rm -e RACK_ENV=test -e RAILS_ENV=test container bundle exec rspec

FROM ruby

#...

Now I can run the following, much shorter, command to run the rspec tests:

wdocker rspec

We also use cucumber for some other tests, which is even longer to type in, adding the cucumber command is easy:

#wd# rspec: docker-compose run --rm -e RACK_ENV=test -e RAILS_ENV=test container bundle exec rspec
#wd# cucumber: docker-compose run --rm -e RACK_ENV=test -e RAILS_ENV=test container bundle exec cucumber

FROM ruby

# ...

Now I can run wdocker cucumber as well.

The latest git version of wdocker passes any arguments after the command name directly to the command to be executed. So if I need to run tests in a single spec file I can just do:

wdocker rspec spec/models/mymodel_spec.rb

We have two commands defined now that are 90% the same. I always use the --rm switch to remove the started container after it's done, I don't want a lot of containers piling up. I also always have to use bundle exec to run commands, since the containers don't use rvm or add the script directories to $PATH. We can extract them to some variables:

#wd# run = docker-compose run --rm
#wd# exec = bundle exec
#wd# test = -e RACK_ENV=test -e RAILS_ENV=test

#wd# rspec: {run} {test} container {exec} rspec
#wd# cucumber: {run} {test} container {exec} cucumber

FROM ruby

# ...

Right now these commands always use the container service defined in docker-compose.yml. I could add it to the run command, but I might need to run some commands on another container, but I can define another variable:

#wd# run = docker-compose run --rm
#wd# test = -e RACK_ENV=test -e RAILS_ENV=test
#wd# run-test-container = {run} {test} container
#wd# exec = bundle exec

#wd# rspec: {run-test-container} {exec} rspec
#wd# cucumber: {run-test-container} {exec} cucumber

FROM ruby

# ...

Now you also see that variables can be nested in other variables.

If you ever forget what you defined or if the mix of commands and variables becomes too much for you, you can call the wdocker command without arguments to see the commands you defined and the shell commands they'll run.


Using DisPass to manage your passwords

Published on:

tl;dr: If you don’t care about any of the back story and just want to know how to use DisPass to manage passwords, skip to Managing passwords for instant gratification.

Introduction

DisPass is a project that was started, and is still maintained, by a friend and former colleague of mine. I've been using it for quite some time. It helps me feel safe online, knowing that all my accounts have different and strong passwords.

DisPass uses algorithms to make reproducible passphrases. Making it a kind-of functional password manager, just like Haskell is a functional programming language and Guix is a functional package manager. Given the same input DisPass will always produce the same output. This means that the generated passphrases are never stored anywhere and cannot be discovered by crackers1 and the like.

The input for DisPass consists of a label, algorithm, length, possibly a sequence number (depending on the algorithm used) and finally a password. All but the label and password have some default value, but can also be specified through command-line switches.

The Labelfile

Being a functional anything usually means that whatever you're using doesn't maintain any state. This can be true for DisPass, but isn't necessarily so. It can be a challenge to remember the size, algorithm and sequence number for a large number of labels, so there is the labelfile.

The labelfile is normally located in either $XDG_CONFIG_HOME/dispass/labels or $HOME/.dispass/labels, but can also be specified on the command-line. It contains the metadata for the labels, and the labels themselves. This lets you run something like:

dispass generate foobar

And it'll know the size, algorithm and sequence number for the label “foobar”, assuming you’ve saved it to the labelfile. The labelfile is unencrypted, but this information is useless as long as nobody knows the password(s) you use to generate the passphrases.

Setting up

DisPass is easy to install if you have either Archlinux or pip installed. Windows is a bit more problematic and I don’t even know how to get started on a Mac personally, but there is no reason it can’t work. It doesn’t have many dependencies, so you don’t need to install anything else first.

The latest release is quite old, but a new release should be coming soon. There haven’t been too many developments since version 0.3.0-dev because it basically does what it needs to do, and the user base is currently very small, so bugs might not be encountered too quickly. Don’t think that it’s an abandoned project, if you look at it’s github page you’ll see that it’s seen a bit of development again as of late.

In the case of Archlinux I’ve provided packages in the AUR for both python2-dispass version 0.2.0 and python2-dispass-git. Installing either of these like any regular old aur package will get you set up. Incidentally, if you’re using Archlinux on x8664 and have the testing package repository enabled, you could also use my package repository, though no guarantees that it’ll ever work are given there.

For a general pip installation it should be as easy as running:

sudo pip install dispass

UIs

Seeing as how my friend would like it to be generally useful, and he’s a VIM user, there is both a GUI and CLI interface. Since I’m an Emacs user I’ve created an Emacs and a Conkeror interface for it as well.

CLI

The CLI is what gets the most attention and gets developed the most. I will be working with this in the Managing passwords section.

GUI

There is a basic GUI included with dispass, it can be started with either the gdispass or the dispass gui commands. It requires tkinter to be installed. It doesn't do everything the CLI does, but there are plans to improve it and use a different gui library (such as Qt). In some situations it can copy the generated passphrases directly to the clipboard, but this is only true on GNU/Linux, not on Windows.

Emacs

I wrote an Emacs interface when I started using DisPass. It tries to copy the generated passwords directly to the clipboard, instead of needing the user to copy it manually as the CLI does. It can also insert generated passphrases into a buffer, such as the minibuffer.

It's available on github.

Conkeror

I also wrote a Conkeror interface some time later, because I didn't want to keep copying and pasting the passphrases through one of the other interfaces (usually Emacs). It inserts the generated passphrases into the focused input.

It's also available on github.

Wishlist

As I mentioned, the idea is to expand the GUI and use a different gui library for it, to make it look a little better. The functionality should also be extended to do everything the CLI does.

A Firefox extension is also still on the list of desirable interfaces. I'm not sure how plausible it is with the new WebExtension plugin api, I haven't looked into it yet. I don't think chrom(e|ium) allows developers to call external programs, which is an obstacle, but I haven't looked at this either.

Managing passwords

Now for the real fun. Generating passphrases is simple. Use the generate command:

dispass generate foobar

If no entry exists in the labelfile for foobar, it uses the defaults, which at the time of writing are a length of 30, and the algorithm dispass1. This algorithm doesn't use a sequence number. It can generate more than one passphrase at a time.

The generated passphrases are presented in an ncurses screen so they aren't kept in your terminal emulator's scrollback history, at least in some cases. You can use the -o switch to do away with the ncurses screen and just output a line for each generated passphrase. Together with something like awk this can be used to directly send some command the passphrase it needs. For example, if the program foo needs a password from stdin, you could use:

dispass generate -o foobar | awk '{ print $2 }' | foo

You can specify a different length, algorithm and sequence number by using command line switches. For example, I normally prefer the dispass2 algorithm since it adds a sequence number. For some crazy reason the place I use the passphrase limits it to a length of 16 characters and I've had to change my password twice, so I use a sequence number of 3. I could use:

dispass generate -l 16 -a dispass2 -s 3 foobar

It would be difficult to remember all this, so I personally would add it to the labelfile. To do this I can use the add command. Basically this is:

dispass add foobar

This creates an entry in the label file with the same default values as the generate command: a length of 30 and using the dispass1 algorithm. To use the values we used before we can instead do:

dispass add foobar:16:dispass2:3

This way we can add multiple entries with different values at once:

dispass add foo:16 bar::dispass2:2

This would add the foo label with a length of 16, using the default algorithm and the label bar with the default length, using the dispass2 algorithm and the sequence number 2. As you can see you can omit any trailing parameters and leave any parameters in between empty to use their default values.

If you added it before I showed you the extended add syntax you can use update to change an existing entry in the labelfile:

dispass update foobar 13:dispass2:3

Unlike the add command, the update command only updates one label at a time.

Now, the place I use my password was cracked by crackers1, my password was stolen. That's no biggie. I use the list command to check what my sequence number is:

dispass list

Then I can update my labelfile and use a new sequence number:

dispass update foobar ::4

I could also use the convenient increment command:

dispass increment foobar

Everytime the sequence number is changed the input changes and so does the passphrase. So a simple call to the increment command will completely change your passphrase. This is nice, because otherwise I'd have to change either the label or the password used to generate the passphrase.

Actually, I just quit the job where I used my foobar label. I still use many other labels and don't want my list to get too big. I also don't want to delete the label in case I ever need to get back in there, so I just disable it:

dispass disable foobar

This keeps it in the labelfile, but commands such as list don't show it anymore. But then they really need me back, and since I'm now a freelance worker I can accommodate them, so I enable my label again:

dispass enable foobar

But now the place where I use the foobar label has gone out of business (I mean, come on, using a maximum password length of 16 and getting cracked by crackers all the time, are you really surprised?) and their site has been taken offline. Now I really have no reason to keep this label around, so I remove it:

dispass remove foobar

Cons

Yes, this is an excellent project and I'm not just saying that because a friend of mine wrote it. There are some things that it just isn't suited for.

When sharing a single account with someone else (don't do this!), you can't expect the other party to use the same label and password to generate the passphrase, if they're even tech-savvy enough to use DisPass just like you. It also increases the amount of information you need to remember to use DisPass. There are better programs to store pre-generated passwords.

Due to the way the current algorithms are implemented there is a limit to the length of the passphrases and that limit isn't entirely consistent. This is only a problem when you need passphrases of more than 100 characters, and I haven't had that problem yet.

Footnotes:

1

I refuse to use the term hackers, because to me that means something completely different, and I hope to you as well.

This article is tagged as: dispass

Yoshi Theme 6

Published on:

According to github I released version 6 of my Yoshi theme 8 days ago. Consequently I uploaded it to Marmalade. I felt I should mention that, and it also gives me an excuse to write something.

This new version brings a newly added CHANGELOG, inspired by Keep a CHANGELOG. It doesn't completely follow the format suggested in there because it's in Org mode format and because I felt I could style this format better than the one they propose, at least when using Org mode. I still need to export it to somewhere nice.

Since the last release, which I don't think anyone has ever seen, there have been some major changes in magit's faces, so those have been added. The original ones are also still there in the hopes that anyone using an older version will still have nice colors.

I've also removed some color variations because I felt they were impurities in the theme. I don't think they actually bring anything to the table and it looks cleaner with just the basic set of colors.

If you have any suggestions, wishes, questions or insults you want to throw my way, please do so in the issue tracker.


Introducing ox-coleslaw

Published on:

I have a big problem: I can't write a blog in anything other than Org mode. I have another problem: I haven't found a good way to write a blog only in Org mode. This always keeps me going back and forth between blogging systems. I've used tekuti, WordPress, and I've tried a few others. Currently I'm using Coleslaw. I haven't written anything lately though because it supports Markdown and HTML and I was getting antsy for some Org mode again. So I've been on the lookout for something new.

Well… I've had enough. I'm not going away this time. I'm going to fix my problems and commit to this system. I picked Coleslaw because it's written en Common Lisp and has some interesting features. I'm going to write an exporter for org to whatever Coleslaw needs!

I've known that it's pretty easy to write an exporter for Org mode for some time, but I've never actually tried to write one. I modified some bits and bobs on org-blog, but that didn't really work out. Today though, while reading an old(er) post on Endless Parentheses, I ran into ox-jekyll. Jekyll has a pretty similar page/post definition syntax to Coleslaw, so it seemed easy to read what they're doing and copy the relevant parts. It's a very small Emacs Lisp file, which made it very easy. So congrats to them and the people writing Org mode for making some very clear code.

So I wrote (or copied) ox-coleslaw based on ox-jekyll. It's slightly smaller than ox-jekyll because, frankly, it offers less. I just need a simple way to export a .org file to a .post file, nothing fancy.

To write posts I will use Org mode. Once ox-coleslaw is loaded I use the org export function to export it to an HTML file with the proper header. You can also do this non-interactively from, for example, a Makefile, but that is a story for another time.

This document is the first attempt at publishing a blog post using ox-coleslaw.


Installing HLA on Archlinux

Published on:

I recently started reading The Art of Assembly Language, 2nd Edition. It uses High-Level Assembly language in its code examples and this requires a special compiler, or assembler, to turn your code into machine code.

Fixing the PKGBUILD

The compiler, hla, is available on the Archlinux User Repository here. At the time of writing, though, that PKGBUILD doesn't work entirely. By default pacman removes all static libraries from the created packages, which took me a while to find out. Adding the following line to the PKGBUILD fixes it:

options=(staticlibs)

I also placed a comment on the AUR page, but there has been no sign of acknowledgment so far.

Running on x86_64

After having installed the compiler I got a lot of errors compiling my very simple hello world application, as typed over from the book. The gist of them was that it couldn't create 64-bit executables, which isn't very surprising as HLA seems to be only for x86 (32-bit) architecture. Another comment on the AUR page helped that though. One should add the -lmelf_i386 switch to the hla command-line. So I put in my ~/.zshrc:

alias hla="hla -lmelf_i386"

This discovery only came after a few other attempts to install HLA.

Alternative: Using Vagrant

Before I'd read about the -lmelf_i386 command-line switch I was looking at ways to run a 32-bit operating system inside my Archlinux installation. There are a few options I'm familiar with: lxc, Docker and Vagrant.

At first I tried to create a 32-bit Archlinux container, but the installation script failed, so I couldn't get that started. Then I went on to Vagrant, which worked pretty quickly.

I used the ubuntu/trusty32 box, which can be downloaded by calling:

vagrant box add ubuntu/trusty32

A very short Vagrantfile:

# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure(2) do |config|
  config.vm.box = "ubuntu/trusty32"
  config.vm.provision :shell, path: "vagrant.sh"
end

and then the provision in vagrant.sh:

wget http://www.plantation-productions.com/Webster/HighLevelAsm/HLAv2.16/linux.hla.tar.gz
tar --directory / --extract --file linux.hla.tar.gz

cat > /etc/profile.d/hla.sh <<EOF
#!/usr/bin/bash

export hlalib=/usr/hla/hlalib
export hlainc=/usr/hla/include
export hlatemp=/tmp
export PATH="${PATH}:/usr/hla"
EOF

After that you can just call vagrant up, wait a while and then have fun playing around with HLA in an Ubuntu 14.04 environment.


display-graphic-p, cl-letf and themes

Published on:

I had a problem with my theme a while where the colors would be different depending on whether I was starting Emacs as a daemon or not. The colors module has some code that changes the way it works depending on whether a graphical or non-graphical display is run. This was messing with some of the colors in my theme.

My first thought was that it should just wait until the first frame is created before setting the theme. For a while this worked.

(if (daemonp)
    (add-hook 'after-make-frame-functions
              (lambda (frame) (load-theme 'yoshi t)))
  (add-hook 'emacs-startup-hook (lambda () (load-theme 'yoshi t))))

Some time later, that stopped working, because apparently display-graphic-p stopped returning true during the execution of the after-make-frame-functions hook. So I set out to change it so it would work again. This would have been easy in earlier versions of Emacs, where flet wasn't yet deprecated.

(if (daemonp)
    (add-hook 'after-make-frame-functions
              (lambda (frame)
                (flet ((display-graphic-p () t))
                  (load-theme 'yoshi t))))
  (add-hook 'emacs-startup-hook (lambda () (load-theme 'yoshi t))))

Unfortunately, as stated, flet is deprecated. Whenever you compile it you're warned and told that you should use cl-flet or cl-letf instead. I couldn't quite figure out how, though. cl-flet does almost the same thing as flet, but lexically, so you can't dynamically rebind display-graphic-p that way. cl-letf did't seem to be anything like flet at all, so I didn't know how to ever use that to accomplish this.

Thankfully, there was noflet. This, for my purposes at least, works the same way as flet did. It is not part of Emacs, however. But in this case I didn't know what else to do.

(if (daemonp)
    (add-hook 'after-make-frame-functions
              (lambda (frame)
                (noflet ((display-graphic-p (&optional display) t))
                  (load-theme 'yoshi t))))
  (add-hook 'emacs-startup-hook (lambda () (load-theme 'yoshi t))))

This worked perfectly, but I really don't like depending on external packages for starting up emacs.

Then Artur Malabarba wrote a post about using cl-letf and things became very clear.

(if (daemonp)
    (add-hook 'after-make-frame-functions
              (lambda (frame)
                (cl-letf (((symbol-function 'display-graphic-p))
                          (lambda (&optional display) t))
                  (load-theme 'yoshi-theme t))))
  (add-hook 'emacs-startup-hook (lambda () (load-theme 'yoshi-theme t))))

It's a bit more involved than the noflet solution, but it does remove a requirement from my init.el. What I failed to realize before reading Artur's post is that cl-letf works a lot like Emacs' new setf, which works a lot like Common Lisp's setf, which makes it very powerful.

By now, my theme doesn't use the colors module anymore.


Small Recent Posts

Published on:

I've been reading Professional WordPress Plugin Development the last few weeks and I finally got around to putting some of what I've read (a tiny bit) to use.

I've written my first WordPress plugin. It's part of a larger project of mine to replace my [[http://orgmode.org]] powered static site with WordPress and learn about WordPress plugin development along the way.

Right now, it only shows the 5 latest posts, much like the widgets in the side-bar and admin area do (except with less options I'm sure). My plans are simple, just a number of posts and possibly a tag or category.

For now I don't have any place for you to see it in action, but hopefully soon. To use it just add [[recent]] anywhere in your post or page. It is meant to be used in pages, but there is no technical reason why it couldn't also be used in posts.


Quick normal-state

Published on:

I realized today that most of the time (over 90%) of the time I save a file I feel I have finished typing something. Typing this out makes it sound very obvious, actually. One other thing I noticed is that I still forget to return to normal-state after I'm done editing. Emacs being hackable and not being content with twisting my brain to suit my editor but rather twisting my editor to suit me, I thought I might automate it a little.

My first idea was to have evil automatically revert to normal-state whenever I save and also whenever I switch buffers. I haven't found a hook that runs when switching buffers, so I'll need to brush up on my advising skills and have it run just before switching, perhaps even when execute-extended-command or smex run, so any explicit minibuffer action returns to normal state.

For now it's just the save file, and only for buffers that aren't in the emacs-state as default list:

(defun modes-starting-in (state)
  "Get a list of modes whose default state is STATE."
  (symbol-value (evil-state-property state :modes)))

(defun maybe-switch-to-normal-state ()
  "Switch the current buffer to normal state.

Only do this when the mode is not in emacs state by
default."
  (unless (memql major-mode (modes-starting-in 'emacs))
    (evil-normal-state)))

(with-eval-after-load 'evil
  (add-hook 'after-save-hook
            #'maybe-switch-to-normal-state))

I personally only use either normal-state or emacs-state as default states when a mode loads, if you want to check more you'll have to add some more calls to memq and change emacs to, for example insert or visual or whichever you need.


Short excursion

Published on:

I took a short excursion to the Ghost blogging platform, which I installed locally, just to try it out. I've been keeping up my posts there, though I did decrease the number. Posting every day is too much of a strain still, so I've decided to post once every two days, tuesdays, thursdays and saturdays. I've just copied the posts over from Ghost to here.

This article is tagged as: meta

Clean completions

Published on:

Today's post is very short. In my quest to make GNU Emacs look ever better I thought about something new (to me). Not all buffers need have a mode line. As such I will now turn off the mode line for completion-list-mode, since that only shows up for a short while and I've so far never had any trouble distinguishing it from other buffers.

(add-hook 'completion-list-mode-hook
          (lambda () (setq mode-line-format nil)))

Now, whenever a completion buffer pops up, it'll use all the space available, including the line where the mode line used to be. If it shows up just above the minibuffer (which for me it always has done) it looks more like a part of the same thing instead of two separate windows.


This blog covers archlinux, avandu, avandu-lua, cask, ci, clark, common-lisp, config, conkeror, diff, dispass, dispass.el, editors, elisp, emacs, eshell, evil, exherbo, experiments, file-synchronization, games, git, github, gnus, hla, html, javascript, lisp, lua, markam, meta, mpd, notion, org-mode, ox-coleslaw, projects, rc, sbcl, small-recent-posts, software, stumpwm, systemd, tasks, tekuti, testing, tips, todo, ttrss, utility, vagrant, vc, vim, visual, wdocker docker docker-compose, wm, wordpress, yoshi-theme

View content from 2016-02, 2015-09, 2015-08, 2014-12, 2014-10, 2014-08, 2014-07, 2014-06, 2014-04, 2014-01, 2013-11, 2013-10, 2013-08, 2013-05, 2013-04, 2013-02, 2013-01