Moving from KDE to Emacs

When I started using EXWM, I thought that using it together with KDE would be the best combination. But then my Plasma configuration self-destructed by chance, I had to recover it from backup and I started thinking what and how much I actually need from KDE exactly.

The basic principle is simple: Things are easier to do in KDE but what you get is basically all you get. Arranging the complete desktop in Emacs would mean knowing what’s happening and making it flexible, for the price of more initial work. I decided to try it and I switched from KDE to Emacs completely.

Here is what I had to do to make it happen.

Key bindings

This is the easiest part. All key bindings are simply managed by Emacs.

Multiple keyboards

Emacs input methods, which are much better and simpler to customize than X keyboards, can be used in applications supporting XIM. Mainstream applications generally work with Emacs input methods. setxkbmap can be used for those that don’t if needed.

I have already discussed setting up EXWM for input methods in a previous post.

Multiple monitors

This is harder. In KDE, it works some way automatically and there is a user friendly configuration tool. Although it may happen the configured settings change from time to time and the configuration must be fixed. EXWM works well together with KDE; without it, it must be handled using other tools.

On the contrary to what is written in EXWM documentation, no exwm-randr-screen-change-hook arrangements were needed. But there is still configuration needed. Especially when changing to a new monitor setup for the first time, the given setup must be arranged manually using xrandr, ARandR, or another tool. Then it can be saved using autorandr.

In order to make switching between the configured monitor setups automatically on monitor connections and disconnections, and when opening or closing the laptop lid, system environment must be configured accordingly. This is what works for me on NixOS:

services.autorandr.enable = true;
services.acpid.enable = true;
services.acpid.lidEventCommands = "/run/current-system/sw/bin/autorandr --batch --change";

It’s also highly advisable to configure exwm-randr-workspace-monitor-plist accordingly.

Desktop notifications

Emacs Desktop Notification Center (EDNC) handles them and opens new amazing possibilities to process them. I have already discussed it in a previous post. Obviously, KDE absence avoids the conflict with KDE notification handling and it’s no longer needed to apply the plasmashell workaround.

Panel

Panel widgets provide useful functionality such as displaying information about the system or access to various settings.

There are several Emacs add-ons to display system information. I got attracted by lemon, which allows displaying information in the echo area when Emacs is idle. This is a good use of an otherwise unused screen area. But it’s not without problems. The set of provided monitors in lemon is limited, due to EIEIO limitations it’s not always easy to customize the provided monitors or even to customize my own monitors (broken multiple inheritance!), it’s uncomfortable both waiting for the lemon output to appear and having it appearing prematurely, replacing minibuffer or echo area contents (e.g. in isearch or when a function help or an evaluated expression is displayed), and lemon may stop updating the echo area when in a non-Emacs application buffer. I still think lemon is a great idea but it doesn’t match my needs well enough.

So I decided to abandon lemon and to sacrifice one line of the screen space and use it for the tab bar. I use tabs, but I don’t display them in the tab bar, I use the tab bar for displaying global information (e.g. sound settings, system info, notifications, time, battery status). It’s possible to use the mode line for such purposes, but there is much less space there and I keep only buffer-specific information there, which makes more sense.

One problem with using the tab bar is that if something you put there produces an error, even once, then the tab bar disappears and cannot be restored until Emacs is restarted. Annoying.

System settings

Without the KDE panel, an alternative access to some system settings is needed.

As for network, I use nmcli now. It’s not that bad as it sounds and with a couple of Eshell aliases for common actions (handling WiFi, VPN, …), it may be more comfortable in some sense than using the dialogs. And network connections can be still edited using GUI tools if needed.

As for Bluetooth, I use Emacs Bluetooth Mode. It has some quirks but Bluetooth setup has never been easy in (not only) GNU/Linux, in any environment.

As for sound, I had to write my own Emacs add-on, pipewire-0, to manage PipeWire settings. I tailored it for my own needs, so it serves me well. 🙂

GnuPG password entry

Whether using KDE or not, GnuPG password entry doesn’t work well with EXWM when using external dialogs. Several EXWM tutorials contain tips, not exactly the same, how to fix it. Here is my version.

Emacs pinentry must be enabled by putting

allow-emacs-pinentry

line to ~/.gnupg/gpg-agent.conf.

NixOS has a corresponding configuration option (and pinentry-emacs package):

programs.gnupg.agent.pinentryFlavor = "emacs";

And an option to start the agent itself:

programs.gnupg.agent.enable = true;

Finally, Emacs pinentry must be configured in Emacs configuration:

(setenv "GPG_AGENT_INFO" nil)           ; when not using the NixOS option
(use-package pinentry
  :init
  (setq epg-pinentry-mode 'loopback)
  :config
  (pinentry-start))

Using Java applications

Java is infamous for its window manager related problems. Fortunately, it contains a workaround that can be activated by setting an environment variable:

export _JAVA_AWT_WM_NONREPARENTING=1

Starting the desktop

It’s distribution dependent and it was tricky to find it out on NixOS. In theory, a simple setting

services.xserver.windowManager.exwm.enable = true;

might work. But since NixOS makes hard to access additional C libraries and Python packages from Emacs, I must start Emacs with nix-shell, with a help of my custom shell script. Instead of the setting above, I must use the following:

services.xserver.displayManager.session = [
  { manage = "desktop";
    name = "emacs";
    start = ''
      /my/Emacs/startup/script.sh &
      waitPID=$!
    '';
  }
];
services.xserver.displayManager.defaultSession = "emacs";

Autostarting applications after logging in

This can be arranged in various, less or more smart, ways in ~./emacs.d/ using start-process-shell-command.

Now, when applications are always started from Emacs, they are guaranteed to get down together with it. This has two problems.

First, throughout the years with Emacs, I got used to understand the Emacs confirmation question about terminating running processes on exit just as a protection against accidentally hitting C-x C-c. I respond “yes” automatically and this terminates applications forcefully. I must learn to quit them cleanly with kill-buffer before exiting Emacs.

Second, Emacs restart requires restarting the other applications too. This is quite annoying although actually not that big deal since the only non-Emacs application I run most of the time nowadays is a web browser.

Screen saver

There are several screen savers available. I tried using i3lock with xautolock but it didn’t work very well, the screen could get blanked during meetings or while watching videos. The old good XScreenSaver works better although autolocking and switching screen off automatically doesn’t seem to work completely as expected.

Conclusion

My desktop is completely managed by Emacs now (except for the log in screen). I like it, it’s a clean environment, under my full control. It was a non-trivial effort though and it’s always tempting to improve something (unlike with KDE where it’s not possible). It depends on one’s needs whether it’s worth it or not. For us who have Emacs as a hobby, perhaps it is; the others may happily continue to use Emacs with KDE.

emacs-desktop.jpg

More fun with EXWM

I have recently started using EXWM, an Emacs window manager. It’s very useful as a window manager itself but it also provides additional functionality and opens new possibilities.

Starting EXWM automatically in KDE

EXWM allows, similarly to other window managers, replacing an already running window manager. Now I’m confident about it enough and start it automatically, by creating ~/.config/plasma-workspace/env/exwm.sh file with the following content:

export KDEWM=~/bin/start-emacs.sh

Why to use a script rather than starting Emacs directly? We’ll see later.

EXWM and input methods

EXWM allows transforming keys sent to applications. I don’t have currently a good use case for transforming key shortcuts or so but I’m absolutely amazed by the possibility to use Emacs input methods in X applications.

The predefined X.org keyboard layouts are far from perfect and everybody who tried to use a customized keyboard using XKB knows the related problems. On the other hand, defining custom keyboard layouts in Emacs is relatively easy, doesn’t require changing system configuration, and offers significantly more flexibility.

Yes, with EXWM, it’s possible to use Emacs input methods in other applications! The following must be added to the exwm.sh file created above:

export XMODIFIERS=@im=exwm-xim
export GTK_IM_MODULE=xim
export QT_IM_MODULE=xim
export CLUTTER_IM_MODULE=xim

On the Emacs side, it’s desirable to add the shortcut for switching input methods, before EXWM is started:

(push ?\C-\\ exwm-input-prefix-keys)

And the last thing needed is to enable exwm-xim:

(autoload 'exwm-xim-enable "exwm-xim")
(exwm-xim-enable)

Using Emacs input methods in e.g. a web browser makes typing there much simpler (at least for those of us who use two or more languages). Then the only missing thing is the possibility to use different input methods in different pages or tabs displayed in the browser, similarly to Emacs buffers. (Spoiler: There is a non-Emacs web browser that allows doing this.)

Editing input in Emacs

It’s also possible to edit input fields in applications directly in Emacs using exwm-edit package. It’s another useful feature but it must be used with some caution, the edited text may get lost e.g. when the input widget in the application is no longer focused once the input text is sent to the application.

Desktop notifications

I use EXWM together with KDE. There is no reason to replace absolutely everything with Emacs. At the dreary times when GNOME was already unusable and KDE 4 not yet usable, I used to use StumpWM and I really missed things such as a normal panel with its widgets and easy access to functions such as network configuration, or a control center for settings.

But one KDE functionality (besides kwin) is worth to move to Emacs: desktop notifications. The desktop notifications do their job in KDE, but nothing more. The notifications are displayed for a while in a popup window and they are also later (not so well) accessible in the panel. But can you easily identify what’s new when looking at the screen at any given moment? Absolutely no. It very calls for a better notification management. Imagine you want to dismiss all e-mail notifications when you switch to your e-mail client. Or you are forced to use a crappy proprietary chat application without a usable API, accessible only via a web browser interface, and providing useful information to the outer world only in the form of temporarily displayed notifications (this is unfortunately a real example!). Etc.

Here the Emacs Desktop Notification Center (EDNC) comes to help. Just enable it:

(use-package ednc)
(ednc-mode 1)

and then you have all the notifications in your hands. Display, format, close, do whatever you want with them in the ways you like. Very useful and another step to a distraction free desktop. No need to use a secondary screen to watch your e-mail or chat messages, or polling their presence by switching to the corresponding applications regularly. With EXWM, one never leaves Emacs, the mode line is always here and notifications about desktop notifications can be displayed there. Now you can detect easily which inbox has new mail or who wrote a chat message or that something else happened. The details are then accessible in the EDNC log buffer or elsewhere, only once you want to see them. And it may be another use case for pip-frame.el, to display selected incoming notifications temporarily.

There is one problem though. I haven’t found a good way to disable the KDE notification service. And while it is running, EDNC cannot be started. I use this ugly (but working) hack:

(shell-command "kill $(pgrep plasmashell)")
(ednc-mode 1)
(start-process-shell-command "plasmashell" nil "plasmashell")

When plasmashell is killed, the notification service is stopped and EDNC can be started. Then plasmashell can be started again.

But this means plasmashell is started from Emacs and is taken down when Emacs is finished. This is not what I want and it is the reason why I use a script in KDEWM environment variable. The script not only starts Emacs but it also starts plasmashell again when Emacs is finished.

Using EXWM

I sometimes miss useful things. For example, many years ago, I missed emerging Org Mode, not recognizing its potential, only to discover some years later how essential tool it is. And the same happened to me with EXWM, an Emacs window manager.

I’ve always thought that making a window manager in Emacs is a crazy cool idea, which is not very useful in practice. If anything bad happens to Emacs or Emacs needs to be restarted then the whole desktop goes down. It may be a good reason itself to avoid it and there is an additional question why would one use it instead of a standard well tuned window manager. I use a tiling approach to window management, with a single window displayed over the whole screen most of the time, and I don’t need much functionality from the window manager after all.

As with many Emacs tools, there is a very good reason to use it. What I missed (my fault, discarding the idea before looking at it properly) was that windows in EXWM behave similarly to regular Emacs buffers. And Emacs already has a powerful built-in window management.

I discovered EXWM because I looked for a way to easily display a web browser window (or occasionally some other application window) side by side with an Emacs frame. This is not difficult to achieve but what if I want to switch to another Emacs workspace and back? What if I want to pop up a help window instead of the web browser window and then return back to the browser? Etc. It starts to be complicated even with advanced tiling window managers like StumpWM. (And no, window manager desktops are not a solution.) While Emacs provides similar functionalities instantly with pop-to-buffer and all the other window management facilities. When realizing this all, I couldn’t understand how I could live without EXWM. Even when not considering added bonuses, such as the possibility to use normal key bindings (compared to e.g. KDE key bindings limited to single-key bindings only and unable to use Super and Hyper modifiers).

And how about the supposed problems? Emacs usually doesn’t crash and also doesn’t block that often. In practice, there don’t seem to be real problems. Whether other applications must be restarted on Emacs restart depends on whether they are started from Emacs or outside it, which is actually not different from using other window managers.

I use Emacs / EXWM just as a replacement for the window manager, not the whole desktop environment. I keep using KDE while using EXWM instead of kwin. This means I can still use panels (with some limitations such as that it’s not possible to use autohiding panels and it’s possible to configure the panels only when EXWM is not running), general key bindings managed by KDE (useful, among other, to be able to run basic applications when Emacs is not running), KDE settings, multiple monitors (they work well without further arrangements in this environment), notifications, etc.

When setting up the EXWM environment, I realized I need a PIP (picture in picture) facility for Emacs buffers. I couldn’t find anything like this, so I wrote a simple utility called pip-frame.el implementing a floating frame displaying Emacs buffers.

Emacs tabs as workspaces

I used to use Emacs frames as workspaces. But reading through Emacs 28 NEWS reminded me that tab bars could be probably better used for the purpose. And indeed, using tab bars is simpler and looks like a very good fit.

First some initial settings:

(tab-bar-mode 1)
(tab-bar-history-mode 1)
(custom-set-variables '(tab-bar-show nil))
(tab-bar-rename-tab "emacs")

They enable the tab bar mode, tab bar history mode (not needed but useful) and hide the tab bar (not to waste screen space for no good purpose). We also want to give a static name to the initial tab.

Now we need to retrieve the tab names. I don’t know whether there is an official way to do it but the following functions serve the purpose:

(defun my-workspace-name ()
  (alist-get 'name (assq 'current-tab (funcall tab-bar-tabs-function))))

(defun my-workspace-all-names ()
  (mapcar #'(lambda (tab) (alist-get 'name tab))
          (funcall tab-bar-tabs-function)))

Workspace switching is much simpler with tabs:

(defun my-workspace-switch (name)
  (interactive "sWorkspace: ")
  (if (member name (my-workspace-all-names))
      (progn
        (tab-bar-switch-to-tab name)
        nil)
    (tab-bar-new-tab)
    (tab-bar-rename-tab name)
    t))

There is no need to track and handle frames anymore. Switching to the last workspace can be done simply with tab-bar-switch-to-recent-tab function.

The rest remains the same as previously with the frames:

(defvar my-workspaces
  ;; name
  ;; new-fn
  ;; refresh-fn
  '(("emacs"
     nil
     (lambda ()
       (require 'bookmark)
       (unless bookmark-alist
         (bookmark-maybe-load-default-file))
       (bookmark-jump "init.d"))) 
    ("gnus"
     gnus
     (lambda ()
       (let ((group-buffer (get-buffer "*Group*")))
         (if group-buffer
             (switch-to-buffer group-buffer)
           (gnus)
           (cd "~")))))
    ("irc"
     my-erc-connect
     nil)
    ("org"
     my-org-agenda
     my-org-agenda)
    ("roam"
     org-roam-find-file
     org-roam-find-file)
    ("system"
     my-run-eshell
     (lambda ()
       (call-interactively
        (lambda ()
          (interactive)
          (ido-buffer-internal ido-default-buffer-method nil nil nil "eshell:")))))))

(defun my-switch-to-workspace (&optional key)
  (interactive)
  (unless key
    (setq key (logand last-command-event 255)))
  (let* ((prefix (char-to-string key))
         (current-name (my-workspace-name))
         ;; This will be explained later:
         (predefined (cl-assoc prefix my-workspaces :test #'string-prefix-p)))
    (if predefined
        (cl-destructuring-bind (name new-fn refresh-fn) predefined
          (if (equal current-name name)
              (when refresh-fn
                (funcall refresh-fn))
            (when (and (my-workspace-switch name)
                       new-fn)
              (funcall new-fn))))
      (let* ((candidates
              (remove current-name
                      (cl-remove-if-not
                       #'(lambda (n) (string-prefix-p prefix n))
                       (my-all-workspace-names))))
             (n (length candidates)))
        (cond
         ((= n 0)
          (my-workspace-switch
           (read-from-minibuffer "Switch to workspace: ")))
         ((= n 1)
          (my-workspace-switch
           (car candidates)))
         (t
          (my-workspace-switch
           (completing-read "Switch to workspace: " candidates nil t prefix))))))))

(global-set-key (kbd "<s-return>") 'my-last-workspace)
(dotimes (i 26)
  (global-set-key (kbd (format "s-%c" (+ ?a i))) 'my-switch-to-workspace))

(defun my-switch-to-workspace-key (key)
  (interactive "c")
  (my-switch-to-workspace key))  
(global-set-key (kbd "C-c z") 'my-switch-to-workspace-key)

Of course, it’s possible to use tabs as workspaces directly, without these add-ons and with taking advantage of tab-bar-tab-post-open-functions variable. Especially people who like using mouse for tab switching may like using the tab bar mode as it is. But I still like the bits of extra functionality for a bit more comfortable tab/workspace switching.

Learning Haskell

Learning Haskell has been on my to-do for maybe about 10 years. I’ve already made several attempts but have never had an opportunity to finish them, starting again and again from time to time. Haskell is not something one can learn in two afternoons. My latest attempt has finally succeeded and I have basic knowledge of the language now. This post summarizes my first impressions.

Why to learn Haskell? It’s a programming language with interesting and quite specific concepts (functional programming with pure functions, lazy evaluation, strict static typing, etc.) while being practically useful and supported, for many years. Something that should belong to common knowledge of every advanced programmer.

There are many learning resources for Haskell but it’s not that easy to find really good ones for beginners. Eventually, I relied mostly on the book Practical Haskell by Alejandro Serrano Mena. The book has 600 pages and each of the pages is worth reading (at least from those I’ve already read, I haven’t read the book completely yet). I used also lots of other resources but the book served me as a basic learning guide.

I can say now that Haskell is indeed interesting and is a cool toy. From practical perspectives, it’s more difficult. There are good things and not so good things. As I explained in my previous post, I’m strongly influenced by Common Lisp. If I weren’t familiar with Common Lisp, I would be probably much more excited about Haskell. That means that although the following text may sound skeptical, I still see Haskell as an interesting general purpose programming language, more advanced than most common programming languages. Just not at the level I’ve got used to demand.

How difficult was to learn Haskell?

Let’s look at some struggles I experienced when learning basics of Haskell.

Cryptic syntax

In most programming languages, when looking at their source code, one can usually get some idea what it does even without being familiar with the language in any way. That’s not the case with Haskell. One must understand a lot of weird symbols like ::, :, ->, <-, =>, \, $, <$>, ~., !!, ++, `op`, >>, >>=, <*> to get the basics and a lot more to really understand the code. Additionally, keywords such as class or do have very different meanings from those common in other languages.

Even now, when I have learned the basics, I often look at some Haskell source code with “what the …?!”. One must be really proficient with the language to understand it easily.

The only clear syntactic construct in Haskell is if then else. Almost everything else is either cryptic or confusing. I have met only two similar examples so far: Perl, which is fortunately legacy these days (thanks to Python, I believe), and TeX, which is limited to typesetting.

Oh, and Haskell is yet another language with the annoying camelCase (AKA ICantReadThis) naming convention.

Very high initial barrier

There is a reason why it took me about ten years to learn the basics of Haskell eventually. In many programming languages, one can start programming in them after two or three afternoons and then learn additional stuff incrementally. With Haskell, it’s a journey, it’s necessary to learn a lot before being able to write something meaningful. It’s not a matter of a couple of evenings, it requires a large initial free-time block, which is hard to get for such a thing once one is no longer a university student. Honestly, I’m not willing to take a week-long vacation to spend it solely learning Haskell.

The primary barrier is probably all the fancy algebraic stuff, which is huge and not so easy to explain and to understand, at the same time being necessary to do any I/O and other things. The cryptic syntax creates an additional hurdle, making it difficult to return to the language study after a longer break.

Thousands of basic functions

Languages not supporting optional and keyword arguments and with insufficient polymorphism naturally tend to encode type and other information into function names, leading to explosion of the number of function or even type names. It is ugliness by itself, but additionally good luck to any beginner looking into reference documentation and trying to grasp the most useful essentials from it.

The type system: a friend or an enemy?

The strong typing system looks very important for safety and it actually is. But it doesn’t seem to be that great in practice. It still doesn’t prevent a lot of bugs and, at least at the beginner level, I find struggling myself with it more than it would help me in non-Haskell environments. It will probably get better once I know and practice more, but at the moment it’s another learning barrier.

In Common Lisp, it’s possible to specify types but it’s not used that often. I usually specify types only for structure/class members and when being asked by the compiler to provide hints for optimization (typically discriminating between different numeric types). The compiler identifies some basic mistakes itself and most remaining type errors are caught quickly before the program starts being used. Although type errors are experienced from time to time in production code, they are not that frequent, because the code is usually clear enough to see easily what belongs where.

That said, this may not apply to other Lisp dialects. For example, I think Emacs Lisp would benefit from better compile time checks.

In Haskell, I specify types manually everywhere except for local definitions (let & where). I wouldn’t dare to omit them because things are often entangled and not very clear in Haskell code. But sometimes I meet situations where I exactly know what I’m doing but I find it hard to specify it to Haskell.

In my first non-trivial Haskell program, I wanted to use something similar to the following:

import Data.Function
import Data.List

class SomeState state where
  nextSteps :: state -> [step]
  applyStep :: state -> step -> state

class Step step where
  value :: step -> Int

step :: SomeState state => state -> state
step state = step' (nextSteps state) state
  where
    step' [] state = state
    step' steps state = applyStep state $ maximumBy (compare `on` value) steps

Here we want to use some general type (SomeState), together with another type (Step) that guarantees some properties (applicability of value function). This would work easily in common programming languages but not in Haskell. The intention is clear, there is no real bug in the program, but the type specifications are insufficient and the program doesn’t compile. A bit of disappointment after being proud about putting together the last line of the code above. Since Haskell, unlike other popular modern languages, even strongly typed, doesn’t support dynamic type dispatch, we have to improve the type specifications.

But how? Obviously, step must be specified. A naive straightforward approach doesn’t work:

class Step step => SomeState state step where
  nextSteps :: state -> [step]
  applyStep :: state -> step -> state

The compiler objects:

Too many parameters for class ‘SomeState’
(Enable MultiParamTypeClasses to allow multi-parameter classes)

Let’s follow the advice:

{-# LANGUAGE MultiParamTypeClasses #-}

class Step step => SomeState state step where
  nextSteps :: state -> [step]
  applyStep :: state -> step -> state

step :: SomeState state step => state -> state

Still not good:

Could not deduce (SomeState state step0)

The compiler knows that step satisfies Step. But searching the web discovers that

nextSteps :: state -> [step]

is meaningless, because it specifies that nextSteps should return all the possible types of step, which is of course not possible. Quite surprising to what we are used to in other programming languages. Desperation increases — as before, the intention is clear, the program is correct, but the type checker is still not happy. Further search discovers there is a thing called functional dependencies:

{-# LANGUAGE FunctionalDependencies #-}
{-# LANGUAGE MultiParamTypeClasses #-}

class Step step => SomeState state step | state -> step where

This works! Following the final suggestion of a linter to use ConstrainedClassMethods rather than MultiParamTypeClasses, we get the final, type correct, version:

{-# LANGUAGE ConstrainedClassMethods #-}
{-# LANGUAGE FunctionalDependencies #-}

import Data.Function
import Data.List

class Step step => SomeState state step | state -> step where
  nextSteps :: state -> [step]
  applyStep :: state -> step -> state

class Step step where
  value :: step -> Int

step :: (SomeState state step) => state -> state
step state = step' (nextSteps state) state
  where
    step' [] state = state
    step' steps state = applyStep state $ maximumBy (compare `on` value) steps

Most likely obvious for an experienced Haskell programmer, but hard to achieve for a beginner trying to write his first simple program. After hours of searching, reading, trying to understand and using three language extensions, the goal of specifying the obvious has been achieved.

Generating random numbers

Trying to use random numbers in Haskell illustrates some of the learning struggles mentioned above.

Let’s start with random numbers in Common Lisp. A search term such as „common lisp random“ leads to the corresponding documentation page, which explains that generating random numbers is as easy as expected. For instance

(random 10)

returns a random integer in the range from 0 to 9 or

(random 10.0)

returns a non-negative floating number less than 10.

Anytime, anywhere.

There is a little culprit in that some stuff around the random state is implementation dependent. This is because Common Lisp has an official ANSI standard and having an overspecified standard could cause trouble. The implementation specific things are clarified in the SBCL manual (SBCL is to Common Lisp what GHC is to Haskell). An important information there is that the initial random state is the same each time SBCL is started. This is good for testing and debugging but to have truly random numbers, one can make the random state random:

(setf *random-state* (make-random-state t))

And this is all one needs to know for 90% of non-cryptographic random number uses.

How about random numbers Haskell? Well, there is in principle no such thing as a mutable state, even less a global mutable state, in Haskell. Moreover, to obtain true random numbers, we need to access a system source of random information. That means we will have to use the Haskell state tracking machinery, including IO.

No surprise that it’s not a part of Prelude and is implemented as a special library. Uh, OK. Let’s look at its documentation. It starts with some examples using random number generators initialized with particular seeds. They are followed by many, many variations of functions and instances. Let’s try to dig out something from them.

First, we need a random number generator initialized from random data. A good candidate seems to be the global generator but according to the documentation, its use is discouraged in libraries. What to use instead? initStdGen looks like the function that we need.

Now, what to do with the generator? It’s necessary to track its changes when generating random numbers. This has some implications. There is a trouble to use an infinite list, one of the Haskell idioms, of random numbers because then we don’t have access to the generator anymore. Either the list is the only random source we need (not applicable if we need random values of different types) or we should use a separately initialized random number generator to create it. Neither of the options is attractive in my case, so let’s give up on infinite lists. Another implication is that the need of tracking together with the typical type pattern there, g -> (a, g), strongly suggests using monads, which is also addressed by the documentation.

initStdGen already wraps the generator in MonadIO but it’s beyond my current knowledge how to utilize it and the documentation is not helpful in this respect. It looks like the generator should be extracted and wrapped in another monad. The documentation mentions StateGenM, which should be good enough, but it seems it can be used basically only as:

main = do
  g <- initStdGen
  let (r, g) = runStateGen g randomM :: (Int, StdGen)
  
  let (r, g) = runStateGen g randomM :: (Int, StdGen)
  

Which brings no progress over using and tracking the generator directly, without monads. So let’s use another wrapper. newIOGenM is used most often in the documentation and it seems to be the least overkill from the available options although I don’t need to perform any IO when generating random numbers. But it apparently serves for tracking the generator state some way.

main = do
  g <- initStdGen >>= newIOGenM
  r <- randomM g :: IO Int
  
  r <- randomM g :: IO Int
  

This looks better. But I don’t want to generate random numbers in main, I want to pass the generator to another function. The case above looks interesting, it seems we have a truly mutable object g — could we pass it to a function simply as a mutable argument?!

randomList :: IOGenM StdGen -> Int -> [Int]

Apparently not, the functions applied on the generator return their values in monadic contexts. And since IOGenM should be bound to IO some way (possibly explaining the apparent mutability), let’s try to use IO as the context:

randomList :: IOGenM StdGen -> Int -> IO [Int]
randomList g n = do
  replicateM n $ randomM g

It works! We could also take the advantage of the “free slot” in IO to return a value from the function. Otherwise we had to use monad transformers, I’m afraid. So having IO there is not that useless in the end result.

It’s still needed to pass the generator as an extra argument, but in order to not complicate the things already complicated enough, let’s live with it.

The last remaining bit is that using Int may perhaps cause some expensive conversions and one of the Word types should be used instead, together with the corresponding functions.

This example illustrates how knowledgeable one must be before starting with basic programming in Haskell. And it’s only a summary, omitting all the blind alleys I met before reaching the given point.

Compared to other programming languages

Haskell is a better language than e.g. Java or Go. It’s safer and more expressive. Haskell can probably, purely technically, serve as a good replacement in many typical usage areas.

Then there is Rust, which has clearly strong Haskell origins and many similarities. There are nevertheless crucial differences such as explicit memory management versus garbage collection, mutability and strict evaluation versus pure and lazy evaluation, or simplicity versus higher level programming means in Haskell. This makes the languages suitable each for different areas of application. Although there are overlaps too one cannot say one would be a universal substitute for the other.

But when compared to Common Lisp, there is a big question: Why Haskell? Everything is simpler in Common Lisp and Common Lisp additionally provides much more programming means (e.g. mutable variables and structures, optional and keyword arguments, a powerful and relatively easy to use macro system, fully generic functions and object oriented programming). And some of the Haskell properties crucial and basically necessary in Haskell (strong typing, pureness, laziness) are of much lesser importance in Common Lisp, leaving them optional there, used only when needed. I tried to look for Haskell advantages over Common Lisp on the web and the best argument I could find was “I like the way Haskell does things”. Which is a valid argument but not very convincing.

So is learning Haskell worth it?

I think so otherwise I would give up learning it a long time ago.

For first, Haskell is a classical topic and getting familiar with its paradigms is useful by itself.

For second, many of the programming languages created in the last decades are sorts of a subset of Haskell and Lisp. Knowing both Haskell and Common Lisp means knowing most most of the modern programming concepts and helps understand their strengths and weaknesses in other programming languages. (But I’ve found out that learning Rust before Haskell is an easier path than vice versa.)

For third, it’s a better language than many mainstream languages, providing both better safety and more programming means. It’s also, although not being mainstream, kinda in these days (unlike Common Lisp, which seems to be continually decreasing in popularity) and there are professional opportunities related to Haskell.

Will I be actually programming in Haskell? It depends. I’d like to keep my Haskell knowledge alive. There may be professional opportunities to work in Haskell. I guess Haskell programs are likely to be often written better than programs written in mainstream languages, making working on them more pleasant. Maybe my learning pet project in Haskell will evolve (I doubt it, because of time constraints, but it’s still a possibility). Maybe I’ll have a reason to contribute to something written in Haskell. These may be good reasons to program in Haskell although Haskell is unlikely to become my first choice language. At least as long as Common Lisp remains sufficiently usable (considering there are enough key Common Lisp people in my generation, this should hopefully hold for the rest of my life).

Learning Rust

Many programmers like learning programming languages. It’s fun to discover new concepts and possibilities and to be able to use them to write more elegant, more powerful or more efficient programs.

I had enjoyed this activity too, until I discovered Common Lisp many years ago. Once I learned Common Lisp, my passion for learning new programming languages disappeared. Common Lisp is so powerful that there is little to discover elsewhere. Learning new programming languages is no longer about discovering new worlds for me but rather about necessity or pragmatism.

Still, I have recently learned two new programming languages despite no immediate need: Rust and Haskell. I’ll write about my Haskell experience in another post, let’s talk about Rust now.

Why to learn Rust? Because it’s a C replacement done apparently right and getting quite popular. It’s safe, it’s safe, it’s safe, it’s much more powerful and easier to use, while it still produces very efficient code and is well suitable for system programming, IoT, etc. Unless object oriented programming is really needed, Rust can also replace C++: again, it’s safe, simpler (in a good sense) and easier to use. It’s also superior to Go in all aspects except for garbage collection. As Rust closes some remaining gaps, it should be a good fit for low to mid-level programming and it’s likely to become mainstream in certain specific and important areas. It’s good to be familiar with it.

It’s also important that Rust provides sufficiently powerful, although cumbersome, macro systems (yes, several ones, which is apparently workaround for the macro-unfriendly language syntax). This allows compensating for missing features in the language. It is actually used this way and prevents the language from becoming bloated or obsolete easily.

Rust was easy to learn. There is a very nice introductory documentation (especially The Book), the compiler is the most helpful and beginner friendly one I’ve ever used and there is a good integration with Language Server Protocol. Well, there are also some obstacles, most notably getting used to lifetimes, especially when being used to use languages equipped with garbage collection, but this is not a big issue.

There are also not so good things about Rust. The language is not very complete and relies on third party libraries to add missing features (for example, I had to use itertools, lazy_static and paste in my first Rust program to get the needed features). The language is also still significantly evolving, sometimes motivating library authors to use the newest features and requiring very recent versions of the compiler. The Rust ecosystem also follows the infamous modern trend of relying on a heap of libraries and the related dependency hell rather than having a curated standard library covering the most common needs. Haskell is better in all those aspects.

I’m also not much happy about the messy strings and CamelCase (fortunately used only in CapitalizedIdentifiers but still annoying).

Overall, Rust looks promising to me and I expect its growing usage in certain areas. I’m going to keep eye on it.

Software problem: Deleting Btrfs subvolumes

Internet is no longer full of Btrfs horror stories so I decided to use Btrfs when I reinstalled my desktop computer with NixOS. It has some nice features, like subvolumes, compression and cooperation with Podman. I still prefer ZFS but due to its weird license it’s not included in Linux and one can fall into trouble when using it as the root file system and something goes wrong with a kernel module upgrade (as happened to me couple of times in the past). Btrfs doesn’t have this problem so it’s preferable for me on desktop computers.

Now I experienced the first problem with Btrfs. When I tried to remove some containers and prune Podman images, I got complaints about files that couldn’t be deleted from the storage. This has led me to the fact that Btrfs permits non-root users to create subvolumes but not to delete them:

$ btrfs subvolume create $HOME/test
Create subvolume '/home/pdm/test'

$ btrfs subvolume delete $HOME/test
WARNING: cannot read default subvolume id: Operation not permitted
Delete subvolume (no-commit): '/home/pdm/test'
ERROR: Could not destroy subvolume/snapshot: Operation not permitted
WARNING: deletion failed with EPERM, send may be in progress

Huh? At least root can still delete the subvolume:

$ sudo btrfs subvolume delete $HOME/test
Delete subvolume (no-commit): '/home/pdm/test'

And removing the volume by simply removing its directory also works, even for a non-root user. But it looks like Podman wants, not surprisingly, delete the volume. I’m not the first one being hit by this problem. It should be fixed but it still doesn’t work for me.

There is a mount option user_subvol_rm_allowed, unfortunately disabled by default, mostly for legacy reasons. After I had added it to my / and /home subvolumes, deletion of subvolumes started to work.

But not so Podman. I was no longer able to create or rebuild containers. For example:

$ podman run -it --rm debian
Error: unlinkat /home/pdm/.local/share/containers/storage/btrfs/subvolumes/6a7bcf44cc576f9ba66b978fc7472380ba0a8e5db318d547f6484acd3e49995e/bin: permission denied

Should I delete the subvolume? Let’s try:

$ btrfs subvol delete /home/pdm/.local/share/containers/storage/btrfs/subvolumes/6a7bcf44cc576f9ba66b978fc7472380ba0a8e5db318d547f6484acd3e49995e
WARNING: cannot read default subvolume id: Operation not permitted
Delete subvolume (no-commit): '/home/pdm/.local/share/containers/storage/btrfs/subvolumes/6a7bcf44cc576f9ba66b978fc7472380ba0a8e5db318d547f6484acd3e49995e'
ERROR: Could not destroy subvolume/snapshot: Permission denied

OK, still not all right, so I removed the subvolume as root. But then:

$ podman run -it --rm debian
Error: stat /home/pdm/.local/share/containers/storage/btrfs/subvolumes/6a7bcf44cc576f9ba66b978fc7472380ba0a8e5db318d547f6484acd3e49995e: no such file or directory

Oops. It looked like the most efficient way to get rid of the mess was to remove all the containers, all the subvolumes and to rebuild the containers again (fortunately, I haven’t had anything valuable in them yet).

After some more trouble (even podman system reset hadn’t worked), I ended up with removing ~/.local/share/containers/ completely and Podman finally started to work.

OK, mostly problems of category 1. The lesson learned is to always use user_subvol_rm_allowed with Btrfs.

Software problem: Compiling Emacs vterm module on NixOS

I started using NixOS again some time ago. The fact that NixOS doesn’t use Filesystem Hierarchy Standard is sometimes a big source of pain and complications. One non-obvious thing for beginners is how to compile a C program.

One C program that is very useful and must be compiled locally is emacs-libvterm C module, providing a visual terminal for Emacs. There are several good reasons to use emacs-libvterm:

  • It’s sometimes useful to run visual terminal programs.
  • Many command line programs don’t work well with line terminals.
  • If a command line program uses its own command line interface, it’s occasionally useful to take advantage of it. Additionally, Eshell command line editing doesn’t work with programs that enter their own command line interfaces.
  • External, non-Emacs, terminal emulators work well but their capabilities are quite limited and they are not integrated with Emacs.
  • Built-in Emacs visual terminal emulators (term, ansi-term) don’t work very well and are unusable with many applications and utilities.

emacs-libvterm solves all the problems. It works, it can use all the Emacs editing capabilities and it is integrated with Emacs. On standard systems, emacs-libvterm compiles its module automatically, as long as libvterm development package is installed. But it’s more difficult on NixOS.

The first problem is how to compile a C program on NixOS. As far as I understand it, nix-shell environment must be used. The second problem is that libvterm package in NixOS contains something old and apparently unmaintained and libvterm-neovim must be used instead. It took me quite long to realize it, I wasted a lot of time investigating problems related to libvterm and possibly other broken NixOS packages. Which was a problem of category 1, my ignorance, combined with a problem of category 2, broken obsolete packages in the distribution.

So how to compile Emacs vterm module on NixOS? Assuming gcc and gnumake packages are already installed on the system, it’s needed to create the following shell.nix file:

with import <nixpkgs> {};
stdenv.mkDerivation {
  name = "emacsenv";
  nativeBuildInputs = [ cmake ];
  buildInputs = [ libvterm-neovim ];
}

Then nix-shell is run in the same directory. When Emacs is started in the given nix-shell environment, emacs-libvtem compiles and installs fine.

Software problem: Music players on Android

One might thing that making a program that plays music stored locally on a computer shouldn’t be that difficult. It must be able to find music files, to play them correctly, to play songs in the correct order, and provide a decent user interface, with some search capabilities. Considering that the most difficult tasks can be outsourced to libraries, there should be no special challenge. Yet all the music players I’ve tried on desktop computers or mobile devices suffered from significant problems of various kinds.

On Android, almost all the music players (or at least those from F-Droid) cannot read tags from music files correctly. That means some tags containing non-ASCII characters are read incorrectly, resulting in displaying album/artist/song names garbled and, what is worse, arbitrarily splitting songs of a single album to two sets, one with a garbled album name and one with the right album name. Additionally, the track numbers may be interpreted incorrectly, resulting in a wrong order of the songs. It looks like almost all Android music players use some common, broken Android library.

A special exception is Vanilla Music, which apparently uses its own, working tag reading library. But even this player is not without problems. It has already happened to me in the past that Vanilla Music couldn’t find new songs uploaded to a phone, whatever I tried, including rescanning the whole media library and reboots. And now I experienced another problem. This time, Vanilla Music search function stopped working, it couldn’t find anything in the displayed lists of artists, albums and songs. After rescanning the whole media library (which takes quite a lot of time), all the lists were split in two parts, sorted separately of each other (i.e. A … Z A … Z instead of A … Z) and one of them being searchable while the other one not. So I ended up with split lists and only one part of each searchable. After more rescanning and reboot attempts, I gave up.

A possible explanation could be that Vanilla Music got damaged by update of my phone from Android 10 to Android 11. So I uninstalled Vanilla Music, installed it again and it started working. Apparently a problem of category 3, a bug somewhere. The lesson learned is that whenever I experience problems with Vanilla Music database, I should reinstall the application instead of attempting to fix its database.