Monospace Mentor

AI took my job as a coder. Then it made me a better software engineer.

2026-06-29T09:10:00+01:00

I’ve been doing agent-supported coding for a few weeks now, and it definitely has changed the way I work. It’s not worse, just different. Which is remarkable, given that many programmers are having a disturbingly different experience: LLMs generating bad code, introducing bugs, even destroying data. I also read accounts from people lamenting skill atrophy, loss of job satisfaction, and a feeling of disconnection from their own codebase. It sounds like for these developers, the LLM became the mythical Monkey’s Paw that gives you what you want, and then takes away something dear to your heart.

I don’t want to find myself in this dark place eventually. I also don’t want to find myself stuck when LLM prices skyrocket, or the whole AI bubble bursts. The reason why I don’t think I will end up stranded is my attitude towards coding, or rather, to software engineering.

Passive and active guidance

Working with an agent is when good coding standards pay off. I got to enjoy that satisfaction before. It was when we introduced linting, unit and integration tests to our infra code a few years into our operation. We quickly realized how our confidence in the code we deployed was growing, while the amount of nightly emergency fixes was going down. Now, tests and linting rules are the guardrails that prevent agents from introducing regressions and becoming too creative with their new code.

Even with these guardrails in place, I don’t rely on the agent to find its way alone. My agent sessions look like pair programming, with the agent as the driver and me as the observer. I see other developers switch their attention to other things while the agent does its thing. I follow every step it takes. Not just because I don’t fully trust it. I also keep asking questions about its approach, and intervene to help it find the best solution. Every task results in a merge request that we review thoroughly before we integrate the changes into our codebase. (I just found a neat way to do these reviews in the terminal, about which I’ll write soon.)

Many years of experience have taught me that this kind of investment into “boring” engineering practices pays dividends. I don’t see why that should change with LLMs.

From programming to engineering

I don’t care how software is made, I care about what it does in the end. Loris Cro put it well in his article “My Software North Star:

Software should be useful to the end user
Software should be correct
Software should be maintainable and efficient

But how do you achieve that?

Before I started to use coding agents regularly, my many responsibilities at work left me little time to practice what I’d call “engineering discipline”. My process was: do a bit of thinking in advance, start coding, review, deploy. It did the job, but there wasn’t much “engineering” to it, and it always felt a bit shallow.

The software engineering methods I read about in the 80s — SA/SD, Jackson SD, SADT — are long dead, and deservedly so. But their intent didn’t die. Today’s methods have different names, and they tend to be leaner: discovery instead of requirements analysis, TDD instead of formal verification, ADRs instead of binders full of design docs. The problem is, they’re only “light” relative to 1985, but still not at all light for a team of two. Learning these engineering methods, let alone practising them regularly, always felt like a luxury we couldn’t afford. AI agents changed these economics.

Get it in writing

Like everyone, I started out by asking the LLM to implement small new features. (I found it hilarious how much better it was at Rails coding than me.) Soon, I picked up other people’s practice of ending prompts with the question of what else it needed to know. This alone eliminated a lot of backtracking and do-overs, and evolved naturally into having the agent generate a comprehensive plan first, and implement that plan in a second phase.

Plans require decisions, during both their inception and their implementation. Realizing the advantage of documenting our technological direction, I started dabbling with Architecture Decision Records a few years ago. ADRs are documents that preserve the thoughts behind decisions with significance for the shape of a software. Usually, those are also the kind of decisions that are difficult to revert later. In 16 years at the same company, I learned to appreciate the value of organizational memory. If only writing those documents wouldn’t take so much effort, especially if you’re not writing in your native language! I don’t think I ever wrote more than a handful. Thanks to a coding partner that takes great care of that, I now have a dedicated repository just for ADRs.

Once I got more proficient at guiding that coding partner, my next step was to break out of the feature-building box. I started involving the agent even before launching new projects. An extensive back-and-forth that I call “crucible sessions” results in clear documentation of a project’s motivations, goals, and most importantly, its scope. Staying focused has always been a struggle for our team. There’s always the temptation of tackling random issues we discover during project work. It’s easy to rationalize the distraction as “a good opportunity”. Starting with a clear definition of a feature and its scope helps us tremendously with shipping it without detours. In the process of refining this workflow, I learned that this kind of document is called a PRD. The agent saves them on Linear as project documents.

You might notice a pattern: all these improvements preserve knowledge and create transparency. This pattern now repeats like a fractal on all levels of my work. I’m writing less code, but I’m creating more value.

There’s more than just code

Like Mark Betz, “I don’t miss writing code”. I don’t feel disenfranchised by the agent that does it for me.

It’s probably an identity thing. My job has never been to deliver code, or to provision servers, or to type words into a blog editor. My job is to keep our business on a healthy trajectory, technologically and otherwise. I’m enjoying being able to do that job more effectively than before.

Did it turn me into an AI Booster? Hell no. I struggle to reconcile the many quandaries around AI usage with my family’s addiction to food and shelter. My message is not “Use AI, it’s great!” It’s: “When you have good software engineering practices, everything works better.” With or without LLMs.

I don’t believe this new era is “Game Over” for developers. It’s a forcing function to get clear about what we’re building, and to communicate well how we’re doing it. If that resonates with your situation, The Server Room is where we can figure it out together.

The motion that changed everything

2026-06-21T16:42:20+01:00

I switched to Vim a long ago after using a series of GUI editors like TextMate and Sublime Text. The urge for novelty has me still try out GUI editors sporadically, the most recent one being Zed. But I always come back quickly. I’m doing all my engineering work from the terminal, and by now my tooling is more or less complete and highly optimized for efficiency.

What brought me to Vim was the thought, “It’s installed on all my Unix and Linux servers, I should have at least some basic proficiency.” What made me stay was its “command grammar”; first and foremost, Vim motions.

I have to admit that even after decades, I only rarely use the number-based motions. Instead of 4dd, I mark lines for deletion in visual mode. And although I have relative line numbers switched on, I still end up holding k like an animal.

What makes Vim great for me are the motions that are based on text. For example, quickly jumping to the next colon with f:. Or selecting the current word with * and immediately being able to jump to its next occurrence with n.

Another neat feature isn’t exactly part of Vim motions, but text objects are closely related. It’ll never stop to make me smile when I can edit the contents of a pair of quotes or parentheses as easy as ci) or da".

BTW, if you’re looking for a comprehensive Vim tutorial, try the freely available chapter “Text Editing Fundamentals”, part of my community course “Basic Linux System Administration”!

This post was inspired by and written for “Hyde’s Vim Carnival”.

Fixing Flatpak Obsidian CLI

2026-05-08T21:18:11+01:00

I just tried to use the new CLI tool for Obsidian for the first time. To set it up, enabling a toggle switch in the “General” section of the settings should suffice. But if Obsidian is installed on Linux as a FlatPak, all you get when you execute the command is:

$ obsidian
The CLI is unable to find Obsidian. Please make sure Obsidian is running and try again.

The problem is that FlatPak isolates each application for safety reasons. In consequence, the Obsidian socket gets set up, but not at the location where the CLI expects it. This socket is a special file through which the CLI tool and the Obsidian application communicate. That’s why the CLI suspects that Obsidian might not even be running, even if it is.

The fix is simple: We put a symbolic link where the CLI expects the socket, and let the link point to where the socket actually is:

ln -sf /run/user/$(id -u)/.flatpak/md.obsidian.Obsidian/xdg-run/.obsidian-cli.sock /run/user/$(id -u)/.obsidian-cli.sock

With this fix, the CLI is happy:

$ obsidian version
1.12.7 (installer 1.12.7)

Now I can explore my Obsidian vaults from the shell!

By the way, you can read my DevOps notes that I manage in Obsidian right here on my website!

And if you’d like to learn more about symbolic links and the Linux filesystem in general, check out my course “Basic Linux System Administration”.

Going back to the roots for our community chat

2026-04-26T16:18:43+01:00

When Discord announced the introduction of age verification, my first thought was, “That’s what I’ve been preparing for.” I logged into one of my machines and set up an IRC server.

Like many open-source communities, The Server Room has been using Discord as a chat platform for years. But I always felt like I was building on a brittle foundation. I knew that it was only a matter of time until I had to find a more future-proof solution for our real-time conversations.

When Discord announced the introduction of age verification, potentially requiring official identification and photographic evidence, that time had come. Many of our community members are working in (or adjacent to) cybersecurity. There was no way they’d be happy to provide deeply personal information to an opaque business (or to its even more questionable business partners) just to have a chat.

Even though Discord backed off their initial plan, I’m happy that I made the change.

The Problem With Building on Someone Else’s Land

The barrier to entry to Discord is low, for server operators, and even more so for community members. In comparison to other chat platforms, its less ephemeral “forum” channels that retain messages for later reading seemed ideal for a community platform that is all about sharing and preserving knowledge. But there’s a catch. (There’s always a catch.)

Discord is a business, businesses need to make money, and it was clear from the start that it won’t offer its service for free without any downsides forever. Discord looks like the ideal candidate for the Enshittification effect - a sudden push for monetization, harming its users once it reaches a certain level of popularity and funds starts running out.

The TSR community is at the heart of my whole operation. Not only is my teaching work funded by membership fees, it’s based on connections between humans. Members don’t just study my course materials in isolation, they help each other grow. I can’t build this community on a product that can take a nosedive at any time. Joan Westenberg wrote about this kind of dependency in “The creator economy trap”. I’ve seen communities switch platforms for all kinds of reasons, and every time, they lost members in the process. For a fledgling community like mine, this could be an extinction-level event. Owning your community platform not only impacts cost and control, it also is the prerequisite for continuity.

For that reason, I set up a forum using the popular open-source software Discourse (the similarity in names is so confusing) in January 2024. Other than Discord, the forum gave me confidence that the useful conversations in my community would be preserved and easy to retrieve at any time. Within a short time, the forum became the centre of our community’s activities. With this part in the bag, real-time chat needed a reliable solution, too.

Why IRC, Though?

While Discord dominates the market of chat solutions, there are alternatives. When you look for open-source solutions, you’ll find Matrix, Mattermost, Zulip, and Discord clones like Stoat. And even among proprietary group chat services, there are well-liked options such as Signal. Why on earth did I choose the oldest chat platform of them all?

One reason is certainly that I’m even older than IRC, and I used it daily in the early years of me having access to the internet. The experience of being able to chat with people in distant countries was magical, and it informed my understanding of the internet as a “global village”.

While I did receive a gentle mocking as “the old sysadmin” when I mentioned IRC as a chat alternative, there was little pushback. People rather seemed intrigued that IRC might still have its place in internet-based communication.

As a first experiment, I created a channel on Libera.Chat. Additionally, I set up a bridge between it and our main Discord channel, et voila: people were able to chat from their IRC client applications. This only took me an hour, and most of that went into the initial bridge configuration.

Another reason for my choice was that setting up an IRC server is easy. With a modern implementation like Ergo, you can get started within an hour. You don’t have to deploy a web application backed by multiple database services, there’s no need for a complex Docker stack.

Then, IRC clients are available for every popular desktop, tablet and phone OS. In addition to the GUI apps, there are (of course!) well-maintained terminal-based ones as well. I’m using weechat, for example.

Finally, IRC is still actively developed. The latest protocol revision IRCv3 brings quality-of-life improvements such as being able to retrieve chat history when connecting. With earlier IRC versions, you’ll either miss what happened in chat while you were offline, or you’ll have to set up a client application (also known as a “bouncer”) that keeps a continuous server connection. Ergo has chat history enabled by default, and modern clients such as Halloy take advantage of it.

And as a bonus, I just found out that the IRCv3 project has a Code of Conduct, which is a good sign that they’re a reliable part of the online community.

All this, together with lively IRC communities like Libera.Chat, proves that IRC is still going strong after four decades. What’s not to like?

The technical details

Getting the IRC server started on one of my virtual private servers didn’t take longer than it took setting up our Discord server back in the day. Launching a Discord server takes a few mouse clicks. Installing Ergo from the Fedora package repository took a few keystrokes.

I admit that I took a detour setting up TLS that cost me another hour. First, I tried to put the IRC server behind the Nginx Proxy Manager that I already had running for serving websites, but I couldn’t get the Gamja web interface to connect successfully. From what I found online, IRC requires a few esoteric proxy settings that are easier to configure when you can modify the nginx configuration directly, instead of having to use the limited options of the NginxPM web interface. In the end, I configured Ergo to open a TLS port itself, and installed Certbot to handle the certificate renewal with Let’s Encrypt.

Installing Gamja was a quick job, and here I was able to successfully employ NginxPM. Thanks to Gamja, community members who don’t want to install a local IRC client have the option to access chat from their web browser.

Matterbridge is a nifty self-hosted service that can cross-post messages from one chat service to another. That way, there’s no split between the people adopting IRC and the ones still stuck on Discord. Discord and IRC are only two items in the long list of services that Matterbridge supports. Since I had already been using it to bridge our channel on Libera.Chat, I just had to modify the connection details to switch to our own IRC server. Then, I added bridges for two more channels, just to have the most important ones covered.

I used the Discord API and a few bots to build admin workflows, posting of news and social media messages, and a few other fun features. IRC has been using bots for the same purposes for much longer. With Limnoria, I was able to replace all the features I wanted to have from the start. The list of available Limnoria plugins is long and provides a lot of useful (or simply fun) features. Here are some of the plugins I set up myself:

SedRegex

allows you to correct a previous message using a quick

s/what you typed/what you meant to type/

. (Did you know that Skype had this built in?)
With

Later

, you can leave offline members a message that they’ll see when they post their first message after returning.
The

Karma

plugin keeps tally of up- and downvotes for a member or another thing. A neat way to show appreciation or annoyance.
The 3rd-party

Grapnel hooks

plugin installs a web API that I can use to post messages from the outside, mainly from automations I build on

Make.com

.

In total, I spent less than a workday on getting our new community chat platform to a sufficiently usable state. That’s perfectly okay, don’t you think?

Conclusion

With little effort, I was able to set up an online chat platform for my DevOps community that gives members a choice of either the convenience of Discord, the flexibility of their own IRC client, or the ease of an IRC web interface. In case I have to ditch Discord after all, there’ll be a well-established alternative chat service in place already. It’s nice to be able to focus on our conversations without worries.

And you’re welcome to join them, either on IRC or on Discord!

The systemd age attestation controversy

2026-04-01T13:29:03+01:00

In yesterday’s Office Hour, we discussed systemd, especially systemd timers. Then the question was raised why there is so much backlash against systemd in some circles. Here’s a short summary of my answer:

systemd is becoming kind of an “OS within the OS”. That this runs counter to the Unix philosophy’s “small programs working together” rubs people, especially the more puristic ones, the wrong way.
systemd author Lennart Pöttering has strong opinions, and some people seem to think that they have to offer even stronger counter-arguments, to put it lightly.

Technical differences can and should be debated. We have a wealth of talented developers in the open-source community. By putting their heads together, they have created, and will keep creating, remarkable pieces of software.

What should never, ever, happen, is that developers get death threats for their decisions. That actually happened after Dylan M. Taylor submitted a pull request to add a field to store (NB: and only store) a user’s birthdate in systemd’s JSON user records. Here’s a gift link to an LWN article that lays out the whole story well.

I find behaviour as described in the article absolutely unacceptable. In Open Source, we’re a community, and mob mentality has no place in it.

Bringing Linux Administration to Everyone: Free Online Course Starting Soon

2025-09-02T11:58:30+01:00

I had planned to start the cohort-based course “Basic Linux System Administration” later this week. When this cohort unfortunately didn’t fill up, I realized this was the perfect opportunity to do something I’ve wanted to try for a while: offer my Linux System Administration course completely free as a livestreaming experience.

Starting this month, I’ll be delivering my complete “Basic Linux System Administration” course through weekly live lectures on my streaming channels. This isn’t just a recording of classroom sessions—it’s a fully interactive learning experience designed specifically for the online format.

What You’ll Get

The course covers everything you need to get a solid foundation for a system administration or DevOps role. We’ll start with the fundamentals—installing Linux, mastering the command line, and understanding how Linux actually works—then progress through practical skills like user management, process control, networking, and security.

Each week includes live Q&A sessions where you can get direct answers to your questions, plus assessment quizzes to verify your understanding. The hands-on labs use real Linux systems, so you’ll gain practical experience with the same tools used in production environments.

Why I’m Doing This

After decades in software development and managing Linux systems, I’ve seen how transformative these skills can be for someone’s career. The demand for Linux expertise continues to grow, but quality education shouldn’t be locked behind high course fees.

This streaming format lets me reach people who might not otherwise access this training—whether due to location, schedule constraints, or budget limitations. Plus, I’m genuinely excited to experiment with live online education and build a learning community around practical Linux skills.

Get Started

All the details are on my course page. You can sign up there to receive notifications about lecture schedules and other important updates.

The first session covers Linux fundamentals and walks you through your first Fedora installation. Whether you’re entirely new to Linux or have some experience but want to formalize your knowledge, this course will give you the solid foundation you need for a successful career in system administration.

Let’s build something great together. See you in the stream!

—Jochen, The Monospace Mentor

A homelab is a skill booster

2025-08-09T12:50:04+01:00

In her article “What’s the biggest unsolved problem within Site Reliability Engineering?”, Paige Cruz identifies a critical challenge facing our industry: how do we pass the hard-won knowledge of veteran system administrators down to a new generation that has never worked outside cloud infrastructure?

Cruz makes a compelling case that this knowledge transfer gap represents SRE’s biggest unsolved problem. However, I take issue with one of her conclusions. She argues that “It’s also not acceptable or sustainable or accessible in my opinion to continue to expect people set up home-labs and experiment themselves. We don’t ask veterinarians to practice surgery in their free time at home and I don’t think we should expect that from future SRE.”

I respectfully disagree. A home lab represents the best investment a future SRE can make to secure their career growth.

The home lab advantage

The veterinarian comparison doesn’t hold up under scrutiny. Veterinarians need live animals, surgical equipment, and sterile environments – resources that are neither practical nor ethical to maintain at home. Particle physicists need billion-dollar accelerators. But what does a sysadmin or SRE need to experiment with infrastructure? A few decommissioned Lenovo ThinkCentre boxes and a network switch.

The barrier to entry is remarkably low, and the learning potential is enormous.

Scientific research consistently shows that skill development requires more than passive consumption of learning materials. Growing expertise demands practical exploration – running into problems, analyzing failures, and working through solutions. A home lab provides exactly this environment, free from the constraints and pressures of production systems.

You don’t need a data centre in your basement. One or a few small networked PCs will serve you well. These machines consume minimal power and don’t need to run around the clock. Fire them up when you want to experiment with a new service mesh, test disaster recovery procedures, or break something just to see what happens.

For those who can’t or don’t want to maintain physical hardware, alternatives exist. “Pubmox” – the shared Proxmox Virtual Environment I maintain for members of “The Server Room” – offers free access to virtual infrastructure for hands-on learning.

The learning laboratory

A home lab creates space for the kind of curiosity-driven exploration that builds deep technical intuition. When you’re troubleshooting a networking issue at 2 AM in your home lab, you’re not worried about affecting customers or violating change management policies. You can take risks, make mistakes, and learn from the consequences.

This environment encourages the experimental mindset that separates great SREs from those who merely follow runbooks. You learn to ask “what if” questions and then actually test your hypotheses.

Knowledge transfer requires community

While I advocate strongly for home labs as skill boosters, they’re not the complete answer to Cruz’s original question about knowledge preservation. The solution to transferring veteran sysadmin wisdom doesn’t lie in solitary exploration – it requires collaborative knowledge transfer.

Home labs excel at building technical skills, but they can’t replace the mentorship and context that comes from working alongside experienced practitioners. The real magic happens when hands-on experimentation combines with guided learning from those who’ve solved these problems before.

This is why I’m launching “Basic Linux System Administration” – an instructor-led course backed by “The Server Room” community. Participants get free access to Pubmox, allowing them to practice what they learn while benefiting from my decades of system administration experience.

It’s the best of both worlds: the freedom to experiment and break things, combined with the guidance to understand why systems fail and how to make them more reliable.

A home lab won’t solve SRE’s knowledge transfer problem on its own. But for individual practitioners looking to deepen their skills and advance their careers, it remains an invaluable investment. The combination of practical experimentation and collaborative learning creates the foundation for the next generation of systems experts our industry desperately needs.

Linux Shell for Beginners: The ls command

2025-05-21T09:06:13+01:00

Hello DevOps friends! In this tutorial, we’re going to take a look at the ls command.

There is no Unix or Linux user who doesn’t use it on a daily basis, so I’ll teach you what you need to know.

The ls command is how you see what’s on your Linux disks. Type ls in your terminal, and it shows you all the files and folders right there. It’s usually one of the first commands you learn. Pretty simple, but super useful for finding your way around. This does not sound spectacular, right, not something to make a tutorial about?

Well, I’ve been using Linux since 1993. Would you believe that in preparation of this tutorial, I still learned something new about ls? So, even if you are a veteran Linux user, you might take away one or two interesting titbits from this.

And at the end, I’ll introduce you to a command that takes a fresh and even more powerful approach to what ls has been doing for decades.

So let’s dive right in!

Intro

Because it’s so essential, the ls command is included in the Core Utilities package that gets installed right when you set up your Linux distribution of choice.

This is how ls works by default: When you execute it on its own, it will list the files and folders in your current working directory.

Hidden files, as the name implies, will not be shown. On Linux, all files whose names start with a dot are treated as hidden files.

As you can see, the output of the command is sorted alphabetically, and it’s not immediately obvious which names are files and which are directories. We can improve this by using command options.

Options are additional arguments that you add after the command name. They change its behaviour, and they all start with a dash.

Long listing format

The option -l requests the “long listing format”. This format lists each file and directory on its own line with a whole lot of detail information. I’ll cover those details later.

Let’s see what the long listing format looks like.

That’s quite a bit more information than before! But apart from the file name and the modification date, it’s not quite obvious what all of this means. I’ll point it out for you.

Every line in the long format starts with a series of letters and dashes. It represents the file type and its access permissions. We’ll get to them in a minute. The solitary number that follows is the number of links related to the file. You won’t need this information often. The next two items designate the owners of the file. Every file can be assigned to one user and to one user group, and they are both listed here. Finally, there’s the file size and the last modification time before the file name is printed in the last column.

Let’s tease apart what’s hidden in the permissions pattern at the start of the line!

File permissions

The file permissions field is a combination of characters, and it consists of several groups, each of which has its own meaning. The first single character signifies the file type. A d stands for “directory”, while regular data files are represented with a dash -. There are other types as well, but they’re not relevant at this stage.

Following the file type character, there are three groups or sections of three characters each. They display the file permissions. The first section shows the permissions of the owner user, the second one the permissions of the owner group, and the third section the permissions of everyone else. This last section is often referred to as “others”.

We can see that each of these sections is composed of the letters ‘r’, ‘w’, ‘x’, and the dash ‘-‘. What do they mean?

The ‘r’ stands for ‘read’ permission, which allows users to read the contents of the file or directory.

Similarly, the ‘w’ allows users to make changes. In case of a directory, making changes to the contents of it means creating and deleting files inside it.

The ‘x’ permission on a file makes it executable. With this permission, you can execute it from the Linux shell like any other shell command. In the case of a directory, the ‘x’ permission has a slightly different meaning. It allows users to enter the directory and thus make it their working directory.

If one of these permissions is not given for a specific section, a dash ‘-‘ takes its place.

These three permissions can be combined freely, but only a few of these combinations are commonly used. For files, you usually see r-- if a file is read-only, or rw- if write permission is given. For executable files, the x pops up in the third position. It’s similar for directories. Usually, the x permission goes hand in hand with read rights, so the two common combinations are r-x and rwx.

(Internally, these combinations are represented as octal numbers, but that’s something I’ll cover in detail in my Linux courses, not here.)

Hidden files

In the early days of Unix, the operating system developers noticed that more and more system and configuration files were accumulating in their home directories. Some of these files even contained sensitive information, for example, login credentials to some FTP server. To minimize exposure and clutter of the directory listing, the fathers of Unix devised a simple solution: files whose names started with a dot would be treated as hidden files and not displayed by ls if not explicitly requested.

This explicit request has the form of the -a option.

On closer inspection, you will notice that among the files this option reveals are always two special directories whose names only consist of dots. The “single dot” directory always represents the directory it’s located in. It allows the user to explicitly reference the current working directory. Similarly, the “double dot” directory stands for the parent of the current working directory. It makes it easy to reference or even switch to the directory one level higher in the directory tree.

Human-readable file sizes

Among other things, the long listing format also shows the file size. However, it shows it in bytes, and for large files, that’s impractical. “Wait, is that number in the million or in the ten million bytes range?” With the -h option, you get file sizes in human-readable form, with useful units like MB or GB.

This option only works in conjunction with the -l option because otherwise, there’s no file size to display.

As I mentioned earlier, you can pass multiple options to ls as separate arguments, or you can combine them into a single multi-option. In both cases, the order in which you pass in the options will not matter. ls -lh works identically to ls -hl, for example.

Sorting

The default sort order by name is useful when you’re looking for a specific file. However, other administration tasks will require a different order. ls has a few options in store.

With -t as in “time”, ls will sort the file list by modification time, starting with the newest file and ending with the file that hasn’t been touched in the longest time.

When you’re looking for candidates to delete to free up storage space, the -S with a capital S comes in handy. It’ll list files by size in descending order.

Speaking of order, you can always reverse the sort order you picked by adding the -r option. For example, ls -lr will give you a listing in long format in inverse alphabetical order.

Recursive listing

Sometimes, you need to see the contents not only of your current working directory but also its subdirectories. Maybe you’re looking for a specific file that’s in there somewhere. Maybe you just want to see how much stuff there is.

The -R option with a capital R will help you with that. The R stands for “recursive” because ls will go down into the first subdirectory and list its contents, then further down until it reaches the bottom, and then will do the same with any other subdirectory until it has traversed them all. Depending on the size of the filesystem hierarchy you’re in, this can take some time and yield quite a lot of output.

Output in colour

At the beginning of this tutorial, I highlighted the fact that with the default output of ls, it’s hard to tell which names are files and which are directories. Colour coding makes it easier. To enable it, use the special option --color=auto. (Note the American spelling with just an “o”.)

You can even change the default colour scheme, but I’ll leave that as a research exercise for you.

Listing specific files and directories

The ls command accepts not only options as its arguments but also file and directory names. When you pass in the name of a directory, ls will list its contents. In the case of a file name, you’ll see its details. Of course, if you don’t also pass in the -l option, the only detail you’ll see is the file’s name; that’s not very useful.

Wildcards

Not always can we be totally specific in naming a file or directory. Maybe we only remember the beginning of the name. Or maybe we just want the details of files of a specific type. This is where wildcards come in, or as they’re commonly named, “globbing characters”. They’re characters with a special meaning that act as placeholders for a larger set of possibilities.

The most simple globbing characters are the asterisk and the question mark. The latter stands for any single character, the former even encompasses any number of arbitrary characters. Note that this number may be zero.

For example, ls a* will list all files whose name is “a” or starts with “a”.

ls a? will show you any file whose name has two characters, the first of which is an “a”.

You can, of course, combine both wildcard characters. The output of the command ls *.?? will be a list of any files that have an extension consisting of two characters, for example abstract.md.

If we don’t just want to allow any character but a specific set, we can pass it to ls in square brackets. ls a[12] will only match the file names a1 and a2.

Character sets can also be specified as ranges. For example, the set 0-5 in square brackets will match any digit between 0 and 5. However, as you can see, there isn’t an a4 or a5.

And by the way, this “globbing” is actually not built into the ls command. It’s a feature of the shell from which you execute ls. But that’s a topic for my Linux and shell scripting courses.

As you already know, the command ls -l followed by a file name will display detail information on the file. However, with a directory name, it will list the details of all the files inside the directory. This can be confusing when you’re working with wildcards because it’s hard to predict if there are matching subdirectories. To make sure that you’ll get filesystem details on both files and directories, you add the -d option.

As you can see, the first command listed the details of the contents of matching directories, while the second only showed their own details.

Quiz

Let’s do a quick quiz! Take a look at this combination of options. Can you tell how ls is going to behave?

This command will list all the contents of the directory /var/log; you can tell that it’s a directory because the path argument ends with a slash. The listing will be in long format with human-readable file sizes. And since it’s sorted by time in reverse order, the files that were changed most recently will be at the bottom.

Advanced options

There are a number of advanced options that you won’t use as often but are useful to know.

If you want to know the inode related to a file, use the -i option. If you don’t know what an inode is, don’t worry about it. I’ll cover the topic in my Linux courses.

The option -F with a capital F appends indicator symbols after each file name to make discerning its type easier. It adds a slash (/) for directories, an asterisk (*) after each file that is executable, and an at sign (@) after each symbolic link. There are other symbols for lesser-used file types that I’ll omit here.

There is also an option that doesn’t have a single-letter form but is useful nonetheless. If you want to see directories listed separately from regular files, try --group-directories-first.

Aliases

Once you work in a Linux shell more often, you’ll notice quickly that you’re using ls very regularly with certain sets of options. Therefore, it’s useful to define shell aliases for them so you can execute them more easily using a shorthand command.

Some of these aliases are so common that they might be already part of the default configuration on your Linux distribution. If not, I recommend you add them, since, for example, typing ltr is much quicker than typing ls -ltr.

Here are a few more suggestions for useful aliases.

lsd gives you the details of not only files but also directories that you name in the arguments.

The lsz alias is easy to remember; it focuses on file size, after all.

In the lsx alias, you’ll find an option that I haven’t covered up to now. The -X option with a capital X does something very useful: It groups files with the same extension. For example, if you have a directory full of different file formats, lsx will give you a quick overview.

Alternatives

As you can see, ls can do a lot of things. But it’s by far not the only command you can use to explore your storage systems. Let’s look at a few companions, or even replacements!

With tree, you can get a quick overview of the directory hierarchy. In practice, you might start with a quick invocation of tree, and then use ls to zoom in on one or more specific items. Since it’s not installed by default on my machine, I have to install it first.

I guess it’s not hard to guess what the find command does.

And it does it well! Instead of trying to scan the output of ls to find the needle in the haystack of your filesystem, you can tell find exactly what you’re looking for, and it’ll go and search it for you. I’ll cover the find command in its own tutorial.

Finally, there are also modern implementations that cover the functionality of ls and then some. lsd is one of them. Note that this is not the shell alias I mentioned earlier. It’s a command that you have to install yourself.

The same goes for exa, and I’d like to give you a few more details on this useful add-on command.

exa is a modern alternative to ls with enhanced features. It supports all of the common ls options. Additionally, it comes with useful features enabled by default. For example, it can use colours and file type icons to make its output much easier to scan.

Like many modern takes on traditional Unix commands, exa is written in Rust, so you don’t have to worry that its bells and whistles come at the expense of performance.

You can install exa from Github, but by now, it’s found its way into many of the common Linux distributions. On my Debian system, I can simply install it via apt install exa.

By the way, that’s the reason why I’m still covering exa here, although unfortunately, it’s not maintained any more. The community has already filled the gap with a fork named eza with a “zed”, but its installation isn’t quite as straightforward yet.

Let’s look at a few examples of what exa does better than ls.

exa has its own long output format, of course. But it includes additional information that ls doesn’t provide. In this example, it outputs a list of files that has an explanatory header at the top and, in addition to the familiar details, includes each file’s inode number. As DevOps people, you will especially appreciate that it also shows each file’s current Git status.

I mentioned the tree command earlier. exa can replace it, too.

In this example, it traverses the directory hierarchy up to two levels down, and display all its files with their details as a neatly formatted tree.

Conclusion

All right! That’s the ls command. It’s an essential command on every Unix and Linux system. Its options allow you to customize its output. Check out the system manual page for ls to learn more. With shell redirection and pipes, you can process this output further. Check out my Linux courses on how this works exactly.

And with modern re-implementations like eza, you get additional features such as file type icons, tree display and colour shading by file size.

That’s it for this time; I hope that it was useful for you. For more DevOps learning content, check out my website monospacementor.com.

If you’d like to grow your skills quickly with the support of me and other DevOps practitioners, consider becoming a member of The Server Room, my online community. Your membership makes it possible for me to do more teaching in the open source community. For details on all the learning content and community events waiting for you, visit theserverroom.sh.

Our community forum is the ideal place to ask any questions you might have.

Until next time! Take care.

A community that becomes your cohort of mentors

2025-05-06T12:57:42+01:00

I do offer mentorship and coaching, but I’m well aware that 1:1 coaching isn’t for everyone. That’s why I was nodding heavily while I read Jacob Evans’s article “Mentorship To Me” earlier today. In it, he explains that he learned from a lot of people, but most weren’t “people who signed up for the job”. What he means is, they didn’t call themselves “mentors”. They simply chose to share their knowledge online, and were happy to take others along on their own learning journey.

Jacob appreciates that he can tap many sources to feed his personal growth:

Being part of Cloudflare, T3 ecosystem, Theo’s, Kent C Dodd’s, and the various dev communities I’m in has taught me that the best mentorship is often ambient, it’s absorbing knowledge from many sources rather than one guru.

This is exactly why I founded The Server Room, my DevOps community of practice, two years ago. I’m on our community forum and our chat every day to help community members grow. But this community is different to my Office Hour live streams, where people come to ask questions and I try to give them useful answers. Our community has a far more complex and beneficial structure of knowledge flow and peer support.

The following list contains just a few examples of the interactions happening in The Server Room. None of them require anyone wearing a “mentor” label.

On the forum, a community member shares their growth goals. Others not only give helpful pointers, but also follow up later to hold them accountable.
During a “Hot Seat” session, a community member presents different options they’re considering as a solution to a problem. The other participants give them feedback based on their own experience.
In chat, a community member shares their latest learnings, and others congratulate them on their progress. They also suggest possible next steps to tackle.
A community member writes a forum post about the fun they’re having with a certain technology. Someone replies that it gave them the confidence to finally give it a try themselves.
During a community workshop, a small sidenote turns out to be the missing puzzle piece that a participant had been searching for weeks.
A community member describes a conflict they’re having at work. Others who had similar experiences provide them with both emotional support and actionable advice.

When people join The Server Room, they don’t sign up to a support forum where some lone veteran engineer answers their questions. They join a community of peers from which they can get help, and in turn are expected to make their own contributions themselves. This way, we create the proverbial rising tide that floats all boats.

Jacob put it really well in his conclusion (emphasis mine):

So yeah, traditional mentorship has its place, but don’t sleep on the power of community-based learning and peer networks. Sometimes the best guidance comes from the people right next to you in the trenches.

If this insight resonates with you, as it did with me, please consider joining The Server Room!

Self-Hosting Web Applications Using Podman

2025-02-25T15:52:36+00:00

Container-based deployment is a neat way to self-host an application. However, I always try not to go overboard with complexity. That’s why I won’t immediately join the chorus of Kubernetes disciples when members of my DevOps community ask me how to deploy container workloads. Searching for viable alternatives, I recently explored the much simpler alternative of Podman and Quadlet during a few of my live streams. In this post, I will tell you about the results:

why I use a URL-shortener
why I host it myself
how to run Kutt in a Podman pod
how to expose it to the internet using Nginx Proxy Manager
how to manage it all using

systemd
how to safely store secrets in Podman

Are you with me? Let’s go!

Why I use a URL shortener

In my blog posts, course materials, and in chats, I use links to other useful information all the time. But their URLs are not always easy to type or remember, neither for me nor for my audience. A URL-shortener is a useful service that allows me to construct URLs that are much easier to memorize and type. Usually, they consist of one or a few simple words, prepended with a short domain name. The service then redirects visitors of these short URLs to the actual content. It’s a quality-of-life improvement for everyone.

Why I run it myself

URL-shorteners are not new. They started to be widely used when Twitter popularized micro-posts as a new type of social media. Its constraint with the 280-character limit discouraged users from including long URLs. Google’s introduction of UTM Tracking via query parameters made this issue even worse for marketers. URL-shorteners like bit.ly offered a workaround.

So why do I not simply use one of the existing URL shorteners? After all, I usually try to resist the NIH Syndrome, and rather make running services I won’t monetize Someone Else’s Problem™.

Well, turns out that those other people increasingly tend to make it my problem again. The two most common ways they do so is The Sunset and Platform Decay (aka Enshittification). Last year, Google announced the shutdown of their popular URL shortener goog.le, which will cause a tremendous amount of “link rot”, which is a term describing links in existing web documents turning invalid and unusable. And just the other day, bit.ly informed its users that links created with its free plan will soon stop sending an immediate redirect and instead show an interstitial page with ads.

In this day and age, we can rely on corporate web products less and less, which is why we have to take matters into our own hands. “Self-hosting” is the way to go if we want to stay in control of how our web content is distributed.

Run your own URL shortener with Kutt

Kutt is a great open source URL-shortening service that you can host yourself. You can use it to define your own shortcuts, and you can also modify an existing shortcut’s destination URL. So not only does the longevity of your URLs with Kutt depend solely on how long you’re willing to run your own service, it’s also extended by your ability to update them when their target URLs need to change.

Running Kutt is straightforward by using its container image. I’m using Podman to run it on a cheap virtual private server (VPS) on Hetzner Cloud. I chose Podman because it requires the least amount of effort and resources of all the container deployment solutions I’ve used so far.

In addition to Kutt, we also need a proxy server that exposes the service on the web safely with TLS encryption. We’re going to use the popular Nginx Proxy Manager for this purpose. It comes with a nice web UI that’ll make it easy to add more services at a later time.

Setting up Podman

By now, most current Linux distributions provide Podman out of the box. You will probably be able to install it simply using your distribution’s package manager. I installed my VPS with Fedora because it provides very recent versions of the tool.

A Kutt in a pod

For the deployment of Kutt, I found a well-built Compose file by YouTuber DB Tech. Compose files are a neat way to define a set of containers that together form an application. In this case, we have containers for the Kutt web application and for the Redis database it uses for caching. Here is the full Compose file. It’s in YAML format and defines two “services”, server (the Kutt web application), and redis.

services:
  server:
    image: kutt/kutt:latest
    volumes:
      - db_data_sqlite:/var/lib/kutt
      - custom:/kutt/custom
    environment:
      DB_FILENAME: "/var/lib/kutt/data.sqlite"
      REDIS_ENABLED: 'true'
      REDIS_HOST: redis
      REDIS_PORT: 6379
      JWT_SECRET: 0F5B2J0CcoaFIDehV14ngwd6Lp2YbyJM7YeFVkILB1 #random string of characters
      DEFAULT_DOMAIN: yourwebsite.com #no http or https
    ports:
      - 3000:3000
    depends_on:
      redis:
        condition: service_started
  redis:
    image: redis:alpine
    restart: always
    expose:
      - 6379
volumes:
  db_data_sqlite:
  custom:

I copied this definition to a file named compose.yml in a project directory I called kutt.

Inside the original file, I had to make a slight alteration to line 9 with the REDIS_ENABLED setting because of an issue that requires boolean values to be quoted when using Podman.

Podman’s compose sub-command supports Compose files by invoking either docker-compose, if it’s installed, or podman-compose. We’ll use the latter because it supports “pods”. Podman can group the service containers of an application in a so-called pod, just like its bigger cousin Kubernetes. Containers inside a pod can talk to each other via localhost, which simplifies many things. (And complicates others, as we’ll see later.)

First, let’s make sure we’ll be using podman-compose:

export PODMAN_COMPOSE_PROVIDER=/usr/bin/podman-compose

Now we can start the pod:

podman compose --in-pod true up -d

Because I named the project directory kutt, Podman automatically created a pod called pod_kutt:

$ podman pod ls
POD ID        NAME        STATUS      CREATED     INFRA ID    # OF CONTAINERS
94387ff1aabd  pod_kutt    Running     2 minutes ago              2

Inside this pod, Podman launched the two service containers, kutt_redis_1 and kutt_server_1:

$ podman compose ps
CONTAINER ID  IMAGE                           COMMAND               CREATED        STATUS        PORTS                   NAMES
2bdf802c7739  docker.io/library/redis:alpine  redis-server          2 minutes ago  Up 2 minutes  6379/tcp                kutt_redis_1
8863e07584ae  docker.io/kutt/kutt:latest      /bin/sh -c npm ru...  2 minutes ago  Up 2 minutes  0.0.0.0:3000->3000/tcp  kutt_server_1

At this point, I was able to access Kutt via http://localhost:3000 from both my browser and the command line:

$ curl http://localhost:3000
Found. Redirecting to /create-admin

Success!

Adding Nginx Proxy Manager

For deploying Nginx Proxy Manager, I followed the same approach. I took the Compose file from the project website, copied it to its own subdirectory, and spun it up within a new pod.

However, I first had to take care of two requirements: The Compose file expects two subdirectories as storage volumes, and it needs to be able to bind the web application to the host’s ports 80 and 443, which requires privileges a normal user doesn’t have.

The subdirectories are easy to create:

mkdir data letsencrypt

To allow the port allocation, we need root privileges:

sudo sh -c "echo 0 > /proc/sys/net/ipv4/ip_unprivileged_port_start"

Now we can deploy Nginx Proxy Manager:

podman compose --in-pod true up -d

Let’s test the web access:

$ curl http://localhost
...
<h1>Congratulations!</h1>
...

Another success! But we’re not going to set up our URL-shortener just yet. Let’s first hand over management to systemd.

Integrating containers into systemd with Quadlet

While the deployment with podman compose worked, we don’t want to have to manually issue commands every time we restart our host machine. From a system administration standpoint, an application that’s deployed in containers ideally should be handled just like any other service. It should be managed by systemd, which will allow you to do common tasks like systemctl start kutt to start the service, and systemctl enable kutt to get it started automatically at boot time. And fortunately, Podman has this integration with systemd in the form of Quadlet.

systemd uses so-called unit files to define the system services it has to manage. With Quadlet, we get a few new types of unit files that interface directly with Podman to run container-based infrastructure. Quadlet files live in /etc/containers/systemd.

Each Quadlet file in turn references a service unit file that can, among other things, run a single container or a whole pod. In summary, we will need a pod unit definition for each service, and a Quadlet file to have it managed by systemd. Since we want to deploy both Kutt and Nginx Proxy Manager, we will need four files in total.

However, Quadlet doesn’t use Compose files as service units. Instead, it reads pod manifests, a different YAML format used by the resource orchestration software Kubernetes. Fortunately, Podman makes it easy to generate such a file right from an already running pod. We just need its name or ID.

$ podman pod ls
POD ID        NAME         STATUS      CREATED         INFRA ID    # OF CONTAINERS
9414b40c01c7  pod_nginxpm  Running     11 minutes ago              1
94387ff1aabd  pod_kutt     Running     18 minutes ago              2

Let’s generate the pod definition file for Kutt:

podman kube generate pod_kutt | sudo tee /etc/containers/systemd/kutt.yml

And for Nginx Proxy Manager:

podman kube generate pod_nginxpm | sudo tee /etc/containers/systemd/nginxpm.yml

Making everything pretty

These pod definition files will work as generated by Podman, but I found the automatic naming less than ideal. First, I removed the initial comments and annotations, and shortened the pod and container names. I also assigned uniform volume names, and made sure their associated physical volume claims (PVC), which is how Kubernetes provides storage space for volumes, followed Kubernetes naming rules.

Here’s my kutt.yml:

---
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: "2025-02-20T15:53:07Z"
  labels:
    app: kutt
  name: kutt
spec:
  containers:
  - args:
    - redis-server
    image: docker.io/library/redis:alpine
    name: redis1
    volumeMounts:
    - mountPath: /data
      name: kutt-redis-pvc
  - args:
    - /bin/sh
    - -c
    - npm run migrate && npm start
    env:
    - name: REDIS_HOST
      value: redis
    - name: JWT_SECRET
      value: 0F5B2J0CcoaFIDehV14ngwd6Lp2YbyJM7YeFVkILB1
    - name: DEFAULT_DOMAIN
      value: mspc.sh
    - name: DB_FILENAME
      value: /var/lib/kutt/data.sqlite
    - name: REDIS_ENABLED
      value: "true"
    - name: REDIS_PORT
      value: "6379"
    image: docker.io/kutt/kutt:latest
    name: server
    ports:
    - containerPort: 3000
      hostPort: 8000
      hostIP: 127.0.0.1
    volumeMounts:
    - mountPath: /var/lib/kutt
      name: kutt-sqlite-pvc
    - mountPath: /kutt/custom
      name: kutt-custom-pvc
  volumes:
  - name: kutt-redis-pvc
    persistentVolumeClaim:
      claimName: kutt-redis
  - name: kutt-sqlite-pvc
    persistentVolumeClaim:
      claimName: kutt-sqlite
  - name: kutt-custom-pvc
    persistentVolumeClaim:
      claimName: kutt-custom

I set the value for DEFAULT_DOMAIN (line 28) to “mspc.sh”; a URL shortener should have a short domain name, after all. I also changed the host port mapping to port 8000 because I think it’s a better fit for HTTP-based services.

Speaking of port mapping, you can see that I also added hostIP: 127.0.0.1 in line 40. This connects the web application to the host’s localhost interface, but not to its external NIC. This way, it will not be reachable from the internet; we want to let Nginx Proxy Manager take care of that.

For the Nginx Proxy Manager pod, I did the same clean-up, and also replaced the directory-based volumes with physical volume claims:

---
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: "2025-02-20T15:59:42Z"
  labels:
    app: nginxpm
  name: nginxpm
spec:
  containers:
  - image: docker.io/jc21/nginx-proxy-manager:latest
    name: nginxpm
    ports:
    - containerPort: 80
      hostPort: 80
    - containerPort: 81
      hostPort: 81
    - containerPort: 443
      hostPort: 443
    volumeMounts:
    - mountPath: /data
      name: nginxpm-data-pvc
    - mountPath: /etc/letsencrypt
      name: nginxpm-letsencrypt-pvc
  volumes:
  - name: nginxpm-data-pvc
    persistentVolumeClaim:
      claimName: nginxpm-data
  - name: nginxpm-letsencrypt-pvc
    persistentVolumeClaim:
      claimName: nginxpm-letsencrypt

At this point, we can get rid of the Compose deployments we used to get here. Simply execute podman compose down for both Kutt and Nginx Proxy Manager, and remove the volume directories as well.

To launch the two services using our newly created pod definition files, we use the command podman kube play:

$ sudo podman kube play /etc/containers/systemd/kutt.yml
...
Pod:
465c57697148a9a737bf966b272a01b785d2674da8de990d0095093d34546128
Containers:
c84fdb5d07d7f16de2a79c606910bbe825fd7f09dcd76a542e02e824fc00f4e9
430eb4c1adb85096727f43dd159187e748d3babf59bab830ebd94e2d6ec2c36b
$ sudo podman kube play /etc/containers/systemd/nginxpm.yml
...
Pod:
b2d82d01d13fa4dae26bd177d2a3b88f6084907238ecb98c54150466c4c9f207
Container:
8e21f38468807eec2d0024c8b8f49724c8a8f97f0a23ee311f85ef0ee192c84d

As you can tell from the output, Podman created a pod and two containers for Kutt, and another pod and a single container for Nginx Proxy Manager; just as expected.

Turning a pod into a systemd unit

The final step is to hand over management of these pods to systemd. All we need for this are two Quadlet files. Since they reference pod definitions, their names both end in .kube. I had to create both files as root because the /etc directory is superuser territory.

For Kutt, there’s /etc/containers/systemd/kutt.kube:

[Unit]
Description=The kutt URL shortener
Wants=network-online.target
After=network-online.target

[Kube]
Yaml=kutt.yml

[Service]
TimeoutStartSec=900

[Install]
WantedBy=multi-user.target

Similarly, we have /etc/containers/systemd/nginxpm.kube:

[Unit]
Description=Nginx Proxy Manager
Wants=network-online.target
After=network-online.target

[Kube]
Yaml=nginxpm.yml
Network=host

[Service]
TimeoutStartSec=900

[Install]
WantedBy=multi-user.target

This second file has a tiny but important difference to the first. It prevents the pod from being isolated in its own container network; instead, it is bound to the host machine’s network interfaces. This is necessary because, as I explained above, the port 8000 that is connected with Kutt is bound exclusively to the host’s localhost interface. Without the Network=host setting in line 8, the Nginx Proxy Manager pod would get its own localhost interface, and this one wouldn’t know anything about Kutt.

Here’s a diagram that visualizes how the kutt pod has its own pod-internal localhost interface that the kutt and redis containers use to communicate. Only port 8000 gets forwarded from the Podman host into this pod. The nginxpm pod, on the other hand, is connected directly to the host’s network interfaces.

This allows us to set up a proxy host in Nginx Proxy Manager that forwards incoming requests to the Kutt container in the other pod.

Putting it all together

Now that we have both Quadlet files and pod manifests in place, we can let systemd take over:

$ sudo systemctl daemon-reload
$ sudo systemctl restart kutt
$ sudo systemctl restart nginxpm

As the final installation step after doing a quick initial setup of Nginx Proxy Manager by visiting my host machine in the browser, I set up a proxy host for the Kutt container bound to localhost:8000 with my desired domain name. Since I also enabled SSL support, everyone can now safely access my URL-shortener.

Handling secrets properly

As a bonus, we can make a small security improvement. One thing I didn’t like about the Kutt manifest is that it contained JWT_SECRET in plain text (just like the original Compose file did). I don’t regard this as good operations practice, given that Podman can store secrets in a safer way.

To do so, I created a Kubernetes secret definition file named secret.yml containing a long, randomly generated jwtSecret in line 3:

apiVersion: v1
data:
  jwtSecret: ZWlUbzl0aG9vZmFoR2gyUXVh5nb0g2YWhkb2hhZXoK
kind: Secret
metadata:
  creationTimestamp: null
  name: kutt-jwt-secret

After I had given the file’s contents into Podman’s safety, it wasn’t needed any more.

$ podman kube play secret.yml
Secrets:
3bfa1f97029464c220cd699ea
$ rm secret.yml

In kutt.yml, I then replaced the hard-coded value of JWT_SECRET with a reference to the key jwtSecret of the kutt-jwt-secret. The resulting env section now looks like this:

    env:
    - name: REDIS_HOST
      value: redis
    - name: JWT_SECRET
      valueFrom:
        secretKeyRef:
          name: kutt-jwt-secret
          key: jwtSecret
    - name: DEFAULT_DOMAIN
      value: mspc.sh
    - name: DB_FILENAME
      value: /var/lib/kutt/data.sqlite
    - name: REDIS_ENABLED
      value: "true"
    - name: REDIS_PORT
      value: "6379"

That’s much better than exposing the value in an unencrypted YAML file.

Conclusion

In this article, I explained all the steps that I went through to run my own URL-shortener using Podman. I laid out how to deploy the necessary applications using Compose files, how to derive pod manifests from the running containers, and finally, how to integrate them with systemd.

If you’d like to use Podman for self-hosting your own container-based applications, I hope this will be useful!