<feedxmlns="http://www.w3.org/2005/Atom"><title>Sysadmining. All day. Every day.</title><linkhref="https://captainark.net/"rel="alternate"></link><linkhref="https://captainark.net/rss.xml"rel="self"></link><id>https://captainark.net/</id><updated>2016-03-26T00:00:00+01:00</updated><entry><title>WebDAV with nginx</title><linkhref="https://captainark.net/webdav-with-nginx.html"rel="alternate"></link><published>2016-03-26T00:00:00+01:00</published><updated>2016-03-26T00:00:00+01:00</updated><author><name>Antoine Joubert</name></author><id>tag:captainark.net,2016-03-26:/webdav-with-nginx.html</id><summarytype="html"><p>This website has been hosted on an <a href="https://www.online.net">Online.net</a> dedicated server since its creation. I've been one of their customers for the past 3 years now, and I still don't have anything bad to say about them.</p>
<p>They recently upgraded their personnal range, and I took the opportunity to upgrade …</p></summary><contenttype="html"><p>This website has been hosted on an <a href="https://www.online.net">Online.net</a> dedicated server since its creation. I've been one of their customers for the past 3 years now, and I still don't have anything bad to say about them.</p>
<p>They recently upgraded their personnal range, and I took the opportunity to upgrade from a single server running all of my services to 2 servers running LXC containers that are hosting my services.</p>
<p>It took me 2 days to migrate everything, but it was worth it. If I decide to switch servers again, I'll have to migrate the containers instead of the services themselves. Considering they are stored on a separate BTRFS volume, it shouldn't take me more than a few hours at most.</p>
<p>During the migration, I realized that I needed to make files that were hosted on one server accessible to the other. I could have gone with CIFS or NFS, but I wanted to have encryption built-in instead of having to rely on a VPN for that. Since I figured it was a good opportunity to learn something new, I ended up going with WebDAV.</p>
<p>In this tutorial, I'll explain how I've configured a read-only WebDAV share using <a href="https://www.nginx.com/">nginx</a> and <a href="https://letsencrypt.org/">Let'sEncrypt</a> SSL certificates between two Debian Jessie containers.</p>
<h2>Server configuration</h2>
<h3>Installing the required packages</h3>
<p>First thing first, we need to install the packages we'll need for this configuration :</p>
<p>We now need to configure nginx by adding the following in the <code>/etc/nginx/sites-available/default</code> file, anywhere in the <code>server{}</code> block that is configured to listen on port 80.</p>
<p><em>Please do modify www.example.com by your server's FQDN, and please note that the letsencrypt servers need to be able to resolve that name to your server's IP.</em></p>
<p>If everything goes well, your certificates will be generated and stored in the /etc/letsencrypt folder.</p>
<h3>WebDAV configuration</h3>
<p>Now that we've obtained our certificate from letsencrypt, we can begin configuring nginx.</p>
<p>First, we need to comment two SSL directives from the default nginx configuration :</p>
<p><em>This configuration will work if you're using a single certificate on your server. If not, you'll have to remove the <code>ssl_certificate</code>, <code>ssl_certificate_key</code> and <code>ssl_trusted_certificate</code> directives from this file and move them to the correct <code>server{}</code> block.</em></p>
<p>We now need to generate a <code>dhparam.pem</code> file :</p>
<p>We now have to create a <code>/etc/nginx/sites-available/example</code> file that will contain our actual webdav configuration. This example makes a <code>data</code> folder stored in <code>/var/www/</code> accessible.</p>
<p>That's it for the WebDAV configuration server-side !</p>
<h3>nginx monitoring</h3>
<p>If you're using monit, you can easily monitor the nginx daemon by copying the following in <code>/etc/monit/conf.d/nginx</code> :</p>
<div class="highlight"><pre><span></span>check process nginx
with pidfile &quot;/run/nginx.pid&quot;
start program = &quot;/bin/systemctl start nginx&quot;
stop program = &quot;/bin/systemctl stop nginx&quot;
alert monit@example.com
</pre></div>
<h3>Certificates auto-renewal</h3>
<p>This goes beyond the scope of the article, but since letsencrypt certficates are only valid for 3 months, you'll need to renew them regularily. You can do so manually or you can setup a cron that does it for you.</p>
<p>I personnaly use the following script :</p>
<span class="nb">echo</span><span class="s2">&quot;Something went wrong while renewing the certificates on </span><span class="k">$(</span>hostname -f<span class="k">)</span><span class="s2"></span>
<span class="s2"> Manual action needed.&quot;</span><span class="p">|</span> mail -s <span class="s2">&quot;Letsencrypt error on </span><span class="k">$(</span>hostname -f<span class="k">)</span><span class="s2">&quot;</span><span class="nv">$MAILDEST</span>
<span class="k">fi</span>
</pre></div>
</td></tr></table>
<p>You can add multiple domains in the script. As long as you add all 3 lines for each domain, it will not automatically reload nginx if one or more certificate could not be renewed and will send an e-mail to the address configured in the <code>MAILDEST</code> variable.</p>
<p>You can configure this script in the root user crontab using the <code>crontab -e</code> command :</p>
<p>If like me, you want to mount your webdav share in a LXC container, you'll first need to make sure that the following line is present in its configuration file :</p>
<div class="highlight"><pre><span></span>lxc.cgroup.devices.allow = c 10:229 rwm
</pre></div>
<p>You'll also need to create the <code>/dev/fuse</code> node in the container :</p>
<div class="highlight"><pre><span></span>mknod /dev/fuse c 10 229
</pre></div>
<p>In any case, we have to edit the <code>/etc/davfs2/secrets</code> file to add the mount point, username and password that will be used to mount the share :</p>
<p>You might need to edit the parameters depending on which users you want to make the share available to.</p>
<h3>Mouting the share on boot</h3>
<p>A davfs volume can be mounted via the <code>/etc/fstab</code> file, but I decided to use monit instead so that the volume would be mounted again automatically should my WebDAV server reboot.</p>
<p>In order to do so, I first created a <code>davfs.txt</code> file in the <code>/var/www/data</code> folder on my WebDAV server :</p>
<p>The last thing I did was create a <code>/etc/monit/conf.d/davfs</code> file with the following content :</p>
<div class="highlight"><pre><span></span>check file davfs with path /data/davfs.txt
alert monit@example.com
if does not exist then exec &quot;/root/bin/mount_davfs&quot;
</pre></div>
<p>That way, if monit notices that the <code>/data/davfs.txt</code> file becomes inaccessible for some reason, it will try remouting the share.</p>
<p>That's all ! Hopefully this has been useful to someone. Please do comment below if you have any question or if this has been helpful !</p></content></entry><entry><title>MySQL backup script</title><linkhref="https://captainark.net/mysql-backup-script.html"rel="alternate"></link><published>2016-03-13T00:00:00+01:00</published><updated>2016-03-13T00:00:00+01:00</updated><author><name>Antoine Joubert</name></author><id>tag:captainark.net,2016-03-13:/mysql-backup-script.html</id><summarytype="html"><p>I wrote a MySQL database backup script a while back. I known they are more than enough of them already floating around the internet, but hey, I figured I'd share it here anyway.</p>
<h2>The script</h2>
<p>For the script to work, you'll need to edit a few variable to match your …</p></summary><contenttype="html"><p>I wrote a MySQL database backup script a while back. I known they are more than enough of them already floating around the internet, but hey, I figured I'd share it here anyway.</p>
<p>For the script to work, you'll need to edit a few variable to match your configuration.</p>
<ul>
<li><code>BACKUPDIR</code> is the path of the directory where you want your backups to be stored.</li>
<li><code>BACKUPUSR</code> is the user that will connect to MySQL to dump the databases. It should have access to all you databases without needing a password.</li>
<li><code>EXCLUDELIST</code> is a list of databases that should not be backed-up. Leaving it as is is probably fine.</li>
<p>I personnaly have this script running once a week, in my user's personnal crontab (editable using the <code>crontab -e</code> command) :</p>
<p>You've probably noticed that the script erases the previous backup when a new one is made.</p>
<p>I don't need to keep multiple versions of the same database backup on my servers because they are all saved remotely on a daily basis using <a href="http://rsnapshot.org/">Rsnapshot</a>. I'll probably write an article on the subject in the future.</p>
<p>As usual, feedback is always appreciated !</p></content></entry><entry><title>Postfix Admin</title><linkhref="https://captainark.net/postfix-admin.html"rel="alternate"></link><published>2016-03-06T00:00:00+01:00</published><updated>2016-03-06T00:00:00+01:00</updated><author><name>Antoine Joubert</name></author><id>tag:captainark.net,2016-03-06:/postfix-admin.html</id><summarytype="html"><p>As I explained in <a href="https://www.captainark.net/setting-up-a-mail-server.html">this previous tutorial</a>, I've been running my own mail server without any issue for some time now.</p>
<p>However, every time I've wanted to add a domain, create a new mailbox or change a user's password, I've had to do it manually from a SQL shell. As …</p></summary><contenttype="html"><p>As I explained in <a href="https://www.captainark.net/setting-up-a-mail-server.html">this previous tutorial</a>, I've been running my own mail server without any issue for some time now.</p>
<p>However, every time I've wanted to add a domain, create a new mailbox or change a user's password, I've had to do it manually from a SQL shell. As fun as it may be, it does get old very fast, so I've decided to install a web frontend to manage this database.</p>
<p>After a bit a googling, I've settled on <a href="http://postfixadmin.sourceforge.net/">Postfix Admin</a>.</p>
<p>The latest stable version of Postfix Admin was released in 2009. Version 3.0 has been in the works for some time now and the project can be cloned from their <a href="https://github.com/postfixadmin/postfixadmin">Github repo</a>.</p>
<p>I've also tried <a href="http://www.vimbadmin.net/">ViMbAdmin</a>, but it felt a little heavy considering what I was going to use it for.</p>
<p>You'll need a web server with PHP support to run Postfix Admin. I personnaly run nginx with php5-fpm, but I won't explain how to configure it here. I'll simply explain how to migrate your current database to one managed with Postfix Admin with as little downtime as possible.</p>
<h1>Creating a new database</h1>
<p>Since the database managed by Postfix Admin does not use the same schema as the one we've created in my previous tutorial, we'll have to create a new one. We will give all privileges on that database to the same user as before, <code>'mail'@'localhost'</code>.</p>
<p>At this point, you can clone the Postfix Admin project from Github and go through the installation process.</p>
<p>While editing the config.inc.php file (or config.local.php file if you've decided to copy it), make sure that the <code>database_name</code> option is set to use the <code>mailnew</code> database we've just created.</p>
<p>Also, make sure that the <code>encrypt</code> option is set to <code>dovecot:SHA512-CRYPT</code>.</p>
<p>The installation process will create all the necessary tables in the database.</p>
<p><strong>At this point, you'll have to recreate all domains, mailboxes and aliases that you have configured in your current mail database using the Postfix Admin interface.</strong></p>
<h1>Postfix configuration</h1>
<p>Once you're done with Postfix Admin, it's time to configure Postfix to use its schema.</p>
<p>First thing first, let's backup our current configuration :</p>
<p>Next, we have to edit the 3 files we've just backed-up. The only line that actually changes is the one beginning with <code>query</code>.</p>
<p>The first file is /etc/postfix/mysql-virtual-mailbox-domains.cf :</p>
<div class="highlight"><pre><span></span>user = mail
password = mailpassword
hosts = 127.0.0.1
dbname = mail
query = SELECT 1 FROM domain WHERE domain=&#39;%s&#39; AND active=&#39;1&#39;
</pre></div>
<p>The second one is /etc/postfix/mysql-virtual-mailbox-maps.cf :</p>
<div class="highlight"><pre><span></span>user = mail
password = mailpassword
hosts = 127.0.0.1
dbname = mail
query = SELECT 1 FROM mailbox WHERE username=&#39;%s&#39; AND active=&#39;1&#39;
</pre></div>
<p>And the last one is /etc/postfix/mysql-virtual-alias-maps.cf :</p>
<div class="highlight"><pre><span></span>user = mail
password = mailpassword
hosts = 127.0.0.1
dbname = mail
query = SELECT goto FROM alias WHERE address=&#39;%s&#39; AND active=&#39;1&#39;
</pre></div>
<h1>Dovecot configuration</h1>
<p>Same as with Postfix, we now need to configure Dovecot to use the Postfix Admin schema.</p>
<p>First, let's backup our current configuration :</p>
<div class="highlight"><pre><span></span>cp -a /etc/dovecot/sql.conf /etc/dovecot/sql.conf.bak
</pre></div>
<p>Next, we have to edit the /etc/dovecot/sql.conf file. The only line that changes is the one beginning with <code>password_query</code>.</p>
<div class="highlight"><pre><span></span>driver = mysql
<p>At this point, Postfix and Dovecot are using the Postfix Admin schema in the mail database.</p>
<p>The last thing we have to do is to edit Postfix Admin's config.inc.php file to use the mail database as well instead of the mailnew database that it should be currently using.</p>
<h1>Cleanup</h1>
<p>Once you've confirmed that everything is working properly, you can delete the backup files we've created :</p>
<p>That's all ! As always, please do leave a comment if this article has been of any use to you !</p></content></entry><entry><title>My tmux configuration</title><linkhref="https://captainark.net/my-tmux-configuration.html"rel="alternate"></link><published>2016-02-02T00:00:00+01:00</published><updated>2016-02-02T00:00:00+01:00</updated><author><name>Antoine Joubert</name></author><id>tag:captainark.net,2016-02-02:/my-tmux-configuration.html</id><summarytype="html"><p><a href="https://tmux.github.io/">tmux</a> is a terminal mutiplexer. It lets you have multiples shells running in a single terminal emulator window and it keeps those shells running in the background should you need to close your terminal emulator.</p>
<p>I've played around with the configuration quite a bit to find settings that suit my …</p></summary><contenttype="html"><p><a href="https://tmux.github.io/">tmux</a> is a terminal mutiplexer. It lets you have multiples shells running in a single terminal emulator window and it keeps those shells running in the background should you need to close your terminal emulator.</p>
<p>This screenshot was done on Mac OS X, using the Terminal app and this <a href="https://github.com/tomislav/osx-terminal.app-colors-solarized">Solarized theme</a>.</p>
<p>I figured I'd share my tmux configuration here !</p>
<h2>Installing tmux</h2>
<p>tmux is available on Debian. I suggest using the <a href="https://packages.debian.org/jessie-backports/tmux">jessie backports</a> version :</p>
<p>I used screen before tmux, so I configured the prefix key on C-a instead of C-b. tmux has the advantage of being <em>much</em> simpler to configure than screen.</p>
<p>If you want to use this configuration, simply copy the following in ~/.tmux.conf. This file is read by default when tmux starts.</p>
<p>If you simply want to try it out, copy it in a file somewhere else and have tmux load with the -f parameter (<code>tmux -f ~/tmux-test.conf</code>).</p>
<p>The second one changes the tmux window name whenever I ssh to a remote host, and switches the window name back to the name of my computer when I logout from the host.</p>
<p>That's all ! As always, please do leave a comment if you've found something useful in this article !</p></content></entry><entry><title>Debian updates with Ansible</title><linkhref="https://captainark.net/debian-updates-with-ansible.html"rel="alternate"></link><published>2016-01-31T00:00:00+01:00</published><updated>2016-01-31T00:00:00+01:00</updated><author><name>Antoine Joubert</name></author><id>tag:captainark.net,2016-01-31:/debian-updates-with-ansible.html</id><summarytype="html"><p>I've recently bought a <a href="http://www8.hp.com/us/en/products/proliant-servers/product-detail.html?oid=5379860">HP Proliant Microserver Gen8</a> to play around with LXC and try new stuff.</p>
<p>From the 4 Debian machines I had to keep up-to-date, I now have 7, so it became quite time-consumming to manually SSH to each of them whenever an update became available.</p>
<p>I ended …</p></summary><contenttype="html"><p>I've recently bought a <a href="http://www8.hp.com/us/en/products/proliant-servers/product-detail.html?oid=5379860">HP Proliant Microserver Gen8</a> to play around with LXC and try new stuff.</p>
<p>From the 4 Debian machines I had to keep up-to-date, I now have 7, so it became quite time-consumming to manually SSH to each of them whenever an update became available.</p>
<p>I ended up looking at <a href="http://www.ansible.com/">Ansible</a> to speed up the process and, within an hour, I had a working playbook that updates the debian packages, pip packages and git repos installed on all of my servers with a single command.</p>
<p>I figured I'd share the playbook I use to update the Debian packages !</p>
<h2>The playbook</h2>
<p>I modified <a href="https://gist.github.com/maethor/380676f6b1cec8cc7439">this gist</a> to only use apt-get instead of both apt-get and aptitude.</p>
<span class="p p-Indicator">-</span><span class="l l-Scalar l-Scalar-Plain">name</span><span class="p p-Indicator">:</span><span class="l l-Scalar l-Scalar-Plain">check what the new version is</span>
<span class="p p-Indicator">-</span><span class="l l-Scalar l-Scalar-Plain">name</span><span class="p p-Indicator">:</span><span class="l l-Scalar l-Scalar-Plain">notify distribution version upgrade</span>
<span class="l l-Scalar l-Scalar-Plain">debug</span><span class="p p-Indicator">:</span><span class="l l-Scalar l-Scalar-Plain">msg=&quot;Debian has been upgraded from {{ ansible_lsb.release }} to {{ new_release.stdout }}&quot;</span>
<span class="p p-Indicator">-</span><span class="l l-Scalar l-Scalar-Plain">name</span><span class="p p-Indicator">:</span><span class="l l-Scalar l-Scalar-Plain">/wheezy/ install the debian-goodies package if it is missing</span>
<span class="p p-Indicator">-</span><span class="l l-Scalar l-Scalar-Plain">name</span><span class="p p-Indicator">:</span><span class="l l-Scalar l-Scalar-Plain">/jessie/ install the needrestart package if it is missing</span>
<p>That's all ! Please leave a comment if you've found this playbook helpful !</p></content></entry><entry><title>Private Git Repo</title><linkhref="https://captainark.net/private-git-repo.html"rel="alternate"></link><published>2016-01-31T00:00:00+01:00</published><updated>2016-01-31T00:00:00+01:00</updated><author><name>Antoine Joubert</name></author><id>tag:captainark.net,2016-01-31:/private-git-repo.html</id><summarytype="html"><p>I've decided to migrate this blog to <a href="http://blog.getpelican.com/">Pelican</a>. I've been playing around with it over the week-end, and it turns out to be way easier to manage than <a href="https://jekyllrb.com/">Jekyll</a>. Themes are much easier to install and configure, so it ends up looking better as well !</p>
<p>Since I'm basically recreating this …</p></summary><contenttype="html"><p>I've decided to migrate this blog to <a href="http://blog.getpelican.com/">Pelican</a>. I've been playing around with it over the week-end, and it turns out to be way easier to manage than <a href="https://jekyllrb.com/">Jekyll</a>. Themes are much easier to install and configure, so it ends up looking better as well !</p>
<p>Since I'm basically recreating this blog from scratch, I've decided to delete the old git repo that was hosting it and to create a new one.</p>
<p>Setting up your own private git repo is pretty easy to achieve and is already well-documented on the <a href="https://git-scm.com/book/en/v2/Git-on-the-Server-Setting-Up-the-Server">Git</a> website.</p>
<p>Every time I want to create a new repo, I've had time to forget how to do it and I end up looking for that page, so I figured I'd write a few lines on the subject.</p>
<p>In this tutorial, I'll configure a git repo on a distant server running Debian 8 (Jessie). This repo will be remotely accessible using SSH. Two users will be able to connect to it : me and the www-data user on my webserver.</p>
<h2>SSH Keys</h2>
<p>If you don't have one already, you'll need a ssh-key to connect to the git repo.</p>
<p>On your computer, in a shell, as your usual user :</p>
<p>For security reasons, configuring a passphrase is recommended. On Mac OS X and most desktop environnements on Linux, you can store this passphrase for the duration of your session using the <code>ssh-add</code> command, so you won't have to type it every time you want to connect to a host.</p>
<p>On the server, we also have to create a ssh-key for the user that is running our webserver (you'll need to have sudo installed) :</p>
<p>This will create a system user (UID &lt; 1000) with a /home/git home directory. If you want to host your git repos somewhere else on your filesystem, you should add a <code>-d /home/directory/for/git</code> in the previous command.</p>
<p>This user will use the git-shell shell. This limits remote connection to that user to git commands (like the rssh shell can limit remote connection to a user to scp or rsync commands).</p>
<p>We have to configure our system to allow the use of this shell :</p>
<p>You can now copy/paste the content of the two <code>$HOME/.ssh/id_rsa.pub</code> files we've created earlier using the <code>ssh-keygen</code> command in <code>/home/git/.ssh/authorized_keys</code>.</p>
<p>The last thing we have to do is to create our first git repo. In this example, my project will be called 'captainarkdotnet' as it will be hosting this blog :</p>
<p>We're done with the server configuration. Let's now actually push stuff to our repo !</p>
<h3>Initial push</h3>
<p>The files for my blog are store in the ~/Documents/projects/captainarkdotnet on my computer. Before doing anything else, we first have to make sure that we currently are in that folder :</p>
<p>Please note that you'll need to edit <strong>git.captainark.net</strong> to the FQDN or IP of your git server, and <strong>captainarkdotnet.git</strong> to the name of the git project on your server.</p>
<p>If everything went well, the last command should give you the following output :</p>
<p>That's it, we've now pushed our first commit to our server !</p>
<h2>First pull</h2>
<p>Alright, time to pull the files we've just pushed on our webserver. I personally store my web content in <code>/var/www</code> ; if you don't, you'll have to adjust the path accordingly :</p>
<p>SSH will ask you to type 'yes' since it's the first time the www-data user connects to the server. If everything goes well, you should have the following output :</p>
<div class="highlight"><pre><span></span>Cloning into <span class="s1">&#39;captainarkdotnet&#39;</span>...
<p>That's it ! We now have a working private git repo ! I won't go into details into the git commands in this tutorial, but here's a quick overwiew of the ones I use the most :</p>
<ul>
<li><code>git add .</code> recursively adds all files from the directory to the repo ;</li>
<li><code>git commit -a -m 'This is a comment'</code> commits the current state of your local repo with the 'This is a comment' comment ;</li>
<li><code>git push</code> pushes your commits to the distant repo ;</li>
<li><code>git pull</code> pulls the latest version of the distant repo locally ;</li>
<li><code>git branch -av</code> shows all available branches for the repo ;</li>
<li><code>git checkout -b testing remotes/origin/testing</code> create a local 'testing' branch based on the remote 'remotes/origin/testing' branch ;</li>
<li>once a branch has been copied locally, you can switch to it with the <code>git checkout {branch}</code> command.</li>
</ul>
<p>For more information on git a command, use <code>man git-{command}</code> !</p>
<p>If you've found this tutorial in any way helpful, please feel free to leave a comment !</p></content></entry><entry><title>Flexget init script</title><linkhref="https://captainark.net/flexget-init-script.html"rel="alternate"></link><published>2015-05-05T00:00:00+02:00</published><updated>2015-05-05T00:00:00+02:00</updated><author><name>Antoine Joubert</name></author><id>tag:captainark.net,2015-05-05:/flexget-init-script.html</id><summarytype="html"><p>I've been using <a href="http://flexget.com/">Flexget</a> for the past two years or so as a download automator.</p>
<p>Since I wrote an <a href="http://flexget.com/wiki/Daemon/Startup#InsservscriptDebiancompatible">init script</a> for it a while back, and it is compatible with Debian Jessie / systemd, I figured I'd share it here.</p>
<h2>The script</h2>
<p>All of the following should be done as …</p></summary><contenttype="html"><p>I've been using <a href="http://flexget.com/">Flexget</a> for the past two years or so as a download automator.</p>
<p>Since I wrote an <a href="http://flexget.com/wiki/Daemon/Startup#InsservscriptDebiancompatible">init script</a> for it a while back, and it is compatible with Debian Jessie / systemd, I figured I'd share it here.</p>
<h2>The script</h2>
<p>All of the following should be done as the root user.</p>
<p>First, create a /etc/default/flexget file with the following content :</p>
<p>Please note that the FGUSER variable needs to be defined for the daemon to start. It can be set to your current user, or you can run flexget as its own user.</p>
<p>You can create a flexget user with the following command :</p>
log_action_msg <span class="s2">&quot;</span><span class="nv">$DESC</span><span class="s2">: FGUSER not set in /etc/default/</span><span class="nv">$NAME</span><span class="s2">. Exiting.&quot;</span>
log_action_msg <span class="s2">&quot;</span><span class="nv">$DESC</span><span class="s2">: Not currently running. Aborting.&quot;</span>
log_action_msg <span class="s2">&quot;</span><span class="nv">$DESC</span><span class="s2">: Not currently running.&quot;</span>
<p>That's all ! If you are using this script, please let me know in the comment section below !</p></content></entry><entry><title>Setting up a mail server</title><linkhref="https://captainark.net/setting-up-a-mail-server.html"rel="alternate"></link><published>2015-04-24T00:00:00+02:00</published><updated>2015-04-24T00:00:00+02:00</updated><author><name>Antoine Joubert</name></author><id>tag:captainark.net,2015-04-24:/setting-up-a-mail-server.html</id><summarytype="html"><p>In this first tutorial, I'll explain how I've configured my mail server using the following :</p>
<ul>
<li>A server running Linux Debian (jessie) ;</li>
<li>Postfix ;</li>
<li>Postfix-policyd-spf-python ;</li>
<li>Dovecot ;</li>
<li>Spamassassin ;</li>
<li>OpenDKIM ;</li>
<li>OpenDMARC ;</li>
<li>Monit ;</li>
<li>Rainloop.</li>
</ul>
<p>I'm assuming you have some basic knowledge of Linux and DNS configuration.</p>
<p>You can host this server at home, but you …</p></summary><contenttype="html"><p>In this first tutorial, I'll explain how I've configured my mail server using the following :</p>
<li>A server running Linux Debian (jessie) ;</li>
<li>Postfix ;</li>
<li>Postfix-policyd-spf-python ;</li>
<li>Dovecot ;</li>
<li>Spamassassin ;</li>
<li>OpenDKIM ;</li>
<li>OpenDMARC ;</li>
<li>Monit ;</li>
<li>Rainloop.</li>
</ul>
<p>I'm assuming you have some basic knowledge of Linux and DNS configuration.</p>
<p>You can host this server at home, but you might have issues with your ISP not allowing outbound traffic on TCP port 25, and your emails might be considered to be spam by other providers if your IP is dynamic and/or you can't configure a reverse DNS record on it.</p>
<p>The cheapest VMs from <a href="https://www.digitalocean.com/?refcode=1cd69e4c3389">DigitalOcean</a> or <a href="http://www.vultr.com/?ref=6804947">Vultr</a> are powerful enough to have this configuration running smoothly.</p>
<p>We'll also need a SSL certificate for this configuration. You can create an auto-signed one or get a free valid one from <a href="http://www.startssl.com/">StartSSL</a>. For the purpose of this tutorial, I'll consider you've chosen the latter.</p>
<p>You'll also need a domain name. I've chosen <a href="http://www.namecheap.com/?aff=85990">Namecheap</a> as a registrar. I won't go into details on how to configure it, but you'll need at the very least a A record on your server's IP as well as a MX record pointing to it.</p>
<p>I use the captainark.net domain as an example throughout this tutorial. You'll have to use your actual domain for your configuration to work !</p>
<p><em>Note: links in this section are sponsored.</em></p>
<h2>Initial configuration</h2>
<h3>Installing the required packages</h3>
<p>First thing first, we need to install the packages we'll need for this configuration :</p>
<p>During its installation, Postfix will prompt you with configuration questions. Choose "Internet Site", and when asked about your System mail name, provide it with your server's FQDN (it should be the output of the <code>hostname -f</code> command on your server).</p>
<p>You'll also have to set-up a password for the MySQL root user.</p>
<h3>Additional configuration</h3>
<p>The PTR records on your server's IPv4 and/or IPv6 should match your server's FQDN (a <code>dig -x</code> on your server's IP should match a <code>hostname -f</code> on your server).</p>
<p>You'll have to open the following TCP ports on your server for this configuration to work : 25, 465, 587 and 993.</p>
<p>If you don't want to have to remember the root user MySQL password, you can create a .my.cnf file in your current user home directory containing the following lines :</p>
<p>I also like to change the default MySQL shell to see what database I'm using at any given time. Since I use bash, I achieve this the following way :</p>
<p>You'll have to logout from the current shell for the modification to be taken into account (if you're using SSH, log out and back into your server).</p>
<p>You should now be able to log into MySQL without specifying a password, and it should look like this :</p>
<h2>Configuring the MySQL database</h2>
<h3>Initial configuration</h3>
<p>We now need to configure the MySQL database Postfix and Dovecot will be using. In this tutorial, we'll be calling it "mail", but you can name it whatever you want.</p>
<p>First, in a mysql shell, let's create the MySQL database :</p>
<p>Now, we are going to create the user that Postfix and Dovecot will be using to access the database. We will only be granting this user select permission :</p>
<p>Now, all messages sent to alias@captainark.net will be forwarded to example@captainark.net.</p>
<p>Use the same syntax to create additional domains, users and aliases. If you have more than one domains configured, be sure to associate your users and aliases with the correct domain_id.</p>
<h2>Configuring Postfix</h2>
<p>Next, we are going to configure <a href="http://www.postfix.org/">Postfix</a>.</p>
<h3>Configuration backup</h3>
<p>First, let's backup the original configuration files :</p>
<p>Purists will probably want to store their certificates in /etc/ssl/private. If you choose to do so, you'll have to adapt the path of those files for the remainder of this tutorial.</p>
<p>If you've decided to create a certificate with StartSSL, you'll end up with two files, a .crt and a .key. I'll name those files server.crt and server-with-passphrase.key. Put both these files in the folder we've just created.</p>
<p>Now, let's remove the passphrase from the key :</p>
<p>That's it for Postfix, for now ; Dovecot is next !</p>
<h2>Configuring Dovecot</h2>
<h3>Dovecot global configuration</h3>
<p>By default, on Debian, <a href="http://www.dovecot.org/">Dovecot</a> uses multiple configuration files in /etc/dovecot/conf.d. I found it annoying to maintain, and I ended up only using the /etc/doveconf.conf file.</p>
<p>As always, let's start by backing up the original configuration file :</p>
<p>Dovecot will use the same SSL certificate as Postfix.</p>
<p>Using this configuration, your virtual users' emails will be stored in /var/mail/$domain/$user/ and will be owned by the vmail user.</p>
<p>For this to work, we have to create the domain folder :</p>
<p>Dovecot will create the virtual users' folders automatically.</p>
<h3>Dovecot access to the MySQL database</h3>
<p>We now need to allow Dovecot to connect to the mail database we have populated earlier. To do so, we are going to create a /etc/dovecot/sql.conf file with the following content :</p>
password_query = SELECT email as user, password FROM virtual_users WHERE email=&#39;%u&#39;;
</pre></div>
<p>You'll have to change the password to the one you have defined earlier. Since this file contains a password, let's make sure it's not world-readable :</p>
<p>The last thing we need to configure here is sieve. The idea is to have all messages flagged as spam automatically moved to the mailbox Junk folder.</p>
<p>To do so, let's first create the required folders :</p>
<p>If you want to have sieve rules for a specific user, simply create $user@$domain.sieve file in the users folder (example@captainark.net in my case).</p>
<p>All .sieve files in the before folder will be used for all your virtual users, before their individual configuration ; the .sieve files in the after folder will be used, well, you guessed it, after.</p>
<p>Let's create a filter.sieve file in the /var/mail/sieve/before folder with the following content :</p>
<li>Password: the password you chose for your virtual user ;</li>
<li>IMAP: your server's FQDN, port 993 (SSL/TLS with normal password) ;</li>
<li>SMTP: your server's FQDN, port 465 (SSL/TLS with normal password).</li>
</ul>
<h2>Configuring SpamAssassin</h2>
<h3>The alternatives</h3>
<p>Next thing we have to do is to configure the actual anti-spam. I tried a few, but I ended up sticking with <a href="http://spamassassin.apache.org/">SpamAssassin</a>. Here's why :</p>
<ul>
<li><a href="http://dspam.nuclearelephant.com/">DSPAM</a> is <a href="http://sourceforge.net/p/dspam/mailman/message/32585111/">no longer maintained</a> and <a href="https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=754810">has been removed from Debian Jessie</a> ;</li>
<li><a href="https://rspamd.com/">Rspamd</a> is interesting, has been <a href="https://packages.debian.org/source/jessie/rspamd">integrated in Debian Jessie</a>, but is poorly documented at this time ;</li>
<li><a href="http://bogofilter.sourceforge.net/">Bogofilter</a> does not seem to have the greatest server integration.</li>
</ul>
<h3>The actual configuration</h3>
<p>SpamAssassin's configuration is pretty straightforward. First, let's edit the /etc/default/spamassassin file :</p>
<p>Next, to have Postfix send incoming emails through SpamAssassin, we have to edit the /etc/postfix/master.cf file. At the very beginning, we have to add a line under the smtp definition :</p>
<p>That's all for SpamAssassin ! To check if it is working, send yourself an email from another provider. You should see the following headers in it :</p>
<h3>Allowing your server to send emails for your domain</h3>
<p><a href="http://www.openspf.org/">SPF</a> (Sender Policy Framework) is a mechanism that confirms that your server's IP is allowed to send emails for your domain. Technically, it is a TXT DNS record which looks something like this :</p>
<p>This DNS record lets other mail servers know that hosts that have a MX record for my domain are also allowed to send emails for it.</p>
<p>For more information on SPF syntax, you can consult the <a href="http://www.openspf.org/SPF_Record_Syntax">official documentation</a>.</p>
<p>Without a properly configured SPF record, other mail servers might flag your emails as spam or outright drop them.</p>
<h3>Checking SPF record for inbound mail</h3>
<p>Now that we have set up our own SPF record, let's configure Postfix to check that other mail servers communicating with us have done the same.</p>
<p>First, let's add the two following lines at the end of /etc/postfix-policyd-spf-python/policyd-spf.conf :</p>
<p>Let's now edit the /etc/postfix/main.cf. In the "smtpd_recipient_restrictions" section, add the "check_policy_service" line as seen below :</p>
<p><a href="http://www.dkim.org/">DKIM</a> (DomainKeys Identified Mail) is a mechanism that validates a domain name identity for an email through cryptographic authentication.</p>
<p>While not mandatory, setting up DKIM improves the odds of emails sent from your server not being flagged as spam by other providers.</p>
<p>With this configuration, OpenDKIM will also check the key for inbound emails.</p>
<h3>Software side</h3>
<p>First, let's backup the original configuration file and create a folder for the configuration files :</p>
<p>For DKIM to work, you have to configure a DNS TXT record in your zone. This record was automatically generated by OpenDKIM in the mail.txt file mentioned earlier :</p>
<div class="highlight"><pre><span></span>mail._domainkey IN TXT &quot;v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCkJq0CW3tl2XHZ1CN5XdbqRDU7KfXOJ70nlwI09bHmDU63/Yz3J5rl863S0t2ncVHfIudZANj0OaiJe5HRR7WCsjuNIhQFfPFGIWLNClpxqdQVQURI38sAGeyn7Ed/Cor1AiWABzFWzel0kvXILw8K/NTzxaAPeSa9ttwQEgSmowIDAQAB&quot; ; ----- DKIM key mail for captainark.net
<p>All you have to do is to copy and paste this record in your DNS zone file.</p>
<p>To make sure that OpenDKIM is working, you can send an empty email to <a href="mailto:check-auth@verifier.port25.com">check-auth@verifier.port25.com</a>. You should receive a response with the following content :</p>
<p>It lets the owner of a domain name indicate that his email is protected by SPF and/or DKIM and what other providers should do with emails that do not pass those checks.</p>
<h3>Software side</h3>
<p>Once again, let's backup the original configuration file :</p>
<p>This tells other providers to not reject or quarantine emails should a SPF or DKIM check fail, but to send a daily report of those checks to postmaster@captainark.net.</p>
<p>For more information on the DMARC syntax, here is an <a href="https://support.google.com/a/answer/2466563?hl=en">article from Google</a>.</p>
<h2>Configuring Monit</h2>
<p><a href="http://mmonit.com/monit/">Monit</a> is a daemon that makes sure that other daemons are running. If they crash, it restarts them automatically. Is is not directly related to a mail server per say, but it's pretty easy to set up.</p>
<p>First, as always, let's backup the original configuration file :</p>
<p><a href="http://www.rainloop.net/">Rainloop</a> is a web-based email client. I won't go into details on how to configure it in this tutorial ; here's a link to the <a href="http://www.rainloop.net/docs/installation/">official documentation</a>.</p>
<p>You'll need a web server with PHP 5.3+ to run Rainloop. You do not have to run Rainloop on the same host as your mail server. No database is required.</p>
<h2>Conclusion</h2>
<p>We now have a mail server that should be running pretty smoothly. It could still be improved by setting up things such as greylisting or virus detection.</p>
<p>If you have found this tutorial useful, if you've found an error in it or if you have any question, please feel free to leave a comment below or to contact me on <a href="https://twitter.com/captainark">Twitter</a>.</p>
<h2>References</h2>
<p>Here are the tutorials I used to set up my own mail server :</p>
<ul>
<li><a href="http://sealedabstract.com/code/nsa-proof-your-e-mail-in-2-hours/">A complete tutorial on setting up a mail server</a></li>
<li><a href="https://www.digitalocean.com/community/tutorials/how-to-configure-a-mail-server-using-postfix-dovecot-mysql-and-spamassasin">A third tutorial from DigitalOcean</a></li>
<li><a href="https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-dkim-with-postfix-on-debian-wheezy">A tutorial on setting up OpenDKIM</a></li>
<li><a href="https://guillaume.vaillant.me/?p=481">A tutorial on setting up OpenDMARC</a> (in french)</li>
