All posts by Bjørn

Certbot and apache

I promised a blog post detailing changes I needed to make to my apache config in order to move BRBcoffee to Https, but in hindsight there isn’t much to write about it, it’s basically just a refactor.

Certbot, the tool from EFF (written in Python, yay!) that gets ssl certs from Let’s Encrypt, doesn’t work with monolithic conf files with multiple hosts. I run all my projects on the same server, routing traffic based on the site address using apache’s VirtualHost directive. It used to look like this:

<VirtualHost *:80>
    DocumentRoot "/var/www/blog"
    ServerName blog.brbcoffee.com
    # Some more directives
</VirtualHost>
<VirtualHost *:80>
    DocumentRoot "/var/www/archive"
    ServerName archive.brbcoffee.com
    # Some more directives
</VirtualHost>
<VirtualHost *:80>
    ProxyPreserveHost On
    ProxyRequests Off
    ServerName cv.brbcoffee.com
    ProxyPass / http://localhost:5000/
    ProxyPassReverse / http://localhost:5000
</VirtualHost>

So what you need to do is rip all of that out, you don’t want it in there. In their place you want this:

IncludeOptional conf.d/*.conf

Depending on your packager, it may be that this directive is already somewhere in your httpd.conf file. If it is, great, just leave it be. After that you want to take each VirtualHost that you ripped out of the main httpd.conf, and place them in individual files, like so:

<VirtualHost *:80>
    DocumentRoot "/var/www/blog"
    ServerName blog.brbcoffee.com
    # Some more directives
</VirtualHost>

blog.conf

The configuration doesn’t change, it just needs to be in separate file for certbot to be able to do its thing. You see, after certbot goes out and gets your certificates it needs to add some rules to each vhost for redirecting traffic to ssl, which I guess they didn’t want to write a lot of ugly parsing code to be able to do in a program that really isn’t about that (although it should be trivial with BeautifulSoup.

Anyway, before, running certbot –apache probably didn’t work, it got the certs for you, but couldn’t edit your conf to do what we want. Now, when you run it, it’ll complete just fine. If you chose to set all traffic to go to https, it will add three redirect lines to your conf files, and it will create a new file as well, in my case, blog-le-ssl.conf. It’s identical to the old conf file, except that it is on port 443, and that it checks that mod_ssl is loaded. All of this is stuff we could have done ourselves, of course, but it’s a convenience thing.

So that’s all there is to it. Refactor your httpd.conf by separating each virtualhost into a different file, and run certbot –apache again.

Https

Welcome to Https BRBcoffee, and thank you to Let’s Encrypt and certbot for making it a breeze, mostly.
Tomorrow there will be a blog post up about the changes I needed to make to my Apache configuration in order to get certbot to play nice.

For now, glory in that green padlock! And don’t go to the archive if you want it to stay green. WordPress doesn’t automatically update image links, so I’ll need to fix that at some point. The CV though I had no trouble with, even though it’s a Flask app that Apache just redirects traffic to. Good job, Apache, good job, python!

Typescript is genius

I’m teaching myself typescript, because why not, and right off the bat I’m blown away by the genius that is using public as a prefix to constructor arguments.
If you haven’t seen it before, it looks like this:

class Human {
    constructor(public name, public age, public job){
        // do more constructor things here
    }
}


And that’s the same as doing this:

class Human {
    constructor(name, age, job){
        this.name = name;
        this.age = age;
        this.job = job;
    }
}


It just takes the argument and sets a field with that same name in the object! It’s not a revolutionary feature, but it saves so much time in the long run. Anyway, back to it. Look forward to reading about my journey through NativeScript soon, possibly, maybe.

Switching from screen to tmux

Hello avid readers of yesteryear!

I’ve recently moved from working in a Linux/Windows environment to a Linux/Windows/OS X environment, and as such I’ve had to make some small changes to my workflows. I’m here to tell you what went wrong and how to fix it (hint: it’s in the title)

XKCD comic about using old software configured for you
Relevant XKCD title text: 2078: He announces that he’s finally making the jump from screen+irssi to tmux+weechat.

Now I’m the guy who just has it set up the way I want. I use screen in linux, and I use screen in the amazingly named Bash on Ubuntu on Windows. It works how I need it to work and I’m able to get things done. Now we introduce Mac OS X to the mix, and a seemingly tiny problem arises:

screenshot of vim
vim in normal terminal session
Screenshot of vim with different colors
vim inside a screen session

Try to spot the difference. I’ll wait.

The problem is a vim plugin called airline, which uses a lot of colors while enhancing vim’s normal ui. Something happens with the way screen identifies itself, that confuses airline, and makes the colors change slightly. No big deal, but it also makes the text less readable, which can be a bigger problem. Now there exists a separately compiled gnu screen for mac, specifically made to fix problems with screen colors. That screenshot was taken using that binary, and it looks identical to the native one. I spent about 2 hours trying to figure out a workaround for this problem, but in the end I decided to just finally give tmux a try, I’d been meaning to get around to that anyway.

vim in tmux session
That was easy

Okay, so tmux can handle colors better than screen, but what about all the other features from screen that we’re used to and love? Well once you’ve remapped your escape key to the one you’re used to from Gnu screen, you should be totally fine. To do this just add unbind C-b with C-a, replacing ‘a’ with whatever key you prefer. Splitting panes is done with ‘%’ and ‘”‘ in tmux, but you can simply unbind and bind to whatever you’re used to. Detaching is the same as always, attaching is done with “attach” instead of “-r”, so fairly easy to remember. Mouse mode is just as easy to enable as before, just replace mousetrack on from .screenrc with set -g mouse on in .tmux.conf.

All in all there isn’t much to write about when moving from screen to tmux. They do the same job, but tmux does it better, since it’s been built for a modern world than the literally 30 year old GNU Screen. If you’re using screen still, give tmux a try. You may spend a little time in the config file at first remapping things, but I swear it’s worth it.

Apple made a watch (and told you it was a revolution)

So Apple decided they needed to jump on the wearable tech bandwagon, and revealed the Apple Watch at their recent keynote. I guess iWatch would be a misleading name? As a tech blogger and rehabilitated fanboy, I feel obliged to write about this. They also released two new phones. They’re bigger, and can now be turned sideways for landscape mode. Such innovation. There, I’ve covered the new phones.

Completely scrambled RGB data
Apple Watch, according to their own press photo

I’m a good law abiding guy, so instead of just taking some picture from Google, I went to the source and downloaded the second official press image. When I tried to open it in an image viewer, it wouldn’t even let me. I tried opening it in a document viewer, and what you see above is the output. Thinking the download had become corrupted, I tried again, with the same result. I’m on Linux, so I figured this was some proprietary Apple format silliness, and spent a few minutes trying to figure it out. Then I ran another test, downloading a press photo for their Magic Trackpad, and I could open that just fine. Their keynote was completely botched as it was, and now this. Wow.

This is the Watch
This is the Watch

There, that’s better. That’s a jpeg file, copied from the same site as the tiff. It’s lower res, and I didn’t agree to their Image User Agreement to get it, because of course they have one. It’s not unreasonable though, just says I can’t edit it or use it to sell stuff. Shown in that image is both what I see as their biggest innovation and biggest mistake.

It’s a square watch.

Come on, Apple, you can do better. Nobody wants a square watch. It’s got your logo on it, so it’ll sell, but seriously, come on. You went and made a new homescreen implementation, that wonderful app cloud, perfect for a round display, but then you just went and made it square anyway. This is the opposite of what Motorola did with their Moto 360, which is that they made a beautiful round watch, but some of the UI is obviously tuned for a square display. With this you might as well have stuck with the app grid we’re used to.

Aside from that, there are a few things I’d like to mention. I’m not a big fan of this release, and I didn’t expect I’d be, but somewhere deep inside I keep hoping that Apple will become revolutionary once more. Instead they settle for telling us how revolutionary they are. They take great pride in their seriously brand new super innovative digital crown you guys this thing is the shit. It’s supposed to make their touchscreen almost unnecessary, only it doesn’t. You can use it to scroll, zoom, and for input, but when demonstrated the guy just kept tapping the screen, using the crown mostly to set the clock, which, in fairness, is what it’s supposed to do.

Oh my God you guys we make the best chargers ever! The Watch charger is magical, you just put the charger on the back and it snaps into place, just like magsafe! Only it uses induction charging! So innovative.

It’s a charger. It hopefully works. We’ve got induction chargers for the other smartwatches already, and they’re on the market. You know what would make this innovative? If you actually shipped when you announced it, you know, like you used to?

Holy hell, you won’t believe how clever we’ve been with the hardware! It’s a quantum leap in technology,  nobody has ever done anything like this before. We can’t even put in words how amazingly we’ve packaged our hardware. Everything is in one module. This is insane, an industry first!

It’s just that we’ve got that. We’ve had it for a long time. There’s System on Chip, or SoC, which puts a lot of sensors and radios and stuff on one piece of silicon. We’ve got System in package, or SIP, which packages chips inside modules to deliver supercompact working systems. How is this not that? How is this innovation and an industry first, guys? You can’t just say you’re the first to do something when you aren’t, there are laws against that kind of ting, I think. Don’t believe me? This is a direct quote from Apple.com:

Configuring an entire computer system on a single chip is an industry first and represents a singular feat of engineering and miniaturization.

I sincerely hope that wearables take off in a big way, and I hope that Apple will help achieve that, but this is not the revolution we’re waiting for. This is another subpar watch, touted as the most brilliant thing we’ll ever see until Apple makes The New Apple Watch. There’s more to the Watch than the things I’ve mentioned, but I only write about things that get me excited, and there’s nothing else. This Apple Watch is mostly boring.

Yes, you can quote me on that.
All of it.

Configuring a linux firewall

So you’ve got your Linux server going, it’s configured the way you want it , and everything is coming up roses. You try to ping it, but the server doesn’t seem to exist. You’ve been blocked by some of the best/most insane firewall in the galaxy: iptables. A firewall’s job is to block unwanted traffic, and iptables takes its job seriously. By default, it. drops. everything. Got a http request incoming? Psh, drop that packet. I don’t care if you’ve got apache running. FTP request? Same story. Mysql? Nope.

A cat shoving things off a desk
This is iptables

Ssh is usually fine though, so we can log in and edit the rules. Iptables rules are added to rule chains. The only chain we’re interested in is the INPUT chain for now; We want to be able to receive http requests to our server, ssh connections, and nothing else. We’ll also want to allow existing connections to persist. These are the switches we’ll be using (you can find all these in the manpages, of course, but some are in the iptables-extensions manpage).

  • -F flushes the rulechains. This means exactly what you’d think.
  • -A [chain] adds a rule to the specified chain.
  • -I [chain] [number] same as -A but inserts rule at a given point in the chain.
  • -i [interface] specifies the interface the rule will act on. If you don’t specify this, the rule will act on all interfaces, including loopback (more on this later).
  • -p [protocol] specifies whether the rule is for tcp, udp, or whathaveyou.
  • --dport [port] further narrows down the packets to look at by checking which port they’re headed for.
  • -m [match] this is an extension that looks at the type of traffic the packet belongs to. We use it with:
  • --state [state], which asks a different module called conntrack whether the connection is INVALID, NEW, ESTABLISHED, RELATED, or UNTRACKED. This is magic, I have only a vague understanding of how it works.
  • -j [policy] says whether to accept or drop the packet.

Alright, let’s get to it. You can think of iptables as a sieve, where every rule along the way checks out a packet and decides whether to keep it or discard it. If the rule doesn’t match the packet, it moves further down the sieve to be checked by other rules. Therefore, at the end of it, if the packet doesn’t match any of our rules, we will just discard it. A good policy for internet traffic is that if you don’t know what it is, don’t touch it. Every rule we add gets added last in the chain/sieve.

A script demonstrating the use of iptables

And that’s it. We’ve configured our firewall. It will reset every time you reboot your server, but that isn’t often. I just keep a script like the one above to reconfigure it. You can get NetworkManager to configure it for you on boot, but I don’t really see the point unless you reboot your server all the time, which, I mean, why would you do such a thing?

Plugin rot and how to do regex replaces in mysql

I’ve spent a few hours over the past week restoring my old blog, which can now be reached at brbcoffee.com/archive/.  This will be the final chapter in that saga.

My old blog is sort of old. It started out on MySpace, moved to Google’s BlogSpot platform, then moved to Tumblr for a while because I got some new authors on board. Finally we moved to a WordPress solution.  From there on BRBcoffee has been on WordPress, but it’s moved around hosts like an alcoholic goes from bar to bar trying to get his fix. I’ve employed a bunch of plugins to do various things for me, from importing and exporting posts to providing language switching capabilities to advanced caching to image scaling to… you get the picture. This is all fine and dandy, but it does pose a few problems when you move around like I do.

  1. The image scaling edits your wp_posts database table, naturally. When you move, the scaled images don’t necessarily come with, resulting in a lot of broken links.
  2. The translation plugin just has it’s own tags in that same database table, and keeps two posts in one field. When the plugin isn’t there anymore, or isn’t compatible with your WordPress version, the tags become visible on the site, and you get two languages at once.
  3. Plugins are often developed by lone wolf devs, and they are never up to date when you check up on them years later.

So, we need to clean up the mess they’ve created, and we don’t install Yet Another Plugin™ to try to fix the problem. We need to dig into our database and fix things ourselves. There’s only one problem: Mysql is made for storing, finding, updating, removing, etc. It isn’t the tool for the job when you need to do complex things like searching for a pattern and replacing only the part of the string that matches that pattern. It can be done, I’m sure, but why would you do it when there are better tools for the job? Here’s how I do it:

  1. Export the table you need to edit with
    mysqldump -u USER -p DATABASE TABLE > dumpfile.sql
  2. Do work on the dumpfile with specialized tools like sed, awk, or even vim when in a pinch.
  3. Empty the table before reimporting the modified dump with
    mysql -u USER -p DATABASE -e "delete from TABLE;"
  4. Finally reimport the table with
    mysql -u USER -p DATABASE < dumpfile.sql

Mysql also comes with a tool called mysqlimport, but I had a problem with it, so I just used the standard mysql command in the end.  So what do those switches mean?

  • -p means this user has a password, ask me for it before doing anything
  • -u USER means use this username. Pretty obvious.
  • -e means execute. Execute the following quoted code.

The only weird thing is that -u takes an argument, but -p doesn’t. It looks like DATABASE should be the argument to -p, but it isn’t. DATABASE is simply the database you want to do work on. In my case, it was called “archive,” while yours might be “blog,” “sales_records,” or even “naughty_movies.” That’s commandline switches for you. In my opinion, a single dash should be reserved for switches, double for arguments, like so:

  • -p [ask me for a password]
  • --user <name> [I'm logging in as name, how quaint]

Oh well, you get used to it. That’s how you should do many complex updates on a migrated database. This isn’t for when you want do do one small change on a live system. The 3rd step removes all records from the specified table. Your system will practically be offline until you’ve reimported the changed data.

If you wish this were simpler, you’re not alone. Behavior like this has been a feature request in mysql since 2007. Go there and click affects me, and maybe in another 7 years we’ll have it built in. Till then, use the method described above, and for heaven’s sake, don’t let the plugin rot get your blog too. If you can do it without a plugin, do it without a plugin.

And the archive is restored!

I did it! I partially restored the dead disk, and I did the necessary doctoring on the old WordPress database dump. Sadly, my OffTopic blog wasn’t in the backup tarball, so that’s gone forever. It only had a handful of posts anyway, so no big loss.

Anyway, brbcoffee.com/archive/ has everything dating back to february 2009. I was 18 when I wrote that stuff. Don’t judge. Some of the images are gone, as is the nature of the internet. Sorry about that.

Bye