@ wrote... (2 years ago)

For various reasons I don't really like black even if it's very good at what it does.

The readability of a Django model with aligned fields vs after running black is a good example of why I'd never use it by default.

Anyhow…

It would be nice to use black occasionally on snippets of code from inside my editor (vim) but that almost never works as the first line is already indented.

more…

Category: tech, Tags: python
Comments: 0
@ wrote... (2 years, 7 months ago)

Not gonna lie, getting this to work was hard, plus I'm still not super happy with the results but it's good enough. You can set a static ip in the Proxmox cloud-init disk and that's what you'll end up with on first boot.

Note: this probably also applies to Ubuntu 18.04

more…

Category: tech, Tags: devops, linux, proxmox, ubuntu
Comments: 6
@ wrote... (2 years, 7 months ago)

We currently host some of our customers on our private cloud and as a result we need to manage an ssl certificate for them as well. Traditionally we've used a standard SSL provider but this was not only pricey, more importantly it was a lot of work for the support staff to get certs, update certs, deploy certs, etc.

more…

Category: tech, Tags: devops, nginx
Comments: 0
@ wrote... (3 years, 3 months ago)

I just figured out how to compress any file to a fixed size. I can't even begin to tell you how revolutionary this is.

This will change the world over night:

  • you'll be able to download a 4k movie in a Tweet
  • upload an image of your hard drive in milliseconds
  • compression speed is O(n), as fast as your disk can read
  • written in python, zero external dependencies
  • any file compresses to ~ 130 bytes, yes, you read that right

Because I believe in Open Source I'm releasing this to the world. Pull requests for decompress() are welcome.

more…

Category: tech, Tags: compsci, sarcasm
Comments: 0
@ wrote... (3 years, 5 months ago)

Here's a crappy little script that automates the steps in Fix proxmox containers that won't start

#!/bin/bash

if [ -z "$1" ]; then
    containers=`pct list | grep stopped | awk '{print $1 " "}'`
else
    containers=$1
fi

echo "restarting containers: $containers"

for m in $containers; do
    lxc-start -n $m &
    echo "force started $m"
done

echo "sleeping for 2 minutes while containers start"
sleep 120

for m in $containers; do
    echo "shutting down $m"
    pct shutdown $m;
    echo "starting $m"
    pct start $m &
done
Category: tech, Tags: lxc, proxmox
Comments: 0
@ wrote... (3 years, 6 months ago)

With the version v0.10.0 of Nomad a very important new feature has landed, network namespaces. Network namespaces allows integration with Consul Connect and that allows you to connect to remote services without having to know their address and port. Plus some security stuff but I care more about my services working than being securely broken.

You can read more official information here:

Consul Connect in Nomad are called sidecars. However… sidecars are an advanced feature and the docs and examples are very confusing. At least I found them very confusing. So here is a slightly modified version of the example job file with lots of comments that hopefully explains how to use this great new feature.

more…

Category: tech, Tags: consul, nomad
Comments: 0
@ wrote... (3 years, 7 months ago)

My cluster was throwing warning Legacy BlueStore stats reporting detected and we could just not abide that.

Here's a simple way to upgrade:

cd /var/lib/ceph/osd
ls                                   # note your osd numbers

ceph osd set noout

for n in 7 8 9 10 11 12 ; do 
  systemctl stop ceph-osd@$n.service; 
  ceph-bluestore-tool repair --path ceph-$n; 
  systemctl start ceph-osd@$n.service; 
  systemctl status ceph-osd@$n.service;
done

# I like to wait until the cluster goes back to green 
# before doing the same on the next host

ceph osd unset noout
Category: tech, Tags: ceph
Comments: 0
@ wrote... (3 years, 8 months ago)

Sometimes an lxc container will refuse to start, usually after something goes wrong on the host. The key is to manually start the container and wait for some timeout or something, then the container will start properly.

Update: this does more or less the same but with less work.

pct list   # note the vmid, replace 100 101 below as appropriate

for m in 100 101; do lxc-start -n $m & done

# after a minute or so you'll see the background tasks finish in the console
pct list   # lxc containers should be running

# Note, that `systemd` will still show the tasks as failed since it didn't
# start them, so now lets stop/start them properly
for m in 100 101; do pct shutdown $m; pct start $m & done

The following is still valid but I now use the above method.

[root@proxmox1 ~]
# pct start 125
pJob for pve-container@125.service failed because a timeout was exceeded.
See "systemctl status pve-container@125.service" and "journalctl -xe" for details.
command 'systemctl start pve-container@125' failed: exit code 1

[root@proxmox1 ~]
# /usr/bin/lxc-start -n 125 -F

# good long wait, over a minute
# login and then `poweroff`

[root@proxmox1 ~]
# pct start 125 && echo $?
0

tl;dr

  1. lxc-start -n 125 -F
  2. login and poweroff
  3. pct start 125
Category: tech, Tags: lxc, proxmox
Comments: 0
@ wrote... (3 years, 10 months ago)

Before upgrading to Proxmox 6 you need to upgrade to Corosync 3. Here's an ansible playbook that will automate that…

more…

Category: tech, Tags: ansible, proxmox
Comments: 0
@ wrote... (4 years, 1 month ago)

We recently upgraded our network to 10 Gbit and were really hoping to see monumental speed increases in our ceph cluster.

One of our benchmarks was pgbench and to say we were sad would be an understatement…

more…

Category: tech, Tags: ceph, postgresql
Comments: 6