@ wrote... (14 years, 3 months ago)

BurgundyWall is located in Calgary, Canada, and the inspiration for the domain name is was located at the end of my living room. If you're really curious you can read (a very little) about me.

If you're interested in any of my articles or if you need a skilled developer then send me an email or check out my resume.

I am available for part-time contract work and have a varied professional experience. So if you need help with embedded Linux application development, backend development, and general Linux (and networking) system administration please get in touch.

If you have some IT related problems that you want to go away I can probably make that happen.

Category: uncategorized
Comments: 0
@ wrote... (5 days, 20 hours ago)

My cluster was throwing warning Legacy BlueStore stats reporting detected and we could just not abide that.

Here's a simple way to upgrade:

cd /var/lib/ceph/osd
ls                                   # note your osd numbers

ceph osd set noout

for n in 7 8 9 10 11 12 ; do 
  systemctl stop ceph-osd@$n.service; 
  ceph-bluestore-tool repair --path ceph-$n; 
  systemctl start ceph-osd@$n.service; 
  systemctl status ceph-osd@$n.service;
done

# I like to wait until the cluster goes back to green 
# before doing the same on the next host

ceph osd unset noout
Category: tech, Tags: ceph
Comments: 0
@ wrote... (1 week, 2 days ago)

Sometimes an lxc container will refuse to start, usually after something goes wrong on the host. The key is to manually start the container and wait for some timeout or something, then the container will start properly.

[root@proxmox1 ~]
# pct start 125
pJob for pve-container@125.service failed because a timeout was exceeded.
See "systemctl status pve-container@125.service" and "journalctl -xe" for details.
command 'systemctl start pve-container@125' failed: exit code 1

[root@proxmox1 ~]
# /usr/bin/lxc-start -n 125 -F

# good long wait, over a minute
# login and then `poweroff`

[root@proxmox1 ~]
# pct start 125 && echo $?
0

tl;dr

  1. lxc-start -n 125 -F
  2. login and poweroff
  3. pct start 125
Category: tech, Tags: lxc, proxmox
Comments: 0
@ wrote... (2 months, 3 weeks ago)

Before upgrading to Proxmox 6 you need to upgrade to Corosync 3. Here's an ansible playbook that will automate that…

more…

Category: tech, Tags: ansible, proxmox
Comments: 0
@ wrote... (6 months ago)

We recently upgraded our network to 10 Gbit and were really hoping to see monumental speed increases in our ceph cluster.

One of our benchmarks was pgbench and to say we were sad would be an understatement…

more…

Category: tech, Tags: ceph, postgresql
Comments: 0
@ wrote... (7 months, 1 week ago)

There are lots of posts about setting up CD with Jenkins and Kubernetes but I haven't found any describing how to do it with Nomad and Gitlab.

So here's how I did it…

more…

Category: tech, Tags: cd, ci, gitlab, hashistack, nomad
Comments: 2
@ wrote... (9 months, 2 weeks ago)

I also found the docs for consul connect to be confusing. They don't clearly differentiate the difference between the client and server proxy.

Some declarations that are worth stating explicitly:

  • consul acl needs to be setup first, see consul acl for more info
  • acl and intention are used somewhat interchangeably here
  • client side consul connect proxies can only talk to other consul connect proxies
  • client side consul connect proxies can not talk directly to a service
  • the docs explaining -service vs -listen vs -upstream are terrible
  • I'll use the term proxy to mean consul connect process
  • the term service refers to the actual service (eg. redis)
  • the term server proxy refers to the proxy that connects to a real service
  • the term client proxy refers to the proxy that clients connect to

Having said all that, service mesh sounds like they're worth having.

Mitchell Hashimoto at least partly agrees with me.

more…

Category: tech, Tags: consul
Comments: 0
@ wrote... (9 months, 2 weeks ago)

I found the otherwise great consul docs to be very obtuse and confusing and maybe even wrong.

I'm running these commands against my home setup which only has a single consul server. In a more realistic setup you'll need to duplicate the config changes on all your consul servers and then restart them one at a time.

Ran against consul 1.4.0

more…

Category: tech, Tags: consul
Comments: 2
@ wrote... (9 months, 3 weeks ago)

I opened an issue on GitHub several months ago against the awesome fabio asking for a simple feature. To help foster community and contributors (I assume) the maintainer showed me the file to edit.

The problem, and why I ignored it for several months, was that I don't know Go. The patch itself is crazy simple, just editing an html template. The problem is the Go environment. How do you build, how do you test, etc…

Anyhow… today was the day I'd tackle this… what could go wrong?

more…

Category: tech, Tags: hatecomputers
Comments: 0
@ wrote... (9 months, 4 weeks ago)

On Tuesday, December 11, 2018 I received a phising email, redacted version on pastebin.

Thankfully I used a unique password because there it was, in the clear. The unique password showed me that it was for http://osnews.com.

I reached out to them immediately and got a response promptly but it didn't mention anything about disclosure. So I then asked David if he was planning to tell his users about the breach and he replied he would by the end of the week.

Here's a snippet of his response:

The very old custom CMS that OSNews runs on hasn’t been meticulously
updated, and it does appear that someone got ahold of our user data.

On Monday, with still no announcement I sent another email and asked again. David replied he would announce by end of day. Although several content posts have been added in the last week there has still been no announcement of the security breach.

So, after one week, I'm announcing for them.

  • osnews.com has been hacked
  • osnews.com kept user passwords in the clear
  • those email/password tuples are now in the wild

Damn.

Category: tech, Tags: osnews
Comments: 3