@ wrote... (4 months, 4 weeks ago)

The curl manpage isn't super clear on this, but to read from stdin you need to prepend the dash with a @.

json_producer | curl -d @- -H 'Content-Type: application/json' http://example.com/json_consumer
Category: tech, Tags: linux, shell
Comments: 0
@ wrote... (6 months, 1 week ago)

Uploading to Minio (or S3) in a script is a bit tricky.

Update: now on GitHub with some Python versions.

#!/bin/bash

# usage: ./minio-upload my-bucket my-file.zip

bucket=$1
file=$2

host=minio.example.com
s3_key='secret key'
s3_secret='secret token'

resource="/${bucket}/${file}"
content_type="application/octet-stream"
date=`date -R`
_signature="PUT\n\n${content_type}\n${date}\n${resource}"
signature=`echo -en ${_signature} | openssl sha1 -hmac ${s3_secret} -binary | base64`

curl -v -X PUT -T "${file}" \
          -H "Host: $host" \
          -H "Date: ${date}" \
          -H "Content-Type: ${content_type}" \
          -H "Authorization: AWS ${s3_key}:${signature}" \
          https://$host${resource}
Category: tech, Tags: linux, python, shell
Comments: 0
@ wrote... (6 months, 1 week ago)

I highly doubt I've invented something new, but I did independently invent it, so I have that going for me, which is nice. Plus I'm stitching together lots of great open source pieces which we all have going for us.

Anyhow, this is kind of difficult to explain. And complicated… with lots of moving parts… but is super useful once you get it to work.

As high level as possible, when I do a git push, another machine reloads new configuration files. And notifies me. All within a second.

more…

Category: tech, Tags: devops, docker, git, linux
Comments: 0
@ wrote... (1 year, 6 months ago)

I seem to need to find files and sort them on modification time just often enough to never remember how but knowing that there's a good way to do it.

Here's the best way I've found so far, it handles spaces correctly and executes a command (ls in this case) on a per file basis. Change (or delete) -n1 to execute the command on multiple files simultaneously.

find . -type f -printf "%T@ %p\0" \ # list all files "seconds-since-epoch filename NULL"
  | sort -z -nr \                   # reverse sort based on seconds
  | cut -z -d' ' -f2- \             # only print filename
  | grep -zZ txt$ \                 # only keep txt$
  | xargs -0 -n1 ls                 # for each line, execute ls

or on one easy copy-paste line:

find . -type f -printf "%T@ %p\0" | sort -z -nr | cut -z -d' ' -f2- | grep -zZ txt$ | xargs -0 -n1 ls

The key here is \0 to output a null character at the end of each filename and then having each command in the pipe honour that null character (via -z|-Z|-0).

Category: tech, Tags: linux
Comments: 0
@ wrote... (1 year, 7 months ago)

Now that's is so easy for me to create new containers and VMs with Proxmox, that's exactly what I've been doing. However, I like to be a good netizen and use https.

Here's my nginx config that lets me easily add a new LetsEncrypt certificate to a new vm/container for a new webapp.

more…

Category: tech, Tags: linux, nginx
Comments: 0
@ wrote... (1 year, 7 months ago)

When I was setting up Deluge to run headless on my linux server the deluge-web wasn't saving any settings and nothing was working. Turned out to be an easy fix if you know how.

The problem was the that the web-ui wasn't auto connecting to the deluged backend. This caused the connection manager to always pop up.

Anyhow, assuming your web-ui and deluged are running on the same machine edit the web.conf file and make sure that default_daemon is populated.

# /var/lib/deluge/config/web.conf -- your path will likely be different
...
"default_daemon": "localhost:58846",
...
Category: tech, Tags: linux
Comments: 0
@ wrote... (1 year, 8 months ago)

I've been making LXC containers in Proxmox like a fiend. I'm toally loving Proxmox, if you want several virtual machines I highly recommend it.

Anyhow, trying to run avahi-daemon in the containers often fails. I'm not the first to notice this but the answers were unsatisfying until I found a suggestion to try running with --no-rlimits. That seems to do the trick!

But how to get systemd to run it that way? Very simply as it turns out.

systemd

systemctl edit avahi-daemon.service

Add then in the text editor that opens up, enter the following:

[Service]
ExecStart=
ExecStart=/usr/sbin/avahi-daemon -s --no-rlimits

see comment #2 for a script friendly way to do this

libnss-mdns

libnss-mdns sometimes doesn't install properly though. If you can't ping/lookup other .local hosts then edit /etc/nsswitch.conf and change…

hosts:          files dns

to

hosts:          files mdns4_minimal [NOTFOUND=return] dns

If anybody wants to write me an ansible script to do that I would totally buy you a beer.

Too late, I had to write it myself, container.yml.

Category: tech, Tags: linux, lxc
Comments: 2
@ wrote... (1 year, 11 months ago)

While trying to proxy my main nginx instance to a GitLab docker container I wasted hours and hours and hours trying to fix the following error:

fatal: unable to access 'https://gitlab.burgundywall.com/kneufeld/myproject.git/': \
SSL read: error:00000000:lib(0):func(0):reason(0), errno 54

It turns out that nginx config option ssl_session_cache is super f'n important to not screw up. I'm not totally sure what the problem is, but in my main server clause i had

ssl_session_cache shared:SSL:1m;

and I didn't have any such option in my server gitlab stanza. So something something something I could not do any git commands via https.

And even with logging everything looked okay

GIT_CURL_VERBOSE=1 git clone https://gitlab.burgundywall.com/kneufeld/myproject.git
Cloning into 'myproject'...
* Couldn't find host gitlab.burgundywall.com in the .netrc file; using defaults
*   Trying 192.168.5.6...
* Connected to gitlab.burgundywall.com (192.168.5.6) port 443 (#0)
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /usr/local/etc/openssl/cert.pem
  CApath: none
* NPN, negotiated HTTP1.1
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: CN=gitlab.burgundywall.com
*  start date: Sep  3 16:53:00 2016 GMT
*  expire date: Dec  2 16:53:00 2016 GMT
*  issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
*  SSL certificate verify ok.
> GET /kneufeld/myproject.git/info/refs?service=git-upload-pack HTTP/1.1
Host: gitlab.burgundywall.com
User-Agent: git/2.9.3
Accept: */*
Accept-Encoding: gzip
Pragma: no-cache

* SSL read: error:00000000:lib(0):func(0):reason(0), errno 54
* Closing connection 0
fatal: unable to access 'https://gitlab.burgundywall.com/kneufeld/myproject.git/': SSL read: error:00000000:lib(0):func(0):reason(0), errno 54

except it didn't work.

Anyhow, when I finally figured out that ssl_session_cache was the issue and did some reading I just made sure that each ssl server has it's own cache.

ssl_session_cache shared:SSL-gitlab:1m;

Hours.

Category: tech, Tags: git, linux, nginx
Comments: 0
@ wrote... (2 years, 3 months ago)

At work I upgraded to Ubuntu 16.04 LTS and every time I ran gvim I got a bunch of errors.

(gvim:19805): Gtk-WARNING **: /usr/lib/x86_64-linux-gnu/gtk-2.0/2.10.0/immodules/im-fcitx.so: cannot open shared object file: No such file or directory
(gvim:19805): Gtk-WARNING **: Loading IM context type 'fcitx' failed

more…

Category: tech, Tags: linux, ubuntu
Comments: 0
@ wrote... (2 years, 4 months ago)

It took a bit of doing but I was able to install Webzash in a Docker container behind a nginx web server. Here's how I did it.

Plus some notes on database permissions.

more…

Category: tech, Tags: accounting, linux, nginx
Comments: 0