Alkali v0.5.5 has now escaped.

Code at github.com and docs at readthedocs.org.

Major features are:

• adding BoolField
• adding CSVStorage, load/save csv files
• queries are bit more efficient/fast, don't copy as many objects by default
• some doc cleanups

You can now nicely render markdown in your console/terminal with ConsoleMD in glorious color.

Note the subtlety changing hues of the subtopics, the brilliance of auto-incrementing list counters, the dulcet notes of the embedded python, the elegant italics of the block quote. ConsoleMD has it all! At least with regards to my small test files.

Code is on github.com

Alkali v0.5.4 has now escaped.

Code at github.com and docs at readthedocs.org.

Major features are:

• implemented Query.distinct
• implemented Query.annotate
• implemented Query.aggregate functions: Sum, Count, Min, Max
• Fields are now descriptors on Model instance
• added ForeignKey field
• models now cascade delete when ForeignKey instance is deleted
• minor speed ups
• IntField now has auto_increment property
• Query returns copies of Model instances
• added signals on model creation/deleting/saving/etc
• better documentation

I had a hell of a time trying to make a nice help command while using click.

I wanted the ability to have a help sub-command for each real command.

mycommand --help
mycommand help

mycommand foo --help
mycommand help foo

It's actually fairly easy (once you know how).

CTX_SETTINGS=dict(help_option_names=['-h','--help'])

@click.group(context_settings=CTX_SETTINGS)
@click.pass_context
def cli(ctx, **kw):
    ...

@cli.command()
@click.pass_context
def foo(ctx):
    ...

@cli.command()
@click.argument('topic', default=None, required=False, nargs=1 )
@click.pass_context
def help(ctx, topic, **kw):
    if topic is None:
        print ctx.parent.get_help()
    else:
        print cli.commands[topic].get_help(ctx)

So it turns out I hadn't updated my Fedora installation for over two years (protip: don't run Fedora on a server kids) so I did a quick series of upgrades and went from Fedora 21 to Fedora 25.

Unfortunately that means that my PostgreSQL database was at version 9.3 but the installed software was version 9.5.

So can you upgrade from 9.3 to 9.5? No. But it's not impossible either.

more…

It turns out that after I imported a bunch of databases from mysql all the ownerships were wrong so any future django migrations would fail.

Based on this stackoverflow.com answer I made a quick script to mass change several databases and its tables all at once.

more…

I'm in the process of merging databases with identical schemas. This will cause primary key collisions which one generally wants to avoid.

So there are a few things you have to do:

more…

I kept getting Incorrect string value warnings when trying to import UTF-8 strings into Mysql. It turns out that Mysql has pretty questionable utf-8 support as seen from this stackoverflow.com question.

Well enough was enough so I decided to migrate some datbases to Postgresql. Here are some notes/scripts that helped me.

more…

Packages change pretty frequently when you're using Homebrew on OSX, including Python.

I noticed that I had several instances of lots of large packages installed (Boot, MacVim, Python, etc) so I ran brew cleanup and that freed up approx 9 gigs of disk! Unfortunately it also broke many virtual environments since the version of python that they were pointing at no longer existed.

What to do what to do…

Based on this Stack Overflow question I wrote a script to update a single virtual environment or all of them.

Note, this only updates virtual enviroments created with virtualenvwrapper. Also, you may have to reinstall your packages, but hopefully not.

#!/bin/bash

if [ -z "$1" ]; then
    echo "usage: $0 virtualenv|all"
    exit 1
fi

if [ -z "$WORKON_HOME" ]; then
    echo "you must export WORKON_HOME"
    exit 1
fi

if [ "$1" = "all" ]; then
    echo "udpating all virtualenvs"
    cd $WORKON_HOME

    for name in $(find . -type d -maxdepth 1 -mindepth 1|xargs -n1 basename); do
        $0 $name
    done

    exit 0
fi

cd $WORKON_HOME
virtenv="$WORKON_HOME/$1"

echo "deleteing $1"

rm -f  $virtenv/.Python
rm -f  $virtenv/bin/pip
rm -f  $virtenv/bin/pip2
rm -f  $virtenv/bin/pip2.7
rm -f  $virtenv/bin/python
rm -f  $virtenv/bin/python2
rm -f  $virtenv/bin/python2.7
rm -fr $virtenv/include
rm -f  $virtenv/lib/python2.7/* 2> /dev/null
rm -fr $virtenv/lib/python2.7/distutils
rm -f  $virtenv/lib/python2.7/site-packages/easy_install.*
rm -fr $virtenv/lib/python2.7/site-packages/pip
rm -fr $virtenv/lib/python2.7/site-packages/pip-*.dist-info
rm -fr $virtenv/lib/python2.7/site-packages/setuptools
rm -fr $virtenv/lib/python2.7/site-packages/setuptools-*.dist-info

source `which virtualenvwrapper.sh`

echo "creating $1"
mkvirtualenv -q $1

I seem to need to find files and sort them on modification time just often enough to never remember how but knowing that there's a good way to do it.

Here's the best way I've found so far, it handles spaces correctly and executes a command (ls in this case) on a per file basis. Change (or delete) -n1 to execute the command on multiple files simultaneously.

find . -type f -printf "%T@ %p\0" \ # list all files "seconds-since-epoch filename NULL"
  | sort -z -nr \                   # reverse sort based on seconds
  | cut -z -d' ' -f2- \             # only print filename
  | grep -zZ txt$ \                 # only keep txt$
  | xargs -0 -n1 ls                 # for each line, execute ls

or on one easy copy-paste line:

find . -type f -printf "%T@ %p\0" | sort -z -nr | cut -z -d' ' -f2- | grep -zZ txt$ | xargs -0 -n1 ls

The key here is \0 to output a null character at the end of each filename and then having each command in the pipe honour that null character (via -z|-Z|-0).