GitLab CI for
Omnis Studio
testing, ci, docker and you
EurOmnis 2018
by Kurt Neufeld @ Artsman
About Me
- Programmed a bunch in C++
- Then programmed a bunch in Python
- Been running Linux at home and work for 20+ years
- Been at Artsman for just over a year in Ops role
- Expert copy-paster from StackOverflow, I guess
you could say I'm full stack
- Ran Omnis Studio for the first time 2 days ago
Salient Points
- Where the industry came from
- Where the industry is going
- Where Artsman came from
- Where Artsman is going
- Examples & Demo?
Industry History
Industry History
- OK, I'm getting bored of this now
- 1990 - file servers, client/server architecture
- 2000 - the web explodes
- 2010 - virtualization and the cloud
Industry Prognostication
Or guessing if English isn't your first language
- containers
- containers
- probably some more containers
- more cloud
Artsman History
- started 30 years ago by a Scot on a dare
- we sell ticketing software
- Omnis for native OSX & Windows
- virtual OSX on our collocated datacenter
Artsman Prognostication
I have some input on this one
- Omnis now supports Linux
- cloud Linux
- containers!
So...
What does all this Linux and containers have
to do with Omnis and us here today?
Glad you asked
Definitions
- CI
Continuous Integration
- Continuous Integration
automated build and testing of all commits
- Container
like a VM but shares the host kernel
Modern Stack
Applications |
Omnis, etc |
Container |
Docker, LXC |
OS |
Linux, no other viable options |
Machine |
Physical, Amazon, Google, VMWare |
Artsman Stack
Apps |
LXC moving towards Docker |
Orchestration |
Nomad; Kubernetes (not Rancher) |
LXC or VM |
apps in LXC, Docker in VMs |
OS |
Proxmox; VMWare, Amazon |
Machines |
2 datacenters, east and west |
Artsman Stack (future)
Docker |
running apps |
Orchestration |
probably Kubernetes |
OS |
Proxmox (or similar) |
Hybrid Cloud |
Private & Public |
Artsman Stack
- Hashi Stack
- Consul
- Nomad
- Vault (future)
|
- Monitoring
- Nagios
- Prometheus (sorta)
- Grafana (sorta)
|
- ELK Stack
- Elastic Search
- Log Stash
- Kibana
|
|
Artsman Stack
Other things we can't live without
- Pfsense
- PostgreSQL
- Python
- Redis
- Ceph
- FreeIPA
- nginx
|
- Rocket.Chat
- Gitlab
- Fabio
- Minio
- Open Source
- Omnis Studio!
|
Artsman Stack (future)
More specifically...
- Get Theatre Manager Server running on Linux
- Get off of virtual OSX
- Radically reduces hardware requirements
- Check up on things once a week from a beach
How to get there...
That's a lot of changes, sounds dangerous
It can be, but with lots of tests (which we have)
then it's manageable.
So specifically, how to do that?
Workflow
- write some sweet code
- git commit
- git push this is where magic happens
- a new artifact is built
- tests automatically get run
- you could auto deploy here
Write Some Sweet Code
You're on your own with this one.
git commit
You are using git right?!?
git push
I recommend GitLab (self hosted or paid service)
or GitHub (paid service).
Through the magic of webhooks, scripts
are run when pushes are made. Some of those scripts can:
- run tests
- build artifacts
- deploy code
- notify via chat, email, etc
So Many Moving Parts
- gitlab to initiate everything
- docker to run gitlab-runners
- gitlab-runner accepts jobs from gitlab
- jobs get containers from docker-registry
- jobs do work; run tests, build artifacts
- artifacts get saved in gitlab
- containers get pushed to docker-registry
- nomad re-deploys new containers
- etc etc etc
I did all that before lunch
.gitlab-ci.yml (simplified)
stages:
- build
- test
build_deploy_image:
stage: build
image: ${docker_registry}/public/alpine:3
script:
- docker build --pull -t ${tag_branch} -f Dockerfile .
- docker push ${docker_registry}/${docker_image}:${CI_COMMIT_REF_SLUG}
pytest:
stage: test
image: ${docker_registry}/${docker_image}:${CI_COMMIT_REF_SLUG}
services:
- postgres:10
script:
- ./make_database
- pytest -s
Automated Tests
- computers don't forget to run them
- stop deploy chain if they fail
- notify developer immediately on problem
- can't start building on broken foundation
Demo Preconditions
- gitlab, nomad, consul, fabio
- base docker image with homnis in it
Demo
deploy new version of omnis app
- edit and commit omnis app
- ci will build new docker image
- manually deploy new docker image
Demo Deficiencies
- should never check-in generated code (ie: .lbs)
- should compile dir tree into .lbs
- no tests written (homnis has issues)
Production Differences
- dns entry for service
- config in load balancer
- random port