My wife runs an after-school pottery program in various elementary schools around town. Parents sign their kids up for the classes, and then my wife’s staff go out to the various schools and teach basic ceramics to the kids. All the existing student registration systems for this are both wildly expensive, and they suck. She asked me if I could solve at least one of those problems for her.
So while going about learning how to use kubernetes, the fact that it requires a registry didn’t escape my notice. For my personal stuff I don’t mind just using Docker Hub but for projects that are a bit more sensitive, like this one it would be nice to just have a private registry. Docker makes it pretty easy to setup a registry but I figured I should write down the process as we go along.
First I create a new container for the registry. I have a profile (
that sets the permissions on the LXC containers so that docker can happily run
inside the container.
config: security.nesting: "true" user.user-data: | #cloud-config package_upgrade: true apt: preserve_sources_list: true sources: docker.list: source: deb [arch=amd64] https://download.docker.com/linux/ubuntu $RELEASE stable keyid: 7EA0A9C3F273FCD8 packages: - docker-ce - docker-ce-cli - containerd.io description: Docker LXD profile devices: eth0: nictype: bridged parent: lxdbr0 type: nic root: path: / pool: default type: disk name: docker
A few tricks to this, first
security.nesting needs to be enabled so that
docker has permissions to run. Second the
user.user-data is a cloud-init yaml
script that installs docker. By default the container is attacked to the
private bridged network that lxc sets up. Using
lxc profile edit docker will
let you paste the above file into your lxc settings.
With that setup we’re ready to launch the registry.
lxc launch -p docker ubuntu: docker-registry
We’ll need to wait a few minutes for lxc to finish running the cloud-init script, but when we’re done we should have a new lxc container with docker installed. We can test this with:
lxc exec docker-registry -- docker run hello-world
If that outputs successfully then we’re good to move on to installing the
registry. Installing a registry is pretty simple, really at it’s base it’s
docker run -d --restart=always --name registry registry, but that leaves us
with a bare container running in a private network inside your LXC. If all of
your other services are on the same LXC bridge network, then that may be enough
If however you need to access it from outside the LXC network we’ll need to do
a few things. First let’s start with exposing the LXC container to the public.
We’re gonna use the
macvlan profile we setup in part 3 to expose our
container to a public IP.
lxc profile add docker-registry macvlan lxc restart docker-registry
We will need to update the guest OS in the container to use the new IP. I’m using the latest LTS of ubuntu so this means editing the netplan files to look something like:
network: version: 2 ethernets: eth0: dhcp4: no addresses: - 192.168.0.3/24 # USE YOUR REAL IP gateway: 192.168.0.1 # USE YOUR REAL GATEWAY nameservers: addresses: - 220.127.116.11 - 18.104.22.168
Getting this configuration into my container just required the following.
lxc exec docker-registry -- bash -l vi /etc/netplan/50-cloud-init.yaml # paste in the above YAML netplan generate netplan apply
If everything worked correctly, you should be able to ping from the above host,
and ping the above host from an outside address. Next we’ll need to create an A
record for this address, I configured
docker.tamarou.com via the Web GUI at
my domain registrar and it’s beyond my ability to advise you on how to do it
Assuming that the A record is live, and you can now setup an SSL certificate for the docker registry using Letsencrypt.
lxc exec docker-registry -- bash -l docker run -it --rm --name certbot -p 80:80 \ -v "/etc/letsencrypt:/etc/letsencrypt" \ -v "/var/lib/letsencrypt:/var/lib/letsencrypt" \ certbot/certbot certonly
If this works correctly it will kick off an interactive session asking you a
few questions. You want to choose
Spin up a temporary webserver and
provide it with the domain you set up with the A record.
Securing communication is great, but still anybody can push anything into our
repo. By default docker only allows
htpasswd authentication, requiring you to
drop to a web-server like Nginx to get anything fancier. That’s fine for our
purposes here, integrating with your own system is left as a lemma for the
Create a new
htpasswd file somewhere,
htpasswd -cB /etc/htpasswd perigrin
If we need to add another user (let’s say for our scrum master it’s as simple as running the following and supplying a password:
lxc exec docker-registry -- htpasswd -B /etc/htpasswd trog
Now we can setup the docker registry to point at this. I went and created a startup script so I was sure to get all these arguments correct, you’re welcome to attempt to type it all correctly or just copy-and-paste.
docker run -d --name registry --restart=always \ -p 443:5000 \ -e REGISTRY_HTTP_ADDR=0.0.0.0:5000 \ -e REGISTRY_HTTP_TLS_CERTIFICATE=/etc/letsencrypt/live/docker.tamarou.com/fullchain.pem \ -e REGISTRY_HTTP_TLS_KEY=/etc/letsencrypt/live/docker.tamarou.com/privkey.pem \ -e REGISTRY_AUTH=htpasswd \ -e REGISTRY_AUTH_HTPASSWD_REALM="Docker Registry Realm" \ -e REGISTRY_AUTH_HTPASSWD_PATH="/etc/htpasswd" -v /etc/letsencrypt:/etc/letsencrypt \ -v /var/lib/docker/registry:/var/lib/registry \ -v /etc/htpasswd:/etc/htpasswd \ registry
Assuming everything worked we can push a test image to the repo this way. On a
machine that docker is installed on (could be the lxc container we just built,
lxc exec docker-registry -- bash -l is your friend) do the following:
docker run hello-world docker tag hello-world docker.tamarou.com/hello-world # replace with your domain docker push docker.tamarou.com/hello-world
This should report that the image was pushed. From anywhere with
curl you can do the following:
curl -X GET https://docker.tamarou.com/v2/_catalog # again make this match your own domain
It will return a JSON listing of the repositories, hopefully now including
Much of this post was based on a series of posts by exoscale. I’ve just re-organized thins a bit so that the process is cleaner and discussed how all of this interacts with my LXC setup.