111 lines
3.3 KiB
Markdown
111 lines
3.3 KiB
Markdown
|
# Harbor Desk Production Deployment
|
||
|
|
|
||
|
|
This deployment targets Ubuntu amd64 at `daniel@10.0.0.33` and serves
|
||
|
|
`threads-tool-dev.30cm.net`.
|
||
|
|
|
||
|
|
## Layout
|
||
|
|
|
||
|
|
- Host Nginx serves immutable static web releases.
|
||
|
|
- Gateway uses blue/green systemd slots on `127.0.0.1:8888` and `:8889`.
|
||
|
|
- Worker uses matching systemd slots and guarded Mongo/Redis leases.
|
||
|
|
- Docker Compose runs MongoDB, Redis, MinIO, Prometheus, Grafana, Loki,
|
||
|
|
Alertmanager, Alloy, node-exporter, cAdvisor, and blackbox-exporter.
|
||
|
|
- Runtime secrets stay in `/etc/harbor/harbor.env` with mode `0600`.
|
||
|
|
- Releases stay under `/opt/harbor/releases`; `/opt/harbor/current` changes
|
||
|
|
atomically.
|
||
|
|
|
||
|
|
## First Install
|
||
|
|
|
||
|
|
The dedicated SSH key is expected at `~/.ssh/harbor_deploy`.
|
||
|
|
|
||
|
|
```bash
|
||
|
|
./deploy/prod/bootstrap-server.sh
|
||
|
|
./deploy/prod/release.sh
|
||
|
|
```
|
||
|
|
|
||
|
|
Bootstrap asks for remote sudo once. It then grants `daniel` passwordless sudo
|
||
|
|
only for the validated activate, rollback, and status scripts.
|
||
|
|
|
||
|
|
Edit external provider credentials after bootstrap:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
ssh -i ~/.ssh/harbor_deploy daniel@10.0.0.33
|
||
|
|
sudoedit /etc/harbor/harbor.env
|
||
|
|
```
|
||
|
|
|
||
|
|
Restart the active services after changing runtime env values.
|
||
|
|
|
||
|
|
## Normal Release
|
||
|
|
|
||
|
|
```bash
|
||
|
|
./deploy/prod/release.sh
|
||
|
|
```
|
||
|
|
|
||
|
|
The command runs backend tests/vet, frontend tests, builds Linux amd64 binaries
|
||
|
|
and static assets, uploads a checksummed archive, migrates forward, starts the
|
||
|
|
inactive gateway, checks health, reloads Nginx, then hands work to the new
|
||
|
|
worker. The old gateway remains available for Nginx connection draining.
|
||
|
|
|
||
|
|
If an earlier inactive gateway still has established connections, activation
|
||
|
|
refuses to replace it. Retry after those requests finish.
|
||
|
|
|
||
|
|
To deploy an already built artifact:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
./deploy/prod/release.sh deploy/prod/artifacts/harbor-*.tar.gz
|
||
|
|
```
|
||
|
|
|
||
|
|
## Rollback
|
||
|
|
|
||
|
|
```bash
|
||
|
|
./deploy/prod/rollback.sh
|
||
|
|
```
|
||
|
|
|
||
|
|
Rollback switches application code only. Never automatically run a down
|
||
|
|
migration; production migrations must remain backward compatible.
|
||
|
|
|
||
|
|
## TLS
|
||
|
|
|
||
|
|
The public hostname is Cloudflare-proxied while the origin has a private IP.
|
||
|
|
Use a restricted Cloudflare DNS API token to obtain a DNS-01 certificate:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
sudo install -m 0600 /dev/null /etc/letsencrypt/cloudflare.ini
|
||
|
|
sudoedit /etc/letsencrypt/cloudflare.ini
|
||
|
|
# dns_cloudflare_api_token = REPLACE_WITH_RESTRICTED_TOKEN
|
||
|
|
sudo certbot certonly --dns-cloudflare \
|
||
|
|
--dns-cloudflare-credentials /etc/letsencrypt/cloudflare.ini \
|
||
|
|
-d threads-tool-dev.30cm.net
|
||
|
|
sudo /opt/harbor/deploy/remote/enable-tls.sh
|
||
|
|
```
|
||
|
|
|
||
|
|
Ensure Cloudflare Tunnel, router forwarding, or split DNS sends origin traffic
|
||
|
|
to `10.0.0.33`. A public Cloudflare DNS record alone cannot reach a private IP.
|
||
|
|
|
||
|
|
## Monitoring
|
||
|
|
|
||
|
|
All monitoring ports bind to localhost. Open Grafana through an SSH tunnel:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
ssh -i ~/.ssh/harbor_deploy -L 3000:127.0.0.1:3000 daniel@10.0.0.33
|
||
|
|
```
|
||
|
|
|
||
|
|
Then browse `http://127.0.0.1:3000`. The generated Grafana password is in
|
||
|
|
`/etc/harbor/harbor.env` and should be read with sudo.
|
||
|
|
|
||
|
|
Useful commands:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
ssh -i ~/.ssh/harbor_deploy daniel@10.0.0.33 \
|
||
|
|
sudo /opt/harbor/deploy/remote/status.sh
|
||
|
|
ssh -i ~/.ssh/harbor_deploy daniel@10.0.0.33 \
|
||
|
|
journalctl -u 'harbor-gateway@*' -f
|
||
|
|
```
|
||
|
|
|
||
|
|
## Backups
|
||
|
|
|
||
|
|
`harbor-backup.timer` creates daily Mongo and MinIO backups under
|
||
|
|
`/var/backups/harbor` and retains 14 days. This protects against accidental
|
||
|
|
deletion but not loss of the machine or disk. Add an off-host copy before using
|
||
|
|
the service for irreplaceable production data.
|