Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
# local mTLS client certificates (e.g. Red Hat entitlements)
*.pem

# build and testing artifacts
/bin/
/cache/
Expand Down
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,22 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).

### Added

- Per-repository `cdn` config field to proxy vendor CDNs that publish no public mirrors
- Per-repository `mtls` config field with client `cert`, `key` and optional `ca` for CDNs using mutual TLS
- Support for proxying entitled Red Hat content from `cdn.redhat.com` without client-side certificates
- Client config snippet for Red Hat Enterprise Linux on the landing page and in the README
- Top-level `branding` config field to customize the landing page title and description
- Landing page now shows the running pkgproxy version
- Container image now runs `serve` by default and loads bundled config from `$KO_DATA_PATH`
- `PKGPROXY_TRUST_PROXY` env var (and `--trust-proxy` flag) to opt in to X-Forwarded-For trust
- `PKGPROXY_HOST` env var to set the listen address without passing `--host` on the command line

### Changed

- Repositories must now define exactly one of `mirrors` or `cdn`; setting both is rejected
- Upstream URLs are validated at startup: they must be absolute and use `http` or `https`
- **Breaking:** `remote_ip` in access logs now reflects the direct connecting peer by default; set `PKGPROXY_TRUST_PROXY` to restore XFF-based IP extraction when running behind a reverse proxy
- **Breaking:** Removed the `--public-host` flag and `PKGPROXY_PUBLIC_HOST` env var; the landing page now fills in config snippet hostnames automatically — server-side from the request's `Host` header (works for `curl` too), further corrected client-side to the browser's own URL when that differs (e.g. behind a TLS-terminating reverse proxy)
- Upgraded Echo web framework to v5.1.1
- Config-file errors now list all default paths attempted, not just the last one

Expand Down
5 changes: 3 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,8 @@ $(if $(filter debian,$(1)),TestDebian,\
$(if $(filter ubuntu,$(1)),TestUbuntu,\
$(if $(filter archlinux,$(1)),TestArch,\
$(if $(filter gentoo,$(1)),TestGentoo,\
$(error Unknown DISTRO: $(1). Use one of: fedora centos-stream almalinux rockylinux debian ubuntu archlinux gentoo))))))))))
$(if $(filter rhel,$(1)),TestRHEL,\
$(error Unknown DISTRO: $(1). Use one of: fedora centos-stream almalinux rockylinux debian ubuntu archlinux gentoo rhel)))))))))))
endef

.PHONY: e2e
Expand Down Expand Up @@ -157,7 +158,7 @@ run: format vet generate ## Run the application from your host
$(info *************************************************)
$(info ********** EXECUTING 'run' MAKE TARGET **********)
$(info *************************************************)
PKGPROXY_CONFIG=./configs/pkgproxy.yaml PKGPROXY_PUBLIC_HOST=$(shell hostname):8080 CGO_ENABLED=$(CGO_ENABLED) go run . serve --host 0.0.0.0 --debug
PKGPROXY_CONFIG=./configs/pkgproxy.yaml CGO_ENABLED=$(CGO_ENABLED) go run . serve --host 0.0.0.0 --debug

PLATFORMS := $(shell echo $(ARCHS) | sed 's/,/ /g' | sed 's/[^ ]\+/linux\/&/g' | tr ' ' ',')

Expand Down
127 changes: 124 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,12 +33,32 @@ podman run --rm -p 8080:8080 -e PKGPROXY_HOST=0.0.0.0 --volume ./cache:/ko-app/c
| `--cachedir` | | `cache` | Path to the local cache directory |
| `--host` | `PKGPROXY_HOST` | `localhost` | Listen address |
| `--port` | | `8080` | Listen port |
| `--public-host` | `PKGPROXY_PUBLIC_HOST` | | Public hostname (or `host:port`) shown in landing page config snippets. When set, the listen port is not appended. Useful when running behind a reverse proxy. |
| `--trust-proxy` | `PKGPROXY_TRUST_PROXY` | | Comma-separated list of trusted proxy sources for X-Forwarded-For. Accepted values: `none`, `loopback`, `private`, a CIDR (e.g. `10.0.0.0/8`), or a bare IP (promoted to `/32`/`/128`). Unset or empty means no XFF trust. |
| `--debug` | | `false` | Enable debug logging |

Any flag with an env variable listed above can be set via the environment instead of passing the flag.

### Landing page hostname

The config snippets shown on the landing page (`GET /`) need pkgproxy's own
address, e.g. `baseurl=http://<pkgproxy>/fedora/...`. Rather than relying on a
server-side setting, this is filled in automatically, with no configuration
needed:

- **Server-side, from the request's `Host` header.** Every response — including
`curl` and other non-browser clients — already contains a working address
built from the `Host` header the request itself carried (the same header a
reverse proxy forwards by default). No JavaScript required.
- **Client-side, from the page's own URL.** In a browser, a small inline script
additionally corrects the address to `window.location.origin` if it differs
from the server-rendered one — which matters behind a reverse proxy that
changes the scheme (e.g. TLS termination), since the `Host` header alone
can't reveal that.

If a reverse proxy in front of pkgproxy does not forward the original `Host`
header, `curl` (or a browser with JavaScript disabled) will see whatever host
pkgproxy itself observed instead.

### Trusting X-Forwarded-For

By default pkgproxy ignores the `X-Forwarded-For` header and uses the direct connecting IP address for the `remote_ip` access-log field. This is the safe behavior when pkgproxy faces the internet directly or runs in a container without a reverse proxy in front of it.
Expand Down Expand Up @@ -66,8 +86,94 @@ Each repository supports the following options:
|-----|----------|-------------|
| `suffixes` | yes | File suffixes that are eligible for caching (e.g. `.rpm`, `.deb`). Use `"*"` to cache all files. |
| `exclude` | no | List of file names to exclude from caching, even when they match a suffix. Useful with the `"*"` wildcard suffix. |
| `mirrors` | yes | Ordered list of upstream mirror URLs |
| `retries` | no | Number of attempts per mirror before moving to the next one (default: `1`) |
| `mirrors` | yes* | Ordered list of upstream mirror URLs |
| `cdn` | yes* | Single upstream CDN URL, used instead of `mirrors` |
| `mtls` | no | Client certificate (`cert`), private key (`key`) and optional CA bundle (`ca`) used with a `cdn` requiring mutual TLS |
| `retries` | no | Number of attempts per upstream before moving to the next one (default: `1`) |

\* Each repository must define exactly one of `mirrors` or `cdn`; setting both is rejected.

### Landing page branding

The top-level `branding` key customizes the title and description shown on the
landing page (and the HTML `<title>`) served at `/`:

```yaml
branding:
title: Acme Package Mirror
description: Internal package cache for Acme Corp.

repositories:
...
```

Both fields are optional and independent — omitting `branding` entirely, or
leaving one of the two fields unset, falls back to the default "pkgproxy" title
and "Caching forward proxy for Linux package repositories." description. The
landing page also always shows the running pkgproxy version below the
description.

### CDN upstreams

Some vendors do not publish public mirrors and serve their packages from a single
CDN instead. Use `cdn` in place of `mirrors` for those repositories:

```yaml
repositories:
rhel:
suffixes:
- .rpm
cdn: https://cdn.redhat.com/
```

Requests are mapped the same way as for mirrors: the repository name is stripped
from the request path and the remainder is appended to the CDN URL, so
`/rhel/content/dist/rhel9/9/x86_64/baseos/os/` is fetched from
`https://cdn.redhat.com/content/dist/rhel9/9/x86_64/baseos/os/`.

### CDN client certificates (mTLS)

When the CDN requires mutual TLS — as the Red Hat CDN does for entitled content —
add an `mtls` block with the client certificate and its private key. pkgproxy
presents them during the TLS handshake with the CDN:

```yaml
repositories:
rhel:
suffixes:
- .drpm
- .rpm
cdn: https://cdn.redhat.com/
mtls:
cert: /etc/pki/entitlement/1234567890123456789.pem
key: /etc/pki/entitlement/1234567890123456789-key.pem
ca: /etc/rhsm/ca/redhat-uep.pem
```

On a subscribed Red Hat host the entitlement certificate and its key are the
`.pem` file pair in `/etc/pki/entitlement/`.

The optional `ca` points at a CA bundle used to verify the CDN's *own* server
certificate, and is added to the system trust store rather than replacing it.
It is required for `cdn.redhat.com`, whose certificate is issued by a private
Red Hat CA that public trust stores do not contain — without it every request
fails with `x509: certificate signed by unknown authority`.

Notes:

- `mtls` is only valid together with `cdn`, and both `cert` and `key` are required.
- Relative paths are resolved against the working directory of the pkgproxy
process. Prefer absolute paths, especially for container deployments.
- pkgproxy **refuses to start** if the certificate, key or CA bundle cannot be
loaded. Proxying without them would only produce opaque TLS or authorization
errors from the CDN.
- The certificate is scoped to the configured CDN host. If the CDN redirects to a
different host, the redirect is followed **without** the client certificate so
the credential is never sent elsewhere.
- Clients talking to pkgproxy need no entitlement certificate of their own — this
is the point of proxying an entitled CDN for a local network. Protect access to
pkgproxy accordingly, since it will serve entitled content to anyone who can
reach it.

### Mirror retries

Expand Down Expand Up @@ -214,6 +320,21 @@ For Enterprise distributions the URL suffix `epel-$releasever-$basearch` must be
baseurl=http://<pkgproxy>:8080/rockylinux/$releasever/BaseOS/$basearch/os/
```

### Red Hat Enterprise Linux

Requires a `rhel` repository configured with `cdn` and `mtls` (see [CDN client
certificates](#cdn-client-certificates-mtls)). Disable the subscription-manager
managed repositories, then e.g. `/etc/yum.repos.d/rhel.repo` (adjust other
repositories accordingly):
```
[rhel-baseos-rpms]
# baseurl=https://cdn.redhat.com/content/dist/rhel$releasever/$releasever/$basearch/baseos/os
baseurl=http://<pkgproxy>:8080/rhel/content/dist/rhel$releasever/$releasever/$basearch/baseos/os
```

The client needs no `sslclientcert`/`sslclientkey` of its own — pkgproxy holds the
entitlement certificate and authenticates against the CDN on the client's behalf.

### Ubuntu

E.g. Ubuntu 24.04 Noble Numbat: `/etc/apt/sources.list` (substitute your release codename):
Expand Down
32 changes: 16 additions & 16 deletions cmd/config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,20 +24,20 @@ func writeConfig(t *testing.T, dir, name string) string {

func TestResolveConfigPath(t *testing.T) {
tests := []struct {
name string
localExists bool
localIsDir bool
koDataSet bool
koFileExists bool
wantPath func(koDir string) string
wantCandidates func(koDir string) []string
name string
localExists bool
localIsDir bool
koDataSet bool
koFileExists bool
wantPath func(koDir string) string
wantCandidates func(koDir string) []string
}{
{
name: "local file wins over ko fallback",
localExists: true,
koDataSet: true,
koFileExists: true,
wantPath: func(_ string) string { return defaultConfigPath },
name: "local file wins over ko fallback",
localExists: true,
koDataSet: true,
koFileExists: true,
wantPath: func(_ string) string { return defaultConfigPath },
wantCandidates: func(_ string) []string { return []string{defaultConfigPath} },
},
{
Expand All @@ -51,10 +51,10 @@ func TestResolveConfigPath(t *testing.T) {
},
},
{
name: "both missing returns default path",
localExists: false,
koDataSet: false,
wantPath: func(_ string) string { return defaultConfigPath },
name: "both missing returns default path",
localExists: false,
koDataSet: false,
wantPath: func(_ string) string { return defaultConfigPath },
wantCandidates: func(_ string) []string { return []string{defaultConfigPath} },
},
{
Expand Down
24 changes: 5 additions & 19 deletions cmd/serve.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,6 @@ import (
var (
listenAddress string
listenPort uint16
publicHost string
trustProxy string
ipExtractor echo.IPExtractor
resolvedTrustProxy string
Expand All @@ -35,7 +34,6 @@ const (
defaultAddress = "localhost"
defaultPort = 8080
hostEnvVar = "PKGPROXY_HOST"
publicHostEnvVar = "PKGPROXY_PUBLIC_HOST"
trustProxyEnvVar = "PKGPROXY_TRUST_PROXY"
)

Expand All @@ -59,7 +57,6 @@ func newServeCommand() *cobra.Command {
}
c.PersistentFlags().StringVar(&listenAddress, "host", defaultAddress, "listen address of the pkgproxy.")
c.PersistentFlags().Uint16Var(&listenPort, "port", defaultPort, "listen port of the pkgproxy.")
c.PersistentFlags().StringVar(&publicHost, "public-host", "", "public hostname (or host:port) shown in landing page config snippets; overrides PKGPROXY_PUBLIC_HOST.")
c.PersistentFlags().StringVar(&trustProxy, "trust-proxy", "", "comma-separated list of trusted proxy addresses for X-Forwarded-For: none, loopback, private, CIDR, or IP; overrides PKGPROXY_TRUST_PROXY.")

return c
Expand All @@ -76,19 +73,6 @@ func resolveListenHost(flagChanged bool, flagValue, envValue string) string {
return defaultAddress
}

// resolvePublicAddr determines the address rendered in landing page config snippets.
// The CLI flag takes precedence over the environment variable. If neither is set,
// the listen host:port is used.
func resolvePublicAddr(flagValue string, listenAddr string, port uint16) string {
if flagValue != "" {
return flagValue
}
if v := os.Getenv(publicHostEnvVar); v != "" {
return v
}
return fmt.Sprintf("%s:%d", listenAddr, port)
}

// resolveTrustProxy determines the trust-proxy value using flag → env var → default precedence.
func resolveTrustProxy(flagChanged bool, flagValue, envValue string) string {
if flagChanged {
Expand Down Expand Up @@ -221,12 +205,14 @@ func startServer(_ *cobra.Command, _ []string) error {
})
app.Use(middleware.Recover())

pkgProxy := pkgproxy.New(&pkgproxy.PkgProxyConfig{
pkgProxy, err := pkgproxy.New(&pkgproxy.PkgProxyConfig{
CacheBasePath: cacheDir,
RepositoryConfig: &repoConfig,
})
publicAddr := resolvePublicAddr(publicHost, listenAddress, listenPort)
app.GET("/", pkgproxy.LandingHandler(&repoConfig, publicAddr))
if err != nil {
return fmt.Errorf("unable to initialize proxy: %w", err)
}
app.GET("/", pkgproxy.LandingHandler(&repoConfig, Version))
app.Use(pkgProxy.Cache)
app.Use(pkgProxy.ForwardProxy)

Expand Down
51 changes: 0 additions & 51 deletions cmd/serve_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -275,54 +275,3 @@ func TestParseTrustProxy(t *testing.T) {
})
}
}

func TestResolvePublicAddr(t *testing.T) {
tests := []struct {
name string
flagValue string
envValue string
listenAddr string
listenPort uint16
want string
}{
{
name: "flag takes precedence over env var",
flagValue: "myproxy.lan",
envValue: "other.host",
listenAddr: "localhost",
listenPort: 8080,
want: "myproxy.lan",
},
{
name: "env var used when flag is empty",
flagValue: "",
envValue: "myproxy.lan",
listenAddr: "localhost",
listenPort: 8080,
want: "myproxy.lan",
},
{
name: "defaults to listen host:port when neither is set",
flagValue: "",
envValue: "",
listenAddr: "localhost",
listenPort: 8080,
want: "localhost:8080",
},
{
name: "flag with embedded port used verbatim",
flagValue: "myproxy.lan:9090",
envValue: "",
listenAddr: "localhost",
listenPort: 8080,
want: "myproxy.lan:9090",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Setenv(publicHostEnvVar, tt.envValue)
got := resolvePublicAddr(tt.flagValue, tt.listenAddr, tt.listenPort)
assert.Equal(t, tt.want, got)
})
}
}
13 changes: 13 additions & 0 deletions configs/pkgproxy.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
---
branding:
title: Pkgproxy Application
description: Caching forward proxy for Linux package repositories

repositories:
almalinux:
suffixes:
Expand Down Expand Up @@ -87,6 +91,15 @@ repositories:
- https://mirror.init7.net/rockylinux/
- https://mirror.puzzle.ch/rockylinux/
- https://dl.rockylinux.org/pub/rocky/
rhel:
suffixes:
- .drpm
- .rpm
cdn: https://cdn.redhat.com/
mtls:
cert: entitlement.pem
key: entitlement-key.pem
ca: /etc/rhsm/ca/redhat-uep.pem
ubuntu:
suffixes:
- .deb
Expand Down
Loading