Selfhosted @lemmy.world bjornsno @lemm.ee 1mo ago

Help with authentik and traefik random drops

Hello self hosters! I am hoping some of you wizards can help me troubleshoot my setup with authentik and traefik.

First about my setup. I have a synology nas that is running a docker compose stack. Synology is notoriously bad at keeping their docker version fresh, but hopefully that isn't relevant to this issue. I'm running traefik for reverse proxy, and authentik for auth. In authentik land I've split the outpost work into its own container, named authentikproxy. Any request to a service with the authentik-basic@file or authentik@file middleware labels applied should be routed through the authentikproxy service for auth. If it detects that one isn't authed, it will in turn send you to the authentik frontend for SSO.

The issue is that authentik randomly stops working for random routes, or randomly fails to start working for random routes. Every time this happens I need to restart my authentikproxy and traefik containers over and over until it randomly decides to work for all my routes. When this happens I am just sent straight to the app unauthenticated. I'll have to either input http basic credentials or use the app's login page, whichever it has. I have found nothing in the logs after months of this going on, neither authentik nor traefik seem to be aware that anything is amiss.

I suspect the issue is to do with the docker networks but that's honestly just a hunch.

My docker-compose file is hundreds of lines long, so I've stripped environment and volume info while preserving traefik labels to try to keep the info more or less concise. It is certainly still too much info but I did not want to accidentally delete something crucial. Here follows my setup.

docker-compose.yml

services:
  traefik:
    profiles:
      - prod
    container_name: traefik
    image: traefik:v2.11
    command:
      - "--entrypoints.websecure.http.tls.domains[0].main=${BASE_DOMAIN}"
      - "--entrypoints.websecure.http.tls.domains[0].sans=*.${BASE_DOMAIN}"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./traefik/middlewares.yml:/app/myconf/middlewares.yml
      - ./traefik/traefik.yml:/traefik.yml
    restart: unless-stopped
    networks:
      default:
        aliases:
          # Allow xcontainernet services to resolve authentik
          - "authentik.${BASE_DOMAIN-home}"
    ports:
      - 80:80
      - 443:443
    labels:
      - "traefik.enable=true"
      - "traefik.http.middlewares.redirectssl.redirectscheme.scheme=https"
      - "traefik.http.routers.traefik.rule=Host(`traefik.${BASE_DOMAIN-home}`)"
      - "traefik.http.routers.traefik.middlewares=redirectssl@docker"
      - "traefik.http.routers.traefiksecure.rule=Host(`traefik.${BASE_DOMAIN-home}`)"
      - "traefik.http.services.traefik.loadbalancer.server.port=8080"

  transmission:
    image: lscr.io/linuxserver/transmission
    container_name: transmission
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.torrents.rule=Host(`torrents.${BASE_DOMAIN-home}`)"
      - "traefik.http.routers.torrents.middlewares=redirectssl@docker"
      - "traefik.http.routers.torrentssecure.rule=Host(`torrents.${BASE_DOMAIN-home}`)"
      - "traefik.http.routers.torrentssecure.entrypoints=websecure"
      - "traefik.http.routers.torrentssecure.middlewares=authentik@file"

  sabnzbd:
    image: lscr.io/linuxserver/sabnzbd
    container_name: sabnzbd
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.nzb.rule=Host(`nzb.${BASE_DOMAIN-home}`)"
      - "traefik.http.routers.nzb.middlewares=redirectssl@docker"
      - "traefik.http.routers.nzbsecure.rule=Host(`nzb.${BASE_DOMAIN-home}`)"
      - "traefik.http.routers.nzbsecure.entrypoints=websecure"
      - "traefik.http.routers.nzbsecure.middlewares=authentik@file"
      - "traefik.http.services.nzb.loadbalancer.server.port=8080"

  sonarr:
    image: ghcr.io/linuxserver/sonarr:latest
    container_name: sonarr
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.sonarr.rule=Host(`sonarr.${BASE_DOMAIN-home}`)"
      - "traefik.http.routers.sonarr.middlewares=redirectssl@docker"
      - "traefik.http.routers.sonarrsecure.rule=Host(`sonarr.${BASE_DOMAIN-home}`)"
      - "traefik.http.routers.sonarrsecure.entrypoints=websecure"
      - "traefik.http.routers.sonarrsecure.middlewares=authentik-basic@file"
      - "traefik.http.services.sonarr.loadbalancer.server.port=8989"

  radarr:
    image: ghcr.io/linuxserver/radarr:latest
    container_name: radarr
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.radarr.rule=Host(`radarr.${BASE_DOMAIN-home}`)"
      - "traefik.http.routers.radarr.middlewares=redirectssl@docker"
      - "traefik.http.routers.radarrsecure.rule=Host(`radarr.${BASE_DOMAIN-home}`)"
      - "traefik.http.routers.radarrsecure.entrypoints=websecure"
      - "traefik.http.routers.radarrsecure.middlewares=authentik-basic@file"
      - "traefik.http.services.radarr.loadbalancer.server.port=7878"

  readarr:
    image: lscr.io/linuxserver/readarr:nightly
    container_name: readarr
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.readarr.rule=Host(`readarr.${BASE_DOMAIN-home}`)"
      - "traefik.http.routers.readarr.middlewares=redirectssl@docker"
      - "traefik.http.routers.readarrsecure.rule=Host(`readarr.${BASE_DOMAIN-home}`)"
      - "traefik.http.routers.readarrsecure.entrypoints=websecure"
      - "traefik.http.routers.readarrsecure.middlewares=authentik-basic@file"
      - "traefik.http.services.readarr.loadbalancer.server.port=8787"

  bazarr:
    image: ghcr.io/linuxserver/bazarr:latest
    container_name: bazarr
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.bazarr.rule=Host(`bazarr.${BASE_DOMAIN-home}`)"
      - "traefik.http.routers.bazarr.middlewares=redirectssl@docker"
      - "traefik.http.routers.bazarrsecure.rule=Host(`bazarr.${BASE_DOMAIN-home}`)"
      - "traefik.http.routers.bazarrsecure.entrypoints=websecure"
      - "traefik.http.routers.bazarrsecure.middlewares=authentik-basic@file"
      - "traefik.http.services.bazarr.loadbalancer.server.port=6767"

  prowlarr:
    image: lscr.io/linuxserver/prowlarr:latest
    container_name: prowlarr
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.prowlarr.rule=Host(`prowlarr.${BASE_DOMAIN-home}`)"
      - "traefik.http.routers.prowlarr.middlewares=redirectssl@docker"
      - "traefik.http.routers.prowlarrsecure.rule=Host(`prowlarr.${BASE_DOMAIN-home}`)"
      - "traefik.http.routers.prowlarrsecure.entrypoints=websecure"
      - "traefik.http.routers.prowlarrsecure.middlewares=authentik-basic@file"
      - "traefik.http.services.prowlarr.loadbalancer.server.port=9696"

  jellyfin:
    image: linuxserver/jellyfin:latest
    container_name: jellyfin
    networks:
      default:
      xcontainernet:
        ipv4_address: 192.168.0.201
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.jellyfin.rule=Host(`tv.${BASE_DOMAIN-home}`)"
      - "traefik.http.routers.jellyfin.middlewares=redirectssl@docker"
      - "traefik.http.routers.jellyfinsecure.rule=Host(`tv.${BASE_DOMAIN-home}`)"
      - "traefik.http.routers.jellyfinsecure.entrypoints=websecure"
      - "traefik.http.services.jellyfin.loadbalancer.server.port=8096"

  authentikserver:
    image: ghcr.io/goauthentik/server:2024.2.2
    command: server
    depends_on:
      - postgresql
      - redis
    labels:
      - "traefik.enable=true"
      ## HTTP Routers
      - "traefik.http.routers.authentik.rule=Host(`authentik.${BASE_DOMAIN-home}`)"
      - "traefik.http.routers.authentik.entrypoints=web"
      - "traefik.http.routers.authentik.middlewares=redirectssl@docker"
      - "traefik.http.routers.authentiksecure.rule=Host(`authentik.${BASE_DOMAIN:-home}`)"
      - "traefik.http.routers.authentiksecure.entrypoints=websecure"
      ## HTTP Services
      - "traefik.http.routers.authentiksecure.service=authentik-svc"
      - "traefik.http.services.authentik-svc.loadbalancer.server.port=9000"

  authentikproxy:
    image: ghcr.io/goauthentik/proxy:2024.2.2
    labels:
      - "traefik.http.routers.authentik-proxy-outpost.rule=HostRegexp(`{subdomain:[a-z0-9-]+}.${BASE_DOMAIN:-home}`) && PathPrefix(`/outpost.goauthentik.io/`)"
      - "traefik.http.routers.authentik-proxy-outpost.entrypoints=websecure"
      - "traefik.http.services.authentik-proxy-outpost.loadbalancer.server.port=9000"

  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    depends_on:
      - redis
      - immich-database
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.immich.rule=Host(`photos.${BASE_DOMAIN-home}`)"
      - "traefik.http.routers.immich.middlewares=redirectssl@docker"
      - "traefik.http.routers.immichsecure.rule=Host(`photos.${BASE_DOMAIN-home}`)"
      - "traefik.http.routers.immichsecure.entrypoints=websecure"
      - "traefik.http.services.immich.loadbalancer.server.port=3001"

networks:
  default:
    ipam:
      config:
        - subnet: 172.22.0.0/24
  xcontainernet:
    name: xcontainernet
    driver: macvlan
    driver_opts:
      parent: eth0
    ipam:
      config:
        - subnet: "192.168.0.0/24"
          ip_range: "192.168.0.200/29"
          gateway: "192.168.0.1"

traefik/traefik.yml

providers:
  docker:
    exposedByDefault: false
    network: homeservices_default
  file:
    directory: /app/myconf
    watch: true

entryPoints:
  web:
    address: ":80"
  websecure:
    address: ":443"
    http:
      tls:
        certResolver: dnsresolver

traefik/middlewares.yml

http:
  middlewares:
    https-redirect:
      redirectScheme:
        scheme: https
        permanent: true

    authentik-basic:
      forwardAuth:
        address: "http://authentikproxy:9000/outpost.goauthentik.io/auth/traefik"
        trustForwardHeader: true
        authResponseHeaders:
          - Authorization

    authentik:
      forwardAuth:
        address: "http://authentikproxy:9000/outpost.goauthentik.io/auth/traefik"
        trustForwardHeader: true
        authResponseHeaders:
          - X-authentik-email
          - X-authentik-groups
          - X-authentik-jwt
          - X-authentik-meta-app
          - X-authentik-meta-jwks
          - X-authentik-meta-outpost
          - X-authentik-meta-provider
          - X-authentik-meta-version
          - X-authentik-name
          - X-authentik-uid
          - X-authentik-username

4 comments

Sorry, I couldn't manage to read the whole post. Anyway, you may want to try authentik 2024.6 or later. They "reworked proxy provider redirect" in that version. I find it much more stable, but still not perfect. OAuth works great though. Note that 2024.6 requires Postgre db upgrade.
- I'll try that, but since I haven't been able to find any related issues I'm pretty sure it's a configuration error on my part. Hehe the regretfully long post. Next step will probably be to open an issue on authentik's GitHub but since I think it's a pebkac I would prefer not to waste their time.
I've got multiple apps using LDAP, oauth, and proxy on authentik, I've not had this happen.

I also use traefik as reverse proxy.

I didn't manually create an outpost. Not sure what advantage there is unless you have a huge organization and run multiple redundant containers. Regardless there might be some bug here because I otherwise have the same setup as you.

I would definitely try uploading everything to the latest container version first
- I honestly just did it to try to get cleaner logs having the container only be responsible for the proxying.