Skip Navigation
OdiousStooge OdiousStooge @lemmy.ml
Posts 8
Comments 10
Docker container logging
  • Huge thank you! I had a feeling something like this was going on but had no idea how to troubleshoot/fix.

    My pictrs and lemmy containers were the biggest between 3-8 GB (significant for a smaller instance) after a couple weeks.

    For anyone who finds this, in addition to what OP provided here, another command I found helpful (since I am a docker noob πŸ™‚) to find the name of the currently running containers:

    docker ps --format '{{.Name}}'

  • Tips for "Care and Feeding" of my Instance - Disk Usage Rising
  • After some tinkering, **yes ** this indeed was my issue. The logs for pictrs and lemmy in particular were between 3 and 8 gb only after a couple weeks of info level logging.

    Steps to fix (the post above has more detail but adding my full workflow in case that helps folks, some of this wasn't super apparent to me) - these steps assume a docker/ansible install:

    1. SSH to your instance.

    2. Change to your instance install dir

    most likely: cd /srv/lemmy/{domain.name}

    1. List currently running containers

    docker ps --format '{{.Name}}'

    Now for each docker container name:

    1. Find the path/name of the associated log file:

    docker inspect --format='{{.LogPath}}' {one of the container names from above}

    1. Optionally check the file size of the log

    ls -lh {path to log file from the inspect command}

    1. Clear the log

    truncate -s 0 {path to log file from the inspect command}

    After you have cleared any logs you want to clear:

    1. Modify docker-compose.yml adding the following to each container:
    logging:
          driver: "json-file"
          options:
            max-size: "100m"
    
    1. Restart the containers

    docker-compose restart

  • Tips for "Care and Feeding" of my Instance - Disk Usage Rising
  • UPDATE:

    If anyone else is running into consistently rising disk I am pretty sure this is my issue (RE logs running with no cap):

    https://lemmy.eus/post/172518

    Trying out ^ and will update with my findings if it helps.

  • Tips for "Care and Feeding" of my Instance - Disk Usage Rising
  • Gotcha thanks! Thats good to know. Based on the originating ticket: https://github.com/LemmyNet/lemmy/issues/1133

    Sounds like it might be safe for me to purge that table a bit more often as well.

    Dumb question, how are you profiling (RE your mention of getting a better idea of which tables might be bloated) your DB? Just SSHing into your box and direct connecting to DB? Or are there other recommended workflows?

  • Lemmy Support @lemmy.ml OdiousStooge @lemmy.ml

    Tips for "Care and Feeding" of my Instance - Disk Usage Rising

    Any helpful tips for general care and feeding I should be doing on a regular basis?

    I know I need to keep an eye on updates and re-run my ansible setup form time to time to stay up to date.

    But I have also been keeping an eye on my VPS metrics to see when/if I need to beef up the server.

    One thing I am noticing is a steadily increasing disk utilization (which mostly makes sense except its seeming a bit faster than I expected as most all media is links to external sites rather than uploading media directly to my instance).

    Anything I can do to manage that short of just adding more space? Like are there logs/cached content that need to be purged from time to time?

    Thank you!

    5

    Scary Busey

    0

    Classic HC Plans?

    Folks been planning their HC toons/profs for when official drops?

    Been trialing a few on unofficial Bloodsail and having a blast.

    Feeling good about Priest with ench (until GMW) and then tailoring/skinning.

    0

    Anyone else a D4 alt-o-holic?

    I get to like 20/30 and then want to try another class. Loving the mechanics on all of them! Hard to settle.

    10
    Any way to "allow all" on a new instance?
  • πŸ’– man ima dumb dumb. Most excellent, thank you!

  • Lemmy Support @lemmy.ml OdiousStooge @lemmy.ml

    Any way to "allow all" on a new instance?

    I have just been hand-jamming lemmy domains I know about in the allowed instances section of my new instance.

    Is there a way to just allow all? Or maybe a running list somewhere of domains I can copy/paste?

    Seeing a bunch of:

    WARN Error encountered while processing incoming HTTP request: lemmy_server::root_span_builder: Domain is not in allowlist

    Flowing through my logs and from what I can tell, it doesn't seem to tell me WHICH domain I should be trying to add πŸ₯²

    Any thoughts? Thank you!

    4
    General @burggit.moe OdiousStooge @lemmy.ml

    How to make NSFW Communities Discoverable?

    It is my (perhaps incorrect) understanding that NSFW communities are hidden from the global search unless you are registered/logged in at that instance.

    This makes it tough for federated users to find your stuff.

    Any tips on making that easier? For example, I am registered at lemmy.ml but wanna see y'alls

    goodies πŸ‘οΈπŸ‘„πŸ‘οΈ

    I also run an instance that allows NSFW content so curious for helping discoverability of that instance as well. Was thinking of maybe adding a 'site map-esque' concept to a general/welcome community's sidebar with the appropriate search strings.

    ie, something like:

    If you'd like to subscribe to a community put this in your search bar...

    Community 1 is blah

    !{community_name_1}@{instance_url}

    Community 2 is blem

    !{community_name_2}@{instance_url}

    Community 3 is shmlep

    !{community_name_3}@{instance_url}

    Any better way?

    3
    *Permanently Deleted*
  • Thanks!. Yeah I gotta figure that out. Something funky going on.

    There are several communities in there atm but I'm getting some "odd" federation behavior.

    If you are interested in federating there are two ways I have seen it work (again, warning, note that it is NSFW πŸ˜€ ):

    • use your search feature on your main instance and enter [email protected] then you can subscribe

    • or, from your main instance, alter your url to: {your main instance}/c/[email protected] so for example if your main instance is lemmy.ml that would be: https://lemmy.ml/c/[email protected]

    ^ butt_frenzy is one of the communities on booty.world for example.

    Gotta be a better way, and I am probs doing something wrong. Will work on it. Open to feedback if anyone has any advice though haha.

    Thanks.

  • Help Troubleshooting Email Verification on Self Hosted
  • Noice many thanks! I'll give it a shot.

  • Help Troubleshooting Email Verification on Self Hosted
  • Oh snap, yes I am using Digital Ocean.

    Looks like you might have forgotten to add the link, mind chucking that in when you get a chance?

    Thanks, this seems like a good lead as well.

  • Help Troubleshooting Email Verification on Self Hosted
  • Cool thanks! Last thing for now, and then I'll go do some debugging.

    Dumb Q: The lemmy install includes an SMTP server correct? Like I don't need to use an external service for that?

  • Help Troubleshooting Email Verification on Self Hosted
  • Nice! Thanks your mention of the config.hjson makes me wonder lol. I probably goofed that too.

    I did the ansible install which I believe just adds orchestration on top of the Docker install. I'll ssh in and try the docker-compose command.

    Do you know, if I did goof the email config, can I just tweak the config locally and then re-run ansible? Or do I need to do some manual tweaks to the deployed solution? Or I suppose at this point it might be easier to just blow the instance away and start fresh.

  • Lemmy Support @lemmy.ml OdiousStooge @lemmy.ml

    Help Troubleshooting Email Verification on Self Hosted

    Any good resources/Discord servers (to discuss with other devs) for troubleshooting user login/email verification issues?

    I have a self-hosted instance and was able to make the admin user, but when trying to create a new test user, the user does not get the verification email.

    I assume its related to this issue because I see the same issue but on login rather than sign up: https://github.com/LemmyNet/lemmy-ui/issues/1080

    I am able to create the new user but get no email verification and then when I try to login the login button gets stuck with the spinner. No frontend console logs so I assume something is dying server-side and not bubbling the error.

    Really just looking for some guidance/links to docs on how I can access logs to troubleshoot server-side errors (and my gut tells me maybe an email issue but I am a novice in this space so not sure).

    Any help is appreciated.

    Thank you.

    30

    Outrageous prices

    1