CompuVerse Update + Matrix Server!
Hey all!
I can confirm that update was performed yesterday to 0.19.0 yesterday / today.
Unfortunately we hit some issues during the upgrade, but these have now been resolved and CompuVerse is now running 0.19.0.
I have also put provisions in place to allow for better longevity in terms of disk usage. Meaning we should now be able to operate off of the current amount of storage for much longer!
As part of this upgrade, I have also now setup a Matrix instance.
We now have a Matrix homeserver running on https://conduit.compuverse.uk (powered by Conduit)
We also have our own hosted instance of Cinny for a ready-to-go Matrix client if you don't already have one you like to use! You can access this on https://chat.compuverse.uk
As per the suggestions from @[email protected] , I plan to use this space to keep everyone up to date in case of any downtime!
The Matrix server is hosted on a completely different server, and thus should be resilient to failing at the same time as the Lemmy instance.
Thanks, Cameron
Hi,
Thanks for your suggestions!
I can confirm that an update was performed yesterday to 0.19.0 yesterday / today. We hit some issues during the upgrade, but these have now been resolved and CompuVerse is running 0.19.0.
I can also confirm that we now have a Matrix server! I will post an announcement shortly, but for reference, the homeserver is now located at https://conduit.compuverse.uk/
We have a client hosted also on https://chat.compuverse.uk/
Feel free to join! :)
Thanks, Cameron
I think the issue here is likely that any activity that occurred in the rest of the Lemmyverse whilst this instance was offline was never federated to us, so this instance doesn't know about the comments or which were posted before it came back online etc.
This will be problematic right now since lots of active threads will have been created or had comments and votes prior to this instance coming back online, but I believe any new content should come through correctly.
I am investigating federation issues.
Do you have any exact examples which I could use as a starting point to investigate this please?
They would be greatly appreciated :)
Thanks,
Cameron
Hi, thanks for your support of Compuverse.
I have just posted an announcement regarding this downtime and aim to improve things for future.
We came back online after resolving the host issues a couple of days ago.
Downtime
Hi,
There's recently been some downtime on Compuverse and I apologise for this.
The host machine was down for a while, and I was unable to bring it back online, no data was lost, it was just shut down and unable to be started.
I'm aiming to create a fail-over solution to allow for an easy transition to a secondary server should the primary fail but currently lack the time and resources required to implement such a thing.
The downtime was excessive and inexcusable so I hope to get things in a much better state as soon as I can.
Thanks & regards,
Cameron
I would like games to arrive as a single complete package, but I was relatively fond of expansion packs from games like RollerCoaster Tycoon.
They took the game you already had, and pretty much doubled it, they were fantastic!
Far better than any modern DLC for sure.
Alexandrite UI
Hi all!
Another new interface is here.
If you're interested, we now host Alexandrite on https://a.compuverse.uk!
Feel free to use this interface if you like :)
WARN: Error encountered while processing the incoming HTTP request Spam (from lemmy.world & lemmy.ml)
Hi, I run the Lemmy instance over at https://compuverse.uk
I seem to be getting spammed with literally thousands of incoming activity requests (primarily coming in from lemmy.world and lemmy.ml). (I measured about 1 new request ever 0.01 -> 0.02 seconds)
The error message in the Lemmy logs is very unhelpful, giving 0 detail about the actual error itself.
The UUIDs in the activities seem to be all unique, rather than duplicates. Does anybody have any idea what's going on and why there is such a large torrent of requests coming in?
This was so bad it essentially caused a DoS attack on my instance, and brought it down. I have had to move the PostgreSQL server onto a more powerful machine in order to keep up with the request rate!
Any help or advice is greatly appreciated :)
Thanks,
Cameron
Hi, many thanks for the suggestions.
There's been a few notable events recently, and I like to keep important news up for a couple of weeks.
I've now cleaned up those that are no longer relevant however. So this should no longer be as problematic.
I think the general Welcome and Rules posts are always going to be relevant to new users, however I would like to be able to get rid of them for older users.
I may have a look into a solution that will allow me to do this. But, not sure how successful I'll be on that front!
That's fantastic news! I guess they realised that the way it was previously handled was HUGELY unsustainable!
Thanks for letting me know about this :)
From what I gather, the way it works is that once a user here has interacted with a remote community at least once, and subscribed to it, from then on, all activity that happens in that community is then automatically pulled across and stored inside this instance also.
So if I subscribed to the "technology" community on "lemmy.world", every post and comment, edit and deletion and I believe even votes and such made afterwards will then be synchronised across to this instance.
Checking the database however, it actually seems that the post and comment data etc isn't too large.
The majority of the storage is actually taken by the "Activity" table.
From what I can see, this table is used to basically store a log of everything the server has been told about. The actual contents are taken out into other tables. (Which are only a couple of hundred MB, rather than 20GB!)
Lemmy does have an automatic cleanup of this table, but it only removes content older than 6 months.
Since we only started in June, we've got an entire 4 months more data to go, and by the way things are going, that's liable to total to potentially hundreds of gigabytes of data.
The activity table had more data in it from the last 2 weeks than it did for the entire month and a half preceeding! (12 million activity records in 2 weeks, compared to only 11 million between the start of June and 2 weeks ago)
PostgreSQL Moved
Hi all, as a result of the recent disk space issue, I have moved the PostgreSQL database onto a secondary server.
This does now mean that database calls are being made "over the wire" so to speak, so performance may be slightly lower than previously. But the server which is running the PSQL database is a rather beefy dedicated server, with much more storage space available.
Hopefully performance isn't impacted too badly. But if you do notice any particularly bad performance, please do let me know and I'll see what I can do.
This new server should last us for MUCH longer (with about 6 times the storage space available).
I did actually set up a Mastodon server previously, but decided to swap over to hosting Lemmy instead.
However, for future downtime, I've created a maintenance page, which I'll switch over to, and provide whatever information I can.
You can see it over at https://holding.compuverse.uk/ :)
Hi Steve, Many thanks for this.
The storage required for lemmy is indeed quite significant!
I have no doubt further optimisations could be done there, but at present it's an unfortunate truth that lemmy likes its storage!
So far it's eaten 21GB of database space in less than 2 months, and again, that's all text!
I'm a software developer and my largest production database I've ever had has only been around 100GB after around 15 years of usage.
Posts and messages from hundreds of thousands of users globally just take a lot of space I guess haha! (Though it would be nice if lemmy only cached remote posts for say, a day or a week, and then wiped the content from its local cache and called out to the remote server thereafter)
I'm very averse to asking for donations! I don't want anyone to have to pay to access CompuVerse, and donations, whilst greatly appreciated would feel wrong to take. Plus there's then further complications regarding how to receive said donations, any taxes involved etc. etc. which I quite frankly can't be bothered to deal with haha!
I really do want to avoid taking donations if at all possible :)
CompuVerse Downtime
Firstly, I really do apologise! CompuVerse has been down for a number of hours now, and I'm really sorry.
I'll be 100% transparent on this, the server ran out of disk space! This sent it into a sort of "safe mode", which unfortunately took down CompuVerse.
I have migrated all image storage over to an external server with a large amount more storage.
However, even with this, just the text content and metadata for CompuVerse is still rather large.
I'm going to investigate solutions but am averse to running a DB server over mounted storage as this is very susceptible to latency and speeds.
By my approximations we have around 2 months' worth of disk space remaining now that Pictrs is removed. I'm looking for solutions in the meantime that will balance performance and cost effectively.
Edit: The PostgreSQL database has been moved. Please read https://compuverse.uk/post/277751 for more info.
New Old CompuVerse interface!
Would you look at that! If you're the kind of person that enjoys the Old Reddit interface, you can now use a very similar interface to access compuverse!
Accessible from https://old.compuverse.uk/
Feel free to use the new interface, I'll try to keep it up to date also, but it's likely not going to be the primary focus.
If you're a user of Reddit Enhancement Suite, you can also get a lot of the same functionality with the following GreaseMonkey script:
https://github.com/rystaf/mlmym/issues/5
I hope you enjoy!
Intel pulls plug on mini-PC NUCs
How much NUC would a Pat G chuck if a Pat G could chuck NUC?
That's a bit of a shame.
I quite liked NUCs!
Hopefully something else comes about with a similar premise!
Lemmy XSS Bug - CompuVerse's Response
TLDR;
CompuVerse was unaffected by this issue, as we did not have any custom emoji.
Your accounts are safe, and no information has been leaked from this instance.
Long-Form
It's recently been brought to my attention that Lemmy had a bug in regarding the handling of custom emoji. More information can be found at this link, however the key takeaways are that the exploit relied on custom emoji. CompuVerse does not have (and has not had) any custom emoji. So thankfully our instance has been unaffected.
Out of an abundance of caution, I have rotated the JWT secret, so if you needed to login again, this is why, and I apologise for the inconvenience!
Custom emoji which were federated over from other servers apparently do not cause the same issue, so we are safe on this front also.
I have also updated the instance to the latest available version of Lemmy to patch the bug properly.
Unfortunately, this has the side effect of breaking our custom CSS theme, so I have reset to the default theme for the time being. I am working on updating the theme to suit as we speak.
So, as a quick recap:
- CompuVerse was unaffected by this issue
- We have rotated JWT secrets as a precaution
- We have updated the instance to a version with this security flaw removed
- The update has broken the custom CSS theme. I am working on updating this as quickly as I can!
Falcon 40B LLM
An intriguing video discussing Falcon 40B, another LLM that seems to perform really quite well, especially given its much smaller size than models like GPT 4.
10,000,000 Users on Meta's Twitter alternative "Threads"
The Meta chief is hoping Threads, his new Instagram-linked app, will beat Twitter on numbers.
Seems Meta is having a modicum of success with their Twitter replacement which they've called "Threads".
10,000,000 users from the first day.
Would be very odd for the main social media networks to be... Facebook, Facebook 2 (Instagram) Facebook 3 (Threads)
All social media ruled by Zuckerberg 🤔
Not sure I like the way the Internet is heading.
Wow, quite the impressive hardware setup there!
I rent a couple of virtual servers (including the one this is running on) from a few providers.
I also host a couple of game servers on a Dedicated machine.
Don't have any physical hosting myself, don't have the resources for such a thing!
I used to self-host a few utilities like a cloud storage solution, link shortener, paste provider, image host etc. but got fed up of maintaining it all ha!
Nowadays I just run a small number of game servers, and CompuVerse!
Highly likely. I suppose it'd add something actually worth having to the subscription because goodness knows none of the other features are worth it ha!
Hmm,
Now this is very interesting.
I'm going to have to take a look into this. It's great to see how well it performs!
I use the OpenAI API for a couple of things at cost atm.
Self-hosting this could be really neat.
Marvel's Spider-Man Remastered & Miles Morales | £28.46 / £22.76 | eTail.Market
eTail.market currently have both of the recent Spider-Man games currently on offer, according to IsThereAnyDeal, these are the lowest prices currently available for the games.
Could be a good time to grab them if you haven't already :)
Twitter Limits Free Users to View 600 Posts / Day
Elon Musk has set a limit on the number of tweets users can read each day. Musk announced the new program Saturday as thousands of users reported problems attempting to access the site. In a tweet …
In a fantastic example of yet further self-implosion by Twitter, Elon Musk has announced a "temporary" limit on the number of posts users can view per day.
Unpaid users can now only view 600 posts per day, before being hit by rate limits. New unpaid users can now only view 300 posts per day.
Paid users are able to view 6000 views per day.
Ah well, at least he's sorted out doom scrolling 👍!
Boost for Lemmy
Discover and join communities on Lemmy and the Fediverse
So, it's official. Reddit has now blocked apps such as Apollo, Boost etc. from using their APIs without copious expense.
Thankfully it seems some of these apps are moving over to Lemmy!
Including my personal favourite third-party Reddit app, Boost!
You can now pre-register for Boost for Lemmy, which should hopefully bring the awesome Boost interface to Lemmy (including this instance).
I highly recommend you check it out and pre-register if interested.
MX Keys S - A fantastically satisfying keyboard, that's quiet enough for the office!
Experience a new level of flow state with the MX Keys S. A high-performance wireless keyboard that is engineered for speed and precision.
So, I've recently been hit with a problem.
My previous keyboard at the office (a cheap Dell membrane keyboard) has given up the ghost, and I was trying to find a nicer replacement.
I have numerous mechanical keyboard (which are generally my preferred type of keyboard), however these are a little too noisy for the office environment!
During my research, I discovered the MX Keys / MX Keys S by Logitech.
I have recently acquired one of these and I highly recommend it.
It has a very satisfying feel when pressing the keys, but is not loud enough to blow out the eardrums of your co-workers.
One of my favourite features of the keyboard is that each key has a small circular indent on the key, this almost, automatically realigns your fingers as you push them, which I've found has increased my typing accuracy quite considerably!
The MX Keys S is a new version of the MX Keys, which only has very minor changes. The main change is switching from the older Logitech Unified Receiver, to a newer Logitech Bolt Receiver, but honestly, the keyboard connects and works perfectly over Bluetooth anyway, so I wouldn't even bother with either unless absolutely necessary.
The other main change is the new inclusion of ambient light sensing, to automatically change the brightness of the backlight. But this is easily managed manually with keys on the keyboard anyway, so again, isn't really necessary.
The MX Keys and Keys S both share the same core design, and same keys, so either is a great choice.
If you want to purchase the keyboard, you can find it on numerous retailers. Including the Logitech site itself (which this post also links to).
However, if you fancy supporting me (and thus CompuVerse), at no extra cost to yourself, please consider using the following Amazon affiliate link, which will automatically provide monetary compensation to myself: MX Keys S on Amazon (Affiliated)
If you don't want to use the affiliated link, no worries! The same link without the affiliation is here: MX Keys S on Amazon (Unaffiliated)
SummoningSalt always makes fantastic videos. Will definitely have to give this one a watch!
This is a good question, I'm not actually sure to be honest with you.
So far I've heard nothing about the main project receiving any sort of threats, and even the Steam page wasn't officially DMCAd or the like. It was more of a threat saying they WOULD officially DMCA if the project wasn't taken down.
It's very interesting that it was played like this rather than just officially issuing a DMCA takedown request. Almost like they weren't 100% convinced the official DMCA would actually hold up?
It is the latest attempt by the parent company of Facebook to make its VR business profitable.
Looks like Meta are trying to introduce a subscription for their VR headsets. Wonder if that'll go anywhere or not...
At present, it will create a post per deal. Which would be way too spammy!
However, the bot is currently offline whilst I'm trying to make some changes. I want to get it into a state where it aggregates multiple deals into a single post. Rather than posting a post per deal that is found!
I don't know if this is something that is possible to do unfortunately. It's not something I've seen available in any of the settings and such I'm afraid :(
Hi, many thanks for the compliments! The bot is having a couple of problems atm which I'm trying to solve, so it's not live at present. The downvote button being enabled / disabled is set by the home instance you are using. For example, the downvote button is enabled on my own compuverse.uk instance, however it appears Beehaw does not enable the downvote option, and thus you don't have the option there.
Spare components will be limited to the US, Canada, and France initially.
A nice win for repairing your own devices. The lack of repairability of Surface devices has always been one of their weakest points.
Good to see MS making at least some steps to rectify this.
The takeover, which has split global regulators, would be the largest in the video games industry.
Seems the US has now decided to block the MS / Activision merger! Following from similar blocks by the UK's Competition & Marketing Authority!
Wonder if this acquisition will ever go through!
Source Code
I do plan on making the source code for this bot public eventually.
It's a lot better than it was when I first started, but still an absolute travesty with a lot of hardcoded nonsense.
Once the code is cleaned up, I will be posting the source code on my GitHub over at https://github.com/CameronRedmore
CompuVerse Deals Bot
Hi all!
In order to give people automated access to loads of game deals, I've created the CompuVerse Deals Bot!
This bot will automatically post new deals as they come in from https://isthereanydeal.com!
To try to keep the quality of the bot's posts at least moderately good, there are a few limits in place.
The bot will only post deals which follow these rules:
- Offer title is at least 30% off
- Offer title has at least 75% positive reviews on Steam
- Offer title is not a DLC
The bot will automatically post the highest cut it has found in the title of the post, along with Steam review scores for the game.
The main body of each post will consist of basic review information on the game, and then a table listing current best known prices in GBP, USD and EUR (other currencies can be added in future if this is something people are interested in!)
I hope you find the bot useful.
Please let me know any suggestions or improvements you may have regarding this bot :)
Thanks,
Cameron
How GitHub's Database Self-Destructed in 43 Seconds
A fascinating (and humorous) watch on a major outage at GitHub, caused by some replication oddities!