Update:
The comments from this post will not be removed as to preserve the discussion around the announcement. Any continued discussions outside of this thread that violate server rules will be removed. We feel that everyone that has an opinion, and wanted to vent, has been heard.

————-

Original post:
Yesterday, we received information about the planned federation by Hexbear. The announcement thread can be found here: https://www.hexbear.net/post/280770. After reviewing the thread and the comments, it became evident that allowing Hexbear to federate would violate our rules.

Our code of conduct and server rules can be found here.

The announcement included several concerning statements, as highlighted below:

• “Please try to keep the dirtbag lib-dunking to hexbear itself. Do not follow the Chapo Rules of Posting, instead try to engage utilizing informed rhetoric with sources to dismantle western propaganda. Posting the western atrocity propaganda and pig poop balls is hilarious but will pretty quickly get you banned and if enough of us do it defederated.”
• “The West's role in the world, through organizations such as NATO, the IMF, and the World Bank - among many others - are deeply harmful to the billions of people living both inside and outside of their imperial core.”
• “These organizations constitute the modern imperial order, with the United States at its heart - we are not fooled by the term "rules-based international order." It is in the Left's interest for these organizations to be demolished. When and how this will occur, and what precisely comes after, is the cause of great debate and discussion on this site, but it is necessary for a better world.”

The rhetoric and goal of Hexbar are clear based on their announcement: to "dismantle western propaganda" and "demolish organizations such as NATO” shows that Hexbar has no intention of "respecting the rules of the community instance in which they are posting/commenting.” It’s to push their beliefs and ideology.

In addition, several comments from a Hexbear admin, demonstrate that instance rules will not be respected.

Here are some examples:

“I can assure you there will be no lemmygrad brigades, that energy would be better funneled into the current war against liberalism on the wider fediverse.”

“All loyal, honest, active and upright Communists must unite to oppose the liberal tendencies shown by certain people among us, and set them on the right path. This is one of the tasks on our ideological front.”

• https://lemmy.world/comment/121850
• https://lemmy.world/comment/1487168
• https://lemmy.world/comment/1476084
• https://lemmy.world/comment/171595
• https://www.hexbear.net/comment/3648500

Overall community comments:

• https://www.hexbear.net/comment/3526128
• https://www.hexbear.net/comment/3526086
• https://www.hexbear.net/comment/3652828

To clarify, for those who have inquired about why Hexbear versus Lemmygrad, it should be noted that we are currently exploring the possibility of defederating from Lemmygrad as well based on similar comments Hexbear has made.

• https://lemmygrad.ml/post/158656
• https://lemmygrad.ml/comment/882559
• https://lemmygrad.ml/comment/540170
• https://lemmygrad.ml/comment/446529

Defederation should only be considered as a last resort. However, based on their comments and behavior, no positive outcomes can be expected.

We made the decision to preemptively defederate from Hexbear for these reasons. While we understand that not everyone may agree with our decision, we believe it is important to prioritize the best interests of our community.

Earlier, after review, we blocked and removed several communities that were providing assistance to access copyrighted/pirated material, which is currently not allowed per Rule #1 of our Code of Conduct. The communities that were removed due to this decision were:

• !piracy@lemmy.dbzer0.com
• !piracy@lemmy.ml
• !steamdeckpirates@lemmy.dbzer0.com

We took this action to protect lemmy.world, lemmy.world's users, and lemmy.world staff as the material posted in those communities could be problematic for us, because of potential legal issues around copyrighted material and services that provide access to or assistance in obtaining it.

This decision is about liability and does not mean we are otherwise hostile to any of these communities or their users. As the Lemmyverse grows and instances get big, precautions may happen. We will keep monitoring the situation closely, and if in the future we deem it safe, we would gladly reallow these communities.

The discussions that have happened in various threads on Lemmy make it very clear that removing the communites before we announced our intent to remove them is not the level of transparency the community expects, and that as stewards of this community we need to be extremely transparent before we do this again in the future as well as make sure that we get feedback around what the planned changes are, because lemmy.world is yours as much as it is ours.

Hello World,

Today, after careful consideration and evaluation of recent events, we have decided to defederate from Lemmygrad.

Regrettably, we have observed a significant increase in hate speech and calls to violence originating from the Lemmygrad instance. Due to the severity of the posts and comments, we are not waiting for the next Lemmy update that will allow users to block instances.

At Lemmy.world, we have always strived to foster an inclusive and welcoming user environment. However, recent posts and comments from Lemmygrad have clearly violated our server rules and, more importantly, our core values. We firmly believe that hate speech and incitement of violence have no place in our community, regardless of personal beliefs or affiliations.

As always, we encourage all users to report any content they deem inappropriate or harmful. No matter one's stance in any conflict, Lemmy.world will always take immediate action to remove and ban any posts or comments that incite violence or propagate hatred.

We encourage everyone to continue engaging in discussions within the boundaries of respect and understanding. As we move forward with this decision, we remain committed to providing all community members with a safe and welcoming space. We appreciate your continued support and cooperation in upholding our shared principles.

Thank you,

The Lemmy.World Team

While I was asleep, apparently the site was hacked. Luckily, (big) part of the lemmy.world team is in US, and some early birds in EU also helped mitigate this.

As I am told, this was the issue:

• There is an vulnerability which was exploited
• Several people had their JWT cookies leaked, including at least one admin
• Attackers started changing site settings and posting fake announcements etc

Our mitigations:

• We removed the vulnerability
• Deleted all comments and private messages that contained the exploit
• Rotated JWT secret which invalidated all existing cookies

The vulnerability will be fixed by the Lemmy devs.

Details of the vulnerability are here

Many thanks for all that helped, and sorry for any inconvenience caused!

Update While we believe the admins accounts were what they were after, it could be that other users accounts were compromised. Your cookie could have been 'stolen' and the hacker could have had access to your account, creating posts and comments under your name, and accessing/changing your settings (which shows your e-mail).

For this, you would have had to be using lemmy.world at that time, and load a page that had the vulnerability in it.

Hello everyone,

We unfortunately have to close the !lemmyshitpost community for the time being. We have been fighting the CSAM (Child Sexual Assault Material) posts all day but there is nothing we can do because they will just post from another instance since we changed our registration policy.

We keep working on a solution, we have a few things in the works but that won't help us now.

Thank you for your understanding and apologies to our users, moderators and admins of other instances who had to deal with this.

Edit: @Striker@lemmy.world the moderator of the affected community made a post apologizing for what happened. But this could not be stopped even with 10 moderators. And if it wasn't his community it would have been another one. And it is clear this could happen on any instance.

But we will not give up. We are lucky to have a very dedicated team and we can hopefully make an announcement about what's next very soon.

Edit 2: removed that bit about the moderator tools. That came out a bit harsher than how we meant it. It's been a long day and having to deal with this kind of stuff got some of us a bit salty to say the least. Remember we also had to deal with people posting scat not too long ago so this isn't the first time we felt helpless. Anyway, I hope we can announce something more positive soon.

As requested by some users: 'old' style now accessible via https://old.lemmy.world

Code can be found here: https://github.com/rystaf/mlmym , created by Ryan (Is he here?) (Yes he appears to be! @nnrx@sh.itjust.works ! Thanks for this awesome front-end!)

Today, like the past few days, we have had some downtime. Apparently some script kids are enjoying themselves by targeting our server (and others). Sorry for the inconvenience.

Most of these 'attacks' are targeted at the database, but some are more ddos-like and can be mitigated by using a CDN. Some other Lemmy servers are using Cloudflare, so we know that works. Therefore we have chosen Cloudflare as CDN / DDOS protection platform for now. We will look into other options, but we needed something to be implemented asap.

For the other attacks, we are using them to investigate and implement measures like rate limiting etc.

Another day, another update.

More troubleshooting was done today. What did we do:

• Yesterday evening @phiresky@phiresky@lemmy.world did some SQL troubleshooting with some of the lemmy.world admins. After that, phiresky submitted some PRs to github.
• @cetra3@lemmy.ml created a docker image containing 3PR's: Disable retry queue, Get follower Inbox Fix, Admin Index Fix
• We started using this image, and saw a big drop in CPU usage and disk load.
• We saw thousands of errors per minute in the nginx log for old clients trying to access the websockets (which were removed in 0.18), so we added a return 404 in nginx conf for /api/v3/ws.
• We updated lemmy-ui from RC7 to RC10 which fixed a lot, among which the issue with replying to DMs
• We found that the many 502-errors were caused by an issue in Lemmy/markdown-it.actix or whatever, causing nginx to temporarily mark an upstream to be dead. As a workaround we can either 1.) Only use 1 container or 2.) set ~~proxy_next_upstream timeout;~~ max_fails=5 in nginx.

Currently we're running with 1 lemmy container, so the 502-errors are completely gone so far, and because of the fixes in the Lemmy code everything seems to be running smooth. If needed we could spin up a second lemmy container using the ~~proxy_next_upstream timeout;~~ max_fails=5 workaround but for now it seems to hold with 1.

Thanks to @phiresky@lemmy.world , @cetra3@lemmy.ml , @stanford@discuss.as200950.com, @db0@lemmy.dbzer0.com , @jelloeater85@lemmy.world , @TragicNotCute@lemmy.world for their help!

And not to forget, thanks to @nutomic@lemmy.ml and @dessalines@lemmy.ml for their continuing hard work on Lemmy!

And thank you all for your patience, we'll keep working on it!

Oh, and as bonus, an image (thanks Phiresky!) of the change in bandwidth after implementing the new Lemmy docker image with the PRs.

Edit So as soon as the US folks wake up (hi!) we seem to need the second Lemmy container for performance. So that's now started, and I noticed the proxy_next_upstream timeout setting didn't work (or I didn't set it properly) so I used max_fails=5 for each upstream, that does actually work.

Status update July 4th

Just wanted to let you know where we are with Lemmy.world.

Issues

As you might have noticed, things still won't work as desired.. we see several issues:

Performance

• Loading is mostly OK, but sometimes things take forever
• We (and you) see many 502 errors, resulting in empty pages etc.
• System load: The server is roughly at 60% cpu usage and around 25GB RAM usage. (That is, if we restart Lemmy every 30 minutes. Else memory will go to 100%)

Bugs

• Replying to a DM doesn't seem to work. When hitting reply, you get a box with the original message which you can edit and save (which does nothing)
• 2FA seems to be a problem for many people. It doesn't always work as expected.

Troubleshooting

We have many people helping us, with (site) moderation, sysadmin, troubleshooting, advise etc. There currently are 25 people in our Discord, including admins of other servers. In the Sysadmin channel we are with 8 people. We do troubleshooting sessions with these, and sometimes others. One of the Lemmy devs, @nutomic@lemmy.ml is also helping with current issues.

So, all is not yet running smoothly as we hoped, but with all this help we'll surely get there! Also thank you all for the donations, this helps giving the possibility to use the hardware and tools needed to keep Lemmy.world running!

Hello World!

We've made some changes today, and we'd like to announce that our Code of Conduct is no longer in effect. We now have a new Terms of Service, in effect starting from today(October 19, 2023).

The "LAST REVISION DATE:" on the page also signifies when the page was last edited, and it is updated automatically. Details of specific edits may be viewed by following the "Page History" reference at the bottom of the page. All significant edits will also be announced to our users.

The new Terms of Service can be found at https://legal.lemmy.world/

In this post our community mods and users may express their questions, concerns, requests and issues regarding the Terms of Service, and content moderation in Lemmy.World. We hope to discuss and inform constructively and in good faith.

Hello World!

The last week or so we have seen quite a big 'boost' in the amount of new users signing up so we thought it would be a good time to highlight some things that are of interest to new users.

CODE OF CONDUCT

Lemmy World is not a free speech instance, there are a couple of ground rules that need to be followed. If you're new, I would advise you to read our Code of Conduct.

NEW USER QUESTIONS

If you are new to the fediverse as a whole, it might all be a bit overwhelming. What is Lemmy? What is federation? What even is an instance? For those questions I would suggest you have a look at the getting starting guide. It should cover most of your questions.

STILL HAVE QUESTIONS?

You can head over to the !support@lemmy.world community. This community should be used for questions regarding Lemmy World and is not the support community for the Lemmy software this site uses.

Our Admin @quinten recently made a post covering the most recurring questions there too. Read about that here.

ALTERNATIVE USER INTERFACES

Lemmy World hosts a few custom User Interfaces which give you a completely different experience both on the desktop as on mobile.

• https://a.lemmy.world - Alexandrite is a beautiful and convenient desktop-first alternate web UI for Lemmy. Official site https://alexandrite.app
• https://p.lemmy.world - Photon A sleek client for Lemmy with powerful mod and admin tools. The only alternative client with feature parity to the official client. Official site https://phtn.app
• https://m.lemmy.world - Voyager was one of the first alternative UI's that became available. Optimized for Mobile. Official site https://vger.app
• https://old.lemmy.world - a familiar desktop experience for lemmy. Official site https://mlmym.org

THIRD PARTY APPS

There are a lot of Third Party apps available for Lemmy. From Paid to Open Source, you will find something that suits you easily.

For a complete list of apps have a look at https://lemmyapps.netlify.app/ (Thanks Blaze@discuss.tchncs.de).

EDIT: Updated the apps list. Also some more interesting links in @otter@lemmy.ca's post here: https://lemmy.world/comment/3962001

EDIT 2: Instead of https://photon.lemmy.world you can now just go to https://p.lemmy.world. You can thank @Rootiest@lemmy.world laziness for that.

Hello World!

As we've all known and talked about quite a lot, we previously blocked several piracy-focused communities. These communities, as announced, were:

• !piracy@lemmy.dbzer0.com,
• !steamdeckpirates@lemmy.dbzer0.com,
• !piracy@lemmy.ml, and their local counterparts as follow up actions.

In our removal announcement, we stated that we will continue to look into this more in detail, and re-allow these communities if and when we deem it safe. It was a solid concern at the time, because we were already receiving takedown requests as well as constant attacks, and didn't want to put our volunteer team at risk. We had zero measures in place, and the tools we had were insufficient to deal with anything at scale.

Well, after back and forth with some very cool people, and starting to have proper measures as well as tooling to protect ourselves, we decided it's time to welcome these communities back again. Long live the IT nerds!

We know it's been a rough ride with everything, and we'd like to thank every one of you who were understanding of us, and stayed with us all the way. Please know that as users, you are what makes this platform what it is, and damned we be if we ever forget it.

With love, and as always, stay safe in the high seas!

Lemmy.world Team

❤️

Hello world!

We would like to start by saying thank you ❤, no really 🙏 THANK YOU to ALL the moderators out there!

Without you folks, we would have no one to help keep our community safe and help build the communities both here on Lemmy.World and on other fine instances. To this end, we want to make sure your voices are heard 📣 loud and clear📣.

So, in the spirit of transparency, we would like everyone to know that we are looking to help out the folks working on Sublinks. Over the last several months we have grown to be more than just Lemmy.World. We've added platforms such as Pixelfed and Sharkey to help offer our users more diverse options for expressing themselves online. We still are very committed to Mastodon as well.

We DO NOT plan on moving away from Lemmy as a software platform at this time. Any changes in our core services would need to be discussed extensively internally AND externally with our community members. We firmly believe in the growth of the Fediverse and without the users, there would only be software, and that's no fun!

Sooo...

The Sublinks team has written up a little survey, which we feel is both thorough and inclusive. It covers a wide range of topics, such as user privacy, and community engagement, along with trying to gauge things that are difficult when moderating.

Also please be aware the information collected by this survey is completely anonymous. As many of us in the social sciences background know, if you want the REAL feelings of individuals, they need to feel safe to express themselves.

👉Moderation Survey HERE👈

Please feel free to comment in this thread, we will do our best to respond to any genuine questions.

We look forward to hearing from each and every one of you!

=Sincerely,
Fedihosting Foundation

PS ... also if this sounds like a corporate press release to you folks, we still punk 🤘😜🤘

[Update - March 3rd 2024 , 21:14 CET]

The update seems to have worked well. The database migration took some time, but after that it looks everything works as far as we tested. Let us know what you think! And if you like our work, check the donations links in the sidebar ;-)

(As you might notice, the new comments are shown YELLOW, that's not a bug, it's apparently a new feature in Lemmy...)

Hey everyone,

We wanted to announce that we will be attempting to upgrade Lemmy.World to version 0.19.3 on Sunday. The update will start at 2024-03-03 1900 UTC and should hopefully take no more than 2 hours. In the event there are issues, we will roll back to 0.18.5. In the event of a roll back, any data created by users will be lost, so we'll try our best to avoid this!

Hopefully downtime should be minimal <3

During the upgrade, please feel free to make yourself at home on our Matrix and Discord chat rooms.

Lemmy.World Pub

Matrix

Please follow Lemmy.World (@LemmyWorld@mastodon.world) - Mastodon for updates.

As usual, status can be tracked by visiting https://dash.lemmy.world and https://status.lemmy.world

Thanks for your patience in waiting for us. Stability is king for us!

-LW Tech Group

Lately we have been dealing with a few abusive members from Feddit.nl and we were unable to get in touch with the instance administrator.

Part of the problem is the instance's open registrations which do not require you to enter an e-mail address during signup. This in combination with an inactive admin is a recipe for abuse.

We hope this is only temporary but we have to do this to protect our users.

Edit: we use fediseer, have a look https://gui.fediseer.com/instances/detail/lemmy.world

Edit 2: We got in touch with the Feddit.nl admin. Email requirements were added to the sign-up process and we're setting up a communication channel. So that means we are federating with Feddit.nl again!

For those who find it interesting, enjoy!

The results are in!

We welcome and to the community.

To use the emoji's simply type :duckass: :silver: :trollface: :thonk: :pikachu: :popcorn: :sad: :smash: or :facepalm:

Hello World,

Nothing as exciting as emojis, right? We would like to make it easier for you to express yourselves, so we'll be adding custom emojis to Lemmy.World. The question is, which ones? Let's pick together!

So, we're reaching out to everyone with a Lemmy.World account: Please provide a link to the image you'd like to suggest and upvote the ones you'd like to see! The most upvoted emoji's will, after review, be added to our list of custom emojis.

The emojis can be used by anyone on Lemmy.World in comments and posts.

We just added Alexandrite to the server, it's an alternative desktop UI for Lemmy created by Sheodox who worked tirelessly to make the necessary changes to we could host it ourselves here. So go to https://a.lemmy.world and have a look!

He continues to update it constantly, you can follow the development on his github page or in his community. If you like what you see and want to support him, why not buy him a coffee? :)

For those who don't have Lemmy World as their home instance and want to use Alexandrite, either ask your instance admins to add it or go to https://alexandrite.app!

Edit: I should probably have mentioned that Alexandrite is meant for desktop!

Lemmy.world is temporarily disabling open signups and moving to an application-required signup process, due to ongoing issues with malicious bot accounts.

We know this is a major step to take, but we believe that it’s the right one for both us and our community right now.

We’re working on a better long-term technical solution to these bots, but that will take time to create, test, and verify that it doesn’t cause any problems with federation and how our users use our site, and we’d rather make sure we get it right than have a site that’s broken.

We’re making this change on 28 Aug 2023, and don’t have a specific timeline for how long registrations will require an application, but we will post an update once our new anti-abuse measures are in place and working.

Take care, LW Team

Hello there!

It has been a while since our last update, but it's about time to address the elephant in the room: downtimes. Lemmy.World has been having multiple downtimes a day for quite a while now. And we want to take the time to address some of the concerns and misconceptions that have been spread in chatrooms, memes and various comments in Lemmy communities.

So let's go over some of these misconceptions together.

"Lemmy.World is too big and that is bad for the fediverse".

While one thing is true, we are the biggest Lemmy instance, we are far from the biggest in the Fediverse. If you want actual numbers you can have a look here: https://fedidb.org/network

The entire Lemmy fediverse is still in its infancy and even though we don't like to compare ourselves to Reddit it gives you something comparable. The entire amount of Lemmy users on all instances combined is currently 444,876 which is still nothing compared to a medium sized subreddit. There are some points that can be made that it is better to spread the load of users and communities across other instances, but let us make it clear that this is not a technical problem.

And even in a decentralised system, there will always be bigger and smaller blocks within; such would be the nature of any platform looking to be shaped by its members. 

"Lemmy.World should close down registrations"

Lemmy.World is being linked in a number of Reddit subreddits and in Lemmy apps. Imagine if new users land here and they have no way to sign up. We have to assume that most new users have no information on how the Fediverse works and making them read a full page of what's what would scare a lot of those people off. They probably wouldn't even take the time to read why registrations would be closed, move on and not join the Fediverse at all. What we want to do, however, is inform the users before they sign up, without closing registrations. The option is already built into Lemmy but only available on Lemmy.ml - so a ticket was created with the development team to make these available to other instance Admins. Here is the post on Lemmy Github.

Which brings us to the third point:

"Lemmy.World can not handle the load, that's why the server is down all the time"

This is simply not true. There are no financial issues to upgrade the hardware, should that be required; but that is not the solution to this problem.

The problem is that for a couple of hours every day we are under a DDOS attack. It's a never-ending game of whack-a-mole where we close one attack vector and they'll start using another one. Without going too much into detail and expose too much, there are some very 'expensive' sql queries in Lemmy - actions or features that take up seconds instead of milliseconds to execute. And by by executing them by the thousand a minute you can overload the database server.

So who is attacking us? One thing that is clear is that those responsible of these attacks know the ins and outs of Lemmy. They know which database requests are the most taxing and they are always quick to find another as soon as we close one off. That's one of the only things we know for sure about our attackers. Being the biggest instance and having defederated with a couple of instances has made us a target.  

"Why do they need another sysop who works for free"

Everyone involved with LW works as a volunteer. The money that is donated goes to operational costs only - so hardware and infrastructure. And while we understand that working as a volunteer is not for everyone, nobody is forcing anyone to do anything. As a volunteer you decide how much of your free time you are willing to spend on this project, a service that is also being provided for free.

We will leave this thread pinned locally for a while and we will try to reply to genuine questions or concerns as soon as we can.

Join the Adventure on the official Lemmy.world Minecraft Server!

Calling all Minecraft adventurers and builders! We're thrilled to invite you to embark on an incredible journey on the lemmy.world Minecraft Server. Get ready for an immersive experience like no other, where creativity knows no bounds, and the possibilities are endless.

Minecraft Version: 1.20.x
Address: minecraft.lemmy.world

You can ask questions, request features, share your stories on the discord, matrix and in the official lemmy community

The Server has some rules

Other info:

Server FAQ: https://lemmy.world/post/5467019

Server Info: https://lemmy.world/post/5468646

Update The upgrade was done, DB migrations took around 5 minutes. We'll keep an eye out for (new) issues but for now it seems to be OK.

Original message We will upgrade lemmy.world to 0.18.3 today at 20:00 UTC+2 (Check what this isn in your timezone). Expect the site to be down for a few minutes. ""Edit"" I was warned it could be more than a few minutes. The database update might even take 30 minutes or longer.

Release notes for 0.18.3 can be found here: https://github.com/LemmyNet/lemmy/blob/main/RELEASES.md

(This is unrelated to the downtimes we experienced lately, those are caused by attacks that we're still looking into mitigating. Sorry for those)

We've installed Voyager and it's reachable at https://m.lemmy.world, you can browse Lemmy, and login there (also if your account isn't on lemmy.world)

PS Thanks go out to @stux@stux@geddit.social , he came up with the idea (see https://m.geddit.social).