I’m not sure that’s possible with images being allowed. If Google, Facebook, Instagram, and YouTube all struggle with it I think it will be an issue anywhere images are allowed. Maybe there’s an opening for an AI to handle the task these days but any dataset for something like that could obviously be incredibly problematic

I'd be fine with not hosting images entirely. I don't think people come to beehaw primarily to look at pictures

People keep talking about going to another platform. Personally I think a better idea would be to develop lemmy to deal with these issues. This must be a fediverse wide problem. So some discussion with other admins and the developers is probably the way to go on many of these things. Moreover you work with https://opencollective.com/, can they help. Beyond this, especially CSAM, there must be large funding agencies where one could get a grant to get some real professional programming put into this problem. Perhaps we could raise funds ourselves to help with this too.

So frankly I would like to see Beehaw solve the issues with lemmy, rather then just move to some other platform that will have its own issues. The exception may be if the Beehaw people think that being a safe space creates too big a target that you have to leave the Threadiverse to be safe. That to me seems like letting the haters win. It is exactly what they want. My vote will always be to solve the threadiverse issues rather then run away.

Just my feeling. There may be more short term practical issues that take precedence and frankly it is all up to you guys where you want to take this project.

I just want to say, I am so so so sorry you had to see that.

I accidentally saw some CSAM in the 1990s and you are right, it is burnt into your mind. It's the real limit case of "what has been seen cannot be unseen" - all I could do was learn to avoid accessing those memories.

If you can access counselling for this, that might be a good option. Vicarious trauma is a real phenomenon.

I'm sure those repugnant assholes do it "for the lulz" and if they want to mess with you they'll do it anywhere.

There's this study that says playing Tetris helps ease recently acquired trauma https://www.ox.ac.uk/news/2017-03-28-tetris-used-prevent-post-traumatic-stress-symptoms

And the admin from his eponymous instance dbzero created an interesting script to get rid of CSAM without having to review it manually, take a look -> https://github.com/db0/lemmy-safety

This post seems highly reactive to me. I'm sorry to hear of you being exposed to such disturbing material, but I fail to see at true connection of that happening and using Lemmy as the platform. I absolutely agree that nobody should have to experience what you did, but I disagree with the platform change proposition.

Sadly, the only 100% way to never have that kind of material ever touch your servers is to not allow image uploads from the public. Whether it's on Lemmy or another social site, or something you control entirely on your own. Maybe sooner than we think, AI could deal with the moderation of it so a human never has to witness that filth, but it's not quite there yet.

Lemm.ee, another instance I am in, isn't hosting images anymore or letting people upload images directly due to this issue. When your platform is supposed to be 100% open source and decentralized, there are bound to be issues like this, and they should be dealt with, even if proprietary tech is necessary for it. I'm sorry to hear about this.

A software platform that makes it nearly impossible for Beehaw to host, in any way, CSAM.

I hate to say it, but you'll need to find a text-only platform. Allowing any image uploads opens the door to things like this.

Besides that, if your concern is that no moderator should be exposed to anything like that, well on a text-only site you might have to deal with disguised spam links to gore, scam, etc. You'll still have to click on links to effectively moderate.

Maybe you should consider if this is a position that you want to put yourself in again. It sounds like this may just not be for you.

As long as you can post links or upload images, there is an avenue for CSAM to be spammed. Beehaw should probably start with a whitelist and slowly expand. Refuse to federate with anyone that has open registration.

As others have suggested, I think temporarily suspending images until you guys can settle on a safe alternative to lemmy is a good idea.

Im sorry you had to see something like this, i hope you are able to seek out some counceling asap, talk to someone about it. Even something like https://www.7cups.com/ might be helpful.

I don't think there is a way to have both the option to host images and have zero risk of getting such image uploads. You either completely disable image hosting, or you mitigate the risk by the way image uploads are handled. Even if you completely disable the image uploads, someone might still link to such content. The way I see this there are two different aspects. One is the legal danger you place yourself when you open your instance to host images uploaded by users. The other is the obvious (and not so obvious) and undeniable harmful effects contact with such material has for most of us. The second, is pretty impossible to guarantee 100% on the internet. The first you can achieve by simply not allowing image uploads (and I guess de-federating with other instances to avoid content replication).

The thing is, when you host an instance of a technology that allows for better moderation (i.e. allowing certain kinds of content, such as images, only after a user reaches a certain threshold of activity), actually helps in a less obvious manner. CSAM is not only illegal to exist on the server-side. It's also illegal and has serious consequences for the people who actually upload it. The more activity history you have on a potential uploader, the easier it becomes to actually track him. Requiring more time for an account before allowing it to post images, makes concealing the identity harder and raises the potential risk for the uploader to the extend that it will be very difficult to go through the process only to cause problems to the community.

Let me also state this clearly: I don't have an issue with disabling image uploads here, or changing the default setting of instance federation to a more limiting one. Or both. I don't mind linked images to external sites.

I am sorry you had to see such content. No, it doesn't seem to go away. At least it hasn't for me, after almost 2 decades :-/

So, so sorry you had to see that, and thank you for protecting the rest of us from seeing it.

On traditional forums, you'd have a lot of control over the posting of images.

If you don't wish to block images entirely, you could block new members from uploading images, or even from sharing links. You could set things up so they'd have to earn the right to post by being active for a randomised amount of time, and have made a randomised number of posts/comments. You could add manual review to that, so that once a member has ostensibly been around long enough and participated enough, admin look at their activity pattern as well as their words to assess if they should be taken off probation or not... Members who have been inactive for a while could have image posting abilities revoked and be put through a similar probation if they return. You could totally block all members from sharing images & links via DM, and admin email accounts could be set to reject images.

It is probably possible to obtain the means to reject images which could contain any sexual content (checked against a database of sexual material which does not involve minors), and you could probably also reject images which could contain children and which might not be wholesome (checked against a database of normal images of children).

Aside from the topic in hand, a forum might decide to block all images of children, because children aren't really in a position to consent to their images being shared online. That gets tricky when it comes to late teens & early 20s, but if you've successfully filtered out infants, young children, pre-teens & early teens as well as all sexual content, it is very unlikely that images of teenagers being abused would get through.

Insisting that images are not uploaded directly, but via links to image hosting sites, might give admin an extra layer of protection, as the hosting sites have their own anti-CSAM mechanisms. You'd probably want to whitelist permitted sites. You might also want a slight delay between the posting of an image link and the image appearing on Beehaw - this would allow time for the image hosting site to find & remove any problem images before they could appear on Beehaw (though I'd imagine these things are pretty damn fast by now).

You could also insist that members who wish to post images or links to images can only do so if they have their VPN and other privacy preserving methods disabled. Most members wouldn't be super-enthused about this, until they've developed trust in the admin of the site, but anyone hoping to share images of children being abused or other illegal content will just go elsewhere.

Admin would probably need to be able to receive images of screenshots from members trying to report technical issues, but those should be relatively easy to whitelist with a bot of some sort? Or maybe there's some nifty plugin for this?

Really though, blocking all images is going to be your best bet. I like the idea of just having the Beehaw bee drawings. You could possibly let us have access to a selection of avatars to pick, or have a little draw plugin so members can draw their own. On that note, those collaborative drawing plugin things can be a fun addition to a site... If someone is very keen for others to see a particular image, they can explain how to find it, or they can organise to connect with each other off Beehaw.

Those images are burnt into my mind and I would love to get rid of them but I don’t know how or if it is possible

I'm very sorry this happened to you, and I wish I could offer you some advice... but that's the main reason I stopped hosting open community stuff many years ago. I thought I was hardened enough, but nope; between the spam, the "shock imagery" (NSFL gore, CSAM), the doxxing, and toxic users in general.... even having some ads was far from making it all worthwhile. There is a reason why "the big ones" like Facebook or Google churn through 3rd world mods who can't take it for more than a few months before getting burnt out.

I wish I could tell you that you'll eventually forget what you've seen... but I still remember stuff from 30 years ago. Also don't want to scare you, but it's not limited to images... some "fanfiction" with text imagery is evil shit that I still can't forget either.

Nowadays, you can find automated CSAM identification services, like the one run by Microsoft, so if you integrated that, you could err on the side of caution and block any image it marks as even suspicious. This may or may not work in your jurisdiction, with some requiring you to "preserve the proof" and submit it to authorities (plus different jurisdictions having different definitions of what is an what isn't breaking the law, and laws against swamping them with false positives... so you basically can't win). This will also do nothing for the NSFL or text based imagery.

A way to "shield yourself" from all of this as an admin, is to go to an encrypted platform where you can't even see what's getting posted, so you never run the risk of seeing that kind of content... but then you end up with zero moderation tools, pushing all the burden onto your users, so not suitable for a safe space.

Honestly, I don't think there is an effective solution for this yet. It's been a great time ~~abusing the good will of the admins and mods~~ staying on Beehaw, but if you can't find a reasonable compromise... oh well.

How would moving platform help with this? Can't someone just post them to whatever platform beehaw moves (and dies) on?

Regardless, nobody should ever have to go through that satanic evil that is Child Sexual Abuse or even view the imagery of it.

Does that mean a platform that does not allow any images to be uploaded? Or a platform that has better access control and remediation controls?

Beehaw is such a special effort. I am so regretful that people have to be subjected to the darkest parts of humanity in order to protect the beehaw project. I don't need images. If that is the necessary course of action, then so be it.

Most importantly, I am so sorry to you as one human to another. I'm sorry you saw that. I'm sorry humans are hurting each other like that. And I'm sorry that your good faith efforts have been taken advantage of.

I mentioned this in discord a while back, but there are image-matching databases for known instances of CSAM that you can apply for access to, as an admin of a forum or social media site. If you had access, you could scan each image uploaded or linked to in a post or comment, and compare to the database for matches. I think that mastodon is adding some hooks for this kind of checking during the upload phase, but I’m not sure what the status is with Lemmy.

I’m happy to help facilitate a solution like this, as it’s something I also care about. Feel free to find me on discord if you want to talk.

Also, as others have said - I’m sorry you had to go through that. The same thing happened to me many years ago and it definitely affected me for a long time.

First of all, I'm so sorry that you have been exposed to such horrors. I hope you can handle that, or find help to.

I don't have a solution, I'd just like to share some thoughts.

1. Some people suggested that AIs could detect this kind of content. I would be reluctant to use such tools, because lots of AI projects exploit unprotected workers in poor countries for data labeling.

2. An zero-image policy could be an effective solution, but it would badly impact @dyi@beehaw.org, @creative@beewah.org and @greenspace@beehaw.org.

3. correct me if I'm wrong, but on the fediverse, when a picture is posted on an instance, it is duplicated on all federated instances? If I'm right, it means that even if beehaw found a way to totally avoid CSAM posting, you could still end up with duplicated CSAM on your server? (with consequences on your mental health, and possibly legal risks for owning such pictures)

Hey, Im not a user from this instance, but I feel worried about you OP.

Perhaps take a break, even if you could not erase your memory's as some people have said please take care of yourself. If it is not restrictive please go to therapy or in case that it is too expensive there is always a free service in a lot of countries.

I don't know what kind of platform would make it impossible to not host csam. Text only, without linking?

Or is the problem here that the post was available to the wider audience, so other than moderation front was able to see it? Lemmy doesn't do restricted posting? Jesus this platform is at its infancy then.

A few observations/thoughts.

• There's an awful lot of posts basically saying "this is a part of the job of moderation" and I don't think that's a particularly empathetic or useful observation. I've been on the internet and moderating for long enough to have been exposed to a lot of this, but this is not an inevitability. It's an outcome of the system we've designed, of regulation and law that we have, and of not prioritizing this as a problem strongly enough. Being dismissive of an emotional experience and trauma isn't particularly helpful.
• I'm not technical enough to explain this, but there are technical and legal issues with CSAM and the lemmy platform that we've ran into. For one, there's no automated scanning tools for this kind of content. My understanding is that even implementing or creating said tools would be difficult because of the way pict-rs and rust are storing images in the first place. You cannot turn off image federation, at all. At best, you can clear the content, but doing so may violate CSAM laws depending on the country and reporting requirements. Someone on the technical side can explain better than I can.
• This isn't a thread to discuss who's to blame for CSAM. Please cease all discussions fighting about religion in the comments. I will be removing these comments.

Sorry to hear that mate! That's one of the biggest reasons I've never wanted to move towards IT forensics even though I think I'd enjoy the actual work. But having to regularly sift through the absolute worst humanity has to offer sounds awful.

Hope the immediate pain of it settles as soon as possible!

This might not be what people want but since beehaw is going to leave Lemmy anyway, couldn't you just completely defederate and run as an isolated instance? Then you'd have control of what her life gets published without having to deal with federated nastiness?

Others have said it better but sorry that you had to experience that

If lemmy doesn't support expanding remote hosted images from platforms like imgur or whatever that's probably a good feature to look for in a new platform. If it does, things are obviously getting to a point where you need to reduce what you need to worry about, to help focus on moderation and keeping things running. Block listing urls is easier than running and hoping on an ai.

Otherwise I think disabling images or giving the ability only to financial supporters or strongly verified accounts via upvote counts, number of reputable or discussed posts, or and age of account are the only sane moves.

Personally I don't mind clicking through for an image if it interests me.

i am so sorry you had to see that kind of thing.

I can’t imagine how terrible you must feel. You should get some external help… I hear that hypnosis is used for traumatic experiences.

I need to think about what you said. I really want to see the community blossom. We should maybe make an IRC community?

EDIT: I think it’s called EMDR

Im sorry you had to witness that. I grew up on forums and message boards in the mid 2000s and onward and Ive still not forgotten a lot of the shock images and other vile things people would post on forums.

You essentially sacrificed a part of your innocence to aid the community and i find that incredibly selfless and respected, although again I am truly sorry it ever had to come to that.

I know it's not really my place as a stranger to give unsolicited advice but if you find yourself struggling, there is a form of therapy called EMDR that is supposedly very successful with getting the brain to fully digest traumatic events.

I’m sorry this happened to you. I think you should disable images on beehaw. Not worth compromising your mental health and it’s too big a job to moderate that level of stuff - which you will have to do if images can be hosted.

I’m really sorry again that someone traumatized you and others with csam. Some people are beyond effed up - I’m really angry hearing about this.

You have my vote.