this post was submitted on 22 Sep 2025
1126 points (99.1% liked)
Microblog Memes
9330 readers
3288 users here now
A place to share screenshots of Microblog posts, whether from Mastodon, tumblr, ~~Twitter~~ X, KBin, Threads or elsewhere.
Created as an evolution of White People Twitter and other tweet-capture subreddits.
Rules:
- Please put at least one word relevant to the post in the post title.
- Be nice.
- No advertising, brand promotion or guerilla marketing.
- Posters are encouraged to link to the toot or tweet etc in the description of posts.
Related communities:
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Feature Visualization How neural networks build up their understanding of images
https://distill.pub/2017/feature-visualization/
This method is definitely a great way to achieve some degree of explainability for images, but it is based on the assumption that nearby pixels will have correllated meanings. When AI is making connections between far-away features, or worse, in a feature space that cannot be readily visualized like images can, it can be very hard to decouple the nonlinear outputs into singular linear features. While AI explainability has come a long way in the last few years, the decision-making processes of AI are so different from human thought that even when it can "show its work" by showing which neurons contributed to the final result, it doesn't necessarily make any intuitive sense to us.
For example, an image-identification AI might identify subtle lens blur data to determine the brand of camera that took a photograph, and then use that data to make an educated guess about which country the image was taken in. It's a valid path of reasoning. But it would take a lot of effort for a human analyst to notice that the AI is using this process to slightly improve its chances of getting the image identification correct, and there are millions of such derived features that combine in unexpected ways, some logical and some irrationally overfitting to the training data.