this post was submitted on 12 Aug 2026
10 points (100.0% liked)
Web Development
5787 readers
4 users here now
Welcome to the web development community! This is a place to post, discuss, get help about, etc. anything related to web development
What is web development?
Web development is the process of creating websites or web applications
Rules/Guidelines
- Follow the programming.dev site rules
- Keep content related to web development
- If what you're posting relates to one of the related communities, crosspost it into there to help them grow
- If youre posting an article older than two years put the year it was made in brackets after the title
Related Communities
- !html@programming.dev
- !css@programming.dev
- !uiux@programming.dev
- !a11y@programming.dev
- !react@programming.dev
- !vuejs@programming.dev
- !webassembly@programming.dev
- !javascript@programming.dev
- !typescript@programming.dev
- !nodejs@programming.dev
- !astro@programming.dev
- !angular@programming.dev
- !tauri@programming.dev
- !sveltejs@programming.dev
- !pwa@programming.dev
Wormhole
Some webdev blogs
Not sure what to post in here? Want some web development related things to read?
Heres a couple blogs that have web development related content
- https://frontendfoc.us/ - [RSS]
- https://wesbos.com/blog
- https://davidwalsh.name/ - [RSS]
- https://www.nngroup.com/articles/
- https://sia.codes/posts/ - [RSS]
- https://www.smashingmagazine.com/ - [RSS]
- https://www.bennadel.com/ - [RSS]
- https://web.dev/ - [RSS]
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
why html? just saving the json it fetches in the first place would be a lot easier to work with IMO, but it depends on what you're doing with it
I second that - can't you just use the API and request the post and its comments as json?
Unless you want to explicitly archive the post as a human viewable HTML, the structured json from the API is almost always superior to further analyze, archive,... the posts.
Archiving thwmin a human readable form is exactly what I want to do.
Then I'm not sure if curl is up to the task in this case - most modern websites are PWAs that dynamically request data from their Backen and display it. Curl just fetches the initial HTML but doesn't execute the JS and thus might not see the content of the page as it isn't there yet the moment it gets saved.
Some PWAs "render" the first page contents in your HTML so that it is included and do not need to be requested.
Maybe try the alternative frontends if some of them work better? Also remember that feddit.org and many other instances deploy Anubis and might lock you out from making simple requests with curl
Got it working with wget