this post was submitted on 30 Mar 2026
10 points (100.0% liked)
Programming
26331 readers
162 users here now
Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!
Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.
Hope you enjoy the instance!
Rules
Rules
- Follow the programming.dev instance rules
- Keep content related to programming in some way
- If you're posting long videos try to add in some form of tldr for those who don't want to watch videos
Wormhole
Follow the wormhole through a path of communities !webdev@programming.dev
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Depends on programming language and built in methods that are being used. I described in a more fundamental way how it may work assuming, that OS itself will eventually use at least 1 thread to read a file. From my perspective of view, this will be our main body thread in which will be a cycle, that reads file line by line and gives ready chunks for other threads to process. As I described in other comment somewhere here, we can simplify this pipeline into firstly read the file into RAM splitting it into pieces. And only then process in parallel. I agree that second approach is more convenient one and easier to implement.