Excess opioid use can cause dependence and increase tolerance to the painkilling effects faster than tolerance is built to the effects it has on respiration. As such, certain types of opioids are exceptionally dangerous when abused without mechanisms to deal with overdoses. Therefore, we say that abusing opioids is bad for health.

On the other hand, we can take the example of the early human who found joy in gathering food. Similar to the argument about "pathways being influenced", we see that neural pathways are reinforced because of repeated concentration on the same goal. However, treating this as an addiction means that once this "addiction" is cured, humans will no longer want to get food. That means that humans will suffer adverse effects due to giving up the desire to gather food. Someone particularly ignorant could even extend this argument to call water and oxygen addictive.

Clearly, a line must be drawn to distinguish between things that are addictive and those that are not. You gave the example of reading and said that excessive concentration causes reading to behave in mechanisms similar to drugs and I totally agree with that statement. However, the fact remains that reading does not cause negative impacts on health despite repeated exposure to reading whereas the same is not true for drugs.

Since you asked for a narrower and non-speculative explanation of the fact that drugs do damage and books do not, let me ask a concrete question in reply. You stated that you have taken psychedelics in the past. Do you feel that if you had encountered a bad episode, you would have had the ability to leave the episode immediately? Would there have been a way to flush all psychedelics from your body? Clearly, with books, you can just stop reading the book, throw it in a paper shredder, or burn it to ash. Can you do the same with all drugs? Is reversibility really that easy for every single drug?

One could argue that binge reading is harmful and I totally agree. But the overall benefits of reading are sufficiently powerful as compared to the extremely low rate of addiction.

In fact, if looking at DSM-5 criteria, we can almost entirely ignore all points related to social impairment as reading is a major social obligation in a lot of places. Similarly, tolerance does not build up when reading. Another example of DSM-5 criteria we can ignore is the fact that physical and psychological problems do not occur. In fact, we can say that the only meaningful criteria are those related to withdrawal and those related to impaired control.

In books, the rate of impaired control is generally negligible as is the rate of withdrawal. Similar to how someone who drinks fifty litres of water a day is generally considered addicted to water, so is a person who reads instead of eating, taking care of personal hygiene, and sleeping. Yet the general rate of both water addiction and reading addiction is absurdly low when compared to the benefits.

Generally, books are considered non-addictive because they enhance one's quality of life without causing negative health effects. It is a non-speculative fact that books have very little adverse effects. Requiring concentration alone is not sufficient to call something addictive. While it is true that anything done in excess is bad (e.g., getting too much oxygen or water), most people read in moderation. Something addictive needs to be damaging to the general quality of life. That is precisely why all pharmaceutical drugs given for medical purposes are given with one question in mind: will taking a given drug increase or decrease the quality of life in the short and long term.

NOTHING IN THIS REPLY CONSISTS OF MEDICAL ADVICE

I have not added inline citations as I do know which of these points are likely to be challenged. For further reading, please read about the DSM-5 criteria.

It depends on the drug honestly. But yes, generally there is irreversible neural and physical damage caused to the body.

Take huffing for instance. It is a very powerful addiction that causes a lot of brain damage. But is it really the chemical being huffed or is it another mechanism like oxygen deprivation.

This is why I say it depends and once again I think most the damage would come from the results of addiction (which is just pathways being reinforced in the mind). A lot of time with drugs like meth most of the damage comes from people not taking care of themselves (not eating, sleeping, etc.).