I'm digging into the pure math here because I'm suspicious of whoever came up with that estimate. Not in a malicious way, but in plain theoretically inaccurate way. 2 million years does not sound reasonable. If total distance wasn't involved, it sounds reasonable just from a self-duplication standpoint. For the number of self-duplication cycles, 2^37 is approx 140 billion and 2^39 is approx 550 billion, which covers our galaxy's estimated 100-400 billion stars. Add an iteration for the initial launch and you figure the range is 35-40 iterations. That allows 50,000-57,000 years average per cycle to travel to the next star. Then, you figure if we can build one self-replicating probe, we can build 4 or even 8 to start. 8^13 is 550 billion, same as 2^39 right. So now, we're talking up to 150,000 years per interstellar trip. Voyager 1's steady interstellar speed would cover the ~4ly gap to Proxima Centauri in ~40,000 years. As it sits now, though, it still hasn't even reached 1 light-day from Sol. While we should have better thrust tech now, it's probably still within an order of magnitude due to Voyager's excellent use of gravitational slingshots in a rare planetary arrangement. Smells OK so far.

But hold on. The galaxy is not linear, it's circular (for the most part), which means we have to consider 2D area, not a 1D line. Even though we're squaring the probes, the area covered is going up by a square as well. As far as distance goes, the squares cancel each other proportionately. So we do have to look at one linear consideration: distance to the other side. At about 105,000ly across and Earth sitting approve 26kly from both the nearest edge and the galactic center, it's about 87,000ly to the other side. Covering that distance in 2 million years would take a speed of 0.0435c - 29,000,000mph, or 700x that of Voyager. But I guess no thing's wrong with 20 million years of exploration or 200 million years, in the grand scheme of the universe, dropping the required interstellar speed to just 7x that of voyage. Make it 2 billion years and ti's attainable with current technology, with 0.7x Voyager being about 28,000mph.

Regardless, I still have major doubts about this theoretical probe's ability to slow at the next star, find suitable solid resources, stop to mine them, distribute its payload, manufacture 2 new probes (or 1 new and prepare itself), and be able to launch 2 probes with enough speed to escape the current system. Voyager 1 and Voyager 2 have escaped the heliosphere as of 2012 and 2018, respectively.That is all we've sent so far. That's all we have in motion so far. Escaping our planet is such a major hurdle that of the ~200 probes sent beyond Earth's gravitational dominance, only 2 have left the solar system. And while 28kmph sure sounds a lot like escape velocity from Earth, the peak speed was achieved at the time it left - over 90,000mph. That is a substantially greater amount of thrust to leave. That's not the velocity needed to leave Earth, that's the velocity needed to make sure it leaves the sun. After that is where the probes used slingshots to gain meaningful speed to reach the heliopause within our lifetimes instead of stalling in some Plutonian orbit.

So, as to why we're not flush with probes, I think it comes down to the basic concept that such a trip far exceeds not only a life time, but all human concepts of time. The oldest known hominin tool is about 3 million years old. The oldest wooden structure, about 500,000 years. Jewelry, 150k years, cave paintings, 65k, and written history is just 5,000 years old. To complete this expedition in 2 million years means it'd exceed the the existence of our species in its entirety. At 2 billion years, it'd exceed the time that Earth has had multicellular life.

Even with all that said, it's be a one-way trip, a one-way message. The first few iterations of replication would likely exceed any type of life as we know it on Earth. It may not even remain in any kind of historical record. Humans may be gone. Surviving life likely will not be sentient/intelligent enough to receive any kind of return message, if they even had the technology and the knowledge to know what to listen for. Just a message from the core would take 26,000 years to reach Earth - and we're back to the law of squares where the message beam will be expanding, and weakening, with a squared ratio to distance traveled. We're struggling to communicate with Voyager as it is.

So, the question is: why bother? Conceptually easy task with no tangible payback. It'd only satisfy some manifest destiny, likely of religious or nationalist origin. That doesn't exactly resonate with the general science community and it'd be extraordinarily difficult to get governmental funding to support a life-spreading probe with their little, universally-meaningless flag attached.

There could be multiple alien probes in our star System now and we would not know it. Space, even in our star System, is huge and probes are, most likely, small. We have huge problems locating, finding and tracking all the asteroids in our star system and we are still not sure if we have found all the planets, why should we be able to locate a alien probe?

how does it self replicate though? I guess it can guarantee finding solar power, but how does it find and process the materials it needs when it reaches the next star

However we do come back to Fermi's Paradox: the universe is 13800million years old. So far we have no evidence a probe has reached our star system. Where are they? Maybe we just haven't stumbled across one yet. Or maybe life really is very rare?

Well there are actually some pretty convincing explanations for for the Fermi Paradox. The one I like most is that despite the 13800 million years, we're pretty early for complex life, and we may just be the first technological civilization in the Milky Way.

The theory goes that while there are a lot of older stars out there, there aren't a lot of 3rd generation stars. When some stars get to the end of their life, they collapse and then go supernova, leaving behind a dust cloud in which other stars can then form. Here's the thing, many heavy elements are only produced through nuclear fusion during a supernova. (Basically everything heavier than iron, about 3/4 of the periodic table). So that second generation of stars contain more heavy elements within the star. When the second generation goes supernova, more heavy elements are created.

Life exists on earth thanks to all of the complex chemical reactions that can take place on earth. But all that may only be possible because we're a 3rd generation solar system with all our fancy heavy elements, and 3rd Gen star systems are relatively new, we're quite early on the scene.

So because of that, we're certainly an early civilization, and we might be... the first.