Opinions please - dodgy, or reading too much into it?

I ordered a new router (a couple hundred bucks) off Amazon. Because of the value, I need to provide a pin code on delivery (pretty sure it is so I can't argue I never received it). They say you can only use a pin code, but I know from past experience that delivery drivers have a button to deliver with the phone number on file instead of the pin code. I guess in case somebody doesn't have their pin code

The drivers have a button to call the number on file via amazons number in Sydney, but they don't see your number, just name. I got 5 calls (missed the first 4 from the driver, watering the garden and left my phone inside) and when I answered he asked for my phone number. I was worried about giving it over the phone, because I'm fairly sure if I did, he could mark it delivered to himself then Amazon would tell me I'm SoL. When I refused, he got a bit snarky and said "ok you don't want package then that your choice", even though I said I'd be home, then he hung up

Seems dodgy, right? I don't know how their systems work, but I'm pretty sure they don't even get the button to call you until they scan the barcode on your box, which I don't think they'd do until delivery time. So I'm kinda wondering if maybe he tried to deliver it to himself and called me to get my number so he could. But I also know they are basically glorified uber eats drivers and get paid per parcel, so perhaps the less cynical assumption is he wanted my number so he could deliver it as soon as it arrives and doesn't need to wait the 3 seconds for me to answer the door, which probably does negatively impact averages. Idk. Just feels sus