So, no... but also yes.

You're correct that it's unlikely that the device connecting to the network would be able to reach the outside Internet, but it would still be able to reach any local resource to itself, which is to say any other device which is in its network segment and also in the same state (DHCPless) that it is, via what's referred to as a link-local address. These are in the 169.254.x.x/16 or fe80::/10 space and allow devices to self-assign addresses independent of upstream connectivity for communication on the local network segment. Usually, these aren't useful, but these address are consistent, and can be used to directly contact known local hosts from your machine without DHCP. As to whether or not they can reach upstream hosts in this state, the answer is 'probably not', but that's not the same as what you said.