So with my nexus 5 im easily able to do something like this to connect adb over wifi:
adb tcpip 5555
adb connect 192.xxx.x.xx
and then disconne
For me, it looks like this was a bug with ADB server version (39). I updated to (41) and the issue went away.
It's super weird, because I was using version 39 on two different PCs and on one adb/wi-fi worked, and on the other, as soon as I unplugged the USB cable the Wi-Fi device would become disconnected as well.