I think there is room to add a higher quality phone in at the smaller tab. Personally right now I am in between phones.I have the Moto G but it has been subpar would like an I phone 5s but its outta my price range. The phones that are in my price just are not good enough
Nexus 5 is still a good option for Tab M. It's like a souped up Moto G with LTE, bigger display, and more storage.
At least budget phones these days like the Moto G, Core, and Lumia 635 are much better than they were a few years ago (One V, Ace/Ace II X, etc.) either way, yes, you generally do get what you pay for.
If you're willing to go off Tab, another option is to look into manufacturers like Xiaomi or Asus with their Zenfone series. They offer some compelling Android specs for reasonable prices.