What we can call the “desert argument” for a land value tax goes something like this. The owner of land does nothing to deserve increases in the value of that land. For instance, if the land becomes more valuable because a bunch of job opportunities pop up nearby, that’s not something the owner can claim credit for. So, the argument goes, it’s not unjust to tax the value of that land.
I’m sympathetic to this argument, but it’s unlikely to persuade libertarian land value tax skeptics. Suppose Todd is 7 feet tall and lives somewhere that suddenly starts caring a lot about basketball. Now Todd’s height has become significantly more valuable, but it’s not due to anything he has done. So we can say that Todd doesn’t deserve the economic value of his height.
Here again, I find this claim plausible. But most (deontological) libertarians will argue that Todd is entitled to the economic value of his height even though he did nothing to deserve it. After all, you can be entitled to things even if you did nothing to deserve them (for instance, your kidneys).
So the desert argument for a land value tax doesn’t pressure the deontological libertarian to a bite a bullet that they haven’t already bitten. Of course, maybe they shouldn’t bite that bullet, but that’s an issue for another post.
As a Georgist, I actually agree that the typical framing of the desert argument doesn't quite work. If people gain but that gain doesn't harm anyone else, we don't mind. The real problem is that private ownership of land allows you to take from others by gatekeeping access. If land was produced this would be fine bc that gatekeeping is compensation for the costs of production. But land is non-produced. So you don't deserve to appropriate the value of it for yourself.
(I'm typing this up quickly so it's not fully nuanced but this is the gist)
I would argue that it less about "deserving" (though I have made this argument as well) and more about efficiency. As far as taxes go, LVT offers unparalleled advantages over other kinds of taxation: https://www.lianeon.org/p/just-tax-the-land