The "right to privacy on one's own property" is only "a powerful and long-established rule" as far as one's home is concerned. For land it is a comparatively modern concept. Land was once regarded as a communal asset, over which most members of the community owned some rights. The idea that ownership of one of those rights, the freehold of the soil, should convey the right to exclude everyone else would have seemed monstrously perverse and unjust to our ancestors.
Unregulated access to land is now neither possible nor desirable, because of the immense population of our island, but the limitations of access should be determined by what gives the greatest benefit to the greatest number, not on a false understanding of the very nature of land ownership.