How much reliance on third party am I subjecting myself by using Tailscale? What happens if I make a local connection to a machine/service running on Tailscale, does it still go out of the local network? If so, is the bulk of the payload transferred locally? Is there any advantage on using it if the machine/service is easily accessible over ipv6?
It will route directly over the local network when possible.
It will be encrypted through the VPN, so there will be some overhead. But will be as direct as it can be. It only routes through tailscales servers as a last resort, when it can’t find a direct route at all (usually because NAT holepunching fails somehow). Their “DERP” relay servers just relay the encrypted connection. I think you can use your own relay servers, but I don’t know if that feature can be disabled entirely.
Headscale can be entirely self-hosted. It still uses the tailscale client applications- but is compatible.
That's what Tailscale is built for, when it can it sets up a P2P connection, it only ever sends data through Tailscales servers if you are in a restrictive network environment (i.e. corp network that controls all inbound and outbound traffic)
good questions, pretty well answered by other commenters. if you are happy with the level of encryption you have on your 'plain' ipv6 connection, sure, use that.
additionally the acl/auth system, their dns and service discovery thing is nice, though not essential.