This is close, after getting the tests working and taking it out of Draft PR, I had come up with issues so I made the tests fail again so it wouldn't be merged prematurely.

Right now with the new code, if the private hosted zone doesn't exist, OctoDNS will attempt to create it, but as a public hosted zone. One issue with automatically creating the private hosted zone is that you have to specify a VPC ID that it goes with. I'm thinking the best approach for this is just to raise an Exception and say creation of private hosted zones is unsupported, and that it must be created manually before it can be managed by OctoDNS

The other thing I was thinking of is that this could be a "breaking" change for someone who uses the current version of the Route53Provider with a mix of public and private hosted zones, but none that have the same name.

In that case, this new code would no longer find the private hosted zones and would attempt to make public hosted zones with the same names. A user would probably have to create a new section in their configuration for a route53_private version of the provider, and swap all their private hosted zones to use it.

I'm not sure if that's acceptable or not. I'd have to do some thinking to figure out a solution that doesn't create that breaking behavior. The public/private with the same name make it really hard, as the zone name is the unique ID throughout OctoDNS, and other providers don't have the concept of private hosted_zones.

One issue with automatically creating the private hosted zone is that you have to specify a VPC ID that it goes with ...

There's been a couple things come up that want/need the ability to attach metadata at the zone level, this seems like it'd be another ++ on that idea (don't remember the others offhand.) Something like:

  foo.internal.net.:
    _octodns:
        route53:
          vpc-id: vpc-123456 # implies private

That may be more of a change than makes sense to add here. The blow up if it doesn't exist and tell them to create it seems fine (and we can circle back someday and improve things with something like the above.) IIRC the main issue is that there's not really a logical place for the data to be hydrated/attached as the way the manager works with zones currently doesn't involve a Zone object that matches the needed scope (been a long time since I thought about this so could be mis-remembering.) I think it should be possible to address this, just didn't seem like a trivial change at the time.

The other thing I was thinking of is that this could be a "breaking" change for someone who uses the current version of the Route53Provider with a mix of public and private hosted zones, but none that have the same name. ...

That's a good question/point. Maybe the default (nothing mentioned) has to continue to work the way it currently does and then things explicitly have to say public vs private to get the new behavior here?