Is it possible to limit access to particular emails using the built-in authorization server?

I don't think so. For this, you should use an external auth server configured in your IdP.

thanks