Hey Vlad, these are good points.
Yes, you do need to consider your audience. Some providers (e.g. Auth0, Azure AD B2C) allow users to create local accounts (with username and password specific to your service), so that can mitigate the issue.
As for the cost, you are correct. However, if you do have that number of users, then you are probably facing very high costs for your infrastructure already. I would argue also that the money spent for the auth provider is money well spent: the more users, the more appealing of a target you become to attackers, and potential breaches become costlier too.