I found this page in the documentation http://www.amember.com/docs/Redirect_Settings but I can't find where these redirect settings are. I would like to change it from the default of showing an error page if the user can't access the protected content but I can't find where to do it, even though according to this page, this option is available, I've gone through all the menus in setup and config and in the protected directories.
I suspect you found that with a search, because the page is actually in the WordPress setup section. That page only refers to the WordPress integration.
First of all, I mainly do WordPress integrations and do not have a lot of experience in using aMember as a stand alone application. However, it is my understanding that when a visitor (not logged in) tries to access a file inside a protected folder they will be redirected to the login page by default. Note that there is a "signup" link at the top of the login form for those that need to signup. If a logged in person tries to access a page from inside a folder they do not have a subscription to, they are redirected to the signup page so they can purchase the necessary subscription. After a person is logged in and they are redirected to the signup page, only products are shown, So effectively a signup page becomes a "renew/purchase new products" page after a person is logged in.
If someone tries to go to a protected page and isn't logged in, you are correct, it goes to the login page, however if the person is logged in and can't get to a protected page, it just sends them to a page that says they can't get to that page because they don't have permission and they need to order that subscription and gives them a little text link to click. It's ugly and awkward and gives them the full url of where they were trying to go. I would much rather that it defaults to the signup page so they can order the subscription, but I can't find where to do that or if it is possible.
The wording, and action, is set in application > default > views > no-access.phtml To do what I think you are trying to achieve replace the contents of this file with a redirect to your sign-up page or a custom page encouraging them to sign up to the subscription