I tried adding only the IP.
Worked only for root.
But when i try log as user i get acces denied, example:
email@example.com > OK
firstname.lastname@example.org > Access Denied
> Access Denied
On my setup like i did with root i will need add the users one by one?