Tenant didn't sign lease and is not paying rent?

Yes, you can evict him - especially since there is no lease. A lease protects the tenant as well as it does you. I'd give him a written notice that if he doesn't pay by a certain day, he will be evicted. Then change the locks and put his stuff out on the porch and have him come get it.