Is affirmative action fair?

As a minority myself, I understand that it was needed a few years back as a pick up for all the injustice minorities had gone through. Now, I feel like it should be open for ALL citizens who've had less chances. (Even White men )We have poor citizens from all races and gender. Why should A.A. just be for minorities.