Man served as 'token white guy' on selection committee, landed in hot water when he chose most qualified candidates