This is a paradox which has bothered me for some time. Our first immigration law was explicitly racist, but brought about by the mostly immigrant-irish labor movement of the late 19th century. The pro-labor "progressive" movement of the early 20th Century was infused with this racism.
https://medium.com/the-shadow/the-paradox-of-lucile-eaves-5e59b825608b