It seemed Latino men were the workers that made it possible for the Hamptons to exist as a walled garden for White people.