Broad Street, usually buzzing with activity, is quieter as people fear immigration raids