I immediately pulled up the whole row of Broccoli, every one of the plants were infested. Really disgusting!
After doing some research on the web I found that the only good way to control these critters organically is with beneficial nematodes. So I found some fresh nematodes at Garden Fever in Portland and sprayed them on the soil.
It takes a while for them to eat all the worms and larvae, hopefully I acted soon enough to save my kohlrabi and Kale.