Evening/nightfall by far the most productive time, worms will do the trick.
Usual spots: beaver dams, deep pools adjacent to fast running water.
During the day, you have to drift your worm right under the beaver dam, as though you were trying to snag yourself in the brush. Usually the brownies are nice enough to prevent the snag from happening as they gobble it down.
Undercut banks are also a good target during the morning/day, but be sure to sneak up very carefully, and cast from as far away from the bank as you can.