Chaff is the outer covering of the grain and has to be separated from it. The proverb means that merit is often obscured by worthlessness. George Wither, the Jacobean poet, wrote this:
Till from the straw the flail the corn doth beat,
Until the chaff is purged from the wheat,
Yea, till the mill the grains in pieces tear,
The richness of the flour will scarce appear.