Yes, and I think what he means by afflictions would be what later theologians refer to as imperfections.

There were some saints who probably committed a couple of venial sins in their entire lives, so if it were speaking strictly about sin proper, then it would likely be untrue, since sin generally requires a deliberate act of will made with sufficient reflection, whereas imperfections don't require that degree of will or intent, where it could just be a habit done unthinkingly, or a weakness in resisting temptation, or some shortcoming by omission (like not having as much love for God or neighbor as we ought), which are all failings but not sins in the technical sense.  St. Augustine did a great deal to distinguish between things like mortal and venial sins, and here he's also speaking about imperfections.  I think that before he worked it out, there was a sense of needing to confess mortal sins, but not too much theological reflection on venial sin, or imperfections.  St. Augustine did a lot to clarify these things due to the depth of his introspective thinking.