I'm hoping to understand how venial sins are counted against us.
Let's say I went to confession on a Sunday and confessed all the sins, both mortal and venial,
that I could recall. The priest gives me absolution but a few days later I die. In those few days I didn't
commit any mortal sins but a number of venial sins. 
Will my time in purgatory be based on just these handful of venial sins or all the sins I have accuмulated
throughout my life?

I don't know about that, Nadir.  I always thought that even though your sins HAVE been forgiven, we still have to suffer for the offense we've given to God, either in this world or the next.  Yes, it's true that God does "forget" your sins in the sense that past mortal sins that have been absolved through confession won't be held against you at the moment of your death, but you are still accountable for the action. 

To the OP, who can say how God will judge your time in Purgatory?  There are so many factors besides just the sins you commit!  Your faults, your acts of charity, your prayers, state of life, etc are all considered.  I would suggest you read the book "Purgatory".  It gives an excellent explanation of it, as well as many interesting anecdotes.  Actually, I'd recommend every Catholic read it. 

I don't know about that, Nadir.  I always thought that even though your sins HAVE been forgiven, we still have to suffer for the offense we've given to God, either in this world or the next.  Yes, it's true that God does "forget" your sins in the sense that past mortal sins that have been absolved through confession won't be held against you at the moment of your death, but you are still accountable for the action.

To the OP, who can say how God will judge your time in Purgatory?  There are so many factors besides just the sins you commit!  Your faults, your acts of charity, your prayers, state of life, etc are all considered.  I would suggest you read the book "Purgatory".  It gives an excellent explanation of it, as well as many interesting anecdotes.  Actually, I'd recommend every Catholic read it.

God makes the rules, it's his universe and we're his creatures, however I don't quite understand having to pay for sins that I've been forgiven of through the act of contrition, especially by means of indescribable torture lasting centuries. 
"though your sins be as scarlet, they shall be as white as snow" Isaiah 1:18 
This doesn't exactly line up with what a lot of the visions various saints have had of purgatory. 
If your child disobeys you and he is truly sorry, asks for forgiveness, you forgive him. Do you then 
give him the equivalent of a million spankings and call it mercy?
What am I missing here?
I get that no impure thing can enter heaven or God's presence but at the risk of sounding like a protestant, it sounds like Christ is paying only the debt for our mortal sins while we have to expiate venial sins in horrific ways for possible centuries(as the seers of Fatima were informed about their deceased friend).
Sometimes I wonder why God did not expand upon such an important stage of the afterlife in the Bible itself, rather than by means of private revelations by saints over a thousand years after the Bible was completed.

The time in Purgatory would depend on the amount of temporal punishment you have at the moment of your death.

Each of us has a running account of temporal punishment. When you commit a mortal sin and are absolved, you still have to do some penance to atone for the temporal punishment left over from that. Also, venial sins incur temporal punishment for us too.

If you confess your sins, mortal and venial, in confession, you are put in sanctifying grace, and the sacrament removes some of your temporal punishment for the venial sins, but not necessarily all of it. And mortal sins come with a lot more temporal punishment since they are much more serious. This temporal punishment must be atoned for as well, even after absolution.

You pay down your debt of temporal punishment by attending Mass (the most effective way), voluntary mortifications, patiently enduring suffering for the love of God, getting indulgences, good works, receiving the sacraments, and similar good actions.

When you die, any temporal punishment you have not yet paid off becomes the suffering you have to endure in Purgatory. And it's important to pay off as much temporal punishment as you can in this world, because you get a lot less credit for the pains of Purgatory, in terms of canceling temporal punishment, than you do for suffering in this world.

Other people have already explained. I will only phrase it in a different way:

You will pay for every sin you commit. Mortal or venial. In this life or in the next.

There are things that can reduce your time in Purgatory, if you are fortunate enough to get there, that are mortifications, good works and indulgences.

If you receive a plenary indulgence right after confession and then die, you can go straight to Heaven.

So, Christ paid for our sins but we have to pay for them too? Doesn't that diminish Christ's ultimate sacrifice, that we can in anyway add to it? 
The forgiveness of our sins is predicated upon our sincere contrition, yet...they are apparently not entirely forgiven as we will be punished nonetheless and can only mitigate this punishment by expiation in the form of indulgences, prayers, charity, etc. in this life plus a huge amount of suffering in the afterlife?
Well, I need to readjust my concept of mercy I suppose. 

I do understand that our sins are forgiven in confession because the church has the power to bind and loose but from where do we get the idea that God forgets?
If He does forget, then are we judged on only the sins we die with?
At the final judgement or even the particular judgement, have all our sins not been recorded?

 I am he that blot out thy iniquities for my own sake, and I will not remember thy sins. Isaiah 43:25
So, God blots out our sins and does not remember them, yet keeps a record of every single sin so we can have everyone who ever lived see and hear them in high definition?
I don't consider myself a theologian but something is at odds with passages like this, at least to my feeble mind

This 3-part article on Purgatory is pretty good:

https://www.traditioninaction.org/religious/e051-Purg_1.htm

https://www.traditioninaction.org/religious/e053-Purg_2.htm

https://www.traditioninaction.org/religious/e054-Purg_3.htm

Understanding something of the Church's teaching on the spiritual life has helped me to understand Purgatory better. Here's one of the shortest books on the subject that I am aware of:

https://archive.org/details/threewaysofspiri00garr/page/n8/mode/1up
(you will need to have an account, which is free, to view the entire work)