Subversion and Trac don't take up very much of your system resources especially when they're just sitting there waiting for you to use them.
I'm working on a small project, just 2 full-time developers, but I find I use our bug tracking software (Jira in our case, similar to Bugzilla), even just to keep track of my own bugs and issues. Easier than trying to remember the details of a bug when you get time to work on it later, and there's a very satisfying feeling to checking off issues after you've resolved them.
I'd stick with something simple in your case, even just a dedicated to-do task list in Outlook would probably suffice if you don't want to spend too much development time setting up a dedicated system. The important thing is to have a way to know for sure you've solved all of the bugs that were reported.