You could use
WPF Performance Suite
Optimizing WPF Application Performance
A similar issue haunts me.. (something like)
At application startup I query the database and retrieve all employees and related information to be shown in the ListBox. This is kept in memory the entire time.
May be you could modify your code to follow WPF: Data Virtualization