General strategies for memory/speed problems

Question

I have a c++ code which runs through about 200 ASCII files, does some basic data processing, and outputs a single ASCII file with (basically) all of the data.

The progra

Answer 1

Analyze your code with callgrind, part of the valgrind suite. You can graphically browse the results with kcachegrind. (Despite its name, it works on callgrind output too.) It's free and will give you awesome detail.

You can also externally turn data collection off and on. So start with it off, wait until your program gets slow, turn it on during the problem time, then turn it off. You'll see where the CPU was going. If necessary, do the same thing in reverse watching only when it's fast and compare.

Usually, the problem will stick out like a sore thumb.

Answer 2

Could you share your program ?

1. One thing to look for is whether you are using data structures that do not scale with increasing number of elements.

e.g. using lists to hold a million elements would be extremely slow to traverse/ search (O(n)) as opposed to say using a binary search tree (nlog(n)) or a hashing (O(1)).

2. You should look at whether you are holding on to the data at the end of each cycle (/burn/run) . Ideally you should release all the resources at the end of each run.

3. Sounds like there may be a handle leak ?

Answer 3

This is a total shot in the dark. You've got:

bool getDirectoryContents(const string dirName, vector<string> *conts) {
    ...
    copy(directory_iterator(p), directory_iterator(), back_inserter(v));

How does the performance change if you instead make that:

bool getDirectoryContents(const string dirName, vector<string> *conts) {
    ...
    // note: preincrementing the iterator
    for (directory_iterator it((p)); it!=directory_iterator(); ++it) {
       v.push_back(*it);
    }

My thought is that std::copy is specified to use postincrement. And boost::filesystem::directory_iterator is an InputIterator: it shouldn't really support postincrement. boost::filesystem::directory_iterator may not be happy being postincremented.

Answer 4

Without more information to go on, I'd guess that what you're dealing with is a Schlemiel the Painter's algorithm: (Original) (Wikipedia). They're incredibly easy to fall into doing string processing. Let me give you an example.

I want to read every line in a file, process each line somehow, run it through some intermediate processing. Then I want to gather up the the results, and maybe write it back to disk. Here's a way to do that. I make a single huge mistake that can be easy to miss:

// proc.cpp
class Foo
{
  public:
  std::string chew_on(std::string const& line_to_chew_on) {...}
  ...
};

Foo processor;
std::string buffer;

// Read/process
FILE *input=fopen(..., "r");
char linebuffer[1000+1];
for (char *line=fgets(linebuffer, 1000, input); line; 
     line=fgets(linebuffer, 1000, input) ) 
{
    buffer=buffer+processor.chew_on(line);  //(1)
}
fclose(input);

// Write
FILE *output=fopen(...,"w");
fwrite(buffer.data(), 1, buffer.size(), output);
fclose(output);

The problem here which can be easy to miss at first glance, is that each time line (1) is run, the entire contents of buffer is copied. If there are 1000 lines with 100 characters each, you end up spending time copying 100+200+300+400+....+100,000=5,050,000 byte copies to run this. Increase to 10,000 lines? 500,500,000. That paint can is getting further and further away.

In this particular example, the fix is easy. Line (1) should read:

    buffer.append(processor.chew_on(line)); // (2)

or equivalently: (thanks Matthieu M.):

    buffer += processor.chew_on(line);

This manages to help because (usually) std::string won't need to make a full copy of buffer to perform the append function, whereas in (1), we're insisting that a copy be made.

More generally, suppose (a) the processing you're doing keeps state, (b) you reference all or most of it often, and (c) that state grows over time. Then there's a fair to good chance that you've written a Θ(n²) time algorithm, which will exibit exactly the type of behavior you're talking about.

---

Edit

Of course, the stock answer to "why is my code slow?" is "run a profile." There are a number of tools and techniques for doing this. Some options include: