Optimizating my code simulating a database

Question

I have been working on a program, simulating a small database where I could make queries, and after writing the code, I have executed it, but the performance is quite bad. I

Answer 1

Though I advise you to please use a profiler to find out which parts of your code are worth optimizing, here is how I would write your program:

Read the entire text file into one string (or better, memory-map the file.) Scan the string once to find all | and \n (newline) characters. The result of this scan is an array of byte offsets into the string.

When the user then queries item M of row N, retrieve it with code something like this:

char* begin = text+offset[N*items+M]+1; 
char* end = text+offset[N*items+M+1];

If you know the number of records and fields before the data is read, the array of byte offsets can be a std::vector. If you don't know and must infer from the data, it should be a std::deque. This is to minimize costly memory allocation and deallocation, which I imagine is the bottleneck in such a program.

Answer 2

No need to reinvent the wheel again, use FirebirdSQL embedded database instead. That combined with IBPP C++ interface gives you a great foundation for any future needs.

http://www.firebirdsql.org/

http://www.ibpp.org/

Answer 3

One obvious issue is that your get-functions return vectors by value. Do you need to have a fresh copy each time? Probably not.

If you try to return a const reference instead, you can avoid a lot of copies:

const vector<Table>& getPointer();

and similar for the nested get's.

Answer 4

Whenever you have performance problems, the first thing you want to do is to profile your code. Here is a list of free tools that can do that on windows, and here for linux. Profile your code, identify the bottlenecks, and then come back and ask a specific question.

Also, like I said in my comment, can't you just use SQLite? It supports in-memory databases, making it suitable for testing, and it is lightweight and fast.

Answer 5

I have not done the job, but you may analyse the complexity of your algorithm. The reference says that access an item is in constant time, but when you create loops, the complexity of your program increases:

for (i=0;i<1000; ++i) // O(i)
  for (j=0;j<1000; ++j) // O(j)
     myAction(); // Constant in your case

The program complexity is O(i*j), so how big may be i an j? What if myAction is not constant in time?