allocate more memory to void * on iteration and then add the value

问题

I have a memory allocation problem with an implementation of a column-store database system. This, in particular is part of implementing a MySql connector to import a table into my database.

Say I have the following vector:

 std::vector <void *> data;

I have a function that checks for types to add data to the pointer as follows: As you can see, for each column we have a void * added to the vector.

for(int c = 0; c != numCols; c++){

      // ...
      case BOOL_TYPE: {
                 bool *b;
                 data.push_back((void *) b);
             }
             break;
      // ... 
}

now another function that read the table rows needs to add the data

    while (mysqlpp::Row row = res.fetch_row()) {
                    for (int c = 0; c != numCols; c++) {
                        // ...
                        switch (colType){
                    case BOOL_TYPE: {
                        if(!isNull){
                            bool val = row[c];
                            data[c] = new char[sizeof(val)];
                            data[c] = val;
                        }
                    }
                    break;

                   //... more types

                }



                    }
   }

I am concerned about these lines:

 data[c] = new char[sizeof(val)];
 data[c] = val;

I am very new to C++ and memory management so I am not sure how I can allocate more memory to the void * each time and then add the value?

EDIT This is for a column-store database where I am storing the data in columns not rows. To be clear, I need a clean way of adding the data from mySQL which is a row-store database into my columns.

An easy solution to my problem would be if I can get the number of rows in a table using MySql++? Then I can allocate that many rows * (size of datattype) bytes initially and fill this memory in.

回答1:

Using raw dynamic allocated arrays gets inconvenient quite quickly and it is memory leak prone.

I suggest using a variant type to store the values without having to do manual memory management. E.g. boost::variant:

typedef boost::variant<
    std::nullptr_t, // for DB NULL
    bool,           // for DB BOOL
    intmax_t,       // for DB Integers
    double,         // for DB Reals
    std::string     // for DB varchar
    > Value;

And then:

//...
data.push_back(Value()); // nullptr value

Or just data.resize(column_count).

and later:

// ...
bool val = row[c];
data[c] = val;

There are two major ways to treat database data:

Strongly-typed. In which case your in-memory columns store the exact type, e.g. std::vector<int> for an Integer DB column. Or, row-wise, a record struct is defined for each query, rows stored as std::vector<Record>.
Weakly-typed. Something like std::vector<Variant> for columns, or std::vector<std::vector<Variant>> for rows. It is hard to make a better variant type than boost::variant though.

Looks like you use the weakly-typed way with void* being the variant type. void* is type unsafe, meaning no errors are caught at compile time, e.g. memory management errors, like casting void* to a wrong type; or forgetting to free the memory or freeing it more than once. Using void* considerably increases the risk of a crash or corruption for no apparent reason. You may like to rethink your design sooner rather than later.

回答2:

For a quick (and in my opinion) dirty solution that might work with the code you have, is to copy the data, not overwrite the pointer and thereby getting a memory leak (as well as the possibility of having the data it points to disappear any moment).

First of all, instead of just initializing it you allocate memory that you can later free:

case BOOL_TYPE: {
    data.push_back(new bool);
    break;

Then when setting proper data, you first free the existing memory, then reallocate it while setting the value:

case BOOL_TYPE: {
    if(!isNull){
        delete data[c];
        data[c] = new bool(row[c]);
    }
}
break;

来源：https://stackoverflow.com/questions/22077283/allocate-more-memory-to-void-on-iteration-and-then-add-the-value

标签

c++

memory-management

mysql++