Dealing with char buffers

Question

As a C++ programmer I sometimes need deal with memory buffers using techniques from C. For example:

char buffer[512];
sprintf(buffer, \"Hello %s!\", userName.c_s         


        

Answer 1

Is passing the buffer as &buffer[0] better programming style than passing buffer? (I prefer &buffer[0].)

&buffer[0] makes the code less readable to me. I have to pause for a second and wonder why someone used it instead of just passing buffer. Sometimes you have to use &buffer[0] (if buffer is a std::vector), but otherwise, stick with the standard C style.

Is there a maximum size that is considered safe for stack allocated buffers?

I doubt there's any practical limit, as long as you're using the stack reasonably. I've never had any problems in my development.

If I'm reading MSDN correctly, threads on Windows default to 1MB of stack size. This is configurable. Other platforms have other limits.

Is static char buffer[N]; faster? Are there any other arguments for or against it?

On the one hand, it might reduce the need to commit memory pages for stack, so your app might run faster. On the other hand, going to the BSS segment or equivalent might reduce cache locality compared to the stack, so your app might run slower. I seriously doubt you'd notice the difference either way.

Using static is not threadsafe, while using the stack is. That's a huge advantage to the stack. (Even if you don't think you'll be multithreaded, why make life harder if that changes in the future?)

When using static buffers you can have your function return have the const char * return type. Is this a good idea? (I do realize that the caller will need to make his own copy to avoid that the next call would change the previous return value.)

Const correctness is always a good thing.

Returning pointers to static buffers is error-prone; a later call might modify it, another thread might modify it, etc. Use std::string instead or other auto-allocated memory instead (even if your function needs to deal internally with char buffers such as your GetCurrentDirectory example.)

What about using static char * buffer = new char[N]; and never deleting the buffer? (Reusing the same buffer each call.)

Less efficient than just using static char buffer[N], since you need a heap allocation.

I understand that heap allocation should be used when (1) dealing with large buffers or (2) maximum buffer size is unknown at compile time. Are there any other factors that play in the stack/heap allocation decision?

See Justin Ardini's answer.

Should you prefer the sprintf_s, memcpy_s, ... variants? (Visual Studio has been trying to convince me of this for a long time, but I want a second opinion :p )

This is a matter of some debate. Personally, I think these functions are a good idea, and if you're targeting Windows exclusively, then there's some benefit to taking the preferred Windows approach and using those functions. (And they're fairly straightforward to reimplement if you later need to target something other than Windows, as long as you don't rely on their error-handling behavior.) Others think that the Secure CRT functions are no more secure than properly used C and introduce other disadvantages; Wikipedia links to a few arguments against them.

Answer 2

• It's up to you, just doing buffer is more terse but if it were a vector, you'd need to do &buffer[0] anyway.
• Depends on your intended platform.
• Does it matter? Have you determined it to be a problem? Write the code that's easiest to read and maintain before you go off worrying if you can obfuscate it into something faster. But for what it's worth, allocation on the stack is very fast (you just change the stack pointer value.)
• You should be using std::string. If performance becomes a problem, you'd be able to reduce dynamic allocations by just returning the internal buffer. But the std::string return interface is way nicer and safer, and performance is your last concern.
• That's a memory leak. Many will argue that's okay, since the OS free's it anyway, but I feel it terrible practice to just leak things. Use a static std::vector, you should never be doing any raw allocation! If you're putting yourself into a position where you might leak (because it needs to be done explicitly), you're doing it wrong.
• I think your (1) and (2) just about cover it. Dynamic allocation is almost always slower than stack allocation, but you should be more concerned about which makes sense in your situation.
• You shouldn't be using those at all. Use std::string, std::stringstream, std::copy, etc.

Answer 3

• Buffer or &Buffer[0] is exactly the same. You could even write Buffer+0. Personally I prefer just to write Buffer (and I think most developers also prefer this), but this is your personal choice
• The maximum depends on how big and how deep your stack is. If you are already 100 functions deep in the stack, the maximum size will be smaller. If you can use C++, you could write a buffer class that dynamically chooses whether to use the stack (for small sizes) or the heap (for large sizes). You will find the code below.
• A static buffer is faster since the compiler will reserve the space for you beforehand. A stack buffer is also fast. For a stack buffer the application just has to increase the stack pointer. For a heap buffer, the memory manager has to find free space, ask the operating system for new memory, afterwards free it again, do some bookkeeping, ...
• If possible use C++ strings to avoid memory leaks. Otherwise, the caller has to know whether he has to free the memory afterwards or not. The downside is that C++ strings are slower than static buffers (since they are allocated on the heap).
• I wouldn't use memory allocation on global variables. When are you going to delete it? And can you be sure that no other global variable will need the allocated memory (and be used before your static buffer is allocated)?
• Whatever kind of buffer you use, try to hide the implementation from the caller of your function. You could try to hide the buffer-pointer in a class let the class remember whether the buffer is dynamically allocated or not (and thus should delete it in its destructor or not). Afterwards it's easy to change the type of the buffer, which you can't do if you just return a char-pointer.
• Personally I prefer the normal sprintf variants, but that's probably because I still have lots of older code and I don't want a mixed situation. In any case, consider using snprintf, where you can pass the buffer size.

Code for dynamic stack/heap buffer:

template<size_t BUFSIZE,typename eltType=char>
class DynamicBuffer
   {
   private:
      const static size_t MAXSIZE=1000;
   public:
      DynamicBuffer() : m_pointer(0) {if (BUFSIZE>=MAXSIZE) m_pointer = new eltType[BUFSIZE];}
      ~DynamicBuffer() {if (BUFSIZE>=MAXSIZE) delete[] m_pointer;};
      operator eltType * () { return BUFSIZE>=MAXSIZE ? m_pointer : m_buffer; }
      operator const eltType * () const { return BUFSIZE>=MAXSIZE ? m_pointer : m_buffer; }
   private:
      eltType m_buffer[BUFSIZE<MAXSIZE?BUFSIZE:1];
      eltType *m_pointer;
   };

Answer 4

Is passing the buffer as &buffer[0] better programming style than passing buffer? (I prefer &buffer[0].)

It depends on the coding standards. I personally prefer: buffer + index instead of &buffer[index] but it's a matter of taste.

Is there a maximum size that is considered safe for stack allocated buffers?

It depends on the stack size. If the amount of stack needed for your buffer exceeds the amount available on the stack, it result a stack-overflow.

Is static char buffer[N]; faster? Are there any other arguments for or against it?

Yes, it should be faster. See also this question: Is it bad practice to declare an array mid-function

When using static buffers you can have your function return have the const char * return type. Is this a good idea? (I do realize that the caller will need to make his own copy to avoid that the next call would change the previous return value.)

Not sure what static means in this case but:

1. If variable is declared on stack( char buf[100] ): You should not return references to stuff that is declared on the stack. They will be trashed at next function call/declaration (e.g. when the stack is used again).

2. If the variable is declared as static static it will make your code non-reentrant. strtok is an example in this case.

What about using static char * buffer = new char[N]; and never deleting the buffer? (Reusing the same buffer each call.)

It is a possibility, though not recommended because it makes your code non-reentrant.

I understand that heap allocation should be used when (1) dealing with large buffers or (2) maximum buffer size is unknown at compile time. Are there any other factors that play in the stack/heap allocation decision?

Stack size of the running thread is too small to fit stack declaration (previously mentioned).

Should you prefer the sprintf_s, memcpy_s, ... variants? (Visual Studio has been trying to convince me of this for a long time, but I want a second opinion :p )

If you want your code to be portable: No. But the effort in creating a portable macro is quite small in this case:

// this is not tested - it is just an example
#ifdef _WINDOWS
 #define SPRINTF sprintf_s
#else
 #define SPRINTF sprintf
#endif

Answer 5

1. Stay away from static buffers if you ever want to use your code re-entrantly.

2. use snprintf() instead of sprintf() so you can control buffer overruns.

3. You never know how much stack space is left in the context of your call -- so no size is technically 'safe'. You have a lot of headroom to play with most of the time. But that one time will get you good. I use a rule of thumb to never put arrays on the stack.

4. Have the client own the buffer and pass it and its size to your function. That makes it re-entrant and leaves no ambiguity as to who needs to manage the life of the buffer.

5. If you're dealing with string data, double check your string functions to make sure they terminate especially when they hit the end of the buffer. The C library is very inconsistent when it comes to handling string termination across the various functions.

Answer 6

If a function gives you a method of knowing how many characters it will return, use it. Your sample GetCurrentDirectory is a good example:

DWORD length = ::GetCurrentDirectory(0, NULL);

Then you can use a dynamically allocated array (either string or vector) to get the result:

std::vector<TCHAR> buffer(length, 0);
// assert(buffer.capacity() >= length);  // should always be true
GetCurrentDirectory(length, &buffer[0]);