问题
I am writing some code to read bitmap files.
Here is the struct I am using to read the bitmap header. See also:
https://msdn.microsoft.com/en-us/library/windows/desktop/dd183374(v=vs.85).aspx
struct BITMAPFILEHEADER
{
WORD bfType; // 2
DWORD bfSize; // 6
WORD bfReserved1; // 8
WORD bfReserved2; // 10
DWORD bfOffBits; // 14
}; // should add to 14 bytes
If I put the following code in my main function:
std::cout << "BITMAPFILEHEADER: " << sizeof(BITMAPFILEHEADER) << std::endl;
the program prints:
BITMAPFILEHEADER: 16
It appears to be re-aligning the data in the struct on 4-byte boundaries, presumably for efficiency. Of course this renders me unable to read a bitmap... Even though microsoft, and others, specifiy this is the way to do it...
How can I prevent the structure re-alignment?
回答1:
The solution I found which works on gcc compilers, under linux:
struct BITMAPFILEHEADER
{
WORD bfType;
DWORD bfSize;
WORD bfReserved1;
WORD bfReserved2;
DWORD bfOffBits;
} __attribute__((packed));
There is probably a better, more cross compiler/platform way of doing things, but I don't know what it is.
回答2:
To avoid this, you can obviously designate the compiling granularity. Just use this switch:
##pragma pack(1)
This tells the compiler to align to 1-byte boundaries (do nothing)
To resume normal padding (from before the previous #pragma pack):
#pragma pack(pop)
回答3:
How can I prevent the structure re-alignment?
In C++, I believe you can not prevent the compiler from realigning "struct BITMAPFILEHEADER".
With g++-5 on my Ubuntu 15.10, the same padding happens in all of -O0, -O2, -O3, -Os. I have not yet found a compiler option to achieve this goal
Some of the answers in the reference further state that you can not control how optimizations may drive padding and alignment.
See also https://stackoverflow.com/a/19471823/2785528
See also Why does gcc generate 15-20% faster code if I optimize for size instead of speed?
Two other answers have suggested the use of pragmas. Any pragma I have tried has worked reliably, for that particular compiler, using the options chosen for that application. Pragmas are not standardized, and therefore (IMHO) are not a part of C++. Same with compiler extensions ... C++ developers should not use either.
Simply stated, C++ is fully capable of doing what is needed.
It appears to be re-aligning the data in the struct on 4-byte boundaries, presumably for efficiency.
I agree that the padding is most likely added for some 'compilers' (or compiler writer's) notion of 'efficiency'. And because the struct is clearly not more space-efficient, then perhaps the alignment allows the production of 'faster' code. We have all traded space for performance, right? This is generally a good idea, unless you need a 'packing' that the compiler does not provide.
Below I submit a possible implementation (with code) which provides absolute control of data size and padding, and, in this C++ design, (without pragma, without compiler extensions) produces an object of exactly 14 bytes.
I found one comment or answer that might be similar to the approach, but it was without code.
I call this approach 'Technique 1 (of 5)'.
See also Note 1 after the code.
Note 0 - I have all the following code in one file (all the parts are available below, but re-arranged.).
The problem: sizeof(struct BITMAPFILEHEADER) is reported as 16 bytes:
// sizeof() reports 16 bytes -- the compiler adds pad
struct BITMAPFILEHEADER
{ // from MS docs
WORD bfType; // 2 - The file type; must be BM (bit mapped?)
DWORD bfSize; // 4 - The size, in bytes, of the bitmap file.
WORD bfReserved1; // 2 - Reserved; must be zero.
WORD bfReserved2; // 2 - Reserved; must be zero.
DWORD bfOffBits; // 4 - The offset, in bytes, from the beginning of
}; // this (14 byte) structure to the bitmap bits
this renders me unable to read a bitmap
I think you are saying that the binary read from your source creates an invalid object because of the alignment issue (though you do not show your 'problem' code).
The following proposed implementation, based on an array of 14 bytes, provides absolute control of data size and padding, and produces an object of exactly 14 bytes.
// sizeof reports 14 bytes
class BITMAPFILEHEADER_t
{
private:
enum Constraints : size_t
{
// bf[indx] Size
Type = 0, TSz = 2, // 1 WORD
Size = 2, SSz = 4, // 2 WORD
Reserved1 = 6, RSz = 2, // 1 WORD
Reserved2 = 8, // 1 WORD
OffBits = 10, OSz = 4, // 2 WORD
//
BfSz = 14, EndConstraints
};
char bf[BfSz]; // 14 byte array
// reinterpret_cast a) explicitly allows conversion to char* from T*
// b) generates no code (compile time directive)
// vvv---short form verbose form---vvvvvvvvvvvvvvvv
char* ric (WORD* w) { return reinterpret_cast<char*>(w); }
char* ric (DWORD* dw) { return reinterpret_cast<char*>(dw); }
char* ric (BITMAPFILEHEADER_t* hdr14) { return reinterpret_cast<char*>(hdr14); }
char* ric (BITMAPFILEHEADER* hdr16) { return reinterpret_cast<char*>(hdr16); }
void memCpy (char* to, char* from, size_t count) {
for (uint i = 0; i < count; ++i) { to[i] = from[i]; }
}
// compute 'distance' between two addrs
uint64_t delta(char* highAddr, char* lowAddr) {
uint64_t dlta = (highAddr - lowAddr);
if(false) { std::cerr << "\n diag: delta is " << dlta << std::flush; }
return(dlta);
}
public:
BITMAPFILEHEADER_t() = delete;
BITMAPFILEHEADER_t(WORD utype, DWORD usize, DWORD uoff)
{
confirmFieldPlacement(); // check layout
setFields (utype, usize, uoff); // fill fields
if(false)
std::cout << "\n ctor 68 " << __PRETTY_FUNCTION__
<< std::flush;
if(false)
std::cout << "\n " << dumpFieldHex ("dumpFieldHex 91 \n");
if(false)
std::cout << " "
<< ::dumpByteHex (ric(this), BfSz, "dumpByteHex 94 \n", 6)
<< std::endl;
} // BITMAPFILEHEADER_t(w, dw, dw)
// convert
// to 14 byte from 16 byte
BITMAPFILEHEADER_t (BITMAPFILEHEADER& bmfh)
{
confirmFieldPlacement(); // ctor must check
setFields(bmfh.bfType, bmfh.bfSize, bmfh.bfOffBits);
}
~BITMAPFILEHEADER_t() = default; // all POD
// convert
// to 16 byte from 14 byte
BITMAPFILEHEADER toBMFH()
{
BITMAPFILEHEADER bmfh;
// to from count
memCpy (ric(&bmfh.bfType), &bf[Type], TSz);
memCpy (ric(&bmfh.bfSize), &bf[Size], SSz);
memCpy (ric(&bmfh.bfReserved1), &bf[Reserved1], RSz);
memCpy (ric(&bmfh.bfReserved2), &bf[Reserved2], RSz);
memcpy (ric(&bmfh.bfOffBits), &bf[OffBits], OSz);
return bmfh;
}
void setFields (WORD utype,
DWORD usize,
DWORD uoff)
{
// to from count
memCpy (&bf[Type], ric(&utype), TSz); // bfType
memCpy (&bf[Size], ric(&usize), SSz); // bfSize
DWORD zero = 0; // reserved values must be 0
memCpy (&bf[Reserved1], ric(&zero), RSz); // bfReserved1
memCpy (&bf[Reserved2], ric(&zero), RSz); // bfReserved2
memcpy (&bf[OffBits] , ric(&uoff) , OSz); // bfOffBits
} // void BITMAPFILEHEADER_t::setFields()
// reads 1x 14 bytes into bf[]
void read_14_Bytes (std::stringstream& ss14)
{
(void) ss14.read (&bf[0], 14);
// ^^^^ binary
}
// writes 1x 14 bytes from bf[]
void write_14_Bytes (std::ostream& ss14)
{
(void) ss14.write (&bf[0], 14);
// ^^^^^ binary
}
std::string dumpFieldHex(std::string lbl)
{
std::stringstream ss;
ss << lbl << std::flush << std::hex
// from len
<< ::dumpByteHex( &bf[Type], TSz, " bfType : ") // 1 WORD
<< ::dumpByteHex( &bf[Size], SSz, " bfSize : ") // 1 DWORD
<< ::dumpByteHex( &bf[Reserved1], RSz, " bfReserved1 : ") // 1 DWORD
<< ::dumpByteHex( &bf[Reserved2], RSz, " bfReserved2 : ") // 1 DWORD
<< ::dumpByteHex( &bf[OffBits], OSz, " bfOffBits : ") // 1 DWORD
<< std::dec << std::flush;
return(ss.str());
} // std::string BITMAPFILEHEADER_t::dumpFieldHex(lbl)
std::string dumpByteHex (std::string label)
{
return
::dumpByteHex (ric(this), // char* startAddr,
BfSz, // size_t len,
label, // std::string label,
0); // int indent);
} // std::string BITMAPFILEHEADER_t::dumpByteHex(lbl)
private:
void confirmFieldPlacement()
{ // distance ( to field, from start addr )
assert( delta ( &bf[Type], ric(this) ) == Type); // 0, len 2
assert( delta ( &bf[Size], ric(this) ) == Size); // 2, len 4
assert( delta ( &bf[Reserved1], ric(this) ) == Reserved1); // 6, len 2
assert( delta ( &bf[Reserved2], ric(this) ) == Reserved2); // 8, len 2
assert( delta ( &bf[OffBits], ric(this) ) == OffBits); // 10, len 4
// expected magic numbers ---------------------^^^^-----------^^
assert( sizeof(BITMAPFILEHEADER_t) == BfSz);
// sizeof whole instance 14 bytes
} // void BITMAPFILEHEADER_t::confirmFieldPlacement()
}; // class BITMAPFILEHEADER_t
The only data in BITMAPFILEHEADER_t is bf. Exactly 14 bytes. No pads.
For unit test, I typically minimize 'main':
int main(int , char** )
{
T528_t t528; // unit test 528
Time_t start_us = HRClk_t::now();
int retVal = t528.exec(); // run unit tests
auto duration_us =
std::chrono::duration_cast<US_t>(HRClk_t::now() - start_us);
std::cout << "\n T528_t::exec() duration "
<< duration_us.count() << " us" << std::endl;
return(retVal);
}
For your review, the unit test code "t528.exec()" provide:
a) a demo using "void read_14_Byte_records (std::stringstream& ss14)" and "void write_14_Byte_records (std::ostream& ss14)"
b) two conversions, between struct (16 bytes) and class (14 bytes). See also "T528::conversion_therapy()".
c) a demo that illustrates standard techniques to make visible the 'hidden' compiler inserted pad.
d) a demo to make visible that the class has no padding.
e) a C++ implementation of dumpByteHex().
class T528_t
{
private:
// reinterpret_cast a) explicitly allows conversion to char* from T*
// b) generates no code (compile time directive)
// vvv---short form verbose form---vvvvvvvvvvvvvvvv
char* ric(BITMAPFILEHEADER_t* hdr14) { return(reinterpret_cast<char*>(hdr14)); }
char* ric(BITMAPFILEHEADER* hdr16) { return(reinterpret_cast<char*>(hdr16)); }
public:
T528_t()
{
std::cout << "\n " << __PRETTY_FUNCTION__ << " ctor ";
std::cout << "\n sizeof (struct BITMAPFILEHEADER ) "
<< sizeof(BITMAPFILEHEADER); // reports 16 bytes
std::cout << "\n sizeof (class BITMAPFILEHEADER_t) "
<< sizeof(BITMAPFILEHEADER_t) // reports 14 bytes
<< std::endl;
}
~T528_t() = default;
int exec()
{
std::cout << "\n " << __PRETTY_FUNCTION__ << std::endl;
{
std::stringstream ss14; // for testing binary i/o
write_14_Byte_records (ss14); // push_back some number of
// 14 byte records into ss14
read_14_Byte_records(ss14); // read/show all 14 byte records in ss14
}
struct_pad_demo(); // using struct
class_no_pad_demo(); // using class
conversion_therapy();
return(0);
} // int exec()
private: // methods
void write_14_Byte_records (std::stringstream& ss14)
{
std::cout << "\n " << __PRETTY_FUNCTION__
<< " START =" << std::flush;
{
BITMAPFILEHEADER_t hdr14 (0x1122, 0x33445566, 0x778899AA);
hdr14.write_14_Bytes (ss14);
if(!ss14.good()) {
std::cerr << " Err: hdr14.write_14_Bytes(ss14) (!ss14.good())"
<< std::endl;
assert(0); // TBR
}
}
std::cout << "=" << std::flush; // one per record
WORD w = 0x1010;
DWORD dw = 0x10101010;
for (int i=0; i<10; ++i)
{
w = static_cast<WORD> ( w + 0x0101);
dw = static_cast<DWORD>(dw + 0x01010101);
{
BITMAPFILEHEADER_t hdr14 ( w, dw, dw );
hdr14.write_14_Bytes (ss14);
if(!ss14.good()) {
std::cerr << " Err: hdr14.write_14_Bytes(ss14) (!ss14.good())"
<< std::endl;
assert(0); // TBR
}
}
// one per record:
std::cout << ((1 == (i % 2)) ? '=' : '.') << std::flush;
}
{
BITMAPFILEHEADER_t hdr14 (0x2211, 0x66554433, 0xAA998877);
hdr14.write_14_Bytes (ss14);
if(!ss14.good()) {
std::cerr << " hdr14.write_14_Bytes(ss14) (!ss14.good())"
<< std::endl;
assert(0); // TBR
}
}
std::cout << "=" << std::flush; // one per record
std::cout << "\n " << __PRETTY_FUNCTION__
<< " END =============\n" << std::endl;
} // void T528::write_14_Byte_records (std::stringstream&)
void read_14_Byte_records (std::stringstream& ss14)
{
std::cout << " " << __PRETTY_FUNCTION__;
for (int i = 0; true; ++i) // infinite loop, break out at ss14.eof()
{
// construct a work buffer to receive stream data
BITMAPFILEHEADER_t workBuff(0, 0, 0);
if(false) { // diag only, buff contents after ctor, before read_14_Bytes()
std::cout << "\n " << workBuff.dumpFieldHex("pre-read:\n"); // err 8,9?
std::cout << " " << workBuff.dumpByteHex ("pre-read:\n ");
}
// binary read of 14 byte data into workBuff
workBuff.read_14_Bytes (ss14);
if(ss14.eof()) { // no more data!
std::cout << "\n\n ss14.eof() after " << std::dec << i
<< " records." << std::endl;
break;
}
assert(ss14.good()); // tbr
std::cout << "\n\n post-read hex-dump of record [" << std::dec
<< i << "] (from ss14)" << std::endl;
std::cout << " " << workBuff.dumpFieldHex("fields within class\n");
std::cout << " " << workBuff.dumpByteHex ("class\n ");
} // for () ss14.eof()
// stringstream has no member named ‘close’.
} // void T528::read_14_Byte_records (std::stringstream&)
void struct_pad_demo()
{
std::cout << "\n\n\n " << __PRETTY_FUNCTION__
<< " START =================================="
<< std::endl;
BITMAPFILEHEADER hdr16; // a struct
// default ctor does nothing, so data is uninitialized
{
size_t sz = sizeof(BITMAPFILEHEADER); // 16
// show uninitialized
std::cout << "\n "
<< dumpByteHex ( ric(&hdr16), sz, " uninitialized struct hdr16"
" may have 'noise'\n ")
<< std::endl;
// because all elements are POD,
// we can use std::memset to fill hdr16 with 0xff
// to val count
std::memset(&hdr16, 0xff, sz);
std::cout << " "
<< dumpByteHex(ric(&hdr16), sz,
" struct after memset(0xff) \n ")
<< std::endl;
// zero the fields
hdr16.bfType = 0; // 2
hdr16.bfSize = 0; // 4
hdr16.bfReserved1 = 0; // 2
hdr16.bfReserved2 = 0; // 2
hdr16.bfOffBits = 0; // 4
// NO visible pad, so any pad is still 0xff
std::cout << " "
<< dumpByteHex( ric(&hdr16), sz,
" struct fields zero'd (no field named"
" 'pad', so any such stay 0xff ) \n ")
<< " ^^ ^^" << std::endl;
// fields to unique values
hdr16.bfType = 0x1122; // 2
hdr16.bfSize = 0x33445566; // 4
hdr16.bfReserved1 = 0xdead; // 2
hdr16.bfReserved2 = 0xbeef; // 2
hdr16.bfOffBits = 0x778899AA; // 4
std::cout << " "
<< dumpByteHex(ric(&hdr16), sz,
" struct fields unique, pad still 0xff"
"\n ")
<< " ^^ ^^" << std::endl;
std::cout << " " << __PRETTY_FUNCTION__
<< " END ==================================\n"
<< std::endl;
}
} // void T528::struct_pad_demo()
void class_no_pad_demo()
{
std::cout << "\n\n " << __PRETTY_FUNCTION__
<< " START =================================="
<< std::endl;
BITMAPFILEHEADER_t hdr14(0, 0, 0); // 14
// dummy field values--^--^--^ as fill
{
size_t sz = sizeof(BITMAPFILEHEADER_t); // 14
// show dummy initialized
std::cout << "\n "
<< dumpByteHex(ric(&hdr14), sz, " class hdr14 after ctor,"
" fields have default values \n ")
<< std::endl;
// because all elements are POD,
// we can use std::memset to fill hdr14 with 0xff
// to val count
std::memset (&hdr14, 0xff, sz); // works on class, also
std::cout << " "
<< dumpByteHex(ric(&hdr14), sz,
" class after memset(0xff) \n ")
<< std::endl;
// zero the private fields (no pad, and bfReserved zero'd)
hdr14.setFields(0,0,0); // bfType, bfSize, bfOffBits
// no pads, no 0xff shown
std::cout << " "
<< dumpByteHex(ric(&hdr14), sz,
" class fields zero'd (no 0xff shows there are no pads), 14 bytes\n ")
<< std::endl;
// fields to unique values
hdr14.setFields(0x2211, 0x66554433, 0xAA998877);
// bfType bfSize bfOffBits
std::cout << " "
<< dumpByteHex(ric(&hdr14), sz,
" class fields unique, reserved's are 0\n ")
<< std::endl;
std::cout << " " << __PRETTY_FUNCTION__
<< " END ==================================\n"
<< std::endl;
}
} // void T528::class_no_pad_demo()
void conversion_therapy()
{
std::cout << "\n\n " << __PRETTY_FUNCTION__
<< " START =================================="
<< std::endl;
BITMAPFILEHEADER hdr16; // a struct
std::memset(&hdr16, 0xff, 16); // fill with 0xff
BITMAPFILEHEADER_t hdr14 (hdr16); // conversion ctor
std::cout << "\n "
<< dumpByteHex(ric(&hdr14), 14,
" class fields filled from struct of 0xff, "
"reserved's are 0, no pad\n ")
<< std::endl;
// unique private fields (no pad, and bfReserved zero'd)
hdr14.setFields(0x1234, 0x56789abc, 0xdef0beef); // bfType, bfSize, bfOffBits
hdr16 = hdr14.toBMFH();
std::cout << " "
<< dumpByteHex(ric(&hdr16), 16,
" struct fields filled from unique fields class, "
"reserved's are 0, pad un-touched, may be garbage \n ")
<< " ^^ ^^---may be garbage, but not accessible in struct \n" << std::endl;
std::cout << " " << __PRETTY_FUNCTION__
<< " END ==================================\n"
<< std::endl;
} // void T528::conversion_therapy()
}; // class T528_t
refactor guide: the includes and the C++ utility function dumpByteHex(), along with sequence hints for stitching this code back in order
#include <chrono>
// 'reduced' chrono footprint --------------vvvvvvv
typedef std::chrono::high_resolution_clock HRClk_t; // std-chrono-hi-res-clk
typedef HRClk_t::time_point Time_t;
typedef std::chrono::milliseconds MS_t; // std-chrono-milliseconds
typedef std::chrono::microseconds US_t; // std-chrono-microseconds
typedef std::chrono::nanoseconds NS_t; // std-chrono-nanoseconds
using namespace std::chrono_literals; // suffixes include 100ms, 2s, 30us
#include <iostream> // std::cout, std::cerr
#include <iomanip> // setw(), setfill()
#include <sstream>
#include <string>
#include <cstring> // memset
#include <cassert> // assert
// forward
std::string dumpByteHex (char* startAddr,
size_t len,
std::string label = "",
int indent = 0);
// Linux substitutes
typedef uint16_t WORD;
typedef uint32_t DWORD;
// reports 16 bytes -- compiler adds unlabled pad
struct BITMAPFILEHEADER ....
// sizeof() reports 14 bytes
class BITMAPFILEHEADER_t ....
//int main(int argc, char* argv[])
int main(int , char** ) ....
// C++ support function
std::string dumpByteHex (char* startAddr, // reinterpret_cast explicitly
size_t len, // allows to char* from T*
std::string label, // = "",
int indent) // = 0
{
std::stringstream ss;
if(len == 0) {
std::cerr << "\n dumpByteHex() err: data length is 0? " << std::endl << std::dec;
assert(len != 0);
}
// Output description
ss << label << std::flush;
unsigned char* kar = reinterpret_cast<unsigned char*>(startAddr); // signed to unsigned
// unsigned char* kar = static_cast<unsigned char*>(startAddr); // invalid cast
std::string echo; // holds input chars until eoln
size_t indx;
size_t wSpaceAdded = false;
for (indx = 0; indx < len; indx++)
{
if((indx % 16) == 0)
{
if(indx != 0) // echo is empty the first time through for loop
{
ss << " " << echo << std::endl;
echo.erase();
}
// fields are typically < 8 bytes, so skip when small
if(len > 7) {
if (indent) { ss << std::setw(indent) << " "; }
ss << std::setfill('0') << std::setw(4) << std::hex
<< indx << " " << std::flush;
} // normally show index
}
// hex code
ss << " " << std::setfill('0') << std::setw(2) << std::hex
<< static_cast<int>(kar[indx]) << std::flush;
if((indx % 16) == 7) { ss << " "; wSpaceAdded = true; } // white space for readability
// defer the echo-of-input, capture to echo
if (std::isprint(kar[indx])) { echo += kar[indx]; }
else { echo += '.'; }
}
// finish last line when < 17 characters
if (((indx % 16) != 0) && wSpaceAdded) { ss << " "; indx++; } // when white space added
while ((indx % 16) != 0) { ss << " "; indx++; } // finish line
// the last echo
ss << " " << echo << '\n';
return ss.str();
} // void dumpByteHex()
Note 1:
My code shows a typical C++ technique that provides absolute control of data size. "Technique 1" above is one of 5.
The ideas were used during the upgrade of the SCU (shelf controller unit), which was overwhelmed by marketing success. The SCU could no longer handle (insufficient cpu cycles / second) a full-shelf of the newest feature cards. Too many status per second, too much PM (performance monitoring) per second, too many alarms per second, etc.
The default solution for these maladies is more processing power and more memory. Thus ... we ported the SCU app: from C to C++ (a new compiler), from 1 MIP (cisc) to 15 mip (risc) (new processor), from 1 MBytes to 16 MBytes, and a new OS (vxWorks) and added ethernet, and SNMP and telnet control and access.
The primary requirement is backward compatibility. The installed base of existing feature cards was big. The new SCU had to work with any age feature card, anywhere installed.
Technique 1 (of 5) is both the easiest to understand and suitable for small field count data. It can be tedious with larger messages. The biggest field count of any message was probably close to 100. This large number of fields inspired at least 2 of the other 4 techniques.
Note 2:
Clearly, automation support is possible for applying Technique 1.
Note 3:
IMHO, I think there is a rationale for the use of pragmas or compiler extensions ... these short cuts can generate code to support the 'bigger scheme of things'. i.e. allows integration to continue, support development of non-message issues, etc.
来源：https://stackoverflow.com/questions/44620755/c-the-compiler-is-changing-the-alignment-of-my-structures-how-can-i-prevent-t