converting little endian to big endian

[sedj]

Hi,

I have a bunch of binary data stored in an Oracle database, which is running on an x86 little endian architecture.

The client machine is a big endian solaris box.

The data comes back (via Oracle Call Interface) as an array of Oracle's 'ub1' datatype - ie unsigned char (8 bit).

Some of the data is 8 bit data, some of it 32 bit (int) data.

Its a propriety format, so I know what the format of the data is. For example
In linux, I do something like :

ub1* ub1Data = GetData();
int i = (int)*ub1Data; // Grab 32 bytes off the data
ub1Data+=4; // shift the pointer on for the next data

Thats all well and good in a little endian architecture - but it gets a SIGSEGV in Solaris.


I think I need to convert the data from big endian to little endian - but I'm unsure of how to do it ...

Anybody got a clue of how to alter the endiness ?

Cheers for any help !

--------------------------------------------------
Free Java/J2EE Database Connection Pooling Software

http://www.primrose.org.uk

 

[cpjust]

I've never tried myself, but here's what I found on Google:

http://www.codeguru.com/forum/showthread.php?t=292902

 

[xwb]

The code you are using for conversion looks dodgy. Are you sure it works on all the data

// This bit is OK
ub1* ub1Data = GetData();

// You may get alignment problems here.  Probably the source of your crash.  Is ub1Data returning an address that is divisible by 4?  Check your stack trace
int i = (int)*ub1Data;

// This skips the next 4 ub1s.  If you want the next one it should be ub1Data++
ub1Data+=4; // shift the pointer on for the next data

 

[chipperMDW]

I'm not sure what sides of the transfer you actually have control over in your situation, but typically, when transferring data between machines, you convert it to and from Network Byte Order (NBO).

That is, the sending side converts data from its host byte order to NBO before transmitting, and the recieving side converts the data it gets back into its host byte-order before using it.

In Linux, you should have access to the htonl and ntohl functions, which respectively do host-to-network-order and network-to-host-order conversion on a long (32-bit) data type. My man page says they conform to SUSv3, so SOlaris probably has them, too.


However, on machines where the host byte-order is the same as NBO, those two functions are both no-ops. Since NBO is (I believe) big-endian, and your Solaris is big-endian, that conversion won't do anything. The Intel box should already be converting to NBO, so even if the Solaris box forgets to convert, the data should already be in the right order for it anyway... not that that's an excuse to "forget" to convert on a big-endian box.

At any rate, it sounds like either the Intel box is forgetting to do the conversion or your problem lies elsewhere (like pointer arithmetic, as xwb pointed out).

 

[sedj]

Thanks for the replies.

Due to oddities with the format of the data, htonl and friends is not possible, because any dereferencing of the ub1* data causes a seg fault (I believe its because of how solaris deals with data that is not byte aligned (which my data unfortunately is).

For that same reason, I cannot use the functions posted by cpjust (which I had found also), because dereferencing the first 32 bit (ub4 / unsigned int) structure causes a seg fault because it is not byte aligned.

The structure of the data in the ub1* is :

ub1 - a char marker
ub1 - another char marker
ub1 - another char marker
ub4 - an int value
ub1 - other data ....
As you can see the first ub4 is not byte aligned. Linux does not care about byte aligning.

Here is the solution I ended up using :

	m_cFormat = (char)*pRawBytes++;
	m_cType = (char)*pRawBytes++;
	m_cDivision = (char)*pRawBytes++;

#ifdef SOLARIS_PLATFORM
	ub4_endian length = AppUtils::ub4EndianSwap(pRawBytes);
	m_iTotalLength = (int)length.dword;
#else
	m_iTotalLength = (int)(*(ub4*)(pRawBytes));
#endif

where the endian swap function is :

typedef union _ub4
{
	ub1 DwordBits[sizeof(ub4)];
	ub4 dword;
} ub4_endian;



ub4_endian AppUtils::ub4EndianSwap(const ub1* pDataIn) {
	ub4_endian retData;

	retData.DwordBits[0] = pDataIn[3];
	retData.DwordBits[3] = pDataIn[0];

	retData.DwordBits[1] = pDataIn[2];
	retData.DwordBits[2] = pDataIn[1];

	return retData;
}

Thanks again for all the suggestions.

Cheers

--------------------------------------------------
Free Java/J2EE Database Connection Pooling Software

http://www.primrose.org.uk