Sending "larger than 8-bit" quantities over an 8-bit link

Question

I have data that I logically manipulate in 12-bit quantities which I need to move over an UART link. (actually soft-serial from Mbed to PC over USB cable, but that's irrelevant)

I was thinking about using the common 8N1 (8 data bits, no parity, 1 stop bit) UART setup, which just leaves me with the question of how to "chop up" my 12-bit values in to two 8-bit "packets".

I could of course implement something trivial myself like sending the 8 lowest bits first and then sending the 4 highest bits in the second packet, with the other 4 bits of the second packet not used. Or I could chop it up in to two 6-bit pieces, with the other 2 bits of each packet either unused or designated for meta-information (like whether the current packet is the first or second half of a 12-bit quantity).

Before doing that I wanted to see if there was any standard or a "standard industry practice" way of chopping up data larger then 8 bits in to 8 bit quantities?

Link speed should is not a issue since my requirements are about 3 kbps.

(In the absence of a "low level communications protocol"-Stack-Exchange I feel this is the most suitable platform for this type of question, per various discussions on Meta.)

EDIT: All answers were valid and good. I marked Olin's answer as correct since it's the way I went with my project.

Answer 1

No, there is no standard or even generally accepted common practise.

If you are really sure all you will ever send over this link is a stream of 12 bit values, then you can put 6 bits in each byte and use the remaining 2 bits to identify whether this byte contains the high or low 6 bits of the 12 bit word. That leaves two more codes for a little expansion.

However, that's not what I'd, and certainly not what I usually do. No matter how much you think to the contrary now, chances are good that sooner or later you'll wish you could send something else over the link, even if just for debugging. Since you say bandwidth is not a problem, do it right now so that expanding the protocol will be easy.

What I usually do is send data in packets that start with a opcode byte, which is then followed by whatever data goes along with that opcode. I have done this many times. For clarity of documentation, I call packets from the host to the micro commands, and from the micro to the host responses, but that doesn't mean responses must be only a result of commands.

This scheme is easy to implement in firmware. It's easy on the host too, but that doesn't matter. With the infinite processing power and memory of a PC, any protocol you cook up that is tractable on the micro will be easy on the host. The point is to keep it simple on the small resource-limited system and have the other system adapts as necessary.

On receiving, I use a jump table for the opcode byte. That dispatches to the routine unique for that command, which knows how many more input stream bytes to read and what to do with them. When done, command routines jump back to a common return point, which fetches the next opcode, dispatches thru the table, which runs the next command routine, etc.

Once you have this mechanism set up and a test program on the host to read and write this protocol, it is really simple to add another command or response. Being so simple, you'll start to use this facility in ways you might not have envisioned at the start. For example, you can make a command that sets a debug bit in the firmware which causes other code to do something special. Or you realize there are some options you would like to control, maybe read back data memory, a special command that does something one-off for testing, etc, etc. This sort of stuff comes in really handy a lot.

To solve the particular case you asked about, you'd have a response that is followed by 2 bytes which contain the 12 bit number. In the micro, that's as simple as sending the response opcode byte followed by the two data bytes periodically. Meanwhile it leaves open the option of sending other things without messing up the stream of 12 bit values.

Answer 2

If the only thing you are every going to send over the link is your 12-bit value, it may be helpful to send it as two bytes which are chosen in such a way that code which receives two consecutive bytes will be able to identify them as being part of the same reading, or parts of two consecutive readings. A common way to do this is to use one bit to identify whether a given byte represents the start of a data item or a continuation thereof. For example, you could specify that a reading will consist of a byte whose top 3 bits are 100 and whose bottom five bits represent the top 5 bits of a 12-bit reading, followed by a byte whose top bit is zero and whose bottom 7 bits represent the bottom 7 bits of a 12-bit reading. Note that this approach can work well if there are multiple sensors. For example, one could specify that a top-3-bits value of 100, 101, or 110 will represent the start of one of three 12-bit sensors' 12-bit readings, while leaving room for further expansion.

Answer 3

When the total information to be transferred is less than an integral number of bytes, the idea of reserving one bit for framing is indeed an attractive one, which I've used in the past.

There are of course other framing schemes in use, which can make sense when more data is to be moved. ASCII representation with a terminating character is also worth considering, since in addition to solving the framing problem it gives you human readable data, though at a 2x overhead.

Since you mention USB though, it may be worth considering if a USB-serial transport can give you implicit framing of a sort not present on a true serial link. Data in the actual USB packets will have position preserved, (ie, both ends can identified which is the first byte in a packet's payload) so the real question there is if particular host drivers accept/present data from/to applications with consistency that can be used for framing. Of course if the embedded end has a translator feeding an actual UART, framing information (beyond ambiguous idle detection) is lost at that point.

Answer 4

I love using c UNIONS for stuff like this. Handle the data as int16s and shift them across as int8s without issues.

Answer 5

If your focus is on finding a standard way to transmit data over a UART link there are plenty of protocols to do that. Modbus which works with 16-bit registers comes to mind. There will be some overhead associated with forming packets, but it's a standard that is widely used in industrial automation environments. If bandwidth is a primary concern then the methods described in other answers here would work well.