BMW E39 Navigation Retrofit Project

Table of Contents

• Main
• Software Design
• Arduino Power Board
• Video Switch + Backup Camera Install
• Rpi Construction
• Bill of Materials
• References

Software Design

Software Overview (larger)

Overview

https://github.com/linster/e39-rpi

The app is a plain-old Kotlin/JVM application. It consists of a serial port reader and writer which run on their own thread, and a handful of “Services” which all share a coroutine.

Pictured above is the interaction with DBus, BlueZ, and PulseAudio, which allows the RPi to listen to steering wheel control IBus messages and change the track on the phone, as well as stream audio to the RPi.

E38/E39 IBus Network Design

IBUS Topology (larger)

The E39 uses a LIN-based bus called IBus (Infotainment Bus) to connect the radio, amplifier equalizer control, navigation computer, head unit, and steering wheel buttons.

Messages from any peripheral are labelled with a Source Device, a Destination Device, and contain a byte array of data. This design makes it easy to listen to all traffic on the bus and intercept key press events, as well as spoof devices on the bus.

The documentation on these message formats has been well-known for a while. I consulted this resource most frequently.

Printing the track info on the screen

To print the track info on the screen, I injected “title” messages onto the IBus. I also made provisions for the two different types of display drivers: the MK4 navigation system, and the TV Module display drivers.

Mk4 Title Layout

T0 is set to “CDC 3-“. T1 through T6 are shown in the top right. The items shown as P1..P6 are the “Index Fields”

TV Module Title Layout

The appropriate Display Driver is selected by setting a value in the global DeviceConfiguration object, since each type of menu system has different text length constraints.

The index areas allow for the construction of a menuing system. Input events for the selection of a particular index area are pretty easy to read.

Demo Video

Full video at Youtube

Serial Port Reader (Kotlin Flow + OKIO)

I’m really proud of this section of code that begins here:

JSerialCommsAdapter.kt, Line 62

override fun readMessages(): Flow<IBusMessage> {
    return rawSerialPackets.consumeAsFlow()
        .flowOn(flowDispatcher)
        .mapNotNull { it.toIbusMessage() }
        .onStart { setupJSerialComm() }
        .onCompletion { 
            readerJob?.cancel(
                Platform.PlatformShutdownCancellationException()
            ) 
        }
}

I needed to:

• Read a stream of bytes, and parse it as it comes in into a flow of byte arrays that are the correct length for the message described
• Parse the flow of byte arrays into an IBusMessage object

Protocol

The tricky part with this protocol is that it is a variable-length serial packet with no marker byte(s) to indicate the beginning or end of the packet.

This meant that when I started reading the bus, the serial buffer could begin with a partial packet.

The protocol is clever: it uses a 1-byte length field (Lange) and a XOR checksum.

Algorithm

The algorithm goes like this to split the incoming stream of bytes into packet-length ByteArrays:

• Pop the first byte off the queue. Keep it around for later.
• Pop the second byte off the queue. Assume this is the length byte.
• Read the next L bytes.
• Assemble a packet with the bytes from Step 1, Step 2, and the data read from Step 3
• Perform a CRC on the ByteArray from Step 4 (not including the CRC byte at the end). If the calculated CRC is the same as the expected CRC, keep the ByteArray and send it for further parsing. If not, throw it all out, and repeat the algorithm.

If you start reading in the middle of a packet, then the length field will be wrong, because it could just be any old data in the packet. However, the length field will tell you when to expect the CRC. With the CRC, then we’ll know whether the packet is valid or not. Eventually, the buffer will settle to reading packets from only their beginning.

There’s another trick the bus uses to reduce multiple reads starting from the middle of the packet: It uses time-based bus contention prevention to ensure there’s adequate time between any two sequential messages on the bus. This means that even if your length byte randomly happens to be huge, then eventually the serial buffer will not have enough data to construct the packet.

OKIO Buffers

To achieve the above algorithm, I used OKIO to provide a streaming byte buffer that supported single-byte reads. I emitted the packet-length ByteArrays to a Channel, which was then consumed in a flow and parsed to a Flow<IBusMessage>.

The code exists on GitHub here

private fun breakBufferIntoPackets(buffer: Buffer) : Flow<UByteArray> = flow {
        val debugBuffer = buffer.copy()
        while (!buffer.exhausted()) {
            if (buffer.size < 4) {
                //No source, len, dest, xor checksum
                return@flow
            }

            if (buffer.size < buffer.get(1) + 2) { //Need the +2 to get source + length bytes.
                //We haven't collected the amount of data this packet says it should have.
                //logger.v(TAG, "Buffer underrun -- waiting for more bytes. Expected ${buffer.get(1)} got ${buffer.size}")
                return@flow
            }

            val sourceDevice = buffer.readByte().toUByte()
            val packetLength = buffer.readByte().toUByte().toInt()
            val destDevice = buffer.readByte().toUByte()

            val data = if (packetLength <= 2) {
                ubyteArrayOf()
            } else {
                // subtract 2 because length includes checksum and dest address
                buffer.readByteArray(packetLength.toLong() - 2).toUByteArray()
            }
            val givenCrc = buffer.readByte().toUByte()

            var actualCrc = 0x00
            ubyteArrayOf(sourceDevice, packetLength.toUByte(), destDevice, *data).forEach { byte -> actualCrc = actualCrc xor byte.toInt() }

            val reAssembledPacket = ubyteArrayOf(sourceDevice, packetLength.toUByte(), destDevice, *data, givenCrc)

            if (false) {
                logger.v(
                    "BYTE READER",
                    "Read raw packet : " +
                            "[${sourceDevice.toDeviceIdString()}] " +
                            "[${packetLength.toString(10)}] " +
                            "[${destDevice.toDeviceIdString()}] " +
                            "<${data.size} bytes data> " +
                            "[CRC g/a : $givenCrc / $actualCrc ]"
                )
            }

            if (packetLength != data.size + 2) {
                logger.w("BYTE READER", "Data size mismatch. [e/a] ${packetLength - 2}/${data.size}")
            }
            if (givenCrc == actualCrc.toUByte()) {
                emit(reAssembledPacket)
            }
        }
    }