Adetunji Dahunsi

TJ Dahunsi

Jul 08 2019 · 7 min read

We’re spoiled for choice these days when it comes to choosing a Home Automation system. We could go voice controlled with Alexa or Google Assistant, or choose from well established ecosystems like that from Phillip’s Hue. All these systems have one thing in common, proprietary hubs that act as the center of our smart homes. Some more extensible than others, however it’d be nice if we could build our own hubs independent of these proprietary systems and administer them as we wish.

Android is an extremely versatile operating system with APIs for a smattering range of technologies used in IOT from Bluetooth to WiFi, along with USB host mode to interface with other peripherals quite conveniently, making it a great candidate to build a Home Automation hub with. If that weren’t enough, Android Things is even less stringent than Android on phones and tablets as it eschews permission prompts for Bluetooth and USB, implicitly granting them on request on the device it runs on. So, let’s start cooking.

Parts & Supplies

1. Raspberry Pi 3, please do not get the B+, only the B is supported for Android Things ($35).

2. SD Card with Android Things image, at least 16 GB ($5)

3. Rf Transmitter and Receiver for RF modules ($7)

4. RF wall plugs from Etekcity (3 pack $24)

5. Texas Instruments CC2531 ZigBee Dongle ($14)

6. CC Debugger to flash CC2531 ($15) the official one from Texas Instruments costs $50, but you can find generic ones like that linked above for less. If you don’t mind springing more, I’d advise to get the official debugger.

7. Gledopto multicolor ZigBee Bulb ($25). Hue bulbs should work too, but a Hue bulb is almost twice the price!

Optional:

1. Touch Screen for Raspberry Pi ($65)

2. Case for Raspberry Pi and touch screen ($26)

Server-Client Liaisons

First, there needs to be a way for client devices (Android phones owned by you or your guests) to send instructions to the hub. Since this is a home automation system, let’s keep things local; Android supports Network Service Discovery or NSD, which is a fantastic way for realtime communications with devices on the same local network. It’s based on DNS-SD which is how wireless printing works on a LAN. Essentially each client will open a socket on the hub, and write instructions to it directly, and receive responses on that same socket allowing for duplex communication.

1private class ServerThread internal constructor(private val serverSocket: ServerSocket) : Thread(), Closeable {
2
3        @Volatile
4        internal var isRunning: Boolean = false
5        private val portMap = ConcurrentHashMap<Int, Connection>()
6
7        private val protocol = ProxyProtocol(ConsoleWriter(this::broadcastToClients))
8        private val pool = Executors.newFixedThreadPool(5)
9
10        override fun run() {
11            isRunning = true
12
13            Log.d(TAG, "ServerSocket Created, awaiting connections.")
14
15            while (isRunning) {
16                try {
17                    Connection( // Create new connection for every new client
18                            serverSocket.accept(), // Block this ServerThread till a socket connects
19                            this::onClientWrite,
20                            this::broadcastToClients,
21                            this::onConnectionOpened,
22                            this::onConnectionClosed)
23                            .apply { pool.submit(this) }
24                } catch (e: Exception) {
25                    Log.e(TAG, "Error creating client connection: ", e)
26                }
27            }
28
29            Log.d(TAG, "ServerSocket Dead.")
30        }
31
32        private fun onConnectionOpened(port: Int, connection: Connection) {
33            portMap[port] = connection
34            Log.d(TAG, "Client connected. Number of clients: ${portMap.size}")
35        }
36
37        private fun onConnectionClosed(port: Int) {
38            portMap.remove(port)
39            Log.d(TAG, "Client left. Number of clients: ${portMap.size}")
40        }
41
42        private fun onClientWrite(input: String?): String {
43            Log.d(TAG, "Read from client stream: $input")
44            return protocol.processInput(input).serialize()
45        }
46
47        @Synchronized
48        private fun broadcastToClients(output: String) {
49            Log.d(TAG, "Writing to all connections: ${JSONObject(output).toString(4)}")
50            pool.execute { portMap.values.forEach { it.outWriter.println(output) } }
51        }
52
53        override fun close() {
54            isRunning = false
55
56            for (key in portMap.keys) catcher(TAG, "Closing server connection with id $key") { portMap[key]?.close() }
57
58            portMap.clear()
59            catcher(TAG, "Closing server socket.") { serverSocket.close() }
60            catcher(TAG, "Shutting down execution pool.") { pool.shutdown() }
61        }
62    }

ServerThread managing concurrent communication with multiple clients

Note that the SocketThread is blocked waiting for a client to connect. This however doesn’t stop clients from writing back to the server as the other methods are called on their own separate threads.

1/**
2     * Connection between [ServerNsdService] and it's clients
3     */
4    private class Connection internal constructor(
5            private val socket: Socket,
6            private val inputProcessor: (input: String?) -> String,
7            private val outputProcessor: (output: String) -> Unit,
8            private val onOpen: (port: Int, connection: Connection) -> Unit,
9            private val onClose: (port: Int) -> Unit
10    ) : Runnable, Closeable {
11
12        val port: Int = socket.port
13        lateinit var outWriter: PrintWriter
14
15        override fun run() {
16            if (!socket.isConnected) return
17
18            onOpen.invoke(port, this)
19
20            try {
21                outWriter = createPrintWriter(socket)
22                val reader = createBufferedReader(socket)
23
24                // Initiate conversation with client
25                outputProcessor.invoke(inputProcessor.invoke(CommsProtocol.PING))
26
27                while (true) {
28                    val input = reader.readLine() ?: break
29                    val output = inputProcessor.invoke(input)
30                    outputProcessor.invoke(output)
31
32                    if (output == "Bye.") break
33                }
34            } catch (e: IOException) {
35                e.printStackTrace()
36            } finally {
37                try {
38                    close()
39                } catch (e: IOException) {
40                    e.printStackTrace()
41                }
42            }
43        }
44
45        @Throws(IOException::class)
46        override fun close() {
47            onClose.invoke(port)
48            socket.close()
49        }
50    }

Connection class managing IO between the Server and its clients

Client side, the class responsible for managing the socket is the ClientThread.

1private class ClientThread internal constructor(
2            internal var service: NsdServiceInfo,
3            internal var clientNsdService: ClientNsdService) : Thread(), Closeable {
4
5        var currentSocket: Socket? = null
6        private var out: PrintWriter? = null
7        private val pool = Executors.newSingleThreadExecutor()
8
9        override fun run() = try {
10            Log.d(TAG, "Initializing client-side socket. Host: " + service.host + ", Port: " + service.port)
11
12            currentSocket = Socket(service.host, service.port)
13
14            out = createPrintWriter(currentSocket)
15            val `in` = createBufferedReader(currentSocket)
16
17            Log.d(TAG, "Connection-side socket initialized.")
18
19            clientNsdService.connectionState = ACTION_SOCKET_CONNECTED
20
21            if (!clientNsdService.messageQueue.isEmpty()) {
22                out?.println(clientNsdService.messageQueue.remove())
23            }
24
25            while (true) {
26                val fromServer = `in`.readLine() ?: break
27
28                Log.i(TAG, "Server: $fromServer")
29
30                val serverResponse = Intent()
31                serverResponse.action = ACTION_SERVER_RESPONSE
32                serverResponse.putExtra(DATA_SERVER_RESPONSE, fromServer)
33
34                Broadcaster.push(serverResponse)
35
36                if (fromServer == "Bye.") {
37                    clientNsdService.connectionState = ACTION_SOCKET_DISCONNECTED
38                    break
39                }
40            }
41        } catch (e: IOException) {
42            e.printStackTrace()
43            clientNsdService.connectionState = ACTION_SOCKET_DISCONNECTED
44        } finally {
45            close()
46        }
47
48        internal fun send(message: String) {
49            out?.let {
50                if (it.checkError()) close()
51                else pool.submit { it.println(message) }
52            }
53        }
54
55        override fun close() {
56            App.catcher(TAG, "Exiting message thread") { currentSocket?.close() }
57            clientNsdService.connectionState = ACTION_SOCKET_DISCONNECTED
58        }
59    }

Note again that the client uses a separate ExecutorService to send messages to the server, while the main thread itself manages the socket opened between it and the server and broadcasts any server responses using the Broadcaster class which is just a wrapper around an Rx PublishProcessor.

Both the ServerThread and ClientThread are hosted in instances of Android Service classes to keep them running in the background.

Since communication will be over a raw input and output streams, there needs to be a a method of serializing and deserializing each communication Payload. We can use a simple Kotlin data class of the same name to do this:

1data class Payload(val key: String) : Serializable {
2    var data: String? = null
3    var action: String? = null
4    var response: String? = null
5
6    val commands = LinkedHashSet<String>()
7
8    fun addCommand(command: String) = commands.add(command)
9}

Simple Payload class

Next we need an abstract way for our hub to communicate with whatever peripherals it may care about, both synchronously to respond directly to client requests, and asynchronously if a peripheral broadcasts an update. Synchronously, our previous Payload data class class works, a Payload comes in as input, is processed, and an output Payload goes out. For asynchronous commands, each instance of a CommsProtocol class has a PrintWriter it can directly write out of. It’s important that whatever is written out is a serialized representation of a Payload, else the client won’t be able to interpret it.

1abstract class CommsProtocol internal constructor(internal val printWriter: PrintWriter) : Closeable {
2
3    val appContext: Context = App.instance
4
5    abstract fun processInput(payload: Payload): Payload
6
7    fun processInput(input: String?): Payload = processInput(when (input) {
8        null, PING -> Payload(CommsProtocol::class.java.name).apply { action = PING }
9        RESET -> Payload(CommsProtocol::class.java.name).apply { action = RESET }
10        else -> input.deserialize(Payload::class)
11    })
12
13    protected fun getString(@StringRes id: Int): String = appContext.getString(id)
14
15    protected fun getString(@StringRes id: Int, vararg args: Any): String = appContext.getString(id, *args)
16
17    companion object {
18
19        const val PING = "Ping"
20        internal const val RESET = "Reset"
21
22        val sharedPool: ExecutorService = Executors.newFixedThreadPool(5)
23    }
24
25}

The Base class for responding to client request server side

Server-Peripheral Liaisons

Now that data can be sent to and received from the client, implementations of the CommsProtocol can be written. Simple RF wall sockets and ZigBee bulbs cover my basic home automation needs for now, and so that’s all I have implemented, however, the system is designed to be extensible to support whatever peripheral.

SerialRFPRotocol

Android Things unfortunately, is quite slow, making it impossible to interface directly with RF transceivers and essentially duplicate the packets sent out by cheap wireless switches. An Arduino however, is perfect for this, and it’s as easy as adding a USB serial dependency to use serial over USB to communicate with an Arduino; and that’s exactly what this protocol does. It sends commands to the Arduino to sniff switches, and when the packet sniffed, it replicates it letting you use your phone as a universal remote. The video below is the original implementation that uses BLE instead of a wired serial connection, which still works but is a bit over engineered and contrived. I’ve since moved to a direct wired connection.

ZigBeeProtocol

This protocol creates a ZigBee network that ZigBee peripherals can join. It again uses Serial over USB to communicate with Texas Instruments’ CC2531 dongle running their Z-STACK ZigBee 3.0 solution. All of the heavy lifting is done by ZSmartSystems’ ZigBee Cluster library. It essentially ports their CLI application to an Android GUI along with conveniences for switching light bulbs on and off. The real fun part was writing an implementations of their ZigBeePort for Android. Again it uses the excellent USB Serial for Android library for managing serial communications. A video of its operation is shown below.

So, that’s one way to build a home automation hub for a little under $100. Full source follows: