This blog will detail the physical screen used as the face for our lamp. The screen used is from a company called Nextion. Nextion includes a hardware part (the display) and a software part (the Nextion editor).
The face team went with Nextion as they had the largest display that we could find that would fit into the lamp. It also came with an editor so getting a hello world would be much faster than trying to attempt it from scratch with code.
The Hardware consists of the Nextion NX8048T050_011 which is a 5'' touchscreen with 800x480 pixals. The interface consists of only four wires: Power, Ground, Tx & Rx. Since we are not making use of the touchscreen aspect of the display, the Tx wire can be removed and only three wire are then needed interface with the screen.
The Nextion Display works by first designing a GUI on the Nextion editor. This GUI is then compiled and loaded onto a SD card. The SD is then loaded in the display and is turned on. The display detects the SD card in it and loads up the compiled GUI. Once loaded the power is removed and then the SD card. When powered back up the GUI designed on the editor is now present on the display.
Nextion editor has a load of components such as sliders, gauges, buttons etc. as it is designed for Human interface and not as a simple visual display. The idea is that it is used on the likes of vending machines, info display, control panels on machines etc. or as a status display on a fish tank shown below.
The way we use it is as an LCD screen, with different images such as a happy face or ball which represents an eye. From that we would move the ball around the screen depending on the X & Y values we receive from the vision team. Also depending on how far away a person is from the lamp a different face or colour ball will be displayed to represent emotions. Too close for example and an angry face or red ball will be displayed.
We encountered a few teething problems when we first used the Nextion editor. The first of which is every image we use has to be loaded on the memory on the display board it's self. This normally wouldn't be a problem, but with the display only having 16MB of on board memory, we quickly ran into memory problems.
From the example on the editor, the way they changed pictures was to call a different image into an image location. Think of it as having a framed picture but constantly replacing the picture in the frame.
We looked at having a picture frame the size of the screen and having images that were the size of the screen, but the the actual face or ball on it would take up a fraction of the image with a lot of white space making up the rest of the image. Then several almost identical images would be created with the face or ball moved a bit to the left or right. Then using the X & Y coordinates from the vision team, different images would be called and it would look like the face is looking left or right. The problem with this however, is that with only 16MB of memory only a max of around 20 images could be stored on the display. With several different emotions needed and several images per emotion, the amount of images grew exponentially and this was not feasible.
The next solution we tried was having many different picture frames and call the one image to a specific picture location. We found that picture locations take up much less memory than the pictures themselves. So by calling an image into a specific frame the X & Y movement could be achieved. The problem with this solution is that every single picture location had to be manually placed and named uniquely, and with 800X480 pixels, that's a lot of picture locations and unique names and this was not feasible.
We finally had a breakthrough when we discovered that we could call picture frames at specific locations. This allowed us to have one picture frame and one image to cover every possible X Y location, which is a massive improvement on the previous two solutions. This call can then be made through serial.
The serial comms involve sending a string to the display.
The string that is sent is
pic : meaning picture frame,
x : meaning X coordinate,
y : meaning Y coordinate,
id : meaning picture id e.g different images.
The final and most inconvenient issue we found is that the refresh rate on the display is very slow. The fastest we could send messages to the display was around one message every 200ms. Any faster than this and the display would not be able to keep up and there would be a backlog of images. If serial then stopped the display would continue to update until the backlog was caught up. Because of this we had to limit how fast our code sent messages to the display.
Any other commands can be found on the wiki below: