Live networking experiments in hybrid cultural events

As an online visitor to cultural events, you can often do little more than ask a question in chat, and chat with other online visitors. How can online visitors feel seen, become more involved in or even have agency over what happens on-site? This is something we’re going to explore during this workshop.

H&D together with The Hmm developed a tool that makes use of standalone wifi hotspots (ESP32 modules) and live networking protocols (MQTT) that allows online input to be translated into something physical, and vice versa. For example, every time an online visitor opens the livestream page, a spotlight shines in the physical space. There are many use cases possible, but how can it help an event really be influenced by the presence of an online audience? How can it stimulate interaction between online and on-site audiences?

A few examples that give an idea of possible experimentation (each example has more detailed instructions on Github for the code, libraries, a wiring diagram and a breadboard view):

Led - A led is connected to the ESP32 module. When the online audience types a specific message in the chat the led will light up.
Neopixel - A more elaborate variation on the first example. A specifically crafted message on the live streaming platform can set the color of a led connected to the ESP32.
Relay - By connecting a relay (an electrically operated switch) we can turn on (and off) almost any device, for instance a wacky flailing inflatable tube man.
Servo - A servo is a motor that can be controlled to rotate from 0 to 180 degrees and anywhere in between. The message send from the live streaming platform controls the servo.
Button - When the button is pressed a message is sent to the live streaming server. If the server receives this message it will show a specific emote. This allows the physical audience to interact with the audience on the live streaming platform.

In November 2022 H&D and The Hmm organised a first version of the workshop Emoji Proxies & Ghost Messengers as part of the project Going Hybrid of Institute of Network Cultures, a second iteration took place during the symposium In-Between Media: Hybrid Tactics in The Crisis Era in March 2023.