Self-described “maker of issues” Christopher Moravec has turned the parable of individuals’s smartphones and voice assistants consistently listening in to their non-public conversions into actuality — so as to robotically generate topical art work.
“The WhisperFrame listens to conversations in our lounge after which generates artwork primarily based on these conversations,” Moravec writes of his undertaking. “[It] generates a brand new picture after each 5 minutes of energetic dialog. When there hasn’t been any speaking, it would revert to displaying randomly chosen photos generated previously.”
The core idea of the undertaking, which has an always-on microphone recording snippets of close by dialog, brings up the pervasive however always-unproven delusion of corporations utilizing smartphones and voice assistants to watch close by conversations for matters which might be data-mined and monetized. This time round, although, the very-real conversational recordings are being mined for thematic content material which might be fed to a generative synthetic intelligence (AI) system to create synthetic artwork.
The recordings are made in 15-20 second loops, then submitted to OpenAI’s Whisper software programming interface (API) for automated transcription into textual content. When 5 minutes has elapsed, these extracts are fed to OpenAI’s GPT-4 giant language mannequin (LLM) with the immediate to extract one key subject and switch it right into a immediate for an image-generating mannequin — which is, in flip, fed to Steady Diffusion, the ensuing image downloaded, and the show up to date.
The imagery generated by Steady Diffusion is keyed to a single subject, drawn from the final 5 minutes by GPT-4. (📷: Christopher Moravec)
“It’s a bit self-fulfilling in that as individuals discuss concerning the picture it drew, it turns into extra seemingly that it tries for example that one once more, as the subject is extra prone to be chosen by GPT-4,” Moravec admits. “Nevertheless it’s nonetheless superior! I even created a second one for my workplace that generates photos throughout conferences! It would even be a brand new method to make assembly notes, a listing of photos representing the assembly as an alternative of motion objects. It in all probability received’t catch on, although!”
The complete write-up is on the market on Moravec’s web site; all generated photos can be found to browse on a devoted web site.