Seeed Studio has introduced the launch of the Native Voice Chatbot, an NVIDIA Riva- and LLaMa-2-based giant language mannequin (LLM) chatbot with voice recognition capabilities — operating totally regionally on NVIDIA Jetson gadgets, together with the corporate’s personal reComputer vary.
“In a world the place synthetic intelligence is evolving at an creative tempo, the mode of human-computer interplay has taken a revolutionary flip in direction of voice interplay. This shift is especially evident in sensible houses, private assistants, and customer support help, the place the demand for seamless and responsive voice chatbots is on the rise,” claims Seeed Studio’s Kunzang Cheki.
“Nonetheless, the reliance on cloud-based options has caused issues associated to knowledge privateness and community latency. In response to those challenges, we current an revolutionary Native Voice Chatbot mission that operates regionally, addressing privateness points and making certain swift responses.”
The Seeed Native Voice Chatbot builds atop two present initiatives: NVIDIA’s Riva, a hardware-accelerated computerized speech recognition (ASR) and speech synthesis engine, and Meta AI’s LLaMa-2 giant language mannequin (LLM). The concept is easy: speech is picked up by a microphone and transformed to textual content by Riva’s ASR; the textual content is fed to LLaMa-2, which generates a believable text-based response; and the response is then fed by way of the Riva text-to-speech engine to render it audible.
“Conventional voice chatbots closely depend upon cloud computing providers, elevating legitimate issues about knowledge privateness and community latency. Our mission focuses on deploying a voice chatbot that operates totally {hardware}, mitigating privateness issues and providing a sooner response time,” Cheki claims. “The general structure ensures a safe, non-public and fast-responding voice interplay system with out counting on cloud providers, addressing knowledge privateness and community latency issues.”
The LLM runs regionally on-device, which means no price limits or subscriptions required. (📷: Seeed Studio)
Working every little thing regionally does come at a price, in fact: whereas the software program itself is suitable with any mannequin of NVIDIA Jetson, the memory-hungry LLM will not work correctly on something with lower than 16GB of RAM — which means the pocket-friendly Jetson Nano vary is shut out of the mission. “I accomplished all experiments utilizing [a] Jetson AGX Orin 32GB H01 Equipment,” Cheki notes.
The mission is documented in full on the Seeed Studio wiki.