IC Taiwan Grand Challenge Batch 2 Winner Introduction – TurboNext.ai

The computational demands of Large Language Models (LLMs) and other AI workloads are escalating rapidly, which will raise both the total operational cost and hardware requirements to make sure the model can function properly. This will force users to consider their LLM strategy to optimize operational cost and prevent unnecessary downtime. TurboNext.ai’s Heterogeneous Compute Platform enables efficient execution across diverse GPUs, while its silicon features an LLM accelerator and terabytes of local memory for faster AI processing.

Current conventional AI inference hardware focuses on GPUs, which can cost as high as USD 300,000 per rack, and a model with high user traffic might need more than 1 rack, which can be prohibitively expensive, especially for startups. AI-focused enterprises often design their LLM infrastructure for peak workloads, leading to needless purchases of resources and an inability to adapt when the user traffic changes. TurboNext.ai’s Heterogeneous Compute system allows users to optimize their existing hardware resources with workload-specific allocation and performance adjustments.

TurboNext.ai has also developed their NPU chip which offers high throughput for transformer architecture LLM models, scalability for tensor & intra-model parallelism, large memory for LLM requirement using DRAM, and balances compute and memory bandwidth optimized for LLM inference. The platform effectively combines software, NPU, and system; utilizing the existing GPUs as co-processors to offer the highest LLM performance at scale and the world’s most efficient token generation rate.