Skip to main content

SambaNova RDU

Vendor: SambaNova

Category: ASIC Dedicated Accelerator

Architecture: RDU (Reconfigurable Dataflow Unit)

Introduction​

SambaNova SN40 RDU (Reconfigurable Dataflow Unit) features a reconfigurable dataflow architecture supporting large model training and inference. The SN40 integrates 104 billion transistors and 64 reconfigurable nodes.

Specifications​

ModelComputeMemoryInterfaceTDPProcess
SN40L1,024 TFLOPS (FP8)64GB on-chip memoryFabric interconnect800W5nm
SN30330 TFLOPS (FP16)32GB on-chip memoryFabric interconnect400W7nm

Official Website​

Visit Official Website

Driver Downloads​

Linux​

OS Support​

WindowsLinuxmacOSAndroid
❌✅❌❌

Version History​

VersionRelease DateDescription
SambaFlow 1.52024SN40L support

Performance Benchmarks​

ModelTaskPerformance Metric
SN40LLlama 2 70B Inference~130 tok/s
SN40LGPT-3 175B InferencePipeline-optimized high throughput

Pricing​

ModelReference PriceNotes
SN40LCloud APISambaNova Cloud
SN40LContact vendorEnterprise deployment

Quick Installation​

SambaNova Cloud (API)​

pip install sambanova-sdk

SN40L is primarily accessed through SambaNova Cloud API or enterprise deployment.

Code Examples​

Python (SambaNova API)​

from sambanova import SambaNova

# OpenAI API-compatible interface
client = SambaNova(api_key="your-key")
response = client.chat.completions.create(
    model="Meta-Llama-3.1-70B-Instruct",
    messages=[{"role": "user", "content": "Hello"}]
)

Architecture Highlights​

  • RDU (Reconfigurable Dataflow Unit): Dataflow-reconfigurable architecture where the compiler determines data paths and hardware reorganizes on demand
  • SN40L: Supports composite execution of 1.2 trillion parameter models, integrated HBM3e
  • Compiled Dataflow: Model hardware dataflow is determined at compile time, zero runtime overhead

Model Compatibility​

Model/FrameworkSupportNotes
Llama Series✅ NativeOfficial SambaNova deployment
Large Language Models✅API inference
Custom Training⚠️Requires SambaNova contact

If you're evaluating alternatives, the following products may also fit your scenario: