Skip to main content

NPU (Neural Processing Unit) Architecture

What is an NPU​

NPU (Neural Processing Unit) is a processor purpose-built for neural network inference. Broadly, it includes all "non-GPU, non-CPU" AI accelerators. Narrowly, it specifically refers to edge/on-device low-power AI chips (distinct from data center ASIC/WSE/LPU).

Typical NPUs: Huawei Ascend, Intel Gaudi, Hailo, Apple Neural Engine (SoC-integrated).

NPU vs GPU vs TPU​

DimensionNPUGPUTPU
Primary positioningInference (some training)Training + inferenceTraining + inference
Deployment scenarioEdge / on-device / data centerData center / workstationGoogle Cloud only
Power< 100W100-1000W100-700W
EcosystemFragmented (vendor SDKs)CUDA matureJAX/TF
General-purposeMediumStrongMedium
ProgrammingVendor SDKCUDAXLA

Major NPU Vendors​

Huawei Ascend (Da Vinci Architecture)​

  • Ascend 910B / 910C / 910D: Data center training + inference
  • Ascend 310: Edge inference (10-22W)
  • MindSpore ecosystem
  • Dominant in China market

Intel Gaudi (Habana Labs)​

  • Gaudi 1 / 2 / 3: Data center training + inference
  • SynapseAI SDK
  • Cost-effectiveness approach

Hailo​

  • Hailo-8 / 8L / 15: Edge vision AI inference
  • Dataflow Architecture (similar to WSE)
  • < 10W ultra-low power

Apple Neural Engine​

  • Integrated into M-Series / A-Series SoCs
  • 16-76 TOPS (by SoC generation)
  • On-device Apple Intelligence

Edge NPUs (PC Integrated)​

  • Intel Meteor Lake / Lunar Lake / Arrow Lake-H / Panther Lake
  • AMD Ryzen AI (XDNA) (based on Xilinx)
  • Qualcomm Hexagon NPU (Snapdragon X Elite)
  • Apple M-Series ANE

Mainstream NPU Spec Comparison​

NPUComputeTDPMemoryDeployment
Huawei Ascend 910C780 TFLOPS BF16310W128GB HBM2eData center
Huawei Ascend 310P22 TOPS INT88W24GB LPDDR4XEdge
Intel Gaudi 31,835 TFLOPS BF16900W128GB HBM2eData center
Hailo-826 TOPS2.5W4GBEdge
Apple M4 ANE38 TOPSIntegratedShared UMAOn-device
Qualcomm Hexagon NPU45 TOPSIntegratedSharedOn-device PC

NPU Use Cases​

  • ✅ Edge / on-device AI inference (low power, always-on)
  • ✅ Custom data center AI acceleration (Huawei, Intel)
  • ✅ Large model inference optimization (specific models)
  • ✅ On-device GenAI (Apple Intelligence, Copilot+ PC)
  • ❌ General AI training (use GPU / TPU)
  • ❌ Large model training (H100/B200 superior)

Detailed Product Pages​

Huawei Ascend (Da Vinci Architecture)​

Intel Gaudi (Habana Labs)​

  • Intel Gaudi 1 - 2020 first generation
  • Intel Gaudi 2 - 2022 second generation, 24-core Tensor processor 432 GB/s
  • Intel Gaudi 3 - 2024 third generation, 128GB HBM2e 1835 BF16 TFLOPS
  • Intel Gaudi 4 - 2026-Q2 estimated 3nm 192GB HBM3E 3.7 PF FP8 18A 24× 200GbE RoCE

Intel Core Ultra (On-Device PC NPU)​

Qualcomm (On-Device PC + Mobile NPU)​

MediaTek (Mobile NPU)​

  • MediaTek Dimensity 9400 - 3nm N3E, Cortex-X925, NPU 890 8th gen, world's first Agentic AI engine, on-device LoRA training

Apple (On-Device ANE)​

Korean AI Trio (Data Center + Edge NPU)​

Chinese AI Startups (Data Center NPU)​

Edge NPU​

  • Hailo-15 - 2024-Q4 7-20 TOPS AI-ISP Gen3 vision AI
  • Hailo-8 - 2019, 26 TOPS 2.5W edge evergreen