Skip to main content

AWS Inferentia (First Generation)

Overview​

AWS Inferentia is an AI inference ASIC designed by Amazon Annapurna Labs and released in 2019. It delivers 128 TOPS INT8 compute at 35W TDP and is available through Inf1 instances.

Core Specifications​

ItemSpec
ArchitectureInferentia (NeuronCore-v1)
NeuronCore4 per chip
INT8 Compute128 TOPS
FP16/BF1664 TFLOPS
TDP35 W
Form FactorInf1 instances

Vendor Information​

ItemDetail
ManufacturerAmazon (Annapurna Labs)
Websitehttps://aws.amazon.com/machine-learning/inferentia/
Product Pagehttps://aws.amazon.com/ec2/instance-types/inf1/
SDKhttps://awsdocs-neuron.readthedocs-hosted.com/

Use Cases​

  • Low-latency AI inference
  • Recommendation systems
  • NLP inference