25x Faster AI Inferencing! VersaLogic’s “Sabertooth AI”

Press Office, VersaLogic Corporation, 12/18/24

Why AI Performance Matters in Embedded Computing
The performance of AI is essential for the success of embedded computing systems, particularly in real-time edge AI applications. Quick and reliable inferencing — making predictions based on trained models — is vital for the effective operation of edge devices, especially in scenarios like autonomous vehicles, robotics, and industrial automation. The speed and accuracy of these decisions can greatly impact application success.

In these fields, real-time decision-making is critical. AI performance relies not only on processing power but also on how quickly a system can adapt to its environment. Faster AI processing reduces latency and enhances reliability, directly influencing safety and efficiency.

Head-to-Head Comparison
To evaluate the AI performance of embedded platforms like VersaLogic’s Sabertooth AI and the NVIDIA Jetson AGX Orin, a variety of testing criteria and benchmarks were used to measure inferencing performance. The benchmarks are designed to simulate real-world AI workloads and assess how well a platform can handle tasks like image recognition, object detection, and sensor data analysis.

Utilizing AI-Benchmark, VersaLogic ran tests to compare the inference score of their Sabertooth AI and the NVIDIA Jetson AGX Orin.

* Comparison using “AI-Benchmark v.0.1.2” tool, measuring a wide range of inferencing performance.

The AI inferencing score is a comprehensive metric that assesses the performance of an embedded AI platform across several key dimensions:

• Inference Latency: How long it takes for the platform to process a given input and provide a prediction.
• Throughput: The number of inferences the platform can process per second.
• Power Efficiency: A measure of how efficiently the platform uses energy while performing inference tasks.
• Model Support and Flexibility: The ability of the platform to run a diverse range of pre-trained AI models with minimal optimization or retraining.

VersaLogic’s Sabertooth AI system outperformed the NVIDIA Jetson AGX Orin by an impressive margin, achieving AI inference speeds that were 25 times faster. The Sabertooth AI’s enhanced speed in processing AI models means it can handle more data in less time, leading to faster and more accurate predictions in critical applications.

What Makes the Sabertooth AI Faster?
The Sabertooth AI system is powered by the hex-core Intel® Xeon®-E CPU, together with an NVIDIA RTX® 2000 ADA GPU for high-performance AI inferencing. It features TPM 2.0 security, 32 GB error-correcting memory, high-speed SSD Storage (NVMe), and lots of computing power in a compact package.

The NVIDIA GPU, built on the innovative Lovelace Architecture, provides:

• 3,072 CUDA Cores, responsible for executing parallel processing tasks typically required in AI computations. The more CUDA cores, the faster and more efficiently the GPU can process tasks like graphics or data.
• 96 Tensor Cores, designed to help the GPU process and calculate large amounts of data very quickly, making it much faster when running AI models.
• 24 RT Cores that enhance real-time ray tracing capabilities.
• 12.99 TFLOPS (trillions of floating-point operations per second) — this GPU can handle nearly 13 trillion calculations every second.

In addition, the Sabertooth AI offers energy-efficient operation, designed to meet the demands of edge computing environments. This ensures that the system can run continuously without overheating or excessive power draw — an important consideration for embedded systems deployed in remote or mobile locations.

Rugged, Reliable, and Compact
The Sabertooth AI board offers significant advantages for industries that rely on high-performance, edge-based AI processing. Its efficiency and reliability are crucial for applications such as industrial automation, defense, security, and autonomous systems, enabling real-time monitoring and predictive maintenance to prevent system failures and boost operational uptime.

One of the standout features of the Sabertooth AI board is its compact size: 90 x 96 x 63 mm (3.5 x 3.8 x 2.5”). This makes it an excellent choice for applications where space is limited.

The Sabertooth AI system is designed and tested for industrial temperature operation (-40° to +85°C) and meets MIL-STD-202H specifications for shock and vibration.

As with all VersaLogic products, the Sabertooth AI was designed for long-term availability (10+ year typical production lifecycle).

VersaLogic’s Advantage in Embedded AI Computing
The 25 times performance advantage highlights the Sabertooth AI’s ability to excel in demanding AI workloads, offering a competitive edge where low latency and high throughput are essential. For OEMs looking to implement edge AI solutions with strict performance requirements, the Sabertooth AI’s exceptional speed makes it a standout choice.

VersaLogic is committed to providing world-class customer support throughout the entire lifecycle of embedded system ownership. Whether you’re designing a complex application or integrating a new solution, their expert team is committed to providing the assistance you need from start to finish.

Want to learn more? VersaLogic’s Embedded Applications Consultants are available to talk about how the Sabertooth AI can be integrated into your application.