- Admin
- February 24, 2025
Case Study
AI Hardware Accelerator Benchmarking
Client
A North America based AI Hardware (HW) accelerator chip company approached Ignitarium to perform power-performance-accuracy (PPA) benchmarking on new age AI accelerator devices to measure and compare PPA metrics.
Scope
• Find latency and throughput of state-of-the-art (SoTA) DL models using hardware accelerators
• Compare DL model accuracy with SoTA models from original framework
• Measure power consumption under DL load and verify the FPS/watt numbers published
Challenges
While performing the task, Ignitarium was required to address the following challenges:
• The team had to ramp up quickly on the technical expertise to use the newly launched software SDK and APIs
• Bring-up various AI accelerator carrier boards in a timely manner
• Ramping up quickly on various aspects of SDK to ensure model conversions and inference scripts are done accurately
Solution
Ignitarium’s approach consisted of following steps:
• Once the reference application code ran on the device, inference scripts were written for the standard SoTA models
• To maintain the transparency, SOTA models were converted to proprietary format and latency throughput was calculated
• Accuracy, host-accelerator computation split and power requirements under DL load were calculated
Business Impact
Detailed analysis report helped the customer leverage the PPA data to position their AI HW appropriately with potential customers
Customer’s Product Marketing Team got relevant inputs for planning the specifications of their next generation AI HW chip
By leveraging our vast experience in benchmarking of AI HW accelerators, customer was able to get the analysis done well within the time and cost budgeted
