Evaluation Metrics
Evaluation Metrics helps you understand how compression affects your models across different dimensions - output quality and resource requirements. Our metrics fall into two overarching categories:
Efficiency Metrics: for speed (total_time, latency, throughput..), memory (disk, inference, and training_memory…), and energy (energy_consumed, co2_emissions)
Quality Metrics: for Fidelity (FID, CMMD...), Alignment (Clip Score...), Diversity (PSNR, SSIM...), Accuracy (Accuracy, precision, perplexity...), and more.
You can also add your own custom metrics!
For image quality evaluation, you can refer to this blog post.
For general documentation on the evaluation metrics, you can find all the details in the documentation.