This package is designed to compute the theoretical amount of FLOPs(floating-point operations)、MACs(multiply-add operations) and Parameters in all various neural networks, such as Linear、 CNN、 RNN、 GCN、Transformer(Bert、LlaMA etc Large Language Model),including any custom models via torch.nn.function.* as long as based on the Pytorch implementation.
Project description
calculate-flops.pytorch
This tool(calflops) is designed to compute the theoretical amount of FLOPs(floating-point operations)、MACs(multiply-add operations) and Parameters in all various neural networks, such as Linear、 CNN、 RNN、 GCN、Transformer(Bert、LlaMA etc Large Language Model),including any custom models via torch.nn.function.*
as long as based on the Pytorch implementation.
In addition, the implementation process of this package inspired by ptflops and deepspeed libraries, for which I am very grateful for their great efforts, they are both very good work. Meanwhile this package also improves some aspects(more simple use、more model support) based on them.
Install the latest version
From PyPI:
pip install calflops
And you also can download latest calflops-*-py3-none-any.whl
files from https://pypi.org/project/calflops/
pip install calflops-*-py3-none-any.whl
Example
from calflops import calculate_flops
# Deep Learning Model, such as alexnet.
from torchvision import models
model = models.alexnet()
batch_size = 1
flops, macs, params = calculate_flops(model=model,
input_shape=(batch_size, 3, 224, 224),
output_as_string=True,
output_precision=4)
print("alexnet FLOPs:%s MACs:%s Params:%s \n" %(flops, macs, params))
#alexnet FLOPs:1.4297 GFLOPS MACs:714.188 MMACs Params:61.1008 M
# Transformers Model, such as bert.
from transformers import AutoModel
from transformers import AutoTokenizer
batch_size = 1
max_seq_length = 128
model_name = "hfl/chinese-roberta-wwm-ext/"
model_save = "../pretrain_models/" + model_name
model = AutoModel.from_pretrained(model_save)
tokenizer = AutoTokenizer.from_pretrained(model_save)
flops, macs, params = calculate_flops_pytorch(model=model,
input_shape=(batch_size, max_seq_length),
transformer_tokenizer=tokenizer)
print("bert(hfl/chinese-roberta-wwm-ext) FLOPs:%s MACs:%s Params:%s \n" %(flops, macs, params))
#bert(hfl/chinese-roberta-wwm-ext) FLOPs:22.36 GFLOPS MACs:11.17 GMACs Params:102.27 M
# Large Languase Model, such as llama2-7b.
from transformers import LlamaTokenizer
from transformers import LlamaForCausalLM
batch_size = 1
max_seq_length = 128
model_name = "llama2_hf_7B"
model_save = "../model/" + model_name
model = LlamaForCausalLM.from_pretrained(model_save)
tokenizer = LlamaTokenizer.from_pretrained(model_save)
flops, macs, params = calculate_flops(model=model,
input_shape=(batch_size, max_seq_length),
transformer_tokenizer=tokenizer)
print("llama2(7B) FLOPs:%s MACs:%s Params:%s \n" %(flops, macs, params))
#llama2(7B) FLOPs:1.7 TFLOPS MACs:850.00 GMACs Params:6.74 B
calculate_flops API
def calculate_flops(model,
input_shape=None,
transformer_tokenizer=None,
args=[],
kwargs={},
forward_mode="forward",
include_backPropagation=False,
compute_bp_factor=2.0,
print_results=True,
print_detailed=True,
output_as_string=True,
output_precision=2,
ignore_modules=None):
"""Returns the total floating-point operations, MACs, and parameters of a model.
Args:
model ([torch.nn.Module]): The model of input must be a PyTorch model.
input_shape (tuple, optional): Input shape to the model. If args and kwargs is empty, the model takes a tensor with this shape as the only positional argument. Default to [].
transformers_tokenizer (None, optional): Transforemrs Toekenizer must be special if model type is transformers and args、kwargs is empty. Default to None
args (list, optinal): list of positional arguments to the model, such as bert input args is [input_ids, token_type_ids, attention_mask]. Default to []
kwargs (dict, optional): dictionary of keyword arguments to the model, such as bert input kwargs is {'input_ids': ..., 'token_type_ids':..., 'attention_mask':...}. Default to {}
forward_mode (str, optional): To determine the mode of model inference, Default to 'forward'. And use 'generate' if model inference uses model.generate().
include_backPropagation (bool, optional): Decides whether the final return FLOPs computation includes the computation for backpropagation.
compute_bp_factor (float, optional): The model backpropagation is a multiple of the forward propagation computation. Default to 2.
print_results (bool, optional): Whether to print the model profile. Defaults to True.
print_detailed (bool, optional): Whether to print the detailed model profile. Defaults to True.
output_as_string (bool, optional): Whether to print the output as string. Defaults to True.
output_precision (int, optional) : Output holds the number of decimal places if output_as_string is True. Default to 2.
ignore_modules ([type], optional): the list of modules to ignore during profiling. Defaults to None.
Concact Author
Author: MrYXJ
Mail: code.mryxj@gmail.com
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.