代码拉取完成,页面将自动刷新
同步操作将从 MindSpore/mindformers 强制同步,此操作会覆盖自 Fork 仓库以来所做的任何修改,且无法恢复!!!
确定后同步将在后台操作,完成时将刷新页面,请耐心等待。
零样本图像分类:模型在基于图文对的预训练后,可以在给定任意图片与候选标签列表的情况下,完成对图像的分类,而无需任何微调。
相关论文 Alec Radford, Jong Wook Kim, et al., Learning Transferable Visual Models From Natural Language Supervision, 2021.
model | type | datasets | Top1-accuracy | stage | example |
---|---|---|---|---|---|
clip | clip_vit_b_32 clip_vit_b_16 clip_vit_l_14 clip_vit_l_14@336 |
Cifar100 | 57.24% 61.41% 69.67% 68.19% |
eval predict |
link link |
数据集目录格式
└─cifar-100-python
├─meta
├─test
└─train
import mindspore; mindspore.set_context(mode=0, device_id=0)
from mindformers import MindFormerBook
from mindformers.trainer import Trainer
from mindformers.tools.image_tools import load_image
# 显示Trainer的模型支持列表
MindFormerBook.show_trainer_support_model_list("zero_shot_image_classification")
# INFO - Trainer support model list for zero_shot_image_classification task is:
# INFO - ['clip_vit_b_32', 'clip_vit_b_16', 'clip_vit_l_14', 'clip_vit_l_14@336']
# INFO - -------------------------------------
# 初始化trainer
trainer = Trainer(task='zero_shot_image_classification',
model='clip_vit_b_32',
eval_dataset='cifar-100-python'
)
img = load_image("https://ascend-repo-modelzoo.obs.cn-east-2."
"myhuaweicloud.com/XFormer_for_mindspore/clip/sunflower.png")
trainer.evaluate() #下载权重进行评估
# INFO - Top1 Accuracy=57.24%
trainer.predict(input_data=img) #下载权重进行推理
# INFO - output result is saved at ./results.txt
import mindspore; mindspore.set_context(mode=0, device_id=0)
from mindformers import pipeline, MindFormerBook
from mindformers.tools.image_tools import load_image
# 显示pipeline支持的模型列表
MindFormerBook.show_pipeline_support_model_list("zero_shot_image_classification")
# INFO - Pipeline support model list for zero_shot_image_classification task is:
# INFO - ['clip_vit_b_32', 'clip_vit_b_16', 'clip_vit_l_14', 'clip_vit_l_14@336']
# INFO - -------------------------------------
# pipeline初始化
classifier = pipeline("zero_shot_image_classification",
model="clip_vit_b_32"
candidate_labels=["sunflower", "tree", "dog", "cat", "toy"])
img = load_image("https://ascend-repo-modelzoo.obs.cn-east-2."
"myhuaweicloud.com/XFormer_for_mindspore/clip/sunflower.png")
classifier(img)
# 输出
# [[{'score': 0.99995565, 'label': 'sunflower'}, {'score': 2.5318595e-05, 'label': 'toy'},
# {'score': 9.903885e-06, 'label': 'dog'}, {'score': 6.75336e-06, 'label': 'tree'},
# {'score': 2.396818e-06, 'label': 'cat'}]]
此处可能存在不合适展示的内容,页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容,可点击提交进行申诉,我们将尽快为您处理。