Get Started OpenVINO

OpenVINO
OpenVINO 安装
- Development
  - pip 安装
  - 源代码编译安装(没有成功)👹
- Runtime
OpenVINO 开发包下的组件
验证安装
查询 OpenVINO Runtime 设备及指标
模型
- 下载所有模型
- 转换所有下载的模型到 IR 格式
模型查看
Samples
工具套件插件
开发文档
参考资料

OpenVINO

Open Visual Inference and Neural network Optimization

OpenVINO 安装

Development

pip 安装

下载

安装 OpenVINO 开发工具

python -m venv openvino_env

Linux
```
source openvino_env/bin/activate
```
Windows
```
openvino_env\Scripts\activate.bat
```

python -m pip install --upgrade pip
pip install openvino-dev[onnx,pytorch,kaldi,mxnet,caffe,tensorflow2]==2022.1.0 -i https://mirrors.aliyun.com/pypi/simple/

Install OpenVINO™ Development Tools

源代码编译安装(没有成功)👹

Build OpenVINO™ Inference Engine

mkdir openvinotoolkit && cd openvinotoolkit
git clone -b 2022.1.0 https://github.com/openvinotoolkit/openvino.git
git clone https://github.com/openvinotoolkit/open_model_zoo.git

cd openvino
sudo ./install_build_dependencies.sh 
sudo ./scripts/install_dependencies/install_NEO_OCL_driver.sh -y --no_numa
sudo ./scripts/install_dependencies/install_openvino_dependencies.sh -y -c=python

cmake --install . --prefix /opt/intel/openvino

OpenVINO Dockerfile

Runtime

OpenVINO 开发包下的组件

Model Optimizer

mo 训练的模型导入、转换和优化为 OpenVINO 可用的格式（IR）。

Benchmark Tool

benchmark_app 允许您在同步和异步模式下估计受支持设备上的深度学习模型推理性能。

Accuracy Checker 和 Annotation Converter

accuracy_check 深度学习准确性验证工具，可用于收集针对常用数据集的准确性指标。该工具的主要优点是配置的灵活性和一组支持的数据集、预处理、后处理和指标。
convert_annotation 用于准备数据集以使用 accuracy_check 进行评估。

Post-Training Optimization Tool

pot 训练后优化工具允许您使用高级功能（如量化和低精度优化）优化训练的模型，而无需重新训练或微调模型。还可以通过 API 进行优化。

Open Model Zoo tools

omz_downloader 用于访问预训练深度学习公共模型和英特尔训练模型的集合。
omz_converter 使用模型优化器将存储在原始深度学习框架格式中的 Open Model Zoo 模型转换为 OpenVINO 中间表示 (IR)。
omz_quantizer 用于使用训练后优化工具（Post-Training Optimization Tool）将 IR 格式的全精度模型自动量化为低精度版本。
omz_info_dumper 用于将有关模型的信息转储为机器可读格式。
omz_data_downloader 数据集下载器

验证安装

$ python -c "from openvino.inference_engine import IECore; print('Inference Engine Available Devices: ', IECore().available_devices)"
Inference Engine Available Devices:  ['CPU', 'GPU']

查询 OpenVINO Runtime 设备及指标

Hello Query Device Python Sample

$ cd <openvino_dir>
$ python samples/python/hello_query_device/hello_query_device.py
[ INFO ] Available devices:
[ INFO ] CPU :
[ INFO ]        SUPPORTED_PROPERTIES:
[ INFO ]                AVAILABLE_DEVICES:
[ INFO ]                RANGE_FOR_ASYNC_INFER_REQUESTS: 1, 1, 1
[ INFO ]                RANGE_FOR_STREAMS: 1, 12
[ INFO ]                FULL_DEVICE_NAME: Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz
[ INFO ]                OPTIMIZATION_CAPABILITIES: FP32, FP16, INT8, BIN, EXPORT_IMPORT
[ INFO ]                CACHE_DIR:
[ INFO ]                NUM_STREAMS: 1
[ INFO ]                AFFINITY: UNSUPPORTED TYPE
[ INFO ]                INFERENCE_NUM_THREADS: 0
[ INFO ]                PERF_COUNT: False
[ INFO ]                INFERENCE_PRECISION_HINT: UNSUPPORTED TYPE
[ INFO ]                PERFORMANCE_HINT: UNSUPPORTED TYPE
[ INFO ]                PERFORMANCE_HINT_NUM_REQUESTS: 0
[ INFO ]
[ INFO ] GPU :
[ INFO ]        SUPPORTED_PROPERTIES:
[ INFO ]                AVAILABLE_DEVICES: 0
[ INFO ]                RANGE_FOR_ASYNC_INFER_REQUESTS: 1, 2, 1
[ INFO ]                RANGE_FOR_STREAMS: 1, 2
[ INFO ]                OPTIMAL_BATCH_SIZE: 1
[ INFO ]                MAX_BATCH_SIZE: 1
[ INFO ]                FULL_DEVICE_NAME: Intel(R) UHD Graphics 630 (iGPU)
[ INFO ]                DEVICE_TYPE: UNSUPPORTED TYPE
[ INFO ]                DEVICE_GOPS: UNSUPPORTED TYPE
[ INFO ]                OPTIMIZATION_CAPABILITIES: FP32, BIN, FP16
[ INFO ]                GPU_DEVICE_TOTAL_MEM_SIZE: UNSUPPORTED TYPE
[ INFO ]                GPU_UARCH_VERSION: unknown
[ INFO ]                GPU_EXECUTION_UNITS_COUNT: 24
[ INFO ]                GPU_MEMORY_STATISTICS: UNSUPPORTED TYPE
[ INFO ]                PERF_COUNT: False
[ INFO ]                MODEL_PRIORITY: UNSUPPORTED TYPE
[ INFO ]                GPU_HOST_TASK_PRIORITY: UNSUPPORTED TYPE
[ INFO ]                GPU_QUEUE_PRIORITY: UNSUPPORTED TYPE
[ INFO ]                GPU_QUEUE_THROTTLE: UNSUPPORTED TYPE
[ INFO ]                GPU_ENABLE_LOOP_UNROLLING: True
[ INFO ]                CACHE_DIR:
[ INFO ]                PERFORMANCE_HINT: UNSUPPORTED TYPE
[ INFO ]                COMPILATION_NUM_THREADS: 12
[ INFO ]                NUM_STREAMS: 1
[ INFO ]                PERFORMANCE_HINT_NUM_REQUESTS: 0
[ INFO ]                DEVICE_ID: 0
[ INFO ]

模型

下载所有模型

omz_downloader --all

转换所有下载的模型到 IR 格式

omz_converter --all

模型查看

netron 可以查看各种深度学习和机器学习框架训练出来的模型。

安装 netron

pip install netron

查看网络

通过设置 --host 0.0.0.0，局域网内的其它主机也可以通过 http://ip:8080 访问。

netron public/alexnet/FP16/alexnet.xml --host 0.0.0.0

Web 应用 netron

通过页面的下载可以安装本地程序。

Samples

图像分类

异步推理 Image Classification Async Python Sample

同步推理 Hello Classification Python Sample

Image 和 Video 文件 download.01.org

下载预训练模型

omz_downloader --name alexnet

转换模型，如果模型非 IR 或 ONNX 格式就需要转换。

omz_converter --name alexnet

下载图片

wget https://pamsdailydish.com/wp-content/uploads/2015/04/Bunch-Bananas-1.jpg -O banana.jpg

异步模型推理

$ python samples/python/classification_sample_async/classification_sample_async.py -m public/alexnet/FP16/alexnet.xml -i banana.jpg -d CPU
[ INFO ] Creating OpenVINO Runtime Core
[ INFO ] Reading the model: public/alexnet/FP16/alexnet.xml
[ INFO ] Loading the model to the plugin
[ INFO ] Starting inference in asynchronous mode
[ INFO ] Image path: banana.jpg
[ INFO ] Top 10 results: 
[ INFO ] class_id probability
[ INFO ] --------------------
[ INFO ] 954      0.9999748
[ INFO ] 666      0.0000086
[ INFO ] 953      0.0000053
[ INFO ] 939      0.0000025
[ INFO ] 951      0.0000020
[ INFO ] 945      0.0000017
[ INFO ] 941      0.0000016
[ INFO ] 940      0.0000009
[ INFO ] 943      0.0000005
[ INFO ] 950      0.0000002
[ INFO ] 
[ INFO ] This sample is an API example, for any performance measurements please use the dedicated benchmark_app tool

同步模型推理

$ python samples/python/hello_classification/hello_classification.py public/alexnet/FP16/alexnet.xml banana.jpg CPU

图像识别

Hello Reshape SSD Python Sample

下载预训练模型

omz_downloader --name mobilenet-ssd

转换模型，如果模型非 IR 或 ONNX 格式就需要转换。

omz_converter --name mobilenet-ssd

模型推理

$ python samples/python/hello_reshape_ssd/hello_reshape_ssd.py public/mobilenet-ssd/FP16/mobilenet-ssd.xml banana.jpg CPU
[ INFO ] Creating OpenVINO Runtime Core
[ INFO ] Reading the model: public/mobilenet-ssd/FP16/mobilenet-ssd.xml
[ INFO ] Reshaping the model to the height and width of the input image
[ INFO ] Loading the model to the plugin
[ INFO ] Starting inference in synchronous mode
[ INFO ] Found: class_id = 3, confidence = 0.69, coords = (139, 16), (418, 319)
[ INFO ] Image out.bmp was created!

使用源码动态创建模型（不需要 XML 文件）

Model Creation Python Sample

$ python samples/python/model_creation_sample/model_creation_sample.py samples/python/model_creation_sample/lenet.bin CPU
[ INFO ] Creating OpenVINO Runtime Core
[ INFO ] Loading the model using ngraph function with weights from samples/python/model_creation_sample/lenet.bin
[ INFO ] Loading the model to the plugin
[ INFO ] Starting inference in synchronous mode
[ INFO ] Top 1 results: 
[ INFO ] Image 0
[ INFO ] 
[ INFO ] classid probability label
[ INFO ] -------------------------
[ INFO ] 0       1.0000000   0
[ INFO ] 
[ INFO ] Image 1
[ INFO ] 
......
[ INFO ] Image 9
[ INFO ] 
[ INFO ] classid probability label
[ INFO ] -------------------------
[ INFO ] 9       1.0000000   9