基于PyPIServer创建私有Python软件包存储库

  • 客户端不仅可以下载还可以上传（当我们自己开发了Python的软件时）
#创建用户名和密码
sudo apt install apache2-utils -y
sudo mkdir /data/pypi-packages
sudo htpasswd -sc /data/pypi-packages/htpasswd.txt wjj
#当您需要再创建用户名时就不需要加参数 -c
sudo htpasswd -s /data/pypi-packages/htpasswd.txt test
#容器部署
docker run -d --restart=always --name pypiserver -p 8080:8080 \
    -v /data/pypi-packages/:/data/packages \
    pypiserver/pypiserver:latest -P /data/packages/htpasswd.txt
  • 安装
pip3 install tensorflow

Looking in indexes: http://172.16.33.174:8080/simple/, https://mirrors.aliyun.com/pypi/simple/
Collecting tensorflow
  Downloading http://172.16.33.174:8080/packages/tensorflow-2.4.0-cp38-cp38-manylinux2010_x86_64.whl (394.8 MB)
PythonpipPyPIPyPIServerDockerhtpasswd

Apache HTTP Server实践 

# If you just change the port or add more ports here, you will likely also
# have to change the VirtualHost statement in
# /etc/apache2/sites-enabled/000-default.conf

Listen 8081
......
......
  • /etc/apache2/sites-enabled/000-default.conf
sudo nano /etc/apache2/sites-enabled/000-default.conf

<VirtualHost *:8081>
......
......
</VirtualHost>
  • 重启服务
sudo systemctl restart apache2
LinuxCommandApacheLinux 实践

Dockerfile OpenCV4 Ubuntu20.04

  1. ImportError: libgthread-2.0.so.0: cannot open shared object file: No such file or directory
>>> import cv2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/dist-packages/cv2/__init__.py", line 5, in <module>
    from .cv2 import *
ImportError: libgthread-2.0.so.0: cannot open shared object file: No such file or directory

apt install -y libglib2.0-dev

需要交互，不能自动化安装。 apt install libglib2.0-dev Setting up tzdata (2020f-0ubuntu0.20.04.1) ...

DockerDockerfileOpenCVUbuntu

Building ONNX Runtime

  • 拉取容器（编译环境）
docker pull nvidia/cuda:11.1-cudnn8-devel-ubuntu20.04
  • 运行容器
docker run -it --name build-onnxruntime-gpu --runtime nvidia \
    -v $(pwd)/onnxruntime:/onnxruntime -w /onnxruntime \
    nvidia/cuda:11.1-cudnn8-devel-ubuntu20.04
  • 更新apt镜像源
sed -i 's/archive.ubuntu.com/mirrors.aliyun.com/g' /etc/apt/sources.list
apt-get update
  • 安装依赖包
apt-get install language-pack-en git cmake python3 python3-pip -y
  • 修改语言环境
locale-gen en_US.UTF-8
update-locale LANG=en_US.UTF-8
  • 更新pip镜像源
pip3 config set global.index-url https://mirrors.aliyun.com/pypi/simple/
  • 安装numpy
pip3 install numpy
  • 编译
./build.sh --parallel --build_shared_lib --enable_pybind --build_wheel \
    --use_cuda --cudnn_home /usr/include/x86_64-linux-gnu/ \
    --cuda_home /usr/local/cuda/ --config Release
LinuxUbuntuGPUCUDADockerONNXONNXRuntime

Dockerfile ONNXRuntime GPU 

FROM nvidia/cuda:11.1-cudnn8-devel-ubuntu20.04 AS builder
LABEL maintainer="wang-junjian@qq.com"

#E: Failed to fetch https://developer.download.nvidia.cn/compute/cuda/repos/ubuntu2004/x86_64/by-hash/SHA256/f10fc2a7a0d072ddcf141af2ef28f1e97ab4b3a5c3b9bbe34ed845d174fb4979  404  Not Found [IP: 61.155.167.2 443]
#E: Some index files failed to download. They have been ignored, or old ones used instead.
RUN rm /etc/apt/sources.list.d/cuda.list /etc/apt/sources.list.d/nvidia-ml.list

RUN sed -i 's/archive.ubuntu.com/mirrors.aliyun.com/g' /etc/apt/sources.list && \
    apt-get update && \
    apt-get install language-pack-en git python3 python3-pip -y && \
    DEBIAN_FRONTEND=noninteractive apt-get install cmake -y && \
    locale-gen en_US.UTF-8 && \
    update-locale LANG=en_US.UTF-8

RUN pip3 install numpy -i https://mirrors.aliyun.com/pypi/simple/
// ...
DockerDockerfileONNXRuntimeCUDA

VLC Extension Example

  • Mac OS X
/Applications/VLC.app/Contents/MacOS/share/lua/extensions/
  • Linux
/home/$USER/.local/share/vlc/lua/extensions/
/usr/share/vlc/lua/extensions/
/usr/lib/vlc/lua/playlist/
VLCLuaPlayer

命令ffmpeg

  • 生成gif(低质量) -pix_fmt(像素格式) -s(设置帧大小WxH)
ffmpeg -y -i input.mp4 -pix_fmt rgb8 -r 10 -s 320x240 output.gif
ffmpeg -y -i input.mp4 -pix_fmt rgb8 -r 10 -vf 'scale=320:-1' output.gif
  • 生成gif(高质量) -ss(开始时间偏移) -t(持续时间)
ffmpeg -i input.mp4 -vf "fps=10,scale=320:-1:flags=lanczos,split[s0][s1];[s0]palettegen[p];[s1][p]paletteuse" -loop 0 output.gif
ffmpeg -y -ss 5 -t 5 -i input.mp4 -vf "fps=10,scale=320:-1:flags=lanczos,split[s0][s1];[s0]palettegen[p];[s1][p]paletteuse" -loop 0 output.gif
  • 每秒抽取一张图片 -r(设置帧速率)
ffmpeg -i input.mp4 -r 1 -s 1024x768 -f image2 input-%03d.jpeg
LinuxffmpegGPUNVIDIAforbasenameCommand

命令history

  • 持久化设置，可以修改配置文件：.bash_profile 或 .bashrc。执行source命令后，设置生效。你也可以退出后重新登录。
$ nano .bashrc
export HISTCONTROL=ignorespace
$ source .bashrc
CommandLinuxsource.bashrc.bash_profilehistory

在YOLOv5中运行JupyterLab和TensorBoard

  • 安装版本1的TensorBoard。（解决FAQ1的问题：jupyter-tensorboard 0.2.0不支持高于TensorBoard 2.0的版本。YOLOv5镜像中安装的TensorBoard 2.4的版本。）
pip uninstall tensorboard -y && pip install tensorboard==1.15
  • 运行JupyterLab
jupyter lab --no-browser --ip 0.0.0.0 --port 8888
  • 本地浏览器进行访问
http://ip:8888/lab
JupyterJupyterLabTensorBoardYOLOv5

命令find

  • 以详细信息显示 /var/log 目录下最近 24小时内修改的文件
$ find /var/log -mtime 0 -ls
 33575669      4 drwxr-xr-x  15  root     root         4096 7月 29 06:33 /var/log
 34131780    164 -rw-------   1  root     root       166061 7月 29 10:20 /var/log/messages
  • 显示 /var/log 目录下最近 [0 - 24小时] 修改的文件
find /var/log -mtime -1
  • 显示 /var/log 目录下最近 [24 - 48小时] 修改的文件
find /var/log -mtime 1
  • 显示 /var/log 目录下最近 [48 - ] 修改的文件
find /var/log -mtime +1
  • 以详细信息显示当前目录下超过 40M的文件
$ find . -size +40M -ls
 17180574  42480 -rwxr-xr-x   1  root     root      43499520 3月 11  2020 ./kubectl
  • 显示当前目录下超过 40M的文件，通过 ls 命令来显示结果。
$ find . -size +40M -exec ls -lh {} \;
-rwxr-xr-x 1 root root 42M 3月  11 2020 ./kubectl
LinuxfinddeleteDS_StoreCommand

命令grep

  • 忽略字母大小写(-i)
grep -i 'text' hello.txt
  • 搜索多个文件
grep 'text' hello.txt hi.txt
  • 搜索当前目录下所有文件
grep 'text' *
  • 搜索当前目录（包含子目录 -R）下所有文件
grep -R 'text' *
  • 搜索 pip 配置文件的路径(增加过滤)
find ~ -name pip* | xargs -i grep "index-url" {} --color -nH

/home/lnsoft/.config/pip/pip.conf:2:index-url = https://mirrors.aliyun.com/pypi/simple/
  • 只匹配字符串，不使用正则表达式。
find . | grep -F .run
  • 显示当前目录下的文件
ll | grep '^-'
  • 显示当前目录下的所有目录及子目录（遍历）
ll -R | grep '^d'
  • 统计目录数
ll | grep '^d' | wc -l
LinuxgrepfindllCommand

图像格式转换、尺寸调整

  • macOS
brew install imagemagick
  • 按照指定高等比调整
convert test.jpg -resize x640 test.jpg
  • 按照指定宽高等比调整
convert test.jpg -resize 640x640 test.jpg
  • 按照指定宽高调整
convert test.jpg -resize 640x640! test.jpg
  • 方法2
find . -name '*.png' -exec convert {} -resize 640x640 {} \;
LinuxCommandImagefilefindconvertShellScriptfor

在Ubuntu上下载docker和nvidia-docker2离线安装包

以下是容器内操作

  • 分析要下载的依赖安装包
$ apt-get install -s nvidia-docker2
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  libcap2 libnvidia-container-tools libnvidia-container1 nvidia-container-runtime nvidia-container-toolkit
The following NEW packages will be installed:
  libcap2 libnvidia-container-tools libnvidia-container1 nvidia-container-runtime nvidia-container-toolkit nvidia-docker2
0 upgraded, 6 newly installed, 0 to remove and 0 not upgraded.
Inst libcap2 (1:2.32-1 Ubuntu:20.04/focal [amd64])
Inst libnvidia-container1 (1.3.0-1 NVIDIA CORPORATION <cudatools@nvidia.com>:1.0/bionic [amd64])
Inst libnvidia-container-tools (1.3.0-1 NVIDIA CORPORATION <cudatools@nvidia.com>:1.0/bionic [amd64])
Inst nvidia-container-toolkit (1.3.0-1 NVIDIA CORPORATION <cudatools@nvidia.com>:1.0/bionic [amd64])
Inst nvidia-container-runtime (3.4.0-1 NVIDIA CORPORATION <cudatools@nvidia.com>:1.0/bionic [amd64])
Inst nvidia-docker2 (2.5.0-1 NVIDIA CORPORATION <cudatools@nvidia.com>:1.0/bionic [all])
Conf libcap2 (1:2.32-1 Ubuntu:20.04/focal [amd64])
Conf libnvidia-container1 (1.3.0-1 NVIDIA CORPORATION <cudatools@nvidia.com>:1.0/bionic [amd64])
Conf libnvidia-container-tools (1.3.0-1 NVIDIA CORPORATION <cudatools@nvidia.com>:1.0/bionic [amd64])
Conf nvidia-container-toolkit (1.3.0-1 NVIDIA CORPORATION <cudatools@nvidia.com>:1.0/bionic [amd64])
Conf nvidia-container-runtime (3.4.0-1 NVIDIA CORPORATION <cudatools@nvidia.com>:1.0/bionic [amd64])
Conf nvidia-docker2 (2.5.0-1 NVIDIA CORPORATION <cudatools@nvidia.com>:1.0/bionic [all])
UbuntuDockernvidia-docker2offline

Linux上查找系统信息

  • /proc/version
$ cat /proc/version
Linux version 4.18.0-147.5.1.el8_1.x86_64 (mockbuild@kbuilder.bsys.centos.org) (gcc version 8.3.1 20190507 (Red Hat 8.3.1-4) (GCC)) #1 SMP Wed Feb 5 02:00:39 UTC 2020
  • hostnamectl
$ hostnamectl | grep Kernel
            Kernel: Linux 4.18.0-147.5.1.el8_1.x86_64
  • Ubuntu
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 20.04 LTS
Release:	20.04
Codename:	focal
  • CentOS
LSB Version:	:core-4.1-amd64:core-4.1-noarch
Distributor ID:	CentOS
Description:	CentOS Linux release 8.1.1911 (Core) 
Release:	8.1.1911
Codename:	Core
  • /proc/driver/nvidia/version
$ cat /proc/driver/nvidia/version
NVRM version: NVIDIA UNIX x86_64 Kernel Module  450.80.02  Wed Sep 23 01:13:39 UTC 2020
GCC version:  gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)
Linux系统信息GPUCUDAMemory硬盘lsb_releasegrep

GitHub加速

加速下载：git clone https://github.com.cnpmjs.org/microsoft/onnxruntime.git

  • Mac OS X
/etc/hosts
  • Linux
/etc/hosts
GitHubcurlhosts