JobPlus
JobPlus知识库 IT 大数据 文章
在ARM板子上把玩Tensorflow Lite
个人头像
连解淦
2018-05-18 11200人阅读 0人打赏

Tensorflow Lite是在Google去年IO大会上发表的，目前Tensorflow Lite也还在不断的完善迭代中。

Tensorflow Lite在Android和iOS上部署官网有比较详细的介绍以及对应的Demo。而对于ARM板子上的部署及测试，官网及网上的资料则相对较少。本文主要描述如何把Tensorflow Lite编译到ARM板子上，并运行相应的Demo。

0.准备工作：在Ubuntu上准备ARM的交叉编译环境

可以通过apt-get install直接在ubuntu上安装交叉编译环境

sudo apt-get install g++-arm-linux-gnueabihf

sudo apt-get install -y gcc-arm-linux-gnueabihf


1.下载Tensorflow源码

git clone https://github.com/tensorflow/tensorflow


2.下载Tensorflow相关依赖包

先cd到Tensorflow工程的根目录，然后执行下面的脚本

./tensorflow/contrib/lite/download_dependencies.sh


3.编译Tensorflow Lite

注意./tensorflow/contrib/lite/build_rpi_lib.sh中的目标编译平台是ARMV7，这里最好不要改，即使你的目标平台是ARMv8。改为ARMv8能也能编译通过，但是貌似没有把优化编译进去，运行起来会非常慢，亲测。

# Licensed under the Apache License, Version 2.0 (the "License");

# you may not use this file except in compliance with the License.

# You may obtain a copy of the License at

#

#     http://www.apache.org/licenses/LICENSE-2.0

#

# Unless required by applicable law or agreed to in writing, software

# distributed under the License is distributed on an "AS IS" BASIS,

# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

# See the License for the specific language governing permissions and

# limitations under the License.

# ==============================================================================

set -e SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

cd "$SCRIPT_DIR/../../.."

#change CC_PREFIX if u need

CC_PREFIX=arm-linux-gnueabihf- make -j 3 -f tensorflow/contrib/lite/Makefile TARGET=RPI TARGET_ARCH=armv7


在根目录下运行该脚本，如下：

./tensorflow/contrib/lite/build_rpi_lib.sh


编译结束，会在tensorflow/contrib/lite/gen/lib/rpi_armv7目录下产生libtensorflow-lite.a静态库。

4.编译label_imageDemo

第三步的build_rpi_lib.sh脚本实际是调用的./tensorflow/contrib/lite/Makefile对Tensorflow Lite源码进行编译，但是该Makefile并不能编译tensorflow/contrib/lite/examples/label_image目录下的Demo，所以需要修改Makefile把label_image的源码配置到Makefile中，修改方式可以参考Makefile里对MINIMAL Demo的配置。如果你不想自己改，下面是已经修改好的。

# Find where we're running from, so we can store generated files here.

ifeq ($(origin MAKEFILE_DIR), undefined)

     MAKEFILE_DIR := $(shell dirname $(realpath $(lastword $(MAKEFILE_LIST)))) 

endif

# Try to figure out the host system

HOST_OS := 

ifeq ($(OS),Windows_NT)

    HOST_OS = WINDOWS

 else

    UNAME_S := $(shell uname -s)

    ifeq ($(UNAME_S),Linux)

            HOST_OS := LINUX

    endif

    ifeq ($(UNAME_S),Darwin)

        HOST_OS := OSX

    endif

 endif

ARCH := $(shell if [[ $(shell uname -m) =~ i[345678]86 ]]; then echo x86_32; else echo $(shell uname -m); fi)

# Where compiled objects are stored.

OBJDIR := $(MAKEFILE_DIR)/gen/obj/

BINDIR := $(MAKEFILE_DIR)/gen/bin/

LIBDIR := $(MAKEFILE_DIR)/gen/lib/

GENDIR := $(MAKEFILE_DIR)/gen/obj/

# Settings for the host compiler.

CXX := $(CC_PREFIX)gcc

CXXFLAGS := --std=c++11 -O3 -DNDEBUG

CC := $(CC_PREFIX)gcc

CFLAGS := -O3 -DNDEBUG

LDOPTS :=

LDOPTS += -L/usr/local/lib

ARFLAGS := -r


INCLUDES := \

 -I. \

 -I$(MAKEFILE_DIR)/../../../ \

 -I$(MAKEFILE_DIR)/downloads/ \

 -I$(MAKEFILE_DIR)/downloads/eigen \

 -I$(MAKEFILE_DIR)/downloads/gemmlowp \

 -I$(MAKEFILE_DIR)/downloads/neon_2_sse \

 -I$(MAKEFILE_DIR)/downloads/farmhash/src \

 -I$(MAKEFILE_DIR)/downloads/flatbuffers/include \

 -I$(GENDIR)

# This is at the end so any globally-installed frameworks like protobuf don't

# override local versions in the source tree.

INCLUDES += -I/usr/local/include

LIBS := \

 -lstdc++ \

 -lpthread \

 -lm \

 -lz

# If we're on Linux, also link in the dl library.

ifeq ($(HOST_OS),LINUX)

    LIBS += -ldl 

endif

include $(MAKEFILE_DIR)/ios_makefile.inc 

include $(MAKEFILE_DIR)/rpi_makefile.inc

# This library is the main target for this makefile. It will contain a minimal

# runtime that can be linked in to other programs.

LIB_NAME := libtensorflow-lite.a

LIB_PATH := $(LIBDIR)$(LIB_NAME)


# A small example program that shows how to link against the library.

MINIMAL_PATH := $(BINDIR)minimal

LABEL_IMAGE_PATH :=$(BINDIR)label_image

MINIMAL_SRCS := \ 

tensorflow/contrib/lite/examples/minimal/minimal.cc

MINIMAL_OBJS := $(addprefix $(OBJDIR), \ 

$(patsubst %.cc,%.o,$(patsubst %.c,%.o,$(MINIMAL_SRCS))))


LABEL_IMAGE_SRCS := \ 

tensorflow/contrib/lite/examples/label_image/label_image.cc \ 

tensorflow/contrib/lite/examples/label_image/bitmap_helpers.cc 

 LABEL_IMAGE_OBJS := $(addprefix $(OBJDIR), \ 

$(patsubst %.cc,%.o,$(patsubst %.c,%.o,$(LABEL_IMAGE_SRCS))))

# What sources we want to compile, must be kept in sync with the main Bazel

# build files.

CORE_CC_ALL_SRCS := \ 

$(wildcard tensorflow/contrib/lite/*.cc) \ 

$(wildcard tensorflow/contrib/lite/kernels/*.cc) \ 

$(wildcard tensorflow/contrib/lite/kernels/internal/*.cc) \ 

$(wildcard tensorflow/contrib/lite/kernels/internal/optimized/*.cc) \ 

$(wildcard tensorflow/contrib/lite/kernels/internal/reference/*.cc) \ 

$(wildcard tensorflow/contrib/lite/*.c) \ 

$(wildcard tensorflow/contrib/lite/kernels/*.c) \ 

$(wildcard tensorflow/contrib/lite/kernels/internal/*.c) \ 

$(wildcard tensorflow/contrib/lite/kernels/internal/optimized/*.c) \ 

$(wildcard tensorflow/contrib/lite/kernels/internal/reference/*.c) \ 

$(wildcard tensorflow/contrib/lite/downloads/farmhash/src/farmhash.cc) \ 

$(wildcard tensorflow/contrib/lite/downloads/fft2d/fftsg.c)


# Remove any duplicates.

CORE_CC_ALL_SRCS := $(sort $(CORE_CC_ALL_SRCS))

CORE_CC_EXCLUDE_SRCS := \ 

$(wildcard tensorflow/contrib/lite/*test.cc) \ 

$(wildcard tensorflow/contrib/lite/*/*test.cc) \ 

$(wildcard tensorflow/contrib/lite/*/*/*test.cc) \ 

$(wildcard tensorflow/contrib/lite/*/*/*/*test.cc) \ 

$(wildcard tensorflow/contrib/lite/kernels/test_util.cc) \ 

$(MINIMAL_SRCS) \ 

$(LABEL_IMAGE_SRCS)


# Filter out all the excluded files.

TF_LITE_CC_SRCS := $(filter-out $(CORE_CC_EXCLUDE_SRCS), $(CORE_CC_ALL_SRCS))

# File names of the intermediate files target compilation generates.

TF_LITE_CC_OBJS := $(addprefix $(OBJDIR), \ 

$(patsubst %.cc,%.o,$(patsubst %.c,%.o,$(TF_LITE_CC_SRCS))))

LIB_OBJS := $(TF_LITE_CC_OBJS)


# For normal manually-created TensorFlow C++ source files.

$(OBJDIR)%.o: %.cc

    @mkdir -p $(dir $@)

    $(CXX) $(CXXFLAGS) $(INCLUDES) -c $< -o $@

# For normal manually-created TensorFlow C++ source files.

$(OBJDIR)%.o: %.c

    @mkdir -p $(dir $@)

    $(CC) $(CCFLAGS) $(INCLUDES) -c $< -o $@

# The target that's compiled if there's no command-line arguments.

all: $(LIB_PATH)  $(MINIMAL_PATH) $(LABEL_IMAGE_PATH)

# Gathers together all the objects we've compiled into a single '.a' archive.

$(LIB_PATH): $(LIB_OBJS)

    @mkdir -p $(dir $@)

    $(AR) $(ARFLAGS) $(LIB_PATH) $(LIB_OBJS) 

 $(MINIMAL_PATH): $(MINIMAL_OBJS) $(LIB_PATH)

     @mkdir -p $(dir $@)

    $(CXX) $(CXXFLAGS) $(INCLUDES) \

    -o $(MINIMAL_PATH) $(MINIMAL_OBJS) \

    $(LIBFLAGS) $(LIB_PATH) $(LDFLAGS) $(LIBS) 

 $(LABEL_IMAGE_PATH): $(LABEL_IMAGE_OBJS) $(LIB_PATH)

    @mkdir -p $(dir $@)

    $(CXX) $(CXXFLAGS) $(INCLUDES) \

    -o $(LABEL_IMAGE_PATH) $(LABEL_IMAGE_OBJS) \

    $(LIBFLAGS) $(LIB_PATH) $(LDFLAGS) $(LIBS)


# Gets rid of all generated files.clean:

    rm -rf $(MAKEFILE_DIR)/gen

# Gets rid of target files only, leaving the host alone. Also leaves the lib

# directory untouched deliberately, so we can persist multiple architectures

# across builds for iOS and Android.cleantarget:

    rm -rf $(OBJDIR)

    rm -rf $(BINDIR) 

 $(DEPDIR)/%.

d: ; .PRECIOUS: $(DEPDIR)/%.d 

 -include $(patsubst %,$(DEPDIR)/%.d,$(basename $(TF_CC_SRCS)))


修改完成后再次执行./tensorflow/contrib/lite/build_rpi_lib.sh，此时在./tensorflow/contrib/lite/gen/bin/rpi_armv8目录下会产生编译好的label_image二进制文件。

5.拷贝程序到板子上

准备测试图片tensorflow/contrib/lite/examples/label_image/testdata/grace_hopper.bmp，当然用其他的图片测试也行。此外，还需要从https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/g3doc/models.md 地址下载你想要测试的tf lite模型。还需要准备ImageNet的标签文件。最后需要传到板子上的是如下几个文件。

grace_hopper.bmp

label_image 

labels.txt

mobilenet_v1_1.0_224_quant.tflite/mobilenet_v1_1.0_224.tflite

  • 6.在ARM板子上运行Tensorflow Lite

到此准备工作全部完成了，最后可以在板子上测试Tensorflow Lite的性能了，使用姿势如下：

./label_image -v 1 -m ./mobilenet_v1_1.0_224.tflite  -i ./grace_hopper.bmp -l ./labels.txt


运行效果如下：

average time: 855.91 ms 

0.860174: 653 military uniform

0.0481021: 907 Windsor tie

0.00786706: 466 bulletproof vest

0.00644932: 514 cornet

0.00608028: 543 drumstick


接下来可以再测试下量化后的MobileNet效果：

./label_image -v 4 -m ./mobilenet_v1_1.0_224_quant.tflite  -i ./grace_hopper.bmp -l ./labels.txt


效果如下：

average time: 185.988 ms 

0.427451: 653 military uniform

0.305882: 907 Windsor tie

0.0431373: 668 mortarboard

0.0313726: 458 bow tie

0.0235294: 543 drumstick


从性能上看，如果不考虑识别的准确率的话，TF Lite量化后的性能收益还是很不错的。在同等平台下，NCNN跑Mobilenet的耗时290ms左右。所以在非量化的情况下NCNN的性能还是强于TF Lite


如果觉得我的文章对您有用，请随意打赏。您的支持将鼓励我继续创作！

¥ 打赏支持
91人赞 举报
分享到
用户评价(0)

暂无评价，你也可以发布评价哦:)

3 人收藏了这篇文章
个人头像 个人头像 个人头像

相关文章

腾讯云数据库性能卓越稳定可靠，为您解决数据库运维难题
广告

关于我们

公司介绍

人才招聘

联系方式

法律声明

服务条款

权利声明

隐私声明

帮助中心

使用攻略

建议反馈

联系客服

网站地图

二维码
JobPlus官方公众号
微博
JobPlus官方微博
Copyright © jobplus.com.cn All rights reserved
苏州聘十信息技术有限公司 版权所有苏ICP备16046992号
扫码APP

扫描使用APP

扫码使用

扫描使用小程序