從頭開始配置編譯python3.6版本的caffe整整花了10天時間,期間經歷了很多事,所以狀態一直很差,真正的配置時間應該在五天左右.

目前搜索到的caffe配置版本多數是2.7和3.5,關于python3.6的配置基本沒有.但是目前anaconda3用的python版本為3.6,tensorflow最新使用的都是3.6,所以我電腦里存在的就是3.6版本.所以我決定配置python3.6版本的caffe.

step by step!

1,配置opencv 3.4.1

我們需要達到的目標是在python3中可以直接使用import cv2,其中關于編譯這方面的資料很多.主要分為編譯本體庫,以及編譯一個附加下載的庫.但是在編譯完成之后發現,依然無法成功import cv2.程序中的Import cv2提示no modules named cv2錯誤

解決方法:sudo pip3 install opencv-python.注意這里一定要用pip3,之前一直有資料說用pip,這樣是無法成功的.使用pip3后,直接成功.

2,編譯caffe

這里的大前提是已經編譯配置好anaconda3,python的版本為3.6.python3.6的bin和lib文件全部在anaconda3中.

這里主要參考的是這篇博文.https://blog.csdn.net/isuccess88/article/details/70165726,但也有很多不同,他那篇是針對python3.5的.

2.1 安裝依賴

sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev protobuf-compiler

sudo apt-get install --no-install-recommends libboost-all-dev

sudo apt-get install libatlas-base-dev

sudo apt-get install libhdf5-serial-dev

sudo apt-get install libgflags-dev libgoogle-glog-dev liblmdb-dev

然后把CAFFE的源代碼下載下來: git clone https://github.com/BVLC/caffe.git

(當然沒有安裝GIT的得先安裝一下)

下載完成之后,進入CAFFE文件夾, 進入里面的PYTHON文件夾,然后輸入

for req in $(cat requirements.txt); do pip3 install $req; done

(PIP如果沒有安裝得先安裝一下:sudo apt install python-pip)

注意的地方是:pip3安裝的,第二點是sudo apt-get install --no-install-recommends libboost-all-dev這一條安裝編譯的并沒有關于python3.6版本的libboost,我們需要重新編譯.否則caffe無法調用成功.(怎么編譯后面說)

2.2 安裝caffe

到CAFFE文件夾, 使用模板寫個Makefile.config. 具體就是先復制一下模板, 再改一些內容(我喜歡用EMACS).

cp Makefile.config.example Makefile.config

-因為CPU MODE, 所以在CPU_ONLY := 1前面的#要去掉.

-兩個路徑要改成這樣:(添加后面的兩個hdf5的路徑, 否則編譯時報hdf5錯誤)

# Whatever else you find you need goes here.

INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include/usr/include/hdf5/serial

LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib/usr/lib/x86_64-linux-gnu/hdf5/serial

一份完整的python3.6配置文件.（這里黑色加粗字體的位置就是我們要做修改注意的地方）

## Refer to http://caffe.berkeleyvision.org/installation.html

# Contributions simplifying and improving our build system are welcome!

# cuDNN acceleration switch (uncomment to build with cuDNN).

# USE_CUDNN := 1

# CPU-only switch (uncomment to build without GPU support).

CPU_ONLY := 1#（因為我們要配置cpu版本的，所以這里把引號去掉）

# uncomment to disable IO dependencies and corresponding data layers

# USE_OPENCV := 0

# USE_LEVELDB := 0

# USE_LMDB := 0

# uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary)

# You should not set this flag if you will be reading LMDBs with any

# possibility of simultaneous read and write

# ALLOW_LMDB_NOLOCK := 1

# Uncomment if you're using OpenCV 3

OPENCV_VERSION := 3#（我們的opencv是3版本的，所以這里我們把引號去掉，之前opencv3已經安裝并編譯成功）

# To customize your choice of compiler, uncomment and set the following.

# N.B. the default for Linux is g++ and the default for OSX is clang++

# CUSTOM_CXX := g++

# CUDA directory contains bin/ and lib/ directories that we need.

CUDA_DIR := /usr/local/cuda

# On Ubuntu 14.04, if cuda tools are installed via

# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:

# CUDA_DIR := /usr

# CUDA architecture setting: going with all of them.

# For CUDA < 6.0, comment the *_50 through *_61 lines for compatibility.

# For CUDA < 8.0, comment the *_60 and *_61 lines for compatibility.

# For CUDA >= 9.0, comment the *_20 and *_21 lines for compatibility.

CUDA_ARCH := -gencode arch=compute_20,code=sm_20 \

-gencode arch=compute_20,code=sm_21 \

-gencode arch=compute_30,code=sm_30 \

-gencode arch=compute_35,code=sm_35 \

-gencode arch=compute_50,code=sm_50 \

-gencode arch=compute_52,code=sm_52 \

-gencode arch=compute_60,code=sm_60 \

-gencode arch=compute_61,code=sm_61 \

-gencode arch=compute_61,code=compute_61

# BLAS choice:

# atlas for ATLAS (default)

# mkl for MKL

# open for OpenBlas

BLAS := atlas

# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.

# Leave commented to accept the defaults for your choice of BLAS

# (which should work)!

# BLAS_INCLUDE := /path/to/your/blas

# BLAS_LIB := /path/to/your/blas

# Homebrew puts openblas in a directory that is not on the standard search path

# BLAS_INCLUDE := $(shell brew --prefix openblas)/include

# BLAS_LIB := $(shell brew --prefix openblas)/lib

# This is required only if you will compile the matlab interface.

# MATLAB directory should contain the mex binary in /bin.

# MATLAB_DIR := /usr/local

# MATLAB_DIR := /Applications/MATLAB_R2012b.app

# NOTE: this is required only if you will compile the python interface.

# We need to be able to find Python.h and numpy/arrayobject.h.

#PYTHON_INCLUDE := /usr/include/python2.7 \

# /usr/lib/python2.7/dist-packages/numpy/core/include

# Anaconda Python distribution is quite popular. Include path:

# Verify anaconda location, sometimes it's in root.

ANACONDA_HOME := $(HOME)/anaconda3#（我們使用的python3.6版本在anaconda3中，所以這里要取消注釋，修改為你的根目錄位置）

PYTHON_INCLUDE := $(ANACONDA_HOME)/include \

$(ANACONDA_HOME)/include/python3.6m \#（這兩個目錄要看具體anaconda3中目錄的名字，我做了對應修改）

$(ANACONDA_HOME)/lib/python3.6/site-packages/numpy/core/include

# Uncomment to use Python 3 (default is Python 2)

? PYTHON_LIBRARIES := boost_python3 python3.6m#（這里我取消了注釋，并且修改了內容，重新編譯boost python3，后面講）

# PYTHON_INCLUDE := /usr/include/python3.5m \

#? ? ? ? ? ? ? ? /usr/lib/python3.5/dist-packages/numpy/core/include

# We need to be able to find libpythonX.X.so or .dylib.

#PYTHON_LIB := /usr/lib#（把這里注釋掉）

PYTHON_LIB := $(ANACONDA_HOME)/lib#（修改為我這里的位置）

# Homebrew installs numpy in a non standard path (keg only)

# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include

# PYTHON_LIB += $(shell brew --prefix numpy)/lib

# Uncomment to support layers written in Python (will link against Python libs)

# WITH_PYTHON_LAYER := 1

# Whatever else you find you need goes here.

INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include? /usr/include/hdf5/serial

LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu/hdf5/serial

# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies

# INCLUDE_DIRS += $(shell brew --prefix)/include

# LIBRARY_DIRS += $(shell brew --prefix)/lib

# NCCL acceleration switch (uncomment to build with NCCL)

# https://github.com/NVIDIA/nccl (last tested version: v1.2.3-1+cuda8.0)

# USE_NCCL := 1

# Uncomment to use `pkg-config` to specify OpenCV library paths.

# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)

# USE_PKG_CONFIG := 1

# N.B. both build and distribute dirs are cleared on `make clean`

BUILD_DIR := build

DISTRIBUTE_DIR := distribute

# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171

# DEBUG := 1

# The ID of the GPU that 'make runtest' will use to run unit tests.

TEST_GPUID := 0

# enable pretty build (comment to see full commands)

Q ?= @

LINKFLAGS := -Wl,-rpath,$(HOME)/anaconda3/lib#（這里是由于會出現一個關于png的錯誤，我后面講）

按照流程接下來的配置應該是：

準備好了.

make pycaffe -j 4

另外, 這個make默認是用CPU單核運算,如果想要快一點, 比如我想使用四核, 在make后面加上-j4標簽.

如果上面4行某一行報錯之后想要重試,建議先make clean再重新開始.

如果這里出現問題，馬上 make clean.

配置好了之后 ，make all?

全部成功后才算是成功。

2.3 配置環境變量

去到CAFFE文件夾里面的python文件夾, 把當前路徑記錄下來(pwd). 然后輸入以下命令(把記下的路徑放在相應地方)

export PYTHONPATH=/path/to/caffe/python:$PYTHONPATH

2.4 大功告成

命令行中輸入 python， 然后 import caffe ,基本是錯的。接下來講在編譯caffe以及import caffe中出現的各種問題。

3.問題以及解決

3.1 編譯時候，cannot find -lboost_python3, python3.6m

原因是這一行中，PYTHON_LIBRARIES := boost_python3 python3.6m，首先我們boost并沒有編譯3.6版本，其次python3.6m這個link不在 /usr/lib 。

1)，sudo ln -s /home/horsetif/anaconda3/bin/python3? /usr/lib/python3.6m(這里是建立一個軟鏈接，把我本地目錄下的python3.6加到usr/lib/中去。

2)，編譯3.6版本的libboost_python3.so

首先去/usr/lib/x86_64-linux-gnu目錄下查看是否有python3版本的libboost，如果有類似libboost_python35.so但是沒有libboost_python3.so則需要手動建立連接。

方法為：

sudo ln-slibboost_python-py35.so libboost_python3.so

這個方法只是針對與3.5版本，現在我們重新編譯，然后把編譯好的結果的軟鏈接加到/usr/lib/x86_64-linux-gnu這個目錄里面。

1.首先下載安裝包，然后解壓， 切換目錄

wget?-O?boost_1_67_0.tar.gz?http://sourceforge.net/projects/boost/files/boost/1.67.0/boost_1_67_0.tar.gz/download??

tar?xzvf?boost_1_67_0.tar.gz? ? ??

cd?boost_1_67_0/??

安裝包也可以在http://www.boost.org/users/history/version_1_67_0.html下載

2.安裝附加依賴庫

sudo?apt-get?update??

sudo?apt-get?install?build-essential?g++?python-dev?autotools-dev?libicu-dev?build-essential?libbz2-dev?libboost-all-dev?

linux系統自帶python2.7，boost編譯后會生成libboost_python27.so，但有些應用需要libboost_python3，下面我們就來編譯。

進入boost源文件目錄boost_1_67_0，配置僅僅編譯python，python路徑指向anaconda3中的python3.6。

./bootstrap?--with-libraries=python?--with-toolset=gcc??

./b2?--with-python?include="/home/horsetif/anaconda3/include/python3.6m/"??

sudo?./b2?install??

編譯安裝成功后，/usr/local/lib下會有libboost_python36.so和libboost_python36.a，有些應用link時需要的是libboost_python3.so或者libboost_python3.a，我們建個軟鏈：

cd?/usr/local/lib??

sudo?ln?-s?libboost_python-py36.so?libboost_python3.so??

sudo?ln?-s?libboost_python-py36.a?libboost_python3.a??

這時，我們要把/usr/local/lib 中，相關文件，建立相對于名稱的軟鏈接到 /usr/lib/x86_64-linux-gnu中。

sudo cp /usr/local/lib/libboost_python36.a? /usr/lib/x86_64-linux-gnu/libboost_python_python36.a

sudo cp /usr/local/lib/libboost_python36.so.1.67.0? /usr/lib/x86_64-linux-gnu/libboost_python3.so

3.2 多個opencv問題

錯誤：make pycaffe可以通過，但是make all時出現錯誤，具體錯誤我沒有記錄，但是好像是因為系統裝了多個opencv的原因，不管怎么說，至少可以在python環境下import caffe了

make all時出現的錯誤：

.build_release/lib/libcaffe.so: undefined reference to `cv::imread(cv::String const&, int)'

.build_release/lib/libcaffe.so: undefined reference to `cv::imencode(cv::String const&, cv::_InputArray const&, std::vector >&, std::vector > const&)'

.build_release/lib/libcaffe.so: undefined reference to `cv::imdecode(cv::_InputArray const&, int)'

collect2: error: ld returned 1 exit status

Makefile:625: recipe for target '.build_release/tools/upgrade_net_proto_text.bin' failed

make: *** [.build_release/tools/upgrade_net_proto_text.bin] Error 1

make: *** Waiting for unfinished jobs....

解決辦法：也是，的確在這之前我分別在python 3.5和python 2.7下面裝了opencv，而且該教程前面又裝了libopencv-dev，所以我就懷疑是這個，但是我嘗試著卸載了opencv還是import出現同樣的錯誤，最后不斷找解決方案，也嘗試了他們的各種解決方法，都不知道make clean然后又make all了多少次，還是不管用。最后在網站https://github.com/BVLC/caffe/issues/2348#issuecomment-95156848看到有人說”Uncomment

if you're using OpenCV 3”于是我在配置文件中去掉 ”OPENCV_ VERSION :=3“的注釋，我又make clean，接著make all,奇跡終于出現了，上面的錯誤消失了,居然make all成功了。

3.3?matplotlib 問題

3、錯誤：現在可以import caffe了，但是又出現了以下錯誤：

>>> import caffe

Traceback (most recent call last):

File "", line 1, in

File "/home/.../Downloads/caffe-master/python/caffe/__init__.py", line 1, in

from .pycaffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, RMSPropSolver, AdaDeltaSolver, AdamSolver, NCCL, Timer

File "/home/.../Downloads/caffe-master/python/caffe/pycaffe.py", line 15, in

import caffe.io

File "/home/..../Downloads/caffe-master/python/caffe/io.py", line 2, in

import skimage.io

File "/home/.../Downloads/yes/lib/python3.5/site-packages/skimage/io/__init__.py", line 15, in

reset_plugins()

File "/home/.../Downloads/yes/lib/python3.5/site-packages/skimage/io/manage_plugins.py", line 93, in reset_plugins

_load_preferred_plugins()

File "/home/.../Downloads/yes/lib/python3.5/site-packages/skimage/io/manage_plugins.py", line 73, in _load_preferred_plugins

_set_plugin(p_type, preferred_plugins['all'])

File "/home/.../Downloads/yes/lib/python3.5/site-packages/skimage/io/manage_plugins.py", line 85, in _set_plugin

use_plugin(plugin, kind=plugin_type)

File "/home/.../Downloads/yes/lib/python3.5/site-packages/skimage/io/manage_plugins.py", line 255, in use_plugin

_load(name)

File "/home/.../Downloads/yes/lib/python3.5/site-packages/skimage/io/manage_plugins.py", line 299, in _load

fromlist=[modname])

File "/home/.../Downloads/yes/lib/python3.5/site-packages/skimage/io/_plugins/matplotlib_plugin.py", line 3, in

import matplotlib.pyplot as plt

File "/home/.../Downloads/yes/lib/python3.5/site-packages/matplotlib/pyplot.py", line 36, in

from matplotlib.figure import Figure, figaspect

File "/home/.../Downloads/yes/lib/python3.5/site-packages/matplotlib/figure.py", line 40, in

from matplotlib.axes import Axes, SubplotBase, subplot_class_factory

File "/home/..../Downloads/yes/lib/python3.5/site-packages/matplotlib/axes/__init__.py", line 4, in

from ._subplots import *

File "/home/.../Downloads/yes/lib/python3.5/site-packages/matplotlib/axes/_subplots.py", line 10, in

from matplotlib.axes._axes import Axes

File "/home/.../Downloads/yes/lib/python3.5/site-packages/matplotlib/axes/_axes.py", line 23, in

import matplotlib.dates as _? # <-registers a date unit converter

File "/home/.../Downloads/yes/lib/python3.5/site-packages/matplotlib/dates.py", line 126, in

from dateutil.rrule import (rrule, MO, TU, WE, TH, FR, SA, SU, YEARLY, MONTHLY, WEEKLY, DAILY, HOURLY, MINUTELY, SECONDLY)

File "/home/.../Downloads/yes/lib/python3.5/site-packages/dateutil/rrule.py", line 55

raise ValueError, "Can't create weekday with n == 0"

^

SyntaxError: invalid syntax

錯誤解決辦法：從上面可以看出也就是在? import matplotlib.pyplot as plt 時出現錯誤，百度了以下，嘗試了幾種解決方案，原來是matplotlib版本太低了，需要升級，于是我pip3 install matplotlib --upgrade 了以下，果然解決了，但是又出現了下面的問題。

3.4 numpy 問題

錯誤：Intel MKL FATAL ERROR: Cannot load libmkl_avx.so or libmkl_def.so.

Python 3.5.2 |Continuum Analytics, Inc.| (default, Jul? 2 2016, 17:53:06)

[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux

Type "help", "copyright", "credits" or "license" for more information.

>>> import caffe

Intel MKL FATAL ERROR: Cannot load libmkl_avx.so or libmkl_def.so.

解決方案：繼續百度了下，哈哈，原來是因為numpy版本過低導致，于是我 conda update numpy ，哈哈又成功了。。所以，我的anaconda里邊的包必須得升級啊，里邊的包都還是安裝anaconda時系統自帶的，太老了。

3.5 錯誤信息

//home/idc/anaconda3/lib/libpng16.so.16: undefined reference to `inflateValidate@ZLIB_1.2.9'

后面還有幾行，主要是這個。

我的解決方法是在Makefile.config加入下列信息：

LINKFLAGS := -Wl,-rpath,$(HOME)/anaconda3/lib

3.6 錯誤信息

可能會出現下面的錯誤：

Failed to include caffe_pb2, things might go wrong!

Traceback (most recent call last):

? File "", line 1, in

? File "/home/david/download/caffe-master/python/caffe/__init__.py", line 4, in

? ? from .proto.caffe_pb2 import TRAIN, TEST

? File "/home/david/download/caffe-master/python/caffe/proto/caffe_pb2.py", line 7, in

? ? from google.protobuf import reflection as _reflection

? File "/home/david/opt/anaconda3/lib/python3.5/site-packages/google/protobuf/reflection.py", line 68, in

? ? from google.protobuf.internal import python_message

? File "/home/david/opt/anaconda3/lib/python3.5/site-packages/google/protobuf/internal/python_message.py", line 848

? ? except struct.error, e:

? ? ? ? ? ? ? ? ? ? ? ^

SyntaxError: invalid syntax

這是因為 protobuf 不支持 python3，解決方案是安裝pip install protobuf-py3, 一個 python3.x 版本的替代包。‘’

4.錯誤代碼總匯：

opencv 配置問題

make all

.build_release/lib/libcaffe.so: undefined reference to `cv::imencode(cv::String const&, cv::_InputArray const&, std::vector>&, std::vector > const&)'

.build_release/lib/libcaffe.so: undefined reference to `cv::imdecode(cv::_InputArray const&, int)'

collect2: error: ld returned 1 exit status

Makefile:625: recipe for target '.build_release/tools/compute_image_mean.bin' failed

make: *** [.build_release/tools/compute_image_mean.bin] Error 1

python版本與編譯caffe版本問題

from ._caffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, \

ImportError: /home/horsetif/Software/caffe/python/caffe/../../build/lib/libcaffe.so.1.0.0: undefined symbol: _ZN2cv6imreadERKNS_6StringEi

File "/home/horsetif/Software/caffe/python/caffe/pycaffe.py", line 13, in from ._caffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, \

ImportError: dynamic module does not define module export function (PyInit__caffe)

boost 配置問題

CXX/LD -o python/caffe/_caffe.so python/caffe/_caffe.cpp

/usr/bin/ld: cannot find -lboost_python3

collect2: error: ld returned 1 exit status

Makefile:507: recipe for target 'python/caffe/_caffe.so' failed

make: *** [python/caffe/_caffe.so] Error 1

boost修改好了之后沒有放對位置

File "/home/horsetif/Software/caffe/python/caffe/pycaffe.py", line 13, in from ._caffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, \

ImportError: libboost_python36.so.1.67.0: cannot open shared object file: No such file or directory

matplotlib 問題

from matplotlib.axes._axes import Axes File "/home/horsetif/anaconda3/lib/python3.6/site-packages/matplotlib/axes/_axes.py", line 23, inimport matplotlib.dates as _ # <-registers a date unit converter File "/home/horsetif/anaconda3/lib/python3.6/site-packages/matplotlib/dates.py", line 148, in from dateutil.rrule import (rrule, MO, TU, WE, TH, FR, SA, SU, YEARLY,

? File "/home/horsetif/anaconda3/lib/python3.6/site-packages/dateutil/rrule.py", line 55

? ? raise ValueError, "Can't create weekday with n == 0"

? ? ? ? ? ? ? ? ? ? ^

SyntaxError: invalid syntax