乐视深度相机使用指南

本文使用的是乐视的三合一体感摄像头，型号为LeTMC-520，本质上就是Astra Pro摄像头，官方提供两种SDK：

  1. Astra SDK：用于提供包含骨架识别的SDK包，但是有时间限制（骨架识别为30分钟非商用版本），官方有提供[API指南](Astra SDK: API (orbbec.com.cn))
  2. OpenNI2 SDK：提供基本数据流的SDK包和相关工具

本文所使用的环境是：Astra SDK+Ubuntu20.04+VSC

一、环境搭建

从官网下载Astra SDK，这里选的是Linux版本的，之后通过如下方式（具体可以参考/samples/readme.txt以及/install下的两个脚本）进行安装：

1
2
3
4
5
6
7
8
# 解压
tar -xzf ../AstraSDK-v2.1.1-24f74b8b15-20200426T012828Z-Ubuntu16.04-x86_64.tar.gz
mv AstraSDK-v2.1.1-24f74b8b15-20200426T012828Z-Ubuntu16.04-x86_64 AstraSDK-v2.1.1
# 安装必备工具
sudo apt install cmake libsfml-dev pkg-config
# 执行修改摄像头权限脚本
chmod +x ./install/install.sh
sudo ./install/install.sh

  • 如果上述install.sh脚本执行完毕后，打开摄像头仍需要root权限，则需要重启下

  • 【可选】建议在.bash_profile或.bashrc下直接添加该SDK的环境变量：

    1
    2
    export ASTRA_SDK_INCLUDE=/home/null/Code/DepthCamera/AstraSDK/AstraSDK-v2.1.1/include
    export ASTRA_SDK_LIB=/home/null/Code/DepthCamera/AstraSDK/AstraSDK-v2.1.1/lib

上述步骤后即可直接运行相对的例程（SDK中/samples目录下）：

  1. 了解相关的例程目录（相关说明名单详见Astra SDK 代码示例）

    1. /samples/sfml：
    2. /samples/cpp-api：C++例程
    3. /samples/c-api：C例程

  2. 执行（需要在SDK中/samples目录下即可）：

    1
    2
    3
    4
    5
    # 直接在当前目录下构建工程
    cmake .
    # 重新建个文件夹构建工程
    mkdir build && cd build
    cmake ..

  3. 例程将会在bin/下根据CMakeList.txt生成可执行文件

由于本人不太喜欢直接在原来的SDK下修改，所以这里本人并没有采用上述方式来编译，而是通过如下方式：

  1. 将SDK下/samples目录下单独拷贝出来

  2. 通过设置如下环境变量：

    1
    2
    3
    4
    5
    export ASTRA_SDK_INCLUDE=/home/null/Code/DepthCamera/AstraSDK/AstraSDK-v2.1.1/include
    export ASTRA_SDK_LIB=/home/null/Code/DepthCamera/AstraSDK/AstraSDK-v2.1.1/lib

    export ASTRA_ROOT=/home/null/Code/DepthCamera/AstraSDK/AstraSDK-v2.1.1/include
    export PATH=$PATH:/home/null/Code/DepthCamera/AstraSDK/AstraSDK-v2.1.1/lib

  3. 在刚刚拷贝出来的/samples目录下直接运行：

    1
    2
    3
    4
    5
    6
    # 构建目录
    mkdir build && cd build
    # 构建工程 生成makefile
    cmake ..
    # 编译
    make

  4. 此时整个工程的构建目录在/samples/build下，最后的可执行文件在/samples/build/bin下

从之后开始，所有目录如果没有明确说明，默认都是在/samples下

二、第一个工程hello world

  1. 到cpp-api目录下，找到CMakeLists.txt，并增加自己的工程：

    1
    add_subdirectory(helloworld)

  2. 然后到同级目录下，随便找个工程（如：ColorReaderEventCPP），然后直接重命名文件夹为刚刚添加的工程名helloworld，并修改该工程下的CMakeList.txt中第一句为：

    1
    set (_projname "helloworld")

  3. 去build文件夹，执行编译命令（记得提前设置环境变量）

    1
    cmake .. && make

  4. 去build/bin/下执行生成的可执行文件

    1
    ./helloworld

轮询模板

cmake：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
set (_projname "xxxx-SFML")

set (${_projname}_SOURCES
${SAMPLE_COMMON_DIR}/LitDepthVisualizer.hpp
main.cpp
${SAMPLE_COMMON_DIR}/key_handler.h
)

find_package(SFML COMPONENTS graphics window system)

add_executable(${_projname} ${${_projname}_SOURCES})

set_target_properties(${_projname} PROPERTIES FOLDER "${SAMPLE_DIR_FOLDER}sfml")

include_directories(${SFML_INCLUDE_DIR})

target_link_libraries(${_projname} ${ASTRA_LIBRARIES} ${SFML_LIBRARIES})

set_debug_working_dir(${_projname} "$(TargetDir)")

add_astra_sdk_dependency(${_projname})

install_sample(${_projname})

main：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
int main(int argc, char** argv)
{
    astra::initialize();

    astra::StreamSet streamSet;
    astra::StreamReader reader = streamSet.create_reader();

    reader.stream<astra::DepthStream>().start();
    //Stores the maximum number of frames we're going to process in the loop
    const int maxFramesToProcess = 100;
    //Sentinel to count the number of frames that we've processed
    int count = 0;

    //The frame processing loop
    do {
        astra::Frame frame = reader.get_latest_frame();
        const auto depthFrame = frame.get<astra::DepthFrame>();

        const int frameIndex = depthFrame.frame_index();
        const short pixelValue = depthFrame.data()[0];

        std::cout << std::endl
            << "Depth frameIndex: " << frameIndex
            << " pixelValue: " << pixelValue
            << std::endl
            << std::endl;

        count++;
    } while (count < maxFramesToProcess);

    std::cout << "Press any key to continue...";
    std::cin.get();

    astra::terminate();

    std::cout << "hit enter to exit program" << std::endl;
    std::cin.get();

    return 0;
}

监听模板

cmake：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
set (_projname "xxxx-SFML")

set (${_projname}_SOURCES
${SAMPLE_COMMON_DIR}/LitDepthVisualizer.hpp
main.cpp
${SAMPLE_COMMON_DIR}/key_handler.h
)

find_package(SFML COMPONENTS graphics window system)

add_executable(${_projname} ${${_projname}_SOURCES})

set_target_properties(${_projname} PROPERTIES FOLDER "${SAMPLE_DIR_FOLDER}sfml")

include_directories(${SFML_INCLUDE_DIR})

target_link_libraries(${_projname} ${ASTRA_LIBRARIES} ${SFML_LIBRARIES})

set_debug_working_dir(${_projname} "$(TargetDir)")

add_astra_sdk_dependency(${_projname})

install_sample(${_projname})

main：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
class DepthFrameListener : public astra::FrameListener
{
    public:
    //Constructor parameter specifies the total number of frames we're going to process before exiting our loop
    DepthFrameListener(int maxFramesToProcess)
        : maxFramesToProcess_(maxFramesToProcess)
        {}
    //is_finished will be used in a later step to check whether we've looped the maximum number of times or not
    bool is_finished() const { return isFinished_; }

    private:
    void on_frame_ready(astra::StreamReader& reader,
                        astra::Frame& frame) override
    {
        //Gets the depth frame data from our frame
        const astra::DepthFrame depthFrame = frame.get<astra::DepthFrame>();

        //Check to verify that we received a valid frame
        if (depthFrame.is_valid())
        {
            //Prints depth frame information to the console
            print_depth_frame(depthFrame);
            ++framesProcessed_;
        }

        isFinished_ = framesProcessed_ >= maxFramesToProcess_;
    }

    void print_depth_frame(const astra::DepthFrame& depthFrame) const
    {
        const int frameIndex = depthFrame.frame_index();
        const short middleValue = get_middle_value(depthFrame);

        std::printf("Depth frameIndex: %d value: %d \n", frameIndex, middleValue);
    }

    short get_middle_value(const astra::DepthFrame& depthFrame) const
    {
        const int width = depthFrame.width();
        const int height = depthFrame.height();

        //Calculates the index of the middle pixel in our depth frame's data
        const size_t middleIndex = ((width * (height / 2.f)) + (width / 2.f));

        const short* frameData = depthFrame.data();
        //Gets the value of the middle depth frame pixel
        const short middleValue = frameData[middleIndex];

        return middleValue;
    }

    bool isFinished_{false};
    int framesProcessed_{0};
    int maxFramesToProcess_{0};
};

int main(int argc, char** argv)
{
    astra::initialize();

    astra::StreamSet streamSet;
    astra::StreamReader reader = streamSet.create_reader();

    reader.stream<astra::DepthStream>().start();

    const int maxFramesToProcess = 100;
    DepthFrameListener listener(maxFramesToProcess);

    reader.add_listener(listener);

    //The Astra SDK update loop.
    do {
        astra_update();
    } while (!listener.is_finished());

    reader.remove_listener(listener);

    astra::terminate();

    return 0;
}