从点 Cloud 生成深度图-Java 学习之路

我正在尝试从点 Cloud 生成深度图 . 我知道我可以将点 Cloud 投影到图像平面，但是在TangoSupport脚本中已经存在一个函数（ScreenCoordinateToWorldNearestNeighbor），该函数在给定屏幕坐标的情况下找到XYZ点 .

我无法使用此支持功能，似乎我的一个或多个输入无效 . 我正在OnTangoDepthAvailable事件中更新我的深度图纹理 .

public void OnTangoDepthAvailable(TangoUnityDepth tangoDepth)
{
    _depthAvailable = true;
    Matrix4x4 ccWorld = _Camera.transform.localToWorldMatrix;
    bool isValid = false;
    Vector3 colorCameraPoint = new Vector3();
    for (int i = 0; i < _depthMapSize; i++)
    {
        for (int j = 0; j < _depthMapSize; j++)
        {
            if (TangoSupport.ScreenCoordinateToWorldNearestNeighbor(
                _PointCloud.m_points, _PointCloud.m_pointsCount,
                tangoDepth.m_timestamp, 
                _ccIntrinsics,
                ref ccWorld, 
                new Vector2(i / (float)_depthMapSize, j / (float)_depthMapSize),
                out colorCameraPoint, out isValid) == Common.ErrorType.TANGO_INVALID)
            {
                _depthTexture.SetPixel(i, j, Color.red);
                continue;
            }

            if (isValid)
            {
                //_depthTexture.SetPixel(i, j, new Color(colorCameraPoint.z, colorCameraPoint.z, colorCameraPoint.z));
                _depthTexture.SetPixel(i, j,
                    new Color(0,UnityEngine.Random.value,0));
            }
            else
            {
                _depthTexture.SetPixel(i, j, Color.white);
            }
        }
    }
    _depthTexture.Apply();
    _DepthMapQuad.material.mainTexture = _depthTexture;
}

如果我不得不猜测，我会说我传递的是错误的矩阵（ ccWorld ） . 以下是矩阵参数文档中的内容：

彩色相机相对于Unity世界框架的变换矩阵 .

结果是一个白色深度图，这意味着该函数成功返回，但isValid为false意味着它在投影后无法找到任何附近的点 Cloud 点 .

有任何想法吗？另外我注意到即使我的深度图是8x8，性能也非常糟糕 . 当有新的深度数据可用时（OnTangoDepthAvailable内），我不应该更新深度图吗？

Edit ：我能够使函数成功返回，但是现在它在投影后没有找到附近的点 Cloud 点 . 生成的深度图始终为白色 . 我打印出所有的参数，一切看起来都正确，所以我认为我传递的是错误的矩阵 .

1 回答

您应该更新SDK和Project Tango Dev Kit . 这是获取深度图 on Android 的示例，也许您会得到 unity 的提示：

public class MainActivity extends AppCompatActivity {

    private Tango mTango;
    private TangoConfig mTangoConfig;
    private TangoPointCloudManager mPointCloudManager;
    private AtomicBoolean tConnected = new AtomicBoolean(false);
    Random rand = new Random();
    private ImageView imageDepthMap;


    private static final ArrayList<TangoCoordinateFramePair> framePairs = new ArrayList<TangoCoordinateFramePair>();

    {
        framePairs.add(new TangoCoordinateFramePair(
                TangoPoseData.COORDINATE_FRAME_CAMERA_DEPTH,
                TangoPoseData.COORDINATE_FRAME_DEVICE));
    }
        @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);


           //initialize the imageView
            imageDepthMap = (ImageView)findViewById(R.id.imageView);

        //initialize pointCloudManager

        mPointCloudManager = new TangoPointCloudManager();


    }

    @Override
    protected void onResume(){

        super.onResume();
        //obtain the tango configuration

        if(tConnected.compareAndSet(false, true)) {


            try {

                setTango();

            } catch (TangoOutOfDateException tE) {

                tE.printStackTrace();
            }

        }
    }

    @Override
    protected void onPause(){

        super.onPause();

        if(tConnected.compareAndSet(true, false)) {
            try {
                //disconnect Tango service so other applications can use it
                mTango.disconnect();
            } catch (TangoException e) {
                e.printStackTrace();
            }
        }
    }


    private void setTango(){

        mTango = new Tango(MainActivity.this, new Runnable() {
            @Override
            public void run() {

                TangoSupport.initialize();
                mTangoConfig = new TangoConfig();
                mTangoConfig = mTango.getConfig(TangoConfig.CONFIG_TYPE_CURRENT);
                mTangoConfig.putBoolean(TangoConfig.KEY_BOOLEAN_DEPTH, true); //activate depth sensing

                mTango.connect(mTangoConfig);

                mTango.connectListener(framePairs, new Tango.OnTangoUpdateListener() {
                @Override
                public void onPoseAvailable(TangoPoseData tangoPoseData) {

                }

                @Override
                public void onXyzIjAvailable(TangoXyzIjData pointCloud) {

                    // Log.d("gDebug", "xyZAvailable");
                    //TangoXyzIjData pointCloud = mPointCloudManager.getLatestXyzIj();
                    // Update current camera pose

                    if (pointCloud.ijRows * pointCloud.ijCols > 0){
                        try {
                            // Calculate the last camera color pose.
                            TangoPoseData lastFramePose = TangoSupport.getPoseAtTime(0,
                                    TangoPoseData.COORDINATE_FRAME_START_OF_SERVICE,
                                    TangoPoseData.COORDINATE_FRAME_CAMERA_COLOR,
                                    TangoSupport.TANGO_SUPPORT_ENGINE_OPENGL, 0);


                            if (pointCloud != null) {

                                //obtain depth info per pixel
                                TangoSupport.DepthBuffer depthBuf = TangoSupport.upsampleImageNearestNeighbor(pointCloud, mTango.getCameraIntrinsics(TangoCameraIntrinsics.TANGO_CAMERA_COLOR), lastFramePose);

                                //create Depth map
                                int[] intBuff = convertToInt(depthBuf.depths, depthBuf.width, depthBuf.height);                              

                                final Bitmap Image = Bitmap.createBitmap(intBuff, depthBuf.width, depthBuf.height, Bitmap.Config.ARGB_8888);

                                runOnUiThread(new Runnable() {
                                    @Override
                                    public void run() {
                                        imageDepthMap.setImageBitmap(Image);
                                    }
                                });

                            }
                        } catch (TangoErrorException e) {
                            Log.e("gDebug", "Could not get valid transform");
                        }
                }
            }

            @Override
            public void onFrameAvailable(int i) {

                //Log.d("gDebug", "Frame Available from " + i);
            }

            @Override
            public void onTangoEvent(TangoEvent tangoEvent) {

            }
        });
            }
        });


    }

    private int[] convertToInt(FloatBuffer pointCloudData, int width, int height){
        double mulFact = 255.0/5.0;
        int byteArrayCapacity = width * height;
        int[] depthMap = new int[byteArrayCapacity];
        int grayPixVal = 0;

        pointCloudData.rewind();
        for(int i =0; i < byteArrayCapacity; i++){

            //obtain grayscale representation
            grayPixVal = (int)(mulFact * (5.0- pointCloudData.get(i)));
            depthMap[i] = Color.rgb(grayPixVal, grayPixVal, grayPixVal);

        }



        return depthMap;
    }

}

我从已经工作的版本中提取了这段代码 . 尝试修复任何与配置相关的错误 . 该代码假设深度感测范围为0.4m - 5m深度估计 . 将零映射到255允许未估计的区域（值为零）为白色 .

回复于 2024-05-04T19:10:19+08:00

从点 Cloud 生成深度图

1 回答

相关问题