Virtual Space Generating System

Capstone_Project

This system generates point cloud data that has the same geometric shape and color as a real space, just by scanning the space.

It uses Lidar to earn geometric information of the space and stereo camera to earn color information. CAP_sensors Sensors for scanning


When using the system, the user holds the sensors and scans the areas of the space. By doing so, the system generates a colored point cloud of a specific area, which is accumulated to the final output. The status of the final output is displayed with the lab top monitor so the user can see the missing area that needs an additional scan. CAP_scanning Scanning an area


When generating 3D data of the space, colors of the 2D images from the stereo camera are used to color point cloud generated by lidar. Each point of the point cloud is mapped to a specific pixel of the 2D image to get the right color for the point. CAP_algo1 Coloring the point cloud


Additionally, since the number of 3D points generated by lidar is not enough to express a real space with good quality, some of the pixels of the 2D image are converted to 3D points and added to the point cloud to make better final output. CAP_algo2 Adding pixel points


Finally, to minimize the size of the final output, overlapping points are filtered out to make sure that only one point exists in one 3D coordinate. CAP_algo3 Filtering overlapping points



Following is the demonstration video and the results.




CAP_palgak_result Octagon hall

CAP_coding_result Study room

CAP_his_result Cafe




In this project, my main contribution was to come up with the ideas for the core algorithms and to develop the actual program to realize it. The following are the things that I have worked on:

  1. Analyzed the source code of LOAM(Lidar Odometry And Mapping) SLAM to use it as the base of the system.
  2. Came up with the idea of using camera images to color lidar point cloud and realized it as source code.
  3. Came up with the idea of using pixels to add new points to the point cloud and realized it as source code.
  4. Converted the coordinate of new points clouds from local coordinates to world coordinates to accumulate them as a point cloud.
  5. Came up with the idea of the algorithm that prevents overlap of points and realized it as working source code.
  6. Adjusted parameters & coefficients and optimized the system with continuous tests and experiments.

This project has the interesting aspect of using lidar and stereo camera together to recognize and express its surrounding environment with the ability beyond to the one that each sensor has. It was also a very beneficial project for me because it provided the chance to learn and practice the SLAM algorithm that uses lidar.

This system can create 3D data of scanned environments with fine quality. Compared with the systems in the market that create a similar result, this system provides much cheaper and easier usage.