Internship: Deep Learning for PointCloud Segmentation and Object Detection

Come work with us as a Research Intern

At BrainCreators, we're at the forefront of applied AI with many years of successful research internship projects that combine cutting edge science with the challenges of applying AI in the real world. The focus of this year’s AI research internship projects will be on the technical challenges at the heart of our Machine Learning platform, BrainMatter.

What we expect from you

  • A full-time commitment to the research internship project.
  • A solid background in the theoretical subjects relevant for your particular project and ML coding skills in pyTorch.
  • Good communication and presentational skills, and a willingness to learn as much as possible in this exciting year.
  • Your project will have a scientific component on which you are encouraged to work towards a publishable paper at the end of the year.
  • Your project will also have an applied component, the result of which is a functional and documented piece of cutting-edge software that can be integrated into BrainMatter.
  • Bachelor’s degree in Artificial Intelligence or related field.

What we can offer you

  • The opportunity to work in our research team as a full time member.
  • A workplace in our Prinsengracht HQ with access to our compute cluster if required.
  • Support and supervision, including a weekly personal supervision meeting and research team group meeting as well as support for integration into our software stack when needed.
  • Internal weekly workshops about scientific and industrial progress.
  • Become part of a vibrant team of AI realists that know how to get things done.
  • Our best interns will be offered a full time job opportunity after graduation.

Project overview

At least three properties differentiate 3D PointCloud data from 2D images: PointCloud data is inherently unordered, there are more complex interactions among points, and it is invariant under different transformations. A naive Deep Learning approach that e.g., simply applies standard convolution to a voxelization of 3D space has drawbacks, and innovative approaches therefore focus on applying Deep Learning directly on the raw PointCloud data. The focus in this project is on understanding when and why to apply these latter techniques for our clients in the business of physical asset management. 

A core focus of the BrainMatter platform is visual asset inspection and recognition for the task of automated asset management. Currently, the strength of the platform is in 2D images, bounding boxes, bounding polygons, and image segmentation. We would like to extend these functionalities with state of the art PointCloud methods. The research intern will be part of our pioneering efforts to select and integrate Deep Learning for PointCloud methods into the BrainMatter platform. 

To help shape this roadmap we offer a research internship position in the area of object detection in PointCloud data, preferably combined with corresponding 2D visual input.The practical goal of the project is to both kickstart our work on Deep Learning for PointCloud in general, and work towards a software deliverable at the end of the project that has a focus on object detection or PointCloud segmentation functionality. At the same time, the research intern is encouraged to find and advance academic knowledge on this topic and work towards a publishable paper on graduation in collaboration with our team and University researchers. 

The details of the project, including the particular research questions, are to some extent open for discussion. Preferably, the research intern works from a small set of already published results, advancing their achievements, and combining them into an original set of research questions for the thesis. Examples include, but are not limited to, using Deep Learning to obtain compact latent representations (of objects in PointCloud data) that are invariant under suitable transformations; finding novel ways to more efficiently exploit image labels and bounding boxes on 2D data to improve the detection of objects in 3D PointClouds; efficient sorting and retrieval of similar objects from PointCloud datasets given their latent representations, etc. 

Another possible avenue of research and applications is the connection between 3D PointCloud and Building Information Modelling (BIM) [7]. A primary usage pattern of the BrainMatter platform is to compare real-world observations with some normative description of the properties an asset ought to satisfy. Often, our clients provide such descriptions in the form of BIM data, and BrainMatter should be able to integrate and compare PointCloud and visual observations with BIM. Conversely, our clients are also interested in creating or updating BIM data from the observations. Applying Deep Learning 3D PointCloud methods to this set of challenges is an essential part of our goals, and is another possible direction for the research internship. 

As a starting point in the literature the research intern is referred to the 2020 survey on Deep Learning for 3D Point Clouds [1], and others like it, e.g., [4][5]. For an overview of available source code and data sets the research intern is encouraged to visit the excellent overview pages on 3D Machine Learning by Tim Zhang [2] and Yongcheng Liu [6]. Another good source is the 3D Object Detection page from Papers with Code [3]. The research intern is encouraged and expected to find and select relevant articles, ideas, and source code in the early stages of the project. 

[1] Deep Learning for 3D Point Clouds: A Survey     [2] 3D Machine Learning  [3] 3D Object Detection   [4] Deep Learning on Point Clouds and Its Application: A Survey  [5] Review: deep learning on 3D point clouds  

[6] Yochengliu  [7] BIM

Interested?

If you'd like to apply for this internship, send your CV and cover letter to our Head of Research, Maarten Stol.