Bin Picking System

My Master Research Project Blog

Development and Evaluation of a Deep Learning based Robotic Bin Picking Vision System using ROS

The technique used by a robot to detect the objects and estimating their poses to grab, which are randomly placed inside a box or a pallet is called bin picking.

My purpose with this project is to develop a system that is based on learning the appearance model of objects using convolutional neural networks (CNN) is proposed.

Using PyTorch deeplearning framework, a 6DoF Poses of objects in the bin is estimated and it has experimented with the usability of depth sensing cameras in operation room lighting. Creation of the 3D object models is done using Blender tool, which is for Generating synthetic training dataset with the help of Unreal Engine(UE4) and NVidia Deep Learning Data Synthesizer(NDDS) software. By Deep Learning the created model, the object poses with adequate accuracy required for semantic grasping by any robot can be obtained. The overall system is implemented using ROS framework.

I have placed my work in the Github repo : https://github.com/avinashsen707/AUBOi5-D435-ROS-DOPE

Week 0-1 | (29/07 - 10/08)

  • Project Discussion and Topic selection.


Week 2-4 | (12/08 - 31/08)

  • ROS Installation and Tutorials.


Week 5-7 | (02/09 - 21/09)

  • ROS Workshop.
  • Preliminary Report.


Week 8-10 | (23/09 - 12/10)

  • Presentation for the first Review Meeting.
  • Research for the state of the art.
  • First data acquisition from the Intel Realsense D435 Depth Camera.


Week 11-13 | (14/10 - 02/11)

  • Continuation of the research for the state of the art.
  • Developments on the installation of the Realsense D435.
  • Segmentation of the data acquired from the Realsense D435.


Week 14-16 | (04/11 - 23/11)

  • Calibration of the Realsense D435 (Intrinsic Caliberation).
  • Processing of the data acquired from the Realsense D435 to Determine the Centroid of an object and its normal.


Week 17-18 | (25/11 - 07/12)

  • Presentation for the Second Review Meeting.
  • Continuation of the research for the state of the art.


Week 19-21 | (09/12 - 28/12)

  • Support for the Realsense D435 Camera.
  • Installation of the Aubo-i5's Package and coordinate Transformations.
  • Continuation of the research for the state of the art.


Week 22-24 | (30/12 - 18/01)

  • Calibration of the Realsense D435 (Extrinsic Caliberation).


Week 25-26 | (20/01 - 01/02)

  • Continuation of the research for the state of the art.
  • DOPE (Deep Object Pose Estimation) Repo implementing in ROS Enviornment.


Week 27-29 | (03/02 - 22/02)

  • Development of 3D model and Custom dataset.
  • Training and testing the Dataset for Evaluation.


Week 30 | (24/02 - 29/02)

  • Presentation for the Third Review Meeting.


Week 31-32 | (02/03 - 14/03)

  • Preparation for the Paper Submission in International colloquium.


Week 33-34 | (16/03 - 28/03)

  • Publishing the Github Repository for Project.


Week 35-38 | (30/03 - 18/04)

  • Publishing of Github Pages for Project Documentation.


Week 39-45 | (20/04 - 30/05)

  • Online courses for Deep Learning Specialisation and TensorFlow.


Week 46-48 | (01/06 - 20/06)

  • System Integration.
  • Experiments and Results.


Week 49-50 | (22/06 - 04/07)

  • Presentation for the Fourth Review Meeting.
  • Demonstration.


Week 51-53 | (06/07 - 25/07)

  • Project Report Submission.


Week 54-55 | (27/07 - 08/08)

  • Final Viva Voce.
  • Endorsements.

Demonstration of the developed Bin-picking Vision System.