3D Photo in-painting on Google Colab

3 min readDec 18, 2020

For more details and information, you can watch the YouTube video on 3D Photo in-painting in Google Colab.

Video URL : https://youtu.be/11malnCaEj4
Overview

In this tutorial, we will be implementing on In-painting 3D Images on Google Colaboratory using the GPU feature. A single colored RGB image is used as input and the 3D photo is generated in MP4 format, the 3D photo generation is based on the multi-layer representation and the final 3D photo result image can be efficiently rendered with motion parallax using standard graphics engines.

Let’s prepare a environment and import some libraries necessary for converting the photo into 3D image.

!pip3 install torch==1.4.0+cu100 torchvision==0.5.0+cu100 -f https://download.pytorch.org/whl/torch_stable.html
!pip3 install opencv-python==4.2.0.32
!pip3 install vispy==0.6.4
!pip3 install moviepy==1.0.2
!pip3 install transforms3d==0.3.1
!pip3 install networkx==2.3
!sudo apt install sed

Then Let’s download the scripts and pretrained model.

%cd /content/
!git clone https://github.com/vt-vl-lab/3d-photo-inpainting.git
%cd 3d-photo-inpainting
!sh download.sh

Then Switch off off-screen rendering

!sed -i 's/offscreen_rendering: True/offscreen_rendering: False/g' argument.yml

Then generate the platform for uploading the photo of your choice. So that, you can convert it to 3D form.

%cd image
from google.colab import files
uploaded = files.upload()
for fn in uploaded.keys():
  print('User uploaded file "{name}" with length {length} bytes'.format(
      name=fn, length=len(uploaded[fn])))
%cd ..