Writing the vectorized version of the program

Assignment Help Computer Engineering
Reference no: EM131100359

E27: Computer Vision - Spring 2016 - PROJECT 3

PROJECT - STRUCTURED LIGHT AND POINT CLOUD DATA

OVERVIEW

In this lab you will investigate how the first-generation Microsoft Kinect sensor uses structured light to generate dense range data, and learn how to reconstruct 3D point clouds from such data.

TASKS

Background. Before you begin, read the ROS Kinect technical documentation at https://wiki.ros.org/kinect_calibration/technical to get an idea of how the first-generation Kinect works.

Getting started. Begin by downloading the project 3 starter code and files from the course website. Inside the archive, you will find a projector image and a number of simulated camera images of 3D objects built using a simplified model of a Kinect sensor. There are also two Python scripts, starter.py and PointCloudApp.py which help compute disparity maps and visualize 3D data.

You can test the PointCloudApp module by running one of the two commands

python PointCloudApp.py

python PointCloudApp.py cam1_xyz.npz

The first should display a 3D torus, and the second should display the 3D reconstruction corresponding to the cam1.png file. In either case, you can click and drag in the 3D window to rotate the view.

Dense point cloud reconstruction. Given the projector image, and any camera image, you can compute a disparity image for the scene. OpenCV makes this very easy via the StereoSGBM object (see https://goo.gl/U5iW51).

634_Figure.png

Your goal is to extend the starter.py program to generate an n-by-3 array of XYZ data to be used as input to the PointCloudApp module, where n is the useable number of pixels from the disparity image.

To do so, you will need to consider the camera geometry as well as the intrinsic parameters. For the simulated setup, the calibration matrices K for both the projector and the camera are identical, with

81_Figure1.png

and b, the stereo baseline, is 0.05 m.

Remember, K maps a 3D point P in the frame of the camera to a point on the sensor plane:

954_Figure2.png

Hence, mapping each point q through K-1 will return some point P which is proportional to (X, Y, Z). Furthermore, the Z coordinate of each point P can be obtained by examining the disparity value ? at each (u, v) location via

Z = b · f/?

Using this knowledge, it is possible to reconstruct the (X, Y, Z) location for every pixel in the disparity image where ? ≠ 0.

Vectorized code. Due to the inefficiency of interpreted languages, and the ability of underlying libraries to exploit data parallelism, programs in environments like MATLAB and numpy often perform much better if they operate on entire arrays rather than using for loops to explicitly iterate over the data. Although such vectorized implementations are faster, they can sometimes be more difficult to write.

Your final program should be vectorized, containing no explicit iteration (i.e. loops). The bottom part of PointCloudApp.py contains some helpful examples of array operations that avoid loops - in particular, pay attention to the use of numpy.meshgrid and the use of logical masks for array indexing.

You might find it helpful to write and debug an iterative version of your program before writing the vectorized version. For this problem, note that K-1 has a very simple form, which might help guide your implementation.

WHAT TO TURN IN-

In addition to your program source code, you should submit a PDF write-up that addresses the following topics:

1. How did you approach writing the vectorized version of the program? Did you implement the iterative version first and then modify it, or did you do something else?

2. Vary the window_size parameter for stereo matching. Try values of 7, 9, 15, and 21. How do the point clouds change? Are there any benefits to using smaller values? Larger ones?

3. In addition to the projector and IR camera, Kinect sensors also have an RGB camera which records color images. This can be useful for assigning colors to the points in the XYZ point cloud data (the resulting representation is sometimes referred to as XYZRGB).

The starter code for this project includes both data taken from a real Kinect as well as a program load_kinect_data.py to view it. This particular data was taken from a sensor mounted on a TurtleBot 2 (https://www.turtlebot.com/). Given that knowledge, why is there a blank vertical stripe on the right hand side of each point cloud from the real Kinect? What physical feature does the stripe correspond to?

Assignment Data - https://www.dropbox.com/s/3jqfjacbt5mnfr3/Project.zip?dl=0.

Reference no: EM131100359

Questions Cloud

Average of the bowler raw score : Display for each bowler the bowler name and the average of the bowler's raw score. Calcualte the current average and handicap for each bowler.
The owner of a chain of three grocery stores : The owner of a chain of three grocery stores has purchased five crates of fresh strawberries. The estimated probability distribution of potential sales of the strawberries before spoilage differs among the three stores.
Enrolling courses for students and for enrolledclass : A summary of business activities for enrolling courses for students and for enrolledclass lists for faculty members at an online university is as follows:
How your perception is influenced by gestalt principle : Consider a famous painting. Identify at least one Gestalt principle that influences your perceptual organization of this painting. Explain how your perception is influenced by the Gestalt principle. Why is this information important?
Writing the vectorized version of the program : E27: Computer Vision - Spring 2016 - PROJECT 3. How did you approach writing the vectorized version of the program? Did you implement the iterative version first and then modify it, or did you do something else
About solving dynamic programming problems : Consider the following statements about solving dynamic programming problems. Label each statement as true or false, and then justify your answer by referring to specific statements (with page citations) in the chapter.
Identify a best practice in community policing : Identify a best practice in community policing that you believe could be replicated in most midsized police agencies. Why do you believe it has potential in most jurisdictions?
The following project network when applying pert/cpm : Consider the following project network when applying PERT/CPM as described in Chap. 10, where the number over each node is the time required for the corresponding activity. Consider
Problem regarding the relational algebra : A database records information about tutorials in a particular unit offering (such as ICT285 this semester). A student is supposed to attend only one of several available tutorials, each of which is taken by a tutor at a particular day and time. A..

Reviews

Write a Review

 

Computer Engineering Questions & Answers

  Sources must be cited in apa formatnbsp your response

sources must be cited in apa format.nbsp your response should be a minimum of one single-spaced pagenbspto anbspmaximum

  Implement needham-schroeder protocol using python

Implement Needham-Schroeder symmetric key protocol, both original version and fixed version using python.

  Writing down the code

Suppose that the a, b and c are char variables has been declared. Write some code which reads the first character of next line into a.

  Percent of commission

What is Tom's base salary? What percent of commission does he earn? Would you be able to use the same equation for Tom for the next year? Why or Why not?

  Implementing an intrusion detection system

Your company is trying to decide whether to implement an intrusion detection system (IDS), or an intrusion prevention system (IPS).

  Paper on what virtual systems can be used for

Write a one page paper on what virtual systems can be used for

  Make an abstract class called aqualife

Fish has an attribute that stores whether the fish is an herbivore or a carnivore. Its eats method checks whether herbivore or carnivore, and prints 'This fish eats veggies' for herbivores and 'This fish eats other fish' for carnivores.

  What are the advantages of using search engines like google

what are the benefits of using search engines such as google yahoo or bing? what are some of the limitations and

  Explain a network design for a client

explain a network design for a client that has 50 remote locations in addition to a central headquarters

  Define how you feel unit testing should be conducted

define how you feel unit testing should be conducted for a program with lots of conditional expressions.

  What is a post-implementation audit of an information system

What is a post-implementation audit of an information system

  Identify a deployment strategy that will facilitate success

Given an organizational and industry context, identify and suggest a deployment strategy that will facilitate the success of a technologically driven organization.

Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd