Brain MRI Segmentation Research Project

Gradient Descent Optimization

2007-09-17T11:25:00.000-07:00

To help with the optimization of the segmentation using the itk::WaveletTransform I implemented a basic itk::WaveletGradientDescentOptimizer that takes into account each intensity profile separately. It gets from the itk::IntensityProfileMetric information about the gradient for each intensity profile around the shape or mesh being registered, then it creates using that information a new mesh based on those gradients and a step value. This new mesh is then approximated using wavelets by calling methods in the itk::WaveletTransform.

To test the new optimizer I tried tracing the brain with a tracing that has more detail than the ellipse model but less detail than the previous one I discussed. The following images shows information about the tracing that I used.

Then I ran the optimizer using the optimal parameters that I found from the tracing and the following movie shows the results. The movie illustrates how the fit on the right side of the brain is closer then with just the ellipse but on the left the coefficients seem to be standing still, which may be due to some bugs in the optimizer that I have to iron out.

IGUI With Fit

2007-09-16T16:31:00.000-07:00

The following shows a still of the intensity profile GUI with the intensity profile selected fitted to using the algorithm that finds the location of the brain skull interface. In the graph showing the intensity profile the title also contains information on where the location of the brain skull interface is located. This can be used to explore different methods to fit the intensity profiles like seeing how the length of the intensity profile should be changed or what types of landmarks to look for around the areas that the intensity profile is sampled from.

Optimization

2007-09-16T12:58:00.000-07:00

The main purpose of incorporating the itk::WaveletTransform into the segmentation framework is to create deformable registration where the points can fit the outside of the brain with more detail. Some issues with the addition of the itk::WaveletTransform were that it had difficulty being optimized using the itk::OnePlusOneEvolutionaryOptimizer so a few parameters were looked at to make sure that it would work with an optimizer like gradient descent or downhill simplex optimizer. The following graphs represent the metric value of the segmentation as one wavelet parameter is being changed. The graphs also show a diagram of the brain with a region circled in green representing the area that the wavelet parameter is affecting.

For the most part the coefficients have an overall minimum point in the metric value but still has places where the coefficients can get stuck. One way to incorporate gradient descent as the optimizer would be to take the gradient information at each intensity profile and then use that information to find the next set of wavelet coefficients.

Wavelet Transform For Deformable Registration

2007-09-09T22:11:00.000-07:00

I finished working on the itk::WaveletTransform and now have it working with the itk::OnePlusOneEvolutionaryOptimizer it uses the Daubechies wavelets now and is able to do the wavelet transformation much faster. The following video shows the registration moving along using a transform with 128 parameters and 1200 points. Parts of the border of the ellipse can be seen to change their location as the coefficients of the wavelets are changed. Video also here.

This registration was created by using parameters that were found by using a tracing of the border of the skull using MultiTrace. The tracing was inputted into the IGUI program to get the intensity profile data to fit and find optimal parameters. A picture of the IGUI program with the tracing mesh follows.
This is the data that is used to fit the data in the preceding video. Although one consequence of using this exact data is that the models used to fit the intensity profiles may not work anymore since many of the intensity profiles are not normal to the skull anymore. To remedy this models that change the length of the sampling interval may need to be developed and ones that take into account a smaller neighborhood of data and one with more ambiguities. Another improvement would be to change the optimizer to factor in the information from gradient descent, which the itk::IntensityProfileMetric is still calculating. The optimizer would need to be changed to make sure that the there were localized gradient directions calculated or to treat each wavelet function as a seperate and independent optimization problem. Because of the problems with the current optimization methodology the registration depicted in the video seems to stay still and have a hard time finding the correct exact positioning although if run long enough it would probably converge to the correct answer.

Daubechies Wavelet For Brain Tracing

2007-09-05T00:44:00.000-07:00

To test how intricate of a shape the wavelet transform of the brain-skull interface could be, I first started by tracing the edge using MultiTracer. Through that program I was able to export by traced points unto a UCF file so that I could work with the points directly. The following shows the brain slice I used with red outlining where I traced, followed by a plot of just the points I traced in red.

Once the tracing had been found separated the x and y points in the data to be modeled separately and also resampled them so that the total number would be a power of two to help fit in with the level framework in multiresolution analysis. Thus I had a total of 2^13 points. At first I tried to use the Cubic B-spline wavelets that I had tried before to model the ellipses but ran into a few problems. Once the function or in this case the data to be modeled became very intricate and complex the linear system I needed to solve became more troublesome to work with and the pseudo inverse I was using would not suffice to solve the system. To remedy this I tried to make the number of wavelets I had match the number of samples I was using to discretize the wavelets in hopes to make a square matrix that would easily be invertible. But the algorithm that I was using was using huge matrix operations and was thus taking too long to get to levels that would allow for the amount of detail that I needed.

I decided to try using the Daubechies Wavelets since they seemed to be good for encoding polynomials. There also were algorithms created that took advantage of the fast wavelet transform that would enable me to avoid inverting the matrix and instead just calculate the coefficients at a high level and then filter them to lower levels, which proved to be much more efficient. The following graphs demonstrate my results. The first one is the tracing in green modeled by wavelets in red using level j=5 with 2^6 coefficients.
These next images represent the data in green modeled by the wavelets in red at levels j=8 and j=9 with 2^9 and 2^10 coefficients respectively. As the level increases the level of detail that the wavelet transform is able to capture also increases.
The following image represents level j=9 where I used only half the wavelets and the results show that the wavelet model of the data shown in green only covers half of the data. This demonstrated the ability to zero in on certain locations using the wavelets.

ITK Issues

2007-08-24T16:19:00.000-07:00

I was having problems getting the coefficients from Linux early since the coefficients I was calculating seemed to be way off from the answers that they should have been. Initially the problem seemed to stem from Linux having problems with inverting the wavelet matrix when solving the linear system for the coefficients. But after installing the latest version of ITK on the Linux box I found that all of the previous problems disappeared and the coefficients I was getting were fine.

Since the ITK version I was using on the Linux boxes was so old it was probably causing problems in the registration program. There were also other cases when I was getting segmentation faults when using the intens_process class that could have been partly caused by the outdated libraries in ITK, VTK, and CMake.

Additional Wavelet Levels With The Ellipse

2007-08-24T16:01:00.000-07:00

The following show the elliptical points in green with tangential noise added from negative ten to positive ten pixels with a uniform distribution. The graphs start with j = 0, and go up to j = 0,...6, with the wavelet points shown in red.

More Wavelet Levels With The Ellipse

2007-08-24T11:36:00.000-07:00

The following is a series of images that represent the graphs of the elliptical data in green modeled by wavelets in red of varying degrees of resolution added. The first images starts with j = 0, and the last is j = 0,...6.

Wavelet Levels With The Ellipse

2007-08-24T11:07:00.000-07:00

The following graphs show the same effects of changing the levels as the previous post. These are different in that instead of using just the x points, the whole ellipse is used. The first graph is of the ellipse in green modeled by the wavelets in red using all of the j = 0,...6, levels of resolution. Following that graph is one using just the j = 0, level of resolution and finally one where the higher resolution wavelets are filtered so that they only use ones on one side of the interval.

Another issue is still that finding the coefficients of the wavelets has proved difficult when running the wavelet program on Linux. The problem stems from the inversion of a a matrix to calculate all of the coefficients. The matrix in question is 131x512 with a rank of 131, that I convert to a 512x512 matrix by adding zeros. In windows when it inverts the matrix it probably finds the pseudo matrix so I tried the same thing in Linux by using the vnl_svd::pinverse. However, the results from using the pseudo inverse were the sae as before and Linux might have tried to use the pseudo inverse instead of trying to do the regular one when I called vnl_matrix_inverse(). Another method available was vnl_svd::solve(vnl_vector const& y) to solve a linear system, although use of this just caused a segmentation fault. To remedy the problem now I am going to finish an iterative method that uses the inner product of the point data and the wavelet functions to create the appropriate coefficients.

Wavelet Levels

2007-08-22T10:54:00.000-07:00

The following image shows the x points used in the ellipse with noise added and evenly spaced intervals in green with the wavelet fit shown in red that contains all the levels j = 0,...6. Following that image is the same display except this time the wavelet fit only contains level j = 0. As it can be seen the higher levels add greater detail at a smaller scale, while the lower levels give the overall shape of the curve.

The following is the same graph as above except that wavelets of the higher levels are filtered so that only the ones that effect the right part of the function are used and thus as can be seen only the right side has the finer detail.

Representing The Ellipse Using Wavelets

2007-08-22T10:39:00.000-07:00

The following images show the ellipse projected onto the wavelets and then projected back to see how they compare to the data. The graph shows the data in red and the wavelet version in green, and as can be seen the data and wavelets show almost the exact same locations. The first graph is the ellipse points, the second the x points, and the third the y points.

One issue that came up was that the calculations on the wavelets came up with incorrect results when I used the program on a Linux machine. The code implements classes from the VNL library like vnl/algo/vnl_matrix_inverse and vnl/vnl_matrix to complete the computations. When run on a Linux machine the results showed that the wavelets version of the data were off at the edges and when I tried to transform the y points, it gave totally incorrect results like as follows with the data in green and the wavelet fit in red.

Cubic B-Spline Wavelets

2007-08-21T13:00:00.001-07:00

The above is a graph of a few of the wavelets that are being used in the levels j = 0,...,2.

Point Representation Using Wavelets

2007-08-21T12:03:00.000-07:00

The above graph represents all the points currently used at the start of the registration process. The points create an elliptical shape. To transform the points using wavelets, the x and y values were separated as shown in the following graphs.

Then Cubic B-Spline wavelets are created with j = 0,...,6, where j is the number of intervals. Scaling functions for j = 0 where also created. Then the scaling functions and wavelets were all put into a matrix by evaluating the function along the interval at 512 points for each wavelet or scaling function and storing those values along the rows. Then that matrix was multiplied by the actual function needing to be transformed, in this case the x or y values that made up 512 points, the result was a vector of coefficients that could be used to recreate the data by multiplying the scaling and wavelet matrix by the coefficient vector. The following graphs illustrate the results where the one on the left represents the x points and the one on the right represents the projection that resulted from the data being projected onto the wavelets.

The wavelet projection produces a set of points with a similar overall shape but the y axis is more then doubled in magnitude. This might be occurring because the wavelets need to be scaled.

Intensity Profile Fitting Error

2007-08-15T13:27:00.000-07:00

When fitting the intensity profile data a Gaussian and logistic function are used. To begin with the data is quickly looked at to get the initial values of the parameters. The location of the Gaussian is found by approximating the derivative at each point in the data. Using the information about the derivative the location of all the peaks in the data can be found. Then the peaks are sorted from highest to lowest and the highest one is used as the height and location of the Gaussian peak. Then the lowest intensity valued point after the Gaussian is used as the beginning of the logistic function. The height of the flat part of the logistic is found by taking the average of the last twenty points of the data. The rest of the parameters of the data are found in a similar fashion. These initial parameters are reflected in the following graph where the data is shown in blue, the Gaussian function in yellow and the logistic function in pink.
Once the initial parameters have been approximated using the previously described methods, they are optimized using the Levenberg-Marquardt algorithm using the implementation found in VNL by instantiating "vnl_levenberg_marquardt". The following graph represents the optimized parameters in the previous graph. Running the optimization widens the Gaussian, changed how steep the logistic function is, and adjusts the height of the flat part of the logistic. However, the location of the start of the logistic function and the position of the peak stays roughly the same.
In the current segmentation metric implemented in the itk::IntensityProfileMetric the each intensity profile is modeled and the location in between the Gaussian peak and the start of the logistic function,the brain-skull interface, is used as a reference to tell how far off the current intensity profile is from its optimal location. This means that optimizing the parameters does not have a huge impact on the information that is finally extracted from the intensity profiles. Therefore, when the fitting function gives a least-squares error to the fit, it does not strongly correlate to how confident the location of the brain-skull interface was approximated. Another issue that has an impact on the error is the area of the flat part of the Gaussian, which represents all of the folds in the brain matter. This area contributes a great deal to the error in the resulting fit. But the contribution to the error from this area can be misleading since if the brain matter is very stable the error will be lower although the other parameters could still be off by a significant amount. So the use of error could be factored into the segmentation process, but can produce incorrect information as shown Gaussian Scaled Metric post, where when the error was weighted the same amount as the difference in location of the brain-skull interface. What the first plot in that post illustrates is that once the error is factored in, the metric doesn't converge to an optimal value anymore, but instead creates a situation where the metric becomes very uniform and gradient decent will not be able to converge to an optimal solution.

Thus the use of error to weight the effects of each of the intensity profiles could be very deceiving in the final metric value and a method that looks at a neighborhood of the difference in location of the brain-skull interface of the intensity profile in question would be a more suitable alternative.

Different Metric Mappings

2007-08-10T12:39:00.000-07:00

The following plots show different ways that I mapped the metric into different mathematical functions. This was done by mapping all the metric values for a square surrounding the center of the brain MRI image to statistical function. The first is a Gaussian with narrow standard deviation.

The second is a Gaussian with a wider standard deviation. Just the first half of a Gaussian was used so the left hand tail up to the peak, meaning is the standard deviation is wider then there is less of a peak and therefore a greater range of mapped values for metric results with low values. Also, a lower metric value is better and means the ellipse has a better fit onto the skull.

The final plot is a mapping to a Sigmoid function that grows very steeply.

Gaussian Scaled Metric Value

2007-08-09T16:29:00.000-07:00

The above visualization shows the same representation of the metric and coordinates on the brain slice image as the last post, but this time the metric value has been scaled. It was scaled by taking the metric value measured the difference between the brain-skull interface point on a sampled intensity profile compared with the optimal value for that point and scaling it to s Gaussian or normal curve. The same this was done to the fit of each of the intensity profiles and then the two values were added up to create the final metric. The results of this seem to show that the addition of emphasizing the error into each metric value seems to through off the metric and in this particular example it's hard to find where the optimal location of the metric would be located.The above image is just the weighted difference part of the metric excluding the error resulting from the fit to each of the intensity profiles. The way that the bottom of the distribution seems to be all around the same level could be due to crowding happening on the tail end of the normal distribution that it is being mapped to.

Metric Values Around Center

2007-08-08T16:56:00.000-07:00

The following is a visualization of metric values around the center of the MRI brain slice. The location of the yellow dots on the plane parallel to the brain slice represents the two dimensional coordinates of the center of the ellipse with respect to the slice. The value that the points take in the z-direction or the direction normal to the brain slice represents the metric value that each location takes. In this case a lower metric value, meaning a point that has a distance closer to the brain slice in the normal direction is better and means that the ellipse is closer to the optimal location. This visualization is a square of locations with width of sixty pixels centered at the center of the image. It shows a gradual slope in towards a low metric value as the locations become closer to the center.

Uniform Distribution In MCMC

2007-08-08T11:47:00.000-07:00

The following results illustrate the MCMC simulation results using a uniform distribution as the proposal distribution with first a standard deviation of one followed by one using a standard deviation of two.

MCMC Using VNL Random Generator

2007-08-08T01:07:00.000-07:00

I changed the MCMC simulation to use the built in VNL numerics library that comes with ITK and VTK and made it so that I sampled from Gaussian distributions by using the normal() function available. This seemed to improve the results if I stuck with the standard deviation of one. Otherwise with a standard deviation of .5 the results seemed to still be off. The following is the results of using a .5 standard deviation of the Gaussians followed by one using a standard deviation of one.When making the registration work to run on Fedora I needed to change the way the intensity profiles are transfered into the metric class. Before it was passing a pointer to an array that held all of the intensities. This was causing a segmentation fault so to remedy this I needed to pass it my one array to write to. Another issue came about because of the GUI. In Fedora the size of the GUI needed to be increased to make sure that everything could be displayed. If the size was too small the resulting image showed nothing but a blank color.

More MCMC Results

2007-08-07T11:24:00.000-07:00

The following images were created using the MCMC method previously described with the addition of a method to check for convergence so that the algorithm will stop after a certain number of iterations. The method is to take a window for the last few iterations and in that window calculate the means and variations for each of the iterations. Then calculate the variance for those means and variations and when they go below a certain point to stop the segmentation process. The first image was found using a standard deviation of .5 for the Gaussian sampling scheme and the second with 1. They show the most likely location to be northwest of the correct location, which could be explained by that area having a local maximum or the fit to the intensity profiles being off.

Intensity Profile GUI Additions And MCMC Results

2007-08-03T14:04:00.000-07:00

To delineate where exactly the intensities are being taken from mesh I colored the graph to make it easier to see. The red and blue colors of the graph correspond to the red and blue parts of the normal lines emanating from the ellipse. There is also a blue line on the graph that represents where the mesh is located in between the two normals. The following is a still of the GUI in action.The following are the previous MCMC results with the ellipse coloring trying to follow the same coloring as the squares although for some reason when points are used instead of faces in a vtkPolyData the coloring for the opacity doesn't work as when. But it shows a distribution that follows the squares and points better.

MCMC Results

2007-08-03T11:09:00.000-07:00

Using the new intens_process class the following represents running the Markov Chain Monte Carlo for 2000 iterations. The yellow dots represents the centers of the red ellipses that are colored according to their likelihood with darker ones being more probable. There are also red squares that are places on top of the points that are colored the same way as the ellipses and represent the probability of the region they cover. These results show a mean that is very close to the actual one but at this point I haven't implemented a way to stop the simulation in a formal manner so more iterations may have changed the result.Another result of the new intens_process is that the intensity profile fits on the top and bottom of the skull have improved and now are as good as the sides. This is shown in the following images with the arrows colored red being the ones with a very high error in their fit to the intensity profile data. There is still one issue with the fit because of regions near the top left and right where the skull boundary has a thicker boundary and also lighter in comparison to the brain matter. Because of this the fit will sometimes pick a peak within the brain matter because the highest intensity value is in that area. I am still trying to find an appropriate algorithm to remedy this matter.

Intensity Profile GUI

2007-08-02T11:06:00.000-07:00

Some of the results that came up from using the MCMC on the final segmentation seemed to be inconsistent with what the correct location of the ellipse should have been. So I took a closer look at what the intens_process class was grabbing from the image by using the intensity profile GUI. To make the GUI a little more clearer I added the slice being looked at in the image to the background and then added lines that represent the normals going inside the mesh with light blue, and the normal lines going outside the mesh with red. Another issue that came up was that the intensities were being sampled by taking the paths of normals arising from the points in the mesh, not the cells. So, I edited the intes_process class so that it would calculate the centers of each face of the mesh and use those points to take the data for the profiles by using normal lines emanating out from it. The only problem with having the image slice in the background is that it makes the mesh harder to see, although this problem might be alleviated by using a brighter color for the mesh or adding lines to it that emphasize its location. I also added the number of the cell into the graph title.

The results found using these changes improved the intensity profiles by having them make more sense in this context with most of the intensity profiles fitting the two Gaussian logistic model. Although having the intensity profiles sampled from the points of the mesh may not have been that big of an issue since they do coincide somewhat with the locations of the faces of the mesh.

Uncertainty Estimation

2007-07-31T12:00:00.000-07:00

The segmentation program built so far uses an elliptical shape model to delineate the interface between the skull and brain. This location is found by using information along lines normal to the ellipse location using the intensity information through intensity profiles. Optimization is completed using gradient descent. To get the initial starting coordinates a center of mass is calculated for the lighter voxels that represent the brain matter. Now to calculate the uncertainty of the segmentation a function was added to itk::IntensityProfileMetric that implements Markov Chain Monte Carlo (MCMC) using the Metropolis-Hastings algorithm.

The MCMC method quantifies the uncertainty by being able to calculate a variance for the distribution. In this case the distribution concerns two parameters, the two-dimensional coordinates that represent the location of the ellipse in respect to the MRI image.

The following is the output of the MCMC run and shows all the sampled coordinate locations in yellow that represent the locations of the center of the ellipses shape model. The red squares in the background represent the probability associated in the region encompassing the square, with darker red meaning more probable and lighter meaning less. The background in the slice that is being segmented with the yellow center points coinciding to their relative position on the MRI image.The following image represents the MCMC coordinates that were sampled by showing the different ellipses in their appropriate location. When the ellipse is darker that means the location is more probable and when the ellipse is less probable the color is a lighter red.

Up & Down Translation Movie

2007-03-24T14:09:00.000-07:00

Registration Using Intensity Profiles

2007-03-22T23:40:00.000-07:00

The preceding movies are of the registration using intensity profiles around the ellipse ring model that are used in the itk::ItensityProfileMetric to bring the ellipse to the skull brain boundary. The metric takes the intensities along coming from a normal from a face on the ellipse ring model and fits a parametric model to them. The model in most cases is a Gaussian followed by a logistic. The Gaussian represents the area of the skull where the intensities get very high and the logistic represents first the empty space between the skull and the brain and then a leveling off that represents the actual brain matter. Once the model is fit the metric will find the position between the Gaussian and the logistic of the data and compare that position to the optimal location of that point. Using that information it will decide what direction and magnitude to proceed to make the position of the point in between the Gaussian and logistic closer to the optimal position. This is done for all of the faces around the elliptical ring and those directions are multiplied by their magnitude and then added together to get the average direction that the intensity profiles are telling to proceed. The magnitudes of all the intensity profiles are also added together and their average is take and used and the magnitude of the metric. Both the magnitude and direction resulting from the metric are passed to the itk::VersorRigid3DtransformOptimizer as the value and derivative and are used to decide where to move the parameters of the ellipse next. The transform being used is the itk::VersorRigid3Dtransform.

The movies above show the ellipse iteratively moving to its optimal location. The arrows represent the direction and the magnitude that each of the intensity profiles is telling the optimizer to move. They were created by using vtkLineSource for the shaft of the arrows and vtkConeSource for the pointer part. They were all added into a vtkAssembly which was in turn fed into the vtkRenderer. The resolution was set to one to better facilitate the drawing of so many arrows on the display. The arrows start at the centroid of a face translated by whatever the parameters of the current iteration. Only the arrows in blue are being factored in the registration and direction of the optimizer. The red arrows were filtered by taking the mean magnitude of the intensity profiles and seeing which intensities had a magnitude much greater than that and filtering those out. In the future this could be improved by looking at the covariance of the intensity profiles and from that information deciding how the different intensity profiles should be factored in and to also weight how well of a fit the parametric models are to the data.

Another issue that could be causing problems in the directions of the normals of the mesh and how the intens_process class calculated the normals when getting intensities from the image. The graph at the bottom of the display focuses on one particular intensity process, in this case one at the bottom of the ellipse model. Intensities being displayed in the diagram seem to vacillate greatly and this may be caused by there being issues with the normals. This could be analyzed by analyzing what the normal directions are and visually inspecting what the data should look like. But overall the intensity profiles seem to work well at getting the ellipse into the correct position with an initial offset of about twenty pixels.

Derivative & Blurring

2007-01-09T17:05:00.000-08:00

The above is a movie of the registration process with the changes added below.

I got the registration to accept the new derivative that I created. The derivative is calculated by using the normal direction of the face on the mesh that I am interested in multiplied by the magnitude of the metric. The metric is calculated by finding the chi-squared difference between the point in between the Gaussian and the logistic fit positions to the data in the intensities. The derivative has to be negated since in the itk::VersorRigid3DTransformOptimizer it assumes that the derivative is calculated from the gradient and that would point in the direction of the maximum, and in the case I am looking at I am trying to minimize the metric value and therefore the optimizer will try to compensate for that by changing the direction of the derivative. I had issues with the metric going off in a weird direction before, but I found that the problem stemmed from the scaling in the optimizer. I had drowned out the angle and z-parameters in the itk::PointSetToImageRegistrationMethod registration framework by scaling the values with 10000000000000.0, which would divide the derivative value inside the optimizer and therefore make it have no contribution to the registration at all. In some of the ITK examples I had found that they used a translation scaling of 1/1000 for the parameters of interest, specifically the translation parameters. So I thought that I could be having issues with that drowning out the effects of the derivative value that I had created. So I set that value to 1 and the registration finally performed as I expected it to with the derivative that I had been feeding it. In this particular situation I had been experimenting with just one intensity process, but now I need to add the rest of the intensity values back into the registration. The registration now works using translation with one intensity profile from any distance as long as the fitting program is able to work with the intensities that it is passed.

I also figured out where the skipping of the lines was located. It was a line in the intens_process class that would throw out a new line as it were iterating through the cells. And I completed changing the itk::IntensityProfileMetric class so that it only loads the image into the intensity_process class process once in the class constructor. Thus the speed is increased for each iteration, but starting up the program takes a long time because the image is being smoothed using itk::BinomialBlurImageFilter which helps to stabalize the results of the fitting program from iteration to iteration.

Another thing that might be good to complete would be to finish making the visualization that shows the normals and the magnitudes that they contribute to the metric and derivative.

Derivatives & Metric

2007-01-05T16:08:00.000-08:00

Today I was looking into ways to improve the registration. One thing that needs to be analyzed is how the derivative is computed in the itk:IntensityProfileMetric. This value could be making the optimizer go off in the wrong direction. I also need to look at the optimizer to see how it takes into account the derivative and how it looks at the value of the metric to decide on what direction to take next. Another thing that needs to be improved is loading all the necessary components of the itk::IntensityProfileMetric. Right now it instantiates an instance of the intensity process class in each iteration, I need to make them class members and do all of that work once. Another improvement would be to find what is making the output of the command iteration class, that tells what the parameters of the registration are, skip so many lines before it enters another value.

The fit to the values of the intensity profile could also need some help, because in some instances it seems as though it is coming off giving an error that is bringing the ellipse closer to the wrong direction. Smoothing has been implemented, but the intensity profile metric class needs to be adjusted to allow for testing since it is too slow.

Fitting Using Levenberg-Marquardt Algorithm

2006-12-08T02:14:00.001-08:00

The intensity data taken from the 3D MRI was very noisy resulting in a fit with a high error. We created a different metric to alleviate the effects of the high error. In this method, we looked at the midpoint in between the end of the Gaussian and the beginning of the logistic for each intensity profile and found the difference between that and the optimal position for the point. This required that the data be fitted to the Gaussian and logistic model every iteration. My previous method of exhaustively trying all the combinations of parameters would have been too slow for this situation. Therefore, I tried to improve the fitting program by implementing the Levenberg-Marquardt algorithm to speed up the process. This was added to the fitting code and produced results that were very close to the previous fit with a huge boost in efficiency. A movie of the process is shown above.

Once I had completed making the fit program faster, I moved on to adding it into the intensity profile metric where it would analyze the intensities at each iteration and find the point between the Gaussian and logistic and compare that to the optimal position. The registration would work well if the offset was within about ten pixels of the optimal position. If I increased the offset, the optimizer would start heading in the opposite direction away from the brain even though the metric would be increasing. This might have something to do with a setting in the optimizer needing adjusting. A movie of this process follows. The blue line on the ring of points shows the specific intensity that I am looking at and the graph shows the intensity data in orange with the fit in white and the optimal position for the mid point in between the end of the Gaussian and beginning of the logistic with a green line.

Registration Using Intenisty Profiles

2006-11-21T16:45:00.000-08:00

To test out the registration I began by starting the ring that I am using ten pixels away from the optimal location. Then I implemented the intensity profile metric using the intensities at just one point in the ring. The metric value was calculated by taking the intensities at that point during each step of the registration and finding the least squares error between the model I had created for the intensities when the model was in the optimal location of the image. The preceding movie shows the metric smoothly converging to the correct value since in this case I am using just one point in the model instead of all the points the errors in my model fitting aren't overwhelming the metric value which had been a problem.

To create the model I use a Gaussing plus a Logistic function as illustrated in the following equation

To fit all of the parameters I wrote a C++ program that would find all the peaks by looking at the first derivative using surrounding points and then finding the most significant one by looking at the second derivative. The beginning of the logistic was found using a similar method. To find the rest of the parameters I started out using values roughly in the range of the correct ones and then exhaustively tried different combinations to find the optimal fit. The previous post shows some examples of how well the fit is with some of the data.

When gathering data from the intensities around points from the ring, most conform to the Gaussian-Logistic model except the top and bottom of the ring which is more of two Gaussian functions as illustrated in the following graph

To alleviate this issue the fitting code can be augmented to take into account model selection to see whether a different combinations of Gaussian and logistic functions would work better.

Fitting Intensity Profiles

2006-11-14T21:15:00.000-08:00

Using the set up in the receding image, I was able to extract intensities along the red ellipse in the image. I have used a sampling range of 120 pixels to extract data. To accomplish this I implemented the intensity profile class that I had to change slightly to make sure that all the intensities were coming from the outside into the brain so that I could compose a regular model for the data. Once this was done I looked at the graphs of the average intensities that resembled the following graph.

I had initially tried to model the intensities using two Gaussians along with a logistic, but after experimenting with various data I found that some of the data contained up to five Gaussians and some with just one. But usually there was one main Gaussian, so I changed my model to used one Gaussian and one logistic. The following two graphs represent the models that I created in the red color and the data in the green. In some cases I was able to get the model to fit very well in the case of the first graph, but others the match was not as good. I constructed this by making a C++ code that would find the Gaussians by looking at the derivative and second derivative of points to fins it, then I would give a rough estimate for the rest of the parameters in the Gaussian and logistic and many different combinations till I came up with a reasonable one.I also added all the intensity profiles into the registration process where I took a sampling of every tenth face in my ring structure and calculated the intensities of the image at that point compared with the model and then took the least squares error and added it up for each face as the metric value. Right now it seems to be stable if I make the optimizer's movements very minor, but I am not sure whether that is significant.

Visualizing Intensity Profiles

2006-10-30T18:21:00.000-08:00

After completing the conversion of the itkMesh to a vtkPolyData and the calculation of the normals from the mesh, I started working with the intensity profile visualizer to analyze what the intensities look like. I was able to pass the class my itkMesh and vtkMesh directly to set up the visualization. The normals were read in from a file that I had made from the registration process, although this can be changed to get the normals directly from the mesh being looked at. One issue that needed to be dealt with was the offset of the mesh to make sure that it was in range with the image to calculate the intensities. I had kept getting a segmentation fault error that was probably due to the points in the mesh out of the image’s range. Adding an offset to each point in the mesh was able to help keep the mesh in range of the image. There were also some issues with the cells being undefined at certain points that has to due with how I initialized them and the count that I used to iterate through the points.

I was able to view some of the intensity profiles as seen in the above pictures of my three-dimensional ring that uses triangular faces on the edge to get normals to use to find the intensities at certain areas in the image. These initial profiles are not uniformly distributed throughout the image because the points could be better positioned around the skull area. To find the optimal value I will need to run the registration process again using the points to figure out the best possible position so as to get a better idea of where I can extract a general shape of the intensity profiles of the image.

Mesh Conversion And Normals

2006-10-29T14:17:00.000-08:00

To calculate the normals in the mesh created in the registration process, I converted the itkMesh into a vtkPolyData so that I could use vtkPolyDataNormals to easily find the face normals. ITK has some already made classes like itkMeshTovtkPolyData that contained a class to complete the process. I tried to implement an instance of the class to convert my itkMesh, but ran into problems with the itkMeshTovtkPolyData class complaining that the itkMesh that I was passing was of const type. Therefore, I added the functions that I needed directly into the itkIntensityProfileMetric class. I confirmed that the itkMesh had been converted to a vtkPolyData by checking the number of points and cells. Once this had been completed, I set out to calculated the normals for use in the itkIntensityProfileMetric and the GUI that displays the intensities for the faces of the mesh. However, I kept getting an error from the vtkPolyDataNormals class that stated there has been a segmentation fault. This may be due to the vtkPolyData that I created not having enough information to be considered a true Mesh or some the fields are not instantiated correctly.

Because of the problems that I encountered with the vtkPolyDataNormals class, I just calculated the normals myself. I accomplished this by storing the points in a double matrix then iterated through the triangle cells and computed the normal vector from the three points stored in each cell. Once the normals were computed I stored them in their own double matrix and made them unit vectors by dividing them by their length. After storing the normal vectors, I outputted them to a file for use in the intensity profile viewer. Incorporating the normals and the mesh into the viewer involves changing the way the viewer inputs meshes from .wrl files to just passing it a completed itkMesh or vtkPolyData.

Cells In itkMesh

2006-09-26T19:34:00.000-07:00

I have completed inserting the face data into an itkMesh that includes the point data for the ring in the 3D-registration process. By changing the parameters in the registration to use an itkMesh I was able to get the registration successfully running. Then I began to work on the intensity process metric and ran into some problems. Declaring the iterators and different variables in the class using the templated names of the data types has proved difficult. With the inclusion of the itkMesh the iterators for moving through the cells were elusive in the ITK class structure so I had to declare them explicitly. Then I found that there is no simple way to access the cell data in the itkMesh. Some of the reason for this is that ITK allows for a heterogenous sampling of cells to be included in an itkMesh. The methods in the ITK manual include creating visitor classes that seem way too complicated for the task of simply accessing information stored in a cell. I am currently looking for functions that could easily get the point data in the cells so that I will be able to calculate the normals.

3D-Registration

2006-09-19T18:42:00.000-07:00

Using a three-dimensional point set requires the registration to be changed so that it contains a three-dimensional transform, optimizer, and image. I first tried to use the itkRigid3DTransform but found that it was also a base class for more specific transforms. itkVersorRigid3DTransform was one of the transforms derived from itkRigid3DTransform and allowed rotation along with translation. The transform had its own tailored optimizer called itkVersorRigid3DTransformOptimizer that would work over the six parameter in the transform. After inserting the optimizer and transform into the registration process I needed to change all the instances of point sets in the registration and visualization into three-dimension ones. The initial parameters in the transform and the scales in the optimizer needed to be augmented to take into account there now being six parameters.

Once the registration was up and running I started to play around with different coordinates to see whether it was working properly. I started by enabling just translation in the x and y-direction by scaling out all the other parameters. Thus the registration was basically two-dimensional over a slice of the three-dimensional image. But the metric value remained extremely small. Throughout the extant of the process there were not any drastic or significant changes. This could be the result of the image not being read correctly into ITK or the point set being in the incorrect position relative to the image.

Displaying The 2D-Slice

2006-09-18T19:05:00.000-07:00

When I first tried to display the slice that came from the 3D-image, it resulted in VTK displaying a completely deformed image that had the bulk of the image colored blue with a red border. I had originally thought that this had to do with the itkImageToVTKImageFilter distorting the data received from ITK. Therefore, I wrote out the image as a .png file and loaded that into VTK, but the problem persisted. This led to me to think that the problem stemmed from the format of the data that I was passing to the VTK interface. The 3D-image had been loaded with a pixel type of unsigned short. I had used unsigned char when VTK had previously worked to display .png files. So I went about trying to change the pixel format of the slice that I had removed. The first filter I tried was the itkImageToImageFilter that would not instantiate itself as a new object due to it being a base class to be used only by other classes derived from it. Then I tried to use the itkRescaleIntensityImageFilter filter. This was able to convert from unsigned short pixels to unsigned char and output an image in that format. I connected the output from the filter to the itkImageToVTKImageFilter and passed that into my VTK visualization set up to successfully display the slice.

The following is an image of the distorted slice on the left and the corrected version on the right:

itkPointSet And 3D-Images

2006-09-17T14:17:00.000-07:00

The problem with loading the itkPointSet into the Registration process had to do with the point data part of the object being empty. When a point set was outputted from the narrow band filter, it filled the point data for each point with the distance it was from the outside of the band. I am not sure exactly how the registration process uses the data. Although in the case of my ellipse, the registration did not work as well when I inserted large values into the data area of itkPointSet that I was creating. After trying various values, I found the optimal value for the point data to be zero.

To make the ring I need to create a point set with three dimensions. This will require the transform to be changed to something that can handle three dimensional point sets. The image used in the process will also need to be in three dimensions. So I began the process of loading in a three dimensional image. I focused my attention on the visualization part using VTK where a two-dimensional image will be displayed. Getting the slice of the three dimensional image that I wanted was accomplished by using the itkExtractImageFilter. I was able to write the slice out to a .png file, but if I passed it to vtk's visualization machinery using itkImageToVTKImageFilter it produced an image that was completely distorted. To fix this I am trying to pass the two-dimensional slice I have into the itk::ImageToImageFilter to change its pixel representation.

Creating An Ellipse

2006-09-13T13:16:00.000-07:00

To create a set of points to represent an ellipse, I first used the equation (x²/79²) + (y²/95²) = 1, solving for y and inserting x-values from -79 to 79, with increments of 0.5. This created around 600 points. The problem was that the sides of the ellipse contained very few points, due to the relatively high changes in the y-direction along the left and right sides of the ellipse. Using the equation in polar coordinates resolved that problem. This was accomplished by incrementing from zero to 2π and plugging that into r² = (79²95²) / (79²sin²(Θ) + 95²cos²(Θ)) as the angle. After converting the resulting polar coordinates into rectangular ones, I was able to create an ellipse that had a uniform spread of points along the border. The following is an image of the ellipse using the rectangular coordinate representation of the equation on the left, and the one using the polar coordinate representation on the right:

Once I had all the points stored into an itkPointSet, I tried to incorporate that into the registration process, but ended up with a segmentation fault error. Taking the itkPointSet that was the output from the spatial object filter on the old ellipse, and replacing its point data with my newly created points fixed the problem. There is some parameter on the itkPointSet that I need to set to get it to work properly. I used the function "Print( std::cout )", that is a part of each itkPointSet, to print out and compare the details of the two itkPointSet objects. The only differences were in the fields "Requested Number Of Regions" with the old point set having its value as two, and mine have a value of zero, and in the field "Reference Count" with a two for the old point set and one for mine.

itkPointSet And itkMesh

2006-09-12T17:26:00.000-07:00

I had created an itkPointSetNormal class that was derived from itkPointSet, but now it seems that the functionality I was trying to capture is already in the itkMesh class. My task was to create a point set that contained the normals along with the points. Although calculating the normals would be difficult if the object started with was more complicated than a circle. One solution is to create a ring structure made up of triangular faces that would then be used to calculate the normals of the points. The class itkMesh allows for all of the abilities of itkPointSet with the addition of cells that contain information about the connectivity of two, three, or four points. Each cell can also have a piece of data attached to it. This allows there to be cells with three points representing a face along with a normal attached as the data.

In the current registration, I am using an ellipse as the shape. So to create the ring structure, I will use the equation (x²/79²) + (y²/95²) = 1 which will give me a row of points that represents the boundary of the ellipse, and where all the coordinates in the z-direction are zero. Supplementing that will be a duplicate ellipse except that in this one the z-coordinates will be negative one. This permits the two ellipses to connect together through triangular faces on their perimeter. The calculations of the normals will be handled in the metric class, where the direction will be the average of the normals of the faces that are adjacent to a point.

Calculating Normals

2006-09-07T16:11:00.000-07:00

The next step in furthering the registration process is to add intensity processes. Sean Mann, another undergrad in the lab, has already written the retrieval of the intensities from a mesh and image in a class. To incorporate that into the registration I need to pass the point set in as a vtkMesh and calculate the normals from my point set.

I finished writing some functions that calculate the normals from the itkPointSet that I get from using the itkBinaryMaskToNarrowBandPointSetFilter on an itkSpatialObject, which in this case was an ellipse. The functions go through each point and calculate the normal based on the shape being roughly a circle and taking a ray from the center going through a perimeter point. The resulting direction is stored in a float array as a point that is then normalized to a unit vector. This approach seems to work well if the shape is a circle. In the case of other shapes, the normals cannot be found so easily. Another method is to calculate the normals based on looking at points to the left and right of the point in question. Then find a line that goes through those two points and use a line perpendicular to it that goes through the center point to be the normal direction. This method would be difficult in the point set implemented thus far because the points resulting from the itkBinaryMaskToNarrowBandPointSetFilter are not uniformly spaced around the border. Because of the way the points are placed, in some cases two points on the same normal vector, this method would only work if the point set were constructed more carefully. Reducing the width of the filter causes there to be empty areas in the border that would not work effectively. Creating the ellipse by using equations to map out the desired region seems like it could also be one solution.

Aside from the creation of the point set normals, their incorporation into the registration will require either adding them to a point set class as an inherited class, or having them as a separate object. There will also need to be an efficient way to transform the normals that could work interchangeably with the various transforms. Those tools could be a part of the new point set class.

I was unable to get the program that the meshes are fed into to calculate the normals to work on my computer. Errors seem to be stemming from the inclusion of "argtable" into the program. I tried installing an rpm of the program from the Internet but get the error "Missing Dependency: linux-gate.so.1 is needed by package argtable2". However, "linux-gate.so.1" is supposedly a virtual library and cannot be installed. It may have something to do with where the install is looking for it. In my case since I am calculating the normals this issue is not as critical.

Tweaking The Registration

2006-08-15T11:54:00.000-07:00

To improve the performance of the Image Registration process, I have added various changes in its structure and implementation. The addition of a user selectable blurring option was included in hopes of smoothing out the optimizer's path. I have been running the itkBinomialBlurImageFilter at twenty iterations, providing an image with its borders slightly blurred. The point set that represents the ellipse has also been shrunk to a smaller band size. Reducing the size of the ellipse band has made the metric more sensitive to every perturbation. It has resulted in the metric being very choppy. I have implemented the Registration in steps, first the translation, then the angle, and finally translation again. The first translation is allowed to move freely, with the optimizer starting at the ability to jump thirty pixels. After this is run the coordinates of the ellipse are usually in roughly the correct position. Then I run a second Registration where the angle is the parameter in question. Once that is completed, I run a third where the translation is again computed, but this time allowed an initial movement of just two pixels. This has allowed for the ability to fine tune a rough set of parameters gradually and incrementally. Although one pervasive issue throughout the process is the vacillating of the metric. It consistently is very unstable in both the translation and orientation processes.

The following is a movie of the Registration process:

Graphical Transformation Of Points

2006-08-11T16:57:00.000-07:00

Today I completed the implementation of the graphical transformation of the points that represent the ellipse in the Registration process I am currently using. The ellipse is a simple itkSpatialObject that can be used to represent the outline of the brain. In my previous Registration, the ellipse fit the brain exactly in size but was placed a few pixels away from the correct location. Thus the process of the Registration only focused on the translation of the points. Now, I have added functionality by using the itkSimilarity2DTransform which contains scaling, rotation, and translation. Previously I had loaded all the points from the ellipse into a vtkPolyData object and used a vtkActor and vtkPolyDataMapper to display the ellipse. To represent the effects of translation I created a function designed to move the vtkActor with respect to the different locations outputted from the Registration process. This was done by simply changing the position settings on the desired actor. To take into account all the aspects of the itkSimilarity2DTransform, I needed to transform the set of points to new locations in space. This was accomplished by instantiating a new transform object with the parameters from the current iteration. Using that, I transformed each point before I placed it into the vtkPolyData object and then reloaded the object into its respective actor. These additional computations did not noticeably reduce the speed of the Registration process. With the transform implemented I no longer needed to access the coordinates of the translation in the display function, since the positioning of the object was already taken care of within the transform.

The results of the Registration using the itkSimilarity2DTransform varied. At times, using initial conditions where the translation was displaced twenty-five pixels with a rotation of fifteen degrees, the end of the registration would find itself within five pixels of the correct transformation. Other times the Registration would end abruptly having made not significant strides into the correct direction. I tried blurring the image using itkBinomialBlurImageFilter, which provided some improvement. If scaling is allowed to vary greatly, the registration process basically stands still and makes no progress. I will probably try to register each aspect of the transformation in succession and use different degrees of blurring.

Movie Of Registration

2006-08-10T17:02:00.000-07:00

A copy of the Registration process movie can be found here.

To construct a movie of the Registration process I implemented the vtkWindowToImageFilter that would take the output of the renderer and convert that to the VTK image type. Using the image type as the input to vtkTIFFWriter I was able to create a series of images that represented each succeeding step in the registration process. Then, to turn those images into a movie I used the "convert" command which comes with the ImageMagick suite and should be available with most versions of linux. I used the command "convert *.tif reg.mpg" to create the movie but came upon the error: Delegate failed `"mpeg2encode" "%i" "%o". I resolved the problem by finding that the converter lacked the required codecs to encode the MPEG movie and added them using the following commands:

wget ftp://ftp.mpegtv.com/pub/mpeg/mssg/mpeg2vidcodec_v12.tar.gz
tar xvzf mpeg2vidcodec_v12.tar.gz
cd mpeg2
make
cd src/mpeg2dec
cp mpeg2decode /usr/local/bin
cd ../mpeg2enc
cp mpeg2encode /usr/local/bin

For viewing the MPEG movies the Berkeley MPEG Player worked well since it was really easy to set up. My initial movies were much too fast to be useful. To alleviate this issue I constructed a movie using three images depicting the same information for each iteration. This allowed the movie to proceed at a slower pace.

vtkXYPlotActor Working

2006-08-09T15:58:00.000-07:00

The above is a snapshot of the final iteration in the Registration process that features the now working vtkXYPlotActor and all my previous implementation of the point set and the brain image registration.

I was finally able to get vtkPlotActor to work properly. The problem was in the ordering of the linked libraries in the CMakeLists.txt file that is used by CMake. It seems that if a library is listed last, then anything dependent on it will not be able to define itself properly. Another issue could have been that certain objects were being defined multiple times to further cause errors. So once I was able to remove all the 'undeclared reference' errors I could then begin experimenting with what the vtkXYPlotActor was capable of doing. It seems to be a very versatile class with the ability for the axes to be scaled to whatever a user designated or for it to automatically compensate for the appropriate amount of space for the points. Using these tools, I can scale the y-axis whenever there are points that overreach a certain boundary. The class also contains incremental labels for both axes where the user can specify a number that the class will try to adhere. There is also infrastructure to allow for the separate coloring of the labels and the lines. The input of the graph is in the form of a vtkPolyData object where the scalars are the values for each of the y-axis points. So when plotting, it will take the scalars as the y-points and the point number in the data structure as the x-points. Compared to actually building a vtkPolyData object using points, cells, scalars, and connections between points, this is much easier. I was also able to fit it in nicely into my real time registration structure by simply reloading the vtkPolyData every iteration.

Now that these steps have been completed, I am beginning to try to figure out how to introduce the ability to scroll through the x-axis. Basically, be able to increase the length of the graph in the x direction but still have it be contained in the same space. This may be possible by introducing the VTK widgets that allow for user control of certain aspects of the graphics. The other step is to be able to transform the set of points in the registration and then display them. In the metric, this is done one point at a time and that may be the only way. Because of all this added computation, the registration process may turn out to be very slow.

Additions To vtkGraph

2006-08-07T23:51:00.000-07:00

I have now added text into my graph implementation using the vtkTextMapper class. It seems to work very well for the purpose with the ability to change the color and size easily, it even has functions that enable the text to be centered to allow for the appropriate format of the text even if the content changes. One difficulty with this class was using its orientation function that allows for the change of the angle of the text by specifying the desired orientation in degrees. When I tried to edit this there was no apparent result. I had planned to display the y-axis label vertically using this function. Now that the labeling has been pretty much completed, I had moved onto the axis that I first tried to display using vtkAxisActor2D, but I have had a terrible time trying to get it to display. The other solution would be to use vtkLine to create the axis. As far as the scaling of the y-axis was concerned, I plan to scale all of the points by a certain amount every time the points reach an upper boundary.

The vtkXYPlotActor itself has much of the functionality that I am trying to recreate. To alleviate some of the issues that I have been having with it to get it to work with Linux I have declared some of the undeclared referenced classes in the calling program and that has helped with many of the errors. Then I came upon difficulties with the vtkMapper2D that I am having trouble declaring properly. It seems that the problem stems from some of the libraries in VTK are not being declared properly in the context of this class. I tried recompiling VTK with turning off the shared libraries and setting the path name correctly but neither seemed to help.

Graph Using vtkXYPlotActor

2006-08-04T23:20:00.000-07:00

The preceding graph was made using the VTK class vtkXYPlotActor. It is able to represent the points as a line and has a nice way to label the axis. The problem is that I was only able to get the class working in Windows. In Linux the compiler would complain every time it would start linking, I am not sure why it is doing that so I have been taking elements from this graph and putting them into my own graph class. The labels suit the purpose very well in that they are much clearer then using a .png file for the background. One issue with it is that it uses the vtkAxisActor2D, and by looking at the above graph it can readily be seen that it is not working properly. The areas where the label is "HPQ" should be the value of the position at that point in the axis. When I tried using the class alone it would not even display properly. This class, if it does what it purports to do, would save a great deal of time in creating my graph class.

Once these problems have been solved, I can start working on making the desired points into a vtkDataSet that is the input the line would take. Either points or a smooth line can represent the line. These specific graphs classes seem to be less robust then the other more standard VTK classes and it might be due to being platform dependant or the classes it depends on being altered as VTK has been updated.

Creating A Two-Dimensional Plot

2006-08-03T14:56:00.000-07:00

I am now starting to create a class that will visualize a two-dimensional graph using the VTK interface. VTK itself comes with a class that supposedly has a good deal of the functionality that I am trying to create but I have yet to get it to work. Currently I have a basic class created that will display a graph given points, but the background is a .png file which has to be drawn by hand specifically for the data set that will be used for the graph. One aspect that separates my class from the one available in the VTK setup is the ability to display the graph point by point which can be used to create a movie of the graph creation in real time. This is helpful in the process of registration where the parameters are being computed on the fly and the visual interface serves to show how the results are changing.

With the new 2DGraph class that I am creating, I am also looking to add the functionality of having the ability to scale itself in the y-direction if the points are specified at locations above the limit. Another addition would be the ability to scroll in the x-direction and have a grid in the background. One issue with implementing these things is the fact that the resolution of the objects being displayed is so low. This causes problems with labels being unclear and the lack of detail that can be implemented. To assist in these goals I am trying to milk some of the functionality of the vtkXYPlotActor that is provided but so far, it has been difficult to get it up and running. The VTK documentation does provide one example piece of code that was written in python, I converted it to C++ but was still unable to get it to run due to problems with the vtkXYPlotActor itself not being able to compile because of "undeclared references" to the initiation of certain objects.

Below is a picture of the graph interface that I have implemented so far.