Christopher Plumberg
Research Log, MSU Summer REU Program, 2009

June 2, 2009
     Today is our HTML tutorial.  I've been doing IDL tutorials for two days now, and things seem to be running smoothly so far.  I'm doing some research on the side on quantum field theory and relativistic quantum mechanics, although some of the books I was hoping to use are currently checked out for the next three months!  Go figure.  Here is a practice link to Google.

June 3, 2009
     Today the server is running a bit slow.  I'm currently doing some reading up on algebraic reconstruction algorithms.  Ought to be interesting.  After our talk yesterday, Dr. Kankelborg and I decided on attempting a background continuum fit (thereby reworking his algorithm so that such a non-uniform continuum might be effectively subtracted).  This would allow for a more realistic assessment of actual solar features, which do not occur on a completely black background.  To that effect, I'm also doing some reading up on Chebyshev polynomials, nonparametric regression and spline smoothing (preferably multivariate?).  We'd be fitting a 2-D surface, so we could also do a quadratic "sum-of-least-squares' fit to try to match a particular background, if worst came to worst.  More to follow on those developments.

June 4, 2009
     I figured out how to generate movies from individual frames!  I can't believe it took me so long to figure out.  Given images which are to be used as individual frames in the movie, you can simply write it this way:
helios%  convert flux*.png fluxmovie.gif
This command takes all the .png files beginning with flux (it's helpful if they're individually numbered), and sticks them all together in one movie (fluxmovie.gif)!  If you want, you can make it a .mpg file instead, like this:
helios%  convert fluxmovie.gif fluxmovie.mpg
You can also simply play it this way:
helios%  animate fluxmovie.mpg
This makes a number of things much easier.  I'm glad I figured that out.
      Also, I got my hands on Dr. Kankelborg's smart3d program for inverting the coronal tomography problem, and am doing some experimentation with it to get myself accustomed to it.

June 5, 2009
     After IDL presentations this morning (which, I think, went quite well overall), I worked a bit more on automating my IDL script for playing .png files through xstepper.  If I can work in a way for the user to enter the names of the .png files (I suppose I could just assume that all the .png files were intended as frames, but that might be assuming a bit too much), then it will be completely automated, and should be a nice bit of work at that.  However, my discovery from yesterday still stands; I believe this other script will serve only as a default in the absence of the first.
     Also, I am working a bit on some extra reading on the side, including review of Fourier transforms, Laplace transforms, and relativistic quantum mechanics.  Hopefully, I can also catch Dr. Kankelborg sometime today to chat a bit about putting the finishing touches on my project from last year.  More to follow on how all of this develops.
     I have had a conversation with Dr. Kankelborg since writing the first two paragraphs in this entry; we discussed at length some methods for fitting/subtracting the background we expect in our data for the coronal tomography project.  This portion of the project can be potentially be done in a number of ways, and I'm admittedly a bit confused as to how to proceed.  We are considering a number of approximation techniques, including fitting by orthogonal polynomials (probably Legendre, but maybe Chebyshev) or multivariate regression techniques.  Scanning the IDL documentation for ways to implement the latter without reinventing the wheel has possibly left me with more questions than when I started.  I think I need to sleep on it.
     Other related challenges include how to establish which pixels constitute the background of the image and which do not; this may be done via the DILATE() function in IDL, or by explicitly performing a box-car convolution with an NaN pixel to "black out" the foreground pixels.  In any case, we need to figure out how to identify the background pixels before we can fit them by any means.
     There is also the problem of actually choosing a non-trivial background B=B(r, theta, phi) which can be integrated (hopefully analytically) over 3 lines of sight in order to produce realistic images.  These images are then what we subject to the DILATE() function (presumably) and approximation schemes to ultimately model its subtraction.
     Lastly, I am attempting to finish up on my FLUX project from last summer; this is going to add a bit to my plate as well.  More to follow as I do my best at everything my hand finds to do, to the glory of God the Father.  Whew.

June 8, 2009
     I spent this weekend and part of today hunting for a function to do multi-dimensional non-linear regression analysis (as per my previous entry), but to no avail.  Instead, I attempted to write down a closed form for the coefficients of the Legendre expansion of the background pixel counts (and I think I may have succeeded), but Dr. Kankelborg pointed out some potential problems with approaching it via orthogonal polynomials at all.  Fortunately, Tom Rust needed something almost identical for doing background subtraction from his images, and he'd managed to locate an IDL function (SFIT) to do the entire fitting process automatically.  I'm not quite sure what an adequate degree is to use for the approximation order, but I suppose I'll have a bit more to work with once I get to actually creating some images with background and running them through this little process.  Anyway, I'm glad it worked out at any rate and that I can keep moving on to something else.  Here's an example of an image I generated while attempting to model some of my own artificial data: Backsub_testimage.jpg.

June 9, 2009
     Today we got to have our first solar lecture with Dana Longcope, who used basic physics to convert our knowledge of the Sun from a knowledge THAT to a knowledge WHY.  It was a good first lecture.
     After the lecture, I worked on toying with various fitting routines and trying to get something to work.  After having a talk with Dr. Kankelborg this afternoon, I was able to get a little bit more direction on exactly what I'm hoping to accomplish in the near future, and how; I was also able to ascertain the actual meaning of the term "sigma filter."  I also learned some convenient tricks for avoiding those cumbersome embedded for loops, namely, some shortcuts involving vector and matrix multiplication.
     In short, my task is to write two separate routines (using the REAL sigma filter), one which will choose and return the background pixels of a given image, and one which will actually perform a decent polynomial fitting to that specified background.  Then perhaps I can construct a sort of wrapper routines which will call them both as I need them.  If I can manage to do a good enough job on these, I might even be able to submit them to the IDL library!  That would be cool.  No pressure though.  ;-)

June 12, 2009
     I've spent the past few days working on implementing and testing the background subtraction routine.  In particular, since I was unable to reconvert the fitted polynomial image back to the original image scale, I had to find a way to calculate the polynomial itself of an arbitrary degree for each pixel in the original array from the coefficients array return by sfit.pro, and then subtract the result from the original image.  Here's my code for doing that: poly_calc.pro.  Hopefully I will be able to spend this weekend working on some fluxon stuff.  Otherwise, now that I think I've been able to test my background subtraction routine successfully, I can move on to generating a background for the test images (cf. Dr. Kankelborg, magcorona.sav), which I can THEN view from different angles and attempt to integrate with his coronal tomographic algorithm.  More to follow.

June 15, 2009
     I spent today working on implementing a test background by imposing a radially decaying exponential function on the simulation volume during Dr. Kankelborg's rotation process.  Since the function is spherically symmetric, the line integral from any particular direction is a function only of impact parameter, which can also be expressed monoparametrically as an angular radius about a fixed location directly along the minimal line of sight.  Thus, the line integral for a given pixel is analytically calculable from varying one parameter, which can in turn be obtained uniquely from applying the clocking, longitudinal and latitudinal transformations to the original image, and calculating the angular distance from the central minimal point for each pixel.

June 16, 2009
     Today was spent developing my background simulation equations a little more to make their implementation as simple as possible.  I've arrived at two basic methods of performing this implementation: the angular method (which relies on the fundamental angular symmetry from the perspective of any line-of-sight of a general spherically symmetric distribution; this was explained a little more fully yesterday).  The three simulated positions represented in Dr. Kankelborg's angles array (in his test3.pro) can easily be used to modify the angle(s) of symmetry that I'm referring to here, so the generalization is straightforward.  Assuming that a given image has an angular dimension of $\varphi$ by $\psi$ square degrees, the first set of equations converts from a pixelized basis for the image to an angular basis for the image; this yields the (approximate) angular position of each pixel in the coordinates of the image.  These angular coordinates can easily be included with the latitudinal and longitudinal displacements at the beginning of test3.pro.
     The alternative means of performing the background simulation implementation is via the symmetrical impact parameter corresponding to the angle of symmetry discussed above; this is merely a reworking of the above argument, and I noticed that my equations for all of these considerations could be simplified a bit more than they already have been.  The core of my implementation, however, is relatively straightforward.
     One final note: these equations are not grouped in any particular way, and do not all follow from one another.  I created this document today using LATEX, so it's perhaps not as lucid as it might be.  I can certainly take a bit and make it a little cleaner, but this working document should give you the gyst.

June 17, 2009
     I decided to take a small break from working on background subtraction and modeling today, to get some mental distance from the problem and simultaneously try to make some progress on my fluxon project from last year.  I spent a short time familiarizing myself with the nuances of my project that 10 months may have caused to slip from my mind, and concluded that the best way to proceed (in plotting the final results of the DA simulation) was to utilize the lineTracer.pro procedure that I used last year to trace out the final field lines.  The only difference, in this case, is that instead of taking the gradient of a scalar potential to obtain the magnetic field as a function of position, I am aiming to write a new function which will use linear interpolation with a fourth-order Runge-Kutta method to obtain the final magnetic field as a function of position.  To this effect, I found it helpful to read in the relevant data files (the DA final data cube), and represent the entire magnetic field as one colossal array (in a .sav file).  Since the DA results are presented in Bx, By and Bz components, I found it advantageous to return these in one giant array, so that I was forced to find a way to concatenate three three-dimensional arrays into one four-dimensional array.  For example, embedding the relevant constituent arrays in one dimension might look like this:
x=indgen(5)
y=indgen(5)
z=[x,y] simply concatenates these two, so that z is of type INT Array[10]
z=[[x],[y]] performs the desired result, yielding z of type INT Array[5,2]
This shorthand is convenient enough for the one-dimensional case, but given arrays like Bxfield, Byfield and Bzfield of type DOUBLE Array[250,95,95] to be concatenated together into, e.g., Bfield of type DOUBLE Array[250,95,95,3], the shorthand notation is simply inadequate to the task, and I was surprised to find that no corresponding functions were available in the IDL database (that I could find; any suggestions would be appreciated).  Since I couldn't find any ways to actually get this to work, I wrote my own procedure to do it for me (sorry, I'm still working on commenting it): concatenate.pro.  Having finally been able to get my data into a usable format, I expect to spend at least some of tomorrow working on an appropriate interpolation function as as I mentioned above.

June 19, 2009
     I was able to get my interpolation function to work; it is now straightfoward to obtain the magnetic field as a function of position, as required by Dan Bruder's lineTracer.pro.  The downside is, I haven't been able to find a very efficient way to do the interpolation, so the programs are running QUITE slowly.  Perhaps Dr. Kankelborg will have some helpful insights.  Since I'm running long processes, I found it helpful to figure out how to let them keep running even after I had left for the day.  One way, which was to do what I did last year, was simply to leave my computer up with my processes running until they had all completed.  However, this proved inconvenient.  Instead, I found that I could run my things in the following manner:
nohup nice idl < batchfile >& log &
This command given at the UNIX/Linux command line prompt uses the nohup (no hangup) command to allow the process to keep running even after I have terminated my connection with the machine (in this case, helios).  The nice command is simply good manners, meaning that I'm willing to share the computer's resources as needed until my process is finished.  sswidl doesn't work here; only idl.  The arrows indicate I/O redirection, the batchfile is what is being run, the ampersands indicate that this is all done in the background (allowing me to work on other things in the meantime if I wish to), and log is the file to which all output is redirected.  The corresponding syntax for doing this with a general program.pro in IDL is:
nohup echo program | nice idl >& log &

June 22, 2009
     I spent today working on BOTH of my major projects.  In FLUX world, I worked (and am still working on) running relaxations and tracings which will allow me to compare specific fluxons with specific results from the DA data cube.  This will let us see the actual differences between our results in visual form, rather than simply via plots and charts of numbers.  I'd like to see if I can include color in some of these plots to help make them a bit more visually appealing as well.
     On the coronal tomography side of things, I've added the background implementation code into the test program that's run as a test-wrapper for Dr. Kankelborg's smart3d inversion program.  However, the actual background subtraction may not be working as well as we'd hoped.  If the background is not successfully zeroed (completely, or very nearly so), then the smart3d program appears to choke and be unable to converge on a solution.  I'm working on making the background selection and subtraction a bit more effective, but I need to do a bit more brainstorming to ensure that I fully understand the options which are available to me.

June 23, 2009
     Having implemented the wrapper-test program (test5.pro) to include the background, perform the subtraction and perform the inversion algorithm with smart3d, it appears that if the background is not adequately zeroed (or very nearly so), smart3d chokes and is unable to converge to an acceptable solution (however, in some cases, the chi-square statistics were rather poor, but the actual reconstructed results looked quite nice).  I'll be spending today trying to tweak my implementation and help smart3d converge to a solution.
     Also, I'm reviewing a bit of FLUX/PDL terminology and syntax so that I can ultimately generate some images which show a fluxon aligned with its corresponding field line in the DeVore/Antiochos results.  More to follow on how this develops.

June 29, 2009
     I worked on comparing the FLUX results line by line with the DA results in different colors, which I finally did successfully today!  Hooray.  Also, I spent a bit working with Dr. Kankelborg on some alternative ways to model the background for subtraction.  However, I have yet to play with the smoothing parameter in smart3d, which is the next step.

July 2, 2009
     Varying the smoothness parameter did little to affect the resulting convergence (or lack thereof) of smart3d to a reduced chi-square statistic of unity, although the final reconstructed volumes were impressively similar.  I attempted to recalculate the chi-square statistics by comparing the original projections to the sum of the foreground and background model, but to little avail.  Planning to use my time today to work on developing some residual measures of the source of divergent error (e.g., foreground or background?), for the sake of analysis and possibly improvement.  Are we expecting too much from smart3d at this point?  TIme will tell.