This is a collection of mostly astronomical, mostly Python scripts for doing assorted tasks. Since these are almost exclusively intended for my own use, I don't usually bother with any of the niceties of tidying them. They tend to be well-commented though. Using them normally consists of opening the file in a text editor, modifying the user-defined parameters (usually highlighted in their own section at the top of each script), and then just running them as a normal code (e.g. "python nameofthescript.py"). They are not fancy and they are certainly not guaranteed to be robust, so be advised to at least skim through them before running !
For even shorter sections of code that just demonstrate how to do a single operation (and may not even execute on their own), see the snippets page. The scripts here are self-contained, though some are standalone and some are for Blender. The latter are given as .blend files rather than .py files. A very few .blend files are included here which don't include Python scripts but I like 'em anyway.
Some scripts are so old that they use the standalone pyfits module from before it was incorporated into astropy. For these, "import pyfits" can normally just be replaced with "from astropy.io import fits as pyfits" and it should all work.
Astronomy tasks
AGES cataloguing : Takes separate files containing inputs for mbspect and runs mbspect using each of them. Saves a file containing the input parameters and mbspect measurements, the ASCII log file of the spectra, and postscripts. An older version for Python 2 is also available (may have more bugs).
Photometry : Programs to assist in bulk downloading and naming of SDSS FITS files, as well as automatically generating DS9 regions for aperture photometry.
Wedge plotting : Blender 2.49 file. Reads in a file of RA, Dec, velocity and makes a wedge plot (aka pie slice). You'll have to make your own axes or modify the ones provided.
Get SDSS RGBs : download multiple SDSS RGB images from a table of RA/Dec coordinates.
Radial profile : measures the radial profile of a source in a FITS file.
Integrate rings; Iso disc : set up discs of particles with density variations given by a user-input radial profile (can combine with RadialProfile.py). Useful when setting up simulated galaxies with radial density profiles taken from observations.
Particle viewer lite : Blender 2.49 file. SPH/n-body particle viewer, adapted from the version in the old series of FRELLED to use linecache. This can be a lot faster than the standard version and makes for very small .blend files, but linecache needs a lot of memory.
Spectra plotter : Plots artificial spectra to demonstrate how integration drives down random noise but doesn't help with systematic biases. Generates spectra sequentially and averages them to mimic integration time. Noise is random (optionally with some baseline function) with a simple tophat signal injected over a range of channels, set to be too weak to be visible in any individual spectrum.
FITS operations
Cleaning
FITS baseline; FITS baseline S/N : Fits and subtracts nth-order polynomials with sigma clipping on spectral axis of data cubes. The S/N version divides the spectra by their rms, returning a signal-to-noise cube rather than flux (new version uses the robust noise, old version uses the standard rms and gives weirdly inconsistent results). Also available in IDL versions : FitsBaseline.pro, FitsBaseline_SN.pro
FITS masked baseline : Takes two cubes as input, one with sources/RFI masked, one with no masking. Does the polynomial fitting as above on the masked cube, then subtracts the result from the unmasked cube. This can give better results if a large portion of the spectral baseline contains signal.
Fit median baseline : Measure the median of the spatial (x-axis) bandpass and subtracts it.
Simulate MINMED : Chops the spectral bandpass up into boxes and measure the median in each box. Finds the minimum of the medians and subtracts it from the whole scan.
Simulate MEDMED : As above, but uses the median of the medians instead of the minimum of the medians.
User MED : Takes two cubes as input, one with sources masked, one without. Measures the median of the spatial bandpass in the masked cube, then subtracts this from the unmasked cube. Since the user has direct control over what to mask, this can use the maximum possible amount of the bandpass, so may be more accurate than MINMED or MEDMED.
AGES Cleaning : Combines all of the above and more to make it simpler to generate a cleaned, science-ready AGES data cube, which is a lot easier than running half a dozen different programs to get a single output file. Allows Hanning smoothing (using miriad), baseline fitting, S/N conversion, MINMED/MEDMED spatial bandpass correction, and keyword editing (defaults to ensure output is miriad-friendly as AGES cubes sometimes aren't).
Other FITS operations
FITS 2 Text : Converts a FITS file into ASCII file, consisting of rows of the format x, y, z, value.
FITS 2 Text Cube Slicer : As above, converts a FITS cube into a series of ASCII files, but this time one file per channel.
Text 2 FITS : Converts ASCII to FITS. Not very robust though. Assumes format of of the columns of x, y, z, value as above. Missing values are replaced with NaNs.
Flux Inject : Adds flux into a data cube at the specified location range, over a simple set of rectangular coordinates.
Gauss Flux Inject : Adds flux with a 2D Gaussian profile. Simple demonstration that adds a 2D profile over a rectangular region in a single channel.
De Gaussing : Three programs to remove Gaussian-profile data from a cube, e.g. typical unresolved sources.
1) DeGauss removes a Gaussian function. User can specify input parameters or allow them to vary (chi-squared minimization), e.g. centre coordinates, FWHM, peak value. Works on multiple channels in 3D data cubes.
2) DeGauss 2D : Only works on 2D data but can also (unlike DeGauss) iteratively fit ellipticity and position angle.
3) DeGauss 3 : removes Gaussians from a data cube without doing any fitting, just using the user-specified parameters exactly.
Interpolate : Interpolates extra slices of a FITS file along the spectral axis, i.e. interpolates additional channels. Linearly interpolates between values. User can specify by what factor to stretch the cube. Useful for data visualisation when one requires cubes of equal dimensions along all axes.
LatLong –> XYZ : Crudely converts cubes of latitude-longitude format into XY format (see "non-Cartesian data" below).
Blank FITS : Creates a blank FITS file with the same header information as another.
Add Header : Add FITS header keyword. Can also be used to edit existing keywords.
Clear FITS History : Simple modification of the above script. Takes a FITS file, removes all the "history" keywords in the header (which can often be extremely numerous), and outputs a new FITS file with the modified header.
Head Transplant : Another simple modification to FITS files, transplanting the header from one, the image data from another, and outputting the combination as a new third file.
FITS Clip : Sets all values in a FITS above a specified threshold to zero (use this for setting values in an array in general). Outputs a new file, leaving the original alone.
Channel Clip : Set all values in all channels in a FITS file, outside a specified range, to zero.
FITS Edit : Demonstrates basic FITS image editing (e.g. changing image values of specific pixels).
FITS Multiply : Takes two FITS files and multiplies them together.
Log FITS : Converts FITS flux image values to logarithmic units.
Fix Axes : Takes data with 4 axes and converts it to 3, which can be much easier to process.
Measure Image RMS : Measure the rms of a FITS image using the spatial bandpass. Useful for comparing the effects of processing when the spectral rms is unaltered.
Fake Stacking : Three scripts to help illustrate the process of stacking. This is an extremely counter-intuitive process which doesn't improve sensitivity in quite the way that many people assume it does, me included. An article on my blog attempts to clear up some of the confusion, or of course see this webpage on this site.
Fake Stacking 1 : Generates a given number of random-noise spectra of a specified length, stacks them, and outputs the stacked profile.
Fake Stacking 2 : As above but includes the option to include signal injection over a specified channel range. Outputs both the stacked and individual profiles.
Fake Stacking Mass : This is supposed to illustrate the difference in how stacking improves sensitivity to average mass but degrades the sensitivity to total mass, but I can't remember how.
PNG 2 FITS : Takes a sequence of (greyscale ?) PNG images and converts them to a single FITS cube. A version which works with colour images is available here.
Replace NaN : Replaces all NaN's in a FITS image with zero. Very inefficient though !
Check Header Keyword : Check if a keyword is present in the FITS header.
FITS Extract Volume : Masks arbitrary-shaped regions in a FITS file by reading in coordinate data from a file (format x, y, z, value). Designed to be used in conjunction with FRELLED.
Particle Gridders : Grid SPH on n-body particle data in position-velocity FITS cubes. Gf Gridder operates on individual files, while Gf Multi Gridder is for batch processing and works on files in multiple subdirectories.
Non-Cartesian data
LAB Create PPP : Takes a position-velocity FITS cube in Galactic coordinates and converts it into a true distance cube of the specified pixel and physical resolution. Unlike the crude "LatLong - > XYZ" version in the above section, this one operates backwards. First it creates the PPP cube, then it calculates where each pixel should be extracted from in the original data. Calibrated on the coordinate system of the LAB data. Detailed explanation on this blog post.
HI4PI Create PPP : As above, but designed for the gargantuan HI4PI data. The user must first split the data into a series of FITS files each containing a single velocity channel (Miriad is good for this as it's excellent at working with obscenely large data sets even on small systems).
FITS Sequence To PNG : Modified, standalone version of the script in FRELLED for converting FITS files to PNG images. Designed for batch processing a sequence of single-channel FITS files, such as required when processing enormous data cubes.
Spherical Pipeline (.blend file) : FRELLED is designed for working with data in Cartesian coordinates. While the above scripts help convert Galactic data to Cartesian format, it's also possible to display the data directly in its original format. This works by creating a series of nested spheres instead of image planes. Full instructions are given in the file. Accompanies this paper see also this blog post). This file is designed for all-sky data and is relatively easy to work with.
Conical Pipeline (.blend file) : As above, but designed for conical data, not typical in observational astronomy but used in some numerical disc simulations. Slightly more complex to work with but enables the user to provide lookup tables (this example requires this file and this file). This means the input data files don't have to conform to any particular standards and can be stretched in a somewhat arbitrary manner.
Blender operations (Blender versions 2.78 and similar)
Create Matrix Text : Create a grid of Matrix-style text values from an input file. Because why wouldn't you want to make data look like it was from the Matrix ?
Animate Matrix Text : Animate a grid of Matrix-style text from a series of input files.
Animate Visibility : Animate object visibility in the 3D view and/or render display based on current frame. Designed for use with the Matrix text scripts above but could very easily be adapted.
Create Heightfield Meshes : Create a heightfield (a.k.a. height map, displacement map) mesh from an input text file, designed for processed FITS files. There's not normally any point in this since Blender has in-built tools for displacement, though it is useful in specialist cases requiring very detailed control over the scaling.
Animate Heightmap Meshes : Animates heightfield meshes using a series of input text files.
Animate Clip Alpha : The "clip alpha" value is one of the few parameters in Blender that can't be keyframed, so here's a way to work around that.
Simple layer animation : Controls the layers an object is visible on using a "frame changed" handler for animation.
X-ray (.blend file) : Not a script but an example of an "X-ray" style material. Very old technique which is hard to find described properly elsewhere so I want to make sure it's still available.
Modify Modifiers : Script to set the start and end frame of a "build" modifier.
Heightfield lines : Modification of the above heightfield scripts to replace them with individual lines. As above, designed for working with data converted into ASCII from 3D FITS files.
Create Lines : Creates one line per declination slice.
Animate Lines : Animates the visibility of the lines by controlling their layer visibility.
Create Sweeping Lines : As with Create Lines, but offsets each sequential line according to the spectral channel.
Miscellaneous
Batch Image Convert : Resize/reformat multiple images using parallel processing. Useful for storing smaller copies of your photographs (files may be overwritten !).
Drop Time : Super-simple numerical calculation of freefall time under gravity varying with distance. Unnecessary, can be done analytically, but never mind.
G-Force : simple gravity simulator for Blender (non-Blender version here).
Copy and Run Files : Very simple script. Takes the contents of one specified subdirectory and pastes it into all other subdirectories, and then executes code from within each subdirectory. Useful for running batch processes that you didn't originally anticipate requiring.
Matplotlib Plotting Examples
A set of standard scripts I use for generating plots with the matplotlib module. I hate matplotlib. It's one of the least-intuitive standard Python modules of all and in my view the whole thing should just be re-written, but needs must. Anyway the default standards, especially the font, are just ugly and not nice enough for publications, so I use these as my go-to for making things look less clunky. Pretty sure I've got better versions somewhere but these are the ones I could find.
Some of these scripts rely on real data files which are not supplied. Use these only for copying and pasting the relevant code.
Scatter Plot : Plots a series of unconnected points, but see below for a much better example.
Function Plot : Plots a mathematical function as a line.
Multiple Line Plots : Plots multiple lines from different data files, overlaid in one plot.
Bar Chart : Plot data as a series of bars of different heights and fixed widths.
Histogram Plot : Bins the data and plots a histogram.
Better Scatter Plot : Plots scattered, unconnected points and a solid and dashed line. Has a nice serif font for both the axes labels and numerical ticks, and shows tick marks on sides of the plot.
Archival (1) : FITS files in VR with Blender 2.78
These are probably obsolete since FRELLED now includes script to export both volumetric FITS data (image sequences) and isosurfaces to Blender 2.91, which supports VR inspection directly. However if you just need to create pre-rendered VR content, e.g. 180 or 360 degree stereoscopic videos, this might be of some use.
You should probably first be familiar with FRELLED to use these files ! Though these are actually much simpler. This set of files takes image sequences (e.g. .png files) and loads them in Blender with Cycles materials, which you can then render in VR (or normally, that's fine too). This is useful for viewing 3D volumetric data by loading in slices of the data with transparency. All are .blend files, not standalone scripts.
Cycles.blend : Main file. Loads sequences of images with transparency based on colour. Ideally use 3 sequences, one for each orthogonal projection through the data.
ConvertXY.blend, XZ.blend, ZY.blend : Combines separate images for transparency and colour into one sequence for each projection. No script – just edit the two image textures on the plane (and the scaling and image size).
SingleImageMultiVolumeConversion.blend : Designed for multi-volume data in FRELLED, which uses four (or more) images – two pairs of transparency and colour, which could be based on the same or different data sets in order to overlay them. This file converts them into a single image sequence, which uses a lot less resources to display. Again no scripts needed, just edit the image textures on the planes.
FBXConversion.blend : Some guidelines and scripts for converting Blender files into FBX files that can be imported into Steam VR Workshop, which is vastly more powerful than Blender for creating VR content. Blender objects should have shadeless UV mapped image texture materials. This is not a simple one-shot conversion script (I'm not sure that's really possible) but it should make the procedure much more straightforward.
Archival (2) : Basic FITS files in Blender
Old Python scripts for Blender <= 2.49, which load in FITS data cubes using a couple of primitive methods and one which is more sophisticated. But please, don't use these. They're extremely old and FRELLED is ten million times more sophisticated !
Point Clouds : Loads all points in a FITS file above a specified value as simple vertices. A bit more sophisticated than just the binary choice of load or don't load the values though : it effectively bins the data depending on the data values, and loads a different mesh for each bin. Each mesh object is given a different halo material. This means the realtime view looks pretty ugly, but the rendered view can look quite nice.
FITS Cubes : As above, but instead of loading one vertex per voxel, it creates faces connecting each point, i.e. each voxel is shown as a true cube in Blender. Much better real time display, but takes an enormous amount of GPU power. Only really any good for very small data sets.
Basic Volumetrics : The correct way of displaying volumetric data in Blender's real time view, by using textured image planes. This is the basis behind FRELLED. I'm preserving this script for archival reasons only; there really is no point in using this one any more.