AP Corner: Let’s Get You Calibrated!

by Don Selle

How to Capture and Process Astro-Photography Calibration Frames

Let me start out by being blunt. If you want to improve the results of your astrophotography or your EAA, you must learn how to take and apply calibration frames to every target frame you take before you align and stack them. This is such an important step that experienced imagers refer to calibrating their sub-frames as pre-processing them. In other words, calibrating your sub-frames is so important that it must be done before any image processing is done.

 

Raw image above – Calibrated image below

 

Many beginning astro-photographers are either ignorant or ambivalent about taking and applying calibration frames to their hard-won target sub-frames. This is especially true for those of us who set up and take down our imaging rigs (which is most astro-imagers). 

I know this firsthand. Taking and applying darks and bias frames was straightforward for me. I could capture them ahead of time and applying them was easy in any of the astro-image processing apps that I used.

220701km_calibrated.jpg

But flats? That was a different thing all together. It seemed to me that they were difficult to take, adding to my difficulty, I was regularly making changes to my imaging rig, so set up was always a bit of an adventure, 

All Taking flats at evening twilight required being in focus and getting my initial focus required the sky be dark enough that shooting flats was out of the question. 

Dawn flats were also a dodgy option, as I was either taking my rig down and heading back to town before dawn, or (especially in the winter months at the HAS dark site) low clouds or ground fog covered the skies at dawn! 

Fortunately, especially for flats. There are good practices that will help you to efficiently add calibration frames to your imaging workflow and improve your astro-images. But first, a few basics.

What are Calibration Frames?

Calibration frames are sub-frame images which are used to condition each of your target sub-frames to ensure that each target sub-frame contains only the signal in each pixel (which also includes random noise) and ensures that the signal is normalized so that major differences in the signal from individual pixels due to uneven illumination and differences in sensitivity are minimized. 

There are three different types of calibration frames. These are:

  • Bias frames which are used to set the “zero point” for each pixel. These also help eliminate fixed pattern noise which every camera sensor has.
  • Dark frames which are used to eliminate the “thermal noise” which accumulates in each pixel during long exposures.
  • Flat frames are used to eliminate the effects of uneven illumination of the sensor due to the characteristics of your OTA and defects such as dust on the optical surfaces in front of the sensor (called dust donuts).

Why are Calibration Frames Necessary?

If calibration frames are so important, why don’t I need to take them for my daylight photography? The answer is two-fold. 

Firstly, modern digital photography does consider all three types of calibration, though differently than we need to do it for astrophotography. It’s done mostly in the background or is accounted for in post processing software.

Every digital camera, whether it be in your phone, a point and shoot or DSLR camera contains a digital signal processor. Much of the work done in the camera revolves around issues that crop up in normal photography, typically pattern noise removal and color balance at a minimum. Many DSLRs will also include long exposure noise reduction and dust delete routines in the camera. Higher end cameras are also able to adjust for the optical properties of a given lens (even illumination and distortion corrections) in the camera. Lens correction is also very prevalent in photo processing software.

Secondly. Because astro-imaging involves very low light and low contrast levels, any “noise” in your signal can be very significant. When you process your images to make them pleasing to look at, you are stretching and adjusting the contrast in the image. When you do that, any noise in the image will become obvious. The non-random noise generated in the imaging process can be removed and if it is not, it will dominate the total noise in the image. If we can eliminate the non-random noise in our sub-frames using calibration frames, it will improve the image and we should do so.

What are Bias Frames Why do I Need Them?

Electronic imaging sensors whether CCD or CMOS are semiconductor chips which contain a grid of light sensitive sites or pixels. When individual photons strike the pixels, they liberate one or more electrons which due to the design of the chip are trapped in the pixels. As more light liberates more electrons, each pixel develops a small static electric charge (or voltage) which is proportional to the amount of light that has struck each pixel. 

Additional electronics either on the imaging chip in the case of a CMOS sensor or separate from the chip in the case of a CCD sensor, reads out the voltage from each pixel, converts it to a digital number that represents the voltage, and sends than array of digital numbers to be used to create an image. This readout process requires a minimum voltage to work, which is added to the voltage when it is digitized.

This “bias voltage” is what we are trying to remove from our sub-frames. Not only does it slightly boost the value of each pixel, but it also varies slightly as the readout process occurs. This variation in voltage can follow a fixed pattern, and the readout process also adds some random noise. This is typically called read noise or shot noise. 

Bias frames captures only the fixed pattern and read noise generated by each pixel on the sensor, with no signal. They can then be used to subtract this noise from each sub-frame, leaving only the signal (which includes other noise we will deal with later).

How Do I Take Bias Frames?

Bias frames are quite easy to take and can be captured at any time prior to your imaging session. Cover the camera to ensure no light hits the sensor. Though sensor temperature is not a major factor for bias frames, if possible, the camera should be at the same temperature you plan to shoot your target. You should also set camera parameters such as binning, gain or ISO to the same settings you plan to use for your light frames. 

Then shoot 30 to 60 frames at the shortest exposure your camera is capable of. It is good practice to have an interval of 3 to 5 seconds between each exposure so that the camera has some time to settle between them.

Once you have the bias frames, you will create a master bias frame by stacking and combining them. I typically average the pixel values rather than use an algorithm that rejects outliers in the data, such as sigma reject. You can use your master bias frame to calibrate your data from several imaging sessions, though it is good practice to periodically create a new master bias frame camera response can change slightly over time.

All About Dark Frames

In addition to shot noise, digital imaging sensors are also subject to accumulating “dark current” during long exposures. Dark current is the result of thermal energy (rather than photons) stimulating electrons to enter the pixels on the imaging chip. 

During a long exposure, this dark current builds up at a steady rate which increases linearly with the temperature of the imaging chip. This dark signal is non-random, and it also comes with its own fixed pattern noise which comes in the form of hot and cold pixels.

Both CCD and CMOS sensors collect dark current. One of the advantages of CMOS sensors compared to CCDs is that their design allows additional circuitry to be included on the chip that will reduce the amount of dark current the chip collects. While CMOS imaging chips are less noisy, long exposure light frames taken with them still should be dark subtracted. 

In truth, for the cooled CMOS cameras I own, I typically do not apply a master dark to images shorter than 30 seconds, because the amount of dark current is so low that it is not obvious, though I do apply the master bias to remove the fixed pattern noise which can be obvious. (More on this below when we discuss flats). 

With my DSLR which is not cooled, I will use darks for all long exposures for which the in-camera image processor offers the option for me to do so. Failing to do this shows me clearly how well cooling a camera works to reduce dark current.

Taking dark frames with a cooled camera is very similar to taking bias frames and can be done at any time. Make sure that no light gets to the imaging chip and make sure the temperature is the same as which you intend to cool the camera to take your light frames. Be sure the temperature is stable. All other settings for you light images, exposure time binning and gain should be the same as shooting light frames.

I will typically take sixteen darks for each exposure time and camera temperature I plan to use when imaging and create a library of dark frames and master darks. Since the signal to noise ration improves by (dvent.)  the square root of the number of frames stacked, sixteen darks when combined into a master dark improves the signal to noise ratio to be about 4 times better than a single dark frame.

When I am imaging deep sky targets with my DSLR (this also applies to an uncooled astro camera) taking darks is a little bit more involved. It is unlikely that the ambient temperature will be steady so the temperature of camera will change throughout your imaging session, and the dark current will change with it. 

First and foremost, you should have your camera set to save your images in RAW format. If you are controlling your DSLR with an astro-imaging app, storing your light frames in FITS file format is equivalent to RAW. Your DSLR will have a setting on it for long exposure noise reduction (different than Hi ISO NR which you should avoid as it eats stars). In long exposure NR mode, after you take a long shot the camera will automatically take a second shot of the same duration with the shutter closed (a dark!) and subtract it from the first frame before storing the corrected image.

The dventage of this is that the dark is taken at the same time as the light, so temperature issues are minimized. The disadvantages are that this doubles the time spent on a single light frame, and without the benefit of averaging multiple darks, the random shot noise may be increased!

In this case, to get the best possible darks, and to increase the efficiency of my imaging, I take darks during my imaging session as follows. After I have taken my flats, I will take at least 5 dark frames before starting to take my light frames. Since I normally will refocus periodically during my imaging sessions (imaging segments of an hour and a half works for me) I will first take at least 5 more dark frames before I refocus and continue imaging. That way, if the temperature has changed while I was taking light frames, the darks will tend to average out the effect of the temperature change. Additionally, if I am taking 3-minute subs, taking 30 lights and 10 darks takes 2 hours, versus 3 hours if I used the long exposure NR setting.

When I process the light frames, I use the dark frames shot before and after an imaging segment to make a master dark and apply it to the segment the dark frames bracket. It’s a little more work, and does take up time during the imaging session, but the darks are taken at the approximate temperature of the camera during that time segment.

When making your master darks, just like bias frames, it is best to use averaging when stacking and integrating your dark frames.

Flat Frame Basics

220701km_master-flat.pngNow we come to the fun part – flat frames. As noted in my introduction, taking flats has its challenges but with a good workflow, it can become routine.

 

Master Flat which corrected the RAW image above

Every optical system whether it be a telescope, or a camera lens has distortions. Uneven illumination, also known as vignetting, is a distortion where the center of the field of view is much brighter than its edges and corners. In some cases, it can also be very asymmetrical. Fortunately, both vignetting and asymmetrical illumination can be corrected with a flat field image.

Anything in the imaging train can cause uneven illumination. This includes any flattener or focal reducer any filters and even the internal structure of the camera itself. The camera sensor will also have a cover filter of some sort in front of it. All these items can distort the light or may have dust or other debris on them that will cast shadows (dust donuts) on the sensor. The dust donuts vary in appearance, with the very small and dark ones caused by dust near or on the sensor, with the bigger more diffuse ones caused by dust closer to the telescope objective.

A good flat field image captures the uneven illumination. When you look at it, you should be able to clearly see the pattern of the uneven illumination. You will very likely also see dark splotches or “dust donuts” in the flat. When you brighten one of the light frames you took during the same session as the flat, the same pattern and dust donuts will be visible there to. These are the defects the flat frame is meant to correct.

To make the correction, you will take several flat field images, and combine them to make a master flat. When used to calibrate your light frames the software you use will first normalize the brightness values from each pixel in the master flat. The brightest pixels will get a value of 100% while all of the other pixels get a percentage value less than 100% which is relative to the brightest pixels.  When the master flat is applied to a light frame, each pixel in the light frame is divided by the same pixel in the master flat. As a result, the vignetted areas in the light frame are normalized to the brightest parts of the image, and the illumination in the light frame is said to be “flattened out”.

Because we are going to use the flat frame to correct a well-focused light frame it is important that when taking flats, we ensure that we do so with our imaging train set up just as if we were taking light frames. This means:

  • The camera should be at or very close to critical focus. 
  • If you use multiple filters in your imaging train, you will need to take flat frames through each filter. As focus can vary slightly with different filters in line, the camera should be focused for each filter you plan to image with. If you have already determined your filter offsets, this will be very helpful when you shoot your flat frames.
  • The camera and all elements of your imaging train should be the same and in the same position when taking the flats as when you shoot your light frames. If after you shoot your flats, you rotate the camera, your focal reducer or field flattener before you shoot the lights, there is a high risk that your flats will not match your lights. Any dust donut will not be corrected, and your flat may also create bright ghosts of them elsewhere in your light frames!
  • When you take your flats, you want to make sure that they are bright enough but that no pixels are either saturated or too dark. A good rule of thumb is that the maximum pixel brightness should be 1/2 to 2/3 of the maximum ADU count. For 16-bit cameras (65k brightness levels) this means the count should be about 30,000 – 45,000. For 14-bit cameras (16k) this means 8,000 to 12,000.
  • If you also have a cooled camera, it is also good practice to chill it to the temperature you plan to image at. In our humid climate, it is not unknown for the sensor cover lens to dew or frost up. This can result in weird patterns appearing in your flats, and they will be unusable. If you cool your camera well before you plan to take your flats, you will have plenty of time to fix this issue.

220701km_frost.jpg

Flat frame – showing frost or dew on the sensor

There are several ways to capture flat frames. The most common are:

  • Twilight flats – These can be taken at both dusk and dawn
  • T-shirt flats – These are taken in daylight. A white cloth of some kind, usually a clean T-shirt is stretched across the telescope objective to dim and diffuse the light of the daylight sky. A variation on this is a flat box which has a light diffuser on one end, and the other end is placed over the telescope objective with the daylight sky used for illumination.
  • Flat panel – a luminescent panel, such as a may be used to back light signs or a laptop screen are set to display a uniform grey color, and this is placed in front of the telescope objective and use as a flat light source
  • Dome flats – a white surface in front of the telescope objective is evenly illuminated and used to take flat frames. There are many variations of this one. 

I personally use the white card from a photographic white balance card set and place it in front of the objective or camera lens with it fully illuminated by the sun, then take flat frames. (https://www.bhphotovideo.com/c/search?Ntt=vello%20white%20balance%20card%20set&N=0&InitialSearch=yes&sts=ps)

You should always inspect your flat frames before you stack and combine them into a master flat. Including a defective flat frame in the stack can result in incomplete flattening or odd patterns showing up in you calibrated light frames. If you shoot twilight flats, you should use the median combine method. This way any stars that show through the twilight will be removed from the master flat.

If you use a one-shot color (OSC) camera, you should be aware that using the twilight sky may slightly change the color balance of your light frames after they are calibrated using the twilight flats. While you see the twilight sky overhead as a grey color, it is actually still blue. This results in a slight color offset when your light frames are calibrated from what would have resulted if your flat was taken with white light. This is usually not a problem or may not even be noticed, since it can be easily corrected in post processing, and most post workflows include color balancing.

I personally have tried all of the methods above except the flat panel and have decided that I get the best results with twilight flats for my monochrome CMOS camera and my cooled OSC camera and a version of dome flats for when I use my DSLR.

Capturing Flat Frames – A Workflow

Capturing flats is straightforward once you develop a workflow and you follow it whenever you set up your imaging rig. Here is a suggested twilight flat workflow you might consider. The objective is to capture 15 flats of 1- or 2-seconds duration for each filter you plan to use with exposure values such that the max pixel values are between ½ and ¾ of your camera’s saturation value.

For a cooled mono camera or a OSC camera:

  1. Before your imaging session, spend an evening during the full moon focusing your camera several times with every filter and keep track of the focuser position. Use this data to determine the focus offsets for each of your filters referenced to the luminance. You will use these later to focus your system before taking your flat frames.
  2. Start setting up early enough that you are done at least an hour before Civil Twilight. Start slowly cooling your camera and periodically take a very short image to see if your camera is dewing or frosting up. 
  3. Select the darkest filter you plan to use during your imaging session. For my camera and filter wheel, the darkest filter is SII, followed by Ha, OIII, Blue, Red, Green, Luminance. 
  4. As the time is approaching Nautical Twilight, slew your scope to the zenith. Turn your mount tracking off. 
  5. Using the focus data you collected, determine the critical focus point for this filter then move the focuser to that. 
  6. In your imaging software, set up a series of exposures (I prefer 15 flats for each filter) for your darkest filter, and be sure to set up auto-save for them. You may want to add 3-5 seconds delay between each exposure.
  7. Start taking test images periodically (do not save them) so that you can time when to start taking your flats. 
  8. When the exposure gets to the ¾ point, turn auto-save on and start your image series.
  9. When the series completes, slew your telescope about 10 degrees to the west, make sure that the tracking is off. Select your second darkest filter, then repeat from step 5, until you have captured flat frames for all of the filters you planned to use.

This workflow can also be used to shoot dawn flats if required.

For a DSLR:

  1. Set up your imaging system such that you are completed at least an hour before Civil Twilight.
  2. If you are using narrow band or triband filters with your camera, start taking your flats with the filter in place. 
  3. Move the focuser to the critical focus point for the filter you have in place.
  4. Facing east (sun at your back) hold the white card in front of the objective. Be sure that there are no shadows on the white card and that it is large enough and close enough to the objective to more than fill the field of view of the camera.
  5. Take and save at least 15 flat frames with the max pixel value between ½ and ¾ of the camera saturation value.
  6. Repeat for each filter.

If you shoot and save your sub-frames in RAW you will have the added advantage that you can use the best flat as the reference frame to set a custom color balance for your imaging session. 

Applying Your Calibration Frames

Once you have your master calibration files you will need to know how to apply them. 

  • For light frames from a cooled CCD Camera:
    • Take the following sub-frames
      • darks for you light frames (same temperature and duration as your lights)
      • darks for flat frames (same temperature and duration as your flats)
      • Bias frames and use the to make a master bias
      • Flat frames (same temperature as your lights)
    • To create your master dark frames (both lights and flats) – subtract the master bias from each dark frame. Use a sigma reject integration algorithm.
    • To create your master flat frame – subtract your master bias and your bias subtracted master flat dark from each flat frame. If you shot twilight flats, use the median combine algorithm to remove any stars which might have peeked through.
    • For your light frames you should subtract your master bias and dark frames then apply your master flat frame.

220701km_process.jpg

(Flow chart from the DeepSkyStacker website)

  • Cooled CMOS Camera – if you are using a cooled CMOS camera, applying calibration frames is basically the same as for a cooled CCD camera with one exception. Since modern CMOS cameras have built-in dark current limitation,  if your flat frame exposures are relatively short (say 15 seconds or less) you can probably skip taking and applying the flat darks.

220701km_process2.jpg

(Flow chart from the DeepSkyStacker website)

  • Uncooled CMOS Camera or DSLR – for these cameras, the alternate calibration process above can most likely be used, though you should check the dark noise level for short exposures to see if taking and applying flat darks is necessary. Also, recall that the suggested method for capturing darks with these cameras centered on taking them progressively throughout the imaging session to better handle temperature changes. You will need to make a master dark for each imaging segment and apply it only to those light frames bracketed by the dark frames.

With a little bit of understanding and some practice capturing your flat frames, you should have no problem calibrating your light frames and getting them prepared for further processing. 

Happy imaging!

HAS Online Store

Get Connected!

HAS has begun using RainedOut, a text message service, to communicate late-breaking news about events. Click here to learn more and subscribe!

Night Sky Network