Plotting and Interpreting the Celestial Line of Position

This category explains the methods that transform the tabulated and inflight observation values into an aircraft position. The navigator is faced with two tasks: plotting the resultant information onto a chart and resolving this information into an aircraft position. There are two basic methods of obtaining a line of position (LOP): the subpoint method and the intercept method.

Methods of Plotting and Interpreting the Celestial Line of Position

Subpoint Method

A detailed explanation of the theory concerning the subpoint method is in the Computing Altitude and True Azimuth category, and in the Celestial Precomputation category. [Figure 11-1] Following is a summary of the steps involved:

1. Positively identify the body and measure the altitude using a sextant.
2. Because no tabulated information for azimuth or elevation is required for this method, corrections for refraction, parallax, semidiameter, wander error, and sextant correction are applied directly to the Ho.
3. The resultant measurement is subtracted from 90° to obtain the co-altitude (co-alt). To convert to NM (1°= 60 NM), multiply the number of degrees times 60. Any fractional portion of degrees is added to the previous value.

Figure 11-1. The subpoint method.

Example: Vega is observed at an altitude (Ho) of 88° 23′. Sextant correction is –03′.

88° 23′ – 03′ = 88° 20′
90° – 88° 20′ = 1° 40′
1° 40′ = 60′ + 40′ = 100 NM

In this example, 100 NM represents the distance from the observer’s position to the subpoint of the body. The coordinates of the body are its corresponding declination (Dec) and Greenwich hour angle (GHA). For this example, Vega’s Dec is N38° 46′. The GHA is obtained by applying the sidereal hour angle (SHA) of Vega to the GHA of Aries.

Example:

SHA = 080° 59′
GHA Aries = 039° 18′
GHA Vega = 120° 17′

Subpoint of Vega is located at 38° 46′ N l20° 17′ W. The observer is now ready to apply the information:

1. Plot the subpoint on an appropriate chart.
2. With dividers or compass, span the co-alt distance; in this case 100 NM.
3. Use the body’s subpoint (38° 46′ N l20° 17′ W) as the center and 100 NM (co-alt) as the radius. The circle is called the circle of equal altitude and the observer is located on that portion of the circle nearest the dead reckoning (DR) position. There are definite advantages to this method. It requires no precomputation values and plotting is very simple if the observer and body are reasonably close together. When the observer and body are separated by great distances, some disadvantages appear.
4. If a body is observed at 20° above the horizon, the observer is 4,200 NM from its subpoint. To swing a LOP from this subpoint, the subpoint and the arc must be plotted on the same chart. To permit plotting of any LOP, the chart must cover an area extending more than 4,000 miles in every direction from the DR position. This means that the chart must be either of such large size that it cannot be spread out on a table in the aircraft, or of such small scale that plotting on it is inaccurate. To cover an area 8,000 miles across, a chart 4 feet square must be drawn to a scale of about 1:10,000,000. Furthermore, measuring would be difficult because of distortion.
5. Since a celestial LOP cannot always be drawn by the subpoint method, the intercept method, based on the same principles, is often used.

Intercept Method

You can eliminate the need for plotting the body’s subpoint and still draw the arc representing the circle of equal altitude. [Figure 11-2] By using the following formula, you can calculate the altitude and azimuth of the body for the DR position:

Hc = SIN-1 [SIN (DEC’) SIN (LDr) +COS (DEC’)COS (LDr) COS (LHA)]
Z = COS (Z) = [SIN (DEC’) – SIN (LDR) SIN (HC)]/[COS (Hc COS (LDr)]
Zn = Z if SIN (LHA) < 0
Zn = 360 – Z if SIN (LHA) > 0

Figure 11-2. Line of position computed by intercept method. [click image to enlarge]The calculations may be performed quickly using a programmable calculator, or they may be extracted from the appropriate volume of the National Imagery and Mapping Agency’s Sight Reduction Tables for Air Navigation in a publication referred to as Pub. No. 249. This method enables the observer to use any of the navigational bodies available at the appropriate fix time. Here is a brief review:

• Compute a DR for the time of the position, using preflight or inflight data.
• Determine the necessary entering values for the Pub. 249 volume being used (Lat, LHA, Dec contrary, or same) and extract all the necessary values of computed altitude (Hc) and azimuth angle (Z).
• After making all the necessary conversions and corrections (Chapter 10), compare the Ho and corrected Hc. This difference is the intercept. If the Ho equals the corrected Hc, then the circle of equal altitude passed through the plotting position. If the Ho is greater than the Hc, the difference is plotted in the direction of the true azimuth (Zn). The Zn represents the azimuth from the observer’s position to the subpoint. If the Ho is less than the Hc, plot the difference 180° from the Zn.
• NOTE: If HO is MOre, plot TOward the subpoint (HO MO TO)

Example: The assumed position is 38° N, 121° 30′ W for a shot taken at 1015Z on Aldebaran. The Ho is 32° 14′. The Hc is determined to be 32° 29′ and the Zn is 120°. A comparison of Ho and Hc determines the intercept to be 15 NM away (15A).

Plotting LOP Using Zn Method

1. Plot the assumed position and set the intercept distance on the dividers. [Figure 11-3]
2. Draw a dashed line through the assumed position toward the subpoint.
3. Span intercept distance along dashed Zn line.
4. Place plotter perpendicular to Zn.
5. Draw LOP along plotter as shown in Figure 11-3.

Figure 11-3. Celestial line of position using true azimuth method. [click image to enlarge]

Plotting LOP Using Flip-Flop Method

1. Plot the assumed position and set the intercept distance on the dividers. [Figure 11-4]
2. Measure 120° of the Zn with point A of the dividers on the assumed position and place point B of the dividers down. In this case, away from 120° or in the direction of 300° from the assumed position. Slide the plotter along the dividers until the center grommet and the 100/200-mile mark are lined up directly over point B of the dividers marking the intercept point.
3. Remove point A of the dividers from the assumed position, keeping point B in place. Flip point A (that was on the assumed position) across the plotter, at the same time expanding the dividers so that point A can be placed on the chart at the 90°/270° mark of the plotter.
4. Flop the plotter around and place the straight edge against the perpendicular, which is established by the dividers.
5. Draw LOP along the plotter as shown in Figure 11-4.

Figure 11-4. Plotting celestial line of position using flip-flop method. [click image to enlarge]When using the intercept method, remember:

• For some assumed position near the DR position, find the Hc and Zn of this body for the time of the observation. This is done with the aid of celestial tables, such as Pub. No. 249 or a programmable calculator.
• Obtain needed corrections, sextant correction, refraction, etc., and apply these to the Hc by reversing the sign. Remember, we are striving to derive a precomputed value to ensure the correct body is shot. Measure the altitude (Ho) of the celestial body with the sextant and record the midtime of the observation.
• Find the intercept, which is the difference between Ho and Hc. Intercept is toward the subpoint if Ho is greater than Hc, and away from the subpoint if Ho is smaller than Hc.
• From the assumed position, measure the intercept toward or away from the subpoint (in the direction of Zn or its reciprocal) and locate a point on the LOP. Through this point, draw the LOP perpendicular to the Zn.

Additional Plotting Techniques

The preceding techniques involve the basic plotting procedures used on most stars and the bodies of the solar system. However, there are certain techniques of plotting that are peculiar to their own celestial methods; for example, the plotting of LOPs obtained by using Polaris, which is discussed later. Also, certain precomputation techniques lend themselves more readily to other plotting techniques, such as preplotting the true azimuths or plotting the fix on the DR computer.

These last plotting techniques are discussed in Pub. No. 249 in the section on precomputation. Other special techniques are discussed in the section on curves, in which the celestial observation is plotted on a graph rather than on the chart.

Interpretation of an LOP

Navigation has two aspects—the mechanical and the interpretive. The mechanical aspect includes operation and reading of instruments, simple arithmetical calculations, plotting, and log keeping. The interpretive aspect is the analysis of the data that have been gathered mechanically. These data are variable and subject to error. You must convert them into probabilities as to the position, track, and GS of the aircraft and the direction and speed of the wind. The more these data are subject to error, the more careful the interpretations must be and the less mechanical the work can be. LOPs and fixes especially require careful interpretation. It is convenient to think of a fix as the true position of the aircraft and of the LOP as a line passing through this position, but these definitions are optimistic. It is almost impossible to make a perfect observation and plot a perfect LOP. Therefore, a LOP passes some place near this position, but not necessarily through it, and a fix determined by the intersection of LOPs is simply the best estimate of this position on the basis of one set of observations. In reality, a fix is a most probable position (MPP) and a LOP is a line of MPP.

The best interpretation of LOPs and fixes means they are used, to the best advantage, with DR. But good interpretation cannot compensate for poor LOPs, nor can good LOPs compensate for careless DR. To get good results, every precaution must be taken to ensure the accuracy of LOPs and exact DR calculations.

Intelligent interpretation requires fine judgment, which can be acquired from experience. You can be guided, however, by certain well-established, though flexible, rules. The following discussion pertains especially to celestial LOPs and fixes. It also applies to LOPs and fixes established by radio and, to some extent, to those obtained by map reading.

Single LOP

Previous discussions dealt with the basic plotting of a LOP and errors in LOPs, but they did not show the actual mechanics of the plotted corrections that must be applied. The LOP must be corrected for Coriolis or rhumb line correction and also for precession and/or nutation correction if it is based on a Volume 1 star shot. Coriolis or rhumb line correction becomes a very significant correction at higher speeds and latitudes. For example, suppose the correction determined from the Coriolis or rhumb line correction table is 9 NM right (of the track). The LOP must be moved a distance of 9 NM to the right of track. This can be done either by moving the assumed position prior to plotting or by moving the LOP itself after it is plotted. (Remember, the assumed position is not used in the plotting of the LOP obtained from a Polaris observation.) Consider Figure 11-5, which shows a track of 90°. Notice that, in both methods, the corrected LOP is in the same place with respect to the original assumed position and that the intercept value is the same. The resultant LOP is the same regardless of the method used.

Figure 11-5. Two methods of coriolis/rhumb line correction. [click image to enlarge]If, in addition to the Coriolis or rhumb line correction, a precession and/or nutation correction of 3 NM in the direction of 60° is required, it would have been further applied as shown in Figure 11-6. Again, the corrected LOP is the same, using either method, because the intercept and resultant position of the corrected LOP to the original assumed position are the same. The corrected LOP alone gives very little information; hence, a position must be arrived at only after considering the LOP and the DR position for the same time.

Figure 11-6. Two methods of coriolis/rhumb line and precession/nutation correction.

Most Probable Position (MPP) by C-Plot

The MPP is just what the name implies. It is not a fix; however, since it is the best information available, it is treated as such. Notice in Figure 11-7A that the DR position and celestial LOP (for the same time) do not coincide.

Figure 11-7. Most probable position by C-plot. [click image to enlarge]Obviously, the DR information, or celestial information, or both is in error. Notice that the prior fix has no time on it. Suppose this prior fix had been for the time of 1010. It would then be very likely that most of the error is in the celestial information and the probable position is closer to the DR position than to the celestial LOP. On the other hand, suppose the prior fix had been for the time of 0900. Since the accuracy of the celestial information is unaffected by the time from the last fix, it would, in this case, be most likely that the actual position is closer to the LOP than to the DR position.

A formula has been devised to position the observer along the perpendicular to the LOP according to the time factor. The formula is:

where t is time in minutes, p is the perpendicular distance between the DR position and the LOP, and d is the distance from the DR position for the time of the MPP measured along the perpendicular to the LOP. Look at Figure 11-7B and C and see how the formula works for the two problems cited above if the perpendicular is 20 NM in length. In Figure 11-7B, t is 15 minutes and p is 20 NM, so the MPP would be located along the perpendicular about 8½ NM from the DR position.

Now, consider Figure 11-7C where t is 1 hour 25 minutes or 85 minutes, p is 20 NM and, in this case, the MPP would be over 16 NM away from the DR position along the perpendicular to the LOP.

If you prefer not to use the formula, a simple table can be easily constructed to solve for d with entering arguments of t and p. [Figure 11-8] The table could easily be enlarged to handle larger values of t and p. In most fixes, the DR position is so close to the LOP that the midpoint between these two can be considered the MPP. A good rule to use is to take the midpoint of the perpendicular if the total distance between the DR position and the LOP is 10 NM or less. If the value of p is greater than 10 NM, use a table or the formula to determine the MPP. Up to this point, determination of the MPP has been rather mechanical. Experienced navigators frequently further adjust the position of the MPP for other factors not yet considered. For example, if the LOP is carefully obtained under good conditions or if it is the average of several LOPs, you may further weight the MPP in the direction of the LOP by an amount that judgment dictates. However, the reverse may be true if the LOP is obtained under adverse conditions of rough air. In the latter case, you might move the MPP closer to the DR position by some amount determined by sound judgment.

Figure 11-8. To solve for distance.

Further, consider the validity of the DR position in relation to factors other than time. A DR position at the end of 40 minutes would be more reliable with Doppler drift and GS versus one based on metro information. These factors may also adjust the original MPP closer to or farther away from the DR position, along the perpendicular. However, these last mentioned factors are judgment values that come only with experience. In fact, with experience you may mentally calculate all the factors involved and arrive at the final position of the MPP without recourse to a formula or table.

Finding a Celestial Fix Point

Up to this point, only the single celestial LOP and what to do with it have been considered. Now, the celestial fix should be considered. To establish a fix, two or more LOPs must be obtained. Since, in most cases, two or more LOPs cannot be obtained simultaneously, they must be converted to a common time. For example, a LOP obtained at 1010 must be converted to the LOP obtained at the fix time of 1014. There are several methods for making this conversion that are discussed in this chapter. Consideration is also given to the planning of the fix and the final interpretation of the fix itself.

Conversion of LOPs to a Common Time (Part One)

Moving the LOP

One method of converting LOPs to a common time is to move the LOP along the best-known track for the number of minutes of GS necessary for the time conversions. This method is similar to that used in correcting for Coriolis or rhumb line and precession or nutation. For example, suppose the track is 110° and the GS is300 knots. LOPs are for 1500, 1504, and 1508, and a fix is desired at 1508. This means the 1500 LOP must be moved to the time of the fix using the track and 8 minutes of the best known GS. The 1504 LOP must be moved to the time of the fix using the track and 4 minutes of GS. The 1508 LOP is already at the fix time, so it requires no movement. Figure 11-9 shows the method of conversion as it is completed on the chart.

Figure 11-9. Conversion of lines of position to a common time.

If, at any time, the LOP has to be retarded (moved back) to the time of the fix, use the following procedures. Using the reciprocal track and GS, obtain the correction in the regular manner for the number of minutes of difference. For example, suppose the fix is at 1800 and the last shot is at 1802.

Retarding the LOP 2 minutes of GS on a track of 70° would be the same as advancing it 2 minutes of GS on a track of 250°.

Motion of Observer Tables

A second method of conversion of LOPs to a common time is with a Motion of the Observer table such as the one in Pub. No. 249. This table gives a correction to be applied to the Ho or Hc so that the LOP plots in its converted position. The correction obtained from Table 1 in all volumes of Pub. No. 249 is for 4 minutes of time. An additional table allows you to get the correction for the number of minutes needed. For example, suppose the LOP needs to be advanced for 11 minutes and the Ho of the body is 33° 29′ and Zn is 080°.

The track of the aircraft is 020o and the GS is 240 knots. In Table 1, Correction for Motion of the Observer for 4 minutes of Time [Figure 11-10], the entering arguments is Rel Zn and GS. Rel Zn is azimuth relative to course (Zn minus track or track minus Zn). Subtract the smaller angle from the larger and enter the table with the answer. In this case, Zn – track = 080° – 020° = 060° (Rel Zn) and GS is 240 knots. Entering this table with these arguments, the correction listed is +08′ for 4 minutes of time.

Figure 11-10. Entering arguments are relative true azimuth and groundspeed. [click image to enlarge]Figure 11-10. Entering arguments are relative true azimuth and groundspeed (continued). [click image to enlarge]Use the whiz wheel to calculate the total motion for 11 minutes. In this case, the 11-minute correction totals 22′. By applying any other correction (refraction, sextant correction, etc.), a total adjustment is derived. By changing the sign, this total may be applied to the Hc. To apply the correction to the Ho, the sign of the adjustment would remain the same. Apply the adjustment to the intercept as the rules state in Table 1. In each case, the resultant intercept would be the same.

Suppose the Hc was 33° 57′. Applying the correction –22 yields 33° 35′. Comparing this with our Ho 33°29′ results in an intercept of 6 NM away. If you decide to apply the correction to the Ho, 33° 29′ + 22′ yields 33° 5l’. Comparing this to the Hc 33° 57′ yields the same result, 6 NM away. When using the Motion of the Observer table and when the fix time is earlier than the observation (LOP to be retarded), the rule for the sign of the correction is also printed below Table 1.

Moving the Assumed Position

Another method of converting LOPs to a common time is to move the assumed position. This method is recommended for shots 4 minutes apart computed to give all three bodies a single assumed position. However, it is not limited to that type of computation. The assumed position is moved along the best-known track at the best-known GS. For example, again suppose the track is 330° and the GS 300 knots. LOPs are for 1500, 1504, and 1508 and a fix is desired at 1508. [Figure 11-11] Since the first LOP would have to be advanced 40 NM (8 minutes at 300 knots), the same result is realized by advancing the assumed position 40 NM parallel to the best-known track. The 1504 LOP must be advanced 20 NM; therefore, the assumed position is advanced 20 NM miles parallel to the best-known track. The third shot requires no movement, and it is plotted from the original assumed position.

Figure 11-11. Moving assumed positions.

It should be noted that the first shot is always plotted from the assumed position, which is closest to destination. In this method, if observations are precomputed and the assumed position is moved prior to shooting, the following procedure is used when shooting is off schedule. For every minute of time that the shot is taken early, move the assumed position 15 minutes of longitude to the east. For every minute of time that the shot is taken late, move the assumed position 15 minutes of longitude to the west. In addition, the affected LOP must be moved along the best-known track for the number of minutes of GS the observation was early or late. If the shot was early, advance the LOP; if the shot was late, retard the LOP.

Conversion of LOPs to a Common Time (Part Two) Planning the Fix

Planning the Fix

In selecting bodies for observation, one should generally consider azimuth primarily and such factors as brightness, altitude, etc., secondarily. If all observations were precisely correct in every detail, the resulting LOPs would meet at a point. However, this is rarely the case. Three observations generally result in LOPs forming a triangle. If this triangle is not more than 2 or 3 miles on a side under good conditions and 5 to 10 miles under unfavorable conditions, there is normally no reason to suppose that a mistake has been made. Even a point fix, however, is not necessarily accurate. An uncorrected error in time, for instance, would require the entire fix to be moved eastward if observations were early and westward if observations were late, at the rate of 1 minute of longitude for each 4 seconds of time.

In a two-LOP fix, the ideal cut of the LOPs is 90°. In Figure 11-12, a 90° cut with a 5 NM error in one LOP causes a 5 NM error in the fix. If the acute angle between the LOPs is 30°, a 5 NM error in one LOP causes a 10 NM error in the fix. Thus, with a two-LOP fix, an error in one LOP causes at least an equal error in the fix; the smaller the acute angle between the LOPs, the greater the fix error caused by a given error in one LOP. Of course, if both LOPs are in error, the fix may be thrown off even more. In a three-LOP fix, the ideal cut of the LOPs is 60° (star azimuths 120° apart). With this cut, a 3 NM error in any one LOP causes a 2 NM error in the fix. With any other cut, a 3 NM error in any one LOP causes more than a 2 NM error in the fix. In a three-star fix, the cut will be 60° if the azimuths of the stars differ by 60° or if they differ by 120°. If there is any unknown constant error in the observations, all the Hos will be either too great or too small.

Figure 11-12. Effect of cut on accuracy of a fix. [click image to enlarge]Notice in Figure 11-13 that, if stars are selected whose azimuths differ by 120°, this constant error of the Hos causes a displacement of the three LOPs, either all toward the center or all away from the center of the triangle. In either case, the position of the center of the triangle is not affected. If you use any three stars with azimuths outside a 180° range, any constant error in observations tends to cancel out.

Figure 11-13. Effect of azimuth on accuracy of fix.

The three-star fix has two distinct advantages over the two-star fix. First, it is the average of three observations. Second, selecting the stars carefully can counteract the effect of constant errors of observation. There is also a third advantage. Each pair of two LOPs furnishes a rough check on the third. In resolving an observation into a LOP, you might possibly make a gross error; for example, obtaining an LHA that is in error by a whole degree. Such an error might not be immediately apparent. Neither would such a discrepancy come to immediate attention in a two-LOP fix. However, this third advantage does not apply when a single LHA is used in solving all LOPs, such as is done when precomputing and using motion corrections to resolve all LOPs to a common time. Because of these three advantages, it is evident that a three-star fix should be used, rather than a two-star fix, when possible.

Whatever the number of observations, common practice, backed by logic, is to take the center of the figure formed unless there is reason for deviating from this procedure. Center is meant as the point representing the least total error of all lines considered reliable. With three LOPs, the center is considered that point within the triangle equidistant from the three sides. It may be found by bisecting the angles, but is usually located by eye.