Read OpenNGC's addendum, so the Pleiades and the Large Magellanic Cloud are on the map

OpenNGC keeps the objects no NGC or IC number covers in a second file, addendum.csv, with the
same 32 semicolon-separated columns as NGC.csv. The ETL only ever fetched NGC.csv, so the
brightest deep-sky object in the sky, the Large Magellanic Cloud (V 0.29), was missing while
the Small one was drawn, and so were the Pleiades (M45), the Hyades, the Horsehead and the
Coalsack. fetchDeepSky now fetches the addendum beside NGC.csv, caches it as
openngc-addendum.csv, and runs its rows through the same loop.

Of its 64 rows, 27 pass the existing filters: all 22 named or Messier rows except M40, which
OpenNGC types as a double star, and M102, typed as a duplicate of M101; plus seven anonymous
open clusters brighter than V 9 (H05, H20, H21, Mel071, Mel101, Mel105, MWSC3171). deepsky.json
goes from 463 to 490 objects (galaxies 73 -> 85, clusters 294 -> 306, nebulae 96 -> 99), with
no existing record changed. Messier coverage goes from 106 to 107 of 110. 340 of the 490 have
a distance: the Pleiades 135.8 pc and the Coma Star Cluster 85.9 pc from their parallaxes;
the Hyades and the Local Group dwarfs honestly have none.

validateDeepSky now requires the Andromeda Galaxy and the Small Magellanic Cloud from NGC.csv,
the Large Magellanic Cloud and the Pleiades from the addendum, and at least 107 Messier
objects. Both checks were run against the real catalogue with the addendum removed: the ETL
fails on "Deep-sky object ESO056-115 is missing", and with the required list emptied, on
"Only 106 Messier objects were produced".

In the app, on the dev server: 490 sprites. The Pleiades sprite lies 0.00182 deg from Alcyone
(0.00183 deg from the published coordinates); the LMC 18.4209 deg from Canopus and 26.8215 deg
from Achernar (18.4209 and 26.8215 published); the Hyades 2.2591 deg from Aldebaran (2.2591).
The twelve labelled deep-sky objects now open with the Large Magellanic Cloud and the
Pleiades and include Brocchi's Cluster, in place of h Persei, chi Persei and NGC 2516.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

README.md

@@ -281,8 +281,8 @@ So deep-sky records store a **unit direction** on the celestial sphere rather th
 the line of sight is always known precisely, and the objects are drawn as a fixed-radius
 backdrop shell where true distance would be unusable anyway. `distancePc` is optional metadata,
 derived from parallax for galactic objects or the Hubble law for genuinely distant galaxies,
-and left `null` — with its `distanceMethod` — whenever neither is trustworthy. Roughly 330 of
-the 463 cataloged objects get a distance; the rest honestly report none.
+and left `null` — with its `distanceMethod` — whenever neither is trustworthy. 340 of
+the 490 cataloged objects get a distance; the rest honestly report none.
 
 ## Layout
 

tools/etl/build.ts

@@ -193,6 +193,19 @@ function validateExoplanets(exoplanets: ExoplanetRecord[], starIds: Set<number>)
 
 const UNIT_VECTOR_TOLERANCE = 1e-6;
 
+/**
+ * Objects the backdrop cannot ship without, two from each of OpenNGC's files: the Andromeda
+ * Galaxy and the Small Magellanic Cloud from NGC.csv, the Large Magellanic Cloud and the
+ * Pleiades from addendum.csv. The addendum went unread for as long as the ETL has existed,
+ * because 463 objects without the brightest deep-sky object in the sky validated cleanly.
+ */
+const REQUIRED_DEEP_SKY_IDS = ['NGC0224', 'NGC0292', 'ESO056-115', 'Mel022'];
+/**
+ * 107 of the 110 Messier objects. OpenNGC types the other three as what they are: M40 a double
+ * star, M73 an asterism and M102 a duplicate of M101, none of them a deep-sky object to draw.
+ */
+const MIN_MESSIER_OBJECTS = 107;
+
 function validateDeepSky(objects: DeepSkyRecord[]): void {
   assertCondition(objects.length > 0, 'No deep-sky objects were produced.');
 
@@ -223,6 +236,12 @@ function validateDeepSky(objects: DeepSkyRecord[]): void {
     assertCondition(kinds.has(kind), `No deep-sky objects of kind "${kind}" were produced.`);
   }
 
+  for (const id of REQUIRED_DEEP_SKY_IDS) {
+    assertCondition(ids.has(id), `Deep-sky object ${id} is missing — one of OpenNGC's two files was not read.`);
+  }
+  const messier = new Set(objects.map((object) => object.messier).filter((designation) => designation !== null)).size;
+  assertCondition(messier >= MIN_MESSIER_OBJECTS, `Only ${messier} Messier objects were produced (at least ${MIN_MESSIER_OBJECTS} expected).`);
+
   const withDistance = objects.filter((object) => object.distancePc !== null).length;
   console.log(`  ${withDistance}/${objects.length} deep-sky objects have a derived distance.`);
 }

tools/etl/fetchDeepSky.ts

@@ -8,6 +8,12 @@ import { fetchTextCached } from './lib/http';
 import { dataPath, ensureDataDir } from './lib/paths';
 
 const OPENNGC_CSV_URL = 'https://raw.githubusercontent.com/mattiaverga/OpenNGC/master/database_files/NGC.csv';
+/**
+ * OpenNGC's second file, with the same columns, for the objects no NGC or IC number covers:
+ * the Pleiades, the Hyades, the Large Magellanic Cloud, the Horsehead, the Coalsack. Without
+ * it the brightest deep-sky object in the sky is missing while the Small Cloud is drawn.
+ */
+const OPENNGC_ADDENDUM_CSV_URL = 'https://raw.githubusercontent.com/mattiaverga/OpenNGC/master/database_files/addendum.csv';
 /** OpenNGC publishes semicolon-separated files, not comma-separated. */
 const OPENNGC_DELIMITER = ';';
 
@@ -64,8 +70,10 @@ function resolveName(commonName: string | null, messier: string | null, designat
  */
 export async function fetchDeepSky(): Promise<DeepSkyRecord[]> {
   console.log('Fetching OpenNGC deep-sky catalog...');
-  const csv = await fetchTextCached(OPENNGC_CSV_URL, 'openngc.csv');
-  const rows = parseCsvObjects(csv, OPENNGC_DELIMITER);
+  const rows = [
+    ...parseCsvObjects(await fetchTextCached(OPENNGC_CSV_URL, 'openngc.csv'), OPENNGC_DELIMITER),
+    ...parseCsvObjects(await fetchTextCached(OPENNGC_ADDENDUM_CSV_URL, 'openngc-addendum.csv'), OPENNGC_DELIMITER)
+  ];
 
   const records: DeepSkyRecord[] = [];
   let skippedUnclassified = 0;