Widen the star catalogue, and separate what is drawn from what is known

The map held 8750 stars within 50 pc and rendered 371 systems. Both were
lower than they needed to be, for different reasons.

The star catalogue was capped by its own encoding as much as by the
cutoff: one JSON object per star, eight key names repeated each time, 157
bytes a star. At the range HYG actually reaches that is 17 MB to download
and parse before the first frame. So the numbers move into two binary
column stores — positions in stars.bin, which the GPU is handed verbatim,
and id/magnitude/colour/spectral index in stars-meta.bin — and the JSON
keeps only the strings, with 2600 distinct spectral classifications
collapsed to a dictionary. The layout is defined once, in star-catalog.ts,
and the ETL and the app both use it, so the writer and the reader cannot
drift.

The cutoff then goes to 250 pc: 68388 stars, 7.8x as many for 1.7x the
bytes. That is where HYG's measurements stop rather than a round number —
98.6% of its rows are Hipparcos, whose parallaxes are good to about a
milliarcsecond, so beyond 250 pc it would be plotting noise.

Drawing all of them is a separate question from knowing them, and it is
answered separately. The field draws a budget: every star inside 25 pc,
because the nearest are faint red dwarfs and Proxima Centauri is magnitude
11, then the brightest of everything beyond. Search, navigation and the
planet cross-reference still see the whole catalogue. A real GPU would
draw all 68388 without noticing; the budget is for the machines that would
not, and it is one constant.

Systems were limited by something else entirely. The archive data already
shipped named 4735 host stars and only 388 resolved, because the rest lay
outside a 50 pc catalogue — and the cross-reference kept only its own
result, so redoing it meant re-downloading an archive that is not
reachable from here. Host coordinates are now stored with each planet, and
the match is re-resolved at build time against whatever catalogue the run
produced. Even name matching alone, which needs no coordinates and so
works on the records already shipped, rescues 335 planets across 238
systems: 371 renderable systems become 609.

Two selection rules were tuned for a 50 pc bubble and no longer fit.
Tethers followed the Sun's nearest neighbours, which are a speck at this
range, and now follow the brightest; labels were ranked by proximity,
which named whatever sat nearest the middle of the screen, and are now
ranked by brightness — so the view names Canopus, Achernar and Spica
rather than a clump of catalogue designations.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WaySiNst4HhDXBHnMy8p5G

tools/etl/fetchStars.ts

@@ -1,6 +1,7 @@
 import { writeFileSync } from 'node:fs';
 
 import { raDecDistanceToXyz } from '../../src/app/shared/astro/coordinates';
+import { encodeStarCatalog } from '../../src/app/shared/models/star-catalog';
 import { StarRecord, SUN_STAR_ID } from '../../src/app/shared/models/star.model';
 import { parseCsvObjects, parseOptionalNumber } from './lib/csv';
 import { fetchTextCached } from './lib/http';
@@ -15,8 +16,19 @@ const HYG_UNKNOWN_DISTANCE_PC = 100000; // HYG's placeholder for unmeasured/unre
  */
 const UNKNOWN_MAGNITUDE = 15;
 
-/** Stars within this distance (parsecs) of the Sun are kept for the galaxy view. */
-const DISTANCE_CUTOFF_PC = Number(process.env['ETL_STAR_DISTANCE_PC'] ?? 50);
+/**
+ * Stars within this distance (parsecs) of the Sun are kept for the galaxy view.
+ *
+ * Set at the range HYG's own measurements reach rather than at a round number. 98.6% of its
+ * rows carry a Hipparcos identifier, and Hipparcos parallaxes are good to roughly a
+ * milliarcsecond — so at 250 pc (4 mas) a star's distance is uncertain by some tens of per
+ * cent, and beyond it the catalogue is plotting noise. Note that only the *radial* placement
+ * blurs: a star's direction on the sky stays exact at any distance.
+ *
+ * The catalogue is also magnitude-limited, so this is not a volume-complete sample beyond about
+ * 50 pc: it thins to the intrinsically bright, which is the same selection the naked eye makes.
+ */
+const DISTANCE_CUTOFF_PC = Number(process.env['ETL_STAR_DISTANCE_PC'] ?? 250);
 
 function resolveName(row: Record<string, string>): string {
   if (row['proper']) {
@@ -98,15 +110,13 @@ export async function fetchStars(): Promise<StarRecord[]> {
 function writeStarAssets(stars: StarRecord[]): void {
   ensureDataDir();
 
-  const positions = new Float32Array(stars.length * 3);
-  stars.forEach((star, index) => {
-    positions[index * 3] = star.x;
-    positions[index * 3 + 1] = star.y;
-    positions[index * 3 + 2] = star.z;
-  });
+  // The layout lives in `star-catalog.ts`, which the app decodes with — one definition, so the
+  // writer and the reader cannot drift.
+  const { index, positions, meta } = encodeStarCatalog(stars);
 
   writeFileSync(dataPath('stars.bin'), Buffer.from(positions.buffer));
-  writeFileSync(dataPath('stars-index.json'), JSON.stringify(stars));
+  writeFileSync(dataPath('stars-meta.bin'), Buffer.from(meta));
+  writeFileSync(dataPath('stars-index.json'), JSON.stringify(index));
 }
 
 if (require.main === module) {