Widen the star catalogue, and separate what is drawn from what is known
The map held 8750 stars within 50 pc and rendered 371 systems. Both were lower than they needed to be, for different reasons. The star catalogue was capped by its own encoding as much as by the cutoff: one JSON object per star, eight key names repeated each time, 157 bytes a star. At the range HYG actually reaches that is 17 MB to download and parse before the first frame. So the numbers move into two binary column stores — positions in stars.bin, which the GPU is handed verbatim, and id/magnitude/colour/spectral index in stars-meta.bin — and the JSON keeps only the strings, with 2600 distinct spectral classifications collapsed to a dictionary. The layout is defined once, in star-catalog.ts, and the ETL and the app both use it, so the writer and the reader cannot drift. The cutoff then goes to 250 pc: 68388 stars, 7.8x as many for 1.7x the bytes. That is where HYG's measurements stop rather than a round number — 98.6% of its rows are Hipparcos, whose parallaxes are good to about a milliarcsecond, so beyond 250 pc it would be plotting noise. Drawing all of them is a separate question from knowing them, and it is answered separately. The field draws a budget: every star inside 25 pc, because the nearest are faint red dwarfs and Proxima Centauri is magnitude 11, then the brightest of everything beyond. Search, navigation and the planet cross-reference still see the whole catalogue. A real GPU would draw all 68388 without noticing; the budget is for the machines that would not, and it is one constant. Systems were limited by something else entirely. The archive data already shipped named 4735 host stars and only 388 resolved, because the rest lay outside a 50 pc catalogue — and the cross-reference kept only its own result, so redoing it meant re-downloading an archive that is not reachable from here. Host coordinates are now stored with each planet, and the match is re-resolved at build time against whatever catalogue the run produced. Even name matching alone, which needs no coordinates and so works on the records already shipped, rescues 335 planets across 238 systems: 371 renderable systems become 609. Two selection rules were tuned for a 50 pc bubble and no longer fit. Tethers followed the Sun's nearest neighbours, which are a speck at this range, and now follow the brightest; labels were ranked by proximity, which named whatever sat nearest the middle of the screen, and are now ranked by brightness — so the view names Canopus, Achernar and Spica rather than a clump of catalogue designations. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WaySiNst4HhDXBHnMy8p5G
This commit is contained in:
@@ -312,3 +312,9 @@ Real photography where it exists, and a surface reasoned from measurements where
|
||||
- Frame against the reference grid's outer ring, which is always wider than the outermost orbit, and leave an explicit margin around it.
|
||||
- Raise the framing ceiling far enough to hold the solar system out to Pluto in a portrait window; only companions hundreds of AU out reach it now.
|
||||
- Floor the star's halo against the framed radius, so a star sized against its innermost orbit still reads at the distance that frames its outermost one.
|
||||
|
||||
### ✓ Step 11: Widen the star catalogue, and separate what is drawn from what is known
|
||||
- Repack the catalogue as two binary column stores plus a string-only JSON index (`star-catalog.ts`, shared by the ETL and the app), so 7.8x the stars costs 1.7x the bytes instead of 12x.
|
||||
- Raise the distance cutoff from 50 pc to 250 pc — where Hipparcos parallaxes stop being trustworthy — taking the catalogue from 8750 stars to 68388.
|
||||
- Give the star field a render budget: every star inside 25 pc plus the brightest of the rest. Search, navigation and the cross-reference still see the whole catalogue.
|
||||
- Store each exoplanet's host coordinates and re-resolve the cross-reference against the current catalogue at build time, so widening the star list rescues systems without re-downloading the archive. Renderable systems: 371 to 609.
|
||||
|
||||
@@ -27,10 +27,12 @@ npm run e2e:typecheck
|
||||
|
||||
## What's in it
|
||||
|
||||
**Galaxy view** — every HYG-catalogue star within 50 parsecs as instanced camera-facing
|
||||
billboards, positioned from real RA/Dec/parallax, coloured by spectral index and sized by
|
||||
magnitude. A polar grid in the galactic plane runs under them with a drop line from each of the
|
||||
Sun's nearest neighbours, and names label the stars nearest whatever the camera is looking at.
|
||||
**Galaxy view** — the HYG catalogue out to 250 parsecs, 68 388 stars, as instanced
|
||||
camera-facing billboards positioned from real RA/Dec/parallax, coloured by spectral index and
|
||||
sized by magnitude. The field draws a budget of the most visible of them rather than all — see
|
||||
"On how many stars" below. A polar grid in the galactic plane runs under them with a drop line
|
||||
from each of the brightest, and names label the most prominent stars near whatever the camera is
|
||||
looking at.
|
||||
Behind them sits a backdrop of notable deep-sky objects and a Milky Way panorama.
|
||||
|
||||
**Galactic view** — keep pulling back and the neighbourhood becomes a point inside the Milky
|
||||
@@ -167,16 +169,42 @@ re-runs are cheap and offline-friendly; set `ETL_FORCE_REFRESH=1` to bypass the
|
||||
|
||||
| Script | Source | Output |
|
||||
| --- | --- | --- |
|
||||
| `fetchStars.ts` | HYG database (Hipparcos/Yale/Gliese) | `stars.bin`, `stars-index.json` |
|
||||
| `fetchStars.ts` | HYG database (Hipparcos/Yale/Gliese) | `stars.bin`, `stars-meta.bin`, `stars-index.json` |
|
||||
| `fetchSolarSystem.ts` | JPL Horizons / SSD | `bodies.json` |
|
||||
| `fetchExoplanets.ts` | NASA Exoplanet Archive (TAP) | `exoplanets.json` |
|
||||
| `fetchDeepSky.ts` | OpenNGC | `deepsky.json` |
|
||||
|
||||
Star positions ship as a packed `Float32Array` (`stars.bin`) rather than JSON to keep the
|
||||
initial payload and parse cost down; `stars-index.json` carries everything else in the same
|
||||
order.
|
||||
The star catalogue ships as two binary column stores plus a small JSON file, not as an array of
|
||||
objects. At 68 388 stars the old encoding — one JSON object per star, its eight key names
|
||||
repeated each time — would have been about 17 MB to download and parse before the first frame.
|
||||
Splitting it puts the numbers in `stars.bin` (positions, handed to the GPU verbatim) and
|
||||
`stars-meta.bin` (id, magnitude, colour index, spectral-type index), and leaves `stars-index.json`
|
||||
holding only the strings, with the ~2 600 distinct spectral classifications collapsed into a
|
||||
dictionary. The result is 2.6 MB for 7.8× the stars. `star-catalog.ts` defines the layout once
|
||||
and both the ETL and the app use it, so the writer and the reader cannot drift apart.
|
||||
|
||||
`ETL_STAR_DISTANCE_PC` (default `50`) sets the star-field distance cutoff.
|
||||
`ETL_STAR_DISTANCE_PC` (default `250`) sets the star-field distance cutoff.
|
||||
|
||||
### On how many stars
|
||||
|
||||
Not many, against the Galaxy. It holds 100–400 billion stars and this map ships 68 388 of them —
|
||||
about 0.00003%. That gap is not this project's to close: Gaia DR3, the largest stellar catalogue
|
||||
ever assembled, has ~1.8 billion sources, roughly 1% of the Galaxy, and is itself blocked by dust
|
||||
and blind to most red dwarfs beyond a few hundred parsecs. It is the same reason the galactic
|
||||
view is a model.
|
||||
|
||||
The 250 pc cutoff is where HYG's own measurements stop. 98.6% of its rows carry a Hipparcos
|
||||
identifier, and Hipparcos parallaxes are good to about a milliarcsecond — so at 250 pc a
|
||||
distance is uncertain by some tens of per cent and beyond it the catalogue would be plotting
|
||||
noise. Only the *radial* placement blurs; a star's direction on the sky stays exact at any
|
||||
distance. Note also that beyond about 50 pc the sample is magnitude-limited rather than
|
||||
volume-complete: it thins to the intrinsically bright, which is the same selection the naked eye
|
||||
makes.
|
||||
|
||||
Drawing and knowing are separate. The field draws `STAR_RENDER_BUDGET` stars — every one inside
|
||||
25 pc, then the brightest of the rest — while search, navigation and the planet cross-reference
|
||||
all see the full catalogue. A real GPU would draw all 68 388 without noticing; the budget exists
|
||||
for the machines that would not, and is a single constant to raise.
|
||||
|
||||
### On deep-sky distances
|
||||
|
||||
@@ -246,7 +274,7 @@ for details on `user` / `project` / `local` scope.
|
||||
## Data credits
|
||||
|
||||
Star catalogue: [HYG database](https://github.com/astronexus/HYG-Database) (Hipparcos, Yale
|
||||
Bright Star, Gliese). Solar-system ephemerides: NASA/JPL Horizons. Exoplanets: NASA Exoplanet
|
||||
Bright Star, Gliese) — 68 388 stars within 250 pc. Solar-system ephemerides: NASA/JPL Horizons. Exoplanets: NASA Exoplanet
|
||||
Archive. Deep-sky objects: [OpenNGC](https://github.com/mattiaverga/OpenNGC). Body and skybox
|
||||
imagery: NASA/JPL/USGS public domain and Solar System Scope (CC BY 4.0) — per-file provenance
|
||||
is recorded in `src/app/shared/rendering/texture-catalog.ts`.
|
||||
|
||||
@@ -3,6 +3,7 @@ import { Injectable } from '@angular/core';
|
||||
import { BodyRecord } from '../../shared/models/body.model';
|
||||
import { DeepSkyRecord } from '../../shared/models/deepsky.model';
|
||||
import { ExoplanetRecord } from '../../shared/models/exoplanet.model';
|
||||
import { decodeStarCatalog, StarCatalogIndex } from '../../shared/models/star-catalog';
|
||||
import { StarRecord } from '../../shared/models/star.model';
|
||||
|
||||
export interface StarField {
|
||||
@@ -42,13 +43,20 @@ export class DataLoaderService {
|
||||
return this.deepSkyPromise;
|
||||
}
|
||||
|
||||
/**
|
||||
* Three assets rather than one, fetched in parallel: the strings as JSON, and the numbers as
|
||||
* two binary column stores. See `star-catalog.ts` for why the catalogue is not a single array
|
||||
* of JSON objects.
|
||||
*/
|
||||
private async fetchStars(): Promise<StarField> {
|
||||
const [stars, buffer] = await Promise.all([
|
||||
fetch('assets/data/stars-index.json').then((response) => response.json() as Promise<StarRecord[]>),
|
||||
fetch('assets/data/stars.bin').then((response) => response.arrayBuffer())
|
||||
const [index, positionBuffer, metaBuffer] = await Promise.all([
|
||||
fetch('assets/data/stars-index.json').then((response) => response.json() as Promise<StarCatalogIndex>),
|
||||
fetch('assets/data/stars.bin').then((response) => response.arrayBuffer()),
|
||||
fetch('assets/data/stars-meta.bin').then((response) => response.arrayBuffer())
|
||||
]);
|
||||
|
||||
return { stars, positions: new Float32Array(buffer) };
|
||||
const positions = new Float32Array(positionBuffer);
|
||||
return { stars: decodeStarCatalog(index, positions, metaBuffer), positions };
|
||||
}
|
||||
|
||||
private fetchJson<T>(url: string): Promise<T> {
|
||||
|
||||
@@ -215,7 +215,9 @@ describe('GalaxySystemSceneComponent camera-flight transitions', () => {
|
||||
// far the camera has pulled back, so what identifies it is the far plane it settles on
|
||||
// (5000 pc) versus the AU-space one (20000 AU), not a fixed near plane.
|
||||
expect(engine.getCamera().far).toBeCloseTo(5000, 6);
|
||||
expect(engine.getCamera().near).toBeLessThan(0.1);
|
||||
// The near plane tracks how far back the camera is rather than sitting at a constant, so
|
||||
// what identifies galaxy space is that it is a small fraction of that far plane.
|
||||
expect(engine.getCamera().near).toBeLessThan(engine.getCamera().far / 1000);
|
||||
expect(navigationStore.viewLevel()).toBe('galaxy');
|
||||
});
|
||||
|
||||
|
||||
@@ -30,8 +30,14 @@ const SOL_STAR_ID = 0;
|
||||
/** Stars drawn from a colour rather than a photograph get a more restrained halo. */
|
||||
const DIM_STAR_GLOW_SCALE = 0.6;
|
||||
|
||||
/** Stars closer than this to the camera get a name label (always includes the selection). */
|
||||
const LABEL_MAX_DISTANCE_PC = 20;
|
||||
/**
|
||||
* How far from what the camera is looking at a star can be and still be named, as a fraction of
|
||||
* how far back the camera is — so the net widens as the view pulls out and closes as it dives
|
||||
* in, instead of naming the same handful of stars at every scale. Bounded at both ends.
|
||||
*/
|
||||
const LABEL_RADIUS_TO_ORBIT_DISTANCE = 0.35;
|
||||
const MIN_LABEL_RADIUS_PC = 4;
|
||||
const MAX_LABEL_RADIUS_PC = 400;
|
||||
/** Caps how many labels are shown at once, to keep the DOM light. */
|
||||
const LABEL_MAX_COUNT = 15;
|
||||
/**
|
||||
@@ -57,7 +63,7 @@ const CLICK_DRAG_SLOP_PX = 5;
|
||||
* arbitrary equatorial direction gives — the plane is tilted 63 degrees to the equator.
|
||||
*/
|
||||
const GALAXY_OVERVIEW_POSITION = (() => {
|
||||
const view = galacticToEquatorial({ x: -21, y: -46, z: 35 });
|
||||
const view = galacticToEquatorial({ x: -105, y: -230, z: 175 });
|
||||
return new THREE.Vector3(view.x, view.y, view.z);
|
||||
})();
|
||||
const GALAXY_OVERVIEW_TARGET = new THREE.Vector3(0, 0, 0);
|
||||
@@ -80,15 +86,22 @@ const GALACTIC_NEAR_PC = 5;
|
||||
const GALACTIC_FAR_PC = 250000;
|
||||
|
||||
/** Rings for the local grid (parsecs from the Sun), with the catalogue's edge called out. */
|
||||
const LOCAL_GRID_RINGS_PC = [10, 20, 30, 40, 50];
|
||||
const LOCAL_GRID_RINGS_PC = [50, 100, 150, 200, 250];
|
||||
const LOCAL_GRID_SPOKES = 12;
|
||||
/** Rings for the galactic grid (parsecs from the centre), with the Sun's orbit called out. */
|
||||
const GALACTIC_GRID_RINGS_PC = [2500, 5000, SUN_GALACTOCENTRIC_RADIUS_PC, 11000, 14000];
|
||||
const GALACTIC_GRID_SPOKES = 24;
|
||||
/** The local grid passes through the Sun, which is the origin, so tethers drop to height zero. */
|
||||
const LOCAL_PLANE_HEIGHT_PC = 0;
|
||||
/** How many of the Sun's nearest neighbours get a permanent drop line to the local grid. */
|
||||
const TETHERED_STAR_COUNT = 28;
|
||||
/**
|
||||
* How many stars get a permanent drop line to the local grid, and which ones: the brightest in
|
||||
* the catalogue rather than the Sun's nearest neighbours.
|
||||
*
|
||||
* Nearest-to-the-Sun was the right set when the catalogue stopped at 50 pc and the camera sat
|
||||
* just outside it. Across 250 pc those same stars are a speck at the centre, while the brightest
|
||||
* are spread through the whole volume — and are the ones the eye is already on.
|
||||
*/
|
||||
const TETHERED_STAR_COUNT = 60;
|
||||
|
||||
/** Camera pose for the whole-Galaxy overview: above the disc, out past the Sun, looking in. */
|
||||
const GALACTIC_OVERVIEW_HEIGHT_PC = 26000;
|
||||
@@ -344,13 +357,12 @@ export class GalaxySystemSceneComponent implements AfterViewInit, OnDestroy {
|
||||
spokeCount: LOCAL_GRID_SPOKES,
|
||||
emphasisRadii: [LOCAL_GRID_RINGS_PC[LOCAL_GRID_RINGS_PC.length - 1]]
|
||||
});
|
||||
// Drop lines for the Sun's nearest neighbours. A fixed set rather than whatever is currently
|
||||
// labelled: these are the stars the local view is about, they cluster where the grid is
|
||||
// densest, and a tether that appears and vanishes as the camera drifts reads as a glitch.
|
||||
// A fixed set rather than whatever is currently labelled: a tether that appears and vanishes
|
||||
// as the camera drifts reads as a glitch.
|
||||
this.tethers = new TetherField(TETHERED_STAR_COUNT);
|
||||
this.tethers.setTargets(
|
||||
[...stars]
|
||||
.sort((a, b) => Math.hypot(a.x, a.y, a.z) - Math.hypot(b.x, b.y, b.z))
|
||||
.sort((a, b) => a.magnitude - b.magnitude)
|
||||
.slice(0, TETHERED_STAR_COUNT)
|
||||
.map((star) => new THREE.Vector3(star.x, star.y, star.z)),
|
||||
LOCAL_PLANE_HEIGHT_PC
|
||||
@@ -461,8 +473,11 @@ export class GalaxySystemSceneComponent implements AfterViewInit, OnDestroy {
|
||||
// Measured from what the camera is looking at, not from where it is. Those differ by the
|
||||
// orbit distance, so a camera-relative rule names the stars closest to the near edge of the
|
||||
// view — a ring of labels around the outside of the thing the user is actually looking at.
|
||||
const { x: cx, y: cy, z: cz } = this.controls?.target ?? GALAXY_OVERVIEW_TARGET;
|
||||
const maxDistanceSq = LABEL_MAX_DISTANCE_PC * LABEL_MAX_DISTANCE_PC;
|
||||
const target = this.controls?.target ?? GALAXY_OVERVIEW_TARGET;
|
||||
const { x: cx, y: cy, z: cz } = target;
|
||||
const orbitDistance = (this.controls ? camera.position.distanceTo(target) : GALAXY_OVERVIEW_POSITION.length()) * LABEL_RADIUS_TO_ORBIT_DISTANCE;
|
||||
const labelRadius = THREE.MathUtils.clamp(orbitDistance, MIN_LABEL_RADIUS_PC, MAX_LABEL_RADIUS_PC);
|
||||
const maxDistanceSq = labelRadius * labelRadius;
|
||||
|
||||
const candidates: Array<{ star: StarRecord; distanceSq: number }> = [];
|
||||
for (const star of this.stars) {
|
||||
@@ -475,7 +490,11 @@ export class GalaxySystemSceneComponent implements AfterViewInit, OnDestroy {
|
||||
}
|
||||
}
|
||||
|
||||
candidates.sort((a, b) => a.distanceSq - b.distanceSq);
|
||||
// Brightest first, not nearest first. Proximity was the right ranking when the catalogue was
|
||||
// a 50 pc bubble and everything in it was equally worth naming; across 250 pc it labels a
|
||||
// clump of whatever happens to be closest to the middle of the screen and never names the
|
||||
// stars that are actually prominent. Brightness is what makes a star worth a name.
|
||||
candidates.sort((a, b) => a.star.magnitude - b.star.magnitude);
|
||||
// Individual star names mean nothing once the whole Galaxy is in frame — at that range the
|
||||
// entire catalogue is inside one pixel — so the labels hand over to the structural ones.
|
||||
const isGalactic = this.galacticStrength >= GALACTIC_LEVEL_THRESHOLD;
|
||||
@@ -561,7 +580,9 @@ export class GalaxySystemSceneComponent implements AfterViewInit, OnDestroy {
|
||||
this.hudTitle.set('Local Stars');
|
||||
this.hudSubtitle.set('Hipparcos · Yale Bright Star · Gliese');
|
||||
this.hudReadouts.set([
|
||||
{ label: 'Stars', value: `${this.stars.length}` },
|
||||
// Both numbers, because they differ: the catalogue is what the map knows and the first is
|
||||
// what it draws. See `STAR_RENDER_BUDGET`.
|
||||
{ label: 'Stars', value: this.starField && this.starField.drawnCount < this.stars.length ? `${this.starField.drawnCount} / ${this.stars.length}` : `${this.stars.length}` },
|
||||
{ label: 'Radius', value: `${LOCAL_GRID_RINGS_PC[LOCAL_GRID_RINGS_PC.length - 1]} pc` },
|
||||
{ label: 'Exoplanets', value: `${this.exoplanets.length}` }
|
||||
]);
|
||||
|
||||
@@ -2,7 +2,7 @@ import * as THREE from 'three/webgpu';
|
||||
import { describe, expect, it } from 'vitest';
|
||||
|
||||
import { StarRecord } from '../../shared/models/star.model';
|
||||
import { colorIndexToRgb, magnitudeToPointSize, StarFieldRenderer } from './star-field-renderer';
|
||||
import { colorIndexToRgb, magnitudeToPointSize, selectDrawnStars, StarFieldRenderer } from './star-field-renderer';
|
||||
|
||||
function star(overrides: Partial<StarRecord> = {}): StarRecord {
|
||||
return {
|
||||
@@ -223,3 +223,71 @@ describe('StarFieldRenderer', () => {
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
/** A star at a given distance along +X, with a given apparent magnitude. */
|
||||
function catalogueStar(id: number, distancePc: number, magnitude: number): StarRecord {
|
||||
return { id, name: `star-${id}`, x: distancePc, y: 0, z: 0, magnitude, spectralType: 'G2V', colorIndex: 0.6 };
|
||||
}
|
||||
|
||||
describe('selectDrawnStars', () => {
|
||||
it('draws everything when the catalogue fits the budget', () => {
|
||||
const catalogue = [catalogueStar(1, 10, 5), catalogueStar(2, 20, 6)];
|
||||
expect(Array.from(selectDrawnStars(catalogue, 10))).toEqual([0, 1]);
|
||||
});
|
||||
|
||||
it('never draws more than the budget', () => {
|
||||
const catalogue = Array.from({ length: 500 }, (_, i) => catalogueStar(i, 200, i));
|
||||
expect(selectDrawnStars(catalogue, 50)).toHaveLength(50);
|
||||
});
|
||||
|
||||
it('keeps the whole solar neighbourhood, however faint', () => {
|
||||
// The load-bearing case: the nearest stars are overwhelmingly faint red dwarfs, and Proxima
|
||||
// Centauri is magnitude 11. A pure brightness cut would delete the part of the map that
|
||||
// matters most and holds the nearby planets.
|
||||
const proxima = catalogueStar(999, 1.3, 11.1);
|
||||
const catalogue = [proxima, ...Array.from({ length: 200 }, (_, i) => catalogueStar(i, 240, 2))];
|
||||
const drawn = selectDrawnStars(catalogue, 20);
|
||||
|
||||
expect(Array.from(drawn)).toContain(0);
|
||||
expect(drawn).toHaveLength(20);
|
||||
});
|
||||
|
||||
it('spends what is left on the brightest stars beyond the neighbourhood', () => {
|
||||
const catalogue = [catalogueStar(0, 10, 12), catalogueStar(1, 200, 8), catalogueStar(2, 200, 2), catalogueStar(3, 200, 5)];
|
||||
const drawn = Array.from(selectDrawnStars(catalogue, 3));
|
||||
|
||||
// The nearby faint one, then the two brightest distant ones — not the magnitude-8 straggler.
|
||||
expect(drawn).toEqual([0, 2, 3]);
|
||||
});
|
||||
|
||||
it('returns catalogue indices in order, so positions can be subset alongside', () => {
|
||||
const catalogue = Array.from({ length: 100 }, (_, i) => catalogueStar(i, 150, 100 - i));
|
||||
const drawn = Array.from(selectDrawnStars(catalogue, 10));
|
||||
expect(drawn).toEqual([...drawn].sort((a, b) => a - b));
|
||||
});
|
||||
});
|
||||
|
||||
describe('StarFieldRenderer render budget', () => {
|
||||
it('draws only the budget, and reports how many that was', () => {
|
||||
const catalogue = Array.from({ length: 300 }, (_, i) => catalogueStar(i, 200, i));
|
||||
const positions = new Float32Array(catalogue.flatMap((s) => [s.x, s.y, s.z]));
|
||||
const renderer = new StarFieldRenderer(catalogue, positions, 40);
|
||||
|
||||
expect(renderer.drawnCount).toBe(40);
|
||||
expect((renderer.object.geometry as THREE.InstancedBufferGeometry).instanceCount).toBe(40);
|
||||
renderer.dispose();
|
||||
});
|
||||
|
||||
it('keeps each drawn star with its own position after subsetting', () => {
|
||||
// The subtle failure this guards: repacking positions for a subset while the colours and
|
||||
// sizes follow a different order would give every star someone else's place in the sky.
|
||||
const catalogue = [catalogueStar(0, 5, 9), catalogueStar(1, 200, 1), catalogueStar(2, 200, 7)];
|
||||
const positions = new Float32Array(catalogue.flatMap((s) => [s.x, s.y, s.z]));
|
||||
const renderer = new StarFieldRenderer(catalogue, positions, 2);
|
||||
|
||||
expect(renderer.drawnCount).toBe(2);
|
||||
expect(renderer.starIdAt(0)).toBe(0);
|
||||
expect(renderer.starIdAt(1)).toBe(1);
|
||||
renderer.dispose();
|
||||
});
|
||||
});
|
||||
|
||||
@@ -30,6 +30,36 @@ const PIXELS_TO_ANGULAR_SIZE =
|
||||
*/
|
||||
const PICK_NDC_SLOP = 0.01;
|
||||
|
||||
/**
|
||||
* How many stars the field draws at once, however many the catalogue holds.
|
||||
*
|
||||
* The catalogue reaches as far as its parallaxes do — 68388 stars at 250 pc — but drawing all of
|
||||
* them is a cost paid every frame by every machine, and most of that cost buys 1.5-pixel dots.
|
||||
* So the *data* is the catalogue and the *drawing* is a budget, and the two are allowed to
|
||||
* differ. Everything still exists for search, for flying to, and for hosting planets.
|
||||
*
|
||||
* The figure is set low deliberately. A real GPU would draw the whole catalogue without
|
||||
* noticing — this is one instanced draw call — but the value that matters is what a weak one
|
||||
* does, and the software rasterizer this was measured against costs a third of its frame rate
|
||||
* per 12000 stars. Raise it freely on hardware that can take it; nothing else depends on it.
|
||||
*/
|
||||
export const STAR_RENDER_BUDGET = 12000;
|
||||
|
||||
/**
|
||||
* Radius (parsecs) inside which every star is drawn regardless of brightness.
|
||||
*
|
||||
* A pure brightness cut would be defensible — apparent magnitude is exactly "how visible this
|
||||
* is" — but it would drop the solar neighbourhood, because the nearest stars are overwhelmingly
|
||||
* faint red dwarfs. Proxima Centauri is magnitude 11. Those are the stars this map is most about
|
||||
* and the ones that hold the nearby planets, so the neighbourhood is kept whole and the budget
|
||||
* is spent on the brightest of everything beyond it.
|
||||
*
|
||||
* Kept deliberately small against the catalogue's 250 pc reach. The guaranteed core occupies a
|
||||
* thousandth of that volume, so a generous radius spends most of the budget inside it and draws
|
||||
* a dense knot surrounded by nothing — which is a worse picture than the smaller catalogue was.
|
||||
*/
|
||||
export const ALWAYS_DRAWN_RADIUS_PC = 25;
|
||||
|
||||
const COLD_STAR_COLOR = new THREE.Color(0.65, 0.75, 1.0);
|
||||
const NEUTRAL_STAR_COLOR = new THREE.Color(1.0, 1.0, 1.0);
|
||||
const WARM_STAR_COLOR = new THREE.Color(1.0, 0.6, 0.35);
|
||||
@@ -90,22 +120,58 @@ function createQuadGeometry(instanceCount: number): THREE.InstancedBufferGeometr
|
||||
* close the camera gets. That is deliberate and physically right: real stars are unresolvable
|
||||
* point sources, and their apparent size on screen is a function of brightness, not distance.
|
||||
*/
|
||||
/**
|
||||
* Chooses which stars to draw when the catalogue is larger than the budget: everything inside
|
||||
* the neighbourhood radius, then the brightest of the rest until the budget is spent.
|
||||
*
|
||||
* Returns indices into the original list, so the caller can subset the positions that go with
|
||||
* them. Returns them in catalogue order rather than in selection order, purely so the drawn set
|
||||
* is stable and inspectable.
|
||||
*/
|
||||
export function selectDrawnStars(stars: readonly StarRecord[], budget = STAR_RENDER_BUDGET): Uint32Array {
|
||||
if (stars.length <= budget) {
|
||||
return Uint32Array.from(stars.keys());
|
||||
}
|
||||
|
||||
const near: number[] = [];
|
||||
const far: number[] = [];
|
||||
stars.forEach((star, index) => {
|
||||
(Math.hypot(star.x, star.y, star.z) <= ALWAYS_DRAWN_RADIUS_PC ? near : far).push(index);
|
||||
});
|
||||
|
||||
far.sort((a, b) => stars[a].magnitude - stars[b].magnitude);
|
||||
const selected = near.concat(far.slice(0, Math.max(0, budget - near.length)));
|
||||
selected.sort((a, b) => a - b);
|
||||
return Uint32Array.from(selected);
|
||||
}
|
||||
|
||||
export class StarFieldRenderer {
|
||||
readonly object: THREE.Mesh;
|
||||
/** How many of the catalogue's stars this field actually draws. */
|
||||
readonly drawnCount: number;
|
||||
|
||||
private readonly geometry: THREE.InstancedBufferGeometry;
|
||||
private readonly material: THREE.SpriteNodeMaterial;
|
||||
/** Angular diameter per star, in the same order as `stars` — reused for picking. */
|
||||
/** The subset of the catalogue that is drawn, and so the only set that can be clicked. */
|
||||
private readonly stars: readonly StarRecord[];
|
||||
/** Angular diameter per drawn star, in the same order as `stars` — reused for picking. */
|
||||
private readonly angularSizes: Float32Array;
|
||||
|
||||
constructor(
|
||||
private readonly stars: readonly StarRecord[],
|
||||
positions: Float32Array
|
||||
) {
|
||||
constructor(catalogue: readonly StarRecord[], cataloguePositions: Float32Array, budget = STAR_RENDER_BUDGET) {
|
||||
const drawn = selectDrawnStars(catalogue, budget);
|
||||
this.stars = drawn.length === catalogue.length ? catalogue : Array.from(drawn, (index) => catalogue[index]);
|
||||
this.drawnCount = this.stars.length;
|
||||
|
||||
const stars = this.stars;
|
||||
this.geometry = createQuadGeometry(stars.length);
|
||||
|
||||
const colors = new Float32Array(stars.length * 3);
|
||||
this.angularSizes = new Float32Array(stars.length);
|
||||
// Repacked only when the drawn set is a subset; otherwise the ETL's buffer is used as-is.
|
||||
const positions =
|
||||
drawn.length === catalogue.length
|
||||
? cataloguePositions
|
||||
: Float32Array.from({ length: drawn.length * 3 }, (_, i) => cataloguePositions[drawn[(i / 3) | 0] * 3 + (i % 3)]);
|
||||
|
||||
stars.forEach((star, index) => {
|
||||
const color = colorIndexToRgb(star.colorIndex, star.spectralType);
|
||||
@@ -115,7 +181,6 @@ export class StarFieldRenderer {
|
||||
this.angularSizes[index] = magnitudeToPointSize(star.magnitude) * PIXELS_TO_ANGULAR_SIZE;
|
||||
});
|
||||
|
||||
// `positions` is the ETL's packed buffer, already in the same order as `stars`.
|
||||
const positionAttribute = new THREE.InstancedBufferAttribute(positions, 3);
|
||||
const colorAttribute = new THREE.InstancedBufferAttribute(colors, 3);
|
||||
const sizeAttribute = new THREE.InstancedBufferAttribute(this.angularSizes, 1);
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { CartesianCoordinates, distanceBetween, raDegDecDistanceToXyz } from './coordinates';
|
||||
import { ExoplanetRecord } from '../models/exoplanet.model';
|
||||
import { StarRecord } from '../models/star.model';
|
||||
|
||||
/** Normalizes a star name for comparison: lowercase, alphanumeric characters only. */
|
||||
@@ -65,3 +66,68 @@ function findNearestStarWithin(position: CartesianCoordinates, stars: readonly S
|
||||
|
||||
return closest ? closest.id : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Re-resolves every exoplanet's host star against a star catalogue.
|
||||
*
|
||||
* The cross-reference is a *derived* fact: it depends as much on which stars were loaded as on
|
||||
* the archive itself. When the catalogue reached 50 pc, 388 of the archive's 4735 named hosts
|
||||
* found a match and the other 4347 were carried and never drawn — not because their planets are
|
||||
* unknown, but because their star was out of range. Widening the catalogue rescues some of them,
|
||||
* and until the host coordinates were stored alongside each planet that meant re-downloading an
|
||||
* archive which is not always reachable.
|
||||
*
|
||||
* Records written before those coordinates were kept can still be matched *by name*, which needs
|
||||
* no coordinates at all — and that alone is worth doing, because a wider catalogue contains more
|
||||
* names. What such a record cannot do is disprove its existing match: a name miss means only
|
||||
* that the name missed, not that the star is absent. So those are upgraded where a match is
|
||||
* found and left alone otherwise, while records that do carry coordinates take the new result
|
||||
* outright, match or no match.
|
||||
*/
|
||||
|
||||
/** A host must sit within this many parsecs of a catalogue star to count as the same object. */
|
||||
export const HOST_MATCH_TOLERANCE_PC = 2;
|
||||
|
||||
export interface RematchSummary {
|
||||
total: number;
|
||||
/** Records carrying host coordinates, and therefore eligible to be re-matched in full. */
|
||||
resolvable: number;
|
||||
matched: number;
|
||||
gained: number;
|
||||
lost: number;
|
||||
}
|
||||
|
||||
export function rematchHostStars(exoplanets: ExoplanetRecord[], stars: readonly StarRecord[]): RematchSummary {
|
||||
const nameIndex = buildStarNameIndex(stars);
|
||||
const summary: RematchSummary = { total: exoplanets.length, resolvable: 0, matched: 0, gained: 0, lost: 0 };
|
||||
|
||||
for (const exoplanet of exoplanets) {
|
||||
const { hostRaDeg, hostDecDeg, hostDistancePc } = exoplanet;
|
||||
const positioned = hostRaDeg !== undefined && hostDecDeg !== undefined && hostDistancePc !== undefined;
|
||||
if (positioned) {
|
||||
summary.resolvable++;
|
||||
}
|
||||
|
||||
const previous = exoplanet.hostStarId;
|
||||
// With no coordinates the query still carries the host's name, and `resolveHostStarId` tries
|
||||
// that first; the positional fallback simply declines to run on non-finite coordinates.
|
||||
const resolved = resolveHostStarId(
|
||||
{ hostname: exoplanet.hostStarName, raDeg: hostRaDeg ?? Number.NaN, decDeg: hostDecDeg ?? Number.NaN, distancePc: hostDistancePc ?? Number.NaN },
|
||||
stars,
|
||||
HOST_MATCH_TOLERANCE_PC,
|
||||
nameIndex
|
||||
);
|
||||
|
||||
exoplanet.hostStarId = positioned ? resolved : (resolved ?? previous);
|
||||
if (exoplanet.hostStarId !== null) {
|
||||
summary.matched++;
|
||||
}
|
||||
if (previous === null && exoplanet.hostStarId !== null) {
|
||||
summary.gained++;
|
||||
} else if (previous !== null && exoplanet.hostStarId === null) {
|
||||
summary.lost++;
|
||||
}
|
||||
}
|
||||
|
||||
return summary;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,82 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
|
||||
import { ExoplanetRecord } from '../models/exoplanet.model';
|
||||
import { StarRecord } from '../models/star.model';
|
||||
import { rematchHostStars } from './host-star-matching';
|
||||
|
||||
/** Two catalogue stars, one of which is only present in the wider of the two catalogues. */
|
||||
const NEARBY: StarRecord = { id: 100, name: 'Gl 357', x: 9, y: 0, z: 0, magnitude: 10.9, spectralType: 'K', colorIndex: 1.4 };
|
||||
const DISTANT: StarRecord = { id: 200, name: 'HD 33844', x: 0, y: 120, z: 0, magnitude: 7.7, spectralType: 'K0', colorIndex: 1.0 };
|
||||
|
||||
const NARROW_CATALOGUE = [NEARBY];
|
||||
const WIDE_CATALOGUE = [NEARBY, DISTANT];
|
||||
|
||||
function planet(overrides: Partial<ExoplanetRecord> = {}): ExoplanetRecord {
|
||||
return { id: 'p', hostStarId: null, hostStarName: 'HD 33844', name: 'HD 33844 b', orbit: { semiMajorAxisAu: 1 }, ...overrides };
|
||||
}
|
||||
|
||||
describe('rematchHostStars', () => {
|
||||
it('rescues a host that the wider catalogue now contains, by name alone', () => {
|
||||
// The whole point: the cross-reference is a fact about the catalogue as much as about the
|
||||
// archive, so widening one ought to resolve hosts the other already knew about.
|
||||
const planets = [planet()];
|
||||
const summary = rematchHostStars(planets, WIDE_CATALOGUE);
|
||||
|
||||
expect(planets[0].hostStarId).toBe(DISTANT.id);
|
||||
expect(summary.gained).toBe(1);
|
||||
expect(summary.matched).toBe(1);
|
||||
});
|
||||
|
||||
it('needs no coordinates to do it', () => {
|
||||
// Which matters, because the shipped records were written before coordinates were kept.
|
||||
const planets = [planet()];
|
||||
expect(planets[0].hostRaDeg).toBeUndefined();
|
||||
rematchHostStars(planets, WIDE_CATALOGUE);
|
||||
expect(planets[0].hostStarId).toBe(DISTANT.id);
|
||||
});
|
||||
|
||||
it('will not clear an existing match on a name miss when it has no coordinates', () => {
|
||||
// A name miss says the name missed, not that the star is absent — and the earlier match may
|
||||
// have been positional, from data this record no longer carries.
|
||||
const planets = [planet({ hostStarId: 999, hostStarName: 'Some Survey Designation' })];
|
||||
const summary = rematchHostStars(planets, WIDE_CATALOGUE);
|
||||
|
||||
expect(planets[0].hostStarId).toBe(999);
|
||||
expect(summary.lost).toBe(0);
|
||||
expect(summary.matched).toBe(1);
|
||||
});
|
||||
|
||||
it('takes the new answer outright when the record does carry coordinates', () => {
|
||||
// With coordinates the match can be redone in full, so its result is authoritative — a host
|
||||
// that no longer resolves is cleared rather than left pointing at a star that may be gone.
|
||||
const planets = [planet({ hostStarId: 999, hostStarName: 'Nowhere', hostRaDeg: 10, hostDecDeg: 10, hostDistancePc: 500 })];
|
||||
const summary = rematchHostStars(planets, WIDE_CATALOGUE);
|
||||
|
||||
expect(planets[0].hostStarId).toBeNull();
|
||||
expect(summary.resolvable).toBe(1);
|
||||
expect(summary.lost).toBe(1);
|
||||
});
|
||||
|
||||
it('matches a positioned host to the catalogue star at its coordinates', () => {
|
||||
const planets = [planet({ hostStarName: 'unlisted alias', hostRaDeg: 90, hostDecDeg: 0, hostDistancePc: 120 })];
|
||||
rematchHostStars(planets, WIDE_CATALOGUE);
|
||||
expect(planets[0].hostStarId).toBe(DISTANT.id);
|
||||
});
|
||||
|
||||
it('leaves a host that neither catalogue contains unmatched', () => {
|
||||
const planets = [planet()];
|
||||
const summary = rematchHostStars(planets, NARROW_CATALOGUE);
|
||||
|
||||
expect(planets[0].hostStarId).toBeNull();
|
||||
expect(summary.matched).toBe(0);
|
||||
expect(summary.gained).toBe(0);
|
||||
});
|
||||
|
||||
it('counts every record it was given', () => {
|
||||
const planets = [planet(), planet({ id: 'q', hostStarName: 'Gl 357' }), planet({ id: 'r', hostStarName: 'nobody' })];
|
||||
const summary = rematchHostStars(planets, WIDE_CATALOGUE);
|
||||
|
||||
expect(summary.total).toBe(3);
|
||||
expect(summary.matched).toBe(2);
|
||||
});
|
||||
});
|
||||
@@ -20,5 +20,18 @@ export interface ExoplanetRecord {
|
||||
periodDays?: number;
|
||||
/** Host star mass in solar masses (`st_mass`); the fallback when no period is published. */
|
||||
hostStarMassSolar?: number;
|
||||
/**
|
||||
* The host star's own published position (`ra`, `dec`, `sy_dist`) — the coordinates the
|
||||
* cross-reference above is resolved from.
|
||||
*
|
||||
* Kept rather than consumed and discarded. `hostStarId` is the *result* of a match against
|
||||
* whatever star catalogue was loaded at the time, so widening that catalogue ought to rescue
|
||||
* some of the 4347 hosts that currently resolve to nothing — but with only the result stored,
|
||||
* redoing the match meant re-downloading the archive. These three numbers make it a local
|
||||
* operation. See `rematchHostStars`.
|
||||
*/
|
||||
hostRaDeg?: number;
|
||||
hostDecDeg?: number;
|
||||
hostDistancePc?: number;
|
||||
orbit: Partial<OrbitalElements>;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,93 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
|
||||
import { BYTES_PER_STAR_META, BYTES_PER_STAR_POSITION, decodeStarCatalog, encodeStarCatalog } from './star-catalog';
|
||||
import { StarRecord } from './star.model';
|
||||
|
||||
const STARS: StarRecord[] = [
|
||||
{ id: 0, name: 'Sol', x: 0, y: 0, z: 0, magnitude: -26.7, spectralType: 'G2V', colorIndex: 0.656 },
|
||||
{ id: 71456, name: 'Rigil Kentaurus', x: -1.35, y: -0.04, z: -0.98, magnitude: -0.01, spectralType: 'G2V', colorIndex: 0.71 },
|
||||
{ id: 32263, name: 'Sirius', x: -0.49, y: 2.47, z: -0.75, magnitude: -1.44, spectralType: 'A0m...', colorIndex: 0.009 },
|
||||
// The case a plain number cannot carry: about a tenth of the catalogue was never photometered.
|
||||
{ id: 118554, name: 'GJ 3512', x: 20.1, y: -3.4, z: 8.8, magnitude: 15, spectralType: 'Unknown', colorIndex: null }
|
||||
];
|
||||
|
||||
describe('encodeStarCatalog / decodeStarCatalog', () => {
|
||||
const encoded = encodeStarCatalog(STARS);
|
||||
const decoded = decodeStarCatalog(encoded.index, encoded.positions, encoded.meta);
|
||||
|
||||
it('round-trips every field of every star', () => {
|
||||
expect(decoded).toHaveLength(STARS.length);
|
||||
decoded.forEach((star, index) => {
|
||||
const original = STARS[index];
|
||||
expect(star.id).toBe(original.id);
|
||||
expect(star.name).toBe(original.name);
|
||||
expect(star.spectralType).toBe(original.spectralType);
|
||||
expect(star.x).toBeCloseTo(original.x, 4);
|
||||
expect(star.y).toBeCloseTo(original.y, 4);
|
||||
expect(star.z).toBeCloseTo(original.z, 4);
|
||||
expect(star.magnitude).toBeCloseTo(original.magnitude, 4);
|
||||
});
|
||||
});
|
||||
|
||||
it('carries an absent colour index through as null, not as zero', () => {
|
||||
// Zero is a real colour index meaning a hot blue-white A-type star, so it cannot double as
|
||||
// "not measured" — the float column uses NaN, which nothing else can be.
|
||||
expect(decoded[3].colorIndex).toBeNull();
|
||||
expect(decoded[0].colorIndex).toBeCloseTo(0.656, 5);
|
||||
expect(decoded[2].colorIndex).toBeCloseTo(0.009, 5);
|
||||
});
|
||||
|
||||
it('sizes both binaries exactly to the star count', () => {
|
||||
expect(encoded.positions.byteLength).toBe(STARS.length * BYTES_PER_STAR_POSITION);
|
||||
expect(encoded.meta.byteLength).toBe(STARS.length * BYTES_PER_STAR_META);
|
||||
});
|
||||
|
||||
it('hands positions over as a bare xyz buffer, which is what the GPU is given', () => {
|
||||
expect(Array.from(encoded.positions.slice(0, 3))).toEqual([0, 0, 0]);
|
||||
expect(encoded.positions[3]).toBeCloseTo(-1.35, 4);
|
||||
});
|
||||
|
||||
it('stores each distinct spectral type once and refers to it by index', () => {
|
||||
// Two of the four stars are G2V. Across the real catalogue this is 68000 stars sharing
|
||||
// about 2600 strings, which is why the dictionary is worth having.
|
||||
expect(encoded.index.spectralTypes).toEqual(['G2V', 'A0m...', 'Unknown']);
|
||||
});
|
||||
|
||||
it('keeps the index free of anything that is not a string, since the numbers are elsewhere', () => {
|
||||
expect(Object.keys(encoded.index).sort()).toEqual(['count', 'names', 'spectralTypes']);
|
||||
expect(encoded.index.count).toBe(STARS.length);
|
||||
expect(encoded.index.names).toEqual(STARS.map((star) => star.name));
|
||||
});
|
||||
|
||||
it('is smaller than the array of objects it replaced', () => {
|
||||
// The whole reason for the format: the old encoding repeated eight key names per star.
|
||||
const asObjects = JSON.stringify(STARS).length;
|
||||
const asCatalogue = JSON.stringify(encoded.index).length + encoded.positions.byteLength + encoded.meta.byteLength;
|
||||
expect(asCatalogue).toBeLessThan(asObjects);
|
||||
});
|
||||
|
||||
it('handles an empty catalogue without producing a malformed buffer', () => {
|
||||
const empty = encodeStarCatalog([]);
|
||||
expect(empty.positions.byteLength).toBe(0);
|
||||
expect(empty.meta.byteLength).toBe(0);
|
||||
expect(decodeStarCatalog(empty.index, empty.positions, empty.meta)).toEqual([]);
|
||||
});
|
||||
|
||||
it('survives more distinct spectral types than a handful, up to the column width', () => {
|
||||
// The dictionary index is 16-bit, and the real catalogue has about 2600 distinct types.
|
||||
const many: StarRecord[] = Array.from({ length: 5000 }, (_, i) => ({
|
||||
id: i,
|
||||
name: `HYG ${i}`,
|
||||
x: i,
|
||||
y: 0,
|
||||
z: 0,
|
||||
magnitude: 10,
|
||||
spectralType: `S${i}`,
|
||||
colorIndex: null
|
||||
}));
|
||||
const round = decodeStarCatalog(...(({ index, positions, meta }) => [index, positions, meta] as const)(encodeStarCatalog(many)));
|
||||
|
||||
expect(round[4999].spectralType).toBe('S4999');
|
||||
expect(round[4999].id).toBe(4999);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,131 @@
|
||||
import { StarRecord } from './star.model';
|
||||
|
||||
/**
|
||||
* On-disk format for the star catalogue, shared by the ETL that writes it and the app that
|
||||
* reads it so the two cannot drift apart.
|
||||
*
|
||||
* The catalogue outgrew a plain array of JSON objects. At the 50 pc cutoff it held 8750 stars
|
||||
* and cost 157 bytes each — most of that the same eight key names repeated once per star. At
|
||||
* the distance Hipparcos parallaxes actually reach, that same encoding would have been about
|
||||
* 17 MB of JSON to parse before the first frame.
|
||||
*
|
||||
* So the numbers move to a binary column store and the strings stay in JSON, where the two
|
||||
* repetitive ones — spectral types, of which 68000 stars share about 2600 distinct values —
|
||||
* collapse into a dictionary. The result is roughly a quarter of the size for eight times the
|
||||
* stars, and the numeric columns arrive as typed arrays with no parsing at all.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Positions stay in their own file rather than joining the columns below.
|
||||
*
|
||||
* They are the one column handed to the GPU verbatim: `StarFieldRenderer` binds the buffer
|
||||
* straight from `stars.bin` as an instanced attribute, so keeping it a bare `Float32Array` of
|
||||
* xyz triples means the star field costs one fetch and no repacking.
|
||||
*/
|
||||
export const STAR_POSITION_COMPONENTS = 3;
|
||||
export const BYTES_PER_STAR_POSITION = STAR_POSITION_COMPONENTS * Float32Array.BYTES_PER_ELEMENT;
|
||||
|
||||
/**
|
||||
* Columns in `stars-meta.bin`, in order: catalogue id, apparent magnitude, colour index, and an
|
||||
* index into the spectral-type dictionary. Stored column by column rather than record by record
|
||||
* so each one is a single typed-array view over the buffer, with no per-record stride or
|
||||
* alignment padding.
|
||||
*/
|
||||
export const BYTES_PER_STAR_META =
|
||||
Int32Array.BYTES_PER_ELEMENT + Float32Array.BYTES_PER_ELEMENT + Float32Array.BYTES_PER_ELEMENT + Uint16Array.BYTES_PER_ELEMENT;
|
||||
|
||||
/** `stars-index.json`: everything that is a string, plus the count the columns are sized by. */
|
||||
export interface StarCatalogIndex {
|
||||
count: number;
|
||||
/** One per star, in catalogue order. */
|
||||
names: string[];
|
||||
/** Distinct spectral classifications; the meta column holds indices into this. */
|
||||
spectralTypes: string[];
|
||||
}
|
||||
|
||||
interface StarMetaColumns {
|
||||
ids: Int32Array;
|
||||
magnitudes: Float32Array;
|
||||
colorIndices: Float32Array;
|
||||
spectralTypeIndices: Uint16Array;
|
||||
}
|
||||
|
||||
/** Lays typed-array views over the meta buffer at the offsets the format defines. */
|
||||
function metaColumns(buffer: ArrayBuffer, count: number): StarMetaColumns {
|
||||
let offset = 0;
|
||||
const ids = new Int32Array(buffer, offset, count);
|
||||
offset += count * Int32Array.BYTES_PER_ELEMENT;
|
||||
const magnitudes = new Float32Array(buffer, offset, count);
|
||||
offset += count * Float32Array.BYTES_PER_ELEMENT;
|
||||
const colorIndices = new Float32Array(buffer, offset, count);
|
||||
offset += count * Float32Array.BYTES_PER_ELEMENT;
|
||||
const spectralTypeIndices = new Uint16Array(buffer, offset, count);
|
||||
|
||||
return { ids, magnitudes, colorIndices, spectralTypeIndices };
|
||||
}
|
||||
|
||||
/**
|
||||
* Packs the string and numeric halves of a star list into the two files the app loads.
|
||||
*
|
||||
* `colorIndex` is genuinely nullable — about a tenth of the catalogue was never photometered —
|
||||
* and `NaN` carries that through the float column. It is the one value a float can hold that
|
||||
* means "no measurement" without colliding with a real one, and 0 emphatically does not: it is
|
||||
* a real colour index meaning a hot blue-white A-type star.
|
||||
*/
|
||||
export function encodeStarCatalog(stars: readonly StarRecord[]): {
|
||||
index: StarCatalogIndex;
|
||||
positions: Float32Array;
|
||||
meta: ArrayBuffer;
|
||||
} {
|
||||
const count = stars.length;
|
||||
const positions = new Float32Array(count * STAR_POSITION_COMPONENTS);
|
||||
const meta = new ArrayBuffer(count * BYTES_PER_STAR_META);
|
||||
const columns = metaColumns(meta, count);
|
||||
|
||||
const spectralTypes: string[] = [];
|
||||
const spectralTypeIds = new Map<string, number>();
|
||||
const names: string[] = [];
|
||||
|
||||
stars.forEach((star, index) => {
|
||||
positions[index * 3] = star.x;
|
||||
positions[index * 3 + 1] = star.y;
|
||||
positions[index * 3 + 2] = star.z;
|
||||
|
||||
names.push(star.name);
|
||||
|
||||
let spectralTypeId = spectralTypeIds.get(star.spectralType);
|
||||
if (spectralTypeId === undefined) {
|
||||
spectralTypeId = spectralTypes.push(star.spectralType) - 1;
|
||||
spectralTypeIds.set(star.spectralType, spectralTypeId);
|
||||
}
|
||||
|
||||
columns.ids[index] = star.id;
|
||||
columns.magnitudes[index] = star.magnitude;
|
||||
columns.colorIndices[index] = star.colorIndex ?? Number.NaN;
|
||||
columns.spectralTypeIndices[index] = spectralTypeId;
|
||||
});
|
||||
|
||||
return { index: { count, names, spectralTypes }, positions, meta };
|
||||
}
|
||||
|
||||
/** Rebuilds the star records the app works with from the three loaded assets. */
|
||||
export function decodeStarCatalog(index: StarCatalogIndex, positions: Float32Array, meta: ArrayBuffer): StarRecord[] {
|
||||
const columns = metaColumns(meta, index.count);
|
||||
const stars: StarRecord[] = new Array(index.count);
|
||||
|
||||
for (let i = 0; i < index.count; i++) {
|
||||
const colorIndex = columns.colorIndices[i];
|
||||
stars[i] = {
|
||||
id: columns.ids[i],
|
||||
name: index.names[i],
|
||||
x: positions[i * 3],
|
||||
y: positions[i * 3 + 1],
|
||||
z: positions[i * 3 + 2],
|
||||
magnitude: columns.magnitudes[i],
|
||||
spectralType: index.spectralTypes[columns.spectralTypeIndices[i]],
|
||||
colorIndex: Number.isNaN(colorIndex) ? null : colorIndex
|
||||
};
|
||||
}
|
||||
|
||||
return stars;
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
Binary file not shown.
Binary file not shown.
+29
-2
@@ -7,7 +7,9 @@ import { StarRecord, SUN_STAR_ID } from '../../src/app/shared/models/star.model'
|
||||
import { fetchDeepSky } from './fetchDeepSky';
|
||||
import { fetchExoplanets } from './fetchExoplanets';
|
||||
import { fetchSolarSystem } from './fetchSolarSystem';
|
||||
import { BYTES_PER_STAR_META, BYTES_PER_STAR_POSITION, decodeStarCatalog, encodeStarCatalog } from '../../src/app/shared/models/star-catalog';
|
||||
import { fetchStars } from './fetchStars';
|
||||
import { rematchHostStars } from '../../src/app/shared/astro/host-star-matching';
|
||||
import { dataPath } from './lib/paths';
|
||||
|
||||
class ValidationError extends Error {}
|
||||
@@ -30,8 +32,21 @@ function validateStars(stars: StarRecord[]): void {
|
||||
assertCondition([star.x, star.y, star.z].every(Number.isFinite), `Star ${star.id} has a non-finite position.`);
|
||||
}
|
||||
|
||||
const binSize = statSync(dataPath('stars.bin')).size;
|
||||
assertCondition(binSize === stars.length * 3 * 4, `stars.bin size (${binSize}) does not match ${stars.length} stars.`);
|
||||
const positionBytes = statSync(dataPath('stars.bin')).size;
|
||||
assertCondition(positionBytes === stars.length * BYTES_PER_STAR_POSITION, `stars.bin size (${positionBytes}) does not match ${stars.length} stars.`);
|
||||
const metaBytes = statSync(dataPath('stars-meta.bin')).size;
|
||||
assertCondition(metaBytes === stars.length * BYTES_PER_STAR_META, `stars-meta.bin size (${metaBytes}) does not match ${stars.length} stars.`);
|
||||
|
||||
// Round-trips the written assets back through the decoder the app uses, so a format change
|
||||
// that only half-lands fails here rather than as a silently wrong star map.
|
||||
const { index, positions, meta } = encodeStarCatalog(stars);
|
||||
const decoded = decodeStarCatalog(index, positions, meta);
|
||||
assertCondition(decoded.length === stars.length, `Star catalogue round-trip lost records: ${decoded.length} of ${stars.length}.`);
|
||||
for (let i = 0; i < stars.length; i++) {
|
||||
assertCondition(decoded[i].id === stars[i].id && decoded[i].name === stars[i].name, `Star catalogue round-trip altered record ${i}.`);
|
||||
assertCondition(decoded[i].spectralType === stars[i].spectralType, `Star catalogue round-trip lost the spectral type of star ${stars[i].id}.`);
|
||||
assertCondition(decoded[i].colorIndex === null === (stars[i].colorIndex === null), `Star catalogue round-trip changed whether star ${stars[i].id} has a colour index.`);
|
||||
}
|
||||
}
|
||||
|
||||
function validateBodies(bodies: BodyRecord[]): void {
|
||||
@@ -142,6 +157,18 @@ async function build(): Promise<void> {
|
||||
const deepSky = await fetchDeepSky();
|
||||
console.log();
|
||||
|
||||
// The cross-reference depends on the star catalogue as much as on the archive, so it is
|
||||
// resolved again here against whatever catalogue this run produced. A no-op when the two were
|
||||
// fetched together, and the whole point when only one of them was.
|
||||
const rematch = rematchHostStars(exoplanets, stars);
|
||||
console.log(
|
||||
`Cross-referencing exoplanet hosts against ${stars.length} stars...\n` +
|
||||
` ${rematch.matched}/${rematch.total} matched` +
|
||||
(rematch.resolvable < rematch.total ? ` (${rematch.total - rematch.resolvable} records predate stored host coordinates and kept their existing match)` : '') +
|
||||
(rematch.gained || rematch.lost ? `; ${rematch.gained} gained, ${rematch.lost} lost` : '')
|
||||
);
|
||||
console.log();
|
||||
|
||||
console.log('Validating output...');
|
||||
validateStars(stars);
|
||||
validateBodies(bodies);
|
||||
|
||||
@@ -75,6 +75,11 @@ export async function fetchExoplanets(stars?: StarRecord[]): Promise<ExoplanetRe
|
||||
// propagating a planet at its real rate and pretending every host is the Sun.
|
||||
periodDays: parseOptionalNumber(row['pl_orbper']),
|
||||
hostStarMassSolar: parseOptionalNumber(row['st_mass']),
|
||||
// Kept so the cross-reference can be redone without the archive; see the record's own
|
||||
// documentation. Undefined rather than NaN, which JSON cannot represent.
|
||||
hostRaDeg: parseOptionalNumber(row['ra']),
|
||||
hostDecDeg: parseOptionalNumber(row['dec']),
|
||||
hostDistancePc: parseOptionalNumber(row['sy_dist']),
|
||||
orbit: {
|
||||
semiMajorAxisAu: parseOptionalNumber(row['pl_orbsmax']),
|
||||
eccentricity: parseOptionalNumber(row['pl_orbeccen']),
|
||||
|
||||
+19
-9
@@ -1,6 +1,7 @@
|
||||
import { writeFileSync } from 'node:fs';
|
||||
|
||||
import { raDecDistanceToXyz } from '../../src/app/shared/astro/coordinates';
|
||||
import { encodeStarCatalog } from '../../src/app/shared/models/star-catalog';
|
||||
import { StarRecord, SUN_STAR_ID } from '../../src/app/shared/models/star.model';
|
||||
import { parseCsvObjects, parseOptionalNumber } from './lib/csv';
|
||||
import { fetchTextCached } from './lib/http';
|
||||
@@ -15,8 +16,19 @@ const HYG_UNKNOWN_DISTANCE_PC = 100000; // HYG's placeholder for unmeasured/unre
|
||||
*/
|
||||
const UNKNOWN_MAGNITUDE = 15;
|
||||
|
||||
/** Stars within this distance (parsecs) of the Sun are kept for the galaxy view. */
|
||||
const DISTANCE_CUTOFF_PC = Number(process.env['ETL_STAR_DISTANCE_PC'] ?? 50);
|
||||
/**
|
||||
* Stars within this distance (parsecs) of the Sun are kept for the galaxy view.
|
||||
*
|
||||
* Set at the range HYG's own measurements reach rather than at a round number. 98.6% of its
|
||||
* rows carry a Hipparcos identifier, and Hipparcos parallaxes are good to roughly a
|
||||
* milliarcsecond — so at 250 pc (4 mas) a star's distance is uncertain by some tens of per
|
||||
* cent, and beyond it the catalogue is plotting noise. Note that only the *radial* placement
|
||||
* blurs: a star's direction on the sky stays exact at any distance.
|
||||
*
|
||||
* The catalogue is also magnitude-limited, so this is not a volume-complete sample beyond about
|
||||
* 50 pc: it thins to the intrinsically bright, which is the same selection the naked eye makes.
|
||||
*/
|
||||
const DISTANCE_CUTOFF_PC = Number(process.env['ETL_STAR_DISTANCE_PC'] ?? 250);
|
||||
|
||||
function resolveName(row: Record<string, string>): string {
|
||||
if (row['proper']) {
|
||||
@@ -98,15 +110,13 @@ export async function fetchStars(): Promise<StarRecord[]> {
|
||||
function writeStarAssets(stars: StarRecord[]): void {
|
||||
ensureDataDir();
|
||||
|
||||
const positions = new Float32Array(stars.length * 3);
|
||||
stars.forEach((star, index) => {
|
||||
positions[index * 3] = star.x;
|
||||
positions[index * 3 + 1] = star.y;
|
||||
positions[index * 3 + 2] = star.z;
|
||||
});
|
||||
// The layout lives in `star-catalog.ts`, which the app decodes with — one definition, so the
|
||||
// writer and the reader cannot drift.
|
||||
const { index, positions, meta } = encodeStarCatalog(stars);
|
||||
|
||||
writeFileSync(dataPath('stars.bin'), Buffer.from(positions.buffer));
|
||||
writeFileSync(dataPath('stars-index.json'), JSON.stringify(stars));
|
||||
writeFileSync(dataPath('stars-meta.bin'), Buffer.from(meta));
|
||||
writeFileSync(dataPath('stars-index.json'), JSON.stringify(index));
|
||||
}
|
||||
|
||||
if (require.main === module) {
|
||||
|
||||
@@ -0,0 +1,18 @@
|
||||
import { readFileSync, writeFileSync } from 'node:fs';
|
||||
|
||||
import { RematchSummary, rematchHostStars } from '../../src/app/shared/astro/host-star-matching';
|
||||
import { ExoplanetRecord } from '../../src/app/shared/models/exoplanet.model';
|
||||
import { StarRecord } from '../../src/app/shared/models/star.model';
|
||||
import { dataPath } from './lib/paths';
|
||||
|
||||
/**
|
||||
* Reads the written assets, re-resolves every exoplanet's host star against the given catalogue,
|
||||
* and writes the exoplanets back. The matching itself lives with the matcher, in
|
||||
* `host-star-matching.ts`; this is only the file handling around it.
|
||||
*/
|
||||
export function rematchWrittenAssets(stars: readonly StarRecord[]): RematchSummary {
|
||||
const exoplanets = JSON.parse(readFileSync(dataPath('exoplanets.json'), 'utf8')) as ExoplanetRecord[];
|
||||
const summary = rematchHostStars(exoplanets, stars);
|
||||
writeFileSync(dataPath('exoplanets.json'), JSON.stringify(exoplanets));
|
||||
return summary;
|
||||
}
|
||||
Reference in New Issue
Block a user