Draw the whole catalogue, and build the aggregation the rest would need
Two things, one verified and one that cannot be. The render budget is now the whole catalogue: 68388 stars, one instanced draw call, which is what a GPU should be asked to do. The budget itself stays, because the catalogue is meant to grow past what any machine should draw at once — Gaia alone could contribute a million — and at that point the selection is what keeps the field legible rather than a grey wash. A `?stars=` override handles the machines that cannot, including the software rasterizer the end-to-end suite runs against, whose frame rate is two orders of magnitude below a real GPU's and which was measuring the rasterizer rather than the app. The aggregation is the second thing, and none of it has run. Every ESA, NOIRLab, SDSS and Euclid endpoint is unreachable from here — only GitHub raw is, which is why HYG and OpenNGC are the current sources. So this is infrastructure and a Gaia query written against the published DR3 schema, not data. What the framework encodes is that these surveys are not interchangeable. The distinction is not size but whether a catalogue knows how far away its objects are, because a 3D map cannot place a star it only has a direction for. Gaia is the only one of the five that can add stars here, because it is the only one that measures parallaxes. DECaPS2 has fifty times Gaia's object count and photometry alone — not one of its 3.32 billion objects can be placed in depth. Euclid's bulge is 8 kpc away, where a parallax is microarcseconds; its contribution would be imagery. SDSS-V and SAGA are keyed to stars something else already places, so they enrich rather than extend. Those roles are recorded as data the ETL prints, not as prose that can drift. Overlapping catalogues are reconciled on direction rather than on 3D proximity, which is the one non-obvious part. Two surveys agree on a star's direction to within an arcsecond and disagree on its distance by tens of per cent, so a star at 200 pc is 50 pc from itself between catalogues while being unmistakably the same object. Matching in 3D would need a tolerance so loose it swallowed real neighbours. The better parallax wins where both reach; where only one does, the star stays. Names become dense-with-holes with a source dictionary, because a survey catalogue has no proper names — writing "Gaia DR3 4472832130942575872" once per star would cost 25 MB per million to repeat what two adjacent fields already say. An empty entry costs three bytes and is regenerated on load. The Sun needed its own case in the merge: it sits at the origin, has no direction to compare, and appears in every catalogue. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WaySiNst4HhDXBHnMy8p5G
This commit is contained in:
@@ -169,7 +169,7 @@ re-runs are cheap and offline-friendly; set `ETL_FORCE_REFRESH=1` to bypass the
|
||||
|
||||
| Script | Source | Output |
|
||||
| --- | --- | --- |
|
||||
| `fetchStars.ts` | HYG database (Hipparcos/Yale/Gliese) | `stars.bin`, `stars-meta.bin`, `stars-index.json` |
|
||||
| `fetchStars.ts` | HYG database, plus any other positional catalogue wired in (see below) | `stars.bin`, `stars-meta.bin`, `stars-index.json` |
|
||||
| `fetchSolarSystem.ts` | JPL Horizons / SSD | `bodies.json` |
|
||||
| `fetchExoplanets.ts` | NASA Exoplanet Archive (TAP) | `exoplanets.json` |
|
||||
| `fetchDeepSky.ts` | OpenNGC | `deepsky.json` |
|
||||
@@ -185,6 +185,40 @@ and both the ETL and the app use it, so the writer and the reader cannot drift a
|
||||
|
||||
`ETL_STAR_DISTANCE_PC` (default `250`) sets the star-field distance cutoff.
|
||||
|
||||
### On aggregating other surveys
|
||||
|
||||
`tools/etl/sources/registry.ts` lists every catalogue the pipeline knows about, and `npm run etl`
|
||||
prints it. Sources are wired in by role, because the roles are not interchangeable:
|
||||
|
||||
| Source | Role | What it adds |
|
||||
| --- | --- | --- |
|
||||
| HYG | positional | The named, spectrally classified bright-star spine — 68 388 stars within 250 pc. |
|
||||
| Gaia DR3 | positional | Parallaxes fifty times more precise, for 1.8 billion sources. |
|
||||
| DECaPS2 | backdrop | 3.32 billion objects across the southern galactic plane. |
|
||||
| SDSS-V Milky Way Mapper | enrichment | Spectroscopic temperatures, gravities, metallicities, radial velocities. |
|
||||
| Euclid Bulge Survey (Q2) | backdrop | High-resolution imagery and astrometry of the inner bulge. |
|
||||
| SAGA | enrichment | Compiled elemental abundances for metal-poor stars. |
|
||||
|
||||
The distinction that matters is not size — it is whether a catalogue knows how **far away** its
|
||||
objects are, because a 3D map cannot place a star it only has a direction for. **Gaia is the only
|
||||
one of these that can add stars to this map**, because it is the only one that measures
|
||||
parallaxes. DECaPS2 has fifty times Gaia's object count and photometry alone: not one of its
|
||||
3.32 billion objects can be placed in depth, so it can only ever be a direction-only backdrop
|
||||
beside the deep-sky shell. Euclid's bulge sits 8 kpc away, where a parallax is microarcseconds —
|
||||
its natural contribution here is imagery, not positions. SDSS-V and SAGA are keyed to stars
|
||||
another catalogue already places; they enrich what is there and cannot extend it.
|
||||
|
||||
Where two positional catalogues overlap they are reconciled by `star-merge.ts`, which matches on
|
||||
**direction** rather than on 3D proximity. Two surveys agree on a star's direction to within an
|
||||
arcsecond and disagree on its distance by tens of per cent — so a star at 200 pc can be 50 pc
|
||||
from itself between catalogues while being unmistakably the same object. Where both have a star,
|
||||
the one with the better parallax wins; where only one reaches, the star is still there.
|
||||
|
||||
Only HYG has ever run. Every ESA, NOIRLab, SDSS and Euclid endpoint is unreachable from the
|
||||
environment this was developed in, so the Gaia query is written against the published DR3 schema
|
||||
and has not been executed against it. A source that cannot be reached is reported and skipped
|
||||
rather than failing the build.
|
||||
|
||||
### On how many stars
|
||||
|
||||
Not many, against the Galaxy. It holds 100–400 billion stars and this map ships 68 388 of them —
|
||||
|
||||
Reference in New Issue
Block a user