Say so when Gaia is missing, rather than as 68 000 unmatched stars
With the merge gate in place, a Gaia DR3 outage no longer ships a HYG-only catalogue: the ETL skips the unreachable source, and validateMerge then fails on the survivor count. That is the right outcome and the wrong message: "68 000 HYG stars found no Gaia counterpart" sends the reader looking at the merge. Gaia contributing nothing is now checked first, by name. Two comments said Gaia was best-effort, in data-refresh.yml and on the merge in fetchStars. They now say what happens instead. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016jxMkwA2rbicdGxHosecYi
This commit is contained in:
@@ -41,7 +41,8 @@ jobs:
|
|||||||
|
|
||||||
# The ETL's cache directory is gitignored and this is a fresh runner, so every source is
|
# The ETL's cache directory is gitignored and this is a fresh runner, so every source is
|
||||||
# fetched live (~50-100 MB). A failed fetch fails the run by design — no refresh is
|
# fetched live (~50-100 MB). A failed fetch fails the run by design — no refresh is
|
||||||
# better than a partial one — except Gaia, which the ETL itself treats as best-effort.
|
# better than a partial one. That includes Gaia: the ETL skips it when unreachable, and
|
||||||
|
# the merge gate in build.ts then refuses a catalogue it contributed nothing to.
|
||||||
- name: Rebuild the datasets
|
- name: Rebuild the datasets
|
||||||
run: npm run etl
|
run: npm run etl
|
||||||
|
|
||||||
|
|||||||
@@ -74,6 +74,12 @@ const MAX_HYG_SURVIVORS = 15_000;
|
|||||||
const TWIN_TOLERANCE_RAD = (1 / 3600) * (Math.PI / 180);
|
const TWIN_TOLERANCE_RAD = (1 / 3600) * (Math.PI / 180);
|
||||||
|
|
||||||
function validateMerge(stars: StarRecord[]): void {
|
function validateMerge(stars: StarRecord[]): void {
|
||||||
|
// Checked first and on its own: an unreachable Gaia is skipped rather than thrown, and would
|
||||||
|
// otherwise surface below as "68 000 HYG stars found no counterpart" — true, and no help.
|
||||||
|
assertCondition(
|
||||||
|
stars.some((star) => star.source === 'gaia'),
|
||||||
|
'Gaia DR3 contributed no stars — the archive was unreachable or returned nothing, and a catalogue without it is not one to publish.'
|
||||||
|
);
|
||||||
const survivors = stars.filter((star) => star.source === 'hyg').length;
|
const survivors = stars.filter((star) => star.source === 'hyg').length;
|
||||||
assertCondition(
|
assertCondition(
|
||||||
survivors <= MAX_HYG_SURVIVORS,
|
survivors <= MAX_HYG_SURVIVORS,
|
||||||
|
|||||||
@@ -119,9 +119,9 @@ export async function fetchStars(): Promise<StarRecord[]> {
|
|||||||
/**
|
/**
|
||||||
* Unions HYG with every other positional source that is wired in and reachable.
|
* Unions HYG with every other positional source that is wired in and reachable.
|
||||||
*
|
*
|
||||||
* A source that cannot be reached is reported and skipped rather than failing the run. That is
|
* A source that cannot be reached is reported and skipped here rather than thrown, so a run still
|
||||||
* not defensive padding: the archives this would draw on are frequently unavailable, and a build
|
* gets as far as validation and says what it has. Whether that may be published is decided
|
||||||
* that produces a smaller catalogue is far better than one that produces none.
|
* there: `validateMerge` in build.ts refuses a catalogue Gaia contributed nothing to.
|
||||||
*/
|
*/
|
||||||
async function mergeWithOtherSources(hygStars: StarRecord[]): Promise<StarRecord[]> {
|
async function mergeWithOtherSources(hygStars: StarRecord[]): Promise<StarRecord[]> {
|
||||||
const others = positionalSources().filter((source) => source.id !== 'hyg');
|
const others = positionalSources().filter((source) => source.id !== 'hyg');
|
||||||
|
|||||||
Reference in New Issue
Block a user