Claude
4aca223027
Two ETL bugs, both fixed at the source and then re-run against HYG. Star ids,
ordering and positions are all unchanged, so stars.bin is byte-identical and
every exoplanet cross-reference still resolves.
Names. HYG's `gl` column already carries its own catalogue prefix ("Gl 581",
"GJ 3512"), unlike the bare numbers in `hd` and `hip`, so prefixing it again
produced 2331 of 8750 stars named "Gl GJ 1076". That corrupted three surfaces at
once: search, the on-screen labels, and exoplanet host-star name matching, which
compares normalised names and could never match "glgj1076" to "gj1076".
Colours. `Number(row['ci']) || 0` cannot tell a blank cell from a real zero, and
0 is a real B-V colour index meaning a hot blue-white A-type star. All 875
affected stars turned out to be blanks — the catalogue contains no genuine zero
inside the distance cutoff — so several hundred red dwarfs were rendering
blue-white. colorIndex is now `number | null` rather than defaulted, because any
numeric default is indistinguishable from a measurement.
Consumers resolve the gap from the spectral type instead. That needs real
parsing: HYG's `spect` column runs to 134 distinct spellings among the affected
stars alone, including a bare lowercase "m" for 354 of them, plus "k-m" ranges,
"dM4" luminosity prefixes and "K:" uncertainty flags. 622 of the 875 recover a
class this way — 497 of them M-class — and the remaining 253, which carry no
classification at all, fall back to neutral white.
The parse is anchored at the start of the string rather than scanning it. A scan
is the obvious implementation and is quietly wrong: the ETL writes the literal
"Unknown" for unclassified stars, that contains a K, and every one of those 253
would have been classified as an orange K-type. A test covers it.
Also lifts parseOptionalNumber out of fetchExoplanets into lib/csv, where both
fetchers now use it, and gives magnitude a faint default instead of 0 — no
current star is affected, but 0 would mean "as bright as Vega" and render an
unphotometered star as one of the largest points on the map.
Tests: 145 passing, up from 116, including the first coverage of
StarFieldRenderer. Build, both typechecks and the Playwright suite are green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WaySiNst4HhDXBHnMy8p5G
87 lines
2.5 KiB
TypeScript
87 lines
2.5 KiB
TypeScript
/**
|
|
* Minimal RFC4180-ish CSV parser: supports a custom delimiter, quoted fields (with escaped
|
|
* `""`), and embedded newlines inside quotes. Good enough for the HYG/Exoplanet Archive
|
|
* exports used by the ETL, without pulling in an extra dependency.
|
|
*/
|
|
export function parseCsv(text: string, delimiter = ','): string[][] {
|
|
const rows: string[][] = [];
|
|
let row: string[] = [];
|
|
let field = '';
|
|
let inQuotes = false;
|
|
|
|
for (let i = 0; i < text.length; i++) {
|
|
const char = text[i];
|
|
|
|
if (inQuotes) {
|
|
if (char === '"') {
|
|
if (text[i + 1] === '"') {
|
|
field += '"';
|
|
i++;
|
|
} else {
|
|
inQuotes = false;
|
|
}
|
|
} else {
|
|
field += char;
|
|
}
|
|
continue;
|
|
}
|
|
|
|
if (char === '"') {
|
|
inQuotes = true;
|
|
} else if (char === delimiter) {
|
|
row.push(field);
|
|
field = '';
|
|
} else if (char === '\r') {
|
|
// skip; \n (handled below) terminates the row
|
|
} else if (char === '\n') {
|
|
row.push(field);
|
|
rows.push(row);
|
|
row = [];
|
|
field = '';
|
|
} else {
|
|
field += char;
|
|
}
|
|
}
|
|
|
|
if (field.length > 0 || row.length > 0) {
|
|
row.push(field);
|
|
rows.push(row);
|
|
}
|
|
|
|
return rows;
|
|
}
|
|
|
|
/**
|
|
* Reads a numeric CSV cell, treating a missing/blank/unparseable value as absent.
|
|
*
|
|
* Always prefer this to a bare `Number(cell)`: catalogs leave unmeasured fields empty, and
|
|
* `Number('')` is `0` — a finite, entirely plausible-looking value. That single coercion has
|
|
* already produced two separate bugs here, painting 875 unphotometered stars as if they had a
|
|
* measured colour index of 0, and placing exoplanet hosts at the origin where they matched the
|
|
* Sun.
|
|
*/
|
|
export function parseOptionalNumber(value: string | undefined): number | undefined {
|
|
if (value === undefined || value.trim() === '') {
|
|
return undefined;
|
|
}
|
|
const parsed = Number(value);
|
|
return Number.isFinite(parsed) ? parsed : undefined;
|
|
}
|
|
|
|
/** Parses `text` as CSV and maps each data row to an object keyed by the header row. */
|
|
export function parseCsvObjects(text: string, delimiter = ','): Array<Record<string, string>> {
|
|
const rows = parseCsv(text, delimiter).filter((row) => row.some((cell) => cell.length > 0));
|
|
if (rows.length === 0) {
|
|
return [];
|
|
}
|
|
|
|
const [header, ...dataRows] = rows;
|
|
return dataRows.map((row) => {
|
|
const record: Record<string, string> = {};
|
|
header.forEach((column, index) => {
|
|
record[column] = row[index] ?? '';
|
|
});
|
|
return record;
|
|
});
|
|
}
|