AuroraX now loads data automatically

September 22, 2026

The AuroraX platform has received a significant upgrade to its data pipeline: data loading is now fully automated! Until now, new ephemeris and summary data was loaded into AuroraX in periodic batches, which meant there was a lag of months between an auroral event and it becoming searchable. We have been able to improve the system to allow us to remove that lag, much like Swarm-Aurora has operated for the last few years. Data from our instrument networks now flows into AuroraX automatically every few hours, shortly after it arrives on our data archive. Satellite ephemeris data is uploaded at a similar cadence, and availability depends on the data in SSCWeb at the time.

Data in the platform can be explored using the Data Availability page, which is updated almost immediately after new data is ingested.

In practice this means:

• Conjunction Search - Search for conjunctions between ground-based instruments and spacecraft from as recently as last night. Ephemeris records for our all-sky imager networks are loaded automatically as the underlying data is processed, so recent events are available without waiting for a manual update.

• Keogramist and AuroraDome - Browse keograms and movies from the previous night as soon as they have been generated. Woke up to reports of a great display? Have a look at what our imagers saw before your first coffee is finished.

• Event Explorer - Recent nights are now available for exploration alongside the full historical record.

Programmatic users benefit as well. Searches performed through PyAuroraX and IDL-AuroraX run against the same continuously updated database, so scripts that look for recent conjunctions or pull summary data no longer need to account for a loading delay.

New metadata filter: raw_data_exists

Loading data this quickly changes one long-standing assumption. Until now, any record in AuroraX carried an implicit guarantee that the raw data behind it existed and was available for analysis. With automated loading, ephemeris records can now be created from summary data products before the raw data has been ingested. To make this explicit, every ephemeris record now carries a raw_data_exists metadata field. It is false for records derived from summary products, and the record is replaced with raw_data_exists set to true once the raw data is on disk.

You can use this field as a metadata filter in Conjunction Search and ephemeris searches, on the website and in PyAuroraX and IDL-AuroraX, at your discretion. If your analysis needs the full-resolution imagery, filter on raw_data_exists being true to restrict results to events where the data is ready to download. If you only need to know that an instrument was observing at the time, leave the filter off and take advantage of the most recent data.

Historical raw_data_exists values

We are currently re-uploading historical ephemeris data to the platform to fully enable this new field. We are deliberately going slowly so that normal operations of the platform are not impacted, which means older records may not yet carry a raw_data_exists value. We expect the data updates to be complete by the end of September 2026. Please assume that anything before 2026/05 has currently raw data available.