1.2 The Data Processing and Analysis Consortium

1.2.1 Introduction

Author(s): Anthony Brown

The transformation of the raw Gaia data into astrophysically meaningful quantities has been entrusted to the Gaia Data Processing and Analysis Consortium (DPAC). DPAC is funded by national (space) agencies. More details on DPAC are provided in Gaia Collaboration et al. (2016b).

1.2.2 History

Author(s): Anthony Brown

DPAC started its activities in 2006. The history of DPAC is described in Gaia Collaboration et al. (2016b).

1.2.3 Objectives and responsibilities

Author(s): Anthony Brown

The objectives and responsibilities of the DPAC are addressed in Gaia Collaboration et al. (2016b).

1.2.4 Structure

Author(s): Anthony Brown

Coordination Units

DPAC is composed of nine autonomous units, called coordination units (CUs). Each CU is composed of several dozen members, spread around various (academic) institutes in various, mostly European countries (see https://www.cosmos.esa.int/web/gaia/dpac/institutes). The details of the various DPAC data processing systems are provided in Gaia Collaboration et al. (2016b). In Table 1.1, the DPAC Coordination Units are listed by number together with the data processing (sub-)systems that the CU in question is responsible for. Although this is not of direct interest to the reader of this documentation, the CUs are sometimes referred to in the text and this table provides the means to understand the context of such a reference.

Table 1.1: DPAC Coordination Units and the data processing (sub-)systems they are in charge of. The DPAC systems listed are the ones mentioned in Gaia Collaboration et al. (2016b).
CU No. Data processing (sub-)systems
CU1 System architecture, Common tools, Main DataBase, Catalogue integration
CU2 Simulations
CU3 Initial Data Treatment and First Look, Astrometric Verification Unit, Astrometric Global Iterative Solution, Global Sphere Reconstruction, Intermediate Data Update, Relativistic astrometric models, Auxiliary Observations
CU4 Solar-system alerts, Non-Single-Star treatment, Solar-System-Object treatment, Extended Object Analysis
CU5 Photometric pipeline, Source-Environment Analysis, Photometric science alerts
CU6 RVS daily, RVS pipeline
CU7 Variable star analysis
CU8 Astrophysical parameter inference
CU9 Data validation and publication

Data Processing Centres

The actual processing of the Gaia data takes place at six data-processing centres (DPCs) in Europe, located at ESAC (DPCE, Section 1.3.4), Barcelona (DPCB, Section 1.3.4), CNES Toulouse (DPCC, Section 1.3.4), IoA Cambridge (DPCI, Section 1.3.4), Geneva (DPCG, Section 1.3.4), and Turin (DPCT, Section 1.3.4). More details are provided in Gaia Collaboration et al. (2016b).

Executive

DPAC is managed by an executive body (DPACE) consisting of the scientific leaders of the coordination units plus representatives from the data-processing centres. See https://www.cosmos.esa.int/web/gaia/gaia-dpac-executive.

Project Office

The DPAC Executive is supported in its tasks by the DPAC project office (PO). See Gaia Collaboration et al. (2016b) and https://www.cosmos.esa.int/web/gaia/project-office for more details.

1.2.5 Overview of the data processing

Author(s): Anthony Brown

System architecture

The system architecture of Gaia is documented in O’Mullane et al. (2011).

Common tools

A common set of (Java) tools and data sources have been provided for DPAC data processing. These are in place both to ensure consistency of data processing, and to avoid multiple, independent developments of similar concepts. The most important common tools are:

Configuration DataBase (CDB)

this provides data related to the configuration of the spacecraft that could impact on data processing. Included are information such as the along-scan phasing offset of each CCD row and strip; the charge-injection configuration for each CCD; the CCD window geometry; gate information for each CCD; etc.

Gaia Parameter DataBase (GPDB)

this provides useful constants (for instance the value of π and the speed of light in a vacuum to a significant number of digits), as well as mission-specific information and nominal, pre-flight spacecraft and payload design parameters.

GaiaTools

this is the main common software library that provides utilities such as reading/writing data in accordance to the common data model (see Section 1.2.5) and related database querying facilities; plotting tools; mathematical tools (vector/array manipulation and quaternion operations); fitting algorithms; and so on.

There is also a common acronym list, available on https://gaia.esac.esa.int/gpdb/glossary.txt.

Main database and data model

The data-processing centres communicate through the main database (MDB) located at DPCE/ESAC following a hub-and-spokes topology. The MDB also houses the DPAC end data products from which the public data releases are produced. For more details, see Gaia Collaboration et al. (2016b).

Data flow

The data flow in the Gaia science ground segment is described in Hernandez and Hutton (2015).

Daily processing

The daily processing of Gaia data includes the Initial Data Treatment (IDT) and First Look ]DPACP-7, the Astrometric Verification Unit (AVU) processing, the RVS daily processing, the (Photometric) Science Alerts ([P]SAs), and the solar-system science alerts. More details are provided in Gaia Collaboration et al. (2016b).

Cyclic processing

The cyclic processing, encompassing the Intermediate Data Update (IDU), the Astrometric Global Iterative Solution (AGIS), the Global Sphere Reconstruction (AVU-GSR), the Photometric Pipeline (PhotPipe), and the RVS pipeline, is detailed in Gaia Collaboration et al. (2016b).

Astrometric and photometric pre-processing

The astrometric and photometric pre-processing is summarised in Gaia Collaboration et al. (2016b) and detailed in Fabricius et al. (2016). Further documentation is provided in Section 2.

Astrometry

The astrometric processing and validation is summarised in Gaia Collaboration et al. (2016b) and detailed in Lindegren et al. (2016). Further documentation is provided in Section 3 and Section 4.

Photometry

The photometric processing and validation is summarised in Gaia Collaboration et al. (2016b) and detailed in Carrasco et al. (2016); Evans et al. (2017); van Leeuwen et al. (2016). Further documentation is provided in Section 5.

Spectroscopy

Gaia DR1 does not contain spectroscopic results. The spectroscopic processing is summarised in Gaia Collaboration et al. (2016b).

Non-single stars and exoplanets

Gaia DR1 does not cover non-single stars and/or exoplanets. The processing of these objects is summarised in Gaia Collaboration et al. (2016b).

Extended objects

Gaia DR1 does not cover extended objects. The processing of these objects is summarised in Gaia Collaboration et al. (2016b).

Solar-system objects

Gaia DR1 does not cover Solar-system objects. The processing of these objects is summarised in Gaia Collaboration et al. (2016b).

Variability

The variability processing and validation is summarised in Gaia Collaboration et al. (2016b) and detailed in Clementini et al. (2016); Eyer et al. (2017). Further documentation is provided in Section 6.

Astrophysical parameters

Gaia DR1 does not cover astrophysical parameters. The associated processing is summarised in Gaia Collaboration et al. (2016b).

1.2.6 Simulations

Author(s): Anthony Brown

Besides the processing tasks mentioned earlier, DPAC responsibilities also include preparatory simulations, spanning the three levels from pixels in the focal plane (GIBIS; Babusiaux et al. 2011), through simulated telemetry (GASS; Masana et al. 2008), to simulated DPAC data products (GOG; Antiche et al. 2014) which have been used both internally and to support the astronomical community in preparing itself for the Gaia mission (Robin et al. 2012; Luri et al. 2014). For more details, see Gaia Collaboration et al. (2016b).