European Commission logo
Joint Research Center (JRC)
NEMESIS

New Econometric Model of Evaluation by Sectoral Interdependency and Supply

EconomyEnergyEnvironmentClimatemacro-sectoralforecastsquantitative forward-looking scenarioseconomic policiesstructural policies

Quality & Transparency

Quality

Model uncertainties

Models are by definition affected by uncertainties (in input data, input parameters, scenario definitions, etc.). Have the model uncertainties been quantified? Are uncertainties accounted for in your simulations?

yes
Yes in the limit of the possibilities.Uncertainties in econometric models come from data, parametrisation and exogenous variables, some scientific works tried to make a general assessment. For NEMESIS the quantification of all the uncertainties is impossible, for instance, the uncertainties on the data are not available. However, there is an effort to identify uncertain equations, e.g. wage curve, or the more important for the assessment conducted, e.g. performance of research, or leverage effect of subsidies in case of Research Policies.

    Sensitivity analysis

    Sensitivity analysis helps identifying the uncertain inputs mostly responsible for the uncertainty in the model responses. Has the model undergone sensitivity analysis?

    yes
    Sensitivity analysis is performed on the wage curve, especially on the unemployment parameter; for Research Policies, sensitivity analysis has been used on the leverage effect and on the economic performance of research.

      Have model results been published in peer-reviewed articles?

      yes
      First publication in Peer reviewed journal in Research Policy in 2006

        Has the model formally undergone scientific review by a panel of international experts?

        Please note that this does not refer to the cases when model results were validated by stakeholders.

        no

          Model validation

          Has model validation been done? Have model predictions been confronted with observed data (ex-post)?

          yes
          The recent turmoiled period with two major economic crises has affected forecasts. For instance, the 3% was not reached and all the medium-long term scenarios forecasted became obsolete.

            Transparency

            To what extent do input data come from publicly available sources?

            This may include sources accessible upon subscription and/or payment

            Entirely based on publicly available sources

            Is the full model database as such available to external users?

            Whether or not it implies a specific procedure or a fee

            no
            Public databases (EUROSTAT, EU-KLEMS, WIOD, INTAN-invest, OECD, DG TAXUD).

              Have model results been presented in publicly available reports?

              Note this excludes IA reports.

              yes

              Have output datasets been made publicly available?

              Note this could also imply a specific procedure or a fee.

              no
              Not for all the exercises. Some Clients specify the privacy character of results.

                Is there any user friendly interface presenting model results that is accessible to the public?

                For instance: Dashboard, interactive interfaces...

                no

                  Has the model been documented in a publicly available dedicated report or a manual?

                  Note this excludes IA reports.

                  yes

                  Is there a dedicated public website where information about the model is provided?

                  no

                    Is the model code open-source?

                    no

                    Can the code be accessed upon request?

                    no