PyPSA technology-data legacy input data¶
The two hand-curated input CSVs used by PyPSA technology-data, which are a mix of many different data sources.
usa.csv holds USA-specific parameters, mostly compiled from the NREL ATB 2024 and ICCT IRA e-fuels assumptions.
other.csv holds parameters that originally mainly served PyPSA-Eur but have been collected without paying attention to the region or consistency with each other:
They are a list of over 100 technologies from ammonia, electricity, to hydrogen and materials like steel.
This dataset will in the future be superseded by a better curated one.
At a glance¶
| Package key | legacy_input_data (formerly manual_input_usa) |
| Publisher | PyPSA technology-data contributors |
| Original data | inputs/US/manual_input_usa.csv, inputs/manual_input.csv (no archived copies) |
| Underlying sources | cited per entry, multiple. |
| License | CC-BY-4.0 |
| Region | World / other and USA |
| Years | USA: 2020–2050; other: 2002–2050, and 0 for values without a specific year |
| Cases | NREL ATB scenario and financial cases, not_available for the remaining ones |
| Currency | USD_2019, USD_2022, USD_2023; EUR_2004 to EUR_2023 |
| Raw format | two manually curated CSV files |
| Parser | LegacyInputDataParser |
Available versions¶
| Version key | Upstream release | Raw files | Parse options | Status |
|---|---|---|---|---|
v0.13.4 |
PyPSA technology-data v0.13.4 | usa.csv, other.csv |
num_digits=3 |
latest |
Accessing the data¶
>>> from technologydata import DataAccessor
>>> data = DataAccessor(data_source="legacy_input_data", version="v0.13.4").load()
>>> tech = data.technologies.get(name="Fischer-Tropsch", region="USA", year=2020)[0]
>>> print(tech.parameters["hydrogen-input"])
1.43 dimensionless, carrier=hydrogen / fischer_tropsch, heating_value=lower_heating_value
See the tutorial for filtering, unit and currency conversion.
Contents¶
Technologies¶
>>> import pandas as pd
>>> techs = DataAccessor(data_source="legacy_input_data", version="v0.13.4").load().technologies
>>> def join(values):
... return ", ".join(sorted({str(v) for v in values} - {"None"}))
>>> print(
... techs.to_dataframe()
... .groupby(["name", "detailed_technology"], as_index=False)
... .agg(cases=("case", join), years=("year", join))
... .to_markdown(index=False)
... )
| name | detailed_technology | cases | years |
|:------------------------------------------------------|:------------------------------------------------------|:---------------------------------------------------------------------------|:-----------------------------|
| Alkaline electrolyzer large size | Alkaline electrolyzer large size | Advanced - Market, Conservative - Market, Moderate - Market, not_available | 2020, 2025, 2030, 2040, 2050 |
| Alkaline electrolyzer medium size | Alkaline electrolyzer medium size | not_available | 2020, 2025, 2030 |
| Alkaline electrolyzer small size | Alkaline electrolyzer small size | not_available | 2020, 2025, 2030 |
| Ammonia cracker | Ammonia cracker | not_available | 0, 2030, 2050 |
| CH4 (g) fill compressor station | CH4 (g) fill compressor station | not_available | 2040 |
| CH4 (g) pipeline | CH4 (g) pipeline | not_available | 2020, 2050 |
| CH4 (g) submarine pipeline | CH4 (g) submarine pipeline | not_available | 2002, 2015, 2020 |
| CH4 (l) transport ship | CH4 (l) transport ship | not_available | 2030, 2040 |
| CH4 evaporation | CH4 evaporation | not_available | 2030, 2040 |
| CH4 liquefaction | CH4 liquefaction | not_available | 0, 2030, 2040 |
| CO2 liquefaction | CO2 liquefaction | not_available | 0, 2004 |
| CO2 pipeline | CO2 pipeline | not_available | 2020 |
| CO2 storage tank | CO2 storage tank | not_available | 2050 |
| CO2 submarine pipeline | CO2 submarine pipeline | not_available | 2020 |
| Coal integrated retrofit 90%-CCS | Coal integrated retrofit 90%-CCS | not_available | 2030 |
| Coal integrated retrofit 95%-CCS | Coal integrated retrofit 95%-CCS | not_available | 2030 |
| Coal-95%-CCS | Coal-95%-CCS | not_available | 2030 |
| Coal-99%-CCS | Coal-99%-CCS | not_available | 2030 |
| Coal-IGCC | Coal-IGCC | not_available | 2020 |
| Coal-IGCC-90%-CCS | Coal-IGCC-90%-CCS | not_available | 2030 |
| FT fuel transport ship | FT fuel transport ship | not_available | 2035 |
| Fischer-Tropsch | Fischer-Tropsch | not_available | 2020, 2030, 2040, 2050 |
| General liquid hydrocarbon storage (crude) | General liquid hydrocarbon storage (crude) | not_available | 2012 |
| General liquid hydrocarbon storage (product) | General liquid hydrocarbon storage (product) | not_available | 2012 |
| H2 (g) fill compressor station | H2 (g) fill compressor station | not_available | 2020 |
| H2 (g) pipeline | H2 (g) pipeline | not_available | 2020, 2030, 2050 |
| H2 (g) pipeline repurposed | H2 (g) pipeline repurposed | not_available | 2020, 2030, 2050 |
| H2 (g) submarine pipeline | H2 (g) submarine pipeline | not_available | 2015, 2020, 2030, 2050 |
| H2 (g) submarine pipeline repurposed | H2 (g) submarine pipeline repurposed | not_available | 2015, 2020, 2030, 2050 |
| H2 (l) storage tank | H2 (l) storage tank | not_available | 2015 |
| H2 (l) transport ship | H2 (l) transport ship | not_available | 2030 |
| H2 evaporation | H2 evaporation | not_available | 2030, 2050 |
| H2 liquefaction | H2 liquefaction | not_available | 0, 2030, 2050 |
| H2 production biomass gasification | H2 production biomass gasification | not_available | 2020 |
| H2 production biomass gasification CC | H2 production biomass gasification CC | not_available | 2020 |
| H2 production coal gasification | H2 production coal gasification | not_available | 2020, 2025, 2030, 2050 |
| H2 production coal gasification CC | H2 production coal gasification CC | not_available | 2020, 2025, 2030, 2050 |
| H2 production heavy oil partial oxidation | H2 production heavy oil partial oxidation | not_available | 2020 |
| H2 production natural gas steam reforming | H2 production natural gas steam reforming | not_available | 2020, 2025, 2030 |
| H2 production natural gas steam reforming CC | H2 production natural gas steam reforming CC | not_available | 2020, 2025, 2030 |
| H2 production solid biomass steam reforming | H2 production solid biomass steam reforming | not_available | 2020 |
| HVAC overhead | HVAC overhead | not_available | 2030 |
| HVDC inverter pair | HVDC inverter pair | not_available | 2030 |
| HVDC overhead | HVDC overhead | not_available | 2030 |
| HVDC submarine | HVDC submarine | not_available | 2030 |
| HVDC underground | HVDC underground | not_available | 2030 |
| Haber-Bosch | Haber-Bosch | not_available | 0 |
| LNG storage tank | LNG storage tank | not_available | 2019 |
| LOHC chemical | LOHC chemical | not_available | 2035 |
| LOHC dehydrogenation | LOHC dehydrogenation | not_available | 2015 |
| LOHC dehydrogenation (small scale) | LOHC dehydrogenation (small scale) | not_available | 2035 |
| LOHC hydrogenation | LOHC hydrogenation | not_available | 0, 2015 |
| LOHC loaded DBT storage | LOHC loaded DBT storage | not_available | 2012 |
| LOHC transport ship | LOHC transport ship | not_available | 2035 |
| LOHC unloaded DBT storage | LOHC unloaded DBT storage | not_available | 2012 |
| MeOH transport ship | MeOH transport ship | not_available | 2035 |
| Methanol steam reforming | Methanol steam reforming | not_available | 0, 2020 |
| NG 2-on-1 Combined Cycle (F-Frame) | NG 2-on-1 Combined Cycle (F-Frame) | not_available | 2020, 2030 |
| NG 2-on-1 Combined Cycle (F-Frame) 95% CCS | NG 2-on-1 Combined Cycle (F-Frame) 95% CCS | not_available | 2030 |
| NG 2-on-1 Combined Cycle (F-Frame) 97% CCS | NG 2-on-1 Combined Cycle (F-Frame) 97% CCS | not_available | 2030 |
| NG Combined Cycle F-Class integrated retrofit 90%-CCS | NG Combined Cycle F-Class integrated retrofit 90%-CCS | not_available | 2030 |
| NG Combined Cycle F-Class integrated retrofit 95%-CCS | NG Combined Cycle F-Class integrated retrofit 95%-CCS | not_available | 2030 |
| NH3 (l) storage tank incl. liquefaction | NH3 (l) storage tank incl. liquefaction | not_available | 2020 |
| NH3 (l) transport ship | NH3 (l) transport ship | not_available | 2030 |
| PEM electrolyzer small size | PEM electrolyzer small size | Advanced - Market, Conservative - Market, Moderate - Market, not_available | 2020, 2030, 2040, 2050 |
| SOEC | SOEC | Advanced - Market, Conservative - Market, Moderate - Market, not_available | 2020, 2030, 2040, 2050 |
| Steam methane reforming | Steam methane reforming | not_available | 0, 2015 |
| air separation unit | air separation unit | not_available | 0 |
| allam | allam | not_available | 2030 |
| ammonia carbon capture retrofit | ammonia carbon capture retrofit | not_available | 2030 |
| battery inverter | battery inverter | Advanced - Market, Conservative - Market, Moderate - Market, not_available | 2022, 2030, 2040, 2050 |
| battery storage | battery storage | Advanced - Market, Conservative - Market, Moderate - Market, not_available | 2022, 2030, 2040, 2050 |
| biodiesel crops | biodiesel crops | not_available | 2020, 2030, 2040, 2050 |
| bioethanol crops | bioethanol crops | not_available | 2020, 2030, 2040, 2050 |
| biogas manure | biogas manure | not_available | 2020, 2030, 2040, 2050 |
| blast furnace-basic oxygen furnace | blast furnace-basic oxygen furnace | not_available | 2020 |
| cement carbon capture retrofit | cement carbon capture retrofit | not_available | 2030 |
| cement dry clinker | cement dry clinker | not_available | 2006 |
| cement finishing | cement finishing | not_available | 2006 |
| coal | coal | not_available | 2020 |
| csp-tower | csp-tower | not_available | 2020, 2030, 2040, 2050 |
| csp-tower TES | csp-tower TES | not_available | 2020, 2030, 2040, 2050 |
| csp-tower power block | csp-tower power block | not_available | 2020, 2030, 2040, 2050 |
| direct air capture | direct air capture | Advanced - Market, Conservative - Market, Moderate - Market, not_available | 0, 2020 |
| dry bulk carrier Capesize | dry bulk carrier Capesize | not_available | 2020 |
| electric arc furnace | electric arc furnace | not_available | 2020 |
| electric arc furnace with hbi and scrap | electric arc furnace with hbi and scrap | not_available | 2020 |
| electric steam cracker | electric steam cracker | not_available | 2015 |
| electrolysis | electrolysis | not_available | 2020, 2025, 2030, 2040, 2050 |
| ethanol carbon capture retrofit | ethanol carbon capture retrofit | not_available | 2030 |
| ethanol from starch crop | ethanol from starch crop | not_available | 2010, 2020, 2025, 2030 |
| ethanol from sugar crops | ethanol from sugar crops | not_available | 2010, 2020, 2025, 2030 |
| fuelwood | fuelwood | not_available | 2020, 2030, 2040, 2050 |
| gas | gas | not_available | 2020 |
| geothermal | geothermal | not_available | 2020 |
| hydrogen direct iron reduction furnace | hydrogen direct iron reduction furnace | not_available | 2020 |
| hydrogen storage compressor | hydrogen storage compressor | not_available | 2020, 2030 |
| hydrogen storage tank type 1 | hydrogen storage tank type 1 | not_available | 2020, 2030 |
| iron ore DRI-ready | iron ore DRI-ready | not_available | 2020 |
| iron-air battery | iron-air battery | not_available | 2025, 2030, 2035, 2040 |
| iron-air battery charge | iron-air battery charge | not_available | 2025, 2030, 2035, 2040 |
| iron-air battery discharge | iron-air battery discharge | not_available | 2025, 2030, 2035, 2040 |
| lignite | lignite | not_available | 2020 |
| methanation | methanation | not_available | 0, 2020, 2030, 2050 |
| methane storage tank incl. compressor | methane storage tank incl. compressor | not_available | 2014 |
| methanol-to-kerosene | methanol-to-kerosene | not_available | 2020, 2030, 2050 |
| methanol-to-olefins/aromatics | methanol-to-olefins/aromatics | not_available | 2015 |
| methanolisation | methanolisation | not_available | 0, 2020, 2030, 2050 |
| natural gas direct iron reduction furnace | natural gas direct iron reduction furnace | not_available | 2020 |
| nuclear | nuclear | not_available | 2020 |
| offwind-float | offwind-float | not_available | 2020, 2030, 2040, 2050 |
| offwind-float-connection-submarine | offwind-float-connection-submarine | not_available | 2030 |
| offwind-float-connection-underground | offwind-float-connection-underground | not_available | 2030 |
| offwind-float-station | offwind-float-station | not_available | 2030 |
| organic rankine cycle | organic rankine cycle | not_available | 2020 |
| seawater RO desalination | seawater RO desalination | not_available | 0 |
| shipping fuel methanol | shipping fuel methanol | not_available | 2020 |
| steel carbon capture retrofit | steel carbon capture retrofit | not_available | 2030 |
| uranium | uranium | not_available | 2020 |
Parameters¶
entries counts the technology entries (one per name, case and year) that carry the parameter.
>>> params = pd.DataFrame(
... [(key, str(p.units), str(p.carrier)) for t in techs for key, p in t.parameters.items()],
... columns=["parameter", "units", "carriers"],
... )
>>> print(
... params.groupby("parameter", as_index=False)
... .agg(units=("units", join), carriers=("carriers", join), entries=("units", "size"))
... .to_markdown(index=False)
... )
| parameter | units | carriers | entries |
|:------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------:|
| CO2 intensity | metric_ton / megawatt_hour | carbon_dioxide / electricity, carbon_dioxide / thermal | 3 |
| FOM | percent / year | | 190 |
| VOM | EUR_2010 / kilowatt, EUR_2010 / megawatt_hour, EUR_2015 / megawatt_hour, EUR_2015 / metric_ton, EUR_2020 / megawatt_hour, EUR_2023 / megawatt_hour | 1 / cement, 1 / clinker, 1 / electricity, 1 / ethanol, 1 / high_value_chemicals, 1 / hydrogen, 1 / kerosene | 32 |
| ammonia-input | dimensionless | ammonia / hydrogen | 1 |
| capacity | metric_ton | ammonia, dehydrogenated_dibenzyltoluene, fischer_tropsch, hydrogen, methane, methanol | 8 |
| capture_rate | percent | | 13 |
| carbondioxide-input | dimensionless, metric_ton / megawatt_hour | carbon_dioxide / fischer_tropsch, carbon_dioxide / methane, carbon_dioxide / methanol, dimensionless | 8 |
| carbondioxide-output | dimensionless | carbon_dioxide / high_value_chemicals | 2 |
| clinker-input | dimensionless | clinker / cement | 1 |
| coal-input | dimensionless, megawatt_hour / metric_ton | coal / hydrogen, coal / steel | 7 |
| commodity | EUR_2020 / metric_ton | | 1 |
| compression-electricity-input | dimensionless | electricity / hydrogen | 2 |
| district heat surcharge | percent | | 1 |
| district heat-input | dimensionless | dimensionless | 1 |
| economic_lifetime | year | | 5 |
| efficiency | percent | | 28 |
| electricity-input | 1 / meter, dimensionless, megawatt_hour / metric_ton | 1 / carbon_dioxide, electricity / ammonia, electricity / carbon_dioxide, electricity / cement, electricity / clinker, electricity / fischer_tropsch, electricity / high_value_chemicals, electricity / hot_briquetted_iron, electricity / hydrogen, electricity / hydrogenated_dibenzyltoluene, electricity / methane, electricity / methanol, electricity / nitrogen, electricity / steel, electricity / thermal, electricity / water | 68 |
| fuel | EUR_2010 / megawatt_hour, EUR_2020 / megawatt_hour | 1 / thermal | 22 |
| gas-input | dimensionless, megawatt_hour / metric_ton | 1 / carbon_dioxide, natural_gas / clinker, natural_gas / hot_briquetted_iron, natural_gas / hydrogen | 10 |
| hbi-input | dimensionless | hot_briquetted_iron / steel | 2 |
| heat-input | megawatt_hour / metric_ton | thermal / carbon_dioxide | 4 |
| heat-output | dimensionless | thermal / methanol | 1 |
| hydrogen-input | dimensionless, megawatt_hour / metric_ton | dimensionless, hydrogen / ammonia, hydrogen / fischer_tropsch, hydrogen / hot_briquetted_iron, hydrogen / hydrogenated_dibenzyltoluene, hydrogen / kerosene, hydrogen / methane, hydrogen / methanol | 12 |
| investment | EUR_2004 * hour / metric_ton, EUR_2005 / kilowatt, EUR_2010 / kilowatt, EUR_2010 / megawatt_hour, EUR_2011 / kilometer / megawatt, EUR_2011 / megawatt, EUR_2012 / meter ** 3, EUR_2012 / metric_ton, EUR_2013 / metric_ton, EUR_2014 / kilometer / megawatt, EUR_2014 / meter ** 3, EUR_2015, EUR_2015 * hour / kilometer / metric_ton, EUR_2015 * hour / metric_ton, EUR_2015 / kilowatt, EUR_2015 / megawatt, EUR_2015 / megawatt_hour, EUR_2017 / kilometer / megawatt, EUR_2017 / kilowatt, EUR_2017 / megawatt, EUR_2019, EUR_2019 / meter ** 3, EUR_2020, EUR_2020 * hour / metric_ton, EUR_2020 / kilometer / megawatt, EUR_2020 / kilowatt, EUR_2020 / kilowatt_hour, EUR_2020 / megawatt, EUR_2020 / metric_ton, EUR_2022 / kilowatt, EUR_2023 / kilometer / megawatt, EUR_2023 / kilowatt, EUR_2023 / kilowatt_hour, USD_2019 * hour / metric_ton, USD_2022 / kilowatt, USD_2022 / kilowatt_hour, USD_2022 / megawatt, USD_2023 * hour / metric_ton | 1 / ammonia, 1 / carbon_dioxide, 1 / cement, 1 / clinker, 1 / electricity, 1 / ethanol, 1 / fischer_tropsch, 1 / high_value_chemicals, 1 / hot_briquetted_iron, 1 / hydrogen, 1 / kerosene, 1 / methane, 1 / methanol, 1 / steel, 1 / thermal | 213 |
| lifetime | year | | 129 |
| lohc-input | dimensionless | dehydrogenated_dibenzyltoluene / hydrogenated_dibenzyltoluene | 1 |
| methane-input | dimensionless | dimensionless, methane / hydrogen | 2 |
| methanol-input | dimensionless, megawatt_hour / metric_ton | methanol / high_value_chemicals, methanol / hydrogen, methanol / kerosene | 3 |
| min_fill_level | percent | | 2 |
| naphtha-input | megawatt_hour / metric_ton | naphtha / high_value_chemicals | 1 |
| nitrogen-input | metric_ton / megawatt_hour | nitrogen / ammonia | 1 |
| oil-input | dimensionless | oil / hydrogen | 1 |
| ore-input | dimensionless | ore / hot_briquetted_iron, ore / steel | 3 |
| scrap-input | dimensionless | scrap / steel | 2 |
| slag-input | dimensionless | slag / cement | 1 |
| wood-input | dimensionless | wood / hydrogen | 3 |
Field mapping¶
| Raw column | Example raw value | Schema field | Example parsed value |
|---|---|---|---|
technology |
Fischer-Tropsch |
Technology.name, Technology.detailed_technology |
Fischer-Tropsch |
year |
2020 |
Technology.year |
2020 |
scenario, financial_case (usa.csv only) |
Moderate, Market |
Technology.case |
Moderate - Market |
| — (input file) | usa.csv, other.csv |
Technology.region |
USA, other |
parameter |
hydrogen-input |
parameter key | hydrogen-input |
value |
1.43 |
Parameter.magnitude |
1.43 |
unit, currency_year |
MWh_H2/MWh_FT, empty |
Parameter.units, .carrier, .heating_value |
dimensionless, hydrogen / fischer_tropsch, lower_heating_value |
further_description |
0.995 MWh_H2 per output, … |
Parameter.note |
same text |
source |
NREL, 2024 ATB Excel Workbook, … |
not kept |
Naming conventions¶
- Technology names and parameter keys are kept verbatim from PyPSA technology-data (
FOM,investment,hydrogen-input, …), so they match PyPSA naming. - Cases:
"{scenario} - {financial_case}", orscenarioalone; rows without a scenario getnot_available. - Units: malformed units in
other.csvare corrected (tCO2→t_CO2,MWhth→MWh_th,kWel→kW_el,,dpdropped);p.u.andper unitare converted topercent(value × 100);1000kmis converted tom(value ÷ 10⁶);currency_yearis folded into the currency (USD+2022→USD_2022). - Capacity per hour:
EUR/t_CO2/handEUR/(t_CO2/h)are both read as cost per capacity in t/h,EUR * hour / metric_ton. - Compound units such as
MWh_H2/MWh_FTare split into unit, carrier and heating value by a fixed mapping of twelve patterns inLegacyInputDataV0134Parser._extract_units_carriers_heating_value; carrier and heating value names are then expanded by the package registries (H2→hydrogen,LHV→lower_heating_value). - Carriers follow the unit: a carrier in the denominator is inverted (
EUR/t_clinker→1 / clinker); a ratio of the same carrier becomesdimensionless(MWh_H2/MWh_H2). - Carrier aliases of the raw files are renamed to the package names:
t_HLOHC→ hydrogenated dibenzyltoluene,t_LOHC→ dehydrogenated dibenzyltoluene,t_hbi→ hot briquetted iron,t_cl→ clinker.
Assumptions and deviations from the source¶
- Heating value: all fuel energy units are assumed to be on a lower heating value basis; the raw files do not state it.
The heating value follows the fuel energy unit:
lower_heating_valuein the numerator,1 / lower_heating_valuein the denominator (EUR/MWh_H2),lower_heating_valuefor a ratio of two fuels (MWh_H2/MWh_FT). Electricity, heat and units without an energy carrier (MWh/t_CO2) get no heating value. - Financial case:
R&DandMarketrows of the same scenario, technology and year inusa.csvare merged, and the case is labelled- Market. In v0.13.4 both financial cases carry identical values, so no numbers are lost, only theR&Dlabel. - Region: entries from
usa.csvare labelledUSA, entries fromother.csvother; upstream usesother.csvfor all regions other than the USA. - Rounding: values are rounded to three decimals after the unit conversions, keeping at least three significant digits for small values (
1.9e-08 / meter).
Reproduce¶
Run from the root of a repository checkout. The parser overwrites the files shipped with the package.
from technologydata import DataAccessor
DataAccessor(data_source="legacy_input_data", version="v0.13.4").parse(
input_file_names=["usa.csv", "other.csv"],
num_digits=3,
)
With archive_source=False (the default) the existing sources.json is reused; archive_source=True archives the sources on the Wayback Machine and rewrites sources.json.
Known limitations¶
- Inconsistent parameter naming in the data source has been kept and not parsed. E.g. parameters like
investmentlikely refer tospecific investment. - Year
0: 26 parameters ofother.csvhave no year in the raw file and are stored with year0in 13 entries (e.g.Haber-Bosch,H2 liquefaction), separate from the entries of the same technology with a year. - Overwritten values: 3 rows of
other.csvshare technology, year and parameter with another row (electricity-inputofAlkaline electrolyzer medium sizeandsmall size,investmentofAlkaline electrolyzer small size); only the last one is kept.
Citation¶
PyPSA technology-data contributors: technology-data v0.13.4,
inputs/US/manual_input_usa.csvandinputs/manual_input.csv, accessed 2025-10-20. https://github.com/PyPSA/technology-data/tree/v0.13.4
License: CC-BY-4.0; the underlying sources may have their own terms.
Please also cite the underlying sources listed above and technologydata, see Citing.
See also¶
Parser API¶
The parser is only needed to reproduce or update the dataset, see Reproduce; loading the data only needs DataAccessor.load().
The dispatcher selects the parser of the requested version.
LegacyInputDataParser()
Main parser for the technology_data raw/legacy_input_data/other.csv and raw/legacy_input_data/usa.csv dataset.
Dispatches to version-specific parser implementations.
Methods:
-
get_supported_versions–Return a list of supported dataset versions.
-
parse–Parse the specified version of the technology_data raw/legacy_input_data/other.csv and raw/legacy_input_data/usa.csv dataset.
get_supported_versions() -> list[str]
Return a list of supported dataset versions.
parse(version: str, input_path: Path | list[Path], num_digits: int, archive_source: bool, filter_params: bool, export_schema: bool) -> None
Parse the specified version of the technology_data raw/legacy_input_data/other.csv and raw/legacy_input_data/usa.csv dataset.
This method selects the appropriate parser for the given version and delegates the parsing task to it.
Parameters:
-
version(str) –The version of the dataset to parse (e.g., 'v0.13.4').
-
input_path(Path | list[Path]) –Paths to the raw input data CSV files.
-
num_digits(int) –Number of decimals to round numerical values to; values < 1 keep at least num_digits significant digits.
-
archive_source(bool) –If True, archives the source objects on the Wayback Machine and rewrites sources.json
-
filter_params(bool) –Ignored by this parser.
-
export_schema(bool) –If True, exports the Pydantic schema for the data models.
Raises:
-
ValueError–If the specified version is not supported.
Bases: ParserBase
flowchart TD
technologydata.parsers.legacy_input_data.parser_v0134.LegacyInputDataV0134Parser[LegacyInputDataV0134Parser]
technologydata.parsers.data_parser_base.ParserBase[ParserBase]
technologydata.parsers.data_parser_base.ParserBase --> technologydata.parsers.legacy_input_data.parser_v0134.LegacyInputDataV0134Parser
click technologydata.parsers.legacy_input_data.parser_v0134.LegacyInputDataV0134Parser href "" "technologydata.parsers.legacy_input_data.parser_v0134.LegacyInputDataV0134Parser"
click technologydata.parsers.data_parser_base.ParserBase href "" "technologydata.parsers.data_parser_base.ParserBase"
Parser for v0.13.4 of the raw/legacy_input_data/other.csv and raw/legacy_input_data/usa.csv datasets.
Methods:
-
parse–Parse and process version 0.13.4 of the raw/legacy_input_data/usa.csv and raw/legacy_input_data/other.csv datasets.
parse(input_path: Path | list[Path], num_digits: int, archive_source: bool, **kwargs: Any) -> None
Parse and process version 0.13.4 of the raw/legacy_input_data/usa.csv and raw/legacy_input_data/other.csv datasets.
This method reads the raw data from both CSV files (usa.csv and other.csv), cleans and transforms it through a series of steps, and then builds a TechnologyCollection. The processed data is saved to JSON files.
Data processing steps include: - USA data is tagged with region='USA', other data with region='other' - Unit normalization: tCO2 -> t_CO2, MWHh_el -> MWh_el, MWhth -> MWh_th, kWel -> kW_el, MWh_thdh -> MWh_th, t_cl -> t_clinker, t_HLOHC -> t_H18DBT, t_LOHC -> t_H0DBT, t_hbi -> t_HBI - Removal of design point suffix: ,dp removed from units - Distance normalization: 1000km -> m (with value divided by 1e6) - Percentage conversion: 'per unit' -> '%' (with value multiplied by 100) - Rounding of all values to num_digits decimals, keeping at least num_digits significant digits - Currency year integration: currency_year column merged into unit string - Unit/carrier/heating value extraction using regex pattern matching
Parameters:
-
input_path(Path | list[Path]) –List of paths to the raw input data files. Must contain two CSV files: one with 'usa' or 'us' in the filename, and one 'other' file.
-
num_digits(int) –Number of decimals to round numerical values to; values below 1 keep at least num_digits significant digits.
-
archive_source(bool) –If True, archives the source object on the Wayback Machine.
-
**kwargs(bool, default:{}) –export_schema : bool If True, exports the Pydantic schema for the data models. filter_params : bool Ignored by this parser.
Raises:
-
TypeError–If input_path is a single Path instead of a list.
-
ValueError–If the USA file or other file cannot be identified from the filenames.
Returns:
-
Nothing–
Notes
Writes parsed files to src/technologydata/parsers/legacy_input_data/v0.13.4/.