Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionNext revisionBoth sides next revision | ||
filereference:output:start [2015/12/09 22:45] – [Level 3: Metatadata information in EDX file] enviadmin | filereference:output:start [2021/11/14 18:26] – [Text Files: Overview and description] enviadmin | ||
---|---|---|---|
Line 4: | Line 4: | ||
Each ENVI-met simulation creates a huge amount of data which is organized in different files and folders. | Each ENVI-met simulation creates a huge amount of data which is organized in different files and folders. | ||
In general, there are two different types of output files regardless their content: | In general, there are two different types of output files regardless their content: | ||
- | * **Simple Text Files**: Files which contain data in pure ASCII text format. | + | * **Simple Text Files**: Files which contain data in pure ASCII text format |
- | * **Binary Files (EDX/ | + | |
+ | |||
+ | * **Binary Files (EDX/ | ||
+ | |||
+ | **In order to use advanced analysis functions like Time Series in LEONARDO, it is strongly recommended to store only one simulation run in the same folder and not to mix different model runs. While the files remain usable itself, the logic of the different applications might not be able to sort out the structure of your simulation data.** | ||
===== Name and storage logic ===== | ===== Name and storage logic ===== | ||
Line 11: | Line 15: | ||
* Level 1: Storage by folder name | * Level 1: Storage by folder name | ||
- | * Level 2: Classification by filename | + | * Level 2: Classification by filename |
* Level 3: Metadata information in the [[filereference: | * Level 3: Metadata information in the [[filereference: | ||
+ | |||
+ | For simple Text files, Level 3 data are not available, so that the filename and filename extension provide information about the content of the files. | ||
==== Level 1: Storage by folder name ==== | ==== Level 1: Storage by folder name ==== | ||
- | [{{ : | + | {{ : |
- | The uppermost level of file organisation starts by sorting the output files into subfolders of the selected output folders. The figure above shows the folder structure of a typical ENVI-met output folder (some folders are only existing in the Advanced version of ENVI-met). All files in one folder are of the same structure and contain the same set of information, | + | The uppermost level of file organisation starts by sorting the output files into subfolders of the selected output folders. The figure above shows an example of the folder structure of a typical ENVI-met output folder (some folders are only existing in the Advanced version of ENVI-met, others may be added in the course of time). All files in one folder are of the same structure and contain the same set of information, |
- | - **Atmosphere** | + | - **Atmosphere:** All information about the state of the atmosphere at different times of model simulation. Definitely the largest set of data and mostly considered as " |
- | - **Buildings** | + | - **Buildings:** Detailed data for the facades and roofs of the buildings. |
- | - **Inflow** | + | - **[[filereference: |
- | - **Log** | + | - **Log:** A copy of the ENVI-met outputs generated during the model run. |
- | - **Pollutants** | + | - **Pollutants:** Data about pollutant concentration in the model atmosphere (only present if pollutants are used in the model). |
- | - **Radiation** | + | - **Radiation:** Detailed data sets about the different radiation fluxes in the model. |
- | - **Receptors** | + | - **[[filereference: |
- | - **Soils** \\ | + | - **Soil:** |
- | - **SolarAccess** \\ | + | - **Solar Access:** |
- | - **Surface** | + | - **Surface:** |
+ | - **Vegetation: | ||
- | In addition, the BIOMet tool will create further folders, by default labled " | + | In addition, the BIOMet tool will create further folders, by default labled " |
Future versions or additional modules of ENVI-met will create further folders and file, but the general concept of storge will be the same. | Future versions or additional modules of ENVI-met will create further folders and file, but the general concept of storge will be the same. | ||
- | ==== Level 2: Classification by filename | + | ==== Level 2: Classification by file name or extension |
Sorting the output files into folders provides a first system of order in the output data. However, once a file has been moved out of its folder or if several simulations come together, this system is not unique and not persistent. As a solution, the ENVI-met file name generation scheme allows a direct identification of the simulation files and their content. | Sorting the output files into folders provides a first system of order in the output data. However, once a file has been moved out of its folder or if several simulations come together, this system is not unique and not persistent. As a solution, the ENVI-met file name generation scheme allows a direct identification of the simulation files and their content. | ||
- | [{{ : | + | {{ : |
Each ENVI-met output filename consists of 3 parts: | Each ENVI-met output filename consists of 3 parts: | ||
- **Simulation Base Name**: This is the given working name for the simulation. You define this name in the .SIMX simulation configuration file. It should be not too log (as more data are added) and unique. In the example above, | - **Simulation Base Name**: This is the given working name for the simulation. You define this name in the .SIMX simulation configuration file. It should be not too log (as more data are added) and unique. In the example above, | ||
- **Type identifier**: | - **Type identifier**: | ||
- | - **Time of data**: The last part of the filename represents the model time the data were taken. In the example above, the file represents the stae of the atmosphere ('' | + | - **Time of data**: The last part of the filename represents the model time the data were taken. This part of the filename is generated base on the " |
+ | |||
+ | ===Classification by file extension === | ||
+ | For pure text files, which might be processed in other programs or using the Python-based DataStudio, it is not possible to include metadata or other addtional data in the files except a header line defining the variables stored in the different colums of the file. | ||
+ | |||
+ | From ENVI-met Version 5 on, all text files are comma-seperated (CSV) files that can be processed directly in Python/ | ||
Line 50: | Line 64: | ||
- | ==== Sumary | + | ===== Binary Files: Overview and description ===== |
+ | |||
+ | The list below lists and links to the basic main files in binary format generated by the ENVI-met simulations. | ||
+ | |||
+ | <WRAP box left 99%> | ||
+ | ^ID ^Content | ||
+ | | AT | Atmospheric data | 3D Data Main state of the atmosphere with most of the variables in it | | ||
+ | | FX | Surface data and surface fluxes | ||
+ | | SO | Soil data | 3D Data about the state of the soil model, mainly temperature and soil water | [[filereference: | ||
+ | | VEG | Vegetation data | Plant data like Leaf Temperature in the context of the 3D Model, Detailed data for observed plants are additional text files of '' | ||
+ | | BLDG_static | Building data static | ||
+ | | BLDG | Building data dynamic | ||
+ | | POLU | Pollutants | ||
+ | | RD | Radiation Data | 3D Data with detailed information about shortwave and longwave radiation fluxes| | ||
+ | | BIO | BIO-met Data | Data generated by BIO-met | | ||
+ | | SA | Solar Access | ||
+ | | SAFAC | Solar Access | ||
+ | </ | ||
+ | |||
+ | <WRAP clear></ | ||
+ | |||
+ | |||
+ | |||
+ | ===== Text Files: Overview and description ===== | ||
+ | |||
+ | This list gives an overview over the different text output files generated by ENVI-met. All files are CSV text files that can be loaded directly in to Python (Pandas). Not all files may be generated in each simulation. This list refers to ENVI-met V5 and newer. | ||
+ | |||
+ | <WRAP box left 99%> | ||
+ | |||
+ | ^ID | ||
+ | | .AT_1DR | Receptor Atmospheric data (Profile) | Vertical profile atmosphere at receptor at a given model time | | ||
+ | | .AT_1DT | Receptor Atmospheric data (Time Series) | ||
+ | | .FX_1DT | Receptor Surface data (Time Series) | ||
+ | | .SO_1DR | Receptor Soil data (Profile) | ||
+ | | .SO_1DT | Receptor Soil data (Time Series) | Vertical profile soil at receptor for all model times of simulation in one file | | ||
+ | | .BLDG_status | Single Building Status (Time Series) | Actual status of individual buildings for all model times of simulation in one file (one for each building) | | ||
+ | | .BLDG_statistics | Building Statistics | Summary of static parameters for the buildings (one for each building) | | ||
+ | | .BLDG_list | Buildings Inventory | List of all buildings in model (all buildings in one file) | | ||
+ | | .VEG_status | Vegetation Status (Time Series) | ||
+ | | .VEG_list | Vegetation Inventory | ||
+ | | .Inflow1D | Profile Atmosphere Model at inflow boundary | ||
+ | </ | ||
+ | <WRAP clear></ | ||
+ | |||
+ | Please note, for all receptor files, you can produce output files with a higher resolution than the main output file intervall. All other files will be generated when the main model files are saved. | ||
+ | |||
+ | === Import text file into Python Pandas === | ||
+ | |||
+ | To import ENVI-met output text files with time series into Python Pandas, use: | ||
+ | |||
+ | <WRAP box left 99%> | ||
+ | <code Python> | ||
+ | import pandas as pd | ||
+ | |||
+ | # this imports the (ENVI-met) text file " | ||
+ | # Comma "," | ||
+ | # the " | ||
+ | # and also is set as the default index column for data processing | ||
+ | |||
+ | df = pd.read_csv(" | ||
+ | |||
+ | </ | ||
+ | </ | ||
+ | <WRAP clear></ | ||
+ | |||
+ | Each Time Series files now contains a Python-compatible // | ||
- | _AT_ : Atmospheric data file | ||
- | (to be cont.) | + | The LEONARDO DataStudio automatically recognizes the content of the ENVI-met output text files and offers specific scripts matching the file types. |