🌧️ RainData - Hydrological & Commodities Data Explorer

Web application for accessing and analyzing historical precipitation data, hydrological statistics, and commodity prices in Brazil.

🚀 Key Features

• 🗺️ Interactive Map:

  • Geospatial visualization of monitoring stations across Brazil using Folium.
  • State-level visualization of historical average prices for commodities (Soybean, Corn, Coffee, Sugarcane).
  • Intuitive navigation: click on a map point to view station details.

• 📊 Dataset Explorer:

  • Fast exploration of commodity price time-series datasets.
  • Interactive line charts for price evolution by State.

• 📈 Hydrological & SPI Analysis:

  • Standardized Precipitation Index (SPI-1): Detect drought and wet anomalies.
  • Commodities vs SPI: Dual-axis charts comparing a State's average SPI vs local commodity prices over time, normalized for correlation analysis.

• 🌍 Bilingual Support (i18n):

  • Fully translated interface supporting Portuguese (PT-BR) and English (EN).

• ⚡ High Performance:

  • Uses Parquet format for ultra-fast data loading.

• 📥 Export: Download charts (.png) and filtered datasets (.csv).

📡 Data Source

• Climate Data: Extracted from BDMEP (Banco de Dados Meteorológicos para Ensino e Pesquisa), provided by INMET (National Institute of Meteorology - Brazil).
• Commodities Data: Historical price series for Brazilian states (60kg bags / metric units).

🛠️ Tech Stack

• Language: Python 3.12
• Framework: Streamlit
• Data Processing: Pandas, NumPy
• Scientific Analysis: SciPy (Statistical distributions, GEV, Gamma)
• Visualization: Plotly Express, Plotly Graph Objects, Matplotlib, Folium

📂 Project Structure

raindata/
├── app.py                # Application entry point & Navigation
├── src/
│   ├── functions/        # Core logic (data cleaning, hydrology, statistics, charts)
│   └── utils/            # Utilities (i18n translations, wakeup script)
├── pages/
│   ├── home.py                   # Map and spatial overview
│   ├── explorer_page.py          # Commodities time-series explorer
│   └── data_analysis_page.py     # Hydrological statistics & SPI vs Commodities
├── data/
│   ├── rain/             # Parquet files for stations
│   └── commodities/      # Parquet files for agricultural commodities
└── requirements.txt      # Project dependencies

⚙️ Installation & Usage

1. Clone the repository:

   git clone https://github.com/your-username/raindata.git
   cd raindata

2. Create a virtual environment:

   python3 -m venv .venv
   source .venv/bin/activate  # Linux/Mac
   # or
   .venv\Scripts\activate     # Windows

3. Install dependencies:

   pip install -r requirements.txt

4. Prepare Data (ETL):

   • Place your raw .csv files from BDMEP in the rain_datasets folder.
   • Run the convert.ipynb notebook to generate metadata_estacoes.parquet and convert data to Parquet.

5. Run the App:

   streamlit run app.py

🎨 Theme

The application uses a custom dark theme with blue accents for better data visualization. Configuration is located in .streamlit/config.toml.